annotate mercurial/statprof.py @ 40910:4bb6195755b1

test: enable sparse-revlog for test-ssh-proto-unbundle.t We are about to enable sparse-revlog globally. To help with reviewing the tests change, we isolate them in individual changesets. Differential Revision: https://phab.mercurial-scm.org/D5337
author Boris Feld <boris.feld@octobus.net>
date Mon, 12 Nov 2018 01:17:05 +0100
parents 8664fdc1cfb3
children 876494fd967d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
1 #!/usr/bin/env python
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
2 ## statprof.py
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
3 ## Copyright (C) 2012 Bryan O'Sullivan <bos@serpentine.com>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
4 ## Copyright (C) 2011 Alex Fraser <alex at phatcore dot com>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
5 ## Copyright (C) 2004,2005 Andy Wingo <wingo at pobox dot com>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
6 ## Copyright (C) 2001 Rob Browning <rlb at defaultvalue dot org>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
7
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
8 ## This library is free software; you can redistribute it and/or
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
9 ## modify it under the terms of the GNU Lesser General Public
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
10 ## License as published by the Free Software Foundation; either
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
11 ## version 2.1 of the License, or (at your option) any later version.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
12 ##
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
13 ## This library is distributed in the hope that it will be useful,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
14 ## but WITHOUT ANY WARRANTY; without even the implied warranty of
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
15 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
16 ## Lesser General Public License for more details.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
17 ##
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
18 ## You should have received a copy of the GNU Lesser General Public
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
19 ## License along with this program; if not, contact:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
20 ##
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
21 ## Free Software Foundation Voice: +1-617-542-5942
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
22 ## 59 Temple Place - Suite 330 Fax: +1-617-542-2652
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
23 ## Boston, MA 02111-1307, USA gnu@gnu.org
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
24
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
25 """
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
26 statprof is intended to be a fairly simple statistical profiler for
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
27 python. It was ported directly from a statistical profiler for guile,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
28 also named statprof, available from guile-lib [0].
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
29
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
30 [0] http://wingolog.org/software/guile-lib/statprof/
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
31
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
32 To start profiling, call statprof.start():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
33 >>> start()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
34
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
35 Then run whatever it is that you want to profile, for example:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
36 >>> import test.pystone; test.pystone.pystones()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
37
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
38 Then stop the profiling and print out the results:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
39 >>> stop()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
40 >>> display()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
41 % cumulative self
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
42 time seconds seconds name
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
43 26.72 1.40 0.37 pystone.py:79:Proc0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
44 13.79 0.56 0.19 pystone.py:133:Proc1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
45 13.79 0.19 0.19 pystone.py:208:Proc8
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
46 10.34 0.16 0.14 pystone.py:229:Func2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
47 6.90 0.10 0.10 pystone.py:45:__init__
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
48 4.31 0.16 0.06 pystone.py:53:copy
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
49 ...
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
50
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
51 All of the numerical data is statistically approximate. In the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
52 following column descriptions, and in all of statprof, "time" refers
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
53 to execution time (both user and system), not wall clock time.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
54
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
55 % time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
56 The percent of the time spent inside the procedure itself (not
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
57 counting children).
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
58
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
59 cumulative seconds
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
60 The total number of seconds spent in the procedure, including
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
61 children.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
62
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
63 self seconds
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
64 The total number of seconds spent in the procedure itself (not
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
65 counting children).
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
66
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
67 name
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
68 The name of the procedure.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
69
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
70 By default statprof keeps the data collected from previous runs. If you
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
71 want to clear the collected data, call reset():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
72 >>> reset()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
73
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
74 reset() can also be used to change the sampling frequency from the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
75 default of 1000 Hz. For example, to tell statprof to sample 50 times a
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
76 second:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
77 >>> reset(50)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
78
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
79 This means that statprof will sample the call stack after every 1/50 of
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
80 a second of user + system time spent running on behalf of the python
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
81 process. When your process is idle (for example, blocking in a read(),
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
82 as is the case at the listener), the clock does not advance. For this
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
83 reason statprof is not currently not suitable for profiling io-bound
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
84 operations.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
85
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
86 The profiler uses the hash of the code object itself to identify the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
87 procedures, so it won't confuse different procedures with the same name.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
88 They will show up as two different rows in the output.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
89
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
90 Right now the profiler is quite simplistic. I cannot provide
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
91 call-graphs or other higher level information. What you see in the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
92 table is pretty much all there is. Patches are welcome :-)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
93
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
94
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
95 Threading
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
96 ---------
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
97
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
98 Because signals only get delivered to the main thread in Python,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
99 statprof only profiles the main thread. However because the time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
100 reporting function uses per-process timers, the results can be
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
101 significantly off if other threads' work patterns are not similar to the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
102 main thread's work patterns.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
103 """
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
104 # no-check-code
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
105 from __future__ import absolute_import, division, print_function
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
106
30256
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
107 import collections
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
108 import contextlib
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
109 import getopt
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
110 import inspect
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
111 import json
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
112 import os
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
113 import signal
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
114 import sys
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
115 import threading
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
116 import time
30256
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
117
30578
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
118 from . import (
30637
344e68882cd3 py3: replace os.environ with encoding.environ (part 4 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30578
diff changeset
119 encoding,
30578
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
120 pycompat,
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
121 )
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
122
30256
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
123 defaultdict = collections.defaultdict
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
124 contextmanager = contextlib.contextmanager
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
125
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
126 __all__ = ['start', 'stop', 'reset', 'display', 'profile']
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
127
40196
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
128 skips = {
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
129 r"util.py:check",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
130 r"extensions.py:closure",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
131 r"color.py:colorcmd",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
132 r"dispatch.py:checkargs",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
133 r"dispatch.py:<lambda>",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
134 r"dispatch.py:_runcatch",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
135 r"dispatch.py:_dispatch",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
136 r"dispatch.py:_runcommand",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
137 r"pager.py:pagecmd",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
138 r"dispatch.py:run",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
139 r"dispatch.py:dispatch",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
140 r"dispatch.py:runcommand",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
141 r"hg.py:<module>",
2864f8d3fcd6 py3: use raw strings for stack names
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40191
diff changeset
142 r"evolve.py:warnobserrors",
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 31074
diff changeset
143 }
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
144
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
145 ###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
146 ## Utils
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
147
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
148 def clock():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
149 times = os.times()
38259
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
150 return (times[0] + times[1], times[4])
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
151
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
152
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
153 ###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
154 ## Collection data structures
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
155
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
156 class ProfileState(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
157 def __init__(self, frequency=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
158 self.reset(frequency)
38260
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
159 self.track = 'cpu'
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
160
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
161 def reset(self, frequency=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
162 # total so far
38259
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
163 self.accumulated_time = (0.0, 0.0)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
164 # start_time when timer is active
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
165 self.last_start_time = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
166 # a float
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
167 if frequency:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
168 self.sample_interval = 1.0 / frequency
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
169 elif not hasattr(self, 'sample_interval'):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
170 # default to 1000 Hz
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
171 self.sample_interval = 1.0 / 1000.0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
172 else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
173 # leave the frequency as it was
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
174 pass
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
175 self.remaining_prof_time = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
176 # for user start/stop nesting
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
177 self.profile_level = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
178
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
179 self.samples = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
180
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
181 def accumulate_time(self, stop_time):
38259
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
182 increment = (
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
183 stop_time[0] - self.last_start_time[0],
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
184 stop_time[1] - self.last_start_time[1],
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
185 )
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
186 self.accumulated_time = (
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
187 self.accumulated_time[0] + increment[0],
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
188 self.accumulated_time[1] + increment[1],
af402c6b90db statprof: also gather wall time
Boris Feld <boris.feld@octobus.net>
parents: 38164
diff changeset
189 )
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
190
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
191 def seconds_per_sample(self):
38260
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
192 return self.accumulated_time[self.timeidx] / len(self.samples)
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
193
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
194 @property
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
195 def timeidx(self):
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
196 if self.track == 'real':
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
197 return 1
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
198 return 0
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
199
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
200 state = ProfileState()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
201
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
202
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
203 class CodeSite(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
204 cache = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
205
30568
bb35fe8621f5 py3: use unicodes in __slots__
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30299
diff changeset
206 __slots__ = (u'path', u'lineno', u'function', u'source')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
207
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
208 def __init__(self, path, lineno, function):
40484
93501a5fd62b statprof: add a couple of asserts to avoid storing unicodes
Augie Fackler <augie@google.com>
parents: 40385
diff changeset
209 assert isinstance(path, bytes)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
210 self.path = path
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
211 self.lineno = lineno
40484
93501a5fd62b statprof: add a couple of asserts to avoid storing unicodes
Augie Fackler <augie@google.com>
parents: 40385
diff changeset
212 assert isinstance(function, bytes)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
213 self.function = function
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
214 self.source = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
215
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
216 def __eq__(self, other):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
217 try:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
218 return (self.lineno == other.lineno and
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
219 self.path == other.path)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
220 except:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
221 return False
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
222
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
223 def __hash__(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
224 return hash((self.lineno, self.path))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
225
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
226 @classmethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
227 def get(cls, path, lineno, function):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
228 k = (path, lineno)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
229 try:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
230 return cls.cache[k]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
231 except KeyError:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
232 v = cls(path, lineno, function)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
233 cls.cache[k] = v
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
234 return v
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
235
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
236 def getsource(self, length):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
237 if self.source is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
238 lineno = self.lineno - 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
239 fp = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
240 try:
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
241 fp = open(self.path, 'rb')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
242 for i, line in enumerate(fp):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
243 if i == lineno:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
244 self.source = line.strip()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
245 break
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
246 except:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
247 pass
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
248 finally:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
249 if fp:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
250 fp.close()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
251 if self.source is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
252 self.source = ''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
254 source = self.source
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
255 if len(source) > length:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
256 source = source[:(length - 3)] + "..."
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
257 return source
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
258
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
259 def filename(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
260 return os.path.basename(self.path)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
261
40385
cc4586749c8c statprof: fix overflow while skipping boilerplate parts
Yuya Nishihara <yuya@tcha.org>
parents: 40384
diff changeset
262 def skipname(self):
cc4586749c8c statprof: fix overflow while skipping boilerplate parts
Yuya Nishihara <yuya@tcha.org>
parents: 40384
diff changeset
263 return r'%s:%s' % (self.filename(), self.function)
cc4586749c8c statprof: fix overflow while skipping boilerplate parts
Yuya Nishihara <yuya@tcha.org>
parents: 40384
diff changeset
264
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
265 class Sample(object):
30568
bb35fe8621f5 py3: use unicodes in __slots__
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30299
diff changeset
266 __slots__ = (u'stack', u'time')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
267
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
268 def __init__(self, stack, time):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
269 self.stack = stack
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
270 self.time = time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
271
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
272 @classmethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
273 def from_frame(cls, frame, time):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
274 stack = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
275
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
276 while frame:
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
277 stack.append(CodeSite.get(
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
278 pycompat.sysbytes(frame.f_code.co_filename),
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
279 frame.f_lineno,
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
280 pycompat.sysbytes(frame.f_code.co_name)))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
281 frame = frame.f_back
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
282
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
283 return Sample(stack, time)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
284
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
285 ###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
286 ## SIGPROF handler
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
287
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
288 def profile_signal_handler(signum, frame):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
289 if state.profile_level > 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
290 now = clock()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
291 state.accumulate_time(now)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
292
38260
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
293 timestamp = state.accumulated_time[state.timeidx]
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
294 state.samples.append(Sample.from_frame(frame, timestamp))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
295
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
296 signal.setitimer(signal.ITIMER_PROF,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
297 state.sample_interval, 0.0)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
298 state.last_start_time = now
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
299
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
300 stopthread = threading.Event()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
301 def samplerthread(tid):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
302 while not stopthread.is_set():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
303 now = clock()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
304 state.accumulate_time(now)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
305
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
306 frame = sys._current_frames()[tid]
38260
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
307
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
308 timestamp = state.accumulated_time[state.timeidx]
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
309 state.samples.append(Sample.from_frame(frame, timestamp))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
310
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
311 state.last_start_time = now
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
312 time.sleep(state.sample_interval)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
313
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
314 stopthread.clear()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
315
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
316 ###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
317 ## Profiling API
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
318
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
319 def is_active():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
320 return state.profile_level > 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
321
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
322 lastmechanism = None
38260
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
323 def start(mechanism='thread', track='cpu'):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
324 '''Install the profiling signal handler, and start profiling.'''
38260
15a1e37f80bd profiling: introduce a "profiling.time-track" option
Boris Feld <boris.feld@octobus.net>
parents: 38259
diff changeset
325 state.track = track # note: nesting different mode won't work
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
326 state.profile_level += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
327 if state.profile_level == 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
328 state.last_start_time = clock()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
329 rpt = state.remaining_prof_time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
330 state.remaining_prof_time = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
331
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
332 global lastmechanism
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
333 lastmechanism = mechanism
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
334
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
335 if mechanism == 'signal':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
336 signal.signal(signal.SIGPROF, profile_signal_handler)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
337 signal.setitimer(signal.ITIMER_PROF,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
338 rpt or state.sample_interval, 0.0)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
339 elif mechanism == 'thread':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
340 frame = inspect.currentframe()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
341 tid = [k for k, f in sys._current_frames().items() if f == frame][0]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
342 state.thread = threading.Thread(target=samplerthread,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
343 args=(tid,), name="samplerthread")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
344 state.thread.start()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
345
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
346 def stop():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
347 '''Stop profiling, and uninstall the profiling signal handler.'''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
348 state.profile_level -= 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
349 if state.profile_level == 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
350 if lastmechanism == 'signal':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
351 rpt = signal.setitimer(signal.ITIMER_PROF, 0.0, 0.0)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
352 signal.signal(signal.SIGPROF, signal.SIG_IGN)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
353 state.remaining_prof_time = rpt[0]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
354 elif lastmechanism == 'thread':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
355 stopthread.set()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
356 state.thread.join()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
357
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
358 state.accumulate_time(clock())
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
359 state.last_start_time = None
30637
344e68882cd3 py3: replace os.environ with encoding.environ (part 4 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30578
diff changeset
360 statprofpath = encoding.environ.get('STATPROF_DEST')
30255
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
361 if statprofpath:
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
362 save_data(statprofpath)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
363
30299
1e5346313963 statprof: return state from stop()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30258
diff changeset
364 return state
1e5346313963 statprof: return state from stop()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30258
diff changeset
365
30255
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
366 def save_data(path):
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
367 with open(path, 'w+') as file:
38274
99188a7c8717 statprof: fix save and load
Boris Feld <boris.feld@octobus.net>
parents: 38260
diff changeset
368 file.write("%f %f\n" % state.accumulated_time)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
369 for sample in state.samples:
40197
113adb1b3f24 py3: use %d in a few places
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40196
diff changeset
370 time = sample.time
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
371 stack = sample.stack
40201
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
372 sites = ['\1'.join([s.path, b'%d' % s.lineno, s.function])
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
373 for s in stack]
40197
113adb1b3f24 py3: use %d in a few places
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40196
diff changeset
374 file.write("%d\0%s\n" % (time, '\0'.join(sites)))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
375
30255
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
376 def load_data(path):
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
377 lines = open(path, 'rb').read().splitlines()
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
378
38274
99188a7c8717 statprof: fix save and load
Boris Feld <boris.feld@octobus.net>
parents: 38260
diff changeset
379 state.accumulated_time = [float(value) for value in lines[0].split()]
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
380 state.samples = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
381 for line in lines[1:]:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
382 parts = line.split('\0')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
383 time = float(parts[0])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
384 rawsites = parts[1:]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
385 sites = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
386 for rawsite in rawsites:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
387 siteparts = rawsite.split('\1')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
388 sites.append(CodeSite.get(siteparts[0], int(siteparts[1]),
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
389 siteparts[2]))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
390
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
391 state.samples.append(Sample(sites, time))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
392
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
393
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
394
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
395 def reset(frequency=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
396 '''Clear out the state of the profiler. Do not call while the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
397 profiler is running.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
398
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
399 The optional frequency argument specifies the number of samples to
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
400 collect per second.'''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
401 assert state.profile_level == 0, "Can't reset() while statprof is running"
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
402 CodeSite.cache.clear()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
403 state.reset(frequency)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
404
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
405
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
406 @contextmanager
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
407 def profile():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
408 start()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
409 try:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
410 yield
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
411 finally:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
412 stop()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
413 display()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
414
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
415
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
416 ###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
417 ## Reporting API
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
418
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
419 class SiteStats(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
420 def __init__(self, site):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
421 self.site = site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
422 self.selfcount = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
423 self.totalcount = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
424
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
425 def addself(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
426 self.selfcount += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
427
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
428 def addtotal(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
429 self.totalcount += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
430
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
431 def selfpercent(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
432 return self.selfcount / len(state.samples) * 100
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
433
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
434 def totalpercent(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
435 return self.totalcount / len(state.samples) * 100
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
436
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
437 def selfseconds(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
438 return self.selfcount * state.seconds_per_sample()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
439
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
440 def totalseconds(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
441 return self.totalcount * state.seconds_per_sample()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
442
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
443 @classmethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
444 def buildstats(cls, samples):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
445 stats = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
446
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
447 for sample in samples:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
448 for i, site in enumerate(sample.stack):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
449 sitestat = stats.get(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
450 if not sitestat:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
451 sitestat = SiteStats(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
452 stats[site] = sitestat
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
453
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
454 sitestat.addtotal()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
455
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
456 if i == 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
457 sitestat.addself()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
458
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
459 return [s for s in stats.itervalues()]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
460
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
461 class DisplayFormats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
462 ByLine = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
463 ByMethod = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
464 AboutMethod = 2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
465 Hotpath = 3
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
466 FlameGraph = 4
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
467 Json = 5
30929
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
468 Chrome = 6
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
469
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
470 def display(fp=None, format=3, data=None, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
471 '''Print statistics, either to stdout or the given file object.'''
38697
3c569172848d statprof: small if cleanup
Boris Feld <boris.feld@octobus.net>
parents: 38274
diff changeset
472 if data is None:
3c569172848d statprof: small if cleanup
Boris Feld <boris.feld@octobus.net>
parents: 38274
diff changeset
473 data = state
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
474
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
475 if fp is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
476 import sys
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
477 fp = sys.stdout
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
478 if len(data.samples) == 0:
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
479 fp.write(b'No samples recorded.\n')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
480 return
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
481
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
482 if format == DisplayFormats.ByLine:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
483 display_by_line(data, fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
484 elif format == DisplayFormats.ByMethod:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
485 display_by_method(data, fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
486 elif format == DisplayFormats.AboutMethod:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
487 display_about_method(data, fp, **kwargs)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
488 elif format == DisplayFormats.Hotpath:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
489 display_hotpath(data, fp, **kwargs)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
490 elif format == DisplayFormats.FlameGraph:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
491 write_to_flame(data, fp, **kwargs)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
492 elif format == DisplayFormats.Json:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
493 write_to_json(data, fp)
30929
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
494 elif format == DisplayFormats.Chrome:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
495 write_to_chrome(data, fp, **kwargs)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
496 else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
497 raise Exception("Invalid display format")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
498
30929
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
499 if format not in (DisplayFormats.Json, DisplayFormats.Chrome):
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
500 fp.write(b'---\n')
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
501 fp.write(b'Sample count: %d\n' % len(data.samples))
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
502 fp.write(b'Total time: %f seconds (%f wall)\n' % data.accumulated_time)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
503
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
504 def display_by_line(data, fp):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
505 '''Print the profiler data with each sample line represented
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
506 as one row in a table. Sorted by self-time per line.'''
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
507 stats = SiteStats.buildstats(data.samples)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
508 stats.sort(reverse=True, key=lambda x: x.selfseconds())
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
509
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
510 fp.write(b'%5.5s %10.10s %7.7s %-8.8s\n' % (
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
511 b'% ', b'cumulative', b'self', b''))
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
512 fp.write(b'%5.5s %9.9s %8.8s %-8.8s\n' % (
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
513 b"time", b"seconds", b"seconds", b"name"))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
514
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
515 for stat in stats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
516 site = stat.site
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
517 sitelabel = '%s:%d:%s' % (site.filename(),
40201
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
518 site.lineno,
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
519 site.function)
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
520 fp.write(b'%6.2f %9.2f %9.2f %s\n' % (
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
521 stat.selfpercent(), stat.totalseconds(),
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
522 stat.selfseconds(), sitelabel))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
523
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
524 def display_by_method(data, fp):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
525 '''Print the profiler data with each sample function represented
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
526 as one row in a table. Important lines within that function are
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
527 output as nested rows. Sorted by self-time per line.'''
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
528 fp.write(b'%5.5s %10.10s %7.7s %-8.8s\n' %
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
529 ('% ', 'cumulative', 'self', ''))
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
530 fp.write(b'%5.5s %9.9s %8.8s %-8.8s\n' %
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
531 ("time", "seconds", "seconds", "name"))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
532
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
533 stats = SiteStats.buildstats(data.samples)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
534
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
535 grouped = defaultdict(list)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
536 for stat in stats:
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
537 grouped[stat.site.filename() + b":" + stat.site.function].append(stat)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
538
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
539 # compute sums for each function
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
540 functiondata = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
541 for fname, sitestats in grouped.iteritems():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
542 total_cum_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
543 total_self_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
544 total_percent = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
545 for stat in sitestats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
546 total_cum_sec += stat.totalseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
547 total_self_sec += stat.selfseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
548 total_percent += stat.selfpercent()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
549
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
550 functiondata.append((fname,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
551 total_cum_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
552 total_self_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
553 total_percent,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
554 sitestats))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
555
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
556 # sort by total self sec
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
557 functiondata.sort(reverse=True, key=lambda x: x[2])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
558
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
559 for function in functiondata:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
560 if function[3] < 0.05:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
561 continue
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
562 fp.write(b'%6.2f %9.2f %9.2f %s\n' % (
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
563 function[3], # total percent
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
564 function[1], # total cum sec
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
565 function[2], # total self sec
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
566 function[0])) # file:function
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
567
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
568 function[4].sort(reverse=True, key=lambda i: i.selfseconds())
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
569 for stat in function[4]:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
570 # only show line numbers for significant locations (>1% time spent)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
571 if stat.selfpercent() > 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
572 source = stat.site.getsource(25)
40201
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
573 if sys.version_info.major >= 3 and not isinstance(source, bytes):
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
574 source = pycompat.bytestr(source)
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
575
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
576 stattuple = (stat.selfpercent(), stat.selfseconds(),
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
577 stat.site.lineno, source)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
578
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
579 fp.write(b'%33.0f%% %6.2f line %d: %s\n' % stattuple)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
580
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
581 def display_about_method(data, fp, function=None, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
582 if function is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
583 raise Exception("Invalid function")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
584
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
585 filename = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
586 if ':' in function:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
587 filename, function = function.split(':')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
588
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
589 relevant_samples = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
590 parents = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
591 children = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
592
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
593 for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
594 for i, site in enumerate(sample.stack):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
595 if site.function == function and (not filename
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
596 or site.filename() == filename):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
597 relevant_samples += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
598 if i != len(sample.stack) - 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
599 parent = sample.stack[i + 1]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
600 if parent in parents:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
601 parents[parent] = parents[parent] + 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
602 else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
603 parents[parent] = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
604
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
605 if site in children:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
606 children[site] = children[site] + 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
607 else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
608 children[site] = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
609
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
610 parents = [(parent, count) for parent, count in parents.iteritems()]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
611 parents.sort(reverse=True, key=lambda x: x[1])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
612 for parent, count in parents:
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
613 fp.write(b'%6.2f%% %s:%s line %s: %s\n' %
40201
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
614 (count / relevant_samples * 100,
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
615 pycompat.fsencode(parent.filename()),
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
616 pycompat.sysbytes(parent.function),
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
617 parent.lineno,
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
618 pycompat.sysbytes(parent.getsource(50))))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
619
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
620 stats = SiteStats.buildstats(data.samples)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
621 stats = [s for s in stats
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
622 if s.site.function == function and
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
623 (not filename or s.site.filename() == filename)]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
624
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
625 total_cum_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
626 total_self_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
627 total_self_percent = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
628 total_cum_percent = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
629 for stat in stats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
630 total_cum_sec += stat.totalseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
631 total_self_sec += stat.selfseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
632 total_self_percent += stat.selfpercent()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
633 total_cum_percent += stat.totalpercent()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
634
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
635 fp.write(
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
636 b'\n %s:%s Total: %0.2fs (%0.2f%%) Self: %0.2fs (%0.2f%%)\n\n'
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
637 % (
40201
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
638 pycompat.sysbytes(filename or '___'),
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
639 pycompat.sysbytes(function),
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
640 total_cum_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
641 total_cum_percent,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
642 total_self_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
643 total_self_percent
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
644 ))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
645
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
646 children = [(child, count) for child, count in children.iteritems()]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
647 children.sort(reverse=True, key=lambda x: x[1])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
648 for child, count in children:
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
649 fp.write(b' %6.2f%% line %s: %s\n' %
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
650 (count / relevant_samples * 100, child.lineno,
40201
7df42042636d py3: sprinkle statprof.py with utf-8 encoding
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40199
diff changeset
651 pycompat.sysbytes(child.getsource(50))))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
652
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
653 def display_hotpath(data, fp, limit=0.05, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
654 class HotNode(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
655 def __init__(self, site):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
656 self.site = site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
657 self.count = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
658 self.children = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
659
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
660 def add(self, stack, time):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
661 self.count += time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
662 site = stack[0]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
663 child = self.children.get(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
664 if not child:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
665 child = HotNode(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
666 self.children[site] = child
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
667
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
668 if len(stack) > 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
669 i = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
670 # Skip boiler plate parts of the stack
40385
cc4586749c8c statprof: fix overflow while skipping boilerplate parts
Yuya Nishihara <yuya@tcha.org>
parents: 40384
diff changeset
671 while i < len(stack) and stack[i].skipname() in skips:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
672 i += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
673 if i < len(stack):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
674 child.add(stack[i:], time)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
675
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
676 root = HotNode(None)
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
677 lasttime = data.samples[0].time
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
678 for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
679 root.add(sample.stack[::-1], sample.time - lasttime)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
680 lasttime = sample.time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
681
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
682 def _write(node, depth, multiple_siblings):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
683 site = node.site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
684 visiblechildren = [c for c in node.children.itervalues()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
685 if c.count >= (limit * root.count)]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
686 if site:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
687 indent = depth * 2 - 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
688 filename = ''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
689 function = ''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
690 if len(node.children) > 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
691 childsite = list(node.children.itervalues())[0].site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
692 filename = (childsite.filename() + ':').ljust(15)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
693 function = childsite.function
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
694
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
695 # lots of string formatting
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
696 listpattern = ''.ljust(indent) +\
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
697 ('\\' if multiple_siblings else '|') +\
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
698 ' %4.1f%% %s %s'
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
699 liststring = listpattern % (node.count / root.count * 100,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
700 filename, function)
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
701 codepattern = '%' + ('%d' % (55 - len(liststring))) + 's %d: %s'
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
702 codestring = codepattern % ('line', site.lineno, site.getsource(30))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
703
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
704 finalstring = liststring + codestring
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
705 childrensamples = sum([c.count for c in node.children.itervalues()])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
706 # Make frames that performed more than 10% of the operation red
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
707 if node.count - childrensamples > (0.1 * root.count):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
708 finalstring = '\033[91m' + finalstring + '\033[0m'
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
709 # Make frames that didn't actually perform work dark grey
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
710 elif node.count - childrensamples == 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
711 finalstring = '\033[90m' + finalstring + '\033[0m'
40384
fc4c598dd4a0 statprof: fix indent level of fp.write() (issue6004)
Yuya Nishihara <yuya@tcha.org>
parents: 40381
diff changeset
712 fp.write(finalstring + b'\n')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
713
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
714 newdepth = depth
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
715 if len(visiblechildren) > 1 or multiple_siblings:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
716 newdepth += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
717
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
718 visiblechildren.sort(reverse=True, key=lambda x: x.count)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
719 for child in visiblechildren:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
720 _write(child, newdepth, len(visiblechildren) > 1)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
721
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
722 if root.count > 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
723 _write(root, 0, False)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
724
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
725 def write_to_flame(data, fp, scriptpath=None, outputfile=None, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
726 if scriptpath is None:
30637
344e68882cd3 py3: replace os.environ with encoding.environ (part 4 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30578
diff changeset
727 scriptpath = encoding.environ['HOME'] + '/flamegraph.pl'
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
728 if not os.path.exists(scriptpath):
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
729 fp.write(b'error: missing %s\n' % scriptpath)
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
730 fp.write(b'get it here: https://github.com/brendangregg/FlameGraph\n')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
731 return
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
732
38164
aac4be30e250 py3: wrap tempfile.mkstemp() to use bytes path
Yuya Nishihara <yuya@tcha.org>
parents: 35413
diff changeset
733 fd, path = pycompat.mkstemp()
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
734
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
735 file = open(path, "w+")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
736
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
737 lines = {}
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
738 for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
739 sites = [s.function for s in sample.stack]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
740 sites.reverse()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
741 line = ';'.join(sites)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
742 if line in lines:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
743 lines[line] = lines[line] + 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
744 else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
745 lines[line] = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
746
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
747 for line, count in lines.iteritems():
40197
113adb1b3f24 py3: use %d in a few places
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40196
diff changeset
748 file.write("%s %d\n" % (line, count))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
749
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
750 file.close()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
751
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
752 if outputfile is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
753 outputfile = '~/flamegraph.svg'
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
754
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
755 os.system("perl ~/flamegraph.pl %s > %s" % (path, outputfile))
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
756 fp.write(b'Written to %s\n' % outputfile)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
757
30928
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
758 _pathcache = {}
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
759 def simplifypath(path):
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
760 '''Attempt to make the path to a Python module easier to read by
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
761 removing whatever part of the Python search path it was found
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
762 on.'''
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
763
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
764 if path in _pathcache:
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
765 return _pathcache[path]
31074
2912b06905dc py3: use pycompat.fsencode() to convert __file__ to bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30929
diff changeset
766 hgpath = pycompat.fsencode(encoding.__file__).rsplit(os.sep, 2)[0]
30928
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
767 for p in [hgpath] + sys.path:
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
768 prefix = p + os.sep
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
769 if path.startswith(prefix):
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
770 path = path[len(prefix):]
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
771 break
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
772 _pathcache[path] = path
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
773 return path
be3a4fde38eb statprof: add a path simplification function
Bryan O'Sullivan <bryano@fb.com>
parents: 30845
diff changeset
774
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
775 def write_to_json(data, fp):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
776 samples = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
777
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
778 for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
779 stack = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
780
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
781 for frame in sample.stack:
40485
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
782 stack.append(
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
783 (pycompat.sysstr(frame.path),
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
784 frame.lineno,
8664fdc1cfb3 statprof: clean up unicode/bytes a little
Augie Fackler <augie@google.com>
parents: 40484
diff changeset
785 pycompat.sysstr(frame.function)))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
786
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
787 samples.append((sample.time, stack))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
788
40191
4b7eb862692e py3: encode json output to bytes and use write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39092
diff changeset
789 data = json.dumps(samples)
4b7eb862692e py3: encode json output to bytes and use write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39092
diff changeset
790 if not isinstance(data, bytes):
4b7eb862692e py3: encode json output to bytes and use write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39092
diff changeset
791 data = data.encode('utf-8')
4b7eb862692e py3: encode json output to bytes and use write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39092
diff changeset
792
4b7eb862692e py3: encode json output to bytes and use write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39092
diff changeset
793 fp.write(data)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
794
30929
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
795 def write_to_chrome(data, fp, minthreshold=0.005, maxthreshold=0.999):
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
796 samples = []
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
797 laststack = collections.deque()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
798 lastseen = collections.deque()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
799
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
800 # The Chrome tracing format allows us to use a compact stack
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
801 # representation to save space. It's fiddly but worth it.
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
802 # We maintain a bijection between stack and ID.
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
803 stack2id = {}
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
804 id2stack = [] # will eventually be rendered
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
805
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
806 def stackid(stack):
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
807 if not stack:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
808 return
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
809 if stack in stack2id:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
810 return stack2id[stack]
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
811 parent = stackid(stack[1:])
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
812 myid = len(stack2id)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
813 stack2id[stack] = myid
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
814 id2stack.append(dict(category=stack[0][0], name='%s %s' % stack[0]))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
815 if parent is not None:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
816 id2stack[-1].update(parent=parent)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
817 return myid
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
818
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
819 def endswith(a, b):
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
820 return list(a)[-len(b):] == list(b)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
821
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
822 # The sampling profiler can sample multiple times without
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
823 # advancing the clock, potentially causing the Chrome trace viewer
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
824 # to render single-pixel columns that we cannot zoom in on. We
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
825 # work around this by pretending that zero-duration samples are a
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
826 # millisecond in length.
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
827
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
828 clamp = 0.001
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
829
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
830 # We provide knobs that by default attempt to filter out stack
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
831 # frames that are too noisy:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
832 #
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
833 # * A few take almost all execution time. These are usually boring
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
834 # setup functions, giving a stack that is deep but uninformative.
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
835 #
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
836 # * Numerous samples take almost no time, but introduce lots of
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
837 # noisy, oft-deep "spines" into a rendered profile.
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
838
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
839 blacklist = set()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
840 totaltime = data.samples[-1].time - data.samples[0].time
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
841 minthreshold = totaltime * minthreshold
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
842 maxthreshold = max(totaltime * maxthreshold, clamp)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
843
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
844 def poplast():
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
845 oldsid = stackid(tuple(laststack))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
846 oldcat, oldfunc = laststack.popleft()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
847 oldtime, oldidx = lastseen.popleft()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
848 duration = sample.time - oldtime
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
849 if minthreshold <= duration <= maxthreshold:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
850 # ensure no zero-duration events
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
851 sampletime = max(oldtime + clamp, sample.time)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
852 samples.append(dict(ph='E', name=oldfunc, cat=oldcat, sf=oldsid,
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
853 ts=sampletime*1e6, pid=0))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
854 else:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
855 blacklist.add(oldidx)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
856
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
857 # Much fiddling to synthesize correctly(ish) nested begin/end
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
858 # events given only stack snapshots.
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
859
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
860 for sample in data.samples:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
861 tos = sample.stack[0]
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
862 name = tos.function
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
863 path = simplifypath(tos.path)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
864 stack = tuple((('%s:%d' % (simplifypath(frame.path), frame.lineno),
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
865 frame.function) for frame in sample.stack))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
866 qstack = collections.deque(stack)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
867 if laststack == qstack:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
868 continue
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
869 while laststack and qstack and laststack[-1] == qstack[-1]:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
870 laststack.pop()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
871 qstack.pop()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
872 while laststack:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
873 poplast()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
874 for f in reversed(qstack):
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
875 lastseen.appendleft((sample.time, len(samples)))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
876 laststack.appendleft(f)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
877 path, name = f
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
878 sid = stackid(tuple(laststack))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
879 samples.append(dict(ph='B', name=name, cat=path, ts=sample.time*1e6,
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
880 sf=sid, pid=0))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
881 laststack = collections.deque(stack)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
882 while laststack:
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
883 poplast()
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
884 events = [s[1] for s in enumerate(samples) if s[0] not in blacklist]
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
885 frames = collections.OrderedDict((str(k), v)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
886 for (k,v) in enumerate(id2stack))
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
887 json.dump(dict(traceEvents=events, stackFrames=frames), fp, indent=1)
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
888 fp.write('\n')
cb440e7af05d statprof: allow rendering in the Chrome trace viewer format
Bryan O'Sullivan <bryano@fb.com>
parents: 30928
diff changeset
889
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
890 def printusage():
40198
9d3034348c4f py3: switch from print(..., file=) to write()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40197
diff changeset
891 print(r"""
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
892 The statprof command line allows you to inspect the last profile's results in
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
893 the following forms:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
894
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
895 usage:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
896 hotpath [-l --limit percent]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
897 Shows a graph of calls with the percent of time each takes.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
898 Red calls take over 10%% of the total time themselves.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
899 lines
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
900 Shows the actual sampled lines.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
901 functions
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
902 Shows the samples grouped by function.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
903 function [filename:]functionname
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
904 Shows the callers and callees of a particular function.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
905 flame [-s --script-path] [-o --output-file path]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
906 Writes out a flamegraph to output-file (defaults to ~/flamegraph.svg)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
907 Requires that ~/flamegraph.pl exist.
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
908 (Specify alternate script path with --script-path.)""")
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
909
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
910 def main(argv=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
911 if argv is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
912 argv = sys.argv
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
913
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
914 if len(argv) == 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
915 printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
916 return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
917
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
918 displayargs = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
919
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
920 optstart = 2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
921 displayargs['function'] = None
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
922 if argv[1] == r'hotpath':
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
923 displayargs['format'] = DisplayFormats.Hotpath
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
924 elif argv[1] == r'lines':
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
925 displayargs['format'] = DisplayFormats.ByLine
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
926 elif argv[1] == r'functions':
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
927 displayargs['format'] = DisplayFormats.ByMethod
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
928 elif argv[1] == r'function':
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
929 displayargs['format'] = DisplayFormats.AboutMethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
930 displayargs['function'] = argv[2]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
931 optstart = 3
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
932 elif argv[1] == r'flame':
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
933 displayargs['format'] = DisplayFormats.FlameGraph
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
934 else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
935 printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
936 return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
937
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
938 # process options
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
939 try:
30578
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
940 opts, args = pycompat.getoptb(sys.argv[optstart:], "hl:f:o:p:",
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
941 ["help", "limit=", "file=", "output-file=", "script-path="])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
942 except getopt.error as msg:
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
943 print(msg)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
944 printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
945 return 2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
946
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
947 displayargs['limit'] = 0.05
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
948 path = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
949 for o, value in opts:
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
950 if o in (r"-l", r"--limit"):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
951 displayargs['limit'] = float(value)
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
952 elif o in (r"-f", r"--file"):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
953 path = value
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
954 elif o in (r"-o", r"--output-file"):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
955 displayargs['outputfile'] = value
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
956 elif o in (r"-p", r"--script-path"):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
957 displayargs['scriptpath'] = value
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
958 elif o in (r"-h", r"help"):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
959 printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
960 return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
961 else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
962 assert False, "unhandled option %s" % o
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
963
30845
262c2be8ea5a statprof: require input file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30637
diff changeset
964 if not path:
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
965 print(r'must specify --file to load')
30845
262c2be8ea5a statprof: require input file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30637
diff changeset
966 return 1
262c2be8ea5a statprof: require input file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30637
diff changeset
967
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
968 load_data(path=path)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
969
35370
c5853c9a6545 py3: handle keyword arguments correctly in statprof.py
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32291
diff changeset
970 display(**pycompat.strkwargs(displayargs))
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
971
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
972 return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
973
40199
b594db74dc13 py3: use raw strings in statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 40198
diff changeset
974 if __name__ == r"__main__":
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
975 sys.exit(main())