mercurial/statprof.py
author Gregory Szorc <gregory.szorc@gmail.com>
Sat, 24 Dec 2016 15:29:32 -0700
changeset 30764 e75463e3179f
parent 30637 344e68882cd3
child 30845 262c2be8ea5a
permissions -rwxr-xr-x
protocol: send application/mercurial-0.2 responses to capable clients With this commit, the HTTP transport now parses the X-HgProto-<N> header to determine what media type and compression engine to use for responses. So far, we only compress responses that are already being compressed with zlib today (stream response types to specific commands). We can expand things to cover additional response types later. The practical side-effect of this commit is that non-zlib compression engines will be used if both ends support them. This means if both ends have zstd support, zstd - not zlib - will be used to compress data! When cloning the mozilla-unified repository between a local HTTP server and client, the benefits of non-zlib compression are quite noticeable: engine server CPU (s) client CPU (s) bundle size zlib (l=6) 174.1 283.2 1,148,547,026 zstd (l=1) 99.2 267.3 1,127,513,841 zstd (l=3) 103.1 266.9 1,018,861,363 zstd (l=7) 128.3 269.7 919,190,278 zstd (l=10) 162.0 - 894,547,179 none 95.3 277.2 4,097,566,064 The default zstd compression level is 3. So if you deploy zstd capable Mercurial to your clients and servers and CPU time on your server is dominated by "getbundle" requests (clients cloning and pulling) - and my experience at Mozilla tells me this is often the case - this commit could drastically reduce your server-side CPU usage *and* save on bandwidth costs! Another benefit of this change is that server operators can install *any* compression engine. While it isn't enabled by default, the "none" compression engine can now be used to disable wire protocol compression completely. Previously, commands like "getbundle" always zlib compressed output, adding considerable overhead to generating responses. If you are on a high speed network and your server is under high load, it might be advantageous to trade bandwidth for CPU. Although, zstd at level 1 doesn't use that much CPU, so I'm not convinced that disabling compression wholesale is worthwhile. And, my data seems to indicate a slow down on the client without compression. I suspect this is due to a lack of buffering resulting in an increase in socket read() calls and/or the fact we're transferring an extra 3 GB of data (parsing HTTP chunked transfer and processing extra TCP packets can add up). This is definitely worth investigating and optimizing. But since the "none" compressor isn't enabled by default, I'm inclined to punt on this issue. This commit introduces tons of tests. Some of these should arguably have been implemented on previous commits. But it was difficult to test without the server functionality in place.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
#!/usr/bin/env python
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
## statprof.py
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
## Copyright (C) 2012 Bryan O'Sullivan <bos@serpentine.com>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     4
## Copyright (C) 2011 Alex Fraser <alex at phatcore dot com>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     5
## Copyright (C) 2004,2005 Andy Wingo <wingo at pobox dot com>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     6
## Copyright (C) 2001 Rob Browning <rlb at defaultvalue dot org>
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     7
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     8
## This library is free software; you can redistribute it and/or
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     9
## modify it under the terms of the GNU Lesser General Public
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    10
## License as published by the Free Software Foundation; either
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    11
## version 2.1 of the License, or (at your option) any later version.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    12
##
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    13
## This library is distributed in the hope that it will be useful,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    14
## but WITHOUT ANY WARRANTY; without even the implied warranty of
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    15
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    16
## Lesser General Public License for more details.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    17
##
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    18
## You should have received a copy of the GNU Lesser General Public
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    19
## License along with this program; if not, contact:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    20
##
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    21
## Free Software Foundation           Voice:  +1-617-542-5942
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    22
## 59 Temple Place - Suite 330        Fax:    +1-617-542-2652
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    23
## Boston, MA  02111-1307,  USA       gnu@gnu.org
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    24
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    25
"""
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    26
statprof is intended to be a fairly simple statistical profiler for
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    27
python. It was ported directly from a statistical profiler for guile,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    28
also named statprof, available from guile-lib [0].
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    29
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    30
[0] http://wingolog.org/software/guile-lib/statprof/
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    31
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    32
To start profiling, call statprof.start():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    33
>>> start()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    34
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    35
Then run whatever it is that you want to profile, for example:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    36
>>> import test.pystone; test.pystone.pystones()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    37
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    38
Then stop the profiling and print out the results:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    39
>>> stop()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    40
>>> display()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    41
  %   cumulative      self
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    42
 time    seconds   seconds  name
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    43
 26.72      1.40      0.37  pystone.py:79:Proc0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    44
 13.79      0.56      0.19  pystone.py:133:Proc1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    45
 13.79      0.19      0.19  pystone.py:208:Proc8
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    46
 10.34      0.16      0.14  pystone.py:229:Func2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    47
  6.90      0.10      0.10  pystone.py:45:__init__
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    48
  4.31      0.16      0.06  pystone.py:53:copy
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    49
    ...
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    50
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    51
All of the numerical data is statistically approximate. In the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    52
following column descriptions, and in all of statprof, "time" refers
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    53
to execution time (both user and system), not wall clock time.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    54
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    55
% time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    56
    The percent of the time spent inside the procedure itself (not
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    57
    counting children).
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    58
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    59
cumulative seconds
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    60
    The total number of seconds spent in the procedure, including
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    61
    children.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    62
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    63
self seconds
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    64
    The total number of seconds spent in the procedure itself (not
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    65
    counting children).
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    66
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    67
name
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    68
    The name of the procedure.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    69
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    70
By default statprof keeps the data collected from previous runs. If you
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    71
want to clear the collected data, call reset():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    72
>>> reset()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    73
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    74
reset() can also be used to change the sampling frequency from the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    75
default of 1000 Hz. For example, to tell statprof to sample 50 times a
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    76
second:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    77
>>> reset(50)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    78
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    79
This means that statprof will sample the call stack after every 1/50 of
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    80
a second of user + system time spent running on behalf of the python
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    81
process. When your process is idle (for example, blocking in a read(),
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    82
as is the case at the listener), the clock does not advance. For this
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    83
reason statprof is not currently not suitable for profiling io-bound
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    84
operations.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    85
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    86
The profiler uses the hash of the code object itself to identify the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    87
procedures, so it won't confuse different procedures with the same name.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    88
They will show up as two different rows in the output.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    89
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    90
Right now the profiler is quite simplistic.  I cannot provide
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    91
call-graphs or other higher level information.  What you see in the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    92
table is pretty much all there is. Patches are welcome :-)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    93
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    94
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    95
Threading
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    96
---------
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    97
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    98
Because signals only get delivered to the main thread in Python,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
statprof only profiles the main thread. However because the time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   100
reporting function uses per-process timers, the results can be
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   101
significantly off if other threads' work patterns are not similar to the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   102
main thread's work patterns.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   103
"""
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   104
# no-check-code
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   105
from __future__ import absolute_import, division, print_function
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   106
30256
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   107
import collections
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   108
import contextlib
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   109
import getopt
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   110
import inspect
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   111
import json
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   112
import os
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   113
import signal
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   114
import sys
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   115
import tempfile
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   116
import threading
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   117
import time
30256
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   118
30578
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
   119
from . import (
30637
344e68882cd3 py3: replace os.environ with encoding.environ (part 4 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30578
diff changeset
   120
    encoding,
30578
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
   121
    pycompat,
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
   122
)
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
   123
30256
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   124
defaultdict = collections.defaultdict
2ed0b3f9f79e statprof: use absolute_imports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30255
diff changeset
   125
contextmanager = contextlib.contextmanager
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   126
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   127
__all__ = ['start', 'stop', 'reset', 'display', 'profile']
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   128
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   129
skips = set(["util.py:check", "extensions.py:closure",
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   130
             "color.py:colorcmd", "dispatch.py:checkargs",
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   131
             "dispatch.py:<lambda>", "dispatch.py:_runcatch",
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   132
             "dispatch.py:_dispatch", "dispatch.py:_runcommand",
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   133
             "pager.py:pagecmd", "dispatch.py:run",
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   134
             "dispatch.py:dispatch", "dispatch.py:runcommand",
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   135
             "hg.py:<module>", "evolve.py:warnobserrors",
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   136
         ])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   137
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   138
###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   139
## Utils
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   140
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   141
def clock():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   142
    times = os.times()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   143
    return times[0] + times[1]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   144
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   145
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   146
###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   147
## Collection data structures
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   148
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   149
class ProfileState(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   150
    def __init__(self, frequency=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   151
        self.reset(frequency)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   152
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   153
    def reset(self, frequency=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   154
        # total so far
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   155
        self.accumulated_time = 0.0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   156
        # start_time when timer is active
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   157
        self.last_start_time = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   158
        # a float
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   159
        if frequency:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   160
            self.sample_interval = 1.0 / frequency
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   161
        elif not hasattr(self, 'sample_interval'):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   162
            # default to 1000 Hz
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   163
            self.sample_interval = 1.0 / 1000.0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   164
        else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   165
            # leave the frequency as it was
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   166
            pass
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   167
        self.remaining_prof_time = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   168
        # for user start/stop nesting
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   169
        self.profile_level = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   170
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   171
        self.samples = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   172
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   173
    def accumulate_time(self, stop_time):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   174
        self.accumulated_time += stop_time - self.last_start_time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   175
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   176
    def seconds_per_sample(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   177
        return self.accumulated_time / len(self.samples)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   178
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   179
state = ProfileState()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   180
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   181
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   182
class CodeSite(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   183
    cache = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   184
30568
bb35fe8621f5 py3: use unicodes in __slots__
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30299
diff changeset
   185
    __slots__ = (u'path', u'lineno', u'function', u'source')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   186
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   187
    def __init__(self, path, lineno, function):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   188
        self.path = path
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   189
        self.lineno = lineno
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   190
        self.function = function
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   191
        self.source = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   192
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   193
    def __eq__(self, other):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   194
        try:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   195
            return (self.lineno == other.lineno and
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   196
                    self.path == other.path)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   197
        except:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   198
            return False
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   199
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   200
    def __hash__(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   201
        return hash((self.lineno, self.path))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   202
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   203
    @classmethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   204
    def get(cls, path, lineno, function):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   205
        k = (path, lineno)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   206
        try:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   207
            return cls.cache[k]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   208
        except KeyError:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   209
            v = cls(path, lineno, function)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   210
            cls.cache[k] = v
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   211
            return v
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   212
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   213
    def getsource(self, length):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   214
        if self.source is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   215
            lineno = self.lineno - 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   216
            fp = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   217
            try:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   218
                fp = open(self.path)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   219
                for i, line in enumerate(fp):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   220
                    if i == lineno:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   221
                        self.source = line.strip()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   222
                        break
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   223
            except:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   224
                pass
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   225
            finally:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   226
                if fp:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   227
                    fp.close()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   228
            if self.source is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   229
                self.source = ''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   230
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   231
        source = self.source
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   232
        if len(source) > length:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   233
            source = source[:(length - 3)] + "..."
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   234
        return source
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   235
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   236
    def filename(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   237
        return os.path.basename(self.path)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   238
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   239
class Sample(object):
30568
bb35fe8621f5 py3: use unicodes in __slots__
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30299
diff changeset
   240
    __slots__ = (u'stack', u'time')
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   241
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   242
    def __init__(self, stack, time):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   243
        self.stack = stack
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   244
        self.time = time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   245
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   246
    @classmethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   247
    def from_frame(cls, frame, time):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   248
        stack = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   249
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   250
        while frame:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   251
            stack.append(CodeSite.get(frame.f_code.co_filename, frame.f_lineno,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   252
                                      frame.f_code.co_name))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   253
            frame = frame.f_back
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   254
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   255
        return Sample(stack, time)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   256
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   257
###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   258
## SIGPROF handler
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   259
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   260
def profile_signal_handler(signum, frame):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   261
    if state.profile_level > 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   262
        now = clock()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   263
        state.accumulate_time(now)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   264
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   265
        state.samples.append(Sample.from_frame(frame, state.accumulated_time))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   266
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   267
        signal.setitimer(signal.ITIMER_PROF,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   268
            state.sample_interval, 0.0)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   269
        state.last_start_time = now
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   270
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   271
stopthread = threading.Event()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   272
def samplerthread(tid):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   273
    while not stopthread.is_set():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   274
        now = clock()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   275
        state.accumulate_time(now)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   276
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   277
        frame = sys._current_frames()[tid]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   278
        state.samples.append(Sample.from_frame(frame, state.accumulated_time))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   279
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   280
        state.last_start_time = now
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   281
        time.sleep(state.sample_interval)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   282
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   283
    stopthread.clear()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   284
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   285
###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   286
## Profiling API
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   287
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   288
def is_active():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   289
    return state.profile_level > 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   290
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   291
lastmechanism = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   292
def start(mechanism='thread'):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   293
    '''Install the profiling signal handler, and start profiling.'''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   294
    state.profile_level += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   295
    if state.profile_level == 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   296
        state.last_start_time = clock()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   297
        rpt = state.remaining_prof_time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   298
        state.remaining_prof_time = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   299
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   300
        global lastmechanism
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   301
        lastmechanism = mechanism
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   302
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   303
        if mechanism == 'signal':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   304
            signal.signal(signal.SIGPROF, profile_signal_handler)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   305
            signal.setitimer(signal.ITIMER_PROF,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   306
                rpt or state.sample_interval, 0.0)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   307
        elif mechanism == 'thread':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   308
            frame = inspect.currentframe()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   309
            tid = [k for k, f in sys._current_frames().items() if f == frame][0]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   310
            state.thread = threading.Thread(target=samplerthread,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   311
                                 args=(tid,), name="samplerthread")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   312
            state.thread.start()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   313
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   314
def stop():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   315
    '''Stop profiling, and uninstall the profiling signal handler.'''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   316
    state.profile_level -= 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   317
    if state.profile_level == 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   318
        if lastmechanism == 'signal':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   319
            rpt = signal.setitimer(signal.ITIMER_PROF, 0.0, 0.0)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   320
            signal.signal(signal.SIGPROF, signal.SIG_IGN)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   321
            state.remaining_prof_time = rpt[0]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   322
        elif lastmechanism == 'thread':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   323
            stopthread.set()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   324
            state.thread.join()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   325
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   326
        state.accumulate_time(clock())
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   327
        state.last_start_time = None
30637
344e68882cd3 py3: replace os.environ with encoding.environ (part 4 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30578
diff changeset
   328
        statprofpath = encoding.environ.get('STATPROF_DEST')
30255
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
   329
        if statprofpath:
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
   330
            save_data(statprofpath)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   331
30299
1e5346313963 statprof: return state from stop()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30258
diff changeset
   332
    return state
1e5346313963 statprof: return state from stop()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30258
diff changeset
   333
30255
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
   334
def save_data(path):
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
   335
    with open(path, 'w+') as file:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   336
        file.write(str(state.accumulated_time) + '\n')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   337
        for sample in state.samples:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   338
            time = str(sample.time)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   339
            stack = sample.stack
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   340
            sites = ['\1'.join([s.path, str(s.lineno), s.function])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   341
                     for s in stack]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   342
            file.write(time + '\0' + '\0'.join(sites) + '\n')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   343
30255
f42cd5434cc2 statprof: require paths to save or load profile data
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30254
diff changeset
   344
def load_data(path):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   345
    lines = open(path, 'r').read().splitlines()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   346
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   347
    state.accumulated_time = float(lines[0])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   348
    state.samples = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   349
    for line in lines[1:]:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   350
        parts = line.split('\0')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   351
        time = float(parts[0])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   352
        rawsites = parts[1:]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   353
        sites = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   354
        for rawsite in rawsites:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   355
            siteparts = rawsite.split('\1')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   356
            sites.append(CodeSite.get(siteparts[0], int(siteparts[1]),
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   357
                        siteparts[2]))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   358
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   359
        state.samples.append(Sample(sites, time))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   360
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   361
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   362
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   363
def reset(frequency=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   364
    '''Clear out the state of the profiler.  Do not call while the
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   365
    profiler is running.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   366
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   367
    The optional frequency argument specifies the number of samples to
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   368
    collect per second.'''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   369
    assert state.profile_level == 0, "Can't reset() while statprof is running"
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   370
    CodeSite.cache.clear()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   371
    state.reset(frequency)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   372
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   373
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   374
@contextmanager
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   375
def profile():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   376
    start()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   377
    try:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   378
        yield
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   379
    finally:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   380
        stop()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   381
        display()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   382
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   383
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   384
###########################################################################
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   385
## Reporting API
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   386
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   387
class SiteStats(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   388
    def __init__(self, site):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   389
        self.site = site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   390
        self.selfcount = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   391
        self.totalcount = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   392
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   393
    def addself(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   394
        self.selfcount += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   395
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   396
    def addtotal(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   397
        self.totalcount += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   398
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   399
    def selfpercent(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   400
        return self.selfcount / len(state.samples) * 100
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   401
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   402
    def totalpercent(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   403
        return self.totalcount / len(state.samples) * 100
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   404
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   405
    def selfseconds(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   406
        return self.selfcount * state.seconds_per_sample()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   407
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   408
    def totalseconds(self):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   409
        return self.totalcount * state.seconds_per_sample()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   410
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   411
    @classmethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   412
    def buildstats(cls, samples):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   413
        stats = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   414
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   415
        for sample in samples:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   416
            for i, site in enumerate(sample.stack):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   417
                sitestat = stats.get(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   418
                if not sitestat:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   419
                    sitestat = SiteStats(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   420
                    stats[site] = sitestat
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   421
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   422
                sitestat.addtotal()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   423
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   424
                if i == 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   425
                    sitestat.addself()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   426
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   427
        return [s for s in stats.itervalues()]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   428
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   429
class DisplayFormats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   430
    ByLine = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   431
    ByMethod = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   432
    AboutMethod = 2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   433
    Hotpath = 3
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   434
    FlameGraph = 4
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   435
    Json = 5
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   436
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   437
def display(fp=None, format=3, data=None, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   438
    '''Print statistics, either to stdout or the given file object.'''
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   439
    data = data or state
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   440
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   441
    if fp is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   442
        import sys
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   443
        fp = sys.stdout
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   444
    if len(data.samples) == 0:
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   445
        print('No samples recorded.', file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   446
        return
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   447
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   448
    if format == DisplayFormats.ByLine:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   449
        display_by_line(data, fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   450
    elif format == DisplayFormats.ByMethod:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   451
        display_by_method(data, fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   452
    elif format == DisplayFormats.AboutMethod:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   453
        display_about_method(data, fp, **kwargs)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   454
    elif format == DisplayFormats.Hotpath:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   455
        display_hotpath(data, fp, **kwargs)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   456
    elif format == DisplayFormats.FlameGraph:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   457
        write_to_flame(data, fp, **kwargs)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   458
    elif format == DisplayFormats.Json:
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   459
        write_to_json(data, fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   460
    else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   461
        raise Exception("Invalid display format")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   462
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   463
    if format != DisplayFormats.Json:
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   464
        print('---', file=fp)
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   465
        print('Sample count: %d' % len(data.samples), file=fp)
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   466
        print('Total time: %f seconds' % data.accumulated_time, file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   467
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   468
def display_by_line(data, fp):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   469
    '''Print the profiler data with each sample line represented
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   470
    as one row in a table.  Sorted by self-time per line.'''
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   471
    stats = SiteStats.buildstats(data.samples)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   472
    stats.sort(reverse=True, key=lambda x: x.selfseconds())
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   473
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   474
    print('%5.5s %10.10s   %7.7s  %-8.8s' %
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   475
          ('%  ', 'cumulative', 'self', ''), file=fp)
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   476
    print('%5.5s  %9.9s  %8.8s  %-8.8s' %
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   477
          ("time", "seconds", "seconds", "name"), file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   478
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   479
    for stat in stats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   480
        site = stat.site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   481
        sitelabel = '%s:%d:%s' % (site.filename(), site.lineno, site.function)
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   482
        print('%6.2f %9.2f %9.2f  %s' % (stat.selfpercent(),
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   483
                                         stat.totalseconds(),
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   484
                                         stat.selfseconds(),
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   485
                                         sitelabel),
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   486
              file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   487
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   488
def display_by_method(data, fp):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   489
    '''Print the profiler data with each sample function represented
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   490
    as one row in a table.  Important lines within that function are
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   491
    output as nested rows.  Sorted by self-time per line.'''
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   492
    print('%5.5s %10.10s   %7.7s  %-8.8s' %
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   493
          ('%  ', 'cumulative', 'self', ''), file=fp)
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   494
    print('%5.5s  %9.9s  %8.8s  %-8.8s' %
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   495
          ("time", "seconds", "seconds", "name"), file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   496
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   497
    stats = SiteStats.buildstats(data.samples)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   498
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   499
    grouped = defaultdict(list)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   500
    for stat in stats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   501
        grouped[stat.site.filename() + ":" + stat.site.function].append(stat)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   502
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   503
    # compute sums for each function
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   504
    functiondata = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   505
    for fname, sitestats in grouped.iteritems():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   506
        total_cum_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   507
        total_self_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   508
        total_percent = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   509
        for stat in sitestats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   510
            total_cum_sec += stat.totalseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   511
            total_self_sec += stat.selfseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   512
            total_percent += stat.selfpercent()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   513
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   514
        functiondata.append((fname,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   515
                             total_cum_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   516
                             total_self_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   517
                             total_percent,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   518
                             sitestats))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   519
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   520
    # sort by total self sec
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   521
    functiondata.sort(reverse=True, key=lambda x: x[2])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   522
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   523
    for function in functiondata:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   524
        if function[3] < 0.05:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   525
            continue
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   526
        print('%6.2f %9.2f %9.2f  %s' % (function[3], # total percent
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   527
                                         function[1], # total cum sec
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   528
                                         function[2], # total self sec
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   529
                                         function[0]), # file:function
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   530
              file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   531
        function[4].sort(reverse=True, key=lambda i: i.selfseconds())
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   532
        for stat in function[4]:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   533
            # only show line numbers for significant locations (>1% time spent)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   534
            if stat.selfpercent() > 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   535
                source = stat.site.getsource(25)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   536
                stattuple = (stat.selfpercent(), stat.selfseconds(),
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   537
                             stat.site.lineno, source)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   538
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   539
                print('%33.0f%% %6.2f   line %s: %s' % (stattuple), file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   540
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   541
def display_about_method(data, fp, function=None, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   542
    if function is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   543
        raise Exception("Invalid function")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   544
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   545
    filename = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   546
    if ':' in function:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   547
        filename, function = function.split(':')
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   548
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   549
    relevant_samples = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   550
    parents = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   551
    children = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   552
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   553
    for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   554
        for i, site in enumerate(sample.stack):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   555
            if site.function == function and (not filename
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   556
                or site.filename() == filename):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   557
                relevant_samples += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   558
                if i != len(sample.stack) - 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   559
                    parent = sample.stack[i + 1]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   560
                    if parent in parents:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   561
                        parents[parent] = parents[parent] + 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   562
                    else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   563
                        parents[parent] = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   564
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   565
                if site in children:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   566
                    children[site] = children[site] + 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   567
                else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   568
                    children[site] = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   569
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   570
    parents = [(parent, count) for parent, count in parents.iteritems()]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   571
    parents.sort(reverse=True, key=lambda x: x[1])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   572
    for parent, count in parents:
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   573
        print('%6.2f%%   %s:%s   line %s: %s' %
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   574
            (count / relevant_samples * 100, parent.filename(),
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   575
            parent.function, parent.lineno, parent.getsource(50)), file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   576
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   577
    stats = SiteStats.buildstats(data.samples)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   578
    stats = [s for s in stats
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   579
               if s.site.function == function and
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   580
               (not filename or s.site.filename() == filename)]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   581
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   582
    total_cum_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   583
    total_self_sec = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   584
    total_self_percent = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   585
    total_cum_percent = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   586
    for stat in stats:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   587
        total_cum_sec += stat.totalseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   588
        total_self_sec += stat.selfseconds()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   589
        total_self_percent += stat.selfpercent()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   590
        total_cum_percent += stat.totalpercent()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   591
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   592
    print(
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   593
        '\n    %s:%s    Total: %0.2fs (%0.2f%%)    Self: %0.2fs (%0.2f%%)\n' %
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   594
        (
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   595
        filename or '___',
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   596
        function,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   597
        total_cum_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   598
        total_cum_percent,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   599
        total_self_sec,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   600
        total_self_percent
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   601
        ), file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   602
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   603
    children = [(child, count) for child, count in children.iteritems()]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   604
    children.sort(reverse=True, key=lambda x: x[1])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   605
    for child, count in children:
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   606
        print('        %6.2f%%   line %s: %s' %
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   607
              (count / relevant_samples * 100, child.lineno,
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   608
               child.getsource(50)), file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   609
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   610
def display_hotpath(data, fp, limit=0.05, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   611
    class HotNode(object):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   612
        def __init__(self, site):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   613
            self.site = site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   614
            self.count = 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   615
            self.children = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   616
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   617
        def add(self, stack, time):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   618
            self.count += time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   619
            site = stack[0]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   620
            child = self.children.get(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   621
            if not child:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   622
                child = HotNode(site)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   623
                self.children[site] = child
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   624
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   625
            if len(stack) > 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   626
                i = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   627
                # Skip boiler plate parts of the stack
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   628
                while i < len(stack) and '%s:%s' % (stack[i].filename(), stack[i].function) in skips:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   629
                    i += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   630
                if i < len(stack):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   631
                    child.add(stack[i:], time)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   632
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   633
    root = HotNode(None)
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   634
    lasttime = data.samples[0].time
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   635
    for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   636
        root.add(sample.stack[::-1], sample.time - lasttime)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   637
        lasttime = sample.time
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   638
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   639
    def _write(node, depth, multiple_siblings):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   640
        site = node.site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   641
        visiblechildren = [c for c in node.children.itervalues()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   642
                             if c.count >= (limit * root.count)]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   643
        if site:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   644
            indent = depth * 2 - 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   645
            filename = ''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   646
            function = ''
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   647
            if len(node.children) > 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   648
                childsite = list(node.children.itervalues())[0].site
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   649
                filename = (childsite.filename() + ':').ljust(15)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   650
                function = childsite.function
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   651
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   652
            # lots of string formatting
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   653
            listpattern = ''.ljust(indent) +\
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   654
                          ('\\' if multiple_siblings else '|') +\
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   655
                          ' %4.1f%%  %s %s'
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   656
            liststring = listpattern % (node.count / root.count * 100,
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   657
                                        filename, function)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   658
            codepattern = '%' + str(55 - len(liststring)) + 's %s:  %s'
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   659
            codestring = codepattern % ('line', site.lineno, site.getsource(30))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   660
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   661
            finalstring = liststring + codestring
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   662
            childrensamples = sum([c.count for c in node.children.itervalues()])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   663
            # Make frames that performed more than 10% of the operation red
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   664
            if node.count - childrensamples > (0.1 * root.count):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   665
                finalstring = '\033[91m' + finalstring + '\033[0m'
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   666
            # Make frames that didn't actually perform work dark grey
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   667
            elif node.count - childrensamples == 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   668
                finalstring = '\033[90m' + finalstring + '\033[0m'
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   669
            print(finalstring, file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   670
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   671
        newdepth = depth
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   672
        if len(visiblechildren) > 1 or multiple_siblings:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   673
            newdepth += 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   674
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   675
        visiblechildren.sort(reverse=True, key=lambda x: x.count)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   676
        for child in visiblechildren:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   677
            _write(child, newdepth, len(visiblechildren) > 1)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   678
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   679
    if root.count > 0:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   680
        _write(root, 0, False)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   681
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   682
def write_to_flame(data, fp, scriptpath=None, outputfile=None, **kwargs):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   683
    if scriptpath is None:
30637
344e68882cd3 py3: replace os.environ with encoding.environ (part 4 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30578
diff changeset
   684
        scriptpath = encoding.environ['HOME'] + '/flamegraph.pl'
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   685
    if not os.path.exists(scriptpath):
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   686
        print("error: missing %s" % scriptpath, file=fp)
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   687
        print("get it here: https://github.com/brendangregg/FlameGraph",
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   688
              file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   689
        return
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   690
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   691
    fd, path = tempfile.mkstemp()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   692
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   693
    file = open(path, "w+")
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   694
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   695
    lines = {}
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   696
    for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   697
        sites = [s.function for s in sample.stack]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   698
        sites.reverse()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   699
        line = ';'.join(sites)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   700
        if line in lines:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   701
            lines[line] = lines[line] + 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   702
        else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   703
            lines[line] = 1
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   704
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   705
    for line, count in lines.iteritems():
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   706
        file.write("%s %s\n" % (line, count))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   707
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   708
    file.close()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   709
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   710
    if outputfile is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   711
        outputfile = '~/flamegraph.svg'
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   712
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   713
    os.system("perl ~/flamegraph.pl %s > %s" % (path, outputfile))
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   714
    print("Written to %s" % outputfile, file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   715
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   716
def write_to_json(data, fp):
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   717
    samples = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   718
30258
eea89068a98d statprof: pass data structure to display functions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30257
diff changeset
   719
    for sample in data.samples:
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   720
        stack = []
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   721
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   722
        for frame in sample.stack:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   723
            stack.append((frame.path, frame.lineno, frame.function))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   724
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   725
        samples.append((sample.time, stack))
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   726
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   727
    print(json.dumps(samples), file=fp)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   728
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   729
def printusage():
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   730
    print("""
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   731
The statprof command line allows you to inspect the last profile's results in
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   732
the following forms:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   733
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   734
usage:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   735
    hotpath [-l --limit percent]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   736
        Shows a graph of calls with the percent of time each takes.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   737
        Red calls take over 10%% of the total time themselves.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   738
    lines
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   739
        Shows the actual sampled lines.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   740
    functions
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   741
        Shows the samples grouped by function.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   742
    function [filename:]functionname
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   743
        Shows the callers and callees of a particular function.
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   744
    flame [-s --script-path] [-o --output-file path]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   745
        Writes out a flamegraph to output-file (defaults to ~/flamegraph.svg)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   746
        Requires that ~/flamegraph.pl exist.
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   747
        (Specify alternate script path with --script-path.)""")
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   748
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   749
def main(argv=None):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   750
    if argv is None:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   751
        argv = sys.argv
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   752
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   753
    if len(argv) == 1:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   754
        printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   755
        return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   756
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   757
    displayargs = {}
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   758
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   759
    optstart = 2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   760
    displayargs['function'] = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   761
    if argv[1] == 'hotpath':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   762
        displayargs['format'] = DisplayFormats.Hotpath
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   763
    elif argv[1] == 'lines':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   764
        displayargs['format'] = DisplayFormats.ByLine
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   765
    elif argv[1] == 'functions':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   766
        displayargs['format'] = DisplayFormats.ByMethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   767
    elif argv[1] == 'function':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   768
        displayargs['format'] = DisplayFormats.AboutMethod
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   769
        displayargs['function'] = argv[2]
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   770
        optstart = 3
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   771
    elif argv[1] == 'flame':
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   772
        displayargs['format'] = DisplayFormats.FlameGraph
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   773
    else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   774
        printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   775
        return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   776
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   777
    # process options
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   778
    try:
30578
c6ce11f2ee50 py3: make a bytes version of getopt.getopt()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30568
diff changeset
   779
        opts, args = pycompat.getoptb(sys.argv[optstart:], "hl:f:o:p:",
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   780
                                   ["help", "limit=", "file=", "output-file=", "script-path="])
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   781
    except getopt.error as msg:
30257
7428223ed7c2 statprof: use print function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30256
diff changeset
   782
        print(msg)
30253
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   783
        printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   784
        return 2
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   785
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   786
    displayargs['limit'] = 0.05
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   787
    path = None
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   788
    for o, value in opts:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   789
        if o in ("-l", "--limit"):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   790
            displayargs['limit'] = float(value)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   791
        elif o in ("-f", "--file"):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   792
            path = value
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   793
        elif o in ("-o", "--output-file"):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   794
            displayargs['outputfile'] = value
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   795
        elif o in ("-p", "--script-path"):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   796
            displayargs['scriptpath'] = value
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   797
        elif o in ("-h", "help"):
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   798
            printusage()
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   799
            return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   800
        else:
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   801
            assert False, "unhandled option %s" % o
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   802
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   803
    load_data(path=path)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   804
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   805
    display(**displayargs)
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   806
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   807
    return 0
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   808
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   809
if __name__ == "__main__":
b032a7b676c6 statprof: vendor statprof.py
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   810
    sys.exit(main())