mercurial/profiling.py
author Joerg Sonnenberger <joerg@bec.de>
Mon, 08 Jul 2024 22:46:04 +0200
changeset 51896 8583d138f436
parent 51863 f4733654f144
child 51897 499b19683c1b
permissions -rw-r--r--
exchange: improve computation of relevant markers for large repos Compute the candidate nodes with relevant markers directly from keys of the predecessors/successors/children dictionaries of obsstore. This is faster than iterating over all nodes directly. This test could be further improved for repositories with relative few markers compared to the repository size, but this is no longer hot already. With the current loop structure, the obshashrange use works as well as before as it passes lists with a single node. Adjust the interface by allowing revision lists as well as node lists. This helps cases that computes ancestors as it reduces the materialisation cost. Use this in _pushdiscoveryobsmarker and _getbundleobsmarkerpart. Improve the latter further by directly using ancestors(). Performance benchmarks show notable and welcome improvement to no-op push and pull (that would also apply to other push/pull). This apply to push and pull done without evolve. ### push/pull Benchmark parameter # bin-env-vars.hg.flavor = default # benchmark.variants.explicit-rev = none # benchmark.variants.protocol = ssh # benchmark.variants.revs = none ## benchmark.name = hg.command.pull # data-env-vars.name = mercurial-devel-2024-03-22-zstd-sparse-revlog before: 5.968537 seconds after: 5.668507 seconds (-5.03%, -0.30) # data-env-vars.name = tryton-devel-2024-03-22-zstd-sparse-revlog before: 1.446232 seconds after: 0.835553 seconds (-42.23%, -0.61) # data-env-vars.name = netbsd-src-draft-2024-09-19-zstd-sparse-revlog before: 5.777412 seconds after: 2.523454 seconds (-56.32%, -3.25) ## benchmark.name = hg.command.push # data-env-vars.name = mercurial-devel-2024-03-22-zstd-sparse-revlog before: 6.155501 seconds after: 5.885072 seconds (-4.39%, -0.27) # data-env-vars.name = tryton-devel-2024-03-22-zstd-sparse-revlog before: 1.491054 seconds after: 0.934882 seconds (-37.30%, -0.56) # data-env-vars.name = netbsd-src-draft-2024-09-19-zstd-sparse-revlog before: 5.902494 seconds after: 2.957644 seconds (-49.89%, -2.94) There is not notable different in these result using the "rust" flavor instead of the "default". The performance impact on the same operation when using evolve were also tested and no impact was noted.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
# profiling.py - profiling functions
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
#
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
# Copyright 2016 Gregory Szorc <gregory.szorc@gmail.com>
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     4
#
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     7
51863
f4733654f144 typing: add `from __future__ import annotations` to most files
Matt Harbison <matt_harbison@yahoo.com>
parents: 51842
diff changeset
     8
from __future__ import annotations
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     9
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
    10
import contextlib
51761
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
    11
import os
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
    12
import signal
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
    13
import subprocess
51840
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    14
import sys
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    15
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    16
from .i18n import _
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
    17
from .pycompat import (
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
    18
    open,
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43085
diff changeset
    19
)
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    20
from . import (
30820
6a70cf94d1b5 py3: replace pycompat.getenv with encoding.environ.get
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30664
diff changeset
    21
    encoding,
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    22
    error,
32417
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    23
    extensions,
36683
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
    24
    pycompat,
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    25
    util,
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    26
)
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    27
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    28
32417
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    29
def _loadprofiler(ui, profiler):
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    30
    """load profiler extension. return profile method, or None on failure"""
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    31
    extname = profiler
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    32
    extensions.loadall(ui, whitelist=[extname])
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    33
    try:
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    34
        mod = extensions.find(extname)
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    35
    except KeyError:
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    36
        return None
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    37
    else:
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    38
        return getattr(mod, 'profile', None)
f40dc6f7c12f profiling: allow loading profiling extension before everything else
Jun Wu <quark@fb.com>
parents: 30975
diff changeset
    39
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    40
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
    41
@contextlib.contextmanager
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
    42
def lsprofile(ui, fp):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    43
    format = ui.config(b'profiling', b'format')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    44
    field = ui.config(b'profiling', b'sort')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    45
    limit = ui.configint(b'profiling', b'limit')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    46
    climit = ui.configint(b'profiling', b'nested')
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    47
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    48
    if format not in [b'text', b'kcachegrind']:
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43089
diff changeset
    49
        ui.warn(_(b"unrecognized profiling format '%s' - Ignored\n") % format)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    50
        format = b'text'
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    51
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    52
    try:
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    53
        from . import lsprof
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    54
    except ImportError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    55
        raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    56
            _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    57
                b'lsprof not available - install from '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    58
                b'http://codespeak.net/svn/user/arigo/hack/misc/lsprof/'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    59
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    60
        )
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    61
    p = lsprof.Profiler()
51840
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    62
    try:
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    63
        p.enable(subcalls=True)
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    64
    except ValueError as exc:
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    65
        if str(exc) != "Another profiling tool is already active":
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    66
            raise
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    67
        if not hasattr(sys, "monitoring"):
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    68
            raise
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    69
        # python >=3.12 prevent more than one profiler to run at the same
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    70
        # time, tries to improve the report to help the user understand
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    71
        # what is going on.
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    72
        other_tool_name = sys.monitoring.get_tool(sys.monitoring.PROFILER_ID)
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    73
        if other_tool_name == "cProfile":
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    74
            msg = 'cannot recursively call `lsprof`'
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    75
            raise error.Abort(msg) from None
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    76
        else:
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    77
            m = 'failed to start "lsprofile"; another profiler already running: %s'
1bb71046f5e0 profiling: improve 3.12 error message for calling lsprof twice
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 51588
diff changeset
    78
            raise error.Abort(_(m) % other_tool_name) from None
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    79
    try:
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
    80
        yield
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    81
    finally:
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    82
        p.disable()
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    83
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    84
        if format == b'kcachegrind':
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    85
            from . import lsprofcalltree
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    86
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    87
            calltree = lsprofcalltree.KCacheGrind(p)
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    88
            calltree.output(fp)
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    89
        else:
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    90
            # format == 'text'
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    91
            stats = lsprof.Stats(p.getstats())
40192
b8f6a99ad89b py3: convert sorting field to sysstr
Gregory Szorc <gregory.szorc@gmail.com>
parents: 38260
diff changeset
    92
            stats.sort(pycompat.sysstr(field))
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    93
            stats.pprint(limit=limit, file=fp, climit=climit)
51588
1574718fa62f profiler: flush after writing the profiler output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
    94
        fp.flush()
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    95
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
    96
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
    97
@contextlib.contextmanager
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
    98
def flameprofile(ui, fp):
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
    try:
43731
6c8ba31405d9 profiling: disable the import-error warning for the flamegraph module
Matt Harbison <matt_harbison@yahoo.com>
parents: 43506
diff changeset
   100
        from flamegraph import flamegraph  # pytype: disable=import-error
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   101
    except ImportError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   102
        raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   103
            _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   104
                b'flamegraph not available - install from '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   105
                b'https://github.com/evanhempel/python-flamegraph'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   106
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   107
        )
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   108
    # developer config: profiling.freq
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   109
    freq = ui.configint(b'profiling', b'freq')
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   110
    filter_ = None
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   111
    collapse_recursion = True
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   112
    thread = flamegraph.ProfileThread(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   113
        fp, 1.0 / freq, filter_, collapse_recursion
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   114
    )
30975
22fbca1d11ed mercurial: switch to util.timer for all interval timings
Simon Farnsworth <simonfar@fb.com>
parents: 30930
diff changeset
   115
    start_time = util.timer()
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   116
    try:
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   117
        thread.start()
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   118
        yield
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   119
    finally:
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   120
        thread.stop()
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   121
        thread.join()
51588
1574718fa62f profiler: flush after writing the profiler output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
   122
        m = b'Collected %d stack frames (%d unique) in %2.2f seconds.'
1574718fa62f profiler: flush after writing the profiler output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
   123
        m %= (
1574718fa62f profiler: flush after writing the profiler output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
   124
            (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   125
                util.timer() - start_time,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   126
                thread.num_frames(),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   127
                thread.num_frames(unique=True),
51588
1574718fa62f profiler: flush after writing the profiler output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
   128
            ),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   129
        )
51588
1574718fa62f profiler: flush after writing the profiler output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
   130
        print(m, flush=True)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   131
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   132
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   133
@contextlib.contextmanager
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   134
def statprofile(ui, fp):
30316
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   135
    from . import statprof
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   136
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   137
    freq = ui.configint(b'profiling', b'freq')
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   138
    if freq > 0:
29785
88d3c1ab03a7 profiling: don't error with statprof when profiling has already started
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29784
diff changeset
   139
        # Cannot reset when profiler is already active. So silently no-op.
88d3c1ab03a7 profiling: don't error with statprof when profiling has already started
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29784
diff changeset
   140
        if statprof.state.profile_level == 0:
88d3c1ab03a7 profiling: don't error with statprof when profiling has already started
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29784
diff changeset
   141
            statprof.reset(freq)
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   142
    else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   143
        ui.warn(_(b"invalid sampling frequency '%s' - ignoring\n") % freq)
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   144
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   145
    track = ui.config(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   146
        b'profiling', b'time-track', pycompat.iswindows and b'cpu' or b'real'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   147
    )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   148
    statprof.start(mechanism=b'thread', track=track)
30316
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   149
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   150
    try:
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   151
        yield
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   152
    finally:
30316
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   153
        data = statprof.stop()
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   154
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   155
        profformat = ui.config(b'profiling', b'statformat')
30316
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   156
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   157
        formats = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   158
            b'byline': statprof.DisplayFormats.ByLine,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   159
            b'bymethod': statprof.DisplayFormats.ByMethod,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   160
            b'hotpath': statprof.DisplayFormats.Hotpath,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   161
            b'json': statprof.DisplayFormats.Json,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   162
            b'chrome': statprof.DisplayFormats.Chrome,
30316
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   163
        }
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   164
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   165
        if profformat in formats:
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   166
            displayformat = formats[profformat]
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   167
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   168
            ui.warn(_(b'unknown profiler output format: %s\n') % profformat)
30316
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   169
            displayformat = statprof.DisplayFormats.Hotpath
faf1b8923da2 profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29785
diff changeset
   170
30930
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   171
        kwargs = {}
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   172
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   173
        def fraction(s):
32978
41b081ac2145 profiling: cope with configwith default value handling changes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32851
diff changeset
   174
            if isinstance(s, (float, int)):
41b081ac2145 profiling: cope with configwith default value handling changes
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32851
diff changeset
   175
                return float(s)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   176
            if s.endswith(b'%'):
30930
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   177
                v = float(s[:-1]) / 100
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   178
            else:
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   179
                v = float(s)
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   180
            if 0 <= v <= 1:
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   181
                return v
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   182
            raise ValueError(s)
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   183
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   184
        if profformat == b'chrome':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   185
            showmin = ui.configwith(fraction, b'profiling', b'showmin', 0.005)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   186
            showmax = ui.configwith(fraction, b'profiling', b'showmax')
30930
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   187
            kwargs.update(minthreshold=showmin, maxthreshold=showmax)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   188
        elif profformat == b'hotpath':
33195
5d8942dbe49e check-config: syntax to allow inconsistent config values
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32978
diff changeset
   189
            # inconsistent config: profiling.showmin
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   190
            limit = ui.configwith(fraction, b'profiling', b'showmin', 0.05)
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   191
            kwargs['limit'] = limit
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   192
            showtime = ui.configbool(b'profiling', b'showtime')
43506
9f70512ae2cf cleanup: remove pointless r-prefixes on single-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   193
            kwargs['showtime'] = showtime
30930
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   194
517bc1cd7033 profiling: add statprof support for Chrome trace viewer rendering
Bryan O'Sullivan <bryano@fb.com>
parents: 30820
diff changeset
   195
        statprof.display(fp, data=data, format=displayformat, **kwargs)
51588
1574718fa62f profiler: flush after writing the profiler output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50929
diff changeset
   196
        fp.flush()
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   197
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   198
51761
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   199
@contextlib.contextmanager
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   200
def pyspy_profile(ui, fp):
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   201
    exe = ui.config(b'profiling', b'py-spy.exe')
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   202
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   203
    freq = ui.configint(b'profiling', b'py-spy.freq')
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   204
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   205
    format = ui.config(b'profiling', b'py-spy.format')
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   206
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   207
    fd = fp.fileno()
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   208
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   209
    output_path = "/dev/fd/%d" % (fd)
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   210
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   211
    my_pid = os.getpid()
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   212
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   213
    cmd = [
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   214
        exe,
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   215
        "record",
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   216
        "--pid",
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   217
        str(my_pid),
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   218
        "--native",
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   219
        "--rate",
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   220
        str(freq),
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   221
        "--output",
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   222
        output_path,
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   223
    ]
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   224
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   225
    if format:
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   226
        cmd.extend(["--format", format])
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   227
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   228
    proc = subprocess.Popen(
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   229
        cmd,
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   230
        pass_fds={fd},
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   231
        stdout=subprocess.PIPE,
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   232
    )
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   233
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   234
    _ = proc.stdout.readline()
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   235
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   236
    try:
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   237
        yield
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   238
    finally:
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   239
        os.kill(proc.pid, signal.SIGINT)
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   240
        proc.communicate()
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   241
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   242
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   243
class profile:
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   244
    """Start profiling.
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   245
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   246
    Profiling is active when the context manager is active. When the context
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   247
    manager exits, profiling results will be written to the configured output.
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   248
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   249
32785
37ec8f24d912 profile: introduce a knob to control if the context is actually profiling
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32784
diff changeset
   250
    def __init__(self, ui, enabled=True):
32783
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   251
        self._ui = ui
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   252
        self._output = None
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   253
        self._fp = None
32805
2b0fc56840d0 profile: use explicit logic to control file closing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32804
diff changeset
   254
        self._fpdoclose = True
44192
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   255
        self._flushfp = None
32783
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   256
        self._profiler = None
32785
37ec8f24d912 profile: introduce a knob to control if the context is actually profiling
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32784
diff changeset
   257
        self._enabled = enabled
32784
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   258
        self._entered = False
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   259
        self._started = False
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   260
32783
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   261
    def __enter__(self):
32784
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   262
        self._entered = True
32785
37ec8f24d912 profile: introduce a knob to control if the context is actually profiling
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32784
diff changeset
   263
        if self._enabled:
37ec8f24d912 profile: introduce a knob to control if the context is actually profiling
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32784
diff changeset
   264
            self.start()
32786
0ead06d54ffe profile: make the contextmanager object available to the callers
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32785
diff changeset
   265
        return self
32784
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   266
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   267
    def start(self):
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   268
        """Start profiling.
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   269
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   270
        The profiling will stop at the context exit.
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   271
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   272
        If the profiler was already started, this has no effect."""
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   273
        if not self._entered:
43732
882e633ac92c profiling: add a missing argument to the ProgrammingError constructor
Matt Harbison <matt_harbison@yahoo.com>
parents: 43731
diff changeset
   274
            raise error.ProgrammingError(b'use a context manager to start')
32784
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   275
        if self._started:
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   276
            return
086c1ef0f666 profile: introduce a "start" method to the profile context
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32783
diff changeset
   277
        self._started = True
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   278
        profiler = encoding.environ.get(b'HGPROF')
32783
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   279
        proffn = None
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   280
        if profiler is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   281
            profiler = self._ui.config(b'profiling', b'type')
51761
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   282
        if profiler not in (b'ls', b'stat', b'flame', b'py-spy'):
32783
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   283
            # try load profiler from extension with the same name
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   284
            proffn = _loadprofiler(self._ui, profiler)
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   285
            if proffn is None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   286
                self._ui.warn(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   287
                    _(b"unrecognized profiler '%s' - ignored\n") % profiler
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   288
                )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   289
                profiler = b'stat'
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   290
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   291
        self._output = self._ui.config(b'profiling', b'output')
29781
2654a0aac80d profiling: move profiling code from dispatch.py (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   292
32808
336700745a5c profile: close 'fp' on error within '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32807
diff changeset
   293
        try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   294
            if self._output == b'blackbox':
32807
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   295
                self._fp = util.stringio()
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   296
            elif self._output:
46958
5a6a1cd21f09 profiling: use `util.expandpath` instead of `ui.expandpath` for output
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44192
diff changeset
   297
                path = util.expandpath(self._output)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   298
                self._fp = open(path, b'wb')
36683
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   299
            elif pycompat.iswindows:
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   300
                # parse escape sequence by win32print()
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   301
                class uifp:
36683
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   302
                    def __init__(self, ui):
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   303
                        self._ui = ui
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   304
36683
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   305
                    def write(self, data):
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   306
                        self._ui.write_err(data)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   307
36683
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   308
                    def flush(self):
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   309
                        self._ui.flush()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   310
36683
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   311
                self._fpdoclose = False
e39953fdd924 profile: colorize output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 36400
diff changeset
   312
                self._fp = uifp(self._ui)
32807
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   313
            else:
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   314
                self._fpdoclose = False
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   315
                self._fp = self._ui.ferr
44192
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   316
                # Ensure we've flushed fout before writing to ferr.
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   317
                self._flushfp = self._ui.fout
32783
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   318
32807
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   319
            if proffn is not None:
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   320
                pass
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   321
            elif profiler == b'ls':
32807
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   322
                proffn = lsprofile
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   323
            elif profiler == b'flame':
32807
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   324
                proffn = flameprofile
51761
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   325
            elif profiler == b'py-spy':
812a094a7477 profiling: add a py-spy profiling backend
Arseniy Alekseyev <aalekseyev@janestreet.com>
parents: 51588
diff changeset
   326
                proffn = pyspy_profile
32807
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   327
            else:
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   328
                proffn = statprofile
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   329
32807
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   330
            self._profiler = proffn(self._ui, self._fp)
54b356d65079 profile: indent part of '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32806
diff changeset
   331
            self._profiler.__enter__()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   332
        except:  # re-raises
32808
336700745a5c profile: close 'fp' on error within '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32807
diff changeset
   333
            self._closefp()
336700745a5c profile: close 'fp' on error within '__enter__'
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32807
diff changeset
   334
            raise
29783
5d44197c208b profiling: make profiling functions context managers (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29781
diff changeset
   335
32783
4483696dacee profile: upgrade the "profile" context manager to a full class
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32417
diff changeset
   336
    def __exit__(self, exception_type, exception_value, traceback):
32810
6675d23da748 profile: properly propagate exception from the sub-context manager
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32809
diff changeset
   337
        propagate = None
32809
062eb859d3ee profile: close 'fp' in all cases
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32808
diff changeset
   338
        if self._profiler is not None:
44192
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   339
            self._uiflush()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   340
            propagate = self._profiler.__exit__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   341
                exception_type, exception_value, traceback
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42423
diff changeset
   342
            )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   343
            if self._output == b'blackbox':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   344
                val = b'Profile:\n%s' % self._fp.getvalue()
32809
062eb859d3ee profile: close 'fp' in all cases
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32808
diff changeset
   345
                # ui.log treats the input as a format string,
062eb859d3ee profile: close 'fp' in all cases
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32808
diff changeset
   346
                # so we need to escape any % signs.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   347
                val = val.replace(b'%', b'%%')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   348
                self._ui.log(b'profile', val)
32806
3a4c677cbd6e profile: remove now useless indent
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32805
diff changeset
   349
        self._closefp()
32810
6675d23da748 profile: properly propagate exception from the sub-context manager
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32809
diff changeset
   350
        return propagate
32804
c0b2c8f25ad9 profiling: move 'fp' closing logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32788
diff changeset
   351
c0b2c8f25ad9 profiling: move 'fp' closing logic into its own function
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32788
diff changeset
   352
    def _closefp(self):
32805
2b0fc56840d0 profile: use explicit logic to control file closing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32804
diff changeset
   353
        if self._fpdoclose and self._fp is not None:
2b0fc56840d0 profile: use explicit logic to control file closing
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32804
diff changeset
   354
            self._fp.close()
44192
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   355
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   356
    def _uiflush(self):
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   357
        if self._flushfp:
d6d4170882cd profiling: flush stdout before writing profile to stderr
Kyle Lippincott <spectral@google.com>
parents: 43732
diff changeset
   358
            self._flushfp.flush()