contrib/catapipe.py
author Matt Harbison <matt_harbison@yahoo.com>
Mon, 18 Jul 2022 17:19:56 -0400
changeset 49395 747c4fc20886
parent 48875 6000f5b25c9b
permissions -rwxr-xr-x
setup: unconditionally enable the `long-paths-support` option on Windows I don't see anything talking about why this was experimental in the first place, but maybe it was concern about the level of python2 support for it. But now, both `python.exe` and the PyOxidizer build of `hg.exe` have a manifest that enables it, so leaving it off would mean some Mercurial installations could operate on a repo with long paths, and others couldn't. Note that only the wide character functions (XxxW) will have the length restriction lifted. Sadly, distutils applies `/MANIFEST:EMBED` to the linker in a way that can't easily be turned off, so we can't use `/MANIFESTFILE` with `extra_preargs` on `link_executable`. Fortunately, the compiler object provides a path to the `mt.exe` it found during initialization, because the previous incarnation seems to have assumed it is being run within an activated Visual Studio environment. That causes MSYS builds to fail, and probably would have broke the CI environment.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
#!/usr/bin/env python3
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
#
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
# Copyright 2018 Google LLC.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
#
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
"""Tool read primitive events from a pipe to produce a catapult trace.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
40491
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
     9
Usage:
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    10
    Terminal 1: $ catapipe.py /tmp/mypipe /tmp/trace.json
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    11
    Terminal 2: $ HGCATAPULTSERVERPIPE=/tmp/mypipe hg root
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    12
    <ctrl-c catapipe.py in Terminal 1>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    13
    $ catapult/tracing/bin/trace2html /tmp/trace.json  # produce /tmp/trace.html
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    14
    <open trace.html in your browser of choice; the WASD keys are very useful>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    15
    (catapult is located at https://github.com/catapult-project/catapult)
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    16
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
For now the event stream supports
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
  START $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
and
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    23
  END $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    24
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    25
events. Everything after the SESSIONID (which must not contain spaces)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
is used as a label for the event. Events are timestamped as of when
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    27
they arrive in this process and are then used to produce catapult
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    28
traces that can be loaded in Chrome's about:tracing utility. It's
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    29
important that the event stream *into* this process stay simple,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    30
because we have to emit it from the shell scripts produced by
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
run-tests.py.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
Typically you'll want to place the path to the named pipe in the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
HGCATAPULTSERVERPIPE environment variable, which both run-tests and hg
40491
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
    35
understand. To trace *only* run-tests, use HGTESTCATAPULTSERVERPIPE instead.
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
"""
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    37
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    38
import argparse
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    39
import json
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    40
import os
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
    41
import timeit
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    42
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    43
_TYPEMAP = {
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    44
    'START': 'B',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    45
    'END': 'E',
42475
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
    46
    'COUNTER': 'C',
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    47
}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    48
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    49
_threadmap = {}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    50
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
    51
# Timeit already contains the whole logic about which timer to use based on
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
    52
# Python version and OS
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
    53
timer = timeit.default_timer
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
    54
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    55
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    56
def main():
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    57
    parser = argparse.ArgumentParser()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    58
    parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    59
        'pipe',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    60
        type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    61
        nargs=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    62
        help='Path of named pipe to create and listen on.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    63
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    64
    parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    65
        'output',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    66
        default='trace.json',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    67
        type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    68
        nargs='?',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    69
        help='Path of json file to create where the traces ' 'will be stored.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    70
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    71
    parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    72
        '--debug',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    73
        default=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    74
        action='store_true',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    75
        help='Print useful debug messages',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
    76
    )
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    77
    args = parser.parse_args()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    78
    fn = args.pipe[0]
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    79
    os.mkfifo(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    80
    try:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    81
        with open(fn) as f, open(args.output, 'w') as out:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    82
            out.write('[\n')
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
    83
            start = timer()
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    84
            while True:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    85
                ev = f.readline().strip()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    86
                if not ev:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    87
                    continue
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
    88
                now = timer()
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    89
                if args.debug:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    90
                    print(ev)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    91
                verb, session, label = ev.split(' ', 2)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    92
                if session not in _threadmap:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    93
                    _threadmap[session] = len(_threadmap)
42475
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
    94
                if verb == 'COUNTER':
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
    95
                    amount, label = label.split(' ', 1)
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
    96
                    payload_args = {'value': int(amount)}
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
    97
                else:
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
    98
                    payload_args = {}
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    99
                pid = _threadmap[session]
39935
cb9b23ec093c catapipe: fix bug introduced when switching to the timer api
Augie Fackler <augie@google.com>
parents: 39514
diff changeset
   100
                ts_micros = (now - start) * 1000000
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   101
                out.write(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   102
                    json.dumps(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   103
                        {
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   104
                            "name": label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   105
                            "cat": "misc",
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   106
                            "ph": _TYPEMAP[verb],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   107
                            "ts": ts_micros,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   108
                            "pid": pid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   109
                            "tid": 1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   110
                            "args": payload_args,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   111
                        }
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   112
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   113
                )
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   114
                out.write(',\n')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   115
    finally:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   116
        os.unlink(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   117
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
   118
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   119
if __name__ == '__main__':
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   120
    main()