contrib/catapipe.py
author Raphaël Gomès <rgomes@octobus.net>
Thu, 16 Jun 2022 15:09:59 +0200
branchstable
changeset 48896 48510535dadb
parent 43076 2372284d9457
child 48966 6000f5b25c9b
permissions -rwxr-xr-x
Added signature for changeset 0cc5f74ff7f0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
#!/usr/bin/env python3
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
#
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
# Copyright 2018 Google LLC.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
#
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
"""Tool read primitive events from a pipe to produce a catapult trace.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
40541
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
     9
Usage:
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    10
    Terminal 1: $ catapipe.py /tmp/mypipe /tmp/trace.json
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    11
    Terminal 2: $ HGCATAPULTSERVERPIPE=/tmp/mypipe hg root
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    12
    <ctrl-c catapipe.py in Terminal 1>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    13
    $ catapult/tracing/bin/trace2html /tmp/trace.json  # produce /tmp/trace.html
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    14
    <open trace.html in your browser of choice; the WASD keys are very useful>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    15
    (catapult is located at https://github.com/catapult-project/catapult)
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    16
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
For now the event stream supports
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
  START $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
and
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    23
  END $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    24
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    25
events. Everything after the SESSIONID (which must not contain spaces)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
is used as a label for the event. Events are timestamped as of when
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    27
they arrive in this process and are then used to produce catapult
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    28
traces that can be loaded in Chrome's about:tracing utility. It's
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    29
important that the event stream *into* this process stay simple,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    30
because we have to emit it from the shell scripts produced by
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
run-tests.py.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
Typically you'll want to place the path to the named pipe in the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
HGCATAPULTSERVERPIPE environment variable, which both run-tests and hg
40541
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39940
diff changeset
    35
understand. To trace *only* run-tests, use HGTESTCATAPULTSERVERPIPE instead.
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
"""
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    37
from __future__ import absolute_import, print_function
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    38
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    39
import argparse
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    40
import json
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    41
import os
39531
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39530
diff changeset
    42
import timeit
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    43
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    44
_TYPEMAP = {
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    45
    'START': 'B',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    46
    'END': 'E',
42491
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40541
diff changeset
    47
    'COUNTER': 'C',
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    48
}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    49
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    50
_threadmap = {}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    51
39531
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39530
diff changeset
    52
# Timeit already contains the whole logic about which timer to use based on
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39530
diff changeset
    53
# Python version and OS
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39530
diff changeset
    54
timer = timeit.default_timer
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39530
diff changeset
    55
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    56
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    57
def main():
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    58
    parser = argparse.ArgumentParser()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    59
    parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    60
        'pipe',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    61
        type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    62
        nargs=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    63
        help='Path of named pipe to create and listen on.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    64
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    65
    parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    66
        'output',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    67
        default='trace.json',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    68
        type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    69
        nargs='?',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    70
        help='Path of json file to create where the traces ' 'will be stored.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    71
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    72
    parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    73
        '--debug',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    74
        default=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    75
        action='store_true',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    76
        help='Print useful debug messages',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
    77
    )
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    78
    args = parser.parse_args()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    79
    fn = args.pipe[0]
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    80
    os.mkfifo(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    81
    try:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    82
        with open(fn) as f, open(args.output, 'w') as out:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    83
            out.write('[\n')
39531
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39530
diff changeset
    84
            start = timer()
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    85
            while True:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    86
                ev = f.readline().strip()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    87
                if not ev:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    88
                    continue
39531
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39530
diff changeset
    89
                now = timer()
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    90
                if args.debug:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    91
                    print(ev)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    92
                verb, session, label = ev.split(' ', 2)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    93
                if session not in _threadmap:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
    94
                    _threadmap[session] = len(_threadmap)
42491
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40541
diff changeset
    95
                if verb == 'COUNTER':
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40541
diff changeset
    96
                    amount, label = label.split(' ', 1)
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40541
diff changeset
    97
                    payload_args = {'value': int(amount)}
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40541
diff changeset
    98
                else:
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40541
diff changeset
    99
                    payload_args = {}
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   100
                pid = _threadmap[session]
39940
cb9b23ec093c catapipe: fix bug introduced when switching to the timer api
Augie Fackler <augie@google.com>
parents: 39531
diff changeset
   101
                ts_micros = (now - start) * 1000000
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   102
                out.write(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   103
                    json.dumps(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   104
                        {
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   105
                            "name": label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   106
                            "cat": "misc",
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   107
                            "ph": _TYPEMAP[verb],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   108
                            "ts": ts_micros,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   109
                            "pid": pid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   110
                            "tid": 1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   111
                            "args": payload_args,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   112
                        }
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   113
                    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   114
                )
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   115
                out.write(',\n')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   116
    finally:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   117
        os.unlink(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   118
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42491
diff changeset
   119
39280
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   120
if __name__ == '__main__':
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
   121
    main()