annotate contrib/catapipe.py @ 46597:305d74c262de

rust: Add config parsing support for more value types * Rust `str` (ASCII or UTF-8) * Integer * Byte quantities Differential Revision: https://phab.mercurial-scm.org/D10008
author Simon Sapin <simon.sapin@octobus.net>
date Tue, 16 Feb 2021 13:55:31 +0100
parents 2372284d9457
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
1 #!/usr/bin/env python3
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
2 #
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
3 # Copyright 2018 Google LLC.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
4 #
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
7 """Tool read primitive events from a pipe to produce a catapult trace.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
8
40491
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
9 Usage:
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
10 Terminal 1: $ catapipe.py /tmp/mypipe /tmp/trace.json
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
11 Terminal 2: $ HGCATAPULTSERVERPIPE=/tmp/mypipe hg root
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
12 <ctrl-c catapipe.py in Terminal 1>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
13 $ catapult/tracing/bin/trace2html /tmp/trace.json # produce /tmp/trace.html
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
14 <open trace.html in your browser of choice; the WASD keys are very useful>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
15 (catapult is located at https://github.com/catapult-project/catapult)
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
16
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
17 For now the event stream supports
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
18
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
19 START $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
20
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
21 and
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
22
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
23 END $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
24
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
25 events. Everything after the SESSIONID (which must not contain spaces)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
26 is used as a label for the event. Events are timestamped as of when
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
27 they arrive in this process and are then used to produce catapult
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
28 traces that can be loaded in Chrome's about:tracing utility. It's
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
29 important that the event stream *into* this process stay simple,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
30 because we have to emit it from the shell scripts produced by
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
31 run-tests.py.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
32
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
33 Typically you'll want to place the path to the named pipe in the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
34 HGCATAPULTSERVERPIPE environment variable, which both run-tests and hg
40491
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
35 understand. To trace *only* run-tests, use HGTESTCATAPULTSERVERPIPE instead.
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
36 """
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
37 from __future__ import absolute_import, print_function
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
38
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
39 import argparse
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
40 import json
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
41 import os
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
42 import timeit
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
43
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
44 _TYPEMAP = {
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
45 'START': 'B',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
46 'END': 'E',
42475
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
47 'COUNTER': 'C',
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
48 }
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
49
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
50 _threadmap = {}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
51
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
52 # Timeit already contains the whole logic about which timer to use based on
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
53 # Python version and OS
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
54 timer = timeit.default_timer
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
55
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
56
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
57 def main():
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
58 parser = argparse.ArgumentParser()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
59 parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
60 'pipe',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
61 type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
62 nargs=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
63 help='Path of named pipe to create and listen on.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
64 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
65 parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
66 'output',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
67 default='trace.json',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
68 type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
69 nargs='?',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
70 help='Path of json file to create where the traces ' 'will be stored.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
71 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
72 parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
73 '--debug',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
74 default=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
75 action='store_true',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
76 help='Print useful debug messages',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
77 )
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
78 args = parser.parse_args()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
79 fn = args.pipe[0]
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
80 os.mkfifo(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
81 try:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
82 with open(fn) as f, open(args.output, 'w') as out:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
83 out.write('[\n')
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
84 start = timer()
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
85 while True:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
86 ev = f.readline().strip()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
87 if not ev:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
88 continue
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
89 now = timer()
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
90 if args.debug:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
91 print(ev)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
92 verb, session, label = ev.split(' ', 2)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
93 if session not in _threadmap:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
94 _threadmap[session] = len(_threadmap)
42475
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
95 if verb == 'COUNTER':
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
96 amount, label = label.split(' ', 1)
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
97 payload_args = {'value': int(amount)}
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
98 else:
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
99 payload_args = {}
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
100 pid = _threadmap[session]
39935
cb9b23ec093c catapipe: fix bug introduced when switching to the timer api
Augie Fackler <augie@google.com>
parents: 39514
diff changeset
101 ts_micros = (now - start) * 1000000
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
102 out.write(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
103 json.dumps(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
104 {
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
105 "name": label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
106 "cat": "misc",
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
107 "ph": _TYPEMAP[verb],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
108 "ts": ts_micros,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
109 "pid": pid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
110 "tid": 1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
111 "args": payload_args,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
112 }
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
113 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
114 )
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
115 out.write(',\n')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
116 finally:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
117 os.unlink(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
118
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
119
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
120 if __name__ == '__main__':
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
121 main()