annotate contrib/catapipe.py @ 50444:315f537627c1

hidden: add support for --remote-hidden to HTTP peer Test written by Pierre-Yves David.
author Manuel Jacob <me@manueljacob.de>
date Fri, 05 Apr 2019 01:17:15 +0200
parents 6000f5b25c9b
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
1 #!/usr/bin/env python3
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
2 #
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
3 # Copyright 2018 Google LLC.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
4 #
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
7 """Tool read primitive events from a pipe to produce a catapult trace.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
8
40491
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
9 Usage:
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
10 Terminal 1: $ catapipe.py /tmp/mypipe /tmp/trace.json
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
11 Terminal 2: $ HGCATAPULTSERVERPIPE=/tmp/mypipe hg root
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
12 <ctrl-c catapipe.py in Terminal 1>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
13 $ catapult/tracing/bin/trace2html /tmp/trace.json # produce /tmp/trace.html
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
14 <open trace.html in your browser of choice; the WASD keys are very useful>
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
15 (catapult is located at https://github.com/catapult-project/catapult)
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
16
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
17 For now the event stream supports
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
18
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
19 START $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
20
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
21 and
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
22
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
23 END $SESSIONID ...
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
24
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
25 events. Everything after the SESSIONID (which must not contain spaces)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
26 is used as a label for the event. Events are timestamped as of when
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
27 they arrive in this process and are then used to produce catapult
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
28 traces that can be loaded in Chrome's about:tracing utility. It's
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
29 important that the event stream *into* this process stay simple,
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
30 because we have to emit it from the shell scripts produced by
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
31 run-tests.py.
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
32
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
33 Typically you'll want to place the path to the named pipe in the
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
34 HGCATAPULTSERVERPIPE environment variable, which both run-tests and hg
40491
c311424ea579 catapult: add a bit more documentation on how to use catapult tracing
Kyle Lippincott <spectral@google.com>
parents: 39935
diff changeset
35 understand. To trace *only* run-tests, use HGTESTCATAPULTSERVERPIPE instead.
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
36 """
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
37
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
38 import argparse
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
39 import json
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
40 import os
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
41 import timeit
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
42
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
43 _TYPEMAP = {
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
44 'START': 'B',
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
45 'END': 'E',
42475
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
46 'COUNTER': 'C',
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
47 }
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
48
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
49 _threadmap = {}
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
50
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
51 # Timeit already contains the whole logic about which timer to use based on
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
52 # Python version and OS
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
53 timer = timeit.default_timer
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
54
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
55
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
56 def main():
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
57 parser = argparse.ArgumentParser()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
58 parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
59 'pipe',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
60 type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
61 nargs=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
62 help='Path of named pipe to create and listen on.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
63 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
64 parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
65 'output',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
66 default='trace.json',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
67 type=str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
68 nargs='?',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
69 help='Path of json file to create where the traces ' 'will be stored.',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
70 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
71 parser.add_argument(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
72 '--debug',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
73 default=False,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
74 action='store_true',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
75 help='Print useful debug messages',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
76 )
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
77 args = parser.parse_args()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
78 fn = args.pipe[0]
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
79 os.mkfifo(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
80 try:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
81 with open(fn) as f, open(args.output, 'w') as out:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
82 out.write('[\n')
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
83 start = timer()
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
84 while True:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
85 ev = f.readline().strip()
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
86 if not ev:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
87 continue
39514
b29ec19748a7 contrib: use a monotonic timer in catapipe
Boris Feld <boris.feld@octobus.net>
parents: 39513
diff changeset
88 now = timer()
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
89 if args.debug:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
90 print(ev)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
91 verb, session, label = ev.split(' ', 2)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
92 if session not in _threadmap:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
93 _threadmap[session] = len(_threadmap)
42475
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
94 if verb == 'COUNTER':
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
95 amount, label = label.split(' ', 1)
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
96 payload_args = {'value': int(amount)}
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
97 else:
ff562d711919 catapipe: add support for COUNTER events
Augie Fackler <augie@google.com>
parents: 40491
diff changeset
98 payload_args = {}
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
99 pid = _threadmap[session]
39935
cb9b23ec093c catapipe: fix bug introduced when switching to the timer api
Augie Fackler <augie@google.com>
parents: 39514
diff changeset
100 ts_micros = (now - start) * 1000000
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
101 out.write(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
102 json.dumps(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
103 {
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
104 "name": label,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
105 "cat": "misc",
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
106 "ph": _TYPEMAP[verb],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
107 "ts": ts_micros,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
108 "pid": pid,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
109 "tid": 1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
110 "args": payload_args,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
111 }
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
112 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
113 )
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
114 out.write(',\n')
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
115 finally:
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
116 os.unlink(fn)
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
117
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42475
diff changeset
118
39252
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
119 if __name__ == '__main__':
9a81f126f9fa contrib: new script to read events from a named pipe and emit catapult traces
Augie Fackler <augie@google.com>
parents:
diff changeset
120 main()