annotate contrib/perf-utils/perf-revlog-write-plot.py @ 47514:559aee84b889

dirstate-entry: add a `from_p2` property Lets start to define and use more semantic property. Differential Revision: https://phab.mercurial-scm.org/D10956
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Sat, 03 Jul 2021 04:26:28 +0200
parents c102b704edb5
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
45830
c102b704edb5 global: use python3 in shebangs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
1 #!/usr/bin/env python3
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
2 #
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
3 # Copyright 2018 Paul Morelle <Paul.Morelle@octobus.net>
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
4 #
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
7 #
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
8 # This script use the output of `hg perfrevlogwrite -T json --details` to draw
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
9 # various plot related to write performance in a revlog
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
10 #
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
11 # usage: perf-revlog-write-plot.py details.json
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
12 from __future__ import absolute_import, print_function
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
13 import json
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
14 import re
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
15
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
16 import numpy as np
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
17 import scipy.signal
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
18
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
19 from matplotlib import (
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
20 pyplot as plt,
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
21 ticker as mticker,
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
22 )
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
23
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
24
41190
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
25 def plot(data, title=None):
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
26 items = {}
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
27 re_title = re.compile(r'^revisions #\d+ of \d+, rev (\d+)$')
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
28 for item in data:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
29 m = re_title.match(item['title'])
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
30 if m is None:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
31 continue
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
32
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
33 rev = int(m.group(1))
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
34 items[rev] = item
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
35
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
36 min_rev = min(items.keys())
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
37 max_rev = max(items.keys())
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
38 ary = np.empty((2, max_rev - min_rev + 1))
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
39 for rev, item in items.items():
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
40 ary[0][rev - min_rev] = rev
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
41 ary[1][rev - min_rev] = item['wall']
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
42
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
43 fig = plt.figure()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
44 comb_plt = fig.add_subplot(211)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
45 other_plt = fig.add_subplot(212)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
46
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
47 comb_plt.plot(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
48 ary[0], np.cumsum(ary[1]), color='red', linewidth=1, label='comb'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
49 )
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
50
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
51 plots = []
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
52 p = other_plt.plot(ary[0], ary[1], color='red', linewidth=1, label='wall')
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
53 plots.append(p)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
54
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
55 colors = {
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
56 10: ('green', 'xkcd:grass green'),
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
57 100: ('blue', 'xkcd:bright blue'),
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
58 1000: ('purple', 'xkcd:dark pink'),
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
59 }
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
60 for n, color in colors.items():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
61 avg_n = np.convolve(ary[1], np.full(n, 1.0 / n), 'valid')
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
62 p = other_plt.plot(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
63 ary[0][n - 1 :],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
64 avg_n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
65 color=color[0],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
66 linewidth=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
67 label='avg time last %d' % n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
68 )
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
69 plots.append(p)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
70
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
71 med_n = scipy.signal.medfilt(ary[1], n + 1)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
72 p = other_plt.plot(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
73 ary[0],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
74 med_n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
75 color=color[1],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
76 linewidth=1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
77 label='median time last %d' % n,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
78 )
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
79 plots.append(p)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
80
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
81 formatter = mticker.ScalarFormatter()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
82 formatter.set_scientific(False)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
83 formatter.set_useOffset(False)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
84
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
85 comb_plt.grid()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
86 comb_plt.xaxis.set_major_formatter(formatter)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
87 comb_plt.legend()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
88
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
89 other_plt.grid()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
90 other_plt.xaxis.set_major_formatter(formatter)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
91 leg = other_plt.legend()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
92 leg2plot = {}
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
93 for legline, plot in zip(leg.get_lines(), plots):
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
94 legline.set_picker(5)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
95 leg2plot[legline] = plot
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
96
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
97 def onpick(event):
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
98 legline = event.artist
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
99 plot = leg2plot[legline]
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
100 visible = not plot[0].get_visible()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
101 for l in plot:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
102 l.set_visible(visible)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
103
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
104 if visible:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
105 legline.set_alpha(1.0)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
106 else:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
107 legline.set_alpha(0.2)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
108 fig.canvas.draw()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41190
diff changeset
109
41190
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
110 if title is not None:
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
111 fig.canvas.set_window_title(title)
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
112 fig.canvas.mpl_connect('pick_event', onpick)
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
113
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
114 plt.show()
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
115
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
116
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
117 if __name__ == '__main__':
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
118 import sys
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
119
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
120 if len(sys.argv) > 1:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
121 print('reading from %r' % sys.argv[1])
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
122 with open(sys.argv[1], 'r') as fp:
41190
c3e5ce3a9483 contrib: update window title when possible in perf-revlog-write-plot.py
Boris Feld <boris.feld@octobus.net>
parents: 40958
diff changeset
123 plot(json.load(fp), title=sys.argv[1])
40958
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
124 else:
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
125 print('reading from stdin')
abd7b75e80bc contrib: provide a small script that draw performance plot
Paul Morelle <paul.morelle@octobus.net>
parents:
diff changeset
126 plot(json.load(sys.stdin))