Mercurial > hg
annotate contrib/hgperf @ 43044:f9d35f01b8b3
setup: build extensions in parallel by default
The build_ext distutils command in Python 3.5+ has a "parallel"
option that controls whether to build extensions in parallel. It
is disabled by default (None) and can be set to an integer value
for number of cores or True to indicate use all available CPU
cores.
This commit changes our build_ext command override to set
"parallel" to True unless a value has been provided by the caller.
On my machine, this makes `python setup.py build_ext` 1-4s faster.
It is worth noting that at this time, each individual source file
constituting the extension is still built serially. For Mercurial,
this means that we can't build faster than the slowest-to-build
extension, which is the zstd extension by a long shot. This means
that setup.py is still not very efficient at utilizing multiple
cores. But we're better than before.
Differential Revision: https://phab.mercurial-scm.org/D6923
# no-check-commit because of foo_bar naming
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Mon, 30 Sep 2019 17:26:41 -0700 |
parents | 163fa0aea71e |
children | 99e231afc29c |
rev | line source |
---|---|
20839
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
1 #!/usr/bin/env python |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
2 # |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
3 # hgperf - measure performance of Mercurial commands |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
4 # |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
5 # Copyright 2014 Matt Mackall <mpm@selenic.com> |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
6 # |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
7 # This software may be used and distributed according to the terms of the |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
8 # GNU General Public License version 2 or any later version. |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
9 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
10 '''measure performance of Mercurial commands |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
11 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
12 Using ``hgperf`` instead of ``hg`` measures performance of the target |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
13 Mercurial command. For example, the execution below measures |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
14 performance of :hg:`heads --topo`:: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
15 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
16 $ hgperf heads --topo |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
17 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
18 All command output via ``ui`` is suppressed, and just measurement |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
19 result is displayed: see also "perf" extension in "contrib". |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
20 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
21 Costs of processing before dispatching to the command function like |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
22 below are not measured:: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
23 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
24 - parsing command line (e.g. option validity check) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
25 - reading configuration files in |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
26 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
27 But ``pre-`` and ``post-`` hook invocation for the target command is |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
28 measured, even though these are invoked before or after dispatching to |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
29 the command function, because these may be required to repeat |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
30 execution of the target command correctly. |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
31 ''' |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
32 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
33 import os |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
34 import sys |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
35 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
36 libdir = '@LIBDIR@' |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
37 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
38 if libdir != '@' 'LIBDIR' '@': |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
39 if not os.path.isabs(libdir): |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
40 libdir = os.path.join(os.path.dirname(os.path.realpath(__file__)), |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
41 libdir) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
42 libdir = os.path.abspath(libdir) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
43 sys.path.insert(0, libdir) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
44 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
45 # enable importing on demand to reduce startup time |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
46 try: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
47 from mercurial import demandimport; demandimport.enable() |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
48 except ImportError: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
49 import sys |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
50 sys.stderr.write("abort: couldn't find mercurial libraries in [%s]\n" % |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
51 ' '.join(sys.path)) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
52 sys.stderr.write("(check your install and PYTHONPATH)\n") |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
53 sys.exit(-1) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
54 |
33892
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
55 from mercurial import ( |
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
56 dispatch, |
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
57 util, |
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
58 ) |
20839
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
59 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
60 def timer(func, title=None): |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
61 results = [] |
33892
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
62 begin = util.timer() |
20839
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
63 count = 0 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
64 while True: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
65 ostart = os.times() |
33892
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
66 cstart = util.timer() |
20839
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
67 r = func() |
33892
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
68 cstop = util.timer() |
20839
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
69 ostop = os.times() |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
70 count += 1 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
71 a, b = ostart, ostop |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
72 results.append((cstop - cstart, b[0] - a[0], b[1]-a[1])) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
73 if cstop - begin > 3 and count >= 100: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
74 break |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
75 if cstop - begin > 10 and count >= 3: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
76 break |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
77 if title: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
78 sys.stderr.write("! %s\n" % title) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
79 if r: |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
80 sys.stderr.write("! result: %s\n" % r) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
81 m = min(results) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
82 sys.stderr.write("! wall %f comb %f user %f sys %f (best of %d)\n" |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
83 % (m[0], m[1] + m[2], m[1], m[2], count)) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
84 |
33892
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
85 orgruncommand = dispatch.runcommand |
20839
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
86 |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
87 def runcommand(lui, repo, cmd, fullargs, ui, options, d, cmdpats, cmdoptions): |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
88 ui.pushbuffer() |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
89 lui.pushbuffer() |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
90 timer(lambda : orgruncommand(lui, repo, cmd, fullargs, ui, |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
91 options, d, cmdpats, cmdoptions)) |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
92 ui.popbuffer() |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
93 lui.popbuffer() |
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
94 |
33892
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
95 dispatch.runcommand = runcommand |
20839
377a111d1cd2
contrib: add "hgperf" command to measure performance of commands easily
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff
changeset
|
96 |
33892
78f644fdaa2a
hgperf: update to conform with import style checks
Augie Fackler <raf@durin42.com>
parents:
30975
diff
changeset
|
97 dispatch.run() |