Mercurial > hg
annotate tests/test-revlog-mmapindex.t @ 43044:f9d35f01b8b3
setup: build extensions in parallel by default
The build_ext distutils command in Python 3.5+ has a "parallel"
option that controls whether to build extensions in parallel. It
is disabled by default (None) and can be set to an integer value
for number of cores or True to indicate use all available CPU
cores.
This commit changes our build_ext command override to set
"parallel" to True unless a value has been provided by the caller.
On my machine, this makes `python setup.py build_ext` 1-4s faster.
It is worth noting that at this time, each individual source file
constituting the extension is still built serially. For Mercurial,
this means that we can't build faster than the slowest-to-build
extension, which is the zstd extension by a long shot. This means
that setup.py is still not very efficient at utilizing multiple
cores. But we're better than before.
Differential Revision: https://phab.mercurial-scm.org/D6923
# no-check-commit because of foo_bar naming
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Mon, 30 Sep 2019 17:26:41 -0700 |
parents | 6ebe899b6551 |
children | 42d2b31cee0b |
rev | line source |
---|---|
34296
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
1 create verbosemmap.py |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
2 $ cat << EOF > verbosemmap.py |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
3 > # extension to make util.mmapread verbose |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
4 > |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
5 > from __future__ import absolute_import |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
6 > |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
7 > from mercurial import ( |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
8 > extensions, |
35404
6ebe899b6551
py3: pass bytes from extension to ui.write() in test-revlog-mmapindex.t
Pulkit Goyal <7895pulkit@gmail.com>
parents:
35393
diff
changeset
|
9 > pycompat, |
34296
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
10 > util, |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
11 > ) |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
12 > |
34446
b0c97e44576f
test-revlog-mmapindex: make it compatible with chg
Jun Wu <quark@fb.com>
parents:
34296
diff
changeset
|
13 > def extsetup(ui): |
b0c97e44576f
test-revlog-mmapindex: make it compatible with chg
Jun Wu <quark@fb.com>
parents:
34296
diff
changeset
|
14 > def mmapread(orig, fp): |
35404
6ebe899b6551
py3: pass bytes from extension to ui.write() in test-revlog-mmapindex.t
Pulkit Goyal <7895pulkit@gmail.com>
parents:
35393
diff
changeset
|
15 > ui.write(b"mmapping %s\n" % pycompat.bytestr(fp.name)) |
34446
b0c97e44576f
test-revlog-mmapindex: make it compatible with chg
Jun Wu <quark@fb.com>
parents:
34296
diff
changeset
|
16 > ui.flush() |
b0c97e44576f
test-revlog-mmapindex: make it compatible with chg
Jun Wu <quark@fb.com>
parents:
34296
diff
changeset
|
17 > return orig(fp) |
34296
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
18 > |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
19 > extensions.wrapfunction(util, 'mmapread', mmapread) |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
20 > EOF |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
21 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
22 setting up base repo |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
23 $ hg init a |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
24 $ cd a |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
25 $ touch a |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
26 $ hg add a |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
27 $ hg commit -qm base |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
28 $ for i in `$TESTDIR/seq.py 1 100` ; do |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
29 > echo $i > a |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
30 > hg commit -qm $i |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
31 > done |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
32 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
33 set up verbosemmap extension |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
34 $ cat << EOF >> $HGRCPATH |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
35 > [extensions] |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
36 > verbosemmap=$TESTTMP/verbosemmap.py |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
37 > EOF |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
38 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
39 mmap index which is now more than 4k long |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
40 $ hg log -l 5 -T '{rev}\n' --config experimental.mmapindexthreshold=4k |
35393
4441705b7111
tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents:
34446
diff
changeset
|
41 mmapping $TESTTMP/a/.hg/store/00changelog.i |
34296
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
42 100 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
43 99 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
44 98 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
45 97 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
46 96 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
47 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
48 do not mmap index which is still less than 32k |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
49 $ hg log -l 5 -T '{rev}\n' --config experimental.mmapindexthreshold=32k |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
50 100 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
51 99 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
52 98 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
53 97 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
54 96 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
55 |
3c9691728237
revlog: add option to mmap revlog index
Mark Thomas <mbthomas@fb.com>
parents:
diff
changeset
|
56 $ cd .. |