Mercurial > hg
annotate tests/test-profile.t @ 30435:b86a448a2965
zstd: vendor python-zstandard 0.5.0
As the commit message for the previous changeset says, we wish
for zstd to be a 1st class citizen in Mercurial. To make that
happen, we need to enable Python to talk to the zstd C API. And
that requires bindings.
This commit vendors a copy of existing Python bindings. Why do we
need to vendor? As the commit message of the previous commit says,
relying on systems in the wild to have the bindings or zstd present
is a losing proposition. By distributing the zstd and bindings with
Mercurial, we significantly increase our chances that zstd will
work. Since zstd will deliver a better end-user experience by
achieving better performance, this benefits our users. Another
reason is that the Python bindings still aren't stable and the
API is somewhat fluid. While Mercurial could be coded to target
multiple versions of the Python bindings, it is safer to bundle
an explicit, known working version.
The added Python bindings are mostly a fully-featured interface
to the zstd C API. They allow one-shot operations, streaming,
reading and writing from objects implements the file object
protocol, dictionary compression, control over low-level compression
parameters, and more. The Python bindings work on Python 2.6,
2.7, and 3.3+ and have been tested on Linux and Windows. There are
CFFI bindings, but they are lacking compared to the C extension.
Upstream work will be needed before we can support zstd with PyPy.
But it will be possible.
The files added in this commit come from Git commit
e637c1b214d5f869cf8116c550dcae23ec13b677 from
https://github.com/indygreg/python-zstandard and are added without
modifications. Some files from the upstream repository have been
omitted, namely files related to continuous integration.
In the spirit of full disclosure, I'm the maintainer of the
"python-zstandard" project and have authored 100% of the code
added in this commit. Unfortunately, the Python bindings have
not been formally code reviewed by anyone. While I've tested
much of the code thoroughly (I even have tests that fuzz APIs),
there's a good chance there are bugs, memory leaks, not well
thought out APIs, etc. If someone wants to review the code and
send feedback to the GitHub project, it would be greatly
appreciated.
Despite my involvement with both projects, my opinions of code
style differ from Mercurial's. The code in this commit introduces
numerous code style violations in Mercurial's linters. So, the code
is excluded from most lints. However, some violations I agree with.
These have been added to the known violations ignore list for now.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Thu, 10 Nov 2016 22:15:58 -0800 |
parents | 3fd53cc1aad8 |
children | 262c2be8ea5a |
rev | line source |
---|---|
12478 | 1 test --time |
5099
105d4cf7ec24
Test --time, --profile and --lsprof
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
2 |
16933
30143c3dd102
dispatch: lowercase --time message
Martin Geisler <mg@aragost.com>
parents:
16913
diff
changeset
|
3 $ hg --time help -q help 2>&1 | grep time > /dev/null |
12478 | 4 $ hg init a |
5 $ cd a | |
8022
4f3fdfaa3874
profiling: Adding profiling.output config variable
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents:
8021
diff
changeset
|
6 |
16898
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
7 #if lsprof |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
8 |
12478 | 9 test --profile |
8022
4f3fdfaa3874
profiling: Adding profiling.output config variable
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents:
8021
diff
changeset
|
10 |
30259
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
11 $ prof='hg --config profiling.type=ls --profile' |
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
12 |
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
13 $ $prof st 2>../out |
16898
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
14 $ grep CallCount ../out > /dev/null || cat ../out |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
15 |
30259
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
16 $ $prof --config profiling.output=../out st |
16898
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
17 $ grep CallCount ../out > /dev/null || cat ../out |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
18 |
30259
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
19 $ $prof --config profiling.output=blackbox --config extensions.blackbox= st |
26191
39a0b11158d8
profiling: allow logging profile to the blackbox
Durham Goode <durham@fb.com>
parents:
18765
diff
changeset
|
20 $ grep CallCount .hg/blackbox.log > /dev/null || cat .hg/blackbox.log |
39a0b11158d8
profiling: allow logging profile to the blackbox
Durham Goode <durham@fb.com>
parents:
18765
diff
changeset
|
21 |
30259
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
22 $ $prof --config profiling.format=text st 2>../out |
16898
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
23 $ grep CallCount ../out > /dev/null || cat ../out |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
24 |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
25 $ echo "[profiling]" >> $HGRCPATH |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
26 $ echo "format=kcachegrind" >> $HGRCPATH |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
27 |
30259
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
28 $ $prof st 2>../out |
16898
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
29 $ grep 'events: Ticks' ../out > /dev/null || cat ../out |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
30 |
30259
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
31 $ $prof --config profiling.output=../out st |
16898
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
32 $ grep 'events: Ticks' ../out > /dev/null || cat ../out |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
33 |
bb91c602d4ad
tests: change odd uses of 'if hghave' to #if
Mads Kiilerich <mads@kiilerich.com>
parents:
12478
diff
changeset
|
34 #endif |
16913
f2719b387380
tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents:
16898
diff
changeset
|
35 |
29787
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
36 #if lsprof serve |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
37 |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
38 Profiling of HTTP requests works |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
39 |
30259
d06c049695e6
tests: explicitly use ls profiler
Gregory Szorc <gregory.szorc@gmail.com>
parents:
29787
diff
changeset
|
40 $ $prof --config profiling.format=text --config profiling.output=../profile.log serve -d -p $HGPORT --pid-file ../hg.pid -A ../access.log |
29787
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
41 $ cat ../hg.pid >> $DAEMON_PIDS |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
42 $ hg -q clone -U http://localhost:$HGPORT ../clone |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
43 |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
44 A single profile is logged because file logging doesn't append |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
45 $ grep CallCount ../profile.log | wc -l |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
46 \s*1 (re) |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
47 |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
48 #endif |
80df04266a16
hgweb: profile HTTP requests
Gregory Szorc <gregory.szorc@gmail.com>
parents:
26191
diff
changeset
|
49 |
30316
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
50 Install an extension that can sleep and guarantee a profiler has time to run |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
51 |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
52 $ cat >> sleepext.py << EOF |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
53 > import time |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
54 > from mercurial import cmdutil, commands |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
55 > cmdtable = {} |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
56 > command = cmdutil.command(cmdtable) |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
57 > @command('sleep', [], 'hg sleep') |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
58 > def sleep(ui, *args, **kwargs): |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
59 > time.sleep(0.1) |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
60 > EOF |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
61 |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
62 $ cat >> $HGRCPATH << EOF |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
63 > [extensions] |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
64 > sleep = `pwd`/sleepext.py |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
65 > EOF |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
66 |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
67 statistical profiler works |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
68 |
30317
3fd53cc1aad8
profiling: make statprof the default profiler (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30316
diff
changeset
|
69 $ hg --profile sleep 2>../out |
30316
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
70 $ grep Sample ../out |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
71 Sample count: \d+ (re) |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
72 |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
73 Various statprof formatters work |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
74 |
30317
3fd53cc1aad8
profiling: make statprof the default profiler (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30316
diff
changeset
|
75 $ hg --profile --config profiling.statformat=byline sleep 2>../out |
30316
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
76 $ head -n 1 ../out |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
77 % cumulative self |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
78 $ grep Sample ../out |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
79 Sample count: \d+ (re) |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
80 |
30317
3fd53cc1aad8
profiling: make statprof the default profiler (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30316
diff
changeset
|
81 $ hg --profile --config profiling.statformat=bymethod sleep 2>../out |
30316
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
82 $ head -n 1 ../out |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
83 % cumulative self |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
84 $ grep Sample ../out |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
85 Sample count: \d+ (re) |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
86 |
30317
3fd53cc1aad8
profiling: make statprof the default profiler (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30316
diff
changeset
|
87 $ hg --profile --config profiling.statformat=hotpath sleep 2>../out |
30316
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
88 $ grep Sample ../out |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
89 Sample count: \d+ (re) |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
90 |
30317
3fd53cc1aad8
profiling: make statprof the default profiler (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30316
diff
changeset
|
91 $ hg --profile --config profiling.statformat=json sleep 2>../out |
30316
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
92 $ cat ../out |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
93 \[\[\d+.* (re) |
faf1b8923da2
profiling: use vendored statprof and upstream enhancements (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents:
30259
diff
changeset
|
94 |
16913
f2719b387380
tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents:
16898
diff
changeset
|
95 $ cd .. |