Mercurial > hg
annotate tests/test-tools.t @ 30435:b86a448a2965
zstd: vendor python-zstandard 0.5.0
As the commit message for the previous changeset says, we wish
for zstd to be a 1st class citizen in Mercurial. To make that
happen, we need to enable Python to talk to the zstd C API. And
that requires bindings.
This commit vendors a copy of existing Python bindings. Why do we
need to vendor? As the commit message of the previous commit says,
relying on systems in the wild to have the bindings or zstd present
is a losing proposition. By distributing the zstd and bindings with
Mercurial, we significantly increase our chances that zstd will
work. Since zstd will deliver a better end-user experience by
achieving better performance, this benefits our users. Another
reason is that the Python bindings still aren't stable and the
API is somewhat fluid. While Mercurial could be coded to target
multiple versions of the Python bindings, it is safer to bundle
an explicit, known working version.
The added Python bindings are mostly a fully-featured interface
to the zstd C API. They allow one-shot operations, streaming,
reading and writing from objects implements the file object
protocol, dictionary compression, control over low-level compression
parameters, and more. The Python bindings work on Python 2.6,
2.7, and 3.3+ and have been tested on Linux and Windows. There are
CFFI bindings, but they are lacking compared to the C extension.
Upstream work will be needed before we can support zstd with PyPy.
But it will be possible.
The files added in this commit come from Git commit
e637c1b214d5f869cf8116c550dcae23ec13b677 from
https://github.com/indygreg/python-zstandard and are added without
modifications. Some files from the upstream repository have been
omitted, namely files related to continuous integration.
In the spirit of full disclosure, I'm the maintainer of the
"python-zstandard" project and have authored 100% of the code
added in this commit. Unfortunately, the Python bindings have
not been formally code reviewed by anyone. While I've tested
much of the code thoroughly (I even have tests that fuzz APIs),
there's a good chance there are bugs, memory leaks, not well
thought out APIs, etc. If someone wants to review the code and
send feedback to the GitHub project, it would be greatly
appreciated.
Despite my involvement with both projects, my opinions of code
style differ from Mercurial's. The code in this commit introduces
numerous code style violations in Mercurial's linters. So, the code
is excluded from most lints. However, some violations I agree with.
These have been added to the known violations ignore list for now.
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Thu, 10 Nov 2016 22:15:58 -0800 |
parents | 6686ae524f94 |
children | 75be14993fda |
rev | line source |
---|---|
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
1 Tests of the file helper tool |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
2 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
3 $ f -h |
23909
0760f023a34b
test-tools: fix portability issues
Matt Mackall <mpm@selenic.com>
parents:
23867
diff
changeset
|
4 ?sage: f [options] [filenames] (glob) |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
5 |
23913 | 6 ?ptions: (glob) |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
7 -h, --help show this help message and exit |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
8 -t, --type show file type (file or directory) |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
9 -m, --mode show file mode |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
10 -l, --links show number of links |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
11 -s, --size show size of file |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
12 -n NEWER, --newer=NEWER |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
13 check if file is newer (or same) |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
14 -r, --recurse recurse into directories |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
15 -S, --sha1 show sha1 hash of the content |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
16 -M, --md5 show md5 hash of the content |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
17 -D, --dump dump file content |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
18 -H, --hexdump hexdump file content |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
19 -B BYTES, --bytes=BYTES |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
20 number of characters to dump |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
21 -L LINES, --lines=LINES |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
22 number of lines to dump |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
23 -q, --quiet no default output |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
24 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
25 $ mkdir dir |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
26 $ cd dir |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
27 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
28 $ f --size |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
29 size=0 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
30 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
31 $ echo hello | f --md5 --size |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
32 size=6, md5=b1946ac92492d2347c6235b4d2611184 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
33 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
34 $ f foo |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
35 foo: file not found |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
36 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
37 $ echo foo > foo |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
38 $ f foo |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
39 foo: |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
40 |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
41 #if symlink |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
42 $ f foo --mode |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
43 foo: mode=644 |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
44 #endif |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
45 |
26950
6686ae524f94
tests: make 'f' tool open files in binary mode when hexdumping
Matt Harbison <matt_harbison@yahoo.com>
parents:
24361
diff
changeset
|
46 #if no-windows |
24361
8de8cd34f2e3
tests: replace uses of 'seq' with portable 'seq.py'
Matt Harbison <matt_harbison@yahoo.com>
parents:
23957
diff
changeset
|
47 $ python $TESTDIR/seq.py 10 > bar |
26950
6686ae524f94
tests: make 'f' tool open files in binary mode when hexdumping
Matt Harbison <matt_harbison@yahoo.com>
parents:
24361
diff
changeset
|
48 #else |
6686ae524f94
tests: make 'f' tool open files in binary mode when hexdumping
Matt Harbison <matt_harbison@yahoo.com>
parents:
24361
diff
changeset
|
49 Convert CRLF -> LF for consistency |
6686ae524f94
tests: make 'f' tool open files in binary mode when hexdumping
Matt Harbison <matt_harbison@yahoo.com>
parents:
24361
diff
changeset
|
50 $ python $TESTDIR/seq.py 10 | sed "s/$//" > bar |
6686ae524f94
tests: make 'f' tool open files in binary mode when hexdumping
Matt Harbison <matt_harbison@yahoo.com>
parents:
24361
diff
changeset
|
51 #endif |
6686ae524f94
tests: make 'f' tool open files in binary mode when hexdumping
Matt Harbison <matt_harbison@yahoo.com>
parents:
24361
diff
changeset
|
52 |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
53 #if unix-permissions symlink |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
54 $ chmod +x bar |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
55 $ f bar --newer foo --mode --type --size --dump --links --bytes 7 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
56 bar: file, size=21, mode=755, links=1, newer than foo |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
57 >>> |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
58 1 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
59 2 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
60 3 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
61 4 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
62 <<< no trailing newline |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
63 #endif |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
64 |
23914
a2396b0fd608
test-tools: check for unix permissions for hardlinking
Matt Mackall <mpm@selenic.com>
parents:
23913
diff
changeset
|
65 #if unix-permissions |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
66 $ ln bar baz |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
67 $ f bar -n baz -l --hexdump -t --sha1 --lines=9 -B 20 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
68 bar: file, links=2, newer than baz, sha1=612ca68d0305c821750a |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
69 0000: 31 0a 32 0a 33 0a 34 0a 35 0a 36 0a 37 0a 38 0a |1.2.3.4.5.6.7.8.| |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
70 0010: 39 0a |9.| |
23915
5b20e4c32117
test-tools: another vfat fix
Matt Mackall <mpm@selenic.com>
parents:
23914
diff
changeset
|
71 $ rm baz |
23914
a2396b0fd608
test-tools: check for unix permissions for hardlinking
Matt Mackall <mpm@selenic.com>
parents:
23913
diff
changeset
|
72 #endif |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
73 |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
74 #if unix-permissions symlink |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
75 $ ln -s yadda l |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
76 $ f . --recurse -MStmsB4 |
23915
5b20e4c32117
test-tools: another vfat fix
Matt Mackall <mpm@selenic.com>
parents:
23914
diff
changeset
|
77 .: directory with 3 files, mode=755 |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
78 ./bar: file, size=21, mode=755, md5=3b03, sha1=612c |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
79 ./foo: file, size=4, mode=644, md5=d3b0, sha1=f1d2 |
23912
7d0aa6269ece
tests: teach f not to report symlink mode bits
Matt Mackall <mpm@selenic.com>
parents:
23911
diff
changeset
|
80 ./l: link, size=5, md5=2faa, sha1=af93 |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
81 #endif |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
82 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
83 $ f --quiet bar -DL 3 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
84 1 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
85 2 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
86 3 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
87 |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
88 $ cd .. |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
89 |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
90 Yadda is a symlink |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
91 #if symlink |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
92 $ f -qr dir -HB 17 |
23915
5b20e4c32117
test-tools: another vfat fix
Matt Mackall <mpm@selenic.com>
parents:
23914
diff
changeset
|
93 dir: directory with 3 files |
23860
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
94 dir/bar: |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
95 0000: 31 0a 32 0a 33 0a 34 0a 35 0a 36 0a 37 0a 38 0a |1.2.3.4.5.6.7.8.| |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
96 0010: 39 |9| |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
97 dir/foo: |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
98 0000: 66 6f 6f 0a |foo.| |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
99 dir/l: |
bead0c7b4f68
tests: add 'f' tool for cross platform file operations in the tests
Mads Kiilerich <madski@unity3d.com>
parents:
diff
changeset
|
100 0000: 79 61 64 64 61 |yadda| |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
101 #else |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
102 $ f -qr dir -HB 17 |
23957
0c4419faacbc
test-tools: portability tweak
Matt Mackall <mpm@selenic.com>
parents:
23922
diff
changeset
|
103 dir: directory with 2 files (esc) |
23867
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
104 dir/bar: (glob) |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
105 0000: 31 0a 32 0a 33 0a 34 0a 35 0a 36 0a 37 0a 38 0a |1.2.3.4.5.6.7.8.| |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
106 0010: 39 |9| |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
107 dir/foo: (glob) |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
108 0000: 66 6f 6f 0a |foo.| |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
109 #endif |
049a9e3a078d
tests: conditionalize test-tools.t for Windows support
Matt Harbison <matt_harbison@yahoo.com>
parents:
23860
diff
changeset
|
110 |