Mercurial > hg
annotate tests/common-pattern.py @ 40021:c537144fdbef
wireprotov2: support response caching
One of the things I've learned from managing VCS servers over the
years is that they are hard to scale. It is well known that some
companies have very beefy (read: very expensive) servers to power
their VCS needs. It is also known that specialized servers for
various VCS exist in order to facilitate scaling servers. (Mercurial
is in this boat.)
One of the aspects that make a VCS server hard to scale is the
high CPU load incurred by constant client clone/pull operations.
To alleviate the scaling pain associated with data retrieval
operations, I want to integrate caching into the Mercurial wire
protocol server as robustly as possible such that servers can
aggressively cache responses and defer as much server load as
possible.
This commit represents the initial implementation of a general
caching layer in wire protocol version 2.
We define a new interface and behavior for a wire protocol cacher
in repository.py. (This is probably where a reviewer should look
first to understand what is going on.)
The bulk of the added code is in wireprotov2server.py, where we
define how a command can opt in to being cached and integrate
caching into command dispatching.
From a very high-level:
* A command can declare itself as cacheable by providing a callable
that can be used to derive a cache key.
* At dispatch time, if a command is cacheable, we attempt to
construct a cacher and use it for serving the request and/or
caching the request.
* The dispatch layer handles the bulk of the business logic for
caching, making cachers mostly "dumb content stores."
* The mechanism for invalidating cached entries (one of the harder
parts about caching in general) is by varying the cache key when
state changes. As such, cachers don't need to be concerned with
cache invalidation.
Initially, we've hooked up support for caching "manifestdata" and
"filedata" commands. These are the simplest to cache, as they should
be immutable over time. Caching of commands related to changeset
data is a bit harder (because cache validation is impacted by
changes to bookmarks, phases, etc). This will be implemented later.
(Strictly speaking, censoring a file should invalidate caches. I've
added an inline TODO to track this edge case.)
To prove it works, this commit implements a test-only extension
providing in-memory caching backed by an lrucachedict. A new test
showing this extension behaving properly is added. FWIW, the
cacher is ~50 lines of code, demonstrating the relative ease with
which a cache can be added to a server.
While the test cacher is not suitable for production workloads, just
for kicks I performed a clone of just the changeset and manifest data
for the mozilla-unified repository. With a fully warmed cache (of just
the manifest data since changeset data is not cached), server-side
CPU usage dropped from ~73s to ~28s. That's pretty significant and
demonstrates the potential that response caching has on server
scalability!
Differential Revision: https://phab.mercurial-scm.org/D4773
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 26 Sep 2018 17:16:56 -0700 |
parents | 4bd6e444c76f |
children | 8695fbe17f7c |
rev | line source |
---|---|
35070
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
1 # common patterns in test at can safely be replaced |
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
2 from __future__ import absolute_import |
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
3 |
35449
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
4 import os |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
5 |
35070
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
6 substitutions = [ |
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
7 # list of possible compressions |
35234
f0445eea9b93
test: fix common-pattern for pure variant
Boris Feld <boris.feld@octobus.net>
parents:
35233
diff
changeset
|
8 (br'(zstd,)?zlib,none,bzip2', |
35070
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
9 br'$USUAL_COMPRESSIONS$' |
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
10 ), |
35806
ab239e3de23b
tests: add a pattern to fix --pure tests
Augie Fackler <augie@google.com>
parents:
35792
diff
changeset
|
11 (br'=(zstd,)?zlib', |
ab239e3de23b
tests: add a pattern to fix --pure tests
Augie Fackler <augie@google.com>
parents:
35792
diff
changeset
|
12 br'=$BUNDLE2_COMPRESSIONS$' |
ab239e3de23b
tests: add a pattern to fix --pure tests
Augie Fackler <augie@google.com>
parents:
35792
diff
changeset
|
13 ), |
35071
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
14 # capabilities sent through http |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
15 (br'bundlecaps=HG20%2Cbundle2%3DHG20%250A' |
35259
ad5f2b923b0d
push: include a 'check:bookmarks' part when possible
Boris Feld <boris.feld@octobus.net>
parents:
35234
diff
changeset
|
16 br'bookmarks%250A' |
35071
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
17 br'changegroup%253D01%252C02%250A' |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
18 br'digests%253Dmd5%252Csha1%252Csha512%250A' |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
19 br'error%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250A' |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
20 br'hgtagsfnodes%250A' |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
21 br'listkeys%250A' |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
22 br'phases%253Dheads%250A' |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
23 br'pushkey%250A' |
35792
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
24 br'remote-changegroup%253Dhttp%252Chttps%250A' |
36968
2090044a288d
revbranchcache: advertise and use 'rbc' exchange capability
Boris Feld <boris.feld@octobus.net>
parents:
36924
diff
changeset
|
25 br'rev-branch-cache%250A' |
35792
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
26 br'stream%253Dv2', |
35071
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
27 # (the replacement patterns) |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
28 br'$USUAL_BUNDLE_CAPS$' |
b4767ae63b32
test-pattern: register current the bundlecaps string
Boris Feld <boris.feld@octobus.net>
parents:
35070
diff
changeset
|
29 ), |
35792
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
30 (br'bundlecaps=HG20%2Cbundle2%3DHG20%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
31 br'bookmarks%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
32 br'changegroup%253D01%252C02%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
33 br'digests%253Dmd5%252Csha1%252Csha512%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
34 br'error%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
35 br'hgtagsfnodes%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
36 br'listkeys%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
37 br'phases%253Dheads%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
38 br'pushkey%250A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
39 br'remote-changegroup%253Dhttp%252Chttps', |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
40 # (the replacement patterns) |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
41 br'$USUAL_BUNDLE_CAPS_SERVER$' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
42 ), |
35072
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
43 # bundle2 capabilities sent through ssh |
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
44 (br'bundle2=HG20%0A' |
35259
ad5f2b923b0d
push: include a 'check:bookmarks' part when possible
Boris Feld <boris.feld@octobus.net>
parents:
35234
diff
changeset
|
45 br'bookmarks%0A' |
35072
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
46 br'changegroup%3D01%2C02%0A' |
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
47 br'digests%3Dmd5%2Csha1%2Csha512%0A' |
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
48 br'error%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0A' |
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
49 br'hgtagsfnodes%0A' |
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
50 br'listkeys%0A' |
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
51 br'phases%3Dheads%0A' |
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
52 br'pushkey%0A' |
35792
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
53 br'remote-changegroup%3Dhttp%2Chttps%0A' |
36968
2090044a288d
revbranchcache: advertise and use 'rbc' exchange capability
Boris Feld <boris.feld@octobus.net>
parents:
36924
diff
changeset
|
54 br'rev-branch-cache%0A' |
35792
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
55 br'stream%3Dv2', |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
56 # (replacement patterns) |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
57 br'$USUAL_BUNDLE2_CAPS$' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
58 ), |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
59 # bundle2 capabilities advertised by the server |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
60 (br'bundle2=HG20%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
61 br'bookmarks%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
62 br'changegroup%3D01%2C02%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
63 br'digests%3Dmd5%2Csha1%2Csha512%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
64 br'error%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
65 br'hgtagsfnodes%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
66 br'listkeys%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
67 br'phases%3Dheads%0A' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
68 br'pushkey%0A' |
36968
2090044a288d
revbranchcache: advertise and use 'rbc' exchange capability
Boris Feld <boris.feld@octobus.net>
parents:
36924
diff
changeset
|
69 br'remote-changegroup%3Dhttp%2Chttps%0A' |
2090044a288d
revbranchcache: advertise and use 'rbc' exchange capability
Boris Feld <boris.feld@octobus.net>
parents:
36924
diff
changeset
|
70 br'rev-branch-cache', |
35072
c4ec72a10798
test-pattern: register the current the bundle2 capabilities string
Boris Feld <boris.feld@octobus.net>
parents:
35071
diff
changeset
|
71 # (replacement patterns) |
35792
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
72 br'$USUAL_BUNDLE2_CAPS_SERVER$' |
1d118f9f4f57
bundle2: always advertise client support for stream parts
Gregory Szorc <gregory.szorc@gmail.com>
parents:
35449
diff
changeset
|
73 ), |
38728
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
74 ( |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
75 br'bundle2=HG20%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
76 br'bookmarks%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
77 br'changegroup%3D01%2C02%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
78 br'digests%3Dmd5%2Csha1%2Csha512%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
79 br'error%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
80 br'hgtagsfnodes%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
81 br'listkeys%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
82 br'pushkey%0A' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
83 br'remote-changegroup%3Dhttp%2Chttps%0A' |
39722
4bd6e444c76f
bundle2: make server.bundle2.stream default to True
Anton Shestakov <av6@dwimlabs.net>
parents:
38728
diff
changeset
|
84 br'rev-branch-cache%0A' |
4bd6e444c76f
bundle2: make server.bundle2.stream default to True
Anton Shestakov <av6@dwimlabs.net>
parents:
38728
diff
changeset
|
85 br'stream%3Dv2', |
38728
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
86 # (replacement patterns) |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
87 br'$USUAL_BUNDLE2_CAPS_NO_PHASES$' |
88be288e8ac1
tests: refactor common bundle2 capabilities
Joerg Sonnenberger <joerg@bec.de>
parents:
37477
diff
changeset
|
88 ), |
37477
28e8c5949109
tests: add a substitution pattern for HTTP error log dates
Matt Harbison <matt_harbison@yahoo.com>
parents:
37008
diff
changeset
|
89 # HTTP access log dates |
36385
003ae586a02e
tests: add HTTP POST and PUT support to the $LOGDATE$ substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35806
diff
changeset
|
90 (br' - - \[\d\d/.../2\d\d\d \d\d:\d\d:\d\d] "(GET|PUT|POST)', |
003ae586a02e
tests: add HTTP POST and PUT support to the $LOGDATE$ substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35806
diff
changeset
|
91 lambda m: br' - - [$LOGDATE$] "' + m.group(1) |
35073
7f0c9e28a816
test-pattern: substitute the HTTP log timestamp too
Boris Feld <boris.feld@octobus.net>
parents:
35072
diff
changeset
|
92 ), |
37477
28e8c5949109
tests: add a substitution pattern for HTTP error log dates
Matt Harbison <matt_harbison@yahoo.com>
parents:
37008
diff
changeset
|
93 # HTTP error log dates |
28e8c5949109
tests: add a substitution pattern for HTTP error log dates
Matt Harbison <matt_harbison@yahoo.com>
parents:
37008
diff
changeset
|
94 (br' - - \[\d\d/.../2\d\d\d \d\d:\d\d:\d\d] (HG error:|Exception)', |
28e8c5949109
tests: add a substitution pattern for HTTP error log dates
Matt Harbison <matt_harbison@yahoo.com>
parents:
37008
diff
changeset
|
95 lambda m: br' - - [$ERRDATE$] ' + m.group(1) |
28e8c5949109
tests: add a substitution pattern for HTTP error log dates
Matt Harbison <matt_harbison@yahoo.com>
parents:
37008
diff
changeset
|
96 ), |
36924
03368431f014
tests: add a substitution pattern for dates in HTTP headers and LFS payload
Matt Harbison <matt_harbison@yahoo.com>
parents:
36385
diff
changeset
|
97 # HTTP header dates- RFC 1123 |
37008
16203c6079e7
tests: use $HTTP_DATE$ for Date header
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36968
diff
changeset
|
98 (br'([Dd]ate): [A-Za-z]{3}, \d\d [A-Za-z]{3} \d{4} \d\d:\d\d:\d\d GMT', |
16203c6079e7
tests: use $HTTP_DATE$ for Date header
Gregory Szorc <gregory.szorc@gmail.com>
parents:
36968
diff
changeset
|
99 lambda m: br'%s: $HTTP_DATE$' % m.group(1) |
36924
03368431f014
tests: add a substitution pattern for dates in HTTP headers and LFS payload
Matt Harbison <matt_harbison@yahoo.com>
parents:
36385
diff
changeset
|
100 ), |
03368431f014
tests: add a substitution pattern for dates in HTTP headers and LFS payload
Matt Harbison <matt_harbison@yahoo.com>
parents:
36385
diff
changeset
|
101 # LFS expiration value |
03368431f014
tests: add a substitution pattern for dates in HTTP headers and LFS payload
Matt Harbison <matt_harbison@yahoo.com>
parents:
36385
diff
changeset
|
102 (br'"expires_at": "\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ"', |
03368431f014
tests: add a substitution pattern for dates in HTTP headers and LFS payload
Matt Harbison <matt_harbison@yahoo.com>
parents:
36385
diff
changeset
|
103 br'"expires_at": "$ISO_8601_DATE_TIME$"' |
03368431f014
tests: add a substitution pattern for dates in HTTP headers and LFS payload
Matt Harbison <matt_harbison@yahoo.com>
parents:
36385
diff
changeset
|
104 ), |
35446
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
105 # Windows has an extra '/' in the following lines that get globbed away: |
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
106 # pushing to file:/*/$TESTTMP/r2 (glob) |
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
107 # comparing with file:/*/$TESTTMP/r2 (glob) |
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
108 # sub/maybelarge.dat: largefile 34..9c not available from |
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
109 # file:/*/$TESTTMP/largefiles-repo (glob) |
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
110 (br'(.*file:/)/?(/\$TESTTMP.*)', |
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
111 lambda m: m.group(1) + b'*' + m.group(2) + b' (glob)' |
991e4404e910
tests: convert the 'file://\$TESTTMP' rule to an automatic substitution
Matt Harbison <matt_harbison@yahoo.com>
parents:
35259
diff
changeset
|
112 ), |
35070
3abdd7da33bd
test-pattern: substitute common compression list
Boris Feld <boris.feld@octobus.net>
parents:
diff
changeset
|
113 ] |
35230
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
114 |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
115 # Various platform error strings, keyed on a common replacement string |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
116 _errors = { |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
117 br'$ENOENT$': ( |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
118 # strerror() |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
119 br'No such file or directory', |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
120 |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
121 # FormatMessage(ERROR_FILE_NOT_FOUND) |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
122 br'The system cannot find the file specified', |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
123 ), |
35231
b33d4cf38666
tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35230
diff
changeset
|
124 br'$ENOTDIR$': ( |
b33d4cf38666
tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35230
diff
changeset
|
125 # strerror() |
b33d4cf38666
tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35230
diff
changeset
|
126 br'Not a directory', |
b33d4cf38666
tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35230
diff
changeset
|
127 |
b33d4cf38666
tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35230
diff
changeset
|
128 # FormatMessage(ERROR_PATH_NOT_FOUND) |
b33d4cf38666
tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35230
diff
changeset
|
129 br'The system cannot find the path specified', |
b33d4cf38666
tests: add a substitution for ENOTDIR/ERROR_PATH_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35230
diff
changeset
|
130 ), |
35232
a037b6647d8c
tests: add a substitution for ECONNRESET/WSAECONNRESET messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35231
diff
changeset
|
131 br'$ECONNRESET$': ( |
a037b6647d8c
tests: add a substitution for ECONNRESET/WSAECONNRESET messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35231
diff
changeset
|
132 # strerror() |
a037b6647d8c
tests: add a substitution for ECONNRESET/WSAECONNRESET messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35231
diff
changeset
|
133 br'Connection reset by peer', |
a037b6647d8c
tests: add a substitution for ECONNRESET/WSAECONNRESET messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35231
diff
changeset
|
134 |
a037b6647d8c
tests: add a substitution for ECONNRESET/WSAECONNRESET messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35231
diff
changeset
|
135 # FormatMessage(WSAECONNRESET) |
a037b6647d8c
tests: add a substitution for ECONNRESET/WSAECONNRESET messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35231
diff
changeset
|
136 br'An existing connection was forcibly closed by the remote host', |
a037b6647d8c
tests: add a substitution for ECONNRESET/WSAECONNRESET messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35231
diff
changeset
|
137 ), |
35233
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
138 br'$EADDRINUSE$': ( |
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
139 # strerror() |
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
140 br'Address already in use', |
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
141 |
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
142 # FormatMessage(WSAEADDRINUSE) |
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
143 br'Only one usage of each socket address' |
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
144 br' \(protocol/network address/port\) is normally permitted', |
1b22d325089c
tests: add a substitution for EADDRINUSE/WSAEADDRINUSE messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35232
diff
changeset
|
145 ), |
35230
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
146 } |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
147 |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
148 for replace, msgs in _errors.items(): |
feecfefeba25
tests: add a substitution for ENOENT/ERROR_FILE_NOT_FOUND messages
Matt Harbison <matt_harbison@yahoo.com>
parents:
35073
diff
changeset
|
149 substitutions.extend((m, replace) for m in msgs) |
35449
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
150 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
151 # Output lines on Windows that can be autocorrected for '\' vs '/' path |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
152 # differences. |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
153 _winpathfixes = [ |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
154 # cloning subrepo s\ss from $TESTTMP/t/s/ss |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
155 # cloning subrepo foo\bar from http://localhost:$HGPORT/foo/bar |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
156 br'(?m)^cloning subrepo \S+\\.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
157 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
158 # pulling from $TESTTMP\issue1852a |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
159 br'(?m)^pulling from \$TESTTMP\\.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
160 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
161 # pushing to $TESTTMP\a |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
162 br'(?m)^pushing to \$TESTTMP\\.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
163 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
164 # pushing subrepo s\ss to $TESTTMP/t/s/ss |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
165 br'(?m)^pushing subrepo \S+\\\S+ to.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
166 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
167 # moving d1\d11\a1 to d3/d11/a1 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
168 br'(?m)^moving \S+\\.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
169 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
170 # d1\a: not recording move - dummy does not exist |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
171 br'\S+\\\S+: not recording move .+', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
172 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
173 # reverting s\a |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
174 br'(?m)^reverting (?!subrepo ).*\\.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
175 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
176 # saved backup bundle to |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
177 # $TESTTMP\test\.hg\strip-backup/443431ffac4f-2fc5398a-backup.hg |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
178 br'(?m)^saved backup bundle to \$TESTTMP.*\.hg', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
179 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
180 # no changes made to subrepo s\ss since last push to ../tcc/s/ss |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
181 br'(?m)^no changes made to subrepo \S+\\\S+ since.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
182 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
183 # changeset 5:9cc5aa7204f0: stuff/maybelarge.dat references missing |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
184 # $TESTTMP\largefiles-repo-hg\.hg\largefiles\76..38 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
185 br'(?m)^changeset .* references (corrupted|missing) \$TESTTMP\\.*', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
186 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
187 # stuff/maybelarge.dat: largefile 76..38 not available from |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
188 # file:/*/$TESTTMP\largefiles-repo (glob) |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
189 br'.*: largefile \S+ not available from file:/\*/.+', |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
190 ] |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
191 |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
192 if os.name == 'nt': |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
193 substitutions.extend([(s, lambda match: match.group().replace(b'\\', b'/')) |
24528dba0e64
run-tests: add substitution patterns for common '\' path output on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
35446
diff
changeset
|
194 for s in _winpathfixes]) |