Mercurial > hg
annotate tests/test-serve.t @ 46607:e9901d01d135
revlog: add a mechanism to verify expected file position before appending
If someone uses `hg debuglocks`, or some non-hg process writes to the .hg
directory without respecting the locks, or if the repo's on a networked
filesystem, it's possible for the revlog code to write out corrupted data.
The form of this corruption can vary depending on what data was written and how
that happened. We are in the "networked filesystem" case (though I've had users
also do this to themselves with the "`hg debuglocks`" scenario), and most often
see this with the changelog. What ends up happening is we produce two items
(let's call them rev1 and rev2) in the .i file that have the same linkrev,
baserev, and offset into the .d file, while the data in the .d file is appended
properly. rev2's compressed_size is accurate for rev2, but when we go to
decompress the data in the .d file, we use the offset that's recorded in the
index file, which is the same as rev1, and attempt to decompress
rev2.compressed_size bytes of rev1's data. This usually does not succeed. :)
When using inline data, this also fails, though I haven't investigated why too
closely. This shows up as a "patch decode" error. I believe what's happening
there is that we're basically ignoring the offset field, getting the data
properly, but since baserev != rev, it thinks this is a delta based on rev
(instead of a full text) and can't actually apply it as such.
For now, I'm going to make this an optional component and default it to entirely
off. I may increase the default severity of this in the future, once I've
enabled it for my users and we gain more experience with it. Luckily, most of my
users have a versioned filesystem and can roll back to before the corruption has
been written, it's just a hassle to do so and not everyone knows how (so it's a
support burden). Users on other filesystems will not have that luxury, and this
can cause them to have a corrupted repository that they are unlikely to know how
to resolve, and they'll see this as a data-loss event. Refusing to create the
corruption is a much better user experience.
This mechanism is not perfect. There may be false-negatives (racy writes that
are not detected). There should not be any false-positives (non-racy writes that
are detected as such). This is not a mechanism that makes putting a repo on a
networked filesystem "safe" or "supported", just *less* likely to cause
corruption.
Differential Revision: https://phab.mercurial-scm.org/D9952
author | Kyle Lippincott <spectral@google.com> |
---|---|
date | Wed, 03 Feb 2021 16:33:10 -0800 |
parents | ebee234d952a |
children | a8deb9dc39da |
rev | line source |
---|---|
22046
7a9cbb315d84
tests: replace exit 80 with #require
Matt Mackall <mpm@selenic.com>
parents:
20008
diff
changeset
|
1 #require serve |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
2 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
3 $ hgserve() |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
4 > { |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
5 > hg serve -a localhost -d --pid-file=hg.pid -E errors.log -v $@ \ |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
6 > | sed -e "s/:$HGPORT1\\([^0-9]\\)/:HGPORT1\1/g" \ |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
7 > -e "s/:$HGPORT2\\([^0-9]\\)/:HGPORT2\1/g" \ |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
8 > -e 's/http:\/\/[^/]*\//http:\/\/localhost\//' |
37846
89793289c891
tests: remove pid file by default
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37156
diff
changeset
|
9 > if [ -f hg.pid ]; then |
89793289c891
tests: remove pid file by default
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37156
diff
changeset
|
10 > killdaemons.py hg.pid |
89793289c891
tests: remove pid file by default
Gregory Szorc <gregory.szorc@gmail.com>
parents:
37156
diff
changeset
|
11 > fi |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
12 > echo % errors |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
13 > cat errors.log |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
14 > } |
4504
c68e6486f295
Add test reproducing a bug in "hg serve -v"
Joel Rosdahl <joel@rosdahl.net>
parents:
diff
changeset
|
15 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
16 $ hg init test |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
17 $ cd test |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
18 $ echo '[web]' > .hg/hgrc |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
19 $ echo 'accesslog = access.log' >> .hg/hgrc |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
20 $ echo "port = $HGPORT1" >> .hg/hgrc |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
21 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
22 Without -v |
6300
874ca958025b
test-serve: replace copy/paste with shell function
Patrick Mezard <pmezard@gmail.com>
parents:
6262
diff
changeset
|
23 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
24 $ hg serve -a localhost -p $HGPORT -d --pid-file=hg.pid -E errors.log |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
25 $ cat hg.pid >> "$DAEMON_PIDS" |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
26 $ if [ -f access.log ]; then |
16487
4fe874697a4d
tests: fix incorrect markup of continued lines of sh commands
Mads Kiilerich <mads@kiilerich.com>
parents:
16364
diff
changeset
|
27 > echo 'access log created - .hg/hgrc respected' |
4fe874697a4d
tests: fix incorrect markup of continued lines of sh commands
Mads Kiilerich <mads@kiilerich.com>
parents:
16364
diff
changeset
|
28 > fi |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
29 access log created - .hg/hgrc respected |
4504
c68e6486f295
Add test reproducing a bug in "hg serve -v"
Joel Rosdahl <joel@rosdahl.net>
parents:
diff
changeset
|
30 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
31 errors |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
32 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
33 $ cat errors.log |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
34 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
35 With -v |
4835
9858477ed74c
serve: respect settings from .hg/hgrc
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents:
4506
diff
changeset
|
36 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
37 $ hgserve |
31769
594dd384803c
test-serve: make the 'listening at *' lines optional
Matt Harbison <matt_harbison@yahoo.com>
parents:
31008
diff
changeset
|
38 listening at http://localhost/ (bound to *$LOCALIP*:HGPORT1) (glob) (?) |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
39 % errors |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
40 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
41 With -v and -p HGPORT2 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
42 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
43 $ hgserve -p "$HGPORT2" |
31769
594dd384803c
test-serve: make the 'listening at *' lines optional
Matt Harbison <matt_harbison@yahoo.com>
parents:
31008
diff
changeset
|
44 listening at http://localhost/ (bound to *$LOCALIP*:HGPORT2) (glob) (?) |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
45 % errors |
4504
c68e6486f295
Add test reproducing a bug in "hg serve -v"
Joel Rosdahl <joel@rosdahl.net>
parents:
diff
changeset
|
46 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
47 With -v and -p daytime (should fail because low port) |
5971
6d5ecf824a65
tests for hg serve prefix option
Michele Cella <michele.cella@gmail.com>
parents:
5384
diff
changeset
|
48 |
31771
5e92ba77793c
test-serve: disable unfixable tests on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
31769
diff
changeset
|
49 #if no-root no-windows |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
50 $ KILLQUIETLY=Y |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
51 $ hgserve -p daytime |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
52 abort: cannot start server at 'localhost:13': Permission denied |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
53 abort: child process failed to start |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
54 % errors |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
55 $ KILLQUIETLY=N |
20008
e54a078153f7
tests: skip tests that require not having root (issue4089)
Matt Mackall <mpm@selenic.com>
parents:
18602
diff
changeset
|
56 #endif |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
57 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
58 With --prefix foo |
10633
3318431f2ab4
test-serve: Show if port config and option are correctly used
Thomas Arendsen Hein <thomas@intevation.de>
parents:
6461
diff
changeset
|
59 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
60 $ hgserve --prefix foo |
31769
594dd384803c
test-serve: make the 'listening at *' lines optional
Matt Harbison <matt_harbison@yahoo.com>
parents:
31008
diff
changeset
|
61 listening at http://localhost/foo/ (bound to *$LOCALIP*:HGPORT1) (glob) (?) |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
62 % errors |
12076
49463314c24f
mail/hgweb: support service names for ports (issue2350)
Brodie Rao <brodie@bitheap.org>
parents:
10633
diff
changeset
|
63 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
64 With --prefix /foo |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
65 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
66 $ hgserve --prefix /foo |
31769
594dd384803c
test-serve: make the 'listening at *' lines optional
Matt Harbison <matt_harbison@yahoo.com>
parents:
31008
diff
changeset
|
67 listening at http://localhost/foo/ (bound to *$LOCALIP*:HGPORT1) (glob) (?) |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
68 % errors |
5971
6d5ecf824a65
tests for hg serve prefix option
Michele Cella <michele.cella@gmail.com>
parents:
5384
diff
changeset
|
69 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
70 With --prefix foo/ |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
71 |
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
72 $ hgserve --prefix foo/ |
31769
594dd384803c
test-serve: make the 'listening at *' lines optional
Matt Harbison <matt_harbison@yahoo.com>
parents:
31008
diff
changeset
|
73 listening at http://localhost/foo/ (bound to *$LOCALIP*:HGPORT1) (glob) (?) |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
74 % errors |
5971
6d5ecf824a65
tests for hg serve prefix option
Michele Cella <michele.cella@gmail.com>
parents:
5384
diff
changeset
|
75 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
76 With --prefix /foo/ |
5971
6d5ecf824a65
tests for hg serve prefix option
Michele Cella <michele.cella@gmail.com>
parents:
5384
diff
changeset
|
77 |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
78 $ hgserve --prefix /foo/ |
31769
594dd384803c
test-serve: make the 'listening at *' lines optional
Matt Harbison <matt_harbison@yahoo.com>
parents:
31008
diff
changeset
|
79 listening at http://localhost/foo/ (bound to *$LOCALIP*:HGPORT1) (glob) (?) |
13540
3ecadce9173d
tests: convert test-serve to new format
Patrick Mezard <pmezard@gmail.com>
parents:
12578
diff
changeset
|
80 % errors |
16913
f2719b387380
tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents:
16487
diff
changeset
|
81 |
39707
5abc47d4ca6b
tests: quote PYTHON usage
Matt Harbison <matt_harbison@yahoo.com>
parents:
37846
diff
changeset
|
82 $ "$PYTHON" $RUNTESTDIR/killdaemons.py $DAEMON_PIDS |
37156
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
83 |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
84 With out of bounds accesses |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
85 |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
86 $ rm access.log |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
87 $ hg serve -a localhost -p $HGPORT -d --prefix some/dir \ |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
88 > --pid-file=hg.pid -E errors.log |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
89 $ cat hg.pid >> "$DAEMON_PIDS" |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
90 |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
91 $ hg id http://localhost:$HGPORT/some/dir7 |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
92 abort: HTTP Error 404: Not Found |
45839
ebee234d952a
errors: set detailed exit code to 100 for some remote errors
Martin von Zweigbergk <martinvonz@google.com>
parents:
45067
diff
changeset
|
93 [100] |
37156
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
94 $ hg id http://localhost:$HGPORT/some |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
95 abort: HTTP Error 404: Not Found |
45839
ebee234d952a
errors: set detailed exit code to 100 for some remote errors
Martin von Zweigbergk <martinvonz@google.com>
parents:
45067
diff
changeset
|
96 [100] |
37156
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
97 |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
98 $ cat access.log errors.log |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
99 $LOCALIP - - [$LOGDATE$] "GET /some/dir7?cmd=capabilities HTTP/1.1" 404 - (glob) |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
100 $LOCALIP - - [$LOGDATE$] "GET /some?cmd=capabilities HTTP/1.1" 404 - (glob) |
7de7bd407251
server: ensure the incoming request falls under the prefix value
Matt Harbison <matt_harbison@yahoo.com>
parents:
31771
diff
changeset
|
101 |
45034
b1a1702262c9
py3: fix crash when server address is 0.0.0.0 (issue6362)
Manuel Jacob <me@manueljacob.de>
parents:
39707
diff
changeset
|
102 $ "$PYTHON" $RUNTESTDIR/killdaemons.py $DAEMON_PIDS |
b1a1702262c9
py3: fix crash when server address is 0.0.0.0 (issue6362)
Manuel Jacob <me@manueljacob.de>
parents:
39707
diff
changeset
|
103 |
b1a1702262c9
py3: fix crash when server address is 0.0.0.0 (issue6362)
Manuel Jacob <me@manueljacob.de>
parents:
39707
diff
changeset
|
104 issue6362: Previously, this crashed on Python 3 |
b1a1702262c9
py3: fix crash when server address is 0.0.0.0 (issue6362)
Manuel Jacob <me@manueljacob.de>
parents:
39707
diff
changeset
|
105 |
45067
9a062913bab6
tests: stablize test-serve.t on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
45034
diff
changeset
|
106 $ hg serve -a 0.0.0.0 -d --pid-file=hg.pid |
9a062913bab6
tests: stablize test-serve.t on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
45034
diff
changeset
|
107 listening at http://*:$HGPORT1/ (bound to *:$HGPORT1) (glob) (?) |
9a062913bab6
tests: stablize test-serve.t on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
45034
diff
changeset
|
108 |
9a062913bab6
tests: stablize test-serve.t on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
45034
diff
changeset
|
109 $ cat hg.pid > "$DAEMON_PIDS" |
9a062913bab6
tests: stablize test-serve.t on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents:
45034
diff
changeset
|
110 $ "$PYTHON" $RUNTESTDIR/killdaemons.py $DAEMON_PIDS |
45034
b1a1702262c9
py3: fix crash when server address is 0.0.0.0 (issue6362)
Manuel Jacob <me@manueljacob.de>
parents:
39707
diff
changeset
|
111 |
16913
f2719b387380
tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents:
16487
diff
changeset
|
112 $ cd .. |