Mercurial > hg
view tests/test-obsolete-tag-cache.t @ 46607:e9901d01d135
revlog: add a mechanism to verify expected file position before appending
If someone uses `hg debuglocks`, or some non-hg process writes to the .hg
directory without respecting the locks, or if the repo's on a networked
filesystem, it's possible for the revlog code to write out corrupted data.
The form of this corruption can vary depending on what data was written and how
that happened. We are in the "networked filesystem" case (though I've had users
also do this to themselves with the "`hg debuglocks`" scenario), and most often
see this with the changelog. What ends up happening is we produce two items
(let's call them rev1 and rev2) in the .i file that have the same linkrev,
baserev, and offset into the .d file, while the data in the .d file is appended
properly. rev2's compressed_size is accurate for rev2, but when we go to
decompress the data in the .d file, we use the offset that's recorded in the
index file, which is the same as rev1, and attempt to decompress
rev2.compressed_size bytes of rev1's data. This usually does not succeed. :)
When using inline data, this also fails, though I haven't investigated why too
closely. This shows up as a "patch decode" error. I believe what's happening
there is that we're basically ignoring the offset field, getting the data
properly, but since baserev != rev, it thinks this is a delta based on rev
(instead of a full text) and can't actually apply it as such.
For now, I'm going to make this an optional component and default it to entirely
off. I may increase the default severity of this in the future, once I've
enabled it for my users and we gain more experience with it. Luckily, most of my
users have a versioned filesystem and can roll back to before the corruption has
been written, it's just a hassle to do so and not everyone knows how (so it's a
support burden). Users on other filesystems will not have that luxury, and this
can cause them to have a corrupted repository that they are unlikely to know how
to resolve, and they'll see this as a data-loss event. Refusing to create the
corruption is a much better user experience.
This mechanism is not perfect. There may be false-negatives (racy writes that
are not detected). There should not be any false-positives (non-racy writes that
are detected as such). This is not a mechanism that makes putting a repo on a
networked filesystem "safe" or "supported", just *less* likely to cause
corruption.
Differential Revision: https://phab.mercurial-scm.org/D9952
author | Kyle Lippincott <spectral@google.com> |
---|---|
date | Wed, 03 Feb 2021 16:33:10 -0800 |
parents | 34a46d48d24e |
children | 011f5218ff2d |
line wrap: on
line source
$ cat >> $HGRCPATH << EOF > [extensions] > blackbox= > rebase= > mock=$TESTDIR/mockblackbox.py > > [blackbox] > track = command, commandfinish, tagscache > > [experimental] > evolution.createmarkers=True > EOF Create a repo with some tags $ hg init repo $ cd repo $ echo initial > foo $ hg -q commit -A -m initial $ hg tag -m 'test tag' test1 $ echo first > first $ hg -q commit -A -m first $ hg tag -m 'test2 tag' test2 $ hg -q up -r 0 $ echo newhead > newhead $ hg commit -A -m newhead adding newhead created new head $ hg tag -m 'test head 2 tag' head2 $ hg log -G -T '{rev}:{node|short} {tags} {desc}\n' @ 5:2942a772f72a tip test head 2 tag | o 4:042eb6bfcc49 head2 newhead | | o 3:c3cb30f2d2cd test2 tag | | | o 2:d75775ffbc6b test2 first | | | o 1:5f97d42da03f test tag |/ o 0:55482a6fb4b1 test1 initial Trigger tags cache population by doing something that accesses tags info $ hg tags tip 5:2942a772f72a head2 4:042eb6bfcc49 test2 2:d75775ffbc6b test1 0:55482a6fb4b1 $ cat .hg/cache/tags2-visible 5 2942a772f72a444bef4bef13874d515f50fa27b6 042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2 55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1 d75775ffbc6bca1794d300f5571272879bd280da test2 Hiding a non-tip changeset should change filtered hash and cause tags recompute $ hg debugobsolete -d '0 0' c3cb30f2d2cd0aae008cc91a07876e3c5131fd22 -u dummyuser 1 new obsolescence markers obsoleted 1 changesets $ hg tags tip 5:2942a772f72a head2 4:042eb6bfcc49 test1 0:55482a6fb4b1 $ cat .hg/cache/tags2-visible 5 2942a772f72a444bef4bef13874d515f50fa27b6 f34fbc9a9769ba9eff5aff3d008a6b49f85c08b1 042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2 55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1 $ hg blackbox -l 5 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> 2/2 cache hits/lookups in * seconds (glob) 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> writing .hg/cache/tags2-visible with 2 tags 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> blackbox -l 5 Hiding another changeset should cause the filtered hash to change $ hg debugobsolete -d '0 0' d75775ffbc6bca1794d300f5571272879bd280da -u dummyuser 1 new obsolescence markers obsoleted 1 changesets $ hg debugobsolete -d '0 0' 5f97d42da03fd56f3b228b03dfe48af5c0adf75b -u dummyuser 1 new obsolescence markers obsoleted 1 changesets $ hg tags tip 5:2942a772f72a head2 4:042eb6bfcc49 $ cat .hg/cache/tags2-visible 5 2942a772f72a444bef4bef13874d515f50fa27b6 2fce1eec33263d08a4d04293960fc73a555230e4 042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2 $ hg blackbox -l 5 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> 1/1 cache hits/lookups in * seconds (glob) 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> writing .hg/cache/tags2-visible with 1 tags 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> tags exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> blackbox -l 5 Resolving tags on an unfiltered repo writes a separate tags cache $ hg --hidden tags tip 5:2942a772f72a head2 4:042eb6bfcc49 test2 2:d75775ffbc6b test1 0:55482a6fb4b1 $ cat .hg/cache/tags2 5 2942a772f72a444bef4bef13874d515f50fa27b6 042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2 55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1 d75775ffbc6bca1794d300f5571272879bd280da test2 $ hg blackbox -l 5 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> --hidden tags 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> 2/2 cache hits/lookups in * seconds (glob) 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> writing .hg/cache/tags2 with 3 tags 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> --hidden tags exited 0 after * seconds (glob) 1970/01/01 00:00:00 bob @2942a772f72a444bef4bef13874d515f50fa27b6 (5000)> blackbox -l 5