annotate tests/test-transaction-rollback-on-revlog-split.t @ 47285:46b828b85eb7

revlog: update data file record before index rename When migrating from inline to non-inline data storage, the data file is recorded initially as zero sized so that it is removed on failure. But the record has to be updated before the index is renamed, otherwise data is lost on rollback. Differential Revision: https://phab.mercurial-scm.org/D10725
author Joerg Sonnenberger <joerg@bec.de>
date Wed, 19 May 2021 13:46:19 +0200
parents 21ed126bab53
children 18415fc918a1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
1 Test correctness of revlog inline -> non-inline transition
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
2 ----------------------------------------------------------
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
3
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
4 Helper extension to intercept renames.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
5
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
6 $ cat > $TESTTMP/intercept_rename.py << EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
7 > import os
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
8 > import sys
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
9 > from mercurial import extensions, util
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
10 >
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
11 > def extsetup(ui):
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
12 > def close(orig, *args, **kwargs):
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
13 > path = args[0]._atomictempfile__name
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
14 > if path.endswith(b'/.hg/store/data/file.i'):
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
15 > os._exit(80)
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
16 > return orig(*args, **kwargs)
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
17 > extensions.wrapfunction(util.atomictempfile, 'close', close)
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
18 > EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
19
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
20
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
21 Test offset computation to correctly factor in the index entries themselve.
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
22 Also test that the new data size has the correct size if the transaction is aborted
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
23 after the index has been replaced.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
24
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
25 Test repo has one small, one moderate and one big change. The clone has
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
26 the small and moderate change and will transition to non-inline storage when
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
27 adding the big change.
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
28
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
29 $ hg init troffset-computation --config format.revlog-compression=none
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
30 $ cd troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
31 $ printf '% 20d' '1' > file
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
32 $ hg commit -Aqm_
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
33 $ printf '% 1024d' '1' > file
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
34 $ hg commit -Aqm_
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
35 $ dd if=/dev/zero of=file bs=1k count=128 > /dev/null 2>&1
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
36 $ hg commit -Aqm_
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
37 $ cd ..
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
38
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
39 $ hg clone -r 1 troffset-computation troffset-computation-copy --config format.revlog-compression=none -q
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
40 $ cd troffset-computation-copy
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
41
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
42 Reference size:
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
43
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
44 $ f -s .hg/store/data/file*
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
45 .hg/store/data/file.i: size=1174
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
46
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
47 $ cat > .hg/hgrc <<EOF
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
48 > [hooks]
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
49 > pretxnchangegroup = python:$TESTDIR/helper-killhook.py:killme
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
50 > EOF
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
51 #if chg
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
52 $ hg pull ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
53 pulling from ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
54 [255]
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
55 #else
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
56 $ hg pull ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
57 pulling from ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
58 [80]
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
59 #endif
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
60 $ cat .hg/store/journal | tr -s '\000' ' ' | grep data/file | tail -1
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
61 data/file.i 128
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
62
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
63 The first file.i entry should match the size above.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
64 The first file.d entry is the temporary record during the split,
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
65 the second entry after the split happened. The sum of the second file.d
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
66 and the second file.i entry should match the first file.i entry.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
67
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
68 $ cat .hg/store/journal | tr -s '\000' ' ' | grep data/file
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
69 data/file.i 1174
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
70 data/file.d 0
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
71 data/file.d 1046
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
72 data/file.i 128
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
73 $ cd ..
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
74
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
75 Now retry the same but intercept the rename of the index and check that
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
76 the journal does not contain the new index size. This demonstrates the edge case
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
77 where the data file is left as garbage.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
78
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
79 $ hg clone -r 1 troffset-computation troffset-computation-copy2 --config format.revlog-compression=none -q
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
80 $ cd troffset-computation-copy2
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
81 $ cat > .hg/hgrc <<EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
82 > [extensions]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
83 > intercept_rename = $TESTTMP/intercept_rename.py
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
84 > [hooks]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
85 > pretxnchangegroup = python:$TESTDIR/helper-killhook.py:killme
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
86 > EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
87 #if chg
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
88 $ hg pull ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
89 pulling from ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
90 [255]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
91 #else
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
92 $ hg pull ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
93 pulling from ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
94 [80]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
95 #endif
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
96 $ cat .hg/store/journal | tr -s '\000' ' ' | grep data/file
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
97 data/file.i 1174
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
98 data/file.d 0
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
99 data/file.d 1046
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
100 $ cd ..