tests/test-transaction-rollback-on-revlog-split.t
author Joerg Sonnenberger <joerg@bec.de>
Wed, 19 May 2021 13:46:19 +0200
changeset 47285 46b828b85eb7
parent 47284 21ed126bab53
child 47286 18415fc918a1
permissions -rw-r--r--
revlog: update data file record before index rename When migrating from inline to non-inline data storage, the data file is recorded initially as zero sized so that it is removed on failure. But the record has to be updated before the index is renamed, otherwise data is lost on rollback. Differential Revision: https://phab.mercurial-scm.org/D10725
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
     1
Test correctness of revlog inline -> non-inline transition
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
     2
----------------------------------------------------------
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
     3
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
     4
Helper extension to intercept renames.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
     5
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
     6
  $ cat > $TESTTMP/intercept_rename.py << EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
     7
  > import os
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
     8
  > import sys
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
     9
  > from mercurial import extensions, util
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    10
  > 
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    11
  > def extsetup(ui):
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    12
  >     def close(orig, *args, **kwargs):
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    13
  >         path = args[0]._atomictempfile__name
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    14
  >         if path.endswith(b'/.hg/store/data/file.i'):
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    15
  >             os._exit(80)
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    16
  >         return orig(*args, **kwargs)
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    17
  >     extensions.wrapfunction(util.atomictempfile, 'close', close)
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    18
  > EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    19
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    20
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    21
Test offset computation to correctly factor in the index entries themselve.
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    22
Also test that the new data size has the correct size if the transaction is aborted
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    23
after the index has been replaced.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    24
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    25
Test repo has one small, one moderate and one big change. The clone has
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    26
the small and moderate change and will transition to non-inline storage when
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    27
adding the big change.
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    28
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    29
  $ hg init troffset-computation --config format.revlog-compression=none
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    30
  $ cd troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    31
  $ printf '% 20d' '1' > file
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    32
  $ hg commit -Aqm_
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    33
  $ printf '% 1024d' '1' > file
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    34
  $ hg commit -Aqm_
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    35
  $ dd if=/dev/zero of=file bs=1k count=128 > /dev/null 2>&1
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    36
  $ hg commit -Aqm_
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    37
  $ cd ..
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    38
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    39
  $ hg clone -r 1 troffset-computation troffset-computation-copy --config format.revlog-compression=none -q
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    40
  $ cd troffset-computation-copy
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    41
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    42
Reference size:
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    43
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    44
  $ f -s .hg/store/data/file*
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    45
  .hg/store/data/file.i: size=1174
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    46
47284
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    47
  $ cat > .hg/hgrc <<EOF
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    48
  > [hooks]
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    49
  > pretxnchangegroup = python:$TESTDIR/helper-killhook.py:killme
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    50
  > EOF
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    51
#if chg
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    52
  $ hg pull ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    53
  pulling from ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    54
  [255]
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    55
#else
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    56
  $ hg pull ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    57
  pulling from ../troffset-computation
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    58
  [80]
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    59
#endif
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    60
  $ cat .hg/store/journal | tr -s '\000' ' ' | grep data/file | tail -1
21ed126bab53 revlog: fix index computation during inline->non-inline transition
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
    61
  data/file.i 128
47285
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    62
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    63
The first file.i entry should match the size above.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    64
The first file.d entry is the temporary record during the split,
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    65
the second entry after the split happened. The sum of the second file.d
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    66
and the second file.i entry should match the first file.i entry.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    67
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    68
  $ cat .hg/store/journal | tr -s '\000' ' ' | grep data/file
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    69
  data/file.i 1174
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    70
  data/file.d 0
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    71
  data/file.d 1046
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    72
  data/file.i 128
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    73
  $ cd ..
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    74
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    75
Now retry the same but intercept the rename of the index and check that
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    76
the journal does not contain the new index size. This demonstrates the edge case
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    77
where the data file is left as garbage.
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    78
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    79
  $ hg clone -r 1 troffset-computation troffset-computation-copy2 --config format.revlog-compression=none -q
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    80
  $ cd troffset-computation-copy2
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    81
  $ cat > .hg/hgrc <<EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    82
  > [extensions]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    83
  > intercept_rename = $TESTTMP/intercept_rename.py
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    84
  > [hooks]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    85
  > pretxnchangegroup = python:$TESTDIR/helper-killhook.py:killme
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    86
  > EOF
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    87
#if chg
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    88
  $ hg pull ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    89
  pulling from ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    90
  [255]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    91
#else
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    92
  $ hg pull ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    93
  pulling from ../troffset-computation
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    94
  [80]
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    95
#endif
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    96
  $ cat .hg/store/journal | tr -s '\000' ' ' | grep data/file
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    97
  data/file.i 1174
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    98
  data/file.d 0
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
    99
  data/file.d 1046
46b828b85eb7 revlog: update data file record before index rename
Joerg Sonnenberger <joerg@bec.de>
parents: 47284
diff changeset
   100
  $ cd ..