tests/test-largefiles-small-disk.t
author Gregory Szorc <gregory.szorc@gmail.com>
Mon, 10 Dec 2018 18:04:12 +0000
changeset 40923 3ed77780f4a6
parent 37439 556984ae0005
child 41738 c70bdd222dcd
permissions -rw-r--r--
wireprotov2: send linknodes to emitfilerevisions() Previously, linknodes were calculated within emitfilerevisions() by using filectx.introrev(), which would always use the linkrev/linknode as recorded by storage. This is wrong for cases where the receiver doesn't have the changeset the linknode refers to. This commit changes the logic for linknode emission so the mapping of filenode to linknode is computed by the caller and passed into emitfilerevisions(). As part of the change, linknodes for "filesdata" in the haveparents=False case are now correct: the existing code performed a manifest walk and it was trivial to plug in the correct linknode. However, behavior for the haveparents=True case is still wrong because it relies on filtering linkrevs against the outgoing set in order to determine what to send. This will be fixed in a subsequent commit. The change test test-wireproto-exchangev2-shallow.t is a bit wonky. The test repo has 6 revisions. The changed test is performing a shallow clone with depth=1. So, only file data for revision 5 is present locally. So, the new behavior of associating the linknode with revision 5 for every file revision seems correct. Of course, when backfilling old revisions, we'll want to update the linknode. But this problem requires wire protocol support and we'll cross that bridge later. Differential Revision: https://phab.mercurial-scm.org/D5405
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     1
Test how largefiles abort in case the disk runs full
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     2
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     3
  $ cat > criple.py <<EOF
33962
991134261057 tests: update test-largefiles-small-disk to pass our import checker
Augie Fackler <raf@durin42.com>
parents: 31207
diff changeset
     4
  > from __future__ import absolute_import
991134261057 tests: update test-largefiles-small-disk to pass our import checker
Augie Fackler <raf@durin42.com>
parents: 31207
diff changeset
     5
  > import errno
991134261057 tests: update test-largefiles-small-disk to pass our import checker
Augie Fackler <raf@durin42.com>
parents: 31207
diff changeset
     6
  > import os
991134261057 tests: update test-largefiles-small-disk to pass our import checker
Augie Fackler <raf@durin42.com>
parents: 31207
diff changeset
     7
  > import shutil
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     8
  > from mercurial import util
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     9
  > #
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    10
  > # this makes the original largefiles code abort:
31207
1ef37b16b8e8 dirstate: try to use hardlink to backup dirstate
Jun Wu <quark@fb.com>
parents: 30181
diff changeset
    11
  > _origcopyfileobj = shutil.copyfileobj
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    12
  > def copyfileobj(fsrc, fdst, length=16*1024):
31207
1ef37b16b8e8 dirstate: try to use hardlink to backup dirstate
Jun Wu <quark@fb.com>
parents: 30181
diff changeset
    13
  >     # allow journal files (used by transaction) to be written
36113
a42817fede27 py3: use b'' in inline extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34661
diff changeset
    14
  >     if b'journal.' in fdst.name:
31207
1ef37b16b8e8 dirstate: try to use hardlink to backup dirstate
Jun Wu <quark@fb.com>
parents: 30181
diff changeset
    15
  >         return _origcopyfileobj(fsrc, fdst, length)
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    16
  >     fdst.write(fsrc.read(4))
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    17
  >     raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC))
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    18
  > shutil.copyfileobj = copyfileobj
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    19
  > #
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    20
  > # this makes the rewritten code abort:
30181
7356e6b1f5b8 util: increase filechunkiter size to 128k
Mads Kiilerich <madski@unity3d.com>
parents: 20063
diff changeset
    21
  > def filechunkiter(f, size=131072, limit=None):
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    22
  >     yield f.read(4)
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    23
  >     raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC))
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    24
  > util.filechunkiter = filechunkiter
15572
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    25
  > #
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    26
  > def oslink(src, dest):
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    27
  >     raise OSError("no hardlinks, try copying instead")
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    28
  > util.oslink = oslink
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    29
  > EOF
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    30
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    31
  $ echo "[extensions]" >> $HGRCPATH
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    32
  $ echo "largefiles =" >> $HGRCPATH
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    33
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    34
  $ hg init alice
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    35
  $ cd alice
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    36
  $ echo "this is a very big file" > big
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    37
  $ hg add --large big
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    38
  $ hg commit --config extensions.criple=$TESTTMP/criple.py -m big
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    39
  abort: No space left on device
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    40
  [255]
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    41
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    42
The largefile is not created in .hg/largefiles:
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    43
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    44
  $ ls .hg/largefiles
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    45
  dirstate
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    46
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    47
The user cache is not even created:
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    48
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    49
  >>> import os; os.path.exists("$HOME/.cache/largefiles/")
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    50
  False
15572
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    51
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    52
Make the commit with space on the device:
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    53
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    54
  $ hg commit -m big
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    55
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    56
Now make a clone with a full disk, and make sure lfutil.link function
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    57
makes copies instead of hardlinks:
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    58
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    59
  $ cd ..
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    60
  $ hg --config extensions.criple=$TESTTMP/criple.py clone --pull alice bob
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    61
  requesting all changes
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    62
  adding changesets
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    63
  adding manifests
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    64
  adding file changes
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    65
  added 1 changesets with 1 changes to 1 files
34661
eb586ed5d8ce transaction-summary: show the range of new revisions upon pull/unbundle (BC)
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 33962
diff changeset
    66
  new changesets 390cf214e9ac
15572
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    67
  updating to branch default
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    68
  getting changed largefiles
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    69
  abort: No space left on device
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    70
  [255]
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    71
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    72
The largefile is not created in .hg/largefiles:
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    73
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    74
  $ ls bob/.hg/largefiles
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 18459
diff changeset
    75
  dirstate