tests/test-symlink-placeholder.t
author Gregory Szorc <gregory.szorc@gmail.com>
Tue, 28 Aug 2018 15:02:48 -0700
changeset 39411 aeb551a3bb8a
parent 38080 0a10f142299d
child 52060 8b7123c8947b
permissions -rw-r--r--
cborutil: implement sans I/O decoder The vendored CBOR package decodes by calling read(n) on an object. There are a number of disadvantages to this: * Uses blocking I/O. If sufficient data is not available, the decoder will hang until it is. * No support for partial reads. If the read(n) returns less data than requested, the decoder raises an error. * Requires the use of a file like object. If the original data is in say a buffer, we need to "cast" it to e.g. a BytesIO to appease the decoder. In addition, the vendored CBOR decoder doesn't provide flexibility that we desire. Specifically: * It buffers indefinite length bytestrings instead of streaming them. * It doesn't allow limiting the set of types that can be decoded. This property is useful when implementing a "hardened" decoder that is less susceptible to abusive input. * It doesn't provide sufficient "hook points" and introspection to institute checks around behavior. These are useful for implementing a "hardened" decoder. This all adds up to a reasonable set of justifications for writing our own decoder. So, this commit implements our own CBOR decoder. At the heart of the decoder is a function that decodes a single "item" from a buffer. This item can be a complete simple value or a special value, such as "start of array." Using this function, we can build a decoder that effectively iterates over the stream of decoded items and builds up higher-level values, such as arrays, maps, sets, and indefinite length bytestrings. And we can do this without performing I/O in the decoder itself. The core of the sans I/O decoder will probably not be used directly. Instead, it is expected that we'll build utility functions for invoking the decoder given specific input types. This will allow extreme flexibility in how data is delivered to the decoder. I'm pretty happy with the state of the decoder modulo the TODO items to track wanted features to help with a "hardened" decoder. The one thing I could be convinced to change is the handling of semantic tags. Since we only support a single semantic tag (sets), I thought it would be easier to handle them inline in decodeitem(). This is simpler now. But if we add support for other semantic tags, it will likely be easier to move semantic tag handling outside of decodeitem(). But, properly supporting semantic tags opens up a whole can of worms, as many semantic tags imply new types. I'm optimistic we won't need these in Mercurial. But who knows. I'm also pretty happy with the test coverage. Writing comprehensive tests for partial decoding did flush out a handful of bugs. One general improvement to testing would be fuzz testing for partial decoding. I may implement that later. I also anticipate switching the wire protocol code to this new decoder will flush out any lingering bugs. Differential Revision: https://phab.mercurial-scm.org/D4414
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
22046
7a9cbb315d84 tests: replace exit 80 with #require
Matt Mackall <mpm@selenic.com>
parents: 19650
diff changeset
     1
#require symlink
15441
e0348815e806 tests: use 'hghave symlink' for tests using symlinks
Mads Kiilerich <mads@kiilerich.com>
parents: 15348
diff changeset
     2
15348
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     3
Create extension that can disable symlink support:
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     4
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     5
  $ cat > nolink.py <<EOF
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     6
  > from mercurial import extensions, util
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     7
  > def setflags(orig, f, l, x):
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     8
  >     pass
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     9
  > def checklink(orig, path):
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    10
  >     return False
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    11
  > def extsetup(ui):
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    12
  >     extensions.wrapfunction(util, 'setflags', setflags)
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    13
  >     extensions.wrapfunction(util, 'checklink', checklink)
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    14
  > EOF
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    15
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    16
  $ hg init unix-repo
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    17
  $ cd unix-repo
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    18
  $ echo foo > a
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    19
  $ ln -s a b
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    20
  $ hg ci -Am0
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    21
  adding a
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    22
  adding b
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    23
  $ cd ..
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    24
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    25
Simulate a checkout shared on NFS/Samba:
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    26
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    27
  $ hg clone -q unix-repo shared
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    28
  $ cd shared
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    29
  $ rm b
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    30
  $ echo foo > b
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    31
  $ hg --config extensions.n=$TESTTMP/nolink.py status --debug
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    32
  ignoring suspect symlink placeholder "b"
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    33
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    34
Make a clone using placeholders:
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    35
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    36
  $ hg --config extensions.n=$TESTTMP/nolink.py clone . ../win-repo
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    37
  updating to branch default
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    38
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    39
  $ cd ../win-repo
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    40
  $ cat b
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    41
  a (no-eol)
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    42
  $ hg --config extensions.n=$TESTTMP/nolink.py st --debug
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    43
19650
36f48c7d5944 localrepo.status: ignore empty symlink placeholders
Siddharth Agarwal <sid0@fb.com>
parents: 16913
diff changeset
    44
Empty placeholder:
36f48c7d5944 localrepo.status: ignore empty symlink placeholders
Siddharth Agarwal <sid0@fb.com>
parents: 16913
diff changeset
    45
36f48c7d5944 localrepo.status: ignore empty symlink placeholders
Siddharth Agarwal <sid0@fb.com>
parents: 16913
diff changeset
    46
  $ rm b
36f48c7d5944 localrepo.status: ignore empty symlink placeholders
Siddharth Agarwal <sid0@fb.com>
parents: 16913
diff changeset
    47
  $ touch b
36f48c7d5944 localrepo.status: ignore empty symlink placeholders
Siddharth Agarwal <sid0@fb.com>
parents: 16913
diff changeset
    48
  $ hg --config extensions.n=$TESTTMP/nolink.py st --debug
36f48c7d5944 localrepo.status: ignore empty symlink placeholders
Siddharth Agarwal <sid0@fb.com>
parents: 16913
diff changeset
    49
  ignoring suspect symlink placeholder "b"
36f48c7d5944 localrepo.status: ignore empty symlink placeholders
Siddharth Agarwal <sid0@fb.com>
parents: 16913
diff changeset
    50
15348
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    51
Write binary data to the placeholder:
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    52
38080
0a10f142299d py3: suppress the output from .write() calls in few tests
Pulkit Goyal <7895pulkit@gmail.com>
parents: 22046
diff changeset
    53
  >>> open('b', 'w').write('this is a binary\0') and None
15348
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    54
  $ hg --config extensions.n=$TESTTMP/nolink.py st --debug
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    55
  ignoring suspect symlink placeholder "b"
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    56
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    57
Write a long string to the placeholder:
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    58
38080
0a10f142299d py3: suppress the output from .write() calls in few tests
Pulkit Goyal <7895pulkit@gmail.com>
parents: 22046
diff changeset
    59
  >>> open('b', 'w').write('this' * 1000) and None
15348
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    60
  $ hg --config extensions.n=$TESTTMP/nolink.py st --debug
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    61
  ignoring suspect symlink placeholder "b"
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    62
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    63
Commit shouldn't succeed:
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    64
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    65
  $ hg --config extensions.n=$TESTTMP/nolink.py ci -m1
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    66
  nothing changed
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    67
  [1]
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    68
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    69
Write a valid string to the placeholder:
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    70
38080
0a10f142299d py3: suppress the output from .write() calls in few tests
Pulkit Goyal <7895pulkit@gmail.com>
parents: 22046
diff changeset
    71
  >>> open('b', 'w').write('this') and None
15348
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    72
  $ hg --config extensions.n=$TESTTMP/nolink.py st --debug
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    73
  M b
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    74
  $ hg --config extensions.n=$TESTTMP/nolink.py ci -m1
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    75
  $ hg manifest tip --verbose
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    76
  644   a
c681e478c429 windows: sanity-check symlink placeholders
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    77
  644 @ b
16913
f2719b387380 tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents: 15441
diff changeset
    78
f2719b387380 tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents: 15441
diff changeset
    79
  $ cd ..