tests/test-patch-offset.t
author Gregory Szorc <gregory.szorc@gmail.com>
Tue, 28 Aug 2018 15:02:48 -0700
changeset 39411 aeb551a3bb8a
parent 38365 bf953d218a91
child 39707 5abc47d4ca6b
permissions -rw-r--r--
cborutil: implement sans I/O decoder The vendored CBOR package decodes by calling read(n) on an object. There are a number of disadvantages to this: * Uses blocking I/O. If sufficient data is not available, the decoder will hang until it is. * No support for partial reads. If the read(n) returns less data than requested, the decoder raises an error. * Requires the use of a file like object. If the original data is in say a buffer, we need to "cast" it to e.g. a BytesIO to appease the decoder. In addition, the vendored CBOR decoder doesn't provide flexibility that we desire. Specifically: * It buffers indefinite length bytestrings instead of streaming them. * It doesn't allow limiting the set of types that can be decoded. This property is useful when implementing a "hardened" decoder that is less susceptible to abusive input. * It doesn't provide sufficient "hook points" and introspection to institute checks around behavior. These are useful for implementing a "hardened" decoder. This all adds up to a reasonable set of justifications for writing our own decoder. So, this commit implements our own CBOR decoder. At the heart of the decoder is a function that decodes a single "item" from a buffer. This item can be a complete simple value or a special value, such as "start of array." Using this function, we can build a decoder that effectively iterates over the stream of decoded items and builds up higher-level values, such as arrays, maps, sets, and indefinite length bytestrings. And we can do this without performing I/O in the decoder itself. The core of the sans I/O decoder will probably not be used directly. Instead, it is expected that we'll build utility functions for invoking the decoder given specific input types. This will allow extreme flexibility in how data is delivered to the decoder. I'm pretty happy with the state of the decoder modulo the TODO items to track wanted features to help with a "hardened" decoder. The one thing I could be convinced to change is the handling of semantic tags. Since we only support a single semantic tag (sets), I thought it would be easier to handle them inline in decodeitem(). This is simpler now. But if we add support for other semantic tags, it will likely be easier to move semantic tag handling outside of decodeitem(). But, properly supporting semantic tags opens up a whole can of worms, as many semantic tags imply new types. I'm optimistic we won't need these in Mercurial. But who knows. I'm also pretty happy with the test coverage. Writing comprehensive tests for partial decoding did flush out a handful of bugs. One general improvement to testing would be fuzz testing for partial decoding. I may implement that later. I also anticipate switching the wire protocol code to this new decoder will flush out any lingering bugs. Differential Revision: https://phab.mercurial-scm.org/D4414
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
     1
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
     2
  $ cat > writepatterns.py <<EOF
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
     3
  > import sys
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
     4
  > 
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
     5
  > path = sys.argv[1]
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
     6
  > patterns = sys.argv[2:]
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
     7
  > 
36020
bfc9ab6c1bec py3: replace file() with open() in test-patch-offset.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32940
diff changeset
     8
  > fp = open(path, 'wb')
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
     9
  > for pattern in patterns:
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    10
  >     count = int(pattern[0:-1])
38365
bf953d218a91 py3: encode sys.argv to bytes using .encode()
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36020
diff changeset
    11
  >     char = pattern[-1].encode('utf8') + b'\n'
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    12
  >     fp.write(char*count)
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    13
  > fp.close()
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    14
  > EOF
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
    15
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    16
prepare repo
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
    17
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    18
  $ hg init a
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    19
  $ cd a
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    20
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    21
These initial lines of Xs were not in the original file used to generate
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    22
the patch.  So all the patch hunks need to be applied to a constant offset
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    23
within this file.  If the offset isn't tracked then the hunks can be
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    24
applied to the wrong lines of this file.
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
    25
32940
75be14993fda cleanup: use $PYTHON to run python in many more tests
Augie Fackler <augie@google.com>
parents: 23749
diff changeset
    26
  $ $PYTHON ../writepatterns.py a 34X 10A 1B 10A 1C 10A 1B 10A 1D 10A 1B 10A 1E 10A 1B 10A
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    27
  $ hg commit -Am adda
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    28
  adding a
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
    29
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    30
This is a cleaner patch generated via diff
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    31
In this case it reproduces the problem when
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    32
the output of hg export does not
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    33
import patch
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
    34
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    35
  $ hg import -v -m 'b' -d '2 0' - <<EOF
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    36
  > --- a/a	2009-12-08 19:26:17.000000000 -0800
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    37
  > +++ b/a	2009-12-08 19:26:17.000000000 -0800
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    38
  > @@ -9,7 +9,7 @@
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    39
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    40
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    41
  >  B
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    42
  > -A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    43
  > +a
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    44
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    45
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    46
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    47
  > @@ -53,7 +53,7 @@
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    48
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    49
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    50
  >  B
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    51
  > -A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    52
  > +a
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    53
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    54
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    55
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    56
  > @@ -75,7 +75,7 @@
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    57
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    58
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    59
  >  B
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    60
  > -A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    61
  > +a
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    62
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    63
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    64
  >  A
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    65
  > EOF
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    66
  applying patch from stdin
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    67
  patching file a
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    68
  Hunk #1 succeeded at 43 (offset 34 lines).
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    69
  Hunk #2 succeeded at 87 (offset 34 lines).
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    70
  Hunk #3 succeeded at 109 (offset 34 lines).
23749
a387b0390082 localrepo: show headline notes in commitctx before showing filenames
Mads Kiilerich <madski@unity3d.com>
parents: 16913
diff changeset
    71
  committing files:
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    72
  a
23749
a387b0390082 localrepo: show headline notes in commitctx before showing filenames
Mads Kiilerich <madski@unity3d.com>
parents: 16913
diff changeset
    73
  committing manifest
a387b0390082 localrepo: show headline notes in commitctx before showing filenames
Mads Kiilerich <madski@unity3d.com>
parents: 16913
diff changeset
    74
  committing changelog
15194
0705f2ac79d6 import: simplify status reporting logic (and make it more I18N-friendly)
Greg Ward <greg@gerg.ca>
parents: 12477
diff changeset
    75
  created 189885cecb41
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
    76
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    77
compare imported changes against reference file
10135
9a4034b630c4 patch: better handling of sequence of offset patch hunks (issue1941)
Greg Onufer <gonufer@jazzhaiku.com>
parents:
diff changeset
    78
32940
75be14993fda cleanup: use $PYTHON to run python in many more tests
Augie Fackler <augie@google.com>
parents: 23749
diff changeset
    79
  $ $PYTHON ../writepatterns.py aref 34X 10A 1B 1a 9A 1C 10A 1B 10A 1D 10A 1B 1a 9A 1E 10A 1B 1a 9A
12477
e68cd3a90599 tests: unify test-patch-offset
Matt Mackall <mpm@selenic.com>
parents: 10265
diff changeset
    80
  $ diff aref a
16913
f2719b387380 tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents: 15194
diff changeset
    81
f2719b387380 tests: add missing trailing 'cd ..'
Mads Kiilerich <mads@kiilerich.com>
parents: 15194
diff changeset
    82
  $ cd ..