Yuya Nishihara <yuya@tcha.org> [Wed, 05 Sep 2018 20:57:38 +0900] rev 39420
base85: fix leak on error return from b85decode()
Spotted by ASAN.
We don't need to initialize 'out' to NULL, but I decided to do that for
clarity.
Augie Fackler <augie@google.com> [Tue, 04 Sep 2018 12:16:28 -0400] rev 39419
merge with stable
Augie Fackler <raf@durin42.com> [Tue, 04 Sep 2018 11:59:12 -0400] rev 39418
Added signature for changeset
ede3bf31fe63
Augie Fackler <raf@durin42.com> [Tue, 04 Sep 2018 11:59:09 -0400] rev 39417
Added tag 4.7.1 for changeset
ede3bf31fe63
Yuya Nishihara <yuya@tcha.org> [Fri, 31 Aug 2018 21:44:24 +0900] rev 39416
hgweb: load revcount + 1 entries to fill nextentry in log page (
issue5972)
"revcount + 1" is moved to the call site to make it clearer.
Yuya Nishihara <yuya@tcha.org> [Tue, 04 Sep 2018 13:29:21 +0900] rev 39415
revlog: fix size of Python nodetree object
Follows up
9f097214fbf3.
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Sep 2018 23:03:19 +0900] rev 39414
revert: stabilize status message of chunks selected interactively
Unfortunately, patch.filterpatch() doesn't preserve the order of the input
files. We have to sort them manually.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 29 Aug 2018 14:29:01 -0700] rev 39413
cborutil: add a buffering decoder
The sansiodecoder leaves it up to the callers to feed in data that
wasn't fully consumed last time.
This commit implements a decoder that performs buffering of
leftover chunks from the previous invocation. It otherwise
behaves identically to sansiodecoder.
Differential Revision: https://phab.mercurial-scm.org/D4434
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 31 Aug 2018 15:54:17 -0700] rev 39412
cborutil: remove readindefinitebytestringtoiter()
This was implemented as part of implementing streaming encoding.
It was never used outside of tests.
Now that we have a full CBOR decoder, it can be used for incremental
decoding of indefinite-length byte strings.
This also removes the last use of the vendored cbor2 package from this
module.
Differential Revision: https://phab.mercurial-scm.org/D4433
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 28 Aug 2018 15:02:48 -0700] rev 39411
cborutil: implement sans I/O decoder
The vendored CBOR package decodes by calling read(n) on an object.
There are a number of disadvantages to this:
* Uses blocking I/O. If sufficient data is not available, the decoder
will hang until it is.
* No support for partial reads. If the read(n) returns less data than
requested, the decoder raises an error.
* Requires the use of a file like object. If the original data is in
say a buffer, we need to "cast" it to e.g. a BytesIO to appease the
decoder.
In addition, the vendored CBOR decoder doesn't provide flexibility
that we desire. Specifically:
* It buffers indefinite length bytestrings instead of streaming them.
* It doesn't allow limiting the set of types that can be decoded. This
property is useful when implementing a "hardened" decoder that is
less susceptible to abusive input.
* It doesn't provide sufficient "hook points" and introspection to
institute checks around behavior. These are useful for implementing
a "hardened" decoder.
This all adds up to a reasonable set of justifications for writing our
own decoder.
So, this commit implements our own CBOR decoder.
At the heart of the decoder is a function that decodes a single "item"
from a buffer. This item can be a complete simple value or a special
value, such as "start of array." Using this function, we can build a
decoder that effectively iterates over the stream of decoded items and
builds up higher-level values, such as arrays, maps, sets, and indefinite
length bytestrings. And we can do this without performing I/O in the
decoder itself.
The core of the sans I/O decoder will probably not be used directly.
Instead, it is expected that we'll build utility functions for invoking
the decoder given specific input types. This will allow extreme
flexibility in how data is delivered to the decoder.
I'm pretty happy with the state of the decoder modulo the TODO items
to track wanted features to help with a "hardened" decoder. The one
thing I could be convinced to change is the handling of semantic tags.
Since we only support a single semantic tag (sets), I thought it would
be easier to handle them inline in decodeitem(). This is simpler now.
But if we add support for other semantic tags, it will likely be easier
to move semantic tag handling outside of decodeitem(). But, properly
supporting semantic tags opens up a whole can of worms, as many
semantic tags imply new types. I'm optimistic we won't need these in
Mercurial. But who knows.
I'm also pretty happy with the test coverage. Writing comprehensive
tests for partial decoding did flush out a handful of bugs. One
general improvement to testing would be fuzz testing for partial
decoding. I may implement that later. I also anticipate switching the
wire protocol code to this new decoder will flush out any lingering
bugs.
Differential Revision: https://phab.mercurial-scm.org/D4414
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 28 Aug 2018 15:22:06 -0700] rev 39410
tests: remove use of string in CBOR test
We don't use the CBOR string major type in the wire protocol. Let's
not test for it.
Differential Revision: https://phab.mercurial-scm.org/D4413
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 28 Aug 2018 20:27:36 -0700] rev 39409
internals: document CBOR utilization
I spoke with some people at Mozilla about CBOR and they advised me
that we should be careful about the subset of CBOR we use in order
to mitigate security, performance, and compatibility concerns.
This commit establishes a document that attempts to formalize our
use of CBOR.
Its main limitations are on what types are allowed. It explicitly
enumerates which types are supported. Notable missing features
include:
* Indefinite-length arrays and maps
* Text strings (bytes all the way)
* Floats
* Date/time types
* Big integers
* Use of indefinite-length byte strings for map keys, values in
containers.
If we have a need for any of these, we can have a discussion about
them when the time comes.
Differential Revision: https://phab.mercurial-scm.org/D4412
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Sep 2018 13:56:53 +0300] rev 39408
py3: add new passing test to whitelist found by buildbot
Differential Revision: https://phab.mercurial-scm.org/D4451
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Sep 2018 13:53:30 +0300] rev 39407
py3: alias xrange to range in tests/test-fastannotate-revmap.py
xrange is not available on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4450
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Sep 2018 13:52:49 +0300] rev 39406
py3: add r'' prefix to prevent b'' being prepended
The keys of keyword arguments should be str.
# skip-blame because we are adding just r'' prefixes.
Differential Revision: https://phab.mercurial-scm.org/D4449