Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 04 Sep 2018 17:16:29 +0300] rev 39445
py3: don't return the revid as unicode in hgext/convert/subversion.py
I tried digging why u'' was added in first place, and I was unable to found
something relevant. This might be because some API's takes unicodes, I am not
sure.
Differential Revision: https://phab.mercurial-scm.org/D4454
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 04 Sep 2018 17:15:17 +0300] rev 39444
py3: make sure we pass str in os.sysconf in hgext/convert/common.py
# skip-blame because just r'' prefix
Differential Revision: https://phab.mercurial-scm.org/D4453
Augie Fackler <augie@google.com> [Tue, 04 Sep 2018 12:16:28 -0400] rev 39443
merge with stable
Yuya Nishihara <yuya@tcha.org> [Tue, 04 Sep 2018 13:29:21 +0900] rev 39442
revlog: fix size of Python nodetree object
Follows up 9f097214fbf3.
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Sep 2018 23:03:19 +0900] rev 39441
revert: stabilize status message of chunks selected interactively
Unfortunately, patch.filterpatch() doesn't preserve the order of the input
files. We have to sort them manually.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 29 Aug 2018 14:29:01 -0700] rev 39440
cborutil: add a buffering decoder
The sansiodecoder leaves it up to the callers to feed in data that
wasn't fully consumed last time.
This commit implements a decoder that performs buffering of
leftover chunks from the previous invocation. It otherwise
behaves identically to sansiodecoder.
Differential Revision: https://phab.mercurial-scm.org/D4434
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 31 Aug 2018 15:54:17 -0700] rev 39439
cborutil: remove readindefinitebytestringtoiter()
This was implemented as part of implementing streaming encoding.
It was never used outside of tests.
Now that we have a full CBOR decoder, it can be used for incremental
decoding of indefinite-length byte strings.
This also removes the last use of the vendored cbor2 package from this
module.
Differential Revision: https://phab.mercurial-scm.org/D4433
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 28 Aug 2018 15:02:48 -0700] rev 39438
cborutil: implement sans I/O decoder
The vendored CBOR package decodes by calling read(n) on an object.
There are a number of disadvantages to this:
* Uses blocking I/O. If sufficient data is not available, the decoder
will hang until it is.
* No support for partial reads. If the read(n) returns less data than
requested, the decoder raises an error.
* Requires the use of a file like object. If the original data is in
say a buffer, we need to "cast" it to e.g. a BytesIO to appease the
decoder.
In addition, the vendored CBOR decoder doesn't provide flexibility
that we desire. Specifically:
* It buffers indefinite length bytestrings instead of streaming them.
* It doesn't allow limiting the set of types that can be decoded. This
property is useful when implementing a "hardened" decoder that is
less susceptible to abusive input.
* It doesn't provide sufficient "hook points" and introspection to
institute checks around behavior. These are useful for implementing
a "hardened" decoder.
This all adds up to a reasonable set of justifications for writing our
own decoder.
So, this commit implements our own CBOR decoder.
At the heart of the decoder is a function that decodes a single "item"
from a buffer. This item can be a complete simple value or a special
value, such as "start of array." Using this function, we can build a
decoder that effectively iterates over the stream of decoded items and
builds up higher-level values, such as arrays, maps, sets, and indefinite
length bytestrings. And we can do this without performing I/O in the
decoder itself.
The core of the sans I/O decoder will probably not be used directly.
Instead, it is expected that we'll build utility functions for invoking
the decoder given specific input types. This will allow extreme
flexibility in how data is delivered to the decoder.
I'm pretty happy with the state of the decoder modulo the TODO items
to track wanted features to help with a "hardened" decoder. The one
thing I could be convinced to change is the handling of semantic tags.
Since we only support a single semantic tag (sets), I thought it would
be easier to handle them inline in decodeitem(). This is simpler now.
But if we add support for other semantic tags, it will likely be easier
to move semantic tag handling outside of decodeitem(). But, properly
supporting semantic tags opens up a whole can of worms, as many
semantic tags imply new types. I'm optimistic we won't need these in
Mercurial. But who knows.
I'm also pretty happy with the test coverage. Writing comprehensive
tests for partial decoding did flush out a handful of bugs. One
general improvement to testing would be fuzz testing for partial
decoding. I may implement that later. I also anticipate switching the
wire protocol code to this new decoder will flush out any lingering
bugs.
Differential Revision: https://phab.mercurial-scm.org/D4414
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 28 Aug 2018 15:22:06 -0700] rev 39437
tests: remove use of string in CBOR test
We don't use the CBOR string major type in the wire protocol. Let's
not test for it.
Differential Revision: https://phab.mercurial-scm.org/D4413
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 28 Aug 2018 20:27:36 -0700] rev 39436
internals: document CBOR utilization
I spoke with some people at Mozilla about CBOR and they advised me
that we should be careful about the subset of CBOR we use in order
to mitigate security, performance, and compatibility concerns.
This commit establishes a document that attempts to formalize our
use of CBOR.
Its main limitations are on what types are allowed. It explicitly
enumerates which types are supported. Notable missing features
include:
* Indefinite-length arrays and maps
* Text strings (bytes all the way)
* Floats
* Date/time types
* Big integers
* Use of indefinite-length byte strings for map keys, values in
containers.
If we have a need for any of these, we can have a discussion about
them when the time comes.
Differential Revision: https://phab.mercurial-scm.org/D4412
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Sep 2018 13:56:53 +0300] rev 39435
py3: add new passing test to whitelist found by buildbot
Differential Revision: https://phab.mercurial-scm.org/D4451
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Sep 2018 13:53:30 +0300] rev 39434
py3: alias xrange to range in tests/test-fastannotate-revmap.py
xrange is not available on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4450
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 03 Sep 2018 13:52:49 +0300] rev 39433
py3: add r'' prefix to prevent b'' being prepended
The keys of keyword arguments should be str.
# skip-blame because we are adding just r'' prefixes.
Differential Revision: https://phab.mercurial-scm.org/D4449
Sushil khanchi <sushilkhanchi97@gmail.com> [Fri, 31 Aug 2018 23:28:09 +0530] rev 39432
revert: fix the inconsistency of status msgs in --interactive mode
Before this patch we were priting every action msg before actually
performing that action and that was resulting in inconsistencies;
like in --interactive session if user decided to not revert any
changes in a file foo, still there will be a msg on console saying
"reverting foo".
To fix this, I have made some changes to print status msg just
before the action it is going to perform, no matter if --interactive
or not.
Changes made in test-revert-interactive.t reflect the changed behavior.
There are also some changes in test-revert.t because of change in the
order of messages.
Differential Revision: https://phab.mercurial-scm.org/D4380
Sushil khanchi <sushilkhanchi97@gmail.com> [Mon, 03 Sep 2018 10:44:52 +0530] rev 39431
revert: remove unnecessary check as `msg` should never be a callable now
Differential Revision: https://phab.mercurial-scm.org/D4448
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Sep 2018 16:32:11 +0900] rev 39430
run-tests: drop support for --with-python3 and $PYTHON3
This backs out cdbc25306696. We can run many tests on python3 without
this hack.
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Sep 2018 16:26:33 +0900] rev 39429
test-check-py3-compat: don't run py3 part if not build against python3
This test depends on C extensions built for python3, which aren't available
if run-tests.py runs on python2 and no -l/--local is specified.
Yuya Nishihara <yuya@tcha.org> [Sun, 02 Sep 2018 16:23:34 +0900] rev 39428
test-check-py3-compat: don't run py2 part if $PYTHON is python3
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sat, 01 Sep 2018 18:00:38 -0400] rev 39427
resolve: test that we suggest a valid hg command
Differential Revision: https://phab.mercurial-scm.org/D4447
Anton Shestakov <av6@dwimlabs.net> [Sun, 02 Sep 2018 08:40:57 +0800] rev 39426
bash_completion: complete arguments for --color and --pager
Anton Shestakov <av6@dwimlabs.net> [Sun, 02 Sep 2018 08:39:17 +0800] rev 39425
bash_completion: add more global options that receive an argument
Augie Fackler <augie@google.com> [Sat, 01 Sep 2018 11:33:17 -0400] rev 39424
tracing: ignore any IOErrors when writing to pipe
When the pager forks off the main process, we can end up with the pipe
closed prematurely. Rather than break hg entirely when that happens
and tracing is active, just let lingering events disappear as needed.
Differential Revision: https://phab.mercurial-scm.org/D4445
Augie Fackler <augie@google.com> [Sat, 01 Sep 2018 11:06:47 -0400] rev 39423
tests: avoid shellquoting bytes on Python 3
Should fix Python 3 builder.
Differential Revision: https://phab.mercurial-scm.org/D4444
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sat, 01 Sep 2018 17:37:52 -0400] rev 39422
resolve: ui.config -> ui.configbool, for better validation
Differential Revision: https://phab.mercurial-scm.org/D4446
Anton Shestakov <av6@dwimlabs.net> [Wed, 22 Aug 2018 09:12:22 +0800] rev 39421
zsh_completion: handle --rev as well as -r for diff and revert
Completion for hg diff and revert should suggest all files in context of a
non-current revision. The script used to look only for `-r foo`, and now it
also understands `--rev foo`.
Differential Revision: https://phab.mercurial-scm.org/D4427
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 26 Aug 2018 16:42:28 -0400] rev 39420
resolve: add config to make hg resolve not re-merge by default
Before this, calling 'hg resolve' with neither -m, -u or -l will
re-merge. This is highly error prone (it's easy to forget to forget a
-m), and pretty bad when it happens (many people have no idea 'hg
resolve' can re-merge, and end up redoing the work of resolving all
the conflicts, because they have no idea there was a backup of their
work).
Differential Revision: https://phab.mercurial-scm.org/D4379
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 26 Aug 2018 15:52:34 -0400] rev 39419
resolve: add a flag for the default behavior of re-merging
On its own, it's not useful, but the next commit will add an hgrc
config option to make it mandatory.
There is no short option, as -r almost always means --rev and this
option doesn't seem like it would be so common as to mandate a short
option.
Differential Revision: https://phab.mercurial-scm.org/D4378
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Sep 2018 02:01:55 -0400] rev 39418
tests: conditionalize narrow-widen error output for Windows
Matt Harbison <matt_harbison@yahoo.com> [Sat, 25 Aug 2018 13:09:24 -0400] rev 39417
largefiles: use a context manager to control the progress bar lifetime
Matt Harbison <matt_harbison@yahoo.com> [Sat, 25 Aug 2018 12:41:58 -0400] rev 39416
lfs: use a context manager to control the progress bar lifetime
Matt Harbison <matt_harbison@yahoo.com> [Sat, 25 Aug 2018 12:26:44 -0400] rev 39415
hg: ensure the progress bar is completed when copying the store
This is just a block indent under the context manager.
Yuya Nishihara <yuya@tcha.org> [Fri, 31 Aug 2018 21:10:28 +0900] rev 39414
shelve: fix crash on unshelve without .shelve metadata file
Follow up for c67c94c0e7ae and 38373da1af02.
The inline comment says "we should keep track of the unshelve node in case
we need to reuse it." Perhaps such case isn't tested, and this patch does
NOT add a test for the reuse of the unbundled revision.
Also, I have no idea what should be done if new revision is unbundled
because of "node not in repo".
Yuya Nishihara <yuya@tcha.org> [Sat, 01 Sep 2018 10:40:48 +0900] rev 39413
fastannotate: use stringutil.pprint() to dump diffopts to be hashed
Yuya Nishihara <yuya@tcha.org> [Sat, 01 Sep 2018 10:23:13 +0900] rev 39412
test-check-py3-compat: update "python3 check-py3-compat.py" output
Yuya Nishihara <yuya@tcha.org> [Sat, 01 Sep 2018 10:13:48 +0900] rev 39411
test-check-py3-compat: exclude thirdparty and cffi which are known to fail
Otherwise the following errors would be spilled out:
mercurial/cffi/bdiff.py: error importing: <ImportError> cannot import
name '_bdiff' (error at bdiff.py:*)
mercurial/cffi/bdiffbuild.py: error importing: <ModuleNotFoundError> No
module named 'cffi' (error at bdiffbuild.py:*)
mercurial/cffi/mpatch.py: error importing: <ImportError> cannot import
name '_mpatch' (error at mpatch.py:*)
mercurial/cffi/mpatchbuild.py: error importing: <ModuleNotFoundError> No
module named 'cffi' (error at mpatchbuild.py:*)
mercurial/cffi/osutilbuild.py: error importing: <ModuleNotFoundError> No
module named 'cffi' (error at osutilbuild.py:*)
mercurial/thirdparty/concurrent/futures/_base.py: invalid syntax: invalid
syntax (<unknown>, line *)
mercurial/thirdparty/concurrent/futures/process.py: error importing:
<SyntaxError> invalid syntax (_base.py, line 416) (error at __init__.py:*)
mercurial/thirdparty/concurrent/futures/thread.py: error importing:
<SyntaxError> invalid syntax (_base.py, line 416) (error at __init__.py:*)
mercurial/thirdparty/zope/interface/_flatten.py: error importing:
<ImportError> cannot import name 'Declaration' (error at _flatten.py:*)
Some of them can be suppressed by building cffi modules for example, but
I don't think it's worth keeping these modules covered by the compatibility
checker.
Yuya Nishihara <yuya@tcha.org> [Sat, 01 Sep 2018 10:18:38 +0900] rev 39410
test-check-py3-compat: remove false output from "python3 check-py3-compat.py"
If python3 were python2, these errors would be reported, but we're running
check-py3-compat.py on Python 3 here.
Boris Feld <boris.feld@octobus.net> [Wed, 01 Aug 2018 08:52:55 +0200] rev 39409
test: add a test file that displays ssh behavior in front of various errors
Prior to changeset ac0a87160012, these errors were suboptimal since they claim
the repository is missing while the actual issue was about permission.
This extra test file provides additional documented coverage for the various
corner cases.
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 01 Sep 2018 00:40:26 +0530] rev 39408
tests: update test-casefolding.t with recent changes
This is caught by the mac-buildbot. This was not found initially because this
test requires a case insensitive filesystem.
I have edited the test file by hand and I suggest to test this on such a system
before pushing.
Differential Revision: https://phab.mercurial-scm.org/D4430
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 01 Sep 2018 02:16:22 +0530] rev 39407
py3: don't use dict.iterkeys() in hgext/fastannotate/context.py
dict.iterkeys() is not present on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4431
Matt Harbison <matt_harbison@yahoo.com> [Wed, 29 Aug 2018 23:39:58 -0400] rev 39406
run-tests: replace '/dev/null' with os.devnull for Windows
Many of the *.py tests were dying on:
File "c:\Users\Matt\projects\hg\hgdemandimport\tracing.py", line 27, in log
_pipe = open(os.environ['HGCATAPULTSERVERPIPE'], 'w', 1)
IOError: [Errno 2] $ENOENT$: '/dev/null'
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 27 Aug 2018 09:13:58 -0700] rev 39405
stringutil: teach pprint() to indent
This will make data structure dumping in various places a bit
easier to read and diff. Since I wanted this for `hg debugwireproto`
output, I added indentation to it.
A more advanced pretty printer implementation would conditionally
add newlines if output is too long. But it is vastly simpler to
be consistent and always add newlines when indenting.
Again, I'm not crazy about the verbosity of the code and there is
room to consolidate logic for "print a collection." But this isn't
the most complicated code in the world and I'm not convinced it is
worth doing.
Differential Revision: https://phab.mercurial-scm.org/D4399
Anton Shestakov <av6@dwimlabs.net> [Wed, 22 Aug 2018 08:20:51 +0800] rev 39404
zsh_completion: complete merge tools for -t/--tool
Differential Revision: https://phab.mercurial-scm.org/D4362
Boris Feld <boris.feld@octobus.net> [Wed, 06 Jun 2018 01:46:37 +0200] rev 39403
shelve: add an "internal" extra
Keeping shelve changeset around increase the risk of collision with normal
changesets. To prevent such collision and help with overall clarity, we add an
'internal' key in extra that mark the changeset as created by "shelve".
Node changes in tests are expected.
Boris Feld <boris.feld@octobus.net> [Wed, 06 Jun 2018 01:10:01 +0200] rev 39402
shelve: write metadata file on the fly if they are missing
Keeping an explicit reference to the shelve node in order to reuse it directly
if the unshelved is repeated (eg: unshelve --keep).
Boris Feld <boris.feld@octobus.net> [Wed, 06 Jun 2018 00:57:22 +0200] rev 39401
shelve: look for shelved node in the repository before unbundling
This prepares the version of shelve that would not strip the shelved node from
the repository. If we have the node information, search for it in the
repository and only fallback on unbundling if it is missing.
To be able to find such nodes, we operate on an unfiltered repository.
Boris Feld <boris.feld@octobus.net> [Wed, 06 Jun 2018 01:03:10 +0200] rev 39400
shelve: handle shelved node on unfiltered repository
To build a version of shelve that use hiding instead of stripping, we need
shelve to be able to find these revisions. This it does not hurt the bundle
version, so we introduce the necessary code in the relevant place.
Boris Feld <boris.feld@octobus.net> [Tue, 29 May 2018 11:47:32 +0200] rev 39399
shelve: store shelved node in a new data file
It is useful for a version of shelve not based on bundle and strip. Having
more data does not hurt the bundle based case so we introduce it in all cases
before doing more rework.
We also keep storing the patch using another method.
note: We could have this metadata file list all the file involved in the
shelve and use that for cleanup. This would be more future proof than having a
hard-coded list of file to purge. However, this is an adventure for another
series.
Yuya Nishihara <yuya@tcha.org> [Sun, 05 Aug 2018 16:33:30 +0900] rev 39398
templatekw: add {path} keyword to host documentation
It's hidden since I'm not sure if filectx templates can be a thing. The plan
is to add {status}, {size}, etc., which are usable within {files % ...}
context.