Yuya Nishihara <yuya@tcha.org> [Sun, 16 Dec 2018 16:31:31 +0900] rev 41001
context: error out if basefilectx.cmp() is called without self._filenode
The base implementation can't handle such cases because the filelog has no
knowledge about the working directory.
Loading self._filenode should have no extra cost since self.size() would
load it anyway.
Yuya Nishihara <yuya@tcha.org> [Sun, 16 Dec 2018 16:24:45 +0900] rev 41000
context: collapse complex condition to see if filelog have to be compared
It's hard to read. I'd rather make the return statement duplicated.
Augie Fackler <augie@google.com> [Tue, 18 Dec 2018 10:21:25 -0500] rev 40999
merge with stable
Georges Racinet <gracinet@anybox.fr> [Mon, 17 Dec 2018 15:05:52 +0100] rev 40998
rust-cpython: build and support for Python3
Defined Cargo features for Python3, making them overall simpler to
use, hooked them in build and made mercurial.rustext importable.
This is tested with Python 3.6.7.
Differential Revision: https://phab.mercurial-scm.org/D5446
Matt Harbison <matt_harbison@yahoo.com> [Tue, 18 Dec 2018 00:54:32 -0500] rev 40997
py3: convert popen() command arguments in hgclient to str on Windows
This fixes test-commandserver.t and test-keyword.t, which was previously
complaining
TypeError("a bytes-like object is required, not 'str'")
Matt Harbison <matt_harbison@yahoo.com> [Mon, 17 Dec 2018 23:08:26 -0500] rev 40996
py3: byteify tests/pullext.py
# skip-blame for b'' prepending
Matt Harbison <matt_harbison@yahoo.com> [Mon, 17 Dec 2018 21:27:37 -0500] rev 40995
py3: byteify the fakeversion extension in test-http-bad-server.t
# skip-blame for b'' prefixing
Matt Harbison <matt_harbison@yahoo.com> [Mon, 17 Dec 2018 18:25:18 -0500] rev 40994
py3: spawn all python instances with legacy stdio enabled on Windows
This fixes 6 more tests. See ef7119cd4965.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 17 Dec 2018 17:44:45 -0500] rev 40993
setup: avoid attempting to invoke the system-wide hg.exe on Windows
On Windows, the executable in the current directory gets priority over anything
in $PATH (both for cmd.exe and MSYS). That means, the former code was launching
the local hg.exe instead of the system-wide one, if it was previously built. If
that failed, it then fell back to the local hg code, but run through python.exe.
I'm not sure what it is about ef7119cd4965, but that started throwing up a
messagebox that python37.dll couldn't be loaded. (And indeed, python37 is not
in $PATH by default.) Invoking the local hg via the current python avoids that.
Boris Feld <boris.feld@octobus.net> [Mon, 17 Dec 2018 10:46:37 +0100] rev 40992
delta: ignore base whose chains already don't match expectations
If we know the existing chain does not match our criteria, there is no point
to build a delta to append. This is especially useful when dealing with a full
text much smaller than its parent. In that case, the parent chain is probably
already too large.
example affected manifest write
before: 1.421005s
after: 0.815520s (-42%)
Boris Feld <boris.feld@octobus.net> [Mon, 17 Dec 2018 10:42:19 +0100] rev 40991
delta: exclude base candidate much smaller than the target
If a revision's full text is that much bigger than a base candidate full text,
we no longer consider that candidate.
This solves a pathological case we encountered on a very specify repository.
It contains a long series of changesets with a very small manifest (one file)
co-existing with others changesets using a very large manifest.
Without this filtering, we ended up considering a large number of tiny full
snapshots as a potential base. It resulted in very large delta (the size of
the full text) and mercurial spending 99% of its time compressing these
deltas.
The timing of a commit moved from about 400s to about 10s (still slow, but not
ridiculously slow).
Boris Feld <boris.feld@octobus.net> [Mon, 17 Dec 2018 10:37:22 +0100] rev 40990
perfrevflogwrite: clear revlog cache between each write
We want to measure write time from a cold cache (similar to commit). So we
need to clear the cache to prevent computation from rev N-1 to interfere with
rev N.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 16 Dec 2018 17:42:45 -0500] rev 40989
py3: enable legacy stdio mode in exewrapper
This drops the test failure count from 166 to 117. The failures were typically
in the form of `hg serve -d` spawning but crashing immediately, leaving clients
with "bad http status" lines, connection refusals, and so forth. The underlying
message on the server side was either "OSError: [WinError 6] The handle is
invalid" or "OSError: [WinError 1] Incorrect function". Additionally, no output
was rendered if the pager was activated. Thanks to Yuya for diagnosing the
problem.
The failure count drops to 107 when PYTHONLEGACYWINDOWSSTDIO=1 is defined in the
environment. These failures seem to revolve around the dummyssh server process,
and dumbhttp.py. So I'll probably add that to the test runner.
One small regression here (only in py3) is that if hg.exe is already built, a
messagebox appears when building it again saying that python37.dll can't be
loaded. Python3 isn't in PATH by default, and setup.py tries running bare `hg`
first. But MSYS prepends '.' to PATH, so it runs the local hg, but can't find
the library. It falls back to the python used to invoke setup.py, so ultimately
it works. I'm not sure if it's better to strip '.' from PATH or just skip right
to `sys.executable hg` on Windows.
Also, something seems to be wrong with run-tests._usecorrectpython(). I
accidentially left off the 'PYTHON="py -3"' when building (thus making py2
stuff), and yet managed to invoke run-tests.py with "py -3". (And that only had
67 failures.)
Matt Harbison <matt_harbison@yahoo.com> [Sun, 16 Dec 2018 17:36:51 -0500] rev 40988
run-tests: alias hg to hg.exe on Windows
To enable legacy stdio mode on Windows, hg.exe needs to be updated. But before
that, we actually have to use it when running the tests. I *think* what was
happening before was when MSYS invoked `hg`, it looked at the shbang line and
ran python.exe found there. The test harness must be updating $PATH to include
the python used to launch it, and therefore it ran py3. As a side note, this
also fixed `py -3 run-tests.py` (without --local), which complained about the
space in the shbang line before this.
This should also help in WSL, because the explicit '.exe' is needed to invoke a
Windows app instead of invoking the Linux app.
I have no idea why this change capitalizes Lib in the tests, as it was
previously lowercase for both py2 and py3.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 16 Dec 2018 15:47:08 -0500] rev 40987
py3: conditionalize the python version in test-install.t
Boris Feld <boris.feld@octobus.net> [Sun, 16 Dec 2018 18:51:48 +0000] rev 40986
tests: followup on network related errors on Debian 9
First, update test-clonebundles.t which was failing since the introduction of
the `$EADDRNOTAVAIL$` common pattern.
Also update two tests from 8695fbe17f7c with a more compact condition line.
Differential Revision: https://phab.mercurial-scm.org/D5445
Yuya Nishihara <yuya@tcha.org> [Sun, 16 Dec 2018 17:55:08 +0900] rev 40985
py3: unimplement RevlogError.__str__()
On Python 2, str(exc) would crash if __str__() returned a unicode containing
non-ASCII characters.
Yuya Nishihara <yuya@tcha.org> [Sun, 16 Dec 2018 17:53:17 +0900] rev 40984
py3: use forcebytestr() to stringify hook exception
This fixes test-narrow-exchange.t.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 15 Dec 2018 23:14:03 -0500] rev 40983
color: fix a documentation typo
Matt Harbison <matt_harbison@yahoo.com> [Sat, 15 Dec 2018 22:31:54 -0500] rev 40982
py3: quote $PYTHON in test-patchbomb.t for Windows
I couldn't get the quoting right in the environment variable, so now it's a
function.
Georges Racinet <gracinet@anybox.fr> [Tue, 16 Oct 2018 19:58:27 +0200] rev 40981
rust-cpython: testing the bindings from Python
This is easier and more convincing than doing the same tests
from a Rust tests module.
Differential Revision: https://phab.mercurial-scm.org/D5437
Georges Racinet <gracinet@anybox.fr> [Thu, 06 Dec 2018 16:34:22 +0100] rev 40980
rust-cpython: build via HGWITHRUSTEXT=cpython
The existing behaviour, building the direct ffi bindings if
HGIWTHRUSTEXT is just set is unchanged, but if HGWITHRUSTEXT is
cpython, then the cpython bindings (aka mercurial/rustext.so) are
built.
Differential Revision: https://phab.mercurial-scm.org/D5436
Georges Racinet <gracinet@anybox.fr> [Thu, 06 Dec 2018 16:23:20 +0100] rev 40979
rust: better treatment of cargo/rustc errors
Differential Revision: https://phab.mercurial-scm.org/D5435
Georges Racinet <gracinet@anybox.fr> [Mon, 03 Dec 2018 06:52:17 +0100] rev 40978
rust-cpython: start cpython crate bindings
This changeset introduces the hg-cpython crate,
that compiles as a shared library holding a whole
Python package (mercurial.rustext), with only the empty
'ancestor' submodule for now.
Such bindings will be easier and safer to develop and maintain
that those of `hg-direct-ffi`.
They don't involve C code, only unsafe Rust that's mostly isolated
within the cpython crate.
The long-term goal would be to import the provided modules, such
as rustext.ancestor with mercurial.policy.importmod, same as
we already do with cext modules.
Differential Revision: https://phab.mercurial-scm.org/D5434
Georges Racinet <gracinet@anybox.fr> [Mon, 03 Dec 2018 06:54:19 +0100] rev 40977
rust-cpython: exclude hgcli from workspace
hgcli uses a specific rust-cpython commit by indygreg, of which
a PR has been derived which is not merged nor released yet.
But we can't use several versions of the sys-python2.7 crate in
a single workspace: it makes for a build error. Since hgcli does not at
the time being need anything from hg-core, whereas the upcoming hg-cpython
will. So for now we're moving hgcli aside, hoping we could base all of them
on the same version of rust-cpython again in the future.
Differential Revision: https://phab.mercurial-scm.org/D5433
Boris Feld <boris.feld@octobus.net> [Fri, 14 Dec 2018 17:25:41 +0100] rev 40976
sparse-revlog: protect C code against delta chain including nullrev
For unclear reasons, some repositories include nullrev (-1). Re-computing
delta for such repo remove nullrev from all chain, so some older versions have
been creating them.
This currently raise an IndexError with the new C code doing chain slicing as
it expect all item to be positive.
Both python and C code for reading delta chain preserve nullrev, and the Python
code for chain slicing handle the case fine. So we take the safe route and make
the new C code works fine in that case.
Boris Feld <boris.feld@octobus.net> [Fri, 14 Dec 2018 17:24:44 +0100] rev 40975
sparse-revlog: handle nullrev in index_get_length
The more generic index_get method handle nullrev fine, we apply the same logic
here.
Boris Feld <boris.feld@octobus.net> [Fri, 14 Dec 2018 17:23:41 +0100] rev 40974
sparse-revlog: handle nullrev in index_get_start
The more generic index_get method handle nullrev fine, we apply the same logic
here.
Boris Feld <boris.feld@octobus.net> [Fri, 14 Dec 2018 17:22:42 +0100] rev 40973
revlog: introduce a constant for nullrev in `revlog.c`
The value is important enough to be explicitly tracked.
Georges Racinet <gracinet@anybox.fr> [Fri, 30 Nov 2018 00:46:55 +0100] rev 40972
rust: translation of missingancestors
This is as direct as possible a translation of the ancestor.missingancestors
Python class in pure Rust. The goal for this changeset is to make it easy
to compare with the Python version.
We also add to Python tests the cases that helped us develop and debug
this implementation.
Some possible optimizations are marked along the way as TODO comments
Differential Revision: https://phab.mercurial-scm.org/D5416
Paul Morelle <paul.morelle@octobus.net> [Fri, 14 Dec 2018 18:15:19 +0100] rev 40971
contrib: provide a small script that draw performance plot
We have been using this script to look into the result of various runs of the
`hg perfrevlogwrite` command. It seems useful enough to be shared more widely.
Boris Feld <boris.feld@octobus.net> [Thu, 06 Dec 2018 10:39:05 +0100] rev 40970
delta: filter nullrev out first
When picking a potential candidate, we filter them on various criteria. The
"different from nullrev" criteria is very fast to compute and we should
process it first.
Boris Feld <boris.feld@octobus.net> [Thu, 06 Dec 2018 10:38:30 +0100] rev 40969
perf: report more of the higher range in perfrevlogwrite
Since the delta chain length is limited to 1000 revisions, we get a new
snapshot about every 1000 revisions. If we assume that the snapshot will be
most of the slowest revision, the current display (99% and max) are not very
precise in their area. We now include more information about this space in the
default report.
Martin von Zweigbergk <martinvonz@google.com> [Fri, 14 Dec 2018 13:44:46 -0800] rev 40968
help: present boolean arguments as "--[no-]foo"
This should make it much more discoverable (we document it in `hg help
flags`, but most users don't think to look there).
Note that flags that default to None (and not False) will not get this
new presentation. We can change the defaults to False later for flags
where it makes sense (probably almost all boolean flags).
Differential Revision: https://phab.mercurial-scm.org/D5432
Martin von Zweigbergk <martinvonz@google.com> [Fri, 14 Dec 2018 13:33:12 -0800] rev 40967
help: show "default: off" for boolean flags that default to off
Differential Revision: https://phab.mercurial-scm.org/D5431
Martin von Zweigbergk <martinvonz@google.com> [Fri, 14 Dec 2018 13:20:00 -0800] rev 40966
help: use "default: on" instead of "default: True"
"True" feels like a Python thing and not something that users should
see.
Differential Revision: https://phab.mercurial-scm.org/D5430
Martin von Zweigbergk <martinvonz@google.com> [Fri, 14 Dec 2018 13:32:34 -0800] rev 40965
check-commit: disallow capitalization only right after topic
It should be okay to write "topic: change 'default: True' to 'default:
on'".
Differential Revision: https://phab.mercurial-scm.org/D5429
Danny Hooper <hooper@google.com> [Thu, 13 Dec 2018 20:39:25 -0800] rev 40964
absorb: don't prompt to apply changes when there are none to apply
Differential Revision: https://phab.mercurial-scm.org/D5428
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 14 Dec 2018 21:00:33 +0300] rev 40963
py3: add a b'' prefix in tests/test-remotefilelog-pull-noshallow.t
This makes the test pass on Python 3.
# skip-blame because just b'' prefix.
Differential Revision: https://phab.mercurial-scm.org/D5427
Augie Fackler <augie@google.com> [Fri, 14 Dec 2018 12:01:47 -0500] rev 40962
error: implement __str__ on RevlogError to fix some output defects on Py3
We open-code encoding.unimethod here to avoid cycles, and do a local
import of encoding when someone str()s a RevlogError. It's not my
favorite solution, but it gets the job done.
Differential Revision: https://phab.mercurial-scm.org/D5426
Augie Fackler <augie@google.com> [Fri, 14 Dec 2018 12:00:38 -0500] rev 40961
perf: add two more missing b prefixes for Python 3
# skip-blame just two more b prefixes
Differential Revision: https://phab.mercurial-scm.org/D5425
Augie Fackler <augie@google.com> [Fri, 14 Dec 2018 11:37:50 -0500] rev 40960
match: fix doctest to use bytes instead of str
Fixes doctests on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5423
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 14 Dec 2018 19:30:20 +0300] rev 40959
py3: use pycompat.bytechr() instead of chr() in test-fastannotate-revmap.py
This makes the test pass on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5422
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 14 Dec 2018 19:21:28 +0300] rev 40958
py3: add a missing b'' prefix in contrib/perf.py
# skip-blame because just b'' prefixes
This fixes test-contrib-perf.t on Python 3 which started failing.
Differential Revision: https://phab.mercurial-scm.org/D5421
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 14 Dec 2018 19:12:45 +0300] rev 40957
py3: use '%d' for integers instead of '%s'
This should fix test-rebase-inmemory.t which started failing on Python 3 after
recent changes.
Differential Revision: https://phab.mercurial-scm.org/D5420
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 14 Dec 2018 19:10:46 +0300] rev 40956
py3: whitelist another passing test caught by buildbot
Differential Revision: https://phab.mercurial-scm.org/D5419
Yuya Nishihara <yuya@tcha.org> [Fri, 14 Dec 2018 21:19:19 +0900] rev 40955
py3: fix bad escapes of sub() replacement pattern in test-template-basic.t
Python 3.7 starts complaining about it. We have to double the backslash or
'\x5c' to get around.
Georges Racinet <gracinet@anybox.fr> [Thu, 13 Dec 2018 17:10:03 +0100] rev 40954
perfdiscovery: benching findcommonheads()
This works between the local repo and any peer given by its path, and
should be useful for further work on discovery
Differential Revision: https://phab.mercurial-scm.org/D5418
Augie Fackler <augie@google.com> [Thu, 13 Dec 2018 14:44:54 -0500] rev 40953
merge with stable
Yuya Nishihara <yuya@tcha.org> [Wed, 12 Dec 2018 22:45:02 +0900] rev 40952
templatefuncs: specialize "no match" value of search() to allow % operation
If Python had Maybe or Option, the type of the search() result would be
Option<Mapping>, which can be considered as a 0/1 container of a Mapping.
So it makes sense that {search(r'no match pattern', x) % "whatever"} is
mapped to an empty string.
Yuya Nishihara <yuya@tcha.org> [Wed, 12 Dec 2018 22:19:57 +0900] rev 40951
templatefuncs: add regexp search() function that extracts substring
This can be used to extract an issue number from a commit message, for
example:
{search(r'\(issue([0-9]*)\)', desc) % '{1}'}
Georges Racinet <gracinet@anybox.fr> [Fri, 30 Nov 2018 00:44:04 +0100] rev 40950
rust: changed Graph.parents to return [Revision; 2]
This will allow for simple iteration on parent revisions,
such as:
for parent in graph.parents(rev)?.iter().cloned()
This seems to be a zero overhead abstraction once built in
release mode.
Differential Revision: https://phab.mercurial-scm.org/D5415
Georges Racinet <gracinet@anybox.fr> [Tue, 11 Dec 2018 17:31:54 +0100] rev 40949
rust: improved docstring
In the previous wording, rustfmt wanted to cut at the == which is not very
readable in my taste.
Differential Revision: https://phab.mercurial-scm.org/D5414
Anton Shestakov <av6@dwimlabs.net> [Thu, 13 Dec 2018 17:18:57 +0800] rev 40948
revset: move subscript relation functions to its own dict
This will help adding more relation functions in extensions.
We skip short names (that consist of one letter) while raising
UnknownIdentifier because such names cannot be suggested anyway: the similarity
cutoff in dispatch._getsimilar() is currently 0.6.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 13 Dec 2018 00:18:47 -0500] rev 40947
py3: teach run-tests.py to handle exe with spaces when --local isn't specified
This was the reason that no amount of quoting worked in test-hghave.t.
`os.popen()` needed to be swapped out because while the added quoting around
line 3124 worked on py3, it failed on py2. See 38d51371792b. The problem with
`os.system()` was wrongly splitting the command on the space in 'Program Files',
regardless of quoting. It looks like there are a few other instances of
`os.system()` in core code, so presumably those should be replaced?
Georges Racinet <gracinet@anybox.fr> [Tue, 11 Dec 2018 17:13:17 +0100] rev 40946
rust: adapted hg-core tests for iteration over Result
Now AncestorsIterator iters on Result<Revision, GraphError>
Matt Harbison <matt_harbison@yahoo.com> [Tue, 11 Dec 2018 21:57:54 -0500] rev 40945
win32: close the handles associated with a spawned child process
Probably not a big deal because at this point, the call is only used when
spawning a daemonized server. In that case, the parent process goes away first,
so it won't prevent the child from being cleaned up.
Yuya Nishihara <yuya@tcha.org> [Tue, 11 Dec 2018 22:23:39 +0900] rev 40944
rust: remove comment about error handling of AncestorsIterator
To be align with 443eb4bc41af "rust: propagate error of index_get_parents()
properly."
Spotted by Georges Racinet.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 10 Dec 2018 20:01:07 +0000] rev 40943
perf: add perfprogress command
I've noticed that progress bars can add significant overhead to tight
loops. Let's add a perf command that attempts to isolate that overhead.
With a default hgrc, iteration over 1M items appears to take ~3.75s on
my machine. Profiling reveals ~28% of time is spent in ui.configbool()
resolving the value of the progress.debug config option.
Even if I set progress.disable=true, execution still takes ~2.60s, with
~59% of the time spent in ui.configbool().
Differential Revision: https://phab.mercurial-scm.org/D5407
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 10 Dec 2018 18:55:08 +0000] rev 40942
wireprotov2: unify file revision collection and linknode derivation
The old mechanism for choosing which file revisions to send in the
haveparents=True case was buggy in multiple ways - the most severe
of which being that file revisions were excluded when they shouldn't
have been.
This commit unifies the logic for deriving the filenodes that will
be sent by the "filesdata" command. We now consistently read files
data from manifests. The "haveparents" argument now controls whether
we iterate ctx.files() or use the full manifest to derive relevant
files.
The logic here is still woefully lacking to fully support shallow
clones. It will require an API break to fully address. This commit
should at least make the server APIs emit proper data, which is
strictly better than before.
Differential Revision: https://phab.mercurial-scm.org/D5406
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 10 Dec 2018 18:04:12 +0000] rev 40941
wireprotov2: send linknodes to emitfilerevisions()
Previously, linknodes were calculated within emitfilerevisions() by
using filectx.introrev(), which would always use the linkrev/linknode
as recorded by storage. This is wrong for cases where the receiver
doesn't have the changeset the linknode refers to.
This commit changes the logic for linknode emission so the mapping
of filenode to linknode is computed by the caller and passed into
emitfilerevisions().
As part of the change, linknodes for "filesdata" in the
haveparents=False case are now correct: the existing code performed a
manifest walk and it was trivial to plug in the correct linknode.
However, behavior for the haveparents=True case is still wrong
because it relies on filtering linkrevs against the outgoing set in
order to determine what to send. This will be fixed in a subsequent
commit.
The change test test-wireproto-exchangev2-shallow.t is a bit wonky.
The test repo has 6 revisions. The changed test is performing a shallow
clone with depth=1. So, only file data for revision 5 is present
locally. So, the new behavior of associating the linknode with
revision 5 for every file revision seems correct. Of course, when
backfilling old revisions, we'll want to update the linknode. But
this problem requires wire protocol support and we'll cross that
bridge later.
Differential Revision: https://phab.mercurial-scm.org/D5405
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 10 Dec 2018 17:26:12 +0000] rev 40940
tests: add tests for server-side linknode adjustment with wireprotov2
The current implementation of linknode serving in wireprotov2 simply
serves up the linkrev/linknode as stored: it doesn't attempt to
adjust the linknode to what the receiver is aware of. This can result
in the client seeing a linknode referencing a changeset that is unknown
to it.
This commit adds test coverage of that scenario.
The tests in test-wireproto-command-filesdata.t demonstrate two failures.
First, the linknode refers to a changeset not in the available set.
Second, the server doesn't send a file revision that it should have
(because of linkrev filtering).
The test in test-wireproto-exchange.t demonstrates that the lack of
a file revision results in a corrupted repository on the client.
Differential Revision: https://phab.mercurial-scm.org/D5404
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 10 Dec 2018 16:53:09 +0000] rev 40939
tests: fix empty commit in test
This was almost certainly a bug.
Differential Revision: https://phab.mercurial-scm.org/D5403
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 10 Dec 2018 19:41:43 +0000] rev 40938
tests: add sparserevlog requirement to test-sqlitestore.t
Looks like this test was missed when enabling sparse revlogs by default.
Differential Revision: https://phab.mercurial-scm.org/D5402