Ian Moody <moz-ian@perix.co.uk> [Mon, 22 Apr 2019 17:46:01 +0100] rev 42188
phabricator: use templatefilters.json in writediffproperties
Instead of json.dumps, since it makes the code simpler and more readable.
This would have been the better option for
8fd19a7b4ed6 but I wasn't aware of
it at the time.
Differential Revision: https://phab.mercurial-scm.org/D6295
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 09:34:16 -0700] rev 42187
commands: use byteskwargs() in verify()
Otherwise Python 3 complains about the missing key.
Differential Revision: https://phab.mercurial-scm.org/D6294
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 09:29:55 -0700] rev 42186
match: use raw strings to avoid illegal baskslash escape
Python 3.8 was complaining about the invalid escape
sequences. Let's use raw strings to avoid the warning and
double baskslashes.
Differential Revision: https://phab.mercurial-scm.org/D6293
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Apr 2019 00:48:16 +0300] rev 42185
revbranchcache: use context manager in _writerevs() to write to file
The other _writenames() is a bit complicated to use context manager.
Differential Revision: https://phab.mercurial-scm.org/D6292
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Apr 2019 00:44:18 +0300] rev 42184
revbranchcache: factor logic to write names and revs in separate functions
Before this patch, the write function was so populated with upto 4 level of
indentation, it was hard to understand what's going on.
Differential Revision: https://phab.mercurial-scm.org/D6291
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:16:33 -0700] rev 42183
tests: make log style a little easier to read in test-copytrace-heuristics.t
Revision numbers are much shorter and easier to read (especially
compared to the full nodeids that were used here), so I switched to
that. That's also what almost all the commands used (e.g. `hg rebase
-s . -d 1`). I updated the two instances that used nodeids. I also
made some other little cleanups to the log templates.
Differential Revision: https://phab.mercurial-scm.org/D6279
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:23:26 -0700] rev 42182
tests: avoid cryptic nodeids in tests/test-rename-merge1.t
These two nodeids had not been part of any output before, so one can't
know which revision they refer to without adding something like `hg
log` before them. It turned out that '.^' was equivalent for both of
them, so that's what I replaced them with.
Differential Revision: https://phab.mercurial-scm.org/D6280
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:08:58 -0700] rev 42181
tests: defines aliases for `hg log` calls in test-copytrace-heuristics.t
This also makes the test cases more consistent since a few had missed
the ":" in "changeset:" that the others used.
Differential Revision: https://phab.mercurial-scm.org/D6278
Georges Racinet <georges.racinet@octobus.net> [Thu, 14 Mar 2019 17:57:31 +0000] rev 42180
rust-discovery: implementing and exposing stats()
This time, it's simple enough that we can do it in all layers in
one shot.
Differential Revision: https://phab.mercurial-scm.org/D6233
Georges Racinet <georges.racinet@octobus.net> [Wed, 20 Feb 2019 09:04:39 +0100] rev 42179
rust-discovery: cpython bindings for the core logic
As previously done with the ancestors submodule, testing for
the bindings is provided from Python on a trivial case.
Differential Revision: https://phab.mercurial-scm.org/D6232
Georges Racinet <georges.racinet@octobus.net> [Tue, 19 Feb 2019 23:42:31 +0100] rev 42178
rust-discovery: starting core implementation
Once exposed to the Python side, this core object will avoid
costly roundtrips with potentially big sets of revisions.
This changeset implements the core logic of the object only, i.e.,
manipulation of the missing, common and undefined set-like revision
attributes.
Differential Revision: https://phab.mercurial-scm.org/D6231
Georges Racinet <georges.racinet@octobus.net> [Wed, 20 Feb 2019 18:33:53 +0100] rev 42177
rust-dagops: roots
Unsuprisingly, the algorithm is much easier than for heads, provided
we work on a set in the first place.
To improve the signature, a trait for set-likes object would be useful,
but that's not an immediate concern.
Differential Revision: https://phab.mercurial-scm.org/D6230
Georges Racinet <georges.racinet@octobus.net> [Tue, 19 Feb 2019 23:41:57 +0100] rev 42176
rust-dagops: range of revisions
This is a Rust implementation for what reachableroots2() does if
includepath is True.
The algorithmic details and performance notes are included in the
documentation comment.
Our main use case for now is a Rust counterpart of the partialdiscovery
object, so we don't really need bindings yet.
Differential Revision: https://phab.mercurial-scm.org/D6229
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 08:57:01 -0700] rev 42175
setup: tweak error message for Python 3
We now have beta support for Python 3. In my opinion, it isn't
yet stable enough to allow `pip install Mercurial` to work with
Python 3 out of the box: we don't want people accidentally using
Mercurial with Python 3 just yet.
But I do think we should be more friendly about informing people
of their options.
This commit tweaks the error message that users see when running
setup.py with Python 3. We instruct them about the current level
of Python 3 support, point them at the wiki for more info, and
give them instructions on how to bypass the check.
As part of this, I also changed which version value is printed,
as we were printing a named tuple before.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 07:21:08 -0700] rev 42174
setup: remove set and dict comprehensions
Yuya observed in a recent review that it is worthwhile to keep
setup.py parseable with Python 2.6 so a useful error message is
seen when attempting to run with Python 2.6.
This commit removes a set and dict comprehension so setup.py
is parseable with Python 2.6.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 19 Apr 2019 23:13:28 +0300] rev 42173
branchcache: don't verify all nodes while writing
nodes are verified either when they are added or used. In case of commits. we
will load the whole branchmap, only verify nodes for the branch on which we are
committing and then we write.
However before this patch, writing the branchmap was validating all the nodes
whereas it should not. This patch fixes that.
Differential Revision: https://phab.mercurial-scm.org/D6290
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 20 Apr 2019 07:29:07 -0700] rev 42172
setup: properly package distutils in py2exe virtualenv builds
Our in-repo py2exe packaging code uses virtualenvs for managing
dependencies. An advantage of this is that packaging is more
deterministic and reproducible. Without virtualenvs, we need to
install packages in the system Python install. Packages installed
by other consumers of the system Python could leak into the Mercurial
package.
A regression from this change was that py2exe packages contained
the virtualenv's hacked distutils modules instead of the original
distutils modules. (virtualenv installs a hacked distutils module
because distutils uses relative path lookups that fail when running
from a virtualenv.)
This commit introduces a workaround so py2exe packaging uses the
original distutils modules when running from a virtualenv.
With this change, `import distutils` no longer fails from py2exe
builds produced from a virtualenv. This fixes the regression.
Furthermore, we now include all distutils modules. Before, py2exe's
module finding would only find modules there were explicitly
referenced in code. So, we now package a complete copy of distutils
instead of a partial one. This is even better than before.
# no-check-commit foo_bar function name
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 10:49:11 -0700] rev 42171
narrow: also warn when not deleting untracked or ignored files
Differential Revision: https://phab.mercurial-scm.org/D6265
Joerg Sonnenberger <joerg@bec.de> [Wed, 17 Apr 2019 14:37:06 +0200] rev 42170
setdiscovery: fix a few typos
Differential Revision: https://phab.mercurial-scm.org/D6263
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 14:09:18 -0700] rev 42169
copies: delete debug message about "unmatched files new in both"
Same reasoning as previous patch.
Differential Revision: https://phab.mercurial-scm.org/D6251
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 21:41:51 -0700] rev 42168
copies: delete debug message about changes since common ancestor
Same reasoning as previous patch.
Differential Revision: https://phab.mercurial-scm.org/D6250
Martin von Zweigbergk <martinvonz@google.com> [Thu, 11 Apr 2019 23:28:38 -0700] rev 42167
copies: delete debug message about search limit
I'm about to rewrite mergecopies() and this message will no longer be
emitted then. Let's remove the message now to remove a distraction
from that patch.
Differential Revision: https://phab.mercurial-scm.org/D6249
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 22:58:10 -0700] rev 42166
copies: move early return for "no copies" case a little earlier
We can return before the block that prints debug messages. That block
will not be run anyway when there are no copies.
Differential Revision: https://phab.mercurial-scm.org/D6248
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 16:46:41 -0700] rev 42165
copies: fix up "fullcopy" with missing entries from "diverge"
Similar to the previous patch, but this doesn't even affect tests. It
does affect tests if you change them to turn on debug logging. I'm
fixing it here so reviewers of the later rewrite patch can hard-code
debug logging to be on and more easily compare test results.
Differential Revision: https://phab.mercurial-scm.org/D6247
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 16:41:43 -0700] rev 42164
copies: fix up "fullcopy" with missing entries from "copy"
This is just a workaround similar to the previous one. It will make it
easier to follow later patches.
Differential Revision: https://phab.mercurial-scm.org/D6246
Martin von Zweigbergk <martinvonz@google.com> [Sun, 14 Apr 2019 00:46:25 -0700] rev 42163
merge: remove workaround for
issue5020
As I explained in the previous commit, I think the filtering added
there is a better fix for the issue, so the workaround from
41f6af50c0d8 (merge: fix crash on criss cross merge with dir move and
delete (
issue5020), 2017-01-31) should no longer be needed.
Differential Revision: https://phab.mercurial-scm.org/D6245
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 22:03:04 -0700] rev 42162
copies: don't include copies that are not in source in directory move
I've been working on a rewrite of mergecopies(). I compared the output
of the rewritten version with the current version. I noticed that
between FIREFOX_NIGHTLY_59_END and FIREFOX_BETA_60_BASE in the
mozilla-unified repo, there were many copies that the current version
detected that the rewritten version did not. One example was
js/src/gc/Iteration.h -> js/src/gc/PublicIterators.h. Then I realized
that js/src/gc/Iteration.h doesn't even exist in
FIREFOX_NIGHTLY_59_END.
This patch adds a filtering step for the "fullcopy" dict. It turns out
that that change also affects the test for
issue5020 in
test-merge-criss-cross.t. The 'dm' action no longer happens there. At
first I thought that the test case change meant that this patch was
broken, but I think it's actually correct tha the 'dm' action should
not happen there. The result of the bid merge is still the same.
I suspect this filtering is a better solution for the issue than
41f6af50c0d8 (merge: fix crash on criss cross merge with dir move and
delete (
issue5020), 2017-01-31). I also suspect that it was broken
just a few months earlier by
a005c33d0bd7 (mergecopies: add logic to
process incomplete data, 2016-10-04). Note that bid merge had been
enabled for a few years at that point, since
19903277f035 (merge: use
bid merge by default (BC), 2014-10-01).
This patch is still just a workaround. It will be cleaned up soon
(with the rewrite of mergecopies()). But doing this in a separate
patch makes later patches easier to understand and gives a place to
explain why this is changing.
Differential Revision: https://phab.mercurial-scm.org/D6244
Martin von Zweigbergk <martinvonz@google.com> [Sat, 13 Apr 2019 00:24:17 -0700] rev 42161
tests: add test for
issue5343 (grafting with copies)
It seems that
issue5353 resulted in a lot of tests in test-graft.t,
but the bug actually reported in that issue didn't get a test
case. This patch adds one for the "move" and one for the "copy"
version of it. I also added a "copy+modify" case, to show what should
be a merge conflict. I didn't add one for the "backwards" version of
it since the comment says that that was already covered by previous
work.
The tests added by this patch show the broken behavior (the bug is
still open). I suspect the results returned from mergecopies() are not
expressive enough to fix this issue: it has a dict for copies to merge
with, but that can only give one more filename, but here we need two
(one for the path on the remote side and one for the path in the merge
base). I want to have it tested anyway since I'm about to refactor
mergecopies().
Differential Revision: https://phab.mercurial-scm.org/D6242
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Tue, 16 Apr 2019 13:12:21 -0400] rev 42160
chistedit: use context manager to set verbose ui
I'm still not exactly sure why this is necessary -- perhaps setting it
unconditionally would leak this setting in chg invocations.
Regardless, this would have looked very out of place as compared to
how this setting is done everywhere else, so at least for the sake of
style, let's be consistent with the rest of the codebase.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 16 Apr 2019 17:26:38 +0200] rev 42159
setdiscovery: stop limiting the number of local head we initially send
In our testing this limitation provides now real gain and instead triggers
pathological discovery timing for some repository with many heads.
See inline documentation for details.
Some timing below:
Mozilla try repository, (~1M revs, ~35K heads), discovery between 2 clones with
100 head missing on each side
before:
! wall 1.492111 comb 1.490000 user 1.450000 sys 0.040000 (best of 20)
! wall 1.813992 comb 1.820000 user 1.700000 sys 0.120000 (max of 20)
! wall 1.574326 comb 1.573500 user 1.522000 sys 0.051500 (avg of 20)
! wall 1.572583 comb 1.570000 user 1.520000 sys 0.050000 (median of 20)
after:
! wall 1.147834 comb 1.150000 user 1.090000 sys 0.060000 (best of 20)
! wall 1.449144 comb 1.450000 user 1.330000 sys 0.120000 (max of 20)
! wall 1.204618 comb 1.202500 user 1.146500 sys 0.056000 (avg of 20)
! wall 1.194407 comb 1.190000 user 1.140000 sys 0.050000 (median of 20)
pypy (~100 heads, 317 heads) discovery between clones with only 42 common heads
before:
! wall 0.031653 comb 0.030000 user 0.030000 sys 0.000000 (best of 25)
! wall 0.055719 comb 0.050000 user 0.040000 sys 0.010000 (max of 25)
! wall 0.038939 comb 0.039600 user 0.038400 sys 0.001200 (avg of 25)
! wall 0.038660 comb 0.050000 user 0.040000 sys 0.010000 (median of 25)
after:
! wall 0.018754 comb 0.020000 user 0.020000 sys 0.000000 (best of 49)
! wall 0.034505 comb 0.040000 user 0.030000 sys 0.010000 (max of 49)
! wall 0.019631 comb 0.019796 user 0.018367 sys 0.001429 (avg of 49)
! wall 0.019132 comb 0.020000 user 0.020000 sys 0.000000 (median of 49)
Private repository (~1M revs, ~3K heads), discovery from a strip subset, about
100 changesets to be pulled.
before:
! wall 1.837729 comb 1.840000 user 1.790000 sys 0.050000 (best of 20)
! wall 2.203468 comb 2.200000 user 2.100000 sys 0.100000 (max of 20)
! wall 2.049355 comb 2.048500 user 2.002500 sys 0.046000 (avg of 20)
! wall 2.035315 comb 2.040000 user 2.000000 sys 0.040000 (median of 20)
after:
! wall 0.136598 comb 0.130000 user 0.110000 sys 0.020000 (best of 20)
! wall 0.330519 comb 0.330000 user 0.260000 sys 0.070000 (max of 20)
! wall 0.157254 comb 0.155500 user 0.123000 sys 0.032500 (avg of 20)
! wall 0.149870 comb 0.140000 user 0.110000 sys 0.030000 (median of 20)
Same private repo, discovery between two clone with 500 different heads on each
side:
before:
! wall 2.372919 comb 2.370000 user 2.320000 sys 0.050000 (best of 20)
! wall 2.622422 comb 2.610000 user 2.510000 sys 0.100000 (max of 20)
! wall 2.450135 comb 2.450000 user 2.402000 sys 0.048000 (avg of 20)
! wall 2.443896 comb 2.450000 user 2.410000 sys 0.040000 (median of 20)
after:
! wall 0.625497 comb 0.620000 user 0.570000 sys 0.050000 (best of 20)
! wall 0.834723 comb 0.820000 user 0.730000 sys 0.090000 (max of 20)
! wall 0.675725 comb 0.675500 user 0.628000 sys 0.047500 (avg of 20)
! wall 0.671614 comb 0.680000 user 0.640000 sys 0.040000 (median of 20)
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 17:56:30 +0200] rev 42158
peer: introduce a limitedarguments attributes
When set to True, it signal that the peer cannot receive too larges arguments
and that algorithm must adapt. This should only be True for http peer that does
not support argument passed as "post".
This will be useful to unlock better discovery performance in the next
changesets.
I am using a dedicated argument because this is not really a usual
"capabilities" things. An alternative approach would be to adds a
"large-arguments" to all peer, but the http peers. That seemed a bit too hacky
to me.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 15:06:53 +0100] rev 42157
verify: also check full manifest validity during verify runs
Before this changes, `hg verify` only checked if a manifest revision existed and
referenced the proper files. However it never checked the manifest revision
content itself.
Mercurial is expecting manifest entries to be sorted and will crash otherwise.
Since `hg verify` did not attempted a full restoration of manifest entry, it
could ignore this kind of corruption.
This new check significantly increases the cost of a `hg verify` run. This
especially affects large repository not using `sparse-revlog`. For now, this is
hidden behind the `--full` experimental flag.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 01:11:09 +0200] rev 42156
verify: introduce an experimental --full flag
The flag currently has no effect, see next changeset for details. We introduce
the flag as experimental to keep the freedom of changing our mind on the final
UI.
Note: this patch highlight a small but in `hg help`. An option section is
generated even if no option are visible.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 01:12:21 +0200] rev 42155
verify: introduce a notion of "level"
Some checks are slower than others, to help the user to run the checks he needs,
we are about to introduce new flag to select faster vs deeper runs. This put
the scaffolding in place to do this.
Martin von Zweigbergk <martinvonz@google.com> [Sat, 13 Apr 2019 23:18:56 -0700] rev 42154
tests: split out separate test for
issue5020
The test was added to the existing setup in
41f6af50c0d8 (merge: fix
crash on criss cross merge with dir move and delete (
issue5020),
2017-01-31). I'm about to make a change that affects that test and
it's much easier to follow if the test case for
issue5020 is a
separate test case. The separate test case is based on what mpm
provided in comment 12 on the issue.
`hg diff -r
41f6af50c0d8^ tests/test-merge-criss-cross.t` after this
patch is pretty small (besides the added test). It's probably easier
for reviewers to look at that than to try to understand the diff
itself (I don't understand it).
Differential Revision: https://phab.mercurial-scm.org/D6243
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 18:04:54 -0700] rev 42153
tests: avoid a rename/delete conflict when updating in test-narrow-update.t
After the upcoming rewrite of mergecopies(), this test would otherwise
(accurately) start warning about "inside/f1 was deleted and renamed".
Differential Revision: https://phab.mercurial-scm.org/D6254
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 15:28:41 -0700] rev 42152
tests: delete unused function in test-rename-merge2.t
Differential Revision: https://phab.mercurial-scm.org/D6253
Martin von Zweigbergk <martinvonz@google.com> [Sun, 14 Apr 2019 13:46:40 -0700] rev 42151
tests: make merge conflicts explicit in `hg annotate` tests
We were using `true` as merge tool. I think it makes the test easier
to understand if we make the conflicts explcit. It also papered over a
conflict that shouldn't have been a conflict (just a bug in copy
tracing). I've marked that "BROKEN".
Differential Revision: https://phab.mercurial-scm.org/D6252
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 03:05:42 +0530] rev 42150
narrow: make warning about possibly dirty files respect ui.relative-paths
Differential Revision: https://phab.mercurial-scm.org/D6264
Augie Fackler <augie@google.com> [Wed, 17 Apr 2019 14:10:02 -0400] rev 42149
merge: forgot to pull before release
These two changes should be part of 5.0, but we are fine
leaving them out of the RC.
Augie Fackler <raf@durin42.com> [Wed, 17 Apr 2019 13:56:10 -0400] rev 42148
Added signature for changeset
4a8d9ed86475
Augie Fackler <raf@durin42.com> [Wed, 17 Apr 2019 13:56:08 -0400] rev 42147
Added tag 5.0rc0 for changeset
4a8d9ed86475
Augie Fackler <augie@google.com> [Wed, 17 Apr 2019 13:41:18 -0400] rev 42146
merge: default into stable for release candidate
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 16 Apr 2019 15:50:20 +0200] rev 42145
debugdiscovery: include the number of heads in all sets
We already displayed information about heads of the common set that are either
local or remote heads. We now also do so for heads of the common set that are
both local and remote heads too. This is useful because various step in the
set discovery algorithm have head specific optimizations.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 00:37:00 +0200] rev 42144
recover: add a --[no-]verify flag
For trivial cases, the cost of the verify run after `hg recover` is getting in
the way. In addition for very large repositories, the cost is simply too high
to be paid, making `hg recover` an unusable commands.
We introduce a --verify flag, set by default. If is automatically associated
with a --no-verify flag that one can use to skip the verify step.
We might consider changing the default behavior in the future. However this is
out of scope for this series.
Joerg Sonnenberger <joerg@bec.de> [Tue, 02 Apr 2019 19:48:31 +0200] rev 42143
bundle2: handle compression in _forwardchunks
_forwardchunks is used to compensate for getbundle protocol deficits.
Since it transparently decodes the payload, it also needs to remove the
corresponding compression parameter in case the server decides to send
one. This the wire protocol part of issue 5990.
Differential Revision: https://phab.mercurial-scm.org/D6182
Martin von Zweigbergk <martinvonz@google.com> [Wed, 27 Dec 2017 22:05:20 -0800] rev 42142
changelog: parse copy metadata if available in extras
This lets read back the copy metadata we just started writing. There
are still many places left to teach about getting the copy information
from the changeset, but we have enough ({file_copies}, specifically)
that we can add it now and have some test coverage of it.
Differential Revision: https://phab.mercurial-scm.org/D6186
Martin von Zweigbergk <martinvonz@google.com> [Wed, 27 Dec 2017 19:49:36 -0800] rev 42141
copies: add config option for writing copy metadata to file and/or changset
This introduces a config option that lets you choose to write copy
metadata to the changeset extras instead of to filelog. There's also
an option to write it to both places. I imagine that may possibly be
useful when transitioning an existing repo.
The copy metadata is stored as two fields in extras: one for copies
since p1 and one for copies since p2.
I may need to add more information later in order to make copy tracing
faster. Specifically, I'm thinking out recording which files were
added or removed so that copies._chaincopies() doesn't have to look at
the manifest for that. But that would just be an optimization and that
can be added once we know if it's necessary.
I have also considered saving space by using replacing the destination
file path by an index into the "files" list, but that can also be
changed later (but before the feature is ready to release).
Differential Revision: https://phab.mercurial-scm.org/D6183
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 04 Apr 2019 13:46:49 +0200] rev 42140
revsetbenchmark: add some simpler revset for heads and roots
This revset might leverage compiled code in the future so lets start to track
them.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 12 Apr 2019 16:25:59 +0200] rev 42139
repoview: flag `server.view` as experimental
Ideally, the non-experimental version of `experimental.extra-filter-revs` will
cover the use case for `server.view=immutable` well enough than having to have
this dedicated configuration. Since `server.view` is not part of any release, I
would prefer to have it marked as experimental to avoid having it to support it
for ever.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 12 Apr 2019 15:41:32 +0200] rev 42138
repoview: move subsettable in a dedicated module
The dictionary got moved in `branchmap` to avoid import cycle. However, we are
about to needs it in repoview too. So we introduce a now module to define that
that mapping.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 01 Feb 2019 15:51:02 +0100] rev 42137
upgrade: support upgrade to/from zstd storage (
issue6088)
Now that we have an official config option for a shiny format improvement, we
better make it simple to migrate to/from it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 Mar 2019 18:27:03 +0100] rev 42136
compression: introduce an official `zstd-revlog` requirement
This requirement supersedes `exp-compression-zstd`. However, we keep support for
the old requirement.
Strictly speaking, we do not need to add a new requirement, there are no logic
change making "new" repo incompatible with mercurial client using a version
that support `exp-compression-zstd`.
The choice to introduce a new requirement is motivated by the following:
* The previous requirement was explicitly "experimental". Using it by default
could confuse users.
* adding support for a hypothetical third compression engine will requires new
code, and will comes with its own requirement tool.
* We won't use it as the default for a while since I do not think we support
zstd on all platform. I can imagine we'll gain another (unrelated but on my
default) requirement by the time we turn this zstd by default.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 16 Apr 2019 15:10:16 +0200] rev 42135
compression: only declare revlog support for available engine
Even if we know that an engine support revlog compression, the Mercurial process
still won't support it if the compression engine is not available.
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 15 Apr 2019 19:21:41 +0300] rev 42134
branchcache: lazily validate nodes in iteritems()
This saves ~0.30 sec on creating a new branch on our internal repo.
Differential Revision: https://phab.mercurial-scm.org/D6236
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Apr 2019 15:01:33 +0300] rev 42133
branchcache: only iterate over branches which needs to be verified
Otherwise we loop over all the branches and call _verifybranch() even if not
required.
Differential Revision: https://phab.mercurial-scm.org/D6240
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Apr 2019 14:48:48 +0300] rev 42132
branchcache: fix the docstring of _verifybranch()
Initially the function was designed to support verifying all branches but later
I decided to have a separate function. I forget to remove the doc related to
that. Thanks to Yuya for spotting this in review.
Differential Revision: https://phab.mercurial-scm.org/D6239
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Apr 2019 14:39:14 +0300] rev 42131
branchcache: don't verify while creating a copy
The copy will not be mark as verified, so there is no need to verify nodes.
Thanks to Yuya who spotted this while reviewing.
Differential Revision: https://phab.mercurial-scm.org/D6238
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 05 Apr 2019 14:35:33 +0200] rev 42130
pull: improved message issued in case of failed update
When running `hg pull --update`, the update step may fail. Nothing in the error
message help to understand the abort is related to the secondary step (update)
instead of the primary step (pull).
We now add some information to the error message to clarify it comes from the
update part. It is useful in various situation (uncommitted changes blocking the
update, update to hidden destination, etc...)
The pull output is updated from:
$ hg pull ../repo-Bob --rev
956063ac4557 --update
pulling from ../repo-Bob
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 0 changes to 2 files (+1 heads)
(2 other changesets obsolete on arrival)
abort: filtered revision '6'!
to:
$ hg pull ../repo-Bob --rev
956063ac4557 --update
pulling from ../repo-Bob
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 0 changes to 2 files (+1 heads)
(2 other changesets obsolete on arrival)
abort: cannot update to target: filtered revision '6'!
(I am not sure why the actual error, "filtered revision '6'", is not using the
more modern format mentioning the obsolescence fate of '6')
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 05 Apr 2019 15:56:05 +0200] rev 42129
pull: deal with locally filtered changeset passed into --rev
Nowadays, it is possible to explicitly pull a remote revision that end up being
hidden locally (eg: obsoleted locally). However before this patch, some
internal processing where crashing trying to resolve a filtered revision.
Without this patches, the pull output result a confusing output:
$ hg pull ../repo-Bob --rev
956063ac4557
pulling from ../repo-Bob
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 0 changes to 2 files (+1 heads)
(2 other changesets obsolete on arrival)
abort: 00changelog.i@
956063ac4557828781733b2d5677a351ce856f59: filtered node!
Rodrigo Damazio Bovendorp <rdamazio@google.com> [Mon, 15 Apr 2019 22:13:11 -0700] rev 42128
absorb: aborting if another operation is in progress
This increases safety of using absorb by both aborting when another operation
is in progress (since the absorption could confuse any other command a lot)
and holding the locks throughout the reading of the working directory (for
which changes to absorb) and the reading of the repo (for which changes to
absorb into).
Differential Revision: https://phab.mercurial-scm.org/D6237
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 04 Apr 2019 13:58:49 +0200] rev 42127
repoview: keep the branchmap cache for the `served.hidden` view warm
For the same reason we want to keep the cache for the `served` view up to date,
we want to also keep the `served.hidden` view up to date. If some processes with
a readonly access to the repo needs to access it, we better have the cache warm
to avoid computing the same data over and over and over.
In most case (no secret changesets), the "served" and "served.hidden" set will
be identical and no cache will actually have to be updated.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 21 May 2018 17:28:35 +0200] rev 42126
repoview: introduce a filter for serving hidden changesets
There are multiple usecase for being able to explicitly view or pull obsolete
from a server. We need to be able to do so without exposing the secret
changesets. We introduces a dedicated repository "view" to do so. Way to expose
this "view" to the user will come later.
To keep a behavior consistent with expected client/server behavior, the general
idea is for the obsolete access to be explicitly requested by the code
generating the request. In addition, the will be server side configuration to
restrict the access to this feature.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 13 Apr 2019 20:57:50 +0200] rev 42125
repoview: fix conditional around unserved changesets
The conditional could lead to wrong computation since we have more unserved
changesets than just the "secret" phase.
In addition, now that we have efficient caching of phased changesets, we don't
need the conditional anymore.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 05 Apr 2019 16:05:20 +0300] rev 42124
branch: return early if we find an open named branch apart from default
The current code builds a list of all the open named branches except default and
then bool that. This is mostly fine until you get a repo which has thousands of
named branches.
Differential Revision: https://phab.mercurial-scm.org/D6211
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 05 Apr 2019 15:57:09 +0300] rev 42123
branchcache: don't verify closed nodes in _branchtip()
We only do membership testing there.
Differential Revision: https://phab.mercurial-scm.org/D6210
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 05 Apr 2019 15:56:33 +0300] rev 42122
branchcache: don't verify closed nodes in iteropen()
We expect that the nodes passed to iteropen() will be verified. We are only
testing for membership in closed nodes set, so we don't need to verify the whole
closed nodes set.
This will speed up calculating branchheads() when there are lot of closed nodes
related to other branches.
Differential Revision: https://phab.mercurial-scm.org/D6209
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 01 Apr 2019 13:56:47 +0300] rev 42121
branchcache: lazily validate nodes from the branchmap
On my personal hg-repository with 365 entries in .hg/cache/branch2, following
are the numbers for perfbranchmapload.
Before this patch:
! wall 0.000866 comb 0.000000 user 0.000000 sys 0.000000 (best of 2680)
! wall 0.001525 comb 0.000000 user 0.000000 sys 0.000000 (max of 2680)
! wall 0.001107 comb 0.001097 user 0.001086 sys 0.000011 (avg of 2680)
! wall 0.001104 comb 0.000000 user 0.000000 sys 0.000000 (median of 2680)
With this patch:
! wall 0.000530 comb 0.000000 user 0.000000 sys 0.000000 (best of 4240)
! wall 0.001078 comb 0.000000 user 0.000000 sys 0.000000 (max of 4240)
! wall 0.000696 comb 0.000693 user 0.000677 sys 0.000017 (avg of 4240)
! wall 0.000690 comb 0.000000 user 0.000000 sys 0.000000 (median of 4240)
On our internal repository with ~20k entries in branchcache, I see improvement
from 0.125 sec to 0.066 sec which is 47% speed up.
The above are the numbers of perfbranchmapload which shows how much time we
saved by not validating the nodes. But we need to validate some nodes. Following
are timings of some mercurial operations which have speed up because of this
lazy validation of nodes:
No-op `hg update` on our internal repository (Avg on 4 runs):
Before: 0.540 secs
After: 0.430 secs
Setting a branch name which already exists without --force (Avg of 4 runs):
Before: 0.510 secs
After: 0.250 secs
I ran the ASV performance suite and was unable to see any improvements except
there was improvement of perfdirstatewrite() on netbeans which I think was not
related.
I looked into the commit code, the command which I am trying to speedup, it
looks like it uses revbranchcache to update the branchcache.
Differential Revision: https://phab.mercurial-scm.org/D6208
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 19 Mar 2019 16:52:15 +0300] rev 42120
branchcache: add functions to validate changelog nodes
This patch adds functions to validate closed nodes, validate nodes for a certain
branch and for all the branches. These functions will be used in upcoming
patches.
Differential Revision: https://phab.mercurial-scm.org/D6207
Kyle Lippincott <spectral@google.com> [Mon, 15 Apr 2019 14:32:47 -0700] rev 42119
readline: provide styled prompt to readline (
issue6070)
Differential Revision: https://phab.mercurial-scm.org/D6168
Martin von Zweigbergk <martinvonz@google.com> [Tue, 02 Apr 2019 14:49:28 -0700] rev 42118
copies: move comment about implementation of mergecopies() to end
When you start reading about mergecopies(), you want to know what it
is, not that there are different implementations depending on config,
so this patch moves that comment to the end.
By the way, we don't seem to define what "copytracing" is. I'm just
leaving it that way because I don't know what it is myself. It seems
to be referred to only on mergecopies() (and not in pathcopies(), for
example), so maybe "copytracing" is supposed to be exactly what
mergecopies() does?
Differential Revision: https://phab.mercurial-scm.org/D6235
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 23:26:08 -0700] rev 42117
remotefilelog: return expected type from copies overrides
copies._computeforwardmissing() and copies._computenonoverlap() return
sets, so the overrides should also do that.
Differential Revision: https://phab.mercurial-scm.org/D6234
Martin von Zweigbergk <martinvonz@google.com> [Sun, 24 Mar 2019 23:47:01 -0700] rev 42116
changelog: extract a _string_unescape() to mirror _string_escape()
We use our own _string_escape() to encode the "extras" field. Then we
use codecs.escape_decode() to escape it. But there's also a little
workaround for dealing with escaped text that looks like octal numbers
since the fix for
https://bz.mercurial-scm.org/show_bug.cgi?id=3156. This patch extracts
the call to codecs.escape_decode() along with the fix for octal
numbers and puts it in a _string_unescape(). It also updates the test
to check for the octal-number case from the aforementioned bug.
As you may have suspected, I want to be able to reuse this new
function later.
Differential Revision: https://phab.mercurial-scm.org/D6184
Martin von Zweigbergk <martinvonz@google.com> [Wed, 20 Mar 2019 11:42:02 -0700] rev 42115
copies: extract function for deciding whether to use changeset-centric algos
We'll eventually have a "experimental.copies.read-from=changeset-only"
option too and I don't want to spread the logic for determining if we
should use changeset-centric of filelog-centric algorithms.
Differential Revision: https://phab.mercurial-scm.org/D6163
Martin von Zweigbergk <martinvonz@google.com> [Fri, 18 Jan 2019 13:13:48 -0800] rev 42114
getrenamedfn: get copy data from context object if configured
The function returned from getrenamedfn() calls
filelog.renamed(). That won't work when storing copy metadata in the
changeset.
I've just switched to a simple implementation here. We may or may not
need to optimize it later, possibly by optimizing the callers.
No more tests fail with "--extra-config-opt
experimental.copies.read-from=compatibility)" than they did before
this patch.
Differential Revision: https://phab.mercurial-scm.org/D6162
Pulkit Goyal <pulkit@yandex-team.ru> [Sun, 31 Mar 2019 16:27:10 +0300] rev 42113
branchmap: implement __contains__()
We have good occurences of `if branch in branchmap()` in our code. If
__contains__() is not implemented then it will use __iter__() to find whether
the element exists or not which is not good.
I am bit confused that whether I should move existing callers to hasbranch() or
this patch is a good way.
Differential Revision: https://phab.mercurial-scm.org/D6206
Pulkit Goyal <pulkit@yandex-team.ru> [Sun, 31 Mar 2019 16:20:17 +0300] rev 42112
branchmap: prevent using __getitem__() in branchheads()
branchheads() can directly use self._entries instead.
Differential Revision: https://phab.mercurial-scm.org/D6205
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 27 Mar 2019 21:33:04 +0300] rev 42111
branchmap: dynamically resolve type of branchcache class
This is required to support subclassing.
Thanks to Yuya for suggesting this in D6151.
Differential Revision: https://phab.mercurial-scm.org/D6204
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 09:41:08 -0700] rev 42110
copies: print list of divergent renames in sorted order
The current order is not obvious to the user (and I may change it in a
later patch). Lexicographical order seems like the obvious choice
here.
Differential Revision: https://phab.mercurial-scm.org/D6227
timeless <timeless@mozdev.org> [Fri, 12 Apr 2019 12:20:38 -0400] rev 42109
import: suppress accept: header
irccloud and others may perform content negotation and reject (406)
if the accept header does not match the mime type of the object
Augie Fackler <augie@google.com> [Fri, 12 Apr 2019 12:06:13 -0400] rev 42108
rebase: fix bug that prevented dry-run rebases from printing failures
As far as I can tell it should be fine to unconditionally skip
_prepareabortorcontinue if we're in the process of raising an Abort
here.
Differential Revision: https://phab.mercurial-scm.org/D6226
Augie Fackler <augie@google.com> [Fri, 12 Apr 2019 11:41:33 -0400] rev 42107
rebase: demonstrate bug in dry-run mode which causes cycles to not be reported
Differential Revision: https://phab.mercurial-scm.org/D6225
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 06 Apr 2019 17:48:11 +0200] rev 42106
test: minor cleanup to test-server-view.t
While looking into adding error output in this test, I did some cleanup.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 06 Apr 2019 10:44:22 +0200] rev 42105
repoview: improve documentation for `repo.filtered` method
I am sitting next to Joerg Sonnenberger and we are discussion his experience
with repoview. This first effect of this discussion is this documentation
clarification.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 05 Apr 2019 14:30:52 -0400] rev 42104
revset: short docstring for checkstatus
This is where all the action happens for the status-related revsets,
and a little documentation doesn't hurt.
Georges Racinet <georges.racinet@octobus.net> [Thu, 11 Apr 2019 18:10:07 +0200] rev 42103
discovery: stop direct use of attribute of partialdiscovery
Instead of accessing `undecided` directly for ui display purposes,
we introduce a `stats()` method that could be extended in the future with
more interesting information.
This is in preparation for a forthcoming Rust version of this object.
Indeed, attributes and furthermore properties are a bit complicated for
classes in native code.
We could go further and rename `undecided` to mark it private, but `_undecided`
is already taken as support for `_undecided` lazyness.
Martin von Zweigbergk <martinvonz@google.com> [Wed, 10 Apr 2019 17:36:37 -0700] rev 42102
overlayworkingctx: remove misleading trailing slash from directory pattern
The paths passed into the matcher are normalized (this applies to
include patterns and regular patterns, and to both glob kind and path
kind), so the regex for input "foo/" ended up being "foo(?:/|$)".
Once we have a (recursive) pattern kind only for directories, we could
switch to that here and remove the "mfiles[0] == path" check. Until
then, let's at least make it not misleading.
Differential Revision: https://phab.mercurial-scm.org/D6224
Martin von Zweigbergk <martinvonz@google.com> [Wed, 10 Apr 2019 17:31:32 -0700] rev 42101
overlayworkingctx: fix file/dir audit to be repo-relative
Before this patch, test-rebase-inmemory.t would stop erroring out
about the conflict if you added a "cd a" before line 252. That was
because a glob matcher (which are relative) was unintentionally
used. That happened because the matcher was given "include" patterns
(not regular patterns), and "include" patterns are always glob by
default (i.e. unless you write them including the kind prefix). IOW,
the "default='path'" argument passed to ctx.match() was ignored.
Differential Revision: https://phab.mercurial-scm.org/D6223
Martin von Zweigbergk <martinvonz@google.com> [Wed, 10 Apr 2019 16:26:40 -0700] rev 42100
messages: replace some instances of "folder" by "directory"
I'm pretty sure this is our preferred term.
Differential Revision: https://phab.mercurial-scm.org/D6222
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 11 Apr 2019 18:34:56 +0200] rev 42099
match: fix re2 compability broken in
2e2699af5649
When using re2, we call test_match() instead of match() on the
compiled regex object. While match() returns a matcher object or None,
test_match() returns True or False. So since
2e2699af5649 running test
on a machine with a re2 install fails in many places. Instead we make
the code a bit more general and everything goes back to normal.
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 10 Apr 2019 03:10:53 +0530] rev 42098
py3: add b'' prefixes to new doctests in match.py
# skip-blame as just b'' prefixes
Differential Revision: https://phab.mercurial-scm.org/D6221
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 10 Apr 2019 03:02:31 +0530] rev 42097
py3: add one new passing test found by buildbot
Differential Revision: https://phab.mercurial-scm.org/D6220
Yuya Nishihara <yuya@tcha.org> [Tue, 09 Apr 2019 21:59:37 +0900] rev 42096
cext: cast s# arguments of Py_BuildValue() to Py_ssize_t
The doc doesn't state that "s#" of Py_BuildValue() is controlled by
PY_SSIZE_T_CLEAN (unlike the one for PyArg_ParseTuple()), but actually
it's switched to Py_ssize_t.
https://docs.python.org/2/c-api/arg.html#c.Py_BuildValue
https://github.com/python/cpython/blob/2.7/Python/modsupport.c#L432
Follow up for
b01bbb8ff1f2 and
896b19d12c08.
Augie Fackler <augie@google.com> [Mon, 08 Apr 2019 10:52:04 -0400] rev 42095
remotefilelog: correctly reject wdir filenodes
This fixes `hg grep -r 'wdir()'` when remotefilelog is enabled and the working
directory contains uncommitted modifications.
Differential Revision: https://phab.mercurial-scm.org/D6217
Augie Fackler <augie@google.com> [Mon, 08 Apr 2019 10:56:55 -0400] rev 42094
remotefilelog: add tests of `hg grep -r 'wdir()'`
This demonstrates how remotefilelog breaks grepping dirtied working
directories. A future change will introduce a fix.
Differential Revision: https://phab.mercurial-scm.org/D6216
Martin von Zweigbergk <martinvonz@google.com> [Wed, 03 Apr 2019 16:03:41 -0700] rev 42093
config: read configs from directories in lexicographical order
Mercurial currently reads the .rc files specified in HGRCPATH (and the
system-default paths) in directory order, which is unspecified. My
team at work maintains a set of .rc files. So far there has been no
overlap between them, so we had not noticed this behavior. However, we
would now like to release some common .rc files and then have another
one per plaform with platform-specific overrides. It would be nice if
we can determine the load order by choosing names carefully. This
patch enables that by loading the .rc files in lexicographical order.
Before this patch, the added test case would consistently say "30" on
my file system (whatever I have -- some Linux FS).
Differential Revision: https://phab.mercurial-scm.org/D6193
Martin von Zweigbergk <martinvonz@google.com> [Wed, 03 Apr 2019 17:41:58 -0700] rev 42092
remotefilelog: fix crash on `hg addremove` of added-but-deleted file
If you `hg add` a file and then delete it from disk, and then run `hg
addremove`, the file ends up in the "removed" set that gets passed to
the findrenames() override. We then crash because the file is not in
the working copy parent. This patch fixes that.
Differential Revision: https://phab.mercurial-scm.org/D6194
Matt Harbison <matt_harbison@yahoo.com> [Fri, 05 Apr 2019 23:07:11 -0400] rev 42091
packaging: ensure that --python is an absolute path when building on Windows
For whatever reason, even though only python2 is on PATH, passing `python.exe`
causes the later check that it's not py3 to bail out.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 05 Apr 2019 22:47:45 -0400] rev 42090
packaging: don't crash building wix with python3.6 and earlier
`capture_output` was added in 3.7. I was tempted to just check and abort in
build.py, since Windows doesn't have the Linux problem where some distros only
ship an older python. But this is in a library that could be used elsewhere in
the future.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Wed, 03 Apr 2019 23:55:03 -0400] rev 42089
chistedit: add basic colours to diff view
This isn't complete, and it would be nice to show the exact same
colours that `hg diff` would show. That goal is too lofty, so this
just shows some basic colours, on the premise that a little is better
than nothing.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 05 Apr 2019 14:54:45 -0400] rev 42088
chistedit: use default curses colours
Terminals will define default colours (for example, white text on
black background), but curses doesn't obey those default colours
unless told to do so.
Calling `curses.use_default_colors` makes curses obey the default
terminal colours. One of the most obvious effects is that this allows
transparency on terminals that support it.
This also brings chistedit closer in appearance to crecord, which also
uses default colours.
The call may error out if the terminal doesn't support colors, but as
far as I can tell, everything still works. If we need a more careful
handling of lack of colours, blame me for not doing it now.
Denis Laxalde <denis@laxalde.org> [Sun, 07 Apr 2019 16:53:47 +0200] rev 42087
match: let regex match function return a boolean
Match function for regex pattern kind is built through
_buildregexmatch() and _buildmatch() using _rematcher() that returns a
re.match function, which either returns a match object or None. This
does not conform to Mercurial's matcher interface for __call__() or
exact(), which are expected to return a boolean value. We fix this by
building a lambda around _rematcher() in _buildregexmatch().
Accordingly, we update doctest examples to remove bool() calls that are
now useless.
Denis Laxalde <denis@laxalde.org> [Sun, 07 Apr 2019 17:16:58 +0200] rev 42086
match: make arguments of _expandsets() optional
Arguments 'ctx', 'listsubrepos' and 'badfn' are optional in function
body.
Denis Laxalde <denis@laxalde.org> [Sun, 07 Apr 2019 17:14:29 +0200] rev 42085
match: make _donormalize's auditor and warn arguments optional
Argument 'warn' is actually non-required, since there's a 'if warn:'
check before usage. Argument 'auditor' is passed to
pathutil.canonpath(), in which it is optional.
Denis Laxalde <denis@laxalde.org> [Mon, 08 Apr 2019 09:34:50 +0200] rev 42084
match: add doctest examples in match()
Make the docstring raw, as it now includes escape characters.
Denis Laxalde <denis@laxalde.org> [Sat, 06 Apr 2019 18:20:49 +0200] rev 42083
match: complete documentation of match() parameters
Denis Laxalde <denis@laxalde.org> [Sat, 06 Apr 2019 17:54:13 +0200] rev 42082
match: add doctest examples for patkind()
Denis Laxalde <denis@laxalde.org> [Sat, 06 Apr 2019 15:21:55 +0200] rev 42081
match: add a docstring with doctest examples to patternmatcher
Doctest examples aim at illustrating how __call__() and exact() are
different, depending on the pattern kind.
Denis Laxalde <denis@laxalde.org> [Sun, 07 Apr 2019 12:21:23 +0200] rev 42080
match: add doctest examples for exactmatcher
Make the docstring raw, since it now includes escape characters.
Martin von Zweigbergk <martinvonz@google.com> [Fri, 05 Apr 2019 11:24:00 -0700] rev 42079
localrepo: don't allow lookup of working directory revision
It seems that repo.lookup(), which is what supports the "lookup" wire
protocol command, should not allow the working copy revision
input.
This fixes both the pull test and the convert test I just added.
Differential Revision: https://phab.mercurial-scm.org/D6215
Martin von Zweigbergk <martinvonz@google.com> [Fri, 05 Apr 2019 11:22:26 -0700] rev 42078
tests: demonstrate broken pull of "
ffffffffffff" revision
Differential Revision: https://phab.mercurial-scm.org/D6214
Martin von Zweigbergk <martinvonz@google.com> [Fri, 05 Apr 2019 11:12:08 -0700] rev 42077
tests: demonstrate broken `hg convert` if "
ffffffffffff" is in description
Differential Revision: https://phab.mercurial-scm.org/D6213
Martin von Zweigbergk <martinvonz@google.com> [Fri, 05 Apr 2019 11:08:17 -0700] rev 42076
tests: add test of for hash reference translation by `hg convert`
The convert extension translates commit references in the commit
message. We didn't have any explicit testing of this before, so let's
add a test.
Differential Revision: https://phab.mercurial-scm.org/D6212
Matt Harbison <matt_harbison@yahoo.com> [Fri, 05 Apr 2019 18:36:43 -0400] rev 42075
py3: write out hgextindex as bytes in setup.py
I hit this trying to build the py2exe target using python3, just to see what
would happen. After commenting out `py2exe.Distribution` in setup.py and
pointing to a local copy of py2exe that supports python3[1], it complained that
`out` was bytes, not str.
[1] https://github.com/albertosottile/py2exe/releases/tag/v0.9.3.0
Philippe Pepiot <philippe.pepiot@logilab.fr> [Thu, 04 Apr 2019 15:40:48 +0200] rev 42074
setup: fix a possible NameError on rust build
File "setup.py", line 975, in rustbuild
"command: %r, environment: %r" % (self.rustsrcdir, cmd, env))
NameError: global name 'cmd' is not defined
Arun Chandrasekaran <aruncxy@gmail.com> [Mon, 01 Apr 2019 22:11:54 -0700] rev 42073
crecord: new keys g & G to navigate to the top and bottom respectively
This patch introduces two new keys 'g' and 'G' that helps to navigate to the
top and bottom of the file/hunk/line respectively. This is inline with the shortcuts
used in man, less, more and such tools that makes it convenient to navigate
swiftly.
'g' or HOME navigates to the top most file in the ncurses window.
'G' or END navigates to the bottom most file/hunk/line depending on the whether
the fold is active or not.
If the bottom most file is folded, it navigates to that file and stops there.
If the bottom most file is unfolded, it navigates to the bottom most hunk in
that file and stops there. If the bottom most hunk is unfolded, it navigates to
the bottom most line in that hunk.
Differential Revision: https://phab.mercurial-scm.org/D6178
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Thu, 04 Apr 2019 10:41:55 -0400] rev 42072
chistedit: properly show verbose diffs
I'm not sure if that ever worked and it's an internal API breakage,
but `"verbose": True` is not correctly parsed, as most of these
options are parsed by diffopts, whereas verbose is a global option.
Setting the UI to verbose instead does work and does show a verbose
patch, with full commit message.
It also shows all files, which unfortunately are a bit hard to read on
a single line in the default verbose template. Thus, we also change
the default template to use the status template, which shows one file
per line as well as its modification state.
Denis Laxalde <denis.laxalde@logilab.fr> [Thu, 04 Apr 2019 11:35:18 +0200] rev 42071
interactive: do not prompt about files given in command line
For commit and revert commands with --interactive and explicit files
given in the command line, we now skip the invite to "examine changes to
<file> ? [Ynesfdaq?]". The reason for this is that, if <file> is
specified by the user, asking for confirmation is redundant.
In patch.filterpatch(), we now use an optional "match" argument to
conditionally call the prompt() function when entering a new "header"
item. We use .exact() method to compare with files from the "header" in
order to only consider (rel)path patterns.
Add tests with glob patterns for commit and revert, to make sure we
still ask to examine files in these cases.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 17:34:43 -0700] rev 42070
zstandard: vendor python-zstandard 0.11
The upstream source distribution from PyPI was extracted. Unwanted
files were removed.
The clang-format ignore list was updated to reflect the new source
of files.
The project contains a vendored copy of zstandard 1.3.8. The old
version was 1.3.6. This should result in some minor performance wins.
test-check-py3-compat.t was updated to reflect now-passing tests on
Python 3.8.
Some HTTP tests were updated to reflect new zstd compression output.
# no-check-commit because 3rd party code has different style guidelines
Differential Revision: https://phab.mercurial-scm.org/D6199
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 15:24:03 -0700] rev 42069
cext: make osutil.c PY_SSIZE_T_CLEAN
This is needed to avoid a deprecation warning on Python 3.8.
With this change, we no longer see deprecation warnings for
this issue on Python 3.8.
Differential Revision: https://phab.mercurial-scm.org/D6198
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 15:21:30 -0700] rev 42068
cext: make parsers.c PY_SSIZE_T_CLEAN
This is needed to avoid a deprecation warning in Python 3.8. I believe
the conversion of int to Py_ssize_t is harmless in the changed
locations. But this being C code, it should be audited with care.
Differential Revision: https://phab.mercurial-scm.org/D6197
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 15:18:06 -0700] rev 42067
cext: make revlog.c PY_SSIZE_T_CLEAN
Without this, Python 3.8 emits a deprecation warning, as using
int for # values is deprecated. Many existing modules use
PY_SSIZE_T_CLEAN, so this shouldn't be contentious.
I audited the file for all # formatters and verified we are
using Py_ssize_t everywhere now.
Differential Revision: https://phab.mercurial-scm.org/D6196
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 18:20:36 -0700] rev 42066
tests: add optional output for Python 2.7 deprecation
We already had one of these a few lines above. We need it here as
well.
Differential Revision: https://phab.mercurial-scm.org/D6203
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 18:01:48 -0700] rev 42065
setup: use raw string for regular expression
Otherwise Python 3.8 complains about the backslash.
Differential Revision: https://phab.mercurial-scm.org/D6202
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 18:01:02 -0700] rev 42064
automation: use raw strings when there are backslashes
Otherwise Python 3.8 complains.
Differential Revision: https://phab.mercurial-scm.org/D6201
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Apr 2019 17:47:25 -0700] rev 42063
perf: make perf.run-limits code work with Python 3
We need b'' because perf.py isn't run through the source
transformer.
We need to cast the exception to bytes using pycompat.bytestr()
because ValueError can't be %s formatted due to built-in exceptions
lacking __bytes__.
We need to pycompat.sysstr() before the float() and int() cast
so the ValueError message doesn't have b'' in it.
Even with that, it looks like the error message for the ValueError
for float casts added quotes, so we need to account for that in test
output.
Differential Revision: https://phab.mercurial-scm.org/D6200
Martin von Zweigbergk <martinvonz@google.com> [Mon, 25 Dec 2017 05:55:50 -0800] rev 42062
localrepo: rename crev in _filecommit() to cnode, since it's a node
I know we often use "rev" generically, but here's it always a node, so
it helps to be specific.
Differential Revision: https://phab.mercurial-scm.org/D6181
Jerry Montfort <jerry.montfort@fake-box.com> [Fri, 05 Apr 2019 04:09:41 +0530] rev 42061
tests: unset environment variable P in test-revset2.t (
issue6109)
The test tests/test-revset2.t fails the test case
"Test repo.anyrevs with customized revset overrides" (line 1609)
if the environment variable P is set. The test implicitly expects that the
environment, in which it is started, does not export the variable 'P'.
To solve this issue, unset 'P' right before the test commands are run.
Differential Revision: https://phab.mercurial-scm.org/D6195
Philippe Pepiot <philippe.pepiot@logilab.fr> [Thu, 04 Apr 2019 19:08:37 +0200] rev 42060
hgmanpage: use a py2 and py3 compatible iterable protocol
Philippe Pepiot <philippe.pepiot@logilab.fr> [Thu, 04 Apr 2019 19:08:05 +0200] rev 42059
hgmanpage: use range instead of xrange
Philippe Pepiot <philippe.pepiot@logilab.fr> [Thu, 04 Apr 2019 19:06:48 +0200] rev 42058
packaging: allow to run make with python3
Use "?=", otherwise the variable cannot be set from environment.
Martin von Zweigbergk <martinvonz@google.com> [Wed, 03 Apr 2019 11:21:27 -0700] rev 42057
cleanup: use set literals where possible
Differential Revision: https://phab.mercurial-scm.org/D6192
Martin von Zweigbergk <martinvonz@google.com> [Wed, 19 Jul 2017 13:17:49 -0700] rev 42056
tests: rename "u" to more usual "ui" in test-context.py
Differential Revision: https://phab.mercurial-scm.org/D6191
Martin von Zweigbergk <martinvonz@google.com> [Wed, 03 Apr 2019 09:38:08 -0700] rev 42055
tests: better document the graft copy case
Differential Revision: https://phab.mercurial-scm.org/D6190
Augie Fackler <augie@google.com> [Wed, 03 Apr 2019 11:46:29 -0400] rev 42054
py2exe: add workaround to allow bundling of hgext3rd.* extensions
py2exe doesn't know how to handle namespace packages *at all*, so it treats
them like normal packages. As a result, if we try and bundle hgext3rd.evolve
in a py2exe build, it won't work if we install evolve into the virtualenv. In
order to work around this, tortoisehg installs hgext3rd.evolve etc into its
staged hg directory, since it doesn't use a virtualenv. As a workaround for us,
we'll just allow any extra packages users want bundled are part of hg during
the pseudo-install phase that py2exe uses. I'm not happy about this, but it
*works*.
As a sample of how you'd make an MSI with evolve bundled:
import os
import shutil
import subprocess
import tempfile
def stage_evolve(version):
"""Stage evolve for inclusion in py2exe binary."""
with tempfile.TemporaryDirectory() as temp:
evolve = os.path.join(temp, "evolve")
subprocess.check_call([
"hg.exe",
"clone",
"https://www.mercurial-scm.org/repo/evolve/",
"--update",
version,
evolve,
])
dest = os.path.join('..', 'hgext3rd', 'evolve')
if os.path.exists(dest):
shutil.rmtree(dest)
shutil.copytree(os.path.join(evolve, "hgext3rd", "evolve"), dest)
def main():
stage_evolve('tip')
print("\0")
print("hgext3rd")
print("hgext3rd.evolve")
print("hgext3rd.evolve.hack")
print("hgext3rd.evolve.thirdparty")
if __name__ == "__main__":
main()
is a script you can pass to the wix/build.py as --extra-packages-script,
and the resulting .msi will have an hg binary with evolve baked in. users
will still need to enable evolve in their hgrc, so you'd probably also
want to bundle configs in your msi for an enterprise environment, but that's
already easy to do with the support for extra features and wxs files in the
wix build process.
Differential Revision: https://phab.mercurial-scm.org/D6189
Augie Fackler <augie@google.com> [Tue, 02 Apr 2019 23:38:54 -0400] rev 42053
wix: fix the package build when not adding features
Should have used ifdef, not if. Sigh.
Differential Revision: https://phab.mercurial-scm.org/D6187
Rodrigo Damazio Bovendorp <rdamazio@google.com> [Mon, 01 Apr 2019 19:02:24 -0700] rev 42052
histedit: narrow the scope of discarded ui output
In
34165875fa5df813bec3a0cd348932b304d44efb, a lot of the output from
histedit was excluded. This slightly adjusts the scope of that exclusion,
to both discard more uninsteresting messages, and ensure that pre-merge-tool
output gets shown before the external merge tool is executed.
Differential Revision: https://phab.mercurial-scm.org/D6177
Matt Harbison <matt_harbison@yahoo.com> [Fri, 29 Mar 2019 21:53:15 -0400] rev 42051
uncommit: abort if an explicitly given file cannot be uncommitted (BC)
I've gotten burned several times by this in the last few days. The former tests
look simple enough, but if a good file and a bad file are given, the bad files
are silently ignored. Some commands like `forget` will warn about bogus files,
but that would likely get lost in the noise of an interactive uncommit. The
commit command aborts if a bad file is given, so this seems more consistent for
commands that alter the repository.
Navaneeth Suresh <navaneeths1998@gmail.com> [Mon, 25 Mar 2019 12:33:41 +0530] rev 42050
unshelve: disable unshelve during merge (
issue5123)
As stated in the
issue5123, unshelve can destroy the second parent of
the context when tried to unshelve with an uncommitted merge. This
patch makes unshelve to abort when called with an uncommitted merge.
See how shelve.mergefiles works. Commit structure looks like this:
```
... -> pctx -> tmpwctx -> shelvectx
/
/
second
merge parent
pctx = parent before merging working context(first merge parent)
tmpwctx = commited working directory after merge(with two parents)
shelvectx = shelved context
```
shelve.mergefiles first updates to pctx then it reverts shelvectx to pctx with:
```
cmdutil.revert(ui, repo, shelvectx, repo.dirstate.parents(),
*pathtofiles(repo, files),
**{'no_backup': True})
```
Reverting tmpwctx files that were merged from second parent to pctx makes them
added because they are not in pctx.
Changing this revert operation is crucial to restore parents after unshelve.
This is a complicated issue as this is not fixing a regression. Thus, for the
time being, unshelve during an uncommitted merge can be aborted.
(Details taken from http://mercurial.808500.n3.nabble.com/PATCH-V3-shelve-restore-parents-after-unshelve-
issue5123-tt4036858.html#
a4037408)
Differential Revision: https://phab.mercurial-scm.org/D6169
Augie Fackler <raf@durin42.com> [Mon, 01 Apr 2019 20:01:48 -0400] rev 42049
wix: add functionality to inject additional Features into installer
This is the last bit required to be able to glue extra configs etc into
the installer.
Differential Revision: https://phab.mercurial-scm.org/D6180
Augie Fackler <raf@durin42.com> [Mon, 01 Apr 2019 16:21:47 -0400] rev 42048
wix: add support for additional wxs files
As with my previous change for an --extra-prebuiild-script, I'm
assuming this is predominantly useful in an enterprise environment
and am only adding this to wix and not also to inno install scripts.
Differential Revision: https://phab.mercurial-scm.org/D6179
Augie Fackler <augie@google.com> [Wed, 20 Mar 2019 13:18:37 -0400] rev 42047
wix: add a hook for a prebuild script to inject extra libraries
I need this to build packages for Google so we can bundle some
extensions in the installed image. My assumption is that this is most
interesting for the .msi images so I only wired it up there. I'm not
thrilled with the interface this provides, but it was an easy way to
retain debug messages on Windows while also having enough structure to
know what lines are actually module names for py2exe.
Still pending on my end: I need to bundle a couple of config files,
and at least one data file. I'm open to advice on how to do those
things, and how to do this better.
Differential Revision: https://phab.mercurial-scm.org/D6164
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 Mar 2019 18:26:54 +0100] rev 42046
compression: introduce an official `format.revlog-compression` option
This option supersedes the `experiment.format.compression` option. The value
currently supported are zlib (default) and zstd (if Mercurial was compiled with
zstd support).
The option gained an explicit reference to `revlog` since this is the target
usage here. Different storage methods might require different compression
strategies.
In our tests, using zstd give a significant CPU usage improvement (both
compression and decompressing) while keeping similar repository size.
Zstd as other interresting mode (dictionnary, pre-text, etc…) that are probably
worth exploring. However, just plain switching from zlib to zstd provide a large
benefit.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 02 Apr 2019 11:03:46 -0700] rev 42045
compression: display compression level in debugformat
Now that we have options to control the compression level, we teach `hg
debugformat` about them. This is a useful information when comparing
repositories.
Note that we have no trace of the compression level used to store existing
deltas. Actually, it would even varies from one delta to another. So we display
the currently set value.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 Mar 2019 18:35:59 +0100] rev 42044
compression: introduce a `storage.revlog.zstd.level` configuration
This option control the zstd compression level used when compressing revlog
chunk. The usage of zstd for revlog compression has not graduated from
experimental yet, but we intend to fix that soon.
The option name for the compression level is more straight forward to pick, so
this changesets comes first. Having a dedicated option for each compression
engine is useful because they don't support the same range of values.
I ran the same measurement as for the zlib compression level (in the parent
changesets). The variation in repository size is stay mostly in the same (small)
range. The "read/write" performance see smallish variation, but are overall much
better than zlib. Write performance show the same tend of having better write
performance for when reaching high-end compression.
Again, we don't intend to change the default zstd compression level (currently:
3) in this series. However this is worth investigating in the future.
The Performance comparison of zlib vs zstd is quite impressive. The repository
size stay in the same range, but the performance are much better in all
situations.
Comparison summary
==================
We are looking at:
- performance range for zlib
- performance range for zstd
- comparison of default zstd (level-3) to default zlib (level 6)
- comparison of the slowest zstd time to the fastest zlib time
Read performance:
-----------------
| zlib | zstd | cmp | f2s
mercurial | 0.170159 - 0.189219 | 0.144127 - 0.149624 | 80% | 88%
pypy | 2.679217 - 2.768691 | 1.532317 - 1.705044 | 60% | 63%
netbeans | 122.477027 - 141.620281 | 72.996346 - 89.731560 | 58% | 73%
mozilla | 147.867662 - 170.572118 | 91.700995 - 105.853099 | 56% | 71%
Write performance:
------------------
| zlib | zstd | cmp | f2s
mercurial | 53.250304 - 56.2936129 | 40.877025 - 45.677286 | 75% | 86%
pypy | 460.721984 - 476.589918 | 270.545409 - 301.002219 | 63% | 65%
netbeans | 520.560316 - 715.930400 | 370.356311 - 428.329652 | 55% | 82%
mozilla | 739.803002 - 987.056093 | 505.152906 - 591.930683 | 57% | 80%
Raw data
--------
repo alg lvl .hg/store size 00manifest.d read write
mercurial zlib 1 49,402,813 5,963,475 0.170159 53.250304
mercurial zlib 6 47,197,397 5,875,730 0.182820 56.264320
mercurial zlib 9 47,121,596 5,849,781 0.189219 56.293612
mercurial zstd 1 49,737,084 5,966,355 0.144127 40.877025
mercurial zstd 3 48,961,867 5,895,208 0.146376 42.268142
mercurial zstd 5 48,200,592 5,938,676 0.149624 43.162875
mercurial zstd 10 47,833,520 5,913,353 0.145185 44.012489
mercurial zstd 15 47,314,604 5,728,679 0.147686 45.677286
mercurial zstd 20 47,330,502 5,830,539 0.145789 45.025407
mercurial zstd 22 47,330,076 5,830,539 0.143996 44.690460
pypy zlib 1 370,830,572 28,462,425 2.679217 460.721984
pypy zlib 6 340,112,317 27,648,747 2.768691 467.537158
pypy zlib 9 338,360,736 27,639,003 2.763495 476.589918
pypy zstd 1 362,377,479 27,916,214 1.532317 270.545409
pypy zstd 3 354,137,693 27,905,988 1.686718 294.951509
pypy zstd 5 342,640,043 27,655,774 1.705044 301.002219
pypy zstd 10 334,224,327 27,164,493 1.567287 285.186239
pypy zstd 15 329,000,363 26,645,965 1.637729 299.561332
pypy zstd 20 324,534,039 26,199,547 1.526813 302.149827
pypy zstd 22 324,530,595 26,198,932 1.525718 307.821218
netbeans zlib 1 1,281,847,810 165,495,457 122.477027 520.560316
netbeans zlib 6 1,205,284,353 159,161,207 139.876147 715.930400
netbeans zlib 9 1,197,135,671 155,034,586 141.620281 678.297064
netbeans zstd 1 1,259,581,737 160,840,613 72.996346 370.356311
netbeans zstd 3 1,232,978,122 157,691,551 81.622317 396.733087
netbeans zstd 5 1,208,034,075 160,246,880 83.080549 364.342626
netbeans zstd 10 1,188,624,176 156,083,417 79.323935 403.594602
netbeans zstd 15 1,176,973,589 153,859,477 89.731560 428.329652
netbeans zstd 20 1,162,958,258 151,147,535 82.842667 392.335349
netbeans zstd 22 1,162,707,029 151,150,220 82.565695 402.840655
mozilla zlib 1 2,775,497,186 298,527,987 147.867662 751.263721
mozilla zlib 6 2,596,856,420 286,597,671 170.572118 987.056093
mozilla zlib 9 2,587,542,494 287,018,264 163.622338 739.803002
mozilla zstd 1 2,723,159,348 286,617,532 91.700995 570.042751
mozilla zstd 3 2,665,055,001 286,152,013 95.240155 561.412805
mozilla zstd 5 2,607,819,817 288,060,030 101.978048 505.152906
mozilla zstd 10 2,558,761,085 283,967,648 104.113481 497.771202
mozilla zstd 15 2,526,216,060 275,581,300 105.853099 591.930683
mozilla zstd 20 2,485,114,806 266,478,859 95.268795 576.515389
mozilla zstd 22 2,484,869,080 266,456,505 94.429282 572.785537
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 Mar 2019 18:35:27 +0100] rev 42043
compression: introduce a `storage.revlog.zlib.level` configuration
This option control the zlib compression level used when compression revlog
chunk.
This is also a good excuse to pave the way for a similar configuration option
for the zstd compression engine. Having a dedicated option for each compression
algorithm is useful because they don't support the same range of values.
Using a higher zlib compression impact CPU consumption at compression time, but
does not directly affected decompression time. However dealing with small
compressed chunk can directly help decompression and indirectly help other
revlog logic.
I ran some basic test on repositories using different level. I am using the
mercurial, pypy, netbeans and mozilla-central clone from our benchmark suite.
All tested repository use sparse-revlog and got all their delta recomputed.
The different compression level has a small effect on the repository size
(about 10% variation in the total range). My quick analysis is that revlog
mostly store small delta, that are not affected by the compression level much.
So the variation probably mostly comes from better compression of the snapshots
revisions, and snapshot revision only represent a small portion of the
repository content.
I also made some basic timings measurements. The "read" timings are gathered using
simple run of `hg perfrevlogrevisions`, the "write" timings using `hg
perfrevlogwrite` (restricted to the last 5000 revisions for netbeans and
mozilla central). The timings are gathered on a generic machine, (not one of
our performance locked machine), so small variation might not be meaningful.
However large trend remains relevant.
Keep in mind that these numbers are not pure compression/decompression time.
They also involve the full revlog logic. In particular the difference in chunk
size has an impact on the delta chain structure, affecting performance when
writing or reading them.
On read/write performance, the compression level has a bigger impact.
Counter-intuitively, the higher compression levels improve "write" performance
for the large repositories in our tested setting. Maybe because the last 5000
delta chain end up having a very different shape in this specific spot? Or maybe
because of a more general trend of better delta chains thanks to the smaller
chunk and snapshot.
This series does not intend to change the default compression level. However,
these result call for a deeper analysis of this performance difference in the
future.
Full data
=========
repo level .hg/store size 00manifest.d read write
----------------------------------------------------------------
mercurial 1 49,402,813 5,963,475 0.170159 53.250304
mercurial 6 47,197,397 5,875,730 0.182820 56.264320
mercurial 9 47,121,596 5,849,781 0.189219 56.293612
pypy 1 370,830,572 28,462,425 2.679217 460.721984
pypy 6 340,112,317 27,648,747 2.768691 467.537158
pypy 9 338,360,736 27,639,003 2.763495 476.589918
netbeans 1 1,281,847,810 165,495,457 122.477027 520.560316
netbeans 6 1,205,284,353 159,161,207 139.876147 715.930400
netbeans 9 1,197,135,671 155,034,586 141.620281 678.297064
mozilla 1 2,775,497,186 298,527,987 147.867662 751.263721
mozilla 6 2,596,856,420 286,597,671 170.572118 987.056093
mozilla 9 2,587,542,494 287,018,264 163.622338 739.803002
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 Mar 2019 19:34:10 +0100] rev 42042
compression: accept level management for zlib compression
We update the zlib related class to be support setting the compression level.
This changeset focus on updating the internal only. A way to configure this
level will be introduced in the next changeset.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 Mar 2019 16:45:14 +0100] rev 42041
util: extract compression code in `mercurial.utils.compression`
The code seems large enough to be worth extracting. This is similar to what was
done for various module in `mercurial/utils/`.
Since None of the compression logic takes a `ui` objet, issuing deprecation
warning is tricky. Luckly the logic does not seems to have many external users.
Martin von Zweigbergk <martinvonz@google.com> [Sat, 30 Mar 2019 13:13:10 -0700] rev 42040
merge: make "labels" argument to graft() optional, like it is for update()
graft() just passes the argument on to update(), and update() doesn't
require it, so graft() shouldn't either.
Differential Revision: https://phab.mercurial-scm.org/D6175
Martin von Zweigbergk <martinvonz@google.com> [Sun, 31 Mar 2019 09:39:02 -0700] rev 42039
revset: remove comment about linkrev workaround from user-facing docs
I think the code is clear enough so we don't need to keep the comment
at all (by now, most Mercurial developers are probably familiar with
the linkrevs issues).
Differential Revision: https://phab.mercurial-scm.org/D6176
Martin von Zweigbergk <martinvonz@google.com> [Fri, 29 Mar 2019 11:32:02 -0700] rev 42038
shelve: let cmdutil.revert() take care of backing up untracked files
cmdutil.revert() backs up untracked files, so I don't see a reason to
do it shelve.mergefiles(). We have tests for this and they still pass.
Differential Revision: https://phab.mercurial-scm.org/D6174
Martin von Zweigbergk <martinvonz@google.com> [Fri, 29 Mar 2019 11:31:42 -0700] rev 42037
shelve: stop passing list of files to revert
It seems to work just fine to not specify any files here. I suspect it
looked the way it did for historical reasons. It apparently used to
use merge instead of rebase until
1d7a36ff2615 (shelve: use rebase
instead of merge (
issue4068), 2013-10-23) and it makes sense to want
to restrict the set of files then.
I noticed this because of the files.extend(shelvectx.p1().files()). If
the working copy was clean before, then shelvectx.p1() will be the
working copy parent and that ended up adding all the files in that
set. In our Google-internal Mercurial setup (including a FUSE) that
was very noticeably slow when the working copy parent happened to have
many files in large directories.
This patch doesn't yet remove the call to shelvectx.p1().files(). We
also use that set for deciding what to back up. I'm pretty sure it's
safe to back up only the set of files we already back even if we no
longer restrict the set of files to revert, so this patch should be
safe on its own. Regardless, the next patch will delegate the
backing-up to cmdutil.revert().
Incidentally, this also gets rid of a repo.pathto() that I had earlier
wanted to get rid of.
Differential Revision: https://phab.mercurial-scm.org/D6173
Martin von Zweigbergk <martinvonz@google.com> [Wed, 27 Mar 2019 14:55:46 -0700] rev 42036
remotefilelog: prefetch files in deterministic order
I have been troubleshooting some slowness in this area (it's unclear
if it's the client or server that's to blame, but that's beside the
point) and it's a lot easier to do troubleshoot if the files are
prefetched in the same order each time.
Differential Revision: https://phab.mercurial-scm.org/D6172
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 26 Mar 2019 17:35:28 +0100] rev 42035
debugdiscovery: display time elapsed during the discovery step
This is a useful information. Now that we perform more analysing after the
discovery is done, it is worth have a more precise measurement. For serious
timing analysis use `hg perfdiscovery`.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 26 Mar 2019 17:26:54 +0100] rev 42034
debugdiscovery: only list common heads on verbose
The list of common heads is only part of the useful information. In addition on
repository with many heads, the information is very not helpful (just fill a
couple of screen with hash). As a result we hide it behind a --verbose flag.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 26 Mar 2019 17:26:11 +0100] rev 42033
debugdiscovery: drop duplicated information
The old line informing about the local being a superset or subset of the remote
is redundant with the newly introduced data. So we drop it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 26 Mar 2019 17:25:22 +0100] rev 42032
debugdiscovery: display more statistic about the common set
We display a lot more information now. Especially, we display the overlap
between the common heads and the local/remote heads. There are various
optimization geared toward heads, as a result, the less common the heads the
more complex the discovery. Having this information easily accessible help when
working on discovery.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 26 Mar 2019 14:04:33 +0100] rev 42031
debugdiscovery: small internal refactoring
The part of the code displaying statistic is made independant from the one
running the discovery. In the same do, the declaration of the discovery
function is a bit simplified.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 26 Mar 2019 14:02:40 +0100] rev 42030
debugdiscovery: allow to select random seed during debugdiscovery run
The randomness can lead to large timing difference, controling it is important.
Pulkit Goyal <pulkit@yandex-team.ru> [Sun, 17 Mar 2019 18:45:53 +0300] rev 42029
discovery: move cl.hasnode outside of the for-loop
IIUC, resolving attributes for changelog can lead to some overhead. So this
patch moves that to outside of a for-loop.
Differential Revision: https://phab.mercurial-scm.org/D6147
Pulkit Goyal <pulkit@yandex-team.ru> [Sun, 17 Mar 2019 18:43:27 +0300] rev 42028
discovery: prevent deleting items from a dictionary
Removing elements from Python dictionary is expensive. So let's prevent adding
them instead.
I added a newline to make code look a bit better.
Differential Revision: https://phab.mercurial-scm.org/D6146
Pulkit Goyal <pulkit@yandex-team.ru> [Sun, 17 Mar 2019 18:34:28 +0300] rev 42027
discovery: drop some unused sets
Differential Revision: https://phab.mercurial-scm.org/D6145
Pulkit Goyal <pulkit@yandex-team.ru> [Sun, 17 Mar 2019 18:29:23 +0300] rev 42026
discovery: prevent recomputing info about server and outgoing changesets
We already iterate over the outgoing.missing above and lookup repo for them. So
let's reuse info calculated at that time instead of recomputing that again.
Also we calculate the set of remotebranches by doing set(remotemap), so let's
reuse that again.
Upcoming patches will clean things a bit more.
Differential Revision: https://phab.mercurial-scm.org/D6144
Alexander Kobjolke <alex@jakalx.net> [Thu, 21 Mar 2019 21:44:29 +0100] rev 42025
crecord: draw on the whole screen
When starting crecord, one can see that it has a small gap on the rightmost
column which doesn't get used. This is in contrast to other interactive curses
frontends such as chistedit.
Disabling the displaying of the cursor allows drawing on the whole availabe
area and thus some hacky code in align() could be removed.
Differential Revision: https://phab.mercurial-scm.org/D6171
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 15 Mar 2019 11:24:08 -0700] rev 42024
automation: perform tasks on remote machines
Sometimes you don't have access to a machine in order to
do something. For example, you may not have access to a Windows
machine required to build Windows binaries or run tests on that
platform.
This commit introduces a pile of code intended to help
"automate" common tasks, like building release artifacts.
In its current form, the automation code provides functionality
for performing tasks on Windows EC2 instances.
The hgautomation.aws module provides functionality for integrating
with AWS. It manages EC2 resources such as IAM roles, EC2
security groups, AMIs, and instances.
The hgautomation.windows module provides a higher-level
interface for performing tasks on remote Windows machines.
The hgautomation.cli module provides a command-line interface to
these higher-level primitives.
I attempted to structure Windows remote machine interaction
around Windows Remoting / PowerShell. This is kinda/sorta like
SSH + shell, but for Windows. In theory, most of the functionality
is cloud provider agnostic, as we should be able to use any
established WinRM connection to interact with a remote. In
reality, we're tightly coupled to AWS at the moment because
I didn't want to prematurely add abstractions for a 2nd cloud
provider. (1 was hard enough to implement.)
In the aws module is code for creating an image with a fully
functional Mercurial development environment. It contains VC9,
VC2017, msys, and other dependencies. The image is fully capable
of building all the existing Mercurial release artifacts and
running tests.
There are a few things that don't work. For example, running
Windows tests with Python 3. But building the Windows release
artifacts does work. And that was an impetus for this work.
(Although we don't yet support code signing.)
Getting this functionality to work was extremely time consuming.
It took hours debugging permissions failures and other wonky
behavior due to PowerShell Remoting. (The permissions model for
PowerShell is crazy and you brush up against all kinds of
issues because of the user/privileges of the user running
the PowerShell and the permissions of the PowerShell session
itself.)
The functionality around AWS resource management could use some
improving. In theory we support shared tenancy via resource
name prefixing. In reality, we don't offer a way to configure
this.
Speaking of AWS resource management, I thought about using a tool
like Terraform to manage resources. But at our scale, writing a
few dozen lines of code to manage resources seemed acceptable.
Maybe we should reconsider this if things grow out of control.
Time will tell.
Currently, emphasis is placed on Windows. But I only started
there because it was likely to be the most difficult to implement.
It should be relatively trivial to automate tasks on remote Linux
machines. In fact, I have a ~1 year old script to run tests on a
remote EC2 instance. I will likely be porting that to this new
"framework" in the near future.
# no-check-commit because foo_bar functions
Differential Revision: https://phab.mercurial-scm.org/D6142
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 09 Mar 2019 16:36:08 -0800] rev 42023
contrib: PowerShell script to install development dependencies
Configuring a Windows machine to hack on Mercurial is a bit of work
and it isn't documented very well.
This commit introduces a PowerShell script to automate going from
a fresh Windows install to an environment suitable for building
Mercurial, its installers, and running tests.
Differential Revision: https://phab.mercurial-scm.org/D6141
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Tue, 26 Mar 2019 11:53:30 -0400] rev 42022
chistedit: change in-progress message
Saying "running histedit" is an artifact of when chistedit was a
separate thing from histedit. I found the message a bit confusing,
since wasn't I running histedit from the beginning, just from the
curses interface?
The whole thing is now histedit, both the curses interface and the
underlying procedure to apply a plan, so let's use a message that
doesn't make a distinction.
Augie Fackler <augie@google.com> [Tue, 26 Mar 2019 10:21:17 -0400] rev 42021
perf: copyedit a few documentation strings
Differential Revision: https://phab.mercurial-scm.org/D6170
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Sun, 24 Mar 2019 20:13:13 -0400] rev 42020
shelve: add --keep to list of allowables
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 17 Mar 2019 12:30:52 +0000] rev 42019
perf: introduce a `perf.run-limits` options
This options make it possible to configure the number of run that the extensions
will perform. This is useful for automated benchmark or for performance
measurement that need better accuracy.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 Mar 2019 19:11:19 +0000] rev 42018
perf: pass limits as a function argument
The function applying the limit has no access to the configuration. Therefore,
some higher layer will have to pass it as argument.
We do this in an independent change to clarify the next change.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 16 Mar 2019 19:08:27 +0000] rev 42017
perf: more flexible implementation for checking stop conditions
We want to make this logic simpler to configure. The first step is to stop
hard-coding every values.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 25 Mar 2019 08:41:02 -0700] rev 42016
perf: document perfparents
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 25 Mar 2019 13:43:40 +0100] rev 42015
perf: document config options
We have configuration, so we better document it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 25 Mar 2019 13:16:53 +0100] rev 42014
tests: use "perf" as a the extension name in test-contrib-perf.t
This is simpler and seems more "inline" with the name people usually use for
this extensions.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 22 Mar 2019 11:26:47 -0400] rev 42013
shelve: do not update when keeping changes, just move the dirstate
This is to leave the working directory unchanged. We reuse the match
object as an optimisation.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 22 Mar 2019 13:03:26 -0400] rev 42012
shelve: refactor _shelvecreatedcommit's match object into calling site
We might need to use this match object again to move the dirstate in
case the user requested to `--keep` the changes.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 22 Mar 2019 11:24:23 -0400] rev 42011
shelve: new keep option
Does nothing yet. The docstring describes what it will soon be doing.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 21 Mar 2019 21:40:22 -0400] rev 42010
diff: support diffing explicit files in subrepos
Most other commands support implied recursion based on file names already.
Danny Hooper <hooper@google.com> [Thu, 21 Mar 2019 18:27:09 -0700] rev 42009
fix: make the order of the work queue deterministic
This makes any output generated during the parallel phase of execution stable
if parallelism is disabled. This helps write tests like that in the future.
Differential Revision: https://phab.mercurial-scm.org/D6166
Danny Hooper <hooper@google.com> [Thu, 21 Mar 2019 18:35:39 -0700] rev 42008
fix: allow fixing untracked files when given as arguments
It's more useful to fix the file than to silently avoid it when the user does
this:
hg fix --working-dir untracked-file
We will still do nothing to ignored files, even if they are specified. This may
be safer, given that such files can unexpectedly slip into the arguments via a
shell glob or fileset.
Differential Revision: https://phab.mercurial-scm.org/D6165
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 19 Mar 2019 16:26:52 +0300] rev 42007
branchcache: have a hasnode function to validate nodes
Upcoming patches will delay node validation until it's required. So we need to a
way in branchcache class to check whether a node exists or node.
Other options were making repo or changelog an attribute of branchcache object.
But the branchcache depends on filters so I decided to pass a function object.
Differential Revision: https://phab.mercurial-scm.org/D6157
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 19 Mar 2019 16:20:02 +0300] rev 42006
branchcache: add attributes to track which nodes are verified
Half of the cost of loading branchcache comes from verifiying all the nodes it
has. We don't need to verify all the nodes in all the cases. Sometimes we need
to verify only a set of nodes for a set of branches. We can ignore nodes of
other branches as we are not going to read them.
This patch introduces two attributes to branchcache class. _verifiedbranches is
a set which will tell the branches for which it's head nodes are verified.
_closedverified is a boolean which will tell whether all closednodes are
verified or not.
Another idea which I had was to keep a set of nodes which are verified, but it
will be more ugly and difficult to track things on node level.
Differential Revision: https://phab.mercurial-scm.org/D6156
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 18 Mar 2019 19:44:55 +0300] rev 42005
branchcache: make entries a private attribute
Differential Revision: https://phab.mercurial-scm.org/D6155
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 18 Mar 2019 19:31:45 +0300] rev 42004
branchcache: introduce hasbranch()
This will be used to check whether a branch exists or not. This will optimized
in future.
Differential Revision: https://phab.mercurial-scm.org/D6154
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 18 Mar 2019 19:11:55 +0300] rev 42003
branchmap: drop branchcache.setdefault() (API)
All the callers are updated to call setdefault of branchcache.entries
Differential Revision: https://phab.mercurial-scm.org/D6153
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 18 Mar 2019 19:01:29 +0300] rev 42002
branchcache: rename itervalues() to iterheads()
The itervalues() exists because branchcache() had a dict interface. Since it no
longer has a dict interface, it makes sense to have better function names.
If a person does not understand how branchcache stores info, it will be hard for
them to guess what itervalues() does.
Differential Revision: https://phab.mercurial-scm.org/D6152
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 18 Mar 2019 18:59:38 +0300] rev 42001
branchmap: remove the dict interface from the branchcache class (API)
The current branchmap computation involves reading the whole branchmap from
disk, validating all the nodes even if they are not required. This leads to a
lot of time on repos which have large branchmap or a lot of branches. On large
repos, this can validate around 1000's of nodes.
On some operations, like finding whether a branch exists or not, we don't need
to validate all the nodes. Or updating heads for a single branch.
Before this patch, branchcache class was having dict interface and it was hard
to keep track of reads.
This patch removes the dict interface. Upcoming patches will implement lazy
loading and validation of data and implement better API's.
Differential Revision: https://phab.mercurial-scm.org/D6151
Yuya Nishihara <yuya@tcha.org> [Sat, 23 Mar 2019 20:59:07 +0900] rev 42000
test-template: fix stdio mode on Windows
Otherwise, CBOR data would be corrupted. Spotted by Matt Harbison.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 22 Mar 2019 12:30:05 -0400] rev 41999
samplehgrcs: update the list of suggested extensions
Back in the day, this was color and pager, both of which are now
default. Churn isn't that popular, but the other four below
(obviously?) are.
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Fri, 22 Mar 2019 12:28:59 -0400] rev 41998
samplehgrcs: clarify which lines should be uncommented
The original wording has confused at least one person. Hopefully it's
clearer this way.
https://stackoverflow.com/questions/
55288177/adding-hg-strip-to-hgrc-config-file
Yuya Nishihara <yuya@tcha.org> [Sun, 10 Mar 2019 13:07:36 +0900] rev 41997
templatefilters: add {x|cbor} filter for custom CBOR output