Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:16:33 -0700] rev 42221
tests: make log style a little easier to read in test-copytrace-heuristics.t
Revision numbers are much shorter and easier to read (especially
compared to the full nodeids that were used here), so I switched to
that. That's also what almost all the commands used (e.g. `hg rebase
-s . -d 1`). I updated the two instances that used nodeids. I also
made some other little cleanups to the log templates.
Differential Revision: https://phab.mercurial-scm.org/D6279
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:23:26 -0700] rev 42220
tests: avoid cryptic nodeids in tests/test-rename-merge1.t
These two nodeids had not been part of any output before, so one can't
know which revision they refer to without adding something like `hg
log` before them. It turned out that '.^' was equivalent for both of
them, so that's what I replaced them with.
Differential Revision: https://phab.mercurial-scm.org/D6280
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:08:58 -0700] rev 42219
tests: defines aliases for `hg log` calls in test-copytrace-heuristics.t
This also makes the test cases more consistent since a few had missed
the ":" in "changeset:" that the others used.
Differential Revision: https://phab.mercurial-scm.org/D6278
Georges Racinet <georges.racinet@octobus.net> [Thu, 14 Mar 2019 17:57:31 +0000] rev 42218
rust-discovery: implementing and exposing stats()
This time, it's simple enough that we can do it in all layers in
one shot.
Differential Revision: https://phab.mercurial-scm.org/D6233
Georges Racinet <georges.racinet@octobus.net> [Wed, 20 Feb 2019 09:04:39 +0100] rev 42217
rust-discovery: cpython bindings for the core logic
As previously done with the ancestors submodule, testing for
the bindings is provided from Python on a trivial case.
Differential Revision: https://phab.mercurial-scm.org/D6232
Georges Racinet <georges.racinet@octobus.net> [Tue, 19 Feb 2019 23:42:31 +0100] rev 42216
rust-discovery: starting core implementation
Once exposed to the Python side, this core object will avoid
costly roundtrips with potentially big sets of revisions.
This changeset implements the core logic of the object only, i.e.,
manipulation of the missing, common and undefined set-like revision
attributes.
Differential Revision: https://phab.mercurial-scm.org/D6231
Georges Racinet <georges.racinet@octobus.net> [Wed, 20 Feb 2019 18:33:53 +0100] rev 42215
rust-dagops: roots
Unsuprisingly, the algorithm is much easier than for heads, provided
we work on a set in the first place.
To improve the signature, a trait for set-likes object would be useful,
but that's not an immediate concern.
Differential Revision: https://phab.mercurial-scm.org/D6230
Georges Racinet <georges.racinet@octobus.net> [Tue, 19 Feb 2019 23:41:57 +0100] rev 42214
rust-dagops: range of revisions
This is a Rust implementation for what reachableroots2() does if
includepath is True.
The algorithmic details and performance notes are included in the
documentation comment.
Our main use case for now is a Rust counterpart of the partialdiscovery
object, so we don't really need bindings yet.
Differential Revision: https://phab.mercurial-scm.org/D6229
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 10:49:11 -0700] rev 42213
narrow: also warn when not deleting untracked or ignored files
Differential Revision: https://phab.mercurial-scm.org/D6265
Joerg Sonnenberger <joerg@bec.de> [Wed, 17 Apr 2019 14:37:06 +0200] rev 42212
setdiscovery: fix a few typos
Differential Revision: https://phab.mercurial-scm.org/D6263
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 14:09:18 -0700] rev 42211
copies: delete debug message about "unmatched files new in both"
Same reasoning as previous patch.
Differential Revision: https://phab.mercurial-scm.org/D6251
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 21:41:51 -0700] rev 42210
copies: delete debug message about changes since common ancestor
Same reasoning as previous patch.
Differential Revision: https://phab.mercurial-scm.org/D6250
Martin von Zweigbergk <martinvonz@google.com> [Thu, 11 Apr 2019 23:28:38 -0700] rev 42209
copies: delete debug message about search limit
I'm about to rewrite mergecopies() and this message will no longer be
emitted then. Let's remove the message now to remove a distraction
from that patch.
Differential Revision: https://phab.mercurial-scm.org/D6249
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 22:58:10 -0700] rev 42208
copies: move early return for "no copies" case a little earlier
We can return before the block that prints debug messages. That block
will not be run anyway when there are no copies.
Differential Revision: https://phab.mercurial-scm.org/D6248
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 16:46:41 -0700] rev 42207
copies: fix up "fullcopy" with missing entries from "diverge"
Similar to the previous patch, but this doesn't even affect tests. It
does affect tests if you change them to turn on debug logging. I'm
fixing it here so reviewers of the later rewrite patch can hard-code
debug logging to be on and more easily compare test results.
Differential Revision: https://phab.mercurial-scm.org/D6247
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 16:41:43 -0700] rev 42206
copies: fix up "fullcopy" with missing entries from "copy"
This is just a workaround similar to the previous one. It will make it
easier to follow later patches.
Differential Revision: https://phab.mercurial-scm.org/D6246
Martin von Zweigbergk <martinvonz@google.com> [Sun, 14 Apr 2019 00:46:25 -0700] rev 42205
merge: remove workaround for issue5020
As I explained in the previous commit, I think the filtering added
there is a better fix for the issue, so the workaround from
41f6af50c0d8 (merge: fix crash on criss cross merge with dir move and
delete (issue5020), 2017-01-31) should no longer be needed.
Differential Revision: https://phab.mercurial-scm.org/D6245
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 22:03:04 -0700] rev 42204
copies: don't include copies that are not in source in directory move
I've been working on a rewrite of mergecopies(). I compared the output
of the rewritten version with the current version. I noticed that
between FIREFOX_NIGHTLY_59_END and FIREFOX_BETA_60_BASE in the
mozilla-unified repo, there were many copies that the current version
detected that the rewritten version did not. One example was
js/src/gc/Iteration.h -> js/src/gc/PublicIterators.h. Then I realized
that js/src/gc/Iteration.h doesn't even exist in
FIREFOX_NIGHTLY_59_END.
This patch adds a filtering step for the "fullcopy" dict. It turns out
that that change also affects the test for issue5020 in
test-merge-criss-cross.t. The 'dm' action no longer happens there. At
first I thought that the test case change meant that this patch was
broken, but I think it's actually correct tha the 'dm' action should
not happen there. The result of the bid merge is still the same.
I suspect this filtering is a better solution for the issue than
41f6af50c0d8 (merge: fix crash on criss cross merge with dir move and
delete (issue5020), 2017-01-31). I also suspect that it was broken
just a few months earlier by a005c33d0bd7 (mergecopies: add logic to
process incomplete data, 2016-10-04). Note that bid merge had been
enabled for a few years at that point, since 19903277f035 (merge: use
bid merge by default (BC), 2014-10-01).
This patch is still just a workaround. It will be cleaned up soon
(with the rewrite of mergecopies()). But doing this in a separate
patch makes later patches easier to understand and gives a place to
explain why this is changing.
Differential Revision: https://phab.mercurial-scm.org/D6244
Martin von Zweigbergk <martinvonz@google.com> [Sat, 13 Apr 2019 00:24:17 -0700] rev 42203
tests: add test for issue5343 (grafting with copies)
It seems that issue5353 resulted in a lot of tests in test-graft.t,
but the bug actually reported in that issue didn't get a test
case. This patch adds one for the "move" and one for the "copy"
version of it. I also added a "copy+modify" case, to show what should
be a merge conflict. I didn't add one for the "backwards" version of
it since the comment says that that was already covered by previous
work.
The tests added by this patch show the broken behavior (the bug is
still open). I suspect the results returned from mergecopies() are not
expressive enough to fix this issue: it has a dict for copies to merge
with, but that can only give one more filename, but here we need two
(one for the path on the remote side and one for the path in the merge
base). I want to have it tested anyway since I'm about to refactor
mergecopies().
Differential Revision: https://phab.mercurial-scm.org/D6242
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Tue, 16 Apr 2019 13:12:21 -0400] rev 42202
chistedit: use context manager to set verbose ui
I'm still not exactly sure why this is necessary -- perhaps setting it
unconditionally would leak this setting in chg invocations.
Regardless, this would have looked very out of place as compared to
how this setting is done everywhere else, so at least for the sake of
style, let's be consistent with the rest of the codebase.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 16 Apr 2019 17:26:38 +0200] rev 42201
setdiscovery: stop limiting the number of local head we initially send
In our testing this limitation provides now real gain and instead triggers
pathological discovery timing for some repository with many heads.
See inline documentation for details.
Some timing below:
Mozilla try repository, (~1M revs, ~35K heads), discovery between 2 clones with
100 head missing on each side
before:
! wall 1.492111 comb 1.490000 user 1.450000 sys 0.040000 (best of 20)
! wall 1.813992 comb 1.820000 user 1.700000 sys 0.120000 (max of 20)
! wall 1.574326 comb 1.573500 user 1.522000 sys 0.051500 (avg of 20)
! wall 1.572583 comb 1.570000 user 1.520000 sys 0.050000 (median of 20)
after:
! wall 1.147834 comb 1.150000 user 1.090000 sys 0.060000 (best of 20)
! wall 1.449144 comb 1.450000 user 1.330000 sys 0.120000 (max of 20)
! wall 1.204618 comb 1.202500 user 1.146500 sys 0.056000 (avg of 20)
! wall 1.194407 comb 1.190000 user 1.140000 sys 0.050000 (median of 20)
pypy (~100 heads, 317 heads) discovery between clones with only 42 common heads
before:
! wall 0.031653 comb 0.030000 user 0.030000 sys 0.000000 (best of 25)
! wall 0.055719 comb 0.050000 user 0.040000 sys 0.010000 (max of 25)
! wall 0.038939 comb 0.039600 user 0.038400 sys 0.001200 (avg of 25)
! wall 0.038660 comb 0.050000 user 0.040000 sys 0.010000 (median of 25)
after:
! wall 0.018754 comb 0.020000 user 0.020000 sys 0.000000 (best of 49)
! wall 0.034505 comb 0.040000 user 0.030000 sys 0.010000 (max of 49)
! wall 0.019631 comb 0.019796 user 0.018367 sys 0.001429 (avg of 49)
! wall 0.019132 comb 0.020000 user 0.020000 sys 0.000000 (median of 49)
Private repository (~1M revs, ~3K heads), discovery from a strip subset, about
100 changesets to be pulled.
before:
! wall 1.837729 comb 1.840000 user 1.790000 sys 0.050000 (best of 20)
! wall 2.203468 comb 2.200000 user 2.100000 sys 0.100000 (max of 20)
! wall 2.049355 comb 2.048500 user 2.002500 sys 0.046000 (avg of 20)
! wall 2.035315 comb 2.040000 user 2.000000 sys 0.040000 (median of 20)
after:
! wall 0.136598 comb 0.130000 user 0.110000 sys 0.020000 (best of 20)
! wall 0.330519 comb 0.330000 user 0.260000 sys 0.070000 (max of 20)
! wall 0.157254 comb 0.155500 user 0.123000 sys 0.032500 (avg of 20)
! wall 0.149870 comb 0.140000 user 0.110000 sys 0.030000 (median of 20)
Same private repo, discovery between two clone with 500 different heads on each
side:
before:
! wall 2.372919 comb 2.370000 user 2.320000 sys 0.050000 (best of 20)
! wall 2.622422 comb 2.610000 user 2.510000 sys 0.100000 (max of 20)
! wall 2.450135 comb 2.450000 user 2.402000 sys 0.048000 (avg of 20)
! wall 2.443896 comb 2.450000 user 2.410000 sys 0.040000 (median of 20)
after:
! wall 0.625497 comb 0.620000 user 0.570000 sys 0.050000 (best of 20)
! wall 0.834723 comb 0.820000 user 0.730000 sys 0.090000 (max of 20)
! wall 0.675725 comb 0.675500 user 0.628000 sys 0.047500 (avg of 20)
! wall 0.671614 comb 0.680000 user 0.640000 sys 0.040000 (median of 20)
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 17:56:30 +0200] rev 42200
peer: introduce a limitedarguments attributes
When set to True, it signal that the peer cannot receive too larges arguments
and that algorithm must adapt. This should only be True for http peer that does
not support argument passed as "post".
This will be useful to unlock better discovery performance in the next
changesets.
I am using a dedicated argument because this is not really a usual
"capabilities" things. An alternative approach would be to adds a
"large-arguments" to all peer, but the http peers. That seemed a bit too hacky
to me.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 15:06:53 +0100] rev 42199
verify: also check full manifest validity during verify runs
Before this changes, `hg verify` only checked if a manifest revision existed and
referenced the proper files. However it never checked the manifest revision
content itself.
Mercurial is expecting manifest entries to be sorted and will crash otherwise.
Since `hg verify` did not attempted a full restoration of manifest entry, it
could ignore this kind of corruption.
This new check significantly increases the cost of a `hg verify` run. This
especially affects large repository not using `sparse-revlog`. For now, this is
hidden behind the `--full` experimental flag.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 01:11:09 +0200] rev 42198
verify: introduce an experimental --full flag
The flag currently has no effect, see next changeset for details. We introduce
the flag as experimental to keep the freedom of changing our mind on the final
UI.
Note: this patch highlight a small but in `hg help`. An option section is
generated even if no option are visible.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 01:12:21 +0200] rev 42197
verify: introduce a notion of "level"
Some checks are slower than others, to help the user to run the checks he needs,
we are about to introduce new flag to select faster vs deeper runs. This put
the scaffolding in place to do this.
Martin von Zweigbergk <martinvonz@google.com> [Sat, 13 Apr 2019 23:18:56 -0700] rev 42196
tests: split out separate test for issue5020
The test was added to the existing setup in 41f6af50c0d8 (merge: fix
crash on criss cross merge with dir move and delete (issue5020),
2017-01-31). I'm about to make a change that affects that test and
it's much easier to follow if the test case for issue5020 is a
separate test case. The separate test case is based on what mpm
provided in comment 12 on the issue.
`hg diff -r 41f6af50c0d8^ tests/test-merge-criss-cross.t` after this
patch is pretty small (besides the added test). It's probably easier
for reviewers to look at that than to try to understand the diff
itself (I don't understand it).
Differential Revision: https://phab.mercurial-scm.org/D6243
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 18:04:54 -0700] rev 42195
tests: avoid a rename/delete conflict when updating in test-narrow-update.t
After the upcoming rewrite of mergecopies(), this test would otherwise
(accurately) start warning about "inside/f1 was deleted and renamed".
Differential Revision: https://phab.mercurial-scm.org/D6254
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 15:28:41 -0700] rev 42194
tests: delete unused function in test-rename-merge2.t
Differential Revision: https://phab.mercurial-scm.org/D6253
Martin von Zweigbergk <martinvonz@google.com> [Sun, 14 Apr 2019 13:46:40 -0700] rev 42193
tests: make merge conflicts explicit in `hg annotate` tests
We were using `true` as merge tool. I think it makes the test easier
to understand if we make the conflicts explcit. It also papered over a
conflict that shouldn't have been a conflict (just a bug in copy
tracing). I've marked that "BROKEN".
Differential Revision: https://phab.mercurial-scm.org/D6252
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 03:05:42 +0530] rev 42192
narrow: make warning about possibly dirty files respect ui.relative-paths
Differential Revision: https://phab.mercurial-scm.org/D6264
Augie Fackler <raf@durin42.com> [Tue, 09 Jul 2019 10:07:35 -0400] rev 42191
Added signature for changeset 97ada9b8d51b
Augie Fackler <raf@durin42.com> [Tue, 09 Jul 2019 10:07:33 -0400] rev 42190
Added tag 5.0.2 for changeset 97ada9b8d51b
Augie Fackler <augie@google.com> [Mon, 08 Jul 2019 13:12:20 -0400] rev 42189
posix: always seek to EOF when opening a file in append mode
Python 3 already does this, so skip it there.
Consider the program:
#include <stdio.h>
int main() {
FILE *f = fopen("narf", "w");
fprintf(f, "narf\n");
fclose(f);
f = fopen("narf", "a");
printf("%ld\n", ftell(f));
fprintf(f, "troz\n");
printf("%ld\n", ftell(f));
return 0;
}
on macOS, FreeBSD, and Linux with glibc, this program prints
5
10
but on musl libc (Alpine Linux and probably others) this prints
0
10
By my reading of
https://pubs.opengroup.org/onlinepubs/009695399/functions/fopen.html
this is technically correct, specifically:
> Opening a file with append mode (a as the first character in the
> mode argument) shall cause all subsequent writes to the file to be
> forced to the then current end-of-file, regardless of intervening
> calls to fseek().
in other words, the file position doesn't really matter in append-mode
files, and we can't depend on it being at all meaningful unless we
perform a seek() before tell() after open(..., 'a'). Experimentally
after a .write() we can do a .tell() and it'll always be reasonable,
but I'm unclear from reading the specification if that's a smart thing
to rely on. This matches what we do on Windows and what Python 3 does
for free, so let's just be consistent. Thanks to Yuya for the idea.
Anton Shestakov <av6@dwimlabs.net> [Wed, 03 Jul 2019 10:06:39 +0800] rev 42188
move: --force flag forcibly moves, not copies
Anton Shestakov <av6@dwimlabs.net> [Wed, 03 Jul 2019 10:01:51 +0800] rev 42187
copy: correct synopsis by making SOURCE a required argument
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 02 Jul 2019 10:53:29 +0200] rev 42186
debugrevlog: fix average size computation for empty data (issue6167)
If the file has no full snapshot (eg: was always empty), `hg debugrevlog` would
fails when trying to compute their average size.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 29 Jun 2019 23:23:07 -0400] rev 42185
bookmarks: backout the attempt to fix the delete race
This backs out 044045dce23a because it broke a bunch of tests on Windows.
Yuya's theory is that we still rely on in-memory changelog data to be flushed
out of the transaction.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 22 Jun 2019 23:04:52 -0400] rev 42184
help: add a missing blank line to unhide `revlog-compression`
The help was output, but it was elided with "Enabled by default" from the
previous item.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 21 Jun 2019 03:50:40 +0200] rev 42183
bookmarks: actual fix for race condition deleting bookmark
This is a simple but efficient fix to prevent the issue tested in
`test-bookmarks-corner-case.t`. It might be worth pursuing a more generic
approach where filecache learn to depend on each other, but that would not be
suitable for stable.
The issue is complicated enough that I documented the race and its current
solution as inline comment. See this comment for details on the fix.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 21 Jun 2019 03:50:06 +0200] rev 42182
localrepo: introduce a `_refreshchangelog` method
See next changeset for usage and documentation for details.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 Jun 2019 17:26:19 +0200] rev 42181
bookmarks: actually trigger the race deleting bookmark in the test
The previous committed version of the test did not triggered the race, but this
was hidden by a strange behavior from the test runner.
So we are moving the test to a slightly more complex that actually trigger the
issue.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 Jun 2019 17:26:16 +0200] rev 42180
test: add some assert in the bookrace extension
This cannot hurt to have a bit more security in the test extension.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 Jun 2019 05:46:07 +0200] rev 42179
test: factor out the "wait" logic in bookrace
The test is currently not testing the race it is supposed to test. The
synchronisation is still valid, but needs to run at a different point.
We start with extracting the synchronisation logic for clarity.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 Jun 2019 05:45:44 +0200] rev 42178
test: remove dead code in the bookrace extension
This code is the remain of a previous version of the code. It is never ran, so
we can remove it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 Jun 2019 05:37:33 +0200] rev 42177
run-tests: stop matching line for missing feature
Before this change, the following unified test input would silently pass
$ echo foo
foo (false !)
After this change, the "foo" output is properly detected as unexpected.
The output of an handful of test had to be updated from broken conditional (that
ended up working by chance).
Yuya Nishihara <yuya@tcha.org> [Sun, 16 Jun 2019 12:31:07 +0900] rev 42176
cborutil: fix streamencode() to handle subtypes
Otherwise the template filter 'cbor' could crash because of bytes subclass:
ValueError: do not know how to encode
<class 'mercurial.encoding.safelocalstr'>
Martin von Zweigbergk <martinvonz@google.com> [Fri, 31 May 2019 22:37:14 -0700] rev 42175
bookmarks: use correct store for "ambiguity check"
I still don't quite know what the check does, but I clearly got it
wrong in 526750cdd02d (bookmarks: keep bookmarks in .hg/store if new
config set, 2019-05-15). Just compare with the strings we use in
@repofilecache and @storecache. These bugs were then copied to the
stable branch in c2b83c957621 (localrepo: grab mixedrepostorecache
class from 526750cdd02d, 2019-05-20) and 2338bdea4474 (bookmark: also
make bookmark cache depends of the changelog, 2019-05-20). As a
result, test-wireproto-exchangev2.t is flaky on both branches. This
patch fixes that.
Differential Revision: https://phab.mercurial-scm.org/D6469
Augie Fackler <raf@durin42.com> [Wed, 05 Jun 2019 10:14:19 -0400] rev 42174
Added signature for changeset c3484ddbdb96
Augie Fackler <raf@durin42.com> [Wed, 05 Jun 2019 10:14:18 -0400] rev 42173
Added tag 5.0.1 for changeset c3484ddbdb96
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 22:50:11 -0400] rev 42172
manifest: add some documentation to _lazymanifest python code
It was not particularly easy figuring out the design of this class and keeping
track of how the pieces work. So might as well write some of it down for the
next person.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 21:54:24 -0400] rev 42171
manifest: avoid corruption by dropping removed files with pure (issue5801)
Previously, removed files would simply be marked by overwriting the first byte
with NUL and dropping their entry in `self.position`. But no effort was made to
ignore them when compacting the dictionary into text form. This allowed them to
slip into the manifest revision, since the code seems to be trying to minimize
the string operations by copying as large a chunk as possible. As part of this,
compact() walks the existing text based on entries in the `positions` list, and
consumed everything up to the next position entry. This typically resulted in
a ValueError complaining about unsorted manifest entries.
Sometimes it seems that files do get dropped in large repos- it seems to
correspond to there being a new entry that would take the same slot. A much
more trivial problem is that if the only changes were removals, `_compact()`
didn't even run because `__delitem__` doesn't add anything to `self.extradata`.
Now there's an explicit variable to flag this, both to allow `_compact()` to
run, and to avoid searching the manifest in cases where there are no removals.
In practice, this behavior was mostly obscured by the check in fastdelta() which
takes a different path that explicitly drops removed files if there are fewer
than 1000 changes. However, timeless has a repo where after rebasing tens of
commits, a totally different path[1] is taken that bypasses the change count
check and hits this problem.
[1] https://www.mercurial-scm.org/repo/hg/file/2338bdea4474/mercurial/manifest.py#l1511
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 21:39:19 -0400] rev 42170
tests: demonstrate broken manifest generation with the pure module
This will be fixed next. But I don't fully understand how 'b.txt' is actually
removed properly in the second test, given what's broken. Also, I'm not sure
why 'bb.txt' is flagged as not being in the manifest, when it clearly appears
to be.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:08:28 +0200] rev 42169
bookmark: also make bookmark cache depends of the changelog
Since the changelog is also used during the parsing of bookmark data, it should
be listed as a file cache dependency. This fix the race condition we just
introduced a test for.
This is a simple fix that might lead bookmark data to be invalidated more often
than necessary. We could have more complicated code to deal with this race in a
more "optimal" way. I feel it would be unsuitable for stable.
In addition, the performance impact of this is probably minimal and I don't
foresee the more advanced fix to actually be necessary.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:08:17 +0200] rev 42168
localrepo: grab mixedrepostorecache class from 526750cdd02d
On default, Martin von Zweigbergk <martinvonz@google.com> introduced a more
advance filecache decorator. I need this decorator to fix a bug on stable. So I
am grafting the relevant part of 526750cdd02d.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:06:53 +0200] rev 42167
bookmark: add a test for a race condition on push
Bookmark pointing to unknown nodes are ignored. Later these ignored bookmarks
are dropped when writing the file back on disk. On paper, this behavior should
be fine, but with the current implementation, it can lead to unexpected
bookmark deletions.
In theory, to make sure writer as a consistent view, taking the lock also
invalidate bookmark data we already loaded into memory. However this
invalidation is incomplete. The data are stored in a `filecache` that preserve
them if the bookmark related file are untouched. In practice, the bookmark data
in memory also depends of the changelog content, because of the step checking
if the bookmarks refers to a node known to the changelog. So if the bookmark
data were loaded from an up to date bookmark file but filtered with an outdated
changelog file this go undetected.
This condition is fairly specific, but can occurs very often in practice. We
introduce a test recreating the situation. The test comes in an independant
changeset to show it actually reproduce the situation. The fix will come soon
after.
A large share of the initial investigation of this race condition was made by
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 07:11:16 +0200] rev 42166
test: properly gate a zstd section
This part of the test can't run if zstd is not available. This was caught by
--pure test (who don't support zstd).
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 07:11:06 +0200] rev 42165
test: update test for expected test output
In 1fac9b931d46 as new test session was introduced. It did not take in account
some part that only ran for pure.
The test is now fixed.
Augie Fackler <augie@google.com> [Wed, 08 May 2019 16:09:50 -0400] rev 42164
sslutil: fsencode path returned by certifi (issue6132)
By inspection, this is the only codepath that could be returning a
string instead of a bytes on Python 3.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 06 May 2019 22:10:34 -0400] rev 42163
commit: allow --interactive to work again when naming a directory (issue6131)
Yuya Nishihara <yuya@tcha.org> [Fri, 03 May 2019 20:06:03 +0900] rev 42162
parser: fix crash by parsing "()" in keyword argument position
A tree node can be either None or a tuple because x=("group", None) is
reduced to x[1].
Augie Fackler <raf@durin42.com> [Wed, 01 May 2019 14:27:19 -0400] rev 42161
Added signature for changeset 07e479ef7c96
Augie Fackler <raf@durin42.com> [Wed, 01 May 2019 14:27:17 -0400] rev 42160
Added tag 5.0 for changeset 07e479ef7c96
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 25 Apr 2019 19:17:02 +0200] rev 42159
hghave: deal with "rc" release
Without this change, 5.0rc0 is not recognised as 5.0
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 17 Apr 2019 15:06:41 +0300] rev 42158
narrow: send specs as bundle2 data instead of param (issue5952) (issue6019)
Before this patch, when ACL is involved, narrowspecs are send as bundle2
parameter for narrow:spec bundle2 part. The limitation of bundle2 parts are they
cannot send data larger than 255 bytes. Includes and excludes in narrow are not
limited by size and they can grow over 255 bytes.
This patch introduces a new mandatory bundle2 part and send narrowspecs as data
of that. The new bundle2 part is introduced to keep things cleaner and easy to
distinguish related to backward compatibility.
The part is mandatory because without server's narrowspec, the local ACL narrow
repo won't work.
This patch makes clients compatible with servers which have older versions.
However I left a comment that we should drop the other bundle2 part soon as
that's broken and people should not rely on that.
I named the new bundle2 part 'Narrow:responsespec' because:
1) Capital 'N' to make it mandatory
2) 'Narrow:spec' cannot be used because bundle2 enforces that there should not
be two different parts which resolve to same name when lowercased.
3) reponsespec clears that they are specs which are send as reponse by the
server
While I was here, I renamed `narrowhgacl` section to `narrowacl` as suggested by
idlsoft@ and martinvonz@.
Differential Revision: https://phab.mercurial-scm.org/D6310
Matt Harbison <matt_harbison@yahoo.com> [Fri, 26 Apr 2019 23:52:49 -0400] rev 42157
inno: bump keyring to 18.0.1 to avoid AttributeError (issue6043)
The error seems to be harmless, because it happens after closing the connection.
For whatever reason, this isn't bundled with the Wix installer.
https://github.com/jaraco/keyring/issues/386
https://bitbucket.org/Mekk/mercurial_keyring/issues/63/attributeerror-during-process-finish-with
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Apr 2019 19:42:43 +0300] rev 42156
context: check file exists before getting data from _wrappedctx
overlayworkingctx class is used to do in-memory merging. The data() function of
that class has logic to look for data() in the wrappedctx if the file data in
cache is empty and if the file is dirty. This assumes that if a file is dirty
and cache has empty data for it, it will exists in the _wrappedctx.
However this assumption can be False in case when we are merging a file which is
empty in destination. In these cases, the backup file 'foo.orig' created by our
internal merge algorithms will be empty, however it won't be present in
_wrappedctx. This case will lead us to error like the one this patch is fixing.
Let's only fallback to getting data from wrappedctx if cache has 'None' as data.
Differential Revision: https://phab.mercurial-scm.org/D6308
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Apr 2019 19:28:46 +0300] rev 42155
tests: show IMM is broken when merging file empty in destination
When we are doing in-memory merging, and we are merging a file which is empty in
merge destination, it leads to error 'abort: xxx not found in manifest'.
Next patch will fix this error.
Differential Revision: https://phab.mercurial-scm.org/D6307
Antonio Muci <a.mux@inwind.it> [Fri, 19 Apr 2019 02:24:25 +0200] rev 42154
buildrpm: bump bundled Python version to 2.7.16 when building for centos{5,6}
When building rpm packages for centos 5 and 6, we bundle a mercurial-specific
version of Python 2.7 in /opt/python-hg.
This change is analogous to 5e947367606c, and bumps the embedded Python version
from 2.7.14 (released in 2017) to 2.7.16 (latest as of today).
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 08:57:01 -0700] rev 42153
setup: tweak error message for Python 3
We now have beta support for Python 3. In my opinion, it isn't
yet stable enough to allow `pip install Mercurial` to work with
Python 3 out of the box: we don't want people accidentally using
Mercurial with Python 3 just yet.
But I do think we should be more friendly about informing people
of their options.
This commit tweaks the error message that users see when running
setup.py with Python 3. We instruct them about the current level
of Python 3 support, point them at the wiki for more info, and
give them instructions on how to bypass the check.
As part of this, I also changed which version value is printed,
as we were printing a named tuple before.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 07:21:08 -0700] rev 42152
setup: remove set and dict comprehensions
Yuya observed in a recent review that it is worthwhile to keep
setup.py parseable with Python 2.6 so a useful error message is
seen when attempting to run with Python 2.6.
This commit removes a set and dict comprehension so setup.py
is parseable with Python 2.6.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 19 Apr 2019 23:13:28 +0300] rev 42151
branchcache: don't verify all nodes while writing
nodes are verified either when they are added or used. In case of commits. we
will load the whole branchmap, only verify nodes for the branch on which we are
committing and then we write.
However before this patch, writing the branchmap was validating all the nodes
whereas it should not. This patch fixes that.
Differential Revision: https://phab.mercurial-scm.org/D6290
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 20 Apr 2019 07:29:07 -0700] rev 42150
setup: properly package distutils in py2exe virtualenv builds
Our in-repo py2exe packaging code uses virtualenvs for managing
dependencies. An advantage of this is that packaging is more
deterministic and reproducible. Without virtualenvs, we need to
install packages in the system Python install. Packages installed
by other consumers of the system Python could leak into the Mercurial
package.
A regression from this change was that py2exe packages contained
the virtualenv's hacked distutils modules instead of the original
distutils modules. (virtualenv installs a hacked distutils module
because distutils uses relative path lookups that fail when running
from a virtualenv.)
This commit introduces a workaround so py2exe packaging uses the
original distutils modules when running from a virtualenv.
With this change, `import distutils` no longer fails from py2exe
builds produced from a virtualenv. This fixes the regression.
Furthermore, we now include all distutils modules. Before, py2exe's
module finding would only find modules there were explicitly
referenced in code. So, we now package a complete copy of distutils
instead of a partial one. This is even better than before.
# no-check-commit foo_bar function name
Augie Fackler <augie@google.com> [Wed, 17 Apr 2019 14:10:02 -0400] rev 42149
merge: forgot to pull before release
These two changes should be part of 5.0, but we are fine
leaving them out of the RC.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 16 Apr 2019 15:50:20 +0200] rev 42148
debugdiscovery: include the number of heads in all sets
We already displayed information about heads of the common set that are either
local or remote heads. We now also do so for heads of the common set that are
both local and remote heads too. This is useful because various step in the
set discovery algorithm have head specific optimizations.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 00:37:00 +0200] rev 42147
recover: add a --[no-]verify flag
For trivial cases, the cost of the verify run after `hg recover` is getting in
the way. In addition for very large repositories, the cost is simply too high
to be paid, making `hg recover` an unusable commands.
We introduce a --verify flag, set by default. If is automatically associated
with a --no-verify flag that one can use to skip the verify step.
We might consider changing the default behavior in the future. However this is
out of scope for this series.
Augie Fackler <raf@durin42.com> [Wed, 17 Apr 2019 13:56:10 -0400] rev 42146
Added signature for changeset 4a8d9ed86475
Augie Fackler <raf@durin42.com> [Wed, 17 Apr 2019 13:56:08 -0400] rev 42145
Added tag 5.0rc0 for changeset 4a8d9ed86475
Augie Fackler <augie@google.com> [Wed, 17 Apr 2019 13:41:18 -0400] rev 42144
merge: default into stable for release candidate
Joerg Sonnenberger <joerg@bec.de> [Tue, 02 Apr 2019 19:48:31 +0200] rev 42143
bundle2: handle compression in _forwardchunks
_forwardchunks is used to compensate for getbundle protocol deficits.
Since it transparently decodes the payload, it also needs to remove the
corresponding compression parameter in case the server decides to send
one. This the wire protocol part of issue 5990.
Differential Revision: https://phab.mercurial-scm.org/D6182
Martin von Zweigbergk <martinvonz@google.com> [Wed, 27 Dec 2017 22:05:20 -0800] rev 42142
changelog: parse copy metadata if available in extras
This lets read back the copy metadata we just started writing. There
are still many places left to teach about getting the copy information
from the changeset, but we have enough ({file_copies}, specifically)
that we can add it now and have some test coverage of it.
Differential Revision: https://phab.mercurial-scm.org/D6186
Martin von Zweigbergk <martinvonz@google.com> [Wed, 27 Dec 2017 19:49:36 -0800] rev 42141
copies: add config option for writing copy metadata to file and/or changset
This introduces a config option that lets you choose to write copy
metadata to the changeset extras instead of to filelog. There's also
an option to write it to both places. I imagine that may possibly be
useful when transitioning an existing repo.
The copy metadata is stored as two fields in extras: one for copies
since p1 and one for copies since p2.
I may need to add more information later in order to make copy tracing
faster. Specifically, I'm thinking out recording which files were
added or removed so that copies._chaincopies() doesn't have to look at
the manifest for that. But that would just be an optimization and that
can be added once we know if it's necessary.
I have also considered saving space by using replacing the destination
file path by an index into the "files" list, but that can also be
changed later (but before the feature is ready to release).
Differential Revision: https://phab.mercurial-scm.org/D6183
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 04 Apr 2019 13:46:49 +0200] rev 42140
revsetbenchmark: add some simpler revset for heads and roots
This revset might leverage compiled code in the future so lets start to track
them.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 12 Apr 2019 16:25:59 +0200] rev 42139
repoview: flag `server.view` as experimental
Ideally, the non-experimental version of `experimental.extra-filter-revs` will
cover the use case for `server.view=immutable` well enough than having to have
this dedicated configuration. Since `server.view` is not part of any release, I
would prefer to have it marked as experimental to avoid having it to support it
for ever.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 12 Apr 2019 15:41:32 +0200] rev 42138
repoview: move subsettable in a dedicated module
The dictionary got moved in `branchmap` to avoid import cycle. However, we are
about to needs it in repoview too. So we introduce a now module to define that
that mapping.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 01 Feb 2019 15:51:02 +0100] rev 42137
upgrade: support upgrade to/from zstd storage (issue6088)
Now that we have an official config option for a shiny format improvement, we
better make it simple to migrate to/from it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 27 Mar 2019 18:27:03 +0100] rev 42136
compression: introduce an official `zstd-revlog` requirement
This requirement supersedes `exp-compression-zstd`. However, we keep support for
the old requirement.
Strictly speaking, we do not need to add a new requirement, there are no logic
change making "new" repo incompatible with mercurial client using a version
that support `exp-compression-zstd`.
The choice to introduce a new requirement is motivated by the following:
* The previous requirement was explicitly "experimental". Using it by default
could confuse users.
* adding support for a hypothetical third compression engine will requires new
code, and will comes with its own requirement tool.
* We won't use it as the default for a while since I do not think we support
zstd on all platform. I can imagine we'll gain another (unrelated but on my
default) requirement by the time we turn this zstd by default.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 16 Apr 2019 15:10:16 +0200] rev 42135
compression: only declare revlog support for available engine
Even if we know that an engine support revlog compression, the Mercurial process
still won't support it if the compression engine is not available.
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 15 Apr 2019 19:21:41 +0300] rev 42134
branchcache: lazily validate nodes in iteritems()
This saves ~0.30 sec on creating a new branch on our internal repo.
Differential Revision: https://phab.mercurial-scm.org/D6236
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Apr 2019 15:01:33 +0300] rev 42133
branchcache: only iterate over branches which needs to be verified
Otherwise we loop over all the branches and call _verifybranch() even if not
required.
Differential Revision: https://phab.mercurial-scm.org/D6240
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Apr 2019 14:48:48 +0300] rev 42132
branchcache: fix the docstring of _verifybranch()
Initially the function was designed to support verifying all branches but later
I decided to have a separate function. I forget to remove the doc related to
that. Thanks to Yuya for spotting this in review.
Differential Revision: https://phab.mercurial-scm.org/D6239
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Apr 2019 14:39:14 +0300] rev 42131
branchcache: don't verify while creating a copy
The copy will not be mark as verified, so there is no need to verify nodes.
Thanks to Yuya who spotted this while reviewing.
Differential Revision: https://phab.mercurial-scm.org/D6238
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 05 Apr 2019 14:35:33 +0200] rev 42130
pull: improved message issued in case of failed update
When running `hg pull --update`, the update step may fail. Nothing in the error
message help to understand the abort is related to the secondary step (update)
instead of the primary step (pull).
We now add some information to the error message to clarify it comes from the
update part. It is useful in various situation (uncommitted changes blocking the
update, update to hidden destination, etc...)
The pull output is updated from:
$ hg pull ../repo-Bob --rev 956063ac4557 --update
pulling from ../repo-Bob
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 0 changes to 2 files (+1 heads)
(2 other changesets obsolete on arrival)
abort: filtered revision '6'!
to:
$ hg pull ../repo-Bob --rev 956063ac4557 --update
pulling from ../repo-Bob
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 0 changes to 2 files (+1 heads)
(2 other changesets obsolete on arrival)
abort: cannot update to target: filtered revision '6'!
(I am not sure why the actual error, "filtered revision '6'", is not using the
more modern format mentioning the obsolescence fate of '6')
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 05 Apr 2019 15:56:05 +0200] rev 42129
pull: deal with locally filtered changeset passed into --rev
Nowadays, it is possible to explicitly pull a remote revision that end up being
hidden locally (eg: obsoleted locally). However before this patch, some
internal processing where crashing trying to resolve a filtered revision.
Without this patches, the pull output result a confusing output:
$ hg pull ../repo-Bob --rev 956063ac4557
pulling from ../repo-Bob
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 0 changes to 2 files (+1 heads)
(2 other changesets obsolete on arrival)
abort: 00changelog.i@956063ac4557828781733b2d5677a351ce856f59: filtered node!
Rodrigo Damazio Bovendorp <rdamazio@google.com> [Mon, 15 Apr 2019 22:13:11 -0700] rev 42128
absorb: aborting if another operation is in progress
This increases safety of using absorb by both aborting when another operation
is in progress (since the absorption could confuse any other command a lot)
and holding the locks throughout the reading of the working directory (for
which changes to absorb) and the reading of the repo (for which changes to
absorb into).
Differential Revision: https://phab.mercurial-scm.org/D6237
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 04 Apr 2019 13:58:49 +0200] rev 42127
repoview: keep the branchmap cache for the `served.hidden` view warm
For the same reason we want to keep the cache for the `served` view up to date,
we want to also keep the `served.hidden` view up to date. If some processes with
a readonly access to the repo needs to access it, we better have the cache warm
to avoid computing the same data over and over and over.
In most case (no secret changesets), the "served" and "served.hidden" set will
be identical and no cache will actually have to be updated.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 21 May 2018 17:28:35 +0200] rev 42126
repoview: introduce a filter for serving hidden changesets
There are multiple usecase for being able to explicitly view or pull obsolete
from a server. We need to be able to do so without exposing the secret
changesets. We introduces a dedicated repository "view" to do so. Way to expose
this "view" to the user will come later.
To keep a behavior consistent with expected client/server behavior, the general
idea is for the obsolete access to be explicitly requested by the code
generating the request. In addition, the will be server side configuration to
restrict the access to this feature.