Matt Harbison <matt_harbison@yahoo.com> [Mon, 09 Aug 2021 19:24:46 -0400] rev 47811
contrib: log the command and args for every process installing windows deps
This is a little noisier, but makes it simple to debug when things fail.
Differential Revision: https://phab.mercurial-scm.org/D11272
Raphaël Gomès <rgomes@octobus.net> [Fri, 23 Jul 2021 10:47:12 +0200] rev 47810
windows-ci: run Windows CI automatically alongside the others
This will enable us to make Windows Python 3 a first-class citizen for the next
6.0 cycle. We will probably get some flaky tests and we're missing others that
are skipped, but we'll turn them on it future patches.
Differential Revision: https://phab.mercurial-scm.org/D11256
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 05 Aug 2021 12:53:44 +0200] rev 47809
vfs: always use / as file separator (
issue6546)
Various part of vfs already enforce `/` usage and using `\` confuse the encoded
vfs. So we simply use `/` all the time.
Differential Revision: https://phab.mercurial-scm.org/D11260
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 05 Aug 2021 12:53:36 +0200] rev 47808
subrepo: compare normalised vfs path
Otherwise the realpath call can turn `/` into `\` on windows confusing the
check.
(We probably needs this in more location)
Differential Revision: https://phab.mercurial-scm.org/D11259
Raphaël Gomès <rgomes@octobus.net> [Thu, 05 Aug 2021 18:25:35 +0200] rev 47807
pager: account for flakiness in Windows output
This test case is cursed and probably not worth losing more time over. This
makes apparent what the intended behavior is while still removing the flakiness
from the CI.
Differential Revision: https://phab.mercurial-scm.org/D11257
Raphaël Gomès <rgomes@octobus.net> [Fri, 23 Jul 2021 10:45:08 +0200] rev 47806
windows-ci: clean up the Heptapod CI file now that the baseline is solid
Enough work has been done one the CI side, this now works with little effort
on our side. The next patch will remove the manual switch.
Differential Revision: https://phab.mercurial-scm.org/D11254
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Aug 2021 21:22:02 +0200] rev 47805
test-nointerrupt: make "sure" the handler "might" trigger (
issue6558)
We are sure that the signal got sent in the right time frame, however, we still
have race, so either the code is actually buggy or we need some security to make
sure the signal get processed.
We might be affected by https://bugs.python.org/
issue43406 ?
Differential Revision: https://phab.mercurial-scm.org/D11251
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Aug 2021 19:26:26 +0200] rev 47804
testing: make sure write_file is "atomic"
This make sure viewer cannot see the new file with partial content.
This was likely the cause of some flakiness in `test-nointerrupt.t`
Differential Revision: https://phab.mercurial-scm.org/D11250
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 04 Aug 2021 19:45:13 +0200] rev 47803
test: disable test-subrepo-git.t in python2 + chg
I am a couple of days in try to debug that at it seems minor enough with enough
other priority to simply disable it for now.
Differential Revision: https://phab.mercurial-scm.org/D11253
Georges Racinet <georges.racinet@octobus.net> [Tue, 20 Jul 2021 17:20:19 +0200] rev 47802
hgwebdir: avoid systematic full garbage collection
Forcing a systematic full garbage collection upon each request
can serioulsy harm performance. This is reported as
https://bz.mercurial-scm.org/show_bug.cgi?id=6075
With this change we're performing the full collection according
to a new setting, `experimental.web.full-garbage-collection-rate`.
The default value is 1, which doesn't change the behavior and will
allow us to test on real use cases. If the value is 0, no full garbage
collection occurs.
Regardless of the value of the setting, a partial garbage collection
still occurs upon each request (not attempting to collect objects from
the oldest generation). This should be enough to take care of
reference cycles that have been created by the last request
(assessment of this requires changing the setting, not to be 1).
In my experience chasing memory leaks in Mercurial servers,
the full collection never reclaimed any memory, but this is with
Python 3 and biased towards small repositories.
On the other hand, as explained in the Python developer docs [1],
frequent full collections are very harmful in terms of performance if
lots of objects survive the collection, and hence stay in the
oldest generation. Note that `gc.collect()` is indeed trying to
collect the oldest generation [2]. This happens usually in two cases:
- unwanted lingering objects (i.e., an actual memory leak that
the GC cannot do anything about). Sadly, we have lots of those
these days.
- desireable long-term objects, typically in caches (not inner caches
carried by repositories, which should be collected with them). This
is a subject of interest for the Heptapod project.
In short, the flat rate that this change still permits is
probably a bad idea in most cases, and the default value can
be tweaked later on (or even be set to 0) according to experiments
in the wild.
The test is inspired from test-hgwebdir-paths.py
[1] https://devguide.python.org/garbage_collector/#collecting-the-oldest-generation
[2] https://docs.python.org/3/library/gc.html#gc.collect
Differential Revision: https://phab.mercurial-scm.org/D11204
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 03 Aug 2021 18:29:31 +0200] rev 47801
check-module-imports: ignore non-stdlib module installed by distribution
Previously, the check script would detect breezy as part of the stdlib if
installed using the debian package manager.
This silence the following complains:
tests/test-convert-bzr.t:117: imports not lexically sorted: breezy.bzr.bzrdir < sys
tests/test-convert-bzr.t:117: stdlib import "breezy.bzr.bzrdir" follows local import: breezy
tests/test-convert-bzr-ghosts.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys
tests/test-convert-bzr-ghosts.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy
tests/test-convert-bzr-treeroot.t:7: imports not lexically sorted: breezy.bzr.bzrdir < sys
tests/test-convert-bzr-treeroot.t:7: stdlib import "breezy.bzr.bzrdir" follows local import: breezy
Differential Revision: https://phab.mercurial-scm.org/D11249
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Mon, 02 Aug 2021 08:06:27 -0400] rev 47800
remotefilelog: fix what looks like a wrong refactoring
when various store functions started returning a revlog type as the
first element of the tuple.
Differential Revision: https://phab.mercurial-scm.org/D11243
Georges Racinet <georges.racinet@octobus.net> [Sun, 01 Aug 2021 14:39:38 +0200] rev 47799
rust-nodemap: falling back to C impl as mitigation
This is a mitigation for https://bz.mercurial-scm.org/show_bug.cgi?id=6554
We see sometimes almost all data except the most recent revisions
removed from the persistent nodemap, but we don't know how to
reproduce yet.
This has sadly repercussions beyond just needing to reconstruct the
persistent nodemap: for instance, this automatically filters out
all bookmarks pointing to revisions that the nodemap cannot resolve.
If such filtering happens in a transaction, the update of the
bookmarks file that happens at the end of transaction loses all
bookmarks that have been affected. There may be similar consequences
for other data.
So this is a data loss, something that we have to prevent as soon as
possible.
As a mitigation measure, we will now fallback to the C implementation
in case nodemap lookups failed. This will add some latency, e.g., in
discovery, yet less than disabling the persistent nodemap entirely.
We considered implementing the fallback directly on the Python
side, but `revlog.get_rev()` is not systematically used, there are
also several direct calls to the index method (`self.index.rev()` for
a `revlog` instance). It is therefore more direct to implement the
mitigation in the rust-cpython wrapper.
Differential Revision: https://phab.mercurial-scm.org/D11238
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:45:07 +0300] rev 47798
obsolete: disable other evolution config options if createmarkers is off
We used to raise an abort in this case, but recent changes to local clone
command (
377d8fc20e34) resulted in destrepo both caring about
experimental.evolution config options and not initializing extensions.
So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local
clone stopped working after
377d8fc20e34 because evolve sets
experimental.evolution=all, but only on srcrepo, for destrepo the extension is
not initialized. It's possible to make local cloning work by initializing
extensions for destrepo in some cases, but in other cases (e.g. allowdivergence
in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail.
In a discussion with Pierre-Yves David it was decided to simply force other
evolution options to be false if createmarkers is not enabled.
Differential Revision: https://phab.mercurial-scm.org/D11223
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:47:21 +0300] rev 47797
fix: use obsolete.isenabled() to check for experimental.allowdivergence
Now that obsolete.isenabled() can also check if divergence is allowed, let's
use it for consistency. Other experimental.evolution options are already
checked via this function.
Differential Revision: https://phab.mercurial-scm.org/D11222
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:45:41 +0300] rev 47796
rebase: use obsolete.isenabled() to check for experimental.allowdivergence
Now that obsolete.isenabled() can also check if divergence is allowed, let's
use it for consistency. Other experimental.evolution options are already
checked via this function.
Differential Revision: https://phab.mercurial-scm.org/D11221
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:45:07 +0300] rev 47795
obsolete: disable other evolution config options if createmarkers is off
We used to raise an abort in this case, but recent changes to local clone
command (
377d8fc20e34) resulted in destrepo both caring about
experimental.evolution config options and not initializing extensions.
So imagine if you had evolve and allowdivergence enabled in your ~/.hgrc. Local
clone stopped working after
377d8fc20e34 because evolve sets
experimental.evolution=all, but only on srcrepo, for destrepo the extension is
not initialized. It's possible to make local cloning work by initializing
extensions for destrepo in some cases, but in other cases (e.g. allowdivergence
in ~/.hgrc, evolve extension in original-repo/.hg/hgrc) it would still fail.
In a discussion with Pierre-Yves David it was decided to simply force other
evolution options to be false if createmarkers is not enabled.
Differential Revision: https://phab.mercurial-scm.org/D11223
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:47:21 +0300] rev 47794
fix: use obsolete.isenabled() to check for experimental.allowdivergence
Now that obsolete.isenabled() can also check if divergence is allowed, let's
use it for consistency. Other experimental.evolution options are already
checked via this function.
Differential Revision: https://phab.mercurial-scm.org/D11222
Anton Shestakov <av6@dwimlabs.net> [Wed, 28 Jul 2021 13:45:41 +0300] rev 47793
rebase: use obsolete.isenabled() to check for experimental.allowdivergence
Now that obsolete.isenabled() can also check if divergence is allowed, let's
use it for consistency. Other experimental.evolution options are already
checked via this function.
Differential Revision: https://phab.mercurial-scm.org/D11221
Raphaël Gomès <rgomes@octobus.net> [Mon, 02 Aug 2021 16:23:45 +0200] rev 47792
heptapod-ci: enable pytype testing for all pipelines
Now that pytype agrees with the codebase, let's run it by default to catch
regressions.
Differential Revision: https://phab.mercurial-scm.org/D11242
Raphaël Gomès <rgomes@octobus.net> [Mon, 02 Aug 2021 16:21:54 +0200] rev 47791
pytype: add assertions to explain revlogv2 invariants to Pytype
Differential Revision: https://phab.mercurial-scm.org/D11241
Matt Harbison <matt_harbison@yahoo.com> [Fri, 30 Jul 2021 00:11:56 -0400] rev 47790
typing: add several assertions to dirstatemap to appease pytype
(grafted from default to stable)
I think it's been mentioned in IRC that these can't be None in this case. This
fixes:
File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 213, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands]
No attribute '__and__' on None or '__rand__' on int
Called from (traceback):
line 290, in reset_state
File "/mnt/c/Users/Matt/hg/mercurial/dirstatemap.py", line 214, in addfile: unsupported operand type(s) for &: 'None' and 'int' [unsupported-operands]
No attribute '__and__' on None or '__rand__' on int
Called from (traceback):
line 290, in reset_state
Differential Revision: https://phab.mercurial-scm.org/D11235
Matt Harbison <matt_harbison@yahoo.com> [Mon, 02 Aug 2021 10:51:19 -0400] rev 47789
windows: avoid a bytes vs unicode crash reading passwords on py2
This broke in
5b3513177f2b. Specifically, after typing in the password on py2,
it would crash with:
TypeError: putwch() argument 1 must be cannot convert raw buffers, not str
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com> [Sun, 01 Aug 2021 10:54:03 -0400] rev 47788
streamclone: ensure the server sends the right amount of data
Otherwise, the client would fail with some confusing error. I have
seen an error which I think is this, perhaps due to a concurrent
revlog split, which streamclones do not handle correctly and would
result in a short read of the index of the revlog being split.
Differential Revision: https://phab.mercurial-scm.org/D11236
Augie Fackler <augie@google.com> [Thu, 29 Jul 2021 16:23:45 -0400] rev 47787
rewriteutil: fix crash when a rewritten message references f{6,64}
Without this, the rewriteutil logic thinks it's found a reference to the wdir
pseudo-revision, and then tries to look it up and rewrite it. Stop it from
doing that.
Amusingly, I had trouble working with this changeset when I didn't
describe the defect above using a regular expression, because it would
trigger the bug in my installed version of hg.
Differential Revision: https://phab.mercurial-scm.org/D11232
Augie Fackler <augie@google.com> [Thu, 29 Jul 2021 16:18:35 -0400] rev 47786
tests: add explicit coverage for update_hash_refs from rewriteutil
I couldn't find any evidence this is covered by tests in core (but there's a
good chance I missed it). We've seen a cute bug in that code, but first
let's just cover the cases that work correctly.
Differential Revision: https://phab.mercurial-scm.org/D11231
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 27 Jul 2021 19:36:20 +0200] rev 47785
store: document the decoding discrepancy in store.py
This will help future people that might be looking into this.
Differential Revision: https://phab.mercurial-scm.org/D11220
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 27 Jul 2021 19:19:00 +0200] rev 47784
clone: add a file with special character while testing uncompressed
This will make sure we currently do not have any actual impact from
issue6548.
Differential Revision: https://phab.mercurial-scm.org/D11219