Raphaël Gomès <rgomes@octobus.net> [Wed, 04 Jan 2023 12:06:07 +0100] rev 49625
relnotes: add 6.3.2
Raphaël Gomès <rgomes@octobus.net> [Wed, 04 Jan 2023 12:07:55 +0100] rev 49624
Added signature for changeset 59466b13a3ae
Raphaël Gomès <rgomes@octobus.net> [Wed, 04 Jan 2023 12:07:54 +0100] rev 49623
Added tag 6.3.2 for changeset 59466b13a3ae
Martin von Zweigbergk <martinvonz@google.com> [Tue, 03 Jan 2023 13:38:56 -0800] rev 49622
filemerge: fix crash when using filesets in [partial-merge-tools]
Without this patch, you'd get `mercurial.error.ProgrammingError: fileset
expression with no context`.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 06 Dec 2022 17:12:59 -0500] rev 49621
packaging: add dependencies to the PyOxidizer build on macOS
Otherwise, we get a bunch of test failures for missing things like pygments, or
tests skipped entirely. The input file is a copy/paste from the equivalent
Windows file, but with dulwich, pygit2, and pytest-vcr commented out because
the build process errors out with them, flagging them as incompatible with
loading from memory. I have no idea if that's actually true or not, because
I've noticed that if I don't `make clean` after every build, the next build
flags the watchman stuff as incompatible with loading from memory.
The remaining failures are:
Failed test-alias.t: output changed
Failed test-basic.t: output changed
Failed test-check-help.t: output changed
Failed test-commit-interactive.t: output changed
Failed test-extension.t: output changed
Failed test-help.t: output changed
Failed test-i18n.t: output changed
Failed test-log.t: output changed
Failed test-qrecord.t: output changed
Failed test-share-safe.t: output changed
Most of the issues seem related to loading help for disabled extensions from
`hgext.__index__`, namely the full extension help being unavailable, not being
able to resolve what commands are provided by what extension, and not having the
command level help available.
test-log.t, test-commit-interactive.t, and test-i18n.t look like i18n (or lack
thereof) issues.
test-basic.t is just odd:
@@ -55,7 +55,7 @@
On Python 3, stdio may be None:
$ hg debuguiprompt --config ui.interactive=true 0<&-
- abort: Bad file descriptor (no-rhg !)
+ abort: response expected
abort: response expected (rhg !)
[255]
$ hg version -q 0<&-
Matt Harbison <matt_harbison@yahoo.com> [Tue, 06 Dec 2022 15:32:11 -0500] rev 49620
tests: conditionalize path output for in-memory pyoxidizer resources
Matt Harbison <matt_harbison@yahoo.com> [Tue, 06 Dec 2022 13:03:36 -0500] rev 49619
tests: conditionalize test output for in-filesystem pyoxidizer resources
The in-memory pyoxidizer builds apparently behave as expected.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 06 Dec 2022 13:02:54 -0500] rev 49618
hghave: add predicates for embedded and filesystem pyoxidizer resources
There are a handful of tests with different output between the two flavors of
pyoxidizer builds (like the location of the modules and templates), and a few
others that avoid `known-bad-output` cases with the embedded resources that
shouldn't cause the tests to fail.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Dec 2022 11:49:56 -0500] rev 49617
run-tests: support --pyoxidized on macOS
We should definitely be able to test this before releasing it with an installer.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Dec 2022 11:46:00 -0500] rev 49616
make: add a target for building pyoxidizer tests on macOS
The resources seem to be embedded inside the binary, but for some reasons they
aren't read there. And since they are embedded, they aren't staged by the build
in the `lib` directory like on Windows. So copy them from the repo. We can
figure out what's going wrong later.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 02 Jan 2023 11:38:20 -0500] rev 49615
copyright: update to 2023
Anton Shestakov <av6@dwimlabs.net> [Mon, 02 Jan 2023 15:27:55 +0400] rev 49614
bisect: correct message about aborting an in-progress bisect (issue6527)
By using a custom cmdhint message here we're avoiding automatic hint generation
in _statecheck.hint(), which would suggest 'hg bisect --continue' and 'hg
bisect --abort' (neither of which is a valid option).
This patch is only fixing the message about in-progress bisect, it doesn't
modify the unfinished state checking logic.
Anton Shestakov <av6@dwimlabs.net> [Mon, 12 Dec 2022 17:49:48 +0400] rev 49613
tests: document what the other currently enabled pylint check ensures
Anton Shestakov <av6@dwimlabs.net> [Mon, 12 Dec 2022 17:42:30 +0400] rev 49612
hghave: detect newer pylint
Older versions (e.g. 2.7.2) say: "Usage: pylint [options]"
Newer versions (e.g. 2.15.5) say: "usage: pylint [options]"
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Nov 2022 00:18:41 +0100] rev 49611
emitrevision: consider ancestors revision to emit as available base
This should make more delta base valid. This notably affects:
* case where we skipped some parent with empty delta to directly delta against
an ancestors
* case where an intermediate snapshots is stored.
This change means we could sent largish intermediate snapshots over the wire.
However this is actually a sub goal here. Sending snapshots over the wire means
the client have a high odd of simply storing the pre-computed delta instead of
doing a lengthy process that will… end up doing the same intermediate snapshot.
In addition the overall size of snapshot (or any level) is "only" some or the
overall delta size. (0.17% for my mercurial clone, 20% for my clone of Mozilla
try). So Sending them other the wire is unlikely to change large impact on the
bandwidth used.
If we decide that minimising the bandwidth is an explicit goal, we should
introduce new logic to filter-out snapshot as delta. The current code has no
notion explicite of snapshot so far, they just tended to fall into the wobbly
filtering options.
In some cases, this patch can yield large improvement to the bundling time:
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-bundle
# benchmark.variants.revs = last-100000
before: 68.787066 seconds
after: 47.552677 seconds (-30.87%)
That translate to large improvement to the pull time :
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = pull
# benchmark.variants.issue6528 = disabled
# benchmark.variants.revs = last-100000
before: 142.186625 seconds
after: 75.897745 seconds (-46.62%)
No significant negative impact have been observed.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Nov 2022 13:54:15 -0500] rev 49610
sqlitestore: add an `ancestors` method
We will need it during bundling.
The implementation mirror the one in revlog.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 24 Nov 2022 04:04:19 +0100] rev 49609
emitrevision: if we need to compute a delta on the fly, try p1 or p2 first
Falling back to `prev` does not yield any real value on modern storage and
result in pathological changes to be created on the other side. Doing a delta
against a parent will likely be smaller (helping the network) and will be safer
to apply on the client (helping future pulls by Triggering intermediate
snapshop where they will be needed by later deltas).
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 16:27:23 +0100] rev 49608
emitrevision: simplify the fallback to computed delta
Not using the stored delta, or having a full snapshot on disk behave the same
ways, so lets use the same code path for that, this is simpler, and it update
will be simpler.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:59:52 +0100] rev 49607
emitrevision: also check the parents in the availability closure
One of the point of having a closure is to gather the logic in it. So we gather
the logic.
The `parents[:]` part is a bit ugly but will be replaced by better code soon
anyway.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:48:51 +0100] rev 49606
emitrevision: add a small closure to check if a base is usable
We will make more use of this and make it more complex too.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 08 Dec 2022 15:13:17 +0100] rev 49605
chg: scale the timeout in test with the rest
This should avoid some flakiness where the logs reports server shutting down.
Anton Shestakov <av6@dwimlabs.net> [Sun, 11 Dec 2022 16:46:29 +0400] rev 49604
hghave: we might need py310 and py311 at some point
Some tests are already showing slightly different results on Python 3.11. The
better idea would be to make them more portable, but if that's not possible,
now we can use hghave detection for certain lines.
I wonder if there will ever be Python 31.0 and 31.1 though.
Anton Shestakov <av6@dwimlabs.net> [Sun, 11 Dec 2022 16:44:50 +0400] rev 49603
hghave: detect Python 3.10 and 3.11 as well
Noticed because test-contrib-relnotes.t was skipped.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Dec 2022 16:05:04 -0500] rev 49602
extensions: load help from hgext.__index__ as a fallback this time
Prior to 843418dc0b1b, `hgext.__index__` was consulted first if present, which
caused the longer help from the extension modules to be ignored, even when
available. But that change causes a bunch of test failures when the pyoxidized
binary bundles *.pyc in the binary, saying the there's no help topic for
`hg help $disabled_extension` and suggesting the use of `--keyword`, rather than
showing a summary and indicating that it is disabled. Current failures were in
test-check-help.t, test-extension.t, test-help.t, and test-qrecord.t.
Ideally, we would read the various *.pyc files from memory and slurp in the
docstring, but I know that they used to not be readable as resources, and I
can't figure out how to make it work now. So maybe 3.9 and/or the current
PyOxidizer doesn't support it yet. I got closer in py2exe with
`importlib.resources.open_binary("hgext", "rebase.pyc")`, but `open_binary()` on
*.pyc fails in pyoxidizer.[1] Either way, the *.pyc can't be passed to
`ast.parse()` as `extensions._disabledcmdtable()` is doing, so I'm setting that
aside for now.
[1] https://github.com/indygreg/PyOxidizer/issues/649
Matt Harbison <matt_harbison@yahoo.com> [Wed, 07 Dec 2022 11:26:07 -0500] rev 49601
extensions: process disabled external paths when `hgext` package is in-memory
This fixes `hg help -e ambiguous` in test-helpt.t:2055 with the
`ambiguous = !./ambiguous.py` configuration, when `hgext` is not in the
filesystem (e.g. pyoxidizer builds with in-memory resources, or TortoiseHg with
py2exe), but the disabled external extension is. Now instead of aborting with a
suggestion to try `--keyword`, the help command prints text for the extension.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Dec 2022 15:14:33 -0500] rev 49600
hg: show the correct message when cloning an LFS repo with extension disabled
The `extensions._disabledpaths()` doesn't handle fetching help from `__index__`,
so it returns an empty dictionary of paths. That means None is always returned
from `extensions.disabled_help()` when embedding resources inside the pyoxidizer
or py2exe binary, regardless of the arg or if is an external extension stored in
the filesystem. And that means wrongly telling the user with an explicitly
disabled LFS extension that it will be enabled locally upon cloning from an LFS
remote. That causes test-lfs-serve.t:295 to fail.
This effectively reverts most of the rest of 843418dc0b1b, while keeping the
help text change in place (which was specifically identified as a problem).
Matt Harbison <matt_harbison@yahoo.com> [Thu, 08 Dec 2022 21:45:47 -0500] rev 49599
demandimport: fix a crash in LazyFinder.__delattr__
I was tinkering with `with hgdemandimport.deactivated()` wrapped around loading
the keyring module, and got spew that seemed to be confirmed by PyCharm. But I
can't believe we haven't seen this before (and phabricator uses the same
pattern):
** Unknown exception encountered with possibly-broken third-party extension "mercurial_keyring" 1.4.3 (keyring 23.11.0, backend unknown)
** which supports versions unknown of Mercurial.
** Please disable "mercurial_keyring" and try your action again.
** If that fixes the bug please report it to https://foss.heptapod.net/mercurial/mercurial_keyring/issues
** Python 3.9.15 (main, Oct 13 2022, 04:28:25) [GCC 7.5.0]
** Mercurial Distributed SCM (version 6.3.1)
** Extensions loaded: absorb, attorc 20220315, blackbox, eol, extdiff, fastannotate, lfs, mercurial_keyring 1.4.3 (keyring 23.11.0, backend unknown), phabblocker 20220315, phabricator 20220315, purge, rebase, schemes, share, show, strip, uncommit
Traceback (most recent call last):
File "/usr/local/bin/hg", line 59, in <module>
dispatch.run()
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 143, in run
status = dispatch(req)
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 232, in dispatch
status = _rundispatch(req)
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 276, in _rundispatch
ret = _runcatch(req) or 0
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 451, in _runcatch
return _callcatch(ui, _runcatchfunc)
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 461, in _callcatch
return scmutil.callcatch(ui, func)
File "/usr/local/lib/python3.9/site-packages/mercurial/scmutil.py", line 153, in callcatch
return func()
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 441, in _runcatchfunc
return _dispatch(req)
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 1265, in _dispatch
return runcommand(
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 899, in runcommand
ret = _runcommand(ui, options, cmd, d)
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 1277, in _runcommand
return cmdfunc()
File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 1263, in <lambda>
d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
File "/usr/local/lib/python3.9/site-packages/mercurial/util.py", line 1880, in check
return func(*args, **kwargs)
File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 962, in cmd_keyring_check
user, pwd, source, final_url = handler.get_credentials(
File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 497, in get_credentials
keyring_pwd = password_store.get_http_password(keyring_url, actual_user)
File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 287, in get_http_password
return self._read_password_from_keyring(
File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 335, in _read_password_from_keyring
keyring = import_keyring()
>> `with hgdemandimport.deactivated()` inserted here
File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 120, in import_keyring
return _import_keyring()
File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 133, in _import_keyring
mod, was_imported_now = meu.direct_import_ext(
File "/usr/lib/python3.9/site-packages/mercurial_extension_utils.py", line 1381, in direct_import_ext
__import__(module_name)
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
self.loader.exec_module(module)
File "/usr/lib/python3.9/site-packages/keyring/__init__.py", line 1, in <module>
from .core import (
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
self.loader.exec_module(module)
File "/usr/lib/python3.9/site-packages/keyring/core.py", line 11, in <module>
from . import backend, credentials
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
self.loader.exec_module(module)
File "/usr/lib/python3.9/site-packages/keyring/backend.py", line 13, in <module>
from .py312compat import metadata
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
self.loader.exec_module(module)
File "/usr/lib/python3.9/site-packages/keyring/py312compat.py", line 10, in <module>
import importlib_metadata as metadata # type: ignore
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
self.loader.exec_module(module)
File "/usr/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 715, in <module>
class MetadataPathFinder(NullFinder, DistributionFinder):
File "/usr/lib/python3.9/site-packages/importlib_metadata/_compat.py", line 24, in install
disable_stdlib_finder()
File "/usr/lib/python3.9/site-packages/importlib_metadata/_compat.py", line 43, in disable_stdlib_finder
del finder.find_distributions
File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 88, in __delattr__
return delattr(object.__getattribute__(self, "_finder"))
TypeError: delattr expected 2 arguments, got 1
Anton Shestakov <av6@dwimlabs.net> [Mon, 05 Dec 2022 19:37:12 +0400] rev 49598
tests: use `test -f` instead of `ls` to see if a file is present (issue6662)
ls's exit code when file doesn't exist is 2 on Linux and 1 on NetBSD, so let's
use something that's supposedly more portable, since we only care whether the
file is there or not.
Raphaël Gomès <rgomes@octobus.net> [Mon, 05 Dec 2022 17:28:40 +0100] rev 49597
rust-status: fix thread count ceiling
This was forcing 16 threads instead of creating a ceiling, which is wrong
when either the available parallelism of the platform is lower or when
the user wants to set it explicitly (like we do in `run-tests.py`)
Anton Shestakov <av6@dwimlabs.net> [Thu, 01 Dec 2022 18:01:24 +0400] rev 49596
tests: use an all too familiar executable in test-run-tests.t (issue6661)
true(1) sometimes lives in /usr/bin/, and so this test fails on such systems.
We also can't use which(1), because it's apparently not POSIX and Debian
complains about it [1]. We also cannot really create a script and chmod +x it,
because this is a symlink case, execbit case is slightly below. So let's use
something that we know is executable, but not hg itself.
Anton Shestakov <av6@dwimlabs.net> [Thu, 01 Dec 2022 15:27:11 +0400] rev 49595
tests: add the missing space to test-hghave.t (issue6762)
log() in run-tests.py add a space to the end of all messages that it prints,
and I missed this fact in my previous patch.
Anton Shestakov <av6@dwimlabs.net> [Mon, 28 Nov 2022 16:10:30 +0400] rev 49594
tests: expect the message from 1baf0fffd82f in test-hghave.t (issue6762)
Couldn't reproduce locally, but apparently this message may occasionally pop up
when running this test.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 24 Nov 2022 10:34:34 +0100] rev 49593
test: adjust test-push-race.t timeout's to overall test timeout
The generic `tests/testlib/wait-on-file` mechanism scale its timeout with the
value of `HGTEST_TIMEOUT`, the `delaypush.py` in `test-push-race.t` is not doing
this, and we have been seeing more and more timeout from loaded CI worker
lately.
Adding this timeout scaling should help with that.
Raphaël Gomès <rgomes@octobus.net> [Wed, 23 Nov 2022 14:42:11 +0100] rev 49592
python-compat: adapt to Python 3.11 BC breakage with `random.sample`
As per https://docs.python.org/3/whatsnew/3.11.html#porting-to-python-3-11:
"The population parameter of `random.sample()` must be a sequence, and
automatic conversion of sets to lists is no longer supported.
Also, if the sample size is larger than the population size,
a `ValueError` is raised"
Matt Harbison <matt_harbison@yahoo.com> [Tue, 22 Nov 2022 11:55:26 -0500] rev 49591
match: make the FLAG_RE pattern a raw string
PyCharm was complaining about invalid escape sequences since this was added
recently in 3eda36e9b3d6.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 20 Nov 2022 15:55:27 -0500] rev 49590
help: fix a py3 error interpolating Set into b'%s'
I can't reproduce it, but a coworker hit this with `hg help -v` with 6.2.3:
...
File "mercurial\help.pyc", line 865, in helplist
TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'set'
I can confirm that the original expression fails in `hg debugshell`, and the new
one works. The second instance was found by searching for "%s", but PyCharm
detects a lot of variables as Any type, so I have no idea if there are other
lurking problems.
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 16:14:20 +0100] rev 49589
Added signature for changeset c890d8b8bc59
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 16:14:08 +0100] rev 49588
Added tag 6.3.1 for changeset c890d8b8bc59
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 16:00:39 +0100] rev 49587
relnotes: add 6.3.1
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 19 Nov 2022 01:35:01 +0100] rev 49586
memory-usage: fix `hg log --follow --rev R F` space complexity
When running `hg log --follow --rev REVS FILES`, the log code will walk the
history of all FILES starting from the file revisions that exists in each REVS.
Before doing so, it looks if the files actually exists in the target revisions.
To do so, it opens the manifest of each revision in REVS to look up if we find
the associated items in FILES.
Before this changeset this was done in a way that created a changectx for
each target revision, keeping them in memory while we look into each file.
If the set of REVS is large, this means keeping the manifest for each entry in
REVS in memory. That can be large… if REV is in the form `::X`, this can quickly
become huge and saturate the memory. We have seen usage allocating 2GB per
second until memory runs out.
So this changeset invert the two loop so that only one revision is kept in
memory during the operation. This solve the memory explosion issue.
Arseniy Alekseyev <aalekseyev@janestreet.com> [Fri, 18 Nov 2022 13:47:29 +0000] rev 49585
tests: run many tests in $TESTTMP/repo instead of $TESTTMP
This is useful so we can store other files in $TESTTMP
(in particular tests that use docket files (nodemap, dirstate-v2) keep
file uids in $TESTTMP/UID)
Arseniy Alekseyev <aalekseyev@janestreet.com> [Fri, 18 Nov 2022 13:52:18 +0000] rev 49584
tests: fix the detection of dirstate-v2 in hghave.py
Arseniy Alekseyev <aalekseyev@janestreet.com> [Thu, 17 Nov 2022 14:37:43 +0000] rev 49583
dirstate-v2: do not put the dirstate data file in a transaction,
since the transaction reverts the store, while the dirstate is stored separately
Matt Harbison <matt_harbison@yahoo.com> [Fri, 18 Nov 2022 13:43:03 -0500] rev 49582
commit: properly consider file include and exclude options when closing branch
It looks like this is meant to prevent adding another commit that does nothing
but close a branch on top of a commit that already closed the branch. The
matcher building functions want `Dict[bytes, Any]`, not `Dict[str, Any]`, which
was found by adding type hints to the matcher related methods in scmutil.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 18 Nov 2022 14:03:56 -0500] rev 49581
tests: demonstrate a bug blocking a redundant branch close
Arseniy Alekseyev <aalekseyev@janestreet.com> [Thu, 17 Nov 2022 16:31:52 +0000] rev 49580
tests: stop creating temporary files in TESTDIR
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 18:05:19 -0500] rev 49579
cffi: fix a bytes vs str issue on macOS when listing directories
This code hasn't been touched in recent years, and the other implementation
return bytes for the filename, so I assume this is a holdover from the py2 days.
I was unable to test it on mac though, because the `_osutil` import failed.
Jason R. Coombs <jaraco@jaraco.com> [Wed, 02 Nov 2022 12:54:12 -0400] rev 49578
packaging: refresh dependency hashes (issue6750)
Also, add some documentation to the `.in` files.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Nov 2022 15:39:10 +0100] rev 49577
matcher: do not prepend '.*' to pattern using ^ after flags
Since the previous commit (fixing wider issue), the code generated strange
regex. This is now fixed and tested.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Nov 2022 16:38:42 +0100] rev 49576
matcher: fix the issue with regex inline-flag in rust oo
Same problem same solution.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Nov 2022 13:05:01 +0100] rev 49575
matcher: fix issues regex flag contained in pattern (issue6759)
Python 3.11 is now enforcing that flag must be at the beginning of the regex
This creates a serious regression for people using Python 3.11 with an hgignore
using flag in a "relre" pattern.
We now detect any flags in such pattern and "prepend" our ".*" pattern after them.
In addition, we now insert the flag in the regexp to only affect the pattern we
are rewriting. Otherwise, the regex built from the combined pattern would these
flags in the middle of it anyway.
As a side effect of this last change, we fix a bug… before this change regex
flag in a pattern would affect all combined patterns. That was bad and is not
longer the case.
The Rust code needs to be updated to fix that very bug, but we will do it in
another changeset.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Nov 2022 14:40:27 +0100] rev 49574
release: removed the 6.3.0 tag
The revision having two tags confuses some of the build script trying to fetch
the version. So, remove the "bad" tag for now.
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 13:35:56 +0100] rev 49573
hg-core: relax dependencies pinning
Being this rigid makes packagers' job more difficult since they might not
carry the exact version. This hard pinning was introduced in eb02decdf but
wasn't strictly necessary to achieve its compatibility goal.
Anton Shestakov <av6@dwimlabs.net> [Mon, 14 Nov 2022 19:58:44 +0400] rev 49572
tests: make running ls in a no longer existing directory more portable
On Linux, ls -A simply returns nothing and the exit code is 0.
On NetBSD, ls -A complains that . doesn't exist and the exit code is 1.
Sadly, it's not possible to do something like "[1] (?)", so " || true" is the
best I could come up with.
Anton Shestakov <av6@dwimlabs.net> [Mon, 14 Nov 2022 19:38:57 +0400] rev 49571
tests: move some lines inside #if windows-#else block test-removeemptydirs.t
This is done to make it's clear that windows is not affected by this test case,
IOW windows and non-windows platforms are now tested separately, because their
results are very different.
Anton Shestakov <av6@dwimlabs.net> [Mon, 20 Jun 2022 12:53:08 +0400] rev 49570
tests: use ls -A instead of ls -1 in test-removeemptydirs.t
In case the tests are run as root, ls assumes -A by default on some systems
(e.g. NetBSD). Tests probably shouldn't be run as root, but let's use -A just
in case, for convenience.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Nov 2022 10:28:39 +0100] rev 49569
release: add the "6.3" to the 6.3 release
The release was tagged "6.3.0" while the custom have been to omit the third ".0"
for initial release for the history of the project.
For consistency, we add the customary version of the tag to the 6.3 release.
pacien <pacien.trangirard@pacien.net> [Mon, 14 Nov 2022 20:21:53 +0100] rev 49568
tests: fix new git protocol policy in convert-git.t
Recent versions of git restrict the use of the "file" protocol for security
reasons (https://github.com/git/git/commit/a1d4f67).
This broke this test, which failed with the following hidden error:
"fatal: transport 'file' not allowed".
This patch relaxes the git configuration for the test to solve this.
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 10:57:44 +0100] rev 49567
relnotes: add final 6.3 relnotes
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 10:35:09 +0100] rev 49566
Added signature for changeset 04f1dba53c96
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 10:34:17 +0100] rev 49565
Added tag 6.3.0 for changeset 04f1dba53c96
Raphaël Gomès <rgomes@octobus.net> [Wed, 09 Nov 2022 23:28:01 -0500] rev 49564
rust: create wrapper struct to reduce `regex` contention issues
Explanations inline.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Nov 2022 02:38:53 +0100] rev 49563
tags-fnode-cache: skip building a changectx in getfnode
Building a changectx object is costly, doing it just to retrieve the revision
number is suboptimal. Directly fetching the revision number from the changelog
provide a sizeable speedup to `hg debugupdatecache`.
### data-env-vars.name = mercurial-2018-08-01-zstd-sparse-revlog
# benchmark.name = debug-update-cache
# benchmark.variants.pre-state = warm
before: 0.213229 seconds
after: 0.165577 seconds (-22.35%)
# data-env-vars.name = mercurial-filtered-2019-11-22-zstd-sparse-revlog
before: 1.200383 seconds
after: 1.071618 seconds (-10.73%)
# data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
before: 1.465735 seconds
after: 0.923128 seconds (-37.02%)
# data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
before: 6.511771 seconds
after: 4.507316 seconds (-30.78%)
# data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog
before: 1.023007 seconds
after: 0.645026 seconds (-36.95%)
# data-env-vars.name = pypy-2018-08-01-zstd-sparse-revlog
before: 0.381141 seconds
after: 0.268654 seconds (-29.51%)
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Nov 2022 02:38:26 +0100] rev 49562
tags-fnode-cache: do not repeatedly open the filelog in a loop
While getting multiple hgtagsfnodecache entries, we were opening (and closing)
the `.hgtags` filelog for each iteration. The meant repeatedly reading and
parsing the version same information from disk. A quite costly operation.
We no longer do this, leading to a sizable improvement in `hg debugupdatecache`
run for an already warm repositories.
### data-env-vars.name = mercurial-2018-08-01-zstd-sparse-revlog
# benchmark.name = debug-update-cache
# benchmark.variants.pre-state = warm
before: 1.711778 seconds
after: 0.213229 seconds (-87.54%)
# data-env-vars.name = pypy-2018-08-01-zstd-sparse-revlog
before: 4.010817 seconds
after: 0.381141 seconds (-90.50%)
# data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog
before: 13.574141
after: 1.023007 seconds (-92.46%)
# data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
before: 18.884656
after: 1.465735 seconds (-92.24%)
# data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
before: 88.924823
after: 6.511771 seconds (-92.68%)
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Nov 2022 02:30:41 +0100] rev 49561
profile: prevent a crash when line number is unknown
For some unknown reason, `self.lineno` can be None. The previous code crashed in
such case, we now ignore the case, as we do for other error in this function.
We also fallback to using `-1` in the output when this lack of line number
makes it to the display code.
The reason of unknown line-numbers is… unknown.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 06 Nov 2022 12:15:35 -0500] rev 49560
upgrade: byteify requirement changes written to output
ui.write() expects bytes, and internally uses `b''` as the default when getting
the `label` keyword from `*args`. So either we're missing test coverage, or
there's some very subtle conversion from unicode somewhere.
Also, slip in a type hint to flag this in the future.
Raphaël Gomès <rgomes@octobus.net> [Thu, 03 Nov 2022 16:30:35 +0100] rev 49559
rhg: add a config option to fall back immediately
This is useful for debugging the behavior of the "default" `hg` in tests
without having to manually substitute the fallback path.
Raphaël Gomès <rgomes@octobus.net> [Thu, 03 Nov 2022 15:57:37 +0100] rev 49558
rhg: stop shadowing `exit` function
This will be useful for the next patch which needs it.
Raphaël Gomès <rgomes@octobus.net> [Thu, 03 Nov 2022 15:43:04 +0100] rev 49557
config: add alias from `hg help rhg` to `hg help rust`
This will make using `rhg` more user-friendly and features more
discoverable.
Raphaël Gomès <rgomes@octobus.net> [Thu, 03 Nov 2022 15:42:33 +0100] rev 49556
rhg: add `config.rhg` helptext
This will make using `rhg` more user-friendly and features more
discoverable.
Raphaël Gomès <rgomes@octobus.net> [Thu, 03 Nov 2022 15:44:54 +0100] rev 49555
config: fix indentation of some`share-safe` options
This makes the output much more readable.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 16 Oct 2022 04:48:21 +0200] rev 49554
perf-unbundle: do a quick and dirty fix to make it run on more commit
Without this change, the perf commands fails within the
f67741e8264b::18415fc918a1 range (boundary excluded).
Check inline comment for details.
With this fix, the command is able to run on this range, with a slightly
different behavior (as no revset is "uninlined"). However this is still much
better than not being able to run anything in this range. Especially because
that range do see some performance regression for unbundle.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 19 Oct 2022 01:54:04 +0200] rev 49553
perf-unbundle: pre-indent the main block in per::unbundle
This makes the next changeset clearer.
Jason R. Coombs <jaraco@jaraco.com> [Wed, 19 Oct 2022 16:23:42 -0400] rev 49552
shelve: handle empty parents and nodestoremove in shelvedstate (issue6748)
Raphaël Gomès <rgomes@octobus.net> [Thu, 03 Nov 2022 14:58:58 +0100] rev 49551
rhg: fallback when encountering ellipsis revisions
Ellipsis revisions are still experimental and buggy in non-trivial
histories. We currently have no plans to improve this situation nor
to add support for ellipsis revisions in `rhg`.
Falling back should be done carefully (since we may have already done
some work that is visible to the user), but in this case it's highly
unlikely that we're doing anything useful with a repo with ellipsis
revisions, so this should be strictly better, also since the
error message is more explicit.
Raphaël Gomès <rgomes@octobus.net> [Wed, 02 Nov 2022 12:05:34 +0100] rev 49550
dirstate-v2: hash the source of the ignore patterns as well
Fixes the test introduced in the last changeset. This caused the hash
to change, which means that the check in the test had to be adapted.
Since this hash is only done as a caching mechanism, invalidation does
not pose any backwards compatibility issues.
Raphaël Gomès <rgomes@octobus.net> [Wed, 02 Nov 2022 15:24:39 +0100] rev 49549
dirstate-v2: add test that shows a collision in ignore patterns hash
This hash is used for optimizing dirstate `status`. We demonstrate that the
hash is incorrectly ignoring the changes to the semantics of the ignore
files just because the contents (but not their source) haven't changed.
This is fixed in the next changeset.
Raphaël Gomès <rgomes@octobus.net> [Thu, 03 Nov 2022 12:08:02 +0100] rev 49548
testlib: add `--raw-sha1` option to `f`
This will be used in the patch fixing the ignore hash collision in
dirstate-v2 to check the behavior of the new hashing function.
Raphaël Gomès <rgomes@octobus.net> [Wed, 19 Oct 2022 16:28:41 +0200] rev 49547
rust-status: save new dircache even if just invalidated
There was a functional race between invalidating the cache (not acted upon
until the end of the status algorithm) and populating the new cache (which
relies upon an up-to-date version of the cache).
We simply inform the cache populating function that we've just invalidated
the cache for this particular directory since the information is present in
the same scope.
Raphaël Gomès <rgomes@octobus.net> [Wed, 19 Oct 2022 15:11:05 +0200] rev 49546
rust-status: fix typos and add docstrings to dircache related fields
Arseniy Alekseyev <aalekseyev@janestreet.com> [Thu, 22 Sep 2022 15:34:27 -0400] rev 49545
rhg: show a bug where repeated [hg status] is needed to cache everything
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 04 Nov 2022 16:15:12 -0400] rev 49544
upgrade: no longer keep all revlogs in memory at any point
Keeping all object open is unsustainable, so we will open them on demand. This
mean opening them multiple times, but this is a lesser evil.
Each revlog consume a small amount of memory (index content, associated nodemap,
etc). While there are few "big" revlog, the sheer amount of small filelog can
become a significant issue memory wise, consuming multiple GB of memory. If you
combines this extra usage with the use of multiprocessing, this usage can
quickly get out of control. This can effectively block the upgrade of larger
repository. This changeset fixes this issue.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 02 Nov 2022 14:23:09 -0400] rev 49543
demandimport: convert ignored modules from bytes -> str in extensions
The default list of ignored modules are str, and test for bypassing the lazy
import is `module.__name__ in ignores`, so these were effectively NOT ignored.
Most of these date back to the grand byteification in 687b865b95ad, with some
subsequent additions that followed the existing example.
I have no idea if these modules in fact need to bypass lazy importing, but at
least it follows the intent of the code.
Raphaël Gomès <rgomes@octobus.net> [Wed, 26 Oct 2022 18:46:56 +0200] rev 49542
dirstate-v2: fix edge case where entries aren't sorted
See previous commit for more details.
Raphaël Gomès <rgomes@octobus.net> [Wed, 26 Oct 2022 18:24:34 +0200] rev 49541
dirstate-v2: highlight a bug when Python-packed but used in `rhg`
The Python packer creates unsorted entries in the edge case that a file
starts with the same name as a sibling folder.
This bug has no effect on the Python `hg status` since Python ignores
directories. `rhg` assumes that all on-disk entries are sorted (which is
a property of the format) including folder, hence the issue highlighted.
This is also technically broken in Rust-augmented `hg status`, but it
makes setting up the test more complex than necessary, since it requires
the packing to be Python only (which it isn't if you have Rust extensions).
Fix is in the next commit.
Raphaël Gomès <rgomes@octobus.net> [Wed, 26 Oct 2022 12:20:23 +0200] rev 49540
dirstate-v2: correct documented return values of `pack_dirstate`
Raphaël Gomès <rgomes@octobus.net> [Wed, 26 Oct 2022 12:19:47 +0200] rev 49539
dirstate-v2: fix typos in docstrings
Raphaël Gomès <rgomes@octobus.net> [Fri, 04 Nov 2022 14:52:16 -0400] rev 49538
dirstate-v2: update constant that wasn't kept in sync
Despite the best efforts of the comment, this constant wasn't kept in sync
when the flags were being rewritten.
The fact that this doesn't actually break anything in the Rust implementation
too much (which does use directories) relies on the fact that all nodes
can have children and that dirstate traversal is not based on that flag, but
for metadata in optimizations. However the bug could become more serious should
we start encoding stronger guarantees using a combination of flags including
this one.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 18 Oct 2022 13:56:45 -0400] rev 49537
lfs: avoid closing connections when the worker doesn't fork
Probably not much more than an minor optimization, but could be useful in the
case of `hg verify` where missing blobs are fetched one at a time.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 18 Oct 2022 13:36:33 -0400] rev 49536
lfs: fix blob corruption when tranferring with workers on posix
The problem seems to be that the connection used to request the location of the
blobs is sitting in the connection pool, and then when workers are forked, they
all see and attempt to use the same connection. This garbles everything. I
have no clue how this ever worked reliably (but it seems to, even on Linux, with
SCM Manager 1.58). See previous discussion when worker support was added[1].
It shouldn't be a problem on Windows, since the workers are just threads in the
same process, and can see which connections are marked available and which are
in use. (The fact that `mercurial.keepalive.ConnectionManager.set_ready()`
doesn't acquire a lock does give me some pause though.)
[1] https://phab.mercurial-scm.org/D1568#31621
Matt Harbison <matt_harbison@yahoo.com> [Tue, 18 Oct 2022 12:58:34 -0400] rev 49535
keepalive: add `__repr__()` to the HTTPConnection class to ease debugging
By default, this just printed the class name and memory address. By displaying
the address and ports on both sides of the socket, it makes it easier to figure
out what's in the ConnectionManager, and correlate with WireShark traces.
It looks like the two connections mentioned in the previous commit come about
because the LFS POST request to access the blobs opens connection 1, and gets a
401. Then for some reason, the follow up with credentials opens a new socket,
instead of using the existing one in the pool. I have no clue why.
This can be seen with something like this in the blobstore:
```
for h in self.urlopener.handlers:
if hasattr(h, "close_all"):
print('open connections on %s in pid %d' % (type(h), os.getpid()))
for host, conns in h._cm.get_all().items():
for c in conns:
print('connection: %r' % c)
```
Matt Harbison <matt_harbison@yahoo.com> [Tue, 18 Oct 2022 11:54:58 -0400] rev 49534
keepalive: ensure `close_all()` actually closes all cached connections
While debugging why LFS blob downloads are getting corrupted with workers, I
noticed that prior to spinning up the workers, the ConnectionManager has 2
connections to the server and calling `KeepAliveHandler.close_all()` left one
behind. The reason is the value component of `self._cm.get_all().items()` is a
list, and `self._cm.remove()` modifies said list while the caller is iterating
over it. Now `get_all()` is a deep copy of both the dict and lists in all
cases.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 02 Nov 2022 16:46:46 -0400] rev 49533
localrepo: byteify the requirements.DIRSTATE_TRACKED_HINT_Vx warning message
Flagged by PyCharm.
Arseniy Alekseyev <aalekseyev@janestreet.com> [Mon, 31 Oct 2022 16:15:54 +0000] rev 49532
rhg: fallback to slow path on invalid patterns in hgignore
Arseniy Alekseyev <aalekseyev@janestreet.com> [Mon, 31 Oct 2022 16:15:30 +0000] rev 49531
rhg: add a test involving hgignore lookaround
Raphaël Gomès <rgomes@octobus.net> [Mon, 24 Oct 2022 18:07:22 +0200] rev 49530
relnotes: add 6.3