sort-revset: introduce a `random` variant This new `sort` variant allows to shuffle any revset. It also allow for randomly picking element using `first`.

tests: remove flakiness in test-nointerrupt.t The problem was that the reaction to the signal was racing against the completion of the command. Since reaction to the signal is to print a line of warning, we can fix this by waiting for that warning to appear before allowing the command to complete.

perf: properly process formatter option in perf::unbundle Otherwise, the options are not understood.

perf: quiet stdout output in perf::unbundle There a lot of repetitive bundle application message we do not care about.

perf: quiet stderr output in perf::unbundle There a lot of repetitive transaction message we do not care about.

bisect: avoid adding irrelevant revisions to bisect state When adding new revisions to the bisect state, it only makes sense to add information about revisions that are under consideration (i.e., those that are topologically between the known good and bad revisions). However, if the user passes in a revset (e.g., '!merge()' to exclude merge commits), hg will resolve the revset first and add all matching revisions to the bisect state (which in this case would likely be the majority of revisions in the repo). To avoid this, revisions should only be added to the bisect state if they are between the good and bad revisions (and therefore relevant to the bisection). -- Here are the results of some performance tests using the `mozilla-central` repo (since it is one of the largest freely-available hg repositories in the wild). These tests compare the performance of a locally-built `hg` before and after application of this series. Note that `--noupdate` is passed to avoid including update time (which should not vary across cases). Setup (run between each test): $ hg bisect --reset $ hg bisect --noupdate --bad 56c3ad4bde5c70714b784ccf15d099e0df0f5bde $ hg bisect --noupdate --good 57426696adaf08298af3027fa77486fee0633b13 Test using a revset that returns a very large number of revisions: $ time hg bisect --noupdate --skip '!merge()' > /dev/null Before: real 0m9.398s user 0m9.233s sys 0m0.120s After: real 0m1.513s user 0m1.425s sys 0m0.052s Test using a revset that is expensive to compute: $ time hg bisect --noupdate --skip 'desc("Bug")' > /dev/null Before: real 0m49.853s user 0m49.580s sys 0m0.243s After: real 0m4.120s user 0m4.036s sys 0m0.048s

bisect: limit ancestors to revs topologically between good and bad revs Previously, when constructing its dict of revisions to their ancestors, bisect would populate the dict with ALL of the descendents of the good set, which is a bit silly because it is impossible for a revision that is a descendent of the minimum known bad revision to be the first bad rev. Instead it makes more sense to limit the revisions to just those topologically between the good and bad.

bisect: bypass changectx when translating revs to nodes When resolving the revset given by the user into node hashes, use the changelog to perform the translation rather than the repo object. This avoids the overhead of constructing a changectx which is immediately discarded.

rhg: make [rhg status -v] work when it needs no extra output Add support for verbose [status] when no extra output is actually needed. This makes it so that [rhg status] is actually useful when [tweakdefaults] is true. (since tweakdefaults implies verbose status)

perf: introduce a benchmark for delta-find That part is responsible of serious slowdown in some `hg pull/unbundle` case. So lets add a way to benchmark it.

automation: set PATH when building on Windows Sometime in the 6.2 release cycle the Windows building automation broke. Building the wheel and even PyOxidizer based installers now fails with: ``` Exception: PowerShell execution failed: error: subprocess-exited-with-error Getting requirements to build wheel did not run successfully. exit code: 1 [1 lines of output] Unable to find a working hg binary to extract the version from the repository tags [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ``` I have a hunch this is a regression from upgrading pip in 1c00777702da, but I haven't verified this. It may not be, as PyOxidizer has its own bundled Python/pip. So maybe it is something in `setup.py`.

contrib: update Mercurial install in bootstrap environment

automation: transition to Windows Server 2022 Let's keep our Windows build environment modern by upgrading to the latest OS. As part of the upgrade, we pick up a migration to EC2Launch Version 2. This has a different config mechanism. So we need to port how we manage the administrator password. As part of migrating to the new YAML/JSON config file mechanism, we move the code to the powershell script that is run when the instance first launches. This ensures that the config is retained during the reboot we perform as part of building the Windows AMI. The motivation for this is I'm currently unable to build the Windows 2019 AMI due to an issue installing OpenSSH. This _just works_ on Windows Server 2022. I have no clue what the root cause is. I think it might have something to do with Microsoft not publishing the files in the right location. Differential Revision: https://phab.mercurial-scm.org/D12630

automation: refresh requirements I'm hitting errors installing the old version of cffi due to an apparent issue where older versions of cffi aren't compatible with the modern Clang I'm using. So let's upgrade packages to unbreak things and to keep things modern. Differential Revision: https://phab.mercurial-scm.org/D12629

debug-delta-find: introduce a quiet mode In quiet mode, we only print the summary of the search and skip the individual steps.

phase-shelve: also capture the state of shelve prior to unshelve

phase-shelve: Add test for shelve technique config

phase-shelve: Implement a 'shelve.store' experimental config Accepts "internal" or "strip", indicating how the shelved changes are stored. Defaults to "internal", retaining compatibility for repos with "internal-phase" already enabled.

phase-shelve: Extract function for _target_phase

phase-shelve: expand the tests to capture use-cases supported

phase-shelve: honor and prefer obs shelves for existence and modified time

phase-shelve: read patch details from a (possibly internal) node in the repo

phase-shelve: Extract function for _optimized_match for re-use

obsstore: break the repo → obstore → repo loop This should help the garbage collector to do its job. On repository with many markers, the memory pressure from the obsstore can get quite serious.

rust-status: ignored directories are now correctly only listed if opted into This fixes the behavior of `hg purge` removing empty ignored directory even without `--all` or `--ignored`.

test: show how purge removes ignored directories Apparently the rust code path removes ignored directories even though the pure code path doesn't. Show this in tests.

relnotes: mention chg behavior change when given --cwd

dispatch: change cwd when loading local config Previously, the `_getlocal` function would not correctly load the repo config when given a relative `rpath` and an alternate cwd via the `wd` parameter. Normally when `--cwd` is specified, hg changes to the given directory before attempting to load the local config (and therefore does not specify a `wd`). The only time the function is called with `wd` set is when hg is running as a command server (e.g., with chg), in which case each forked worker process will attempt to configure itself via `_getlocal` before responding to the client. When given a relative repo path, the worker fails to load the repo config, detects a config mismatch with the client, and enters a redirect/respawn loop. To fix this, we can simply change to the desired working directory during config loading. (Note that simply concatenating `wd` and `rpath` won't work in all cases. The repo path could be something more complicated than a simple relative path, such as a `union:` repo.)

contrib: add support for rhel9

packagelib: use python3 by default

tests: work around libmagic bug in svn subrepo tests libmagic 5.40 introduced a bug [1] wherein ASCII text files with fewer than 3 distinct character values would be reported as binary data rather than as text. This bug was later fixed in version 5.41 [2]. SVN uses libmagic to determine the MIME type of added files with missing or unknown extensions [3]. This results in test failures on systems with libmagic 5.40 installed: $ echo a > a $ svn add a - A a + A (bin) a A simple workaround is to change the test file's content to include 3 distinct ASCII values (including the terminating newline). [1] https://bugs.astron.com/view.php?id=180 [2] https://bugs.astron.com/view.php?id=261 [3] https://svnbook.red-bean.com/en/1.8/svn.advanced.props.html#idm2649

rust: bump to memmap2 0.5.3, micro-timer 0.4.0, and crossbeam-channel 0.5.0 The merge in 12adf8c695ed had conflicts in rust/Cargo.lock and rust/hg-core/Cargo.toml . Let's ignore rust/Cargo.lock - it is regenerated. For rust/hg-core/Cargo.toml, stable had dd6b67d5c256 "rust: fix unsound `OwningDirstateMap`" which introduced ouroboros (and dropped stable_deref_trait). Default had ec8d9b5a5e7c "rust-hg-core: upgrade dependencies" which had a lot of churn bumping minimum versions - also patch versions. It is indeed a good idea to bump to *allow* use of latest package. That means that major versions should be bumped for packages after 1.0, and for packages below 1.0 minor versions should be bumped too. But it doesn't work to try enforce a policy of using latest patch by bumping versions at arbitrary times. For good or bad, the merge doesn't seem to have resolved the conflicts correctly, and many of the minor "upgrade dependencies" were lost again. Unfortunately, it also lost the bump of memmap2 to 0.5.3, which is needed for Fedora packaging where 0.4 isn't available. Same with micro-timer bump to 0.4 (which already is used in rhg). crossbeam-channel bump was also lost. This change fixes that regression by redoing these "important" lines of the merge "correctly". I propose this for stable, even though dependency changes on stable branches are annoying.

revlog: make _partialmatch fail fast on almost-hex inputs Before this change, resolving a revision like [0123456789^] on a large repo can take multiple seconds because: - hg does not realize this is a revset, so it tries various things, including _partialmatch(b"0123456789^") - after the rust lookup fails, it falls back to pure hg - pure hg takes all-but-last chars and converts them to binary, which *succeeds*, so it does the expensive part.

perf-unbundle: add a perf command to time the unbundle operation Check documentation for details.

perf-bundle: accept --type argument

perf-bundle: accept --rev arguments This is fairly standard nowaday.

perf-bundle: add a new command to benchmark bundle creation time

Added signature for changeset f69bffd00abe

Added tag 6.2.1 for changeset f69bffd00abe

bundle: introduce a --exact option I have been wanting this options for a long time.

bundlespec: add documentation about existing option We have some documentation, lets make it complete.

debug-discovery: apply spelling fixes from Raphaël

tree-discovery: fix the request debug output and progress location This is now associated with each request.

debug-discovery: deal with case where common is empty This code was previously confused by case where: `heads_common == {nullid}`

debug-discovery: do not abort on unrelated repositories This is a useful case to consider, so we should not abort in this case. A warning is still issued.

debug-discovery: gather the right number of roundtrips for tree discovery We where not counting the right amount of request before.

debug-discovery: also gather details on tree-discovery queries type This is useful to understand the algorithm.

debug-discovery: properly apply remote filtering in "old" mode Before this change using `--remote-as-revs` with `--old` had no effect and everything was considered as "common", which is really not what we intended.

debug-discovery: fix a typo in the doc The second option should be `--remote-…` as we just talked about `--local-…` already.

packaging: bump dulwich to 0.20.45 I'm told the new dulwich avoids hg-git test failures.

packaging: update keyring on Windows to avoid spurious stacktraces When challenged for a network password, this would spew on Windows before it actually used the stored password: ``` Error initializing plugin EntryPoint(name='libsecret', value='keyring.backends.libsecret', group='keyring.backends'). Traceback (most recent call last): File "keyring.backend", line 198, in _load_plugins init_func = ep.load() File "importlib.metadata", line 77, in load module = import_module(match.group('module')) File "importlib", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked ModuleNotFoundError: No module named 'keyring.backends.libsecret' Error initializing plugin EntryPoint(name='macOS', value='keyring.backends.macOS', group='keyring.backends'). Traceback (most recent call last): File "keyring.backend", line 198, in _load_plugins init_func = ep.load() File "importlib.metadata", line 77, in load module = import_module(match.group('module')) File "importlib", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked ModuleNotFoundError: No module named 'keyring.backends.macOS' ``` We're kinda threading a needle here because the next version of `keyring` (currently at 23.7.0) requires `importlib-metadata` 3.6+, which PyOxidizer 0.22 doesn't support[1]. [1] https://github.com/indygreg/PyOxidizer/issues/609

setup: use the full executable manifest from `python.exe` The manifest embedded by the build process (before the string here is added) already accounts for the `<requestedExecutionLevel level="asInvoker" ...>` setting. (Note that the PyOxidizer build is missing this, so it will likely trigger the UAC escalation prompt on each run.) However, using `mt.exe` to merge the fragment with what is already in the manifest seems to strip all whitespace, making it unreadable. Since Mercurial can be run via `python.exe`, it makes sense that we would have the same manifest settings (like the supported OS list), though I'm unaware of any functionality this enables. It also has the nice effect of making the content readable from a resource editor. The manifest comes from python 3.9.12. Note that this seems to strip the `<?xml ... ?>` declaration when viewed with ResourceHacker 5.1.7, but this was also the state of things with the previous commit, and `mt.exe "-inputresource:hg.exe;#1" -out:extracted` does contain the declaration and the BOM in both cases. No idea why this differs from other executables.

setup: unconditionally enable the `long-paths-support` option on Windows I don't see anything talking about why this was experimental in the first place, but maybe it was concern about the level of python2 support for it. But now, both `python.exe` and the PyOxidizer build of `hg.exe` have a manifest that enables it, so leaving it off would mean some Mercurial installations could operate on a repo with long paths, and others couldn't. Note that only the wide character functions (XxxW) will have the length restriction lifted. Sadly, distutils applies `/MANIFEST:EMBED` to the linker in a way that can't easily be turned off, so we can't use `/MANIFESTFILE` with `extra_preargs` on `link_executable`. Fortunately, the compiler object provides a path to the `mt.exe` it found during initialization, because the previous incarnation seems to have assumed it is being run within an activated Visual Studio environment. That causes MSYS builds to fail, and probably would have broke the CI environment.

setup: stop shadowing the builtin `dir` symbol I hit this when debugging what's available on the compiler.

subrepo: avoid opening console window for non-native subrepos on Windows Prevent annoying command prompt windows popping up when using TortoiseHG with Git and SVN subrepos by passing creationflags=subprocess.CREATE_NO_WINDOW to subprocess.Popen.

mergestate: action name was str Apparently the standard for them is still to use byte strings. Found while looking at something else

ci: bump pytype to 2022.03.29 This is as far as we can go without running into issues with the vendored `attr` package. I tried updating that to the latest, and not only did it not fix the issue, but test-util.py failed due to some poking at `attr` internals that apparently is no longer valid. The `libcst` package is now pinned to what I have locally because trying to install the latest (0.4.7) complains that it can't find the Rust compiler. We should probably use a requirements file instead (and/or figure out why it can't find the Rust compiler), but I don't feel like dealing with another side quest.

typing: suppress a few attribute errors in url.py These are newly detected by pytype 2022.03.21. Not sure what is going on here- `realhostport` and `headers` are added outside of the constructor, so that makes sense. But PyCharm also thinks the private methods don't exist, though when clicking through the class hierarchy, it shows in the py3.9 source code.

typing: suppress a few pyi-errors with more recent pytype Not sure what's going on here, but these were flagged with pytype 2022.03.21. We can't update to something much more recent, because newer versions complain about various `attr` uses.

sslutil: another use proper attribute to select python 3.7+ The previous attribute was python 3.6+, but guarded a python 3.7+ block Using the correct attribute avoids: + File "/tmp/hgtests.bc0_uk2d/install/lib/python/mercurial/sslutil.py", line 577, in wrapserversocket + sslcontext.minimum_version = ssl.TLSVersion.TLSv1_1 + AttributeError: module 'ssl' has no attribute 'TLSVersion'