Added signature for changeset a3356ab610fc

Added tag 6.3rc0 for changeset a3356ab610fc

branching: merge default into stable This marks the feature freeze for the 6.3 release

shelve: re-wrap now that the line fits

shelve: avoid setting overloading tmpwctx

dirstate-v2: skip evaluation of hgignore regex on cached directories By making the computation of [has_ignored_ancestor] lazy we're eliding its computation in the common case when none of its descendants have changed on disk. On a ~400k files repo, with a cached status, we saw a ~64% reduction in CPU time, resulting in a speedup of ~10-15% (on ZFS), and a speedup of ~38% of XFS (XFS has faster stat operations for some reason).

releasenotes: use re.MULTILINE mode when checking admonitions Release note admonitions must start at the beginning of a line within the changeset description: .. admonitions:: The checkadmonitions function search for and validates admonitions. Unfortunately, since the ctx.description is multi-line, the regex search always fails unless the admonition is on the first line. This changeset adds re.MULTILINE to the re.compile to make the re opbject multi-line.

rhg: parallellize computation of [unsure_is_modified] [unsure_is_modified] is called for every file for which we can't determine its status based on its size and mtime alone. In particular, this happens if the mtime of the file changes without its contents changing. Parallellizing this improves performance significantly when we have many of these files. Here's an example run (on a repo with ~400k files after dropping FS caches) ``` before: real 0m53.901s user 0m27.806s sys 0m31.325s after: real 0m32.017s user 0m34.277s sys 1m26.250s ``` Another example run (a different FS): ``` before: real 3m28.479s user 0m31.800s sys 0m25.324s after: real 0m29.751s user 0m41.814s sys 1m15.387s ```

rhg: enable in case ui.statuscopies=True rhg already has code to support ui.statuscopies, but it's disabled, for seemingly no good reason.

rhg: share some code

rhg: support tweakdefaults

rhg: centralize PlainInfo

rhg: central treatment of PLAIN and PLAINEXCEPT

revset: handle wdir() in `sort(..., -topo)` The last apparent usage of `repo.changelog.parentrevs` in revsets is in `children()`, but since the sets being operated on never include wdir(), it's never called with `wdirrev` and the wdir() arg on the command line is effectively ignored instead of aborting there. I'm not sure how to fix that. Before (on a clone of hg): $ python3.8 hg perf::revset --config extensions.perf=contrib/perf.py 'sort(all(), -topo)' ! wall 0.123663 comb 0.130000 user 0.130000 sys 0.000000 (best of 76) After: $ python3.8 hg perf::revset --config extensions.perf=contrib/perf.py 'sort(all(), -topo)' ! wall 0.123838 comb 0.130000 user 0.130000 sys 0.000000 (best of 75)

revset: handle wdir() in `roots()` This is already handled in `heads()`, and both are needed to determine if a set is contiguous. I'm guessing the `0 <= p` check was to try to filter out the null revision, but it looks like that comes through in the corner case of a new repo with no commits. But that was already the case, as shown by the tests. Before (on a clone of hg): $ python3.8 hg perf::revset --config extensions.perf=contrib/perf.py 'roots(all())' ! wall 0.059301 comb 0.040000 user 0.040000 sys 0.000000 (best of 100) After: $ python3.8 hg perf::revset --config extensions.perf=contrib/perf.py 'roots(all())' ! wall 0.059387 comb 0.060000 user 0.060000 sys 0.000000 (best of 100)

pull_logger: add basic log file rotation based on size

contrib: add pull_logger extension This extension logs the pull parameters, i.e. the remote and common heads, when pulling from the local repository. The collected data should give an idea of the state of a pair of repositories and allow replaying past synchronisations between them. This is particularly useful for working on data exchange, bundling and caching-related optimisations.

branching: merge stable into default

mergetools: don't let meld open all changed files on startup In meld 3.16, a multi-file change with option '-a' results in an overview list being opened. In meld 3.20, a multi-file change with option '-a' results in an overview list AND every changed file being opened. Simply removing '-a' seems to work fine for both cases and also behaves the same as before for single-file changes. As per the release notes, this is due to the following change in meld 3.19.0: "Make the --auto-compare command line flag work again (Kai Willadsen)" See also https://gitlab.gnome.org/GNOME/meld/-/issues/516

tests: fix http-bad-server expected errors for python 3.10 (issue6643) The format of the error message changed with this version of Python. This also removes obsolete Python 3 checks.

status: let `--no-copies` override `ui.statuscopies`

run-tests: display the time it took to install Mercurial It will help make people aware of this critical step and to assess the time it takes in various options (like a CI run for example).

run-tests: deal with distutil deprecation PEP 632 recommend the use of `packaging.version` to replace the deprecated `distutil.version`. So lets do it.

fsmonitor: migrate Python ABCs from collections to collections.abc The Collections Abstract Base Classes in the collections module are deprecated since Python 3.3 in favor of collections.abc, and removed in Python 3.10.

templates: add filter to reverse list The filter supports only lists because for lists, it’s straightforward to implement. Reversing text doesn’t seem very useful and is hard to implement. Reversing the bytes would break multi-bytes encodings. Reversing the code points would break characters consisting of multiple code points. Reversing graphemes is non-trivial without using a library not included in the standard library.

requires: re-use vfs.tryread for simplicity Avoids calling `set` twice or having to re-raise an exception and implements the routine with a single return expression.

bisect: avoid copying ancestor list for non-merge commits During a bisection, hg needs to compute a list of all ancestors for every candidate commit. This is accomplished via a bottom-up traversal of the set of candidates, during which each revision's ancestor list is populated using the ancestor list of its parent(s). Previously, this involved copying the entire list, which could be very long in if the bisection range was large. To help improve this, we can observe that each candidate commit is visited exactly once, at which point its ancestor list is copied into its children's lists and then dropped. In the case of non-merge commits, a commit's ancestor list consists exactly of its parent's list plus itself. This means that we can trivially reuse the parent's existing list for one of its non-merge children, which avoids copying entirely if that commit is the parent's only child. This makes bisections over linear ranges of commits much faster. During some informal testing in the large publicly-available `mozilla-central` repository, this noticeably sped up bisections over large ranges of history: Setup: $ cd mozilla-central $ hg bisect --reset $ hg bisect --good 0 $ hg log -r tip -T '{rev}\n' 628417 Test: $ time hg bisect --bad tip --noupdate Before: real 3m35.927s user 3m35.553s sys 0m0.319s After: real 1m41.142s user 1m40.810s sys 0m0.285s

rhg-status: add support for narrow clones

rust: add support for hints in error messages This will be used by the narrow support code in the next commit.

rust: add Debug constraint to Matcher trait This makes sure we can easily debug which Matcher we're looking at when using trait objects, and is just generally useful. Effort to make the debugging output nicer has been kept to a minimum, please feel free to improve.

rhg: add sparse support

rhg: add debugrhgsparse command to help figure out bugs in rhg

rhg-status: extract a function for printing pattern file warnings This will be reused for the warnings produced by the sparse file parsing functions.

rust-filepatterns: allow overriding default syntax This will be used when parsing pattern files other than .hgignore like the sparse spec.

rhg: don't fallback if `strip` or `rebase` are activated Neither of these extensions do anything other than add commands, so ignoring them opens up more of the test suite to rhg.

rhg: fallback in `debugdata` if repo has `narrow` Narrow uses ellipsis nodes and debugdata does not understand them yet.

rust-status: expose DifferenceMatcher from Rust to Python

rust-matchers: implement DifferenceMatcher This can be generated by the sparse matcher.

rhg: support "!" syntax for disabling extensions This makes it so that calls in test-log.t do not fall back immediately because of the disabled extension, instead going through the CLI parsing code, which breaks because of invalid UTF-8 in a flag. I *think* clap 3.x+ supports this? I'm not sure, and we have to upgrade the minimum Rust version to use clap 3.x anyway which is out of scope for this series, so let's just kick that can down the road a little bit.

revlog: finer computation of "issnapshot" If the parent had an empty diff, we skip of it to compute a diff against the parent base. This create shorter and simpler chain. However these case could be wrongly detected as snapshot. So we improve the code doing this detection. In practice nobody care as when tried on a copy of mozilla-try and we got the same number of snapshot (1315) in both case. Performance where equivalent.

perf: make perf::bundle compatible down to 5.2 A another small change to make it compatible with a wider set of revision. I did not check compatibility in the python-2 territory yet.

perf: make perf::bundle compatible before 61ba04693d65 A small change to make it compatible with a wider set of revision.

branching: merge stable into default

compare-disco: support for `file` nodes specification This leverage the `nodefromfile` feature in core. This make it possible for callers to no longer pay the subset computation cost (and to make sure the subset is the right one, even when the base repository is different)

compare-disco: move case parsing into its own function This is open the way to the next changeset that will allow to specify a list of heads.

compare-disco: add an option to skip the case If we already know the context, we can save a lot of display space by skipping the case. This also open the way to speedup to the way we specify the subsets. (the code is hacky, but this is a quicky and dirty debug script)

compare-disco: display a header by default This help us to understand the output. (the code is hacky, but this is a quicky and dirty debug script)

compare-disco: prepare for primitive argument parsing We need to be able to configure a couple of things, so lets prepare the code for it. (the code is hacky, but this is a quicky and dirty debug script)

phase-shelve: correct unicode string to honor 'shelve.store=internal' In the case of strip-based shelves, there should be no hidden commit found. That's because shelve.store=internal is necessary but not sufficient to enable phase-based shelves; internal-phase must also be set.

status: include `repo` in template context also for resolved paths The `repo` object needs to be in the templater context when using e.g. `relpath`. It has been missing there since it was the unresolved files were added to the templated output in 07ebb567e8bb.

shelve: remove strip and rely on prior state (issue6735)

shelve: in test for trailing whitespace, strip commit (issue6735)

shelve: demonstrate that the state is different across platforms (issue6735)

phase: rename the requirement for internal-phase (BC) The previous requirements covers both `internal` and `archived` phase. However, the `archived` phase is not ready for usage (while the internal one is mostly ready for years). So we split the archived on in a dedicated requirements (see previous changeset for details) and rename the one for internal-phase. This will avoid older client trying to use the archived phase on `internal` only repositories. Since the requirements stayed experimental since its introduction. It seems fine to drop the previous version.

phase: introduce a dedicated requirement for the `archived` phase See inline documentation for details.

phase: introduce a dedicated function to check for the archived phase The internal-phase is "ready to use" since its introduce. However, some question remains around the `archived` phase. So it seem safer to move them to separated configuration and requirements. This changeset is a first of a small series doing this.

rebase: add boolean config item rebase.store-source This allows to use rebase without recording a rebase_source extra field. This is useful for example to build a mirror converted from another SCM (such as svn) by converting only new revisions, and then incrementally add them to the destination by pulling from the newly converted (unrelated) repo and rebasing the new revisions onto the last old already stored changeset. Without this patch the rebased changesets would always receive some rebase_source that would depend on the particular history of the conversion process, instead of only depending on the original source revisions. This is used to implement a hg mirror repo of SvarDOS (a partially nonfree but completely redistributable DOS distribution) in the scripts at https://hg.pushbx.org/ecm/svardos.scr/ In particular, cre.sh creates an svn mirror, upd.sh recreates an entire hg repo from the svn mirror (which takes too long to do in a regular job), and akt.sh uses hg convert with the config item convert.svn.startrev to incrementally convert only the two most recent revisions already found in the mirror destination plus any possible new revisions. If any are found, the temporary repo's changesets are pulled into the destination (as changesets from an unrelated repository). Then the changesets corresponding to the new revisions are rebased onto the prior final changeset. (Finally, the two remaining duplicates of the prior head and its parent are stripped from the destination repository.) Without this patch, the particular rebase_source extra field would depend on the order and times at which akt.sh was used, instead of only depending on the source repository. In other words, whatever sequence of upd.sh and akt.sh is used at whatever times, it is desired that the final output repositories always match each other exactly.

sort-revset: introduce a `random` variant This new `sort` variant allows to shuffle any revset. It also allow for randomly picking element using `first`.

perf: properly process formatter option in perf::unbundle Otherwise, the options are not understood.

perf: quiet stdout output in perf::unbundle There a lot of repetitive bundle application message we do not care about.

perf: quiet stderr output in perf::unbundle There a lot of repetitive transaction message we do not care about.

bisect: avoid adding irrelevant revisions to bisect state When adding new revisions to the bisect state, it only makes sense to add information about revisions that are under consideration (i.e., those that are topologically between the known good and bad revisions). However, if the user passes in a revset (e.g., '!merge()' to exclude merge commits), hg will resolve the revset first and add all matching revisions to the bisect state (which in this case would likely be the majority of revisions in the repo). To avoid this, revisions should only be added to the bisect state if they are between the good and bad revisions (and therefore relevant to the bisection). -- Here are the results of some performance tests using the `mozilla-central` repo (since it is one of the largest freely-available hg repositories in the wild). These tests compare the performance of a locally-built `hg` before and after application of this series. Note that `--noupdate` is passed to avoid including update time (which should not vary across cases). Setup (run between each test): $ hg bisect --reset $ hg bisect --noupdate --bad 56c3ad4bde5c70714b784ccf15d099e0df0f5bde $ hg bisect --noupdate --good 57426696adaf08298af3027fa77486fee0633b13 Test using a revset that returns a very large number of revisions: $ time hg bisect --noupdate --skip '!merge()' > /dev/null Before: real 0m9.398s user 0m9.233s sys 0m0.120s After: real 0m1.513s user 0m1.425s sys 0m0.052s Test using a revset that is expensive to compute: $ time hg bisect --noupdate --skip 'desc("Bug")' > /dev/null Before: real 0m49.853s user 0m49.580s sys 0m0.243s After: real 0m4.120s user 0m4.036s sys 0m0.048s

bisect: limit ancestors to revs topologically between good and bad revs Previously, when constructing its dict of revisions to their ancestors, bisect would populate the dict with ALL of the descendents of the good set, which is a bit silly because it is impossible for a revision that is a descendent of the minimum known bad revision to be the first bad rev. Instead it makes more sense to limit the revisions to just those topologically between the good and bad.

bisect: bypass changectx when translating revs to nodes When resolving the revset given by the user into node hashes, use the changelog to perform the translation rather than the repo object. This avoids the overhead of constructing a changectx which is immediately discarded.

rhg: make [rhg status -v] work when it needs no extra output Add support for verbose [status] when no extra output is actually needed. This makes it so that [rhg status] is actually useful when [tweakdefaults] is true. (since tweakdefaults implies verbose status)

perf: introduce a benchmark for delta-find That part is responsible of serious slowdown in some `hg pull/unbundle` case. So lets add a way to benchmark it.

debug-delta-find: introduce a quiet mode In quiet mode, we only print the summary of the search and skip the individual steps.

phase-shelve: also capture the state of shelve prior to unshelve

phase-shelve: Add test for shelve technique config

phase-shelve: Implement a 'shelve.store' experimental config Accepts "internal" or "strip", indicating how the shelved changes are stored. Defaults to "internal", retaining compatibility for repos with "internal-phase" already enabled.

phase-shelve: Extract function for _target_phase

phase-shelve: expand the tests to capture use-cases supported

phase-shelve: honor and prefer obs shelves for existence and modified time

phase-shelve: read patch details from a (possibly internal) node in the repo

phase-shelve: Extract function for _optimized_match for re-use

obsstore: break the repo → obstore → repo loop This should help the garbage collector to do its job. On repository with many markers, the memory pressure from the obsstore can get quite serious.

relnotes: mention chg behavior change when given --cwd

dispatch: change cwd when loading local config Previously, the `_getlocal` function would not correctly load the repo config when given a relative `rpath` and an alternate cwd via the `wd` parameter. Normally when `--cwd` is specified, hg changes to the given directory before attempting to load the local config (and therefore does not specify a `wd`). The only time the function is called with `wd` set is when hg is running as a command server (e.g., with chg), in which case each forked worker process will attempt to configure itself via `_getlocal` before responding to the client. When given a relative repo path, the worker fails to load the repo config, detects a config mismatch with the client, and enters a redirect/respawn loop. To fix this, we can simply change to the desired working directory during config loading. (Note that simply concatenating `wd` and `rpath` won't work in all cases. The repo path could be something more complicated than a simple relative path, such as a `union:` repo.)

contrib: add support for rhel9

packagelib: use python3 by default

tests: work around libmagic bug in svn subrepo tests libmagic 5.40 introduced a bug [1] wherein ASCII text files with fewer than 3 distinct character values would be reported as binary data rather than as text. This bug was later fixed in version 5.41 [2]. SVN uses libmagic to determine the MIME type of added files with missing or unknown extensions [3]. This results in test failures on systems with libmagic 5.40 installed: $ echo a > a $ svn add a - A a + A (bin) a A simple workaround is to change the test file's content to include 3 distinct ASCII values (including the terminating newline). [1] https://bugs.astron.com/view.php?id=180 [2] https://bugs.astron.com/view.php?id=261 [3] https://svnbook.red-bean.com/en/1.8/svn.advanced.props.html#idm2649

revlog: make _partialmatch fail fast on almost-hex inputs Before this change, resolving a revision like [0123456789^] on a large repo can take multiple seconds because: - hg does not realize this is a revset, so it tries various things, including _partialmatch(b"0123456789^") - after the rust lookup fails, it falls back to pure hg - pure hg takes all-but-last chars and converts them to binary, which *succeeds*, so it does the expensive part.

perf-unbundle: add a perf command to time the unbundle operation Check documentation for details.

perf-bundle: accept --type argument

perf-bundle: accept --rev arguments This is fairly standard nowaday.

perf-bundle: add a new command to benchmark bundle creation time

bundle: introduce a --exact option I have been wanting this options for a long time.

bundlespec: add documentation about existing option We have some documentation, lets make it complete.

setup: use the full executable manifest from `python.exe` The manifest embedded by the build process (before the string here is added) already accounts for the `<requestedExecutionLevel level="asInvoker" ...>` setting. (Note that the PyOxidizer build is missing this, so it will likely trigger the UAC escalation prompt on each run.) However, using `mt.exe` to merge the fragment with what is already in the manifest seems to strip all whitespace, making it unreadable. Since Mercurial can be run via `python.exe`, it makes sense that we would have the same manifest settings (like the supported OS list), though I'm unaware of any functionality this enables. It also has the nice effect of making the content readable from a resource editor. The manifest comes from python 3.9.12. Note that this seems to strip the `<?xml ... ?>` declaration when viewed with ResourceHacker 5.1.7, but this was also the state of things with the previous commit, and `mt.exe "-inputresource:hg.exe;#1" -out:extracted` does contain the declaration and the BOM in both cases. No idea why this differs from other executables.

setup: unconditionally enable the `long-paths-support` option on Windows I don't see anything talking about why this was experimental in the first place, but maybe it was concern about the level of python2 support for it. But now, both `python.exe` and the PyOxidizer build of `hg.exe` have a manifest that enables it, so leaving it off would mean some Mercurial installations could operate on a repo with long paths, and others couldn't. Note that only the wide character functions (XxxW) will have the length restriction lifted. Sadly, distutils applies `/MANIFEST:EMBED` to the linker in a way that can't easily be turned off, so we can't use `/MANIFESTFILE` with `extra_preargs` on `link_executable`. Fortunately, the compiler object provides a path to the `mt.exe` it found during initialization, because the previous incarnation seems to have assumed it is being run within an activated Visual Studio environment. That causes MSYS builds to fail, and probably would have broke the CI environment.

setup: stop shadowing the builtin `dir` symbol I hit this when debugging what's available on the compiler.

subrepo: avoid opening console window for non-native subrepos on Windows Prevent annoying command prompt windows popping up when using TortoiseHG with Git and SVN subrepos by passing creationflags=subprocess.CREATE_NO_WINDOW to subprocess.Popen.

ci: bump pytype to 2022.03.29 This is as far as we can go without running into issues with the vendored `attr` package. I tried updating that to the latest, and not only did it not fix the issue, but test-util.py failed due to some poking at `attr` internals that apparently is no longer valid. The `libcst` package is now pinned to what I have locally because trying to install the latest (0.4.7) complains that it can't find the Rust compiler. We should probably use a requirements file instead (and/or figure out why it can't find the Rust compiler), but I don't feel like dealing with another side quest.

typing: suppress a few attribute errors in url.py These are newly detected by pytype 2022.03.21. Not sure what is going on here- `realhostport` and `headers` are added outside of the constructor, so that makes sense. But PyCharm also thinks the private methods don't exist, though when clicking through the class hierarchy, it shows in the py3.9 source code.

typing: suppress a few pyi-errors with more recent pytype Not sure what's going on here, but these were flagged with pytype 2022.03.21. We can't update to something much more recent, because newer versions complain about various `attr` uses.

sslutil: another use proper attribute to select python 3.7+ The previous attribute was python 3.6+, but guarded a python 3.7+ block Using the correct attribute avoids: + File "/tmp/hgtests.bc0_uk2d/install/lib/python/mercurial/sslutil.py", line 577, in wrapserversocket + sslcontext.minimum_version = ssl.TLSVersion.TLSv1_1 + AttributeError: module 'ssl' has no attribute 'TLSVersion'