pyoxidizer: allow extensions to be loaded from the file system It seems that setting this config is all that's needed to be able to load extensions from the file system (which we clearly want). Thanks for making this work, Gregory Szorc!. Differential Revision: https://phab.mercurial-scm.org/D8122

graft: always allow hg graft --base . (issue6248) `hg graft --base . -r abc` is rejected before this change with a "nothing to merge" error, if `abc` does not descend from `.`. This looks like an artifact of the implementation rather than intended behavior. It makes perfect sense to apply the diff between `.` and `abc` to the working copy (i.e. degenerate into `hg revert`), regardless of what `abc` is. Differential Revision: https://phab.mercurial-scm.org/D8127

revlog-compression: update the config to be a list format.revlog-compression is now a list of engine, the first supported one is to be used. Doing this have several benefits: 1) this is fully backward compatible, config using a single entry will be read as a single item list, not changing any behavior. 2) This open the way to use zstd by default without impacting platform were it is not available. This will be done in a later changesets. Using zstd provide a significant performance boost explained in : bb271ec2fbfb. However zstd is not available in some cases, A notable example is the `--pure` version of Mercurial which doesn't come with zstd support. Differential Revision: https://phab.mercurial-scm.org/D8148

remotefilelog: add 'changelog' arg to shallowcg1packer.generate (issue6269) This cause traceback on widening using narrow extension when remotefilelog is enabled. Differential Revision: https://phab.mercurial-scm.org/D8134

drawdag: abide by new createmarkers() API The `obsolete.createmarkers()` API was changed in 6335c0de80fa (obsolete: allow multiple predecessors in createmarkers, 2018-09-22) to prefer its precursors input to be a tuple instead of a single precursor. Let's fix `drawdag.py` to comply. Differential Revision: https://phab.mercurial-scm.org/D8149

lfutil: provide a hint if the largefiles/lfs cache path cannot be determined A coworker hit this error using an LFS repo in a stripped down environment, and didn't know how to resolve it. The final conditional is a bit fast and loose, but there is currently no 'posix' test in hghave, and it doesn't seem like it's worth adding for this since I think Windows is the only non-POSIX platform we run tests on. Differential Revision: https://phab.mercurial-scm.org/D8145

setup: exclude the __index__ module from itself when generating This module is generated on Windows to hold all of the extension names and the help summaries, so that they are discoverable inside the py2exe zipfile. The problem is this file is generated by dumping the disabled list, and that list comes from walking the filesystem. So once an install from source into a virtualenv created this module, then next build from source from that virtualenv would also see __index__.py in the filesystem, and include it. Clearly that's wrong because this isn't a real extension, so just filter it from the list when generating it. The Mercurial installer was unaffected by this, but the TortoiseHg package was. In the final package, `hg help -v extensions` and the panel of extensions both showed it. Differential Revision: https://phab.mercurial-scm.org/D8142

tests: stabilize test-amend.t on Windows If $TESTTMP isn't quoted in this context, it ends up like `C:Temphgtests.pikkoxchild1test-amend.t-obsstore-off`. Differential Revision: https://phab.mercurial-scm.org/D8144

tests: replace truncate(1) with inline python MSYS doesn't have truncate(1) installed by default, and FreeBSD looked unhappy with the arguments provided. Differential Revision: https://phab.mercurial-scm.org/D8147

tests: stabilize test-rename-merge2.t on Windows I have no idea why, but this shifted in b4057d001760. Differential Revision: https://phab.mercurial-scm.org/D8146

merge with stable

py3: fix EOL detection in commandserver.channeledinput This breaks TortoiseHg's email preview which sends b'\n' while readline request is issued and the loop never ends. Spotted by Matt Harbison.

bookmarks: prevent pushes of divergent bookmarks (foo@remote) Before this change, such bookmarks are write-only: a client can push them but not pull/read them. And because these bookmark can't be read, even pushes are limited (for instance trying to delete such a bookmark fails with a vanilla client because the client thinks the bookmark is neither on the local nor the remote). This change makes the server refuses such bookmarks, and for earlier errors, makes the client refuse to send them. I think the change of behavior is acceptable because I think this is a bug in push/pull, and I don't think we change the behavior of `hg unbundle`, because it doesn't seem that `hg bundle` ever store bookmarks (and even if it did, it would seem weird anyway to try to send divergent bookmarks). Differential Revision: https://phab.mercurial-scm.org/D8117

bookmarks: refactor in preparation for next commit Differential Revision: https://phab.mercurial-scm.org/D8116

bookmarks: avoid traceback when two pushes race to delete the same bookmark `hg push -f -B remote-only-bookmark` can raise server-side in `bookmarks._del` (specifically in `self._refmap.pop(mark)`), if the remote-only bookmark got deleted concurrently. Fix this by simply not deleting the non-existent bookmark in that case. For avoidance of doubt, refusing to delete a bookmark that doesn't exist when the push starts is taking care of elsewhere; no change of behavior there. Differential Revision: https://phab.mercurial-scm.org/D8124

relnotes: add entry about previous `hg recover` change Differential Revision: https://phab.mercurial-scm.org/D8123

darwin: add another preemptive gui() call when using chg Changeset a89381e04c58 added this gui() call before background forks, and Google's extensions do background forks on essentially every invocation for logging purposes. The crash is reliably (though not 100%) reproducible without this change when running `HGPLAIN=1 chg status` in one of our repos. With this fix, I haven't been able to trigger the crash anymore. Differential Revision: https://phab.mercurial-scm.org/D8141

copy: add experimental support for marking committed copies The simplest way I'm aware of to mark a file as copied/moved after committing is this: hg uncommit --keep <src> <dest> # <src> needed for move, but not copy hg mv --after <src> <dest> hg amend This patch teaches `hg copy` a `--at-rev` argument to simplify that into: hg copy --after --at-rev . <src> <dest> In addition to being simpler, it doesn't touch the working copy, so it can easily be used even if the destination file has been modified in the working copy. Differential Revision: https://phab.mercurial-scm.org/D8035

copy: move argument validation a little earlier Argument validation is usually done early and I will want it done before some code that I'm about to add. Differential Revision: https://phab.mercurial-scm.org/D8033

copy: add experimetal support for unmarking committed copies The simplest way I'm aware of to unmark a file as copied after committing is this: hg uncommit --keep <dest> hg forget <dest> hg add <dest> hg amend This patch teaches `hg copy --forget` a `-r` argument to simplify that into: hg copy --forget --at-rev . <dest> In addition to being simpler, it doesn't touch the working copy, so it can easily be used even if the destination file has been modified in the working copy. I'll teach `hg copy` without `--forget` to work with `--at-rev` next. Differential Revision: https://phab.mercurial-scm.org/D8030

copy: add option to unmark file as copied To unmark a file as copied, the user currently has to do this: hg forget <dest> hg add <dest> The new command simplifies that to: hg copy --forget <dest> That's not a very big improvement, but I'm planning to also teach `hg copy [--forget]` a `--at-rev` argument for marking/unmarking copies after commit (usually with `--at-rev .`). Differential Revision: https://phab.mercurial-scm.org/D8029

nodemap: introduce an option to use mmap to read the nodemap mapping The performance and memory benefit is much greater if we don't have to copy all the data in memory for each information. So we introduce an option (on by default) to read the data using mmap. This changeset is the last one definition the API for index support nodemap data. (they have to be able to use the mmaping). Below are some benchmark comparing the best we currently have in 5.3 with the final step of this series (using the persistent nodemap implementation in Rust). The benchmark run `hg perfindex` with various revset and the following variants: Before: * do not use the persistent nodemap * use the CPython implementation of the index for nodemap * use mmapping of the changelog index After: * use the MixedIndex Rust code, with the NodeTree object for nodemap access (still in review) * use the persistent nodemap data from disk * access the persistent nodemap data through mmap * use mmapping of the changelog index The persistent nodemap greatly speed up most operation on very large repositories. Some of the previously very fast lookup end up a bit slower because the persistent nodemap has to be setup. However the absolute slowdown is very small and won't matters in the big picture. Here are some numbers (in seconds) for the reference copy of mozilla-try: Revset Before After abs-change speedup -10000: 0.004622 0.005532 0.000910 × 0.83 -10: 0.000050 0.000132 0.000082 × 0.37 tip 0.000052 0.000085 0.000033 × 0.61 0 + (-10000:) 0.028222 0.005337 -0.022885 × 5.29 0 0.023521 0.000084 -0.023437 × 280.01 (-10000:) + 0 0.235539 0.005308 -0.230231 × 44.37 (-10:) + :9 0.232883 0.000180 -0.232703 ×1293.79 (-10000:) + (:99) 0.238735 0.005358 -0.233377 × 44.55 :99 + (-10000:) 0.317942 0.005593 -0.312349 × 56.84 :9 + (-10:) 0.313372 0.000179 -0.313193 ×1750.68 :9 0.316450 0.000143 -0.316307 ×2212.93 On smaller repositories, the cost of nodemap related operation is not as big, so the win is much more modest. Yet it helps shaving a handful of millisecond here and there. Here are some numbers (in seconds) for the reference copy of mercurial: Revset Before After abs-change speedup -10: 0.000065 0.000097 0.000032 × 0.67 tip 0.000063 0.000078 0.000015 × 0.80 0 0.000561 0.000079 -0.000482 × 7.10 -10000: 0.004609 0.003648 -0.000961 × 1.26 0 + (-10000:) 0.005023 0.003715 -0.001307 × 1.35 (-10:) + :9 0.002187 0.000108 -0.002079 ×20.25 (-10000:) + 0 0.006252 0.003716 -0.002536 × 1.68 (-10000:) + (:99) 0.006367 0.003707 -0.002660 × 1.71 :9 + (-10:) 0.003846 0.000110 -0.003736 ×34.96 :9 0.003854 0.000099 -0.003755 ×38.92 :99 + (-10000:) 0.007644 0.003778 -0.003866 × 2.02 Differential Revision: https://phab.mercurial-scm.org/D7894

rust-dirstatemap: directly return `non_normal` and `other_entries` This cleans up the interface which I previously thought needed to be uglier than in reality. No performance difference, simple refactoring. Differential Revision: https://phab.mercurial-scm.org/D8121

copy: rename `wctx` to `ctx` since it will not necessarily be working copy Differential Revision: https://phab.mercurial-scm.org/D8032

copy: rewrite walkpat() to depend less on dirstate I want to add a `hg cp/mv -r <rev>` option to mark files as copied/moved in an existing commit (amending that commit). The code needs to not depend on the dirstate for that. Differential Revision: https://phab.mercurial-scm.org/D8031

merge with stable

pathutil: resurrect comment about path auditing order It was removed at 51c86c6167c1, but expensive symlink traversal isn't the only reason we should walk path components from the root.

rust-dirstatemap: remove additional lookup in dirstate.matches We use the same trick as the Python implementation Differential Revision: https://phab.mercurial-scm.org/D7119

rust-nodemap: insert method In this implementation, we are in direct competition with the C version: this Rust version will have a clear startup advantage because it will read the data from disk, but the insertion happens all in memory for both. Differential Revision: https://phab.mercurial-scm.org/D7795

recover: don't verify by default The reason is: - it's not that hard to trigger interrupted transactions: just run out of disk space - it takes forever to verify on large repos. Before --no-verify, I told people to C-c hg recover when the progress bar showed up. Now I tell them to pass --no-verify. - I don't remember a single case where the verification step was useful This is technically a change of behavior. Perhaps this would be better suited for tweakdefaults? Differential Revision: https://phab.mercurial-scm.org/D7972

context: use manifest.find() instead of two separate calls I noticed this while debugging an extension that's implementing the manifest interface. Always nice to save a function call. Differential Revision: https://phab.mercurial-scm.org/D8109

rust-matchers: implement `visit_children_set` for `FileMatcher` As per the removed inline comment, this will become useful in a future patch in this series as the `IncludeMatcher` is introduced. Differential Revision: https://phab.mercurial-scm.org/D7914