commitctx: gather more code dealing with copy-in-extra Now that we have a function that deal with the copy-in-extra special case, we can gather more code meant to deal with this special case. Making the rest of the code simpler.

commitctx: rename files to touched in a couple of place This is a clearer word that we will use increasingly over this series. It also make the old variable name available to another usage :-)

commitctx: extract copy information encoding into extra into commit.py The encoding of copy information into extra has multiple subcases and become quite complicated (eg: empty list can be explicitly or implicitly stored for example). In addition, it is niche experimental feature since as it affect the hash, it is only suitable for user who don't mercurial for storage server side (ie: Google). Having this complexity part of the changelog will get in the way of further cleanup. We could have to either move more of that logic into the changelog or to move or extract more of the logic at the higher level. We take the second approach and start gather logic in dedicated function in commit.py.

commitctx: create the new extra dict on its own line A trivial move to make the next changeset easier to read.

commitctx: explicitly pass `manifest` to _commit_manifest As pointed out by Yuya Nishihara.

phabricator: unconditionally pop `test_vcr` to fix debugcallconduit 11592ce6a711 / D8525 accidentally broke debugcallconduit in non-test scenarios because it stopped popping `test_vcr` from `kwargs` unconditionally, so when `--test-vcr` isn't set the empty string still gets passed down as the value of `test_vcr` in `kwargs`. However unlike all the other commands debugcallconduit doesn't have an `**opts` argument to receive it, so it aborts because of invalid arguments. Differential Revision: https://phab.mercurial-scm.org/D8852

phabricator: demonstrate debugcallconduit being broken without --test-vcr This was accidentally broken by 11592ce6a711 / D8525 Differential Revision: https://phab.mercurial-scm.org/D8851

cleanup: fix bad formatting of state.py from D8811 `test-check-format.t` was failing for me. Do I just have the wrong version? Differential Revision: https://phab.mercurial-scm.org/D8841

dirstate: restore original estimation and update comment The former comment didn't reflect the content of the dirstate entries, the two nodes are a fixed header in the file and not per-entry. Split the documented entry into the path part and the fixed header. The heuristic itself is still valid, e.g. for the NetBSD src tree a maximum path size of 142 and an average of 49, resulting in 66 bytes per entry on average. Differential Revision: https://phab.mercurial-scm.org/D8850

merge stable in default

dirstate: revert change to Rust binding The Rust binding uses its own class and is not derived from the regular implementation. As such, it didn't get _nodelen. Differential Revision: https://phab.mercurial-scm.org/D8849

commitctx: extract all the file preparation logic in a new function Before we actually start to create a new commit we have a large block of logic that do the necessary file and manifest commit and that determine which files are been affected by the commit (and how). This is a complex process on its own. It return a "simple" output that can be fed to the next step. The output itself is not that simple as we return a lot of individual items (files, added, removed, ...). My next step (and actual goal for this cleanup) will be to simplify the return by returning a richer object that will be more suited for the variation of data we want to store. After this changeset the `commitctx` is a collection of smaller function with limited scope. The largest one is still `_filecommit` without about 100 lines of code.

commitctx: gather more preparation code within the lock context This is a small change that exist mostly for clarification. I am about to move a large amount of code in its own function. having all that code next to each other will make the next changeset clearer.

commitctx: move a special case about files earlier Same logic as a changeset a bit earlier, the `writefilecopymeta` section is more a post processing details so we move the conditional about `files` value closer to the rest of the code computing `files` value.

commitctx: extract all the manual logic to process the files That branch of the if is significantly more complicated than the other two. Moving it to its own function make it simple to keep the scope limited and to read to the higher level function.

commitctx: treat `filesadded` more like `filesremoved` Accumulating the filename in a list will have a negligible cost and deal with the list of added files like the other ones will make is code cleaning simpler. The two variable with very close name is not great, but my plan is to split most of the code in a separated function which will make the "problem" go away by itself.

commitctx: move `writechangesetcopy` business at the end a code section This code is to handle a specific subcase so we move it a the end. This allow to gather the rest of the "core" code closer to the related logic.

commitctx: move copy meta config reading in a dedicated function The logic is non trivial, make it contained in a function is clearer. It also unlock easy re-use of that logic without having the pass the value around.

commitctx: no longer use the `writecopiesto` variable in the function The `writefilecopymeta` variable already carry the same information, so we can use `writefilecopymeta` in the one conditional where `writecopiesto` was used.

commitctx: extract the function that commit a new manifest The logic is large enough and isolated enough to be extracted, this reduce the size of the main function, making it simpler to follow.

commitctx: stop using weakref proxy for transaction This weakref proxy was introduced in 2007 by 30d4d8985dd8. If I understand it correctly, the logic at that time was relying on the transaction destructor, triggered at garbage collection time to rollback failed transaction. passing the object to sub function directly mean it would live in the function scope and be trapped in the traceback on exception, leading to the transaction rollback too late in some case. Modern transaction usage use explicit opening and closing of transaction and no longer rely on some internal reference counting details. So this weakref proxy is no longer necessary. Absolutely no test are affected when we drop it.

commitctx: document the None return for "touched" value

tests: add tests trying to use dir as style, as %include, and as __base__ Differential Revision: https://phab.mercurial-scm.org/D8800

tests: fix mistaken copy&paste from commit 4489e9a22763 Differential Revision: https://phab.mercurial-scm.org/D8802

manifest: kill one more instance of the old merge hash hack Differential Revision: https://phab.mercurial-scm.org/D8766

dirstate: isolate node len dependency for the pure version When switching to a 256bit hash function, this still needs adjustment, but concentrates the change in one place. Differential Revision: https://phab.mercurial-scm.org/D8815

fix: update documentation to reflect preference for --source over --rev I should have updated the documentation in 5205b46bd887 (fix: add a -s option to format a revision and its descendants, 2020-03-13) and/or a6ef1e8e2f6d (fix: mark -r as advanced, 2020-03-13)... Differential Revision: https://phab.mercurial-scm.org/D8808

tests: make check-py3-compat.py actually load the specified files correctly For most uses, this change is essentially a no-op, as this script is generally only run by test-check-py3-compat.t, which will already put `$TESTDIR/..` in `$PYTHONPATH`. When running outside of tests, however, `$PYTHONPATH` is likely not set, causing check-py3-compat.py to parse the file from the repo, but then import the installed version, and raise any errors about the installed version, not the one currently in the repo. Additionally, this helps users (like me) who have a strange set up where their home directory (and thus their hg repos) happen to be in a subdirectory of sys.prefix (which is /usr on my system). Since the '.' entry added to sys.path takes precedence over the absolute path of `$TESTDIR/..` in `$PYTHONPATH`, the path to the modules that it imports (and that show up in any stack trace) are *relative*, meaning that we don't detect them as starting with `sys.prefix`. Sample non-test invocation, and the difference this change makes (the path for 'error at <path>:<line>' is correct now):: Before: ``` $ python3 contrib/check-py3-compat.py mercurial/win*.py mercurial/win32.py: error importing: <ValueError> _type_ 'v' not supported (error at check-py3-compat.py:65) mercurial/windows.py: error importing: <ModuleNotFoundError> No module named 'msvcrt' (error at check-py3-compat.py:65) ``` After: ``` $ python3 contrib/check-py3-compat.py mercurial/win*.py mercurial/win32.py: error importing: <ValueError> _type_ 'v' not supported (error at win32.py:11) mercurial/windows.py: error importing: <ModuleNotFoundError> No module named 'msvcrt' (error at windows.py:12) ``` Differential Revision: https://phab.mercurial-scm.org/D8814

tests: virtualenv is only used on py2, rename and conditionalize If I have I have the Debian `python3-virtualenv` package installed on my machine, the import succeeds but then I receive an AttributeError because the package is essentially completely different between py2 and py3, and test-hghave fails. Differential Revision: https://phab.mercurial-scm.org/D8812

morestatus: mention --stop even if not using --verbose Differential Revision: https://phab.mercurial-scm.org/D8811

templater: handle None returned from templatedir() My recent 91aa9bba3dc9 (templater: make templatepaths() return a single path, or None, 2020-07-21) didn't account for the fact that `templatedir()` returns `None` in frozen binaries. That is ironic, since the reason I'm working on this is to add support for built-in mapfiles in frozen binaries. This patch updates the callers to handle the `None` case. It's somewhat ugly, but I will have to revisit this soon anyway, since my goal is to make all callers handle that case by trying to read the map file using the resources API instead. Differential Revision: https://phab.mercurial-scm.org/D8810

templater: do search for include of unqualified builtin outside of config code Commit 081b08e4ea13 (templater: look for mapfiles in template paths, 2015-05-15) added support for using things like `%include map-cmdline.default` to include built-in map files without using a valid path to them. This patch rewrites that support by moving it into `_readmapfile()` so it can later be adapted for reading from a non-file resource. Differential Revision: https://phab.mercurial-scm.org/D8792

templater: switch to lower-level config.parse() in _readmapfile() I hope to modify this code to also work with resources loaded from memory (for PyOxidizer support). For that, we'll need to handle the lookup of relative `%include` path (not joined with the base directory of the containing file). This patch prepares for that by using `config.parse()` instead of `config.read()`, since the latter expects a file in the file system. As it happens, this change also lets us clean up the `config` class to not need the `_includepaths` field. That will happen next. Differential Revision: https://phab.mercurial-scm.org/D8791

config: pass both relative and absolute paths to `include` callback The `include` callback is responsible for loading configs from `%include` statements. The callback currently gets passed the absolute path [1] to the config to read. That is created by joining the dirname of the file that contains the `%include` statement. For PyOxidizer support, I'm trying to reduce dependence on paths. This patch helps with that by passing the relative path found in the `%include` statement (but with username expansion, etc.) to the `include` callback. It also turns out that the existing callers can easily adapt to using the relative path. Coming patches will clean that up and then we'll remove the absolute path from the callback. [1] The "absolute path" bit is a bit of a lie -- it's going to be an absolute path if the path that was passed into `config.parse()` was absolute. Differential Revision: https://phab.mercurial-scm.org/D8790

tests: add test for bad template %include and __base__ It doesn't seem like we had any tests for `%include non-existent` and `__base__ = non-existent`. The latter raises an error while the former ignores the include. We should probably also make the former an error. Differential Revision: https://phab.mercurial-scm.org/D8798

tests: add test for unqualified include of a built-in template map The feature was added in 081b08e4ea13 (templater: look for mapfiles in template paths, 2015-05-15), but no test case was added. Differential Revision: https://phab.mercurial-scm.org/D8797

merge with stable

commitctx: extract _filecommit too This function is exclusively used in `commitctx`. So we should extract it too for consistency and to reduce the `localrepo` bloat. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. Differential Revision: https://phab.mercurial-scm.org/D8710

commitctx: extract the function in a dedicated module the function have few callers (< 15) is quite long a mostly independent from the repository itself. It seems like a good candidate to reduce the bloatness of the localrepository class. Extracting it will help us cleaning the code up and splitting it into more reasonable-size function. We don't use a copy trick because the amount of code extract is quite small (<5%) and the de-indent means every single line change anyway. So this is not deemed valuable to do so. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. Differential Revision: https://phab.mercurial-scm.org/D8709

commitctx: document a fast path in _filecommit This block cut off a lot of logic, documenting the why and how seems useful to future reader. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8700

hgweb: simplify now that we always have a single path Both `templatedir()` and `web.templatepath` are now always a single path (or None). Differential Revision: https://phab.mercurial-scm.org/D8788

templater: simplify stylemap() now that templatedir() returns a single path Differential Revision: https://phab.mercurial-scm.org/D8787

templater: make templatepaths() return a single path, or None The function returns either a singleton list or an empty list, so it makes more sense to return a value or None. The plural in the name also doesn't make sense, but `templatepath()` is already taken, so I renamed it to `templatedir()` instead. Differential Revision: https://phab.mercurial-scm.org/D8786

templater: simplify templatepaths() to avoid iterating a singleton list The function iterates over a hard-coded list of one element since d844e220792a (templater: don't search randomly for templates - trust util.datapath, 2014-09-28). Differential Revision: https://phab.mercurial-scm.org/D8785

commitctx: move a tiny else clause above the very long one This is simple to have all the simple case unfold before the 100+ line one. Otherwise it is hard to relate the `else` to the initial conditionnal. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8708

commitctx: consider removed as touched This achieve the same result with clearer code. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8707

commitctx: rename "changed" to touched The variable contains content that are both added and modified. "changed" could be confused with "modified" only, so we pick a less ambiguous naming that will help with more unification. For example, it would make sense to shove the "removed" item in there since this is how the variable is used. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8706

commitctx: more filesremoved assignment closer to removed computation A small change that makes the code flow clearer. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8704

commitctx: compute files added from _filecommit returns When possible, lets avoid recomputing the same information again. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8703

commitctx: return "touched" status from _filecommit Instead of mutating a list passed in argument, we simply return the information from the `_filecommit` function. This make for a cleaner API and allow for richer information to be returned. That richer information will be used in the next commit to avoid duplicated computation. This is part of a larger refactoring/cleanup of the commitctx code to clarify and augment the logic gathering metadata useful for copy tracing. The current code is a tad too long and entangled to make such update easy. We start with easy and small cleanup. Differential Revision: https://phab.mercurial-scm.org/D8702

tests: fix test-check-module-imports.t broken by D9150 Differential Revision: https://phab.mercurial-scm.org/D9177

tests: make test-convert-git.t work across changed default branch name Recent git versions default to "main" instead of "master", which breaks our test. Differential Revision: https://phab.mercurial-scm.org/D9174

tests: skip doctests if not running from a hg repo Differential Revision: https://phab.mercurial-scm.org/D9150

Added signature for changeset 07731064ac41

Added tag 5.5.2 for changeset 07731064ac41

hgdemandimport: exclude more sqlalchemy modules We could potentially exclude the entire sqlalchemy library.

repoview: don't crash if mergestate points to non-existent node Differential Revision: https://phab.mercurial-scm.org/D9078

tests: demonstrate crash caused by pinning of non-existent mergestate node If the mergestate somehow contains a reference to a non-existent node, it causes most commands to crash since aaeccdb6e654 (repoview: pin revisions for `local` and `other` when a merge is active, 2020-09-04). This patch adds a test showing that. Differential Revision: https://phab.mercurial-scm.org/D9077

strip: with --keep, consider all revs "removed" from the wcp (issue6270) The current code was blink to change from other branches when stripping merges that are ancestors of the working copy parents.

test-strip: display more information highlight buggy behavior When using `hg strip --keep` on a set of changeset that contains a merge, the set of file considered when rebuilding the dirstate is missing files updated by changeset that are not directly inside the `new-parent::old-parent` range. We start with updating the test with new output highlighting the issue. This issue was spotted because that dirstate inconsistency made the test flaky. The new command make the test less flaky (but still wrong).