rust: Make the hg-cpython crate default to Python 3 This default is used when running `cargo` manually such as for `cargo test`. `setup.py` and `Makefile` both configure the Python major version explicitly. Differential Revision: https://phab.mercurial-scm.org/D11618

packaging: update the certifi dependency Not sure if this helps with recent certificate issues[1], but we might as well keep this modern. Note that certifi no longer claims py2 support, and there's a PR to add it back in[2]. Py2 support was dropped in 2020.04.05.2 (which predates what's being updated here). None of the *.py files have changed since the 2020.6.20 release, and I was able to call `certifi.where()` in both a virtualenv and with `hg debugshell` from an MSI install with this version, so we shouldn't be any worse off than before. [1] https://foss.heptapod.net/mercurial/tortoisehg/thg/-/issues/5745 [2] https://github.com/certifi/python-certifi/pull/154 Differential Revision: https://phab.mercurial-scm.org/D11609

setup: stop packaging python3.dll and python3X.dll in the wheel distribution Now that exewrapper is smart enough to find the DLLs it needs without help from the build script, backout ed286d150aa8 and 2960b7fac966. Note that this will require deleting the build/lib.win-amd64-3.X directory to actually remove it from the final wheel. Differential Revision: https://phab.mercurial-scm.org/D11455

exewrapper: find the proper python3X.dll in the registry Previously, we relied on the default library lookup[1], which for us is essentially to look on `PATH`. That has issues- the Python installations are not necessarily on `PATH`, so I started copying the DLLs locally in 2960b7fac966 and ed286d150aa8 during the build to work around that. However, it's been discovered that causes `python3.dll` and `python3X.dll` to get slipped into the wheel that gets distributed on PyPI. Additionally, Mercurial would fail to run in a venv if the Python environment that created it isn't on `PATH`, because venv creation doesn't copy the DLLs locally. The logic here is inspired by the `py.exe` launcher[2], though this is simpler because we don't care about the architecture- if this is a 32 bit process running on Win64, the registry reflection will redirect to where the 32 bit Python process wrote its keys. A nice unintended side effect is to also make venvs that don't have their root Python on `PATH` work without all of the code required to read `pyvenv.cfg`[3]. I don't see any reasonable way to create a venv without Python being installed (other than maybe building Python from source?), so punt on trying to read that file for now and save a bunch of string manipulation code. I somehow managed to corrupt my Windows user profile, and that makes the Microsoft Store python not run (even loading the DLL gives an access error), so I'm giving priority to both global and user specific python.org installations. Loading python3.dll is new, but when I went down the rabbit hole of implementing `pyvenv.cfg` support, I saw a comment[4] that led me to think we could have trouble if we don't. The comment in ed286d150aa8 confirms this, so we should probably bail out completely if Python3 can't be loaded from the registry, rather than getting something random on `PATH`. But I'll leave that for the default branch. [1] https://docs.microsoft.com/en-us/windows/win32/Dlls/dynamic-link-library-search-order#standard-search-order-for-desktop-applications [2] https://github.com/python/cpython/blob/adcd2205565f91c6719f4141ab4e1da6d7086126/PC/launcher.c#L249 [3] https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/PC/getpathp.c#L707 [4] https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/PC/getpathp.c#L1098 Differential Revision: https://phab.mercurial-scm.org/D11454

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280

fix: add test to demonstrate how many times tools are executed The current implementation wastes a lot of effort invoking fixer tools more than once on the same file name+content. It is good to track changes in this behavior. Differential Revision: https://phab.mercurial-scm.org/D11279

rhg: add ui.plain() and check it before showing relative paths in status Adds a very basic replica of `ui.plain()`. Differential Revision: https://phab.mercurial-scm.org/D10912

rhg: add relative paths support in `rhg status` Differential Revision: https://phab.mercurial-scm.org/D11614

rhg: refactor function to relativize paths in utils Commands like `files`, `status` supports printing relative paths. Hence we need to re-use this code in other places too. So let's take this out from `rhg files` into a utility function. Next patch will make `rhg status` use it. Differential Revision: https://phab.mercurial-scm.org/D11613

hg: let extensions call the func without populating opts keys This change is to help extensions by not forcing them to populate with opts[b'bundle'] and opts[b'force'] when calling hg.incoming(...) Differential Revision: https://phab.mercurial-scm.org/D11619

rhg: in rhg cat cli, fix the long name of the --rev flag Also tweak the help for the anonymous argument. Differential Revision: https://phab.mercurial-scm.org/D11617

rhg: fallback if tweakdefaults or statuscopies is enabled with status `rhg status` is experimental right now and does not support all functionalities. While the long term target is to implement them, for now we add a fallback to have all tests pass with `rhg status` enabled. Differential Revision: https://phab.mercurial-scm.org/D10906

largefiles: partially undo 61e526585b20e2ff15f19497d0451d18fea02db8 and child Since the largefiles dirstate is now part of transaction, we get rid of this temporary fix which lived for ~7 years. Differential Revision: https://phab.mercurial-scm.org/D11612

largefiles: add tr backup for largefilesdirstate This will help us in automatically restoring the largefilesdirstate if a transaction is rolled back. Differential Revision: https://phab.mercurial-scm.org/D11611

largefiles: pass current transaction to `lfdirstate.write()` Right now, the largefile dirstate is not included in transaction which makes things complex. Next patch will add code to do so, so let's make it mandatory to pass current transaction and pass from all existing callers. Differential Revision: https://phab.mercurial-scm.org/D11610

merge: with stable

dirstate-v2: Add internal documentation It can be viewed by running `hg help internals.dirstate-v2` Since that command rewraps paragraphs, the source text is written with semantic line breaks: https://sembr.org/ Differential Revision: https://phab.mercurial-scm.org/D11546

dirstate-v2: Move data file info in the docket closer together Having `data_size` next to `uuid_size` (and the UUID itself) makes more sense. Differential Revision: https://phab.mercurial-scm.org/D11545

dirstate-item: drop the legacy new_normal constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11608

dirstate-item: replace call to new_normal The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11607

dirstate-item: replace a call to new_normal The constructor is on its way out, so we inline a relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11606

dirstate-item: drop the legacy new_possibly_dirty constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11605

dirstate-item: replace call to new_possibly_dirty The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11604

dirstate-item: drop the legacy new_from_p2 constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11603

dirstate-item: replace call to new_from_p2 The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11602

dirstate-item: drop the legacy new_added constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11601

dirstate-item: replace call to new_added The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11600

dirstate-item: drop the legacy new_merged constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11599

dirstate-item: replace call to new_merged The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11598

dirstate-item: drop the `merged` property It has no user anymore. Differential Revision: https://phab.mercurial-scm.org/D11597

dirstate-item: replace another usage of `merged` We simply spell out the logic here. This was the last usage of `merged`. Differential Revision: https://phab.mercurial-scm.org/D11596

dirstate-item: replace a `merged` usage with `p2_info` It seems more accurate and no test complains (XXX hopefully XXX). Differential Revision: https://phab.mercurial-scm.org/D11595

dirstate-item: drop the `from_p2` property It has no user anymore. Differential Revision: https://phab.mercurial-scm.org/D11594

dirstate-item: directly use `p2_info` in `v1_size` This is simpler. Differential Revision: https://phab.mercurial-scm.org/D11593

dirstate-item: use the `p2_info` property to replace more verbose call Differential Revision: https://phab.mercurial-scm.org/D11592

status: process `from_p2` file the same as `merged` one What matters here is that the file constains information coming from the second parent and should be considered `modified` on plain `hg status. So we can process `from_p2` file sooner. It also highlight that we probably don't need the merged/from_p2 distinction at higher level. Differential Revision: https://phab.mercurial-scm.org/D11591

dirstate: drop an incorrect comment We are actually checking that we are only in a case were the file might needs lookup before doing this. So the comment is not relevant. Differential Revision: https://phab.mercurial-scm.org/D11590

dirstate: drop some duplicated code The same operation is done a handful a line lower. Differential Revision: https://phab.mercurial-scm.org/D11589

dirstate: align the dirstate's API to the lower level ones This conclude the refactoring of this API. We can now finalize the dirstate v2 on disk format. Differential Revision: https://phab.mercurial-scm.org/D11587

dirstate-item: introduce a `p1_tracked` property It is useful to simplify various conditional that use `any_tracked and not added`. Differential Revision: https://phab.mercurial-scm.org/D11586

dirstate-item: introduce a `p2_info` property that combine two others The `merged` and `from_p2` property are always used together so we can expose a combined property instead. Differential Revision: https://phab.mercurial-scm.org/D11585

dirstate: narrow gathering of parent data The parent data are only going to be useful is the file might be clean. And it might only be clean if it is tracked in both p1 and the working copy. Differential Revision: https://phab.mercurial-scm.org/D11584

dirstate: align the dirstatemap's API to the data change We are passing different data, so lets simplify the dirstatemap API too. Differential Revision: https://phab.mercurial-scm.org/D11583

dirstate-v2: Store a bitfield on disk instead of v1-like state Differential Revision: https://phab.mercurial-scm.org/D11558

dirstate-item: change the internal storage and constructor value This should be closer to what we do need and what we can actually reliably record. In practice it means that we abandon the prospect of storing much more refined data for now. We don't have the necessary information nor code using it right now. So it seems safer to just use a clearer version of what we had so far. See the documentation changes for details. Differential Revision: https://phab.mercurial-scm.org/D11557

dirstatemap: drop legacy method on the dirstatemap wrapper They are no longer in use now that the Rust wrapper version of the Dirstatemap are back in line with the Python one. Differential Revision: https://phab.mercurial-scm.org/D11582

dirstatemap: align the Rust wrapper implementation of `setparent` We move to using `drop_merge_data` as intended. Differential Revision: https://phab.mercurial-scm.org/D11581

dirstatemap: fix copymap.pop in Rust to return the value it pops I guess this was overlooked in the initial implementation? Without this, the next patch would, loose copy information in setparent. Differential Revision: https://phab.mercurial-scm.org/D11580

dirstate-item: implement `drop_merge_data` on the Rust DirstateItem It was currently missing and we want to be able to use in it the Rust case too. Differential Revision: https://phab.mercurial-scm.org/D11579

dirstatemap: use a common implement for reset_state Same logic as for `set_untracked` this make sure both implementation are aligned. The `reset_state` implementation for the Rust wrapped had signicantly diverged, this change finally put it back in line. Differential Revision: https://phab.mercurial-scm.org/D11578

dirstatemap: add a common `_drop_entry` method for dirstatemap This method is called to remove DirstateItem from the map. Each variant have a different implementation (which is … the point). Differential Revision: https://phab.mercurial-scm.org/D11577

dirstatemap: use common code for set_clean Same logic before this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11576

dirstatemap: use common code for set_possibly_dirty Same logic before this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11575

dirstatemap: use a common implement for set_tracked Same logic as for `set_untracked` this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11574

dirstatemap: add a common `_insert_entry` method for dirstatemap This method is called to add a new DirstateItem to the map. Each variant have a different implementation (which is … the point). Differential Revision: https://phab.mercurial-scm.org/D11573

dirstatemap: use a common implementation for `dirstatemap.set_untracked` We can now make sure they use the same code, and drop the older, out of sync, implementation of `set_untracked` for the rust wrapper. Differential Revision: https://phab.mercurial-scm.org/D11572

dirstatemap: add a common `_refresh_entry` method for dirstatemap This method is called once a DirstateItem have been modified to apply the change on the dirstatemap if necessary. Each variant have a different implementation (which is … the point). We use `addfile` for the rustmap and not `set_dirstate_item` because we need to keep the internal counter up to date and `set_dirstate_item` does not do it. Differential Revision: https://phab.mercurial-scm.org/D11571

dirstatemap: create `_dirs_incr/_dirs_decr` methods on the common class The Rust wrapper does not need them. However having a default, no-op, implementation will help use to write code used by both implementation. Differential Revision: https://phab.mercurial-scm.org/D11570

dirstatemap: small rework of the `set_untracked` method This shuffle the code a bit to have it flowing more "naturally". This will help us to create a common version of this code in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D11569

dirstatemap: arrange methods by category The dirstatemap code cover various aspects, it grow a bit messy over the years. So we shuffle the code around into some documented categories. This will help use to clean up the code. No code was changed in this changeset, only code move. Differential Revision: https://phab.mercurial-scm.org/D11568