help: update help text for debug-repair-issue6528 The changegroup fix was put in 5.9.1, this is now out of date. Alson, this can maybe encourage people to upgrade? Differential Revision: https://phab.mercurial-scm.org/D11392

dispatch: use detailed exit code 250 for keyboard interrupt Among our users at Google, we're still seeing several percent of commands fail with exit code 255. I suspect keyboard interrupts is an important remaining reason. This is a resend of D10086 with some fixes for pager handling added ahead of it. Differential Revision: https://phab.mercurial-scm.org/D11628

dispatch: ignore failure to flush ui When the pager dies, we get a `SIGPIPE`. That causes `error.SignalInterrupt` to be raised ` (from `ui._catchterm()`). Any further writes or flushes will cause further `SIGPIPE`s and furhter `error.SignalInterrupt`. If we write or flush outside of the try/except that handle `KeyboardInterrupt` (which `error.SignalInterrupt` is a subclass of), then control will escape from the `dispatch` module. Let's fix that by ignoring errors from flushing the ui. I would have rather fixed this by restoring the stdout and stderr streams when the pager dies, but it gets complicated because of multiple ui instances (ui/lui) and different pager setups between regular hg and chg. This changes a test in `test-pager.t`, but I don't understand why. I would have thought that all the output from the command should have gone to the broken pager. Differential Revision: https://phab.mercurial-scm.org/D11627

dispatch: don't change error status if flushing stdio fails If we already have a non-zero exit code, I don't think we should change it to 255 because we fail to flush stdio. This may not matter yet, but it will matter when I make a killed pager result in exit code 250 (it's currently 255). Differential Revision: https://phab.mercurial-scm.org/D11626

dirstate-v2: Use "byte sequence" in docs The patch originally sent as https://phab.mercurial-scm.org/D11546 used "byte string" but that was changed during review to avoid suggesting Unicode or character encodings. However "byte range" sounds to be like a range of *indices* within a byte string/sequence elsewhere. This changes to "byte sequence". Python docs use "sequence" a lot when discussing the `bytes` type: https://docs.python.org/3/library/stdtypes.html Differential Revision: https://phab.mercurial-scm.org/D11623

rust: Make the hg-cpython crate default to Python 3 This default is used when running `cargo` manually such as for `cargo test`. `setup.py` and `Makefile` both configure the Python major version explicitly. Differential Revision: https://phab.mercurial-scm.org/D11618

packaging: update the certifi dependency Not sure if this helps with recent certificate issues[1], but we might as well keep this modern. Note that certifi no longer claims py2 support, and there's a PR to add it back in[2]. Py2 support was dropped in 2020.04.05.2 (which predates what's being updated here). None of the *.py files have changed since the 2020.6.20 release, and I was able to call `certifi.where()` in both a virtualenv and with `hg debugshell` from an MSI install with this version, so we shouldn't be any worse off than before. [1] https://foss.heptapod.net/mercurial/tortoisehg/thg/-/issues/5745 [2] https://github.com/certifi/python-certifi/pull/154 Differential Revision: https://phab.mercurial-scm.org/D11609

setup: stop packaging python3.dll and python3X.dll in the wheel distribution Now that exewrapper is smart enough to find the DLLs it needs without help from the build script, backout ed286d150aa8 and 2960b7fac966. Note that this will require deleting the build/lib.win-amd64-3.X directory to actually remove it from the final wheel. Differential Revision: https://phab.mercurial-scm.org/D11455

exewrapper: find the proper python3X.dll in the registry Previously, we relied on the default library lookup[1], which for us is essentially to look on `PATH`. That has issues- the Python installations are not necessarily on `PATH`, so I started copying the DLLs locally in 2960b7fac966 and ed286d150aa8 during the build to work around that. However, it's been discovered that causes `python3.dll` and `python3X.dll` to get slipped into the wheel that gets distributed on PyPI. Additionally, Mercurial would fail to run in a venv if the Python environment that created it isn't on `PATH`, because venv creation doesn't copy the DLLs locally. The logic here is inspired by the `py.exe` launcher[2], though this is simpler because we don't care about the architecture- if this is a 32 bit process running on Win64, the registry reflection will redirect to where the 32 bit Python process wrote its keys. A nice unintended side effect is to also make venvs that don't have their root Python on `PATH` work without all of the code required to read `pyvenv.cfg`[3]. I don't see any reasonable way to create a venv without Python being installed (other than maybe building Python from source?), so punt on trying to read that file for now and save a bunch of string manipulation code. I somehow managed to corrupt my Windows user profile, and that makes the Microsoft Store python not run (even loading the DLL gives an access error), so I'm giving priority to both global and user specific python.org installations. Loading python3.dll is new, but when I went down the rabbit hole of implementing `pyvenv.cfg` support, I saw a comment[4] that led me to think we could have trouble if we don't. The comment in ed286d150aa8 confirms this, so we should probably bail out completely if Python3 can't be loaded from the registry, rather than getting something random on `PATH`. But I'll leave that for the default branch. [1] https://docs.microsoft.com/en-us/windows/win32/Dlls/dynamic-link-library-search-order#standard-search-order-for-desktop-applications [2] https://github.com/python/cpython/blob/adcd2205565f91c6719f4141ab4e1da6d7086126/PC/launcher.c#L249 [3] https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/PC/getpathp.c#L707 [4] https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/PC/getpathp.c#L1098 Differential Revision: https://phab.mercurial-scm.org/D11454

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280

fix: add test to demonstrate how many times tools are executed The current implementation wastes a lot of effort invoking fixer tools more than once on the same file name+content. It is good to track changes in this behavior. Differential Revision: https://phab.mercurial-scm.org/D11279

rhg: add ui.plain() and check it before showing relative paths in status Adds a very basic replica of `ui.plain()`. Differential Revision: https://phab.mercurial-scm.org/D10912

rhg: add relative paths support in `rhg status` Differential Revision: https://phab.mercurial-scm.org/D11614

rhg: refactor function to relativize paths in utils Commands like `files`, `status` supports printing relative paths. Hence we need to re-use this code in other places too. So let's take this out from `rhg files` into a utility function. Next patch will make `rhg status` use it. Differential Revision: https://phab.mercurial-scm.org/D11613

hg: let extensions call the func without populating opts keys This change is to help extensions by not forcing them to populate with opts[b'bundle'] and opts[b'force'] when calling hg.incoming(...) Differential Revision: https://phab.mercurial-scm.org/D11619

rhg: in rhg cat cli, fix the long name of the --rev flag Also tweak the help for the anonymous argument. Differential Revision: https://phab.mercurial-scm.org/D11617

rhg: fallback if tweakdefaults or statuscopies is enabled with status `rhg status` is experimental right now and does not support all functionalities. While the long term target is to implement them, for now we add a fallback to have all tests pass with `rhg status` enabled. Differential Revision: https://phab.mercurial-scm.org/D10906

largefiles: partially undo 61e526585b20e2ff15f19497d0451d18fea02db8 and child Since the largefiles dirstate is now part of transaction, we get rid of this temporary fix which lived for ~7 years. Differential Revision: https://phab.mercurial-scm.org/D11612

largefiles: add tr backup for largefilesdirstate This will help us in automatically restoring the largefilesdirstate if a transaction is rolled back. Differential Revision: https://phab.mercurial-scm.org/D11611

largefiles: pass current transaction to `lfdirstate.write()` Right now, the largefile dirstate is not included in transaction which makes things complex. Next patch will add code to do so, so let's make it mandatory to pass current transaction and pass from all existing callers. Differential Revision: https://phab.mercurial-scm.org/D11610

merge: with stable

dirstate-v2: Add internal documentation It can be viewed by running `hg help internals.dirstate-v2` Since that command rewraps paragraphs, the source text is written with semantic line breaks: https://sembr.org/ Differential Revision: https://phab.mercurial-scm.org/D11546

dirstate-v2: Move data file info in the docket closer together Having `data_size` next to `uuid_size` (and the UUID itself) makes more sense. Differential Revision: https://phab.mercurial-scm.org/D11545

dirstate-item: drop the legacy new_normal constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11608

dirstate-item: replace call to new_normal The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11607

dirstate-item: replace a call to new_normal The constructor is on its way out, so we inline a relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11606

dirstate-item: drop the legacy new_possibly_dirty constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11605

dirstate-item: replace call to new_possibly_dirty The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11604

dirstate-item: drop the legacy new_from_p2 constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11603

dirstate-item: replace call to new_from_p2 The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11602

dirstate-item: drop the legacy new_added constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11601

dirstate-item: replace call to new_added The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11600

dirstate-item: drop the legacy new_merged constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11599

dirstate-item: replace call to new_merged The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11598

dirstate-item: drop the `merged` property It has no user anymore. Differential Revision: https://phab.mercurial-scm.org/D11597

dirstate-item: replace another usage of `merged` We simply spell out the logic here. This was the last usage of `merged`. Differential Revision: https://phab.mercurial-scm.org/D11596

dirstate-item: replace a `merged` usage with `p2_info` It seems more accurate and no test complains (XXX hopefully XXX). Differential Revision: https://phab.mercurial-scm.org/D11595

dirstate-item: drop the `from_p2` property It has no user anymore. Differential Revision: https://phab.mercurial-scm.org/D11594

dirstate-item: directly use `p2_info` in `v1_size` This is simpler. Differential Revision: https://phab.mercurial-scm.org/D11593

dirstate-item: use the `p2_info` property to replace more verbose call Differential Revision: https://phab.mercurial-scm.org/D11592

status: process `from_p2` file the same as `merged` one What matters here is that the file constains information coming from the second parent and should be considered `modified` on plain `hg status. So we can process `from_p2` file sooner. It also highlight that we probably don't need the merged/from_p2 distinction at higher level. Differential Revision: https://phab.mercurial-scm.org/D11591

dirstate: drop an incorrect comment We are actually checking that we are only in a case were the file might needs lookup before doing this. So the comment is not relevant. Differential Revision: https://phab.mercurial-scm.org/D11590

dirstate: drop some duplicated code The same operation is done a handful a line lower. Differential Revision: https://phab.mercurial-scm.org/D11589

dirstate: align the dirstate's API to the lower level ones This conclude the refactoring of this API. We can now finalize the dirstate v2 on disk format. Differential Revision: https://phab.mercurial-scm.org/D11587

dirstate-item: introduce a `p1_tracked` property It is useful to simplify various conditional that use `any_tracked and not added`. Differential Revision: https://phab.mercurial-scm.org/D11586

dirstate-item: introduce a `p2_info` property that combine two others The `merged` and `from_p2` property are always used together so we can expose a combined property instead. Differential Revision: https://phab.mercurial-scm.org/D11585

dirstate: narrow gathering of parent data The parent data are only going to be useful is the file might be clean. And it might only be clean if it is tracked in both p1 and the working copy. Differential Revision: https://phab.mercurial-scm.org/D11584

dirstate: align the dirstatemap's API to the data change We are passing different data, so lets simplify the dirstatemap API too. Differential Revision: https://phab.mercurial-scm.org/D11583

dirstate-v2: Store a bitfield on disk instead of v1-like state Differential Revision: https://phab.mercurial-scm.org/D11558

dirstate-item: change the internal storage and constructor value This should be closer to what we do need and what we can actually reliably record. In practice it means that we abandon the prospect of storing much more refined data for now. We don't have the necessary information nor code using it right now. So it seems safer to just use a clearer version of what we had so far. See the documentation changes for details. Differential Revision: https://phab.mercurial-scm.org/D11557

dirstatemap: drop legacy method on the dirstatemap wrapper They are no longer in use now that the Rust wrapper version of the Dirstatemap are back in line with the Python one. Differential Revision: https://phab.mercurial-scm.org/D11582

dirstatemap: align the Rust wrapper implementation of `setparent` We move to using `drop_merge_data` as intended. Differential Revision: https://phab.mercurial-scm.org/D11581

dirstatemap: fix copymap.pop in Rust to return the value it pops I guess this was overlooked in the initial implementation? Without this, the next patch would, loose copy information in setparent. Differential Revision: https://phab.mercurial-scm.org/D11580

dirstate-item: implement `drop_merge_data` on the Rust DirstateItem It was currently missing and we want to be able to use in it the Rust case too. Differential Revision: https://phab.mercurial-scm.org/D11579

dirstatemap: use a common implement for reset_state Same logic as for `set_untracked` this make sure both implementation are aligned. The `reset_state` implementation for the Rust wrapped had signicantly diverged, this change finally put it back in line. Differential Revision: https://phab.mercurial-scm.org/D11578

dirstatemap: add a common `_drop_entry` method for dirstatemap This method is called to remove DirstateItem from the map. Each variant have a different implementation (which is … the point). Differential Revision: https://phab.mercurial-scm.org/D11577

dirstatemap: use common code for set_clean Same logic before this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11576

dirstatemap: use common code for set_possibly_dirty Same logic before this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11575

dirstatemap: use a common implement for set_tracked Same logic as for `set_untracked` this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11574

dirstatemap: add a common `_insert_entry` method for dirstatemap This method is called to add a new DirstateItem to the map. Each variant have a different implementation (which is … the point). Differential Revision: https://phab.mercurial-scm.org/D11573

dirstatemap: use a common implementation for `dirstatemap.set_untracked` We can now make sure they use the same code, and drop the older, out of sync, implementation of `set_untracked` for the rust wrapper. Differential Revision: https://phab.mercurial-scm.org/D11572

dirstatemap: add a common `_refresh_entry` method for dirstatemap This method is called once a DirstateItem have been modified to apply the change on the dirstatemap if necessary. Each variant have a different implementation (which is … the point). We use `addfile` for the rustmap and not `set_dirstate_item` because we need to keep the internal counter up to date and `set_dirstate_item` does not do it. Differential Revision: https://phab.mercurial-scm.org/D11571

dirstatemap: create `_dirs_incr/_dirs_decr` methods on the common class The Rust wrapper does not need them. However having a default, no-op, implementation will help use to write code used by both implementation. Differential Revision: https://phab.mercurial-scm.org/D11570

dirstatemap: small rework of the `set_untracked` method This shuffle the code a bit to have it flowing more "naturally". This will help us to create a common version of this code in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D11569

dirstatemap: arrange methods by category The dirstatemap code cover various aspects, it grow a bit messy over the years. So we shuffle the code around into some documented categories. This will help use to clean up the code. No code was changed in this changeset, only code move. Differential Revision: https://phab.mercurial-scm.org/D11568

dirstatemap: move a multiple simple functions in the common class These are small and simple, lets factor them out. Differential Revision: https://phab.mercurial-scm.org/D11567

dirstatemap: rename `_rustmap` to `_map` This match the name of the `map` for the other implementation and will make it simpler to share code between the two. Differential Revision: https://phab.mercurial-scm.org/D11566

dirstatemap: use a common __init__ for dirstatemap This is the first and simplest things to put in common. Differential Revision: https://phab.mercurial-scm.org/D11565

dirstatemap: introduce a common base for the dirstatemap class We have two dirstatemaps class. One for the python version of the dirstate map and one for the Rust version (that has a python wrapper to deal with some aspect of it). We end up with duplicated code between them, so we introduce a common base class to start migrating common code in them. Differential Revision: https://phab.mercurial-scm.org/D11564

errors: raise InputError from revsingle() iff revset provided by the user Same reasoning as for `revrange()` in an earlier patch. Differential Revision: https://phab.mercurial-scm.org/D11562

errors: raise InputError from revpair() iff revset provided by the user Same reasoning as for `revrange()` in an earlier patch. Differential Revision: https://phab.mercurial-scm.org/D11561

errors: raise InputError on bad revset to revrange() iff provided by the user Most callers of `scmutil.revrange()` pass in a revset provided by the user. If there are problems resolving that, it should result in an `InputError` and exit code 10 (when using detailed exit codes). However, there are also some callers that pass in revsets not provided by the user. `InputError` is not appropriate in those cases. This patch therefore introduces a wrapper around `scmutil.revrange()` that simply converts the exception type. I put it in `logcmdutil.py` since that seems to be the lowest-level module in the (poorly defined) UI layer. Differential Revision: https://phab.mercurial-scm.org/D11560

phase: avoid a no-op resolution of revset from revnums I was surprised that `scmutil.revrange()` supports integers in the list of revsets. I think it's clearer to not pass a list that's known to contain only integers into the function. Differential Revision: https://phab.mercurial-scm.org/D11559

dirstate: push back the future a bit in the test The future was set to 2021-01-01, we push it by 10 years.

dirstate-item: point out that `merged` is set only with p1_tracked This is currently True, and we will use this fact to simplify the API in the next commit. However, we add this assertion first to validate that this is True in the whole test-suite.

dirstate-item: update the attribute documentation It was very outdated. We are about to change these attribute so we should has well have them documented so that the change get easier to grasp.

dirstate-item: use `any_tracked` more This simplify more code.

dirstate-item: drop an outdated comments This comment is no longer relevant since we moved away from the `state` internal representation, multiple weeks ago.

dirstate: remove a update_file's special case for `merged` file This case was fishy and can be dealt with by passing more accurate data a higher level. This clarify the API and prepare for a larger rework of the data we feeds to the dirstate.

dirstate: remove a update_file's special case for tracked file with p2 data This case was fishy and can be dealt with by passing more accurate data a higher level. This clarify the API and prepare for a larger rework of the data we feeds to the dirstate.

dirstate: deprecate `__getitem__` access If we want to drop `state` usage, we need to deprecate this. Differential Revision: https://phab.mercurial-scm.org/D11544

dirstate-item: use item's property instead of `state` in largefile Differential Revision: https://phab.mercurial-scm.org/D11543

dirstate-item: use `added` instead of `state` when moving dirstate Differential Revision: https://phab.mercurial-scm.org/D11542

dirstate-item: use item's property instead of `state` in revert Differential Revision: https://phab.mercurial-scm.org/D11541

dirstate-item: use item's property when computing a copies Differential Revision: https://phab.mercurial-scm.org/D11540

dirstate-item: use item's property instead of `state` in copy Differential Revision: https://phab.mercurial-scm.org/D11539

dirstate-item: use `added` in debugrebuilddirstate (instead of `state`) Differential Revision: https://phab.mercurial-scm.org/D11538

dirstate-item: use `maybe_clean` instead of `state` in record Differential Revision: https://phab.mercurial-scm.org/D11537

dirstate-item: use `any_tracked` instead of `state` to apply patches Differential Revision: https://phab.mercurial-scm.org/D11536

dirstate-item: use item's property instead of `state` in addremove Differential Revision: https://phab.mercurial-scm.org/D11535

dirstate-item: use `tracked` instead of `state` during copy detection Differential Revision: https://phab.mercurial-scm.org/D11534

dirstate-item: use `maybe_clean` instead of `state` in `strip` Differential Revision: https://phab.mercurial-scm.org/D11533

dirstate-item: use `tracked` instead of `state` in context.matches Differential Revision: https://phab.mercurial-scm.org/D11532

dirstate-item: use `tracked` instead of the `state` in context's iter Differential Revision: https://phab.mercurial-scm.org/D11531

dirstate-item: use `tracked` instead of the `state` in context Differential Revision: https://phab.mercurial-scm.org/D11530

dirstate-item: use item's property to deal with hgsubstate in mq This is clearer than processing the `state`. Differential Revision: https://phab.mercurial-scm.org/D11529