path: keep the path instance in the `pulloperation` This will allow more pull code to use the path options. Ideally we would modify the peer API to keep the path instance. However that is much more churn that I can deal with for my current goal: adjusting a user facing API for a new feature before we release it in the 6.0 changesets. So I am taking a shortcut that seems reasonable. Differential Revision: https://phab.mercurial-scm.org/D11674

path: return path instance directly from get_pull_paths This means the caller has to do a bit more work, however it give access to the `path` instance and the information it contains. Differential Revision: https://phab.mercurial-scm.org/D11673

path: unify path creation in `get_pull_paths` This remove a special case and will make it possible to return `path` instance directly. Differential Revision: https://phab.mercurial-scm.org/D11672

path: add a new argument to control path validation During pull, unvalidated path might be used, having the option to do so directly will allow use to simplify some code and unlock more `path` usage later in the series. Differential Revision: https://phab.mercurial-scm.org/D11671

rhg: internally, return a structured representation from hg cat The purpose of this change is to make it possible to support limited templating in `hg cat`, so we could print separators between files etc. The templating itself is not implemented yet, so this functionality is unused in `rhg cat`. However, in our fork of hg we're implementing a slightly different command `hg jscat` which makes use of this. So accepting this change will let us minimize the size of the patch we're maintaining on our side. Differential Revision: https://phab.mercurial-scm.org/D11679

rhg: fix `hg cat` interaction with null revision Differential Revision: https://phab.mercurial-scm.org/D11664

dirstate-v2: add an option to prevent unintentional slow dirstate-v2 This is the project policy discussed in November 2020 and already put to use for the persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D11668

rhg: fix formatting error reported by test-check-format-rust.t Differential Revision: https://phab.mercurial-scm.org/D11670

chistedit: add option to show order of commits in opposite order Many users (including me) expect the order of commits in histedit to match the order in `hg log -G` and are confused because it doesnn't. This patch adds an option to show later commits first in the list. I've only added support for it in chistedit for now. As a consequence, I've marked the config option experimental (I think it should apply to both interfaces before it graduates). Differential Revision: https://phab.mercurial-scm.org/D11669

dirstate-v2: Separate HAS_FILE_MTIME and HAS_DIRECTORY_MTIME flags Previously the same flag was used, with its meaning based on whether the node otherwise identifies a file tracked anywhere. In addition to being more explicit, this enables storing a directory mtime if a given path used to be tracked in a parent commit (so the dirstate still has data about it) but became a directory in the working copy. (However this is not done yet as it would require a larger change, replacing the `dirstate_map::NodeData` enum with struct fields.) Differential Revision: https://phab.mercurial-scm.org/D11662

dirstate-v2: Extend node flags to 16 bits Only 7 out of 8 available bits are used right now. Reserve some more. Future versions of Mercurial may assign meaning to some of these bits, with the limitation that then-older versions will always reset those bits to unset when writing nodes. (A new node is written for any mutation in its subtree, leaving the bytes of the old node unreachable until the data file is rewritten entirely.) Differential Revision: https://phab.mercurial-scm.org/D11661

narrow: raise StateError when working copy is stale (for detailed exit code) Differential Revision: https://phab.mercurial-scm.org/D11663

dirstate-v2: Use attributes as intended instead of properties in v2_data() The property return other integer values instead of None, so `is not None` does not work. This fixes test-dirstate-race.t in pure-Python mode, which currently fails on the default branch. Differential Revision: https://phab.mercurial-scm.org/D11660

dirstate: Remove unused variable This fixes test-check-pyflakes.t which is currently failing on the default branch. Differential Revision: https://phab.mercurial-scm.org/D11658

rust: Reformat source code This fixes test-check-rust-format.t which is currently failing on the default branch. Differential Revision: https://phab.mercurial-scm.org/D11657

rhg: do not try to open a nodemap for an inline index This saves an [open] system call per file, which is a small saving, but it showed up in the profile at large file counts (it accounted for 30ms out of 400ms needed for catting 10000 files, on a ZFS filesystem on Linux, so ~3us per syscall). Differential Revision: https://phab.mercurial-scm.org/D11659

rhg: stop manifest traversal when no more files are needed Stopping the traversal early can skip a significant part of the manifest traversal, to avoid some of its cost. The worst-case benchmarks are favorable, as well. Running [hg cat] on the last file in the manifest of a large repo, I'm seeing a ~4ms improvement (150ms -> 146ms), so this time is now almost indistinguishable from the baseline ("brute force") implementation. Running [hg cat] on ~220 files together with the last file of the repo is further improved by ~5ms or so. I suspect the raw performance improvements are caused by splitting the manifest search and the file data access into separate phases, instead of interleaving them. Differential Revision: https://phab.mercurial-scm.org/D11616

rhg: faster hg cat when many files are requested With this patch I'm seeing a ~39ms improvement (220ms -> 181ms) when running [hg cat] on ~220 files in a ~260k-file repo. The timing for [hg cat] on an individual file becomes slightly worse (losing 5ms: 145ms -> 150ms). A follow-up commit is intended to improve that. Differential Revision: https://phab.mercurial-scm.org/D11615

dirstate-v2: Add support when Rust is not enabled This wires into `dirstatemap` the parser and serializer added in previous changesets. The memory representation is still the same, with a flat `dict` for `DirstateItem`s and another one for copy sources. Serialization always creates a new dirstate-v2 data file and does not support (when Rust is not enabled) appending to an existing one, since we don’t keep track of which tree nodes are new or modified. Instead the tree is reconstructed during serialization. Differential Revision: https://phab.mercurial-scm.org/D11520

dirstate-v2: Initial Python serializer This adds code seralizing a `map` and `copy_map` dicts into dirstate-v2 file formate. This is not used yet. Differential Revision: https://phab.mercurial-scm.org/D11519

dirstate-v2: initial Python parser The dirstate-v2 file format should be supported even if Rust extensions are not enabled. This changeset adds parsing code that is not used yet. Differential Revision: https://phab.mercurial-scm.org/D11518

dirstate: Move more methods to the _dirstatemapcommon base class This reduces duplication slightly and will help with supporting dirstate-v2 when Rust is not enabled. Differential Revision: https://phab.mercurial-scm.org/D11621

dirstate-v2: Add storage space for nanoseconds precision in file mtimes For now the sub-second component is always set to zero for tracked files and symlinks. (The mtime of directories for the `readdir`-skipping optimization is a different code path and already uses the full precision available.) This extra storage uses the space previously freed by replacing the 32-bit `mode` field by two bits in the existing `flags` field, so the overall size of nodes is unchanged. (This space had been left as padding for this purpose.) Also move things around in the node layout and documentation to have less duplication. Now that they have the same representation, directory mtime and file mtime are kept in the same field. (Only either one can exist for a given node.) Differential Revision: https://phab.mercurial-scm.org/D11655

status: Extract TruncatedTimestamp from fs::Metadata without SystemTime On Unix, the Rust standard library exposes `mtime` and `mtime_nsec` methods for `std::fs::Metada` whih is exactly what we need to construct a `TruncatedTimestamp`. This skips the computation in the conversion through `SystemTime` and `Result<Duration, Duration>`. Differential Revision: https://phab.mercurial-scm.org/D11654

pyoxidizer: add the user site to `sys.path` on macOS This allows 3rd party extensions that are installed with `pip` to be picked up, similar to what we do on Windows. PyOxidizer has a bug that prevents this from working without this extra help (see 95af358fcdfe), though it appears there's another wrinkle here with `sys._framework` too. I needed this to see if the problem[1] loading the keyring extension on Windows also occurs on macOS (it doesn't). [1] https://github.com/indygreg/PyOxidizer/issues/445 Differential Revision: https://phab.mercurial-scm.org/D11452

pyoxidizer: don't use in-memory for resources on macOS When trying to debug something else on macOS, pyoxidized builds were failing with the following message about pywatchman (and then cext.base85 when I commented out pywatchman): "cannot be loaded from memory but memory loading required" I don't have any immediate plans to do more work on macOS, but maybe this will save someone else time debugging this. Differential Revision: https://phab.mercurial-scm.org/D11451

chistedit: rename a confusingly named variable I would expect `ctxs` contain instances of some class from the `context` module but it actually contains instances of `histeditrule`. Differential Revision: https://phab.mercurial-scm.org/D11653

chistedit: remove some local variable and access state on self instead Now that we've replaced the state dict by a class, some of the local variables that just do `foo = self.foo` seem unnecessary. Differential Revision: https://phab.mercurial-scm.org/D11652

chistedit: move changeview() onto state class Differential Revision: https://phab.mercurial-scm.org/D11649

chistedit: move cycleaction() onto state class Differential Revision: https://phab.mercurial-scm.org/D11648

chistedit: move changeaction() onto state class Differential Revision: https://phab.mercurial-scm.org/D11647

chistedit: move swap() onto state class Differential Revision: https://phab.mercurial-scm.org/D11646

chistedit: move makeselection() onto state class Differential Revision: https://phab.mercurial-scm.org/D11645

chistedit: move changemode() onto state class Differential Revision: https://phab.mercurial-scm.org/D11644

chistedit: move movecursor() onto state class Differential Revision: https://phab.mercurial-scm.org/D11643

chistedit: move patchcontents() onto state class Differential Revision: https://phab.mercurial-scm.org/D11642

chistedit: move event() onto state class Differential Revision: https://phab.mercurial-scm.org/D11641

chistedit: move renderpatch() and dependencies onto state class Differential Revision: https://phab.mercurial-scm.org/D11640

chistedit: move renderrules() onto state class Differential Revision: https://phab.mercurial-scm.org/D11639

chistedit: move layout() and dependencies onto state class Differential Revision: https://phab.mercurial-scm.org/D11638

chistedit: move rendercommit() onto state class Differential Revision: https://phab.mercurial-scm.org/D11637

chistedit: move view state from a dict to a custom class Differential Revision: https://phab.mercurial-scm.org/D11636

rhg: do not fail when the repo is empty Differential Revision: https://phab.mercurial-scm.org/D11651

rhg: handle null changelog and manifest revisions Differential Revision: https://phab.mercurial-scm.org/D11650

rust: update the rust-cpython crate to 0.7.0 This notably brings support for Python 3.10, and includes the panic message when propagating a Rust panic as a Python exception. https://github.com/dgrunwald/rust-cpython/blob/master/CHANGELOG.md#070---2021-10-09 Differential Revision: https://phab.mercurial-scm.org/D11630

dirstate-v2: Name a constant in the Rust implementation We are about to introduce a Python version of this code that will also need this constant. Differential Revision: https://phab.mercurial-scm.org/D11547

dirstate-v2: Replace the 32-bit `mode` field with two bits Previously we stored the entire value from `stat_result.st_mode`, like dirstate-v1 does. However only the executable permission and type of file (only symbolic links and normal files are supported) are relevant to Mecurial. So replace this field with two bits in the existing bitfield byte. For now the unused space is left as padding, as it will be used for something else soon. Differential Revision: https://phab.mercurial-scm.org/D11635

dirstate-v2: Store unsigned integers inside DirstateEntry The negative marker values are not used anymore. Differential Revision: https://phab.mercurial-scm.org/D11634

dirstate-v2: Truncate directory mtimes to 31 bits of seconds … instead of 64 bits, while keeping the sub-second presision. This brings the size of one timestamp from 12 bytes to 8 bytes. 31 bits is chosen instead of 32 because that’s already what happens for the mtime of files and symlinks, because dirstate-v1 uses negative i32 values as markers. Later we’ll add sub-second precision for file/symlink mtimes, making their dirstate-v2 representation the same as for directories. Differential Revision: https://phab.mercurial-scm.org/D11633

dirstate-v2: Separate Rust structs for Timestamp and PackedTimestamp PackedTimestamp is now exclusively for dirstate-v2 serialization purpose. It contains unaligned big-endian integers. Timestamp is used everywhere else and contains native Rust integers. Differential Revision: https://phab.mercurial-scm.org/D11632

dirstate-v2: Change the representation of negative directory mtime Change it from how I previously thought C’s `timespec` works to how it actually works. The previous behavior was also buggy for timestamps strictly before the epoch but less than one second away from it, because two’s complement does not distinguish negative zero from positive zero. Differential Revision: https://phab.mercurial-scm.org/D11629

dirstate-v2: Only convert from SystemTime to Timestamp and not back Converting from Timestamp back to SystemTime was only used for equality comparison, but this can also be done on Timestamp values. Differential Revision: https://phab.mercurial-scm.org/D11631

dirstate-v2: Swap the order of size and mtime on disk This makes the dirstate-v2 file format match dirstate-v1 for the order of `mode`, `size`, and `mtime`. This order does not matter as long as these components are handled through named fields/attributes in code, but in a few places we still have tuples so having the same order everywhere might help avoid a bug that might not be obvious since those components have the same type. Differential Revision: https://phab.mercurial-scm.org/D11620

dirstate-v2: Document flags/mode/size/mtime fields of tree nodes This file format modification was previously left incomplete because of planned upcoming changes. Not all of these changes have been made yet, but documenting what exists today will help talking more widely about it. Differential Revision: https://phab.mercurial-scm.org/D11625

help: update help text for debug-repair-issue6528 The changegroup fix was put in 5.9.1, this is now out of date. Alson, this can maybe encourage people to upgrade? Differential Revision: https://phab.mercurial-scm.org/D11392

dispatch: use detailed exit code 250 for keyboard interrupt Among our users at Google, we're still seeing several percent of commands fail with exit code 255. I suspect keyboard interrupts is an important remaining reason. This is a resend of D10086 with some fixes for pager handling added ahead of it. Differential Revision: https://phab.mercurial-scm.org/D11628

dispatch: ignore failure to flush ui When the pager dies, we get a `SIGPIPE`. That causes `error.SignalInterrupt` to be raised ` (from `ui._catchterm()`). Any further writes or flushes will cause further `SIGPIPE`s and furhter `error.SignalInterrupt`. If we write or flush outside of the try/except that handle `KeyboardInterrupt` (which `error.SignalInterrupt` is a subclass of), then control will escape from the `dispatch` module. Let's fix that by ignoring errors from flushing the ui. I would have rather fixed this by restoring the stdout and stderr streams when the pager dies, but it gets complicated because of multiple ui instances (ui/lui) and different pager setups between regular hg and chg. This changes a test in `test-pager.t`, but I don't understand why. I would have thought that all the output from the command should have gone to the broken pager. Differential Revision: https://phab.mercurial-scm.org/D11627

dispatch: don't change error status if flushing stdio fails If we already have a non-zero exit code, I don't think we should change it to 255 because we fail to flush stdio. This may not matter yet, but it will matter when I make a killed pager result in exit code 250 (it's currently 255). Differential Revision: https://phab.mercurial-scm.org/D11626

dirstate-v2: Use "byte sequence" in docs The patch originally sent as https://phab.mercurial-scm.org/D11546 used "byte string" but that was changed during review to avoid suggesting Unicode or character encodings. However "byte range" sounds to be like a range of *indices* within a byte string/sequence elsewhere. This changes to "byte sequence". Python docs use "sequence" a lot when discussing the `bytes` type: https://docs.python.org/3/library/stdtypes.html Differential Revision: https://phab.mercurial-scm.org/D11623

rust: Make the hg-cpython crate default to Python 3 This default is used when running `cargo` manually such as for `cargo test`. `setup.py` and `Makefile` both configure the Python major version explicitly. Differential Revision: https://phab.mercurial-scm.org/D11618