cext: define S_IFLNK on Python 2.7 and Windows Before this change, building on Python 2.7 on Windows fails due to S_IFLNK being undefined. This regression was introduced by a32a96079e2d / D11518. It worked on Python 3 because its pyport.h (which is included via Python.h) contains effectively the same code as added by this changeset. Differential Revision: https://phab.mercurial-scm.org/D11763

tests: dd status=noxfer is not portable (issue6523) Differential Revision: https://phab.mercurial-scm.org/D11754

tests: use more portable mkdir -p instead of --parents (issue6591) Differential Revision: https://phab.mercurial-scm.org/D11755

tests: fix test-censor by improving regexp (issue6585) Differential Revision: https://phab.mercurial-scm.org/D11753

internals: typo pass on the dirstate-v2 help file Differential Revision: https://phab.mercurial-scm.org/D11747

dirstate-v2: fix confusion between requirement and format config variable None of those outputs failed on the CI because dirstate-v2 is not fully tested yet. This also updates the documentation. Differential Revision: https://phab.mercurial-scm.org/D11746

internals: correct dirstate-v2 format variable in the documentation Differential Revision: https://phab.mercurial-scm.org/D11745

hghave: update dirstate-v2 format config option It wasn't kept in sync with the new change in format name. Differential Revision: https://phab.mercurial-scm.org/D11744

Added signature for changeset a44bb185f6bd

Added tag 6.0rc0 for changeset a44bb185f6bd

merge with default

tests: accept alternative privileged port allocation failure This registers an additional failure message on failed privileged port allocation, equally funcionally valid but previously not handled and causing the test to fail when run in the NixOS sandbox. Differential Revision: https://phab.mercurial-scm.org/D11741

tests: allow the true command to be provided by coreutils The `true` command is sometimes provided as a symbolic link to the `coreutils` single binary. This is the case on NixOS, on which the test was failing because the symbolic link fully resolves to the latter name, equally valid but not previously accepted by the test's golden output. Differential Revision: https://phab.mercurial-scm.org/D11740

tests: add guard check for suid support This adds a check for setuid and setgid support and guards a part of a test requiring this particular filesystem feature. Such feature is notably missing in the Nix/NixOS build sandbox and was causing the whole test to fail for the mercurial package. Differential Revision: https://phab.mercurial-scm.org/D11739

rhg: more efficient `HgPath::join` This commit makes `HgPath::join` slightly more efficient by avoiding one copy. It also avoids a particularly inefficient (quadratic) use of `HgPath::join` by using a new mutating function `HgPathBuf::push` instead. The name for `HgPathBuf::push` is chosen by analogy to `PathBuf::push`. Differential Revision: https://phab.mercurial-scm.org/D11721

relnotes: update 6.0 with the last changes for the release Differential Revision: https://phab.mercurial-scm.org/D11742

rhg: Fix `rhg status` file content comparison This is only used when a file’s metadata make its status ambiguous, which depends on timing of previous command executions. Differential Revision: https://phab.mercurial-scm.org/D11743

rhg: make it possible to opt out of `rhg cat` The reason an opt-out is needed is that there are still behavior differences between `rhg cat` and `hg cat`: - it does not interpret relative paths correctly - it does not interpret patterns correctly, e.g. 're:foobar$' would be interpreted as a verbatim filename - it does not implement the correct semantics of relpath matcher: if given a directory, `hg` concatenates all files in this directory, while `rhg` simply complains Differential Revision: https://phab.mercurial-scm.org/D11723

tests: run the whole hg-core/path_auditor test in a clean temp dir This makes the whole test happen in a clean temporary directory in `$TMPDIR/$random`, and simplifies the test a bit by eliminating unnecessarily dynamic path elements computations. Before this patch, the first part of the test was happening in `/tmp` itself. This allowed coincidentally named files placed in that directory to impact the outcome of the test. Additionally, this made the second part of the test fail on systems on which `$TMPDIR != /tmp`, because the inspected directory was different from the one in which the mock files were being written. This fully fixes the issue only partially solved in db2bc9e667a1. Differential Revision: https://phab.mercurial-scm.org/D11738

fsmonitor: fix traceback after removal of nonnormalset (issue6600) This basically does the same thing it used to do, but in the only place it's needed. Differential Revision: https://phab.mercurial-scm.org/D11707

fsmonitor: fix traceback about bytes and str incompatibility This didn't work anymore in Python 3, switching to a `repr` based reporting does Differential Revision: https://phab.mercurial-scm.org/D11708

backout: backed out changeset 67d14d4e036c Same as the previous patch, this breaks the Windows CI for some yet unknown reason. Differential Revision: https://phab.mercurial-scm.org/D11727

backout: backed out changeset f78d8b8c46d7 This and the following backout exist because the original patches break the Windows CI for some yet unknown reason. Differential Revision: https://phab.mercurial-scm.org/D11726

relnotes: clear next release notes Differential Revision: https://phab.mercurial-scm.org/D11737

relnotes: add 6.0 release notes Differential Revision: https://phab.mercurial-scm.org/D11736

relnotes: fix wrong command name in 5.9 relnotes Differential Revision: https://phab.mercurial-scm.org/D11735

pyoxidizer: use in-memory resources on non-Windows platforms In-memory resources were disabled for macOS in 7bc1beed, and for all platforms in c900d962. Unfortunately this made it so that we were no longer producing standalone binaries on these platforms, and would have to ship the .py and .pyc files alongside the pyoxidized binary. These changes are no longer necessary after f6b04591, which disabled pep517 and solved the issue we were encountering. Differential Revision: https://phab.mercurial-scm.org/D11734

compat: don't rely on cpython-specific builtins manipulation Pierre Augier signaled on the mailing list that this fails on Pypy and pointed out the correct solution. Differential Revision: https://phab.mercurial-scm.org/D11732

rust-format: update rustfmt version This hasn't been updated in a while, and since we're updating the MSRV to 1.48.0, we might as well do this. Differential Revision: https://phab.mercurial-scm.org/D11733

rust: update the minimum version of Rust Debian Bullseye has just been released, and it carries `rustc 1.48.0`. This actually implies a regression that we can't really do anything about in `rhg`. See https://github.com/rust-lang/rust/issues/88825. In short, closed (or bad) standard file descriptors are reopened silently with no way of telling by the Rust runtime before `main` is executed. This means that closed fds are not forwarded to the subprocess we run in case of fallback. This is a bit sad, but probably not something worth worrying too much about. Differential Revision: https://phab.mercurial-scm.org/D11341

dirstate-v2: freeze the on-disk format It seems the format as reached a good balance. With a core of new capabilities that motivated it initially and enough new feature and room for future improvement to be a clear progress we can set a milestone for. Having the format frozen will help the feature to get real life testing, outside of the test suite. The feature itself stay experimental but the config gains a new name to avoid people enable non-frozen version by default. If too many bugs are reported during the RC we might move the format back to experimental and drop its support in future version (in favor of a new one) Differential Revision: https://phab.mercurial-scm.org/D11709

merge: with stable

windows: use raw string in test log paths Otherwise the backslashes will work as escapes on Windows and break everything. Differential Revision: https://phab.mercurial-scm.org/D11725

pull: make the new argument a keyword argument As per feedback from Dan Villiom Podlaski Christiansen in https://phab.mercurial-scm.org/D11674#179866 Differential Revision: https://phab.mercurial-scm.org/D11730

dirstate: make sure that status does not overlook the fallback flags Without this extra checks, file with fallback flags change as the only change would be overlooked. In the future we might store proper data in the dirstate and do less lookup. However, for now this will do to make sure that 6.0 is forward compatible with later version. Differential Revision: https://phab.mercurial-scm.org/D11729

dirstate: use a single closure for get_flags The previous code was overlooking fallback when neither symlink not exec was supported. The number of "variants" is getting too high, so I am consolidating this in a single closure that should be easier to maintains. This also ensure that fallback flags are always taken into account. (they are not user code yet, but small experimentation shown that the feature was working as intended.) A a small side effect we need to check for symlink support more lazily and this show up in the test in a couple of places. Differential Revision: https://phab.mercurial-scm.org/D11728

merge: with stable

typing: add more type hints to the errors module These were found by looking for `Any` types in the generated `*.pyi` file after running the tests. There are some more complicated types that I'm not sure of, and am leaving untyped. I also can't figure out how to get `self.hint` to be anything other than `Any` in most classes. Differential Revision: https://phab.mercurial-scm.org/D11719

typing: add a few assertions to revlog.py to help pytype I've seen this before, but this was seen as potentially `None` with pytype 2021.10.18. Differential Revision: https://phab.mercurial-scm.org/D11718

dirstate: group return logic and clarify each function in flagfunc As suggested by spectral during review of my fix of the missing return, this makes it more obvious that all cases are covered and improves readability of the functions with new names and a small docstring. Differential Revision: https://phab.mercurial-scm.org/D11715

dirstate: add missing return on platforms without exec or symlink Differential Revision: https://phab.mercurial-scm.org/D11713

parsers: don't ask about symlinks on platforms that don't support them Otherwise the compiler gets quite sad. Differential Revision: https://phab.mercurial-scm.org/D11712

parsers: don't ask about the exec bit on platforms that don't have it Otherwise the compiler gets quite sad. Differential Revision: https://phab.mercurial-scm.org/D11711

tests: fix test-convert-git to work w/ "git pull" requiring strategy A recent change to git (031e2f7ae195) made it an error to not specify a strategy (`--rebase`, `--no-rebase`, `--ff-only`), instead of just the warning it was previously. As far as I can tell, `--no-rebase` is the behavior we were getting before, and the only one that makes the test work. Differential Revision: https://phab.mercurial-scm.org/D11714

branching: merge stable into default

hg: remove reserved identifiers Per 7.1.3 paragraph, from C draft: All identifiers that begin with an underscore and either an uppercase letter or another underscore are always reserved for any use.

pyoxidizer: update README.md with several small fixes Currently, pyoxidizer.bzl does not mention the git commit that should be checked out, so these instructions are a bit difficult to follow right now (impossible, technically), so I removed the instruction to `git checkout <Git commit>` and the admonition to use a specific version of PyOxidizer. I don't even know if the project currently builds with the "0.7.0-pre" version that was previously recommended. As fallout from that change to not "pin" to a specific PyOxidizer, I had to update the Python version to use when running the tests. While here, I added a recommendation to use `--release`, as the primary reason for this project is performance, and it may have been leaving some on the table to not have that there. Differential Revision: https://phab.mercurial-scm.org/D11698

pyoxidizer: disable using in-memory resources It's possible that the errors are due to using an incompatible version of PyOxidizer; unfortunately the README.md file in this directory says to fetch a copy of PyOxidizer matching the commit in this pyoxidizer.bzl file, and yet the pyoxidizer.bzl file does not actually have a commit mentioned in it. By disabling in-memory modules, this appears to work on all platforms using the current head version of PyOxidizer, so let's disable them for now. Sample error (during `pyoxidizer build`): ``` error[PYOXIDIZER_PYTHON_EXECUTABLE]: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser> Caused by: extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required --> ./pyoxidizer.bzl:140:5 | 140 | exe.add_python_resources(exe.pip_install(["--verbose", ROOT])) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ add_python_resources() error: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser> Caused by: extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required ``` Differential Revision: https://phab.mercurial-scm.org/D11697

dirstate-v2: reorder flag to group related one together Since the format is not frozen yet, it seems like the right moment to do it. Differential Revision: https://phab.mercurial-scm.org/D11704

dirstate-v2: read the fallback value in Rust This was overlooked in a previous commit. Differential Revision: https://phab.mercurial-scm.org/D11695

dirstate-v2: adjust the meaning of directory flags Tracking directory "explicitly" give use the opportunity to distinct between entry that are untracked because they are part of the directory structure and entry that are ignored/unknown files on the files system. The help is adjusted to the new semantic and the code now comply to it for both read and write. Differential Revision: https://phab.mercurial-scm.org/D11694

dirstate-v2: actually use sub-second mtime precision Instead of zero, set the nanoseconds field to its correct value whenever possible and preserve it across serialization+parsing. Differential Revision: https://phab.mercurial-scm.org/D11702

dirstate: ignore sub-second component when either is zero in mtime When comparing mtimes for equality. Some APIs simply return zero when more precision is not available. When comparing values from different sources, if only one is truncated in that way, doing a simple comparison would cause many false negatives. Differential Revision: https://phab.mercurial-scm.org/D11701

dirstate-v2: add a new MTIME_SECOND_AMBIGUOUS flags This flag will let us use more `mtime` value in the future. For now we have a minimal handling of the flag at read time, but we will never put ourself in a situation where we will needs to writes it. See the flag documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11700

dirstate: store mtimes with nanosecond precision in memory Keep integer seconds since the Unix epoch, together with integer nanoseconds in the `0 <= n < 1e9` range. For now, nanoseconds are still always zero. This commit is about data structure changes. Differential Revision: https://phab.mercurial-scm.org/D11684

rust: remove now-unused DirstateEntry::clear_ambiguous_mtime method Differential Revision: https://phab.mercurial-scm.org/D11696

dirstate: align Rust function name to `need_delay` The rest of the code use this name. It is not a great name, but it is better to stay consistent. Differential Revision: https://phab.mercurial-scm.org/D11699

dirstate: rename a `very_likely_equal` method to `likely_equal` No need to oversell it. Differential Revision: https://phab.mercurial-scm.org/D11691

backout: backed out changeset 2f2107c01dee After discussion on the original phab patch, we've decided to wait until Python 2 support is dropped to do this since the behavior differs for relatively obscure reasons. Differential Revision: https://phab.mercurial-scm.org/D11692

backout: backed out changeset 6edc8800dbc3 Same as the previous changeset. Differential Revision: https://phab.mercurial-scm.org/D11693

dirstate-v2: preserve the fallback values on disk When the fallback values are set, they are now read and written to disk. See format documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11688

dirstate: make DirstateItem constructor accept fallback value This could arguably goes in the previous changeset, but I wanted to keep that previous changeset small to focus more on the user code and the documentation. Differential Revision: https://phab.mercurial-scm.org/D11687

dirstate: add a concept of "fallback" flags to dirstate item The concept is defined and "used" by the flag code, but it is neither persisted nor set anywhere yet. We currently focus on defining the semantic of the attribute. More to come in the next changesets Check the inline documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11686

dirstate-v2: adds two flag to track the presence of some unrecorded files Right now, we don't record ignored or unknown files in the dirstate. However the structure would allow it. So we introduce two flags that can be used to clarify whether all unknown/ignored children are recorded or not. This will allow for more information to be stored in the future if this end up being relevant. Differential Revision: https://phab.mercurial-scm.org/D11682

dirstate-v2: adds a flag to mark a file as modified Right now, a files with a file system state that requires a lookup (same size, different mtime) will requires a lookup. If the result of that lookup is a modified files, it will remains ambiguous, requiring a lookup on the next status run too. To fix this, we introduce a dedicated flag in the new format. Such flag will allow to record such file as "known modified" avoiding an extra lookup later. As None of the associate code currently exist in the status code, we do the minimal implementation: if we read a dirstate entry with this flag set, we make it as "ambiguous" so that the next status code has to look it up. The same as it would have to without this flag existing anyway. Differential Revision: https://phab.mercurial-scm.org/D11681

rhg: simplify split_metadata It turns out that it's possible to implement `FilelogEntry.into_data` on top of `split`, as proposed by @spectral. Differential Revision: https://phab.mercurial-scm.org/D11689

pyoxidizer: default to a meaningless release instead of hardcoding 5.8 This way we don't advertise the wrong version, or if we do it's really obvious. To be honest, I'm not even sure that `VERSION` should be optional, but I also have never used the packaging tools so far, this is simply something I saw. Differential Revision: https://phab.mercurial-scm.org/D11360

copy: recommend `--at-rev .` if target was added in parent commit If the target file was added in the working copy parent, it's much more likely that the user meant to use `--at-rev .` (to record the copy/rename in the that commit) than `--force` (to replace the just-added file by another file). Differential Revision: https://phab.mercurial-scm.org/D11667

fix: appease pyflakes and make unused variables more obvious pyflakes is complaining that `baserevs` is redefined by the list comprehension, and while this lint can be valuable, it's not actually a problem here. Differential Revision: https://phab.mercurial-scm.org/D11683

bookmarks: add a `ignore` variant of the bookmark mode This new mode allow to disable bookmark exchange with some path (or all path). Differential Revision: https://phab.mercurial-scm.org/D11677

bookmarks: add support for `mirror` mode to `incoming` This is more consistent. Differential Revision: https://phab.mercurial-scm.org/D11676

bookmarks: move the `mirror` option to the `paths` section A new `bookmarks` section with a `mirror` option have been added. That option has never been released yet. This new options is limited since it affect all paths without distinction. In case where a repository is interacting with multiple peers, being able to control behavior on a path basis can be quite valuable. In addition, having more variant of behavior would be interesting, especially a mode where no bookmark exchanged is tried at all. Such new mode (implemented later) make a lot of sense for configuration on a path-basis. Configuration of the default behavior is still possible through the usage of generic path configuration. The "old" config, becomes: [bookmarks] mirror=True becomes: [path] *:bookmarks.mode=mirror Differential Revision: https://phab.mercurial-scm.org/D11675

path: keep the path instance in the `pulloperation` This will allow more pull code to use the path options. Ideally we would modify the peer API to keep the path instance. However that is much more churn that I can deal with for my current goal: adjusting a user facing API for a new feature before we release it in the 6.0 changesets. So I am taking a shortcut that seems reasonable. Differential Revision: https://phab.mercurial-scm.org/D11674

path: return path instance directly from get_pull_paths This means the caller has to do a bit more work, however it give access to the `path` instance and the information it contains. Differential Revision: https://phab.mercurial-scm.org/D11673

path: unify path creation in `get_pull_paths` This remove a special case and will make it possible to return `path` instance directly. Differential Revision: https://phab.mercurial-scm.org/D11672

path: add a new argument to control path validation During pull, unvalidated path might be used, having the option to do so directly will allow use to simplify some code and unlock more `path` usage later in the series. Differential Revision: https://phab.mercurial-scm.org/D11671

rhg: internally, return a structured representation from hg cat The purpose of this change is to make it possible to support limited templating in `hg cat`, so we could print separators between files etc. The templating itself is not implemented yet, so this functionality is unused in `rhg cat`. However, in our fork of hg we're implementing a slightly different command `hg jscat` which makes use of this. So accepting this change will let us minimize the size of the patch we're maintaining on our side. Differential Revision: https://phab.mercurial-scm.org/D11679

rhg: fix `hg cat` interaction with null revision Differential Revision: https://phab.mercurial-scm.org/D11664

dirstate-v2: add an option to prevent unintentional slow dirstate-v2 This is the project policy discussed in November 2020 and already put to use for the persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D11668

rhg: fix formatting error reported by test-check-format-rust.t Differential Revision: https://phab.mercurial-scm.org/D11670

chistedit: add option to show order of commits in opposite order Many users (including me) expect the order of commits in histedit to match the order in `hg log -G` and are confused because it doesnn't. This patch adds an option to show later commits first in the list. I've only added support for it in chistedit for now. As a consequence, I've marked the config option experimental (I think it should apply to both interfaces before it graduates). Differential Revision: https://phab.mercurial-scm.org/D11669

dirstate-v2: Separate HAS_FILE_MTIME and HAS_DIRECTORY_MTIME flags Previously the same flag was used, with its meaning based on whether the node otherwise identifies a file tracked anywhere. In addition to being more explicit, this enables storing a directory mtime if a given path used to be tracked in a parent commit (so the dirstate still has data about it) but became a directory in the working copy. (However this is not done yet as it would require a larger change, replacing the `dirstate_map::NodeData` enum with struct fields.) Differential Revision: https://phab.mercurial-scm.org/D11662

dirstate-v2: Extend node flags to 16 bits Only 7 out of 8 available bits are used right now. Reserve some more. Future versions of Mercurial may assign meaning to some of these bits, with the limitation that then-older versions will always reset those bits to unset when writing nodes. (A new node is written for any mutation in its subtree, leaving the bytes of the old node unreachable until the data file is rewritten entirely.) Differential Revision: https://phab.mercurial-scm.org/D11661

narrow: raise StateError when working copy is stale (for detailed exit code) Differential Revision: https://phab.mercurial-scm.org/D11663

dirstate-v2: Use attributes as intended instead of properties in v2_data() The property return other integer values instead of None, so `is not None` does not work. This fixes test-dirstate-race.t in pure-Python mode, which currently fails on the default branch. Differential Revision: https://phab.mercurial-scm.org/D11660

dirstate: Remove unused variable This fixes test-check-pyflakes.t which is currently failing on the default branch. Differential Revision: https://phab.mercurial-scm.org/D11658

rust: Reformat source code This fixes test-check-rust-format.t which is currently failing on the default branch. Differential Revision: https://phab.mercurial-scm.org/D11657

rhg: do not try to open a nodemap for an inline index This saves an [open] system call per file, which is a small saving, but it showed up in the profile at large file counts (it accounted for 30ms out of 400ms needed for catting 10000 files, on a ZFS filesystem on Linux, so ~3us per syscall). Differential Revision: https://phab.mercurial-scm.org/D11659

rhg: stop manifest traversal when no more files are needed Stopping the traversal early can skip a significant part of the manifest traversal, to avoid some of its cost. The worst-case benchmarks are favorable, as well. Running [hg cat] on the last file in the manifest of a large repo, I'm seeing a ~4ms improvement (150ms -> 146ms), so this time is now almost indistinguishable from the baseline ("brute force") implementation. Running [hg cat] on ~220 files together with the last file of the repo is further improved by ~5ms or so. I suspect the raw performance improvements are caused by splitting the manifest search and the file data access into separate phases, instead of interleaving them. Differential Revision: https://phab.mercurial-scm.org/D11616

rhg: faster hg cat when many files are requested With this patch I'm seeing a ~39ms improvement (220ms -> 181ms) when running [hg cat] on ~220 files in a ~260k-file repo. The timing for [hg cat] on an individual file becomes slightly worse (losing 5ms: 145ms -> 150ms). A follow-up commit is intended to improve that. Differential Revision: https://phab.mercurial-scm.org/D11615

dirstate-v2: Add support when Rust is not enabled This wires into `dirstatemap` the parser and serializer added in previous changesets. The memory representation is still the same, with a flat `dict` for `DirstateItem`s and another one for copy sources. Serialization always creates a new dirstate-v2 data file and does not support (when Rust is not enabled) appending to an existing one, since we don’t keep track of which tree nodes are new or modified. Instead the tree is reconstructed during serialization. Differential Revision: https://phab.mercurial-scm.org/D11520

dirstate-v2: Initial Python serializer This adds code seralizing a `map` and `copy_map` dicts into dirstate-v2 file formate. This is not used yet. Differential Revision: https://phab.mercurial-scm.org/D11519

dirstate-v2: initial Python parser The dirstate-v2 file format should be supported even if Rust extensions are not enabled. This changeset adds parsing code that is not used yet. Differential Revision: https://phab.mercurial-scm.org/D11518

dirstate: Move more methods to the _dirstatemapcommon base class This reduces duplication slightly and will help with supporting dirstate-v2 when Rust is not enabled. Differential Revision: https://phab.mercurial-scm.org/D11621

dirstate-v2: Add storage space for nanoseconds precision in file mtimes For now the sub-second component is always set to zero for tracked files and symlinks. (The mtime of directories for the `readdir`-skipping optimization is a different code path and already uses the full precision available.) This extra storage uses the space previously freed by replacing the 32-bit `mode` field by two bits in the existing `flags` field, so the overall size of nodes is unchanged. (This space had been left as padding for this purpose.) Also move things around in the node layout and documentation to have less duplication. Now that they have the same representation, directory mtime and file mtime are kept in the same field. (Only either one can exist for a given node.) Differential Revision: https://phab.mercurial-scm.org/D11655

status: Extract TruncatedTimestamp from fs::Metadata without SystemTime On Unix, the Rust standard library exposes `mtime` and `mtime_nsec` methods for `std::fs::Metada` whih is exactly what we need to construct a `TruncatedTimestamp`. This skips the computation in the conversion through `SystemTime` and `Result<Duration, Duration>`. Differential Revision: https://phab.mercurial-scm.org/D11654

pyoxidizer: add the user site to `sys.path` on macOS This allows 3rd party extensions that are installed with `pip` to be picked up, similar to what we do on Windows. PyOxidizer has a bug that prevents this from working without this extra help (see 95af358fcdfe), though it appears there's another wrinkle here with `sys._framework` too. I needed this to see if the problem[1] loading the keyring extension on Windows also occurs on macOS (it doesn't). [1] https://github.com/indygreg/PyOxidizer/issues/445 Differential Revision: https://phab.mercurial-scm.org/D11452

pyoxidizer: don't use in-memory for resources on macOS When trying to debug something else on macOS, pyoxidized builds were failing with the following message about pywatchman (and then cext.base85 when I commented out pywatchman): "cannot be loaded from memory but memory loading required" I don't have any immediate plans to do more work on macOS, but maybe this will save someone else time debugging this. Differential Revision: https://phab.mercurial-scm.org/D11451

chistedit: rename a confusingly named variable I would expect `ctxs` contain instances of some class from the `context` module but it actually contains instances of `histeditrule`. Differential Revision: https://phab.mercurial-scm.org/D11653

chistedit: remove some local variable and access state on self instead Now that we've replaced the state dict by a class, some of the local variables that just do `foo = self.foo` seem unnecessary. Differential Revision: https://phab.mercurial-scm.org/D11652

chistedit: move changeview() onto state class Differential Revision: https://phab.mercurial-scm.org/D11649

chistedit: move cycleaction() onto state class Differential Revision: https://phab.mercurial-scm.org/D11648

chistedit: move changeaction() onto state class Differential Revision: https://phab.mercurial-scm.org/D11647

chistedit: move swap() onto state class Differential Revision: https://phab.mercurial-scm.org/D11646

chistedit: move makeselection() onto state class Differential Revision: https://phab.mercurial-scm.org/D11645

chistedit: move changemode() onto state class Differential Revision: https://phab.mercurial-scm.org/D11644

chistedit: move movecursor() onto state class Differential Revision: https://phab.mercurial-scm.org/D11643

chistedit: move patchcontents() onto state class Differential Revision: https://phab.mercurial-scm.org/D11642

chistedit: move event() onto state class Differential Revision: https://phab.mercurial-scm.org/D11641

chistedit: move renderpatch() and dependencies onto state class Differential Revision: https://phab.mercurial-scm.org/D11640

chistedit: move renderrules() onto state class Differential Revision: https://phab.mercurial-scm.org/D11639

chistedit: move layout() and dependencies onto state class Differential Revision: https://phab.mercurial-scm.org/D11638

chistedit: move rendercommit() onto state class Differential Revision: https://phab.mercurial-scm.org/D11637

chistedit: move view state from a dict to a custom class Differential Revision: https://phab.mercurial-scm.org/D11636

rhg: do not fail when the repo is empty Differential Revision: https://phab.mercurial-scm.org/D11651

rhg: handle null changelog and manifest revisions Differential Revision: https://phab.mercurial-scm.org/D11650

rust: update the rust-cpython crate to 0.7.0 This notably brings support for Python 3.10, and includes the panic message when propagating a Rust panic as a Python exception. https://github.com/dgrunwald/rust-cpython/blob/master/CHANGELOG.md#070---2021-10-09 Differential Revision: https://phab.mercurial-scm.org/D11630

dirstate-v2: Name a constant in the Rust implementation We are about to introduce a Python version of this code that will also need this constant. Differential Revision: https://phab.mercurial-scm.org/D11547

dirstate-v2: Replace the 32-bit `mode` field with two bits Previously we stored the entire value from `stat_result.st_mode`, like dirstate-v1 does. However only the executable permission and type of file (only symbolic links and normal files are supported) are relevant to Mecurial. So replace this field with two bits in the existing bitfield byte. For now the unused space is left as padding, as it will be used for something else soon. Differential Revision: https://phab.mercurial-scm.org/D11635

dirstate-v2: Store unsigned integers inside DirstateEntry The negative marker values are not used anymore. Differential Revision: https://phab.mercurial-scm.org/D11634

dirstate-v2: Truncate directory mtimes to 31 bits of seconds … instead of 64 bits, while keeping the sub-second presision. This brings the size of one timestamp from 12 bytes to 8 bytes. 31 bits is chosen instead of 32 because that’s already what happens for the mtime of files and symlinks, because dirstate-v1 uses negative i32 values as markers. Later we’ll add sub-second precision for file/symlink mtimes, making their dirstate-v2 representation the same as for directories. Differential Revision: https://phab.mercurial-scm.org/D11633

dirstate-v2: Separate Rust structs for Timestamp and PackedTimestamp PackedTimestamp is now exclusively for dirstate-v2 serialization purpose. It contains unaligned big-endian integers. Timestamp is used everywhere else and contains native Rust integers. Differential Revision: https://phab.mercurial-scm.org/D11632

dirstate-v2: Change the representation of negative directory mtime Change it from how I previously thought C’s `timespec` works to how it actually works. The previous behavior was also buggy for timestamps strictly before the epoch but less than one second away from it, because two’s complement does not distinguish negative zero from positive zero. Differential Revision: https://phab.mercurial-scm.org/D11629

dirstate-v2: Only convert from SystemTime to Timestamp and not back Converting from Timestamp back to SystemTime was only used for equality comparison, but this can also be done on Timestamp values. Differential Revision: https://phab.mercurial-scm.org/D11631

dirstate-v2: Swap the order of size and mtime on disk This makes the dirstate-v2 file format match dirstate-v1 for the order of `mode`, `size`, and `mtime`. This order does not matter as long as these components are handled through named fields/attributes in code, but in a few places we still have tuples so having the same order everywhere might help avoid a bug that might not be obvious since those components have the same type. Differential Revision: https://phab.mercurial-scm.org/D11620

dirstate-v2: Document flags/mode/size/mtime fields of tree nodes This file format modification was previously left incomplete because of planned upcoming changes. Not all of these changes have been made yet, but documenting what exists today will help talking more widely about it. Differential Revision: https://phab.mercurial-scm.org/D11625

help: update help text for debug-repair-issue6528 The changegroup fix was put in 5.9.1, this is now out of date. Alson, this can maybe encourage people to upgrade? Differential Revision: https://phab.mercurial-scm.org/D11392

dispatch: use detailed exit code 250 for keyboard interrupt Among our users at Google, we're still seeing several percent of commands fail with exit code 255. I suspect keyboard interrupts is an important remaining reason. This is a resend of D10086 with some fixes for pager handling added ahead of it. Differential Revision: https://phab.mercurial-scm.org/D11628

dispatch: ignore failure to flush ui When the pager dies, we get a `SIGPIPE`. That causes `error.SignalInterrupt` to be raised ` (from `ui._catchterm()`). Any further writes or flushes will cause further `SIGPIPE`s and furhter `error.SignalInterrupt`. If we write or flush outside of the try/except that handle `KeyboardInterrupt` (which `error.SignalInterrupt` is a subclass of), then control will escape from the `dispatch` module. Let's fix that by ignoring errors from flushing the ui. I would have rather fixed this by restoring the stdout and stderr streams when the pager dies, but it gets complicated because of multiple ui instances (ui/lui) and different pager setups between regular hg and chg. This changes a test in `test-pager.t`, but I don't understand why. I would have thought that all the output from the command should have gone to the broken pager. Differential Revision: https://phab.mercurial-scm.org/D11627

dispatch: don't change error status if flushing stdio fails If we already have a non-zero exit code, I don't think we should change it to 255 because we fail to flush stdio. This may not matter yet, but it will matter when I make a killed pager result in exit code 250 (it's currently 255). Differential Revision: https://phab.mercurial-scm.org/D11626

dirstate-v2: Use "byte sequence" in docs The patch originally sent as https://phab.mercurial-scm.org/D11546 used "byte string" but that was changed during review to avoid suggesting Unicode or character encodings. However "byte range" sounds to be like a range of *indices* within a byte string/sequence elsewhere. This changes to "byte sequence". Python docs use "sequence" a lot when discussing the `bytes` type: https://docs.python.org/3/library/stdtypes.html Differential Revision: https://phab.mercurial-scm.org/D11623

rust: Make the hg-cpython crate default to Python 3 This default is used when running `cargo` manually such as for `cargo test`. `setup.py` and `Makefile` both configure the Python major version explicitly. Differential Revision: https://phab.mercurial-scm.org/D11618

packaging: update the certifi dependency Not sure if this helps with recent certificate issues[1], but we might as well keep this modern. Note that certifi no longer claims py2 support, and there's a PR to add it back in[2]. Py2 support was dropped in 2020.04.05.2 (which predates what's being updated here). None of the *.py files have changed since the 2020.6.20 release, and I was able to call `certifi.where()` in both a virtualenv and with `hg debugshell` from an MSI install with this version, so we shouldn't be any worse off than before. [1] https://foss.heptapod.net/mercurial/tortoisehg/thg/-/issues/5745 [2] https://github.com/certifi/python-certifi/pull/154 Differential Revision: https://phab.mercurial-scm.org/D11609

setup: stop packaging python3.dll and python3X.dll in the wheel distribution Now that exewrapper is smart enough to find the DLLs it needs without help from the build script, backout ed286d150aa8 and 2960b7fac966. Note that this will require deleting the build/lib.win-amd64-3.X directory to actually remove it from the final wheel. Differential Revision: https://phab.mercurial-scm.org/D11455

exewrapper: find the proper python3X.dll in the registry Previously, we relied on the default library lookup[1], which for us is essentially to look on `PATH`. That has issues- the Python installations are not necessarily on `PATH`, so I started copying the DLLs locally in 2960b7fac966 and ed286d150aa8 during the build to work around that. However, it's been discovered that causes `python3.dll` and `python3X.dll` to get slipped into the wheel that gets distributed on PyPI. Additionally, Mercurial would fail to run in a venv if the Python environment that created it isn't on `PATH`, because venv creation doesn't copy the DLLs locally. The logic here is inspired by the `py.exe` launcher[2], though this is simpler because we don't care about the architecture- if this is a 32 bit process running on Win64, the registry reflection will redirect to where the 32 bit Python process wrote its keys. A nice unintended side effect is to also make venvs that don't have their root Python on `PATH` work without all of the code required to read `pyvenv.cfg`[3]. I don't see any reasonable way to create a venv without Python being installed (other than maybe building Python from source?), so punt on trying to read that file for now and save a bunch of string manipulation code. I somehow managed to corrupt my Windows user profile, and that makes the Microsoft Store python not run (even loading the DLL gives an access error), so I'm giving priority to both global and user specific python.org installations. Loading python3.dll is new, but when I went down the rabbit hole of implementing `pyvenv.cfg` support, I saw a comment[4] that led me to think we could have trouble if we don't. The comment in ed286d150aa8 confirms this, so we should probably bail out completely if Python3 can't be loaded from the registry, rather than getting something random on `PATH`. But I'll leave that for the default branch. [1] https://docs.microsoft.com/en-us/windows/win32/Dlls/dynamic-link-library-search-order#standard-search-order-for-desktop-applications [2] https://github.com/python/cpython/blob/adcd2205565f91c6719f4141ab4e1da6d7086126/PC/launcher.c#L249 [3] https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/PC/getpathp.c#L707 [4] https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/PC/getpathp.c#L1098 Differential Revision: https://phab.mercurial-scm.org/D11454

fix: reduce number of tool executions By grouping together (path, ctx) pairs according to the inputs they would provide to fixer tools, we can deduplicate executions of fixer tools to significantly reduce the amount of time spent running slow tools. This change does not handle clean files in the working copy, which could still be deduplicated against the files in the checked out commit. It's a little harder to do that because the filerev is not available in the workingfilectx (and it doesn't exist for added files). Anecdotally, this change makes some real uses cases at Google 10x faster. I think we were originally hesitant to do this because the benefits weren't obvious, and implementing it efficiently is kind of tricky. If we simply memoized the formatter execution function, we would be keeping tons of file content in memory. Also included is a regression test for a corner case that I broke with my first attempt at optimizing this code. Differential Revision: https://phab.mercurial-scm.org/D11280

fix: add test to demonstrate how many times tools are executed The current implementation wastes a lot of effort invoking fixer tools more than once on the same file name+content. It is good to track changes in this behavior. Differential Revision: https://phab.mercurial-scm.org/D11279

rhg: add ui.plain() and check it before showing relative paths in status Adds a very basic replica of `ui.plain()`. Differential Revision: https://phab.mercurial-scm.org/D10912

rhg: add relative paths support in `rhg status` Differential Revision: https://phab.mercurial-scm.org/D11614

rhg: refactor function to relativize paths in utils Commands like `files`, `status` supports printing relative paths. Hence we need to re-use this code in other places too. So let's take this out from `rhg files` into a utility function. Next patch will make `rhg status` use it. Differential Revision: https://phab.mercurial-scm.org/D11613

hg: let extensions call the func without populating opts keys This change is to help extensions by not forcing them to populate with opts[b'bundle'] and opts[b'force'] when calling hg.incoming(...) Differential Revision: https://phab.mercurial-scm.org/D11619

rhg: in rhg cat cli, fix the long name of the --rev flag Also tweak the help for the anonymous argument. Differential Revision: https://phab.mercurial-scm.org/D11617

rhg: fallback if tweakdefaults or statuscopies is enabled with status `rhg status` is experimental right now and does not support all functionalities. While the long term target is to implement them, for now we add a fallback to have all tests pass with `rhg status` enabled. Differential Revision: https://phab.mercurial-scm.org/D10906

largefiles: partially undo 61e526585b20e2ff15f19497d0451d18fea02db8 and child Since the largefiles dirstate is now part of transaction, we get rid of this temporary fix which lived for ~7 years. Differential Revision: https://phab.mercurial-scm.org/D11612

largefiles: add tr backup for largefilesdirstate This will help us in automatically restoring the largefilesdirstate if a transaction is rolled back. Differential Revision: https://phab.mercurial-scm.org/D11611

largefiles: pass current transaction to `lfdirstate.write()` Right now, the largefile dirstate is not included in transaction which makes things complex. Next patch will add code to do so, so let's make it mandatory to pass current transaction and pass from all existing callers. Differential Revision: https://phab.mercurial-scm.org/D11610

merge: with stable

dirstate-v2: Add internal documentation It can be viewed by running `hg help internals.dirstate-v2` Since that command rewraps paragraphs, the source text is written with semantic line breaks: https://sembr.org/ Differential Revision: https://phab.mercurial-scm.org/D11546

dirstate-v2: Move data file info in the docket closer together Having `data_size` next to `uuid_size` (and the UUID itself) makes more sense. Differential Revision: https://phab.mercurial-scm.org/D11545

dirstate-item: drop the legacy new_normal constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11608

dirstate-item: replace call to new_normal The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11607

dirstate-item: replace a call to new_normal The constructor is on its way out, so we inline a relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11606

dirstate-item: drop the legacy new_possibly_dirty constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11605

dirstate-item: replace call to new_possibly_dirty The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11604

dirstate-item: drop the legacy new_from_p2 constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11603

dirstate-item: replace call to new_from_p2 The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11602

dirstate-item: drop the legacy new_added constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11601

dirstate-item: replace call to new_added The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11600

dirstate-item: drop the legacy new_merged constructor Nobody is calling it anymore. Its purposes has been filled. Differential Revision: https://phab.mercurial-scm.org/D11599

dirstate-item: replace call to new_merged The constructor is on its way out, so we inline the last relevant call before dropping it. Differential Revision: https://phab.mercurial-scm.org/D11598

dirstate-item: drop the `merged` property It has no user anymore. Differential Revision: https://phab.mercurial-scm.org/D11597

dirstate-item: replace another usage of `merged` We simply spell out the logic here. This was the last usage of `merged`. Differential Revision: https://phab.mercurial-scm.org/D11596

dirstate-item: replace a `merged` usage with `p2_info` It seems more accurate and no test complains (XXX hopefully XXX). Differential Revision: https://phab.mercurial-scm.org/D11595

dirstate-item: drop the `from_p2` property It has no user anymore. Differential Revision: https://phab.mercurial-scm.org/D11594

dirstate-item: directly use `p2_info` in `v1_size` This is simpler. Differential Revision: https://phab.mercurial-scm.org/D11593

dirstate-item: use the `p2_info` property to replace more verbose call Differential Revision: https://phab.mercurial-scm.org/D11592

status: process `from_p2` file the same as `merged` one What matters here is that the file constains information coming from the second parent and should be considered `modified` on plain `hg status. So we can process `from_p2` file sooner. It also highlight that we probably don't need the merged/from_p2 distinction at higher level. Differential Revision: https://phab.mercurial-scm.org/D11591

dirstate: drop an incorrect comment We are actually checking that we are only in a case were the file might needs lookup before doing this. So the comment is not relevant. Differential Revision: https://phab.mercurial-scm.org/D11590

dirstate: drop some duplicated code The same operation is done a handful a line lower. Differential Revision: https://phab.mercurial-scm.org/D11589

dirstate: align the dirstate's API to the lower level ones This conclude the refactoring of this API. We can now finalize the dirstate v2 on disk format. Differential Revision: https://phab.mercurial-scm.org/D11587

dirstate-item: introduce a `p1_tracked` property It is useful to simplify various conditional that use `any_tracked and not added`. Differential Revision: https://phab.mercurial-scm.org/D11586

dirstate-item: introduce a `p2_info` property that combine two others The `merged` and `from_p2` property are always used together so we can expose a combined property instead. Differential Revision: https://phab.mercurial-scm.org/D11585

dirstate: narrow gathering of parent data The parent data are only going to be useful is the file might be clean. And it might only be clean if it is tracked in both p1 and the working copy. Differential Revision: https://phab.mercurial-scm.org/D11584

dirstate: align the dirstatemap's API to the data change We are passing different data, so lets simplify the dirstatemap API too. Differential Revision: https://phab.mercurial-scm.org/D11583

dirstate-v2: Store a bitfield on disk instead of v1-like state Differential Revision: https://phab.mercurial-scm.org/D11558

dirstate-item: change the internal storage and constructor value This should be closer to what we do need and what we can actually reliably record. In practice it means that we abandon the prospect of storing much more refined data for now. We don't have the necessary information nor code using it right now. So it seems safer to just use a clearer version of what we had so far. See the documentation changes for details. Differential Revision: https://phab.mercurial-scm.org/D11557

dirstatemap: drop legacy method on the dirstatemap wrapper They are no longer in use now that the Rust wrapper version of the Dirstatemap are back in line with the Python one. Differential Revision: https://phab.mercurial-scm.org/D11582

dirstatemap: align the Rust wrapper implementation of `setparent` We move to using `drop_merge_data` as intended. Differential Revision: https://phab.mercurial-scm.org/D11581

dirstatemap: fix copymap.pop in Rust to return the value it pops I guess this was overlooked in the initial implementation? Without this, the next patch would, loose copy information in setparent. Differential Revision: https://phab.mercurial-scm.org/D11580

dirstate-item: implement `drop_merge_data` on the Rust DirstateItem It was currently missing and we want to be able to use in it the Rust case too. Differential Revision: https://phab.mercurial-scm.org/D11579

dirstatemap: use a common implement for reset_state Same logic as for `set_untracked` this make sure both implementation are aligned. The `reset_state` implementation for the Rust wrapped had signicantly diverged, this change finally put it back in line. Differential Revision: https://phab.mercurial-scm.org/D11578

dirstatemap: add a common `_drop_entry` method for dirstatemap This method is called to remove DirstateItem from the map. Each variant have a different implementation (which is … the point). Differential Revision: https://phab.mercurial-scm.org/D11577

dirstatemap: use common code for set_clean Same logic before this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11576

dirstatemap: use common code for set_possibly_dirty Same logic before this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11575

dirstatemap: use a common implement for set_tracked Same logic as for `set_untracked` this make sure both implementation use the same logic for this. Differential Revision: https://phab.mercurial-scm.org/D11574

dirstatemap: add a common `_insert_entry` method for dirstatemap This method is called to add a new DirstateItem to the map. Each variant have a different implementation (which is … the point). Differential Revision: https://phab.mercurial-scm.org/D11573

dirstatemap: use a common implementation for `dirstatemap.set_untracked` We can now make sure they use the same code, and drop the older, out of sync, implementation of `set_untracked` for the rust wrapper. Differential Revision: https://phab.mercurial-scm.org/D11572

dirstatemap: add a common `_refresh_entry` method for dirstatemap This method is called once a DirstateItem have been modified to apply the change on the dirstatemap if necessary. Each variant have a different implementation (which is … the point). We use `addfile` for the rustmap and not `set_dirstate_item` because we need to keep the internal counter up to date and `set_dirstate_item` does not do it. Differential Revision: https://phab.mercurial-scm.org/D11571

dirstatemap: create `_dirs_incr/_dirs_decr` methods on the common class The Rust wrapper does not need them. However having a default, no-op, implementation will help use to write code used by both implementation. Differential Revision: https://phab.mercurial-scm.org/D11570

dirstatemap: small rework of the `set_untracked` method This shuffle the code a bit to have it flowing more "naturally". This will help us to create a common version of this code in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D11569

dirstatemap: arrange methods by category The dirstatemap code cover various aspects, it grow a bit messy over the years. So we shuffle the code around into some documented categories. This will help use to clean up the code. No code was changed in this changeset, only code move. Differential Revision: https://phab.mercurial-scm.org/D11568

dirstatemap: move a multiple simple functions in the common class These are small and simple, lets factor them out. Differential Revision: https://phab.mercurial-scm.org/D11567