store: use StoreEntry API instead of parsing filename in remotefilelog This is more explicit and more robust.

store: use StoreEntry API instead of parsing filename in narrow This is more explicit and more robust.

store: add a `target_id` attribute on RevlogStoreEntry This hold the "target" (file, directory, etc) of a revlog. Having this available will help a lot of code to avoid direct file path access.

store: actually tag tree manifest revlogs as manifest revlogs It turn out we have been mislabeling these for a long while. This is now fixed.

store: also gather files per revlog in `topfiles` This conclude out revlog gathering.

store: also group files by revlog in fncache version of datafiles One more step.

store: add logic to group revlog file together For now each file get its own entry, this will help stopping this, soon™. We use such gathering in the `basicstore` code.

store: change `_walk` return to `(filename, (type, size))` If we are to group file per revlog, having the filename as the "main key" will be useful. This change will make the following changes clearer.

store: lazily get file size on demand for the fncache case We don't have the information in the first place, so we can avoid querying the file system inconditionnaly for use case we don't needs it. This change requires the StoreFile class to be passed a vfs to retrieve the file_size if needed. In the non-fncache case, we already have the information from file system walking, so we keep it and use it.

store: only access is_volatile information through the file object This make sure other code only access this information through the proper API, and it prepare for store entries to be able to agregate multiple files.

store: only access file_size information through the file object This make sure other code only access this information through the proper API, and it prepare for store entries to be able to agregate multiple files.

store: have custom init for entries class This will get useful to add special processing later in this series.

store: use specialized class for store entries We introduce two different classes for revlog and other entries. For now, we still have multiple entry for the same revlog, but we will work toward grouping the different file in a single entry in this series. Having the distinction is a step toward this goal.

store: introduce a EntryFile object to actually access file info For now a StoreEntry match a single file, but the goal is to eventually combine multiple file in a higher level Entry, so we need to introduce this distinction and use it first.

store: use a StoreEntry object instead of tuple for store files We want to make the store return more semantic information instead of a stream of file path. To achieve this, we start with adding a simple object that hold the same information as the tuple it replace, and do a simple update to the user code to fetch and use the same information. From there, we will be able to iteratively upgrade the codebase toward better objects.

store: no longer explicitly mark the splitted index as ignored Since the fncache is going to automatically recognised it as non "normal" revlog file, we don't need to explicitly make it ignored.

store: cleanup what is recognized as a revlog file The temporary file from `censor` are not relevant to the store as they are just temporary. So let `fncache` and `datafiles` forget about them.

store: only inclure revlog related file in `datafiles` This is a "reading" equivalent of the previous patches (see that patch for details).

store: do not record file that are not part of a revlog in fncache The fncache exist to list file to copy/stream when cloning a repository, it should only contains file that are relevant for a revlog in such case. For example, temporary file are not relevant. So we now skip the addiction of non-relevant file to the fn-cache in the first place.

revlog-split: expand the some test to cover the success case too It seems useful to double check the case of a transaction successfully committed before trying the one with transaction abort.

transaction: use a ".bck" extension for all backup file There are too much code in Mercurial that assume file function from extension. Keeping the original extension when doing backup is just creating confusion (for exemple, backup of revlog's confused with new version revlogs). So we change the backup name from now on to be explicilty backup file and prevent this kind of problem. (note that repository using fncache is less affected (but still affected) by this kind of problem as the backup are not listed in the fncache).

repair-issue6528: remove a now useless continue The condition for that continue have been excplicly filtered out of the `files` list a handful of line earlier.

branching: merge stable into default

stabletailgraph: clarify omission of linear parts in test sorts Those are not interesting and are collapsed using globs. Only the tagged nodes are interesting.

stabletailgraph: omit uninteresting linear parts in test log The test uses tags for the nodes of interest. There is no need to print the linear parts which exist only to force the rank.

stabletailgraph: fix typo in test

stabletailgraph: fix terminology in doc

perf: introduce a `perf::stream-locked-section` command This command benchmark the initial part of a stream clone, where the repository is locked.

transaction: tr._names are actually bytes, use byte string to join them Looks like this is never checked in the tests? But trying e.g. `print(tr)` by hand before this change would give: TypeError: sequence item 0: expected str instance, bytes found Looks like tr._names are always bytes (although this isn't actually enforced, but maybe at some point it will at least be type checked e.g. by pytype).

nix: add nix-defined package and devel env (flake) This adds a Nix Flake which defines: - package recipies to build and run Mercurial (C, C+Rust variants), - a meta package to run the test suite in an isolated sandbox, - a development environment with some tools (notably Python and the Black formatter, pinned to match the CI). Python is pinned to the recommended version. The Rust toolchain is a fairly recent one provided by the Nixpkgs channel. It is not yet pinned to the same version as the "reference" Debian package, but this does not seem to cause any issue. Example usage of local commands are provided in the `flake.nix` file. Once merged, it should also be possible to pull and run directly from the source repository directly with something like: `nix run hg+https://foss.heptapod.net/mercurial/mercurial-devel?ref=TOPIC&dir=contrib/nix' -- version`

hgignore: ignore nix result symlink

tests: add test ignore list for nix Some tests cannot run properly in the nix sandbox. This adds a list of tests to ignore when running in that context.

ui: keep the progress bar around when writing if stdout is not a tty

tests: in filterpyflakes, tolerate non-ascii file contents

comments: fix spelling

heptapod-ci: display python version it will be useful to debug in the CI

branching: merge stable into default

sha1dc: Make sure SHA1DC_BIGENDIAN is set on Darwin/PowerPC __BYTE_ORDER__ is not defined on old toolchains so sha1dc gets built in little endian mode on a big endian platform, hilarity ensues. Tested on Mac OS X Tiger 10.4.

hidden: add support to explicitly access hidden changesets with SSH peers This implements support for using --remote-hidden with an SSH server. The remote `hg serve --stdio` call is passed the `--hidden` flag as a request to access hidden changesets. This approach has benefits similar to the one we used for HTTP peers. It * works around the lack of global parameters in wire protocol v1, * reuses the `--hidden` flag (that does not use the wireproto), and * can be safely ignored by older client (fitting the best effort contract). Same as for HTTP, the feature is experimental so we have all the room we needs to update the implementation in the future if deemed necessary. The SSH version of the `--remote-hidden` config uses the same configuration as the HTTP support to control the access to this feature. The name of the user running the command is used for the checking. Test written by Pierre-Yves David.

hidden: support passing --hidden with `serve --stdio` This currently has no effect since the server code will filter the repository again. However, it is clearer to put this change in its own changeset. This will mostly impact ssh peers, clients being now able to pass --hidden when spawning the server. See the next changeset for details on how this is going to be used.

hgweb: move ismember from `hgweb.common` to `scmutil` To implement hidden access control for other protocol than http, we need code outside of hgweb to access this logic.

hidden: add support for --remote-hidden to HTTP peer Test written by Pierre-Yves David.

hidden: add support to explicitly access hidden changesets via HTTP We now detect hidden access request and validate them while handling HTTP wireprotocol command. This changeset only introduces server support. Client support is added in a subsequent changeset. The mechanism we use for the HTTP wireprotocol is the same as the one used to lift filtering for hgweb. The use of the global `access-hidden` parameter in the http url is a simple and effective approach that * works around the lack of global parameters in wire protocol v1, * reuses the mechanism used for hgweb (that does not use the wireproto), and * can be safely ignored by older client (fitting the best effort contract). The feature is still experimental, so we can easily update the implementation details in the future. However, landing a first implementation now will help test the feature in the wild and iterate on alternative implementations if deemed necessary.

hidden: support for explicitly accessing hidden changesets in wireproto server This installs the basic infrastructure to be able to access hidden changeset through the wireprotocol. Each wireprotocol server still needs a way to define how the feature is triggered.

hidden: add support for --remote-hidden to localpeer

pull: add --remote-hidden option and pass it through peer creation This option will allow to pull changesets that are hidden on the remote. This is useful when looking into a changeset’s evolution history, resolving evolution instability or mirroring a repository. The option is best effort and will only affect the pull when it can. The option will be ignored when it cannot be honored. Support for each type of peer is yet to be implemented. They currently all warn about lack of support. The warning code will get removed as peers gain support for this option. The option is still experimental, so we will have freedom to update the UI or implementation before it graduates out of experimental. Based on a changeset by Pierre-Yves David, which added the option.

hgweb: add support to explicitly access hidden changesets This changeset adds a "global" `access-hidden` argument to hgweb. This argument lift the "hidden" filtering. This means the request has access to hidden (eg: obsolete) changesets. Secret changesets remains filtered. This feature has multiple applications. The first main use case is to allow the hgweb interface to display more obsolescence related data, such as the Anton Shestakov work to add `obslog` support to hgweb. The second foreseen usecase is support for a `--remote-hidden` argument to `hg pull` and `hg clone`. This flag will make it possible to retrieve hidden (typically obsolete) changeset under some conditions. This is useful when digging up obsolescence history or when doing full mirroring. More on this feature coming in later changesets. To avoid exposing information by mistake, access to this feature is currently controlled with the `experimental.server.allow-hidden-access` config option. The option works the same way as `web.allow-push`. The current default is to not allow any hidden access. However we might change it before the feature stop being experimental.

clone-bundles: add an option to generate bundles in the background This is what the "on-change" version have been doing from the start, it seems useful to also offer this option in the command.

clone-bundles: add a command to clear all bundles

clone-bundles: add more test cases for automatic generation See inline documentation for details.

clone-bundles: introduce a command to refresh bundle This command allows for a different mode than the one attached to the transation. It might fit some hosting setup better.

clone-bundles: add a configuration to control auto-generation on changes We are about to introduce a "manual" way to deal with automatic clone management but running a command using some internal API. The first step is to introduce a way to control the "on changes" behavior.

clone-bundles: also control automation based on absolute number of revisions

clone-bundles: only regenerate the clone bundle when cached ration is low See inline documentation for details.

clone-bundles: garbage collect older bundle when generating new ones See inline documentation for details.

clone-bundles: add a basic first version of automatic bundle generation See inline documentation for details.

branching: merge stable into default

tests: automatically glob the discovery timing information Time is not stable in tests.

rhg: support `status --print0` This seems very easy to support, and useful because it makes it possible to parse the [hg status] output even if the user creates files with '\n' characters by accident.

stabletailgraph: implement stable-tail sort This adds the computation of the "stable-tail sort", an incremental node sorting method. It is a stepping stone for the implementation of faster label discovery (for example for obs markers) and more caching.

heptapod: add `.gitattributes` file to improve language detection I am fully aware of the irony.

peer: rename makepeer() → _make_peer() In httppeer and sshpeer, there previously were makepeer() and make_peer(), which was confusing. Therefore, this changeset renames one of the functions. makepeer() was the internal function called by make_peer() and some debug command. This function is renamed to _make_peer().

rust: configure MSRV in Clippy This setting makes Clippy never apply lints that are meant for later versions. In case the target precise toolchain is the one running, it does not make a difference, but this gives us a machine-parseable specification that is pretty standard. The README and `hg help rust` are updated to state that `clippy.toml` is the single source of truth about that, also lifting a minor ambiguity: it is fine if the MSRV is lagging behind the version in Debian testing.

rust-readme: mentioned that format check is enforced by CI

rust-readme: mentioning clippy especially since there is a CI check for it.

rust-readme: rst fixes

rustdoc: nodemap doc refreshing Not pretending to be comprehensive. - correcting some inconsistencies - adding a few missing doc-comments - adding more cross references (in some cases it's right beside the current documentation item, but it will nevertheless also be useful, because `rustdoc` will warn us if inconsistencies arise).

rustdoc: fixed or introduced crossrefs in nodemap.rs

rustdoc: summary line for hg_path_to_os_string The main motivation of this change is to avoid the TODO being the summary line, even though this leads to a pretty obvious summary. Then doc-comments for the other functions are introduced for consistency.

rustdoc: wording for checkexec Notably separating the summary line for correct display at module level.

rustdoc: fixed warnings about links This is the minimal fix making those that actually were supposed to be links to work (including in private items).

rust-changelog: introduce ChangelogEntry parent entries accessors Straightforwards now that lifetimes are explicit in `RevlogEntry` parent accessors.

rust-revlog: fix lifetime problem for RevlogEntry parent entries accessors Without this, the lifetime of the result is equated to the lifetime of the `self` reference, preventing callers, e.g., to take a `RevlogEntry` and return its `p1_entry()`, as it looks like returning something that does not outlive the *reference to* the `RevlogEntry`.

rust-revlog: explicit naming for `RevlogEntry` lifetime This matches what has been done in `revlog::changelog::ChangelogRevisionData`, and has the advantage of making things clearer when we introduce other, shorter lived lifetimes.

rust-changelog: introducing an intermediate `ChangelogEntry` Before this change, client code needing to extract, e.g, the Node ID and the description from a changeset had no other choice than calling both `entry_for_rev()` and `data_for_rev()`. This duplicates some (limited) computation, and more importantly imposes bad hygiene for client code: at some point of developement, the client code would have to pass over both entry and data in its internal layers, which at some point of development would raise the question whether they are consistent. We introduce the intermediate `ChangelogEntry` from which both conversion to the generic `RevlogEntry` and extraction of `ChangelogRevisionData` are possible. It might grow some convenience methods in the future. We keep the `data_for_rev()` method of `Changelog` for compatibility, pointing users at the more powerful alternative.

rust-changelog: added a test for `NULL_REVISION` special case The result is due to `Revlog.get_rev_data()` returning an empty byte string for `NULL_REVISION`, followed by special case for emtpty byte strings in `ChangelogRevisionData::new()`.

rust-changelog: made doc-comments more consistent The most important is the one about `data_for_rev`, that looked like a copy-paste leftover (got me confused first time I read this code, before I actually learned there were both `Entry` and RevisionData`. In the comment for the `struct`, "changelog" was probably more about the format in general (as documented elsewhere) than as an identifier. Some of the "Return something" had "of", half had "for".

extras: re-use Projection from jaraco.collections

extras: expose 'retained_extras' for extensions to extend

extras: extract function for projecting names from a gettable object

hgweb: add "children" into the JSON template for a changeset The children are shown in the HTML templates, so I don’t see a reason against including them in the JSON template as well.

branching: merge stable into default

help: fix formatting for http config section

branching: merge stable into default

fastexport: rework newline logic Newlines should only be added when otherwise the stream would look weird without them (on blobs), therefore they are the exception. Flip the logic so they are added, not skipped.

encoding: avoid quadratic time complexity when json-encoding non-UTF8 strings Apparently the code uses "+=" with a bytes object, which is linear-time, so the whole encoding is quadratic-time. This patch makes us use a bytearray object, instead, which has a(n amortized-)constant-time append operation. The encoding is still not particularly fast, but at least a 10MB file takes tens of seconds, not many hours to encode.

bundle: prevent implicite bundling of internal changeset Now that the two mains source of on-disk bundle are preventing the inclusion of internal changesets in their bundling. We can add a lower level check that would prevent any other leakage of internal-phase changesets. We keep the door open to some usage, like the temporary bundle using during strip for example.

bundle: abort if the user request bundling of internal changesets See inline comments for details.

strip: do now include internal changeset in the strip backup See inline comment for details.

revset: add `_internal()` predicate This predicate help core code to select internal changeset (in the internal phase).

strip: drop the assert about bundled revision This is working well, so no need for extract computation. See the previous changeset for the rationnal of still have these assert around for a short interval.

strip: explicitly compute the boundary of the backup bundle We want to make change to the set of backed up revision in a future changeset, we start with a change of the computation without any changes in the semantic to clarify later changeset. The could of costly assert are here to testify that the result is still correct. They will be removed in the next changesets, but I wanted them in this changeset to help in case someone bisect a regression to this changeset in the future.

outgoing: fix common-heads computation from `missingroots` argument When initializing a `outgoing` object, the `common set` can be defined explicitly (with the `commonheads` argument`) or implicitly (with the missingroots arguments). It turns out the logic to compute `commonheads` from `missingroots` is buggy, as it does not consider the parents of enough changesets. Previously, it only considered parents of "missingroots` items, while it need to consider all parents of missing. Here is an example: F |\ C E | | B D |/ A If we use [E] as missing-roots, the missing set is [E, F], and the common-heads are [C, D]. However you cannot only consider the parent of [E] to find them, as [C] is not a parent of [E]. This already fix the bundle generated in one test, and it would prevent many other to misbehave with future change from this series.

bundle: include required phases when saving a bundle (issue6794) We now properly computes and includes phases above secret in bundle, previously, they would be skipped, and then the code computing them would crash. Note that from this changeset, we also include the heads associated with the changegroup's "target" phase. This turned out to be necessary to ensure the movement of changeset included in the bundle, but already known locally. This explain why lines for "secret" heads appears in multiple tests.

bundle: add test bundling changeset in the "archived" phase Having this test highlight that phase bundling above secret is significantly broken. Fixes coming in the next changesets.

bundle: introduce a "v3" spec This is a small update from "v2", we use the freshly out of experimental changelog version "03' format (alignment with bundle version is a coincidence) and we now bundle phases by default. We shall use v3 as the default bundle type later, when changegroup version '03' is a bit more established