scmutil: obsrevs is already a frozenset Differential Revision: https://phab.mercurial-scm.org/D12157

obsolete: always return frozensets from obsolete.getrevs() getrevs function already returns an empty frozenset when there is no obsstore, but let's make sure to return a frozenset in any case. This makes it possible to use the result of this function as a dict key or provide it to hash() built-in function without any conversions. Differential Revision: https://phab.mercurial-scm.org/D12156

revlog: split revlog v1 and revlog v2 handling Explicitly splitting their fields packing and unpacking makes it easier to extend the existing C implemenation to handle the new changelog format, whose fields and offsets are not simply a superset of the revlog. Differential Revision: https://phab.mercurial-scm.org/D12137

revlog: extract entry byte offsets into named constants Labelling the fields pointed by the given offsets shared by revlog v1 and v2. Differential Revision: https://phab.mercurial-scm.org/D12136

doc: inspect.getargspec has been removed in Python 3.11 Fixed by dropping the inspection introduced in cdda48c93676. The 2nd "reporter" parameter to docutils.languages.get_language has been available since 0.8 more than 10 years ago. Reported for Fedora on https://bugzilla.redhat.com/show_bug.cgi?id=2022252#c2 .

branchmap: split a long condition in branchcache.validfor(), add comments Differential Revision: https://phab.mercurial-scm.org/D12138

branchmap: don't add branch entries if there are no heads We definitely don't want any empty entries to be present in repo.branchmap() just for the sake of not breaking test-notify.t. No test changes required because the previous patch made notify extension to not raise any tracebacks in case of RepoLookupErrors. Differential Revision: https://phab.mercurial-scm.org/D12135

notify: don't produce errors if a revision is not found Notify extension has a way to only subscribe to a specific revset, such as "branch(foo)". Before this patch, when there was no branch with that name, it would produce a traceback saying "unknown revision: foo". With this patch it would no longer do that, and instead it'll assume there are no revisions that match this revset. I think this patch is an improvement in general, but there's a reason I'm sending it now. test-notify.t has a test case where it obsoletes the only revision on a branch, and previously that wouldn't produce any complications, because head computation wasn't obsolescence-aware. Now if the only revision on a branch is obsolete, repo should not see that branch at all. That branch will still be present in branchcache (with an empty list of revisions) until the next patch. Differential Revision: https://phab.mercurial-scm.org/D12134

narrow: allow merging non-conflicting change outside of the narrow spec We use the mergestate to carry information about these merge action and reprocess them at commit time to apply the necessary update. The dirstate itself is never affected and remains "pure", with content only in the narrow-spec. This file involved in such merge are therefor not listed in `hg status`. The current testing is based on a modification of the previous testing, that refused to do such merges. As a result it is a bit simple and more extensive code and testing testing will have to be introduced later. I am planning to do this extra testing, soon. In addition, this only works for flat manifest. Support for tree manifest will need more work. I am not currently planning to do this work. Differential Revision: https://phab.mercurial-scm.org/D12119

merge-actions: add some information about the "changes" the action do This will be useful when processing merges action outside of the narrow-spec. "support" outside of narrow file on commit Differential Revision: https://phab.mercurial-scm.org/D12118

merge-actions: have an attribute for narrow safetiness This allow the core doing narrow filtering to process action without explicitely listing all possible actions. This is important to make the actions system more flexible in the future. Differential Revision: https://phab.mercurial-scm.org/D12117

merge-actions: add an explicite "no_op" attribute This make the MergeAction smarter and able to describe themself. This is useful to help introducing more MergeAction object that better the complexity of the situation. Differential Revision: https://phab.mercurial-scm.org/D12116

merge-actions: gather all created action into a set This is mostly to demonstrate we can do this before we start adding more specialized set. Differential Revision: https://phab.mercurial-scm.org/D12115

merge-actions: make merge action a full featured object This open the way for having "smarter" value as action, making the usage code simpler and more flexible. We have to explicitly use __bytes__ call in a couple of place because Python2… Differential Revision: https://phab.mercurial-scm.org/D12114

convert: use the merge action constant The constant exists, lets use them. Otherwise we cannot make these constant more powerful. Differential Revision: https://phab.mercurial-scm.org/D12113

large-file: use the merge action constant The constant exists, lets use them. Otherwise we cannot make these constant more powerful. Differential Revision: https://phab.mercurial-scm.org/D12112

merge: stop using merge action for pathconflict option This is not the b'r' you are looking for. Differential Revision: https://phab.mercurial-scm.org/D12111

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12110

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12109

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12108

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12107

merge: break up two not-so-one-liner for extra readability Differential Revision: https://phab.mercurial-scm.org/D12106

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12105

merge: break up a not-so-one-liner for readability (even if not fully satisfied this time) Differential Revision: https://phab.mercurial-scm.org/D12104

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12103

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12102

merge: break up a not-so-one-liner for readability Differential Revision: https://phab.mercurial-scm.org/D12101

phabricator-refresh: add a magic value to skip it in the CI When maintaining (light) fork, it is useful to be able to disable this. Differential Revision: https://phab.mercurial-scm.org/D12133

interfaces: add missing caches kwarg of localrepo.updatecaches() Added to localrepo: in 1337bfaa88ca, added to the interface: now. Differential Revision: https://phab.mercurial-scm.org/D12092

obsolete: don't import from .node twice See d55b71393907. Differential Revision: https://phab.mercurial-scm.org/D12121

dagop: don't import nullrev from .node twice See 59fa3890d40a. Differential Revision: https://phab.mercurial-scm.org/D12120

stream-clone: allow to change persistent-nodemap format during stream clone Persistent nodemap affect the store format. However it is fairly isolated and fast to generate locally. So not making it a fixed part of the stream clone is useful. This allow clients without persistent-nodemap support (default for client without Rust enabled, or simply older client). So it make it possible to enable persistent nodemap on client, where it can provide a massive boost. without too much consequence. To do so, we stop using it in the advertisement requirements for streaming and let the client add/remove the necessary file depending of its configuration. We still send the files as it seems like a small save to not regenerate them. In addition, the way we match them will overlap with the changelog-v2/revlog-v2 so we can't simply skip the associated patterns. Differential Revision: https://phab.mercurial-scm.org/D12096

stream-clone: add 5000 changesets to test-clone-stream-format This make sure the changelog is not inlined. We needs this to test coming behavior change around persistent-nodemap. Differential Revision: https://phab.mercurial-scm.org/D12095

debugbuilddag: add a flag to allow running it from a non-empty repository Allow that by default seems "dangerous", but having a flag to make it possible will be useful to help building some repository incrementally. The newly introduced support is basic, but already useful. Differential Revision: https://phab.mercurial-scm.org/D12094

persistent-nodemap: properly delete all nodemap files on downgrade Previously, we were only deleting the docket file… Differential Revision: https://phab.mercurial-scm.org/D12093

destutil: if wdp is obsolete, update to the closest non-obsolete ancestor As the original comments suggest, using prune as a model here was an existing idea, and now this patch implements it. I think it would be even better to do what solveobswdp() from evolve does (in short, it allows to update to a successor of the closest ancestor even if that ancestor is obsolete), but that is outside of this series' scope. Differential Revision: https://phab.mercurial-scm.org/D12099

discovery: port _postprocessobsolete() changes from evolve, add tests Let's make this function obsolescence-aware as well. Now we check that revisions are not obsolete first, and only then check that they are going to be common after exchange. The tests make sure that rewriting changesets doesn't confuse head computation. They rely on experimental.single-head-per-branch feature to show that remote correctly sees the expected result (i.e. either aborts the push or allows it to succeed). They are ported from evolve as well. Differential Revision: https://phab.mercurial-scm.org/D12098

branchmap: skip obsolete revisions while computing heads It's time to make this part of core Mercurial obsolescence-aware. Not considering obsolete revisions when computing heads is clearly what Mercurial should do. But there are a couple of small issues: - Let's say tip of the repo is obsolete. There are two ways of finding tiprev for branchcache (both are in use): looking at input data for update() and looking at computed heads after update(). Previously, repo tip would be tiprev of the branchcache. With this patch, an obsolete revision can no longer be tiprev. And depending on what way we use for finding tiprev (input data vs computed heads) we'll get a different result. This is relevant when recomputing cache key from cache contents, and may lead to updating cache for obsolete revisions multiple times (not from scratch, because it still would be considered valid for a subset of revisions in the repo). - If all commits on a branch are obsolete, the branchcache will include that branch, but the list of heads will be empty (that's why there's now `if not heads` when recomputing tiprev/tipnode from cache contents). Having an entry for every branch is currently required for notify extension (and test-notify.t to pass), because notify doesn't handle revsets in its subscription config very well and will throw an error if e.g. a branch doesn't exist. - Cloning static HTTP repos may try to stat() a non-existent obsstore file. The issue is that we now care about obsolescence during clone, but statichttpvfs doesn't implement a stat method, so a regular vfs.stat() is used, and it assumes that file is local and calls os.stat(). During a clone, we're trying to stat() .hg/store/obsstore, but in static HTTP case we provide a literal URL to the obsstore file on the remote as if it were a local file path. On windows it actually results in a failure in test-static-http.t. The first issue is going to be addressed in a series dedicated to making sure branchcache is properly and timely written on disk (it wasn't perfect even before this patch, but there aren't enough tests to demonstrate that). The second issue will be addressed in a future patch for notify extension that will make it not raise an exception if a branch doesn't exist. And the third one was partially addressed in the previous patch in this series and will be properly fixed in a future patch when this series is accepted. filteredhash() grows a keyword argument to make sure that branchcache is also invalidated when there are new obsolete revisions in its repo view. This way the on-disk cache format is unchanged and compatible between versions (although it will obviously be recomputed when switching versions before/after this patch and the repo has obsolete revisions). There's one test that uses plain `hg up` without arguments while updated to a pruned commit. To make this test pass, simply return current working directory parent. Later in this series this code will be replaced by what prune command does: updating to the closest non-obsolete ancestor. Test changes: test-branch-change.t: update branch head and cache update message. The head of default listed in hg heads is changed because revision 2 was rewritten as 7, and 1 is the closest ancestor on the same branch, so it's the head of default now. The cache invalidation message appears now because of the cache hash change, since we're now accounting for obsolete revisions. Here's some context: "served.hidden" repo filter means everything is visible (no filtered revisions), so before this series branch2-served.hidden file would not contain any cache hash, only revnum and node. Now it also has a hash when there are obsolete changesets in the repo. The command that the message appears for is changing branch of 5 and 6, which are now obsolete, so the cache hash changes. In general, when cache is simply out-of-date, it can be updated using the old version as a base. But if cache hash differs, then the cache for that particular repo filter is recomputed (at least with the current implementation). This is what happens here. test-obsmarker-template.t: the pull reports 2 heads changed, but after that the repo correctly sees only 1. The new message could be better, but it's still an improvement over the previous one where hg pull suggested merging with an obsolete revision. test-obsolete.t: we can see these revisions in hg log --hidden, but they shouldn't be considered heads even with --hidden. test-rebase-obsolete{,2}.t: there were new heads created previously after making new orphan changesets, but they weren't detected. Now we are properly detecting and reporting them. test-rebase-obsolete4.t: there's only one head now because the other head is pruned and was falsely reported before. test-static-http.t: add obsstore to the list of requested files. This file doesn't exist on the remotes, but clients want it anyway (they get 404). This is fine, because there are other nonexistent files that clients request, like .hg/bookmarks or .hg/cache/tags2-served. Differential Revision: https://phab.mercurial-scm.org/D12097

obsolete: make sure windows tests pass when stat() is given a URL This is a temporary fix for an issue that's only visible on windows, but exists on other platforms as well. The issue is if we're trying to use obsstore from a remote peer and that peer is a static HTTP repo, vfs tries to os.stat() a remote file using a URL to that file (e.g. http://localhost/repo/.hg/store/obsstore). The next patch in this series makes branchcache obsolescence-aware, so in certain situations exchange process will try os.stat()ing a URL. On windows this will produce an OSError that is not ENOENT, but EINVAL instead (because of `:` symbol, for example). Differential Revision: https://phab.mercurial-scm.org/D12100

transaction: do not rely on a global variable to post_finalize file We can just add a new argument to the `addfilegenerator` function. This is more explicit and therefor clearer and less error prone. Differential Revision: https://phab.mercurial-scm.org/D12125

dirstate: introduce a "tracked-key" feature A new format variant is introduced. When used, a `tracked-key` file will be generated. That file will be update when the set of tracked file might have changed. This will be useful for external automation (e.g. build tool) to be notified when the set of relevant files changes. One of the motivation for this changes is to mitigate effect dirstate-v2 has on such automation. Since the dirstate file is updated much more frequently on dirstate-v2, monitoring update to that file is no longer a viable strategy. See the associated documentation for details about the feature To prevent older client to update the repository without updating that file, a new requirements is introduced. The `postfinalizegenerators` business is a bit weird, so I'll likely clean that up soon. Differential Revision: https://phab.mercurial-scm.org/D12124

dirstate: rename the filegenerator used for writing We will need a different name in the next changesets. Changing the name is actually not that trivial so we do it in its own changeset. Differential Revision: https://phab.mercurial-scm.org/D12123

dirstate: use a context manager when writing the dirstate This make sure the file is closed in a timely manner. We define a lambda for the file opening. It might seems a bit overkill here but a future changeset will need to do more of those. Differential Revision: https://phab.mercurial-scm.org/D12122

branching: merge stable into default

Added signature for changeset 75676122c2bf

Added tag 6.0.2 for changeset 75676122c2bf

relnotes: add 6.0.1 and 6.0.2 notes Differential Revision: https://phab.mercurial-scm.org/D12130

branchmap: stop writing cache for uncommitted data If we are about to write the branch while a transaction is active. we delay that write. After the transaction is closed, we flush all the write we delayed (unless they have been written in between). Differential Revision: https://phab.mercurial-scm.org/D12128

transaction: add a way to know a transaction has been finalized This will be useful to fix the timing of the branchmap on disk caching. Differential Revision: https://phab.mercurial-scm.org/D12127

branchmap: Add a test about writing branchmap and aborted transaction It turn out that we update the branchmap on disk whenever we recompute it… including in a middle of a transaction. This means writing the new branchmap too early (the changeset are not visible yet) and worse… it mean writing branchmap for transaction we rollback. so we introduce a test to highlight the issue (and prevent it to regress in the future). Differential Revision: https://phab.mercurial-scm.org/D12126

dirstate-v2: rename the configuration to enable the format The rename of the old experimental name was overlooked before the 6.0 release. We rename everything to use the new name (and keep the released name as an alias for compatibility). Differential Revision: https://phab.mercurial-scm.org/D12129

arbitraryfilectx: use our existing helpers for reading and writing files Differential Revision: https://phab.mercurial-scm.org/D12090

fix: remove unnecessary and overly strict check for divergence `rewriteutil.precheck()` checks for divergence these days, so we can remove the redundant check in `hg fix`. Differential Revision: https://phab.mercurial-scm.org/D12088

encoding: fix trim() to be O(n) instead of O(n^2) `encoding.trim()` iterated over the possible lengths smaller than the input and created a slice for each. It then calculated the column width of the result, which is of course O(n), so the overall algorithm was O(n). This patch rewrites it to iterate over the unicode characters, keeping track of the length so far. Also, the old algorithm started from the end of the string, which made it much worse when the input is large and the limit is small (such as the typical 72 we pass to it). You can time it by running something like this: ``` time python3 -c 'from mercurial.utils import stringutil; print(stringutil.ellipsis(b"0123456789" * 1000, 5))' ``` That drops from 4.05 s to 83 ms with this patch (and most of that is of course startup time). Differential Revision: https://phab.mercurial-scm.org/D12089

stream-clone: filter possible missing requirements using all supported one The `supportedformat` requirements is missing some important requirements and it seems better to filter out with all requirements we know, not just an "arbitrary" subset. The `supportedformat` set is lacking some important requirements (for example `revlog-compression-zstd`). This is getting fixed on default (for Mercurial 6.1) However, fixing that in 6.1 means the stream requirements sent over the wire will contains more items. And if we don't apply this fix on older version, they might end up complaining about lacking support for feature they actually support for years. This patch does not fix the deeper problem (advertised stream requirement lacking some of them), but focus on the trivial part : Lets use the full set of supported requirement for looking for unsupported ones. This patch should be simple to backport to older version of Mercurial and packager should be encouraged to do so. This is a graft of d9017df70135 from default. Differential Revision: https://phab.mercurial-scm.org/D12091

share-safe: enable by default (BC) The feature have been around for a year (4 version) and is quite important. Lets make it enabled by default. Differential Revision: https://phab.mercurial-scm.org/D11997

share-safe: add support for static-http repository We need to read the second requirements file in the static-http case too. Otherwise, static-http would miss most of the requirements and misbehave. Differential Revision: https://phab.mercurial-scm.org/D11996

legacy-revlog: fix requirement computation when cloning legacy repo The oldest format of repository does not have requirements so we need to treat them differently when cloning such repository. The previous code had issue whenever we start using a working-copy-only requirements. The "legacy" format is signaled using an empty requirements list. If we add working-copy-only requirements to it, this is no longer empty, and no longer legacy. The new code fix this, and will get fully tested in a couple of changeset, once the share-safe become the default. Differential Revision: https://phab.mercurial-scm.org/D11995

requirements: do not warn about dropping share-safe, unless explicitly set If we are just altering the default value, this is "fine". This will get tested once share-safe become the default. Differential Revision: https://phab.mercurial-scm.org/D11994

sparse: take lock before writing requirements with `share-safe`, we will also write file in the store. So we take the `lock` before altering the requirements. Differential Revision: https://phab.mercurial-scm.org/D11993