dagop: don't import nullrev from .node twice See 59fa3890d40a. Differential Revision: https://phab.mercurial-scm.org/D12120

stream-clone: allow to change persistent-nodemap format during stream clone Persistent nodemap affect the store format. However it is fairly isolated and fast to generate locally. So not making it a fixed part of the stream clone is useful. This allow clients without persistent-nodemap support (default for client without Rust enabled, or simply older client). So it make it possible to enable persistent nodemap on client, where it can provide a massive boost. without too much consequence. To do so, we stop using it in the advertisement requirements for streaming and let the client add/remove the necessary file depending of its configuration. We still send the files as it seems like a small save to not regenerate them. In addition, the way we match them will overlap with the changelog-v2/revlog-v2 so we can't simply skip the associated patterns. Differential Revision: https://phab.mercurial-scm.org/D12096

stream-clone: add 5000 changesets to test-clone-stream-format This make sure the changelog is not inlined. We needs this to test coming behavior change around persistent-nodemap. Differential Revision: https://phab.mercurial-scm.org/D12095

debugbuilddag: add a flag to allow running it from a non-empty repository Allow that by default seems "dangerous", but having a flag to make it possible will be useful to help building some repository incrementally. The newly introduced support is basic, but already useful. Differential Revision: https://phab.mercurial-scm.org/D12094

persistent-nodemap: properly delete all nodemap files on downgrade Previously, we were only deleting the docket file… Differential Revision: https://phab.mercurial-scm.org/D12093

destutil: if wdp is obsolete, update to the closest non-obsolete ancestor As the original comments suggest, using prune as a model here was an existing idea, and now this patch implements it. I think it would be even better to do what solveobswdp() from evolve does (in short, it allows to update to a successor of the closest ancestor even if that ancestor is obsolete), but that is outside of this series' scope. Differential Revision: https://phab.mercurial-scm.org/D12099

discovery: port _postprocessobsolete() changes from evolve, add tests Let's make this function obsolescence-aware as well. Now we check that revisions are not obsolete first, and only then check that they are going to be common after exchange. The tests make sure that rewriting changesets doesn't confuse head computation. They rely on experimental.single-head-per-branch feature to show that remote correctly sees the expected result (i.e. either aborts the push or allows it to succeed). They are ported from evolve as well. Differential Revision: https://phab.mercurial-scm.org/D12098

branchmap: skip obsolete revisions while computing heads It's time to make this part of core Mercurial obsolescence-aware. Not considering obsolete revisions when computing heads is clearly what Mercurial should do. But there are a couple of small issues: - Let's say tip of the repo is obsolete. There are two ways of finding tiprev for branchcache (both are in use): looking at input data for update() and looking at computed heads after update(). Previously, repo tip would be tiprev of the branchcache. With this patch, an obsolete revision can no longer be tiprev. And depending on what way we use for finding tiprev (input data vs computed heads) we'll get a different result. This is relevant when recomputing cache key from cache contents, and may lead to updating cache for obsolete revisions multiple times (not from scratch, because it still would be considered valid for a subset of revisions in the repo). - If all commits on a branch are obsolete, the branchcache will include that branch, but the list of heads will be empty (that's why there's now `if not heads` when recomputing tiprev/tipnode from cache contents). Having an entry for every branch is currently required for notify extension (and test-notify.t to pass), because notify doesn't handle revsets in its subscription config very well and will throw an error if e.g. a branch doesn't exist. - Cloning static HTTP repos may try to stat() a non-existent obsstore file. The issue is that we now care about obsolescence during clone, but statichttpvfs doesn't implement a stat method, so a regular vfs.stat() is used, and it assumes that file is local and calls os.stat(). During a clone, we're trying to stat() .hg/store/obsstore, but in static HTTP case we provide a literal URL to the obsstore file on the remote as if it were a local file path. On windows it actually results in a failure in test-static-http.t. The first issue is going to be addressed in a series dedicated to making sure branchcache is properly and timely written on disk (it wasn't perfect even before this patch, but there aren't enough tests to demonstrate that). The second issue will be addressed in a future patch for notify extension that will make it not raise an exception if a branch doesn't exist. And the third one was partially addressed in the previous patch in this series and will be properly fixed in a future patch when this series is accepted. filteredhash() grows a keyword argument to make sure that branchcache is also invalidated when there are new obsolete revisions in its repo view. This way the on-disk cache format is unchanged and compatible between versions (although it will obviously be recomputed when switching versions before/after this patch and the repo has obsolete revisions). There's one test that uses plain `hg up` without arguments while updated to a pruned commit. To make this test pass, simply return current working directory parent. Later in this series this code will be replaced by what prune command does: updating to the closest non-obsolete ancestor. Test changes: test-branch-change.t: update branch head and cache update message. The head of default listed in hg heads is changed because revision 2 was rewritten as 7, and 1 is the closest ancestor on the same branch, so it's the head of default now. The cache invalidation message appears now because of the cache hash change, since we're now accounting for obsolete revisions. Here's some context: "served.hidden" repo filter means everything is visible (no filtered revisions), so before this series branch2-served.hidden file would not contain any cache hash, only revnum and node. Now it also has a hash when there are obsolete changesets in the repo. The command that the message appears for is changing branch of 5 and 6, which are now obsolete, so the cache hash changes. In general, when cache is simply out-of-date, it can be updated using the old version as a base. But if cache hash differs, then the cache for that particular repo filter is recomputed (at least with the current implementation). This is what happens here. test-obsmarker-template.t: the pull reports 2 heads changed, but after that the repo correctly sees only 1. The new message could be better, but it's still an improvement over the previous one where hg pull suggested merging with an obsolete revision. test-obsolete.t: we can see these revisions in hg log --hidden, but they shouldn't be considered heads even with --hidden. test-rebase-obsolete{,2}.t: there were new heads created previously after making new orphan changesets, but they weren't detected. Now we are properly detecting and reporting them. test-rebase-obsolete4.t: there's only one head now because the other head is pruned and was falsely reported before. test-static-http.t: add obsstore to the list of requested files. This file doesn't exist on the remotes, but clients want it anyway (they get 404). This is fine, because there are other nonexistent files that clients request, like .hg/bookmarks or .hg/cache/tags2-served. Differential Revision: https://phab.mercurial-scm.org/D12097

obsolete: make sure windows tests pass when stat() is given a URL This is a temporary fix for an issue that's only visible on windows, but exists on other platforms as well. The issue is if we're trying to use obsstore from a remote peer and that peer is a static HTTP repo, vfs tries to os.stat() a remote file using a URL to that file (e.g. http://localhost/repo/.hg/store/obsstore). The next patch in this series makes branchcache obsolescence-aware, so in certain situations exchange process will try os.stat()ing a URL. On windows this will produce an OSError that is not ENOENT, but EINVAL instead (because of `:` symbol, for example). Differential Revision: https://phab.mercurial-scm.org/D12100

transaction: do not rely on a global variable to post_finalize file We can just add a new argument to the `addfilegenerator` function. This is more explicit and therefor clearer and less error prone. Differential Revision: https://phab.mercurial-scm.org/D12125

dirstate: introduce a "tracked-key" feature A new format variant is introduced. When used, a `tracked-key` file will be generated. That file will be update when the set of tracked file might have changed. This will be useful for external automation (e.g. build tool) to be notified when the set of relevant files changes. One of the motivation for this changes is to mitigate effect dirstate-v2 has on such automation. Since the dirstate file is updated much more frequently on dirstate-v2, monitoring update to that file is no longer a viable strategy. See the associated documentation for details about the feature To prevent older client to update the repository without updating that file, a new requirements is introduced. The `postfinalizegenerators` business is a bit weird, so I'll likely clean that up soon. Differential Revision: https://phab.mercurial-scm.org/D12124

dirstate: rename the filegenerator used for writing We will need a different name in the next changesets. Changing the name is actually not that trivial so we do it in its own changeset. Differential Revision: https://phab.mercurial-scm.org/D12123

dirstate: use a context manager when writing the dirstate This make sure the file is closed in a timely manner. We define a lambda for the file opening. It might seems a bit overkill here but a future changeset will need to do more of those. Differential Revision: https://phab.mercurial-scm.org/D12122

branching: merge stable into default

arbitraryfilectx: use our existing helpers for reading and writing files Differential Revision: https://phab.mercurial-scm.org/D12090

fix: remove unnecessary and overly strict check for divergence `rewriteutil.precheck()` checks for divergence these days, so we can remove the redundant check in `hg fix`. Differential Revision: https://phab.mercurial-scm.org/D12088

encoding: fix trim() to be O(n) instead of O(n^2) `encoding.trim()` iterated over the possible lengths smaller than the input and created a slice for each. It then calculated the column width of the result, which is of course O(n), so the overall algorithm was O(n). This patch rewrites it to iterate over the unicode characters, keeping track of the length so far. Also, the old algorithm started from the end of the string, which made it much worse when the input is large and the limit is small (such as the typical 72 we pass to it). You can time it by running something like this: ``` time python3 -c 'from mercurial.utils import stringutil; print(stringutil.ellipsis(b"0123456789" * 1000, 5))' ``` That drops from 4.05 s to 83 ms with this patch (and most of that is of course startup time). Differential Revision: https://phab.mercurial-scm.org/D12089

share-safe: enable by default (BC) The feature have been around for a year (4 version) and is quite important. Lets make it enabled by default. Differential Revision: https://phab.mercurial-scm.org/D11997

share-safe: add support for static-http repository We need to read the second requirements file in the static-http case too. Otherwise, static-http would miss most of the requirements and misbehave. Differential Revision: https://phab.mercurial-scm.org/D11996

legacy-revlog: fix requirement computation when cloning legacy repo The oldest format of repository does not have requirements so we need to treat them differently when cloning such repository. The previous code had issue whenever we start using a working-copy-only requirements. The "legacy" format is signaled using an empty requirements list. If we add working-copy-only requirements to it, this is no longer empty, and no longer legacy. The new code fix this, and will get fully tested in a couple of changeset, once the share-safe become the default. Differential Revision: https://phab.mercurial-scm.org/D11995

requirements: do not warn about dropping share-safe, unless explicitly set If we are just altering the default value, this is "fine". This will get tested once share-safe become the default. Differential Revision: https://phab.mercurial-scm.org/D11994

sparse: take lock before writing requirements with `share-safe`, we will also write file in the store. So we take the `lock` before altering the requirements. Differential Revision: https://phab.mercurial-scm.org/D11993

largefiles: take lock before writing requirements With `share-safe`, we will also write file in the store. We now take the `lock` as needed. Differential Revision: https://phab.mercurial-scm.org/D11992

lfs: take lock before writing requirements With `share-safe`, we will also write file in the store, so we take the store lock before writing requirements. Differential Revision: https://phab.mercurial-scm.org/D11991

test: simpler requirements matching in test-repo-compengines.t We focus on the requirements relevant for this tests. Differential Revision: https://phab.mercurial-scm.org/D11990

test: simpler requirements matching in test-lfs-largefiles.t We focus on the requirements relevant for this tests. Differential Revision: https://phab.mercurial-scm.org/D11989

test: simpler requirements matching in test-sidedata.t We focus on the requirements relevant for this tests. Differential Revision: https://phab.mercurial-scm.org/D11988

test: simpler requirement matching in test-revlog-v2.t We narrow the matching toward the one we actually care about. Differential Revision: https://phab.mercurial-scm.org/D11987

test-copies: narrow the format check in test-copies-in-changeset.t This will reduce the noise when touching unrelated format variants. Differential Revision: https://phab.mercurial-scm.org/D11986

tests: use debugrequires instead of grepping the file directly With `share-safe`, the requirements are stored in multiple files so it seems better to use the command that retrieve the information wherever it is. Differential Revision: https://phab.mercurial-scm.org/D11985

tests: use debugrequires in test-upgrade-repo.t Whenever possible, we use the `hg debugrequires` command instead of directly grepping the file. Differential Revision: https://phab.mercurial-scm.org/D11984

test: do not use `which` to find an exec path in the test Posix is deprecated `which` in favor of `command -v`… which does not provide the same feature. Debian is warning about this deprecation and this get annoying. Differential Revision: https://phab.mercurial-scm.org/D11983

stream-clone: stop considering working copy only requirements Now that the ducks are properly aligned on there shelve, it becomes easy to fix the list of requirements to consider for stream. We remove the working copy specific ones and add some missing revlog specific ones. Differential Revision: https://phab.mercurial-scm.org/D11982

stream-clone: also filter the requirement we put in the bundle 2 We were wrongly putting irrelevant requirements in the bundle and the receiving side was getting confused, treating them as being missing while still putting them in the `requires` file. Leading do corrupted repositories. This changes fix stream-clone behavior regarding format when bundle-2 is involved, so we now also test this cases. Behavior with older version of Mercurial will be fine as they filter the requirements they get from the bundle on their side anyway. Differential Revision: https://phab.mercurial-scm.org/D12084

requirements: add an official `REVLOG_COMPRESSION_ZSTD` const Such constant was missing and its value was missing from the set of requirements that needs to be preserved through stream clone. This did not had any consequence yet as the "bundle 2 does not filter its requirements" is shadowing the issue. However we are now in a situation where we can fix this issue. So lets do it next. With the preparation work on test, changing the streamreqs value only impact two tests, where checking the full value seems to remains relevant. Important note: Since older version of Mercurial used the old `supportedformat` class attribute to check for stream requirement they supported, older version will consider this requirements to prevent them from using streaming clone. Even as they support this requirements for years. Pack for stable will be send to fix it, but they will have to be backported to older version if needed. Differential Revision: https://phab.mercurial-scm.org/D12083

stream-requirements: smoother matching in test-http-protocol.t The value of `streamreqs` changes depending of the format used, but this is not relevant for this tests. So we doing a smarter matching of the line to avoid this test interfering with format changes in the future. Differential Revision: https://phab.mercurial-scm.org/D12082

stream-requirements: smoother matching in test-ssh-bundle1.t The value of `streamreqs` changes depending of the format used, but this is not relevant for this tests. So we doing a smarter matching of the line to avoid this test interfering with format changes in the future. Differential Revision: https://phab.mercurial-scm.org/D12081

stream-requirements: smoother matching in test-ssh-proto.t The value of `streamreqs` changes depending of the format used, but this is not relevant for this tests. So we doing a smarter matching of the line to avoid this test interfering with format changes in the future. Differential Revision: https://phab.mercurial-scm.org/D12080

stream-requirements: smoother matching in test-hgweb-commands.t The value of `streamreqs` changes depending of the format used, but this is not relevant for this tests. So we doing a smarter matching of the line to avoid this test interfering with format changes in the future. Differential Revision: https://phab.mercurial-scm.org/D12079

stream-requirements: smoother matching in test-treemanifest.t The value of `streamreqs` changes depending of the format used, but this is not relevant for this tests. So we doing a smarter matching of the line to avoid this test interfering with format changes in the future. Differential Revision: https://phab.mercurial-scm.org/D12078

stream-requirements: smoother matching in test-ssh-proto-unbundle.t The value of `streamreqs` changes depending of the format used, but this is not relevant for this tests. So we doing a smarter matching of the line to avoid this test interfering with format changes in the future. Differential Revision: https://phab.mercurial-scm.org/D12077

test-bundle: split some variant in there own section Same rational as the previous commit about test-bundle.t. These line are quite volatile and having dedicated block will make their update clearer and simpler. Differential Revision: https://phab.mercurial-scm.org/D12076

test-bundle: split each variant in there own section Having conditional line matching for this kind of big block is quite inconvenient to update. The new way should be simpler to maintain. Differential Revision: https://phab.mercurial-scm.org/D12075

test-http-bad-server: abstract the `streamreqs` value Now that socket closing are decided much closer to the actual closure, we can use fuzzy matching for this part of the exchange that changes on a regular basis. This should make fixing bug in stream requirements and enabling new format much simpler. Differential Revision: https://phab.mercurial-scm.org/D12074

test-http-bad-server: use the new pattern-reading for a test-case This test case is now less sensitive to change of unrelated bits of the client/server exchange. Since this introduce some churn in the output, we do it independently for each test cases. This patch is the last of such changes, for both sent and recv cases. Differential Revision: https://phab.mercurial-scm.org/D12073

test-http-bad-server: use the new pattern-reading for a test-case This test case is now less sensitive to change of unrelated bits of the client/server exchange. Since this introduce some churn in the output, we do it independently for each test cases. Differential Revision: https://phab.mercurial-scm.org/D12072

test-http-bad-server: use the new pattern-reading for a test-case This test case is now less sensitive to change of unrelated bits of the client/server exchange. Since this introduce some churn in the output, we do it independently for each test cases. Differential Revision: https://phab.mercurial-scm.org/D12071

test-http-bad-server: use the new pattern-reading for a test-case This test case is now less sensitive to change of unrelated bits of the client/server exchange. Since this introduce some churn in the output, we do it independently for each test cases. Differential Revision: https://phab.mercurial-scm.org/D12070

test-http-bad-server: use the new pattern-reading for a test-case This test case is now less sensitive to change of unrelated bits of the client/server exchange. Since this introduce some churn in the output, we do it independently for each test cases. Differential Revision: https://phab.mercurial-scm.org/D12069

test-http-bad-server: introduce socket closing after reading a pattern We introduce the `close-after-recv-patterns` option. It has the same goal as `close-after-send-patterns` with a slightly different implementation. Reading "up to a pattern" is hard. As we can only check the pattern from what we have already read (inlike writing, were we can check what we are about to write). So instead we make the `close-after-recv-patterns` alter the behavior of the existing `close-after-recv-bytes`. The value from `close-after-recv-bytes` only gets into play after we have seen the pattern from `close-after-recv-patterns`. This allow us to achieve the target benefit without changing the read pattern too much. Differential Revision: https://phab.mercurial-scm.org/D12068

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12067

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12066

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12065

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12064

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12063

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12062

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12061

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12060

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12059

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12058

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12057

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12056

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12055

test-http-bad-server: modify some `tail` call fit their goal We want to check that we stop writing when we want to stop writing. Differential Revision: https://phab.mercurial-scm.org/D12054

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12053

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12052

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12051

test-http-bad-server: move on test from bytes-count to pattern This specific test case is now using the new "close when a pattern is seen" approach compared to the more fragile "that many bytes were sent" approach. Since such change are still a bit noisy, we split each change in there own changesets. Differential Revision: https://phab.mercurial-scm.org/D12050

test-bad-http-server: introduce sock closing when writing a pattern Instead of using a fixed number a bytes, a very unstable method. We introduce a new config that can define a regex. One this regex is detected, nothing after it will be written and the socket will be closed. Tests will be migrated to this new method in later changesets. Differential Revision: https://phab.mercurial-scm.org/D12049

test-http-bad-server: document that the value are actually a list I don't really expect this doc to be read a lot, but mentioning that this is a list seems important. Differential Revision: https://phab.mercurial-scm.org/D12048

test-http-bad-server: refactor the writing logic to avoid early return Our ultimate goal is to add another way to define the connection needs to be closed. To do so, we need the "read" code to be more unified. Differential Revision: https://phab.mercurial-scm.org/D12047

test-http-bad-server: replace the default 65537 value in output This is the default block value that we use here and it make the output overall less readable. So we replace it with `~` Differential Revision: https://phab.mercurial-scm.org/D12046

test-http-bad-server: refactor the reading logic to avoid early return Our ultimate goal is to add another way to define the connection needs to be closed. To do so, we need the "read" code to be more unified. Differential Revision: https://phab.mercurial-scm.org/D12045

test-http-bad-server: factor code dealing with "read" in the new object This will make sure both `read` and `readline` do the same processing and make it simpler to update that processing in the future. Differential Revision: https://phab.mercurial-scm.org/D12044

test-http-bad-server: factor code dealing with "write" in the new object This will make sure both `sendall` and `write` do the same processing and make it simpler to update that processing in the future. Differential Revision: https://phab.mercurial-scm.org/D12043

test-http-bad-server: unify log printing for `sendall` and `write` The `write` function was logging before calling the function while the `sendall` function were logging after calling the function. We align the `write` behavior on the `sendall` behavior before unifying the code. Differential Revision: https://phab.mercurial-scm.org/D12042

test-http-bad-server: track close condition in an object In order to make the logic more advanced, we need to unify it. To unify it, we introduce a small object that will be responsible for tracking and enforcing "premature socket close" conditions for both proxy object (socketproxy and fileobjectproxy). More logic will be moved into the object in later changesets. Differential Revision: https://phab.mercurial-scm.org/D12041

test-http-bad-server: use more readable name for variable This make the code more accessible overall. Differential Revision: https://phab.mercurial-scm.org/D12040

test-http-bad-server: align output by using `;` This is the only message using a `,` instead of `;` and that seems unintentional. I am unifying the message before unigying the code. Differential Revision: https://phab.mercurial-scm.org/D12039

test-http-bad-server: highlight each test with section Small "documentation" change to highlight each individual tests better. Differential Revision: https://phab.mercurial-scm.org/D12038

test-http-bad-server: rename config to use `-` This make things much more readable. Differential Revision: https://phab.mercurial-scm.org/D12037

test-http-bad-server: move the extension in `testlib` This seems like a better location for it. Differential Revision: https://phab.mercurial-scm.org/D12036

requirements: remove the `localrepo.supportedformat` attribute It is not longer used anywhere. Differential Revision: https://phab.mercurial-scm.org/D12035

requirements: sort _basesupported This will make the next changeset clearer. Differential Revision: https://phab.mercurial-scm.org/D12034

requirements: move the comment about manifestv2 in the module This is about requirements, so lets put it into the requirements module. We are about to remove code it was previously attached too. Differential Revision: https://phab.mercurial-scm.org/D12033

stream-clone: add a explicit set list requirements relevant to stream clone This set explicitly lists all the requirements that "stream clone" will preserve. Any other one will not be preserved. The role of listing the relevant one was previously filled by `repo.supportedformat`, but it seems clearer to use that such global and explicit set. The `repo.supportedformat` attribute will be cleaned up in a later changeset The true meaning of `repo.supportedformat` vs `repo._basesupported` was lost over time so, the content is currently bad. For example, `dirstate-v2` is currently considered relevant to the stream clone, or internal phase is missing. We kept the same content in this changeset and we will fix them later. Differential Revision: https://phab.mercurial-scm.org/D12032

stream-clone: filter possible missing requirements using all supported one The `supportedformat` requirements is missing some important requirements and it seems better to filter out with all requirements we know, not just an "arbitrary" subset. This is especially relevant as we about to phase out the `supportedformat` class attribute. (a backport of this change to stable should be sent soon). Differential Revision: https://phab.mercurial-scm.org/D12031

requirements: move "bookmark in store" requirements in the right module There seems to be no reasons to not have it with the others. Differential Revision: https://phab.mercurial-scm.org/D12030

stream-clone: add a explicit test for format change during stream clone They are different kind of requirements, the one which impact the data storage and are relevant to the files being streamed and the one which does not. For example some requirements are only relevant to the working copy, like sparse, or dirstate-v2. Since they are irrelevant to the content being streamed, they do not prevent the receiving side to use streaming clone and mercurial skip adverting them over the wire and, ideally, within the bundle. In addition, this let the client decide to use whichever format it desire for the part that does not affect the store itself. So the configuration related to these format are used as normal when doing a streaming clone. In practice, the feature was not really tested and is badly broken with bundle-2, since the requirements are not filtered out from the stream bundle. So we start with adding simple tests as a good base before the fix and adjust the feature. Differential Revision: https://phab.mercurial-scm.org/D12029

stream-clone: factor computation of requirement of a stream clone This gather code duplicated in multiple place and will make it easier to modify it safely in the future. Differential Revision: https://phab.mercurial-scm.org/D12028

stream-clone: factor computation of new clone requirement out This gather code duplicated in multiple place and will make it easier to modify it safely in the future. Differential Revision: https://phab.mercurial-scm.org/D12027

stream-clone: drop some outdated lines in test-stream-bundle-v2.t These line refer to a "dirstate-v2" variant that is no longer in use. The associated line drifted out of sync with reality. Differential Revision: https://phab.mercurial-scm.org/D12026

relnotes: add note about changed labels texts Differential Revision: https://phab.mercurial-scm.org/D12085

graft: attempt to make merge labels more helpful Differential Revision: https://phab.mercurial-scm.org/D12025

shelve: attempt to make merge labels more helpful Differential Revision: https://phab.mercurial-scm.org/D12024

histedit: attempt to make merge labels more helpful Differential Revision: https://phab.mercurial-scm.org/D12023

update: set custom conflict label for base commit Differential Revision: https://phab.mercurial-scm.org/D12022

merge: set custom conflict label for base commit Differential Revision: https://phab.mercurial-scm.org/D12021

rebase: set custom conflict label for base commit Differential Revision: https://phab.mercurial-scm.org/D12020

simplemerge: take over formatting of label from `filemerge` The padding we do of conflict labels depends on which conflict marker style is used. For two-way conflict markers (the default), the length of the base label shouldn't matter. It does before this patch, however. This patch moves the formatting from `filemerge` to `simplemerge`. The latter knows which conflict marker style to use, so it can easily decide about the padding. This change will allow us to use more descriptive "base" labels without causing illogical padding in 2-way markers. I'll do that next. One wrinkle is that we pass the same labels to external merge tools. I decided to change that in this patch to be simpler: no padding, and no ellipsis to fit within 80 columns. My reasoning is that the typical external, 3-or-4-panel merge tool doesn't show the labels on top of each others, so the padding doesn't make sense there. The ellipsis is probably not necessary because the external tools probably have their own way of dealing with long labels. Also, we limit them to "80 - 8" to fit the "<<<<<<< " before, which is almost definitely not what an external tool would put there. Differential Revision: https://phab.mercurial-scm.org/D12019

filemerge: pass `simplemerge.MergeInput` to tool functions Differential Revision: https://phab.mercurial-scm.org/D12018

filemerge: work with `simplemerge.MergeInput` in `filemerge()` We currently pass around pairs of file context objects and labels between functions in the `filemerge` module. I plan to pass around `simplemerge.MergeInput` instead. This patch prepares for that by using the type internally in `filemerge.filemerge()`. Differential Revision: https://phab.mercurial-scm.org/D12017

filemerge: always define a "base" label It simplifies the code and improves the conflict markers to always have the "base" label defined. Differential Revision: https://phab.mercurial-scm.org/D12016

filemerge: set default labels a little earlier By setting the default labels a little earlier, we can rely on them always being set, as far as I can tell. It may actually even be fine to rely on that even if we don't set them earlier, but it makes more sense to me to do it. Differential Revision: https://phab.mercurial-scm.org/D12015

filemerge: remove unused arguments from `_merge()` Differential Revision: https://phab.mercurial-scm.org/D12014

simplemerge: take arguments as annotated context objects The labels we put in conflict markers are formatted so the part before the ':' (typically says things like "local") is padded so the ':' is aligned among the labels. That means that if you specify a long label for "base" but the conflict marker style is "merge" (i.e. 2-way), the other two will have unwanted padding. We often don't specify a label for the base, so we don't notice the problem (and it may very well be that it didn't exist before my D11972). I think the best fix is to pass the labels along with the context objects, so the low-level code that switches on the marker style to use (i.e. `simplemerge`) can do the formatting. This patch starts doing that by passing a fully-formatted label to `simplemerge`. A coming patch will move the formatting to `simplemerge`. Differential Revision: https://phab.mercurial-scm.org/D12013

filemerge: make `labels` a required arg for `_premerge()` We always pass labels, and it's always a non-empty list, so we can simplify a bit. Differential Revision: https://phab.mercurial-scm.org/D12012

filemerge: remove unnecessary check for empty string Looking at the statement just before, the string is clearly never empty (or otherwise falsy). Differential Revision: https://phab.mercurial-scm.org/D12011

tests: also test spaces in conflict marker labels Our fake merge tool printed the arguments using `$1` etc. without quotes, which means that the shell collapsed repeated spaces within the argument. That resulted in tests that did not pick up changes to the number of spaces. Differential Revision: https://phab.mercurial-scm.org/D12010

blackbox: change year in logs to ISO 8601 format AFAIK, year/month/day is not a format than anyone uses. It seems more sensible to me to use ISO 8601, as that's standard and unambiguously year-month-day. Compatibility-wise, I think it's acceptable to change the default format. It's most for human consumption. It's plausible that a few tools parse this format, but it has already changed in the past (for instance to add the current revision). Differential Revision: https://phab.mercurial-scm.org/D12006

blackbox: add milliseconds to blackbox logs by default The current second granularity is often not specific enough to determine whether an hg command is happening before or after some other event. Given that starting a process takes on the order of 1ms (well, for native processes. It's quite a bit more for python processes), microseconds seems like unnecessary noise. This also lines up behavior with the rust version, where we already switched to millisecond precision. Differential Revision: https://phab.mercurial-scm.org/D12005

blackbox: correct date format doc Differential Revision: https://phab.mercurial-scm.org/D12004

branching: merge with stable

rust: Fix outdated comments in OwningDirstateMap OwningDirstateMap used to own a PyBytes, but was generalized to be more generic when it was moved from hg-cpython to hg-core. This fixes some comments that were still referencing PyBytes. Differential Revision: https://phab.mercurial-scm.org/D12003

merge: consider the file merged when using :merge-{local,other} Returning `None` from `simplemerge()` means that the caller interprets it as "no merge was necessary (because two sides matched)". See 6217040b2780 and issue2680 for some background. However, `simplemerge()` shouldn't even get called in such scenarios, and returning `None` means that the file is not considered merged, even though the contents actually were. See the affected test cases. Differential Revision: https://phab.mercurial-scm.org/D11999

simplemerge: delete unused exception class `CantReprocessAndShowBase` The only user was removed in f18830651811 (simplemerge: burn "minimal" feature to the ground, 2014-08-05). Differential Revision: https://phab.mercurial-scm.org/D11998

simplemerge: change _minimize() to minimize a single conflict `_minimize()` is weird in that it changes `a_lines` and `b_lines` but leaves `base_lines` unchanged, which means they'll be inconsistent. That was fine because the caller never used `base_lines`. With the recent refactorings of this code, we can now make this function cleaner by having it minimize a single conflict and not care about `base_lines`. This also makes the code simpler and makes the code for each `render_*()` function more similar. Differential Revision: https://phab.mercurial-scm.org/D11981

simplemerge: make minimize() a free function IMO, `Merge3Text` should be about merging text without knowing about conflict markers. "Conflict minimization" is about making conflict markers smaller, so it should be moved out. This patch does that. I'll refactor it next. Differential Revision: https://phab.mercurial-scm.org/D11980

simplemerge: simplify and rename `render_markers()` `render_markers()` now always renders minimized 2-way markers, so let's simplify and rename it accordingly. Differential Revision: https://phab.mercurial-scm.org/D11979

simplemerge: split out function for rendering :merge3 conflict markers The results in some duplicate, but the code is very straight-forward and I think it's worth it to have each conflict marker style in a separate function. Differential Revision: https://phab.mercurial-scm.org/D11978

simplemerge: move default labels to simplemerge extension I want to remove knowledge of paths from the `simplemerge` core module. As part of that, let's move the default labels from the core module to the extension. Differential Revision: https://phab.mercurial-scm.org/D11977

simplemerge: use same newline detection for :mergediff as for :merge[3] The `:merge` and `:merge3` markers used the newline style detected from the base input. The `:mergediff` style always used LF as newline character. This patch teaches `:mergediff` to use the same detection as the others did. Differential Revision: https://phab.mercurial-scm.org/D11976

simplemerge: clarify names of functions that render conflict markers Differential Revision: https://phab.mercurial-scm.org/D11975

simplemerge: make merge_lines() a free function IMO, the rendering of a conflict as text with merge markers should be separate from the code for resolving conflicts. The latter is what `Merge3Text` is mostly about already. Differential Revision: https://phab.mercurial-scm.org/D11974

simplemerge: convert `merge_lines()` away from generator We always consume all the lines and put them in a list anyway. By making the function not a generator, we can later make it return an additional value (to indicate if there were conflicts). Differential Revision: https://phab.mercurial-scm.org/D11973

simplemerge: use 3-way markers if mode=='merge3', ignoring number of labels Before this patch, we use a combination of `mode` and `labels` to decide which marker style to use. That's mostly my own fault because I was the one who added the `mergediff` as a "mode" instead of adding a separate argument for marker style. Still, let's make these arguments more orthogonal by ignoring the number of labels when deciding which marker style to use. This refactoring makes it a little clearer in the simplemerge *extension* that passing more than 2 `--label` arguments changes the marker style. Differential Revision: https://phab.mercurial-scm.org/D11972

rust: upgrade `rand*` crates `test-check-cargo-lock.t` is failing for me and I was hoping this would help. It doesn't, but we might as well take the upgrade now that I've done the (small amount of) work for it. Differential Revision: https://phab.mercurial-scm.org/D12000

logcmdutil: raise `StateError` when file to follow doesn't exist Differential Revision: https://phab.mercurial-scm.org/D11969