merge: default into stable for release candidate

debugdiscovery: include the number of heads in all sets We already displayed information about heads of the common set that are either local or remote heads. We now also do so for heads of the common set that are both local and remote heads too. This is useful because various step in the set discovery algorithm have head specific optimizations.

recover: add a --[no-]verify flag For trivial cases, the cost of the verify run after `hg recover` is getting in the way. In addition for very large repositories, the cost is simply too high to be paid, making `hg recover` an unusable commands. We introduce a --verify flag, set by default. If is automatically associated with a --no-verify flag that one can use to skip the verify step. We might consider changing the default behavior in the future. However this is out of scope for this series.

bundle2: handle compression in _forwardchunks _forwardchunks is used to compensate for getbundle protocol deficits. Since it transparently decodes the payload, it also needs to remove the corresponding compression parameter in case the server decides to send one. This the wire protocol part of issue 5990. Differential Revision: https://phab.mercurial-scm.org/D6182

changelog: parse copy metadata if available in extras This lets read back the copy metadata we just started writing. There are still many places left to teach about getting the copy information from the changeset, but we have enough ({file_copies}, specifically) that we can add it now and have some test coverage of it. Differential Revision: https://phab.mercurial-scm.org/D6186

copies: add config option for writing copy metadata to file and/or changset This introduces a config option that lets you choose to write copy metadata to the changeset extras instead of to filelog. There's also an option to write it to both places. I imagine that may possibly be useful when transitioning an existing repo. The copy metadata is stored as two fields in extras: one for copies since p1 and one for copies since p2. I may need to add more information later in order to make copy tracing faster. Specifically, I'm thinking out recording which files were added or removed so that copies._chaincopies() doesn't have to look at the manifest for that. But that would just be an optimization and that can be added once we know if it's necessary. I have also considered saving space by using replacing the destination file path by an index into the "files" list, but that can also be changed later (but before the feature is ready to release). Differential Revision: https://phab.mercurial-scm.org/D6183

revsetbenchmark: add some simpler revset for heads and roots This revset might leverage compiled code in the future so lets start to track them.

repoview: flag `server.view` as experimental Ideally, the non-experimental version of `experimental.extra-filter-revs` will cover the use case for `server.view=immutable` well enough than having to have this dedicated configuration. Since `server.view` is not part of any release, I would prefer to have it marked as experimental to avoid having it to support it for ever.

repoview: move subsettable in a dedicated module The dictionary got moved in `branchmap` to avoid import cycle. However, we are about to needs it in repoview too. So we introduce a now module to define that that mapping.

upgrade: support upgrade to/from zstd storage (issue6088) Now that we have an official config option for a shiny format improvement, we better make it simple to migrate to/from it.

compression: introduce an official `zstd-revlog` requirement This requirement supersedes `exp-compression-zstd`. However, we keep support for the old requirement. Strictly speaking, we do not need to add a new requirement, there are no logic change making "new" repo incompatible with mercurial client using a version that support `exp-compression-zstd`. The choice to introduce a new requirement is motivated by the following: * The previous requirement was explicitly "experimental". Using it by default could confuse users. * adding support for a hypothetical third compression engine will requires new code, and will comes with its own requirement tool. * We won't use it as the default for a while since I do not think we support zstd on all platform. I can imagine we'll gain another (unrelated but on my default) requirement by the time we turn this zstd by default.

compression: only declare revlog support for available engine Even if we know that an engine support revlog compression, the Mercurial process still won't support it if the compression engine is not available.

branchcache: lazily validate nodes in iteritems() This saves ~0.30 sec on creating a new branch on our internal repo. Differential Revision: https://phab.mercurial-scm.org/D6236

branchcache: only iterate over branches which needs to be verified Otherwise we loop over all the branches and call _verifybranch() even if not required. Differential Revision: https://phab.mercurial-scm.org/D6240

branchcache: fix the docstring of _verifybranch() Initially the function was designed to support verifying all branches but later I decided to have a separate function. I forget to remove the doc related to that. Thanks to Yuya for spotting this in review. Differential Revision: https://phab.mercurial-scm.org/D6239

branchcache: don't verify while creating a copy The copy will not be mark as verified, so there is no need to verify nodes. Thanks to Yuya who spotted this while reviewing. Differential Revision: https://phab.mercurial-scm.org/D6238

pull: improved message issued in case of failed update When running `hg pull --update`, the update step may fail. Nothing in the error message help to understand the abort is related to the secondary step (update) instead of the primary step (pull). We now add some information to the error message to clarify it comes from the update part. It is useful in various situation (uncommitted changes blocking the update, update to hidden destination, etc...) The pull output is updated from: $ hg pull ../repo-Bob --rev 956063ac4557 --update pulling from ../repo-Bob searching for changes adding changesets adding manifests adding file changes added 2 changesets with 0 changes to 2 files (+1 heads) (2 other changesets obsolete on arrival) abort: filtered revision '6'! to: $ hg pull ../repo-Bob --rev 956063ac4557 --update pulling from ../repo-Bob searching for changes adding changesets adding manifests adding file changes added 2 changesets with 0 changes to 2 files (+1 heads) (2 other changesets obsolete on arrival) abort: cannot update to target: filtered revision '6'! (I am not sure why the actual error, "filtered revision '6'", is not using the more modern format mentioning the obsolescence fate of '6')

pull: deal with locally filtered changeset passed into --rev Nowadays, it is possible to explicitly pull a remote revision that end up being hidden locally (eg: obsoleted locally). However before this patch, some internal processing where crashing trying to resolve a filtered revision. Without this patches, the pull output result a confusing output: $ hg pull ../repo-Bob --rev 956063ac4557 pulling from ../repo-Bob searching for changes adding changesets adding manifests adding file changes added 2 changesets with 0 changes to 2 files (+1 heads) (2 other changesets obsolete on arrival) abort: 00changelog.i@956063ac4557828781733b2d5677a351ce856f59: filtered node!

absorb: aborting if another operation is in progress This increases safety of using absorb by both aborting when another operation is in progress (since the absorption could confuse any other command a lot) and holding the locks throughout the reading of the working directory (for which changes to absorb) and the reading of the repo (for which changes to absorb into). Differential Revision: https://phab.mercurial-scm.org/D6237

repoview: keep the branchmap cache for the `served.hidden` view warm For the same reason we want to keep the cache for the `served` view up to date, we want to also keep the `served.hidden` view up to date. If some processes with a readonly access to the repo needs to access it, we better have the cache warm to avoid computing the same data over and over and over. In most case (no secret changesets), the "served" and "served.hidden" set will be identical and no cache will actually have to be updated.

repoview: introduce a filter for serving hidden changesets There are multiple usecase for being able to explicitly view or pull obsolete from a server. We need to be able to do so without exposing the secret changesets. We introduces a dedicated repository "view" to do so. Way to expose this "view" to the user will come later. To keep a behavior consistent with expected client/server behavior, the general idea is for the obsolete access to be explicitly requested by the code generating the request. In addition, the will be server side configuration to restrict the access to this feature.

repoview: fix conditional around unserved changesets The conditional could lead to wrong computation since we have more unserved changesets than just the "secret" phase. In addition, now that we have efficient caching of phased changesets, we don't need the conditional anymore.

branch: return early if we find an open named branch apart from default The current code builds a list of all the open named branches except default and then bool that. This is mostly fine until you get a repo which has thousands of named branches. Differential Revision: https://phab.mercurial-scm.org/D6211

branchcache: don't verify closed nodes in _branchtip() We only do membership testing there. Differential Revision: https://phab.mercurial-scm.org/D6210