context: check file exists before getting data from _wrappedctx overlayworkingctx class is used to do in-memory merging. The data() function of that class has logic to look for data() in the wrappedctx if the file data in cache is empty and if the file is dirty. This assumes that if a file is dirty and cache has empty data for it, it will exists in the _wrappedctx. However this assumption can be False in case when we are merging a file which is empty in destination. In these cases, the backup file 'foo.orig' created by our internal merge algorithms will be empty, however it won't be present in _wrappedctx. This case will lead us to error like the one this patch is fixing. Let's only fallback to getting data from wrappedctx if cache has 'None' as data. Differential Revision: https://phab.mercurial-scm.org/D6308

tests: show IMM is broken when merging file empty in destination When we are doing in-memory merging, and we are merging a file which is empty in merge destination, it leads to error 'abort: xxx not found in manifest'. Next patch will fix this error. Differential Revision: https://phab.mercurial-scm.org/D6307

buildrpm: bump bundled Python version to 2.7.16 when building for centos{5,6} When building rpm packages for centos 5 and 6, we bundle a mercurial-specific version of Python 2.7 in /opt/python-hg. This change is analogous to 5e947367606c, and bumps the embedded Python version from 2.7.14 (released in 2017) to 2.7.16 (latest as of today).

setup: tweak error message for Python 3 We now have beta support for Python 3. In my opinion, it isn't yet stable enough to allow `pip install Mercurial` to work with Python 3 out of the box: we don't want people accidentally using Mercurial with Python 3 just yet. But I do think we should be more friendly about informing people of their options. This commit tweaks the error message that users see when running setup.py with Python 3. We instruct them about the current level of Python 3 support, point them at the wiki for more info, and give them instructions on how to bypass the check. As part of this, I also changed which version value is printed, as we were printing a named tuple before.

setup: remove set and dict comprehensions Yuya observed in a recent review that it is worthwhile to keep setup.py parseable with Python 2.6 so a useful error message is seen when attempting to run with Python 2.6. This commit removes a set and dict comprehension so setup.py is parseable with Python 2.6.

branchcache: don't verify all nodes while writing nodes are verified either when they are added or used. In case of commits. we will load the whole branchmap, only verify nodes for the branch on which we are committing and then we write. However before this patch, writing the branchmap was validating all the nodes whereas it should not. This patch fixes that. Differential Revision: https://phab.mercurial-scm.org/D6290

setup: properly package distutils in py2exe virtualenv builds Our in-repo py2exe packaging code uses virtualenvs for managing dependencies. An advantage of this is that packaging is more deterministic and reproducible. Without virtualenvs, we need to install packages in the system Python install. Packages installed by other consumers of the system Python could leak into the Mercurial package. A regression from this change was that py2exe packages contained the virtualenv's hacked distutils modules instead of the original distutils modules. (virtualenv installs a hacked distutils module because distutils uses relative path lookups that fail when running from a virtualenv.) This commit introduces a workaround so py2exe packaging uses the original distutils modules when running from a virtualenv. With this change, `import distutils` no longer fails from py2exe builds produced from a virtualenv. This fixes the regression. Furthermore, we now include all distutils modules. Before, py2exe's module finding would only find modules there were explicitly referenced in code. So, we now package a complete copy of distutils instead of a partial one. This is even better than before. # no-check-commit foo_bar function name

merge: forgot to pull before release These two changes should be part of 5.0, but we are fine leaving them out of the RC.

debugdiscovery: include the number of heads in all sets We already displayed information about heads of the common set that are either local or remote heads. We now also do so for heads of the common set that are both local and remote heads too. This is useful because various step in the set discovery algorithm have head specific optimizations.

recover: add a --[no-]verify flag For trivial cases, the cost of the verify run after `hg recover` is getting in the way. In addition for very large repositories, the cost is simply too high to be paid, making `hg recover` an unusable commands. We introduce a --verify flag, set by default. If is automatically associated with a --no-verify flag that one can use to skip the verify step. We might consider changing the default behavior in the future. However this is out of scope for this series.

Added signature for changeset 4a8d9ed86475

Added tag 5.0rc0 for changeset 4a8d9ed86475

merge: default into stable for release candidate

bundle2: handle compression in _forwardchunks _forwardchunks is used to compensate for getbundle protocol deficits. Since it transparently decodes the payload, it also needs to remove the corresponding compression parameter in case the server decides to send one. This the wire protocol part of issue 5990. Differential Revision: https://phab.mercurial-scm.org/D6182

changelog: parse copy metadata if available in extras This lets read back the copy metadata we just started writing. There are still many places left to teach about getting the copy information from the changeset, but we have enough ({file_copies}, specifically) that we can add it now and have some test coverage of it. Differential Revision: https://phab.mercurial-scm.org/D6186

copies: add config option for writing copy metadata to file and/or changset This introduces a config option that lets you choose to write copy metadata to the changeset extras instead of to filelog. There's also an option to write it to both places. I imagine that may possibly be useful when transitioning an existing repo. The copy metadata is stored as two fields in extras: one for copies since p1 and one for copies since p2. I may need to add more information later in order to make copy tracing faster. Specifically, I'm thinking out recording which files were added or removed so that copies._chaincopies() doesn't have to look at the manifest for that. But that would just be an optimization and that can be added once we know if it's necessary. I have also considered saving space by using replacing the destination file path by an index into the "files" list, but that can also be changed later (but before the feature is ready to release). Differential Revision: https://phab.mercurial-scm.org/D6183

revsetbenchmark: add some simpler revset for heads and roots This revset might leverage compiled code in the future so lets start to track them.

repoview: flag `server.view` as experimental Ideally, the non-experimental version of `experimental.extra-filter-revs` will cover the use case for `server.view=immutable` well enough than having to have this dedicated configuration. Since `server.view` is not part of any release, I would prefer to have it marked as experimental to avoid having it to support it for ever.

repoview: move subsettable in a dedicated module The dictionary got moved in `branchmap` to avoid import cycle. However, we are about to needs it in repoview too. So we introduce a now module to define that that mapping.

upgrade: support upgrade to/from zstd storage (issue6088) Now that we have an official config option for a shiny format improvement, we better make it simple to migrate to/from it.

compression: introduce an official `zstd-revlog` requirement This requirement supersedes `exp-compression-zstd`. However, we keep support for the old requirement. Strictly speaking, we do not need to add a new requirement, there are no logic change making "new" repo incompatible with mercurial client using a version that support `exp-compression-zstd`. The choice to introduce a new requirement is motivated by the following: * The previous requirement was explicitly "experimental". Using it by default could confuse users. * adding support for a hypothetical third compression engine will requires new code, and will comes with its own requirement tool. * We won't use it as the default for a while since I do not think we support zstd on all platform. I can imagine we'll gain another (unrelated but on my default) requirement by the time we turn this zstd by default.

compression: only declare revlog support for available engine Even if we know that an engine support revlog compression, the Mercurial process still won't support it if the compression engine is not available.

branchcache: lazily validate nodes in iteritems() This saves ~0.30 sec on creating a new branch on our internal repo. Differential Revision: https://phab.mercurial-scm.org/D6236

branchcache: only iterate over branches which needs to be verified Otherwise we loop over all the branches and call _verifybranch() even if not required. Differential Revision: https://phab.mercurial-scm.org/D6240

branchcache: fix the docstring of _verifybranch() Initially the function was designed to support verifying all branches but later I decided to have a separate function. I forget to remove the doc related to that. Thanks to Yuya for spotting this in review. Differential Revision: https://phab.mercurial-scm.org/D6239

branchcache: don't verify while creating a copy The copy will not be mark as verified, so there is no need to verify nodes. Thanks to Yuya who spotted this while reviewing. Differential Revision: https://phab.mercurial-scm.org/D6238

pull: improved message issued in case of failed update When running `hg pull --update`, the update step may fail. Nothing in the error message help to understand the abort is related to the secondary step (update) instead of the primary step (pull). We now add some information to the error message to clarify it comes from the update part. It is useful in various situation (uncommitted changes blocking the update, update to hidden destination, etc...) The pull output is updated from: $ hg pull ../repo-Bob --rev 956063ac4557 --update pulling from ../repo-Bob searching for changes adding changesets adding manifests adding file changes added 2 changesets with 0 changes to 2 files (+1 heads) (2 other changesets obsolete on arrival) abort: filtered revision '6'! to: $ hg pull ../repo-Bob --rev 956063ac4557 --update pulling from ../repo-Bob searching for changes adding changesets adding manifests adding file changes added 2 changesets with 0 changes to 2 files (+1 heads) (2 other changesets obsolete on arrival) abort: cannot update to target: filtered revision '6'! (I am not sure why the actual error, "filtered revision '6'", is not using the more modern format mentioning the obsolescence fate of '6')

pull: deal with locally filtered changeset passed into --rev Nowadays, it is possible to explicitly pull a remote revision that end up being hidden locally (eg: obsoleted locally). However before this patch, some internal processing where crashing trying to resolve a filtered revision. Without this patches, the pull output result a confusing output: $ hg pull ../repo-Bob --rev 956063ac4557 pulling from ../repo-Bob searching for changes adding changesets adding manifests adding file changes added 2 changesets with 0 changes to 2 files (+1 heads) (2 other changesets obsolete on arrival) abort: 00changelog.i@956063ac4557828781733b2d5677a351ce856f59: filtered node!

absorb: aborting if another operation is in progress This increases safety of using absorb by both aborting when another operation is in progress (since the absorption could confuse any other command a lot) and holding the locks throughout the reading of the working directory (for which changes to absorb) and the reading of the repo (for which changes to absorb into). Differential Revision: https://phab.mercurial-scm.org/D6237

repoview: keep the branchmap cache for the `served.hidden` view warm For the same reason we want to keep the cache for the `served` view up to date, we want to also keep the `served.hidden` view up to date. If some processes with a readonly access to the repo needs to access it, we better have the cache warm to avoid computing the same data over and over and over. In most case (no secret changesets), the "served" and "served.hidden" set will be identical and no cache will actually have to be updated.

repoview: introduce a filter for serving hidden changesets There are multiple usecase for being able to explicitly view or pull obsolete from a server. We need to be able to do so without exposing the secret changesets. We introduces a dedicated repository "view" to do so. Way to expose this "view" to the user will come later. To keep a behavior consistent with expected client/server behavior, the general idea is for the obsolete access to be explicitly requested by the code generating the request. In addition, the will be server side configuration to restrict the access to this feature.

repoview: fix conditional around unserved changesets The conditional could lead to wrong computation since we have more unserved changesets than just the "secret" phase. In addition, now that we have efficient caching of phased changesets, we don't need the conditional anymore.