lfs: don't skip locally available blobs when verifying The `skipflags` config was introduced in a2ab9ebcd85b, which specifically calls out downloading and storing all blobs as potentially too expensive. But I don't see any reason to skip blobs that are already available locally. Hashing the blob is the only way to indirectly verify the rawdata content stored in the revlog. (The note in that commit about skipping renamed is still correct, but the reason given about needing fulltext isn't.) Differential Revision: https://phab.mercurial-scm.org/D7712

lfs: add a switch to `hg verify` to ignore the content of blobs Trying to validate the fulltext of an external revision causes missing blobs to be downloaded and cached. Since the downloads aren't batch prefetched[1] and aren't compressed, this can be expensive both in terms of time and space. I made this a tri-state instead of a simple bool because there's an existing (undocumented) config to handle this, and it would be weird if `hg verify` were to suddenly start ignoring that config but an `hg recover` initiated verify honors it. Since this uses the same config setting, it too will skip rename verification (which requires fulltext, but not for LFS). [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116118.html Differential Revision: https://phab.mercurial-scm.org/D7708

revlog: run rustfmt nightly I'm a little nervous about folding this back (might be nightly rustfmt mismatches?) so I want someone to review this. Differential Revision: https://phab.mercurial-scm.org/D7813

examples: specify rustfmt nightly using a $() construct This is ugly, but it's how we have to configure rustfmt for now as we require nightly rustfmt. Differential Revision: https://phab.mercurial-scm.org/D7812

hg-core: rustfmt path.rs The file as vendored does not conform to our source formatting conventions. Let's reformat it so it does. # skip-blame automated code reformatting Differential Revision: https://phab.mercurial-scm.org/D7580

hg-core: vendor Facebook's path utils module The added file was imported from https://github.com/facebookexperimental/eden/blob/d1d8fb939a39aa331ae7f162c39cbcaa511d474b/eden/scm/lib/util/src/path.rs without modifications. The file is not yet integrated into our project. This will be done in subsequent commits. Differential Revision: https://phab.mercurial-scm.org/D7573

revlog-native: introduced ABI version in capsule Concerns that an inconsistency could arise between the actual contents of the capsule in revlog.c and the Rust consumer have been raised after the switch to the array of data and function pointers in f384d68d8ea8. It has been suggested that the `version` from parsers.c could be use for this. In this change, we introduce instead a separate ABI version number, which should have the following advantages: - no need to change the consuming Rust code for changes that have nothing to do with the contents of the capsule - the version number in parsers.c is not explicitely flagged as ABI. It's not obvious to me whether an ABI change that would be invisible to Python would warrant an increment The drawback is that developers now have to consider two version numbers. We expect the added cost of the check to be negligible because it occurs at instantiation of `CIndex` only, which in turn is tied to instantiation of Python objects such as `LazyAncestors` and `MixedIndex`. Frequent calls to `Cindex::new` should also probably hit the CPU branch predictor. Differential Revision: https://phab.mercurial-scm.org/D7856

phases: make phasecache._phasesets immutable Previously, some code paths would mutate the cache itself, which could give weird results if multiple revsets got evaluated through that path. Differential Revision: https://phab.mercurial-scm.org/D7854

phases: reduce code duplication in phasecache.getrevset This is a functional NOP other than reducing some of the duplication in that method. Differential Revision: https://phab.mercurial-scm.org/D7853

scmutil: fix an unbound variable with progressbar debug enabled Differential Revision: https://phab.mercurial-scm.org/D7852

hgext: replace references to hashlib.sha1 with hashutil.sha1 When in a non-pure build of Mercurial, this will provide protections against SHA1 collision attacks. Differential Revision: https://phab.mercurial-scm.org/D7851

sslutil: migrate to hashutil.sha1 instead of hashlib.sha1 This is a straight-line replacement like the others, but I split it out since it's used in a network context and I'm not sure this is appropriate (we should probably drop support for sha1 fingerprints over TLS) and wanted this to be easily dropped. Differential Revision: https://phab.mercurial-scm.org/D7850

core: migrate uses of hashlib.sha1 to hashutil.sha1 Differential Revision: https://phab.mercurial-scm.org/D7849

hashutil: new package for hashing-related features Right now this just tries to use our sha1dc and if it's missing (eg a --pure build) we fall back to hashlib. I imagine in the future we'll want some other things in here for detecting what hasher is in use as we transition off sha1. Differential Revision: https://phab.mercurial-scm.org/D7848

sha1dc: initial implementation of Python extension A future change will use this when available to avoid sha1 collision issues until we can get moved to something else. Differential Revision: https://phab.mercurial-scm.org/D7815

sha1dc: import latest version from github After the recent SHA1 news, the attacks are serious enough we should be more proactive. This code will at least allow detection of attacks early. It's already widely deployed in Git. This is git revision 855827c583bc30645ba427885caa40c5b81764d2 of the sha1collisiondetection repo[0], with most of the files omitted. A follow-up change will introduce Python bindings for this code. 0: https://github.com/cr-marcstevens/sha1collisiondetection Differential Revision: https://phab.mercurial-scm.org/D7814

transaction: add a `hasfinalize` method The method allow code to check if an existing callback exists. It allow them to skip potentially expensive setup for a callback. Differential Revision: https://phab.mercurial-scm.org/D7832

changelog: fix the diverted opener to accept more kwargs The current code prevent the use of `atomictemp` file with the changelog opener. I do not see a good reason for this limitation. Differential Revision: https://phab.mercurial-scm.org/D7831

revlog: reorder a conditionnal about revlogio if we are using REVLOGV0, we will not use a rust based index. This small line movement make it clearer. Differential Revision: https://phab.mercurial-scm.org/D7830

rebase: delete seemingly unnecessary needupdate() This seemed to be about checking that the user hasn't updated away when we asked them to resolve merge conflicts. These days we call `cmdutil.checkunfinished()` and refuse to update, so the user shouldn't be able to get into this state. `test-rebase-interruptions.t` actually has some tests where it disables the rebase extension in order to be allowed to do some of these updates. That still passes, but I wouldn't personally haved cared if that failed. Differential Revision: https://phab.mercurial-scm.org/D7825

workingctx: move setparents() logic from localrepo to mirror overlayworkingctx It would be nice to later be able to call `wctx.setparents()` whether `wctx` is a `workingctx` or an `overlayworkingctx`. Differential Revision: https://phab.mercurial-scm.org/D7823

overlayworkginctx: implement a setparents() to mirror dirstate.setparents() This lets us make the in-memory and on-disk code a bit more similar. I'll soon also implement setparents() on the regular workingctx. Differential Revision: https://phab.mercurial-scm.org/D7822

overlayworkingctx: default branch to base context's branch This matches what the dirstate does (it reuses working copy parent's branch unless told otherwise). By moving the default out of `rebase.commitmemorynode()`, it will let us clean that up better later. Differential Revision: https://phab.mercurial-scm.org/D7821

grep: speed up `hg grep --all-files some/path` by using ctx.matches(match) ctx.matches(match) avoids walking unintersting parts of the tree when using tree manifests. That can make a very big difference when grepping in a small subset of the tree (2.0s -> 0.7s in my case, but can of course be made more extreme by picking a smaller subset of files). Differential Revision: https://phab.mercurial-scm.org/D7820