vfs: use 'vfs' module directly in 'mercurial.hg' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: extract 'vfs' class and related code to a new 'vfs' module (API) The 'scmutil' is growing large (1500+ lines) and 2/5 of it is related to vfs. We extract the 'vfs' related code in its own module get both module back to a better scale and clearer contents. We keep all the references available in 'scmutil' for now as many reference needs to be updated.

vfs: replace 'scmutil.opener' usage with 'scmutil.vfs' The 'vfs' class is the first class citizen for years. We remove all usages of the older API. This will let us remove the old API eventually.

fsmonitor: remove use of repo.opener This has been deprecated, so we need to switch to the appropriate vfs apis.

help: align description of 'base rev' with reality [issue5488] The text about revlogs seems to be wrong about -1 being used to indicate the start of a delta chain. Attempt to correct this.

help: fix internals.changegroups Add information about tree manifests, copy edit the text and fix up a few ambiguities. The document also contains a few additional fixes from Siddharth Agarwal <sid0@fb.com>, who used it to build a parser for changegroups in Rust.

share: drop 'relshared' requirement as well

share: fix typo to drop 'shared' requirement on unshare This must be a typo and it seems correct to drop the requirement since the repo is no longer a shared repository.

similar: compare between actual file contents for exact identity Before this patch, similarity detection logic (for addremove and automv) depends entirely on SHA-1 digesting. But this causes incorrect rename detection, if: - removing file A and adding file B occur at same committing, and - SHA-1 hash values of file A and B are same This may prevent security experts from managing sample files for SHAttered issue in Mercurial repository, for example. https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html https://shattered.it/ Hash collision itself isn't so serious for core repository functionality of Mercurial, described by mpm as below, though. https://www.mercurial-scm.org/wiki/mpm/SHA1 This patch compares between actual file contents after hash comparison for exact identity. Even after this patch, SHA-1 is still used, because it is reasonable enough to quickly detect existence of "(almost) same" file. - replacing SHA-1 causes decreasing performance, and - replacement of it has ambiguity, yet Getting content of removed file (= rfctx.data()) at each exact comparison should be cheap enough, even though getting content of added one costs much. ======= ============== ===================== file fctx data() reads from ======= ============== ===================== removed filectx in-memory revlog data added workingfilectx storage ======= ============== =====================

localrepo: handle rename with hardlinks properly In "aftertrans", we rename "journal.*" to "undo.*". We expect "journal.*" files to disappear after renaming. However, if "journal.foo" and "undo.foo" refer to a same file (hardlink), rename may be a no-op, leaving both files on disk, according to Linux manpage [1]: If oldpath and newpath are existing hard links referring to the same file, then rename() does nothing, and returns a suc‐ cess status. The POSIX specification [2] is not very clear about what to do. To be safe, remove "undo.*" before the rename so "journal.*" cannot be left on disk. [1]: http://man7.org/linux/man-pages/man2/rename.2.html [2]: http://pubs.opengroup.org/onlinepubs/9699919799/

dirstate: avoid unnecessary load+dump during backup Previously, dirstate.savebackup unconditionally dumps the dirstate map to disk. It may require loading dirstate first to be able to dump it. Those operations could be expensive if the dirstate is big, and could be avoided if we know the dirstate file is up-to-date. This patch avoids the read and write if the dirstate is clean. In that case, we just do a plain copy without any serialization. This should make commands which use transactions but do not touch dirstate faster. For example, "hg bookmark -r REV NAME".

dirstate: try to use hardlink to backup dirstate This should be more efficient once util.copyfile has real hardlink support.

dirstate: track updated files to improve write time Previously, dirstate.write() would iterate over the entire dirstate to find any entries that needed to be marked 'lookup' (i.e. if they have the same timestamp as now). This was O(working copy) and slow in large repos. It was most visible when rebasing or histediting multiple commits, since it gets executed once per commit, even if the entire rebase/histedit is wrapped in a transaction. The fix is to track which files have been editted, and only check those to see if they need to be marked as 'lookup'. This saves 25% on histedit times in very large repositories. I tested this by adding temporary debug logic to verify that the old files processed in the loop matched the new files processed in the loop and running the test suite.

hook: give exthooks tags for blocking time The ui.system autogenerated tag isn't really useful - as they're named, let's use the name the user gave us.

filemerge: tag merge tool for blocked times Merge tools can take a while - let's ensure that they're appropriately tagged

hgk: set a blocked tag when the user invokes view

transplant: set a blockedtag when invoking external filter

config: set blockedtag when invoking configuration edit

bisect: set a blockedtag when running the check command So that the hg bisect data clearly shows the bisect command separately to the main data set.

dispatch: set a blockedtag when running an external alias

patch: set a blockedtag when running an external filter

sshpeer: set a blockedtag when starting ssh So that the data is readable.

revert: always display hunks reversed when reverting to parent When reverting to the parent of working directory, operation is "discard" so we want hunks to be presented in the same order as the diff (i.e. "reversed"). So we do not query the experimental.revertalternateinteractivemode option in this case and always set "reversehunks" to True.

fileset: add a 'status(...)' predicate to control evaluation context Same as 'revs', this predicate does not select files but switches the evaluation context. This allow to match file according arbitrary status call. We can now express the same query as 'hg status'. The API (two 'revsingle' class) have been picked instead of a single 'revs' revset for multiple reasons: * it is less confusing to express * it allow to express more query (eg: backward status, cross branch status)

fileset: allow to specify a basectx for status This will be used for a predicates that defines the status range of a match.

fileset: add revs(revs, fileset) to evaluate set in working directory Unlike other functions, "revs()" does not select files but switches the evaluation context. This allow to match file with property in another revision that the one currently evaluated. This changeset is based on work from Yuya Nishihara.

fileset: add function to switch revision where fileset will be evaluated If the subset isn't filtered yet, i.e. if fullmatchctx, the new subset is recalculated from scratch. Otherwise, it is narrowed by the existing subset.

fileset: extract function that builds status tuple only if necessary This function will be used to recalculate status when mctx.ctx is switched.

fileset: build initial subset in fullmatchctx class

fileset: extract function that builds initial subset from ctx or status This function will be used to recalculate subset when mctx.ctx is switched.

fileset: add class to host special handling of initial subset Future patches will add a function to switch mctx.ctx object so that we can forcibly evaluate a fileset expression in a specified revision. For example, new "revs()" function will be used to match predicate agains another revision $ hg revert 'set:revs(42, added())' fullmatchctx class is similar to revset.fullreposet. It will allow us to recalculate the subset only if it is not filtered yet.

patchbomb: add config knob to generate flags by template (issue5354) This can be used to flag patches by branch or topic automatically. Flags optionally given by --flag option are exported as {flags} template keyword, so you can add --flag V2.