vfs: use 'vfs' module directly in 'mercurial.debugcommand' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.simplemerge' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.cmdutil' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.subrepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.archival' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.store' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.patch' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.repair' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.localrepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

chg: forward user-defined signals SIGUSR1 and SIGUSR2 are reserved for user-defined behaviors. They may be redefined by an hg extension [1], but cannot be easily redefined for chg. Since the default behavior (kill) is not that useful for chg, let's forward them to hg, hoping it got redefined there and could be more useful. [1] https://bitbucket.org/facebook/hg-experimental/commits/e7c883a465

chg: document why we send SIGHUP and SIGINT to process group This makes the code more consistent - other signals are documented.

tests: make test-shelve.t timing-independent It was sometimes taking 2s for me (not the "1s" the test expected).

tests: make run-tests.py run on Python 3 again

rebase: move actual rebase into a single transaction Previously, rebasing would open several transaction over the course of rebasing several commits. Opening a transaction can have notable overhead (like copying the dirstate) which can add up when rebasing many commits. This patch adds a single large transaction around the actual commit rebase operation, with a catch for intervention which serializes the current state if we need to drop back to the terminal for user intervention. Amazingly, almost all the tests seem to pass. On large repos with large working copies, this can speed up rebasing 7 commits by 25%. I'd expect the percentage to be a bit larger for rebasing even more commits. There are minor test changes because we're rolling back the entire transaction during unexpected exceptions instead of just stopping mid-rebase, so there's no more backup bundle. It also leave an unknown file in the working copy, since our clean up 'hg update' doesn't delete unknown files.

rebase: allow aborting if last-message.txt is missing Previously, if .hg/rebasestate existed but .hg/last-message.txt was missing, 'hg rebase --abort' would say there's no rebase in progress but 'hg checkout foo' would say 'abort: rebase in progress'. It turns out loading the collapse message will throw a "no rebase in progress" error if the file doesn't exist, even though .hg/rebasestate obviously indicates a rebase is in progress. The fix is to only throw an exception if we're trying to --continue, and to just eat the issues if we're doing --abort. This issue is exposed by us writing the rebase state earlier in the process. This will be used by later patches to ensure the user can appropriately 'hg rebase --abort' if there's a crash before the first the first commit has finished rebasing. Tests cover all of this. The only negative affect is we now require a hg rebase --abort in a very specific exception case, as shown in the test.

rebase: add storestatus support for transactions This let's the status writing logic support transactions. This will be useful in a later patch where we add a transaction around the entire rebase.

rebase: move storestatus onto rebaseruntime The rebaseruntime class already has the restorestatus function, so let's make it own the store status function too. This get's rid of a lot of unnecessary argument passing and will make a future patch cleaner that refactors storestatus to support transactions.

rebase: clear updatestate during rebase --abort in more cases Previously, rebase --abort would only call update if you were on a node that had already been rebased. This meant that if the rebase failed during the rebase of the first commit, the working copy would be left dirty (with a .hg/updatestate file) and rebase --abort would not have update to clean it up. The fix is to also perform an update if you're still on the target node or on the original working copy node (since the working copy may be dirty, we still need to do the update). We don't want to perform an update in all cases though because of issue4009. A subsequent patch makes this case much more common, since it causes the entire rebase transaction to rollback during unexpected exceptions. This causes the existing test-rebase-abort.t to cover this case.

repoview: convert attribute names to unicodes on Python 3 In Python 3, the attribute names must be strings i.e. unicodes.

parsers: alias long to int on Python 3

store: slice over a bytestring to get characters instead of ascii values On Python 2, >>> a = b'abc' >>> a[1] 'b' Whereas on python 3, >>> a = b'abc' >>> a[1] 98 >>> a[1:2] b'b' This does not change behaviour on python 2.

vfs: use 'vfs' module directly in 'mercurial.hg' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: extract 'vfs' class and related code to a new 'vfs' module (API) The 'scmutil' is growing large (1500+ lines) and 2/5 of it is related to vfs. We extract the 'vfs' related code in its own module get both module back to a better scale and clearer contents. We keep all the references available in 'scmutil' for now as many reference needs to be updated.

vfs: replace 'scmutil.opener' usage with 'scmutil.vfs' The 'vfs' class is the first class citizen for years. We remove all usages of the older API. This will let us remove the old API eventually.

fsmonitor: remove use of repo.opener This has been deprecated, so we need to switch to the appropriate vfs apis.

help: align description of 'base rev' with reality [issue5488] The text about revlogs seems to be wrong about -1 being used to indicate the start of a delta chain. Attempt to correct this.

help: fix internals.changegroups Add information about tree manifests, copy edit the text and fix up a few ambiguities. The document also contains a few additional fixes from Siddharth Agarwal <sid0@fb.com>, who used it to build a parser for changegroups in Rust.

share: drop 'relshared' requirement as well

share: fix typo to drop 'shared' requirement on unshare This must be a typo and it seems correct to drop the requirement since the repo is no longer a shared repository.

similar: compare between actual file contents for exact identity Before this patch, similarity detection logic (for addremove and automv) depends entirely on SHA-1 digesting. But this causes incorrect rename detection, if: - removing file A and adding file B occur at same committing, and - SHA-1 hash values of file A and B are same This may prevent security experts from managing sample files for SHAttered issue in Mercurial repository, for example. https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html https://shattered.it/ Hash collision itself isn't so serious for core repository functionality of Mercurial, described by mpm as below, though. https://www.mercurial-scm.org/wiki/mpm/SHA1 This patch compares between actual file contents after hash comparison for exact identity. Even after this patch, SHA-1 is still used, because it is reasonable enough to quickly detect existence of "(almost) same" file. - replacing SHA-1 causes decreasing performance, and - replacement of it has ambiguity, yet Getting content of removed file (= rfctx.data()) at each exact comparison should be cheap enough, even though getting content of added one costs much. ======= ============== ===================== file fctx data() reads from ======= ============== ===================== removed filectx in-memory revlog data added workingfilectx storage ======= ============== =====================

localrepo: handle rename with hardlinks properly In "aftertrans", we rename "journal.*" to "undo.*". We expect "journal.*" files to disappear after renaming. However, if "journal.foo" and "undo.foo" refer to a same file (hardlink), rename may be a no-op, leaving both files on disk, according to Linux manpage [1]: If oldpath and newpath are existing hard links referring to the same file, then rename() does nothing, and returns a suc‐ cess status. The POSIX specification [2] is not very clear about what to do. To be safe, remove "undo.*" before the rename so "journal.*" cannot be left on disk. [1]: http://man7.org/linux/man-pages/man2/rename.2.html [2]: http://pubs.opengroup.org/onlinepubs/9699919799/

dirstate: avoid unnecessary load+dump during backup Previously, dirstate.savebackup unconditionally dumps the dirstate map to disk. It may require loading dirstate first to be able to dump it. Those operations could be expensive if the dirstate is big, and could be avoided if we know the dirstate file is up-to-date. This patch avoids the read and write if the dirstate is clean. In that case, we just do a plain copy without any serialization. This should make commands which use transactions but do not touch dirstate faster. For example, "hg bookmark -r REV NAME".

dirstate: try to use hardlink to backup dirstate This should be more efficient once util.copyfile has real hardlink support.

dirstate: track updated files to improve write time Previously, dirstate.write() would iterate over the entire dirstate to find any entries that needed to be marked 'lookup' (i.e. if they have the same timestamp as now). This was O(working copy) and slow in large repos. It was most visible when rebasing or histediting multiple commits, since it gets executed once per commit, even if the entire rebase/histedit is wrapped in a transaction. The fix is to track which files have been editted, and only check those to see if they need to be marked as 'lookup'. This saves 25% on histedit times in very large repositories. I tested this by adding temporary debug logic to verify that the old files processed in the loop matched the new files processed in the loop and running the test suite.

hook: give exthooks tags for blocking time The ui.system autogenerated tag isn't really useful - as they're named, let's use the name the user gave us.

filemerge: tag merge tool for blocked times Merge tools can take a while - let's ensure that they're appropriately tagged

hgk: set a blocked tag when the user invokes view

transplant: set a blockedtag when invoking external filter

config: set blockedtag when invoking configuration edit

bisect: set a blockedtag when running the check command So that the hg bisect data clearly shows the bisect command separately to the main data set.

dispatch: set a blockedtag when running an external alias

patch: set a blockedtag when running an external filter

sshpeer: set a blockedtag when starting ssh So that the data is readable.

revert: always display hunks reversed when reverting to parent When reverting to the parent of working directory, operation is "discard" so we want hunks to be presented in the same order as the diff (i.e. "reversed"). So we do not query the experimental.revertalternateinteractivemode option in this case and always set "reversehunks" to True.

fileset: add a 'status(...)' predicate to control evaluation context Same as 'revs', this predicate does not select files but switches the evaluation context. This allow to match file according arbitrary status call. We can now express the same query as 'hg status'. The API (two 'revsingle' class) have been picked instead of a single 'revs' revset for multiple reasons: * it is less confusing to express * it allow to express more query (eg: backward status, cross branch status)

fileset: allow to specify a basectx for status This will be used for a predicates that defines the status range of a match.

fileset: add revs(revs, fileset) to evaluate set in working directory Unlike other functions, "revs()" does not select files but switches the evaluation context. This allow to match file with property in another revision that the one currently evaluated. This changeset is based on work from Yuya Nishihara.

fileset: add function to switch revision where fileset will be evaluated If the subset isn't filtered yet, i.e. if fullmatchctx, the new subset is recalculated from scratch. Otherwise, it is narrowed by the existing subset.

fileset: extract function that builds status tuple only if necessary This function will be used to recalculate status when mctx.ctx is switched.

fileset: build initial subset in fullmatchctx class

fileset: extract function that builds initial subset from ctx or status This function will be used to recalculate subset when mctx.ctx is switched.

fileset: add class to host special handling of initial subset Future patches will add a function to switch mctx.ctx object so that we can forcibly evaluate a fileset expression in a specified revision. For example, new "revs()" function will be used to match predicate agains another revision $ hg revert 'set:revs(42, added())' fullmatchctx class is similar to revset.fullreposet. It will allow us to recalculate the subset only if it is not filtered yet.

patchbomb: add config knob to generate flags by template (issue5354) This can be used to flag patches by branch or topic automatically. Flags optionally given by --flag option are exported as {flags} template keyword, so you can add --flag V2.

patchbomb: pass around ui and revs that are needed for flag template See the next patch for why.

patchbomb: build patch texts by _getpatchmsgs() Now _getpatchmsgs() knows revision numbers, which allows us to generate flags by applying a template to changectx objects.

patchbomb: drop internal option for pbranch extension (API) I want to move _getpatches() to _getpatchmsgs() to make sure each patch text is tied with the corresponding revision number. This helps adding templater support. IIRC, the pbranch extension doesn't work with the recent Mercurial versions, so the removal of this option wouldn't hurt.

patchbomb: factor out function that builds a prefix string to patch subject I'll add templating support.

formatter: add argument to change output file of non-plain formatter This allows us to build data not written to the console. That would be doable by ui.pushbuffer()/popbuffer(), but changing the file object seems cleaner.

schemes: move re construction to module-level and python3-ify This makes the schemes extension load correctly in Python 3.

dispatch: cope with sys.version being unicode on Python 3