vfs: use 'vfs' module directly in 'mercurial.statichttprepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.bundlerepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.debugcommand' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.simplemerge' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.cmdutil' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.subrepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.archival' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.store' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.patch' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.repair' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.localrepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

chg: forward user-defined signals SIGUSR1 and SIGUSR2 are reserved for user-defined behaviors. They may be redefined by an hg extension [1], but cannot be easily redefined for chg. Since the default behavior (kill) is not that useful for chg, let's forward them to hg, hoping it got redefined there and could be more useful. [1] https://bitbucket.org/facebook/hg-experimental/commits/e7c883a465

chg: document why we send SIGHUP and SIGINT to process group This makes the code more consistent - other signals are documented.

tests: make test-shelve.t timing-independent It was sometimes taking 2s for me (not the "1s" the test expected).

tests: make run-tests.py run on Python 3 again

rebase: move actual rebase into a single transaction Previously, rebasing would open several transaction over the course of rebasing several commits. Opening a transaction can have notable overhead (like copying the dirstate) which can add up when rebasing many commits. This patch adds a single large transaction around the actual commit rebase operation, with a catch for intervention which serializes the current state if we need to drop back to the terminal for user intervention. Amazingly, almost all the tests seem to pass. On large repos with large working copies, this can speed up rebasing 7 commits by 25%. I'd expect the percentage to be a bit larger for rebasing even more commits. There are minor test changes because we're rolling back the entire transaction during unexpected exceptions instead of just stopping mid-rebase, so there's no more backup bundle. It also leave an unknown file in the working copy, since our clean up 'hg update' doesn't delete unknown files.

rebase: allow aborting if last-message.txt is missing Previously, if .hg/rebasestate existed but .hg/last-message.txt was missing, 'hg rebase --abort' would say there's no rebase in progress but 'hg checkout foo' would say 'abort: rebase in progress'. It turns out loading the collapse message will throw a "no rebase in progress" error if the file doesn't exist, even though .hg/rebasestate obviously indicates a rebase is in progress. The fix is to only throw an exception if we're trying to --continue, and to just eat the issues if we're doing --abort. This issue is exposed by us writing the rebase state earlier in the process. This will be used by later patches to ensure the user can appropriately 'hg rebase --abort' if there's a crash before the first the first commit has finished rebasing. Tests cover all of this. The only negative affect is we now require a hg rebase --abort in a very specific exception case, as shown in the test.

rebase: add storestatus support for transactions This let's the status writing logic support transactions. This will be useful in a later patch where we add a transaction around the entire rebase.

rebase: move storestatus onto rebaseruntime The rebaseruntime class already has the restorestatus function, so let's make it own the store status function too. This get's rid of a lot of unnecessary argument passing and will make a future patch cleaner that refactors storestatus to support transactions.

rebase: clear updatestate during rebase --abort in more cases Previously, rebase --abort would only call update if you were on a node that had already been rebased. This meant that if the rebase failed during the rebase of the first commit, the working copy would be left dirty (with a .hg/updatestate file) and rebase --abort would not have update to clean it up. The fix is to also perform an update if you're still on the target node or on the original working copy node (since the working copy may be dirty, we still need to do the update). We don't want to perform an update in all cases though because of issue4009. A subsequent patch makes this case much more common, since it causes the entire rebase transaction to rollback during unexpected exceptions. This causes the existing test-rebase-abort.t to cover this case.

repoview: convert attribute names to unicodes on Python 3 In Python 3, the attribute names must be strings i.e. unicodes.

parsers: alias long to int on Python 3

store: slice over a bytestring to get characters instead of ascii values On Python 2, >>> a = b'abc' >>> a[1] 'b' Whereas on python 3, >>> a = b'abc' >>> a[1] 98 >>> a[1:2] b'b' This does not change behaviour on python 2.

vfs: use 'vfs' module directly in 'mercurial.hg' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: extract 'vfs' class and related code to a new 'vfs' module (API) The 'scmutil' is growing large (1500+ lines) and 2/5 of it is related to vfs. We extract the 'vfs' related code in its own module get both module back to a better scale and clearer contents. We keep all the references available in 'scmutil' for now as many reference needs to be updated.

vfs: replace 'scmutil.opener' usage with 'scmutil.vfs' The 'vfs' class is the first class citizen for years. We remove all usages of the older API. This will let us remove the old API eventually.

fsmonitor: remove use of repo.opener This has been deprecated, so we need to switch to the appropriate vfs apis.

help: align description of 'base rev' with reality [issue5488] The text about revlogs seems to be wrong about -1 being used to indicate the start of a delta chain. Attempt to correct this.

help: fix internals.changegroups Add information about tree manifests, copy edit the text and fix up a few ambiguities. The document also contains a few additional fixes from Siddharth Agarwal <sid0@fb.com>, who used it to build a parser for changegroups in Rust.

share: drop 'relshared' requirement as well

share: fix typo to drop 'shared' requirement on unshare This must be a typo and it seems correct to drop the requirement since the repo is no longer a shared repository.

similar: compare between actual file contents for exact identity Before this patch, similarity detection logic (for addremove and automv) depends entirely on SHA-1 digesting. But this causes incorrect rename detection, if: - removing file A and adding file B occur at same committing, and - SHA-1 hash values of file A and B are same This may prevent security experts from managing sample files for SHAttered issue in Mercurial repository, for example. https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html https://shattered.it/ Hash collision itself isn't so serious for core repository functionality of Mercurial, described by mpm as below, though. https://www.mercurial-scm.org/wiki/mpm/SHA1 This patch compares between actual file contents after hash comparison for exact identity. Even after this patch, SHA-1 is still used, because it is reasonable enough to quickly detect existence of "(almost) same" file. - replacing SHA-1 causes decreasing performance, and - replacement of it has ambiguity, yet Getting content of removed file (= rfctx.data()) at each exact comparison should be cheap enough, even though getting content of added one costs much. ======= ============== ===================== file fctx data() reads from ======= ============== ===================== removed filectx in-memory revlog data added workingfilectx storage ======= ============== =====================

localrepo: handle rename with hardlinks properly In "aftertrans", we rename "journal.*" to "undo.*". We expect "journal.*" files to disappear after renaming. However, if "journal.foo" and "undo.foo" refer to a same file (hardlink), rename may be a no-op, leaving both files on disk, according to Linux manpage [1]: If oldpath and newpath are existing hard links referring to the same file, then rename() does nothing, and returns a suc‐ cess status. The POSIX specification [2] is not very clear about what to do. To be safe, remove "undo.*" before the rename so "journal.*" cannot be left on disk. [1]: http://man7.org/linux/man-pages/man2/rename.2.html [2]: http://pubs.opengroup.org/onlinepubs/9699919799/

dirstate: avoid unnecessary load+dump during backup Previously, dirstate.savebackup unconditionally dumps the dirstate map to disk. It may require loading dirstate first to be able to dump it. Those operations could be expensive if the dirstate is big, and could be avoided if we know the dirstate file is up-to-date. This patch avoids the read and write if the dirstate is clean. In that case, we just do a plain copy without any serialization. This should make commands which use transactions but do not touch dirstate faster. For example, "hg bookmark -r REV NAME".

dirstate: try to use hardlink to backup dirstate This should be more efficient once util.copyfile has real hardlink support.

dirstate: track updated files to improve write time Previously, dirstate.write() would iterate over the entire dirstate to find any entries that needed to be marked 'lookup' (i.e. if they have the same timestamp as now). This was O(working copy) and slow in large repos. It was most visible when rebasing or histediting multiple commits, since it gets executed once per commit, even if the entire rebase/histedit is wrapped in a transaction. The fix is to track which files have been editted, and only check those to see if they need to be marked as 'lookup'. This saves 25% on histedit times in very large repositories. I tested this by adding temporary debug logic to verify that the old files processed in the loop matched the new files processed in the loop and running the test suite.

hook: give exthooks tags for blocking time The ui.system autogenerated tag isn't really useful - as they're named, let's use the name the user gave us.

filemerge: tag merge tool for blocked times Merge tools can take a while - let's ensure that they're appropriately tagged

hgk: set a blocked tag when the user invokes view

transplant: set a blockedtag when invoking external filter

config: set blockedtag when invoking configuration edit

bisect: set a blockedtag when running the check command So that the hg bisect data clearly shows the bisect command separately to the main data set.

dispatch: set a blockedtag when running an external alias

patch: set a blockedtag when running an external filter

sshpeer: set a blockedtag when starting ssh So that the data is readable.

revert: always display hunks reversed when reverting to parent When reverting to the parent of working directory, operation is "discard" so we want hunks to be presented in the same order as the diff (i.e. "reversed"). So we do not query the experimental.revertalternateinteractivemode option in this case and always set "reversehunks" to True.

fileset: add a 'status(...)' predicate to control evaluation context Same as 'revs', this predicate does not select files but switches the evaluation context. This allow to match file according arbitrary status call. We can now express the same query as 'hg status'. The API (two 'revsingle' class) have been picked instead of a single 'revs' revset for multiple reasons: * it is less confusing to express * it allow to express more query (eg: backward status, cross branch status)

fileset: allow to specify a basectx for status This will be used for a predicates that defines the status range of a match.

fileset: add revs(revs, fileset) to evaluate set in working directory Unlike other functions, "revs()" does not select files but switches the evaluation context. This allow to match file with property in another revision that the one currently evaluated. This changeset is based on work from Yuya Nishihara.

fileset: add function to switch revision where fileset will be evaluated If the subset isn't filtered yet, i.e. if fullmatchctx, the new subset is recalculated from scratch. Otherwise, it is narrowed by the existing subset.

fileset: extract function that builds status tuple only if necessary This function will be used to recalculate status when mctx.ctx is switched.

fileset: build initial subset in fullmatchctx class

fileset: extract function that builds initial subset from ctx or status This function will be used to recalculate subset when mctx.ctx is switched.

fileset: add class to host special handling of initial subset Future patches will add a function to switch mctx.ctx object so that we can forcibly evaluate a fileset expression in a specified revision. For example, new "revs()" function will be used to match predicate agains another revision $ hg revert 'set:revs(42, added())' fullmatchctx class is similar to revset.fullreposet. It will allow us to recalculate the subset only if it is not filtered yet.

patchbomb: add config knob to generate flags by template (issue5354) This can be used to flag patches by branch or topic automatically. Flags optionally given by --flag option are exported as {flags} template keyword, so you can add --flag V2.

patchbomb: pass around ui and revs that are needed for flag template See the next patch for why.

patchbomb: build patch texts by _getpatchmsgs() Now _getpatchmsgs() knows revision numbers, which allows us to generate flags by applying a template to changectx objects.

patchbomb: drop internal option for pbranch extension (API) I want to move _getpatches() to _getpatchmsgs() to make sure each patch text is tied with the corresponding revision number. This helps adding templater support. IIRC, the pbranch extension doesn't work with the recent Mercurial versions, so the removal of this option wouldn't hurt.

patchbomb: factor out function that builds a prefix string to patch subject I'll add templating support.

formatter: add argument to change output file of non-plain formatter This allows us to build data not written to the console. That would be doable by ui.pushbuffer()/popbuffer(), but changing the file object seems cleaner.

schemes: move re construction to module-level and python3-ify This makes the schemes extension load correctly in Python 3.

dispatch: cope with sys.version being unicode on Python 3

dispatch: allow testedwith to be bytes or str

ui: fix ui.traceback on Python 3

ui: fix opts labeling on ui.warn et al for Python 3 This is a step towards fixing extension load warnings on Python 3. Note that I suspect there are still some bugs in this area and that things like color won't work, but the code at least executes and prints text to the console correctly now.

config: add sanity assert that files are opened as binary This helps with some debugging in Python 3, and shouldn't hurt anything in Python 2. The unusual construction using getattr is done so that StringIO/BytesIO instances can be used as well as real files.

update: for "noconflict" updates, print "conflicting changes" on conflict With experimental.updatecheck=noconflict, if the update is aborted because of conlicts, "uncommitted changes" is not quite accurate. Let's use "conflicting changes" instead. Also fix the hint to recomment --clean, not --merge, since that's what we do for other failed updates.

tests: fix test-update-branches to remove non-conflicting file I was clearly very sloppy when I wrote the test case for experimental.updatecheck=noconflict. The test case that checks that one can move between commits with a removed file was deleting a file that was modified between the source and target commits, which resulted in a "change/delete" conflict. Since that is a conlict, the update correctly failed. Let's fix the test by removing a file that is not modified between the two commits.

branches: populate all template keywords in formatter This is a usage example of fm.context().

formatter: add support for changeset templating Some formatter-based commands provide fields that are identical to the ones defined in templatekw, but we had to specify them manually to support all changeset-based template keywords. This patch adds fm.context() that populates all templatekw. These keywords are available only in template output, so we still need to set important keywords via fm.data() if they should be available in e.g. JSON output. Currently fm.context() takes only 'ctx' argument. It will eventually be extended to take 'fctx' to support file-based keywords (e.g. {path}) seen in hgweb.

templatekw: move defaulttmpl constant from changeset_templater These templates are used when rendering inner lists of some template keywords, so it makes sense to define them in templatekw. This allows us to reuse them to create a templateformatter knowing changectx.

formatter: drop filters argument from maketemplater() It's unused now. I want to keep the high-level API simple.

templater: port formatnode filter from changeset_templater This slightly reduces the difference between changeset_templater and formatter, and helps extending formatter to support changeset templating. New formatnode() is not a template filter, but a function since a filter cannot access to ui. And it's marked as DEPRECATED since I think it exists only for compatibility reasons.

update: allow setting default update check to "noconflict" The new value allows update (linear or not) as long as they don't result in file merges. I'm hoping that this value can some day become the default.

update: add experimental config for default way of handling dirty wdir This allows the user to set e.g. experimental.updatecheck=abort to abort update if the working directory is dirty, but still be able to override the behavior with e.g. --merge when needed. I considered adding a --mergelinear option to get back the old behavior even when experimental.updatecheck=abort is set, but I couldn't see why anyone would prefer that over --merge. The default is read in hg.updatetotally(), which means it also applies to "hg pull -u" and "hg unbundle -u".

update: accept --merge to allow merging across topo branches (issue5125)

merge: combine the "merge" cases in docstring table

merge: combine "dirty" cases in docstring table

merge: clarify non-linear default updates in docstring table Non-linear updates won't happen by default, regardless of -c/-C.

merge: combine the two "can't happen" cases in docstring table

merge: move "incompatible options" case first in docstring table

merge: make "linear" an input in docstring table Instead of having two ouputs, it seem simpler to have an addition input. This will allow further simplification.

merge: drop redundant column in docstring table The "same" and "cross" columns now have the same values, so let's combine them into "non-linear".

merge: drop obsolete non-linear cases from docstring table Since 6b1fc09c699a (update: change default destination to tipmost descendant (issue4673) (BC), 2016-02-02), non-linear updates can no longer happen if the user doesn't specify a destination, so we can drop these case from the table in the docstring of merge.update().

revert: move code dealing with deletions closer together

ui: fix configlist on Python 3 Since we're working on bytestrings, we have to use [offset:offset+1] to get consistent behavior on Python 2 and 3. I've only tested the _parse_plain closure, not the _parse_quote one, but I have no real reason to expect the latter to be broken since the fixes were fairly mechanical.

config: pass some optional args as keywords This makes it a little more obvious that self.read is being passed as include, which took me a moment to figure out.

config: load included config files in binary mode I guess we've been getting lucky that this works in Python 2. This fixes loading included config files in Python 3 (it used to fail on the "somebody set up us the BOM" check.)

manifest: remove _repo from manifestctx objects We were storing the repo on the manifestctx objects so that they could access the manifestlog via repo.manifestlog, which would refresh the structure if it became out of date. This caused probems however when we want to have multiple manifest logs in memory at once, like when transitioning to tree manifest from flat manifests, since a tree manifest would try to access sub-trees via repo.manifestlog[node], which was the flat manifest. The solution is to just not store the repo, and instead store the manifestlog that created this context. This removes the invalidation when the in memory manifestlog becomes out of date, but people should probably not be keeping ctx's around that long anyway.

phases: remove experimental.nativephaseskillswitch The native code has been used for a long time. Therefore drop the experimental config option.

manifest: allow specifying the revlog filename Previously we had hardcoded the manifest filename to be 00manifest.i. In our external treemanifest extension, we want to allow writing a treemanifest side by side with a flat manifest, so we need to be able to store the root revisions at a different location (in our extension we use 00manifesttree.i). This patches moves the revlog name to a parameter so we can adjust it.

py3: drop unrequired code from __init__.py Once we are using pycompat.open(), we don't need this hack to convert the second argument to unicodes. So removing this.

py3: add pycompat.open and replace open() calls open() requires mode argument as unicodes on Python 3. This patch introduces pycompat.open() which is inserted to files using transformer and replaces builtins.open() calls.

localrepo: deprecate 'repo.opener' (API) The "new" 'repo.vfs' attribute have been around for almost 5 years. I think we can deprecate the old form now ;-)

statichttp: use 'repo.vfs' as the main attribute We are about to deprecate the repo.opener attribute, we prepare the static http code to be ready for this change.

clonebundle: use 'repo.vfs' instead of 'repo.opener' The later is a 5 years old form.