branchmap: stringify int in a portable way We actually need a bytes in Python 3, and thanks to our nasty source loader this will portably do the right thing.

branchmap: don't use buffer() on Python 3 This is certainly slower than the Python 2 code, but it works, and we can revisit it later if it's a problem.

py3: use bytearray() instead of array('c', ...) constructions Portable from 2.6-3.6.

summary: don't explicitly str() something we're about to %s str() is wrong on Python 3 here, and %s implicitly calls str() anyway, so this was just extra dancing for no reason.

context: implement both __bytes__ and __str__ for Python 3 They're very similar, for obvious reasons.

context: work around `long` not existing on Python 3 I can't figure out what this branch is even trying to accomplish, and it was introduced in ac89a23ca814 which doesn't really shed any insight into why longs are treated differently from ints.

phases: explicitly evaluate list returned by map On Python 3 map() returns a generator, which bool()s to true even if it had an empty input set. Work around this by using list() on the map() result.

ui: check for --debugger in sys.argv using r-string to avoid bytes on py3 Our source loader was errantly turning this --debugger into a bytes, which was then causing me to still get a pager when I was using the debugger on py3. That made life hard.

minirst: use bytes.strip instead of str.strip bytes.strip exists in Python 2.6 and Python 2.7 also.

smcposix: pass unicode as first argument to array.array This is an instance where we can safely convert the first argument, rest are the cases except one where we are using 'c' which is not there in Python 3. So that needs to be handled differently. This will help in making `hg help` run on Python 3.

util: pass encoding.[encoding|encodingmode] as unicodes We need to pass str to encode() and decode().

win32text: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

transplant: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

shelve: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

share: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

mq: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

largefiles: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

keyword: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

journal: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

histedit: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

eol: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

convert: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

test-bundle2-remote-changegroup: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

subrepo: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

repair: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

merge: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

hg-mod: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

commands: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

cmdutil: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

localrepo: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

minirst: make encoding.encoding unicodes to pass into encode() and decode()

minirst: make regular expressions bytes

setup: convert setupversion to unicode Something deep in the bowels of distutils expects "version" passed to setup() to be a str/unicode. So, convert the type. This still works on Python 2 because the string is ascii and an implicit coercion back to str/bytes should work without issue. If it does cause problems, we can always make the unicode conversion dependent on running Python 3. This change makes `python3.5 setup.py install` work.

util: add debugstacktrace depth limit Useful when you don't care about the start of the stack, but only want to see the last entries.

util: strip trailing newline from debugstacktrace message This makes the function more convenient to use as drop-in replacement for ui.write & co.

tests: rework util.debugstacktrace tests Prepare for adding another test.

vfs: use repo.vfs.rename

vfs: use repo.vfs.unlinkpath

merge: use repo.wvfs.unlinkpath

vfs: use repo.wvfs.unlinkpath

policy: try and always have a bytes for module policy debuginstall now runs cleanly in Python 3.

init: zstd is already python3-ready, so don't run it through our importer

config: guard against setconfig specifying unicode values on py3 This was leading to some difficult to trace problems because the values were set in one place, but then blew up much later in the program. Exploding violently with an assertion seems reasonable here.

dispatch: enforce bytes when converting boolean flags to config items This fixes --verbose on Python 3.

py3: add a test that proves %include and missing-extension code works

extensions: tapdance to get reasonable import error formatting I'm not thrilled with this, but it seems to work.

rebase: abort if *any* commit in rebase set is public

merge with stable

httpconnection: rename config to groups Because that is what it is.

httpconnection: don't use dict() Not sure the history here. But we don't do this elsewhere.

formatter: set _first on formatter, not ui The _first field is used for tracking when to emit a separator between items. It seems like it's clearly formatter state, not ui state, so let's move it there.

rebase: unhide original working directory node as well (issue5219) By including the working directory revision at the start of rebase in the repo._rebaseset, we make sure it's not hidden when we update back to it at the end of the rebase. This feels like abusing the set a bit given its name (_rebaseset), but I couldn't think of another name that's clearly better.

rebase: pass in a regular set to _setrebasesetvisibility() Trivial refactoring to simplify the next patch.

tests: demonstrate broken rebase from obsolete commit This shows the bug I reported in issue 5219.

treemanifest: make node reuse match flat manifest behavior In a flat manifest, a node with the same content but different parents is still considered a new node. In the current tree manifests however, if the content is the same, we ignore the parents entirely and just reuse the existing node. In our external treemanifest extension, we want to allow having one treemanifest for every flat manifests, as a way of easeing the migration to treemanifests. To make this possible, let's change the root node treemanifest behavior to match the behavior for flat manifests, so we can have a 1:1 relationship. While this sounds like a BC breakage, it's not actually a state users can normally get in because: A) you can't make empty commits, and B) even if you try to make an empty commit (by making a commit then amending it's changes away), the higher level commit logic in localrepo.commitctx() forces the commit to use the original p1 manifest node if no files were changed. So this would only affect extensions and automation that reached passed the normal localrepo.commit() logic straight into the manifest logic.

help: update help.internalstable for new censor docs

parsers: drop old nonnormalentries method This is okay to do because the Python will fall back transparently if the method is missing.

parsers: avoid leak of nonnset and otherpset Py_BuildValue increments the refcount, rather than stealing the reference, which I missed in code review.

sslutil: issue warning when [hostfingerprint] is used Mercurial 3.9 added the [hostsecurity] section, which is better than [hostfingerprints] in every way. One of the ways that [hostsecurity] is better is that it supports SHA-256 and SHA-512 fingerprints, not just SHA-1 fingerprints. The world is moving away from SHA-1 because it is borderline secure. Mercurial should be part of that movement. This patch adds a warning when a valid SHA-1 fingerprint from the [hostfingerprints] section is being used. The warning informs users to switch to [hostsecurity]. It even prints the config option they should set. It uses the SHA-256 fingerprint because recommending a SHA-1 fingerprint in 2017 would be ill-advised. The warning will print itself on every connection to a server until it is fixed. There is no way to suppress the warning. I admit this is annoying. But given the security implications of sticking with SHA-1, I think this is justified. If this patch is accepted, I'll likely send a follow-up to start warning on SHA-1 certificates in [hostsecurity] as well. Then sometime down the road, we can drop support for SHA-1 fingerprints. Credit for this idea comes from timeless in issue 5466.

setup: use setuptools on Windows (issue5400) We've had a long, complicated history with setuptools. We want to make it the universal default. But when we do, it breaks things. `python setup.py build` is broken on Windows today. Forcing the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things. Since the previous bustage with making setuptools the default was on !Windows, it seems safe to move ahead with the setuptools transition on Windows. So this patch does that.

schemes: use br'' literal to define bytes regexp

help: fix layout of pre-formatted text

help: fix example of revs() fileset

filecache: make 'join' abstract All subclasses redefine this method, so we can make it abstract.

filecache: explicitly test 'repofilecache' The tests is actually about testing a repofilecache (it uses a fake repo). We make this clear to prevent blockers while cleaning theses API.

repofilecache: directly use 'repo.vfs.join' The 'vfs' attribute already have all methods we need, the value of going through the repository for this is low. so we removes it.

repofilecache: define a 'join' method We are about to turn the 'join' method of the base class Abstract, so we need on to be defined in the localrepo. The ultimate goal here is to be able to stop relying for the 'localrepo' class to have a 'join' methods (there is above one hundred methods on 'localrepo'. This change make te 'repo' file cache have its own code so that we can prepare this change to the repostory class. explicite join

wix: add censor docs to installer script Spotted by Matt Harbison.

internals: add some brief documentation about censor

localrepo: rename proxycls to filteredrepo When debugging in a Python shell, the type of "repo" is "proxycls", which could confuse new people. In [1]: repo Out[1]: <mercurial.localrepo.proxycls at 0x7f65d4b976d0> Let's rename it to "filteredrepo" to make it clearer.

dirstate: track otherparent files same as nonnormal Calling dirstate.setparents() is expensive in a large repo because it iterates over every file in the dirstate. It does so to undo any merge state or otherparent state files. Merge state files are already covered by dirstate._nonnormalset, so we just need to track otherparent files in a similar manner to avoid the full iteration here. Fixing this shaves 20-25% off histedit in large repos. I tested this by adding temporary debug logic to verify that the old files processed in the loop matched the new files processed in the loop and running the test suite.

pycompat: verify sys.argv exists before forwarding it (issue5493) ISAPI_WSGI doesn't set up sys.argv, so we have to look for the attribute before assuming it exists.

hgweb: use patch.diffhunks in webutil.diffs to simplify the algorithm Function patch.diffhunks yields items for a "block" (i.e. a file) as a whole so take advantage of this to simplify the algorithm and avoid parsing diff lines to determine whether we're starting a new "block" or not. Thus we drop to external block counter and rely on diffhunks iterations instead. We also take advantage of the fact that patch.diffhunks() yields *lines* of hunks (instead of a string) to avoid building a list that is ''.join-ed into a string that is then split. As lines in 'header' returned by patch.diffhunks() have no trailing new line, we need to insert it ourselves to match template expectations.

hgweb: start enumerate at 1 in webutil.diffs's inner function prettyprintlines

patch: add a diffhunks function yielding (diffheaders, hunks) trydiff function now yield (header, hunks) tuple that are processed by diffhunks(). Then diff() is a wrapper around diffhunks().

mdiff: let unidiff return (diffheader, hunks) This will be used to make it possible to filter diff hunks based on this range information. Now unidiff returns a 'hunks' generator that yield tuple (hunkrange, hunklines) coming from _unidiff() with 'newline at end of file' processing.

mdiff: extract a checknonewline inner function in unidiff()

mdiff: distinguish diff headers from hunks in unidiff() Let unidiff return the list of headers it produces (lines '--- <original>' and '+++ <new>') apart from diff hunks. In patch.diff(), we combine headers generated there (not specific to unified format) with those from unidiff(). By returning a list of header lines, we do not append new lines in datetag inner function of unidiff() so that all header lines are '\n'.join-ed in a similar way.

test: end printed diff "hunks" with an empty string in test-context.py So that the resulting diff is correct and does not include a spurious empty line between lines "diff --git a/foo b/foo" and "--- a/foo".

mdiff: let _unidiff yield hunks as (<range information>, <hunk lines>) Now _unidiff yields each hunk lines packed into a tuple with the "range information" `(s1, l1, s2, l2)` that is used to build the typical hunk header '@@ -s1,l1 +s2,l2 @@'. This will be used to make it possible to filter diff hunks based on this range information. The new "range information" is ignored in unidiff() (only caller of _unidiff) for now.

mdiff: turn the comment above _unidiff into a docstring

mdiff: compute newlines-splitted texts within _unidiff There is no reason to compute splitted texts l1, l2 in unidiff() before calling _unidiff as they are only used with the latter function.

dispatch: add pagination of two more help cases I missed these in the last round anf Yuya spotted them in review. Thanks!

help: avoid mutating passed-in `keep` list in `formattedhelp`

extensions: use [0:1] slice on config path instead of [0] This behaves the same in Python 2 and Python 3, even though the path is a bytes.

extensions: use inspect module instead of func_code.co_argcount Fixes the extsetup argspec check on Python 3.

treemanifest: add tests covering hg diff of partial trees Previously the hg files tests also covered the logic (i.e. treemanifest.matches) that governed how hg diff limited its diff. In a future patch we will be switching treemanifest.diff() to have a custom implementation, so let's go ahead and add equivalent test coverage for hg diff.

context: remove uses of manifest.matches This removes the uses of manifest.matches in context.py in favor of the new manifest.diff(match) api. This is part of removing manifest.matches since it is O(manifest). To drop the dependency on ctx._manifestmatches(s) we transfer responsibilty for creating status oriented manifests over to ctx._buildstatusmanifest(s). This already existed for workingctx, we just need to implement a simple version for basectx. The old _manifestmatches functionality is basically identical to the _buildstatusmanifest functionality (minus the matching part), so no behavior should be lost.

context: remove assumptions about manifest creation during _buildstatus Previously we called self.manifest() in some cases to preload the first manifest. This relied on the assumption that the later _manifestmatches() call did not duplicate any work the original self.manifest() call did. Let's remove that assumption, since it bit me during my refactors of this area and is easy to remove.

context: move _manifest from committablectx to workingctx committablectx had a _manifest implementation that was only used by the derived workingctx class. The other derived versions, like memctx and metadataonlyctx, define their own _manifest functions. Let's move the function down to workingctx, and let's break it into two parts, the _manifest part that reads from self._status, and the part that actually builds the new manifest. This separation will let us reuse the builder code in a future patch to answer _buildstatus with varying status inputs, since workingctx has special behavior for _buildstatus that the other ctx's don't have.

status: handle more node indicators in buildstatus There are several different node markers that indicate different working copy states. The context._buildstatus function was only handling one of them, and this patch makes it handle all of them (falling back to file content comparisons when in one of these states). This affects a future patch where we get rid of context._manifestmatches as part of getting rid of manifest.matches(). context._manifestmatches is currently hacky in that it uses the newnodeid for all added and modified files, which is why the current newnodeid check is sufficient. When we get rid of this function and use the normal manifest.diff function, we start to see the other indicators in the nodes, so they need to be handled or else the tests fail.

merge: remove uses of manifest.matches This gets rid of the manifest.matches calls in merge.py in favor of the new api. This is part of getting rid of manifest.matches since it is O(manifest).

copies: remove use of manifest.matches Convert the existing use of manifest.matches to use the new api. This is part of getting rid of manifest.matches, since it is O(manifest).

manifest: add match argument to diff and filesnotin As part of removing manifest.matches (since it is O(manifest)), let's start by adding match arguments to diff and filesnotin. As we'll see in later patches, these are the only flows that actually use matchers, so by moving the matching into the actual functions, other manifest implementations can make more efficient algorithsm. For instance, this will allow treemanifest diff's to only iterate over the files that are different AND meet the match criteria. No consumers are changed in this patches, but the code is fairly easy to verify visually. Future patches will convert consumers to use it. One test was affected because it did not use the kwargs version of the clean parameter.

fileset: drop false function signatures from revs() and status() docs They should be inserted by @predicate helper.

py3: factor out bytechr() function I also changed xrange(127) to range(127) as the number is relatively small.

vfs: use 'vfs' module directly in 'test-clone-uncompressed' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'test-filecache' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'test-parseindex' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'test-lock' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'contrib/undumprevlog' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'hgext.largefile' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'hgext.convert' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'hgext.transplant' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'hgext.shelve' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'hgext.mq' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.unionrepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.statichttprepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.bundlerepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.debugcommand' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.simplemerge' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.cmdutil' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.subrepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.archival' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.store' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.patch' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.repair' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: use 'vfs' module directly in 'mercurial.localrepo' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

chg: forward user-defined signals SIGUSR1 and SIGUSR2 are reserved for user-defined behaviors. They may be redefined by an hg extension [1], but cannot be easily redefined for chg. Since the default behavior (kill) is not that useful for chg, let's forward them to hg, hoping it got redefined there and could be more useful. [1] https://bitbucket.org/facebook/hg-experimental/commits/e7c883a465

chg: document why we send SIGHUP and SIGINT to process group This makes the code more consistent - other signals are documented.

tests: make test-shelve.t timing-independent It was sometimes taking 2s for me (not the "1s" the test expected).

tests: make run-tests.py run on Python 3 again

rebase: move actual rebase into a single transaction Previously, rebasing would open several transaction over the course of rebasing several commits. Opening a transaction can have notable overhead (like copying the dirstate) which can add up when rebasing many commits. This patch adds a single large transaction around the actual commit rebase operation, with a catch for intervention which serializes the current state if we need to drop back to the terminal for user intervention. Amazingly, almost all the tests seem to pass. On large repos with large working copies, this can speed up rebasing 7 commits by 25%. I'd expect the percentage to be a bit larger for rebasing even more commits. There are minor test changes because we're rolling back the entire transaction during unexpected exceptions instead of just stopping mid-rebase, so there's no more backup bundle. It also leave an unknown file in the working copy, since our clean up 'hg update' doesn't delete unknown files.

rebase: allow aborting if last-message.txt is missing Previously, if .hg/rebasestate existed but .hg/last-message.txt was missing, 'hg rebase --abort' would say there's no rebase in progress but 'hg checkout foo' would say 'abort: rebase in progress'. It turns out loading the collapse message will throw a "no rebase in progress" error if the file doesn't exist, even though .hg/rebasestate obviously indicates a rebase is in progress. The fix is to only throw an exception if we're trying to --continue, and to just eat the issues if we're doing --abort. This issue is exposed by us writing the rebase state earlier in the process. This will be used by later patches to ensure the user can appropriately 'hg rebase --abort' if there's a crash before the first the first commit has finished rebasing. Tests cover all of this. The only negative affect is we now require a hg rebase --abort in a very specific exception case, as shown in the test.

rebase: add storestatus support for transactions This let's the status writing logic support transactions. This will be useful in a later patch where we add a transaction around the entire rebase.

rebase: move storestatus onto rebaseruntime The rebaseruntime class already has the restorestatus function, so let's make it own the store status function too. This get's rid of a lot of unnecessary argument passing and will make a future patch cleaner that refactors storestatus to support transactions.

rebase: clear updatestate during rebase --abort in more cases Previously, rebase --abort would only call update if you were on a node that had already been rebased. This meant that if the rebase failed during the rebase of the first commit, the working copy would be left dirty (with a .hg/updatestate file) and rebase --abort would not have update to clean it up. The fix is to also perform an update if you're still on the target node or on the original working copy node (since the working copy may be dirty, we still need to do the update). We don't want to perform an update in all cases though because of issue4009. A subsequent patch makes this case much more common, since it causes the entire rebase transaction to rollback during unexpected exceptions. This causes the existing test-rebase-abort.t to cover this case.

repoview: convert attribute names to unicodes on Python 3 In Python 3, the attribute names must be strings i.e. unicodes.

parsers: alias long to int on Python 3

store: slice over a bytestring to get characters instead of ascii values On Python 2, >>> a = b'abc' >>> a[1] 'b' Whereas on python 3, >>> a = b'abc' >>> a[1] 98 >>> a[1:2] b'b' This does not change behaviour on python 2.

vfs: use 'vfs' module directly in 'mercurial.hg' Now that the 'vfs' classes moved in their own module, lets use the new module directly. We update code iteratively to help with possible bisect needs in the future.

vfs: extract 'vfs' class and related code to a new 'vfs' module (API) The 'scmutil' is growing large (1500+ lines) and 2/5 of it is related to vfs. We extract the 'vfs' related code in its own module get both module back to a better scale and clearer contents. We keep all the references available in 'scmutil' for now as many reference needs to be updated.

vfs: replace 'scmutil.opener' usage with 'scmutil.vfs' The 'vfs' class is the first class citizen for years. We remove all usages of the older API. This will let us remove the old API eventually.

fsmonitor: remove use of repo.opener This has been deprecated, so we need to switch to the appropriate vfs apis.

help: align description of 'base rev' with reality [issue5488] The text about revlogs seems to be wrong about -1 being used to indicate the start of a delta chain. Attempt to correct this.

help: fix internals.changegroups Add information about tree manifests, copy edit the text and fix up a few ambiguities. The document also contains a few additional fixes from Siddharth Agarwal <sid0@fb.com>, who used it to build a parser for changegroups in Rust.

share: drop 'relshared' requirement as well

share: fix typo to drop 'shared' requirement on unshare This must be a typo and it seems correct to drop the requirement since the repo is no longer a shared repository.

similar: compare between actual file contents for exact identity Before this patch, similarity detection logic (for addremove and automv) depends entirely on SHA-1 digesting. But this causes incorrect rename detection, if: - removing file A and adding file B occur at same committing, and - SHA-1 hash values of file A and B are same This may prevent security experts from managing sample files for SHAttered issue in Mercurial repository, for example. https://security.googleblog.com/2017/02/announcing-first-sha1-collision.html https://shattered.it/ Hash collision itself isn't so serious for core repository functionality of Mercurial, described by mpm as below, though. https://www.mercurial-scm.org/wiki/mpm/SHA1 This patch compares between actual file contents after hash comparison for exact identity. Even after this patch, SHA-1 is still used, because it is reasonable enough to quickly detect existence of "(almost) same" file. - replacing SHA-1 causes decreasing performance, and - replacement of it has ambiguity, yet Getting content of removed file (= rfctx.data()) at each exact comparison should be cheap enough, even though getting content of added one costs much. ======= ============== ===================== file fctx data() reads from ======= ============== ===================== removed filectx in-memory revlog data added workingfilectx storage ======= ============== =====================

localrepo: handle rename with hardlinks properly In "aftertrans", we rename "journal.*" to "undo.*". We expect "journal.*" files to disappear after renaming. However, if "journal.foo" and "undo.foo" refer to a same file (hardlink), rename may be a no-op, leaving both files on disk, according to Linux manpage [1]: If oldpath and newpath are existing hard links referring to the same file, then rename() does nothing, and returns a suc‐ cess status. The POSIX specification [2] is not very clear about what to do. To be safe, remove "undo.*" before the rename so "journal.*" cannot be left on disk. [1]: http://man7.org/linux/man-pages/man2/rename.2.html [2]: http://pubs.opengroup.org/onlinepubs/9699919799/

dirstate: avoid unnecessary load+dump during backup Previously, dirstate.savebackup unconditionally dumps the dirstate map to disk. It may require loading dirstate first to be able to dump it. Those operations could be expensive if the dirstate is big, and could be avoided if we know the dirstate file is up-to-date. This patch avoids the read and write if the dirstate is clean. In that case, we just do a plain copy without any serialization. This should make commands which use transactions but do not touch dirstate faster. For example, "hg bookmark -r REV NAME".

dirstate: try to use hardlink to backup dirstate This should be more efficient once util.copyfile has real hardlink support.

dirstate: track updated files to improve write time Previously, dirstate.write() would iterate over the entire dirstate to find any entries that needed to be marked 'lookup' (i.e. if they have the same timestamp as now). This was O(working copy) and slow in large repos. It was most visible when rebasing or histediting multiple commits, since it gets executed once per commit, even if the entire rebase/histedit is wrapped in a transaction. The fix is to track which files have been editted, and only check those to see if they need to be marked as 'lookup'. This saves 25% on histedit times in very large repositories. I tested this by adding temporary debug logic to verify that the old files processed in the loop matched the new files processed in the loop and running the test suite.

hook: give exthooks tags for blocking time The ui.system autogenerated tag isn't really useful - as they're named, let's use the name the user gave us.

filemerge: tag merge tool for blocked times Merge tools can take a while - let's ensure that they're appropriately tagged

hgk: set a blocked tag when the user invokes view

transplant: set a blockedtag when invoking external filter

config: set blockedtag when invoking configuration edit

bisect: set a blockedtag when running the check command So that the hg bisect data clearly shows the bisect command separately to the main data set.

dispatch: set a blockedtag when running an external alias

patch: set a blockedtag when running an external filter

sshpeer: set a blockedtag when starting ssh So that the data is readable.

revert: always display hunks reversed when reverting to parent When reverting to the parent of working directory, operation is "discard" so we want hunks to be presented in the same order as the diff (i.e. "reversed"). So we do not query the experimental.revertalternateinteractivemode option in this case and always set "reversehunks" to True.

fileset: add a 'status(...)' predicate to control evaluation context Same as 'revs', this predicate does not select files but switches the evaluation context. This allow to match file according arbitrary status call. We can now express the same query as 'hg status'. The API (two 'revsingle' class) have been picked instead of a single 'revs' revset for multiple reasons: * it is less confusing to express * it allow to express more query (eg: backward status, cross branch status)

fileset: allow to specify a basectx for status This will be used for a predicates that defines the status range of a match.

fileset: add revs(revs, fileset) to evaluate set in working directory Unlike other functions, "revs()" does not select files but switches the evaluation context. This allow to match file with property in another revision that the one currently evaluated. This changeset is based on work from Yuya Nishihara.

fileset: add function to switch revision where fileset will be evaluated If the subset isn't filtered yet, i.e. if fullmatchctx, the new subset is recalculated from scratch. Otherwise, it is narrowed by the existing subset.

fileset: extract function that builds status tuple only if necessary This function will be used to recalculate status when mctx.ctx is switched.

fileset: build initial subset in fullmatchctx class

fileset: extract function that builds initial subset from ctx or status This function will be used to recalculate subset when mctx.ctx is switched.

fileset: add class to host special handling of initial subset Future patches will add a function to switch mctx.ctx object so that we can forcibly evaluate a fileset expression in a specified revision. For example, new "revs()" function will be used to match predicate agains another revision $ hg revert 'set:revs(42, added())' fullmatchctx class is similar to revset.fullreposet. It will allow us to recalculate the subset only if it is not filtered yet.

patchbomb: add config knob to generate flags by template (issue5354) This can be used to flag patches by branch or topic automatically. Flags optionally given by --flag option are exported as {flags} template keyword, so you can add --flag V2.

patchbomb: pass around ui and revs that are needed for flag template See the next patch for why.

patchbomb: build patch texts by _getpatchmsgs() Now _getpatchmsgs() knows revision numbers, which allows us to generate flags by applying a template to changectx objects.

patchbomb: drop internal option for pbranch extension (API) I want to move _getpatches() to _getpatchmsgs() to make sure each patch text is tied with the corresponding revision number. This helps adding templater support. IIRC, the pbranch extension doesn't work with the recent Mercurial versions, so the removal of this option wouldn't hurt.

patchbomb: factor out function that builds a prefix string to patch subject I'll add templating support.

formatter: add argument to change output file of non-plain formatter This allows us to build data not written to the console. That would be doable by ui.pushbuffer()/popbuffer(), but changing the file object seems cleaner.

schemes: move re construction to module-level and python3-ify This makes the schemes extension load correctly in Python 3.

dispatch: cope with sys.version being unicode on Python 3

dispatch: allow testedwith to be bytes or str

ui: fix ui.traceback on Python 3

ui: fix opts labeling on ui.warn et al for Python 3 This is a step towards fixing extension load warnings on Python 3. Note that I suspect there are still some bugs in this area and that things like color won't work, but the code at least executes and prints text to the console correctly now.

config: add sanity assert that files are opened as binary This helps with some debugging in Python 3, and shouldn't hurt anything in Python 2. The unusual construction using getattr is done so that StringIO/BytesIO instances can be used as well as real files.

update: for "noconflict" updates, print "conflicting changes" on conflict With experimental.updatecheck=noconflict, if the update is aborted because of conlicts, "uncommitted changes" is not quite accurate. Let's use "conflicting changes" instead. Also fix the hint to recomment --clean, not --merge, since that's what we do for other failed updates.

tests: fix test-update-branches to remove non-conflicting file I was clearly very sloppy when I wrote the test case for experimental.updatecheck=noconflict. The test case that checks that one can move between commits with a removed file was deleting a file that was modified between the source and target commits, which resulted in a "change/delete" conflict. Since that is a conlict, the update correctly failed. Let's fix the test by removing a file that is not modified between the two commits.

branches: populate all template keywords in formatter This is a usage example of fm.context().

formatter: add support for changeset templating Some formatter-based commands provide fields that are identical to the ones defined in templatekw, but we had to specify them manually to support all changeset-based template keywords. This patch adds fm.context() that populates all templatekw. These keywords are available only in template output, so we still need to set important keywords via fm.data() if they should be available in e.g. JSON output. Currently fm.context() takes only 'ctx' argument. It will eventually be extended to take 'fctx' to support file-based keywords (e.g. {path}) seen in hgweb.

templatekw: move defaulttmpl constant from changeset_templater These templates are used when rendering inner lists of some template keywords, so it makes sense to define them in templatekw. This allows us to reuse them to create a templateformatter knowing changectx.

formatter: drop filters argument from maketemplater() It's unused now. I want to keep the high-level API simple.

templater: port formatnode filter from changeset_templater This slightly reduces the difference between changeset_templater and formatter, and helps extending formatter to support changeset templating. New formatnode() is not a template filter, but a function since a filter cannot access to ui. And it's marked as DEPRECATED since I think it exists only for compatibility reasons.

update: allow setting default update check to "noconflict" The new value allows update (linear or not) as long as they don't result in file merges. I'm hoping that this value can some day become the default.

update: add experimental config for default way of handling dirty wdir This allows the user to set e.g. experimental.updatecheck=abort to abort update if the working directory is dirty, but still be able to override the behavior with e.g. --merge when needed. I considered adding a --mergelinear option to get back the old behavior even when experimental.updatecheck=abort is set, but I couldn't see why anyone would prefer that over --merge. The default is read in hg.updatetotally(), which means it also applies to "hg pull -u" and "hg unbundle -u".

update: accept --merge to allow merging across topo branches (issue5125)

merge: combine the "merge" cases in docstring table

merge: combine "dirty" cases in docstring table

merge: clarify non-linear default updates in docstring table Non-linear updates won't happen by default, regardless of -c/-C.

merge: combine the two "can't happen" cases in docstring table

merge: move "incompatible options" case first in docstring table

merge: make "linear" an input in docstring table Instead of having two ouputs, it seem simpler to have an addition input. This will allow further simplification.

merge: drop redundant column in docstring table The "same" and "cross" columns now have the same values, so let's combine them into "non-linear".

merge: drop obsolete non-linear cases from docstring table Since 6b1fc09c699a (update: change default destination to tipmost descendant (issue4673) (BC), 2016-02-02), non-linear updates can no longer happen if the user doesn't specify a destination, so we can drop these case from the table in the docstring of merge.update().

revert: move code dealing with deletions closer together

ui: fix configlist on Python 3 Since we're working on bytestrings, we have to use [offset:offset+1] to get consistent behavior on Python 2 and 3. I've only tested the _parse_plain closure, not the _parse_quote one, but I have no real reason to expect the latter to be broken since the fixes were fairly mechanical.

config: pass some optional args as keywords This makes it a little more obvious that self.read is being passed as include, which took me a moment to figure out.

config: load included config files in binary mode I guess we've been getting lucky that this works in Python 2. This fixes loading included config files in Python 3 (it used to fail on the "somebody set up us the BOM" check.)

manifest: remove _repo from manifestctx objects We were storing the repo on the manifestctx objects so that they could access the manifestlog via repo.manifestlog, which would refresh the structure if it became out of date. This caused probems however when we want to have multiple manifest logs in memory at once, like when transitioning to tree manifest from flat manifests, since a tree manifest would try to access sub-trees via repo.manifestlog[node], which was the flat manifest. The solution is to just not store the repo, and instead store the manifestlog that created this context. This removes the invalidation when the in memory manifestlog becomes out of date, but people should probably not be keeping ctx's around that long anyway.

phases: remove experimental.nativephaseskillswitch The native code has been used for a long time. Therefore drop the experimental config option.

manifest: allow specifying the revlog filename Previously we had hardcoded the manifest filename to be 00manifest.i. In our external treemanifest extension, we want to allow writing a treemanifest side by side with a flat manifest, so we need to be able to store the root revisions at a different location (in our extension we use 00manifesttree.i). This patches moves the revlog name to a parameter so we can adjust it.

py3: drop unrequired code from __init__.py Once we are using pycompat.open(), we don't need this hack to convert the second argument to unicodes. So removing this.

py3: add pycompat.open and replace open() calls open() requires mode argument as unicodes on Python 3. This patch introduces pycompat.open() which is inserted to files using transformer and replaces builtins.open() calls.

localrepo: deprecate 'repo.opener' (API) The "new" 'repo.vfs' attribute have been around for almost 5 years. I think we can deprecate the old form now ;-)

statichttp: use 'repo.vfs' as the main attribute We are about to deprecate the repo.opener attribute, we prepare the static http code to be ready for this change.

clonebundle: use 'repo.vfs' instead of 'repo.opener' The later is a 5 years old form.

localrepo: deprecated 'repo.wopener' (API) The "new" 'repo.wvfs' attribute have been around for almost 5 years. I think we can deprecate the old form now ;-)

localrepo: add some comment about role of various vfs object This should make things clearer for most people.

tests: un-nest the `order` repo in test-rebase-scenario-global This wasn't hurting anything, but it's more in line with how we manage other tests.

osx: install bash and zsh completions by default The zsh location appears to be on the default $fpath for zsh. bash, on the other hand, appears to have no default location for completion scripts, so we follow the lead of Apple's Git distribution and select a semi-arbitrary place in /usr/local for the file.

util: add allowhardlinks module variable To enable extensions to enable hardlinks for certain environments, let's move the 'if False' to be an 'if allowhardlinks' and let extensions modify the allowhardlinks variable. Tests on linux ext4 pass with it set to True and to False.

merge with stable

Added signature for changeset 25703b624d27

Added tag 4.1.1 for changeset 25703b624d27

merge with i18n

i18n-pt_BR: synchronized with 7074589cf22a

i18n-ja: synchronized with 7074589cf22a

revert: remove set(mf) because it's O(manifest) The revert code had a 'set(manifest)' line in it, which has a runtime equivalent to the size of the manifest. With alternative manifest implementations, like treemanifest, this can be extra expensive. Let's rewrite it to be O(changes) instead of O(manifest size).

share: add --relative flag to store a relative path to the source Storing a relative path the source repository is useful when exporting repositories over the network or when they're located on external drives where the mountpoint isn't always fixed. Currently, Mercurial interprets paths in `.hg/shared` relative to $PWD. I suspect this is very much unintentional, and you have to manually edit `.hg/shared` in order to trigger this behaviour. However, on the off chance that someone might rely on it, I added a new capability called 'relshared'. In addition, this makes earlier versions of Mercurial fail with a graceful error. I should note that I haven't tested this patch on Windows.

minirst: support passing admonitions into findadmonitions() and parse() This will allow consumers to declare a custom list of admonitions to parse. Without this patch, custom admonitions would get removed when prunecomments() is run. We could add an argument controlling whether prunecomments() is run. However, it is better to convert the "paragraph" block to an "admonition" block so consumers don't have to parse for custom admonitions.

minirst: dynamically compile admonitions regexp Currently, parsing admonitions uses a static regular expression created from a pre-defined list of admonitions. A future patch will introduce a feature that needs to parse custom admonitions. Prepare for this by compiling the admonitions regular expression during each function invocation. Strictly speaking, there is a slight performance loss here. But we only run this code as part of displaying help text. I don't think the loss will be noticeable and I don't think we care if it were.

minirst: detect bullet lists using asterisks Previously, the "bullet" regular expression excluded the asterisk ('*') as a character denoting a bulleted list. Why I'm not sure because the asterisk seems to be the canonical bullet character in reST these days. This patch makes asterisk-prefixed lines parse as bulleted lists.

color: update the help table We also need to reference the new topic in the great old help table.

ui: remove superfluous indent in _write()

smartset: reorder initialization of baseset in more intuitive way What we want to do is to assign either _set or _list per the given data type.

config: update the Windows example config file We move from the color extensions to the 'ui.color' config.

help: use 'churn' instead of 'color' as an example extension The 'color' extensions is now deprecated.

config: suggest the 'ui.color' instead of the 'color' extension The extensions is deprecated now so we should offer the core way to handle color instead.

color: update main documentation Now that the feature no longer lives in the extension, we document it in the help of the core config. This include the new 'ui.color' option introduced in the previous changesets. As a result the color extensions can now be deprecated. This is a documentation patch only; color is still disabled by default.

pager: drop the 'color' dependant code The 'color' implementation is in core and no longer wrap '_runcommand'. We drop the extra complexity.

color: cleanup 'debugcolor' logic Now that style are carried by the 'ui' object, we no longer need complicated logic to restore the original style. We just need to copy the 'ui' and work on the copied version.

color: move 'debugcolor' into the 'debugcommands' modules This is the last bits we needed to move out of the extensions. 'hgext/color.py' now only contains logic to changes the default color behavior to 'auto'. However, more cleanups are on the way and we need to document the new config directly in core.

merge with stable

worker: flush messages written by child processes before exit I found some child outputs were lost while testing the previous patch. Since os._exit() does nothing special, we need to do that explicitly.

worker: add basic test to ensure child processes are managed well This should catch the bug fixed by "worker: ignore meaningless exit status indication returned by os.waitpid()." Before, worker.py was untested since test repositories are relatively small.

color: rename '_styles' to '_defaultstyles' for clarity This should make it clear the dict is only used for new config. Extensions should not modify it directly anyway since we have 'extraloader' logic for loading '_styles' too.

color: move 'styles' definition on the 'ui' object Same logic as for '_terminfoparams'. The content depends on the config so it should be specific to each 'ui instance.

color: pass 'ui' to 'win32print' Same logic as before,'win32print' relies on some data we will move on the 'ui' object soon, we update the API beforehand for clarity.

color: move the dict with terminfo parameters on the ui object This dictionnary is affected by the content of the config, so we should have one for each ui config. We rename the global dict to '_baseterminfoparams' to make the situation clearer.

color: add ui to effect rendering We'll carry more and more color specific data on the ui object. This will help isolating different color configuration from each other. For example repository config might configure special style that should not affect other ui object. The first step is to make sure the ui object is available were we will needs it.

color: initialize color for the localrepo ui The 'ui' object dedicated to a 'localrepo' is independent from the one available in dispatch (and 'uisetup'). In addition, it is created from the 'baseui' (apparently for good reason). As a result, we need to run the color setup on it after the local repository config is read. This was overlooked when the rest of the initialization changed but did not had impact yet because all setup is still global. We fix it before it is too late.

color: add a 'ui.color' option to control color behavior This new option control whether or not color will be used. It mirror the behavior of '--color'. I usually avoid adding new option to '[ui]' as the section is already filled with many option. However, I feel like 'color' is central enough to deserves a spot in this '[ui]' section. For now the option is not documented so it is still marked as experimental. Once it get documented and official, we should be able to deprecate the color extensions. There is more cleanup to do before that documentation is written, but we need this option early to made them. Having that option will allow for more cleanup of the initialisation process and proper separation between color configuration.

color: reinvent dictionary move the module lever dictionary declaration to a more standard (and practical indentation)