pushkey: run hook after the lock release The pushkey operation used to be in its own wireprotocol command and (in practice) always be lock free when running the hook. With bundle2, it happen in a greater scheme and a hook running locking command would get stuck. We now run such hooks after the lock release as similar hook do. Bundle2 test are altered to ensure we are lockfree at hook running time.

setup: use changes since latest tag instead of just distance For a Mercurial built on the merge from stable into default right after 3.2.2 was released -- 19ebd2f88fc7 -- the version number produced was "3.2.2+4". This is potentially misleading, since in reality the built Mercurial includes many more changes compared to 3.2.2. Change the versioning scheme so that we take into consideration all the changes present in the current revision that aren't present in the latest tag. For 19ebd2f88fc7 the new versioning scheme results in a version number of "3.2.2+256". This gives users a much better idea of how many changes have actually happened since the latest release. Since changessincelatesttag is always greater than or equal to the latesttagdistance, this will produce version numbers that are always greater than or equal to the old scheme. Thus there's minimal compatibility risk.

setup: use changessincelatesttag from archive if present changessincelatesttag gives one a better idea of how much the code has changed since. Since changessincelatesttag is always greater than or equal to the latesttagdistance (see previous patch for why), this will always produce version numbers greater than or equal to the previous scheme.

archive: store number of changes since latest tag as well This is different from latesttagdistance in that while latesttagdistance is defined to be the length of the longest path to the latest tag, changessincelatesttag is the number of changes contained in @ that aren't contained in the latest tag. So, if 't' is the latest tag in the repository below: t | v --o--o----o \ \ ..o..o..@ then latesttagdistance is 2, but changessincelatesttag is 4. Note that changessincelatesttag is always greater than or equal to the latesttagdistance -- that's because changessincelatesttag counts all the changes in the longest path since the latest tag, and possibly others. This is an important fact that we'll take advantage of in upcoming patches.

merge with stable

demandimport: blacklist distutils.msvc9compiler (issue4475) This module depends on _winreg, which is windows-only. Recent versions of setuptools load distutils.msvc9compiler and expect it to ImportError immediately when on non-Windows platforms, so we need to let them do that. This breaks in an especially mystifying way, because setuptools uses vars() on the imported module. We then throw an exception, which vars doesn't pick up on well. For example: In [3]: class wat(object): ...: @property ...: def __dict__(self): ...: assert False ...: In [4]: vars(wat()) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-4-2781ada5ffe6> in <module>() ----> 1 vars(wat()) TypeError: vars() argument must have __dict__ attribute Which is similar to the problem we run into.

largefiles: don't duplicate 'actions' into 'actionbyfile'

merge: make calculateupdates() return file->action dict This simplifies largefiles' overridecalculateupdates(), which no longer has to do the conversion it started doing in 38e55e55ae4d (largefiles: rewrite merge code using dictionary with entry per file, 2014-12-09). To keep this patch small, we'll leave the name 'actionbyfile' in overrides.py. It will be renamed in the next patch.

merge: let _forgetremoved() work on the file->action dict By moving the conversion from the file->action dict after _forgetremoved(), we make that method shorter by removing the need for the confusing 'xactions' variable.

merge: let _resolvetrivial() work on the file->action dict By moving the conversion from the file->action dict after _resolvetrivial(), we greatly simplify and clarify that method.

merge: let bid merge work on the file->action dict By moving the conversion from the file->action dict after the bid merge code, bid merge can be simplified a little. A few tests are affected by this change. Where we used to iterate over the actions first in order of the action type ('g', 'r', etc.) [1], we now iterate in order of filename. This difference affects the order of debug log statements. [1] And then in the non-deterministic order of files in the manifest dictionary (the order returned from manifest.diff()).

merge: write manifestmerge() using dictionary with entry per file In the same vein as 38e55e55ae4d (largefiles: rewrite merge code using dictionary with entry per file, 2014-12-09), rewrite manifestmerge() itself as dictionary with the filename as key. This will let us simplify some of the other code in merge.py and eventually drop the conversion in the largefiles code. No difference in speed could be detected (well within the noise level when run in Mozilla repo).

repoview: backout ced3ecfc2e57 Monkey patching repoview does not really work and making it really work will be really hard. So we better have it broken without complexity than broken with extra complexity.

largefile: explain why no monkey patching on a repoview The comment requested for investigations, here they are.

largefile: backout ca54fb3d71ce The hack for method monkey patching on repoview has been ruled out as fragile, so we are rolling it back. We'll expand the explanation in the next changeset.

incoming: handle phases the same as pull Now that bundlerepo can move phases safely, 'hg incoming' can share its phase handling code with pull to better reflect what would actually show up.

bundlerepo: retract phase boundary This patch makes bundrepo retract the phase boundary for new commits to 'draft' status, which is consistent with the behavior of 'hg unbundle'. The old behavior was for commits to appear with the same phase as their nearest ancestor in the base repository. This affects several classes of operation: * Inspecting a bundle with the -B flag * Treating a bundle file as a peer (old: everything public, new: everything draft) * Incoming command (neither old or new behavior is sensible -- fixed in next patch)

bundlerepo: implement safe phasecache This patch makes bundlerepo use a subclass of phasecache that will allow phase boundaries to be moved around, but will never write them to the underlying repository.

localrepo.__getitem__: add slicing support This allows the python slice syntax to be used with revision numbers when indexing into a repository, such as repo[8:]

ignore: resolve ignore files relative to repo root (issue4473) (BC) Previously these would be considered to be relative to the current working directory. That behavior is both undocumented and doesn't really make sense. There are two reasonable options for how to resolve relative paths: - relative to the repo root - relative to the config file Resolving these files relative to the repo root matches existing behavior with hooks. An earlier discussion about this is available at http://mercurial.markmail.org/thread/tvu7yhzsiywgkjzl. Thanks to Isaac Jurado <diptongo@gmail.com> for the initial patchset that spurred the discussion.

test-hgignore: add testing for ui.ignore I couldn't find any tests for this, and we're going to make changes here in upcoming patches.

tests: add missing globs for Windows I couldn't figure out how to glob the first chunk for Windows, so it's been conditionalized.

share: use the 'sharedpath' attr on repo instead of reloading from the file Seems like a useful optimization, and now the exact file content is not a concern.

share: fix source repo lookup on Windows The stored path contains platform specific separators, so splitting on '/.hg' returned the string unmodified on Windows. The source was then looked for in $source/.hg/.hg, which obviously fails. This caused cascading errors in test-share.t relating to the recent bookmark support.

help: suggest '-v -e' to get built-in aliases for extensions (issue4461) If extension name matches one of command names, suggest user to type 'hg help -v -e <extension>' to get full list of built-in aliases

test-check-commit-hg: clarify misleading "commit message rules" error The test case doesn't only check the commit message, but also the patch, which can result in confusing output like + Revision df6f06d17100 does not comply to commit message rules + ------------------------------------------------------ + 32: adds double empty line + + even when there are no double blank lines in the commit message. Drop the "commit message" part to make it less confusing.

keyword: handle resolve to either parent Merged files are considered modified at commit time even if only 1 parent differs. In this case we must use the change context of this parent for expansion. The issue went unnoticed for long because it is only apparent until the next update to the merge revision - except in test-keyword where it was always staring us in the face.

keyword: update test file syntax

branches: deprecate -a

largefiles: fix a spurious missing file warning with forget (issue4053) If an uncommitted and deleted file was forgotten, a warning would be emitted, even though the operation was successful. See the previous patch for 'remove -A' for the exact circumstances, and details about the cause.

largefiles: fix a spurious missing file warning with 'remove -A' (issue4053) The bug report doesn't mention largefiles, but the given recipe doesn't fail unless the largefiles extension is loaded. The problem only affected normal files, whether or not any largefiles are committed, and only files that have not been committed yet. (Files with an 'a' state are dropped from dirstate, not marked removed.) Further, if the named normal file never existed, the warning would be printed out twice. The problem is that the core implementation of remove() calls repo.status(), which eventually triggers a dirstate.walk(). When the file isn't seen in the filesystem during the walk, the exception handling finds the file in dirstate, so it doesn't complain. However, the largefiles implementation called status() again with all of the original files (including the normal ones, just dropped). This time, the exception handler doesn't find the file in dirstate and does complain. This simply excludes the normal files from the second repo.status() call, which the largefiles extension has no interest is processing anyway.

largefiles: introduce the 'composelargefilematcher()' method This is a copy/paste (with the necessary tweaks) of the composenormalfilematcher method currently on default, which does the inverse- this trims the normal files out of the matcher. It will be used in the next patch.

context: return dirstate parents in workingctx.ancestors() workingctx.ancestors() was not returning the dirstate parents as part of the result set. The only place this function is used is for copy detection when committing a file, and that code already checks the parents manually, so this change has no affect at the moment. I found it while playing around with changing how copy detection works.

backout: add --commit option Mercurial backout command makes a commmit by default only when the backed out revision is the parent of working directory and doesn't commit in any other case. The --commit option changes behaviour of backout to make a commit whenever possible (i.e. there is no unresolved conflicts). This behaviour seems more intuitive to many use (especially git users migrating to hg).

share: add option to share bookmarks This patch adds the -B/--bookmarks option to the share command added by the share extension. All it does for now is create a marker, 'bookmarks.shared', that will be used by future code to implement the sharing functionality.

highlight: ignore Unicode's extra linebreaks (issue4291) Unicode and Python's unicode.splitlines() treat several extra legacy ASCII codepoints as linebreaks, even though the vast bulk of computing and Python's own str.splitlines() do not. Rather than introduce line numbering confusion, we filter them out when highlighting.

test: fix typo in test-help.t

templatekw: remove unneeded showtags Now that we have the machinery of namespaces in-place, we use that instead of hand-rolling our own template function. Note, this can only be used for tags because both branches and bookmarks have special case logic for 'default' and the current bookmark (which is something outside the namespace api for now).

namespaces: generate template keyword when registering a namespace For any namespace, we generate a template keyword. For example, given a namespace 'babar', we automatically have the ability to use it in a template: hg log -r . -T '{babars % "King: {babar}\n"}' Furthermore, we only generate this keyword for a namespace if one doesn't already exist. This is necessary for 'branches' and 'bookmarks' since both of those have concepts of 'current' (something outside the namespace api) and also allows extensions to override default behavior if desired.

templatekw: add helper method to generate a template keyword for a namespace This marks our second feature of the namespace api: automatic template keyword. This patch adds a method that takes in a namespace and uses the node-to-name map to output the list of names.

namespaces: add names method to return list of names for a given node In the previous patch, we added a node-to-name map property. This patch just exposes that interface to the api.

namespaces: add nodemap property This patch adds a node-to-name map property to the namespace. This is necessary because we cannot simply invert the name-to-node map because we do not assume the name-to-node map to be unique (for example, consider named branches: many nodes have one branch name). The node-to-name is helpful in log commands where we are already iterating over a set of nodes and want to display some kind of naming information to the user.

namespaces: add method to get template name of namespace This patch adds the public api for getting the template name of a namespace so that the next patch can use it to generate a template keyword automatically.

namespaces: add template name of a namespace The template name property will be used in upcoming patches to automatically generate a template keyword. For example, given a namespace called 'babars', we will automatically generate a template keyword 'babar' such that we can use it in the following way, $ hg log -r . -T '{babars % "King: {babar}\n"}'

memctx: remove redundant test for issue4470 from test-commit.t Because: - the test to avoid regression for issue4470 was already added to test-commit-amend.t by previous patch It is also a part of test series about manifest calculation issues of memctx in test-commit-amend.t. - this is the only test using "commit --amend" in test-commit.t

memctx: calculate manifest more efficiently Before this patch, "memctx._manifest" updates all entries in the (parent) manifest. But this is inefficiency, because almost all files may be clean in that context. On the other hand, just updating entries for changed "files" specified at construction causes unexpected abortion, when there is at least one newly removed file (see issue4470 for detail). To calculate manifest more efficiently, this patch replaces "pman.iteritems()" for the loop by "self._status.modified" to avoid updating entries for clean or removed files Examination of removal is also omitted, because removed files aren't treated in this loop (= "self[f]" returns not None always).

merge with stable

Added signature for changeset 6dad422ecc5a

Added tag 3.2.3 for changeset 6dad422ecc5a

pathauditor: check for Windows shortname aliases

pathauditor: check for codepoints ignored on OS X

darwin: omit ignorable codepoints when normcase()ing a file path This lets us avoid some nasty case collision problems in OS X with invisible codepoints.

encoding: add hfsignoreclean to clean out HFS-ignored characters According to Apple Technote 1150 (unavailable from Apple as far as I can tell, but archived in several places online), HFS+ ignores sixteen specific unicode runes when doing path normalization. We need to handle those cases, so this function lets us efficiently strip the offending characters from a UTF-8 encoded string (which is the only way it seems to matter on OS X.)

test-casefolding.t: demonstrate a bug with HFS+ ignoring some codepoints

manifest: disallow setting the node id of an entry to None manifest.diff() uses None as a special value to denote the absence of a file, so setting a file node to None means you then can't trust manifest.diff(). This should also make future manifest work slightly easier.

context: stop setting None for modified or added nodes Instead use a magic value, so that we can identify modified or added nodes correctly when using manifest.diff(). Thanks to Martin von Zweigbergk for catching that we have to update _buildstatus as well. That part eluded my debugging for some time.

largefiles: don't actually remove largefiles in an addremove dry run The addlargefiles() method already properly handled dry runs.

bundle2: lowercase part types Since the capitalization no longer carries any meaning (previous diff), this patch normalizes all of the bundle2 part type strings to lower case.

bundle2.bundlepart: make mandatory part flag explicit in API This makes all bundle2 parts mandatory unless they are expressly made advisory via the keyword parameter or the bundlepart.mandatory property.

memctx: calculate manifest correctly with newly-removed files (issue4470) Before this patch, "memctx._manifest" tries to get (and use normally) filectx also for newly-removed files, even though "memctx.filectx()" returns None for such files. To calculate manifest correctly even with newly-removed files, this patch does: - replace "man.iteritems()" for the loop by "self._status.modified" to avoid accessing itself to newly removed files this also reduces loop cost for large manifest. - remove files in "self._status.removed" from the manifest In this patch, amending is confirmed twice to examine both (1) newly removed files and (2) ones already removed in amended revision.

memctx: calculate manifest including newly added files correctly Before this patch, "memctx._manifest" calculates the manifest according to the 1st parent. This causes the disappearance of newly added files from the manifest. For example, if newly added files aren't listed up in manifest of memctx, they aren't listed up in "added" field of "status" returned by "ctx.status()", and "{diff()}" (= "patch.diff") in "committemplate" shows nothing for them. To calculate manifest including newly added files correctly, this patch puts newly added files (= ones in "self._status.added") into the manifest. Some details of changes for "test-commit-amend.t" in this patch: - "touch foo" is replaced by "echo foo > foo", because newly added empty file can't be shown in "diff()" output without "diff.git" configuration - amending is confirmed twice to examine both (1) newly added files and (2) ones already added in amended revision

memctx: calculate exact status being committed from specified files Before this patch, "memctx._status" is initialized by "(files, [], [], [], [], [], [])" and this causes "memctx.modified" to include not only modified files but also added and removed ones incorrectly. This patch adds "_status" method to calculate exact status being committed according to "files" specified at construction time. Exact "_status" is useful to share/reuse logic of committablectx. This patch is also preparation for issues fixed by subsequent patches. Some details of changes for tests in this patch: - some filename lines are omitted in "test-convert-svn-encoding.t", because they are correctly listed up as "removed" files those lines are written out in "localrepository.commitctx" for "modified" and "added" files by "ui.note". - "| fixbundle" filterring in "test-histedit-fold.t" is omitted to check lines including "added" correctly "fixbundle" discards all lines including "added".

bundle2._processpart: forcing lower-case compare is no longer necessary Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. Since the parttype and mandatory bit are separated in bundle2.unbundlepart (see previous patch), there is no longer a need to remove the mandatory bit before working with the parttype.

bundle2.unbundlepart: decouple mandatory from parttype Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. This patch separates the two pieces of information when reading the part header so that it's unnecessary to know how they were combined during transmission.

memctx: fix manifest for removed files (issue4470) filectxfn returns None for removed files, so we have to check for None before computing the new file content hash for the manifest. Includes a test that proves this works, by demonstrating that we can show the diff of an amended commit in the committemplate.

convert: on svn failure, note libsvn version (issue4043) We have our own fast-path logic to see if something passes a sniff test for being a Subversion repository, but it's possible for a user to svnsync a repo using svn 1.8 and then use svn 1.7 bindings (as in the bug) to try and convert the repo. If we at least tell the user the version of libsvn that we used, they might get enough of a hint to check on their own for format incompatibilities between their svn{admin,sync} and the libsvn used by hg.

vfs: add a 'split' method This method has the same behavior as the 'os.path.split' function, but having it in vfs will allow handling of tricky encoding situations in the future. In the same patch, we replace the use of 'os.path.split' in the transaction code.

vfs: add a 'reljoin' function for joining relative paths The vfs.join method only works for absolute paths. We need something that works for relative paths too when transforming filenames. Since os.path.join may misbehave in tricky encoding situations, encapsulate the new join method in our vfs abstraction. The default implementation remains os.path.join, but this opens the door to other VFSes doing something more intelligent based on their needs. In the same go, we replace the usage of 'os.path.join' in transaction code.

subrepo: fix git subrepo ui argument

subrepo: drop the 'ui' parameter to revert() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in revert is to emit status and warning messages, and to check the verbose flag prior to printing the action to be performed on a file. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call. Unlike other methods where the matcher is passed along and narrowed, a new matcher is created in each repo, and therefore the bad() method already used the local repo's ui.

subrepo: drop the 'ui' parameter to removefiles() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in remove is to emit status and warning messages, and to check the verbose flag prior to printing files to be removed. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call.

subrepo: drop the 'ui' parameter to forget() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in forget is to emit status and warning messages, and to check the verbose flag prior to printing files to be forgotten. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call.

subrepo: drop the 'ui' parameter to cat() This no longer needs to be explicitly passed because the subrepo object tracks a 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in cat is to emit a status message when a subrepo is missing. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object.

subrepo: drop the 'ui' parameter to archive() The current state of subrepo methods is to pass a 'ui' object to some methods, which has the effect of overriding the subrepo configuration since it is the root repo's 'ui' that is passed along as deep as there are subrepos. Other subrepo method are *not* passed the root 'ui', and instead delegate to their repo object's 'ui'. Even in the former case where the root 'ui' is available, some methods are inconsistent in their use of both the root 'ui' and the local repo's 'ui'. (Consider hg._incoming() uses the root 'ui' for path expansion and some status messages, but also calls bundlerepo.getremotechanges(), which eventually calls discovery.findcommonincoming(), which calls setdiscovery.findcommonheads(), which calls status() on the local repo 'ui'.) This inconsistency with respect to the configured output level is probably always hidden, because --verbose, --debug and --quiet, along with their 'ui.xxx' equivalents in the global and user level hgrc files are propagated from the parent repo to the subrepo via 'baseui'. The 'ui.xxx' settings in the parent repo hgrc file are not propagated, but that seems like an unusual thing to set on a per repo config file. Any 'ui.xxx' options changed by --config are also not propagated, because they are set on repo.ui by dispatch.py, not repo.baseui. The goal here is to cleanup the subrepo methods by dropping the 'ui' parameter, which in turn prevents mixing subtly different 'ui' instances on a given subrepo level. Some methods use more than just the output level settings in 'ui' (add for example ends up calling scmutil.checkportabilityalert() with both the root and local repo's 'ui' at different points). This series just goes for the low hanging fruit and switches methods that only use the output level. If we really care about not letting a subrepo config override the root repo's output level, we can propagate the verbose, debug and quiet settings to the subrepo in the same way 'ui.commitsubrepos' is in hgsubrepo.__init__. Archive only uses the 'ui' object to call its progress() method, and gitsubrepo calls status().

subrepo: use 'self.ui' instead of 'self._repo.ui' They are now equivalent, but the former is slightly more readable.

subrepo: reset 'self.ui' to the subrepo copy of 'ui' in the hgsubrepo class Creation of the subrepo's '_repo' object creates a new 'ui' by combining the parent repo's 'baseui' and reading in the subrepo's hgrc file. This simply avoids 'self.ui' and 'self._repo.ui' pointing to different objects, which seems like a potential source of bugs. Git and Svn subrepos are unchanged, because they don't have their own ui, and have always used their parent's for their configuration.

subrepo: rename the '_ui' member to 'ui' The localrepository class has a 'ui' member, so keeping the names the same will allow for duck typing with subrepo instances when accessing 'ui'. Changing this is easier than finding all of the localrepository instance uses and renaming that to '_ui'.

subrepo: drop the '_ui' member in the subrepo subclasses This member has existed in the base class since fcbc66b5da6a.

graphmod: attempt to clarify documentation of groupbranchiter() Thanks to Pierre-Yves for checking my cleanups here and helping me understand the algorithm well enough to help document it.

graphmod: rename graph-topological config to graph-group-branches The latter suggests what the change is slightly better.

groupbranchiter: allow callers to select the first branch Instead of just bootstrapping the algorithm with the first revision we see, allow callers to pass revs that should be displayed first. All branches are retained until we can display such revision. Expected usage is to display the current working copy parent first.

groupbranchiter: support for non-contiguous revsets The algorithm now works when some revisions are skipped. We now use "first included ancestors" instead of just "parent" to link changesets with each other.

groubranchhiter: indent most of the inner code We are going to add an additional layer of indentation to support non-contiguous revset. We do it in a pure code movement changeset to help the readability of the next changeset.

graphlog: add a way to test the 'groupbranchiter' function We add an experimental config option to use the topological sorting. I first tried to hook the 'groupbranchiter' function in the 'sort' revset but this was useless because graphlog enforces revision number sorting :( As the goal is to advance on the topological iteration logic, I see this experimental option as a good way to move forward. We have to use turn the iterator into a list because the graphlog is apparently not ready for pure iterator input yet.

graphmod: add a function for topological iteration This changeset introduces a function to perform topological (one branch after the other) iteration over a set of changesets. This first version has a lot of limitations, but the approach should be flexible enough to allow many improvements in the future. This changeset aims to set the first stone more than providing a complete solution. The algorithm does not need to know the whole set of nodes involved before emitting revision. This makes it a good candidate for usage in place like `hg log` or graphical tools that need a fast first result time.

namespaces: add branches Note that the exception-catching from the previous branchtip check is moved up to catch exceptions from the try block surrounding the namespace lookup.

namespaces: add tags

namespaces: remove weakref; always pass in repo It turns out that maintaining a reference of any sort (even weak!) to the repo when constructed doesn't work because we may at some point pass in a repoview filtered by something other than what the initial repo was.

changectx: use names api to simplify and extend node lookup Previously, changectx had to know about each type of name (bookmark, tag, etc.) to look up. Now, we use repo.namenodes to simplify (and extend) this.

namespaces: add a method to the first matching node for a given name

namespaces: add bookmarks to the names data structure This marks the first use of abstracting our different types of named objects (bookmarks, tags, branches, etc.) and upcoming patches will use this to simplify logic.

namespaces: add a convenience property for the weakref _repo

namespaces: pass repo to __init__ We store repo as a weakref so that we can prevent a cycle between namespaces and localrepo.

namespaces: add a function to turn single results into lists

namespaces: add a method to register new namespaces

namespaces: introduce a generic way to map between names and nodes This patch begins the work to provide a way to register a namespace to handle 'names'. Benefits of this would be, - improved templating: This would provide {name} which could output any branch, bookmark, tag, or any extension registered namespace all without having the extension doing any extra work - improved tab completion: Since this provides a single source of all 'names', tab completion would not need to know of each namespace - changeset lookup: Similar to before, a unified place to get all 'names' will allow finding changesets without any extension code having to reimplement this Also, d226fe36e362 has shown us that for internal code which expects a certain type of method or behavior, we should provide an easy way for extensions to check this behavior.

statichttprepo: update profile of __call__ in mock vfs object New code paths could fail because the old statichttprepo profile couldn't handle the usual parameters. Instead, reuse a more generic profile also used in readonlyvfs.

tests: avoid git diffstat singular/plural output difference 'git diff --stat' output changed with regard to the amount of changes/insertions/deletions shown. In older git versions (1.7.7.6), output was shown as: 2 files changed, 2 insertions(+), 1 deletions(-) In newer versions, output is shown as: 2 files changed, 2 insertions(+), 1 deletion(-) This patch uses a regex to handle both cases.

subrepo: add revert support without backup for git subrepos Previously, git subrepos did not support reverting. This change adds basic support for reverting when '--no-backup' is specified. A warning is given (and the current state is kept) when a revert is done without the '--no-backup' flag.

match: make 'always' and 'exact' functions, not classes There is no reason for classes 'always' and 'exact' not to be just functions that return instances the plain 'match' class.

share: implement shared bookmark functionality This does not cause any behavioral change unless a 'bookmarks.shared' marker file exists. A future change will add UI to create this file when a repository is shared.

debugindex: respect --debug flag to show full nodeids

localrepo.clone: add a way to override server preferuncompressed Without this patch, if the server sets preferuncompressed, there's no way for clients to override that and force a non-streaming clone. With this patch, we extend the meaning of --pull to also override preferuncompressed and force a non-streaming clone.

hg.clone: set 'stream' depending on whether --pull was requested or not In an upcoming patch we'll differentiate between the two in localrepo.

merge: perform case-collision checking on final set of actions When there are multiple common ancestors, we should check for case collisions only on the resulting actions after bid merge has run. To do this, move the code until after bid merge. Move it past _resolvetrivial() too, since that might update actions. If the remote changed a file and then reverted the change, while the local side deleted the file and created a new file with a name that case-folds like the old file, we should fail before this patch but not after. Although the changes to the actions caused by _forgetremoved() should have no effect on case collisions, move it after that, too, so the next person reading the code won't have to think about it. Moving it past these blocks of code takes it to the end of calculateupdates(), so let's even move it outside of the method, so we also check collisions in actions produced by extensions overriding the method.

spelling: fixes from proofreading of spell checker issues

tests: allow more flexibility in git diffstat output The output of 'git diff --stat' changed in git 1.7.10 and 1.7.11. To ensure the tests work with earlier versions of git as well, the output is now wrapped with a whitespace regex.

merge: move cd/dc prompts after largefiles prompts By moving the cd/dc prompts out of calculateupdates(), we let largefiles' overridecalculateupdates() so the unresolved values (i.e. 'cd' or 'dc' rather than 'g', 'r', 'a' and missing). This allows overridecalculateupdates() to ask the user whether to keep the normal file or the largefile before the user gets the cd/dc prompt. Whichever answer the user gives, we make overridecalculateupdates() replace 'cd' or 'dc' action, saving the user one annoying (and less clear) question.

addremove: automatically process a subrepository's subrepos Since addremove on the top of a directory tree will recursively handle sub directories, it should be the same with deep subrepos, once the user has explicitly asked to process a subrepo. This really only has an effect when a path that is a subrepo (or is in a subrepo) is given, since -S causes all subrepos to be processed already. An addremove without a path that crosses into a subrepo, will still not enter any subrepos, per backward compatibility rules.

addremove: support addremove with explicit paths in subrepos Git and svn subrepos are currently not supported.

addremove: add support for the -S flag Git and svn subrepos are currently not supported. It doesn't look like git or svn have these commands natively, so that's an area for a git or svn expert.

commit: propagate --addremove to subrepos if -S is specified (issue3759) The recursive addremove operation occurs completely before the first subrepo is committed. Only hg subrepos support the addremove operation at the moment- svn and git subrepos will warn and abort the commit.

subrepo: store the ui object in the base class This will be used in the next patch to print a warning from the base class. It seems better than having to explicitly pass it to a new method, since a lot of existing methods also require it.

commit: abort if --addremove is specified, but fails This will be required when subrepo support is added, in order to ensure consistent commits when a subrepo flavor doesn't support addremove.

addremove: warn when addremove fails to operate on a named path It looks like a bad path is the only mode of failure for addremove. This warning is probably useful for the standalone command, but more important for 'commit -A'. That command doesn't currently abort if the addremove fails, but it will be made to do so prior to adding subrepo support, since not all subrepos will support addremove. We could just abort here, but it looks like addremove has always silently ignored bad paths, except for the exit code.

scmutil: pass a matcher to scmutil.addremove() instead of a list of patterns This will make it easier to support subrepository operations.

tests: fix a typo in test-walkrepos.py

merge: extract _resolvetrivial() function We would eventually like to move the resolution of modify/delete and delete/modify conflicts to the resolve phase. However, we don't want to move the checks for identical content that were added in 902554884335 (merge: before cd/dc prompt, check that changed side really changed, 2014-12-01). Let's instead move these out to a new _resolvetrivial() function that processes the actions from manifestmerge() and replaces any false cd/dc conflicts. The function will also provide a natural place for us to later add code for resolving false 'm' conflicts.

largefiles: start by finding files of interest Instead of iterating over 'g' action, first find the set of all files that are largefiles in p1. Then iterate over these files. This prepares for considering actions other than 'g'.

largefiles: rewrite merge code using dictionary with entry per file In overridecalculateupdates(), we currently only deal with conflicts that result in a 'g' action for either the largefile or a standin. We will soon want to deal cases with 'cd' and 'dc' actions here. It will be easier to reason about such cases if we rewrite it using a dict from filename to action. A side-effect of this change is that the output can only have one action per file (which should be a good change). Before this change, when one of the tests in test-issue3084 received this input (the 'a' in the input was a result of 'cd' conflict resolved in favor of the modified file): 'g': [('.hglf/f', ('',), 'remote created')], 'a': [('f', None, 'prompt keep')], and the user chose to keep the local largefile, it produced this output: 'g': [('.hglf/f', ('',), 'remote created')], 'r': [('f', None, 'replaced by standin')], 'a': [('f', None, 'prompt keep')], Although 'a' actions are processed after 'r' actions by recordupdates(), it still worked because 'a' actions have no effect on merges (only on updates). After this change, the output is: 'g': [('.hglf/f', ('',), 'remote created')], 'r': [('f', None, 'replaced by standin')], Similarly, there are several tests in test-largefiles-update that get inputs like: 'a': [('.hglf/large2', None, 'prompt keep')], 'g': [('large2', ('',), 'remote created')], and when the user chooses to keep the local largefile, they produce this output: 'a': [('.hglf/large2', None, 'prompt keep'), ('.hglf/large2', None, 'keep standin')], 'lfmr': [('large2', None, 'forget non-standin largefile')], In this case, it was not a merge but an update, so the 'a' action does have an effect. However, since dirstate.add() is idempotent, it still has no obserable effect. After this change, the output is: 'a': [('.hglf/large2', None, 'keep standin')], 'lfmr': [('large2', None, 'forget non-standin largefile')],

largefiles: put same 'action' object back in 'newglist' The items we put in 'newglist' are always the same as what we found in actions['g'], so let's just put the same item into the list instead of creating a new one.

largefiles: don't unnecessarily sort merge action lists The action lists returned from calculateupdates() (in merge.py) are not required to be sorted. In fact, since they result from iteration over the unordered manifest, they are unlikely to be sorted. Moreover, some of the lists are appended to after they are returned from manifestmerge(). The lists are instead sorted in applyupdates(). Therefore, let's not sort the lists generated in largefiles' overridecalculateupdates().

merge: don't treat 'diverge' and 'renamedelete' like actions See earlier patch for motivation.

merge: move dr/rd warning messages out of applyupdates() As preparation for making 'dr' and 'rd' actions no longer actions, move the reporting from applyupdates() to its caller update(). This way we won't have to pass additonal arguments to applyupdates() when they are no longer actions. Also, the warnings are equally unrelated to applyupdates() as they are to recordupdates(), as they don't result in any changes to either the working copy or the dirstate. See earlier patch for additional motivation.

merge: don't report progress for dr/rd actions It is easier to reason about certain algorithms in terms of a file->action mapping than the current action->list-of-files. Bid merge is already written this way (but with a list of actions per file), and largefiles' overridecalculateupdates() will also benefit. However, that requires us to have at most one action per file. That requirement is currently violated by 'dr' (divergent rename) and 'rd' (rename and delete) actions, which can exist for the same file as some other action. These actions are only used for displaying warnings to the user; they don't change anything in the working copy or the dirstate. In this way, they are similar to the 'k' (keep) action. However, they are even less action-like than 'k' is: 'k' at least describes what to do with the file ("do nothing"), while 'dr' and 'rd' or only annotations for files for which there may exist other, "real" actions. As a first step towards separating these acitons out, stop including them in the progress output, just like we already exclude the 'k' action.

subrepo: add partial diff support for git subrepos So far, git subrepositories were silently ignored for diffs. This patch adds support for git subrepositories, with the remark that --include and --exclude are not supported. If --include or --exclude are used, the subrepo is ignored.

subrepo: extend git version check to 3 digits This allows more flexibility when a version check is required. Some git features are introduced in a version where only the 3rd digit changes.

subrepo: move git version check into a separate method This allows checking the git version in other methods, instead of only being able to check if the version is ok or not.

rebase: show a note for updated mq patches It deserves more than a debug message. Show a note like: updating mq patch p0.patch to 5:9ecc820b1737 The message could also refer to "qrefresh" instead. Same same.

rebase: improve debug messages while processing the list of rebases

rebase: show warning when rebase creates no changes to commit Similar to graft: note: rebase of 6:eea13746799a created no changes to commit

rebase: show more useful status information while rebasing Show status messages while rebasing, similar to what graft do: rebasing 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: make 'saved backup' globbing less narrow in rebase tests Globbing the hash made it harder to maintain tests with run-tests -i when it was so far by the generated test output. The hashes are stable and we just need to add a (glob).

tests: stabilise mq rebase tests Prepare for including hashes in output ... and less globbing make the tests easier to update.

test-graft: use strip extension instead of mq extension It only needs strip, no reason to load all of mq.

transaction: remove the 'onabort' mechanism It has no known users. If someones needs similar functionality, a new 'addabort' method similar to 'addfinalize' should be added.

transaction: remove the redundant 'onclose' mechanism It is superseded by the 'addfinalize' function and all its user have been migrated.

fncache: document the fact fncache is outdate at hook run time Using 'addfinalize' to generate 'fncache' means that no pending version of the file will be generated for the hooks. We would have to use the 'addfilegenerator' method to get such result. However the 'fncachevfs' (who decide that a write is necessary) have no access to the transaction to register such file generation at add time. Having the transaction accessible to the 'vfs' is too much trouble for no benefit. This outdated 'fncache' file at hook time is not expected to be an issue. The previous move from 'onclose' to 'addfinalize' had no impact on this timing. I'm documenting it now because I looked at it.

fncache: drop dedicated 'onclose' function in favor of 'tr.addfinalize' Now that we have a shiny generic mechanism, we can use it.

merge with stable

graft: drop cset description from empty commit message This is either already redundant in the output or too verbose in quiet mode.

graft: show hashes in user-facing messages Graft was in various places just showing the revision number in status messges. Instead, also show the stable and more useful short hash.

graft: give helpful warning for empty grafts It was just showing a status message with the internal revision number. Instead, show a warning like note: graft of 27:3aaa8b6725f0 "28" created no changes to commit (message tweaked in-flight by mpm)

graft: show more useful status information while grafting Show status messages with first line of commit description and names, like grafting 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: test coverage for empty graft A future change will add a warning to the quiet rebase.

largefiles: drop the unfiltered repo usage in overridepurge() Now that repoview supports replacing methods, we don't need this hack.

repoview: allow methods on the proxy class to be replaced It doesn't seem to be a common idiom for repo instances, but the status() method is replaced in largefiles' purge() override. Since __setattr__ is implemented in repoview to setattr() on the unfiltered repo, the replacement method wouldn't get called unless it was invoked with the unfiltered repo, because the filtered repo remains unchanged. Since this doesn't seem to be commonly used, I didn't bother to filter out methods that perhaps shouldn't be replaced, such as changelog().

log: fix log revset instability The log/graphlog revset was not producing stable results since it was iterating over a dict. Now we sort before iterating to guarantee a fixed order. This fixes some potential flakiness in the tests.

log: fix log -f slow path to actually follow history The revset created when -f was used with a slow path (for patterns and directories) did not actually contain any logic to enforce follow. Instead it was depending on the passed in subset to already be limited (which was limited to :. but not ::.). This fixes it by adding a '& ::.' to any -f log revset. hg log -f <file> is still broken, in that it can return results that aren't actually ancestors of the current file, but fixing that has major perf implications, so we'll deal with it later.

obsstore: cache size computation for fm1 node We have two different types of node type (sha1 and sha256, only sha1 is used now) and therefor different sizes for them. We now compute the value once instead of redoing the computation every loop. This has no visible performance impact.

obsstore: prefetch struct.calcsize This function is widely used and worth but be at module level. No specific performance boost is visible, but this is more consistent.

obsstore: disable garbage collection during initialization (issue4456) Python garbage collection is triggered by container creation. So code that creates a lot of tuples tends to trigger GC a lot. We disable the gc during obsolescence marker parsing and associated initialization. This provides an interesting speedup (25%). Load marker function on my 58758 markers repo: before: 0.468247 seconds after: 0.344362 seconds The benefit is a bit less visible overall. With python2.6 on my system I see: after: 0.60 before: 0.53 The difference is probably explained by the delaying of a costly GC. (but there is still a win). Marking involved tuples, lists and dicts as ignorable by the garbage collector should give us more benefit. But this is another adventure. Thanks goes to Siddharth Agarwal for the lead.

dirstate: use the 'nogc' decorator Now that we have a generic way to disable the gc, we use it. however, we have too use it in a baroque way. See inline comment for details.

util: add a 'nogc' decorator to disable the garbage collection Garbage collection behave pathologically when creating a lot of containers. As we do that more than once it become sensible to have a decorator for it. See inline documentation for details.

generate-working-copy-states: open() in binary mode when writing content This avoids changes to the hashes on Windows in test-merge-force.t like so: @@ -594,12 +594,12 @@ content2 M missing_content2_content3_content4-tracked - <<<<<<< local: 443153eb5b88 - test: local\r (esc) - content4 - ||||||| base - ======= - content2 - >>>>>>> other: 9b2ccd328a08 - test: remote\r (esc) + <<<<<<< local: 0447570f1af6 - test: local + content4 + ||||||| base + ======= + content2 + >>>>>>> other: 85100b8c675b - test: remote missing_content2_content3_content4-tracked.orig: content4

largefiles: don't use 'r' action for standin that doesn't exist When merging and the remote has turned a normal file into a largefile and the user chooses to keep the local largefile, we use the 'r' action for the remote largefile standin. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

largefiles: don't use 'r' action for normal file that doesn't exist When merging and the remote has turned a largefile into a normal file and the user chooses to keep the local largefile, we use the 'r' action for the remote normal file. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

rebase: handle revtodo as a special value when storing/restoring state Revtodo happens to share its value with nullrev, but this is an implementation details, so we move away from it. After this changeset one can successfully change the values for all the constants and the tests still pass, but doing so would require more refactoring if we want to avoid breaking backward compatibility on the state file.

rebase: add a 'revtodo' constant The state mapping is using '-1' to mark revisions that have not been rebased yet. We introduce and use a constant for that purpose. This will help emphasize the fact the value means something other than nullrev.

rebase: use '>= 0' to know is a revision was rebased The fact that the state for "not yet rebased" is -1 is an implementation details. So we change the comparisons to some semantically more correct.

patchbomb: introduce a 'patchbomb.confirm' option When set to true, this option will make patchbomb always ask for confirmation before sending the email. Confirmation is a powerful way to prevent stupid mistakes when the sending patches. This should let me get rid of my global alias adding --confirm to hg email. I know that some people may get bitten when moving from a machine with confirm configured to a machine where it is not, but I think it is worth the risk.

patchbomb: add a 'patchbomb.intro' option This option allows the user to control the default behavior for including an introduction message. This avoids having to tirelessly skip the intro for people contributing to Mercurial. The three possibles values are: - always, - auto (default, current behavior), - never. I was thinking of ("true", "false", "") (empty value being auto) but I ruled it out as too confusing. This new config option reuses the pre-existing 'patchbomb' section.

patchbomb: extract 'getoutgoing' closure into its own function This continues my crusade against closure complication.

perf: add a perfloadmarkers command It is used to benchmark the obsstore initialization time.

rebase: move base calculation from rebasenode() to defineparents() We want to collect all calculation in one place.

largefiles: remove redundant checks for false modify/delete conflicts In a34a99181f36 (largefiles: don't show largefile/normal prompts if one side is unchanged, 2014-12-01), overridecalculateupdates() started checking for false modify/delete conflicts in large files and their standins. Then, in the very next changeset, 902554884335 (merge: before cd/dc prompt, check that changed side really changed, 2014-12-01), calculateupdates() itself started checking for false modify/delete conflicts in all files. Since "large files and their standins" is a subset of "all files", we can now drop the checks in overridecalculateupdates().

merge: make 'keep' message more descriptive Most merge action messages don't describe the action itself, they describe the reason the action was taken. The only exeption is the 'k' action, for which the message is just "keep" and instead there is a code comment folling it that says "remote unchanged". Let's move that comment into the merge action message.

addremove: restore the relative path printing when files are named This fixes the previously mentioned issue with 3778884197f0, and undoes its corresponding test change. The test change demonstrates the correctness when a file is specified (i.e. the glob is required on Windows because relative paths use '\' and absolute paths use '/'). It is admittedly very subtle, but there will be a more robust test in the addremove -S v3 series.

match: introduce uipath() to properly style a file path Several methods print files relative to the repo root, unless files are named on the command line, in which case they are printed relative to cwd. Since the check relies on the 'pats' parameter, which needs to be replaced by a matcher when adding subrepo support, this logic gets folded into the matcher to tidy up the callers. Prior to 3778884197f0, this style decision was based off of whether or not the 'pats' list was empty. That change altered the check to test match.anypats() instead, in order to make paths printed consistent when -I/-X is specified. That however, changed the style when a file is given to the command. So now we test the pattern list to get the old behavior for files, as well as test -I/-X to get the consistency for patterns.

merge with stable

update: don't overwrite untracked ignored files on update When looking for untracked files that would conflict with a tracked file in the target revision (or the remote side of a merge), we explcitly exclude ignored files. The code was added in 7e30f5f2285f (merge: refactor unknown file conflict checking, 2012-02-09), but it seems like only unknown, not ignored, files were considered since the beginning of time. Although ignored files are mostly build outputs and backup files, we should still not overwrite them. Fix by simply removing the explicit check.

update: add tests for untracked local file We don't seem to have any tests for updating to another revision when there are untracked files on the local side that conflict with the those on the remote side, so let's add tests. This shows how we overwrite untracked ignored files when updating to a revision that tracks the file.

merge: don't overwrite conflicting file in locally renamed directory When the local side has renamed a directory from a/ to b/ and added a file b/c in it, and the remote side has added a file a/c, we end up overwriting the local file b/c with the contents of remote file a/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename get ('dg') action.

merge: don't ignore conflicting file in remote renamed directory When the remote side has renamed a directory from a/ to b/ and added a file b/c in it, and the local side has added a file a/c, we end up moving a/c to b/c without considering the remote version of b/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename ('dm') action.

merge: duplicate 'if f in copied' into each branch

merge: branch code into {n1 and n2, n1, n2} top-level cases There are three high-level cases that are of interest in manifestmerge(): 1) The file exists on both sides, 2) The file exists only on the local side, and 3) The file exists only on the remote side. Let's make this clearer in the code. The 'if f in copied' case will be broken up into the two applicable branches in the next patch.

update: remove redundant and misplaced merge tests The "nothing to merge" case is covered by test-merge-default.t. The "uncommitted changes" case is covered by test-merge1.t (and others). The "merge -f" case is covered by test-merge-force.t.

largefiles: use clearer debug messages in actions In overridecalculateupdates(), 'g' (get) actions may be converted into other actions. In most of these cases, it does not make sense to keep the action's message. For example, 'remote created' does not make sense for an 'r' (remove) action.

largefiles: don't clobber merge action message with user message The message in the action is used for debugging and should not be the same as the question presented to the user. Use a different variable for the user message, so the 'msg' variable already in scope does not get overwritten.

bookmarks: factor out repository lookup from writing bookmarks file This will allow the share extension to extend bookmarks functionality to share bookmarks between repositories.

parsers: ensure revlog index node tree is initialized before insertion Currently, the revlog index C implementation assumes its node tree will be initialized before a new element is inserted by revnum. For example, revlog.py executes 'self.index.insert(-1, e)' in _addrevision(). This is only safe because the node tree has been initialized by a "node in self.nodemap" check made in addrevision(). (For context, this was discovered while developing an experimental revlog mixin which stores "elided nodes" via a separate code path from _addrevision(); that new code path segfaults without this patch.)

revset: add 'only' to DoS-safe symbols There is no reason to disable it in hgweb because the same query can be written without using 'only()'.

revset: move 'only' so that functions are sorted alphabetically

tests: drop unnecessary glob that causes a warning on Windows This was triggered by 3778884197f0.

merge: add test with conflicting file and remote directory rename Changset 88629daa727b (merge: demonstrate that directory renames can lose local file content, 2014-12-02) should clearly have added the reverse version of the test: where the remote side renamed a directory, added a new file in that directory, and the local directory added a conflicting file in the source directory. Add such a test now, and also touch up the ones already added slightly (e.g. 'local' was a stupid value for content that can be on either side of a merge).

graft: use a real conflict for the tests One of the graft tests grafts a changeset that changes a file's content from 'a' to 'b' onto a branch that has changed the file's content from 'a', via 'b', and then back to 'a' again. To prepare for not considering this a file in need of merging, let's use 'c' as the file's new content to make sure it has to be considered conflicting. There's a second similar case further down where an ancestor is grafted. Make sure that is also considered a conflict.

add: use lexists so that broken symbolic links are added This restores the add behavior prior to d8cdd46f426d and matches the behavior of addremove.

rebase: avoid redundant repo[rev].rev() - just keep working in rev space

rebase: always store rev in state, also when collapsing

rebase: clarify naming of variables holding node hashes - don't call them rev

bookmarks: factor out bookmark file opening for easier extensibility

merge with stable

diff: explicitly honor all diffopts 'hg diff' should naturally honor all diffopts.

annotate: explicitly only honor whitespace diffopts The whitespace ones are the only ones the annotate logic cares about anyway, so there's no visible impact.

notify: explicitly honor all diffopts The notify output doesn't seem to be parseable anyway, what with the maxdiff config option. Plus it is designed mainly for servers where hopefully the admins are doing sensible things.

jsonchangeset: don't honor whitespace and format-changing diffopts JSON is meant to be parsed by computers, and format changes can break them.

transplant: don't honor whitespace and format-changing diffopts transplant uses diff purely internally.

hgk: don't honor whitespace and format-changing diffopts The diff-tree command is a plumbing command in Git, which means it is expected to have a stable output format. Ensure that our output is stable too.

patchbomb: don't honor whitespace and format-changing diffopts (BC) The whitespace diffopts break lossless transmission, and the format-changing ones make import harder. We expect parsers to be able to read git-style diffs, though.

merge: add more thorough tests for --force With generate-working-copy-states.py generalized to support arbitrarily many changesets, we can use it for generating test cases for merge: use one changeset each for base, remote and local. With the various working copy states, this is a total of 104 cases. The first candidate for additional testing is 'hg merge --force'. Even though the force option is deprecated, it is convenient for testing because it can be tested without first needing to revert any changes. Except for the lack of checking for uncommitted changes, it differs in only a few cases from unforced merge. The new tests cover all the cases in the existing test-merge-force.t, except for the unforced merge case, which is covered in several other files anyway, so nothing remains of the file after this patch.

merge: display modify/delete conflict prompts in sorted order The order is determined by manifest.diff(), which currently is not sorted. There are currently no tests for this, but we will soon add some that would be flaky without this patch.

generate-working-copy-states: accept depth arguments on command line Add a parameter to generate-working-copy-states.py that indicates how many changesets are wanted. This number impacts all the subcommands. The current 'filelist' subcommand becomes 'filelist 2', the current 'base' and 'parent' subcommands become 'state 2 1' and 'state 2 2' respectively, while 'wc' becomes 'state 2 wc'. See earlier patch for motivation.

generate-working-copy-states: generalize for depth The script can currently generate filenames and contents for exactly two changesets plus the working copy. For some tests (e.g. of plain dirstate status), only one changeset is needed, while for others (e.g. merge), three changesets are needed. Let's prepare for such tests by generalizing the code for any number of changesets.

generate-working-copy-states: make 'tracked' a string The only places the 'tracked' variable is only used is where it's converted to a string, so let's simplify by using the string value directly.

merge: demonstrate that directory renames can lose local file content When a directory has been renamed on the local branch and a file has been added in the old location on a remote branch, we move that new file to the new location. Unfortunately, if there is already a file there, we overwrite it with the contents from the remote branch. For untracked local files, we should probably abort, and for tracked local files, we should merge the contents. To start with, let's add a test to demonstrate the breakage. Also note that while files merged in from a remote branch are normally (and unintuitively) reported as modified, these files are reported as added.

mq: drop updateheader - inserthgheader and insertplainheader is enough

mq: smarter handling of plain headers 6333412245ec and 5ccced6eab0b fixed issue4453 with a simple insertplainheader function that fixed the regression but didn't make the implementation more stable. Now we introduce plain header handling similar to how we handle hg patches. The whole header is scanned for fields to update while determining the best position for inserting the field if it is missing. It also makes sure there is an empty line between headers and body.

largefiles: drop the override for 'fetch' The fetch extension has been calling cmdutil.bailifchanged() since a014fdc97154, so this is redundant. Add test coverage to prevent regression. It doesn't look like there is any testing for fetch with largefiles.

rebase: ignore negative state when updating back to original wc parent The state mapping also contains some magic negative values (detached parent, ignored revision). Blindly reading the state thus lead to unfortunate usage of the negative value as an update destination. We now filter them out. We do a minor alteration of the test to catch this.

bundle2-push: provide transaction to reply unbundler This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. This diff adds an experimental config option "bundle2.pushback" which provides a transaction to the reply unbundler during a push operation. This behavior is opt-in because of potential security issues: the response can contain any part type that has a handler defined, allowing the server to make arbitrary changes to the local repository.

bundle2.processbundle: let callers request default behavior This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. The default transaction getter for processbundle is a private function that raises an exception; this diff lets calling code pass None as the transaction getter to explicitly request this default behavior. The next diff will check a config option to determine whether to provide a transaction to the reply bundle processor. If one shouldn't be provided, the code needs a way to specify that the default behavior should be used.

push: elevate phase transaction to cover entire operation This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Most pushes already open a transaction in order to sync phase information. This diff replaces that transaction with one that spans the entire push operation. This transaction will be used in a later patch to guard repository changes made during the reply handler.

pull: extract transaction logic into separate object This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Aside from the transaction logic, the pulloperation class is used primarily as a logic-free data structure for storing state information. This diff extracts the transaction logic into its own class that can be shared with push operations.

record: don't honor format-changing diffopts (issue4459) record does support the whitespace options, so honor those. In upcoming patches we'll replace all the other uses.

patch.difffeatureopts: add a feature for format-changing diffopts Typically, callers that are interested in this set are also interested in the others. We still add it as a first-class flag for convenience.

patch.difffeatureopts: add a feature for whitespace diffopts These aren't exactly format-breaking features -- just ones for which patches applied to a repo will produce incorrect commits, In any case, some commands like record and annotate only care about this feature.

patch.difffeatureopts: add a feature for diff.git This deserves to be its own feature -- indeed, this is often the only feature several commands care about.

patch: rename diffopts to diffallopts This is meant to be used when all features are explicitly requested. diffopts is kept as an alias for now -- it will go away soon.

patch: add a new function to initialize diffopts by feature Not all callers are interested in all diffopts -- for example, commands like record (which use diff internally) break when diffopts like noprefix are enabled. This function will allow us to add flags that callers can use to enable only the features they're interested in.

patch.diffopts: use a dict for initialization In upcoming patches we'll conditionally add to and remove from the dict.

largefiles: split the creation of a normal matcher out of its install method Refactoring addremove to support subrepos will need the ability to keep passing the same matcher and narrowing it, instead of monkey patching scmutil's matcher.

addremove: print relative paths when called with -I/-X (BC) For "hg addremove 'glob:*.py'", we print any paths added or removed as relative to the current directory, but when "hg addremove -I 'glob:*.py'" is used, we use the absolute path (relative from the repo root). It seems like they should be the same, so change it so we use relative paths in both cases. Continue to use absolute paths when no patterns are given.

merge with stable

merge with i18n

i18n-pt_BR: synchronized with 3bd577a3283e

i18n-ja: synchronized with edf29f9c15f0

Added signature for changeset 902554884335

Added tag 3.2.2 for changeset 902554884335

merge: before cd/dc prompt, check that changed side really changed Before, merging would in some cases ask "wrong" questions about "changed/deleted" conflicts ... and even do it before the resolve phase where they can be postponed, re"resolved" or answered in bulk operations. Instead, check that the content of the changed file really did change. Reading and comparing file content is expensive and should be avoided before the resolve phase. Prompting the user is however even more expensive. Checking the content here is thus better. The 'f in ancestors[0]' should not be necessary but is included to be extra safe.

largefiles: don't show largefile/normal prompts if one side is unchanged

tests: add test-issue3084.t cases for 'changed but same' as for 'unchanged' Use suffix -same for cases where file changed but content is the same - that is the case where manifestmerge doesn't detect that a file is unchanged. (The suffix -id is already used for cases where the file didn't change - that is the trivial case where manifestmerge detects that the file is unchanged.) These new tests are good but the results are bad. There shouldn't be any merge conflicts or prompts when one side didn't change.

tests: clean-up of largefiles tests in test-issue3084.t Prepare for adding more test cases to the systematic testing, moving the test from ac3b3a2d976d to another section.

pushkey: gracefully handle prepushkey hook failure (issue4455) This allow to gracefully report the failure of the bookmark push and carry on. Before this change set. Local push would plain quit and wireprotocol would failed in various ungraceful way.

hook: raise a more specialized HookAbort exception when a hook fails We need to gracefully handle some aborts for pushkey, especially because it leads to a user-facing crash over the wireprotocols. So we need a more specialized exception to catch.

help: fix typo in files help

revset: fix first and last for generatorset (issue4465) The code was just plain wrong.

mq: fix update of headers that occur in the "wrong" order Mq tried to insert headers in the right order. Sometimes it would stop searching before checking all headers and it could thus duplicate a header instead of replacing it.

subrepo: add status support for ignored files in git subrepos Retrieving the status of a git subrepo did not show ignored files. Using 'git ls-files', we can retrieve these files and display the correct status.

manifest: document the extra letter in working copy manifest node As the second developer to get confused by this in November, I'm adding some documentation for the next poor soul.

hgweb: send proper HTTP response after uncaught exception This patch fixes a bug where hgweb would send an incomplete HTTP response. If an uncaught exception is raised when hgweb is processing a request, hgweb attempts to send a generic error response and log that exception. The server defaults to chunked transfer coding. If an uncaught exception occurred, it was sending the error response string / chunk properly. However, RFC 7230 Section 4.1 mandates a 0 size last chunk be sent to indicate end of the entity body. hgweb was failing to send this last chunk. As a result, properly written HTTP clients would assume more data was coming and they would likely time out waiting for another chunk to arrive. Mercurial's own test harness was paving over the improper HTTP behavior by not attempting to read the response body if the status code was 500. This incorrect workaround was added in ba6577a19656 and has been removed with this patch.

tests: test rebase of merge of target ancestors

tests: test coverage of parent order with p1 outside rebase set

tests: add test for rebasing merges with ancestors of the rebase destination This shows sub-optimal behaviour. The user gets a merge prompt that it is very hard to justify.

merge: 0 is a valid ancestor different from None Most internal functions can take either a hash or an integer. Merge did however not handle 0 as revision 0. Now it does.

merge with stable

revert: look for copy information for all local modifications Renaming a file over an existing one marks the file as modified. So we track rename source in modified file too.

rename: properly report removed and added file as modified (issue4458) The result of 'hg rm' + 'hg rename' disagreed with the one from 'hg rename --force'. We align them on 'hg move --force' because it agrees with what 'hg status' says after the commit. Stopping reporting a modified file as added puts an end to the hg revert confusion in this situation (issue4458). However, reporting the file as modified also prevents revert from restoring the copy source. We fix this in a later changeset. Git diff also stop reporting the add in the middle of the chain as add. Not sure how important (and even wrong) it is.

manifest: fix a bug where working copy file 'add' mark was buggy Because the same dictionary was used to (1) get node from parent and (2) store annotated version, we could end up with buggy values. For example with a chain of renames: $ hg mv b c $ hg mv a b The value from 'b' would be updated as "<old-a>a", then the value of c would be updated as "<old-b>a'. With the current dictionary sharing this ends up with: '<new-c>' == '<old-a>aa' This value is double-wrong as we should use '<old-b>' and a single 'a'. We now use a read-only value for lookup. The 'test-rename.t' test is impacted because such a chained added file is suddenly detected as such.

merge: be precise about what merged into what in short desc

docker: add Docker files for running an Apache mod_wsgi server I frequently find myself wanting to run hgweb in a production-like environment, with a real HTTP server and multiple WSGI workers. This patch introduces a Docker environment for running Mercurial under Apache + mod_wsgi. With just a few command executions, it is possible to spin up a Docker container running hgweb. The container is tailored for Mercurial developers wanting to run Mercurial from a source checkout. It is **not** meant to be something suitable for production use. The container provides a default hgweb environment with an empty repository that allows pushes. You can thus start a container and push your favorite repository there for quick testing. The container is designed to allow customizations. Users can provide their own hgweb configurations and mount existing directories containing repositories into the container. The behavior of the container and how to control things is documented in the README.rst file.

merge: use None as filename for base in 'both created' conflicts Instead of using a file that we know is not in the common ancestor's maniffest, let's use None. This is safe as the only place that cares about the value (applyupdates) already checks if the item exists in the ancestor.

merge: break out "both renamed a -> b" case We can further limit the scope of the 2-way merge case by breaking out the case where the file was not created from scratch on both sides but rather renamed in the same way (and is therefore a 3-way merge). This involves copying some code, but it makes it clearer which case the "Note:" in the code refers to.

merge: separate out "both created" cases When 'f' is not in 'ma', 'a' will be 'nullid' and all the if/elif conditions that check whether some one nodeid is equal to 'a' will fail, and the else-clause will instead apply. We can make that more explicit by creating a separate 'm' action for the case where 'a' is 'nullid'. While it does mean copying some code, perhaps it makes it a little clearer which codepaths are possible, and which cases the "Note:" in the code refers to. It also lets us make the debug action messages a little more specific.

merge: indent to prepare for next patch

merge with stable

factotum: convert Plan 9 quoted string to Python string

plan9: setup check for current python build

add: check for the existence of a file matched inexactly before adding it The change in 10697f29af2b created a problem on Windows and OS X: --- /usr/local/mercurial/tests/test-issue660.t +++ /usr/local/mercurial/tests/test-issue660.t.err @@ -47,6 +47,8 @@ Should succeed - shadow removed: $ hg add b + adding b/b + b/b does not exist! Prior to the failing 'hg add', the file 'b/b' was added and committed, then 'b' was recursively deleted from the filesystem, file 'b' was created and the delete was recorded with 'hg rm --after'. This add is attempting to record the existence of file 'b'. A filesystem that is not case sensitive prevents dirstate.walk() from skipping its step 3, and step 3 has the effect of inserting removed files into the walk list. The Linux code doesn't run through step 3, and didn't exhibit the problem. It's not clear why a non case sensitive filesystem triggers step 3, given that the path normalization occurs in step 2. Prior to 10697f29af2b, part of the check here was 'f not in repo.dirstate' instead of 'f not in wctx'. Files in the 'r' state are filtered out of context.__contains__() but not dirstate.__contains__(). Therefore the removed file name wasn't added to the list of files to add when checking against dirstate. That change was to allow removed files to be readded, but adding a file that doesn't exist is nonsensical. If the user specifies a missing file, it will be an exact match and will still fail.

tests: choose the proper environment variable style for the platform Windows was printing out 'commit $HG_NODE' instead of the actual hash.

tests: fix another multi-statement hook for Windows The double quotes are necessary, otherwise it tries to pipe into a program named 'short'. An '&' could serve as a command separator on Windows instead of ';', but I don't see any obvious way to swap these depending on the platform. In this case though, there really wasn't a need for multiple statements.

run-tests: set a default largefiles usercache in the default hgrc file This fixes a test failure introduced in 4be754832829 on Windows and OS X, where the cached largefile wasn't being deleted because the named .cache directory didn't exist. It only existed on Linux because the test suite sets $HOME to the directory of the test being run, and Linux uses $HOME/.cache by default. Most of the other largefiles tests explicitly set this value at the top of their scripts, but test-largefiles-update.t didn't pick that up when it was created. Those scripts that do set a value will override this. We could just set the parameter in the test-largefiles-update.t script, but there are a few other non obvious tests that exercise largefiles too. These largefiles end up being cached in the user's real cache, so proper hygiene dictates that this not be left to each individual test script.

merge: remove obsolete check for untracked files in 'dm' action Since 0776a6cababe (merge: don't use unknown(), 2012-02-09), untracked files are no longer included in the manifest diff, so there is no need to check exclude them when renaming files for directory moves with the 'dm' action.

merge: remove dead assignment in applyupdates()

merge: move calculateupdates() before applyupdated() calculateupdates() happens before applyupdates(), so move it before in the code. That also moves it close to manifestmerge(), which is a good location as calculateupdates() is the only caller of manifestmerge().

merge: remove unused variables from _checkcollision()

largefiles: avoid exec-bit examination on the platform being unaware of it Changeset 24600c9d7f4e introduced the examination of exec bit of largefiles in "hg status --rev REV" case, but it doesn't avoid it on the platform being unaware of exec-bit (e.g. on NTFS of Windows).

merge with stable

changegroup: fix file linkrevs during reorders (issue4462) Previously, if reorder was true during the creation of a changegroup bundle, it was possible that the manifest and filelogs would be reordered such that the resulting bundle filelog had a linkrev that pointed to a commit that was not the earliest instance of the filelog revision. For example: With commits: 0<-1<---3<-4 \ / --2<--- if 2 and 3 added the same version of a file, if the manifests of 2 and 3 have their order reversed, but the changelog did not, it could produce a filelog with linkrevs 0<-3 instead of 0<-2, which meant if commit 3 was stripped, it would delete that file data from the repository and commit 2 would be corrupt (as would any future pulls that tried to build upon that version of the file). The fix is to make the linkrev fixup smarter. Previously it considered the first manifest that added a file to be the first commit that added that file, which is not true. Now, for every file revision we add to the bundle we make sure we attach it to the earliest applicable linkrev.

merge: consistently use single quotes for non-user-facing strings Because I'm getting tired of searching for both 'O' and "O".

localrepo: add a currenttransaction method This method returnx the current transaction or None: it will allow a cache writer to hook in an existing transaction.

repoview: extract actual hidden cache writing in its own function This will allow the generation of this cache within the transaction. Relying on the transaction will reduce the chance of reader seeing bad cache.

templates: fix broken "less" & "more" links in paper style (issue4460) "/search", which is an invalid command in hgweb, was mistakenly used for "[show] more [revsets]" and "[show] less [revsets]" links on search page in templates "paper" (and those which inherit paper, such as coal) before and worked fine until 6e1fbcb18a75, which made hgweb more strict about invalid commands.

push: stop independent usage of bundle2 in syncphase (issue4454) The phase-syncing code was using bundle2 if the remote supported it. It was doing so without regard to bundle2 activation on the client. Moreover, the phase push is now properly included in the unified bundle2 push, so having extra code in syncphase should be useless. If the remote is bundle2-enabled, the phases should already be synced. The buggy verification code was leading to a crash when a 3.2 client was pushing to a 3.1 server. The real bundle2 path detected that their versions were incompatible, but the syncphase code failed to, sending an incompatible bundle2 to the server. We drop the useless and buggy code as a result. The "else" clause is de-indented in the process.

dirstate: speed up repeated missing directory checks In a mozilla repo with tip at bb3ff09f52fe, hg update tip~1000 && time hg revert -nq -r tip . displays ~4:20 minutes. With tip~100, it runs in ~11 s. With revision 100000, it did not finish in 12 minutes. Revert calls dirstate.status() with a matcher that matches each file in the target revision. The main problem [1] lies in dirstate._walkexplicit(), which looks for matching deleted directories by checking whether each path is prefix of any path in the dirstate. With m files in the dirstate and n files in the target revision that are not in the dirstate, this is clearly O(m*n). Let's improve by keeping a lazily initialized set of all the directories in the dirstate, so the time becomes O(m+n). After this patch, the 4:20 minutes become 5.5 s, while for a single missing path, it slows down from 1.092 s to 1.150 s (best of 4). The >12 min case becomes 5.8 s. [1] A narrower optimization would be to make revert take the fast path for '.' and '--all'.

revert: access status fields by name rather than index For better readability.

subrepo: remove "_getstorehashcachepath" referred by no other code paths

subrepo: replace direct file APIs around "writelines" by "vfs.writelines" This patch also replaces "self._getstorehashcachepath" (building absolute path up) by "self._getstorehashcachename" (building relative path up), because "vfs.writelines" requires relative path.

vfs: add "writelines" This patch allows "writelines" to take "mode" and "notindexed" arguments, because subsequent patch for subrepo requires both.

vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode This patch uses "False" as default value of "notindexed" argument, even though "vfs.makedir()" uses "True" for it, because "os.mkdir()" doesn't set "_FILE_ATTRIBUTE_NOT_CONTENT_INDEXED" attribute to newly created directories.

subrepo: replace direct file APIs around "readlines" by "vfs.tryreadlines" This patch also replaces "self._getstorehashcachepath" (building absolute path up) by "self._getstorehashcachename" (building relative path up), because "vfs.tryreadlines" requires relative path. This patch makes "_readstorehashcache()" return "[]" (returned by "vfs.tryreadlines()"), when cache file doesn't exist, even though "_readstorehashcache()" returned '' (empty string) in such case before this patch. "_readstorehashcache()" is invoked only by the code path below in "_storeclean()": for filehash in self._readstorehashcache(path): if filehash != itercache.next(): clean = False break In this case, "[]" and '' don't differ from each other, because both of them cause avoiding iteration of "for loop".

vfs: add "readlines" and "tryreadlines" This patch allows "readlines" and "tryreadlines" to take "mode" argument, because "subrepo" requires to read files not in "rb" (binary, default for vfs) but in "r" (text) mode in subsequent patch.

subrepo: add "_cachestorehashvfs" to handle cache store hash files via vfs This "vfs" object will be used by subsequent patches to handle cache store hash files without direct file APIs. This patch decorates "_cachestorehashvfs" with "@propertycache" to delay vfs creation, because it is used only for cooperation with other repositories. In this patch, "/" is used as the path separator, even though "self._repo.join" uses platform specific path separator (e.g. "\\" on Windows). But it is reasonable enough, because "store" and other management file handling already include such implementation, and they work well.

subrepo: remove "_calcfilehash" referred by no other code paths

subrepo: replace "_calcfilehash" invocation by "vfs.tryread" "_calcfilehash" can be completely replaced by simple "vfs.tryread" invocation. def _calcfilehash(filename): data = '' if os.path.exists(filename): fd = open(filename, 'rb') data = fd.read() fd.close() return util.sha1(data).hexdigest() Building absolute path "absname" up by "self._repo.join" for files in "filelist" is avoided, because "vfs.tryread" does so internally.

subrepo: replace "os.path.exists" by "exists" via wvfs of the parent Existance of specified "path" should be examined by "exists" via wvfs of the parent repository, because the working directory of the parent repository may be in UTF-8 mode. Wide API should be used via wvfs in such case. In this patch, "/" is used as the path separator, even though "path" uses platform specific path separator (e.g. "\\" on Windows). But it is reasonable enough, because "store" and other management file handling already include such implementation, and they work well.

subrepo: avoid redundant "util.makedirs" invocation "util.makedirs" for the (sub-)repository root of "hgsubrepo" is also executed in the constructor of "localrepository", if "create" is True and ".hg" of it doesn't exist. This patch avoids redundant "util.makedirs" invocation in the constructor of "hgsubrepo".

merge: remove confusing comment about --force manifestmerge() has a piece of code that's roughly: if not force and different: abort else: # if different: old untracked f may be overwritten and lost ... The comment only talks about what happens when 'different' is true, and in combination with the if-block above, that must mean that it is only about what happens when 'force and different'. It seems quite fine that files are overwritten when 'force' is true, so let's remove the comment. As it stands, it can easily be interpreted as a TODO (which is how I interpreted it at first).

phases: read pending data when appropriate If we are called by a hook and pending data exists, read those.

bookmark: read pending data when appropriate If we are called by a hook and pending data exists, read it.

test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.

transaction: write pending generated files Such file are generated with a .pending prefix. It is up to the reader to implement the necessary logic for reading pending files. We add a test to ensure pending files are properly cleaned-up in both success and error cases.

transaction: have _generatefile return a boolean The function returns True if any files were generated. This will be used to know if any pending files have been written.

transaction: allow generating files with a suffix This will allow us to generate temporary pending files. Files generated with a suffix are assumed temporary and will be cleaned up at the end of the transaction.

transaction: fix some docstring grammar

transaction: accept a 'location' argument for registertmp This will allow generation of temporary files outside of store. This will be useful for bookmarks.

tests: handle differences between missing file error strings on Windows and Unix

run-tests: don't warn on unnecessary globs mandated by check-code.py When test output is processed, if os.altsep is defined (i.e. on Windows), TTest.globmatch() will cause a warning later on if a line has a glob that isn't necessary. Unfortunately, the regex checking in check-code.py doesn't have this context. Therefore we ended up with cases where the test would get flagged with a warning only on Windows because a glob was present, because check-code.py would warn if it wasn't. For example, from test-subrepo.t: $ hg -R issue1852a push `pwd`/issue1852c pushing to $TESTTMP/issue1852c (glob) The glob isn't necessary here because the slash is shown as it was provided. However, check-code mandates one to handle the case where the default path has backslashes in it. Break the cycle by checking against a subset of the check-code rules before flagging the test with a warning, and ignore the superfluous glob if it matches a rule. This change fixes warnings in test-largefiles-update.t, test-subrepo.t, test-tag.t, and test-rename-dir-merge.t on Windows. I really hate that the rules are copy/pasted here (minus the leading two spaces) because it would be nice to only update the rules once, in a single place. But I'm not sure how else to do it. I'm open to suggestions. Splitting some of the rules out of check-code.py seems wrong, but so does moving check-code.py out of contrib, given that other checking scripts live there. There are other glob patterns that could be copied over, but this is enough to make the current tests run on Windows.

update: remove unnecessary check for unknown files with --check As far as I and the test suite can tell, the checks in manifestmerge() already report the errors (whether or not --check is given), so we don't need to call merge.checkunknown(). Since this is the last call to the method, also remove the method.

merge with stable

tests: move a multi-statement debuglocks hook into a shell script for Windows Before this patch, a part of "test-push-hook-lock.t" fails unexpectedly on Windows environment, because semicolon (";") isn't recognized as the command separator by "cmd.exe". This is fixed the same way as a similar issue in 7c253c23de3b.

tests: fix globs for Windows test-largefiles-update.t, test-subrepo.t, test-tag.t, and test-rename-dir-merge.t still warn about no result returned because of unnecessary globs that test-check-code-hg.t wants, relating to output for pushing to, pulling from and moving X to Y.

run-tests: include quotes in the HGEDITOR value when storing sys.executable This fixes test-install.t on Windows that broke in 2122b82b6987 when shlex.split() was added to the debuginstall command: @@ -7,8 +7,11 @@ checking installed modules (*mercurial)... (glob) checking templates (*mercurial?templates)... (glob) checking commit editor... + Can't find editor 'c:\Python27\python.exe -c "(omitted)"' in PATH + (specify a commit editor in your configuration file) checking username... - no problems detected + 1 problems detected, please check your install! + [1] What happens is that shlex.split() on Windows turns this: c:\Python27\python.exe -c "import sys; sys.exit(0)" into this: ['c:Python27python.exe', '-c', 'import sys; sys.exit(0)'] While technically a regression, most programs on Windows live in some flavor of 'Program Files', and therefore the environment variable needs to contain quotes anyway to handle the space. This wasn't handled prior to the shlex() change, because it tested the whole environment variable to see if it was an executable, or split on the first space and tested again.

mq: when adding headers in plain mode, separate them from message (issue4453) c87f2a5a6e49 did a clean-up in one direction ... but we want it in the other direction.

mq: introduce insertplainheader - same naive implementation as before

mq: when setting message in plain mode, separate it from header (issue4453) Fix inconsistent handling of plain header separation in mq patcheader - and contrary to c87f2a5a6e49, do it in the direction of having an empty line between header and description. Plain patches are like mails and should thus have an empty line between headers and body in compliance with RFC 822 3.1.

setdiscovery: avoid a full changelog graph traversal We were definitely being suboptimal here: we were constructing two full sets, one with the full set of common nodes (i.e. a graph traversal) and one with all nodes. Then we subtract one set from the other. This whole process is O(commits) and causes discovery to be significantly slower than it should be. Instead, keep track of common incrementally and keep undecided as small as possible. This makes discovery massively faster on large repos: on one such repo, 'hg debugdiscovery' over SSH with one commit missing on the client and five on the server went from 4.5 seconds to 1.5. (An 'hg debugdiscovery' with no commits missing on the client, i.e. connection startup time, was 1.2 seconds.)

ancestor: add a way to remove ancestors of bases from a given set This and missingancestors can share state, which will turn out to be perfect for set discovery.

ancestor: add a way to add to bases of a missing ancestor object This will be useful for setdiscovery, since with that we incrementally add to our knowledge of common nodes.

ancestor: add a way to test whether a missing ancestor object has bases This is pretty trivial so there's no unit test coverage for it. This will be used by setdiscovery.

ancestor: remove now-unused missingancestors function Callers should use revlog.incrementalmissingrevs instead.

revlog: switch findmissing* methods to incrementalmissingrevs This will allow us to remove ancestor.missingancestors in an upcoming patch.

revlog: add a method to get missing revs incrementally This will turn out to be useful for discovery.

test-ancestor: add support for multiple tests against one incremental object In upcoming patches we'll add more operations to the object, and this prepares for testing those operations.

test-ancestor: move naive missing ancestor algorithm into a class This mirrors the change to the real missing ancestor algorithm in a previous patch.

ancestor.missingancestors: turn into a state-keeping class This allows multiple efficient missing ancestor queries against the same set of bases. In upcoming patches we'll also define ways to grow the set of bases. The fact that the test output hasn't changed establishes this patch's correctness.

ancestor.missingancestors: calculate start point after filtering revsvisit Any revs that are filtered out are also in basesvisit, which means they wouldn't be returned in the missing list anyway. There's no need to explore such revs or their ancestors. The 'if not revsvisit' check moves down because we can't call max() on an empty set.

ancestor.missingancestors: don't discard from basesvisit We only actually care about whether revsvisit is empty, so we can let basesvisit grow to arbitrary size. It turns out that this actually helps performance. For a large repo with hundreds of thousands of commits, hg perfrevset 'only(0, tip)' (basically the worst case, involving a full DAG traversal) goes from 1.63 seconds to 1.50. hg perfrevset 'only(tip, 0)' remains unchanged at 1.98 seconds.

test-ancestor: use random testing for missing ancestors We're going to make changes to the missing ancestor algorithm, and random testing will give us much more confidence than a fixed set of tests.

test-ancestor: define a main function We're going to add to it in upcoming patches.

test-ancestor: test iteration for lazyancestors This has some test coverage in test-revlog-ancestry.py, but not very much.

ancestor.lazyancestors: take parentrevs function rather than changelog Principle of least privilege, and it also brings this in line with missingancestors.

remove: avoid a bogus warning about no tracked files when removing '.' Previously, any files relative to the root of the repo that match the -I patterns would be deleted, but the command exited with 1 after printing a warning: $ hg remove -S -I 're:.*.txt' . removing sub1/sub2/folder/test.txt removing sub1/sub2/test.txt not removing .: no tracked files

remove: support remove with explicit paths in subrepos

remove: recurse into subrepositories with --subrepos/-S flag Like 'forget', git and svn subrepos are currently not supported. Unfortunately the name 'remove' is already used in the subrepo classes, so we break the convention of naming the subrepo function after the command.

cmdserver: protect pipe server streams against corruption caused by direct io Because pipe-mode server uses stdio as IPC channel, other modules should not touch stdio directly and use ui instead. However, this strategy is brittle because several Python functions read and write stdio implicitly. print 'hello' # should use ui.write() # => ch = 'h', size = 1701604463 'ello', data = '\n' This patch adds protection for such mistakes. Both stdio files and low-level file descriptors are redirected to /dev/null while command server uses them.

cmdserver: postpone creation of pipe server until run() This makes it easy to swap file descriptors while running command server.

cmdserver: use given streams as pipe channels like other commands Because commandserver itself is an hg subcommand, it shouldn't use stdio directly in principle.

revset.only: use cl.findmissingrevs ancestor.missingancestors is really an implementation detail.

manifestmerge: use already existing fl2 synonym for m2.flags(f) Probably not a noticeable performance gain, but shortens the code slightly.

merge: drop underscore prefix from _checkunknown() The method has been called from commands.py since 3eab42088be4 (update: just merge unknown file collisions, 2012-02-09), so drop the underscore prefix that suggests that it's private.

transaction: drop special handling for phases and bookmarks generation We are still doing double backups, but now that we have proper location handling this is less of an issue. Dropping this simplifies the code before we add some pending-related logic. This also ensures we actually test the new 'location' mechanism.

transaction: use 'location' instead of 'vfs' objects for file generation The argument is now a location name. The location must be present in the 'vfsmap' provided to the transaction at creation time.

transaction: use 'location' instead of 'vfs' in the addbackup method This unlock the backup of file outside of store (eg: bookmarks).

addbackup: handle file in subdirectory The current naming scheme ('journal.backups.<file>') resulted is bad directory name when 'file' was in a subdirectory. We now extract the directory name and create the backupfile within it. We plan to use file in a subdirectory for cachefile.

addbackup: use the vfs for the backup destination too The backup file location was always computed using the opener, bypassing the 'location' setting. (And making the feature broken.)

transaction: set backupentries version to proper value Now that all mechanisms are in place, we can advertise it with a proper new version.

transaction: support cache file in backupentries We do not want to abort if anything wrong happen while handling a cache file. Cache file have way to be invalidated and if old/bad version stay no misbehavior will happen. Proper value will eventually be computed and the wrong will be righten. This changeset use the transaction reporter (usually writing on stderr) to write details about failed cache handling. This will only apply to write operation using a transaction. The usual update during read only operation will stay a debug message. I was on the way to bring these message back to debug level when I realised it could be a feature. People with write access to the repository are likely to have the power to fix error related to cache (and it is valuable to fix them). So let the things as is for now.

transaction: use the location value when doing backup We finally use the 'location' value coupled with the 'vfsmap' to restore backup for the right file.

transaction: pass a vfs map to the transaction The goal is to allow access to file outside ofthe store directory from the transaction. The obvious target are the `bookmarks` file. But we can envision usage for cache too. We keep passing a main opener explicitly because a lot of code rely on this default opener. The main opener (operating on store) is using an empty key ''.

transaction: change the on disk format for backupentries We need to store new data to improve the current transaction logic: - location: We want to generate and backup file outside of the 'store' (eg: bookmarks, or various cache files). This requires knowing and preserving where each file is located. The value of this new field is a string. It will be used as a key for a vfs mapping. - cache: We would like to handle cache file in the transaction code. This Will help to have cache consistent with the repository state and avoid performance issue on big repository like Mozilla. However, failure to handle cache file should not result in a transaction failure. We add a new field that carry this information. The value is boolean, A True value mean any error while handling this file can be ignored. Those two mechanisms are not implemented yet, but they are now persisted in the on disk file. Support for new mechanisms is coming in later changeset. We update the file format now and will introduce the new features in later changeset. The format version is set to 0 until we actually support the new feature. This will prevent misunderstanding between incomplete and final client. Support for reading both version 1 and (future) version 2 could be achieved (using default value when reading version 1) but has not been seen as necessary for now.

silenttestrunner: add environment variable to make tests noisy again As I've been working on complicated extension code it's been handy to be able to get standard unittest verbose output so I can find crashers more efficiently.

largefiles: update comments to refer to the right overridden method This cleans up leftovers from b228ad1f79d7 (largefiles: override calculateupdates instead of manifestmerge, 2014-03-02).

revlog: cache chain info after calculating it for a rev (issue4452) This dumb cache works surprisingly well: on a repository with typical delta chains ~50k in length, unbundling a linear series of 5000 revisions (changelogs and manifests only) went from 60 seconds to 3.

manifest: add matches() method Move the code in context._manifestmatches() into a new manifest.matches(). It's a natural place for the code to live and it allows other callers to easily use it. It should also make it easier to optimize the new method in alternative implementations of the manifest (same reasoning as with manifest.diff()).

context.status: pass status tuple into _buildstatus By passing a status tuple (instead of the current list), we can access the status fields by name and make it a little more readable.

context.status: avoid de- and reconstructing status tuple We can just modify the status tuple we got from dirstate.status() instead of deconstructing it and constructing a new instance, thereby simplifying the code a little.

context.status: make _dirstatestatus() return an status tuple Letting _dirstatestatus() return an scmutil.status instance also means that _buildstatus() will always return such an instance, so we can remove the conversion from the call sites.

context.status: wipe deleted/unknown/ignored fields when reversed It makes no sense to request reverse status (i.e. changes from the working copy to its parent) and then look at the deleted, unknown or ignored fields. If you do, you would get the result from the forward status (changes from parent to the working copy). Instead of giving a nonsensical answer to a nonsensical question, it seems a little saner to return empty lists. It might be best if we could prevent the caller accessing these lists, but it's doubtful it's worth the trouble.

patch.trydiff: add support for noprefix

mdiff.unidiff: add support for noprefix

diff: add a --noprefix option See previous patch descriptions for the motivation. The tests reflect the current state of the world -- as we add support we'll see changes in the test output.

patch.diffopts: add support for noprefix In an upcoming patch we'll enable support as an option to 'hg diff' as well. The tests reflect the current state of the world -- as we add support we'll see changes in the test output.

patch.diffopts: allow a setting to be forced in plain mode Upcoming patches will add an option that will almost certainly break diff output parsers when enabled. Add support for forcing an option to something in plain mode, as a fallback. Options passed in via the CLI are not affected, though -- it is assumed that any script passing the option in explicitly knows what it is doing.

patch.diffopts: break get function into if statements We're going to add another condition here, and with the current structure that becomes just too confusing.

mdiff.diffopts: add a new noprefix option By popular demand, we introduce an option to disable the 'a/' and 'b/' prefixes in diff output. This makes copying and pasting filenames from diff output easier. This option will be implemented and documented in upcoming patches. To ensure that existing scripts that parse output don't break, we will ensure that this prefix is disabled in plain mode. A straight 'hg export | hg import' without HGPLAIN=1 will still be broken though, but there's little that can be done about that.

mdiff.diffopts: add doc comment for nobinary

changelog: register changelog.i.a as a temporary file The file is registered to make sure the transaction is cleaned up in all cases.

transaction: allow registering a temporary transaction file During the transaction, files may be created to store or expose data involved in the transaction (eg: changelog index data are written in a 'changelog.i.a' for hooks). But we do not have an official way to record such file creation and make sure they are cleaned up. The lack of clean-up is currently okay because there is a single file involved and a single producer/consumer. However, as we want to expose more data (bookmarks, phases, obsmarker) we need something more solid. The 'backupentries' mechanism could handle that. Temporary files can be encoded as a backup of nothing '('', <temporarypath>)'. We "need" to attach it to the same mechanism as we use to be able to use temporary transaction files outside of .'store/' and 'backupentries' is expected to gain such feature. This changeset makes it clear that we should rename 'backupentries' to something more generic.

transaction: always generate file on close The conditionnal was buggy and file were only generated if "onclose" was defined. By luck, "onclose" was always defined.

remove: move most of the implementation into cmdutils.remove() This will allow access to the reusable parts from subrepos, similar to add(), forget(), etc.

revlog: increase I/O bound to 4x the amount of data consumed This doesn't affect normal clones since they'd be bound by the CPU bound below anyway -- it does, however, improve generaldelta clones significantly. This also results in better deltaing for generaldelta clones -- in generaldelta clones, we calculate deltas with respect to the closest base if it has a higher revision number than either parent. If the base is on a significantly different branch, this can result in pointlessly massive deltas. This reduces the number of bases and hence the number of bad deltas. Empirically, for a highly branchy repository, this resulted in an improvement of around 15% to manifest size.

revlog: bound based on the length of the compressed deltas This is only relevant for generaldelta clones.

revlog: compute length of compressed deltas along with chain length In upcoming patches to the revlog, we're going to split up the notions of bounding I/O and bounding CPU.

revlog: store fulltext when compressed delta is bigger than it This is a very silly case and not particularly likely to happen in the wild, but it turns out we can hit it in a couple of places. As we tune the storage parameters we're likely to hit more such cases. The affected test cases all have smaller revlogs now.

revlog: make a predicate clearer with parens

transaction: extract backupentry registration in a dedicated function We are about to use the 'backupentry' mechanism to allow cleaning up transaction-related temporary files (such as 'changelog.i.a'). We start by extracting the entry registration into its own method for easy reuse. At that point, I would like to rename the backup-file related variable to something generic but I'm a bit short of ideas.

transaction: pass the transaction to 'postclose' callback This mirrors the API for 'pending' and 'finalize' callbacks. I do not have immediate usage planned for it, but I'm sure some callback will be happy to access transaction related data.

transaction: pass the transaction to 'finalize' callback The callback will likely need to perform some operation related to the transaction (eg: registering file update). So we better pass the current transaction as the callback argument. Otherwise callback that needs it has to rely on horrible weak reference trick. This allow already allow us to slay a wild weak reference usage.

transaction: pass the transaction to 'pending' callback The callback will likely need to perform some operation related to the transaction (eg: backing files up). So we better pass the current transaction as the callback argument. Otherwise callback that needs it has to rely on horrible weak reference trick. The first foreseen user of this is changelog._writepending. We would like it to register the temporary file it create for cleanup purpose.

transaction: gather backupjournal logic together in the __init__ The initialisation of file-backup related variable were a bit scattered, we gather them together.

transaction: handle missing file in backupentries (instead of using entries) The case where a backup of a missing file was requested was previously handled by the 'entries' list. As the 'backupentries' is about to gain ability to backup files outside of '.hg/store', we want it to be able to handle the missing file too. Reminder: using 'addbackup' on a missing file means that such file needs to be deleted if we rollback the transaction.

test-hup: use ls instead of echo Having one file entry per line makes it easier to analyse diffs.

largefiles: move "copyalltostore" invocation into "markcommitted" Before this patch, while "hg convert", largefiles avoids copying largefiles in the working directory into the store area by combination of setting "repo._isconverting" in "mercurialsink{before|after}" and checking it in "copytostoreabsolute". This avoiding is needed while "hg convert", because converting doesn't update largefiles in the working directory. But this implementation is not efficient, because: - invocation in "markcommitted" can easily ensure updating largefiles in the working directory "markcommitted" is invoked only when new revision is committed via "commit" of "localrepository" (= with files in the working directory). On the other hand, "commitctx" may be invoked directly for in-memory committing. - committing without updating the working directory (e.g. "import --bypass") also needs this kind of avoiding For efficiency of this kind of avoiding, this patch does: - move "copyalltostore" invocation into "markcommitted" - remove meaningless procedures below: - hooking "mercurialsink{before|after}" to (un)set "repo._isconverting" - checking "repo._isconverting" in "copytostoreabsolute" This patch invokes "copyalltostore" also in "_commitcontext", because "_commitcontext" expects that largefiles in the working directory are copied into store area after "commitctx". In this case, the working directory is used as a kind of temporary area to write largefiles out, even though converted revisions are committed via "commitctx" (without updating normal files).

largefiles: avoid printing messages while transplanting by "_lfstatuswriters" Putting "lambda *msg, **opts: None" (= avoid printing messages always) into "_lfstatuswriters" while transplanting makes explicit passing "printmessage = False" for "updatelfiles()" useless. This patch also removes setting/unsetting "repo._istransplanting" in "overridetransplant", because there is no code path referring it.

largefiles: update standins only at the 1st commit of "transplant --continue" Before this patch, "hg transplant --continue" may record incorrect standins, because largefiles extension always avoid updating standins while transplanting, even though largefiles in the working directory may be modified manually at the 1st commit of "hg transplant --continue". But, on the other hand, updating standins should be avoided at subsequent commits for efficiency reason. To update standins only at the 1st commit of "hg transplant --continue", this patch uses "automatedcommithook", which updates standins by "lfutil.updatestandinsbymatch()" only at the 1st commit of resuming. Even after this patch, "repo._istransplanting = True" is still needed to avoid some status report while updating largefiles in "lfcommands.updatelfiles()". This is reason why this patch omits not "repo._istransplanting = True" in "overriderebase" but examination of "getattr(repo, "_istransplanting", False)" in "updatestandinsbymatch".

largefiles: avoid redundant "updatelfiles" invocation in "overridetransplant" At "hg transplant --merge REV", largefiles newly coming from the 2nd parent (= REV) are marked as "a"(dded) by "patch.patch()", and have to be marked as "n"(ormal) after commit. But until changeset 3100d1cbce32, such largefiles were still marked as "a" unexpectedly even after commit, because no additional entry is added to filelog of such largefiles and they aren't listed in "repo[newnode].files()" in this case: "newnode" is one of newly committed changeset (= result of "repo.commit()"). "updatelfiles" invocation in "overridetransplant" shadows this problem by forcibly synchronizing lfdirstate to dirstate. Now, "updatelfiles" invocation in "overridetransplant" is redundant, because changeset 3100d1cbce32 made "markcommitted" use "ctx.files()" to get targets of "synclfdirstate" instead of "repo[newnode].files()".

merge with stable

util.system: remove unused handling of onerr=ui In our code, onerr is None or util.Abort. It smells bad to overload ui and exception class.

util.system: use ui.system() in place of optional ui.fout parameter

ui: introduce util.system() wrapper to make sure ui.fout is used This change is intended to avoid future problem of data corruption under command server. out=ui.fout is mandatory as long as command server uses stdout as IPC channel.

hook: remove redundant code to redirect http hook output to client stream out=ui and out=ui.fout should be the same here. ui.fout was introduced at afccc64eea73, which was not available when out=ui was added at c37f35d7f2f5.

hgk: forward command output to ui.fout consistently Nobody would want to run hgk in command server, but it should work in principle. This fixes possible data corruption of command-server channel.

merge with stable

Added signature for changeset 643c58303fb0