largefiles: eliminate a duplicate message when removing files in verbose mode There is no --after for addremove, so the printing for addremove can be hoisted out of the 'not after' check. The difference between the two remove messages reflects the existing difference between core remove and core addremove styles for printing the file. There are still some pre-existing issues here. Core addremove only prints on inexact matches or when verbose. But since the largefiles that are being removed are passed to removelargefiles() as a pattern list, there is never an inexact match, which would keep the largefiles from being printed at all unless verbose is specified. Therefore, the output is a little more aggressive than core. The addremove print style here is also inconsistent with core- it should use matcher.uipath(f) instead of f. These can be fixed once a matcher is passed in.

largefiles: ensure that the standin files are available in getlfilestoupload() The function only adds the hash content of the file to the set to upload if the file in the ctx is a standin. It is called by overrides.summaryremotehook(), which is called in the summary method. The largefiles extension switches 'lfstatus' on in summary, so the standins shouldn't be visible when obtaining a context there. The reason this wasn't noticed before is that the 'lfstatus' attribute is only being set on the unfiltered repo because of how repoview delegates attribute assignment. Therefore any filtered view will return a context containing standins, whether or not 'lfstatus' was set in the various overrides methods. That will be fixed in the next patch. But without this change, the next patch would have test failures for 'summary --large' stating there are no files to upload.

merge: move checking of unknown files out of manifestmerge() This moves most reading of filelogs out of manifestmerge, making it easy for a narrow clone extension to filter out or translate unwanted actions before any filelogs are read. The only call left is inside of copies.mergecopies(), which can be overridden separately at a lower level.

merge: extract method for checking for conflicting untracked file Now that the functionality is collected in one place, let's extract it to a method.

merge: create 'cm' action for 'get or merge' case We still have one case of a call to _checkunknownfile() in manifestmerge(): when force=True and branchmerge=True and the remote side has a file that the local side doesn't. This combination of arguments is used by 'hg merge --force', but also by rebase and unshelve. In this scenario, we try to create the file from the contents from the remote, but if there is already a local untracked file in place, we merge it instead.

merge: don't overwrite untracked file at directory rename target When a directory was renamed and a new untracked file was added in the new directory and the remote directory added a file by the same name in the old directory, the local untracked file gets overwritten, as demonstrated by the broken test case in test-rename-dir-merge. Fix by checking for unknown files for 'dg' actions too. Since _checkunknownfile() currently expects the same filename in both contexts, we need to add a new parameter for the remote filename to it.

merge: remove constant tuple element from 'aborts' The second element of the tuples in the 'aborts' list is always 'ud', so let's remove it.

merge: collect checking for unknown files at end of manifestmerge() The 'c' and 'dc' actions include creating a file on disk and we need to check that no conflicting file exists unless force=True. Move two of the calls to _checkunknownfile() to a single place at the end of manifestmerge(). This removes some of the reading of filelogs from the heart of manifestmerge() and collects it in one place close to where its output (entries in the 'aborts' list) is used. Note that this removes the unnecessary call to _checkunknownfile() when force=True in one of the code paths.

merge: introduce 'c' action like 'g', but with additional safety _checkunknownfile() reads the filelog of the remote side's file. For narrow clones, the filelog will not exist for all files and we need a way to avoid reading them. While it would be easier for the narrow extension to just override _checkunknownfile() and make it ignore files outside the narrow clone, it seems cleaner to have manifestmerge() not care about filelogs (considering its name). In order to move the calls to _checkunknownfile() out, we need to be able to tell in which cases we should check for unknown files. Let's start by introducing a new action distinct from 'g' for this purpose. Specifically, the new action will be just like 'g' except that it will check that for conflicting unknown files first. For now, just add the new action type and convert it to 'g'.

merge: structure 'remote created' code to match table This does duplicate the call to _checkunknownfile(), but it will simplify future patches.

pushkey: run hook after the lock release The pushkey operation used to be in its own wireprotocol command and (in practice) always be lock free when running the hook. With bundle2, it happen in a greater scheme and a hook running locking command would get stuck. We now run such hooks after the lock release as similar hook do. Bundle2 test are altered to ensure we are lockfree at hook running time.

setup: use changes since latest tag instead of just distance For a Mercurial built on the merge from stable into default right after 3.2.2 was released -- 19ebd2f88fc7 -- the version number produced was "3.2.2+4". This is potentially misleading, since in reality the built Mercurial includes many more changes compared to 3.2.2. Change the versioning scheme so that we take into consideration all the changes present in the current revision that aren't present in the latest tag. For 19ebd2f88fc7 the new versioning scheme results in a version number of "3.2.2+256". This gives users a much better idea of how many changes have actually happened since the latest release. Since changessincelatesttag is always greater than or equal to the latesttagdistance, this will produce version numbers that are always greater than or equal to the old scheme. Thus there's minimal compatibility risk.

setup: use changessincelatesttag from archive if present changessincelatesttag gives one a better idea of how much the code has changed since. Since changessincelatesttag is always greater than or equal to the latesttagdistance (see previous patch for why), this will always produce version numbers greater than or equal to the previous scheme.

archive: store number of changes since latest tag as well This is different from latesttagdistance in that while latesttagdistance is defined to be the length of the longest path to the latest tag, changessincelatesttag is the number of changes contained in @ that aren't contained in the latest tag. So, if 't' is the latest tag in the repository below: t | v --o--o----o \ \ ..o..o..@ then latesttagdistance is 2, but changessincelatesttag is 4. Note that changessincelatesttag is always greater than or equal to the latesttagdistance -- that's because changessincelatesttag counts all the changes in the longest path since the latest tag, and possibly others. This is an important fact that we'll take advantage of in upcoming patches.

merge with stable

demandimport: blacklist distutils.msvc9compiler (issue4475) This module depends on _winreg, which is windows-only. Recent versions of setuptools load distutils.msvc9compiler and expect it to ImportError immediately when on non-Windows platforms, so we need to let them do that. This breaks in an especially mystifying way, because setuptools uses vars() on the imported module. We then throw an exception, which vars doesn't pick up on well. For example: In [3]: class wat(object): ...: @property ...: def __dict__(self): ...: assert False ...: In [4]: vars(wat()) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-4-2781ada5ffe6> in <module>() ----> 1 vars(wat()) TypeError: vars() argument must have __dict__ attribute Which is similar to the problem we run into.

largefiles: don't duplicate 'actions' into 'actionbyfile'

merge: make calculateupdates() return file->action dict This simplifies largefiles' overridecalculateupdates(), which no longer has to do the conversion it started doing in 38e55e55ae4d (largefiles: rewrite merge code using dictionary with entry per file, 2014-12-09). To keep this patch small, we'll leave the name 'actionbyfile' in overrides.py. It will be renamed in the next patch.

merge: let _forgetremoved() work on the file->action dict By moving the conversion from the file->action dict after _forgetremoved(), we make that method shorter by removing the need for the confusing 'xactions' variable.

merge: let _resolvetrivial() work on the file->action dict By moving the conversion from the file->action dict after _resolvetrivial(), we greatly simplify and clarify that method.

merge: let bid merge work on the file->action dict By moving the conversion from the file->action dict after the bid merge code, bid merge can be simplified a little. A few tests are affected by this change. Where we used to iterate over the actions first in order of the action type ('g', 'r', etc.) [1], we now iterate in order of filename. This difference affects the order of debug log statements. [1] And then in the non-deterministic order of files in the manifest dictionary (the order returned from manifest.diff()).

merge: write manifestmerge() using dictionary with entry per file In the same vein as 38e55e55ae4d (largefiles: rewrite merge code using dictionary with entry per file, 2014-12-09), rewrite manifestmerge() itself as dictionary with the filename as key. This will let us simplify some of the other code in merge.py and eventually drop the conversion in the largefiles code. No difference in speed could be detected (well within the noise level when run in Mozilla repo).

repoview: backout ced3ecfc2e57 Monkey patching repoview does not really work and making it really work will be really hard. So we better have it broken without complexity than broken with extra complexity.

largefile: explain why no monkey patching on a repoview The comment requested for investigations, here they are.

largefile: backout ca54fb3d71ce The hack for method monkey patching on repoview has been ruled out as fragile, so we are rolling it back. We'll expand the explanation in the next changeset.

incoming: handle phases the same as pull Now that bundlerepo can move phases safely, 'hg incoming' can share its phase handling code with pull to better reflect what would actually show up.

bundlerepo: retract phase boundary This patch makes bundrepo retract the phase boundary for new commits to 'draft' status, which is consistent with the behavior of 'hg unbundle'. The old behavior was for commits to appear with the same phase as their nearest ancestor in the base repository. This affects several classes of operation: * Inspecting a bundle with the -B flag * Treating a bundle file as a peer (old: everything public, new: everything draft) * Incoming command (neither old or new behavior is sensible -- fixed in next patch)

bundlerepo: implement safe phasecache This patch makes bundlerepo use a subclass of phasecache that will allow phase boundaries to be moved around, but will never write them to the underlying repository.

localrepo.__getitem__: add slicing support This allows the python slice syntax to be used with revision numbers when indexing into a repository, such as repo[8:]

ignore: resolve ignore files relative to repo root (issue4473) (BC) Previously these would be considered to be relative to the current working directory. That behavior is both undocumented and doesn't really make sense. There are two reasonable options for how to resolve relative paths: - relative to the repo root - relative to the config file Resolving these files relative to the repo root matches existing behavior with hooks. An earlier discussion about this is available at http://mercurial.markmail.org/thread/tvu7yhzsiywgkjzl. Thanks to Isaac Jurado <diptongo@gmail.com> for the initial patchset that spurred the discussion.

test-hgignore: add testing for ui.ignore I couldn't find any tests for this, and we're going to make changes here in upcoming patches.

tests: add missing globs for Windows I couldn't figure out how to glob the first chunk for Windows, so it's been conditionalized.

share: use the 'sharedpath' attr on repo instead of reloading from the file Seems like a useful optimization, and now the exact file content is not a concern.

share: fix source repo lookup on Windows The stored path contains platform specific separators, so splitting on '/.hg' returned the string unmodified on Windows. The source was then looked for in $source/.hg/.hg, which obviously fails. This caused cascading errors in test-share.t relating to the recent bookmark support.

help: suggest '-v -e' to get built-in aliases for extensions (issue4461) If extension name matches one of command names, suggest user to type 'hg help -v -e <extension>' to get full list of built-in aliases

test-check-commit-hg: clarify misleading "commit message rules" error The test case doesn't only check the commit message, but also the patch, which can result in confusing output like + Revision df6f06d17100 does not comply to commit message rules + ------------------------------------------------------ + 32: adds double empty line + + even when there are no double blank lines in the commit message. Drop the "commit message" part to make it less confusing.

keyword: handle resolve to either parent Merged files are considered modified at commit time even if only 1 parent differs. In this case we must use the change context of this parent for expansion. The issue went unnoticed for long because it is only apparent until the next update to the merge revision - except in test-keyword where it was always staring us in the face.

keyword: update test file syntax

branches: deprecate -a

largefiles: fix a spurious missing file warning with forget (issue4053) If an uncommitted and deleted file was forgotten, a warning would be emitted, even though the operation was successful. See the previous patch for 'remove -A' for the exact circumstances, and details about the cause.

largefiles: fix a spurious missing file warning with 'remove -A' (issue4053) The bug report doesn't mention largefiles, but the given recipe doesn't fail unless the largefiles extension is loaded. The problem only affected normal files, whether or not any largefiles are committed, and only files that have not been committed yet. (Files with an 'a' state are dropped from dirstate, not marked removed.) Further, if the named normal file never existed, the warning would be printed out twice. The problem is that the core implementation of remove() calls repo.status(), which eventually triggers a dirstate.walk(). When the file isn't seen in the filesystem during the walk, the exception handling finds the file in dirstate, so it doesn't complain. However, the largefiles implementation called status() again with all of the original files (including the normal ones, just dropped). This time, the exception handler doesn't find the file in dirstate and does complain. This simply excludes the normal files from the second repo.status() call, which the largefiles extension has no interest is processing anyway.

largefiles: introduce the 'composelargefilematcher()' method This is a copy/paste (with the necessary tweaks) of the composenormalfilematcher method currently on default, which does the inverse- this trims the normal files out of the matcher. It will be used in the next patch.

context: return dirstate parents in workingctx.ancestors() workingctx.ancestors() was not returning the dirstate parents as part of the result set. The only place this function is used is for copy detection when committing a file, and that code already checks the parents manually, so this change has no affect at the moment. I found it while playing around with changing how copy detection works.

backout: add --commit option Mercurial backout command makes a commmit by default only when the backed out revision is the parent of working directory and doesn't commit in any other case. The --commit option changes behaviour of backout to make a commit whenever possible (i.e. there is no unresolved conflicts). This behaviour seems more intuitive to many use (especially git users migrating to hg).

share: add option to share bookmarks This patch adds the -B/--bookmarks option to the share command added by the share extension. All it does for now is create a marker, 'bookmarks.shared', that will be used by future code to implement the sharing functionality.

highlight: ignore Unicode's extra linebreaks (issue4291) Unicode and Python's unicode.splitlines() treat several extra legacy ASCII codepoints as linebreaks, even though the vast bulk of computing and Python's own str.splitlines() do not. Rather than introduce line numbering confusion, we filter them out when highlighting.

test: fix typo in test-help.t

templatekw: remove unneeded showtags Now that we have the machinery of namespaces in-place, we use that instead of hand-rolling our own template function. Note, this can only be used for tags because both branches and bookmarks have special case logic for 'default' and the current bookmark (which is something outside the namespace api for now).

namespaces: generate template keyword when registering a namespace For any namespace, we generate a template keyword. For example, given a namespace 'babar', we automatically have the ability to use it in a template: hg log -r . -T '{babars % "King: {babar}\n"}' Furthermore, we only generate this keyword for a namespace if one doesn't already exist. This is necessary for 'branches' and 'bookmarks' since both of those have concepts of 'current' (something outside the namespace api) and also allows extensions to override default behavior if desired.

templatekw: add helper method to generate a template keyword for a namespace This marks our second feature of the namespace api: automatic template keyword. This patch adds a method that takes in a namespace and uses the node-to-name map to output the list of names.

namespaces: add names method to return list of names for a given node In the previous patch, we added a node-to-name map property. This patch just exposes that interface to the api.

namespaces: add nodemap property This patch adds a node-to-name map property to the namespace. This is necessary because we cannot simply invert the name-to-node map because we do not assume the name-to-node map to be unique (for example, consider named branches: many nodes have one branch name). The node-to-name is helpful in log commands where we are already iterating over a set of nodes and want to display some kind of naming information to the user.

namespaces: add method to get template name of namespace This patch adds the public api for getting the template name of a namespace so that the next patch can use it to generate a template keyword automatically.

namespaces: add template name of a namespace The template name property will be used in upcoming patches to automatically generate a template keyword. For example, given a namespace called 'babars', we will automatically generate a template keyword 'babar' such that we can use it in the following way, $ hg log -r . -T '{babars % "King: {babar}\n"}'

memctx: remove redundant test for issue4470 from test-commit.t Because: - the test to avoid regression for issue4470 was already added to test-commit-amend.t by previous patch It is also a part of test series about manifest calculation issues of memctx in test-commit-amend.t. - this is the only test using "commit --amend" in test-commit.t

memctx: calculate manifest more efficiently Before this patch, "memctx._manifest" updates all entries in the (parent) manifest. But this is inefficiency, because almost all files may be clean in that context. On the other hand, just updating entries for changed "files" specified at construction causes unexpected abortion, when there is at least one newly removed file (see issue4470 for detail). To calculate manifest more efficiently, this patch replaces "pman.iteritems()" for the loop by "self._status.modified" to avoid updating entries for clean or removed files Examination of removal is also omitted, because removed files aren't treated in this loop (= "self[f]" returns not None always).

merge with stable

Added signature for changeset 6dad422ecc5a

Added tag 3.2.3 for changeset 6dad422ecc5a

pathauditor: check for Windows shortname aliases

pathauditor: check for codepoints ignored on OS X

darwin: omit ignorable codepoints when normcase()ing a file path This lets us avoid some nasty case collision problems in OS X with invisible codepoints.

encoding: add hfsignoreclean to clean out HFS-ignored characters According to Apple Technote 1150 (unavailable from Apple as far as I can tell, but archived in several places online), HFS+ ignores sixteen specific unicode runes when doing path normalization. We need to handle those cases, so this function lets us efficiently strip the offending characters from a UTF-8 encoded string (which is the only way it seems to matter on OS X.)

test-casefolding.t: demonstrate a bug with HFS+ ignoring some codepoints

manifest: disallow setting the node id of an entry to None manifest.diff() uses None as a special value to denote the absence of a file, so setting a file node to None means you then can't trust manifest.diff(). This should also make future manifest work slightly easier.

context: stop setting None for modified or added nodes Instead use a magic value, so that we can identify modified or added nodes correctly when using manifest.diff(). Thanks to Martin von Zweigbergk for catching that we have to update _buildstatus as well. That part eluded my debugging for some time.

largefiles: don't actually remove largefiles in an addremove dry run The addlargefiles() method already properly handled dry runs.

bundle2: lowercase part types Since the capitalization no longer carries any meaning (previous diff), this patch normalizes all of the bundle2 part type strings to lower case.

bundle2.bundlepart: make mandatory part flag explicit in API This makes all bundle2 parts mandatory unless they are expressly made advisory via the keyword parameter or the bundlepart.mandatory property.

memctx: calculate manifest correctly with newly-removed files (issue4470) Before this patch, "memctx._manifest" tries to get (and use normally) filectx also for newly-removed files, even though "memctx.filectx()" returns None for such files. To calculate manifest correctly even with newly-removed files, this patch does: - replace "man.iteritems()" for the loop by "self._status.modified" to avoid accessing itself to newly removed files this also reduces loop cost for large manifest. - remove files in "self._status.removed" from the manifest In this patch, amending is confirmed twice to examine both (1) newly removed files and (2) ones already removed in amended revision.

memctx: calculate manifest including newly added files correctly Before this patch, "memctx._manifest" calculates the manifest according to the 1st parent. This causes the disappearance of newly added files from the manifest. For example, if newly added files aren't listed up in manifest of memctx, they aren't listed up in "added" field of "status" returned by "ctx.status()", and "{diff()}" (= "patch.diff") in "committemplate" shows nothing for them. To calculate manifest including newly added files correctly, this patch puts newly added files (= ones in "self._status.added") into the manifest. Some details of changes for "test-commit-amend.t" in this patch: - "touch foo" is replaced by "echo foo > foo", because newly added empty file can't be shown in "diff()" output without "diff.git" configuration - amending is confirmed twice to examine both (1) newly added files and (2) ones already added in amended revision

memctx: calculate exact status being committed from specified files Before this patch, "memctx._status" is initialized by "(files, [], [], [], [], [], [])" and this causes "memctx.modified" to include not only modified files but also added and removed ones incorrectly. This patch adds "_status" method to calculate exact status being committed according to "files" specified at construction time. Exact "_status" is useful to share/reuse logic of committablectx. This patch is also preparation for issues fixed by subsequent patches. Some details of changes for tests in this patch: - some filename lines are omitted in "test-convert-svn-encoding.t", because they are correctly listed up as "removed" files those lines are written out in "localrepository.commitctx" for "modified" and "added" files by "ui.note". - "| fixbundle" filterring in "test-histedit-fold.t" is omitted to check lines including "added" correctly "fixbundle" discards all lines including "added".

bundle2._processpart: forcing lower-case compare is no longer necessary Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. Since the parttype and mandatory bit are separated in bundle2.unbundlepart (see previous patch), there is no longer a need to remove the mandatory bit before working with the parttype.

bundle2.unbundlepart: decouple mandatory from parttype Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. This patch separates the two pieces of information when reading the part header so that it's unnecessary to know how they were combined during transmission.

memctx: fix manifest for removed files (issue4470) filectxfn returns None for removed files, so we have to check for None before computing the new file content hash for the manifest. Includes a test that proves this works, by demonstrating that we can show the diff of an amended commit in the committemplate.

convert: on svn failure, note libsvn version (issue4043) We have our own fast-path logic to see if something passes a sniff test for being a Subversion repository, but it's possible for a user to svnsync a repo using svn 1.8 and then use svn 1.7 bindings (as in the bug) to try and convert the repo. If we at least tell the user the version of libsvn that we used, they might get enough of a hint to check on their own for format incompatibilities between their svn{admin,sync} and the libsvn used by hg.

vfs: add a 'split' method This method has the same behavior as the 'os.path.split' function, but having it in vfs will allow handling of tricky encoding situations in the future. In the same patch, we replace the use of 'os.path.split' in the transaction code.

vfs: add a 'reljoin' function for joining relative paths The vfs.join method only works for absolute paths. We need something that works for relative paths too when transforming filenames. Since os.path.join may misbehave in tricky encoding situations, encapsulate the new join method in our vfs abstraction. The default implementation remains os.path.join, but this opens the door to other VFSes doing something more intelligent based on their needs. In the same go, we replace the usage of 'os.path.join' in transaction code.

subrepo: fix git subrepo ui argument

subrepo: drop the 'ui' parameter to revert() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in revert is to emit status and warning messages, and to check the verbose flag prior to printing the action to be performed on a file. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call. Unlike other methods where the matcher is passed along and narrowed, a new matcher is created in each repo, and therefore the bad() method already used the local repo's ui.

subrepo: drop the 'ui' parameter to removefiles() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in remove is to emit status and warning messages, and to check the verbose flag prior to printing files to be removed. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call.

subrepo: drop the 'ui' parameter to forget() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in forget is to emit status and warning messages, and to check the verbose flag prior to printing files to be forgotten. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call.

subrepo: drop the 'ui' parameter to cat() This no longer needs to be explicitly passed because the subrepo object tracks a 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in cat is to emit a status message when a subrepo is missing. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object.

subrepo: drop the 'ui' parameter to archive() The current state of subrepo methods is to pass a 'ui' object to some methods, which has the effect of overriding the subrepo configuration since it is the root repo's 'ui' that is passed along as deep as there are subrepos. Other subrepo method are *not* passed the root 'ui', and instead delegate to their repo object's 'ui'. Even in the former case where the root 'ui' is available, some methods are inconsistent in their use of both the root 'ui' and the local repo's 'ui'. (Consider hg._incoming() uses the root 'ui' for path expansion and some status messages, but also calls bundlerepo.getremotechanges(), which eventually calls discovery.findcommonincoming(), which calls setdiscovery.findcommonheads(), which calls status() on the local repo 'ui'.) This inconsistency with respect to the configured output level is probably always hidden, because --verbose, --debug and --quiet, along with their 'ui.xxx' equivalents in the global and user level hgrc files are propagated from the parent repo to the subrepo via 'baseui'. The 'ui.xxx' settings in the parent repo hgrc file are not propagated, but that seems like an unusual thing to set on a per repo config file. Any 'ui.xxx' options changed by --config are also not propagated, because they are set on repo.ui by dispatch.py, not repo.baseui. The goal here is to cleanup the subrepo methods by dropping the 'ui' parameter, which in turn prevents mixing subtly different 'ui' instances on a given subrepo level. Some methods use more than just the output level settings in 'ui' (add for example ends up calling scmutil.checkportabilityalert() with both the root and local repo's 'ui' at different points). This series just goes for the low hanging fruit and switches methods that only use the output level. If we really care about not letting a subrepo config override the root repo's output level, we can propagate the verbose, debug and quiet settings to the subrepo in the same way 'ui.commitsubrepos' is in hgsubrepo.__init__. Archive only uses the 'ui' object to call its progress() method, and gitsubrepo calls status().

subrepo: use 'self.ui' instead of 'self._repo.ui' They are now equivalent, but the former is slightly more readable.

subrepo: reset 'self.ui' to the subrepo copy of 'ui' in the hgsubrepo class Creation of the subrepo's '_repo' object creates a new 'ui' by combining the parent repo's 'baseui' and reading in the subrepo's hgrc file. This simply avoids 'self.ui' and 'self._repo.ui' pointing to different objects, which seems like a potential source of bugs. Git and Svn subrepos are unchanged, because they don't have their own ui, and have always used their parent's for their configuration.

subrepo: rename the '_ui' member to 'ui' The localrepository class has a 'ui' member, so keeping the names the same will allow for duck typing with subrepo instances when accessing 'ui'. Changing this is easier than finding all of the localrepository instance uses and renaming that to '_ui'.

subrepo: drop the '_ui' member in the subrepo subclasses This member has existed in the base class since fcbc66b5da6a.

graphmod: attempt to clarify documentation of groupbranchiter() Thanks to Pierre-Yves for checking my cleanups here and helping me understand the algorithm well enough to help document it.

graphmod: rename graph-topological config to graph-group-branches The latter suggests what the change is slightly better.

groupbranchiter: allow callers to select the first branch Instead of just bootstrapping the algorithm with the first revision we see, allow callers to pass revs that should be displayed first. All branches are retained until we can display such revision. Expected usage is to display the current working copy parent first.

groupbranchiter: support for non-contiguous revsets The algorithm now works when some revisions are skipped. We now use "first included ancestors" instead of just "parent" to link changesets with each other.

groubranchhiter: indent most of the inner code We are going to add an additional layer of indentation to support non-contiguous revset. We do it in a pure code movement changeset to help the readability of the next changeset.

graphlog: add a way to test the 'groupbranchiter' function We add an experimental config option to use the topological sorting. I first tried to hook the 'groupbranchiter' function in the 'sort' revset but this was useless because graphlog enforces revision number sorting :( As the goal is to advance on the topological iteration logic, I see this experimental option as a good way to move forward. We have to use turn the iterator into a list because the graphlog is apparently not ready for pure iterator input yet.

graphmod: add a function for topological iteration This changeset introduces a function to perform topological (one branch after the other) iteration over a set of changesets. This first version has a lot of limitations, but the approach should be flexible enough to allow many improvements in the future. This changeset aims to set the first stone more than providing a complete solution. The algorithm does not need to know the whole set of nodes involved before emitting revision. This makes it a good candidate for usage in place like `hg log` or graphical tools that need a fast first result time.

namespaces: add branches Note that the exception-catching from the previous branchtip check is moved up to catch exceptions from the try block surrounding the namespace lookup.

namespaces: add tags

namespaces: remove weakref; always pass in repo It turns out that maintaining a reference of any sort (even weak!) to the repo when constructed doesn't work because we may at some point pass in a repoview filtered by something other than what the initial repo was.

changectx: use names api to simplify and extend node lookup Previously, changectx had to know about each type of name (bookmark, tag, etc.) to look up. Now, we use repo.namenodes to simplify (and extend) this.

namespaces: add a method to the first matching node for a given name

namespaces: add bookmarks to the names data structure This marks the first use of abstracting our different types of named objects (bookmarks, tags, branches, etc.) and upcoming patches will use this to simplify logic.

namespaces: add a convenience property for the weakref _repo

namespaces: pass repo to __init__ We store repo as a weakref so that we can prevent a cycle between namespaces and localrepo.

namespaces: add a function to turn single results into lists

namespaces: add a method to register new namespaces

namespaces: introduce a generic way to map between names and nodes This patch begins the work to provide a way to register a namespace to handle 'names'. Benefits of this would be, - improved templating: This would provide {name} which could output any branch, bookmark, tag, or any extension registered namespace all without having the extension doing any extra work - improved tab completion: Since this provides a single source of all 'names', tab completion would not need to know of each namespace - changeset lookup: Similar to before, a unified place to get all 'names' will allow finding changesets without any extension code having to reimplement this Also, d226fe36e362 has shown us that for internal code which expects a certain type of method or behavior, we should provide an easy way for extensions to check this behavior.

statichttprepo: update profile of __call__ in mock vfs object New code paths could fail because the old statichttprepo profile couldn't handle the usual parameters. Instead, reuse a more generic profile also used in readonlyvfs.

tests: avoid git diffstat singular/plural output difference 'git diff --stat' output changed with regard to the amount of changes/insertions/deletions shown. In older git versions (1.7.7.6), output was shown as: 2 files changed, 2 insertions(+), 1 deletions(-) In newer versions, output is shown as: 2 files changed, 2 insertions(+), 1 deletion(-) This patch uses a regex to handle both cases.

subrepo: add revert support without backup for git subrepos Previously, git subrepos did not support reverting. This change adds basic support for reverting when '--no-backup' is specified. A warning is given (and the current state is kept) when a revert is done without the '--no-backup' flag.

match: make 'always' and 'exact' functions, not classes There is no reason for classes 'always' and 'exact' not to be just functions that return instances the plain 'match' class.

share: implement shared bookmark functionality This does not cause any behavioral change unless a 'bookmarks.shared' marker file exists. A future change will add UI to create this file when a repository is shared.

debugindex: respect --debug flag to show full nodeids

localrepo.clone: add a way to override server preferuncompressed Without this patch, if the server sets preferuncompressed, there's no way for clients to override that and force a non-streaming clone. With this patch, we extend the meaning of --pull to also override preferuncompressed and force a non-streaming clone.

hg.clone: set 'stream' depending on whether --pull was requested or not In an upcoming patch we'll differentiate between the two in localrepo.

merge: perform case-collision checking on final set of actions When there are multiple common ancestors, we should check for case collisions only on the resulting actions after bid merge has run. To do this, move the code until after bid merge. Move it past _resolvetrivial() too, since that might update actions. If the remote changed a file and then reverted the change, while the local side deleted the file and created a new file with a name that case-folds like the old file, we should fail before this patch but not after. Although the changes to the actions caused by _forgetremoved() should have no effect on case collisions, move it after that, too, so the next person reading the code won't have to think about it. Moving it past these blocks of code takes it to the end of calculateupdates(), so let's even move it outside of the method, so we also check collisions in actions produced by extensions overriding the method.

spelling: fixes from proofreading of spell checker issues

tests: allow more flexibility in git diffstat output The output of 'git diff --stat' changed in git 1.7.10 and 1.7.11. To ensure the tests work with earlier versions of git as well, the output is now wrapped with a whitespace regex.

merge: move cd/dc prompts after largefiles prompts By moving the cd/dc prompts out of calculateupdates(), we let largefiles' overridecalculateupdates() so the unresolved values (i.e. 'cd' or 'dc' rather than 'g', 'r', 'a' and missing). This allows overridecalculateupdates() to ask the user whether to keep the normal file or the largefile before the user gets the cd/dc prompt. Whichever answer the user gives, we make overridecalculateupdates() replace 'cd' or 'dc' action, saving the user one annoying (and less clear) question.

addremove: automatically process a subrepository's subrepos Since addremove on the top of a directory tree will recursively handle sub directories, it should be the same with deep subrepos, once the user has explicitly asked to process a subrepo. This really only has an effect when a path that is a subrepo (or is in a subrepo) is given, since -S causes all subrepos to be processed already. An addremove without a path that crosses into a subrepo, will still not enter any subrepos, per backward compatibility rules.

addremove: support addremove with explicit paths in subrepos Git and svn subrepos are currently not supported.

addremove: add support for the -S flag Git and svn subrepos are currently not supported. It doesn't look like git or svn have these commands natively, so that's an area for a git or svn expert.

commit: propagate --addremove to subrepos if -S is specified (issue3759) The recursive addremove operation occurs completely before the first subrepo is committed. Only hg subrepos support the addremove operation at the moment- svn and git subrepos will warn and abort the commit.

subrepo: store the ui object in the base class This will be used in the next patch to print a warning from the base class. It seems better than having to explicitly pass it to a new method, since a lot of existing methods also require it.

commit: abort if --addremove is specified, but fails This will be required when subrepo support is added, in order to ensure consistent commits when a subrepo flavor doesn't support addremove.

addremove: warn when addremove fails to operate on a named path It looks like a bad path is the only mode of failure for addremove. This warning is probably useful for the standalone command, but more important for 'commit -A'. That command doesn't currently abort if the addremove fails, but it will be made to do so prior to adding subrepo support, since not all subrepos will support addremove. We could just abort here, but it looks like addremove has always silently ignored bad paths, except for the exit code.

scmutil: pass a matcher to scmutil.addremove() instead of a list of patterns This will make it easier to support subrepository operations.

tests: fix a typo in test-walkrepos.py

merge: extract _resolvetrivial() function We would eventually like to move the resolution of modify/delete and delete/modify conflicts to the resolve phase. However, we don't want to move the checks for identical content that were added in 902554884335 (merge: before cd/dc prompt, check that changed side really changed, 2014-12-01). Let's instead move these out to a new _resolvetrivial() function that processes the actions from manifestmerge() and replaces any false cd/dc conflicts. The function will also provide a natural place for us to later add code for resolving false 'm' conflicts.

largefiles: start by finding files of interest Instead of iterating over 'g' action, first find the set of all files that are largefiles in p1. Then iterate over these files. This prepares for considering actions other than 'g'.

largefiles: rewrite merge code using dictionary with entry per file In overridecalculateupdates(), we currently only deal with conflicts that result in a 'g' action for either the largefile or a standin. We will soon want to deal cases with 'cd' and 'dc' actions here. It will be easier to reason about such cases if we rewrite it using a dict from filename to action. A side-effect of this change is that the output can only have one action per file (which should be a good change). Before this change, when one of the tests in test-issue3084 received this input (the 'a' in the input was a result of 'cd' conflict resolved in favor of the modified file): 'g': [('.hglf/f', ('',), 'remote created')], 'a': [('f', None, 'prompt keep')], and the user chose to keep the local largefile, it produced this output: 'g': [('.hglf/f', ('',), 'remote created')], 'r': [('f', None, 'replaced by standin')], 'a': [('f', None, 'prompt keep')], Although 'a' actions are processed after 'r' actions by recordupdates(), it still worked because 'a' actions have no effect on merges (only on updates). After this change, the output is: 'g': [('.hglf/f', ('',), 'remote created')], 'r': [('f', None, 'replaced by standin')], Similarly, there are several tests in test-largefiles-update that get inputs like: 'a': [('.hglf/large2', None, 'prompt keep')], 'g': [('large2', ('',), 'remote created')], and when the user chooses to keep the local largefile, they produce this output: 'a': [('.hglf/large2', None, 'prompt keep'), ('.hglf/large2', None, 'keep standin')], 'lfmr': [('large2', None, 'forget non-standin largefile')], In this case, it was not a merge but an update, so the 'a' action does have an effect. However, since dirstate.add() is idempotent, it still has no obserable effect. After this change, the output is: 'a': [('.hglf/large2', None, 'keep standin')], 'lfmr': [('large2', None, 'forget non-standin largefile')],

largefiles: put same 'action' object back in 'newglist' The items we put in 'newglist' are always the same as what we found in actions['g'], so let's just put the same item into the list instead of creating a new one.

largefiles: don't unnecessarily sort merge action lists The action lists returned from calculateupdates() (in merge.py) are not required to be sorted. In fact, since they result from iteration over the unordered manifest, they are unlikely to be sorted. Moreover, some of the lists are appended to after they are returned from manifestmerge(). The lists are instead sorted in applyupdates(). Therefore, let's not sort the lists generated in largefiles' overridecalculateupdates().

merge: don't treat 'diverge' and 'renamedelete' like actions See earlier patch for motivation.

merge: move dr/rd warning messages out of applyupdates() As preparation for making 'dr' and 'rd' actions no longer actions, move the reporting from applyupdates() to its caller update(). This way we won't have to pass additonal arguments to applyupdates() when they are no longer actions. Also, the warnings are equally unrelated to applyupdates() as they are to recordupdates(), as they don't result in any changes to either the working copy or the dirstate. See earlier patch for additional motivation.

merge: don't report progress for dr/rd actions It is easier to reason about certain algorithms in terms of a file->action mapping than the current action->list-of-files. Bid merge is already written this way (but with a list of actions per file), and largefiles' overridecalculateupdates() will also benefit. However, that requires us to have at most one action per file. That requirement is currently violated by 'dr' (divergent rename) and 'rd' (rename and delete) actions, which can exist for the same file as some other action. These actions are only used for displaying warnings to the user; they don't change anything in the working copy or the dirstate. In this way, they are similar to the 'k' (keep) action. However, they are even less action-like than 'k' is: 'k' at least describes what to do with the file ("do nothing"), while 'dr' and 'rd' or only annotations for files for which there may exist other, "real" actions. As a first step towards separating these acitons out, stop including them in the progress output, just like we already exclude the 'k' action.

subrepo: add partial diff support for git subrepos So far, git subrepositories were silently ignored for diffs. This patch adds support for git subrepositories, with the remark that --include and --exclude are not supported. If --include or --exclude are used, the subrepo is ignored.

subrepo: extend git version check to 3 digits This allows more flexibility when a version check is required. Some git features are introduced in a version where only the 3rd digit changes.

subrepo: move git version check into a separate method This allows checking the git version in other methods, instead of only being able to check if the version is ok or not.

rebase: show a note for updated mq patches It deserves more than a debug message. Show a note like: updating mq patch p0.patch to 5:9ecc820b1737 The message could also refer to "qrefresh" instead. Same same.

rebase: improve debug messages while processing the list of rebases

rebase: show warning when rebase creates no changes to commit Similar to graft: note: rebase of 6:eea13746799a created no changes to commit

rebase: show more useful status information while rebasing Show status messages while rebasing, similar to what graft do: rebasing 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: make 'saved backup' globbing less narrow in rebase tests Globbing the hash made it harder to maintain tests with run-tests -i when it was so far by the generated test output. The hashes are stable and we just need to add a (glob).

tests: stabilise mq rebase tests Prepare for including hashes in output ... and less globbing make the tests easier to update.

test-graft: use strip extension instead of mq extension It only needs strip, no reason to load all of mq.

transaction: remove the 'onabort' mechanism It has no known users. If someones needs similar functionality, a new 'addabort' method similar to 'addfinalize' should be added.

transaction: remove the redundant 'onclose' mechanism It is superseded by the 'addfinalize' function and all its user have been migrated.

fncache: document the fact fncache is outdate at hook run time Using 'addfinalize' to generate 'fncache' means that no pending version of the file will be generated for the hooks. We would have to use the 'addfilegenerator' method to get such result. However the 'fncachevfs' (who decide that a write is necessary) have no access to the transaction to register such file generation at add time. Having the transaction accessible to the 'vfs' is too much trouble for no benefit. This outdated 'fncache' file at hook time is not expected to be an issue. The previous move from 'onclose' to 'addfinalize' had no impact on this timing. I'm documenting it now because I looked at it.

fncache: drop dedicated 'onclose' function in favor of 'tr.addfinalize' Now that we have a shiny generic mechanism, we can use it.

merge with stable

graft: drop cset description from empty commit message This is either already redundant in the output or too verbose in quiet mode.

graft: show hashes in user-facing messages Graft was in various places just showing the revision number in status messges. Instead, also show the stable and more useful short hash.

graft: give helpful warning for empty grafts It was just showing a status message with the internal revision number. Instead, show a warning like note: graft of 27:3aaa8b6725f0 "28" created no changes to commit (message tweaked in-flight by mpm)

graft: show more useful status information while grafting Show status messages with first line of commit description and names, like grafting 12:2647734878ef "fork" (tip) This gives more context for the user when resolving conflicts.

tests: test coverage for empty graft A future change will add a warning to the quiet rebase.

largefiles: drop the unfiltered repo usage in overridepurge() Now that repoview supports replacing methods, we don't need this hack.

repoview: allow methods on the proxy class to be replaced It doesn't seem to be a common idiom for repo instances, but the status() method is replaced in largefiles' purge() override. Since __setattr__ is implemented in repoview to setattr() on the unfiltered repo, the replacement method wouldn't get called unless it was invoked with the unfiltered repo, because the filtered repo remains unchanged. Since this doesn't seem to be commonly used, I didn't bother to filter out methods that perhaps shouldn't be replaced, such as changelog().

log: fix log revset instability The log/graphlog revset was not producing stable results since it was iterating over a dict. Now we sort before iterating to guarantee a fixed order. This fixes some potential flakiness in the tests.

log: fix log -f slow path to actually follow history The revset created when -f was used with a slow path (for patterns and directories) did not actually contain any logic to enforce follow. Instead it was depending on the passed in subset to already be limited (which was limited to :. but not ::.). This fixes it by adding a '& ::.' to any -f log revset. hg log -f <file> is still broken, in that it can return results that aren't actually ancestors of the current file, but fixing that has major perf implications, so we'll deal with it later.

obsstore: cache size computation for fm1 node We have two different types of node type (sha1 and sha256, only sha1 is used now) and therefor different sizes for them. We now compute the value once instead of redoing the computation every loop. This has no visible performance impact.

obsstore: prefetch struct.calcsize This function is widely used and worth but be at module level. No specific performance boost is visible, but this is more consistent.

obsstore: disable garbage collection during initialization (issue4456) Python garbage collection is triggered by container creation. So code that creates a lot of tuples tends to trigger GC a lot. We disable the gc during obsolescence marker parsing and associated initialization. This provides an interesting speedup (25%). Load marker function on my 58758 markers repo: before: 0.468247 seconds after: 0.344362 seconds The benefit is a bit less visible overall. With python2.6 on my system I see: after: 0.60 before: 0.53 The difference is probably explained by the delaying of a costly GC. (but there is still a win). Marking involved tuples, lists and dicts as ignorable by the garbage collector should give us more benefit. But this is another adventure. Thanks goes to Siddharth Agarwal for the lead.

dirstate: use the 'nogc' decorator Now that we have a generic way to disable the gc, we use it. however, we have too use it in a baroque way. See inline comment for details.

util: add a 'nogc' decorator to disable the garbage collection Garbage collection behave pathologically when creating a lot of containers. As we do that more than once it become sensible to have a decorator for it. See inline documentation for details.

generate-working-copy-states: open() in binary mode when writing content This avoids changes to the hashes on Windows in test-merge-force.t like so: @@ -594,12 +594,12 @@ content2 M missing_content2_content3_content4-tracked - <<<<<<< local: 443153eb5b88 - test: local\r (esc) - content4 - ||||||| base - ======= - content2 - >>>>>>> other: 9b2ccd328a08 - test: remote\r (esc) + <<<<<<< local: 0447570f1af6 - test: local + content4 + ||||||| base + ======= + content2 + >>>>>>> other: 85100b8c675b - test: remote missing_content2_content3_content4-tracked.orig: content4

largefiles: don't use 'r' action for standin that doesn't exist When merging and the remote has turned a normal file into a largefile and the user chooses to keep the local largefile, we use the 'r' action for the remote largefile standin. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

largefiles: don't use 'r' action for normal file that doesn't exist When merging and the remote has turned a largefile into a normal file and the user chooses to keep the local largefile, we use the 'r' action for the remote normal file. This is wrong, since that file does not exist in the parent of the working copy. Use 'k', which does nothing but debug logging, instead.

rebase: handle revtodo as a special value when storing/restoring state Revtodo happens to share its value with nullrev, but this is an implementation details, so we move away from it. After this changeset one can successfully change the values for all the constants and the tests still pass, but doing so would require more refactoring if we want to avoid breaking backward compatibility on the state file.

rebase: add a 'revtodo' constant The state mapping is using '-1' to mark revisions that have not been rebased yet. We introduce and use a constant for that purpose. This will help emphasize the fact the value means something other than nullrev.

rebase: use '>= 0' to know is a revision was rebased The fact that the state for "not yet rebased" is -1 is an implementation details. So we change the comparisons to some semantically more correct.

patchbomb: introduce a 'patchbomb.confirm' option When set to true, this option will make patchbomb always ask for confirmation before sending the email. Confirmation is a powerful way to prevent stupid mistakes when the sending patches. This should let me get rid of my global alias adding --confirm to hg email. I know that some people may get bitten when moving from a machine with confirm configured to a machine where it is not, but I think it is worth the risk.

patchbomb: add a 'patchbomb.intro' option This option allows the user to control the default behavior for including an introduction message. This avoids having to tirelessly skip the intro for people contributing to Mercurial. The three possibles values are: - always, - auto (default, current behavior), - never. I was thinking of ("true", "false", "") (empty value being auto) but I ruled it out as too confusing. This new config option reuses the pre-existing 'patchbomb' section.

patchbomb: extract 'getoutgoing' closure into its own function This continues my crusade against closure complication.

perf: add a perfloadmarkers command It is used to benchmark the obsstore initialization time.

rebase: move base calculation from rebasenode() to defineparents() We want to collect all calculation in one place.

largefiles: remove redundant checks for false modify/delete conflicts In a34a99181f36 (largefiles: don't show largefile/normal prompts if one side is unchanged, 2014-12-01), overridecalculateupdates() started checking for false modify/delete conflicts in large files and their standins. Then, in the very next changeset, 902554884335 (merge: before cd/dc prompt, check that changed side really changed, 2014-12-01), calculateupdates() itself started checking for false modify/delete conflicts in all files. Since "large files and their standins" is a subset of "all files", we can now drop the checks in overridecalculateupdates().

merge: make 'keep' message more descriptive Most merge action messages don't describe the action itself, they describe the reason the action was taken. The only exeption is the 'k' action, for which the message is just "keep" and instead there is a code comment folling it that says "remote unchanged". Let's move that comment into the merge action message.

addremove: restore the relative path printing when files are named This fixes the previously mentioned issue with 3778884197f0, and undoes its corresponding test change. The test change demonstrates the correctness when a file is specified (i.e. the glob is required on Windows because relative paths use '\' and absolute paths use '/'). It is admittedly very subtle, but there will be a more robust test in the addremove -S v3 series.

match: introduce uipath() to properly style a file path Several methods print files relative to the repo root, unless files are named on the command line, in which case they are printed relative to cwd. Since the check relies on the 'pats' parameter, which needs to be replaced by a matcher when adding subrepo support, this logic gets folded into the matcher to tidy up the callers. Prior to 3778884197f0, this style decision was based off of whether or not the 'pats' list was empty. That change altered the check to test match.anypats() instead, in order to make paths printed consistent when -I/-X is specified. That however, changed the style when a file is given to the command. So now we test the pattern list to get the old behavior for files, as well as test -I/-X to get the consistency for patterns.

merge with stable

update: don't overwrite untracked ignored files on update When looking for untracked files that would conflict with a tracked file in the target revision (or the remote side of a merge), we explcitly exclude ignored files. The code was added in 7e30f5f2285f (merge: refactor unknown file conflict checking, 2012-02-09), but it seems like only unknown, not ignored, files were considered since the beginning of time. Although ignored files are mostly build outputs and backup files, we should still not overwrite them. Fix by simply removing the explicit check.

update: add tests for untracked local file We don't seem to have any tests for updating to another revision when there are untracked files on the local side that conflict with the those on the remote side, so let's add tests. This shows how we overwrite untracked ignored files when updating to a revision that tracks the file.

merge: don't overwrite conflicting file in locally renamed directory When the local side has renamed a directory from a/ to b/ and added a file b/c in it, and the remote side has added a file a/c, we end up overwriting the local file b/c with the contents of remote file a/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename get ('dg') action.

merge: don't ignore conflicting file in remote renamed directory When the remote side has renamed a directory from a/ to b/ and added a file b/c in it, and the local side has added a file a/c, we end up moving a/c to b/c without considering the remote version of b/c. Add a check for this case and use the merge ('m') action in this case instead of the directory rename ('dm') action.

merge: duplicate 'if f in copied' into each branch

merge: branch code into {n1 and n2, n1, n2} top-level cases There are three high-level cases that are of interest in manifestmerge(): 1) The file exists on both sides, 2) The file exists only on the local side, and 3) The file exists only on the remote side. Let's make this clearer in the code. The 'if f in copied' case will be broken up into the two applicable branches in the next patch.

update: remove redundant and misplaced merge tests The "nothing to merge" case is covered by test-merge-default.t. The "uncommitted changes" case is covered by test-merge1.t (and others). The "merge -f" case is covered by test-merge-force.t.

largefiles: use clearer debug messages in actions In overridecalculateupdates(), 'g' (get) actions may be converted into other actions. In most of these cases, it does not make sense to keep the action's message. For example, 'remote created' does not make sense for an 'r' (remove) action.

largefiles: don't clobber merge action message with user message The message in the action is used for debugging and should not be the same as the question presented to the user. Use a different variable for the user message, so the 'msg' variable already in scope does not get overwritten.

bookmarks: factor out repository lookup from writing bookmarks file This will allow the share extension to extend bookmarks functionality to share bookmarks between repositories.

parsers: ensure revlog index node tree is initialized before insertion Currently, the revlog index C implementation assumes its node tree will be initialized before a new element is inserted by revnum. For example, revlog.py executes 'self.index.insert(-1, e)' in _addrevision(). This is only safe because the node tree has been initialized by a "node in self.nodemap" check made in addrevision(). (For context, this was discovered while developing an experimental revlog mixin which stores "elided nodes" via a separate code path from _addrevision(); that new code path segfaults without this patch.)

revset: add 'only' to DoS-safe symbols There is no reason to disable it in hgweb because the same query can be written without using 'only()'.

revset: move 'only' so that functions are sorted alphabetically

tests: drop unnecessary glob that causes a warning on Windows This was triggered by 3778884197f0.

merge: add test with conflicting file and remote directory rename Changset 88629daa727b (merge: demonstrate that directory renames can lose local file content, 2014-12-02) should clearly have added the reverse version of the test: where the remote side renamed a directory, added a new file in that directory, and the local directory added a conflicting file in the source directory. Add such a test now, and also touch up the ones already added slightly (e.g. 'local' was a stupid value for content that can be on either side of a merge).

graft: use a real conflict for the tests One of the graft tests grafts a changeset that changes a file's content from 'a' to 'b' onto a branch that has changed the file's content from 'a', via 'b', and then back to 'a' again. To prepare for not considering this a file in need of merging, let's use 'c' as the file's new content to make sure it has to be considered conflicting. There's a second similar case further down where an ancestor is grafted. Make sure that is also considered a conflict.

add: use lexists so that broken symbolic links are added This restores the add behavior prior to d8cdd46f426d and matches the behavior of addremove.

rebase: avoid redundant repo[rev].rev() - just keep working in rev space

rebase: always store rev in state, also when collapsing

rebase: clarify naming of variables holding node hashes - don't call them rev

bookmarks: factor out bookmark file opening for easier extensibility

merge with stable

diff: explicitly honor all diffopts 'hg diff' should naturally honor all diffopts.

annotate: explicitly only honor whitespace diffopts The whitespace ones are the only ones the annotate logic cares about anyway, so there's no visible impact.

notify: explicitly honor all diffopts The notify output doesn't seem to be parseable anyway, what with the maxdiff config option. Plus it is designed mainly for servers where hopefully the admins are doing sensible things.

jsonchangeset: don't honor whitespace and format-changing diffopts JSON is meant to be parsed by computers, and format changes can break them.

transplant: don't honor whitespace and format-changing diffopts transplant uses diff purely internally.

hgk: don't honor whitespace and format-changing diffopts The diff-tree command is a plumbing command in Git, which means it is expected to have a stable output format. Ensure that our output is stable too.

patchbomb: don't honor whitespace and format-changing diffopts (BC) The whitespace diffopts break lossless transmission, and the format-changing ones make import harder. We expect parsers to be able to read git-style diffs, though.

merge: add more thorough tests for --force With generate-working-copy-states.py generalized to support arbitrarily many changesets, we can use it for generating test cases for merge: use one changeset each for base, remote and local. With the various working copy states, this is a total of 104 cases. The first candidate for additional testing is 'hg merge --force'. Even though the force option is deprecated, it is convenient for testing because it can be tested without first needing to revert any changes. Except for the lack of checking for uncommitted changes, it differs in only a few cases from unforced merge. The new tests cover all the cases in the existing test-merge-force.t, except for the unforced merge case, which is covered in several other files anyway, so nothing remains of the file after this patch.

merge: display modify/delete conflict prompts in sorted order The order is determined by manifest.diff(), which currently is not sorted. There are currently no tests for this, but we will soon add some that would be flaky without this patch.

generate-working-copy-states: accept depth arguments on command line Add a parameter to generate-working-copy-states.py that indicates how many changesets are wanted. This number impacts all the subcommands. The current 'filelist' subcommand becomes 'filelist 2', the current 'base' and 'parent' subcommands become 'state 2 1' and 'state 2 2' respectively, while 'wc' becomes 'state 2 wc'. See earlier patch for motivation.

generate-working-copy-states: generalize for depth The script can currently generate filenames and contents for exactly two changesets plus the working copy. For some tests (e.g. of plain dirstate status), only one changeset is needed, while for others (e.g. merge), three changesets are needed. Let's prepare for such tests by generalizing the code for any number of changesets.

generate-working-copy-states: make 'tracked' a string The only places the 'tracked' variable is only used is where it's converted to a string, so let's simplify by using the string value directly.

merge: demonstrate that directory renames can lose local file content When a directory has been renamed on the local branch and a file has been added in the old location on a remote branch, we move that new file to the new location. Unfortunately, if there is already a file there, we overwrite it with the contents from the remote branch. For untracked local files, we should probably abort, and for tracked local files, we should merge the contents. To start with, let's add a test to demonstrate the breakage. Also note that while files merged in from a remote branch are normally (and unintuitively) reported as modified, these files are reported as added.

mq: drop updateheader - inserthgheader and insertplainheader is enough

mq: smarter handling of plain headers 6333412245ec and 5ccced6eab0b fixed issue4453 with a simple insertplainheader function that fixed the regression but didn't make the implementation more stable. Now we introduce plain header handling similar to how we handle hg patches. The whole header is scanned for fields to update while determining the best position for inserting the field if it is missing. It also makes sure there is an empty line between headers and body.

largefiles: drop the override for 'fetch' The fetch extension has been calling cmdutil.bailifchanged() since a014fdc97154, so this is redundant. Add test coverage to prevent regression. It doesn't look like there is any testing for fetch with largefiles.

rebase: ignore negative state when updating back to original wc parent The state mapping also contains some magic negative values (detached parent, ignored revision). Blindly reading the state thus lead to unfortunate usage of the negative value as an update destination. We now filter them out. We do a minor alteration of the test to catch this.

bundle2-push: provide transaction to reply unbundler This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. This diff adds an experimental config option "bundle2.pushback" which provides a transaction to the reply unbundler during a push operation. This behavior is opt-in because of potential security issues: the response can contain any part type that has a handler defined, allowing the server to make arbitrary changes to the local repository.

bundle2.processbundle: let callers request default behavior This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. The default transaction getter for processbundle is a private function that raises an exception; this diff lets calling code pass None as the transaction getter to explicitly request this default behavior. The next diff will check a config option to determine whether to provide a transaction to the reply bundle processor. If one shouldn't be provided, the code needs a way to specify that the default behavior should be used.

push: elevate phase transaction to cover entire operation This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Most pushes already open a transaction in order to sync phase information. This diff replaces that transaction with one that spans the entire push operation. This transaction will be used in a later patch to guard repository changes made during the reply handler.

pull: extract transaction logic into separate object This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Aside from the transaction logic, the pulloperation class is used primarily as a logic-free data structure for storing state information. This diff extracts the transaction logic into its own class that can be shared with push operations.

record: don't honor format-changing diffopts (issue4459) record does support the whitespace options, so honor those. In upcoming patches we'll replace all the other uses.

patch.difffeatureopts: add a feature for format-changing diffopts Typically, callers that are interested in this set are also interested in the others. We still add it as a first-class flag for convenience.

patch.difffeatureopts: add a feature for whitespace diffopts These aren't exactly format-breaking features -- just ones for which patches applied to a repo will produce incorrect commits, In any case, some commands like record and annotate only care about this feature.

patch.difffeatureopts: add a feature for diff.git This deserves to be its own feature -- indeed, this is often the only feature several commands care about.

patch: rename diffopts to diffallopts This is meant to be used when all features are explicitly requested. diffopts is kept as an alias for now -- it will go away soon.

patch: add a new function to initialize diffopts by feature Not all callers are interested in all diffopts -- for example, commands like record (which use diff internally) break when diffopts like noprefix are enabled. This function will allow us to add flags that callers can use to enable only the features they're interested in.

patch.diffopts: use a dict for initialization In upcoming patches we'll conditionally add to and remove from the dict.

largefiles: split the creation of a normal matcher out of its install method Refactoring addremove to support subrepos will need the ability to keep passing the same matcher and narrowing it, instead of monkey patching scmutil's matcher.

addremove: print relative paths when called with -I/-X (BC) For "hg addremove 'glob:*.py'", we print any paths added or removed as relative to the current directory, but when "hg addremove -I 'glob:*.py'" is used, we use the absolute path (relative from the repo root). It seems like they should be the same, so change it so we use relative paths in both cases. Continue to use absolute paths when no patterns are given.

merge with stable

merge with i18n

i18n-pt_BR: synchronized with 3bd577a3283e

i18n-ja: synchronized with edf29f9c15f0

Added signature for changeset 902554884335

Added tag 3.2.2 for changeset 902554884335

merge: before cd/dc prompt, check that changed side really changed Before, merging would in some cases ask "wrong" questions about "changed/deleted" conflicts ... and even do it before the resolve phase where they can be postponed, re"resolved" or answered in bulk operations. Instead, check that the content of the changed file really did change. Reading and comparing file content is expensive and should be avoided before the resolve phase. Prompting the user is however even more expensive. Checking the content here is thus better. The 'f in ancestors[0]' should not be necessary but is included to be extra safe.

largefiles: don't show largefile/normal prompts if one side is unchanged

tests: add test-issue3084.t cases for 'changed but same' as for 'unchanged' Use suffix -same for cases where file changed but content is the same - that is the case where manifestmerge doesn't detect that a file is unchanged. (The suffix -id is already used for cases where the file didn't change - that is the trivial case where manifestmerge detects that the file is unchanged.) These new tests are good but the results are bad. There shouldn't be any merge conflicts or prompts when one side didn't change.

tests: clean-up of largefiles tests in test-issue3084.t Prepare for adding more test cases to the systematic testing, moving the test from ac3b3a2d976d to another section.

pushkey: gracefully handle prepushkey hook failure (issue4455) This allow to gracefully report the failure of the bookmark push and carry on. Before this change set. Local push would plain quit and wireprotocol would failed in various ungraceful way.

hook: raise a more specialized HookAbort exception when a hook fails We need to gracefully handle some aborts for pushkey, especially because it leads to a user-facing crash over the wireprotocols. So we need a more specialized exception to catch.

help: fix typo in files help

revset: fix first and last for generatorset (issue4465) The code was just plain wrong.

mq: fix update of headers that occur in the "wrong" order Mq tried to insert headers in the right order. Sometimes it would stop searching before checking all headers and it could thus duplicate a header instead of replacing it.

subrepo: add status support for ignored files in git subrepos Retrieving the status of a git subrepo did not show ignored files. Using 'git ls-files', we can retrieve these files and display the correct status.

manifest: document the extra letter in working copy manifest node As the second developer to get confused by this in November, I'm adding some documentation for the next poor soul.

hgweb: send proper HTTP response after uncaught exception This patch fixes a bug where hgweb would send an incomplete HTTP response. If an uncaught exception is raised when hgweb is processing a request, hgweb attempts to send a generic error response and log that exception. The server defaults to chunked transfer coding. If an uncaught exception occurred, it was sending the error response string / chunk properly. However, RFC 7230 Section 4.1 mandates a 0 size last chunk be sent to indicate end of the entity body. hgweb was failing to send this last chunk. As a result, properly written HTTP clients would assume more data was coming and they would likely time out waiting for another chunk to arrive. Mercurial's own test harness was paving over the improper HTTP behavior by not attempting to read the response body if the status code was 500. This incorrect workaround was added in ba6577a19656 and has been removed with this patch.

tests: test rebase of merge of target ancestors

tests: test coverage of parent order with p1 outside rebase set

tests: add test for rebasing merges with ancestors of the rebase destination This shows sub-optimal behaviour. The user gets a merge prompt that it is very hard to justify.

merge: 0 is a valid ancestor different from None Most internal functions can take either a hash or an integer. Merge did however not handle 0 as revision 0. Now it does.

merge with stable

revert: look for copy information for all local modifications Renaming a file over an existing one marks the file as modified. So we track rename source in modified file too.

rename: properly report removed and added file as modified (issue4458) The result of 'hg rm' + 'hg rename' disagreed with the one from 'hg rename --force'. We align them on 'hg move --force' because it agrees with what 'hg status' says after the commit. Stopping reporting a modified file as added puts an end to the hg revert confusion in this situation (issue4458). However, reporting the file as modified also prevents revert from restoring the copy source. We fix this in a later changeset. Git diff also stop reporting the add in the middle of the chain as add. Not sure how important (and even wrong) it is.

manifest: fix a bug where working copy file 'add' mark was buggy Because the same dictionary was used to (1) get node from parent and (2) store annotated version, we could end up with buggy values. For example with a chain of renames: $ hg mv b c $ hg mv a b The value from 'b' would be updated as "<old-a>a", then the value of c would be updated as "<old-b>a'. With the current dictionary sharing this ends up with: '<new-c>' == '<old-a>aa' This value is double-wrong as we should use '<old-b>' and a single 'a'. We now use a read-only value for lookup. The 'test-rename.t' test is impacted because such a chained added file is suddenly detected as such.

merge: be precise about what merged into what in short desc

docker: add Docker files for running an Apache mod_wsgi server I frequently find myself wanting to run hgweb in a production-like environment, with a real HTTP server and multiple WSGI workers. This patch introduces a Docker environment for running Mercurial under Apache + mod_wsgi. With just a few command executions, it is possible to spin up a Docker container running hgweb. The container is tailored for Mercurial developers wanting to run Mercurial from a source checkout. It is **not** meant to be something suitable for production use. The container provides a default hgweb environment with an empty repository that allows pushes. You can thus start a container and push your favorite repository there for quick testing. The container is designed to allow customizations. Users can provide their own hgweb configurations and mount existing directories containing repositories into the container. The behavior of the container and how to control things is documented in the README.rst file.

merge: use None as filename for base in 'both created' conflicts Instead of using a file that we know is not in the common ancestor's maniffest, let's use None. This is safe as the only place that cares about the value (applyupdates) already checks if the item exists in the ancestor.

merge: break out "both renamed a -> b" case We can further limit the scope of the 2-way merge case by breaking out the case where the file was not created from scratch on both sides but rather renamed in the same way (and is therefore a 3-way merge). This involves copying some code, but it makes it clearer which case the "Note:" in the code refers to.

merge: separate out "both created" cases When 'f' is not in 'ma', 'a' will be 'nullid' and all the if/elif conditions that check whether some one nodeid is equal to 'a' will fail, and the else-clause will instead apply. We can make that more explicit by creating a separate 'm' action for the case where 'a' is 'nullid'. While it does mean copying some code, perhaps it makes it a little clearer which codepaths are possible, and which cases the "Note:" in the code refers to. It also lets us make the debug action messages a little more specific.

merge: indent to prepare for next patch

merge with stable

factotum: convert Plan 9 quoted string to Python string

plan9: setup check for current python build

add: check for the existence of a file matched inexactly before adding it The change in 10697f29af2b created a problem on Windows and OS X: --- /usr/local/mercurial/tests/test-issue660.t +++ /usr/local/mercurial/tests/test-issue660.t.err @@ -47,6 +47,8 @@ Should succeed - shadow removed: $ hg add b + adding b/b + b/b does not exist! Prior to the failing 'hg add', the file 'b/b' was added and committed, then 'b' was recursively deleted from the filesystem, file 'b' was created and the delete was recorded with 'hg rm --after'. This add is attempting to record the existence of file 'b'. A filesystem that is not case sensitive prevents dirstate.walk() from skipping its step 3, and step 3 has the effect of inserting removed files into the walk list. The Linux code doesn't run through step 3, and didn't exhibit the problem. It's not clear why a non case sensitive filesystem triggers step 3, given that the path normalization occurs in step 2. Prior to 10697f29af2b, part of the check here was 'f not in repo.dirstate' instead of 'f not in wctx'. Files in the 'r' state are filtered out of context.__contains__() but not dirstate.__contains__(). Therefore the removed file name wasn't added to the list of files to add when checking against dirstate. That change was to allow removed files to be readded, but adding a file that doesn't exist is nonsensical. If the user specifies a missing file, it will be an exact match and will still fail.

tests: choose the proper environment variable style for the platform Windows was printing out 'commit $HG_NODE' instead of the actual hash.

tests: fix another multi-statement hook for Windows The double quotes are necessary, otherwise it tries to pipe into a program named 'short'. An '&' could serve as a command separator on Windows instead of ';', but I don't see any obvious way to swap these depending on the platform. In this case though, there really wasn't a need for multiple statements.

run-tests: set a default largefiles usercache in the default hgrc file This fixes a test failure introduced in 4be754832829 on Windows and OS X, where the cached largefile wasn't being deleted because the named .cache directory didn't exist. It only existed on Linux because the test suite sets $HOME to the directory of the test being run, and Linux uses $HOME/.cache by default. Most of the other largefiles tests explicitly set this value at the top of their scripts, but test-largefiles-update.t didn't pick that up when it was created. Those scripts that do set a value will override this. We could just set the parameter in the test-largefiles-update.t script, but there are a few other non obvious tests that exercise largefiles too. These largefiles end up being cached in the user's real cache, so proper hygiene dictates that this not be left to each individual test script.

merge: remove obsolete check for untracked files in 'dm' action Since 0776a6cababe (merge: don't use unknown(), 2012-02-09), untracked files are no longer included in the manifest diff, so there is no need to check exclude them when renaming files for directory moves with the 'dm' action.

merge: remove dead assignment in applyupdates()

merge: move calculateupdates() before applyupdated() calculateupdates() happens before applyupdates(), so move it before in the code. That also moves it close to manifestmerge(), which is a good location as calculateupdates() is the only caller of manifestmerge().

merge: remove unused variables from _checkcollision()

largefiles: avoid exec-bit examination on the platform being unaware of it Changeset 24600c9d7f4e introduced the examination of exec bit of largefiles in "hg status --rev REV" case, but it doesn't avoid it on the platform being unaware of exec-bit (e.g. on NTFS of Windows).

merge with stable

changegroup: fix file linkrevs during reorders (issue4462) Previously, if reorder was true during the creation of a changegroup bundle, it was possible that the manifest and filelogs would be reordered such that the resulting bundle filelog had a linkrev that pointed to a commit that was not the earliest instance of the filelog revision. For example: With commits: 0<-1<---3<-4 \ / --2<--- if 2 and 3 added the same version of a file, if the manifests of 2 and 3 have their order reversed, but the changelog did not, it could produce a filelog with linkrevs 0<-3 instead of 0<-2, which meant if commit 3 was stripped, it would delete that file data from the repository and commit 2 would be corrupt (as would any future pulls that tried to build upon that version of the file). The fix is to make the linkrev fixup smarter. Previously it considered the first manifest that added a file to be the first commit that added that file, which is not true. Now, for every file revision we add to the bundle we make sure we attach it to the earliest applicable linkrev.

merge: consistently use single quotes for non-user-facing strings Because I'm getting tired of searching for both 'O' and "O".

localrepo: add a currenttransaction method This method returnx the current transaction or None: it will allow a cache writer to hook in an existing transaction.

repoview: extract actual hidden cache writing in its own function This will allow the generation of this cache within the transaction. Relying on the transaction will reduce the chance of reader seeing bad cache.

templates: fix broken "less" & "more" links in paper style (issue4460) "/search", which is an invalid command in hgweb, was mistakenly used for "[show] more [revsets]" and "[show] less [revsets]" links on search page in templates "paper" (and those which inherit paper, such as coal) before and worked fine until 6e1fbcb18a75, which made hgweb more strict about invalid commands.

push: stop independent usage of bundle2 in syncphase (issue4454) The phase-syncing code was using bundle2 if the remote supported it. It was doing so without regard to bundle2 activation on the client. Moreover, the phase push is now properly included in the unified bundle2 push, so having extra code in syncphase should be useless. If the remote is bundle2-enabled, the phases should already be synced. The buggy verification code was leading to a crash when a 3.2 client was pushing to a 3.1 server. The real bundle2 path detected that their versions were incompatible, but the syncphase code failed to, sending an incompatible bundle2 to the server. We drop the useless and buggy code as a result. The "else" clause is de-indented in the process.

dirstate: speed up repeated missing directory checks In a mozilla repo with tip at bb3ff09f52fe, hg update tip~1000 && time hg revert -nq -r tip . displays ~4:20 minutes. With tip~100, it runs in ~11 s. With revision 100000, it did not finish in 12 minutes. Revert calls dirstate.status() with a matcher that matches each file in the target revision. The main problem [1] lies in dirstate._walkexplicit(), which looks for matching deleted directories by checking whether each path is prefix of any path in the dirstate. With m files in the dirstate and n files in the target revision that are not in the dirstate, this is clearly O(m*n). Let's improve by keeping a lazily initialized set of all the directories in the dirstate, so the time becomes O(m+n). After this patch, the 4:20 minutes become 5.5 s, while for a single missing path, it slows down from 1.092 s to 1.150 s (best of 4). The >12 min case becomes 5.8 s. [1] A narrower optimization would be to make revert take the fast path for '.' and '--all'.

revert: access status fields by name rather than index For better readability.

subrepo: remove "_getstorehashcachepath" referred by no other code paths

subrepo: replace direct file APIs around "writelines" by "vfs.writelines" This patch also replaces "self._getstorehashcachepath" (building absolute path up) by "self._getstorehashcachename" (building relative path up), because "vfs.writelines" requires relative path.

vfs: add "writelines" This patch allows "writelines" to take "mode" and "notindexed" arguments, because subsequent patch for subrepo requires both.

vfs: add "notindexed" argument to invoke "ensuredir" with it in write mode This patch uses "False" as default value of "notindexed" argument, even though "vfs.makedir()" uses "True" for it, because "os.mkdir()" doesn't set "_FILE_ATTRIBUTE_NOT_CONTENT_INDEXED" attribute to newly created directories.

subrepo: replace direct file APIs around "readlines" by "vfs.tryreadlines" This patch also replaces "self._getstorehashcachepath" (building absolute path up) by "self._getstorehashcachename" (building relative path up), because "vfs.tryreadlines" requires relative path. This patch makes "_readstorehashcache()" return "[]" (returned by "vfs.tryreadlines()"), when cache file doesn't exist, even though "_readstorehashcache()" returned '' (empty string) in such case before this patch. "_readstorehashcache()" is invoked only by the code path below in "_storeclean()": for filehash in self._readstorehashcache(path): if filehash != itercache.next(): clean = False break In this case, "[]" and '' don't differ from each other, because both of them cause avoiding iteration of "for loop".

vfs: add "readlines" and "tryreadlines" This patch allows "readlines" and "tryreadlines" to take "mode" argument, because "subrepo" requires to read files not in "rb" (binary, default for vfs) but in "r" (text) mode in subsequent patch.

subrepo: add "_cachestorehashvfs" to handle cache store hash files via vfs This "vfs" object will be used by subsequent patches to handle cache store hash files without direct file APIs. This patch decorates "_cachestorehashvfs" with "@propertycache" to delay vfs creation, because it is used only for cooperation with other repositories. In this patch, "/" is used as the path separator, even though "self._repo.join" uses platform specific path separator (e.g. "\\" on Windows). But it is reasonable enough, because "store" and other management file handling already include such implementation, and they work well.

subrepo: remove "_calcfilehash" referred by no other code paths

subrepo: replace "_calcfilehash" invocation by "vfs.tryread" "_calcfilehash" can be completely replaced by simple "vfs.tryread" invocation. def _calcfilehash(filename): data = '' if os.path.exists(filename): fd = open(filename, 'rb') data = fd.read() fd.close() return util.sha1(data).hexdigest() Building absolute path "absname" up by "self._repo.join" for files in "filelist" is avoided, because "vfs.tryread" does so internally.

subrepo: replace "os.path.exists" by "exists" via wvfs of the parent Existance of specified "path" should be examined by "exists" via wvfs of the parent repository, because the working directory of the parent repository may be in UTF-8 mode. Wide API should be used via wvfs in such case. In this patch, "/" is used as the path separator, even though "path" uses platform specific path separator (e.g. "\\" on Windows). But it is reasonable enough, because "store" and other management file handling already include such implementation, and they work well.

subrepo: avoid redundant "util.makedirs" invocation "util.makedirs" for the (sub-)repository root of "hgsubrepo" is also executed in the constructor of "localrepository", if "create" is True and ".hg" of it doesn't exist. This patch avoids redundant "util.makedirs" invocation in the constructor of "hgsubrepo".

merge: remove confusing comment about --force manifestmerge() has a piece of code that's roughly: if not force and different: abort else: # if different: old untracked f may be overwritten and lost ... The comment only talks about what happens when 'different' is true, and in combination with the if-block above, that must mean that it is only about what happens when 'force and different'. It seems quite fine that files are overwritten when 'force' is true, so let's remove the comment. As it stands, it can easily be interpreted as a TODO (which is how I interpreted it at first).

phases: read pending data when appropriate If we are called by a hook and pending data exists, read those.

bookmark: read pending data when appropriate If we are called by a hook and pending data exists, read it.

test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.

transaction: write pending generated files Such file are generated with a .pending prefix. It is up to the reader to implement the necessary logic for reading pending files. We add a test to ensure pending files are properly cleaned-up in both success and error cases.

transaction: have _generatefile return a boolean The function returns True if any files were generated. This will be used to know if any pending files have been written.

transaction: allow generating files with a suffix This will allow us to generate temporary pending files. Files generated with a suffix are assumed temporary and will be cleaned up at the end of the transaction.

transaction: fix some docstring grammar

transaction: accept a 'location' argument for registertmp This will allow generation of temporary files outside of store. This will be useful for bookmarks.

tests: handle differences between missing file error strings on Windows and Unix

run-tests: don't warn on unnecessary globs mandated by check-code.py When test output is processed, if os.altsep is defined (i.e. on Windows), TTest.globmatch() will cause a warning later on if a line has a glob that isn't necessary. Unfortunately, the regex checking in check-code.py doesn't have this context. Therefore we ended up with cases where the test would get flagged with a warning only on Windows because a glob was present, because check-code.py would warn if it wasn't. For example, from test-subrepo.t: $ hg -R issue1852a push `pwd`/issue1852c pushing to $TESTTMP/issue1852c (glob) The glob isn't necessary here because the slash is shown as it was provided. However, check-code mandates one to handle the case where the default path has backslashes in it. Break the cycle by checking against a subset of the check-code rules before flagging the test with a warning, and ignore the superfluous glob if it matches a rule. This change fixes warnings in test-largefiles-update.t, test-subrepo.t, test-tag.t, and test-rename-dir-merge.t on Windows. I really hate that the rules are copy/pasted here (minus the leading two spaces) because it would be nice to only update the rules once, in a single place. But I'm not sure how else to do it. I'm open to suggestions. Splitting some of the rules out of check-code.py seems wrong, but so does moving check-code.py out of contrib, given that other checking scripts live there. There are other glob patterns that could be copied over, but this is enough to make the current tests run on Windows.

update: remove unnecessary check for unknown files with --check As far as I and the test suite can tell, the checks in manifestmerge() already report the errors (whether or not --check is given), so we don't need to call merge.checkunknown(). Since this is the last call to the method, also remove the method.

merge with stable

tests: move a multi-statement debuglocks hook into a shell script for Windows Before this patch, a part of "test-push-hook-lock.t" fails unexpectedly on Windows environment, because semicolon (";") isn't recognized as the command separator by "cmd.exe". This is fixed the same way as a similar issue in 7c253c23de3b.

tests: fix globs for Windows test-largefiles-update.t, test-subrepo.t, test-tag.t, and test-rename-dir-merge.t still warn about no result returned because of unnecessary globs that test-check-code-hg.t wants, relating to output for pushing to, pulling from and moving X to Y.

run-tests: include quotes in the HGEDITOR value when storing sys.executable This fixes test-install.t on Windows that broke in 2122b82b6987 when shlex.split() was added to the debuginstall command: @@ -7,8 +7,11 @@ checking installed modules (*mercurial)... (glob) checking templates (*mercurial?templates)... (glob) checking commit editor... + Can't find editor 'c:\Python27\python.exe -c "(omitted)"' in PATH + (specify a commit editor in your configuration file) checking username... - no problems detected + 1 problems detected, please check your install! + [1] What happens is that shlex.split() on Windows turns this: c:\Python27\python.exe -c "import sys; sys.exit(0)" into this: ['c:Python27python.exe', '-c', 'import sys; sys.exit(0)'] While technically a regression, most programs on Windows live in some flavor of 'Program Files', and therefore the environment variable needs to contain quotes anyway to handle the space. This wasn't handled prior to the shlex() change, because it tested the whole environment variable to see if it was an executable, or split on the first space and tested again.

mq: when adding headers in plain mode, separate them from message (issue4453) c87f2a5a6e49 did a clean-up in one direction ... but we want it in the other direction.

mq: introduce insertplainheader - same naive implementation as before

mq: when setting message in plain mode, separate it from header (issue4453) Fix inconsistent handling of plain header separation in mq patcheader - and contrary to c87f2a5a6e49, do it in the direction of having an empty line between header and description. Plain patches are like mails and should thus have an empty line between headers and body in compliance with RFC 822 3.1.

setdiscovery: avoid a full changelog graph traversal We were definitely being suboptimal here: we were constructing two full sets, one with the full set of common nodes (i.e. a graph traversal) and one with all nodes. Then we subtract one set from the other. This whole process is O(commits) and causes discovery to be significantly slower than it should be. Instead, keep track of common incrementally and keep undecided as small as possible. This makes discovery massively faster on large repos: on one such repo, 'hg debugdiscovery' over SSH with one commit missing on the client and five on the server went from 4.5 seconds to 1.5. (An 'hg debugdiscovery' with no commits missing on the client, i.e. connection startup time, was 1.2 seconds.)

ancestor: add a way to remove ancestors of bases from a given set This and missingancestors can share state, which will turn out to be perfect for set discovery.

ancestor: add a way to add to bases of a missing ancestor object This will be useful for setdiscovery, since with that we incrementally add to our knowledge of common nodes.

ancestor: add a way to test whether a missing ancestor object has bases This is pretty trivial so there's no unit test coverage for it. This will be used by setdiscovery.

ancestor: remove now-unused missingancestors function Callers should use revlog.incrementalmissingrevs instead.

revlog: switch findmissing* methods to incrementalmissingrevs This will allow us to remove ancestor.missingancestors in an upcoming patch.

revlog: add a method to get missing revs incrementally This will turn out to be useful for discovery.

test-ancestor: add support for multiple tests against one incremental object In upcoming patches we'll add more operations to the object, and this prepares for testing those operations.

test-ancestor: move naive missing ancestor algorithm into a class This mirrors the change to the real missing ancestor algorithm in a previous patch.

ancestor.missingancestors: turn into a state-keeping class This allows multiple efficient missing ancestor queries against the same set of bases. In upcoming patches we'll also define ways to grow the set of bases. The fact that the test output hasn't changed establishes this patch's correctness.

ancestor.missingancestors: calculate start point after filtering revsvisit Any revs that are filtered out are also in basesvisit, which means they wouldn't be returned in the missing list anyway. There's no need to explore such revs or their ancestors. The 'if not revsvisit' check moves down because we can't call max() on an empty set.

ancestor.missingancestors: don't discard from basesvisit We only actually care about whether revsvisit is empty, so we can let basesvisit grow to arbitrary size. It turns out that this actually helps performance. For a large repo with hundreds of thousands of commits, hg perfrevset 'only(0, tip)' (basically the worst case, involving a full DAG traversal) goes from 1.63 seconds to 1.50. hg perfrevset 'only(tip, 0)' remains unchanged at 1.98 seconds.

test-ancestor: use random testing for missing ancestors We're going to make changes to the missing ancestor algorithm, and random testing will give us much more confidence than a fixed set of tests.

test-ancestor: define a main function We're going to add to it in upcoming patches.

test-ancestor: test iteration for lazyancestors This has some test coverage in test-revlog-ancestry.py, but not very much.

ancestor.lazyancestors: take parentrevs function rather than changelog Principle of least privilege, and it also brings this in line with missingancestors.

remove: avoid a bogus warning about no tracked files when removing '.' Previously, any files relative to the root of the repo that match the -I patterns would be deleted, but the command exited with 1 after printing a warning: $ hg remove -S -I 're:.*.txt' . removing sub1/sub2/folder/test.txt removing sub1/sub2/test.txt not removing .: no tracked files

remove: support remove with explicit paths in subrepos

remove: recurse into subrepositories with --subrepos/-S flag Like 'forget', git and svn subrepos are currently not supported. Unfortunately the name 'remove' is already used in the subrepo classes, so we break the convention of naming the subrepo function after the command.

cmdserver: protect pipe server streams against corruption caused by direct io Because pipe-mode server uses stdio as IPC channel, other modules should not touch stdio directly and use ui instead. However, this strategy is brittle because several Python functions read and write stdio implicitly. print 'hello' # should use ui.write() # => ch = 'h', size = 1701604463 'ello', data = '\n' This patch adds protection for such mistakes. Both stdio files and low-level file descriptors are redirected to /dev/null while command server uses them.

cmdserver: postpone creation of pipe server until run() This makes it easy to swap file descriptors while running command server.

cmdserver: use given streams as pipe channels like other commands Because commandserver itself is an hg subcommand, it shouldn't use stdio directly in principle.

revset.only: use cl.findmissingrevs ancestor.missingancestors is really an implementation detail.

manifestmerge: use already existing fl2 synonym for m2.flags(f) Probably not a noticeable performance gain, but shortens the code slightly.

merge: drop underscore prefix from _checkunknown() The method has been called from commands.py since 3eab42088be4 (update: just merge unknown file collisions, 2012-02-09), so drop the underscore prefix that suggests that it's private.

transaction: drop special handling for phases and bookmarks generation We are still doing double backups, but now that we have proper location handling this is less of an issue. Dropping this simplifies the code before we add some pending-related logic. This also ensures we actually test the new 'location' mechanism.

transaction: use 'location' instead of 'vfs' objects for file generation The argument is now a location name. The location must be present in the 'vfsmap' provided to the transaction at creation time.

transaction: use 'location' instead of 'vfs' in the addbackup method This unlock the backup of file outside of store (eg: bookmarks).

addbackup: handle file in subdirectory The current naming scheme ('journal.backups.<file>') resulted is bad directory name when 'file' was in a subdirectory. We now extract the directory name and create the backupfile within it. We plan to use file in a subdirectory for cachefile.

addbackup: use the vfs for the backup destination too The backup file location was always computed using the opener, bypassing the 'location' setting. (And making the feature broken.)

transaction: set backupentries version to proper value Now that all mechanisms are in place, we can advertise it with a proper new version.

transaction: support cache file in backupentries We do not want to abort if anything wrong happen while handling a cache file. Cache file have way to be invalidated and if old/bad version stay no misbehavior will happen. Proper value will eventually be computed and the wrong will be righten. This changeset use the transaction reporter (usually writing on stderr) to write details about failed cache handling. This will only apply to write operation using a transaction. The usual update during read only operation will stay a debug message. I was on the way to bring these message back to debug level when I realised it could be a feature. People with write access to the repository are likely to have the power to fix error related to cache (and it is valuable to fix them). So let the things as is for now.

transaction: use the location value when doing backup We finally use the 'location' value coupled with the 'vfsmap' to restore backup for the right file.

transaction: pass a vfs map to the transaction The goal is to allow access to file outside ofthe store directory from the transaction. The obvious target are the `bookmarks` file. But we can envision usage for cache too. We keep passing a main opener explicitly because a lot of code rely on this default opener. The main opener (operating on store) is using an empty key ''.

transaction: change the on disk format for backupentries We need to store new data to improve the current transaction logic: - location: We want to generate and backup file outside of the 'store' (eg: bookmarks, or various cache files). This requires knowing and preserving where each file is located. The value of this new field is a string. It will be used as a key for a vfs mapping. - cache: We would like to handle cache file in the transaction code. This Will help to have cache consistent with the repository state and avoid performance issue on big repository like Mozilla. However, failure to handle cache file should not result in a transaction failure. We add a new field that carry this information. The value is boolean, A True value mean any error while handling this file can be ignored. Those two mechanisms are not implemented yet, but they are now persisted in the on disk file. Support for new mechanisms is coming in later changeset. We update the file format now and will introduce the new features in later changeset. The format version is set to 0 until we actually support the new feature. This will prevent misunderstanding between incomplete and final client. Support for reading both version 1 and (future) version 2 could be achieved (using default value when reading version 1) but has not been seen as necessary for now.

silenttestrunner: add environment variable to make tests noisy again As I've been working on complicated extension code it's been handy to be able to get standard unittest verbose output so I can find crashers more efficiently.

largefiles: update comments to refer to the right overridden method This cleans up leftovers from b228ad1f79d7 (largefiles: override calculateupdates instead of manifestmerge, 2014-03-02).

revlog: cache chain info after calculating it for a rev (issue4452) This dumb cache works surprisingly well: on a repository with typical delta chains ~50k in length, unbundling a linear series of 5000 revisions (changelogs and manifests only) went from 60 seconds to 3.

manifest: add matches() method Move the code in context._manifestmatches() into a new manifest.matches(). It's a natural place for the code to live and it allows other callers to easily use it. It should also make it easier to optimize the new method in alternative implementations of the manifest (same reasoning as with manifest.diff()).

context.status: pass status tuple into _buildstatus By passing a status tuple (instead of the current list), we can access the status fields by name and make it a little more readable.

context.status: avoid de- and reconstructing status tuple We can just modify the status tuple we got from dirstate.status() instead of deconstructing it and constructing a new instance, thereby simplifying the code a little.

context.status: make _dirstatestatus() return an status tuple Letting _dirstatestatus() return an scmutil.status instance also means that _buildstatus() will always return such an instance, so we can remove the conversion from the call sites.

context.status: wipe deleted/unknown/ignored fields when reversed It makes no sense to request reverse status (i.e. changes from the working copy to its parent) and then look at the deleted, unknown or ignored fields. If you do, you would get the result from the forward status (changes from parent to the working copy). Instead of giving a nonsensical answer to a nonsensical question, it seems a little saner to return empty lists. It might be best if we could prevent the caller accessing these lists, but it's doubtful it's worth the trouble.

patch.trydiff: add support for noprefix

mdiff.unidiff: add support for noprefix

diff: add a --noprefix option See previous patch descriptions for the motivation. The tests reflect the current state of the world -- as we add support we'll see changes in the test output.

patch.diffopts: add support for noprefix In an upcoming patch we'll enable support as an option to 'hg diff' as well. The tests reflect the current state of the world -- as we add support we'll see changes in the test output.

patch.diffopts: allow a setting to be forced in plain mode Upcoming patches will add an option that will almost certainly break diff output parsers when enabled. Add support for forcing an option to something in plain mode, as a fallback. Options passed in via the CLI are not affected, though -- it is assumed that any script passing the option in explicitly knows what it is doing.

patch.diffopts: break get function into if statements We're going to add another condition here, and with the current structure that becomes just too confusing.

mdiff.diffopts: add a new noprefix option By popular demand, we introduce an option to disable the 'a/' and 'b/' prefixes in diff output. This makes copying and pasting filenames from diff output easier. This option will be implemented and documented in upcoming patches. To ensure that existing scripts that parse output don't break, we will ensure that this prefix is disabled in plain mode. A straight 'hg export | hg import' without HGPLAIN=1 will still be broken though, but there's little that can be done about that.

mdiff.diffopts: add doc comment for nobinary

changelog: register changelog.i.a as a temporary file The file is registered to make sure the transaction is cleaned up in all cases.

transaction: allow registering a temporary transaction file During the transaction, files may be created to store or expose data involved in the transaction (eg: changelog index data are written in a 'changelog.i.a' for hooks). But we do not have an official way to record such file creation and make sure they are cleaned up. The lack of clean-up is currently okay because there is a single file involved and a single producer/consumer. However, as we want to expose more data (bookmarks, phases, obsmarker) we need something more solid. The 'backupentries' mechanism could handle that. Temporary files can be encoded as a backup of nothing '('', <temporarypath>)'. We "need" to attach it to the same mechanism as we use to be able to use temporary transaction files outside of .'store/' and 'backupentries' is expected to gain such feature. This changeset makes it clear that we should rename 'backupentries' to something more generic.

transaction: always generate file on close The conditionnal was buggy and file were only generated if "onclose" was defined. By luck, "onclose" was always defined.

remove: move most of the implementation into cmdutils.remove() This will allow access to the reusable parts from subrepos, similar to add(), forget(), etc.

revlog: increase I/O bound to 4x the amount of data consumed This doesn't affect normal clones since they'd be bound by the CPU bound below anyway -- it does, however, improve generaldelta clones significantly. This also results in better deltaing for generaldelta clones -- in generaldelta clones, we calculate deltas with respect to the closest base if it has a higher revision number than either parent. If the base is on a significantly different branch, this can result in pointlessly massive deltas. This reduces the number of bases and hence the number of bad deltas. Empirically, for a highly branchy repository, this resulted in an improvement of around 15% to manifest size.

revlog: bound based on the length of the compressed deltas This is only relevant for generaldelta clones.

revlog: compute length of compressed deltas along with chain length In upcoming patches to the revlog, we're going to split up the notions of bounding I/O and bounding CPU.

revlog: store fulltext when compressed delta is bigger than it This is a very silly case and not particularly likely to happen in the wild, but it turns out we can hit it in a couple of places. As we tune the storage parameters we're likely to hit more such cases. The affected test cases all have smaller revlogs now.

revlog: make a predicate clearer with parens

transaction: extract backupentry registration in a dedicated function We are about to use the 'backupentry' mechanism to allow cleaning up transaction-related temporary files (such as 'changelog.i.a'). We start by extracting the entry registration into its own method for easy reuse. At that point, I would like to rename the backup-file related variable to something generic but I'm a bit short of ideas.

transaction: pass the transaction to 'postclose' callback This mirrors the API for 'pending' and 'finalize' callbacks. I do not have immediate usage planned for it, but I'm sure some callback will be happy to access transaction related data.

transaction: pass the transaction to 'finalize' callback The callback will likely need to perform some operation related to the transaction (eg: registering file update). So we better pass the current transaction as the callback argument. Otherwise callback that needs it has to rely on horrible weak reference trick. This allow already allow us to slay a wild weak reference usage.

transaction: pass the transaction to 'pending' callback The callback will likely need to perform some operation related to the transaction (eg: backing files up). So we better pass the current transaction as the callback argument. Otherwise callback that needs it has to rely on horrible weak reference trick. The first foreseen user of this is changelog._writepending. We would like it to register the temporary file it create for cleanup purpose.

transaction: gather backupjournal logic together in the __init__ The initialisation of file-backup related variable were a bit scattered, we gather them together.

transaction: handle missing file in backupentries (instead of using entries) The case where a backup of a missing file was requested was previously handled by the 'entries' list. As the 'backupentries' is about to gain ability to backup files outside of '.hg/store', we want it to be able to handle the missing file too. Reminder: using 'addbackup' on a missing file means that such file needs to be deleted if we rollback the transaction.

test-hup: use ls instead of echo Having one file entry per line makes it easier to analyse diffs.

largefiles: move "copyalltostore" invocation into "markcommitted" Before this patch, while "hg convert", largefiles avoids copying largefiles in the working directory into the store area by combination of setting "repo._isconverting" in "mercurialsink{before|after}" and checking it in "copytostoreabsolute". This avoiding is needed while "hg convert", because converting doesn't update largefiles in the working directory. But this implementation is not efficient, because: - invocation in "markcommitted" can easily ensure updating largefiles in the working directory "markcommitted" is invoked only when new revision is committed via "commit" of "localrepository" (= with files in the working directory). On the other hand, "commitctx" may be invoked directly for in-memory committing. - committing without updating the working directory (e.g. "import --bypass") also needs this kind of avoiding For efficiency of this kind of avoiding, this patch does: - move "copyalltostore" invocation into "markcommitted" - remove meaningless procedures below: - hooking "mercurialsink{before|after}" to (un)set "repo._isconverting" - checking "repo._isconverting" in "copytostoreabsolute" This patch invokes "copyalltostore" also in "_commitcontext", because "_commitcontext" expects that largefiles in the working directory are copied into store area after "commitctx". In this case, the working directory is used as a kind of temporary area to write largefiles out, even though converted revisions are committed via "commitctx" (without updating normal files).

largefiles: avoid printing messages while transplanting by "_lfstatuswriters" Putting "lambda *msg, **opts: None" (= avoid printing messages always) into "_lfstatuswriters" while transplanting makes explicit passing "printmessage = False" for "updatelfiles()" useless. This patch also removes setting/unsetting "repo._istransplanting" in "overridetransplant", because there is no code path referring it.

largefiles: update standins only at the 1st commit of "transplant --continue" Before this patch, "hg transplant --continue" may record incorrect standins, because largefiles extension always avoid updating standins while transplanting, even though largefiles in the working directory may be modified manually at the 1st commit of "hg transplant --continue". But, on the other hand, updating standins should be avoided at subsequent commits for efficiency reason. To update standins only at the 1st commit of "hg transplant --continue", this patch uses "automatedcommithook", which updates standins by "lfutil.updatestandinsbymatch()" only at the 1st commit of resuming. Even after this patch, "repo._istransplanting = True" is still needed to avoid some status report while updating largefiles in "lfcommands.updatelfiles()". This is reason why this patch omits not "repo._istransplanting = True" in "overriderebase" but examination of "getattr(repo, "_istransplanting", False)" in "updatestandinsbymatch".

largefiles: avoid redundant "updatelfiles" invocation in "overridetransplant" At "hg transplant --merge REV", largefiles newly coming from the 2nd parent (= REV) are marked as "a"(dded) by "patch.patch()", and have to be marked as "n"(ormal) after commit. But until changeset 3100d1cbce32, such largefiles were still marked as "a" unexpectedly even after commit, because no additional entry is added to filelog of such largefiles and they aren't listed in "repo[newnode].files()" in this case: "newnode" is one of newly committed changeset (= result of "repo.commit()"). "updatelfiles" invocation in "overridetransplant" shadows this problem by forcibly synchronizing lfdirstate to dirstate. Now, "updatelfiles" invocation in "overridetransplant" is redundant, because changeset 3100d1cbce32 made "markcommitted" use "ctx.files()" to get targets of "synclfdirstate" instead of "repo[newnode].files()".

merge with stable

util.system: remove unused handling of onerr=ui In our code, onerr is None or util.Abort. It smells bad to overload ui and exception class.

util.system: use ui.system() in place of optional ui.fout parameter

ui: introduce util.system() wrapper to make sure ui.fout is used This change is intended to avoid future problem of data corruption under command server. out=ui.fout is mandatory as long as command server uses stdout as IPC channel.

hook: remove redundant code to redirect http hook output to client stream out=ui and out=ui.fout should be the same here. ui.fout was introduced at afccc64eea73, which was not available when out=ui was added at c37f35d7f2f5.

hgk: forward command output to ui.fout consistently Nobody would want to run hgk in command server, but it should work in principle. This fixes possible data corruption of command-server channel.

merge with stable

Added signature for changeset 643c58303fb0

Added tag 3.2.1 for changeset 643c58303fb0

run-tests: use a try/except ladder instead of looking for a specific version This ensures we get json instead of simplejson in as many places as possible.

hghave: use a less brittle have-json check

sortdict: add insert method Future patches will allow extensions to choose which order a namespace should output in the log, so we add a way for sortdict to insert to a specific location.

sortdict: add iteritems method Future patches will start using sortdict for log operations where order is important. Adding iteritems removes the headache of having to remember to use items() if the object is a sortdict.

addremove: add back forgotten files (BC) After running "hg forget README && hg addremove", README will still be reported as removed, while "hg forget README && hg add README" adds it back so it gets reported as clean. It seems like they should behave the same. Furthermore, it seems like no files should remain untracked after 'hg addremove && hg commit' (or 'hg commit -A'). For these reasons, change the behavior of addremove so it does add forgotten files back. The problem is with scmutil._interestingfiles(), which reports the file as removed, so scmutil.addremove() does not add it. Fix by teaching _interestingfiles() to report forgotten files separately from removed files and make addremove() add forgotten files back. However, do not treat forgotten files as sources for rename detection. Note that since removed and forgotten files are treated the same before this change, forgotten files were considered sources for rename detection. Also update the other caller, marktouched(), in the same way as addremove().

add: add back forgotten files even when not matching exactly (BC) I accidentally did 'hg forget .' and tried to undo the operation with 'hg add .'. I expected the files to be reported as either modified or clean, but they were still reported as removed. It turns out that forgotten files are only added back if they are listed explicitly, as shown by the following two invocations. This makes it hard to recover from the mistake of forgetting a lot of files. $ hg forget README && hg add README && hg status -A README C README $ hg forget README && hg add . && hg status -A README R README The problem lies in cmdutil.add(). That method checks that the file isn't already tracked before adding it, but it does so by checking the dirstate, which does have an entry for forgotten files (state 'r'). We should instead be checking whether the file exists in the workingctx. The workingctx is also what we later call add() on, and that method takes care of transforming the add() into a normallookup() on the dirstate. Since we're changing repo.dirstate into wctx, let's also change repo.walk into wctx.walk for consistency (repo.walk calls wctx.walk, so we're simply inlining the call).

context.status: explain "caching reasons" more fully Where we "load earliest manifest first for caching reasons", elaborate on what "caching reasons" refers to. Text provided by Matt in http://thread.gmane.org/gmane.comp.version-control.mercurial.devel/73235/focus=73578.

localrepo: rename revlog.maxchainlen to format.maxchainlen This is more consistent with other option names, as spotted by Pierre-Yves. Thanks!

revlog: add config variable for limiting delta-chain length The current heuristic for deciding between storing delta and full texts is based on ratio of (sizeofdeltas)/(sizeoffulltext). In some cases (for example a manifest for ahuge repo) this approach can result in extremely long delta chains (~30,000) which are very slow to read. (In the case of a manifest ~500ms are added to every hg command because of that). This commit introduces "revlog.maxchainlength" configuration variable that will limit delta chain length.

debugrevlog: fix computing chain length in debugrevlog -d The chain length was computed correctly only when generaldelta feature was enabled. Now it's fixed. When generaldelta is disabled the base revision in revlog index is not the revision we have delta against - it's always previous revision. Instead of incorrect chainbaseandlen in command.py we are now using two single-responsibility functions in revlog.py: - chainbase(rev) - chainlen(rev) Only chainlen(rev) was missing so it was written to mimic the way the chain of deltas is actually found during file reconstruction.

transaction: factorise append-only file registration The addition is done in two different places but differs slightly. We factorise this addition to ensure it is consistent in all places.

transaction: document `tr.add`

transaction: drop backupentries logic from startgroup and endgroup The `startgroup` and `endgroup` methods are used in a very specific context to wrap a very specific operation (revlog truncation). It does not make sense to perform any other operations during such a "group" (eg:file backup). There is currently no user of backupfile during a "group" so we drop the group-specific code and restrict authorized operations during "group".

transaction: document startgroup and endgroup These enigmatic methods are only used in repair. We document them to clarify there purpose and user.

transaction: mark backup-related attributes private As the transaction is gaining more functions and attributes, it is important to clarify what is part of the public API.

transaction: document the contents of `tr.backupentries` Now that all items are known we can document it.

transaction: drop the third item in `tr.backupentries` This third item is always None and never used.

rebase: fix rebase with no common ancestors (issue4446) The new rebase revset didn't check for the case when there are no common ancestors. Now it does. The new behavior should be the same as the pre-3.2 behavior. Added a test.

test-run-tests: accept more levels of precision and trailing ws (issue4440) simplejson produces slightly different output from the built-in json module, specifically: * It uses 0.000 instead of 0.0000 * It likes to put a trailing space after a comma This change works around both of those variations.

merge with stable

changegroup.cg2packer: lookup 'group' via inheritance chain This lets extensions insert themselves in the class hierarchy.

context.status: only filter suspect symlinks in the dirstate status We don't care about filtering out symlinks that have already been committed with full content, only those that have been accidentally resolved in the working directory.

context.status: inline _poststatus() By inlining _poststatus() into _buildstatus(), it becomes clearer that it is only called for the workingctx.

context.status: remove now-empty _prestatus()

context.status: call _dirstatestatus() from within _buildstatus() By making the call to _dirstatestatus() within _buildstatus(), it becomes clearer that it's called only for the workingctx.

context.status: move manifest caching trick to _buildstatus() In basectx._buildstatus(), we read the manifests for the two revisions being compared. For "caching reasons" unknown to me, it is better to read the earlier manifest first, which basectx._prestatus() takes care of. However, if the 'self' context is a committablectx and the 'other' context is the parent of the working directory (as in the very common case of plain "hg status"), there is no need to read any manifests at all -- all that's needed is the dirstate status. To avoid reading the manifests, _prestatus() is overridden in committablectx and avoids calling its super method, and _buildstatus() calls its super method only if the 'other' context is not the parent of the working directory. It seems easier to follow what's happening if we move the pre-fetching to _buildstatus() just before the place where the manifests are fetched. We just need to add an extra check that the revision is not None to handle the case that was previously handled by subclass overriding. That also makes it safe for committablectx._prestatus() to call its parent, although the latter now becomes empty, so we won't bother.

context.status: remove unused arguments from _matchstatus()

context.status: remove overriding in workingctx The workingctx method simply calls the super method. The only effect it has is that it uses a different default argument for the 'other' argument. The only in-tree caller is patch.diff, which always passes an argument to the method, so it should be safe to remove the overriding. Having the default argument depend on the type seems rather dangerous anyway.

synthrepo: when adding files, ensure new path is not a directory

synthrepo: synthesized dates must be positive, fit in 32-bit signed ints

discovery: test coverage for issue4438 / 86c35b7ae300 / 73cfaa348650 The randomness in the discovery protocol made this problem hard to reproduce. The test mocks random.sample to make sure we hit the problem every time. The set iteration order also made the output unstable ... but with the issue fixed, it is stable.

hgweb: fix a crash when using web.archivesubrepos A matcher is required when enabling the subrepo option on archival.archive(), because that calls match.narrowmatcher(), which accesses fields on the object. It's therefore probably a bad idea to default the matcher to None on archive(), but that's a fix for default.

tests: introduce a subrepository to test-archive.t This will be used in an upcoming patch to add coverage for web.archivesubrepos.

test-status-rev: add tests for plain dirstate and inter-revision status We have tests for the status across from '.^' to the working copy. It makes sense to have the similar tests for the inter-revision status between '.^' and '.' and for the dirstate status in the same place.

test-status-rev: remove unnecessary initial commit The initial commit was there when we had a group of tests that compared against an empty base, but since those tests no longer exist, we can drop the empty commit.

test-status-rev: use one glob for each expected status It's getting a little hard to read the ~30 calls to 'hg status' with one per file. Instead, let's use one glob for each expected status. For example, modified files can be listed with 'glob:content1_*_content[23]-tracked'. That also nicely becomes an explanation for why each status is expected.

test-status-rev: remove duplicate tests The second group of tests in test-status-rev compare to an empty revision. The first group of tests that compare to the first commit should be testing all the same states with the missing_* files, so drop the second group of tests.

changegroup: sparsely populate fnodes Previously, fnodes had a key and empty dict value for every element in changedfiles. This is somewhat wasteful. Empty dicts in CPython consume a lot more memory than you would expect - 280 bytes. On mozilla-central, which has ~190,000 files/fnodes keys, the previous loop populating fnodes allocated 91,924 KB of memory, most of that for the empty dicts. With this patch in place, our peak RSS during mozilla-central clone drops: before: 364,356 KB after: 326,008 KB delta: -38,348 KB When combined with the previous patch, total peak RSS decrease is now 190,116 KB.

changegroup: don't store unused value on fnodes (issue4443) The contents of fnodes are only accessed once per key. It is wasteful to cache the value since nobody will use it. Before this patch, the caching of unused data in fnodes was effectively causing a memory leak during the file streaming part of bundle creation. On mozilla-central (which has ~190,000 entries in fnodes), this patch has a significant impact on RSS at the end of generate(): before: 516,124 KB after: 364,356 KB delta: -151,768 KB The origin of this code can be traced back to 627cd7842e5d and has been with us since the 2.7 release.

changegroup: don't define lookupmf() until it is needed lookupmf() is currently defined earlier than when it is needed. Future patches further refactoring this code will be easier to read when lookupmf() is in its new home.

mail: actually use the verifycert config value The mail module only verifies the smtp ssl certificate if 'verifycert' is enabled (the default). The 'verifycert' can take three possible values: - 'strict' - 'loose' - any "False" value, eg: 'false' or '0' We tested the validity of the third value, but never converted it to actual falseness, making 'False' an equivalent for 'loose'. This changeset fixes it.

exchange: use the postclose API on transaction As with changegroup, we should wait for the transaction to be really closed before scheduling hook execution.

changegroup: use the 'postclose' API on transaction The post-transaction hooks run after the lock release (because hooks may want to touch the repository), but they must only run if the transaction is successfully closed. We use the new 'addpostclose' method on transaction to register a callback installing this post-lock-release call.

transaction: allow registering a post-close callback The addchangegroup code considers the transaction done after a 'tr.close()' call and schedules the hook's execution for after lock release. In the nested transaction case, the transaction is not yet committed and we must delay this scheduling. We add an 'addpostclose' method (like the 'addpending' and 'addfinalize' ones) that registers code to be run if the transaction is successfully committed.

exchange: swap "push" for "pull" in pulloperation docstring

exchange: prepare kwargs for bundle2 part generation exactly once

exchange: fix indentation in _pullchangeset

dagutil: fix id/ix typos in docstrings

patchbomb: extract 'getpatchmsgs' closure into its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'makeintro' closure into its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'getbundlemsgs' closure in its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'getdescription' closure in its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'getbundle' closure in its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'getpatches' closure in its own function Keep marching toward the promised land of simplification!

patchbomb: extract 'getoutgoing' closure into its own function The patchbomb command is a gigantic 300 line function full of closures. As a first step to simplify it in smaller bits, I'm extracting the closures into full featured functions. The first victim is 'getoutgoing'. It gains a docstring in the process.

bundle2: handle empty 'b2x:changegroup' value in push and pull Changeset e4dc2b0be056 added advertising of supported changegroup version through the new 'b2x:changegroup' capability. However, this capability is not new and has been around since 3.1 with an empty value. This makes new clients unable to push to 3.2 servers through bundle2 as they cannot find a common changegroup version to use from and empty list. Treating empty 'b2x:changegroup' value as old client fixes it.

bundle2: drop duplicated definition of 'b2x:exchange' This bundle2 capability is going to be dynamically computed in 'getrepocaps'. We do not need to include it in the static value.

convert: use git diff-tree -Cn% instead of --find-copies=n% for older git The option --find-copies was added in a later git version than the one included in Debian squeeze-lts (1.7.2.5), probably around 1.7.4.

changelog: rely on transaction for finalization Instead of calling 'cl.finalize()' by hand (possibly at a bogus time) we register it in the transaction during 'delayupdate' and rely on 'tr.close()' to call it at the right time.

transaction: allow registering a finalization callback The new 'addfinalize' method allows people to register a callback to be triggered when the transaction is closed. This aims to get rid of explicit calls to 'changelog.finalize'. This also obsoletes the 'onclose' function but removing it is not in the scope of this series.

changelog: handle writepending in the transaction The 'delayupdate' method now takes a transaction object and registers its '_writepending' method for execution in 'transaction.writepending()'. The hook can then use 'transaction.writepending()' directly. At some point this will allow the addition of other file creation during writepending.

transaction: add 'writepending' logic The contents of the transaction must be flushed to disk before running a hook. But it must be flushed to a special file so that the normal reader does not use it. This logic is currently in the changelog only. We add some facility to register such operations in the transaction itself.

changelog: rework the delayupdate mechanism The current way we use the 'delayupdate' mechanism is wrong. We call 'delayupdate' right after the transaction retrieval, then we call 'finalize' right before calling 'tr.close()'. The 'finalize' call will -always- result in a flush to disk, making the data available to all readers. But the 'tr.close()' may be a no-op if the transaction is nested. This would result in data: 1) exposed to reader too early, 2) rolled back by other part of the transaction after such exposure So we need to end up in a situation where we call 'finalize' a single time when the transaction actually closes. For this purpose we need to be able to call 'delayupdate' and '_writepending' multiple times and 'finalize' once. This was not possible with the previous state of the code. This changeset refactors the code to makes this possible. We buffer data in memory as much as possible and fall-back to writing to a ".a" file after the first call to '_writepending'.

merge with stable

bookmarks: fix formatting of exchange message (issue4439) The message formatting was crashing when doing explicit pulling `hg pull -B X`. This changeset fix it and improved the test coverage.

test-status-rev: document one more broken test The status for missing_content2_content2-untracked doesn't get reported at all. Since the file does exist in the working copy, it should reported as unknown. Document that in the test.

test-status-rev: use common script for generating file history Start using the generate-working-copy-states.py script that's shared with test-revert.t, instead of creating the states manually in the test. This adds several states that are currently missing. We will start checking those states later.

test-status-rev: use same names as from generate-working-copy-states To prepare for using generate-working-copy-states.py for generating the files and their content, let's start by renaming the files according to the naming scheme used by that script.

test-revert: move embedded script to its own file Move the gen-revert-cases.py out of test-revert.t into its own file so we can reuse it from other tests (specifically test-status-rev.t).

test-revert: simplify generation of files With the recent change in naming of the generated files, it becomes much easier to generate the files by iterating over all the possible states than over the state transitions.

merge with stable

discovery: indices between sample and yesno must match (issue4438) 3ef893520a85 changed 'sample' from a list to a set. The iteration order is thus undefined and the yesno indices are not stable. To solve this, repeat the listification and comment from elsewhere in the code. Note: the randomness in the discovery protocol can make this problem hard to reproduce.

discovery: limit 'all local heads known remotely' to real 'all' (issue4438) 3ef893520a85 made it possible that the initial head check didn't include all heads. If that is the case, don't use the early exit just because this random sample happened to be 'all known'. Note: the randomness in the discovery protocol can make this problem hard to reproduce.

largefiles: avoid printing messages while rebasing by "_lfstatuswriters" Putting "lambda *msg, **opts: None" (= avoid printing messages always) into "_lfstatuswriters" while rebasing makes explicit passing "printmessage = False" for "updatelfiles()" useless. This patch also removes setting/unsetting "repo._isrebasing" in "overriderebase", because there is no code path referring it.

largefiles: get function to write status messages via "getstatuswriter()" This patch makes "updatelfiles()" get appropriate function to write largefiles specific status messages via "getstatuswriter()". This patch introduces None as "print messages if needed", because True (forcibly writing) and False (forcibly ignoring) are already used for "printmessage" of "updatelfiles". Subsequent patch will move "avoid printing messages only while automated committing" decision from caller of "updatelfiles()" into "getstatuswriter()".

largefiles: introduce "_lfstatuswriters" to customize status reporting "lfutil.getstatuswriter" is the utility to get appropriate function to write largefiles specific status out from "repo._lfstatuswriters". This patch uses "stack" with an element instead of flag like "_isXXXXing" or so, because: - the former works correctly even when customizations are nested, and - ensuring at least one element can ignore empty check

largefiles: update standins only at the 1st commit of "hg rebase --continue" Before this patch, "hg rebase --continue" may record incorrect standins, because largefiles extension always avoid updating standins while rebasing, even though largefiles in the working directory may be modified manually at the 1st commit of "hg rebase --continue". But, on the other hand, updating standins should be avoided at subsequent commits for efficiency reason. To update standins only at the 1st commit of "hg rebase --continue", this patch introduces state-full callable object "automatedcommithook", which updates standins by "lfutil.updatestandinsbymatch()" only at the 1st commit of resuming. Even after this patch, "repo._isrebasing = True" is still needed to avoid some status report while updating largefiles in "lfcommands.updatelfiles()". This is reason why this patch omits not "repo._isrebasing = True" in "overriderebase" but examination of "getattr(repo, "_isrebasing", False)" in "updatestandinsbymatch".

largefiles: introduce "_lfcommithooks" to abstract pre-committing procedures This changes allows to customize pre-committing procedures according to conditions. This patch uses "stack" with an element instead of flag like "_isXXXXing" or so, because: - the former works correctly even when customizations are nested, and - ensuring at least one element can ignore empty check

largefiles: factor out procedures to update standins for pre-committing This patch factors out procedures to update standins for pre-committing. This is one of preparations to avoid execution of such procedures according to invocation context. For example, resuming automated committing (e.g. "hg rebase --continue") should update standins at the 1st commit, because largefiles in the working directory may be modified manually. But on the other hand, it should avoid updating standins at subsequent committings for efficiency reason. For simplicity, this patch just moves procedures mechanically only with replacing below. - "self" => "repo" - "lfutil." => (none) - "orig" invocation => returning "match" Using "fstandin" instead "standin" as the name of local variable for the loop below is the only special care, because the latter shadows the same name function in "lfutil.py". [before] for standin in standins: lfile = lfutil.splitstandin(standin) if lfdirstate[lfile] != 'r': lfutil.updatestandin(self, standin) [after] for fstandin in standins: lfile = splitstandin(fstandin) if lfdirstate[lfile] != 'r': updatestandin(repo, fstandin)

largefiles: factor out procedures to update lfdirstate for post-committing Before this patch, procedures to update lfdirstate for post-committing are scattered in "lfilesrepo.commit". In the case of "hg commit" with patterns for target files ("Case 2"), lfdirstate is updated BEFORE real committing. This patch factors out procedures to update lfdirstate for post-committing into "lfutil.markcommitted", and makes it callable via "markcommitted" of the context passed to "lfilesrepo.commitctx". "markcommitted" of the context is called, only when it is committed successfully. Passing original "markcommitted" of the context is meaningless in this patch, but required in subsequent one to prepare something before invocation of it.

largefiles: remove meaningless code path for "hg pull --rebase" This patch removes "--rebase" specific code path for "hg pull" in "overridepull", because previous patch makes it meaningless: now, "rebase.rebase" ("orig" invocation in this patch) can update/commit largefiles safely without "repo._isrebasing = True". As a side effect of removing "rebase.rebase" invocation in "overridepull", this patch removes "nothing to rebase ..." message in "test-largefiles.t", which is shown only when rebase extension is enabled AFTER largefiles: before this patch: 1. "dispatch" invokes "pullrebase" of rebase as "hg pull" at first, because rebase wraps "hg pull" later 2. "pullrebase" invokes "overridepull" of largefiles as "orig", even though rebase assumes that "orig" is "pull" of commands 3. "overridepull" executes "pull" and "rebase" directly 3.1 "pull" pulls changesets and creates new head "X" 3.2 "rebase" rebases current working parent "Y" on "X" 4. "overridepull" returns to "pullrebase" 5. "pullrebase" tries to rebase, but there is nothing to be done, because "Y" is already rebased on "X". then, it shows "nothing to rebase ..." after this patch: 1. "dispatch" invokes "pullrebase" of rebase as "hg pull" 2. "pullrebase" invokes "overridepull" of largefiles as "orig" 3. "overridepull" executes "pull" as "orig" 4. "overridepull" returns to "pullrebase" 5. revision "Y" is not yet rebased, so "pullrebase" doesn't shows "nothing to rebase ..." As another side effect of removing "rebase.rebase" invocation, this patch fixes issue3861, which occurs only when rebase extension is enabled BEFORE largefiles: before this patch: 1. "dispatch" invokes "overridepull" of largefiles at first, because largefiles wrap "hg pull" later 2. "overridepull" executes "pull" and "rebase" explicitly 2.1 "pull" pulls changesets and creates new head "X" 2.2 "rebase" rebases current working parent, but fails because no revision is checked out in issue3861 case 3. "overridepull" returns to "dispatch" with exit code 1 returned from "rebase" at (2.2) 4. "hg pull" terminates with exit code 1 unexpectedly after this patch: 1. "dispatch" invokes "overridepull" of largefiles at first 2. "overridepull" invokes "pullrebase" of rebase as "orig" 3. "pullrebase" invokes "pull" as "orig" 4. "pullrebase" invokes "rebase", and it fails 5. "pullrebase" returns to "overridepull" with exit code 0 (because "pullrebase" ignores result of "pull" and "rebase") 6. "overridepull" returns to "dispatch" with exit code 0 returned from "rebase" at (5) 7. "hg pull" terminates with exit code 0

largefiles: wrap "rebase.rebase" for functions using it directly Before this patch, largefiles extension wraps only "rebase" in the command table by "extensions.wrapcommand". But there are some functions using "rebase.rebase" directly. Without special care for them, largefiles extension can't work correctly with such functions. In addition to it, "special care" often becomes complicated and awkward. For example: - "unshelve" can't get correct result of "rebase.rebase", because of lack of special care - special care for "hg pull --rebase" causes issue3861 This patch wraps "rebase.rebase" for functions using it directly. For simplicity, this patch keeps 'special care for "hg pull --rebase"'. It is removed in the subsequent patch.

changegroup: introduce cg2packer/unpacker cg2 supports generaldelta in changegroups, to be used in bundle2. Since generaldelta is handled directly in cg2, reordering is switched off by default.

push: send highest changegroup format supported by both side When using bundle2, we find the common subset of supported changegroup-packers and we pick the max of them. This allow to use generaldelta aware changegroups through bundle2.

getbundle: send highest changegroup format supported by both side When using bundle2, we find the common subset of supported changegroup-packers and we pick the max of them. This allow to use generaldelta aware changegroup through bundle2.

changegroup: allow use of different cg#packer in getchangegroupraw This will allow the use of general delta aware changegroup formats.

changegroup: introduce "raw" versions of some commands The commands getchangegroup, getlocalchangegroup and getsubset now each have a version ending in -raw. The raw versions return the chunk generator from the changegroup packer directly, without wrapping it in a chunkbuffer and unpacker. This avoids extra chunkbuffers in the bundle2 code path. Also, the raw versions can be extended to support alternative packers in the future, to be used from bundle2.

merge with stable

win32: remove Mercurial.ini file from Inno Setup installer (issue4435)

win32: fix win32 installers generation cd0c51c07e5f introduced a typo preventing a proper generation of the installers. Also remove ConcatenateFiles() function as it is no more required.

patchbomb: add label and color to the confirm output The current output is mostly a wall of text. This makes it hard to actually check something for people with lazy eyes. We use labels and colors to make it more joyful (and get the patch summaries to stand out). The colors have been arbitrarily choosen. They can be changed later if someone has a more scientific choice.

tests: write hgrc of more than two lines by using shell heredoc Here document should be readable than repeating echo commands.

perf: use a formatter for output We use a `formatter` object in the perf extensions. This allow the use of formatted output like json. To avoid adding logic to create a formatter and pass it around to the timer function in every command, we add a `gettimer` function in charge of returning a `timer` function as simple as before but embedding an appropriate formatter. This new `gettimer` function also return the formatter as it needs to be explicitly closed at the end of the command. example output: $ hg --config ui.formatjson=True perfvolatilesets visible obsolete [ { "comb": 0.02, "count": 126, "sys": 0.0, "title": "obsolete", "user": 0.02, "wall": 0.0199398994446 }, { "comb": 0.02, "count": 117, "sys": 0.0, "title": "visible", "user": 0.02, "wall": 0.0250301361084 } ]

bundle2: support a "version" argument in `changegroup` part When included, this mandatory parameter (mandatory == cannot be ignored) lets the part handler select the right cgunpacker class.

bundle2caps: advertise the available versions for changegroup packer This will let the bundle2 client and server detect what packer they should be using. This detection part is not done. I expect it to be done with the addition of the second packer (with generaldelta support).

changegroup: add a "packermap" dictionary to track different packer versions We only have "01" right now, but we should get general delta in soon. Bundle2 is expected to make use of this to advertise and select the right packer to use on both sides.

templater: don't overwrite the keyword mapping in runsymbol() (issue4362) This keyword remapping was introduced in e06e9fd2d99f as part of converting generator based iterators into list based iterators, mentioning "undesired behavior in template" when a generator is exhausted, but doesn't say what and introduces no tests. The problem with the remapping was that it corrupted the output for keywords like 'extras', 'file_copies' and 'file_copies_switch' in templates such as: $ hg log -r 142b5d5ec9cc --template "{file_copies % ' File: {file_copy}\n'}" File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) File: mercurial/changelog.py (mercurial/hg.py) What was happening was that in the first call to runtemplate() inside runmap(), 'lm' mapped the keyword (e.g. file_copies) to the appropriate showxxx() method. On each subsequent call to runtemplate() in that loop however, the keyword was mapped to a list of the first item's pieces, e.g.: 'file_copy': ['mercurial/changelog.py', ' (', 'mercurial/hg.py', ')'] Therefore, the dict for the second and any subsequent items were not processed through the corresponding showxxx() method, and the first item's data was reused. The 'extras' keyword regressed in de7e6c489412, and 'file_copies' regressed in 0b241d7a8c62 for other reasons. The common thread of things fixed by this seems to be when a list of dicts are passed to the templatekw._hybrid class.

revset-matching: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet but it is less visible because the matching process itself is very expensive: revset) matching(100) before) wall 6.413281 comb 6.420000 user 5.910000 sys 0.510000 (best of 3) after) wall 6.173608 comb 6.170000 user 5.750000 sys 0.420000 (best of 3) However for some complex list, this provide a massive speedup revset) matching(parents(100)) before) wall 23.890740 comb 23.890000 user 23.450000 sys 0.440000 (best of 3) after) wall 6.382280 comb 6.390000 user 5.930000 sys 0.460000 (best of 3)

revset-parentspec: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet: revset) 100^1 before) wall 0.002694 comb 0.000000 user 0.000000 sys 0.000000 (best of 897) after) wall 0.000997 comb 0.000000 user 0.000000 sys 0.000000 (best of 2324) revset) parents(100)^1 before) wall 0.003832 comb 0.000000 user 0.000000 sys 0.000000 (best of 587) after) wall 0.001034 comb 0.000000 user 0.000000 sys 0.000000 (best of 2309) revset) (100^1)^1 before) wall 0.005616 comb 0.000000 user 0.000000 sys 0.000000 (best of 405) after) wall 0.001030 comb 0.000000 user 0.000000 sys 0.000000 (best of 2258)

revset-children: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet: revset) children(tip~100) before) wall 0.007469 comb 0.010000 user 0.010000 sys 0.000000 (best of 338) after) wall 0.003356 comb 0.000000 user 0.000000 sys 0.000000 (best of 755)

revset-ancestorspec: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefits of the fullreposet: revset) 100~5 before) wall 0.002712 comb 0.000000 user 0.000000 sys 0.000000 (best of 918) after) wall 0.000996 comb 0.000000 user 0.000000 sys 0.000000 (best of 2493) revset) parents(100)~5 before) wall 0.003812 comb 0.010000 user 0.010000 sys 0.000000 (best of 667) after) wall 0.001038 comb 0.000000 user 0.000000 sys 0.000000 (best of 2361) revset) (100~5)~5 before) wall 0.005614 comb 0.000000 user 0.000000 sys 0.000000 (best of 446) after) wall 0.001035 comb 0.000000 user 0.000000 sys 0.000000 (best of 2424)

revset-rangeset: call 'getset' on a 'fullreposet' Calling 'baseset(repo.changelog)' builds a list for all revisions in the repo. And we already have the lazy and efficient 'fullreposet' class for this purpose. This gives us the usual benefit ofs the fullreposet: revset) 10:100 before) wall 0.002774 comb 0.000000 user 0.000000 sys 0.000000 (best of 797) after) wall 0.001977 comb 0.000000 user 0.000000 sys 0.000000 (best of 1244) revset) parents(10):parents(100) before) wall 0.005054 comb 0.000000 user 0.000000 sys 0.000000 (best of 481) after) wall 0.002060 comb 0.000000 user 0.000000 sys 0.000000 (best of 1056)

test-revert: make sure all 'tracked' files are really tracked When a file is missing in the 'parent' version and is tracked but missing in the working directory, which happens by the 'missing' or 'removed' types, and the 'clean' type in the working directory, the file does not exist in the working directory (unlike it would had the 'deleted' type been used). Thus, the *_missing_missing_tracked are not actually tracked and they end up testing the same state as *_missing_missing_untracked. To make them tracked, add a temporary file, just like we do for the delete case. For simplicity's sake, let's make sure the gen-revert-cases.py script always puts a file in the working directory, whether or not it's going to be deleted.

test-revert: sort by output filename again Future patches will change how the output of 'gen-revert-cases.py filelist' is generated, so now we want the order to depend on just the filename again.

test-revert: name files by state, not by state transition This is the main patch in a series. See motivation in earlier patch. In this patch, we actually change the names of the generated files. For example, the file that is currently called missing_clean becomes missing_missing_missing-tracked and it's clearer that it should be tracked. It turns out that since the state was not previously clear, it ended up testing an untracked state, which was the same as for missing_clean. We'll fix this in a later patch. Let's also change the content from (base,parent,wc) to (content1,content2,content3) to make them all the same length so they line up when displayed.

test-revert: temporarily sort by input states instead of output filename The next patch will change the names of the files produced by the script in test-revert. In order to reduce the size and increase the clarity of the next patch, make the order produced by the internal 'gen-revert-cases.py filelist' command independent of the filenames.

test-revert: put content, not keys, into 'combination' By putting the file content rather than keys in the 'combination' list, we restrict the knowledge of 'ctxcontent' and 'wccontent' to the loop generating the combinations. That will make it easier to replace the generation code.

test-revert: replace 'removed' in working copy with 'untracked-deleted' The 'wccontent' variable has eight different states, four of them tracked, and the other four untracked (at least when the file existed in the parent revision). Among these eight states, 'removed' sticks out by lacking the 'untracked-' prefix despite resulting in an untracked state. To make the symmetry clearer, and to prepare for future patches, rename 'removed' to 'untracked-deleted', which is exactly what it is. Note that, unlike 'remove', 'deleted' is configured in gen-revert-cases.py to have content in the working directory and that that content is instead expected to be removed in the test script. However, no changes are needed to the test script, since it already contains 'hg forget *untracked*' and 'rm *deleted*', which together have the same effect as 'hg remove'. See additional motivation in earlier patch.

test-revert: removing a missing file has no effect The tests for removed_deleted and removed_removed test the same state as removed_clean and removed_untracked-clean, respectively. Drop the duplicate tests. See additional motivation in earlier patch.

test-revert: reverting an addition is the same as removing The tests for added_revert and added_untracked-revert test the same state as added_deleted and added_removed, respectively. Drop the duplicate tests. See additional motivation in earlier patch.

test-revert: reverting no change means it's clean This is the first step in a series that aims to put the state, not the state transitions, in the filenames of the files generated by the gen-revert-cases.py script. The possible state of a file in a revision and in the working copy is only whether it exists and what its content is (the tests don't care check flags). In the dirstate, the only state is whether it's tracked or not. With the new naming, the file that is currently called modified_untracked-clean now becomes content1_content2_content2-untracked, for example. By putting these states in the filename, it becomes easier to see that we're not missing or duplicating any state, and to check that the state is what we think it is. For example, the file that is currently called missing_clean becomes missing_missing_missing-tracked and it's clearer that it should be tracked. Putting the content in the filename will also make the tests of file content (e.g. "cat ../content-parent.txt") very obvious. When we put the state in the filename, the filenames clearly need to be unique. However, it turns out that some states are currently tested multiple times. The 'revert' transition in the script means to take the content from the grandparent. If the parent is the same as the grandparent, there is no change compared to the parent, which is exactly what 'clean' means. Avoid testing the same state twice.

merge with stable

extdiff: drop the command alias without options example in the help text In the dropped example, the extension would look for 'vdiff.diffargs' in the configuration, and not finding it, would run kdiff3 without the configured options. That's not obvious to a new user who sees a kdiff3 configuration in the prepackaged mergetools.rc file, and sees that kdiff3 still runs. While it is conceivable that the user wants a kdiff3 command that runs without the preconfigured options, it is more likely what they want is this, which uses the canned options: [alias] vdiff = kdiff3 [extdiff] kdiff3 = We could mention alias here, but that seems like it belongs elswhere.

extdiff: allow a preconfigured merge-tool to be invoked There are three ways to configure an extdiff tool: 1) cmd.tool = (/path/to/exe optional) 2) tool = (path/to/exe optional) 3) tool = sometool someargs Previously, if no executable is specified in the first two forms, the named tool must be in $PATH, or the invocation fails. Since the [merge-tools] section already has the path to the diff executable, and/or the registry keys to find the executable on Windows, reuse that configuration for forms 1 and 2 instead of failing. We already fallback to [diff-tools] and then [merge-tools] for program arguments if they aren't specified in the [extdiff] section. Since this additional lookup only occurs if an executable is not on the $PATH for the named tool, this is backwards compatible. For now, we assume the user knows what he is doing if a path is provided. This change allows a configuration file like this (assuming beyondcompare3 is configured in merge-tools), instead of hardcoding system specific a path: [extdiff] beyondcompare3 =

extdiff: sort files when snapshotting This fixes output stability and is generally filesystem-performance-friendly.

filemerge: split the logic for finding an external tool to its own function This will be used by extdiff in an subsequent patch.

largefiles: simplify check for lack of path arguments Instead of checking for a partial merge by checking that the matches has no files and no patterns, check that it's not an always-matcher. Except for being shorter, it also catches the rare case of an exact-matcher with no files.

largefiles: shortcircuit status code also for non-matching patterns We currently shortcircuit the checking for large file standins if only patterns of type 'path' are given on the command line. That makes e.g. "hg st 'glob:foo/**'" unnecessarily slow when the only large files are in a sibling directory. Relax the check to be that it is not an always-matcher and that no large files match the patterns given on the command line. Note that before this change, only the latter of the following two would show the status of files in .hglf (since the -I makes match.anypats() true). After this change, they both display the status. This behavior doesn't seem correct, but it would be a separate change to explicitly filter out .hglf even in the shortcircuit case. hg st .hglf/$file hg st .hglf/$file -I .

largefiles: remove confusing 'or None' from predicate The match function that is overriden returns a boolean value, so adding 'or None' is both unnecessary and confusing.

largefiles: drop unnecessary setting of matcher._always In two very similar segments of code, an existing matcher is modified by changing its _files attribute through a map and a filter operation. Neither operation can cause an empty list to become non-empty, so a matcher that always matches can not stop always matching. Drop the setting of the attribute, so we don't unnecessarily prevent the fast paths to be taken where these matchers end up being used.

config: move mergetools configuration from contrib to default configuration The merge tool configuration is an essential part of a good initial user experience. 'make osx' installers and direct 'make' installation did not have merge tool configuration. Now they have. Note: The installer fixes for windows have been done blindly and might require additional changes.

config: introduce "built-in" default configuration settings in default.d This helps providing a more consistent user experience on all platforms and with all packaging. The exact location of default.d depends on how Mercurial is installed and whether it is 'frozen'. The exact location should never be relevant to users and is intentionally not explained in details in the documentation. It will however always be next to the help and templates files. Note that setting HGRCPATH also disables these defaults. I don't know if that should be considered a bug or a feature.

contrib: buildrpm checking of md5 checksums of downloaded Python and Docutils

contrib: update build defaults to latest Python and docutils versions Use Python 2.7.8 and Docutils 0.12 for Makefile.python and buildrpm (CentOS 5 rpms).

spelling: fixes from proofreading of spell checker issues

extdiff: quote user-supplied options passed to shell $ hg extdiff -p cmd -o "name <user@example.com>" resulted in a shell redirection error (due to the less-than sign), rather than passing the single option to cmd. This was due to options not being quoted for passing to the shell, via util.system(). Apply util.shellquote() to each of the user-specified options (-o) to the comparison program before they are concatenated and passed to util.system(). The requested external diff command (-p) and the files/directories being compared are already quoted correctly. The discussion at the time of changeset be98c5ce4022 correctly noted that this course of action breaks whitespace-separated options specified for external diff commands in the configuration. The lower part of the patch corrects this by lexing options read from the configuration file into separate options rather than reading them all into the first option. Update test to cover these conditions. Related changesets (reverse-chronological): - be98c5ce4022 (fix reverted to make configuration file options work) - 453097750fbf (issue fixed but without fix for configuration file)

serve: correct meta variable of --daemon-pipefds option It was changed to lock path at e22695b4472f.

tests: have dumbhttp.py use cmdutil.service() to wait for child to listen() Because the original dumbhttp.py exited without waiting for listen(), several tests could fail with "abort: error: Connection refused" if subsequent hg command is fast enough.

test-revert: remove obsolete comment about known misbehavior It seems like the last known misbehvior that the comment was referring to was dealt with in 2ff28e07d7d6 (revert: properly back up added files with local modification, 2014-08-31).

test-revert.t: fix wc check-code false positive

merge with stable

Added signature for changeset ced632394371

Added tag 3.2 for changeset ced632394371

setdiscovery: limit the size of all sample (issue4411) Further digging on this issue show that the limit on the sample size used in discovery never works for heads. Here is a quote from the code itself: desiredlen = size - len(always) if desiredlen <= 0: # This could be bad if there are very many heads, all unknown to the # server. We're counting on long request support here. The long request support never landed and evolution make the "very many heads, all unknown to the server" case quite common. We implement a simple and stupid hard limit of sample size for all query. This should prevent HTTP 414 error with the current state of the code.

hook: protect commit hooks against stripping of temporary commit (issue4422) History rewriting commands like histedit tend to use temporary commits. They may schedule hook execution on these temporary commits for after the lock has been released. But temporary commits are likely to have been stripped before the lock is released (and the hook run). Hook executed for missing revisions leads to various crashes. We disable hooks execution for revision missing in the repo. This provides a dirty but simple fix to user issues.

mq: do not call [0] on revset The __getitem__ method have been removed. The "first" method is to be used instead. Test have been extended to test this code path.

addset: fix `first` and `last` on sorted addset (issue4426) The lazy sorting were not enforced on addset. This was made visible through MQ.

clone: properly mark branches closed with --uncompressed (issue4428) On streaming clone, we were priming the local branch cache with the remote branchmap, without checking which heads were closed. This fixes an issue introduced in: changeset: 17740:e6067bec18da user: Tomasz Kleczek <tomasz.kleczek@fb.com> date: Wed Oct 03 13:19:53 2012 -0700 summary: branchcache: fetch source branchcache during clone (issue3378) that was exposed in 2.9 by: changeset: 20192:38fad5e76ee8 user: Brodie Rao <brodie@sf.io> date: Mon Sep 16 01:08:29 2013 -0700 summary: branches: simplify with repo.branchmap().iterbranches()

Makefile: update .PHONY Based on sed -n 's/^\([a-z0-9-]*\):\(\s.*\)\?$/\1/gp' Makefile | xargs echo add check, check-code, update-pot, some packaging targets

buildrpm: fix use of invalid $PLATFORM in mercurial.repo

debuglocks: add missing usage summary

help: don't crash on help for 'sections' with multiple '.'

help: fix config description of ui.reportoldssl

i18n-ja: synchronized with f3baaeffe841

i18n-pt_BR: synchronized with 0b7853f969ac

merge with i18n

i18n-ja: synchronized with 5459b30aa498

clone: fix copying bookmarks in uncompressed clones (issue4430) ef62c66bee1b broke bookmarks getting copied during uncompressed clones. Since most of the pull logic has been moved into exchange.py, lets just call exchange.pull to fix up the repo with the latest bits after the streaming clone has bootstrapped the repo. This keeps us from having to duplicate the bookmark logic.

revset: don't recreate matcher for every revision The matcher variable 'm' in checkstatus() is reset to None on each call, so the caching of the matcher no longer happens as it was intended. This seems to be a regression in ed7b674824a3 (revset: added lazyset implementation to checkstatus, 2014-01-03). Fix by moving the cached matcher into the enclosing function so it's actually cached across calls. This speeds up hg log -r 'modifies(mercurial/context.py)' >/dev/null from 7.5s to 4s. Also see similar fix in f2aeff8a87b6 (revset: avoid recalculating filesets, 2014-10-22).

help: use ":hg:`command`" instead of incorrect ":hg:'command'" notation

i18n: add i18n comment to error messages of filesets predicates

i18n: add i18n comment to error messages of template functions

help: refer ":merge3" instead of "internal:merge3" According to warning message (introduced by 38e0363dcbe0) in filemerge.py, the former should be used as official name.

i18n: make hint message of exception translatable

help: use "hg files" instead of "hg locate" in "hg help filesets" The latter command is already deprecated.

files: refer "hg help filesets" instead of "hg help revsets" in help text "specifying FILE patterns" should refer the former.

tests: silence output race in test-run-tests.t

merge with i18n

i18n-pt_BR: synchronized with 8d43c6bb38c0

i18n-pt_BR: synchronized with c312ef382033

transaction: only generate file when we actually close the transaction Before this change, the file were written for every call to `tr.close()` exposing data to reader far too early.

transaction: extract file generation into its own function We extract the code generating files into its own function. We are about to move this code around to fix a bug. We'll need it in a function soon to reuse it for "pending" logic. So we move the code into a function instead of moving it twice.

amend: abort early if no username is configured with evolve enabled (issue4211) Amend will reuse the original username if a new one is not provided with -U, but obsolete.createmarkers() only considers ui.username() for the obsolete marker's metadata. Allowing the metadata field to be spoofed seems to defeat the point of the field in the first place. This covers 'evolve amend' and 'ci --amend' with evolve enabled. Without this, the transaction aborts but the parent changeset is set to -1. The corresponding test will be added to evolve separately.

revset: fix O(2^n) perf regression in addset hg log -r 1 ... -r 100 was never returning due to a regression in the way addset computes __nonzero__. It used 'bool(self._r1 or self._r2)' which required executing self._r1.__nonzero__ twice (once for the or, once for the bool). hg log with a lot of -r's happens to build a one sided addset tree of N length, which ends up being 2^N performance. This patch fixes it by converting to bool before or'ing. This problem can be repro'd with something as simple as: hg log `for x in $(seq 1 50) ; do echo "-r $x "; done` Adding '1 + 2 + ... + 20' to the revsetbenchmark.txt didn't seem to repro the problem, so I wasn't able to add a revset benchmark for this issue.

tests: add missing glob for Windows

test-convert-svn-sink: properly isolate symlink section This was fixed earlier by moving all the symlink bits to a section to the end of the file, but then it was broken (by the same person) by adding more tests at the end.

util.fspath: use a dict rather than a linear scan for lookups Previously, we'd scan through the entire directory listing looking for a normalized match. This is O(N) in the number of files in the directory. If we decide to call util.fspath on each file in it, the overall complexity works out to O(N^2). This becomes a problem with directories a few thousand files or larger. Switch to using a dictionary instead. There is a slightly higher upfront cost to pay, but for cases like the above this is amortized O(1). Plus there is a lower constant factor because generator comprehensions are faster than for loops, so overall it works out to be a very small loss in performance for 1 file, and a huge gain when there's more. For a large repo with around 200k files in it on a case-insensitive file system, for a large directory with over 30,000 files in it, the following command was tested: ls | shuf -n $COUNT | xargs hg status This command leads to util.fspath being called on $COUNT files in the directory. COUNT before after 1 0.77s 0.78s 100 1.42s 0.80s 1000 6.3s 0.96s I also tested with COUNT=10000, but before took too long so I gave up.

test-clone.t: drop message about listing bookmarks with no hardlinks

tests: don't try to test unix sockets on vfat

tests: change obsolete timestamp to avoid "gmtime()" problem on Windows Before this patch, "test-obsolete.t" fails on Windows environment, because strings corresponded to "tm_wday" (day of the week) field are incorrect. On POSIX environment, "gmtime()" returns correct "tm_wday" value even for negative "time_t" value. On the other hand, it returns incorrect one on Windows environment. At least, "gmtime()" of the Windows runtime library bundled with Python 2.7.3 does. According to 9a7d0f7e0561 introducing original timestamp value '56 120', it shouldn't cause negative "time_t" value. test-obsolete: remove subminute timezone in test Obsmarker format "1" does not supports sub minute timezone. So we change the test to something slightly more sensible. It replaced "-d '56 12'" by "-d '56 120'".

tests: use "%HG_ARGS%" in shell alias on Windows instead of "$HG_ARGS" Before this patch, a part of "test-alias.t" fails unexpectedly on Windows environment, because "cmd.exe" can't evaluate "$HG_ARGS" expression in shell alias correctly. This patch uses "%HG_ARGS%" in shell alias on Windows instead of "$HG_ARGS" to expand it correctly.

tests: introduce "checkeditform-n-cat.sh" script to invoke "cat" in it safely Before this patch, a part of "test-transplant.t" fails unexpectedly on Windows environment, because semicolon (";") in HGEDITOR isn't recognized as the command separator by "cmd.exe". This patch newly introduces "checkeditform-n-cat.sh" script to invoke "cat" in it safely anywhere.

doc: change 'revision or range' to 'revision or revset' The phrase "revision or range" comes from a pre-revset era. Since the documentation for ranges now is under the revset docs, and as a helpful hint nudging users towards revsets, I think it's better to say "revision or revset"

largefiles: add examination of exec bit in "hg status --rev REV" case Before this patch, "hg status --rev REV" doesn't list largefiles up with "M" mark, even if exec bit of them is changed, because "lfilesrepo.status" doesn't examine exec bit in such case.

largefiles: ignore removal status of files not managed in the target context Before this patch, "hg status --rev REV" listed largefiles removed in the working directory up with "R" mark, even if they aren't managed in the REV. Normal files aren't listed up in such case. When "lfilesrepo.status" is invoked for "hg status --rev REV", it treats files on conditions below as "removed" (to avoid manifest full scan in "ctx.status" ?): - marked as "R" in lfdirstate, or - files managed in the target revision but unknown in the manifest of the working context (= not including "R" files) But the former can include files not managed in the target context. To ignore removal status of files not managed in the target context, this patch drops files unknown in the target revision from "removed" list.

changelog: use headrevsfiltered 2b5940f64750 introduced use of the new filtering headrevs C implementation. It caught TypeError to detect when to fall back to the implementation that was compatible with old extensions. That method was however not reliable. Instead, use the new headrevsfiltered function when passing a filter. It will reliably fail with AttributeError when an old extension that predates headrevsfiltered is used.

parsers: introduce headrevsfiltered in C extension All extensions that have this function do support filtering. The existing headrevs function may support filtering but we cannot reliably detect whether it does.

httppeer: close the temporary bundle file after two-way streaming it This fixes several push tests in test-bundle2-exchange.t that were failing on Windows with messages like the following: $ hg -R main push http://localhost:$HGPORT2/ -r 32af7686d403 \ --bookmark book_32af pushing to http://localhost:$HGPORT2/ searching for changes remote: adding changesets remote: adding manifests remote: adding file changes remote: added 1 changesets with 1 changes to 1 files remote: 1 new obsolescence markers updating bookmark book_32af abort: The process cannot access the file because it is being used by another process: 'C:\path\to\tmp\bundle.hg' [255]

status: make 'hg status --rev' faster when there are deleted files In order not to avoid listing files as both added and deleted, for example, we check for every file in the manifest if it is in the _list_ of deleted files. This can get quite slow when there are many deleted files. Change it to a set to make the containment check faster. On a somewhat contrived example of the Mozilla repo with the entire testing/ directory deleted (~14k files), this makes 'hg status --rev .^' go from 26s to 2s.

setdiscovery: limit the size of the initial sample (issue4411) The set discovery start by sending a "known" command with all local heads. When the number of local heads is massive (eg: using hidden changesets) such request becomes too large. This lead to 414 error over http, aborting the whole process. We limit the size of the sample used by the first query to fix this. The test are impacted because they do test massive number of heads. But they do not test it over real world http setup.

setdiscovery: extract sample limitation in a `_limitsample` function We need to reuse this logic for the initial query. We extract it in a function to unsure sample limiting is applied consistently in all cases.

exchange: don't report failure from identical bookmarks b901645a8784 regressed the behavior of pushing an unchanged bookmark to a remote. Before that commit, pushing a unchanged bookmark would result in "exporting bookmark @" being printed. After that commit, we now see an incorrect message "bookmark %s does not exist on the local or remote repository!" This patch fixes the regression introduced by b901645a8784 by having the bookmark error reporting code filter identical bookmarks and adds a test for the behavior.

bookmarks: explicitly track identical bookmarks bookmarks.compare() previously lumped identical bookmarks in the "invalid" bucket. This patch adds a "same" bucket. An 8-tuple for holding this state is pretty gnarly. The return value should probably be converted into a class to increase readability. But that is beyond the scope of a patch intended to be a late arrival to stable.

context.status: remove incorrect swapping of added/removed in workingctx The comment in workingctx.status() says that "calling 'super' subtly reveresed the contexts", but that is simply not true, so we should not be swapping added and removed fields.

keyword: use parent of working copy as base for status Instead of calling repo[None].status(), use the more common form that uses the parent of the working copy as the base: repo['.'].status(). Note that the former defaults to comparing to revision '.', while the latter defaults to revision None, so the contexts being compared are the same. It might seem like this would result in a reverse diff, but it turns out that workingctx.status() incorrectly reverses the result. That bug will be fixed in a later commit.

revset: avoid O(n) lookup of invalid revision in rev() ba89f7b542c9 was not the final version of that patch. It was really slow because `l not in repo.changelog` iterates revisions up to `l`. Instead, rev() should utilize spanset.__contains__(). revset #0: rev(210000) 0) wall 0.000039 comb 0.000000 user 0.000000 sys 0.000000 (best of 67978) 1) wall 0.002721 comb 0.000000 user 0.000000 sys 0.000000 (best of 1055) 2) wall 0.000059 comb 0.000000 user 0.000000 sys 0.000000 (best of 45599) (0: 3.2-rc, 1: ba89f7b542c9, 2: this patch) Note that the benchmark result described in ba89f7b542c9 is wrong because it is the one of the initial version.

run-tests: output diffs with only forward slashes This removes some gratuitous variance when testing on Windows with test-run-tests.t.

i18n: add hint to digest mismatch message

help/config: fix typo

files: fix example list syntax

parsers: use 'k' format for Py_BuildValue instead of 'n' because Python 2.4 'n' was introduced in Mercurial in 2b5940f64750 and broke Python 2.4 support in mysterious ways that only showed failure in test-glog.t. Py_BuildValue failed because of the unknown format and a TypeError was thrown ... but it never showed up on the Python side and it happily continued processing with wrong data. Quoting https://docs.python.org/2/c-api/arg.html : n (integer) [Py_ssize_t] Convert a Python integer or long integer to a C Py_ssize_t. New in version 2.5. k (integer) [unsigned long] Convert a Python integer or long integer to a C unsigned long without overflow checking. This will use unsigned long instead of Py_ssize_t. That is not a good solution, but good is not an option when we have to support Python 2.4.

rebase: improve base revset performance The old revset had pretty terrible performance on large repositories (12+ seconds). This new revset achieves the same result in only 0.7s. As we improve the underlying revset APIs we can probably get this revset down to 'only(base, dest)::', but at the moment that version still takes 2s.

amend: fix amending rename commit with diverged topologies (issue4405) This addresses the bug described in issue4405: when obsolescence markers are enabled, amending a commit with a file move can lead to the copy information being lost. However, the bug is more general and can be reproduced without obsmarkers as well, as demonstracted by Pierre-Yves and put into the updated test. Specifically, graph topology divergences between the filelogs and the changelog can cause copy information to be lost during amends.

hgweb: disable SSLv3 serving (BC) Because of recent attacks[0] on SSLv3, let's just drop support entirely. 0: http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html

sslutil: only support TLS (BC) In light of the POODLE[0] attack on SSLv3, let's just drop the ability to use anything older than TLSv1 entirely. This only fixes the client side. Another commit will fix the server side. There are still a few SSLv[23] constants hiding in httpclient, but I'll fix those separately upstream and import them when we're not in a code freeze. 0: http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html

eol: fix crash when handling removed files ci --amend would in some cases fail after 650b5b6e75ed failed to refactor the eol extension too.

bundle2: transmit exception during part generation If an exception is raised during a bundle2 part payload generation it is now recorded in the bundle. If such exception occurs, we capture it, transmit an abort exception through the bundle, cleanly close the current part payload and raise it again. This allow to generate valid bundle even in case of exception so that the consumer does not wait forever for a dead producer. This also allow to raise the exception during unbundling at the exact point it happened during bundling make debugging easier.

bundle2: add a interrupt mechanism It is now possible to emit a single part in the middle of a payload production. This part will be processed with limitation (only access to a `ui` object). The goal is to let the server raise exception and output while a part is being processed. The source motivation is to transmit exception that occurs while generating a part. This change is was the motivation to bump the bundle2 format from HG2X to HG2Y. Somehow, the format bump made it into 3.2 without it. So this change go on stable. It is low risk as bundle2 is still disabled by default.

transactions: change backupfiles format to use newlines Previously the journal.backupfiles file was delimited by \0. Now we delimit it using \n (same as the journal file). This allows us to change the number of values in each line more easily, rather than relying on the count of \0's.

transactions: add version number to journal.backupfiles The transaction format will be changing a bit over the next releases, so let's go ahead and add a version number to make backwards compatibility easier. This whole file format was broken prior to 3.2 (see previous patch), so changing it now is pretty low risk.

transactions: fix hg recover with fncache backups The transaction backupfiles logic was broken for 'hg recover'. The file format is XXX\0XXX\0YYY\0YYY\0 but the parser did a couple things wrong. 1) It went one step beyond the final \0 and tried to read past the end of the array. 2) array[i:i+1] returns a single item, instead of two items as intended. Added a test to catch it, which turns out to be the first actual 'hg recover' test.

revset: have rev() drop out-of-range or filtered rev explicitly (issue4396) The recent optimization of "and" operation relies on the assumption that the rhs set does not contain invalid revisions. So rev() has to remove invalid revisions. This is still faster than using `.filter(lambda r: r == l)`. revset #0: rev(25) 0) wall 0.026341 comb 0.020000 user 0.020000 sys 0.000000 (best of 113) 1) wall 0.000038 comb 0.000000 user 0.000000 sys 0.000000 (best of 66567) 2) wall 0.000062 comb 0.000000 user 0.000000 sys 0.000000 (best of 43699) (0: bbf4f3dfd700^, 1: 3.2-rc, 2: this patch)

revset: avoid recalculating filesets This fixes a regression in 8dabcc889e33 that moved matcher building into a callback, thus causing it be rebuilt for each revision matched against.

tests: open file in binary mode to use POSIX end-of-line style anywhere Before this patch, "test-context.py" fails on Windows environment, because "diff" output contains unexpected "\r" character. Opening the target file in text mode causes this automatic end-of-line conversion.

tests: add "(glob)" for l10n messages in test-clone.t for Windows This patch follows the style of other tests avoiding same kind of issue.

tests: add "(glob)" for l10n messages in test-largefiles-update.t for Windows This patch follows the style of other tests avoiding same kind of issue.

tests: add "(glob)" to paths in test-revset-outgoing.t for Windows

tests: add "(glob)" to paths in test-hgrc.t for Windows

tests: add "(glob)" to paths in test-url-rev.t for Windows

tests: adjust expected prompt choice outputs in test-record.t for Windows Changeset 9ab18a912c44 introduced "showing prompt choice if input is not a tty but is forced to be interactive" and adjusted expected output in test-record.t. But some tests for no-execbit platform (= Windows) were not adjusted by 9ab18a912c44. This causes unexpected failure of test-record on Windows environment. This patch adds below to prompt lines for such tests: - prompt choice at the end of prompt line, and - empty line after prompt line

ui: separate option to show prompt echo, enabled only in tests (issue4417) The problem in commandserver was addressed by 524b786bd54f, but it is tricky to reuse ui.nontty option to disable echo back. Instead, this patch introduces new option to enable echoing of prompt response. Prompt echoing is changed to be off by default, which should avoid possible breakage of output parsing in user scripts. For now, this option is undocumented because it exists for internal use.

Added signature for changeset 7f8d16af8cae

Added tag 3.2-rc for changeset 7f8d16af8cae

merge with i18n

i18n-ru: synchronized with 6b4dc7968bf0

merge default into stable for 3.2 freeze

hook: schedule run "b2x-transactionclose" for after lock release Hooks that run after the transaction need to be able to touch the repository. So we need to run them after the lock release. This is similar to what the "changegroup" hook is doing in the `addchangegroup` function.

repoview: issue a special message when filtering hidden changesets Hidden changesets are by far the most common error case and is the only one[1] that can reach the user. We move to a friendlier message with a hint about how to access the data anyway. We should probably point to a help topic instead but we do not have such a topic yet. Example of the new output abort: hidden revision '4'! (use --hidden to access hidden revisions) [1] Actually, filtering from "served" can also reach the user during certain exchange operations.

repoview: include the filter name in filtered revision error messages This will help user to debug. A more precise message will be issued for the most common case ("visible" filter) in the next changesets. example output: - abort: filtered revision '4'! + abort: filtered revision '4' (not in 'visible' subset)!

largefiles: inline redundant toname function in status Simpler and an optimization.

largefiles: inline redundant inctx function in status

ssl: only use the dummy cert hack if using an Apple Python (issue4410) The hack for using certificate store in addition to the provided CAs resides in Apple's OpenSSL. Apple's own Pythons will use it, but other custom built Pythons might use a custom built OpenSSL without that hack and will fail when exposed to the dummy cacert introduced in d7f7f1860f00. There do not seem to be a simple way to check from Python if we are using a patched OpenSSL or if it is an Apple OpenSSL. Instead, check if the Python executable resides in /usr/bin/python* or in /System/Library/Frameworks/Python.framework/ and assume that all Pythons found there will be native Pythons using the patched OpenSSL. Custom built Pythons will not get the benefit of using the CAs from the certificate store.

largefiles: move initialization of standins variable to clarify its "scope"

largefiles: the update override only needs lfdirstate and status for --check

largefiles: remove confusing rev parameter for lfdirstatestatus Dirstate only works on the repo wctx.

largefiles: replace repo._isaddremove hack with a simple function parameter

status: add more complete tests for --rev The 'status --rev' code is not very well tested, which has bitten us as recently as in issue4321. Let's add some more tests, some of which uncover bugs. Remove the few existing tests that are now covered in a more thorough and consistent way.

cmdserver: include pid of server handling requests in hello message Because unix-mode server forks child process per connection, client does not know the pid of the server that will handle requests. The pid is necessary to interrupt hung process: 1. client connects to socket server 2. server accepts the connection, forks, and tells pid 3. client requests "runcommand pull" .. hung .. 4. client sends SIGINT to the (forked) server 5. server returns from I/O wait Note that getsockopt(SO_PEERCRED) of Linux cannot be used because the server fork()s after accept().

run-tests: clean up unused globals that were removed by tests-refactor series They were demoted to instance variables by cb88d4a04f58, 1ad7aabba14e, 93511a595766, 2e1aa8c1ee37, 8e7b0f4d6ac7 and 02087bc4f143.

docker: use official centos5 image I guess it didn't exist when centos5 support was introduced.

docker: add centos7 target for CentOS / Red Hat 7 support

lock: while releasing, unlink lockfile even if the release function throws Consider a hypothetical bug in the release function that causes it to raise an exception. Also consider the bisect command, which saves its state in a finally clause. Saving the state requires acquiring the wlock. If we don't unlink the lockfile when the exception is thrown, we'll try to acquire the wlock again. We're going to try and acquire a lock again while our old lockfile is on disk. The PID on disk is our own, and of course we're still running, so we won't take over the lock. Hence we'll be stuck waiting for a lock that we left behind ourselves. To avoid this, always unlink the lockfile. This preserves the invariant that self.held > 0 is equivalent to the lockfile existing on disk.

i18n: cache the result of every gettext call In looking at profiler output for 'hg log' on mozilla-central, I noticed we spent a _huge_ amount of time in gettext relative to what it's doing. Caching provides a roughly 15% performance improvement even on repositories as small as hg. == hg repo on linux == Before: % cumulative self time seconds seconds name 5.05 0.19 0.19 i18n.py:62:gettext 4.84 0.18 0.18 revlog.py:88:decompress 2.95 0.17 0.11 changelog.py:201:node 2.32 0.09 0.09 ui.py:577:write 2.11 0.08 0.08 i18n.py:72:gettext 2.11 0.08 0.08 obsolete.py:196:_fm0readmarkers 1.89 0.07 0.07 obsolete.py:569:_load 1.68 0.63 0.06 localrepo.py:29:__get__ real 0m4.026s user 0m3.993s sys 0m0.034s After: % cumulative self time seconds seconds name 8.05 0.26 0.26 revlog.py:88:decompress 2.68 0.22 0.09 color.py:395:write 2.20 0.07 0.07 obsolete.py:196:_fm0readmarkers 1.95 0.06 0.06 obsolete.py:174:_fm0readmarkers 1.95 0.06 0.06 ui.py:577:write 1.95 0.06 0.06 util.py:1228:datestr 1.71 0.06 0.06 utf_8.py:16:decode 1.71 0.06 0.06 revlog.py:273:__len__ real 0m3.519s user 0m3.447s sys 0m0.073s == mozilla-central repo on linux == Before: % cumulative self time seconds seconds name 7.72 2.35 2.35 revlog.py:88:decompress 4.46 1.36 1.36 i18n.py:62:gettext 2.22 0.67 0.67 i18n.py:72:gettext 2.19 1.14 0.67 changelog.py:201:node 2.16 0.66 0.66 ui.py:577:write 1.96 0.60 0.60 utf_8.py:16:decode 1.93 1.97 0.59 color.py:395:write 1.85 0.81 0.56 changelog.py:136:tip real 0m30.822s user 0m30.660s sys 0m0.149s After: % cumulative self time seconds seconds name 9.82 2.49 2.49 revlog.py:88:decompress 2.67 1.31 0.68 localrepo.py:29:__get__ 2.57 0.65 0.65 utf_8.py:16:decode 2.48 1.01 0.63 changelog.py:201:node 2.10 0.82 0.53 changelog.py:136:tip 2.01 0.51 0.51 ui.py:577:write 1.91 0.49 0.49 util.py:1232:datestr 1.85 1.65 0.47 color.py:395:write real 0m25.619s user 0m25.446s sys 0m0.166s == cpython repo on os x = Before: % cumulative self time seconds seconds name 5.05 1.35 1.35 cmdutil.py:982:_show 4.59 1.22 1.22 revlog.py:274:__len__ 3.98 1.06 1.06 i18n.py:62:gettext 3.91 1.04 1.04 revlog.py:1016:revision 3.68 0.98 0.98 revlog.py:337:parents 3.45 0.92 0.92 revlog.py:88:decompress 2.91 0.78 0.78 revlog.py:309:rev 2.62 0.70 0.70 revlog.py:1033:revision real 0m30.414s user 0m28.145s sys 0m0.541s After: % cumulative self time seconds seconds name 7.98 1.66 1.66 cmdutil.py:982:_show 6.83 1.42 1.42 changelog.py:46:decodeextra 5.18 1.08 1.08 revlog.py:274:__len__ 3.94 0.82 0.82 revlog.py:1016:revision 3.41 0.71 0.71 revlog.py:309:rev 3.32 0.69 0.69 revlog.py:88:decompress 2.99 0.63 0.62 revlog.py:1033:revision 2.69 0.56 0.56 revlog.py:341:start real 0m22.811s user 0m21.883s sys 0m0.397s

util.system: avoid buffering of subprocess output when it is piped util.system() copies subprocess' output through pipe if output file is not stdout. Because a file iterator has internal buffering, output won't be flushed until enough data is available. Therefore, it could easily miss important messages such as "waiting for lock".

bundle2: client side support for a part to import external bundles Bundle2 opens doors to advanced features allowing to reduce load on mercurial servers, and improve clone experience for users on unstable or slow networks. For instance, it could be possible to pre-generate a bundle of a repository, and give a pointer to it to clients cloning the repository, followed by another changegroup with the remainder. For significantly big repositories, this could come as several base bundles with e.g. 10k changesets, which, combined with checkpoints (not part of this change), would prevent users with flaky networks from starting over any time their connection fails. While the server-side support for those features doesn't exist yet, it is preferable to have client-side support for this early-on, allowing experiments on servers only requiring a vanilla client with bundle2 enabled.

bundle2: remove _getbundleextrapart Extensions can now add new part generators with the getbundle2partsgenerator decorator instead.

glossary: add deprecated and experimental

test-merge5: remove obsolete '-y' and its motivating comment The '-y' in 'hg update -y' was once needed to answer questions about modify/delete conflicts. That is no longer needed, so remove the '-y' and the comment justifying its use.

test-resolve.t: use redirection to /dev/null instead of grep -q

resolve: run happily after conflict-free merge

test-resolve: add tests for re-merge

test-resolve: add more tests for in conflict-free states We already have a test for 'hg resolve -m' when there is no merge in progress. Add one for 'hg resolve --all' as well. Also add tests for both --all and -m when there is a merge without conflicts in progress. They should both be successful, just as if there had been conflicts that had been marked resolved. However, that is currently broken, so mark the tests broken for now. The behavior will be fixed in a later patch.

test-resolve: add test resolving one of two files The tests for resolve are missing a lot of cases. Let's start by adding another file to the test repo, so we can test resolving one of two files.

test-resolve: clarify test descriptions and consistently use "should"

revset-phases: prefetch attributes in phasesrelated revsets Pre-fetching attributes gives a significant performance boost. Such is Python. draft() 0) wall 0.011661 comb 0.010000 user 0.010000 sys 0.000000 (best of 205) 1) wall 0.009804 comb 0.000000 user 0.000000 sys 0.000000 (best of 231) draft() - ::bookmark() 0) wall 0.014173 comb 0.010000 user 0.010000 sys 0.000000 (best of 177) 1) wall 0.012966 comb 0.010000 user 0.010000 sys 0.000000 (best of 182)

revset-phases: do not cache phase-related filters The phase retrieval is fast enough to not require caching the result of the functions. draft() 0) wall 0.017209 comb 0.020000 user 0.020000 sys 0.000000 (best of 149) 1) wall 0.011654 comb 0.010000 user 0.010000 sys 0.000000 (best of 186) public() 0) wall 0.018687 comb 0.010000 user 0.010000 sys 0.000000 (best of 128) 1) wall 0.013290 comb 0.010000 user 0.010000 sys 0.000000 (best of 181) secret() 0) wall 0.017464 comb 0.020000 user 0.020000 sys 0.000000 (best of 127) 1) wall 0.011499 comb 0.000000 user 0.000000 sys 0.000000 (best of 196) draft() - ::bookmark() 0) wall 0.020099 comb 0.020000 user 0.020000 sys 0.000000 (best of 127) 1) wall 0.014399 comb 0.020000 user 0.020000 sys 0.000000 (best of 169)

changectx: issue a FilteredRepoLookupError when applicable We capture FilteredxxxError and issue a FilteredRepoLookupError instead with a sightly different messsge. The message will likely get more improvement in the future. error: filtered revision '4'

repoview: add a FilteredRepoLookupError This exception is a more precise RepoLookupError that will allow us to issue a special message when we end up accessing a filtered revision.

repoview: add a FilteredLookupError class This exception is a more precise LookupError that will allow us to issue a special message when we end up accessing a filtered revision.

repoview: add a FilteredIndexError class This exception is a more precise IndexError that will allow us to issue a special message when we end up accessing a filtered revision.

changectx: move `IndexError` handling in the top level try except This one can be handled by the top level catching.

changectx: wrap the `changeid` processing in a try/except We are going to introduce more precise exception classes for filtered nodes. So we will have to upgrade them to the `RepoLookupError` level here. We wrap the whole thing into a try/except to ease this future catching. Some of the current exception catching will be moved in this one. But the current changeset focuses on code movement only.

bundle2: detect and disallow a negative chunk size We have no usage planned for 2/3 of them and the support for the planned usecase is not here yet. So we raise a BundleValueError when encountered

bundle2: add an UnsupportedPartError We need the BundleValueError for format errors not related to part support. So we add a specific class for part-support errors.

bundle2: change header size and make them signed (new format) We are changing all integers that denote the size of a chunk to read to int32. There are two main motivations for that. First, we change everything to the same width (32 bits) to make it possible for a reasonably agnostic actor to forward a bundle2 without any extra processing. With this change, this could be achieved by just reading int32s and forwarding chunks of the size read. A bit a smartness would be logic to detect the end of stream but nothing too complicated. Second, we need some capacity to transmit special information during the bundle processing. For example we would like to be able to raise an exception while a part is being read if this exception happend while this part was generated. Having signed integer let us use negative numbers to trigger special events during the parsing of the bundle. The format is renamed for B2X to B2Y because this breaks binary compatibility. The B2X format support is dropped. It was experimental to allow this kind of things. All elements not directly related to the binary format remain flagged "b2x" because they are still compatible.

bundle2: extract processing of part into its own function This is code movement only. This will be useful to have it separated for reuse purposes. We plan to introduce a new feature to the bundle format that allow inserting a part in the middle of another part payload. This will be useful to transmit a exception raised during a part generation.

bundle2: add a test for exceptions raised during the generation process We would like exceptions raised during the generation process to be gracefully handled on the receiver side. We add a test for it. It shows that we are not doing it yet.

dagwalker: drop a useless intermediate variable The variable used to contain revs.set() but as the `.set()` has been removed it is now useless.

revset-node: speedup by a few hundred fold Instead of checking all elements of the subset against a single rev, just check if this rev is in the subset. The old way was inherited from when the subset was a list. Non surprise, this provide massive speedup. id("d82e2223f132") before) wall 0.008205 comb 0.000000 user 0.000000 sys 0.000000 (best of 302) after) wall 0.000069 comb 0.000000 user 0.000000 sys 0.000000 (best of 34518) revset #1: public() and id("d82e2223f132") before) wall 0.019763 comb 0.020000 user 0.020000 sys 0.000000 (best of 124) after) wall 0.000101 comb 0.000000 user 0.000000 sys 0.000000 (best of 20130)

revset-only: use `subset &` instead of filtering The & version is more likely to be optimised. only(.) before) wall 0.003216 comb 0.000000 user 0.000000 sys 0.000000 (best of 768) after) wall 0.001086 comb 0.000000 user 0.000000 sys 0.000000 (best of 2231) only(default, stable) before) wall 0.018469 comb 0.020000 user 0.020000 sys 0.000000 (best of 138) after) wall 0.015888 comb 0.010000 user 0.010000 sys 0.000000 (best of 156)

revset-_ancestor: use & instead of filter The & operation is more likely optimised. ::10 before) wall 0.028189 comb 0.030000 user 0.030000 sys 0.000000 (best of 100) after) wall 0.001050 comb 0.000000 user 0.000000 sys 0.000000 (best of 2326) ::tip before) wall 0.081132 comb 0.080000 user 0.080000 sys 0.000000 (best of 100) after) wall 0.055418 comb 0.050000 user 0.050000 sys 0.000000 (best of 100)

obsstore: record data as floating point in fm0 format For python struct module, "d" is double. But for python string formating, "d" is integer. We want to preserve the floating point nature of the data, so we store it in the metadata as floating point. We use "%r" to make sure we get as many significant digitis as necessary to restore the float to the exact same value on the other side. The fm1 is transmitting the information as float. The lack of this made fm1-stored markers not survive a round-trip to fm0 leading to duplicated markers (or two markers very alike).

bundle2: add a comment about addchangegroup source and url

pull: call the `b2x-transactionclose` hook when closing the transaction We need a wider set of hooks to process all the changes that happened during the pull transaction. We reuse the experimental `b2x-transactionclose` hook set from server's unbundle for consistency. This hook is experimental and will not remains as-is forever, but this will open the door for experimentation in 3.2.

revset-only: use __nonzero__ to check if a revset is empty For some smartsets, computing length is more expensive than checking if the set is empty.

_spanset: drop __getitem__ implementation It is expensive and not part of the official smartset API.

filteredset: drop __getitem__ implementation It is expensive and not part of the official smartset API.

generatorset: implement __len__ It was the only smartset class without a `__len__` implementation.

revset: make __len__ part of the offical API It is common for code to ask for the length of a revset. In fact, all but generatorset already implement it.

cmdserver: add service that listens on unix domain socket and forks process Typical use case of 'unix' mode is a background hg daemon. $ hg serve --cmdserver unix --cwd / -a /tmp/hg-`id -u`.sock Unlike 'pipe' mode in which parent process keeps stdio channel, 'unix' server can be detached. So clients can freely connect and disconnect from server, saving Python start-up time. It might be better to write "--cmdserver socket -a unix:/sockpath" instead of "--cmdserver unix -a /sockpath" in case hgweb gets the ability to listen on unix domain socket.

test-commandserver: add connector for unix domain socket server The next patch will introduce --cmdserver unix.

test-commandserver: allow check() to make connection in different way The next patch will add connector for 'unix' mode server.

test-commandserver: remove unused repopath argument from check() Instead of repopath, check() will receive connect() function as argument. It will allow to connect to server of different mode.

cmdserver: make server streams switchable In 'unix' mode, server instance will be created per connection, and fin/fout are set to socket files.

cmdserver: switch service objects by mode server class will be changed to accept fin/fout pair instead of mode string so that it can interact with socket files.

cmdserver: wrap 'pipe' mode server by service object This is the stub for new mode that will listen for connections on unix domain socket. Though --daemon option is not banned in 'pipe' mode, it is useless because the detached 'pipe' mode server exits immediately due to null stdin. Should it abort if --daemon is specified with --cmdserver pipe or --stdio?

histedit: miscellaneous style cleanups Fix some mis-wrapped lines, re-wrap some lines to be more compact, remove superfluous parens.

histedit: update docstring on histeditstate.read() The docstring showed the previous class-less heritage of read(). Now that it's classy, update the docstring.

histedit: remove now-superfluous repo argument from processreplacement Spotted by Olle on the mailing list during review.

histedit: move locks into state Allow action functions to control the locks. This is necessary for an implementation of x/exec or similar.

histedit: read state from histeditstate Read the state in histeditstate. This allows us to correctly update internal variables when necessary without having to recreate a new state. When we read a state in _histedit state while we will already have state passed from histedit(), we can read the state in place and don't have to merge two histeditstates.

histedit: pass state to action functions

histedit: pass state to processreplacement

histedit: pass state to boostrapcontinue Pass the state to boostrapcontinue and remove the unecessary passing of repo.

histedit: let the state expose a context but serialize correctly to nodes The histedit code often expects a context. However histedit hands around the tuple for the serialization and therefore hand over a parentctxnode. This leads to code having to return a context based on the parentctxnode. We let the state only return a context but correctly serialize and deserialze to a node.

histedit: add clear method to remove state Encapsulate the unlinking histedit-state and use the vfs layer instead of os.unlink.

histedit: use state object where necessary We are using the properties from the state object where necessary and ensure that the state is set correctly.

histedit: add histedit state class Add an histeditstate class that is intended to hold the current state. This allows us encapsulate the state and avoids passing around a tuple which is based on the serialization format. In particular this will give actions more control over the state and allow external sources to have more control of histedits behavior, e.g. an external implementation of x/exec.

import-checker: check modules for pure Python build correctly Before this patch, "import-checker.py" just replaces "/" in specified filenames by ".". This makes modules for pure Python build belong to "mercurial.pure" package, and prevents "import-checker.py" from correctly checking about cyclic dependency in them. This patch discards "pure" component from fully qualified name of such modules. To avoid discarding "pure" from the module name of standard libraries unexpectedly, this patch allows "dotted_name_of_path" to discard "pure" only from Mercurial specific modules, which are specified via command line arguments.

import-checker: treat "from mercurial import XXXX" style correctly Before this patch, "import-checker.py" assumes that the name of Mercurial module recognized by "imported_modules" doesn't have package part: for example, "util". This is reason why "import-checker.py" always builds fully qualified module name up relatively, if the given module doesn't belong to standard Python library. But in fact, modules imported in "from mercurial import XXXX" style already have fully qualified name: for example, "mercurial.util" module imported by "mercurial.parsers" is treated as "mercurial.mercurial.util" because of building module name up relatively. This prevents "import-checker.py" from correctly checking about cyclic dependency in them. This patch avoids building module name up relatively, also if module name starts with "mercurial.", to treat modules imported in "from mercurial import XXXX" style correctly.

encoding: avoid cyclic dependency around "parsers" in pure Python build 80f2b63dd83a brought "asciilower" and "import parsers" into "encoding.py". This works fine with "parsers" module in C implementation, but doesn't with one in pure Python implementation, because the latter causes cyclic dependency below and aborting execution: util => i18n => encoding => parsers => util This patch delays importing "parsers" module until it is really needed, to avoid cyclic dependency around "parsers" in pure Python build.

pull: add source information to the transaction The source information can, should be applied once when opening the transaction for the pull. This will lets element processed within a bundle2 be aware of them and open the door to running a set of hooks when closing this pull transaction. This is similar to what is done in server's unbundle call.

changegroup: store source and url in the `hookargs` dict We store the source and url of the current data into `transaction.hookargs` this let us inherit it from upper layers that may have created a much wider transaction. We have to modify bundle2 at the same time to register the source and url in the transaction. We have to do it in the same patch otherwise, the `addchangegroup` call would fill these values and the hook calling will crash because of the duplicated 'source' and 'url' arguments passed to the hook call.

prechangegroup: use hook argument from the transaction There can be useful data in there (eg: bundle2 related one)

addchangegroup: call `prechangegroup` hook after transaction retrieval We want to reused some possible information stored in the transaction `hookargs` dict that may be stored by something handling the transaction at an upper level (eg: bundle2) So we move the running of the hooks after transaction creation. This has no visible effects (but an empty transaction roolback if the hook fails) because nothing had happened in the transaction yet.

addchangegroup: get the `node` argument of `incoming` hook from transaction The transaction is now carrying hook-related informations. So we use it to retrieve the `node` argument. This will also carry around all kinds of other useful informations (like: "are we in a bundle2 processing")

dicthelpers: delete now that they are no longer used

manifest: transpose pair of pairs from diff() It makes more sense for the file nodeids and returned from diff() to be ((n1,fl1),(n2,fl2)) than ((n1,n2),(fl1,fl2)), so change it to the former.