linkrev: also adjust linkrev when bootstrapping 'follow' revset The follow revset (used by `hg log --follow`) now uses the new 'introrev' method to bootstrap its traversal. This catches issues from linkrev-shadowing of the changesets introducing the version of a file in source changeset. A new test has been added to display pathological cases. Another test is affected because it was meant to test this behavior but actually failed to do so for half of the output. The output are now similar.

linkrev: introduce an 'introrev' method on filectx The previous changeset properly fixed the ancestors computation, but we need to ensure that the initial filectx is also using the right changeset. When asking for log or annotation from a certain point, the first step is to define the changeset that introduced the current file version. We cannot just pick the "starting point" changesets as it may just "use" the file revision, unchanged. Currently, we were using 'linkrev' for this purpose, but this exposes us to unexpected branch-jumping when the revision introducing the starting point version is itself linkrev-shadowed. So we need to take the topology into account again. Therefore, we introduce an 'introrev' function, returning the changeset which introduced the file change in the current changeset. This function will be used to fix linkrev-related issues when bootstrapping 'hg log --follow' and 'hg annotate'. It reuses the '_adjustlinkrev' function, extending it to allow introspection of the initial changeset too. In the previous usage of the '_adjustlinkrev' the starting rev was always using a children file revisions, so it could be safely ignored in the search. In this case, the starting point is using the revision of the file we are looking, and may be the changeset we are looking for.

filectx.parents: enforce changeid of parent to be in own changectx ancestors Because of the way filenodes are computed, you can have multiple changesets "introducing" the same file revision. For example, in the changeset graph below, changeset 2 and 3 both change a file -to- and -from- the same content. o 3: content = new | | o 2: content = new |/ o 1: content = old In such cases, the file revision is create once, when 2 is added, and just reused for 3. So the file change in '3' (from "old" to "new)" has no linkrev pointing to it). We'll call this situation "linkrev-shadowing". As the linkrev is used for optimization purposes when walking a file history, the linkrev-shadowing results in an unexpected jump to another branch during such a walk.. This leads to multiple bugs with log, annotate and rename detection. One element to fix such bugs is to ensure that walking the file history sticks on the same topology as the changeset's history. For this purpose, we extend the logic in 'basefilectx.parents' so that it always defines the proper changeset to associate the parent file revision with. This "proper" changeset has to be an ancestor of the changeset associated with the child file revision. This logic is performed in the '_adjustlinkrev' function. This function is given the starting changeset and all the information regarding the parent file revision. If the linkrev for the file revision is an ancestor of the starting changeset, the linkrev is valid and will be used. If it is not, we detected a topological jump caused by linkrev shadowing, we are going to walk the ancestors of the starting changeset until we find one setting the file to the revision we are trying to create. The performance impact appears acceptable: - We are walking the changelog once for each filelog traversal (as there should be no overlap between searches), - changelog traversal itself is fairly cheap, compared to what is likely going to be perform on the result on the filelog traversal, - We only touch the manifest for ancestors touching the file, And such changesets are likely to be the one introducing the file. (except in pathological cases involving merge), - We use manifest diff instead of full manifest unpacking to check manifest content, so it does not involve applying multiple diffs in most case. - linkrev shadowing is not the common case. Tests for fixed issues in log, annotate and rename detection have been added. But this changeset does not solve all problems. It fixes -ancestry- computation, but if the linkrev-shadowed changesets is the starting one, we'll still get things wrong. We'll have to fix the bootstrapping of such operations in a later changeset. Also, the usage of `hg log FILE` without --follow still has issues with linkrev pointing to hidden changesets, because it relies on the `filelog` revset which implement its own traversal logic that is still to be fixed. Thanks goes to: - Matt Mackall: for nudging me in the right direction - Julien Cristau and Rémi Cardona: for keep telling me linkrev bug were an evolution show stopper for 3 years. - Durham Goode: for finding a new linkrev issue every few weeks - Mads Kiilerich: for that last rename bug who raise this topic over my anoyance limit.

context: remove unreliable accessor methods from committablectx There are two caching routes for (propertycache-ed) "_status" below in committablectx: - invoking "status()": "dirstate.status()" is invoked, and the result of it is cached into "_status". In this case, any of "listignored", "listclean" and "listunknown" may be True. - accessing "_status" directly before "status()": Own "status()" is invoked, but all of "listignored", "listclean" and "listunknown" arguments are False, in this case. "ignored"/"clean"/"unknown" accessor methods of "committablectx" use corresponded fields of "_status", but these fields aren't reliable, because these fields are empty when: - "_status" method is executed before accessors, or - "status()" is executed with "list*=False" before accessors In addition to it, these accessors aren't used in the recent Mercurial implementation. At least, removing them doesn't cause any test failures.

context: cache self._status correctly at workingctx.status Before this patch, "workingctx.status" always replaces "self._status" by the recent result, even though: - status isn't calculated against the parent of the working directory, or - specified "match" isn't "always" one (status is only visible partially) If "workingctx" object is shared between some procedures indirectly referring "ctx._status", this incorrect caching may cause unexpected result: for example, "ctx._status" is used via "manifest()", "files()" and so on. To cache "self._status" correctly at "workingctx.status", this patch overwrites "self._status" in "workingctx._buildstatus" only when: - status is calculated against the parent of the working directory, and - specified "match" is "always" one This patch can be applied (and effective) only on default branch, because procedure around "basectx.status" is much different between stable and default: for example, overwriting "self._status" itself is executed not in "workingctx._buildstatus" but in "workingctx._poststatus", on stable branch.

filectx.parents: also fetch the filelog of rename source too we are going to need this filelog for the linkrev adjustment, so we better normalise the list and have the filelog in all case. This is done in a previous changeset to help readability.

Added signature for changeset 1265a3a71d75

Added tag 3.2.4 for changeset 1265a3a71d75

largefiles: backout f72d73937853 - linear updates handle m -> a differently f72d73937853 introduced a significant performance regression: All largefiles were marked 'normallookup' by linear (or noop) updates and had to be rehashed by the next command. The previous change introduced a different solution to the problem f72d73937853 solved and we can thus back it out again.

largefiles: mark lfile as added in lfdirstate when the standin is added This is an alternative solution to the problem addressed by f72d73937853. This implementation has the advantage that it doesn't mark clean largefiles as normallookup. We can thus avoid repeated rehashing of all largefiles when f72d73937853 is backed out. This implementation use the existing 'lfmr' actions that 23fe278bde43 introduced for handling another part of the same cases.

tests: add test coverage for lfdirstate invalidation of linear update f72d73937853 introduced a significant performance regression: All largefiles are marked 'normallookup' in lfdirstate by linear (or noop) updates and has to be rehashed by the next command. To avoid such regressions, keep an eye on the dirstate content after a plain 'hg up'.

test-subrepo-git: ignore global git config This was causing a test failure for people with company-wide settings. Still need a way to ignore local config.

tests/autodiff.py: explicitly only honor feature diffopts This test extension manages the opts it cares about on its own anyway.

cmdutil.changeset_printer: explicitly honor all diffopts This is used in hg log -p so the output is expected to be the same as that of hg diff.

export: explicitly honor all diffopts This is slightly more controversial than diff, but we hope that HGPLAIN=1 covers all the format-breaking ones. A possible alternative here that breaks BC is to honor all opts except the whitespace ones.

webcommands.annotate: explicitly only honor whitespace diffopts The whitespace ones are the only ones the annotate logic cares about anyway, so there's no visible impact.

filectx.parents: filter nullrev parent sooner We are going to introduce a linkrev-correction phases when computing parents. It will be more convenient to have the nullid parent filtered out earlier. I had to make a minimal adjustment to the rename handling logic to keep it functional. That logic have been documented in the process since it took me some time to check all the cases out.

context: catch FilteredRepoLookupError instead of RepoLookupError Now that we have a more specialised exception, lets use it when we meant to catch the more specialised case.

narrowmatcher: propagate the rel() method The full path is propagated to the original match object since this is often used directly for printing a file name to the user. This is cleaner than requiring each caller to join the prefix with the file name prior to calling it, and will lead to not having to pass the prefix around separately. It is also consistent with the bad() and abs() methods in terms of the required input. The uipath() method now inherits this path building property. There is no visible change in path style for rel() because it ultimately calls util.pathto(), which returns an os.sep based path. (The previous os.path.join() was violating the documented usage of util.pathto(), that its third parameter be '/' separated.) The doctest needed to be normalized to '/' separators to avoid test differences on Windows, now that a full path is returned for a short filename. The test changes are to drop globs that are no longer necessary when printing an absolute file in a subrepo, as returned from match.uipath(). Previously when os.path.join() was used to add the prefix, the absolute path to a file in a subrepo was printed with a mix of '/' and '\'. The absolute path for a file not in a subrepo, as returned from match.uipath(), is still purely '/' based.

match: add the abs() method This is a utility to make it easier for subrepos to convert a file name to the full path rooted at the top repository. It can replace the various path joining lambdas, and doesn't require the prefix to be passed into the method that wishes to build such a path. The name is derived from the following pattern in annotate() and other places: name = ((pats and rel) or abs) The pathname separator is not os.sep in part to avoid confusion with variables named 'abs' or similar that _are_ '/' separated, and in part because some methods like cmdutils.forget() and maybe cmdutils.add() need to build a '/' separated path to the root repo. This can replace the manual path building there.

merge with stable

posix: quote the specified string only when it may have to be quoted This patch makes "posix.shellquote" examine the specified string and quote it only when it may have to be quoted for safety, like as the previous patch for "windows.shellquote". In fact, on POSIX environment, quoting itself doesn't cause issues like issue4463. But (almost) equivalent quoting policy can avoid examining test result differently on POSIX and Windows (even though showing command line with "%r" causes such examination in "test-extdiff.t"). The last hunk for "test-extdiff.t" in this patch isn't needed for the previous patch for "windows.shellquote", because the code path of it is executed only "#if execbit" (= avoided on Windows).

windows: quote the specified string only when it has to be quoted Before this patch, "windows.shellquote" (as used as "util.shellquote") always quotes specified strings with double quotation marks, for external process invocation. But some problematic applications can't work correctly, when command line arguments are quoted: see issue4463 for detail. On the other hand, quoting itself is needed to specify arguments containing whitespaces and/or some special characters exactly. This patch makes "windows.shellquote" examine the specified string and quote it only when it may have to be quoted for safety.

extdiff: rename the name of an argument for readability To reduce amount of changes for review-ability, previous patch uses "args" as argument name of "dodiff()", even though "args" includes also the name of command to be executed (or full-path of it). This patch replaces "args" by more appropriate name "cmdline".

extdiff: avoid unexpected quoting arguments for external tools (issue4463) Before this patch, all command line arguments for external tools are quoted by the combination of "shlex.split" and "util.shellquote". But this causes some problems. - some problematic commands can't work correctly with quoted arguments For example, 'WinMerge /r ....' is OK, but 'WinMerge "/r" ....' is NG. See also below for detail about this problem. https://bitbucket.org/tortoisehg/thg/issue/3978/ - quoting itself may change semantics of arguments For example, when the environment variable CONCAT="foo bar baz': - mydiff $CONCAT => mydiff foo bar baz (taking 3 arguments) - mydiff "$CONCAT" => mydiff "foo bar baz" (taking only 1 argument) For another example, single quoting (= "util.shellquote") on POSIX environment prevents shells from expanding environment variables, tilde, and so on: - mydiff "$HOME" => mydiff /home/foobar - mydiff '$HOME' => mydiff $HOME - "shlex.split" can't handle some special characters correctly It just splits specified command line by whitespaces. For example, "echo foo;echo bar" is split into ["echo", "foo;echo", "bar"]. On the other hand, if quoting itself is omitted, users can't specify options including space characters with "--option" at runtime. The root cause of this issue is that "shlex.split + util.shellquote" combination loses whether users really want to quote each command line elements or not, even though these can be quoted arbitrarily in configurations. To resolve this problem, this patch does: - prevent configurations from being processed by "shlex.split" and "util.shellquote" only (possibly) "findexe"-ed or "findexternaltool"-ed command path is "util.shellquote", because it may contain whitespaces. - quote options specified by "--option" via command line at runtime This patch also makes "dodiff()" take only one "args" argument instead of "diffcmd" and "diffopts. It also omits applying "util.shellquote" on "args", because "args" should be already stringified in "extdiff()" and "mydiff()". The last hunk for "test-extdiff.t" replaces two whitespaces by single whitespace, because change of "' '.join()" logic causes omitting redundant whitespaces.

subrepo: add forgotten annotation for reverting git subrepos Support for reverting git subrepos was added earlier, but the annotation to handle any subrepo errors was forgotten.

subrepo: add full revert support for git subrepos Previously, revert was only possible if the '--no-backup' switch was specified. Now, to support backups, we explicitly go over all modified files in the subrepo.

setup: don't fail when Python doesn't have the cygwinccompiler package Some Python installations like the ones available from the optware project for the Synology DiskStation NASes don't have that package, which means running the setup script will crash and exit right away. Instead, we now just use an empty/fake class for the HackedMingw32CCompiler, which we likely won't use anyway.

tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)

tests: run 'cvs init' only on non-existent directories (issue4482) Do not create cvsroot directory since cvs-1.12 errors out if CVSROOT exists before 'cvs init'.

remove: use vfs instead of os.path + match.rel() for filesystem checks

forget: use vfs instead of os.path + match.rel() for filesystem checks

tests: make a multi-statement hook in bundle2-exchange Windows compatible This is similar to the fix in 7dd1870120b2.

sshpeer: more thorough shell quoting This fixes an issue spotted by Jesse Hertz.

merge with stable

i18n-ja: synchronized with 3b84bde06d17

i18n-pt_BR: synchronized with 53a65929ef1f

localrepo: use the vfs join method to implement the localrepo join method This will make it possible to customize the behavior of the join method by changing the vfs class (e.g. by using the altvfs" class introduced recently). Note that we could have modified the VFS join methods to acept a set of optional paths in the same way thta the localrepo join method does. However it seemed simpler to simply call os.path.join before calling self.vfs.join.

localrepo: introduce shared method to check if a repository is shared Up until now we compared the "path" and "sharedpath" repository properties to check if a repository is shared. This was relying an implementation detail of shared repositories. In order to make it possible to change the way shared repositories are implemented, we encapsulate this check into its own localrepo method, called shared. This new method returns None if the repository is shared, and something else (for now a string describing the short of share) otherwise. The reason why I did not call this method "isshared" and made it return a boolean is that I plan to introduce a new type of shared repository soon. # NOTES: This is the first patch in a series whose purpose is to add support for creating "full repository shares", which are repositories that share everything with the repository source except their current revision, branch and bookmark. This series is RFC because I am not very sure of some of the solutions I took. Comments are welcome!

trydiff: use 'ctx1.flags()' for symmetry with 'ctx2.flags()'

trydiff: use 'not in addedset' for symmetry with 'not in removedset' With the previous change in place, we can safely use 'addedset'.

trydiff: simplify checking for additions In the body of the loop in trydiff(), there are conditions like: addedset or (f in modifiedset and to is None) The second half of that expression is to account for the fact that merge-in additions appear as additions. By instead fixing up the sets of modified and added files to compensate for this fact, we can simplify the body of the loop. It also fixes one case where the addedset was checked without the additional check (the "have we already reported a copy above?" case in the code, also see fixed test case). The similar condition with 'removedset' in it seems to have served no purpose even before this change, so it could have been simplified even before.

trydiff: extract 'date2' variable like existing 'date1' Note that there is a comment saying "ctx2 date may be dynamic". The comment was introduced in dccb83241dd0 (patch: use contexts for diff, 2006-12-25), but it seems like it stopped being dynamic in that very changeset -- before that changeset, the date seems to have been the file's mtime, but after the changeset, it seems to be the changeset's timestamp (current time for workingctx). Since no one seems to have missed the "dynamicness", let's simplify and extract a date2 for symmetry with date1.

trydiff: use sets, not lists, for containment checks This only has a noticeable effect on diffs touching A LOT of files. For example, it takes hg diff -r FIREFOX_AURORA_30_BASE -r FIREFOX_AURORA_35_BASE from 1m55.465s to 1m32.354s. That diff has over 50k files.

largefiles: fix unused import Caught by pyflakes.

largefiles: look at unfiltered().lfstatus to allow status() to be filtered The comment about status being buggy with a repo proxy seems to be that status wasn't being redirected to testing the largefiles themselves- lfstatus was always False, and so the original status method was being called instead of doing the largefiles processing. This is because when the various largefile command overrides call 'repo.lfstatus = True', __setattr__() in repoview is in turn setting the value on the unfiltered repo, and the value in the filtered repo remains False. Explicitly looking at the attribute on the unfiltered repo keeps all views in sync.

largefiles: eliminate a duplicate message when removing files in verbose mode There is no --after for addremove, so the printing for addremove can be hoisted out of the 'not after' check. The difference between the two remove messages reflects the existing difference between core remove and core addremove styles for printing the file. There are still some pre-existing issues here. Core addremove only prints on inexact matches or when verbose. But since the largefiles that are being removed are passed to removelargefiles() as a pattern list, there is never an inexact match, which would keep the largefiles from being printed at all unless verbose is specified. Therefore, the output is a little more aggressive than core. The addremove print style here is also inconsistent with core- it should use matcher.uipath(f) instead of f. These can be fixed once a matcher is passed in.

largefiles: ensure that the standin files are available in getlfilestoupload() The function only adds the hash content of the file to the set to upload if the file in the ctx is a standin. It is called by overrides.summaryremotehook(), which is called in the summary method. The largefiles extension switches 'lfstatus' on in summary, so the standins shouldn't be visible when obtaining a context there. The reason this wasn't noticed before is that the 'lfstatus' attribute is only being set on the unfiltered repo because of how repoview delegates attribute assignment. Therefore any filtered view will return a context containing standins, whether or not 'lfstatus' was set in the various overrides methods. That will be fixed in the next patch. But without this change, the next patch would have test failures for 'summary --large' stating there are no files to upload.

merge: move checking of unknown files out of manifestmerge() This moves most reading of filelogs out of manifestmerge, making it easy for a narrow clone extension to filter out or translate unwanted actions before any filelogs are read. The only call left is inside of copies.mergecopies(), which can be overridden separately at a lower level.

merge: extract method for checking for conflicting untracked file Now that the functionality is collected in one place, let's extract it to a method.

merge: create 'cm' action for 'get or merge' case We still have one case of a call to _checkunknownfile() in manifestmerge(): when force=True and branchmerge=True and the remote side has a file that the local side doesn't. This combination of arguments is used by 'hg merge --force', but also by rebase and unshelve. In this scenario, we try to create the file from the contents from the remote, but if there is already a local untracked file in place, we merge it instead.

merge: don't overwrite untracked file at directory rename target When a directory was renamed and a new untracked file was added in the new directory and the remote directory added a file by the same name in the old directory, the local untracked file gets overwritten, as demonstrated by the broken test case in test-rename-dir-merge. Fix by checking for unknown files for 'dg' actions too. Since _checkunknownfile() currently expects the same filename in both contexts, we need to add a new parameter for the remote filename to it.

merge: remove constant tuple element from 'aborts' The second element of the tuples in the 'aborts' list is always 'ud', so let's remove it.

merge: collect checking for unknown files at end of manifestmerge() The 'c' and 'dc' actions include creating a file on disk and we need to check that no conflicting file exists unless force=True. Move two of the calls to _checkunknownfile() to a single place at the end of manifestmerge(). This removes some of the reading of filelogs from the heart of manifestmerge() and collects it in one place close to where its output (entries in the 'aborts' list) is used. Note that this removes the unnecessary call to _checkunknownfile() when force=True in one of the code paths.

merge: introduce 'c' action like 'g', but with additional safety _checkunknownfile() reads the filelog of the remote side's file. For narrow clones, the filelog will not exist for all files and we need a way to avoid reading them. While it would be easier for the narrow extension to just override _checkunknownfile() and make it ignore files outside the narrow clone, it seems cleaner to have manifestmerge() not care about filelogs (considering its name). In order to move the calls to _checkunknownfile() out, we need to be able to tell in which cases we should check for unknown files. Let's start by introducing a new action distinct from 'g' for this purpose. Specifically, the new action will be just like 'g' except that it will check that for conflicting unknown files first. For now, just add the new action type and convert it to 'g'.

merge: structure 'remote created' code to match table This does duplicate the call to _checkunknownfile(), but it will simplify future patches.

pushkey: run hook after the lock release The pushkey operation used to be in its own wireprotocol command and (in practice) always be lock free when running the hook. With bundle2, it happen in a greater scheme and a hook running locking command would get stuck. We now run such hooks after the lock release as similar hook do. Bundle2 test are altered to ensure we are lockfree at hook running time.

setup: use changes since latest tag instead of just distance For a Mercurial built on the merge from stable into default right after 3.2.2 was released -- 19ebd2f88fc7 -- the version number produced was "3.2.2+4". This is potentially misleading, since in reality the built Mercurial includes many more changes compared to 3.2.2. Change the versioning scheme so that we take into consideration all the changes present in the current revision that aren't present in the latest tag. For 19ebd2f88fc7 the new versioning scheme results in a version number of "3.2.2+256". This gives users a much better idea of how many changes have actually happened since the latest release. Since changessincelatesttag is always greater than or equal to the latesttagdistance, this will produce version numbers that are always greater than or equal to the old scheme. Thus there's minimal compatibility risk.

setup: use changessincelatesttag from archive if present changessincelatesttag gives one a better idea of how much the code has changed since. Since changessincelatesttag is always greater than or equal to the latesttagdistance (see previous patch for why), this will always produce version numbers greater than or equal to the previous scheme.

archive: store number of changes since latest tag as well This is different from latesttagdistance in that while latesttagdistance is defined to be the length of the longest path to the latest tag, changessincelatesttag is the number of changes contained in @ that aren't contained in the latest tag. So, if 't' is the latest tag in the repository below: t | v --o--o----o \ \ ..o..o..@ then latesttagdistance is 2, but changessincelatesttag is 4. Note that changessincelatesttag is always greater than or equal to the latesttagdistance -- that's because changessincelatesttag counts all the changes in the longest path since the latest tag, and possibly others. This is an important fact that we'll take advantage of in upcoming patches.

merge with stable

demandimport: blacklist distutils.msvc9compiler (issue4475) This module depends on _winreg, which is windows-only. Recent versions of setuptools load distutils.msvc9compiler and expect it to ImportError immediately when on non-Windows platforms, so we need to let them do that. This breaks in an especially mystifying way, because setuptools uses vars() on the imported module. We then throw an exception, which vars doesn't pick up on well. For example: In [3]: class wat(object): ...: @property ...: def __dict__(self): ...: assert False ...: In [4]: vars(wat()) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-4-2781ada5ffe6> in <module>() ----> 1 vars(wat()) TypeError: vars() argument must have __dict__ attribute Which is similar to the problem we run into.

largefiles: don't duplicate 'actions' into 'actionbyfile'

merge: make calculateupdates() return file->action dict This simplifies largefiles' overridecalculateupdates(), which no longer has to do the conversion it started doing in 38e55e55ae4d (largefiles: rewrite merge code using dictionary with entry per file, 2014-12-09). To keep this patch small, we'll leave the name 'actionbyfile' in overrides.py. It will be renamed in the next patch.

merge: let _forgetremoved() work on the file->action dict By moving the conversion from the file->action dict after _forgetremoved(), we make that method shorter by removing the need for the confusing 'xactions' variable.

merge: let _resolvetrivial() work on the file->action dict By moving the conversion from the file->action dict after _resolvetrivial(), we greatly simplify and clarify that method.

merge: let bid merge work on the file->action dict By moving the conversion from the file->action dict after the bid merge code, bid merge can be simplified a little. A few tests are affected by this change. Where we used to iterate over the actions first in order of the action type ('g', 'r', etc.) [1], we now iterate in order of filename. This difference affects the order of debug log statements. [1] And then in the non-deterministic order of files in the manifest dictionary (the order returned from manifest.diff()).

merge: write manifestmerge() using dictionary with entry per file In the same vein as 38e55e55ae4d (largefiles: rewrite merge code using dictionary with entry per file, 2014-12-09), rewrite manifestmerge() itself as dictionary with the filename as key. This will let us simplify some of the other code in merge.py and eventually drop the conversion in the largefiles code. No difference in speed could be detected (well within the noise level when run in Mozilla repo).

repoview: backout ced3ecfc2e57 Monkey patching repoview does not really work and making it really work will be really hard. So we better have it broken without complexity than broken with extra complexity.

largefile: explain why no monkey patching on a repoview The comment requested for investigations, here they are.

largefile: backout ca54fb3d71ce The hack for method monkey patching on repoview has been ruled out as fragile, so we are rolling it back. We'll expand the explanation in the next changeset.

incoming: handle phases the same as pull Now that bundlerepo can move phases safely, 'hg incoming' can share its phase handling code with pull to better reflect what would actually show up.

bundlerepo: retract phase boundary This patch makes bundrepo retract the phase boundary for new commits to 'draft' status, which is consistent with the behavior of 'hg unbundle'. The old behavior was for commits to appear with the same phase as their nearest ancestor in the base repository. This affects several classes of operation: * Inspecting a bundle with the -B flag * Treating a bundle file as a peer (old: everything public, new: everything draft) * Incoming command (neither old or new behavior is sensible -- fixed in next patch)

bundlerepo: implement safe phasecache This patch makes bundlerepo use a subclass of phasecache that will allow phase boundaries to be moved around, but will never write them to the underlying repository.

localrepo.__getitem__: add slicing support This allows the python slice syntax to be used with revision numbers when indexing into a repository, such as repo[8:]

ignore: resolve ignore files relative to repo root (issue4473) (BC) Previously these would be considered to be relative to the current working directory. That behavior is both undocumented and doesn't really make sense. There are two reasonable options for how to resolve relative paths: - relative to the repo root - relative to the config file Resolving these files relative to the repo root matches existing behavior with hooks. An earlier discussion about this is available at http://mercurial.markmail.org/thread/tvu7yhzsiywgkjzl. Thanks to Isaac Jurado <diptongo@gmail.com> for the initial patchset that spurred the discussion.

test-hgignore: add testing for ui.ignore I couldn't find any tests for this, and we're going to make changes here in upcoming patches.

tests: add missing globs for Windows I couldn't figure out how to glob the first chunk for Windows, so it's been conditionalized.

share: use the 'sharedpath' attr on repo instead of reloading from the file Seems like a useful optimization, and now the exact file content is not a concern.

share: fix source repo lookup on Windows The stored path contains platform specific separators, so splitting on '/.hg' returned the string unmodified on Windows. The source was then looked for in $source/.hg/.hg, which obviously fails. This caused cascading errors in test-share.t relating to the recent bookmark support.

help: suggest '-v -e' to get built-in aliases for extensions (issue4461) If extension name matches one of command names, suggest user to type 'hg help -v -e <extension>' to get full list of built-in aliases

test-check-commit-hg: clarify misleading "commit message rules" error The test case doesn't only check the commit message, but also the patch, which can result in confusing output like + Revision df6f06d17100 does not comply to commit message rules + ------------------------------------------------------ + 32: adds double empty line + + even when there are no double blank lines in the commit message. Drop the "commit message" part to make it less confusing.

keyword: handle resolve to either parent Merged files are considered modified at commit time even if only 1 parent differs. In this case we must use the change context of this parent for expansion. The issue went unnoticed for long because it is only apparent until the next update to the merge revision - except in test-keyword where it was always staring us in the face.

keyword: update test file syntax

branches: deprecate -a

largefiles: fix a spurious missing file warning with forget (issue4053) If an uncommitted and deleted file was forgotten, a warning would be emitted, even though the operation was successful. See the previous patch for 'remove -A' for the exact circumstances, and details about the cause.

largefiles: fix a spurious missing file warning with 'remove -A' (issue4053) The bug report doesn't mention largefiles, but the given recipe doesn't fail unless the largefiles extension is loaded. The problem only affected normal files, whether or not any largefiles are committed, and only files that have not been committed yet. (Files with an 'a' state are dropped from dirstate, not marked removed.) Further, if the named normal file never existed, the warning would be printed out twice. The problem is that the core implementation of remove() calls repo.status(), which eventually triggers a dirstate.walk(). When the file isn't seen in the filesystem during the walk, the exception handling finds the file in dirstate, so it doesn't complain. However, the largefiles implementation called status() again with all of the original files (including the normal ones, just dropped). This time, the exception handler doesn't find the file in dirstate and does complain. This simply excludes the normal files from the second repo.status() call, which the largefiles extension has no interest is processing anyway.

largefiles: introduce the 'composelargefilematcher()' method This is a copy/paste (with the necessary tweaks) of the composenormalfilematcher method currently on default, which does the inverse- this trims the normal files out of the matcher. It will be used in the next patch.

context: return dirstate parents in workingctx.ancestors() workingctx.ancestors() was not returning the dirstate parents as part of the result set. The only place this function is used is for copy detection when committing a file, and that code already checks the parents manually, so this change has no affect at the moment. I found it while playing around with changing how copy detection works.

backout: add --commit option Mercurial backout command makes a commmit by default only when the backed out revision is the parent of working directory and doesn't commit in any other case. The --commit option changes behaviour of backout to make a commit whenever possible (i.e. there is no unresolved conflicts). This behaviour seems more intuitive to many use (especially git users migrating to hg).

share: add option to share bookmarks This patch adds the -B/--bookmarks option to the share command added by the share extension. All it does for now is create a marker, 'bookmarks.shared', that will be used by future code to implement the sharing functionality.

highlight: ignore Unicode's extra linebreaks (issue4291) Unicode and Python's unicode.splitlines() treat several extra legacy ASCII codepoints as linebreaks, even though the vast bulk of computing and Python's own str.splitlines() do not. Rather than introduce line numbering confusion, we filter them out when highlighting.

test: fix typo in test-help.t

templatekw: remove unneeded showtags Now that we have the machinery of namespaces in-place, we use that instead of hand-rolling our own template function. Note, this can only be used for tags because both branches and bookmarks have special case logic for 'default' and the current bookmark (which is something outside the namespace api for now).

namespaces: generate template keyword when registering a namespace For any namespace, we generate a template keyword. For example, given a namespace 'babar', we automatically have the ability to use it in a template: hg log -r . -T '{babars % "King: {babar}\n"}' Furthermore, we only generate this keyword for a namespace if one doesn't already exist. This is necessary for 'branches' and 'bookmarks' since both of those have concepts of 'current' (something outside the namespace api) and also allows extensions to override default behavior if desired.

templatekw: add helper method to generate a template keyword for a namespace This marks our second feature of the namespace api: automatic template keyword. This patch adds a method that takes in a namespace and uses the node-to-name map to output the list of names.

namespaces: add names method to return list of names for a given node In the previous patch, we added a node-to-name map property. This patch just exposes that interface to the api.

namespaces: add nodemap property This patch adds a node-to-name map property to the namespace. This is necessary because we cannot simply invert the name-to-node map because we do not assume the name-to-node map to be unique (for example, consider named branches: many nodes have one branch name). The node-to-name is helpful in log commands where we are already iterating over a set of nodes and want to display some kind of naming information to the user.

namespaces: add method to get template name of namespace This patch adds the public api for getting the template name of a namespace so that the next patch can use it to generate a template keyword automatically.

namespaces: add template name of a namespace The template name property will be used in upcoming patches to automatically generate a template keyword. For example, given a namespace called 'babars', we will automatically generate a template keyword 'babar' such that we can use it in the following way, $ hg log -r . -T '{babars % "King: {babar}\n"}'

memctx: remove redundant test for issue4470 from test-commit.t Because: - the test to avoid regression for issue4470 was already added to test-commit-amend.t by previous patch It is also a part of test series about manifest calculation issues of memctx in test-commit-amend.t. - this is the only test using "commit --amend" in test-commit.t

memctx: calculate manifest more efficiently Before this patch, "memctx._manifest" updates all entries in the (parent) manifest. But this is inefficiency, because almost all files may be clean in that context. On the other hand, just updating entries for changed "files" specified at construction causes unexpected abortion, when there is at least one newly removed file (see issue4470 for detail). To calculate manifest more efficiently, this patch replaces "pman.iteritems()" for the loop by "self._status.modified" to avoid updating entries for clean or removed files Examination of removal is also omitted, because removed files aren't treated in this loop (= "self[f]" returns not None always).

merge with stable

Added signature for changeset 6dad422ecc5a

Added tag 3.2.3 for changeset 6dad422ecc5a

pathauditor: check for Windows shortname aliases

pathauditor: check for codepoints ignored on OS X

darwin: omit ignorable codepoints when normcase()ing a file path This lets us avoid some nasty case collision problems in OS X with invisible codepoints.

encoding: add hfsignoreclean to clean out HFS-ignored characters According to Apple Technote 1150 (unavailable from Apple as far as I can tell, but archived in several places online), HFS+ ignores sixteen specific unicode runes when doing path normalization. We need to handle those cases, so this function lets us efficiently strip the offending characters from a UTF-8 encoded string (which is the only way it seems to matter on OS X.)

test-casefolding.t: demonstrate a bug with HFS+ ignoring some codepoints

manifest: disallow setting the node id of an entry to None manifest.diff() uses None as a special value to denote the absence of a file, so setting a file node to None means you then can't trust manifest.diff(). This should also make future manifest work slightly easier.

context: stop setting None for modified or added nodes Instead use a magic value, so that we can identify modified or added nodes correctly when using manifest.diff(). Thanks to Martin von Zweigbergk for catching that we have to update _buildstatus as well. That part eluded my debugging for some time.

largefiles: don't actually remove largefiles in an addremove dry run The addlargefiles() method already properly handled dry runs.

bundle2: lowercase part types Since the capitalization no longer carries any meaning (previous diff), this patch normalizes all of the bundle2 part type strings to lower case.

bundle2.bundlepart: make mandatory part flag explicit in API This makes all bundle2 parts mandatory unless they are expressly made advisory via the keyword parameter or the bundlepart.mandatory property.

memctx: calculate manifest correctly with newly-removed files (issue4470) Before this patch, "memctx._manifest" tries to get (and use normally) filectx also for newly-removed files, even though "memctx.filectx()" returns None for such files. To calculate manifest correctly even with newly-removed files, this patch does: - replace "man.iteritems()" for the loop by "self._status.modified" to avoid accessing itself to newly removed files this also reduces loop cost for large manifest. - remove files in "self._status.removed" from the manifest In this patch, amending is confirmed twice to examine both (1) newly removed files and (2) ones already removed in amended revision.

memctx: calculate manifest including newly added files correctly Before this patch, "memctx._manifest" calculates the manifest according to the 1st parent. This causes the disappearance of newly added files from the manifest. For example, if newly added files aren't listed up in manifest of memctx, they aren't listed up in "added" field of "status" returned by "ctx.status()", and "{diff()}" (= "patch.diff") in "committemplate" shows nothing for them. To calculate manifest including newly added files correctly, this patch puts newly added files (= ones in "self._status.added") into the manifest. Some details of changes for "test-commit-amend.t" in this patch: - "touch foo" is replaced by "echo foo > foo", because newly added empty file can't be shown in "diff()" output without "diff.git" configuration - amending is confirmed twice to examine both (1) newly added files and (2) ones already added in amended revision

memctx: calculate exact status being committed from specified files Before this patch, "memctx._status" is initialized by "(files, [], [], [], [], [], [])" and this causes "memctx.modified" to include not only modified files but also added and removed ones incorrectly. This patch adds "_status" method to calculate exact status being committed according to "files" specified at construction time. Exact "_status" is useful to share/reuse logic of committablectx. This patch is also preparation for issues fixed by subsequent patches. Some details of changes for tests in this patch: - some filename lines are omitted in "test-convert-svn-encoding.t", because they are correctly listed up as "removed" files those lines are written out in "localrepository.commitctx" for "modified" and "added" files by "ui.note". - "| fixbundle" filterring in "test-histedit-fold.t" is omitted to check lines including "added" correctly "fixbundle" discards all lines including "added".

bundle2._processpart: forcing lower-case compare is no longer necessary Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. Since the parttype and mandatory bit are separated in bundle2.unbundlepart (see previous patch), there is no longer a need to remove the mandatory bit before working with the parttype.

bundle2.unbundlepart: decouple mandatory from parttype Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. This patch separates the two pieces of information when reading the part header so that it's unnecessary to know how they were combined during transmission.