filectx.parents: filter nullrev parent sooner We are going to introduce a linkrev-correction phases when computing parents. It will be more convenient to have the nullid parent filtered out earlier. I had to make a minimal adjustment to the rename handling logic to keep it functional. That logic have been documented in the process since it took me some time to check all the cases out.

context: catch FilteredRepoLookupError instead of RepoLookupError Now that we have a more specialised exception, lets use it when we meant to catch the more specialised case.

narrowmatcher: propagate the rel() method The full path is propagated to the original match object since this is often used directly for printing a file name to the user. This is cleaner than requiring each caller to join the prefix with the file name prior to calling it, and will lead to not having to pass the prefix around separately. It is also consistent with the bad() and abs() methods in terms of the required input. The uipath() method now inherits this path building property. There is no visible change in path style for rel() because it ultimately calls util.pathto(), which returns an os.sep based path. (The previous os.path.join() was violating the documented usage of util.pathto(), that its third parameter be '/' separated.) The doctest needed to be normalized to '/' separators to avoid test differences on Windows, now that a full path is returned for a short filename. The test changes are to drop globs that are no longer necessary when printing an absolute file in a subrepo, as returned from match.uipath(). Previously when os.path.join() was used to add the prefix, the absolute path to a file in a subrepo was printed with a mix of '/' and '\'. The absolute path for a file not in a subrepo, as returned from match.uipath(), is still purely '/' based.

match: add the abs() method This is a utility to make it easier for subrepos to convert a file name to the full path rooted at the top repository. It can replace the various path joining lambdas, and doesn't require the prefix to be passed into the method that wishes to build such a path. The name is derived from the following pattern in annotate() and other places: name = ((pats and rel) or abs) The pathname separator is not os.sep in part to avoid confusion with variables named 'abs' or similar that _are_ '/' separated, and in part because some methods like cmdutils.forget() and maybe cmdutils.add() need to build a '/' separated path to the root repo. This can replace the manual path building there.

merge with stable

posix: quote the specified string only when it may have to be quoted This patch makes "posix.shellquote" examine the specified string and quote it only when it may have to be quoted for safety, like as the previous patch for "windows.shellquote". In fact, on POSIX environment, quoting itself doesn't cause issues like issue4463. But (almost) equivalent quoting policy can avoid examining test result differently on POSIX and Windows (even though showing command line with "%r" causes such examination in "test-extdiff.t"). The last hunk for "test-extdiff.t" in this patch isn't needed for the previous patch for "windows.shellquote", because the code path of it is executed only "#if execbit" (= avoided on Windows).

windows: quote the specified string only when it has to be quoted Before this patch, "windows.shellquote" (as used as "util.shellquote") always quotes specified strings with double quotation marks, for external process invocation. But some problematic applications can't work correctly, when command line arguments are quoted: see issue4463 for detail. On the other hand, quoting itself is needed to specify arguments containing whitespaces and/or some special characters exactly. This patch makes "windows.shellquote" examine the specified string and quote it only when it may have to be quoted for safety.

extdiff: rename the name of an argument for readability To reduce amount of changes for review-ability, previous patch uses "args" as argument name of "dodiff()", even though "args" includes also the name of command to be executed (or full-path of it). This patch replaces "args" by more appropriate name "cmdline".

extdiff: avoid unexpected quoting arguments for external tools (issue4463) Before this patch, all command line arguments for external tools are quoted by the combination of "shlex.split" and "util.shellquote". But this causes some problems. - some problematic commands can't work correctly with quoted arguments For example, 'WinMerge /r ....' is OK, but 'WinMerge "/r" ....' is NG. See also below for detail about this problem. https://bitbucket.org/tortoisehg/thg/issue/3978/ - quoting itself may change semantics of arguments For example, when the environment variable CONCAT="foo bar baz': - mydiff $CONCAT => mydiff foo bar baz (taking 3 arguments) - mydiff "$CONCAT" => mydiff "foo bar baz" (taking only 1 argument) For another example, single quoting (= "util.shellquote") on POSIX environment prevents shells from expanding environment variables, tilde, and so on: - mydiff "$HOME" => mydiff /home/foobar - mydiff '$HOME' => mydiff $HOME - "shlex.split" can't handle some special characters correctly It just splits specified command line by whitespaces. For example, "echo foo;echo bar" is split into ["echo", "foo;echo", "bar"]. On the other hand, if quoting itself is omitted, users can't specify options including space characters with "--option" at runtime. The root cause of this issue is that "shlex.split + util.shellquote" combination loses whether users really want to quote each command line elements or not, even though these can be quoted arbitrarily in configurations. To resolve this problem, this patch does: - prevent configurations from being processed by "shlex.split" and "util.shellquote" only (possibly) "findexe"-ed or "findexternaltool"-ed command path is "util.shellquote", because it may contain whitespaces. - quote options specified by "--option" via command line at runtime This patch also makes "dodiff()" take only one "args" argument instead of "diffcmd" and "diffopts. It also omits applying "util.shellquote" on "args", because "args" should be already stringified in "extdiff()" and "mydiff()". The last hunk for "test-extdiff.t" replaces two whitespaces by single whitespace, because change of "' '.join()" logic causes omitting redundant whitespaces.

subrepo: add forgotten annotation for reverting git subrepos Support for reverting git subrepos was added earlier, but the annotation to handle any subrepo errors was forgotten.

subrepo: add full revert support for git subrepos Previously, revert was only possible if the '--no-backup' switch was specified. Now, to support backups, we explicitly go over all modified files in the subrepo.

setup: don't fail when Python doesn't have the cygwinccompiler package Some Python installations like the ones available from the optware project for the Synology DiskStation NASes don't have that package, which means running the setup script will crash and exit right away. Instead, we now just use an empty/fake class for the HackedMingw32CCompiler, which we likely won't use anyway.

tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)

tests: run 'cvs init' only on non-existent directories (issue4482) Do not create cvsroot directory since cvs-1.12 errors out if CVSROOT exists before 'cvs init'.

remove: use vfs instead of os.path + match.rel() for filesystem checks

forget: use vfs instead of os.path + match.rel() for filesystem checks

tests: make a multi-statement hook in bundle2-exchange Windows compatible This is similar to the fix in 7dd1870120b2.

sshpeer: more thorough shell quoting This fixes an issue spotted by Jesse Hertz.

merge with stable

i18n-ja: synchronized with 3b84bde06d17

i18n-pt_BR: synchronized with 53a65929ef1f

localrepo: use the vfs join method to implement the localrepo join method This will make it possible to customize the behavior of the join method by changing the vfs class (e.g. by using the altvfs" class introduced recently). Note that we could have modified the VFS join methods to acept a set of optional paths in the same way thta the localrepo join method does. However it seemed simpler to simply call os.path.join before calling self.vfs.join.

localrepo: introduce shared method to check if a repository is shared Up until now we compared the "path" and "sharedpath" repository properties to check if a repository is shared. This was relying an implementation detail of shared repositories. In order to make it possible to change the way shared repositories are implemented, we encapsulate this check into its own localrepo method, called shared. This new method returns None if the repository is shared, and something else (for now a string describing the short of share) otherwise. The reason why I did not call this method "isshared" and made it return a boolean is that I plan to introduce a new type of shared repository soon. # NOTES: This is the first patch in a series whose purpose is to add support for creating "full repository shares", which are repositories that share everything with the repository source except their current revision, branch and bookmark. This series is RFC because I am not very sure of some of the solutions I took. Comments are welcome!

trydiff: use 'ctx1.flags()' for symmetry with 'ctx2.flags()'

trydiff: use 'not in addedset' for symmetry with 'not in removedset' With the previous change in place, we can safely use 'addedset'.

trydiff: simplify checking for additions In the body of the loop in trydiff(), there are conditions like: addedset or (f in modifiedset and to is None) The second half of that expression is to account for the fact that merge-in additions appear as additions. By instead fixing up the sets of modified and added files to compensate for this fact, we can simplify the body of the loop. It also fixes one case where the addedset was checked without the additional check (the "have we already reported a copy above?" case in the code, also see fixed test case). The similar condition with 'removedset' in it seems to have served no purpose even before this change, so it could have been simplified even before.

trydiff: extract 'date2' variable like existing 'date1' Note that there is a comment saying "ctx2 date may be dynamic". The comment was introduced in dccb83241dd0 (patch: use contexts for diff, 2006-12-25), but it seems like it stopped being dynamic in that very changeset -- before that changeset, the date seems to have been the file's mtime, but after the changeset, it seems to be the changeset's timestamp (current time for workingctx). Since no one seems to have missed the "dynamicness", let's simplify and extract a date2 for symmetry with date1.

trydiff: use sets, not lists, for containment checks This only has a noticeable effect on diffs touching A LOT of files. For example, it takes hg diff -r FIREFOX_AURORA_30_BASE -r FIREFOX_AURORA_35_BASE from 1m55.465s to 1m32.354s. That diff has over 50k files.

largefiles: fix unused import Caught by pyflakes.

largefiles: look at unfiltered().lfstatus to allow status() to be filtered The comment about status being buggy with a repo proxy seems to be that status wasn't being redirected to testing the largefiles themselves- lfstatus was always False, and so the original status method was being called instead of doing the largefiles processing. This is because when the various largefile command overrides call 'repo.lfstatus = True', __setattr__() in repoview is in turn setting the value on the unfiltered repo, and the value in the filtered repo remains False. Explicitly looking at the attribute on the unfiltered repo keeps all views in sync.

largefiles: eliminate a duplicate message when removing files in verbose mode There is no --after for addremove, so the printing for addremove can be hoisted out of the 'not after' check. The difference between the two remove messages reflects the existing difference between core remove and core addremove styles for printing the file. There are still some pre-existing issues here. Core addremove only prints on inexact matches or when verbose. But since the largefiles that are being removed are passed to removelargefiles() as a pattern list, there is never an inexact match, which would keep the largefiles from being printed at all unless verbose is specified. Therefore, the output is a little more aggressive than core. The addremove print style here is also inconsistent with core- it should use matcher.uipath(f) instead of f. These can be fixed once a matcher is passed in.

largefiles: ensure that the standin files are available in getlfilestoupload() The function only adds the hash content of the file to the set to upload if the file in the ctx is a standin. It is called by overrides.summaryremotehook(), which is called in the summary method. The largefiles extension switches 'lfstatus' on in summary, so the standins shouldn't be visible when obtaining a context there. The reason this wasn't noticed before is that the 'lfstatus' attribute is only being set on the unfiltered repo because of how repoview delegates attribute assignment. Therefore any filtered view will return a context containing standins, whether or not 'lfstatus' was set in the various overrides methods. That will be fixed in the next patch. But without this change, the next patch would have test failures for 'summary --large' stating there are no files to upload.

merge: move checking of unknown files out of manifestmerge() This moves most reading of filelogs out of manifestmerge, making it easy for a narrow clone extension to filter out or translate unwanted actions before any filelogs are read. The only call left is inside of copies.mergecopies(), which can be overridden separately at a lower level.

merge: extract method for checking for conflicting untracked file Now that the functionality is collected in one place, let's extract it to a method.

merge: create 'cm' action for 'get or merge' case We still have one case of a call to _checkunknownfile() in manifestmerge(): when force=True and branchmerge=True and the remote side has a file that the local side doesn't. This combination of arguments is used by 'hg merge --force', but also by rebase and unshelve. In this scenario, we try to create the file from the contents from the remote, but if there is already a local untracked file in place, we merge it instead.

merge: don't overwrite untracked file at directory rename target When a directory was renamed and a new untracked file was added in the new directory and the remote directory added a file by the same name in the old directory, the local untracked file gets overwritten, as demonstrated by the broken test case in test-rename-dir-merge. Fix by checking for unknown files for 'dg' actions too. Since _checkunknownfile() currently expects the same filename in both contexts, we need to add a new parameter for the remote filename to it.

merge: remove constant tuple element from 'aborts' The second element of the tuples in the 'aborts' list is always 'ud', so let's remove it.

merge: collect checking for unknown files at end of manifestmerge() The 'c' and 'dc' actions include creating a file on disk and we need to check that no conflicting file exists unless force=True. Move two of the calls to _checkunknownfile() to a single place at the end of manifestmerge(). This removes some of the reading of filelogs from the heart of manifestmerge() and collects it in one place close to where its output (entries in the 'aborts' list) is used. Note that this removes the unnecessary call to _checkunknownfile() when force=True in one of the code paths.

merge: introduce 'c' action like 'g', but with additional safety _checkunknownfile() reads the filelog of the remote side's file. For narrow clones, the filelog will not exist for all files and we need a way to avoid reading them. While it would be easier for the narrow extension to just override _checkunknownfile() and make it ignore files outside the narrow clone, it seems cleaner to have manifestmerge() not care about filelogs (considering its name). In order to move the calls to _checkunknownfile() out, we need to be able to tell in which cases we should check for unknown files. Let's start by introducing a new action distinct from 'g' for this purpose. Specifically, the new action will be just like 'g' except that it will check that for conflicting unknown files first. For now, just add the new action type and convert it to 'g'.

merge: structure 'remote created' code to match table This does duplicate the call to _checkunknownfile(), but it will simplify future patches.

pushkey: run hook after the lock release The pushkey operation used to be in its own wireprotocol command and (in practice) always be lock free when running the hook. With bundle2, it happen in a greater scheme and a hook running locking command would get stuck. We now run such hooks after the lock release as similar hook do. Bundle2 test are altered to ensure we are lockfree at hook running time.

setup: use changes since latest tag instead of just distance For a Mercurial built on the merge from stable into default right after 3.2.2 was released -- 19ebd2f88fc7 -- the version number produced was "3.2.2+4". This is potentially misleading, since in reality the built Mercurial includes many more changes compared to 3.2.2. Change the versioning scheme so that we take into consideration all the changes present in the current revision that aren't present in the latest tag. For 19ebd2f88fc7 the new versioning scheme results in a version number of "3.2.2+256". This gives users a much better idea of how many changes have actually happened since the latest release. Since changessincelatesttag is always greater than or equal to the latesttagdistance, this will produce version numbers that are always greater than or equal to the old scheme. Thus there's minimal compatibility risk.

setup: use changessincelatesttag from archive if present changessincelatesttag gives one a better idea of how much the code has changed since. Since changessincelatesttag is always greater than or equal to the latesttagdistance (see previous patch for why), this will always produce version numbers greater than or equal to the previous scheme.

archive: store number of changes since latest tag as well This is different from latesttagdistance in that while latesttagdistance is defined to be the length of the longest path to the latest tag, changessincelatesttag is the number of changes contained in @ that aren't contained in the latest tag. So, if 't' is the latest tag in the repository below: t | v --o--o----o \ \ ..o..o..@ then latesttagdistance is 2, but changessincelatesttag is 4. Note that changessincelatesttag is always greater than or equal to the latesttagdistance -- that's because changessincelatesttag counts all the changes in the longest path since the latest tag, and possibly others. This is an important fact that we'll take advantage of in upcoming patches.

merge with stable

demandimport: blacklist distutils.msvc9compiler (issue4475) This module depends on _winreg, which is windows-only. Recent versions of setuptools load distutils.msvc9compiler and expect it to ImportError immediately when on non-Windows platforms, so we need to let them do that. This breaks in an especially mystifying way, because setuptools uses vars() on the imported module. We then throw an exception, which vars doesn't pick up on well. For example: In [3]: class wat(object): ...: @property ...: def __dict__(self): ...: assert False ...: In [4]: vars(wat()) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-4-2781ada5ffe6> in <module>() ----> 1 vars(wat()) TypeError: vars() argument must have __dict__ attribute Which is similar to the problem we run into.

largefiles: don't duplicate 'actions' into 'actionbyfile'

merge: make calculateupdates() return file->action dict This simplifies largefiles' overridecalculateupdates(), which no longer has to do the conversion it started doing in 38e55e55ae4d (largefiles: rewrite merge code using dictionary with entry per file, 2014-12-09). To keep this patch small, we'll leave the name 'actionbyfile' in overrides.py. It will be renamed in the next patch.