memctx: calculate manifest more efficiently Before this patch, "memctx._manifest" updates all entries in the (parent) manifest. But this is inefficiency, because almost all files may be clean in that context. On the other hand, just updating entries for changed "files" specified at construction causes unexpected abortion, when there is at least one newly removed file (see issue4470 for detail). To calculate manifest more efficiently, this patch replaces "pman.iteritems()" for the loop by "self._status.modified" to avoid updating entries for clean or removed files Examination of removal is also omitted, because removed files aren't treated in this loop (= "self[f]" returns not None always).

merge with stable

Added signature for changeset 6dad422ecc5a

Added tag 3.2.3 for changeset 6dad422ecc5a

pathauditor: check for Windows shortname aliases

pathauditor: check for codepoints ignored on OS X

darwin: omit ignorable codepoints when normcase()ing a file path This lets us avoid some nasty case collision problems in OS X with invisible codepoints.

encoding: add hfsignoreclean to clean out HFS-ignored characters According to Apple Technote 1150 (unavailable from Apple as far as I can tell, but archived in several places online), HFS+ ignores sixteen specific unicode runes when doing path normalization. We need to handle those cases, so this function lets us efficiently strip the offending characters from a UTF-8 encoded string (which is the only way it seems to matter on OS X.)

test-casefolding.t: demonstrate a bug with HFS+ ignoring some codepoints

manifest: disallow setting the node id of an entry to None manifest.diff() uses None as a special value to denote the absence of a file, so setting a file node to None means you then can't trust manifest.diff(). This should also make future manifest work slightly easier.

context: stop setting None for modified or added nodes Instead use a magic value, so that we can identify modified or added nodes correctly when using manifest.diff(). Thanks to Martin von Zweigbergk for catching that we have to update _buildstatus as well. That part eluded my debugging for some time.

largefiles: don't actually remove largefiles in an addremove dry run The addlargefiles() method already properly handled dry runs.

bundle2: lowercase part types Since the capitalization no longer carries any meaning (previous diff), this patch normalizes all of the bundle2 part type strings to lower case.

bundle2.bundlepart: make mandatory part flag explicit in API This makes all bundle2 parts mandatory unless they are expressly made advisory via the keyword parameter or the bundlepart.mandatory property.

memctx: calculate manifest correctly with newly-removed files (issue4470) Before this patch, "memctx._manifest" tries to get (and use normally) filectx also for newly-removed files, even though "memctx.filectx()" returns None for such files. To calculate manifest correctly even with newly-removed files, this patch does: - replace "man.iteritems()" for the loop by "self._status.modified" to avoid accessing itself to newly removed files this also reduces loop cost for large manifest. - remove files in "self._status.removed" from the manifest In this patch, amending is confirmed twice to examine both (1) newly removed files and (2) ones already removed in amended revision.

memctx: calculate manifest including newly added files correctly Before this patch, "memctx._manifest" calculates the manifest according to the 1st parent. This causes the disappearance of newly added files from the manifest. For example, if newly added files aren't listed up in manifest of memctx, they aren't listed up in "added" field of "status" returned by "ctx.status()", and "{diff()}" (= "patch.diff") in "committemplate" shows nothing for them. To calculate manifest including newly added files correctly, this patch puts newly added files (= ones in "self._status.added") into the manifest. Some details of changes for "test-commit-amend.t" in this patch: - "touch foo" is replaced by "echo foo > foo", because newly added empty file can't be shown in "diff()" output without "diff.git" configuration - amending is confirmed twice to examine both (1) newly added files and (2) ones already added in amended revision

memctx: calculate exact status being committed from specified files Before this patch, "memctx._status" is initialized by "(files, [], [], [], [], [], [])" and this causes "memctx.modified" to include not only modified files but also added and removed ones incorrectly. This patch adds "_status" method to calculate exact status being committed according to "files" specified at construction time. Exact "_status" is useful to share/reuse logic of committablectx. This patch is also preparation for issues fixed by subsequent patches. Some details of changes for tests in this patch: - some filename lines are omitted in "test-convert-svn-encoding.t", because they are correctly listed up as "removed" files those lines are written out in "localrepository.commitctx" for "modified" and "added" files by "ui.note". - "| fixbundle" filterring in "test-histedit-fold.t" is omitted to check lines including "added" correctly "fixbundle" discards all lines including "added".

bundle2._processpart: forcing lower-case compare is no longer necessary Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. Since the parttype and mandatory bit are separated in bundle2.unbundlepart (see previous patch), there is no longer a need to remove the mandatory bit before working with the parttype.

bundle2.unbundlepart: decouple mandatory from parttype Encoding whether or not a part is mandatory in the capitalization of the parttype is unintuitive and error-prone. This sequence of patches separates these concerns in the API to reduce programmer error and pave the way for a potential change in how this information is transmitted over the wire. This patch separates the two pieces of information when reading the part header so that it's unnecessary to know how they were combined during transmission.

memctx: fix manifest for removed files (issue4470) filectxfn returns None for removed files, so we have to check for None before computing the new file content hash for the manifest. Includes a test that proves this works, by demonstrating that we can show the diff of an amended commit in the committemplate.

convert: on svn failure, note libsvn version (issue4043) We have our own fast-path logic to see if something passes a sniff test for being a Subversion repository, but it's possible for a user to svnsync a repo using svn 1.8 and then use svn 1.7 bindings (as in the bug) to try and convert the repo. If we at least tell the user the version of libsvn that we used, they might get enough of a hint to check on their own for format incompatibilities between their svn{admin,sync} and the libsvn used by hg.

vfs: add a 'split' method This method has the same behavior as the 'os.path.split' function, but having it in vfs will allow handling of tricky encoding situations in the future. In the same patch, we replace the use of 'os.path.split' in the transaction code.

vfs: add a 'reljoin' function for joining relative paths The vfs.join method only works for absolute paths. We need something that works for relative paths too when transforming filenames. Since os.path.join may misbehave in tricky encoding situations, encapsulate the new join method in our vfs abstraction. The default implementation remains os.path.join, but this opens the door to other VFSes doing something more intelligent based on their needs. In the same go, we replace the usage of 'os.path.join' in transaction code.

subrepo: fix git subrepo ui argument

subrepo: drop the 'ui' parameter to revert() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in revert is to emit status and warning messages, and to check the verbose flag prior to printing the action to be performed on a file. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call. Unlike other methods where the matcher is passed along and narrowed, a new matcher is created in each repo, and therefore the bad() method already used the local repo's ui.

subrepo: drop the 'ui' parameter to removefiles() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in remove is to emit status and warning messages, and to check the verbose flag prior to printing files to be removed. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call.

subrepo: drop the 'ui' parameter to forget() This no longer needs to be explicitly passed because the subrepo object tracks the 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in forget is to emit status and warning messages, and to check the verbose flag prior to printing files to be forgotten. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object. The local repo's ui was already being used to print a warning message in wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status() call.

subrepo: drop the 'ui' parameter to cat() This no longer needs to be explicitly passed because the subrepo object tracks a 'ui' reference since fcbc66b5da6a. See the change to 'archive' for details about the differences between the output level in the root repo and subrepo 'ui' object. The only use for 'ui' in cat is to emit a status message when a subrepo is missing. The bad() method on the matcher still uses the root repo's ui, because narrowing the matcher doesn't change the ui object.

subrepo: drop the 'ui' parameter to archive() The current state of subrepo methods is to pass a 'ui' object to some methods, which has the effect of overriding the subrepo configuration since it is the root repo's 'ui' that is passed along as deep as there are subrepos. Other subrepo method are *not* passed the root 'ui', and instead delegate to their repo object's 'ui'. Even in the former case where the root 'ui' is available, some methods are inconsistent in their use of both the root 'ui' and the local repo's 'ui'. (Consider hg._incoming() uses the root 'ui' for path expansion and some status messages, but also calls bundlerepo.getremotechanges(), which eventually calls discovery.findcommonincoming(), which calls setdiscovery.findcommonheads(), which calls status() on the local repo 'ui'.) This inconsistency with respect to the configured output level is probably always hidden, because --verbose, --debug and --quiet, along with their 'ui.xxx' equivalents in the global and user level hgrc files are propagated from the parent repo to the subrepo via 'baseui'. The 'ui.xxx' settings in the parent repo hgrc file are not propagated, but that seems like an unusual thing to set on a per repo config file. Any 'ui.xxx' options changed by --config are also not propagated, because they are set on repo.ui by dispatch.py, not repo.baseui. The goal here is to cleanup the subrepo methods by dropping the 'ui' parameter, which in turn prevents mixing subtly different 'ui' instances on a given subrepo level. Some methods use more than just the output level settings in 'ui' (add for example ends up calling scmutil.checkportabilityalert() with both the root and local repo's 'ui' at different points). This series just goes for the low hanging fruit and switches methods that only use the output level. If we really care about not letting a subrepo config override the root repo's output level, we can propagate the verbose, debug and quiet settings to the subrepo in the same way 'ui.commitsubrepos' is in hgsubrepo.__init__. Archive only uses the 'ui' object to call its progress() method, and gitsubrepo calls status().

subrepo: use 'self.ui' instead of 'self._repo.ui' They are now equivalent, but the former is slightly more readable.