filemerge: refactoring of 'filemerge()' current 'filemerge.filemerge()' implementation is verfy complicated. - it is not easy to add new internal merge tools (only by patching on 'filemerge()', or replacing it completely) - cleanup of temporary files is unsatisfactory ('internal:dump' does not, in fact) this is patch for refactoring of 'filemerge()' to isolate each internal merge tool implementations from 'filemerge()', and clean up common part in it.

patch: fuzz more aggressively to match patch(1) behaviour The previous code was assuming a default context of 3 lines. When fuzzing, it would take this value in account to reduce the amount of removed line from hunks top or bottom. For instance, if a hunk has only 2 lines of bottom context, fuzzing with fuzz=1 would do nothing and with fuzz=2 it would remove one of those lines. A hunk with one line of bottom context could not be fuzzed at all. patch(1) has apparently no such restrictions and takes the fuzz level at face value. - test-import.t: fuzz/offset changes at the beginning of file are explained by the new fuzzing behaviour and match patch(1) ones. Patching locations are different but those of my patch(1) do not make a lot of sense right now (patched output are the same) - test-import-bypass.t: more agressive fuzzing makes a patching supposed to fail because of context, succeed. Change the diff to avoid this. - test-mq-merge.t: more agressive fuzzing would allow the merged patch to apply with fuzz, but fortunately we disallow this behaviour. The new output is kept. I have not enough experience with patch(1) fuzzing to know whether aligning our implementation on it is a good or bad idea. Until now, it has been the implementation reference. For instance, "qpush" tolerates fuzz (test-mq-merge.t runs the special case of pushing merge revisions where fuzzing is forbidden).

patch: fix fuzzing of hunks without previous lines (issue3264) When applying hunks such as: @@ -2,1 +2,2 @@ context +change fuzzing would empty the "old" block and make patchfile.apply() traceback. Instead, we apply the new block at specified location without testing. The "bottom hunk" test was removed as patch(1) has no problem applying hunk with no context in the middle of a file.

patch: make hunk.fuzzit() compute the fuzzed start locations - It moves hunks processing weirdness where it belongs - It helps reusing said weirdness whenever fuzzit() is called, like during the actual hunk fuzzing.

patch: fuzz old and new lines at the same time In theory, the fuzzed offsets for old and new lines should be exactly the same as they are based on hunk parsing. Make it true in practice.

largefiles: optimize update speed by only updating changed largefiles Historically, during 'hg update', every largefile in the working copy was hashed (which is a very expensive operation on big files) and any largefiles that did not have a hash that matched their standin were updated. This patch optimizes 'hg update' by keeping track of what standins have changed between the old and new revisions, and only updating the largefiles that have changed. This saves a lot of time by avoiding the unecessary calculation of a list of sha1 hashes for big files. With this patch, the time 'hg update' takes to complete is a function of how many largefiles need to be updated and what their size is. Performance tests on a repository with about 80 largefiles ranging from a few MB to about 97 MB are shown below. The tests show how long it takes to run 'hg update' with no changes actually being updated. Mercurial 2.1 release: $ time hg update 0 files updated, 0 files merged, 0 files removed, 0 files unresolved getting changed largefiles 0 largefiles updated, 0 removed real 0m10.045s user 0m9.367s sys 0m0.674s With this patch: $ time hg update 0 files updated, 0 files merged, 0 files removed, 0 files unresolved real 0m0.965s user 0m0.845s sys 0m0.115s The same repsoitory, without the largefiles extension enabled: $ time hg update 0 files updated, 0 files merged, 0 files removed, 0 files unresolved real 0m0.799s user 0m0.684s sys 0m0.111s So before the patch, 'hg update' with no changes was approximately 9.25s slower with largefiles enabled. With this patch, it is approximately 0.165s slower.

mq: make qimport --push push all imported patches (issue3130) Only the first imported one was pushed.

convert: tolerate spaces between splicemap parent ids (issue3203) Splicemap lines are documented in hg help convert like: key parent1, parent2 but parsed like: key, parents = line.strip().rsplit(' ', 1) parents = parents.replace(',', ' ').split() The rsplit() call was introduced to handle spaces in keys for the generic mapfile format. Spaces can appear in svn identifiers since they contain path components. This logic makes less sense with splicemap since svn identifiers can also appear on the right side, even if it is a bit less likely. Given the parsing is theorically broken, I would rather follow what is documented already and is correct in the main case where all identifiers are hg hashes. Also, using svn identifiers in a splicemap sounds difficult as they are not easily accessible.

test-commandserver: flush stdout

localrepo: clear _filecache on rollback (issue3261) Files are being replaced by rollback but the corresponding data in localrepo isn't actually updated for things like bookmarks, phases, etc. Then when rollback is done, the cache is updated thinking it has the most up-to-date data, where in fact it is still pre-rollback. We clear _filecache to force everything to be recreated.

scmutil: update cached copy when filecached attribute is assigned (issue3263) When assigning a new object to filecached properties, the cached object that was kept in the _filecache map was still holding the old object. By implementing __set__, we track these changes too and update the cached copy as well.

cmdserver: invalidate the dirstate when running commands (issue3271) The dirstate is invalidated separately outside of invalidate() which is already being called (other callers of invalidate() seems to suggest the separation is there for a reason).

localrepo: reset _phasesdirty flag after writing

import: handle git renames and --similarity (issue3187) There is no reason to discard copy sources from the set of files considered by addremove(). It was done to handle the case where a first patch would create 'a' and a second one would move 'a' to 'b'. If these patches were applied with --no-commit, 'a' would first be marked as added, then unlinked and dropped from the dirstate but still passed to addremove(). A better fix is thus to exclude removed files which ends being dropped from the dirstate instead of removed. Reported by Jason Harris <jason@jasonfharris.com>

context: make workingctx.forget() really warn about untracked files

largefiles: check whether specified patterns are related to largefiles strictly current 'lfiles_repo.status()' implementation examines whether specified patterns are related to largefiles in working directory (not to STANDIN) or not by NOT-EMPTY-NESS of below list: [f for f in match.files() if f in lfdirstate] but it can not be assumed that all in 'match.files()' are file itself exactly, because user may only specify part of path to match whole under subdirectories recursively. above examination will mis-recognize such pattern as 'not related to largefiles', and executes normal 'status()' procedure. so, 'hg status' shows '?'(unknown) status for largefiles in working directory unexpectedly. this patch examines relation of pattern to largefiles by applying 'match()' on each entries in lfdirstate and checking wheter there is no matched entry. it may increase cost of examination, because it causes of full scan of entries in lfdirstate. so this patch uses normal for-loop instead of list comprehensions, to decrease cost when matching is found.

merge with stable

log: remove caching of all visited revisions (issue3253) Not only does this eat all available memory for some users, it's slower.

pull: backout change to return code This bit a number of people.

convert: use splicemap entries when sorting revisions (issue1748) When sorting revisions before converting them, we have to edit the revision graph using splicemap entries. Otherwise, a spliced revision may be converted before its synthetic parents. Invalid splicemap revisions are now detected before starting the conversion.

convert: turn splicemap into a simple dictionary Parsing the splicemap as a mapfile was a pain because map does not let us override its parsing code and splicemap entries are not key/values. Besides we had no need for mapfiles extra features. Just parse the splicemap and return a dictionary.

debugrevspec: mention --verbose to print the parsed tree

largefiles: only cache largefiles in new heads This fixes a serious performance regression in largefiles introduced in Mercurial 2.1. Caching new largefiles on pull is necessary, because otherwise largefiles will be missing (and unable to be downloaded) when the user tries to merge or rebase a new head with an old one. But this is an expensive operation and should only be done for heads that are new from the pull, rather than on all heads in the repository.

merge with stable

mq: restore _branchtags() fast path (issue3223) Since a5917346c72e, mq saves the nodeid of the first applied patch to cache/branchheads, which breaks the optimized cache handling introduced in fbf8320f25c8. The problem is the revision being committed is appended to mqrepo.applied after the commit succeeds, which means mqrepo._branchtags() performs a regular update and write the first applied patch to the branch cache. One solution is to set a context variable _committingpatch on the mqrepo while it is committing a patch and to take it in account when deciding to fast-path mqrepo._branchtags(). Not really elegant but it works. The changes to test-mq-caches.t reverse changes introduced by a5917346c72e. The cache should not have been updated with mq records. The changes to test-keyword.t are indirectly caused by a5917346c72e. Reported and analyzed by Yuya Nishihara <yuya@tcha.org> Notes: - qpush still makes a slow path _branchtags() call when checking heads. Maybe this can be optimized. - be careful when merging this patch in default as secretcommit() was renamed newcommit() right after the end of the code freeze.

mq: ensure all mq commits are made with secretcommit() Having a common code path helps fixing things globally.

convert/bzr: ignore nested repos when listing branches (issue3254) Reported by A.S. Budden <abudden@gmail.com>

tests: tighten checks for octal escapes in shell printf. printf on AIX default shell ksh (89) says \1 is an invalid escape. It insists on at least 2 digits. This causes failures in test-keyword.t and test-status.t. check-code.py already looks out for \NNN and recommends using Python for outputting octal values. Extend the check to \NN and \N and fix up resulting failures.

phase: when phase cannot be reduced, hint at --force and return 1 (BC) Before, trying to change the phase of a "public" node to "draft" would result in: $ hg phase --draft . no phases changed [0] With this patch: $ hg phase --draft . cannot move 1 changesets to a more permissive phase, use --force no phases changed [1] On partial failures, the exit status is now 1 instead of 0 and the number of changed nodes is displayed while it was only with --verbose. It seems useful to tell the user that despite the apparent failure, something changed. $ hg phase --draft '5 or 7' cannot move 1 changesets to a more permissive phase, use --force phase changed for 1 changesets [1]

revset: fix alias substitution recursion (issue3240) The revset aliases expansion worked like: expr = "some revset" for alias in aliases: expr = alias.process(expr) where "process" was replacing the alias with its *unexpanded* substitution, recursively. So it only worked when aliases were applied in proper dependency order. This patch rewrites the expansion process so all aliases are expanded recursively at every tree level, after parent alias rewriting and variable expansion.

update: just merge unknown file collisions The unknown file collision rule was introduced as an extension of the "should be clean when merging" rule. Unfortunately, it got applied to the normal update path, which should be happy to merge local changes. This patch gives us merges for unknown file collisions on update, while preserving abort for merge and update -c.

merge: don't use unknown() This removes use of unknown files for building the synthetic working directory manifest used by manifestmerge. Instead, we adopt the strategy used by _checkunknown. Side-effect: unknown files are no longer moved by remote directory renames, and now are left alone like ignored files.

merge: refactor unknown file conflict checking Previously, we would do a full working directory walk including unknown files to perform a merge. In many cases, this was painful because unknown files greatly outnumbered tracked files and generally had no useful effect on the merge. Here we instead wait until we find a file in the destination that's not tracked locally and detect if it exists and is not ignored. This is usually cheaper but can be -more- expensive in the case where we're adding a huge number of files. On the other hand, the cost of statting the new files should be dwarfed by the cost of eventually writing them. In this version, case collisions are detected implicitly by os.path.exists and wctx[f] lookup.

update: use normal update path with --check (issue2450) This avoids clobbering unknown files on update by not using overwrite mode.

fetch: use update rather than clean when updating (issue3246) We already verify the working directory is "clean" before starting so there's no advantage to clobbering the working directory.

merge with stable

mdiff: adjust hunk offsets with --ignore-blank-lines (issue3234) When diffing the following documents with --ignore-blank-lines (-B): $ cat > a <<EOF > > > > b > x > d > EOF and: $ cat > b <<EOF > b > y > d > EOF the context lines are taken from the first document, even if the lines differ (with -w or -b) or if the number of lines differ (with -B). In the second case, we have to adjust the hunk new lines offsets or we end with inconsistent diffs like (see the @@ offsets): diff -r 0e66aa54f318 a --- a/a +++ b/a @@ -1,4 +1,3 @@ b -x +y d Note that having different context lines in a and b means the diff can be applied but is not invertible. Reported by Nicholas Riley <com-selenic@sabi.net>

merge with stable

merge with i18n

merge with i18n

i18n-de: fix rst markup in command output (issue3247)

merge with i18n stable

i18n-ru: fix hg command rst markup

i18n-ja: fix rst markup in hg command output

i18n-de: fix rst markup in command output (issue3247)

merge with i18n stable

i18n-de: fix hg command rst markup and quoting (issue3247)

merge with i18n stable

i18n-de: fix rst markup in command output (issue3247)

windows: use 'str.replace()' instead of combination of split() and join() 'str.replace()' is twice (or more) as fast as combination of split() and join(). combination of split() and join() in current 'util.pconvert()' implementation comes only from historical reason when win32mbcs extension was introduced.

largefiles: reduce redundant splitstandin/standin combination assumptions: - 'lfutil.splitstandin(f)' strips 'f' of '.hglf/', if it can - 'lfile(f)' examines 'lfutil.standin(f) in manifest' - 'lfutil.standin(f)' adds '.hglf/' to 'f' and - 'f' should start with '.hglf/' there, because it is already examined by 'lfutil.isstandin(f)' then: lfile(lfutil.splitstandin(f)) => lfile(<f without '.hglf/'>) => lfutil.standin(<f without '.hglf/'>) in manifest => <<f without '.hglf/'> with '.hglf/'> in manifest => f in manifest

largefiles: reduce OR-ing of same conditions

subrepo: rewrite handling of subrepo state at commit (issue2403) When the contents of .hgsubstate are stale (either because they've manually been tweaked or partial updates have confused it), we get confused about whether it actually needs committing. So instead, we actively consult the parent's substate and compare it the actual current state when deciding whether it needs committing. Side effect: lots of "committing subrepo" messages that didn't correspond with real commits disappear. This change is fairly invasive for a fairly obscure condition, so it's kept on the default branch.

subrepo: add basestate method This will allow us to sort out what's going on with state .hgsubstate files more precisely.

bdiff: fix malloc(0) issue in fixws() If fixws() is called on a zero-length string, malloc(0) is called and expected to return a pointer. Which it does on e.g. Linux. AIX returns NULL, which it is also legal, but the malloc() is then assumed to have failed. So ensure a valid pointer is always returned.

forget: show warning messages for forgetting in subrepo correctly in 'cmdutil.forget()': for f in match.files(): if match.exact(f) or not explicitonly: .... is equal to: for f in match.files(): if True: .... because 'f' from 'match.files()' should 'match.exact(f)': - 'match.files()' returns 'self._files' - 'match.exact(f)' examines 'f in self._fmap', - 'self._fmap' of match is 'set(self._files)' then, 'explicitonly' wants to suppress warning messges, if it is true (= 'cmdutil.forget()' is invoked from 'subrepo.forget()'). so, current code should be fixed as: if not explicitonly: for f in match.files(): ....

tests: fix command line overflow on ksh "<command> `hg manifest`" overflows the puny 1024 byte command line length limit on AIX default shell ksh. Replace with "hg manifest | xargs <command>".

use util.localpath() instead of 'str.replace()' to unify path conversion

i18n: use util.pconvert() instead of 'str.replace()' for problematic encoding some problematic encodings use backslash as part of multi-byte characters. util.pconvert() can treat strings in such encodings correctly, if win32mbcs is enabled, but str.replace() can not.

i18n: use util.pconvert() instead of 'str.replace()' for problematic encoding some problematic encodings use backslash as part of multi-byte characters. util.pconvert() can treat strings in such encodings correctly, if win32mbcs is enabled, but str.replace() can not.

merge with stable

mq: make qprev return the previous applied patch (issue3245) The current behaviour is to return the previous one in the series but at the same time the implementation is buggy because it does not take guarded patches in account.

mq: fix qnext when all remaining patches are guarded When all remaining patches are guarded, qnext used to return the last of the queue anyway.

convert/bzr: test tags conversion