context: add followfirst arg to filectx and workingfilectx When _followfirst() revset was introduced it seemed to be the sole user of such an argument, so filectx.ancestors() was duplicated and modified instead. It now appears this argument could be used when computing the set of files to be considered when --patch or --stat are passed along with --follow FILE.

graphlog: fix --removed "hg log --removed FILE" does not return changesets where FILE was removed, but ones where FILE was changed and possibly removed. The flag is really here to disable walkchangerevs() fast path, which cannot see file removals by scanning filelogs.

test-glog: rewrite more tests using testlog() function

match: consider filesets as "anypats" Matt suggested this on IRC, I do not think the choice is obvious, but this one makes things simpler because while filesets are turned into a list of files into the match objects, it would more be difficult to tell invalid files passed in pats from those expanded from filesets.

graphlog: evaluate FILE/-I/-X filesets on the working dir This subtlety is not documented yet but: - pats/--include/--exclude filesets are evaluated against the working directory - --rev filesets are reevaluated against every revisions

graphlog: implement --copies

test-check-code-hg: skip test if not in a working dir (issue3248).

copies: use ctx.dirs() for directory rename detection

copies: fix mergecopies doc mapping direction

merge with stable

log: restore cache used by --copies The {filelog -> linkrev -> copyfrom} cache was refactored and broken by: changeset: 10060:f780b1098efc user: Patrick Mezard <pmezard@gmail.com> date: Sun Dec 13 18:06:24 2009 +0100 summary: templatekw: change {file_copies} behaviour, add {file_copies_switch} With --copies, this cache is accessed for every touched file of every revision. Unfortunately it is recreated for every revision, which means filelogs are parsed again. This patch makes the cache global again for all revisions. A couple of indicative timings of "hg log --copies", before and after: hg: 44s / 5s mozilla --limit 10000: 3m51s / 2m32s mozilla: 23m46s / 12m23s I do not know any good tool to trace memory consumption of these runs for comparisons. Watching the full mozilla run in top, the process did not seem to misbehave.

graphlog: implement --follow-first log --graph --follow-first FILE cannot be compared with the regular version because it never worked: --follow-first is not taken in account in walkchangerevs() fast path and is explicitely bypassed in FILE case in walkchangerevs() nested iterate() function.

graphlog: implement --follow with file arguments

test-glog: test multiple --prune values

graphlog: restore FILE glob expansion on Windows On platforms not supporting shell expansion, scmutil.match() performs glob expansion on 'pats' arguments. But _matchfiles() revset calls match.match() directly, bypassing this behaviour. To avoid duplicating scmutil.match(), a secondary scmutil.matchandpats() is introduced returning both the match object and the expanded inputs. Note the expanded pats are also needed in the fast path, and will be used by --follow code path.

glog: restore multiple --rev test lost in rebasing

copies: remove checkdirs options This removes the undocumented merge.followdirs option, which has always been true.

copies: add docstring for mergecopies

addremove: return 1 if we failed to handle any explicit files

merge with stable

log: fix --follow FILE ancestry calculation Currently, --follow FILE looks for a FILE filelog, scans it and collects linkrevs and renames, then filters them. The problem is the filelog scan does not start at FILE filenode in parent revision but at the last filelog revision. So: - Files not in the parent revision can be followed, the starting node is unexpected - Files in the parent revision can be followed from an incorrect starting node. This patch makes log --follow FILE fail if FILE is not in parent revision, and computes ancestors of the parent revision FILE filenode.

test-glog: extend a test before fixing --follow issues The changes add a lot of noise to the test output, I prefer to separate it from the changes which are to be introduced by --follow fixes.

graft: use proper revisions for copy detection (issue3265)

convert: fix typos in error messages

graphlog: paths/-I/-X handling requires a new revset The filtering logic of match objects cannot be reproduced with the existing revsets as it operates at changeset files level. A changeset touching "a" and "b" is matched by "-I a -X b" but not by "file(a) and not file(b)". To solve this, a new internal "_matchfiles(...)" revset is introduced. It works like "file(x)" but accepts more than one argument and its arguments are prefixed with "p:", "i:" and "x:" to be used as patterns, include patterns or exclude patterns respectively. The _matchfiles revset is kept private for now: - There are probably smarter ways to pass the arguments in a user-friendly way - A "rev:" argument is likely appear at some point to emulate log command behaviour with regard to filesets: they are evaluated for the parent revision and applied everywhere instead of being reevaluated for each revision.

graphlog: imitate log slowpath when inputs are explicit files

graphlog: paths arguments must be or'ed

graphlog: rewrite --rev like all other options

graphlog: --branch and --only-branch are the same Handling the aliasing in fancyopts would be cleaner but I do not want to make this change for stable.

test-check-code-hg: fix xargs exit status on OSX When xargs subcommand invocation fails in a normal way, GNU xargs returns 123 and BSD one returns 1.

largefiles: respect store.createmode and avoid extra file copy Before, a tempfile was used to create a temp file was created with 600 permissions and the uploaded data was written into it. This file was then *copied* to .hg/largefiles/<hash>. We now simply use atomictempfile to write the data to a temp file with the right permissions and then rename that into place.

largefiles: respect store.createmode in basestore.get This replaces another use of tempfile with atomictempfile. The problem with tempfile is that it creates files with 600 permissions instead of respecting repo.store.createmode.

largefiles: use repo.store.createmode for new files in .hg/largefiles Before, the mode was copied from the file in the working copy. This is inconsistent with how Mercurial normally creates files inside the .hg folder. This can lead to a situation where you can read the files under .hg/store but not under .hg/largefiles and so a clone will fail if it needs to access a largefile.

mq: expand qimport summary

merge with stable

graphlog: multiple --keyword must be or'ed

graphlog: multiple --branch must be or'ed

graphlog: multiple --user must be or'ed

graphlog: explicitely join multivalue parameters This will let use override the "join" value (and/or) depending on the option considered. The option revset arity is now deduced from the revset and the option value type, to simplify opt2revset definition.

test-glog.t: use printrevset extension to trace rewritten revsets Using "hg log -G --print-revset" prints the revset generated by graphlog and exits. This helps debugging and writing shorter tests. It has been suggested to handle these tests with doctests. I think the extension approach is better because: - It tests the actual parameter set passed to graphlog.revset(), not what we expect it to be. 'branch' and 'only-branch' are currently distinct options but nothing prevents fancyopts to grow a notion of option aliasing one day, where both options would be merged before reaching the command. - It can be used as debug output interleaved with real log calls. v2: - Use a test extension instead of a global deprecated new option

context: use 'changectx.dirs()' in 'walk()' for directory patterns this patch uses 'changectx.dirs()' instead of nested loop, to examine whether specified pattern is related to the context as a directory or not.

localrepository: use 'changectx.dirs()' in 'status()' for directory patterns when pattern which does not match against any files in working context is specified, current implementation of 'localrepository.status()' decides whether warning message about it should be shown or not by 'f not in context' this works correctly for 'file pattern', but not for 'directory pattern', because 'f not in context' always returns True for directories, even if they are related to the context. this patch uses 'changectx.dirs()' to examine whether specified pattern is related to the context as a directory or not.

context: add 'dirs()' to changectx/workingctx for directory patterns this patch adds 'dirs()' to changectx/workingctx, which returns map of all directories deduced from manifest, to examine whether specified pattern is related to the context as directory or not quickly. 'workingctx.dirs()' uses 'dirstate.dirs()' rather than building another copy of it.

status: fix format field thinko

largefiles: don't break filesets

help: sort hgrc related "Sections" chapters alphabetically

doc: minor fixes to [graph] section documentation

hgweb: refactor graph customization javascript - Avoid flipping lineWidth state around the edge() call, pass it to the function instead. - Pass the line width and color appended to the other parameters instead of in a dictionary. The javascript code is simpler, no need to check for all containers existence, and the JSON output is smaller. - Reindent setColor() comments and fix code spacing.

templates: move Graph.edge() implementation in mercurial.js All implementation in graph.tmpl are the same. It can still be overriden if necessary. There is no clear reason to keep it separated from mercurial.js.

formatter: convert status command

ui: add formatter method

formatter: add basic formatters

encoding: introduce utf8-b helpers

graphmod: add config cache Before, we'd lookup the branch for every edge segment in the entire graph: extremely expensive. This happened even when no per-branch settings existed. Now we define a revision -> config cache function that's LRU-cached and is a no-op when no configuration exists. Still not terribly fast, but hopefully only one real branch lookup per revision. This might degenerate for wide graphs as the LRU is hard-coded to 20 elements.

graphmod: rewrite graph config validation Our goal is not to strictly disallow _invalid_ input, simply disallow _hostile_ input. Avoid using re Avoid creating empty dicts when no branch parameters are recognized

graph: in hgrc specify line color for main branch You can specify color to visually distinguish main branch (trunk) on hgweb's graph page. If color specified, all branch heads will share same color. Settings format is branch_name.color = value, where color is six hexadecimal digits e.g.: [graph] default.color = FF0000

graph: in hgrc specify line width for main branch You can specify width to visually distinguish main branch (trunk) on hgweb's graph page. Settings format is branch_name.width = value, where width in px e.g.: [graph] default.width = 3

merge with stable

filemerge: remove some redundancy in decorators/docstrings

filemerge: create detail of internal merge tools from documentation string this patch introduces 'internaltoolsmarker' which creates detail of each internal merge tools from documentation string for 'hg help merge-tools'.

filemerge: refactoring of 'filemerge()' current 'filemerge.filemerge()' implementation is verfy complicated. - it is not easy to add new internal merge tools (only by patching on 'filemerge()', or replacing it completely) - cleanup of temporary files is unsatisfactory ('internal:dump' does not, in fact) this is patch for refactoring of 'filemerge()' to isolate each internal merge tool implementations from 'filemerge()', and clean up common part in it.

patch: fuzz more aggressively to match patch(1) behaviour The previous code was assuming a default context of 3 lines. When fuzzing, it would take this value in account to reduce the amount of removed line from hunks top or bottom. For instance, if a hunk has only 2 lines of bottom context, fuzzing with fuzz=1 would do nothing and with fuzz=2 it would remove one of those lines. A hunk with one line of bottom context could not be fuzzed at all. patch(1) has apparently no such restrictions and takes the fuzz level at face value. - test-import.t: fuzz/offset changes at the beginning of file are explained by the new fuzzing behaviour and match patch(1) ones. Patching locations are different but those of my patch(1) do not make a lot of sense right now (patched output are the same) - test-import-bypass.t: more agressive fuzzing makes a patching supposed to fail because of context, succeed. Change the diff to avoid this. - test-mq-merge.t: more agressive fuzzing would allow the merged patch to apply with fuzz, but fortunately we disallow this behaviour. The new output is kept. I have not enough experience with patch(1) fuzzing to know whether aligning our implementation on it is a good or bad idea. Until now, it has been the implementation reference. For instance, "qpush" tolerates fuzz (test-mq-merge.t runs the special case of pushing merge revisions where fuzzing is forbidden).

patch: fix fuzzing of hunks without previous lines (issue3264) When applying hunks such as: @@ -2,1 +2,2 @@ context +change fuzzing would empty the "old" block and make patchfile.apply() traceback. Instead, we apply the new block at specified location without testing. The "bottom hunk" test was removed as patch(1) has no problem applying hunk with no context in the middle of a file.

patch: make hunk.fuzzit() compute the fuzzed start locations - It moves hunks processing weirdness where it belongs - It helps reusing said weirdness whenever fuzzit() is called, like during the actual hunk fuzzing.

patch: fuzz old and new lines at the same time In theory, the fuzzed offsets for old and new lines should be exactly the same as they are based on hunk parsing. Make it true in practice.

largefiles: optimize update speed by only updating changed largefiles Historically, during 'hg update', every largefile in the working copy was hashed (which is a very expensive operation on big files) and any largefiles that did not have a hash that matched their standin were updated. This patch optimizes 'hg update' by keeping track of what standins have changed between the old and new revisions, and only updating the largefiles that have changed. This saves a lot of time by avoiding the unecessary calculation of a list of sha1 hashes for big files. With this patch, the time 'hg update' takes to complete is a function of how many largefiles need to be updated and what their size is. Performance tests on a repository with about 80 largefiles ranging from a few MB to about 97 MB are shown below. The tests show how long it takes to run 'hg update' with no changes actually being updated. Mercurial 2.1 release: $ time hg update 0 files updated, 0 files merged, 0 files removed, 0 files unresolved getting changed largefiles 0 largefiles updated, 0 removed real 0m10.045s user 0m9.367s sys 0m0.674s With this patch: $ time hg update 0 files updated, 0 files merged, 0 files removed, 0 files unresolved real 0m0.965s user 0m0.845s sys 0m0.115s The same repsoitory, without the largefiles extension enabled: $ time hg update 0 files updated, 0 files merged, 0 files removed, 0 files unresolved real 0m0.799s user 0m0.684s sys 0m0.111s So before the patch, 'hg update' with no changes was approximately 9.25s slower with largefiles enabled. With this patch, it is approximately 0.165s slower.

mq: make qimport --push push all imported patches (issue3130) Only the first imported one was pushed.

convert: tolerate spaces between splicemap parent ids (issue3203) Splicemap lines are documented in hg help convert like: key parent1, parent2 but parsed like: key, parents = line.strip().rsplit(' ', 1) parents = parents.replace(',', ' ').split() The rsplit() call was introduced to handle spaces in keys for the generic mapfile format. Spaces can appear in svn identifiers since they contain path components. This logic makes less sense with splicemap since svn identifiers can also appear on the right side, even if it is a bit less likely. Given the parsing is theorically broken, I would rather follow what is documented already and is correct in the main case where all identifiers are hg hashes. Also, using svn identifiers in a splicemap sounds difficult as they are not easily accessible.

test-commandserver: flush stdout

localrepo: clear _filecache on rollback (issue3261) Files are being replaced by rollback but the corresponding data in localrepo isn't actually updated for things like bookmarks, phases, etc. Then when rollback is done, the cache is updated thinking it has the most up-to-date data, where in fact it is still pre-rollback. We clear _filecache to force everything to be recreated.

scmutil: update cached copy when filecached attribute is assigned (issue3263) When assigning a new object to filecached properties, the cached object that was kept in the _filecache map was still holding the old object. By implementing __set__, we track these changes too and update the cached copy as well.

cmdserver: invalidate the dirstate when running commands (issue3271) The dirstate is invalidated separately outside of invalidate() which is already being called (other callers of invalidate() seems to suggest the separation is there for a reason).

localrepo: reset _phasesdirty flag after writing

import: handle git renames and --similarity (issue3187) There is no reason to discard copy sources from the set of files considered by addremove(). It was done to handle the case where a first patch would create 'a' and a second one would move 'a' to 'b'. If these patches were applied with --no-commit, 'a' would first be marked as added, then unlinked and dropped from the dirstate but still passed to addremove(). A better fix is thus to exclude removed files which ends being dropped from the dirstate instead of removed. Reported by Jason Harris <jason@jasonfharris.com>

context: make workingctx.forget() really warn about untracked files

largefiles: check whether specified patterns are related to largefiles strictly current 'lfiles_repo.status()' implementation examines whether specified patterns are related to largefiles in working directory (not to STANDIN) or not by NOT-EMPTY-NESS of below list: [f for f in match.files() if f in lfdirstate] but it can not be assumed that all in 'match.files()' are file itself exactly, because user may only specify part of path to match whole under subdirectories recursively. above examination will mis-recognize such pattern as 'not related to largefiles', and executes normal 'status()' procedure. so, 'hg status' shows '?'(unknown) status for largefiles in working directory unexpectedly. this patch examines relation of pattern to largefiles by applying 'match()' on each entries in lfdirstate and checking wheter there is no matched entry. it may increase cost of examination, because it causes of full scan of entries in lfdirstate. so this patch uses normal for-loop instead of list comprehensions, to decrease cost when matching is found.

merge with stable

log: remove caching of all visited revisions (issue3253) Not only does this eat all available memory for some users, it's slower.

pull: backout change to return code This bit a number of people.

convert: use splicemap entries when sorting revisions (issue1748) When sorting revisions before converting them, we have to edit the revision graph using splicemap entries. Otherwise, a spliced revision may be converted before its synthetic parents. Invalid splicemap revisions are now detected before starting the conversion.

convert: turn splicemap into a simple dictionary Parsing the splicemap as a mapfile was a pain because map does not let us override its parsing code and splicemap entries are not key/values. Besides we had no need for mapfiles extra features. Just parse the splicemap and return a dictionary.

debugrevspec: mention --verbose to print the parsed tree

largefiles: only cache largefiles in new heads This fixes a serious performance regression in largefiles introduced in Mercurial 2.1. Caching new largefiles on pull is necessary, because otherwise largefiles will be missing (and unable to be downloaded) when the user tries to merge or rebase a new head with an old one. But this is an expensive operation and should only be done for heads that are new from the pull, rather than on all heads in the repository.

merge with stable

mq: restore _branchtags() fast path (issue3223) Since a5917346c72e, mq saves the nodeid of the first applied patch to cache/branchheads, which breaks the optimized cache handling introduced in fbf8320f25c8. The problem is the revision being committed is appended to mqrepo.applied after the commit succeeds, which means mqrepo._branchtags() performs a regular update and write the first applied patch to the branch cache. One solution is to set a context variable _committingpatch on the mqrepo while it is committing a patch and to take it in account when deciding to fast-path mqrepo._branchtags(). Not really elegant but it works. The changes to test-mq-caches.t reverse changes introduced by a5917346c72e. The cache should not have been updated with mq records. The changes to test-keyword.t are indirectly caused by a5917346c72e. Reported and analyzed by Yuya Nishihara <yuya@tcha.org> Notes: - qpush still makes a slow path _branchtags() call when checking heads. Maybe this can be optimized. - be careful when merging this patch in default as secretcommit() was renamed newcommit() right after the end of the code freeze.

mq: ensure all mq commits are made with secretcommit() Having a common code path helps fixing things globally.

convert/bzr: ignore nested repos when listing branches (issue3254) Reported by A.S. Budden <abudden@gmail.com>

tests: tighten checks for octal escapes in shell printf. printf on AIX default shell ksh (89) says \1 is an invalid escape. It insists on at least 2 digits. This causes failures in test-keyword.t and test-status.t. check-code.py already looks out for \NNN and recommends using Python for outputting octal values. Extend the check to \NN and \N and fix up resulting failures.

phase: when phase cannot be reduced, hint at --force and return 1 (BC) Before, trying to change the phase of a "public" node to "draft" would result in: $ hg phase --draft . no phases changed [0] With this patch: $ hg phase --draft . cannot move 1 changesets to a more permissive phase, use --force no phases changed [1] On partial failures, the exit status is now 1 instead of 0 and the number of changed nodes is displayed while it was only with --verbose. It seems useful to tell the user that despite the apparent failure, something changed. $ hg phase --draft '5 or 7' cannot move 1 changesets to a more permissive phase, use --force phase changed for 1 changesets [1]

revset: fix alias substitution recursion (issue3240) The revset aliases expansion worked like: expr = "some revset" for alias in aliases: expr = alias.process(expr) where "process" was replacing the alias with its *unexpanded* substitution, recursively. So it only worked when aliases were applied in proper dependency order. This patch rewrites the expansion process so all aliases are expanded recursively at every tree level, after parent alias rewriting and variable expansion.

update: just merge unknown file collisions The unknown file collision rule was introduced as an extension of the "should be clean when merging" rule. Unfortunately, it got applied to the normal update path, which should be happy to merge local changes. This patch gives us merges for unknown file collisions on update, while preserving abort for merge and update -c.

merge: don't use unknown() This removes use of unknown files for building the synthetic working directory manifest used by manifestmerge. Instead, we adopt the strategy used by _checkunknown. Side-effect: unknown files are no longer moved by remote directory renames, and now are left alone like ignored files.

merge: refactor unknown file conflict checking Previously, we would do a full working directory walk including unknown files to perform a merge. In many cases, this was painful because unknown files greatly outnumbered tracked files and generally had no useful effect on the merge. Here we instead wait until we find a file in the destination that's not tracked locally and detect if it exists and is not ignored. This is usually cheaper but can be -more- expensive in the case where we're adding a huge number of files. On the other hand, the cost of statting the new files should be dwarfed by the cost of eventually writing them. In this version, case collisions are detected implicitly by os.path.exists and wctx[f] lookup.

update: use normal update path with --check (issue2450) This avoids clobbering unknown files on update by not using overwrite mode.

fetch: use update rather than clean when updating (issue3246) We already verify the working directory is "clean" before starting so there's no advantage to clobbering the working directory.

merge with stable