copies: refactor checkcopies() into a top level method This moves checkcopies() out of mergecopies() and makes it a top level function in the copies module. This allows extensions to override it. For example, I'm developing a filelog replacement that doesn't have rev numbers so all the rev number dependent implementation here needs to be replaced by the extension. No logic is changed in this commit.

store: move top file walk to a separate function Some extensions find it useful to be able to walk the non-data files in the repo, so I'm moving that part of the walk to a separate function. In particular, this allows an extension to interact with only the non-filelog store data (for instance, when cloning everything but filelogs).

clone: move file stream walk to a separate function Moves the file walk out of the stream method so that extensions can override it. This allows an extension to decide what files should be streamed, and in particular allows a stream without filelogs.

dirstate._walkexplicit: inline dirsnotfound.append

dirstate._walkexplicit: rename work to dirsfound Now that this code is factored out, work is too specific a name.

dirstate.walk: refactor explicit walk into separate function This enables this code to be reused by extensions that implement the other, more time-consuming bits of walk in different ways.

dirstate.walk: pull skipstep3 out of the explicit walk code This is a move towards factoring out this code into a separate function.

dirstate.walk: move dirignore filter out of explicit walk code This is a move towards factoring this code out into a separate function.

dirstate.walk: maintain a list of dirs not found Upcoming patches will factor out the walk over explicit files done in step 1. This helps us get there.

highlight: fix page layout with empty first and last lines Repeated newlines were stripped by pygmentize, now give the option not to do so.

check-code: add another Windows pathsep rule

merge with stable

tests: quiet strip to avoid commandserver pathsep issue

tests: fix another Windows path issue

tests: fix unionrepo path issue on msys (issue3927)

merge with stable

blackbox: fix literal block syntax

largefiles: check unknown files with case awareness of the filesystem Before this patch, largefiles extension checks unknown files in the working directory always case sensitively. This causes failure in updating from the revision X consisting of '.hglf/A' (and "A" implicitly) to the revision Y consisting of 'a' (not ".hglf/A") on case insensitive filesystem, because "A" in the working directory is treated as colliding against and different from 'a' on the revision Y. This patch uses "repo.dirstate.normalize()" to check unknown files with case awareness of the filesystem.

largefiles: check existence of the file with case awareness of the filesystem Before this patch, largefiles extension always unlinks largefiles untracked on the target context in merging/updating after updating working directory. For example, it is assumed that the revision X consists of ".hglf/A" (and "A" implicitly) and revision Y consists of "a" (not ".hglf/A"). In the case of updating from X to Y, largefiles extension tries to unlink "A" after updating "a" in working directory. This causes unexpected unlinking "a" on the case insensitive filesystem. This patch checks existence of the file in the working context with case awareness of the filesystem to prevent from such unexpected unlinking. "lfcommands._updatelfile()" also unlinks target file in the case "largefile is tracked in the target context, but fails to be fetched". This patch doesn't apply "repo.dirstate.normalize()" in this case, because it should be already ensured in the manifest merging that there is no normal file colliding against any largefiles.

windows: check target type before actual unlinking to follow POSIX semantics Creation and writing into target file via vfs (a.k.a opener) is done after "unlink()" target file, if it exists. For example, it is assumed that the revision X consists of file 'A', and the revision Y consists of file 'A/B'. Merging revision X into Y tries to "unlink()" on directory 'A' of 'A/B', before creation of file 'A'. On POSIX environment, directories should be removed by "rmdir(2)", and "unlink(2)" on directories fails. "unlink()" of Mercurial (and Python) uses "unlink(2)" directly, so unlinking in the merge case above would fail. In the other hand, on Windows environment, "unlink()" of Mercurial tries to rename before actual unlinking, to follow POSIX semantics: already opened file can be unlinked safely. This causes unexpected success in unlinking in the merge case above, even though directory 'A' is renamed to another. This confuses users. This patch checks whether target is directory or not before renaming, and raises IOError(errno.EPERM) if so, to follow POSIX semantics.

Merge with stable.

manifestmerge: local unknown, remote created: don't traverse symlinks To figure out what to do with locally unknown files, Mercurial attempts to read them if they exist. When an attempt is made to read a file that exists but traverses a symlink, Mercurial aborts. With this patch, we first ensure that the file doesn't traverse a symlink before opening it. This is fine because a file being "remote created" means the symlink doesn't exist remotely, which means it will be deleted in the apply phase.

subrepo: open files in 'rb' mode to read exact data in (issue3926) Before this patch, "subrepo._calcfilehash()" opens files by "open()" without any mode specification. This implies "text mode" on Windows. When target file contains '\x00' byte, "read()" in "text mode" reads file contents in without data after '\x00'. This causes invalid SHA1 hash calculation in "subrepo._calcfilehash()". This patch opens files in 'rb' mode to read exact data in.

patch: use scmutil.marktouched instead of scmutil.addremove addremove required paths relative to the cwd, which meant a lot of extra code that transformed paths into relative ones. That code is now gone as well.

scmutil: add a function to mark that files have been operated on Several places use scmutil.addremove as a means to declare that certain files have been operated on. This is ugly because: - addremove takes patterns relative to the cwd, not paths relative to the root, which means extra contortions for callers. - addremove doesn't make clear what happens to files whose status hasn't changed. This new method accepts filenames relative to the repo root, and has a much clearer contract. It also allows future modifications that do more with files whose status hasn't changed.

scmutil.addremove: factor out code to mark added/removed/renames An upcoming patch will reuse this code in another function.

scmutil.addremove: factor out code to find renames This code will be used in a different context in upcoming patches.

scmutil.addremove: rename local 'copies' to 'renames' An upcoming patch will refactor some code out into a method called _findrenames. Having a line saying "copies = _findrenames..." is confusing. Besides, 'renames' is a more precise name for this local anyway.

scmutil.addremove: factor out dirstate walk into another function Upcoming patches will reuse and expand on this function for other purposes.

filecontext: use 'is not None' to check for filelog existence Previously we used 'if filelog:' to check if the filelog existed. If the instance did exist, this pattern then calls len() on the filelog to see if it is empty. I'm developing a filelog replacement that doesn't have len() implemented, so it's better to do an explicit 'is not None' check here instead. Also change _changeid() to return the _changeid attribute if it has it. Previously it would try to obtain it from the _changectx(), and if that did not exist it would construct the _changectx() using the linkrev. In the extension I'm working on, filectx's don't have easy access to linkrevs so avoiding this when possible is better.

filelog: use super() for calling base functions filelog had some hardcoded revlog.revlog.foo() calls. This changes it to use super() instead so that extensions can replace the filelog base class.

bookmarks: allow bookmark command to take multiple arguments This change allows setting or deleting multiple bookmarks at once. If more than one is being set and --inactive is not given, the first one is made active.

merge with stable

convert: fix bug of wrong CVS path parsing without port number (issue3678) The cvsps.py:getrepopath suffers from a string parsing bug (it returns "user@server/path/to/repository" if the CVSROOT is given like this: ":pserver:user@server/path/to/repository" ), which gives returnes the wrong value becouse cvsps.py fails to strip the prefix from filenames. With this patch for the same input we get the correct repo path that is: "/path/to/repository"

match: add comments to explain explicitdir and traversedir

match: make explicitdir and traversedir None by default With this, extensions can easily tell when traversedir and/or explicitdir don't need to be called.

dirstate.walk: cache match.explicitdir and traversedir locally

match: drop dir callback dir has been subsumed by explicitdir and traversedir.

inotify: call match.explicitdir inotify only returns explicit directories matched.

purge: hook into match.explicitdir and traversedir

localrepo.commit: hook into match.explicitdir

dirstate.walk: call match.explicitdir or traversedir as appropriate

match: introduce explicitdir and traversedir match.dir is currently called in two different places: (1) noting when a directory specified explicitly is visited. (2) noting when a directory is visited during a recursive walk. purge cares about both, but commit only cares about the first. Upcoming patches will split the two cases into two different callbacks. Why bother? Consider a hypothetical extension that can provide more efficient walk results, via e.g. watching the filesystem. That extension will need to fall back to a full recursive walk if a callback is set for (2), but not if a callback is only set for (1).

merge with stable

run-tests: when running in parallel, delete tmpdirs immediately This is especially important if extensions that use inotify are enabled, because it's very easy to hit the inotify max_user_instances limit without this.

tests: check path separator in moves

tests: fix test-commandserver phase . output for windows

hfs+: rewrite percent-escaper (issue3918) The original code was a bit too clever and got confused by some cp949 Korean text. This rewrite bytes the bullet and manually decodes UTF-8 sequences. Adds some doctests.

merge with stable

revert: ensure that copies and renames are honored (issue3920) Previously, we restored the states of files, but not the additional information the dirstate uses to track copies and renames.

hgignore: fix regression with hgignore directory matches (issue3921) If a directory matched a regex in hgignore but the files inside the directory did not match the regex, they would appear as deleted in hg status. This change fixes them to appear normally in hg status. Removing the ignore(nf) conditional here is ok because it just means we might stat more files than we had before. My testing on a large repo shows this causes no performance regression since the only additional files being stat'd are the ones that are missing (i.e. status=!), which are generally rare.

templater: fix output instability from gsoc patches

log-style: add a log style that is default+phase (issue3436) There is a new style called phases style. Usage:: hg log --style phases Why do we need this new style - in what way is it different from or similar to existing styles? The new style is default + phases information. With the new phases feature the users exhibited their desire for a new style that could help them. Why do this need a new style - couldn't it be folded into an existing style? The default style and the new one are about the same, the difference is the phases tag. The users find both styles useful, this means that the both styles must exist.

templater: show the style list when I try to use a wrong one When someone try to use a wrong style, a list with sugestions will appear. In the test-log.t file it's a test that prove this thing.

test-nested-repo.t: touch another file inside nested repo This helped uncover a bug in a patchset I've been writing.

check-code: add more path sep glob checks

splicemap: improve error handling when source is subversion (issue2084) Implemented error handling on splicemap file when source is subversion (This checks are similar to when source is hg or git). The revision string is expected to be of svn:<uuid><path>@<number> format. the test case has been enhanced to check this format.

splicemap: improve error handling when source is git (issue2084) Implemented similar error handling that is done for hg in an earlier revision. These are: a. add checking for splicemap file format b. add checking for each revision string formats

splicemap: improve error handling when source is hg (issue2084) 1. Introduced 2 levels of error handling for splicemap files a. Check the splicemap file for rules which are same across different types of source repos. This is done through enhancing parsesplicemap function b. Check revision string formats. Each repo may have their own format. This is done usign checkrevformat function c. Implemented the above two for hg

splicemap: move parsesplicemap to convcmd.py (issue2084) parsesplicemap is only referenced from convcmd.py This move is necessary to enable other changes related to this issue

merge with stable

tests: sprinkle globs over largefiles/subrepo tests for Windows

tests: skip permissions tests in test-phases-exchange when fs lacks permissions

merge with stable

Added signature for changeset 23f785b38af3

Added tag 2.6 for changeset 23f785b38af3

bookmarks: fix bug that activated a bookmark even with -r passed If the current revision was the target revision of -r, then the bookmark would be active. Test cases have been updated accordingly.

bookmarks: resolve divergent bookmark when moving across a branch This patch resolves a single divergent bookmark if a divergent bookmark exists in the target revision and it current bookmark is not an ancestor of the target revision, else it would already be handled by the previous patch in this series. Test coverage is added.

bookmarks: resolve divergent bookmarks when moving active bookmark forward This patch resolves divergent bookmarks between the current active bookmark MARK and the new destination. This situation can arise when pulling new changesets, abandoning your current changesets actively bookmarked with MARK via strip, and then doing a bare update. The non-divergent but active bookmark MARK is then moved to a common ancestor of the new changesets and the abandoned changesets. Test coverage is added.

bookmarks: resolve divergent bookmarks when fowarding bookmark to descendant This patch is a follow-up to 56dd55da2f7d that resolves divergent bookmarks between the to-be-forwarded bookmark MARK and the new descendant. This situation can happen when pulling new changesets, updating to the divergent bookmark, abandoning the previous changesets with strip, and then moving MARK to MARK@N. Test coverage is added.

tags: update tag type only if tag node is updated (issue3911) Before this patch, tag type information is always updated, even if tag previously read in has higher priority than one newly read in. This causes that the tag type is displayed as "local", even if global tag overwrites existing local one successfully. This patch updates tag type only if tag node is updated. This patch tests overwriting local tags below: - visible one (normal case) - already removed one (recorded as null)

match: fix root calculation for combining regexps with simple paths The fall-back root for walking is the repo root, not no root. The "roots" do however also end up in m.files() which is used in various ways, for instance to indicate whether matches are exact. The change could thus have other impacts.

icasefs: enhance test to prevent regression by changes in the future This patch adds steps to test case-folding collision detection on the revision tree built up in the order different from the one reproducing issue3452. This may prevent regression by changes around "copy detection" and/or "case-folding collision detection" logic in the future.

icasefs: rewrite case-folding collision detection (issue3452) Before this patch, case-folding collision detection uses "copies.pathcopies()" before "manifestmerge()", and is not aware of renaming in some cases. For example, in the case of issue3452, "copies.pathcopies()" can't detect renaming, if the file is renamed at the revision before common ancestor of merging. So, "hg merge" is aborted unexpectedly on case insensitive filesystem. This patch fully rewrites case-folding collision detection, and relocate it into "manifestmerge()". New implementation uses list of actions held in "actions" and "prompts" to build provisional merged manifest up. Provisional merged manifest should be correct, if actions required to build merge result up in working directory are listed up in "actions" and "prompts" correctly. This patch checks case-folding collision still before prompting for merge, to avoid aborting after some interactions with users. So, this assumes that user would choose not "deleted" but "changed". This patch also changes existing abort message, because sorting before collision detection changes order of checked files.

icasefs: refactor "test-casecollision-merge.t" to reuse in succeeding fixes This patch refactors "test-casecollision-merge.t" to increase reusability in succeeding patches. It is confirmed that changed test also can detect issue3370.

icasefs: ignore removed files at building "dirstate._foldmap" up on icasefs Before this patch, all files in dirstate are used to build "_foldmap" up on case insensitive filesystem regardless of their statuses. For example, when dirstate contains both removed file 'a' and added file 'A', "_foldmap" may be updated finally by removed file 'a'. This causes unexpected status information for added file 'A' at "hg status" invocation. This patch ignores removed files at building "_foldmap" up on case insensitive filessytem. This patch doesn't add any test, because this issue is difficult to reproduce intentionally: it depends on iteration order of "dirstate._map".

i18n-pt_BR: synchronized with 3f5e75c22585

merge with i18n

i18n-ja: synchronized with f01a351db791

dispatch: add support for --option=value to _earlygetopt This fixes a very confusing error message: $ hg --config=pager.enabled=off st abort: option --config may not be abbreviated!

dispatch: add doctests for _earlygetopt

push: make locking of source optional (issue3684) Having the permission to lock the source repo on push is now optional. When the repo cannot be locked, phase are not changed locally. A status message is issue when some actual phase movement are skipped: cannot lock source repo, skipping local public phase update A debug message with the exact reason of the locking failure is issued in all case.

push: factorise phase movement in a simple closure Having all phases movement centralised will help to handle special case when the local repo can not be locked as describe in issue 3684.

merge: increase safety of parallel updating/removing on icasefs "merge.applyupdates()" sorts "actions" in removal first order, and "workeractions" derived from it should be also sorted. If each actions in "workeractions" are executed in serial, this sorting ensures that merging/updating process is collision free, because updating the file in target context is always executed after removing the existing file which causes case-folding collision against the former. In the other hand, if each actions are executed in parallel, updating on a worker process may be executed before removing on another worker process, because "worker.partition()" partitions list of actions regardless of type of each actions. This patch divides "workeractions" into removing and updating, and executes the former first. This patch still scans "actions"/"workeractions" some times for ease of patch review, even though large list may cost much in this way. (total cost should be as same as before) This also changes some tests, because dividing "workeractions" affects progress indication.

hgweb: handle filtered "0" rev in navigation Before this changeset, navigation generation crashed if revision "0" was filtered. We introduce a `_first` methods on revision navigation that return the lowest unfiltered element and use it in two place were the "0" changeset was explicitly referenced. Test case are introduced.

hgweb: fix empty navigation detection For some obscure reason, changelog.node(0) returns nullid if changelog is empty. this break empty navigation detection. We fix this code by using the length of the changelog. Using the length have some issue with revision filtering but this is a small step in the right direction. Proper fix comes in later changeset.

tests: AIX can't handle negative date in test-dirstate.t test-dirstate.t fails on AIX in the absurd date test. AIX touch errors on any date prior to 1970. AIX mktime() gives an error on such dates, so the problem is deeper than touch and attempts to work around touch in Python failed. Give up. Add an AIX test to hghave and skip the absurd date test on AIX.

win32: use explicit path to "python.exe" only if it exists Before this patch, "hg.bat" for Windows environment always uses "%~dp0..\python" as explicit path to "python.exe". This path may not be valid in some cases. For example, on the environment using "virtualenv" python package, both "python.exe" and "hg.bat" are placed in the same directory. In this case, "python.exe" should be found on PATH, because virtualenv activation script puts "python.exe" on the PATH. This patch uses explicit path to "python.exe" only if it exists, and expects that "python.exe" can be found on PATH otherwise.

test-mq-strip.t: add a test for strip --keep with clean working dir This helped uncover a bug in a patchset I've been writing.

largefiles: use repo.wwrite for writing standins (issue3909)

largefiles: drop repo wrapping detection After 257afe5489d4 I see: $ hg id -q largefiles: repo method 'commit' appears to have already been wrapped by another extension: largefiles may behave incorrectly largefiles: repo method 'push' appears to have already been wrapped by another extension: largefiles may behave incorrectly be207d9b7e4b The warning is bad: * The message gives no hint what the problem is and how it can be resolved. The message is useless. * Largefiles do have its share of problems, but I don't think I ever have seen a problem where this warning would have helped. The 'may' in the warning seems like an exaggeration of the risk. Having largefiles enabled in combination with for instance mq, hggit and hgsubversion causes a warning (depending on the configuration order) but do not cause problems. Extensions might of course be incompatible, but they can be that in many other ways. The check and the message are incorrect. It would thus be better to remove the check and the warning completely. Before 257afe5489d4 the check always failed. That change made the check work more like intended ... but the intention was wrong. This change will thus also back that change out.

config: discard "%unset" values defined in the other files read in previously Before this patch, "%unset" can't unset values defined in the other files read in previously, even though online help document says that it can. It can unset only values defined in the same configuration file. For example, the value defined in "~/.hgrc" can't be unset by "%unset" in ".hg/hgrc" of the repository. This patch records "%unset"-ed values in "config.parse()", and discards corresponding values in "config.update()".

tests: rename from test-config-case.t to test-config.t for centralization Before this patch, there is no test script testing configuration handling generally. "test-config-case.t" seems to be specific for testing case sensitive configuration. This patch renames from "test-config-case.t" to "test-config.t" for centralization of tests around configuration handling.

i18n: show the non-ASCII password prompt text correctly Before this patch, the prompt text for asking password is directly passed to "getpass.getpass()" of Python standard library. In "getpass.getpass()" implementation on Windows environment, the prompt text is split into byte sequence and "msvcrt.putch()" is applied on each bytes in it. This splitting causes non-ASCII prompt text to be broken. This patch shows the prompt text for asking password on "ui.getpass()" side, and invokes "getpass.getpass()" with empty prompt text. This prevents non-ASCII prompt text from being broken in "getpass.getpass()" implementation. This patch also sets "ui.prompt" label to prompt text to follow "ui.prompt()" style.

tests: make sed usage in test-unionrepo.t cross-platform Usage of the 'i' command proves tricky. I tried to write a check-code rule, but failed.

check-code: fix sed 'i' command rule newline matching The regular expression was meant to match cases where an 'i' command was not followed by precisely a '\' and then a newline; it failed to match the newline, so cases with a '\' but no newline would erroneously pass.