tests: don't hardcode errno==2 for ENOENT Hurd seems to set ENOENT to 2 + 2**30, unlike everyone else.

osutil: tab damage, how i hate thee

osutil: write a C implementation of statfiles for unix This makes a big difference to performance. In a clean working directory containing 170,000 files, performance of "hg --time diff" improves from 2.38 seconds to 1.69.

merge with i18n

i18n-ja: synchronized with f94ead934067

Added signature for changeset 0c10cf819146

Added tag 2.4.1 for changeset 0c10cf819146

osutil: fix tab damage

Merge with crew

osutil: factor out creation and init of listdir_stat

dirstate: avoid use of zip on big lists In a clean working directory containing 170,000 tracked files, this improves performance of "hg --time diff" from 1.69 seconds to 1.43. This idea is due to Siddharth Agarwal.

dirstate: move file type filtering to its source This prepares us to move to a much faster statfiles implementation on Unix.

clfilter: rename `unfilteredmeth` to `unfilteredmethod` As originally intended.

clfilter: fix a false positive in the test-obsolete.t We push between two repo which once filtered looks unrelated. Weakness in the current implementation allows this push to be done without -f. But later improvement with filtering will make this push fails for unrelatedness. However we want this push to fail for including bumped changeset. So we had a smaller push --force to make them related.

clfilter: ensure that filecache on localrepo is unfiltered All filecache usage on repo is for logic that should be unfiltered. The caches should be common to all filtered instances, and computation must be done unfiltered. A dedicated storecache subclass is created for this purpose.

clfilter: add a propertycache that must be unfiltered Some of the localrepo property caches must be computed unfiltered and stored globally. Some others must see the filtered version and store data relative to the current filtering. This changeset introduces two classes `unfilteredpropertycache` and `filteredpropertycache` for this purpose. A new function `hasunfilteredcache` is introduced for unambiguous checking for cached values on unfiltered repos. A few tweaks are made to the property cache class to allow overriding the way the computed value is stored on the object. Some logic relative to _tagcaches is cleaned up in the process.

largefile: status is buggy on repoproxy, so run unfiltered For some yet-unkown reason, largefile status does not work on repoproxy. As status is not affected by filtering, we run it unfiltered. Na'Tosha Bard's view on this issue: "but, well, largefiles status is kind of an unholy piece of code"

clfilter: mq should not warn about filtered mq patches MQ warns when qstatus contains unknown nodes. With changelog filtering, a node may be unknown because it is filtered. Thus, an unfiltered repo is used for this check.

clfilter: ensure that mq performs commits on unfiltered repos

clfilter: use unfiltered repo for bookmark push logic The remote location of the bookmark may be filtered locally. This changeset ensures that bookmark movement logic has access to all the repo's content.

clfilter: `bookmark.validdest` should run on unfiltered repo The logic recently added to `bookmark.validdest` uses data about obsolete changesets to see if a bookmark destination is valid. Obsolete changesets are likely to be filtered, so we need to work on an unfiltered repository.

clfilter: unfilter some parts of the push logic Computation of common changesets during push needs to be done on the widest set possible. An unfiltered version of the repo is kept for discovery and various revset calls. The discovery code itself enforces the filtering of unserved outgoing changeset.

subrepo: add argument to "diff()" to pass "ui" of caller side (issue3712) (API) Color extension achieves colorization by overriding the class of "ui" object just before command execution. Before this patch, "diff()" of abstractsubrepo and classes derived from it has no "ui" argument, so "diff()" of hgsubrepo uses "self._repo.ui" to invoke "cmdutil.diffordiffstat()". For separation of configuration between repositories, revision 573bec4ab7ba changed the initialization source of "self._repo.ui" from "ui"(overridden) to "baseui"(plain) of parent repository. And this caused break of colorization. This patch adds "ui" argument to "diff()" of abstractsubrepo and classes derived from it to pass "ui" object of caller side.

clfilter: prevent unwanted warning about filtered parents as unknown During changectx __init__ the dirstate's parents MAY be checked. If the repo is filtered, this check will complain "working directory has unknown parents" even if the parents are perfectly known. This may happen when the repo is used for serving and the dirstate has parents that are secret, as those secret changesets will be filtered.

clfilter: strip logic should be unfiltered Strip is a "write" operation that needs to be aware of the whole repo's content before destroying changesets. Only the low level function is altered. The top level command will still process its argument filtered (if any filtering is in place).

clfilter: verify logic should be unfiltered To verify a changelog obviously needs all of it. The verify logic now ensures it works on an unfiltered repository.

clfilter: phases logic should be unfiltered Phase computations and boundary movements need to be aware of all revisions that exist in the repository to return correct results.

clfilter: unfilter computation of obsolescence related computation All obsolescence related sets need to be computed on the full unfiltered version of the repository, in particular because several of them (obsolete, extinct) are used to compute the hidden revisions. On a filtered repo, revset predicates related to these sets will be properly filtered because of revset's own pre-filtering.

clfilter: ensure changeset creation in the repo is run unfiltered This applies to both creation of new commits and application of changegroups.

clfilter: ensure changegroup generation is run unfiltered Changegroup generation logic needs to be aware of the whole repository to work properly.

clfilter: ensure `rollback` is run unfiltered Rollback logic needs to be aware of the whole repository to work properly.

clfilter: ensure cache invalidation is done on the main unfiltered repo The proxy version will not hold any cache for now. But we have to ensure all cache operations are done on the unfiltered version.

clfilter: ensure that tag logic runs unfiltered The current tag logic is not aware of filtering. We keep the status quo, ensuring that the tag cache is computed as before: without any filtering.

clfilter: ensure `branchcache` logic runs unfiltered The current branchcache construction is not aware of filtering. We keep the status quo, ensuring that the branch cache logic is computed as before: without any filtering.

clfilter: introduce an `unfilteredmethod` decorator This decorator ensure the method in run on an unfiltered version of the repository. See follow-up commit for details. This decorator is not named `unfiltered` because it would clash with the `unfilteredmethod` on `localrepo` itself.

clfilter: introduce an "unfiltered" method on localrepo This commit is part of the changelog level filtering effort. It returns the main "unfiltered" version of a repo-like object. For localrepo this means the same localrepo object. But this method will be overwritten by the filtered versions of a repository to return the core unfiltered version of the repo. Introducing this simple method first allows later commits to prepare for the use of a filtered version of a repository. A new repo method is added because a lot of users may call it. At the end of this series of commits, about 40 calls exist in core and hgext.

clfilter: remove usage of `range` and `xrange` in scmutil.revrange For changelog level filtering to take effect it need to be used for any iteration. This changeset removes usage of `range` and `xrange` that survived the first pass.

hgweb: display diff for a changeset against any parents (issue2810) During merge of branches, it is useful to compare merge results against the two parents. This change adds this support to hgweb. To specify which parent to compare to, use rev/12300:12345 where 12300 is a parent changeset number. Two links are added to changeset web page so that one can choose which parent to compare to.

branch: add missing repo argument to checknewlabel scmutil.checknewlabel takes a repo object as its first argument. When the call to this function was added in e689b0d91546, the first argument was mistakenly set to 'None'.

Merge with stable.

rebase: fix pull --rev options clashing with --rebase (issue3619) Rebase also have a plain `--rev` option used to select the rebase set (as `--base` or `--source` would). But the content of the --rev option was intended for the remote repo and is irrelevant for the local rebase operation. We expect `hg pull --rebase` to stick with the default behavior here: hg rebase --base . --dest tip(branch(.)) The `rev` option is dropped from the option passed to rebase.

dirstate: remove obsolete comment from setbranch This comment should have been removed in e689b0d91546, when the call to scmutil.checknewlabel was removed.

merge with stable

tests: fix broken fix of test-phases output

update: allow update to existing branches with invalid names (issue3710) Starting with 361ab1e2086f, users are no longer able to update a working copy to a branch named with a "bad" character (such as ':'). Prior to v2.4, it was possible to create branch names using "bad" characters, so this breaks backwards compatibility. Mercurial must allow users to update to existing branches with bad names. However, it should continue to prevent the creation of new branches with bad names. A test was added to confirm that 'hg update' works as expected. The test uses a bundled repo that was created with an earlier version of Mercurial.

commit: increase perf by building a new addlist instead of editing the old one When commiting to a repo with lots of files (>170000), manifest.py:addlistdelta takes some time because it's editing a large array many times. Changing it to build a new array instead of editing the old one saves around 0.04 seconds on a 1.64 second commit. A 2.5% gain. The gain here is pretty minor, but it was blatantly at the top of the profiler report and the fix is straight forward. I tested it by comparing the arrays produced by the new and old logic while running all of the tests.

template engine: convert generator-based iterator to list-based iterator If a template iterator is implemented with generator, the iterator is exhau= sted after we use it. This leads to undesired behavior in template. This chang= e converts a generator-based iterator to list-based iterator when template en= gine first detects a generator-based iterator. All future usages of iterator wi= ll use list instead.

command: remove phase from the list of basic command This is not a basic command. There is no reason new user should needs to know about it. Thanks to Matt Mackall for pointing this.

merge with stable

phases: fix missing "error" module import (issue3707)

test-mq-qrefresh: test that the patch at qtip is indeed empty ...after qrefreshing a non-existent file

tests: only call check-code once The accepted warnings list is now empty, let's try to keep it that way.

ancestor: fix a comment (followup to 0b03454abae7)

revlog: allow reverse iteration with revlog.revs We often need to perform rev iteration in reverse order. This changeset makes it possible to do so, in order to avoid costly reverse or reversed() calls later.

convert: add config option to use the local time zone The default for the time zone offset in a converted changeset has always been 0 (UTC). With this patch, the converted changeset is modified so that the local offset from UTC is specified as the time zone offset. The option is specified as the boolean convert.localtimezone (default False). Example usage: hg convert -s cvs --config convert.localtimezone=True example-cvs example-hg IMPORTANT: the patch only applies to conversions from cvs or svn. The documentation for the option only appears in those two sections in the convert help text.

rebase: use revlog.findmissingrevs to compute detach set

revlog: add rev-specific variant of findmissing This will be used by rebase in an upcoming commit.

revlog: switch findmissing to use ancestor.missingancestors This also speeds up other commands that use findmissing, like incoming and merge --preview. With a large linear repository (>400000 commits) and with one incoming changeset, incoming is sped up from around 4-4.5 seconds to under 3.

ancestor: faster algorithm for difference of ancestor sets One of the major reasons rebase is slow in large repositories is the computation of the detach set: the set of ancestors of the changesets to rebase not in the destination parent. This is currently done via a revset that does two walks all the way to the root of the DAG. Instead of doing that, to find ancestors of a set <revs> not in another set <common> we walk up the tree in reverse revision number order, maintaining sets of nodes visited from <revs>, <common> or both. For the common case where the sets are close both topologically and in revision number (relative to repository size), this has been found to speed up rebase by around 15-20%. When the nodes are farther apart and the DAG is highly branching, it is harder to say which would win. Here's how long computing the detach set takes in a linear repository with over 400000 changesets, rebasing near tip: Rebasing across 4 changesets Revset method: 2.2s New algorithm: 0.00015s Rebasing across 250 changesets Revset method: 2.2s New algorithm: 0.00069s Rebasing across 10000 changesets Revset method: 2.4s New algorithm: 0.019s

bisect: add example for limiting bisection to specified directories The bisect command does not have an option to limit itself only to subdirectories, but it's possible to use revsets for the --skip option for the same effect. Given the relative obscurity of revsets, it helps to have this as another example for bisect.

subrepo: use posixpath when diffing, for consistent paths This fixes a Windows failure in test-subrepo-recursion.t introduced by c84ef0047a94.

run-tests: fix an unnoticed check-code violation

run-tests: add a --compiler option Without this option, it is not possible to run the test suite on Windows using mingw's gcc as the compiler.

run-tests: make build command line less intimidating Use a dict for parameters to the format string, instead of a ridiculous number of positional parameters.

hooks: be even more forgiven of non-fd descriptors (issue3711) Looks like there are instances where sys.stdout/stderr contain file handles that are invalid. We should be tolerant of this for hook I/O redirection, as our primary concern is not garbling our own output stream.

hooks: delay I/O redirection until we actually run a hook (issue3711) We were attempting to redirect I/O even if no hook was actually getting called. This defers redirection until we've found something to do.

util: make chunkbuffer non-quadratic on Windows The old str-based += collector performed very nicely on Linux, but turns out to be quadratically expensive on Windows, causing chunkbuffer to dominate in profiles. This list-based version has been measured to significantly improve performance with large chunks on Windows, with negligible overall overhead on Linux (though microbenchmarks show it to be about 50% slower). This may increase memory overhead where += didn't behave quadratically. If we want to gather up 1G of data to join, we temporarily have 1G in our list and 1G in our string.

revset: backed out changeset 54cedee86e51 This was causing clones of the hg repo to go from 12.4s to 14.7s.

hgk: specify some colours explicitly in hex Tk 8.6b3 uses web colours, where green is defined as #008000, not #00ff00, and grey is #c0c0c0, not #808080, so specify those colours explicitly.

hgk: no committer please Generate committer only if we really have it. Update test-hgk.t accordingly.

hgk: use Ttk instead of plain Tk Use Ttk (themed Tk) for most of the widgets. Default to xpnative theme on Windows, clam otherwise. Provide a shim for Tk 8.4 without Tile/Ttk.

check-code: move i18n check from warning to error

i18n: wrap false positives for translation detection

eol: don't refer to a random name-captured ui object Previously, we used a ui object captured at the time of monkeypatching.

mq: don't refer to a random name-captured repo object Previously, we used a repo object captured at the time of monkeypatching.

Merge with crew-stable

commit: increase perf by avoiding checks against entire repo subsets When commiting to a repo with lots of history (>400000 changesets) checking the results of revset.py:descendants against the subset takes some time. Since the subset equals the entire changelog, the check isn't necessary. Avoiding it in that case saves 0.1 seconds off of a 1.78 second commit. A 6% gain. We use the length of the subset to determine if it is the entire repo. There is precedence for this in revset.py:stringset.

commit: increase perf by avoiding unnecessary filteredrevs check When commiting to a repo with lots of history (>400000 changesets) the filteredrevs check (added with 5c89e7fa5bc2) in changelog.py takes a bit of time even if the filteredrevs set is empty. Skipping the check in that case shaves 0.36 seconds off a 2.14 second commit. A 17% gain.

merge with crew-stable

grep: remove useless while condition A revised version of 35ba170c0f82 was sent to the list that fixed this <http://markmail.org/message/jmfuiise5igcyh2m>, but the older version of the patch was applied.

merge with mpm

test-pathencode: more aggressively check for python < 2.6

diff: move index header generation to patch In an upcoming patch, we will add index information to all git diffs, not only binary diffs, so this code needs to be moved to a more appropriate place. Also, since this information is used for patch headers, it makes more sense to be in the patch module, along with other patch-related metadata.

patch: make _addmodehdr a function under trydiff addmodehdr is a header helper, same as diffline, so it doesn't need to be a top-level function and can be nested under trydiff. In upcoming patches we will generalize this approach for all headers.

diff: rewrite diffline Make diffline more readable, using strings with placeholders rather than appending to a list from many ifs that makes difficult to understand the actual output format.

diff: swap and simplify diffline args Args order swapped, since a, b are more important than revs (which is only used on non-git format), and change to read opts from context.

diff: change how quiet mode supresses diffline Before, quiet mode produced no diffline header for mercurial as a side effect of not populating "revs". This was a weird side effect, and we will always need revs for git index header that will be added in upcoming patches, so now we just check ui.quiet from diffline directly.

diff: move diffline to patch module diffline is not part of diff computation, so it makes more sense to place it with other header generation in patch module. In upcoming patches we will generalize this approach for all headers added in the patch, including the git index header.

diff: unify calls to diffline diffline was called from trydiff for binary diffs and from unidiff for text diffs. In this patch we unify those calls into one. diffline is also a header, not part of diff mechanisms, so it makes sense to remove that responsibility from the mdiff module. In upcoming patches we will move diffline to patch module and keep grouping responsibilities.

diff: move b85diff to mdiff module b85diff generates a binary diff, so we move this code to mdiff module along with unidiff for text diffs. All diffing mechanisms will be in the same place. In an upcoming patch we will remove the responsibility to print the index header from b85diff and move it back to patch, since it's a patch metadata header, not part of the diff generation.

share: always set default path to work with subrepos (issue3518) set the default path in any case because creating subrepo looks this up. Subrepos are cloned, not shared.

vfs: optimize __call__ by not calling util.split for reads dirname and basename are not used on reads

run-tests: backout 4a4173519b63 This deleted work in progress to eliminate child processes for -j.

test-pathencode: make a 2.4-safe import of collections

tests: add a randomized test for pathencode This is a probabilistic test - it generates different test cases on every run, unless invoked from the command line with a specific seed. The default number of tests run should make the tests take about a second to complete on a semi-modern laptop.

webcommands: allow hgweb's archive to recurse into subrepos Currently when obtaining an archive snapshot of a repository via the web interface, subrepositories are not taken in the snapshot. I introduce an option, archivesubrepos, which allows this.

Merge with crew-stable.

grep: don't search past the end of the searched string '*' causes the resulting RE to match 0 or more repetitions of the preceding RE: >>> bool(re.search('.*', '')) >>> True This causes an infinite loop because currently we're only checking if there was a match without looking at where we are in the searched string.

convert: fix a too-long line nag

convert: add support for converting git submodule (issue3528) Previously, convert aborted upon encountering a git submodule. This patch changes it so that it now succeeds. It modifies convert_git to manually generate '.hgsub' and '.hgsubstate' files for each git revision, so as to convert git sub modules to non-mercurial subrepositories.

run-tests.py: remove runqueue's results parameter since it is now a global

makefile: don't use system hgrc when running hg in-place This suppresses the printing of spurious error messages if a global hgrc refers to extensions that the in-place hg can't access.

setup: print subprocess stderr if there is any I just spent 1.5 days trying to debug a failing buildbot because setup.py was silently dropping the errors that were being printed by in-place hg.

run-tests: open child-parent pipes in binary mode Python's pickle is a binary format.

amend: force editor only if old message is reused (issue3698) This regression was added by 9732473aa24b. It triggered the editor even if --message or --logfile were provided.

grep: don't search past the end of the searched string '*' causes the resulting RE to match 0 or more repetitions of the preceding RE: >>> bool(re.search('.*', '')) >>> True This causes an infinite loop because currently we're only checking if there was a match without looking at where we are in the searched string.

bookmarks: introduce a bmstore to manage bookmark persistence Bookmarks persistence still showed a fair amount of its legacy as a monkeypatching extension. This encapsulates all bookmarks serialization and parsing in a single class, and offers a single location where other bookmarks storage engines can be substituted in. As a result, many files no longer import the bookmarks module, which strikes me as an encapsulation win. This doesn't do anything to the current bookmark state yet, but I'm hoping put that in the bmstore class as well.

run-tests: add --time option to log times for each test --time also prints out the wall-clock time each test takes in descending order.

run-tests: remove resultslock since it serves no useful purpose Each child process has its own copy of the results dict, so all access to the results dict is serial.

run-tests: use pickle to send results from children to parent JSON would probably be preferable, but it isn't available for Python 2.4 or 2.5.

bookmark: remove useless line in `validdest` This case is already covered by the `if old == new:` conditional at the start of the function.

bookmark: issue a single call to `allsuccessors` per loop Update to this code was minimalist when `allsuccessors` argument were changed from a list to a set. As this code is getting my attention again I realised we can drastically simplify this part of the code by issue a single call to `allsuccessors`.

bookmark: simplify mutability check in `validdest` There is no need to check phase by hand. We already have a ctx method for that.

Merge with stable

Merge with mpm

bundle: add revset expression to show bundle contents (issue3487) bundle() revset expression returns all changes that are present in the bundle file (no matter whether they are in the repo or not). Bundle file should be specified via -R option.

tags: add formatter support

manifest: add formatter support

status: use condwrite to avoid zero-width format string hack

formatter: add condwrite method This makes handling conditional output tidier