revset: speed up '_matchfiles' File matching is done by applying the matcher to all elements in the 'file' field of all changesets in the repository. This requires to read/parse all changesets in the repository and do a lot of matching. However about 1/3 of the time of the function is used to create 'changectx' object and retrieve their 'file' field. This is far too much overhead so we are skipping the changectx layer and directly access the data from the changelog. This provide use significant speed up: repository: mozilla central 252524 revisions command: hg perfrevset '_matchfiles("p:browser")' Before: 15.899687s After: 10.011705s Slowdown is even more significant if you have a lot of namespace that slowdown lookup. The time is now spent with this approximate repartition: Matcher: 20% regexp matching: 10% changelog.read: 80% reading revision: 60% checking hash: 15% decompression: 15% reading chunk: 30% changelog parsing: 20% decoding to local: 10% The next easy win is probably to have more of the changelog stack implemented using the CPython api.

mergestate: handle additional record types specially This works around a bug in older Mercurial versions' handling of the v2 merge state. We also add a bunch of tests that make sure that (1) we correctly abort when the merge state has an unsupported record type (2) aborting the merge, rebase or histedit continues to work and clears out the merge state.

test-resolve.t: remove completely unnecessary line I have no idea what I was thinking when I wrote this.

resolve: fix incorrect merge The merge from stable into default was semantically incomplete -- a couple of changes in preceding code had to be rewritten here. This code only triggers for change/delete conflicts, so we can't test it yet. We will soon be able to do it, though.

fileset: add missing() predicate (issue4925) Help of status cmd defines status file of 'missing', what is called in fileset 'deleted'. To stay consistent this patch introduces missing() predicate which in fact is alias to 'deleted'.

webutil: make _siblings into an object with __iter__ and __len__ _siblings is a helper that is used for displaying changeset parents and children in hgweb. Before, when it was a simple generator, it couldn't tell its length without being consumed, and that required a special case when preparing data for changeset template (see 9e1f4c65f5f5). Let's make it into a class (similar to templatekw._hybrid) that allows len(...) without side-effects.

mergestate: move binary format documentation into _readrecordsv2 This is too low-level to be the top-level documentation for mergestate. We're restricting the top-level documentation to only be about what consumers of the mergestate and anyone extending it need to care about.

unshelve: add support for custom merge tools For parity with merge --tool, rebase --tool etc. rebase.rebase overwrites the tool in repo.ui, so we need to explicitly pass it down there too.

unshelve: shed spurious space

unshelve: add -k as short form of --keep For parity with strip -k, rebase -k, etc.

import-checker: allow symbol imports from hgweb.common and .request This seems the convention of hgweb.

perf: un-bitrot perfstatus

util: drop statmtimesec We've globablly forced stat to return integer times which agrees with our extension code, so this is no longer needed. This speeds up status on mozilla-central substantially: $ hg perfstatus ! wall 0.190179 comb 0.180000 user 0.120000 sys 0.060000 (best of 53) $ hg perfstatus ! wall 0.275729 comb 0.270000 user 0.210000 sys 0.060000 (best of 36)

util: disable floating point stat times (issue4836) Alternate fix for this issue which avoids putting extra function calls and exception handling in the fast path. For almost all purposes, integer timestamps are preferable to Mercurial. It stores integer timestamps in the dirstate and would thus like to avoid doing any float/int comparisons or conversions. We will continue to have to deal with 1-second granularity on filesystems for quite some time, so this won't significantly hinder our capabilities. This has some impact on our file cache validation code in that it lowers timestamp resolution. But as we still have to deal with low-resolution filesystems, we're not relying on this anyway. An alternate approach is to use stat[ST_MTIME], which is guaranteed to be an integer. But since this support isn't already in our extension, we can't depend on it being available without adding a hard Python->C API dependency that's painful for people like yours truly who have bisect regularly and people without compilers.

rebase: refactoring to avoid repetition of expression This patch removes the repetition of "(revignored, revprecursor, revpruned)" and replaces its occurences with the more legible "revskipped".

rebase: remove an unused todo Since we handled skipping obsolete revs when it is relevant, this todo is no longer useful. We replace the comment with two useful comments.

rebase: don't rebase obsolete commits with no successor This patch avoids unnecessary conflicts to resolve during rebase for the users of changeset evolution. This patch modifies rebase to skip obsolete commits with no successor. It introduces a new rebase state 'revpruned' for these revisions that are being skipped and a new message to inform the user of what is happening. This feature is gated behind the config flag experimental.rebaseskipobsolete When an obsolete commit is skipped, the output is: note: not rebasing 7:360bbaa7d3ce "O", it has no successor

rebase: fix a typo in test-rebase-obsolete We had left a lonely single quote where it shouldn't be!

merge with stable

hgweb: unify import style of error classes It will be enforced by the import checker.

hgweb: do not import templatefilters.revescape and websub as symbol It will be enforced by the import checker.

hgweb: alias ui module as uimod It will be enforced by the import checker.

mergestate.commit: factor out making the list of records Will aid in testing.

mergestate: move _read() call to read constructor With this patch, mergestate.clean() will no longer abort when it encounters an unsupported merge type. However we hold off on testing it until backwards compatibility is in place.

debugmergestate: explain why we create mergestate objects directly We would normally use the read() constructor, but in this case it's fine because - we implement our own reading layer, so the extra parsing done by read() is unnecessary - read() can raise an exception for unsupported merge state records, but here we'd like to handle that separately - debugmergestate needs to be privy to mergestate internals anyway

debugdirstate: convert keyword parameters to **opts Removes the named arguments and replaces them by accessing opts. This will be used in the next patch in the series because we'll be adding more flags to debugdirstate

convert: changed test's progress output format to ignore estimate On my machine, whenever I run all test with a high -j value, test-convert-git.t would consistently fail by displaying an estimate. This patch removes that value from the output.

mq: use repo._bookmarks.recordchange instead of repo._bookmarks.write Before this patch, mq was using repo._bookmarks.write. This patch replaces this code with the recommended way of saving bookmarks changes: repo._bookmarks.recordchange.

mq: indentation change to make the next patch more legible We put the code to be indented in the next patch in a "if True:" block to make it easier to review.

bookmarks: use repo._bookmarks.recordchange instead of repo._bookmarks.write We move from the old api repo._bookmarks.write to the new api repo._bookmarks.recordchange.

localrepo: put bookmark move following commit in one transaction Before this patch, making a commit on a local repo could move a bookmark and both operations would not be grouped as one transaction. This patch makes both operations part of one transaction. This is necessary to switch to the new api to save bookmarks repo._bookmarks.recordchange if we don't want to change the current behavior of rollback. Dirstate change happening after the commit is done is now part of the transaction mentioned above. This leads to a change in the expected output of several tests. The change to test-fncache happens because both lock are now released in the same finally clause. The lock release is made explicitly buggy in this test. Previously releasing lock would crash triggering release of wlock that crashes too. Now lock release crash does not directly result in the release of wlock. Instead wlock is released at garbage collection time and the error raised at that time "confuses" python.

test-resolve.t: switch to mergestate.read() See previous patches for why we're doing this.

localrepo.commit: switch to mergestate.read() See previous patches for why we're doing this.

fileset: switch to mergestate.read() See previous patches for why we're doing this.

commands.summary: switch to mergestate.read() See previous patch for why we're doing this. We do this with a bit of care -- it would be bad form for 'hg summary' to abort completely if we encounter an unsupported merge record. Instead just warn about that and continue with the rest of the summary.

commands.resolve: switch to mergestate.read() See previous patches for why we're doing this.

shelve: switch to mergestate.read() See previous patches for why we're doing this.

mergestate: add a constructor that reads state from disk At the moment it's the same as just creating a new mergestate, but we'll soon move the _read call out of __init__ and in here.

merge.applyupdates: switch to mergestate.clean() See the previous patches for why we're doing this.

localrepo: switch to mergestate.clean() See the previous patches for why we're doing this.

strip: switch to mergestate.clean() See the previous patches for why we're doing this.

mergestate: add a constructor that sets up a clean merge state Eventually, we'll move the read call out of the constructor. This will: - avoid unnecessary reads when we're going to nuke the merge state anyway - avoid raising an exception if there's an unsupported merge record 'clean' seems like a good name for it because I wanted to avoid anything with the word 'new' in it, and 'reset' is more an action performed on a merge state than a way to get a new merge state. Thanks to Martin von Zweigbergk for feedback about naming this.

mergestate: raise structured exception for unsupported merge records We're going to catch this exception in 'hg summary' to print a better error message. This code is pretty untested, so there are no changes to test output. In upcoming patches we're going to test the output more thoroughly.

error: add a structured exception for unsupported merge records We're going to use this in summary to print a better error message.

dirstate: back out 502b56a9e897 Superseded by the parent of this commit.

osutil: make statfiles check for interrupts periodically This is a simpler and faster fix for issue4878 than the contortions performed in 502b56a9e897.

osutil: don't leak on statfiles error Found using the power of the mighty eyeball.

histedit: extracts _isdirtywc function Checking if working copy is dirty was done in few places, this patch extracts it in _isdirtywc procedure.

node: add 'nullhex', hex-encoded nullid We're going to need this for upcoming changes, because the merge state stores nodes as hex strings.

filemerge: introduce class whose objects represent files not in a context Most code is going to barf at the return values here (particularly from data and size), so we restrict it to the filemerge code. This is already somewhat supported via: ctx.filectx(f, fileid=nullid) Indeed, for add/add conflicts (ancestor doesn't have the file) we use precisely that. However, that is broken in subtle ways: - The cmp() function in filectx returns False (identical) for such a filectx when compared to a zero-length file. - size() returns 0 rather than some sort of value indicating that the file isn't present. - data() returns '' rather than some sort of value indicating that the file isn't present. Given the relatively niche use of such filectxes, this seems to be the simplest way to fix all these issues.

filectx: add isabsent method This will indicate whether this filectx represents a file that is *not* in a changectx. This will be used by merge and filemerge code to know about when a conflict is a change/delete conflict. While this is kind of hacky, it is the least bad of all the alternatives. Other options considered but rejected include: - isinstance(fctx, ...) -- not very Pythonic, doesn't support duck typing - fctx.size() is None -- the 'size()' call on workingfilectxes causes a disk stat - fctx.filenode() == nullid -- the semantics around filenode are incredibly confusing. In particular, for workingfilectxes, filenode() is always None no matter whether the file is present on disk or in either parent. Having different behavior for None versus nullid in the merge code is just asking for pain. Thanks to Pierre-Yves David for early review feedback here.

filectx: allow custom comparators We're going to introduce other sorts of filectxes very soon, and we'd like the cmp method to function properly (i.e. commutatively) for them. The only way to make that happen is for this cmp method to call into that specialized one if that defines a custom comparator.

debugmergestate: print out record type for files We're going to add a separate record type for change/delete conflicts soon. We need to make sure they get stored with the correct record type so that older versions of Mercurial correctly abort when they see change/delete records.

merge.applyupdates: don't return early if merge driver's conclude failed Somewhat silly oversight -- this prevented the progress bar from being reset.

convert: use repo._bookmarks.recordchange instead of repo._bookmarks.write Before this patch, convert was using repo._bookmarks.write, a deprecated API for saving bookmarks. This patch changes the use of repo._bookmarks.write to repo._bookmarks.recordchange.

convert: indentation change to make the next patch more legible We put the code to be indented in the next patch in a "if True:" block to make it easier to review.

strip: renaming local variables Renaming local variables to be more precise, i want to store a different list of bookmarks and it would be hard to understand what marks represents in that change therefore renaming it to repomarks. Renamed bookmarks(module) to bookmarksmod as to free up bookmarks which will be used when pluralizing bookmark.

test-rename-merge2: make selected output less arbitrary It's unclear why everything from the first 'updating:' line should be ignored. The arbitrariness makes it confusing that changing the code so e.g. the 'getting 8/f' line is printed later makes it disappear completely from the ouput. The list of 'preserving x for resolve of y' seems convered by the subsequent for loop in the test case. Perhaps it's only copies that are of interests, so let's keep only that part.

test-convert-git: drop a git warning that doesn't occur on Windows I do see this on Linux with 1.7.7.6, but not on Windows with 1.9.5.msysgit.0. Adding a (?) didn't work to conditionally ignore the line; I'm not sure if the (glob) interferes with that.

tests: fix globs for Windows