dirstate: split the foldmap into separate ones for files and directories Computing the set of directories in the dirstate can be pretty expensive. For 'hg status' without arguments, it turns out we actually never need to figure out the right case for directories in the foldmap. (An upcoming patch explains why.) This patch splits up the directory and file maps into separate ones, allowing for the subsequent optimization in status.

dirstate: introduce function to normalize just filenames This will be used in upcoming patches to stop generating the set of directories in many common cases.

dirstate: factor out code to discover normalized path In upcoming patches we're going to reuse this code. The storemap is currently always the foldmap, but will vary in future patches.

dirstate: don't require exact case when adding dirs on icasefs (issue4578) We don't require it when adding files on a case insensitive filesystem, so don't require it to add directories for consistency. The problem with the previous code was that _walkexplicit() was only returning the normalized directory. The file(s) in the directory are then appended, and passed to the matcher. But if the user asks for 'capsdir1/capsdir', the matcher will not accept 'CapsDir1/CapsDir/AbC.txt', and the name is dropped. Matching based on the non-normalized name is required. If not normalizing, skip the extra string building for efficiency. '.' is replaced with '' so that the path being tested when no file is specified, isn't prefixed with './' (and therefore fail the match).

filemerge: clean up language in mergemarkertemplate help

color: fix crash in cmd.exe When 'term' is None because it isn't in the environment, don't iterate over it. Unfortunately, unsetting $TERM or exporting it as '' doesn't work in the tests, so there's no way to simulate cmd.exe in the test suite.

log: prefer 'wctx' over 'pctx' for working context

merge with stable

tags: remove scary message about corrupt tags cache Caches should be transparent. If a cache is damaged, it should silently be rebuilt, much like if it were invalid. No one seems to have ever hit this in the wild.

hgk: use switch instead of a less efficient if/elseif/if

hgk: set distinct fill and outline colour for non-public and obsolete changesets

hgk: show secret changesets differently (shape and label)

manifestv2: implement slow readdelta() without revdiff For manifest v2, revlog.revdiff() usually does not provide enough information to produce a manifest. As a simple workaround, implement readdelta() by reading both the old and the new manifest and use manifest.diff() to find the difference. This is several times slower than the current readdelta() for v1 manifests, but there seems to be no other simple option, and this is still much faster than returning the full manifest (at least for verify).

manifestv2: disable fastdelta optimization We may add support for the fastdelta optimization for manifest v2 at a later point, but let's disable it for now, so we don't have to implement it right away.

manifestv2: add (unused) config option With tree manifests, hashes will change anyway, so now is a good time to also take up the old plans of a new manifest format. While there should be little or no reason to use tree manifests with the current manifest format (v1) once the new format (v2) is supported, we'll try to keep the two dimensions (flat/tree and v1/v2) separate. In preparation for adding a the new format, let's add configuration for it and propagate that configuration to the manifest revlog subclass. The new configuration ("experimental.manifestv2") says in what format to write the manifest data. We may later add other configuration to choose how to hash it, either keeping the v1 hash for BC or hashing the v2 content. See http://mercurial.selenic.com/wiki/ManifestV2Plan for more details.

manifest: extract method for creating manifest text Similar to the previous change, this one extracts a method for producing a manifest text from an iterator over (path, node, flags) tuples.

manifest: extract method for parsing manifest By extracting a method that generates (path, node, flags) tuples, we can reuse the code for parsing a manifest without doing it via a _lazymanifest like treemanifest currently does. It also prepares for parsing the new manifest format. Note that this makes parsing into treemanifest slower, since the parsing is now always done in pure Python. Since treemanifests will be expected (or even forced) to be used only with the new manifest format, parsing via _lazymanifest was not an option anyway.

dirstate._walkexplicit: don't bother normalizing '.' The overwhelmingly common case is running commands like 'hg diff' with no arguments. Therefore the only file that'll be listed is the root directory. Normalizing that's just a waste of time. This means that for a plain 'hg diff' we'll never need to construct the foldmap, saving us a significant chunk of time. On case-insensitive HFS+ on OS X, for a large repository with over 200,000 files, this brings down 'hg diff' from 2.97 seconds to 2.36.

dirstate._walkexplicit: drop normpath calls The paths the matcher returns are normalized already.

dirstate._walkexplicit: indicate root as '.', not '' '.' is the canonical way to represent the root, and it's apparently the only transformation that normpath makes.

phases: add killswitch for native implementation

phases: move pure phase computation in a function

revset: add hook after tree parsing This will be useful to execute actions after the tree is parsed and before the revset returns a match. Finding symbols in the parse tree will later allow hashes of hidden revisions to work on the command line without the --hidden flag.

hgk: remove unused revlog import

run-tests: obtain replacements inside Test._runcommand Now that command running is part of Test, we no longer need to pass a list of replacements down through various call layers. The impetus for this change is to fetch replacements after command execution, not before. This will allow replacements to be defined as part of test execution.

hgk: remove no longer needed debug-rev-parse command

hgk: remove no longer needed debug-config command

hgk: display obsolete changesets in darkgrey

hgk: pass --hidden switch to hg subprocesses when needed

hgk: remove repetitious (and wrong) command syntax descriptions

run-tests: separate newline normalization from replacements Upcoming patches will change how the replacements system works to make it more flexible. To prepare for this, eliminate the one-off use of replacements to perform newline normalization on Windows.

run-tests: remove arguments from Test._runcommand Now that runcommand is part of the Test class, arguments that were previously coming from Test attributes can now be switched to lookups inline.

run-tests: move run into Test class Future patches will change how replacements work. Since the logic in run() is strongly tied to the operation of individual tests and since there is potential to make the implementation simpler by giving the function access to Test attributes, move it into Test.

run-tests: wait for test threads after first error The test runner has the ability to stop on first error. Tests are executed in new Python threads. The test runner starts new threads when it has capacity to do so. Before this patch, the "stop on first error" logic would return immediately from the "run tests" function, without waiting on test threads to complete. There was thus a race between the test runner thread doing cleanup work and the test thread performing activity. For example, the test thread could be in the middle of executing a test shell script and the test runner could remove the test's temporary directory. Depending on timing, this could result in any number of output from the test runner. This patch eliminates the race condition by having the test runner explicitly wait for test threads to complete before continuing. I discovered this issue as I modified the test harness in a subsequent patch and was reliably able to tickle the race condition.

run-tests: report code coverage from source directory As part of testing code coverage output, I noticed some files were being reported twice: there was an entry for the file in the install location and for the file in the source tree. I'm not sure why this is. But it resulted in under-reporting of coverage data since some lines weren't getting covered in both locations. I also noticed that files in the source directory and outside the "mercurial" and "hgext" packages were getting included in the coverage report. Cosmetically, this seemed odd to me. It's not difficult to filter paths from the report. But I figure this data can be useful (we could start reporting run-tests.py coverage, for example). This patch switches the coverage API to report code coverage from the source directory. It registers a path alias so that data from the install location is merged into data from the source directory. We now get merged results for files that were being reported in multiple locations. Since code coverage reporting now relies on the profiled install now being in sync with the source tree, an additional check to disallow code coverage when --with-hg is specified has been added. This should have been present before, as --local was previously disallowed for the same reasons. Merging the paths raises our aggregate line coverage from ~60 to 81%.

run-tests: collect aggregate code coverage Before this patch, every Python process during a code coverage run was writing coverage data to the same file. I'm not sure if the coverage package even tries to obtain a lock on the file. But what I do know is there was some last write wins leading to loss of code coverage data, at least with -j > 1. This patch changes the code coverage mechanism to be multiple process safe. The mechanism for initializing code coverage via sitecustomize.py has been tweaked so each Python process will produce a separate coverage data file on disk. Unless two processes generate the same random value, there are no race conditions writing to the same file. At the end of the test run, we combine all written files into an aggregate report. On my machine, running the full test suite produces a little over 20,000 coverage files consuming ~350 MB. As you can imagine, it takes several seconds to load and merge these coverage files. But when it is done, you have an accurate picture of the aggregate code coverage for the entire test suite, which is ~60% line coverage.

run-tests: obtain code coverage via Python API Before, we were invoking the "coverage" program provided by the "coverage" module. This patch changes the code to go through the Python API. This makes the next patch a little bit easier to reason about. A side effect of this patch is that writing code coverage reports will be slightly faster, as we won't have to redundantly load coverage data.

commands.debugrevlog: report max chain length This is sometimes useful to know. Report it.

_lazymanifest: drop unnecessary call to sorted() The entries returned from _lazymanifest.iterentries() are already sorted.

test-git-export: add globs the test runner wants on Windows The only difference for the first two was to add the globs, but the third line of output on Windows was '..\dir2\copy'. I'm not sure why 'copy' is output on Windows instead of '*'.

run-tests: explicitly handle unicode when writing xunit file The xunit writer was passing a str to a minidom API. An implicit .decode('ascii') was performed somewhere, causing UnicodeDecodeError if test output contained non-ascii sequences. This patch converts test output to utf-8 before passing it to minidom. We use the "replace" strategy to ensure invalid utf-8 sequences get munged into �.

parsers.c: avoid implicit conversion loses integer warnings These warnings are raised by Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn) and were introduced in 539b3c7eea44

test-annotate: conditionalize error output for Windows It seems better to leave the actual output in place instead of globbing everything but 'abort:', in case it starts aborting for other reasons. It isn't clear the purpose for reversing the file name position, but that originates in windows.posixfile.

test-diffstat: add a glob the test runner wants on Windows The test gets a '~' status without it.

tests: add testing for diff.showfunc The diff.showfunc config knob did not have coverage before.

manifest: make manifest.intersectfiles() internal manifest.intersectfiles() is just a utility used by manifest.matches(), and a future commit removes intersectfiles for treemanifest for optimization purposes. This commit makes the intersectfiles methods on manifestdict and treemanifest internal, and converts its test to a more generic testMatches(), which has the exact same coverage.

win32: add comment about WinError Prevent reintroducing the bug that was added in e34106fa0dc3 (and fixed with a2285e2fc949).

templates: fix "log -q" output of phases style It had the same problem as 6136704b975d, name conflicts of {node} keyword.

record_curses: fix ui bug for newly added file With record's curses interface toggling and untoggling a newly added file would lead to a confusing UI (the header was marked as partial and the hunks as unselected). Tested additionally using the curses interface with newly added, removed and modified files in a test repo.

import-checker: rotatecycle is actually the canonical cycle key So refactor to drop cyclekey().

import-checker: make search algorithm non-recursive breadth-first Breadth-first allows finding the shortest cycle including the starting module. This lets us terminate our search early when we've discovered shorter paths already. This gives a tremendous speed-up to the cycle-finding portion of the test, dropping total runtime from 39s to 3s.

import-checker: drop set() from cyclekey()

import-checker: drop duplicate element from cycle This will allow optimizing cyclekey creation

import-checker: fix rotatecycle It was duplicating the last element sometimes.

dirs.addpath: rework algorithm to search forward This improves performance because it uses strchr rather than a loop. For LLVM/clang version "Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)" on OS X, for a repo with over 200,000 files, this improves perfdirs from 0.248 seconds to 0.230 (7.3%) For gcc 4.4.6 on Linux, for a test repo with over 500,000 files, this improves perfdirs from 0.704 seconds to 0.658 (6.5%).

changeset_printer: use changectx to get status tuple log.parents() can't handle wdir() revision. Because repo.status() creates ctx objects, there would be no benefit to get parent node from changelog.

changeset_printer: replace _meaningful_parentrevs() by changeset_templater's Because changeset_printer needs pctx object anyway, there would be no benefit to avoid creation of pctx in _meaningful_parentrevs().

changeset_printer: use context objects consistently to show parents This prepares for merging changeset_printer._maningful_parentrevs() with changeset_templater's.

children: don't pass filectx to displayer displayer doesn't want a fctx but a ctx. It failed with -Tdefault template. Traceback (most recent call last): ... File "mercurial/templatekw.py", line 212, in showbookmarks bookmarks = args['ctx'].bookmarks() AttributeError: 'filectx' object has no attribute 'bookmarks'

verify: add a note about a paleo-bug In the very early days of hg, it was possible to commit /dev/null because our patch importer was too simple. Repos from this era may still exist, add a note about why we ignore this name.

cmdutil: remove some excess vertical whitespace

revert: move calculation of targetsubs earlier

shelve: add interactive mode This allows us to shelve selectively part of the changes of the workdir

shelve: add interactive mode command line option

record: change return value of recording code It makes it easier to include interactive mode to more commands that require to get a reference to the newly created node

revert: fix --interactive on local modification (issue4576) We were moving files during the backup phase and it was incompatible with the way record/crecord is working

largefiles: remove useless overrideupdate Now, "overrideupdate()" wrapping "hg update" is useless, because "workingctx.dirty() and raising Abort" in "hg update" was replaced by "cmdutil.bailifchanged()" in the previous patch, and the latter can detect changes of largefiles in the working directory.

update: replace workingctx.dirty and raising Abort by cmdutil.bailifchanged This patch makes wrapping "commands.update()" by largefiles extension useless, because "cmdutil.bailifchanged()" can detect changes of largefiles in the working directory. This patch also changes test-update-branches.t, because "cmdutil.bailifchanged()" shows more detailed information about dirty-ness of the working directory than "workingctx.dirty()".

cmdutil: allow bailifchanged to ignore merging in progress In "commands.update()", "cmdutil.bailifchanged()" isn't used for "abort if the working directory is dirty", because it forcibly examines about merging in progress. "workingctx.dirty()" used in "commands.update()" can't detect changes of largefiles in the working directory without "repo.lfstatus = True" wrapping. This is only reason of "commands.update()" wrapping by largefiles extension. On the other hand, "cmdutil.bailifchanged()" already wrapped by largefiles extension can detect changes of largefiles. This patch is a preparations for replacing "workingctx.dirty()" and raising Abort in "commands.update()" by "cmdutil.bailifchanged()". It can remove redundant "commands.update()" wrapping.

subrepo: add bailifchanged to centralize raising Abort if subrepo is dirty This patch also centralizes composing dirty reason message like "uncommitted changes in subrepository 'xxxx'".

subrepo: add dirtyreason to centralize composing dirty reason message This patch newly adds "dirtyreason()" to centralize composing dirty reason message like "uncommitted changes in subrepository 'xxxx'". There are 3 similar messages below, and this patch is a part of preparations for unifying them into (1), too. 1. uncommitted changes in subrepository 'XXXX' 2. uncommitted changes in subrepository XXXX 3. uncommitted changes in subrepo XXXX This patch chooses adding new method "dirtyreason()" instead of making "dirty()" return "reason string", because: - some of existing "dirty()" implementation is too complicated to do so simply, and - ill-mannered 3rd party subrepo classes, of which "dirty()" doesn't return "reason string", cause meaningless message (even though it is rare case)

record_curses: add test for newly added files We have a UI bug where toggling a newly added file twice in the curses interface didn't mark it as selected. This test checks that the underlying logic is working as expected, the next patch of the series fixes the UI bug.

test-manifest.py: don't test .text() with present node suffix When m.text() is called after setting a nodeid with a suffix (such as '+'), manifestdict uses the suffix-less nodeid for the text, while treemanifest includes the suffix. It would perhaps make most sense to raise an exception so the bug is found, but since the two implementations behave differently, let's just not test the behavior for now.

treemanifest: drop 22nd byte for consistency with manifestdict When assigning a 22-byte hash to a nodeid in a manifest, manifestdict drops the 22nd byte, while treemanifest keeps it. Let's make treemanifest drop the 22nd byte as well.

test-manifest.py: rewrite tests in terms of manifestdict By rewriting test-manifest.py in terms of manifestdict instead of _lazymanifest, the tests can be run on treemanifests too. There are still a few tests that fail on treemanifests. They will be addressed in the next few patches.

test-manifest.py: separate out test for double-free after copy() The test that we don't double-free anything after creating a copy is currently mixed with the __setitem__ test. Let's separate them.

revert: evaluate subrepos to revert against the working directory Reverting to a revision where the subrepo didn't exist will now abort, and matching subrepos against the working directory is consistent with how filesets are evaluated since 5b85a5bc5bbb.

revert: handle subrepos missing in the given --rev The list of subrepos to revert is currently based on the given --rev, so there is currently no way for this to fail. Using the --rev context is wrong though, because if the subrepo doesn't exist in --rev, it is skipped, so it won't be changed. This change makes it so that the revert aborts, which is what happens if a plain file is reverted to -1. Finding matches based on --rev is also inconsistent with evaluating files against the working directory (5b85a5bc5bbb). This change is made now, so as to not cause breakage when the context is switched in an upcoming patch.

osutil: mark end of string with null char, not 0 Noticed this while working on other stuff in the area.

osutil: use getdirentriesattr on OS X if possible This is a significant win for large repositories on OS X, especially with a cold cache. Unfortunately we need to keep the lstat-based implementation around for two reasons: - Not all filesystems support this call. - There's an edge case in which it's best to fall back to avoid a retry loop. More about this in the comments. The below tests are all performed on a Mac with an SSD running OS X 10.9, on a repository with over 200k files. The results are best of 5 with simulated best-effort conditions. The gains with a hot cache are pretty impressive: 'hg status' goes from 5.18 seconds to 3.79 seconds. However, a repository that large will probably already be using something like hgwatchman [1], which helps much more (for this repo, 'hg status' with hgwatchman is approximately 1 second). Where this really helps is when the cache is cold [2]: hg status goes from 31.0 seconds to 9.66. See http://lists.apple.com/archives/filesystem-dev/2014/Dec/msg00002.html for some more discussion about this function. This is based on a patch by Sean Farley <sean@farley.io>. [1] https://bitbucket.org/facebook/hgwatchman [2] There appears to be no easy way to clear the file cache (aka "vnodes") on OS X short of rebooting. purge(8) purportedly does that but in my testing had little effect. The workaround I came up with was to assume that vnode eviction was LRU, make sure the kern.maxvnodes sysctl is smaller than the size of the repository, then make sure we'd always miss the cache by running 'hg status' in another clone of the repository before running it in the test repository.

osutil._listdir: rename to _listdir_stat In upcoming patches we'll add another implementation of listdir on OS X. That implementation will have to fall back to this one under some circumstances, though. We'll make _listdir be able to detect those circumstances and use the right function as appropriate.

revset: optimize "x & fullreposet" case If self is a smartset and other is a fullreposet, nothing should be necessary. A small win for trivial query in mozilla-central repo: revset #0: (0:100000) 0) wall 0.017211 comb 0.020000 user 0.020000 sys 0.000000 (best of 163) 1) wall 0.001324 comb 0.000000 user 0.000000 sys 0.000000 (best of 2160)

debugrevspec: show nesting structure of smartsets if verbose This shows how smartsets are constructed from the query. It will be somewhat useful to track problems such as stack overflow.

revset: add __repr__ to all smartset classes This is sometimes useful for debugging.

status: add relative directory help text (issue3835) Previously, it was difficult to find out how to display the status of files relative to your current working directory. This patch adds that knowledge to the help text.

diff: rename --relative option to --root The diff output format is unable to express files outside the directory so it makes sense to name this option --root instead of --relative.

revlog: make converting from inline to non-line work after a strip The checkinlinesize function, which converts inline revlogs to non-inline, uses the current transaction's "data" field to determine how to update the transaction after the conversion. This change works around the missing data field, which is not in the transaction after a strip.

check-code: in C code, prevent space before closing parenthesis

match: remove unused assignment of ctx ctx is consumed in __init__ to build match patterns and never used after that.

revert: comment that filesets are always evaluated against workingctx

revert: take fast path also when not reverting to '.' This speeds up 'hg revert -r .^ --all --dry-run' on the Mozilla repo from 4.081s to 0.826s. Note that 'hg revert -r .^ .' does not get any faster, since '.' does not make match.always() True. I can't think of a reason it would break anything, and if it does, it's clearly not covered by tests.

revert: define 'wctx' a little earlier and use it more

match: add isexact() method to hide internals Comparing a function reference seems bad.

matcher: make e.g. 'relpath:.' lead to fast paths Several commands take the fast path when match.always() is true. However, when the user passes "." on the command line, that results in a matcher for which match.always() == False. Let's make it so such matchers return True, and have an empty list of .files(). This makes e.g. "hg log ." as fast as "hg log" and "hg revert ." as fast as "hg revert --all" (when run from repo root).

revset: add the 'subrepo' symbol This returns the csets where matching subrepos have changed with respect to the containing repo's first parent. The second parent shouldn't matter, because it is either syncing up to the first parent (i.e. it hasn't changed from the current branch's POV), or the merge changed it with respect to the first parent (which already adds it to the set). There's already a 'subrepo' fileset, but it is prefixed with 'set:', so there should be no ambiguity (in code anyway). The only test I see for it is to revert subrepos named by a glob pattern (in test-subrepo.t, line 58). Since it doesn't return a tracked file, neither 'log "set:subrepo()"' nor 'files "set:subrepo()"' print anything. Therefore, it seems useful to have a revset that will return something for log (and can be added to a revsetalias to be chained with 'file' revsets.) It might be nice to be able to filter for added, modified and removed separately, but add/remove should be rare. It might also be nice to be able to do a 'contains' check, in addition to this mutated check. Maybe it is possible to get those with the existing 'adds', 'contains', 'modifies' and 'removes' by teaching them to chase explicit paths into subrepos. I'm not sure if this should be added to the 'modifies adds removes' line in revset.optimize() (since it is doing an AMR check on .hgsubstate), or if it is OK to put into 'safesymbols' (things like 'file' are on the list, and that takes a regex, among other patterns).

tags: improve documentation The documentation for tags.py was making comprehension difficult. This patch rewrites most of the comments in the file to make it easier for mere mortals to understand what's going on.

phase: default to C implementation for phase computation

phase: compute phases in C Previously, the phase computation would grow much slower as the oldest draft commit in the repository grew older (which is very common in repos with evolve on) and the number of commits increase. By rewriting the computation in C we can speed it up from 700ms to 7ms on a large repository whose oldest draft commit is a year old.

manifest: move C bool polyfill into util.h

manifest: use util.h to get Py_ssize_t

clone: add progress support to hardlink clones (issue3059)

util: add progress callback support to copyfiles

revert: evaluate filesets against working directory (issue4497) As the failing revert tests in test-fileset-generated.t show, Revert currently creates one matcher for matching files in the working copy and another matcher for matching files in the target revision. The two matchers are created with different contexts, which means filesets are evaluated differently. Then the union of the sets of files matching the matchers in the two contexts are reverted. It doesn't seem to make sense to use two different matchers; only the context they're applied to should be different. It seems very likely that the user wants the filesets to be evaluated against the working directory, which the tests test-fileset-generated.t also assume, so let's make it so. I willingly admit that the largefiles code was modified by trial and error (according to tests).

largefiles: extract and reuse 'standin' variable in overriderevert()

largefiles: override cmdutil.revert() instead of comands.revert() By overriding the cmdutil method we don't need to override both the function and the command. Also, we get access to the 'ctx' and 'parents' variables, which will soon prove useful. Rename the 'ctx' argument to overridematch() to 'mctx' rather than letting it shadow new 'ctx'.

test-commit-interactive-curses: backout 852801178a63 Now that the imports have been fixed up, and the platform EOL difference eliminated, this test runs on Windows.

test-interactive: use stable EOL in various file generating routines The seq.py and shell echo redirected to a file ends up with platform specific EOL, which throws off the hash when the file is committed to the test repo on Windows. The other option is to glob the hashes, but the ability to see if they have changed might point out problems that would otherwise be missed.

patch.diff: restrict matcher to relative root in certain cases Previously we'd request all results, then filter by relative root. This is clearly inefficient, so we now restrict the matcher to the relative root for certain easy cases. The particular case here is when the matcher matches all files. In that case we can simply create a matcher by the relative root. This is purely an optimization and has no impact on correctness.

commands.diff: add support for diffs relative to a subdirectory Previous patches added all the backend support for this. This exposes this option in the UI.

cmdutil.diffordiffstat: add support for diffs relative to a subdirectory It's possible that some of the match patterns are outside the relative root. We warn in that case. In upcoming patches we'll add tests for this.

test-fileset.t: remove 'bar ' and 'baz\' from tested filenames This test is guarded by a '#if no-windows', but they can't be written out on vfat on Linux either. con.xml can, and for testing that's good enough. Also clean up a stray 'ls' that I mistakenly left in while writing the test.

commands.push: abort when revisions evaluate to empty set (BC) If the "-r" argument is specified to "hg push," the user has expressed an intent for a specific changeset to be present on the remote. If that expression cannot be mapped to a known changeset, the user's intent is ambiguous and cannot be acted upon without making assumptions. Previously, if arguments to `push -r <rev>` evaluated to an empty set (perhaps the user specified a revset that didn't evaluate to anything), the empty "revs" list would be passed down to "exchange.push" where it appears the empty list was being interpreted as "push everything." This patch adds validation to the "-r" argument to the push command. If the argument is specified but doesn't resolve to a changeset, the command will abort instead of doing something potentially unexpected. This patch is technically breaking backwards compatibility. I believe this is justified because the new behavior closes a crack that could result in undefined or under-defined behavior. Also, this patch doesn't drop client capabilities because if users really wanted to push all changesets, they can simply omit the "-r" argument from push completely.

largefiles: fix typo with s/__class/__class__/ Commit f78252429e0a (largefiles: don't create chain of __contains__ calls, 2015-03-11) introduced a typo with __class instead of __class__. Let's fix it. There are (obviously) no tests covering this code path, and I could not figure out a way to trigger it, so it remains untested. Spotted by Drew Gottlieb.

hgweb: pull line numbers out of main flow of source content Pulls the autogenerated line number boxes outside of the padding box of the main content of .sourcelines, allowing the first tab to be properly sized and the line numbers to be outside the main source's margin when text is wrapped.

merge with stable

crecord: don't abort on Windows when curses isn't available This is the hack portion of the previous patch, which can be backed out once we figure out how to deal with curses. Without this, the tests affected by the import problem mentioned in the previous patch fail in exactly the same way, but with the nicer 'module not available' error.

test-fileset: forget non-portable files to avoid failures on Windows The 3 deleted lines appeared on Windows, because the files were never created in the first place.

crecord: conditionalize the imports that are not available on Windows 'fcntl', 'termios' and 'wcurses' are not available on the default Windows python installation, and importing them caused widespread carnage in the test suite. There were 29 different changed test files (on top of unrelated errors), mostly in the form of an ImportError. The failures weren't related to actual crecord use, and followed the import chain: 'localrepo' -> 'subrepo' -> 'cmdutil' -> 'crecord' -> 'fcntl'

cmdutil: drop a duplicate import of crecord It looks like this crept in with 69538481ea9f.

annotate: add option to annotate working-directory files Working revision or node is displayed with "+" suffix in plain output, but null/None in machine-readable format.

committablefilectx: override linkrev() to point to the associated changectx This is necessary to annotate workingctx revision. basefilectx.linkrev() can't be used because committablefilectx has no filelog. committablefilectx looks for parents() from self._changectx. That means fctx is linked to self._changectx, so linkrev() can simply be aliased to rev().

revset: add wdir() function to specify workingctx revision by command The main purpose of wdir() is to annotate working-directory files. Currently many commands and revsets cannot handle workingctx and may raise exception. For example, -r ":wdir()" results in TypeError. This problem will be addressed by future patches. We could add "wdir" symbol instead, but it would conflict with the existing tag, bookmark or branch. So I decided not to. List of commands that will potentially support workingctx revision: command default remarks -------- ------- ----------------------------------------------------- annotate p1 useful archive p1 might be useful cat p1 might be useful on Windows (no cat) diff p1:wdir (default) export p1 might be useful if wctx can have draft commit message files wdir (default) grep tip:0 might be useful identify wdir (default) locate wdir (default) log tip:0 might be useful with -p or -G option parents wdir (default) status wdir (default) This patch includes minimal test of "hg status" that should be able to handle the workingctx revision.

win32: 'raise ctypes.WinError' -> 'raise ctypes.WinError()' WinError is a function that creates an Error, not an Error itself. This is a partial backout of e34106fa0dc3.

patch.diff: add support for diffs relative to a subdirectory For now this implementation is pretty naive -- it filters out files right before passing them into trydiff. In upcoming patches we'll add some more smarts.

patch.trydiff: add support for stripping a relative root This assumes that if relroot is not None, all the files in modified, added and removed start with it. In upcoming patches we'll follow that.

merge with stable

obsolete: mark warning for translation No good reason not to.

subrepo: add basic support to hgsubrepo for the files command Paths into the subrepo are not yet supported. The need to use the workingctx in the subrepo will likely be used more in the future, with the proposed working directory revset symbol. It is also needed with archive, if that code is to be reused to support 'extdiff -S'. Unfortunately, it doesn't seem possible to put the smarts in subrepo.subrepo(), as it breaks various status and diff tests. I opted not to pass the desired revision into the subrepo method explicitly, because the only ones that do pass an explicit revision are methods like status and diff, which actually operate on two contexts- the subrepo state and the explicitly passed revision.

mergecopies: reuse ancestry context when traversing file history (issue4537) Merge copies is traversing file history in search for copies and renames. Since 3.3 we are doing "linkrev adjustment" to ensure duplicated filelog entry does not confuse the traversal. This "linkrev adjustment" involved ancestry testing and walking in the changeset graph. If we do such walk in the changesets graph for each file, we end up with a 'O(<changesets>x<files>)' complexity that create massive issue. For examples, grafting a changeset in Mozilla's repo moved from 6 seconds to more than 3 minutes. There is a mechanism to reuse such ancestors computation between all files. But it has to be manually set up in situation were it make sense to take such shortcut. This changesets set this mechanism up and bring back the graph time from 3 minutes to 8 seconds. To do so, we need a bigger control on the way 'filectx' are instantiated during each 'checkcopies' calls that 'mergecopies' is doing. We add a new 'setupctx' that configure and return a 'filectx' factory. The function make sure the ancestry context is properly created and the factory make sure it is properly installed on returned 'filectx'.

adjustlinkrev: handle 'None' value as source When the source rev value is 'None', the ctx is a working context. We cannot compute the ancestors from there so we directly skip to its parents. This will be necessary to allow 'None' value for '_descendantrev' itself necessary to make all contexts used in 'mergecopies' reuse the same '_ancestrycontext'.

adjustlinkrev: prepare source revs for ancestry only once We'll need some more complex initialisation to handle workingfilectx case. We do this small change in a different patch for clarity.

subrepo: add the parent context to hgsubrepo This brings parity with gitsubrepo and svnsubrepo (which already reference their parent), and will be used in an upcoming patch. I'm a bit concerned that the parent context could get stale (consider what happens when the parent repo is reverted for example). I tried adding the parent context to the substate tuple so that the parent is available everywhere a state change is possible, but that made submerge() unhappy. Even with removing the parent context inside submerge(), I wasn't able to get all of the test diffs fixed. But since the other subrepos reference their parent too, if there is a problem, it is a preexisting one (that nobody seems to be running into). It can be fixed if/when it pops up.

fileset: add a fileset for portable filenames This has mostly the same semantics as the files that the 'ui.portablefilenames' config option would warn or abort about. The only difference is filenames that case-fold to the same string -- given a set of filenames we've already checked we can check whether a new one collides with them, but we don't have a way to tell which filename it collided with.

annotate: reuse ancestry context when adjusting linkrev (issue4532) The linkrev adjustment will likely do the same ancestry walking multiple time so we already have an optional mechanism to take advantage of this. Since 2896f53509a7, linkrev adjustment was done lazily to prevent too bad performance impact on rename computation. However, this laziness created a quadratic situation in 'annotate'. Mercurial repo: hg annotate mercurial/commands.py before: 8.090 after: 36.300 Mozilla repo: hg annotate layout/generic/nsTextFrame.cpp before: 1.190 after: 290.230 So we setup sharing of the ancestry context in the annotate case too. Linkrev adjustment still have an impact but it a much more sensible one. Mercurial repo: hg annotate mercurial/commands.py before: 36.300 after: 10.230 Mozilla repo: hg annotate layout/generic/nsTextFrame.cpp before: 290.230 after: 5.560

treemanifest: make hasdir() faster Same rationale as the previous change.

treemanifest: make filesnotin() faster Same rationale as the previous change.

treemanifest: make diff() faster Containment checking is slower in treemanifest than it is in manifestdict, making the current diff algorithm O(n log n). By traversing both treemanifests in parallel, we can make it O(n). More importantly, once we start lazily loading submanifests, we will be able to easily skip entire submanifest if they have the same nodeid.

treemanifest: store directory path in treemanifest nodes This leads to less concatenation while iterating, and it's useful for debugging.

treemanifest: add configuration for using treemanifest type This change adds boolean configuration option experimental.treemanifest. When the option is enabled, manifests are parsed into the new treemanifest type. Tests can be now run using treemanifest by switching the config option default in localrepo._applyrequirements(). Tests pass even when made to randomly choose between manifestdict and treemanifest, suggesting that the two types produce identical manifests (so e.g. a manifest revlog entry written from a treemanifest can be parsed by the manifestdict code).

treemanifest: create treemanifest class There are a number of problems with large and flat manifests. Copying from http://mercurial.selenic.com/wiki/ManifestShardingPlan: * manifest too large for RAM * manifest resolution too much CPU (long delta chains) * committing is slow because entire manifest has to be hashed * impossible for narrow clone to leave out part of manifest as all is needed to calculate new hash * diffing two revisions involves traversing entire subdirectories even if identical This is a first step in a series introducing a manifest revlog per directory. This change adds a new manifest class: treemanifest, which is a tree where each node has a dict of files (nodeids), a dict of flags, and a dict of subdirectories (treemanifests). So far, it behaves just like manifestdict, but it will later help us write one manifest revlog per directory. The new class is still unused; it will be used after the next change. The code is not yet optimized. Running with it (see below) makes most or all operations slower. Once we start storing manifest revlogs for every directory, it should be possible to make many of these operations much faster. The fastdelta() optimization has been intentionally not implemented for the treemanifests. We can implement it later if necessary. All tests pass when run with the following patch (and without, of couse): --- a/mercurial/manifest.py Thu Mar 19 11:08:42 2015 -0700 +++ b/mercurial/manifest.py Thu Mar 19 11:15:50 2015 -0700 @@ -596,7 +596,7 @@ class manifest(revlog.revlog): return None, None def add(self, m, transaction, link, p1, p2, added, removed): - if p1 in self._mancache: + if False and p1 in self._mancache: # If our first parent is in the manifest cache, we can # compute a delta here using properties we know about the # manifest up-front, which may save time later for the @@ -626,3 +626,5 @@ class manifest(revlog.revlog): self._mancache[n] = (m, arraytext) return n + +manifestdict = treemanifest

bookmarks: rewrite comparing bookmarks in commands.summary() by compare() This patch adds utility function "summary()", to replace comparing bookmarks in "commands.summary()". This replacement finishes centralizing the logic to compare bookmarks into "bookmarks.compare()". This patch also adds test to check summary output with incoming/outgoing bookmarks, because "hg summary --remote" is not tested yet on the repository with incoming/outgoing bookmarks. This test uses "(glob)" to ignore summary about incoming/outgoing changesets.

bookmarks: remove useless diff() Previous patches removed code paths referring it.

bookmarks: add outgoing() to replace diff() for outgoing bookmarks This replacement makes enhancement of "show outgoing bookmarks" easy, because "compare()" can detect more detailed difference of bookmarks between two repositories.

bookmarks: add incoming() to replace diff() for incoming bookmarks This replacement makes enhancement of "show incoming bookmarks" easy, because "compare()" can detect more detailed difference of bookmarks between two repositories.

manifest: avoid intersectfiles for matches > 100 files Previously we tried to avoid manifest.intersectfiles for exact matches with less than 100 files. However, when the left side of the "or" is false, the right side gets evaluated, of course, and the evaluation of "util.all(fn in self for fn in files)" is both costly in itself, and likely to be true, causing intersectfiles() to be called after all. Fix this by moving the check for less than 100 files outside of the "or" expression, thereby also making it apply for a non-exact matcher, should one be passed in.

convert: optimize convert of files that are unmodified from p2 in merges Conversion of a merge starts with p1 and re-adds the files that were changed in the merge or came unmodified from p2. Files that are unmodified from p1 will thus not be touched and take no time. Files that are unmodified from p2 would be retrieved and rehashed. They would end up getting the same hash as in p2 and end up reusing the filelog entry and look like the p1 case ... but it was slow. Instead, make getchanges also return 'files that are unmodified from p2' so the sink can reuse the existing p2 entry instead of calling getfile. Reuse of filelog entries can make a big difference when files are big and with long revlong chains so they take time to retrieve and hash, or when using an expensive custom getfile function (think http://mercurial.selenic.com/wiki/ConvertExtension#Customization with a code reformatter). This in combination with changes to reuse filectx entries in localrepo._filecommit make 'unchanged from p2' almost as fast as 'unchanged from p1'. This is so far only implemented for the combination of hg source and hg sink. This is a refactoring/optimization. It is covered by existing tests and show no changes - which is a good thing.

localrepo: reuse commit of parent filectx entries without rehashing It is currently only amend and debugbuilddag that will pass a filectx to localrepo._filecommit. Amend will usually not hit the case where a the filectx is a parent that just can be reused. Future convert changes will use it more.

obsolete: avoid infinite loop from obs-cycle in divergence (issue4126) As for other currently in place cycle detection, arbitrarily cut the first obsolescence link that create a cycle avoiding the infinite loop. This will have to be made more deterministic in the future but we do not really care right now.

tests: avoid deprecation warning

walkchangerevs: make followfilter a top-level class The class only depends on the 'repo' variable in the closure, so let's move the class out of the function and make it explicit that that (the repo) is all it needs.

patch._applydiff: resolve prefix with respect to the cwd This has several advantages compared to resolving it relative to the root: - '--prefix .' works as expected. - consistent with upcoming 'hg diff' option to produce relative patches (I made sure to put in the (glob) annotations this time!)

test-revert-interactive: eliminate bashism in wildcard pattern The test failed with Debian dash.

devel: also warn about transaction started without a lock Nobody should start a transaction on an unlocked repository. If developer warnings are enabled this will be reported. This use the same config as bad locking order since this is closely related.

contrib: remove the now useless lock-checker.py extension This feature is in core now, and we do not keep backward compability for contrib.

devel: move the lock-checking code into core If the developer warnings are enabled, bad locking order will be reported without the need for the contrib extension.

patch.pathtransform: prepend prefix even if strip is 0 Silly oversight by me.

walkchangerevs: simplify by using match.always() method Since "slowpath = match.anypats() or (match.files() and opts.get('removed'))", we can simplify the condition "not slowpath and not match.files()" to "not match.anypats() and not match.files()", which is equivalent to "match.always()".

localrepo: simplify by using match.always() method

localrepo: remove check for matcher object that's never None

context.walk: walk all files when file and '.' given When both '.' (the working copy root) and an explicit file (or files) are in match.files(), we only walk the explicitly listed files. This is because we remove the '.' from the set too early. Move later and add a test for it. Before this change, the last test would print only "3".

context.walk: call with util.all() a generator, not a list The file set can be large, so avoid going through the entire file set when a file happens not to be in the context.

obsolete: remove last instance of _enabled The _enabled bool has been replaced by obsolete.isenabled(...). This removes the last instance of it so I can remove the _enabled flag entirely shortly.

revbranchcache: write cache even during read operations Previously we would only actually write the revbranchcache to disk if we were in the middle of a write operation (like commit). Now we will also write it during any read operation. The cache knows how to invalidate itself, so it shouldn't become corrupt if multiple writers try at once (and the write-on-read behavior/risk is the same as all our other caches).

revbranchcache: move cache writing to the transaction finalizer Instead of writing the revbranchcache during updatecache (which often happens too early, before the cache is even populated), let's run it as part of the transaction finalizer. It still won't be written for read-only operations, but that's no worse than it is today. A future commit will remove the actual write that happens in updatecache(). This is also good prep for when all caches get moved into the transaction.

revbranchcache: populate cache incrementally Previously the cache would populate completely the first time it was accessed. This could take over a minute on larger repos. This patch changes it to update incrementally. Only values that are read will be written, and it will only rewrite as much of the file as strictly necessary. This adds a magic value of '\0\0\0\0' to represent an empty cache entry. The probability of this matching an actual commit hash prefix is tiny, so it's ok if that's always considered a cache miss. This is also BC safe since any existing entries with '\0\0\0\0' will just be considered misses. Perf numbers: Mozilla-central: hg --time log -r 'branch(mobile)' -T. Cold Cache: 14.7s -> 15.1s (3% worse) Warm Cache: 1.6s -> 2.1s (30% worse) Mozilla-cental: hg perfbranchmap 2s -> 2.4s (20% worse) hg: hg log -r 'branch(stable) & branch(default)' Cold Cache: 3.1s -> 1.9s (40% better - because the old code missed the cache on both branch() revset iterations, so it did twice the work) Warm Cache: 0.2 -> 0.26 (30% worse) internal huge repo: hg --time log -r 'tip & branch(default)' Cold Cache: 65.4s -> 0.2s (327x better) While this change introduces minor regressions when iterating over every commit in a branch, it massively improves the cold cache time for operations which touch a single commit. I feel the better O() is worth it in this case.

revbranchcache: move entry writing to a separate function This moves the actual writing of entries to the cache to a separate function. This will allow us to use it in multiple places. Ex: in one place we will write dummy entries, and in another place we will write real data.

revbranchcache: store repo on the object Previously we would instantiate the revbranchcache with a repo object, use it briefly, then require it be passed in every time we wanted to fetch any information. This seems unnecessary since it's obviously specific to that repo (since it was constructed with it). This patch stores the repo on the revbranchcache object, and removes the repo parameter from the various functions on that class. This has the other nice benefit of removing the double-revbranchcache-read that existed before (it was read once for the branch revset, and once for the repo.revbranchcache).

revbranchcache: move out of branchmap onto localrepo Previously the revbranchcache was a field inside the branchmap. This is bad for a couple reasons: 1) There can be multiple branchmaps per repo (one for each filter level). There can only be one revbranchcache per repo. In fact, a revbranchcache could only exist on a branchmap that was for the unfiltered view, so you could have branchmaps exist for which you couldn't have a revbranchcache. It was funky. 2) The write lifecycle for the revbranchcache is going to be different from the branchmap (branchmap is greedily written early on, revbranchcache should be lazily computed and written). This patch moves the revbranchcache to live as a field on the localrepo (alongside self._branchmap). This will allow us to handle it's lifecycle differently, which will let us move it to be lazily computed in future patches.

revbranchcache: add test for when the cache is not writable The revbranchecache code already handled the case when the cache file wasn't writable, but let's add a test as well so future changes don't regress this.

patch.trydiff: add a docstring It took me a bit to figure out what this function actually does.

keyword: monkeypatch patch.diff more generically This function doesn't need access to any of the args or kwargs, so make the monkeypatching more robust. (In upcoming patches we'll introduce another argument to patch.diff, and this function would break if it weren't for this patch.)

keyword: rename kw_diff to kwdiff in keeping with Mercurial style rules In an upcoming patch we'll change this function's signature. If the name is kept the same, test-check-commit-hg.t complains.

fetch: use an abort hint where appropriate

eol: replace "working copy" with "working directory" in extension help

revset: replace "working copy" with "working directory" in function help

commands: say "working directory" in full spelling

commands: replace "working copy" with "working directory" in help/messages "working directory" is the standard term, we should use it consistently. But I didn't touch the hint, "run 'hg update' to get a working copy", because "get a working directory" sounds a bit odd.

test-commit-interactive-curses: #require 'tic' When run on Windows, this test aborts: @@ -20,7 +20,8 @@ > X > EOF $ hg commit -i -m "a" -d "0 0" - no changes to record + abort: No module named fcntl! + [255] $ hg tip changeset: -1:000000000000 tag: tip Maybe there's another way to get the screen size on Windows (it dies in crecord.gethw()), but for now, quiet the test noise by skipping it if terminal info compiler is unavailable.

check-code: enforce the usage of 'seq.py' instead of 'seq'

tests: replace uses of 'seq' with portable 'seq.py'

tests: introduce 'seq.py' as a portable replacement for 'seq' OS X 10.6.8 doesn't have it, while 10.10 does. I'm not sure when it was added. It may be missing from other platforms as well. This currently doesn't handle the string manipulation options (-f, -s and -w in MinGW anyway), since there is currently no need for it. Since xrange defaults to starting at 0 instead of 1, and treats the end as exclusive instead of inclusive, the args need to be extracted instead of doing: xrange(*[int(a) for a in sys.argv[1:]]) Therefore, the step might as well be added, even though there is no current use.

revert: add flag to make revert interactive

record: move ui.write wrapping where it should be We have to do that for the coloring to work in interactive revert

record: refactor the filtering code

record: consolidate ui.write wrapping in a function

bookmarks: reuse @number bookmark, if it refers changeset referred remotely Before this patch, "@number" suffixed bookmark may be newly created at each "hg pull" from the remote repository, if the bookmark in remote repository diverges from one in local one. This causes unexpected increase of "@number" suffixed bookmarks. This patch reuses "@number" suffixed bookmark, if it refers the changeset which is referred by the same bookmark in the remote repository.

bookmarks: check @pathalias suffix before available @number for efficiency Before this patch, available "@number" suffix is searched before "@pathalias" suffix, even though the latter has higher priority than the former if the latter exits. This patch checks "@pathalias" suffix before available "@number" for efficiency. When an URL has multiple path definitions, the first one is used for "pathalias" after this patch, even though the last one is used before this patch, because: - this choice can terminate loop immediately for efficiency - such case seems to be rare

bookmarks: prevent divergent bookmark from being updated unexpectedly Before this patch, "@99" suffixed bookmark may be updated unexpectedly by the bookmark value on the remote side at "hg pull", if all of "@1" to "@99" suffixed bookmarks exist in the local repository, because variable "n" still refers "@99" suffixed bookmark after the loop to examine "@num" suffixes, even though it already exists in the local repository. This patch prevents divergent bookmark from being updated unexpectedly, and shows warning message in such situation. This patch uses original python script "seq.py" instead of "seq" command to create sequence numbers in the test, because "seq" command may not be available: it isn't defined in recent POSIX specification (POSIX.1-2001 2013 Edition or XPG7)

manifest: include Python.h before standard headers Python.h should be included before any standard headers according to the python docs: https://docs.python.org/2/c-api/intro.html#include-files

crecord: use colwidth instead of ucolwidth

manifest: speed up matches for large sets of files If the number of files being matched is large, the bisection overhead can dominate, which caused a performance regression for revert --all and histedit. This introduces a (fairly arbitrary) cross-over from using bisections to bulk search.

perf: add methods for timing changeset file list reading

censor: add exchange tests, via local push/pull and bundle/unbundle These tests exercise the edge cases of exchanging censored commits between censor-aware Mercurial clients. These tests do not extend to cover backwards-compatible exchange; such tests require a script which builds older versions of Mercurial. For more background on the censorship feature design, see: http://mercurial.selenic.com/wiki/CensorPlan

censor: add censor command to hgext with basic client-side tests The censor command is a core extension which can replace the contents of a historical file revision with a censor "tombstone" which can be exchanged with older clients in place of the real revision data. The command rewrites the filelog by copying revision-by-revision. Care must be taken to expand the fulltext of the children of the censored revision before copying them to the new filelog; they might be stored as deltas against the uncensored revision, and those deltas will be invalidated. For more background on the censorship feature design, see: http://mercurial.selenic.com/wiki/CensorPlan

record: add comparison methods for recordhunk class

record: minor refactoring of dorecord It prepares the way for introducing the flag to reverse hunk selection

record: add tests for the curses recording interface

record: enable curses recording logic with experimental flag

record: change arguments of curses recording function This way the arguments are the same for both curses and non-curses versions.

record: change interface of the filtering function This way filtering functions accept chunks and return chunks

hgweb: replace 'ctx._repo' with 'ctx.repo()'

pvec: replace 'ctx._repo' with 'ctx.repo()'

scmutil: replace 'ctx._repo' with 'ctx.repo()'

templater: replace 'ctx._repo' with 'ctx.repo()'

largefiles: replace 'ctx._repo' with 'ctx.repo()'

obsolete: replace 'ctx._repo' with 'ctx.repo()'

fileset: replace 'ctx._repo' with 'ctx.repo()'

filectx: add a repo accessor This is similar to a07314472a80 in motivation. All contexts now have this method, so the rest of the 'ctx._repo' uses can be converted without worrying about what type of context it is.

run-tests: ignore ENOENT failures when removing old .err results When the same test runs in multiple threads and the previous run was a failure, the threads can race to delete the error output. This fixes that.

run-tests: stop storing start/stop times in a dict by test name This resolves the last breakage in run-tests that prevented me from running a single test many times in several threads in parallel. This will be useful for testing potential fixes to flaky tests.

run-tests: avoid running the same test instance concurrently There's a fair amount of mutable state stored on test case instances. That causes many weird failures if you try to do something like `run-tests.py -j16 --loop test-help.t`. The quick fix is this slightly weird test-reloading dance, which ensures that every time a test is executed it runs on a fresh instance of the TestCase subclass.

run-tests: add --runs-per-test flag This is useful when you're working with a flaky test and want to run it (for example) 500 times to see if it'll false-fail. This currently breaks if you use it with more than one thread, but I'm looking into that now.

convert: adjust progress bar for octopus merges (issue4169) For merges, we walk the files N-1 times, where N is the number of parents. This means that for an octopus merge with 3 parents and 2 changed files, we actually fetch 6 files. This corrects the progress output of the convert command when such commits are encountered.

test-convert-git: show insane progress bar with octopus merge (issue4169) This is done as a separate change from the fix so that it's obvious what is corrected by the bugfix.

manifest: have context use self.hasdir() A couple places in context currently use "x in self._dirs" to check for the existence of the directory, but this requires that all directories be loaded into a dict. Calling hasdir() instead puts the work on the the manifest to check for the existence of a directory in the most efficient manner.

manifest: add hasdir() to context This is a convenience method that calls to its manifest's hasdir(). There are parts of context that check to see if a directory exists, and this method will let implementations of manifest provide an optimal way to find a particular directory.

manifest: add manifestdict.hasdir() method Allows for alternative implementations of manifestdict to decide if a directory exists in whatever way is most optimal.

manifest: have context's dirs() call its manifest's dirs() This lets the context's dirs() method be agnostic towards any alternate manifest implementations.

manifest: add dirs() to manifestdict Manifests should have a method of accessing its own dirs, not just the context that references the manifest. This makes it easier for other optimized versions of manifests to compute their own dirs in the most efficient way.

formatter: convert None to json null It will be used by "annotate" command to represent the workingctx revision.

localrepo: extend "changeid in repo" to return True for workingctx revision This is necessary to implement "wc" symbol for workingctx, that will be used as follows: $ hg annotate -r wc FILE In principle, "rev in repo" should be True if "repo[rev]" can return a context object. But when it was implemented by ea3acaae25bb, lookup() had a long logic to map all sorts of changeids to nodes, and "None in repo" did crash because lookup() could not accept None. So I assume that the case of changeid=None was not considered. Now "None in repo" doesn't crash, it should be True for workingctx revision. Behavior of "changeid in repo": revision "null" existing rev None (workingctx) ---------- ------ ------------ ----------------- original* True True TypeError current True True False this patch True True True (*original: ea3acaae25bb)

merge with stable

crecord: fix another underbar

crecord: fiddle with vertical whitespace

crecord: fix underbar style for orig_stdout

crecord: use encoding.ucolwidth

crecord: more import style

crecord: fix some import style

manifest.c: ensure realloc_if_full() returns 1 or 0 This fixes an MSVC 2008 warning that I don't see with gcc 4.6.3-2: warning C4047: 'return' : 'bool' differs in levels of indirection from 'line *' More importantly, the truncation from pointer to 'unsigned char' would have returned 0 if self->lines pointed to an address divisible by 0xFF, which causes find_lines() to return MANIFEST_OOM. I was able to cause this to happen in a trivial program with the gcc compiler.

record: change recording function to work with crecord

record: add crecord's ui logic to core Code adapted from https://bitbucket.org/edgimar/crecord/src to respect coding convention an record's interface

record: change interface of dorecord to accept new filters This makes it easier to add different filtering logic (record /crecord ...)

record: remove duplicated tests Since the record and commit -i commands are identical we simplify record's test to just cover the help message and minimal smoke testing.

record: make record use commit -i

style: kill ersatz if-else ternary operators Although Python supports `X = Y if COND else Z`, this was only introduced in Python 2.5. Since we have to support Python 2.4, it was a very common thing to write instead `X = COND and Y or Z`, which is a bit obscure at a glance. It requires some intricate knowledge of Python to understand how to parse these one-liners. We change instead all of these one-liners to 4-liners. This was executed with the following perlism: find -name "*.py" -exec perl -pi -e 's,(\s*)([\.\w]+) = \(?(\S+)\s+and\s+(\S*)\)?\s+or\s+(\S*)$,$1if $3:\n$1 $2 = $4\n$1else:\n$1 $2 = $5,' {} \; I tweaked the following cases from the automatic Perl output: prev = (parents and parents[0]) or nullid port = (use_ssl and 443 or 80) cwd = (pats and repo.getcwd()) or '' rename = fctx and webutil.renamelink(fctx) or [] ctx = fctx and fctx or ctx self.base = (mapfile and os.path.dirname(mapfile)) or '' I also added some newlines wherever they seemd appropriate for readability There are probably a few ersatz ternary operators still in the code somewhere, lurking away from the power of a simple regex.

cvsps: use a different tiebreaker to avoid flaky test After adding some sneaky debug printing[0], I determined that this test flaked when a CVS commit containing two files starts too close to the end of a second, thus putting file "a" in one second and "b/c" in the following second. The secondary sort key meant that these changes sorted in a different order when the timestamps were different than they did when they matched. As far as I can tell, CVS walks through the files in a stable order, so by sorting on the filenames in cvsps we'll get stable output. It's fine for us to switch from sorting on the branchpoint as a secondary key because this was already the point when we didn't care, and we're just trying to break ties in a stable way. It's unclear to be if having the branchpoint present matters anymore, but it doesn't really hurt to leave it. With this change in place, I was able to run test-convert-cvs over 650 times in a row without a failure. test-convert-cvcs-synthetic.t appears to still be flaky, but I don't think it's *worse* than it was before - just not better (I observed one flaky failure in 200 runs on that test). 0: The helpful debug hack ended up being this, in case it's useful to future flaky test assassins: --- a/hgext/convert/cvsps.py +++ b/hgext/convert/cvsps.py @@ -854,6 +854,8 @@ def debugcvsps(ui, *args, **opts): ui.write(('Branch: %s\n' % (cs.branch or 'HEAD'))) ui.write(('Tag%s: %s \n' % (['', 's'][len(cs.tags) > 1], ','.join(cs.tags) or '(none)'))) + if cs.comment == 'ci1' and (cs.id == 6) == bool(cs.branchpoints): + ui.write('raw timestamp %r\n' % (cs.date,)) if cs.branchpoints: ui.write(('Branchpoints: %s \n') % ', '.join(sorted(cs.branchpoints)))

templates: fix "log -q" output of default style It was changed at 0ded0f0b1c04 unintentionally due to name conflicts.

merge with stable

subrepo: replace 'ctx._repo' with 'ctx.repo()'

files: replace 'ctx._repo' with 'ctx.repo()'