check-code: suggest policy.importmod This forbids patterns like `from mercurial.cext import parsers` which breaks pure.

test-obsolete-bundle-strip: test bundling in the seventh case Updating all cases in the file at the same time is a bit overwhelming and harder to double check. So each case come in its own patch.

test-obsolete-bundle-strip: test bundling in the sixth case Updating all cases in the file at the same time is a bit overwhelming and harder to double check. So each case come in its own patch.

test-obsolete-bundle-strip: test bundling in the fifth case Updating all cases in the file at the same time is a bit overwhelming and harder to double check. So each case come in its own patch.

test-obsolete-bundle-strip: test bundling in the fourth case Updating all cases in the file at the same time is a bit overwhelming and harder to double check. So each case come in its own patch.

test-obsolete-bundle-strip: test bundling in the third case Updating all cases in the file at the same time is a bit overwhelming and harder to double check. So each case come in its own patch.

test-obsolete-bundle-strip: test bundling in the second case Updating all cases in the file at the same time is a bit overwhelming and harder to double check. So each case come in its own patch.

test-obsolete-bundle-strip: test bundling in the first case We adds a shell function checking that the relevant markers and the bundled markers matches. Updating all cases in the file at the same time is a bit overwhelming and harder to double checks. So each cases come in its own patch.

obsolete: raise richer exception on unknown version We raise a more precise subclass of Abort with details about the faulty version. This will be used to detect this case and display some information in debugbundle.

import-checker: add hgdemandimport to local modules This fixes test-check-module-imports.t.

rebase: drop unnecessary parentchange call We're calling localrepo.setparents here, not dirstate.setparents. localrepo.setparents calls dirstate.parentchange already.

hidden: remove unnecessary guard condition The "if visible" guard is now pretty pointless, because the function call it guards will do almost no work anyway when there are no visible revisions. We can also stop wrapping "visible" in a set since it just needs to be an iterable now.

hidden: subtract pinned revs from hidden earlier The pinned revs are simply revisions that should not be hidden even if hideablerevs() says that should. Let's make that clear by simply setting "hidden = hideablerevs() - pinnedrevs()" early on.

hidden: make _revealancestors() reveal ancestors exclusively I think this seems more expected. It also prepares for the next commit.

hidden: remove _consistencyblockers() Roughly speaking, we currently do this to reveal hidden ancestors of visible revisions: 1. Iterate over all visible non-public revisions and see if they have hidden parents 2. For each revision found in step (1) walk the chain of hidden commits and reveal it We can simplify that by skipping step (1) and doing step (2) from all visible non-public revisions instead. This doesn't seem to have much impact on "perfvolatilesets". Before: ! obsolete ! wall 0.004616 comb 0.000000 user 0.000000 sys 0.000000 (best of 570) ! visible ! wall 0.008235 comb 0.010000 user 0.010000 sys 0.000000 (best of 326) After: ! obsolete ! wall 0.004727 comb 0.010000 user 0.010000 sys 0.000000 (best of 543) ! visible ! wall 0.008371 comb 0.000000 user 0.000000 sys 0.000000 (best of 324)

hidden: pass revs to iterate into _consistencyblockers() Instead of passing the domain into _consistencyblockers() and having the function calculate the set of revisions to iterate over, let the caller do it. This is just a minor refactoring to make future changes simpler.

hidden: remove unnecessary 'domain' parameter from _revealancestors() The function will stop searching as soon as it runs into a non-hidden revision, so there is no need to restrict by the domain (of mutable revisions) as well. This doesn't seem to have much impact on "perfvolatilesets". Before: ! obsolete ! wall 0.004903 comb 0.000000 user 0.000000 sys 0.000000 (best of 535) ! visible ! wall 0.008913 comb 0.010000 user 0.010000 sys 0.000000 (best of 300) After: ! obsolete ! wall 0.004616 comb 0.000000 user 0.000000 sys 0.000000 (best of 570) ! visible ! wall 0.008235 comb 0.010000 user 0.010000 sys 0.000000 (best of 326)

hidden: change _domainancestors() to _revealancestors() This change makes the function actually reveal the ancestors by removing them from the hidden set. This prepares for further simplification. Note that the function will now only reveal contiguous chains of hidden revisions, but that's fine because we always pass it an immediate child of any revision that should be revealed (or the revision itself). This doesn't seem to have much impact on "perfvolatilesets". Before: ! obsolete ! wall 0.004672 comb 0.010000 user 0.010000 sys 0.000000 (best of 590) ! visible ! wall 0.008936 comb 0.010000 user 0.010000 sys 0.000000 (best of 322) After: ! obsolete ! wall 0.004903 comb 0.000000 user 0.000000 sys 0.000000 (best of 535) ! visible ! wall 0.008913 comb 0.010000 user 0.010000 sys 0.000000 (best of 300)

hidden: rename "revealedrevs" to "pinnedrevs" (API) E.g. tags and bookmarks can reveal revisions that would otherwise be hidden. A revision can also be revealed because one if its descendants is visible. Let's use the term "pinned" for the former case (bookmarks etc.).

hidden: drop obsolete comment about cacheability The cache was recently dropped, so I believe the comment is now obsolete.

cat: add formatter support This is an example showing how formatter can handle the --output option. git subrepo isn't supported for now.

cat: use with statement to close output file

cat: stop using makefileobj() Prepares for porting to the formatter API. We won't be able to utilize the abstraction provided by makefilename() because formatter must be instantiated per file.

formatter: add nullformatter This can be used as a placeholder variable.

formatter: add helper to create a formatter optionally backed by file To make things simple, openformatter() and maybereopen() have no support for a plain object API. Callers must use the "with" statement. Unlike cmdutil.makefileobj(), append mode ('ab') isn't supported by these functions. This is because JSON output can't be simply concatenated. Perhaps cmdutil.export() will have to build a {filename: [revs...]} map first and write revs per file.

formatter: add option to redirect output to file object Commands like 'export' have --output=OUTFILESPEC option, so we need a way to write formatter output optionally to a file.

merge with i18n

i18n-pt_BR: synchronized with a0e46f6b248b

zsh_completion: install as _hg not hg The contrib/zsh_completion file itself says to name it _hg. With a name like `hg`, if the user has a line like `autoload ${^fpath}/*(N-.:t)` in their zshrc, it will create a shell function named `hg` that will hide the actual hg command and make hg unusable. Separately from that though, the underscore prefix makes it actually work. The zsh man page states: The convention for autoloaded functions used in completion is that they start with an underscore This does not seem to just be a "convention", though. With the ill-advised line removed from my zshrc and the file named `/usr/local/share/zsh/site-functions/hg` (without the underscore), these completions did not seem to get loaded and the ones from the zsh installation were loaded instead. If I renamed them to be `/usr/local/share/zsh/site-functions/_hg`, however, they were loaded. I manually tested the above statement by starting a new zsh instance with the file in `/usr/local/share/zsh/site-functions` with the following names: - As `hg`, `which _hg_labels` did not show anything - As `_hg`, `which _hg_labels` showed the expected function.

osx: override default exclude filter for pkgbuild To quote `man 1 pkgbuild`: --filter filter-expression By default, --root will include the entire contents of the given root-path in the package payload, except for any .svn or CVS directories, and any .DS_Store files. You can override these default filters by specifying one or more --filter options. Each filter-expression is an re_format(7) ``extended'' expression: any path in the root which matches any of the given expressions will be excluded from the pack- age payload. (Note that specifying even one --filter inhibits the default filters, so you must respecify the default fil- ters if you still want them to be used.) It turns out the default filter these days *also* includes .git and .hg. Notice how that filter expression is a regular expression? That (presumably unintentionally) prevents a file named "chg" or "_hg" from getting included in the distribution. Many many thanks to spectral@ for trying to include a _hg file which led us to figure this bug out. Bug filed with Apple for this as rdar://problem/32437369, mentioning both the gap in documentation and the wrong defaults.

bugzilla: fix typo in help text

help: fix typos

win32mbcs: avoid unintentional failure at colorization Since 176ed32dc159, pycompat.bytestr() wrapped by win32mbcs returns unicode object, if an argument is not byte-str object. And this causes unexpected failure at colorization. pycompat.bytestr() is used to convert from color effect "int" value to byte-str object in color module. Wrapped pycompat.bytestr() returns unicode object for such "int" value, because it isn't byte-str. If this returned unicode object is used to colorize non-ASCII byte-str in cases below, UnicodeDecodeError is raised at an operation between them. - colorization uses "ansi" color mode, or Even though this isn't default on Windows, user might use this color mode for third party pager. - ui.write() is buffered with labeled=True Buffering causes "ansi" color mode internally, regardless of actual color mode. With "win32" color mode, extra escape sequences are omitted at writing data out. For example, with "win32" color mode, "hg status" doesn't fail for non-ASCII filenames, but "hg log" does for non-ASCII text, because the latter implies buffered formatter. There are many "color effect" value lines in color.py, and making them byte-str objects isn't suitable for fixing on stable. In addition to it, pycompat.bytestr will be used to get byte-str object from any types other than int, too. To resolve this issue, this patch does: - replace pycompat.bytestr in checkwinfilename() with newly added hook point util._filenamebytestr, and - make win32mbcs reverse-wrap util._filenamebytestr (this is a replacement of 176ed32dc159) This patch does two things above at same time, because separately applying the former change adds broken revision (from point of view of win32mbcs) to stable branch. "_" prefix is added to "filenamebytestr", because it is win32mbcs specific hook point.

tests: fix typo in "flagprocesor"

contrib: remove unnecessary debug output from editmergeps.ps1

contrib: fix a bug preventing editmergeps.ps1 from running unknonw editors '$ executable' is not a way to run executable in powershell, '& executable' is instead. Found this when testing with regular Windows notepad.

contrib: make editmergeps use -NoNewWindow option in Start-Process cmdlet Running 'Start-Process -Wait "vim" "+10" "filename"' from PowerShell actually spawns a separate cmd window to run vim in. This looks ugly and in most cases not what user wants. During my initial testing I was using the Cmder app, which made me not notice this (it captures new windows as new tabs).

contrib: run editmergeps.ps1 from the same location as editmergeps.bat This change is needed for cases when user does not put editmergeps.bat directly into PATH, but rather uses 'merge-tools.editmergeps.executable' config option to provide a full path to editmergeps.bat. In such cases, editmergeps.ps1 cannot be run simply by name, it needs a full path. In BATCH file %~dp0 stands for the directory in which the original file is located.

help: pass commands module by argument This removes import cycle.

copies: introduce getdstfctx Previously `c2` may had an incorrect linkrev because getsrcfctx set wrong _descendantrev. getsrcfctx() sets descendant rev equals to srcctx.rev() (see _makegetfctx()), but for `c2` descendant rev should be dstctx. While we were lucky it didn't broke copytracing it made it significantly slower in some cases. Besides it broke some external extensions, for example remotefilelog.

copies: rename getfctx to getsrcfctx In the next patch we'll use getdstfctx. Let's rename getfctx to getsrcfctx in this patch.

copies: remove msrc and mdst parameters This function already has lots of parameters. And we can get manifests from contexts. So let's get msrc and mdst parameters from srcctx and dstctx.

copies: add dstctx parameter Add parameter with destination context

copies: rename ctx to srcctx In the next diff we'll pass new dstctx parameter. Let's rename ctx to srcctx in this patch.

copies: rename m2 to mdst Small refactoring to rename m2 to more clearer mdst.

copies: rename m1 to msrc Small refactoring that renames `m1` parameter name to a more clearer name `msrc`.

transaction: delete callbacks after use Before this change, localrepository instances that performed multiple transactions would leak transaction objects. This could occur when running `hg convert`. When running `hg convert`, the leak would be ~90 MB per 10,000 changesets as measured with the Mercurial repo itself. The leak I tracked down involved the "validate" closure from localrepository.transaction(). It appeared to be keeping a reference to the original transaction via __closure__. __del__ semantics and a circular reference involving the repo object may have also come into play. Attempting to refactor the "validate" closure proved to be difficult because the "tr" reference in that closure may reference an object that isn't created until transaction.__init__ is called. And the "validate" closure is passed as an argument to transaction.__init__. Plus there is a giant warning comment in "validate" about how hacky it is. I did not want to venture into the dragon den. Anyway, we've had problems with transactions causing leaks before. The solution then (14e683d6b273) is the same as the solution in this patch: drop references to callbacks after they are called. This not only breaks cycles in core Mercurial but can help break cycles in extensions that accidentally introduce them. While I only tracked down a leak due to self.validator, since this is the 2nd time I've tracked down leaks due to transaction callbacks I figure enough is enough and we should prevent the class of leak from occurring regardless of the variable. That's why all callback variables are now nuked.

match: remove special-casing of always-matching patterns in patternmatcher This moves the optimization for patterns that match everything to the caller, so we can remove it from patternmatcher. Note that we need to teach alwaysmatcher to use relative paths now in cases like "hg files .." from inside mercurial/, because while it still matches everything, paths should be printed relative to the working directory.

match: move normalize() call out of matcher constructors By passing in the result of the normalize() call, we prepare for moving the special handling of patterns that always match out of the patternmatcher. It also lets us remove many of the arguments from the matcher, because they were passed only the the normalize function (we could have removed the arguments by binding them to the function instead of moving the normalize() call out).

match: drop support for empty pattern list in patternmatcher Since the caller now deals with empty pattern lists, we can drop that support in the patternmatcher. It now gets the more logical behavior of matching nothing when no patterns are given (although there is no in-core caller that will pass no patterns).

match: optimize visitdir() for when no explicit files are listed In patternmatcher, we used to say that all directories should be visited if no explicit files were listed, because the case of empty _files usually implied that no patterns were given (which in turns meant that everything should match). However, this made e.g. "hg files -r . rootfilesin:." slower than necessary, because that also ended up with an empty list in _files. Now that patternmatcher does not handle includes, the only remaining case where its _files/_fileset fields will be empty is when it's matching everything. We can therefore treat the always-case specially and stop treating the empty _files case specially. This makes the case mentioned above faster on treemanifest repos.

match: handle everything-matching using new alwaysmatcher Having a special matcher that always matches seems to make more sense than making one of the other matchers handle the case. For now, we just use this new matcher when no patterns were provided.

match: add __repr__ for subdirmatcher Should at least be useful for debugging. Would matter for correctness too if fsmonitor or Facebook's sparse extension worked with subrepos (which I don't know if they do).

tests: make test-manifest.py portable to Python 3 Lots of b prefixes here, and https://bugs.python.org/issue29714 means that this test is still very broken on Python 3.6 and 3.6.1, but 3.6.2 should things (based on testing using tip of the 3.6 branch from git). #cleanup-only

cleanup: rename all iteritems methods to items and add iteritems alias Due to a quirk of our module importer setup on Python 3, all calls and definitions of methods named iteritems() get rewritten at import time. Unfortunately, this means there's not a good portable way to access these methods from non-module-loader'ed code like our unit tests. This change fixes that, which also unblocks test-manifest.py from passing under Python 3. We don't presently define any itervalues methods, or we'd need to give those similar treatment.

help: work around textwrap.dedent() only working on strings

server: write out pid using bytes IO instead of str IO

help: convert dict to strkwargs

util: use sysstr.join instead of bytes.join in textwrap wrapper

tests: port test extension in test-help.t to python 3 The changes required herein suggest to me that we should probably accept ascii-safe unicode strings for command name, flag name, etc.

doc: port check-seclevel.py to be Python 2/3 portable

match: define exactmatcher.matchfn statically This should eliminate the reference cycle, self.matchfn -> self.exact -> self.

match: remove override of prefix() from differencematcher It's exactly the same as basematcher.prefix().

cat: do not start pager if output will be written to file

cat: pass filename template as explicit argument I'll move the handling of the '-' filename to commands.cat().

cmdutil: extract function checking if pattern should be taken as stdin/out This will be used in commands.cat().

cmdutil: drop deprecated hack to pass file object to makefileobj() (API) All callers pass a string 'pat' or None.

encoding: make sure "wide" variable never be referenced from other modules Better to not expose (maybe-) unicode objects.

manifest: use itertools.chain() instead of + for Python 3 compat This is all pure-Python code, so I'm not too worried about perf here, but we can come back and fix it should it be a problem. With this change, the manifest code passes most unit tests on Python 3 (once the tests are corrected with many b prefixes. I've got a little more to sort out there and then I'll mail that change too.

manifest: fix some pure-Python parser bits to work on Python 3

tests: make test-manifest finish importing in Python 3 The test is still broken, but now it executes.

tests: drop assertIn polyfill now that we're 2.7-only

perf: benchmark command for revlog indexes We didn't have explicit microbenchmark coverage for loading revlog indexes. That seems like a useful thing to have, so let's add it. We currently measure the low-level nodemap APIs. There is room to hook in at the actual revlog layer. This could be done as a follow-up. The hackiest thing about this patch is specifying revlog paths. Other commands have arguments that allow resolution of changelog, manifest, and filelog. I needed to hook in at a lower level of the revlog API than what the existing helper functions to resolve revlogs allowed. I was too lazy to write some new APIs. This could be done as a follow-up easily enough. Example output for `hg perfrevlogindex 00changelog.i` on my Firefox repo (404418 revisions): ! revlog constructor ! wall 0.003106 comb 0.000000 user 0.000000 sys 0.000000 (best of 912) ! read ! wall 0.003077 comb 0.000000 user 0.000000 sys 0.000000 (best of 924) ! create index object ! wall 0.000000 comb 0.000000 user 0.000000 sys 0.000000 (best of 1803994) ! retrieve index entry for rev 0 ! wall 0.000193 comb 0.000000 user 0.000000 sys 0.000000 (best of 14037) ! look up missing node ! wall 0.003313 comb 0.000000 user 0.000000 sys 0.000000 (best of 865) ! look up node at rev 0 ! wall 0.003295 comb 0.010000 user 0.010000 sys 0.000000 (best of 858) ! look up node at 1/4 len ! wall 0.002598 comb 0.010000 user 0.010000 sys 0.000000 (best of 1103) ! look up node at 1/2 len ! wall 0.001909 comb 0.000000 user 0.000000 sys 0.000000 (best of 1507) ! look up node at 3/4 len ! wall 0.001213 comb 0.000000 user 0.000000 sys 0.000000 (best of 2275) ! look up node at tip ! wall 0.000453 comb 0.000000 user 0.000000 sys 0.000000 (best of 5697) ! look up all nodes (forward) ! wall 0.094615 comb 0.100000 user 0.100000 sys 0.000000 (best of 100) ! look up all nodes (reverse) ! wall 0.045889 comb 0.050000 user 0.050000 sys 0.000000 (best of 100) ! retrieve all index entries (forward) ! wall 0.078398 comb 0.080000 user 0.060000 sys 0.020000 (best of 100) ! retrieve all index entries (reverse) ! wall 0.079376 comb 0.080000 user 0.070000 sys 0.010000 (best of 100)

perf: rename perfrevlog to perfrevlogrevisions We have a couple of commands beginning with "perfrevlog." The actual "perfrevlog" command actually measures resolving the fulltext of multiple revisions. So let's rename it to reflect what it actually does.

server: use pycompat to get argv

encoding: make wide character class list a sysstr That's what east_asian_width returns, so just match it.

cmdutil: use sorted(dict) instead of x = dict.keys(); x.sort() The former both does less work and has the virtue of working on Python 3.

util: look for empty-sysstr instead of empty-bytesstr in textwrap code Fixes behavior on Python 3.

minirst: look for column delimiters using slices instead of indicies This works on both Python 2 and 3.

minirst: grab a byte, not an int, for the underline style

minirst: use bytes.strip instead of str.strip

py3: use pycompat.bytestr so that we don't get ascii values This fixes `hg files 'set:(**.py)'` which makes test-check-py3-compat.t able to run on Python 3. So if you now do `python3 ./run-tests.py test-check-py3-compat`, the test will actually run.

py3: update test-check-py3-compat.t

loader: pywatchman appears to already be py3 compatible Our loader was doing some confusing things in pywatchman, but it looks like we shouldn't be using it there anyway.

py3: update test expectations for py3-commands test

context: do not cache manifestctx This will make sure when ctx.repo.manifestlog changes, a correct new manifestctx is returned. repo.manifestlog takes care of caching so the manifestctx won't be reconstructed every time.

test-context: add a case demonstrating manifest caching problem This issue was discovered when testing absorb on Windows. What happens are: 1. ctx.p1().manifestctx gets cached. let's call ctx.p1().manifestctx._revlog() "mrevlog1" 2. repo.manifestlog gets invalidated. let's call repo.manifestlog._revlog "mrevlog2" 3. repo.commitctx(ctx) commitctx uses ctx.p1().manifestctx and writes to "mrevlog1" 4. repo[n].manifest() cannot find the manifest node in "mrevlog2" This patch adds a test case to reproduce the issue.

debugbundle: display the content of obsmarkers parts We parse and display the markers in the part when possible.

bundle: add an experimental knob to include obsmarkers in bundle The "hg bundle" command is a good place to test if the inclusion of obsmarkers within a bundle is working well (part exists, content is correct etc). So we add a way to have them included. Ideally, this would be controlled by a change around bundlespec (bundlespec "v3" + arguments). However, my main goal is to have obsmarkers included in bundle created by the 'hg strip' command, not the 'hg bundle' so for now I'm avoiding the detour through bundlespec rework territory. Better debug output for obsmarkers in 'debugbundle' will be added in later changesets. The 'test-obsolete-bundle-strip.t' test will also get updated in a later changeset to keep the current changeset smaller.

bundle2: move function building obsmarker-part in the bundle2 module We move it next to similar part building functions. We will need it for the "writenewbundle" logic. This will allow us to easily include obsmarkers in on-disk bundle, a necessary step before having `hg strip` also operate on markers. (Yes, the bundle2 module was already too large, but there any many interdependencies between its components so it is non-trivial to split, this is a quest for another adventure.)

policy: remove unused policynoc and policynocffi constants

cffi: remove superfluous "if True" blocks

cffi: split modules from pure The copyright lines are updated per change history. cffi/osutil.py isn't tested since I have no access to OS X machine right now, sorry.

policy: extend API version checks for cffi This is just a stub for future extension. I could add a version constant to CFFI modules by putting it to both ffi.set_source() and ffi.cdef(), but that doesn't seem right. So for now, cffi modules will be explicitly unversioned (i.e. version constant must be undefined or set to None.) We can revisit it later when we need to consider CFFI support more seriously.

filterpyflakes: allow reexporting pure symbols from cffi modules cffi modules will do 'from ..pure.<module> import *'.

import-checker: guess names of C extension modules Since extension modules aren't included in the list of source files, they need to be populated somehow. Otherwise the import from cext/cffi would be treated as a global one.

import-checker: convert localmods to a set of module names This makes it easy to add a source-less module name to the set.

import-checker: allow importing symbols from pure modules This allows us to re-export pure functions from cffi modules: # mercurial/cffi/base85.py from ..pure.base85 import *

cffi: put compiled modules into mercurial.cffi package Don't pollute the top-level namespace.

cffi: rename build scripts This frees up cffi package for modules to be split from pure.

match: remove support for includes from patternmatcher Includes (and excludes) are now delegated to the includematcher.

match: simplify includematcher a bit The "include" we have in symbols is redundant and the double negative in visitdir() can be removed.

match: remove support for non-include patterns from includematcher The includematcher will always get at least one include pattern and will never get any non-include patterns, so we can remove most of the code in it. This patch does mostly straight-forward deletions of code. We will clean up further later.

match: split up main matcher into patternmatcher and includematcher At this point the includematcher is an exact copy of the main matcher class. We will specialize and simplify both classes in the following patches. This initial unmodified copy is just to make the differences clearer. We also rename the main matcher to "patternmatcher" for consistency. I may eventually merge this new includematcher back into the main matcher, but I think doing it this way makes the intermediate steps clearer regardless.

match: remove support for exact matching from main matcher class Exact matching is now handled by the exactmatcher class. We can safely remove _files from the __repr__() implementation, because even though the field is set, the patternspat field is enough for the representation to be unambiguous (which was not the case when the matcher could handle exact matches).

match: handle exact matching using new exactmatcher

merge: use intersectmatchers() in "m2-vs-ma optimization" It doesn't seem like this can actually happen, but seems like cleaner anyway.

match: handle includes using new intersectionmatcher

match: move entire uipath() implementation to basematcher Even though most matchers will always want to use the relative path in uipath(), when we add support for intersecting matcher, we will want to control which form to use for any kind of matcher without knowing the type (see next patch), so we need the implementation on the base class. Also rename the attribute from "pathrestricted" to "relativeuipath" since there actually are cases where we match everything but still use relative paths (like when the user runs "hg files .." from inside mercurial/).

local-clone: also copy tags related caches This caches provide a large speedup for some repositories. Keeping it around is valuable.

local-clone: also copy revs-branch-cache files This cache provides a large speedup for some repositories. Keeping it around is valuable.

local-clone: extract the listing of caches to copy Right now, the clone only copies the branchmap caches. There are multiple other valuable caches that we should copy and extensions might add their own. So we add a function to list the cache files to copy from the repository. The repository argument is unused but extensions will want it.

local-clone: extract the closure copying caches Closures often get on the way. They are not much value in having that as a closure so I'm extracting it at the module level.

test: add isolated prune case (to test-obsolete-bundle-strip.t) This adds a test where the prune marker is not related to any other obsmarkers.

test-obsolete-bundle-strip: add a complex split and fold case This is a more complex case that checks the logic used when split and fold gets into play.

test-obsolete-bundle-strip: add cases with prune on missing revs Same as the previously added case, but the prune is no longer known locally. This will mostly matter for the strip testing. Introducing the test early will help clarify patches related to strip.

obsolete: fix relevant-obsmarkers computation on pruned changeset The markers pruning a node was not directly considered relevant for the pruned node, only to its parents. This went unnoticed during obsmarkers exchange because all ancestors of the pruned node would be included in the computation. This still affects obsmarkers exchange a bit since "inline" prune markers would be ignored (see second test case). This went unnoticed, because in such case, we always push another obsolescence markers for that node. We add explicit tests covering this case. (The set of relevant changeset is use in the obsmarkers discovery protocol used in the evolve experimental extension, the impact will be handled on the extension side).

test: add a test file for relevant obsmarkers and its usage The logic around obsmarkers "relevant" to a set of revs have a couple of test around in other places but no systematic testing. In addition, all the current testing focus on the exchange case (we looks at relevant markers for '::heads'). For bundles, we'll need something a bit different. We'll no longer have set of revision going down to the repository roots. So we'll have to test these cases too. In addition, stripping obsmarkers will introduce new logic around obsmarkers that will need testing too. So a new test file make sense here. We start with a simple tests, more advanced cases are coming in the next changesets. The extra testing catch a minor bug (later in the series).

annotate: add a new experimental --skip option to skip revs This option is most useful for mechanical code modifications, especially ones that retain the same number of lines.

annotate: add core algorithm to skip a rev The core algorithm is inspired by git hyper-blame, implemented at https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/master/git_hyper_blame.py. The heuristic is as documented in the comments.

annotate: make pair take all parents to pair against In upcoming patches we'll need to be aware of all parents at the same time. This also exposes a potential bug: if a line can be annotated with both parents of a merge commit, it'll always be annotated with p2, not p1. I'm not sure if that's what we want, but at least the code makes it clear now.

annotate: move pair function to top level We'll want to make this more complicated and have unit tests for it in upcoming patches.

bookmarks: fix check of hash-like name to not abort by ambiguous identifier 'mark in repo' may raise LookupError. I set it to not be warned since bookmark names shorter than 4 chars aren't checked and short names are likely to be ambiguous.

localrepo: document that __contains__() may raise LookupError

hidden: drop outdated comment about "dynamic" performance This comment is now irrelevant since we have a faster algorithm and no cache.

hidden: unify the static and dynamic blocker logic We no longer have cache and they both work the same way. Unifying the logic simplify the code and reduce the amount of set copies.

hidden: drop the hidden cache logic The improvement in time complexitty and the speed-up in computation is large enough that the has little use now. Its update time can even gets in the way. So we drop it. This will allow us to unify the static/dynamic blockers logic in the next changeset.

hidden: simplify the computation of consistency blocker For a couple of years, we now have precomputed set for all mutable phases. We can use this set restrict our search and quickly detect non-hideable children of hideable changesets. This speeds up the hidden computation. See docstring of the new function for details. This new version reuses the '_domainancestors' function to keep the computation of revealed changeset in O(len(visible)) Below are perfvolatilesets timing from two Mozilla repositories with different contents. hidden cache is disabled while obtaining them. 1) Mozilla repository with: * 400667 changesets * 35 hidden changesets (first rev-268334) * 288 visible drafts * 1 unstable changeset Before: ! visible ! wall 0.001744 comb 0.000000 user 0.000000 sys 0.000000 (best of 1563) After: ! visible ! wall 0.000742 comb 0.000000 user 0.000000 sys 0.000000 (best of 3755) The timing above include the computation of obsolete changeset: ! obsolete ! wall 0.000396 comb 0.000000 user 0.000000 sys 0.000000 (best of 6816) So adjusted time give 1.3ms before versus 0.3ms after. A 4x speedup. 2) Mozilla repository with: * 405645 changesets * 4312 hidden changesets (first rev-326004) * 264 visible drafts * 1 unstable changeset Before: ! visible ! wall 0.025476 comb 0.030000 user 0.030000 sys 0.000000 (best of 111) After ! visible ! wall 0.007703 comb 0.010000 user 0.010000 sys 0.000000 (best of 358) The timing above include the computation of obsolete changeset: ! obsolete ! wall 0.006408 comb 0.010000 user 0.010000 sys 0.000000 (best of 404) So adjusted time give 19ms before versus 1.3ms after. A 17x speedup.