changegroup: rename dir to tree to avoid shadowing a built-in Differential Revision: https://phab.mercurial-scm.org/D4235

repository: remove storedeltachains from ifilestorage The ifilestorage interface was bootstrapped from requirements of callers outside the storage implementation (revlogs). I believe we even made some members public so they could be part of the interface! Historically, the changegroup code was a gross offender when it came to accessing low-level storage primitives. There are a handful of members on the ifilestorage interface that are/were used only for changegroup code. With the recent refactor of changegroup code and the establishment of a formal API on the storage interface for producing revision deltas, the changegroup code is no longer accessing these low-level primitives related to delta generation directly. Instead, things are abstracted away in the storage implementation. This means we can remove elements from the storage interface that are no longer needed. We start with "storedeltachains." We remove it from the interface. Then we make it a private attribute and update all references. .. api:: storedeltachains has been dropped from ifilestorage interface .. api:: storedeltachains on revlog classes is now _storedeltachains Differential Revision: https://phab.mercurial-scm.org/D4227

repository: establish API for emitting revision deltas With our revision delta and revision delta request interfaces defined, it is now time to define a method on storage interfaces for using them. So far, the only storage interface that is well-defined and used is file storage. So that is the only interface we need to add a method on. We define an ``emitrevisiondeltas()`` method that takes an iterable of ``irevisiondeltarequest``s and turns them into ``irevisiondelta`` instances. changegroup._handlerevisiondeltarequest() and the looping logic from changegroup.deltagroup() has effectively been moved to revlog.emitrevisiondeltas(). Our filelog wrapper class proxies its emitrevisiondeltas() to the internal revlog instance. The simple store test extension used to verify sanity of storage abstractions has also implemented emitrevisiondeltas() for file storage and the test harness when run with this extension doesn't seem to exhibit any regressions. Rather than create a shared type to represent revision deltas, each storage backend has its own type and the class name identifies where the revision delta was derived from. Differential Revision: https://phab.mercurial-scm.org/D4226

repository: formalize interfaces for revision deltas and requests Now that we've sufficiently abstracted how revision deltas are produced in changegroup code, we can now start the process of formalizing that as part of the interfaces defined in the repository module. This commit essentially converts the revisiondelta and revisiondeltarequest classes into well-defined interfaces. This is not strictly necessary. But I want all types formalized by the storage interface to have interfaces. This makes it much easier to test for interface conformance and for implementing new storage backends. Because the interface is documented, comments and docstrings from changegroup.py have been dropped. Differential Revision: https://phab.mercurial-scm.org/D4225

changegroup: move node sorting into deltagroup() The 3 callers of deltagroup() all sort nodes and convert to integer revs immediately before calling deltagroup(). Stepping back a bit, aside from the constraints on node ordering that always apply (namely that a node must follow its ancestor), it makes sense for sorting to be an implementation detail of the store. Moving the sorting into deltagroup() will open the door for moving sorting into a storage API. The test for the changelog in deltagroup() to impact the sorting method is a bit hacky. I suspect this will be resolved once we establish a formal interface for delta group generation and port changelog to it. Differential Revision: https://phab.mercurial-scm.org/D4224

cext: fix truncation warnings in revlog on Windows

cext: fix revlog compiler error on Windows

shortest: use nodetree for finding shortest node within revset This speeds up `hg log -T '{shortest(node,1)}\n'` in my repo from 12s to 4.5s. That's very close to the 4.1s it takes without the disambiguation revset configured. My repo has 69.5k revisions, of which 550 were in the configured revset ("not public()"). Differential Revision: https://phab.mercurial-scm.org/D4120

index: fix a comment about overflow-checking There's no "argument-checking" done in this method. This was a bad copy paste. Differential Revision: https://phab.mercurial-scm.org/D4339

tests: demonstrate a problem with renames on the p2 side of a conversion I think this is related to the octopus merge being sloppy, and that's having a cascading affect on the fixup merge. If this change is made on p1 (specifically with the 'Added parent file' commit), the failure doesn't occur. The file modification with the rename doesn't seem to be necessary, but it's what's happening in a production repo where I first noticed, so I left it. This is an example of the manifest divergence I'd been seeing, which wasn't fixed by Yuya's recent changes. This is separate from the changelog divergence I was also seeing[1]. Probably nobody cares about bzr anymore, but this will also affect git, since the octopus fixup code is in the hg sink. [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-August/120473.html

tests: add a test to show the added/modified/removed files in a bzr convert

tests: show added/modified/removed files when logging repos converted from bzr There's a bug related to incorrect status values that can be triggered by an octopus fixup merge. Seeing what the current status is (instead of just the aggregate list) will hopefully help track this down.

tests: drop an unnecessary redirect to /dev/null This eliminates the need to conditionalize the color output, because NUL on Windows is a tty. Credit to Yuya for recognizing this.

py3: whitelist another test caught by the ratchet Differential Revision: https://phab.mercurial-scm.org/D4335

index: move raise_revlog_error() further up I will add another caller below it. Differential Revision: https://phab.mercurial-scm.org/D4119

index: make node tree a Python object Differential Revision: https://phab.mercurial-scm.org/D4118

index: rename _fix_index() since it no longer fixes the index Since c0d411ea6639 (index: drop support for negative indexes into the index, 2018-07-20), it always returns the input (unless it raises). Differential Revision: https://phab.mercurial-scm.org/D4334

index: don't include nullid in boundary check in pure code Yet another change that should probably have been part of a3dacabd476b (index: don't allow index[len(index)] to mean nullid, 2018-07-20). Differential Revision: https://phab.mercurial-scm.org/D4333

fastannotate: address pyflakes concerns Two unused imports, one unused local variable assignment. Differential Revision: https://phab.mercurial-scm.org/D4337

fastannotate: move some global state mutation to extsetup() Differential Revision: https://phab.mercurial-scm.org/D4202

fastannotate: rip out specialized support for remotefilelog remotefilelog can choose to collaborate with fastannotate for now if it needs to, and in the future when we make good on our longstanding desire to move remotefilelog-like features in-house we'll make sure things are well-supported via a reasonable interface. Differential Revision: https://phab.mercurial-scm.org/D4201

fastannotate: record some TODO items from indygreg's review Some of these I'll address more or less right away, but some will take a while for us to resolve. Differential Revision: https://phab.mercurial-scm.org/D4200

fastannotate: write out files from server in a predictable order This is mostly for test output stabilization, but it *might* be slightly better with a ton of annotate caches because we'll write to one directory at a time rather than bouncing around randomly. Differential Revision: https://phab.mercurial-scm.org/D3995

fastannotate: initial import from Facebook's hg-experimental I made as few changes as I could to get the tests to pass, but this was a bit involved due to some churn in the blame code since someone last gave fastannotate any TLC. There's still follow-up work here to rip out support for old versions of hg and to integrate the protocol with modern standards. Some performance numbers (all on my 2016 MacBook Pro with a 2.6Ghz i7): Mercurial mercurial/manifest.py traditional blame time: real 1.050 secs (user 0.990+0.000 sys 0.060+0.000) build cache time: real 5.900 secs (user 5.720+0.000 sys 0.110+0.000) fastannotate time: real 0.120 secs (user 0.100+0.000 sys 0.020+0.000) Mercurial mercurial/localrepo.py traditional blame time: real 3.330 secs (user 3.220+0.000 sys 0.070+0.000) build cache time: real 30.610 secs (user 30.190+0.000 sys 0.230+0.000) fastannotate time: real 0.180 secs (user 0.160+0.000 sys 0.020+0.000) mozilla-central dom/ipc/ContentParent.cpp traditional blame time: real 7.640 secs (user 7.210+0.000 sys 0.380+0.000) build cache time: real 98.650 secs (user 97.000+0.000 sys 0.950+0.000) fastannotate time: real 1.580 secs (user 1.340+0.000 sys 0.240+0.000) mozilla-central dom/base/nsDocument.cpp traditional blame time: real 17.110 secs (user 16.490+0.000 sys 0.500+0.000) build cache time: real 399.750 secs (user 394.520+0.000 sys 2.610+0.000) fastannotate time: real 1.780 secs (user 1.530+0.000 sys 0.240+0.000) So building the cache is expensive (but might be faster with xdiff enabled), but the blame results are *way* faster. Differential Revision: https://phab.mercurial-scm.org/D3994

util: improve handling of truncated compressed streams If the compressed stream is over as marked by the reader providing nothing new and the compression engine is not providing data, bail out. This can happen in practise when the server misbehaves and would result in an infinite loop otherwise. Differential Revision: https://phab.mercurial-scm.org/D4297

zsh_completion: update all options It's just too hard to further split this patch. What it does: - adds missing flags - removes flags that are no longer there - updates flag descriptions and argument names - adds * where using the same option many times is okay - groups with () mutually exclusive flags that can't be used together - adds + and = to options that need arguments - removes + and = from options that don't take any arguments - fixes minor issues and a typo Differential Revision: https://phab.mercurial-scm.org/D4267

zsh_completion: run hg with HGPLAINEXCEPT=alias to list commands This is what contrib/bash_completion does, looks like a sensible thing to do. Differential Revision: https://phab.mercurial-scm.org/D4266

zsh_completion: remove completion for obsolete graphlog extension Differential Revision: https://phab.mercurial-scm.org/D4265

zsh_completion: declare appropriate local parameters for ->string form When ->string form is used for _arguments, the function that calls it must declare appropriate local parameters. Managing local return value is needed to tell the completion system if our function succeeded in suggesting something or not, plus without that `hg diff -<TAB>` doesn't look right. While at it, fix a copypaste error (s/diff_files/revert_files/). Docs: http://zsh.sourceforge.net/Doc/Release/Completion-System.html Differential Revision: https://phab.mercurial-scm.org/D4264

zsh_completion: complete files from `hg st -mar` for commit and diff Useful to have and is consistent with our bash completion file. This is also an improvement coming from upstream, downstream or sidestream (depends on how you look at it) hg completion file -- the one that lives in zsh's own repo. Differential Revision: https://phab.mercurial-scm.org/D4263

zsh_completion: drop -w, add -S to _arguments From the docs: http://zsh.sourceforge.net/Doc/Release/Completion-System.html _arguments -w: "allow option stacking even if one or more of the options take arguments", so something like "-xy xarg yarg". Mercurial's command line parser doesn't support that, so the flag got removed in zsh 92584634d3d3 and bf8b61182043. _arguments -S: "do not complete options after a ‘--’ appearing on the line". By options it means things like "-x" and "--foo", but it will still complete arguments like files. Differential Revision: https://phab.mercurial-scm.org/D4262

zsh_completion: rename _hg_style_opts to _hg_template_opts --style is a deprecated alias for --template, and doesn't show up in help without --verbose, so completing it doesn't make much sense. Let's s/style/template/ this variable to be more consistent with cmdutil.py, which is helpful to look at in parallel next time someone wants to update the completion rules. Differential Revision: https://phab.mercurial-scm.org/D4261

pycompat: wrap xrange for py2 to provide efficient __contains__ The C implementation of xrange in Python 2 provides a O(n) membership test, which is noticable on pull-based clones of large repositories. Avoid this by providing a wrapper class with O(1) membership test based on the edges of the range. Differential Revision: https://phab.mercurial-scm.org/D4313

phases: drop dead code in `newheads` There are new code earlier in the function achieving the same result. Spotted by Yuya Nishihara.

phases: fix `nullid` reference in newheads Spotted by Yuya Nishihara.

beautifygraph: don't warn about busted terminal if HGPLAIN is set This was breaking some automation for users that discovered the extension and turned it on. Differential Revision: https://phab.mercurial-scm.org/D4338

merge with stable

merge: do not delete untracked files silently (issue5962) 37450a122128, 2a774cae3a03, and 656ac240f392 weren't enough to prevent data loss. No unknown "files" weren't deleted before 7a8a16f8ea22, "context: also consider path conflicts when clearing unknown files."

node: correct hex representation of pseudo node ids

merge with stable

perf: fix typo in perfphasesremote()

revlog: fix typo in 'buildtext' name There are no such things as 'deltacomputer._buildtext'. I'm not sure why this code never ran, but we better fix the typo.

revlog: fix pure version of _partialmatch() to include nullid Before this patch, test-issue842.t and a few more tests fail when they try to refer to the null revision by using a "000.." prefix of it (or because they use the "shortest" template function which internally does that). This should have been part of my a3dacabd476b (index: don't allow index[len(index)] to mean nullid, 2018-07-20), but I had forgotten to update another part of the pure code there, so it didn't fail until a1f934573c0b (parsers: adjust pure-python version to mimic a3dacabd476b, 2018-08-09) and 65d5de1169dd (revlog: fix pure nodemap to not access missing index entry, 2018-08-17) fixed the other things I had missed. Differential Revision: https://phab.mercurial-scm.org/D4332

tests: conditionalize color output on abort This broke in afc4ad706f9c. I'm not sure why other aborts don't need to be conditionalized, but `hg help foo` does abort in red for both msys and cmd.exe.

tests: conditionalize out tests that msys is mangling This was exposed by ac0a87160012. The paths for convert and ssh respectively are being turned into: $TESTTMP\bzr+ssh:\foobar@selenic.com\baz\.hg 'C:\\C:\\Users\\Matt\\AppData\\Local\\Temp\\hgtests.8awa9x\\child1\\test-ssh-bundle1.t-sshv2\\nonexistent\\.hg The errno value raised is 123, and the message is The filename, directory name, or volume label syntax is incorrect

test-absorb: stabilize for no-execbit platforms

test-graft: add a missing output line for Windows

share: allow more than one level of directories to be created

clone: allow local cloning to create more than one level of directories I figure cloning a remote repository is more common, thus it's more likely that some people might be relying on the less restrictive behavior. Additionally, `hg init` will also create more than one level of missing directories. I also have a use case for reading the subrepos from .hgsub, and sharing them into the normal nested location on the server. SCM Manager doesn't host subrepos in the normal nested location, which is nice for deduplicating the repository data, but confuses `hg verify`. Some of the subrepos are in the root of the repositories, while others are one or two directories deep. So not having to build up the parent path first is desirable.

tests: demonstrate an inconsistency when cloning to a missing directory tree I noticed that `hg share` is unable to create more than one missing directory on the path, and thought it was inconsistent with clone. It turns out that the path for copying/linking the remote store has the same limitation, but cloning from a remote repo doesn't.

tests: stabilize test-merge-tools.t on Windows More fun with `hg import --bypass` to work around Windows limitations. The diffs were generated on Linux, and had a tab to terminate the `+++b/...` lines. But check-code complained about trailing whitespace, and it seems to run without them.

dagutil: remove module The previous commit removed the last consumer of this module. .. api:: dagutil module has been removed Some functionality has been moved to the dagop module. Other functionality can be accomplished via revsets. Differential Revision: https://phab.mercurial-scm.org/D4330

dagop: port revlogdag.linearize() to standalone function The code should functionally be identical. We also port the one consumer in changegroup to use the new standalone function. After this commit, dagutil is no longer used! Differential Revision: https://phab.mercurial-scm.org/D4329

dagutil: use revlog.parentrevs() for resolving parent revisions And remove parents() since it is no longer used. revlog.parentrevs() is almost the same as parents(). The main difference is that parentrevs() can return nullrev. dagop.headrevs() already handles nullrev. We add an inline check for nullrev in the other call site to account for the difference. .. api:: parents() removed from dagutil classes Use parentrevs() on the storage object instead. Differential Revision: https://phab.mercurial-scm.org/D4328

dagop: extract headsetofconnecteds() from dagutil The functionality for resolving the set of DAG heads from a subset simply requires a function to resolve parent revisions. Let's establish a function in the dagop module to do this, which seems to be where generic DAG functionality goes these days. Differential Revision: https://phab.mercurial-scm.org/D4327

setdiscovery: precompute children revisions to avoid quadratic lookup Moving away from dagutil a few commits ago introduced quadratic behavior when resolving children revisions during discovery. This commit introduces a precompute step of the children revisions to avoid the bad behavior. I believe the new code should have near identical performance to what dagutil was doing before. Behavior is still slightly different because we take into account filtered revisions. But this change was made when we moved off dagutil. I added a comment about multiple invocations of this function redundantly calculating the children revisions. I believe this potentially undesirable behavior was present when we used dagutil, as the call to inverse() previously in this function created a new object and required computing children on every invocation. I thought we should document the potential for a performance issue rather than let it go undocumented. Differential Revision: https://phab.mercurial-scm.org/D4326

dagutil: remove unused classes We only directly use revlogdag in changegroup code. We don't need all this abstraction. So remove various classes and levels of inheritance. Differential Revision: https://phab.mercurial-scm.org/D4325

setdiscovery: use revset for resolving DAG heads in a subset This was the final use of dagutil in setdiscovery! For reasons I didn't investigate, feeding a set with nullrev into the heads() revset resulted in a bunch of tests failing. Filtering out nullrev from the input set fixes things. Differential Revision: https://phab.mercurial-scm.org/D4324

dagutil: remove ability to invert instances The previous commit removed the last consumer of this feature. .. api:: remove inverse() methods from classes in dagutil Differential Revision: https://phab.mercurial-scm.org/D4323

setdiscovery: don't use dagutil for parent resolution _updatesample()'s one remaining use of revlogdag is for resolving the parents of a revision. In 2 cases, we actually resolve parents. In 1, we operate on the inverted DAG and resolve children. This commit teaches _updatesample() to receive an argument defining the function to resolve "parent" revisions. Call sites pass in changelog.parentrevs() or a wrapper around changelog.children() accordingly. The use of children() is semantically correct. But it is quadratic, since revlog.children() does a range scan over all revisions starting at its input and effectively calls parentrevs() to build up the list of children. So calling it repeatedly in a loop is a recipe for bad performance. I will be implementing something better in a subsequent commit. I wanted to get the porting off of dagutil done in a way that was simple and correct. Like other patches in this series, this change is potentially impacted but revlogdag's ignorance of filtered revisions. The new code is filtering aware, since changelog's revs() (used by children() will skip filtered revisions and therefore hidden children won't appear. This is potentially backwards incompatible. But no tests fail and I think this code should respect visibility. Differential Revision: https://phab.mercurial-scm.org/D4322

setdiscovery: use revsets for computing a subset's heads and roots revlogdag.headsetofconnecteds() obtains the set of DAG heads in a given set of revs. revlogdag.inverse() inverts the DAG order and makes headsetofconnecteds() obtain the DAG roots in a given subset. Both of these can be expressed with a revset. Like other patches in this series, revlogdag uses revlog.index and thus doesn't take filtering into account. Revsets do. So there is a chance for regressions with this change. But no tests fail. And I think this code should take filtering into account since hidden changesets shouldn't factor into discovery (unless operating on the hidden repository). Differential Revision: https://phab.mercurial-scm.org/D4321

dagutil: remove heads() and localsubset from revlogdag.__init__ The previous commit removed the last consumer of this API. I'm not going to mark as API incompatible because I doubt anybody used this functionality (outside of possibly passing an argument to revlogdag.__init__). I intend to remove revlogdag later in this series and its API annotation will cover this one. Differential Revision: https://phab.mercurial-scm.org/D4320

setdiscovery: pass head revisions into sample functions This eliminates the last remaining consumer of heads() and related functionality in dagutil. Differential Revision: https://phab.mercurial-scm.org/D4319

setdiscovery: pass heads into _updatesample() In preparation for eliminating the use of dagutil. Since _takefullsample() operates on the inverted DAG, it is easier to have the caller pass in the relevant set instead of teaching _updatesample() about when to invert the DAG. We keep the logic identical for now: future commits will remove dagutil. Differential Revision: https://phab.mercurial-scm.org/D4318

setdiscovery: use a revset for finding DAG heads in a subset The march towards moving away from dagutil continues. Like other patches moving us away from dagutil, there is the potential for regressions to occur because revlogdag's headsetofconnecteds() uses revlog.index, which doesn't take filtering into account. The revset layer does. But no tests fail, so we appear to be in the clear. Differential Revision: https://phab.mercurial-scm.org/D4317

setdiscovery: reflect use of revs instead of nodes This code all operates on revision numbers. Update variable names and comments accordingly. Differential Revision: https://phab.mercurial-scm.org/D4316

dagutil: remove descendantset() and ancestorset() descendantset() is unused after the previous commit. And ancestorset() was only used by descendantset(), so it can be removed as well. .. api:: descendantset() and ancestorset() removed from dagutil Use a revset instead when operating on the changelog. Or use various functionality in the ancestor or dagop modules. Differential Revision: https://phab.mercurial-scm.org/D4315

setdiscovery: use a revset instead of dagutil.descendantset() This is the only use of descendantset() in the repo. Strictly speaking, the revset behaves slightly differently than dagutil. The reason is that dagutil is using revlog.index for DAG traversal and this data structure isn't aware of visibility / filtering. So it can operate on revisions it shouldn't operate on. But our test coverage of this code is pretty comprehensive and this change causes no tests to fail. So I think we are good. Also, the revset parser failed to parse `%ld:: - %ld::`, hence the use of descendants(). I'm not sure if that is a feature or a bug. Differential Revision: https://phab.mercurial-scm.org/D4314

setdiscovery: don't use dagutil to compute heads This is a relatively trivial operation to perform. We don't need to use dagutil. This brings us one step closer to eliminating dagutil. We still need to populate the heads on revlogdag because later functionality relies on it. Differential Revision: https://phab.mercurial-scm.org/D4310

dagutil: remove nodeset() AFAICT this was unused since the code was introduced by cb98fed52495 in 2011! Differential Revision: https://phab.mercurial-scm.org/D4309

debugcommands: use a revset instead of dagutil All this code was doing was finding the nodes that are heads from the ancestors of an input set. This can easily be expressed with a revset without having to go through dagutil. This was the last use of ancestorset() outside of dagutil itself. Differential Revision: https://phab.mercurial-scm.org/D4308

dagutil: remove internalize() and internalizeall() The previous commit removed the last consumer. .. api:: removed internalize() and internalizeall() from dagutil Use .rev(node) on storage objects to convert nodes to revisions. Differential Revision: https://phab.mercurial-scm.org/D4307

setdiscovery: don't use dagutil for node -> rev conversion The node -> rev conversion is possible using standard storage APIs and doesn't need to involve the dagutil module. Differential Revision: https://phab.mercurial-scm.org/D4306

dagutil: remove externalize() and externalizeall() They are unused after the previous commit. .. api:: externalize() and externalizeall() removed from dagutil Use .node() on a storage primitive to perform revision to node conversions. Differential Revision: https://phab.mercurial-scm.org/D4305

setdiscovery: don't use dagutil for rev -> node conversions We don't need to use dagutil to perform a simple rev -> node conversion. I haven't measured, but the new code is likely faster, as we avoid extra function calls and avoid some attribute lookups. Differential Revision: https://phab.mercurial-scm.org/D4304

exchange: don't use dagutil We were only using it for simple node -> rev and parent revision lookups. These are exposed via the storage interface and we don't need to go through dagutil. Differential Revision: https://phab.mercurial-scm.org/D4303

revlog: only consider the span of the delta section Since the number of snapshots is limited we can exclude them from the logic checking size and number of reads. Limiting the span computation to the delta section will allow for further optimization.

revlog: ensure intermediate snapshot have decreasing size If the intermediate snapshot is bigger than the previous one, there is likely a better snapshot to be made at a different level.

revlog: bound number of snapshots in a chain To limit the number of snapshot chained, we enforce them to be smaller and smaller. This guarantee the number of snapshot in a chain will be bounded to a small number.

revlog: compute snapshot depth on delta info We need the information to be available when choosing delta.

debugrevlog: display snapshot details per depth This help in understanding the final structure of build manifest. All data about snapshot (full and intermediate) are gathered into a sub-list for clarity. Since we do not produce such snapshots yet, the only thing changing in test output is the way the information is presented.

revlog: add a method to retrieve snapshot depth Some snapshot property (eg: maximum size) will depend on their depth.

debugrevlog: include information about intermediate snapshots As we are about to create intermediate snapshots, we need to have a way to debug them. We start by adding very simple debug output and more detailed output will comes in next changesets.

revlog: also detect intermediate snapshots Also detect intermediate-snapshot done against another previous snapshot. Doing an intermediate snapshot instead of a full one can reduce the number of full snapshots we need. They are especially useful for content with a lot of churn on the same line (eg: the manifest) where having a delta over multiple revisions can end up being significantly smaller than the sum of these revision deltas. A revlog built using intermediate snapshots can be a bit smaller and reuse snapshot much more efficiently. This last property is useful combined with constraints on chain length. Using intermediate snapshot can produce repository with delta chain ten times shorter without impact on the storage size. Shorter chain lengths are faster to restore, greatly improving read performance. This changesets (and the following ones) focus on getting the core principle of intermediate snapshots into Mercurial core. Later changeset will introduce the strategy to create them.

revlog: add a method to tells whether rev is stored as a snapshot For now we only have one type of snapshot: full snapshot versus nullrev. However we are looking into adding intermediate snapshot where a large diff against another snapshot is performed instead of storing a full new text. The conditional is a bit strange and is done in order to help readability of a some later changesets.

debugrevlog: fix for non-manifest object The `filelog` object is no longer an actual revlog. Instead, the actual revlog is stored in the `_revlog` attribute.

merge with stable

remotephase: avoid full changelog iteration (issue5964) Changeset 88efb7d6bcb6 introduced a performance regression by triggering a full ancestors walk. This changeset reworks this logic so that we no longer walk down the full changelog. The motivation for 88efb7d6bcb6, issue5939, is still fixed. mercurial compared to a draft repository ---------------------------------------- 8eeed92475d5: 0.012637 seconds 88efb7d6bcb6: 0.202699 seconds (x16) 46da52f4b820: 0.215551 seconds (+6%) this code: 0.008397 seconds (-33% from base) The payload size reduction we see in `test-bookmarks-pushpull.t` comes from a more aggressive filter of nullid and is harmless.

nodes: expand/comment the magic nodes so they are more easily searchable We just encountered `000000000000modified`, and it was quite annoying to search for these, even though I knew they existed. For those that don't know that they exist, this is essentially impossible to search for :) (Technically we encountered it in its hex form, 3030303030303030303030306d6f646966696564, so I'm adding comments with those forms in case that's helpful to people in the future). Differential Revision: https://phab.mercurial-scm.org/D4331

revlog: obtain the first node at the lowest layer while building pure nodemap Just for clarity. This doesn't matter in practice since changelog.nodemap is accessed *before* filtered revisions get ready.

revlog: fix pure nodemap to not access missing index entry This bug was revealed by a3dacabd476b and a1f934573c0b.

changelog: remove copy of revlog.nodemap() It's been there since 2012, "clfilter: introduce `filteredrevs` attribute on changelog." I don't think we can apply changelog filtering to nodemap at this level, so this patch removes the nodemap stub completely.

remotephase: fast path newheads computation in simple case (issue5964) Changeset 88efb7d6bcb6 fixed the logic of `phases.newheads` but greatly regressed its performance (up to many order of magnitude). The first step to fix the regression is to exit early when there is no work to do. If there are no heads to filter or not roots to filter them, we don't have to do any work. This fixes the regression when talking to an all public changeset. The performance is even better than before. pypy, compared to an all public repo ------------------------------------ 8eeed92475d5: 0.005758 seconds 88efb7d6bcb6: 0.602517 seconds (x104) this code: 0.001508 seconds (-74% from base) mercurial compared to an all public repo ---------------------------------------- 8eeed92475d5: 0.000577 seconds 88efb7d6bcb6: 0.185316 seconds (x321) this code: 0.000150 seconds (-74% from base) The performance of newheads, when actual computations are required, is fixed in the next changeset.

perf: add a perfphasesremote command This command measure the time we spend analysing remote phase during push and pull and display some information relevant to this computation. The `test-contrib-perf.t` expected output has to be updated but I do need these module for this perf command.

sparse-revlog: fix delta validity computation When considering the validity of a delta with sparse-revlog, we check the size of the largest read. To do so, we use some regular logic with the extra delta information. Some of this logic was not handling this extra delta properly, confusing it with "nullrev". This confusion with nullrev lead to wrong results for this computation but preventing a crash. Changeset 781b2720d2ac on default revealed this error, crashing. This changeset fixes the logic on stable so that the computation is correct (and the crash is averted). The fix is made on stable as this will impact 4.7 clients interacting with sparse-revlog repositories (eg: created by later version).

branchmap: explicitly convert file into iterator Follows up 2a4bfbb52111. This is required for httprangereader, which is not an iterable itself.