revbranchcache: add the necessary bit to send 'rbc' data over bundle2 Getbundle is now capable of sending rev-branch-cache information for the changesets it bundle. The data sent are mostly nodes so it is quite compact. The goal of the rev-branch-cache is to speed up branch map computation, especially when the branchmap gets invalidated so we send data for all exchanged changesets. In addition, computing the relevant heads to send in case of partial pulling would be challenging. The feature is still inactive since the capability is not advertised yet.

bundle: include advisory rev branch cache part in bundle2 bundle `hg bundle` command producing bundle2 will now include an optional part containing the revision-branch cache data. The data sent are mostly nodes so it is quite compact. The goal of the rev-branch-cache is to speed up branch map computation, especially when the branchmap gets invalidated so we send data for all exchanged changesets. In addition, computing the relevant heads to send in case of partial pulling would be challenging. As a reminder, the rev branch cache data significantly speed up branch computation. Having it around provides a small speedup to pull/clone and much higher tolerance to branch map cache invalidation that might happens from later commands. On the Mercurial repository, computing the visible branchmap from scratch move from 2.00 seconds to 0.34s (a -83% speedup). Using this new part, Unbundling the full Mercurial repository moves from 25.736 seconds to 24.030 seconds (around -7% speedup). The bundle size increase is around 3% (from 22.43 MB to 23.13MB) On an half a million revision repository with twenty thousand branches, computing the branchmap moves from 75 seconds to 45 second (-40%) if the caches is used. A bundle containing 50 000 changesets in such repository get a 0.5% size increase from such part for a -3% unbundling time speedup.

rev-branch-cache: add a function to generate a part The function is able to produce a rbc part consumed by the function introduced into previous changesets. More details on usage and impact in the next changesets.

revbranchcache: add a bundle2 handler for a rbc part We add the necessary bit to process a part containing rev-branch-cache information. The local rev branch cache is then updated with the received information. Computing branch cache can become a significant part of the time spent pulling.

revbranchcache: add a public function to update the data We want to exchange more cached data over the wire. To do so, we need a clean way to update the cache on the receiving ends.

httppeer: consolidate _requestbuilder assignments and document I collapsed multiple assignments because they don't appear to be necessary. We don't invoke the requestbuilder in anything called during __init__. So there's no reason to not define it earlier AFAICT. This seemingly useless attribute is actually an extension point. Document it as such. (A previous version of this patch removed the attribute because it appeared to just be an alias.) Differential Revision: https://phab.mercurial-scm.org/D2723

commands: don't check for merge.update() truthiness AFAICT ``stats`` is always a tuple in these cases. We don't need to check if the variable has a truthy value. Differential Revision: https://phab.mercurial-scm.org/D2691

httppeer: alias url as urlmod "url" is a common variable name. We do this aliasing elsewhere to avoid shadowing. Differential Revision: https://phab.mercurial-scm.org/D2724

util: prefer "bytesio" to "stringio" The io.BytesIO and io.StringIO types enforce the type of data being operated on. On Python 2, we use cStringIO.StringIO(), which is lax about mixing types. On Python 3, we actually use io.BytesIO. Ideally, we'd use io.BytesIO on Python 2. But I believe its performance is poor compared to cString.StringIO(). Anyway, we canonically define our pycompat type as "stringio." That name is misleading, especially on Python 3. This commit renames the canonical symbols to "bytesio." "stringio" is preserved as an alias for API compatibility. There are a lot of callers in the repo and I hesitate to take away the old name. I also don't feel like changing everything at this time. But at least new callers can use a "proper" name. Differential Revision: https://phab.mercurial-scm.org/D2868

contrib: fix a subtle bug in check-code's regex rewriting We rewrite `\s` to `[ \t]` when preparing our regular expressions, but we previously weren't working to avoid having nested sets. Previously, Python let this slide without incident, but in Python 3.7 wants to make sure you meant an actual [ in a set, and so this warns. This appears to be fortunate for us, because `[\s(]` was getting rewritten to be `[[ \t](]` which doesn't actually match what we expected. See preceding changes that were revealed to be necessary after implementing this fix. Differential Revision: https://phab.mercurial-scm.org/D2866

remotenames: work around move of ABCs in collections This starts warning in Python 3.7, and will break in 3.8. The import is performed in a curious place because `import collections.abc` explodes badly on Python 2.7, and `from collections import abc` flunks our import checker. Differential Revision: https://phab.mercurial-scm.org/D2847

hgweb: explain instabilities of unstable changesets

debug: add debugwhyunstable that explains instabilities This is a port of evolve's feature of listing all unstable changesets in detail (`hg evolve --list`).

obsolete: move marker flags to obsutil Evolve extension expects to have obsolete.bumpedfix (imported in hgext3rd/evolve/evolvecmd.py), so we provide it.

debugupdatecache: also warm rev branch cache We add basic code to have `debugupdatecache` ensure that the rev branch cache is fully warmed. This only affects the `debugupdatecache` command, not normal transaction operation.

debugbundle: do not display detailed part data in --quiet mode This changeset makes `hg debugbundle` respect --quiet and only display data about part headers.

tests: clean up two wayward `python` invocations These are also fallout from the check-code regex rewriting bug. Differential Revision: https://phab.mercurial-scm.org/D2865

contrib: open a hole in the open().read() ban for open().close() It turns out open().close() is both fine and something we occasionally do to verify something can be written. The few cases in the codebase were getting missed due to a regular expression bug (which I discussed in my previous change), but since I'm about to fix the bug, I need to fix the patterns. Differential Revision: https://phab.mercurial-scm.org/D2864

cleanup: fix some latent open(path).read() et al calls we previously missed This pattern was banned by check-code way back in 1b4b82063ce2 (may of 2011), but due to a regular expression rewriting bug in check-code these particular callsites were never detected. Python 3.7 caught the bug, which then exposed these errors. Differential Revision: https://phab.mercurial-scm.org/D2863

uncommit: fix unaligned indentation Differential Revision: https://phab.mercurial-scm.org/D2861

util: clear debugstacktrace call During the renaming of datestr, it seems that I have forget a debugstacktrace in util.py. Remove it. Differential Revision: https://phab.mercurial-scm.org/D2838

peer-request: include more details about batch commands A batch commands encapsulate multiple other commands. We display a bit more details about what is actually batched if peer request tracking is set.

tests: fix test-wireproto.py to work around serverrepo() not having a ui This started failing recently, but it doesn't look important as no actual ui objects have been in play. Differential Revision: https://phab.mercurial-scm.org/D2867

tests: add a cat of `error.log` in subrepo test This made some debugging a lot less painful when something was broken, and it costs us almost nothing, so I figure we may as well leave it in. Differential Revision: https://phab.mercurial-scm.org/D2853

hghave: fix hardlink-whitelisted check on Python 3 Differential Revision: https://phab.mercurial-scm.org/D2846

hghave: fix xdiff check on Python 3 Differential Revision: https://phab.mercurial-scm.org/D2845

commandserver: prefer first-party selectors module from Python 3 to backport Caught by some deprecation warnings on Python 3.7. Differential Revision: https://phab.mercurial-scm.org/D2844

forget: add --dry-run mode

push-discovery: extract the bookmark comparison logic in its own function This will help extensions to alter the behavior as they see fit.

push-discovery: don't turn use generator when comparing bookmarks We want extensions to be able to implement their own logic. Generators can be consume only once, impractical for this purpose.