obsolete: explicitly track folds inside the markers We now record information to be able to recognize "fold" event from obsolescence markers. To do so, we track the following pieces of information: a) a fold ID. Unique to that fold (per successor), b) the number of predecessors, c) the index of the predecessor in that fold. We will now be able to create an algorithm able to find "predecessorssets". We now store this data in the generic "metadata" field of the markers. Updating the format to have a more compact storage for this would be useful. This way of tracking a fold through multiple markers could be applied to split too. This would have two advantages: 1) We get a simpler format, since number of successors is limited to [0-1]. 2) We can better deal with situations where only some of the split successors are pushed to a remote repository. We should look into the relevance of such a change before updating the on-disk format. note: unlike splits, folds do not have to deal with cases where only some of the markers have been synchronized. As they all share the same successor changesets, they are all relevant to the same nodes.

cleanupnodes: update comment to drop mention of filtering Since changeset 1857f50a9643 drop the filtering, we should not longer mention it in code comment.

treemanifests: remove _loadalllazy when doing copies 'before' here is https://phab.mercurial-scm.org/D4845 (not the committed/rebased version) diff --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.329 s +- 0.011 s | 1.320 s +- 0.010 s | 99.3% m-u | | x | 1.316 s +- 0.005 s | 1.334 s +- 0.018 s | 101.4% m-u | x | | 1.330 s +- 0.021 s | 1.322 s +- 0.005 s | 99.4% m-u | x | x | 87.2 ms +- 0.7 ms | 86.9 ms +- 1.5 ms | 99.7% l-d-r | | | 203.3 ms +- 7.8 ms | 199.4 ms +- 1.8 ms | 98.1% l-d-r | | x | 204.6 ms +- 2.8 ms | 201.7 ms +- 2.1 ms | 98.6% l-d-r | x | | 90.5 ms +- 11.0 ms | 86.2 ms +- 1.0 ms | 95.2% l-d-r | x | x | 66.3 ms +- 2.0 ms | 66.4 ms +- 0.9 ms | 100.2% diff -c . --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 239.4 ms +- 2.0 ms | 241.7 ms +- 4.6 ms | 101.0% m-u | | x | 128.9 ms +- 1.9 ms | 130.9 ms +- 7.7 ms | 101.6% m-u | x | | 241.1 ms +- 1.6 ms | 240.1 ms +- 1.4 ms | 99.6% m-u | x | x | 133.4 ms +- 1.5 ms | 133.4 ms +- 1.2 ms | 100.0% l-d-r | | | 84.3 ms +- 1.5 ms | 83.5 ms +- 1.0 ms | 99.1% l-d-r | | x | 200.9 ms +- 6.3 ms | 203.0 ms +- 4.4 ms | 101.0% l-d-r | x | | 108.1 ms +- 1.4 ms | 108.7 ms +- 2.1 ms | 100.6% l-d-r | x | x | 190.2 ms +- 4.8 ms | 191.6 ms +- 2.0 ms | 100.7% rebase -r . --keep -d .^^: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 5.655 s +- 0.029 s | 5.640 s +- 0.036 s | 99.7% m-u | | x | 5.813 s +- 0.038 s | 5.773 s +- 0.028 s | 99.3% m-u | x | | 5.593 s +- 0.043 s | 5.589 s +- 0.028 s | 99.9% m-u | x | x | 648.2 ms +- 19.2 ms | 637.3 ms +- 27.7 ms | 98.3% l-d-r | | | 673.3 ms +- 8.0 ms | 673.2 ms +- 6.8 ms | 100.0% l-d-r | | x | 6.583 s +- 0.030 s | 5.721 s +- 0.028 s | 86.9% <-- l-d-r | x | | 277.8 ms +- 6.7 ms | 276.0 ms +- 2.7 ms | 99.4% l-d-r | x | x | 1.692 s +- 0.013 s | 720.9 ms +- 13.3 ms | 42.6% <-- status --change . --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 220.9 ms +- 1.6 ms | 219.9 ms +- 2.2 ms | 99.5% m-u | | x | 109.2 ms +- 1.0 ms | 109.4 ms +- 0.8 ms | 100.2% m-u | x | | 222.6 ms +- 1.7 ms | 221.4 ms +- 2.1 ms | 99.5% m-u | x | x | 113.4 ms +- 0.5 ms | 113.1 ms +- 1.1 ms | 99.7% l-d-r | | | 82.1 ms +- 1.7 ms | 82.1 ms +- 1.2 ms | 100.0% l-d-r | | x | 199.8 ms +- 4.0 ms | 200.7 ms +- 3.6 ms | 100.5% l-d-r | x | | 85.4 ms +- 1.5 ms | 85.2 ms +- 0.3 ms | 99.8% l-d-r | x | x | 202.6 ms +- 4.4 ms | 208.0 ms +- 4.0 ms | 102.7% status --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.941 s +- 0.014 s | 1.930 s +- 0.009 s | 99.4% m-u | | x | 1.924 s +- 0.007 s | 1.950 s +- 0.010 s | 101.4% m-u | x | | 1.959 s +- 0.085 s | 1.926 s +- 0.009 s | 98.3% m-u | x | x | 96.2 ms +- 1.0 ms | 96.4 ms +- 0.7 ms | 100.2% l-d-r | | | 604.4 ms +- 10.6 ms | 602.6 ms +- 7.1 ms | 99.7% l-d-r | | x | 605.7 ms +- 4.1 ms | 607.4 ms +- 6.1 ms | 100.3% l-d-r | x | | 182.4 ms +- 1.2 ms | 183.4 ms +- 1.2 ms | 100.5% l-d-r | x | x | 150.8 ms +- 2.0 ms | 150.6 ms +- 1.0 ms | 99.9% update $rev^; ~/src/hg/hg{hg}/hg update $rev: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 3.185 s +- 0.027 s | 3.181 s +- 0.017 s | 99.9% m-u | | x | 3.028 s +- 0.021 s | 2.954 s +- 0.010 s | 97.6% m-u | x | | 3.168 s +- 0.010 s | 3.175 s +- 0.023 s | 100.2% m-u | x | x | 317.5 ms +- 3.5 ms | 313.2 ms +- 2.9 ms | 98.6% l-d-r | | | 456.2 ms +- 10.6 ms | 454.4 ms +- 5.8 ms | 99.6% l-d-r | | x | 9.236 s +- 0.063 s | 757.9 ms +- 9.2 ms | 8.2% <-- l-d-r | x | | 257.6 ms +- 2.3 ms | 261.2 ms +- 1.7 ms | 101.4% l-d-r | x | x | 1.614 s +- 0.013 s | 478.0 ms +- 14.3 ms | 29.6% <-- Differential Revision: https://phab.mercurial-scm.org/D4875

treemanifests: store whether a lazydirs entry needs copied after materializing Due to the way that things like manifestlog.get caches its values, without making a copy (if necessary) after calling readsubtree(), we might end up adjusting the state of the same object on different contexts, breaking things like dirty state tracking (and probably other things). Differential Revision: https://phab.mercurial-scm.org/D4874

treemanifests: extract _loaddifflazy from _diff, use in _filesnotin Differential Revision: https://phab.mercurial-scm.org/D4873

identify: show remote bookmarks in `hg id url -Tjson -B` I didn't display bookmarks when `default and not ui.quiet`: it seems strange for templates to depend on --id or -q, and it would take more code for `hg id url -T {node}` to not request remote bookmarks. An alternative I thought of was providing lazy data to the formatter, `fm.data(bookmarks=lambda: fm.formatlist(getbms(), name='bookmark'))`. The plainformatter would naturally not compute it, the templateformatter would compute only what it needs, and the other ones would compute everything, but that's not supported (or I don't see how), so I abandoned this idea. Differential Revision: https://phab.mercurial-scm.org/D4872

showstack: also handle SIGALRM This is looking *very* handy when debugging mysterious hangs in a test: you can wrap a hanging invocation in `perl -e 'alarm shift @ARGV; exec @ARGV' 1` for example, a hanging `hg pull` becomes `perl -e 'alarm shift @ARGV; exec @ARGV' 1 hg pull` where the `1` is the timeout in seconds before the process will be hit with SIGALRM. After making that edit to the test file, you can then use --extra-config-opt on run-tests.py to globaly enable showstack during the test run, so you'll get full stack traces as you force your hg to exit. I wonder (but only a little, not enough to take action just yet) if we should wire up some scaffolding in run-tests itself to automatically wrap all commands in alarm(3) somehow to avoid hangs in the future? Differential Revision: https://phab.mercurial-scm.org/D4870

exchangev2: add progress bar around manifest scanning This can take a long time on large repositories. Let's add a progress bar so we don't have long periods where it isn't obvious what is going on. Differential Revision: https://phab.mercurial-scm.org/D4859

httppeer: report http statistics Now that keepalive.py records HTTP request count and the number of bytes sent and received as part of performing those requests, we can easily print a report on the activity when closing a peer instance! Exact byte counts are globbed in tests because they are influenced by non-deterministic things, such as hostnames and port numbers. Plus, the exact byte count isn't too important anyway. I feel obliged to note that printing the byte count could have security implications. e.g. if sending a password via HTTP basic auth, the length of that password will influence the byte count and the reporting of the byte count could be a side-channel leak of the password length. I /think/ this is beyond our threshold for concern. But if we think it poses a problem, we can teach the byte count logging code to e.g. ignore sensitive HTTP request headers. We could also consider not reporting the byte count of request headers altogether. But since the wire protocol uses HTTP headers for sending command arguments, it is kind of important to report their size. Differential Revision: https://phab.mercurial-scm.org/D4858

keepalive: track number of bytes received from an HTTP response We also bubble the byte count up to the HTTPConnection instance and its parent opener at read time. Unlike sending, there isn't a clear "end of response" signal we can intercept to defer updating the accounting. Differential Revision: https://phab.mercurial-scm.org/D4857