sparse: add local files to temporaryfiles if they exist out of sparse We get the f1 from args if it's merge and check that whether that exists in sparse checkout or not. If that does not, we add that for merging. The error comes from very low-level where we try to read data of a working-filectx which does not exists in the working directory. It will be extremely ugly to plug in logic to update sparse copy with new file at such a low level. We already have logic related to updating the checkout with required files in calculateupdates() and let's handle this case there only. calculateupdates() call sparse.filterupdatesactions() and the logic is added into the latter function. To get the exact traceback, this patch can be backed out and test-sparse-merges.t can be run with ui.traceback=True. Differential Revision: https://phab.mercurial-scm.org/D4341

tests: show that merging with sparse is broken when rename is involved This patch adds test to show that merging with sparse is broken when you have a rename on one side and just modification on another side. Differential Revision: https://phab.mercurial-scm.org/D4340

narrowspec: fix a typoed 'supported' Spotted by martinvonz. Differential Revision: https://phab.mercurial-scm.org/D4374

narrow: build the known set of nodes only when ellipses is enabled We don't need to build the known set in non-ellipses case because we don't have a shallow repo. In this patch, this checks whether the server has ellipses enabled or not using the server capability and then build the known set of nodes. Building the known set of nodes can take ~3-4 minutes on repositories with millions of csets so this patch speeds up extending a non-shallow narrow clone on large repositories. In future, we should first check whether local repository is an ellipses repo using a new ellipses repo requirement and then control all the combinations between local repo requirement and server capability. Differential Revision: https://phab.mercurial-scm.org/D4520

narrow: add narrow and ellipses as server capabilities Right now we don't have a way to differentiate between whether a server can serve ellipsis or not. The way we check whether a server is narrow enabled is by checking bundle2 capability which does not scale outside of bundle2 world. The goal is to use have wireprotocol commands just like remotefilelog for widening the narrow clone, atleast in non-ellipses cases. Having a server capability will help there as we can't rely on bundle2 capability there. Also having a server capability is neat than having a bundle2 capability. There are lot of things we can optimize locally on the client side by knowing before hand that whether the server supports ellipses or not. This will also help us in making sure that a client ellipses repo does not communicate with a server repo without ellipses. Differential Revision: https://phab.mercurial-scm.org/D4521

treemanifest: use visitchildrenset when doing a walk For this series, starting at 'introduce lazy loading of subdirs' and ending with this commit, we get the following timing numbers, using roughly the same methodology and setup that we did in an earlier commit. "before" is 6268fed3. diff --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.585 s +- 0.034 s | 1.574 s +- 0.045 s | 99.3% m-u | | x | 1.591 s +- 0.024 s | 1.601 s +- 0.034 s | 100.6% m-u | x | | 1.579 s +- 0.032 s | 1.603 s +- 0.029 s | 101.5% m-u | x | x | 109.8 ms +- 2.1 ms | 108.8 ms +- 2.2 ms | 99.1% l-d-r | | | 234.6 ms +- 5.1 ms | 240.1 ms +- 7.9 ms | 102.3% l-d-r | | x | 238.3 ms +- 8.1 ms | 232.2 ms +- 6.3 ms | 97.4% l-d-r | x | | 120.0 ms +- 2.7 ms | 119.6 ms +- 2.1 ms | 99.7% l-d-r | x | x | 120.2 ms +- 1.6 ms | 119.1 ms +- 1.6 ms | 99.1% diff -c . --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 356.9 ms +- 5.6 ms | 355.9 ms +- 5.6 ms | 99.7% m-u | | x | 209.1 ms +- 3.3 ms | 205.8 ms +- 2.6 ms | 98.4% m-u | x | | 418.3 ms +- 14.0 ms | 350.8 ms +- 7.7 ms | 83.9% <-- m-u | x | x | 168.3 ms +- 1.5 ms | 168.1 ms +- 2.7 ms | 99.9% l-d-r | | | 100.5 ms +- 2.1 ms | 99.1 ms +- 1.6 ms | 98.6% l-d-r | | x | 5.672 s +- 0.133 s | 4.335 s +- 0.051 s | 76.4% <-- l-d-r | x | | 102.8 ms +- 2.8 ms | 100.4 ms +- 1.7 ms | 97.7% l-d-r | x | x | 1.025 s +- 0.028 s | 875.9 ms +- 12.1 ms | 85.5% <-- rebase -r . --keep -d .^^: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 6.654 s +- 0.137 s | 6.718 s +- 0.046 s | 101.0% m-u | | x | 6.672 s +- 0.104 s | 6.716 s +- 0.143 s | 100.7% m-u | x | | 6.661 s +- 0.072 s | 6.658 s +- 0.066 s | 100.0% m-u | x | x | 696.0 ms +- 49.0 ms | 696.2 ms +- 54.8 ms | 100.0% l-d-r | | | 789.1 ms +- 12.5 ms | 793.5 ms +- 17.3 ms | 100.6% l-d-r | | x | 7.621 s +- 0.194 s | 7.144 s +- 0.116 s | 93.7% <-- l-d-r | x | | 335.4 ms +- 9.7 ms | 339.6 ms +- 10.8 ms | 101.3% l-d-r | x | x | 6.315 s +- 0.085 s | 5.633 s +- 0.105 s | 89.2% <-- status --change . --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 331.2 ms +- 7.0 ms | 335.1 ms +- 6.5 ms | 101.2% m-u | | x | 184.0 ms +- 3.8 ms | 183.1 ms +- 3.4 ms | 99.5% m-u | x | | 330.5 ms +- 7.2 ms | 332.7 ms +- 6.9 ms | 100.7% m-u | x | x | 148.7 ms +- 3.5 ms | 147.5 ms +- 2.2 ms | 99.2% l-d-r | | | 97.3 ms +- 1.3 ms | 98.2 ms +- 2.5 ms | 100.9% l-d-r | | x | 5.196 s +- 0.030 s | 4.281 s +- 0.025 s | 82.4% <-- l-d-r | x | | 101.4 ms +- 3.1 ms | 100.9 ms +- 1.8 ms | 99.5% l-d-r | x | x | 1.015 s +- 0.018 s | 861.2 ms +- 11.2 ms | 84.8% <-- status --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 2.356 s +- 0.032 s | 2.369 s +- 0.027 s | 100.6% m-u | | x | 2.371 s +- 0.041 s | 2.385 s +- 0.028 s | 100.6% m-u | x | | 2.366 s +- 0.036 s | 2.379 s +- 0.030 s | 100.5% m-u | x | x | 121.8 ms +- 3.8 ms | 121.1 ms +- 2.7 ms | 99.4% l-d-r | | | 726.2 ms +- 12.9 ms | 723.1 ms +- 12.7 ms | 99.6% l-d-r | | x | 738.7 ms +- 11.2 ms | 741.3 ms +- 11.1 ms | 100.4% l-d-r | x | | 215.9 ms +- 3.6 ms | 214.7 ms +- 5.9 ms | 99.4% l-d-r | x | x | 213.1 ms +- 2.5 ms | 212.8 ms +- 4.5 ms | 99.9% update $rev^; ~/src/hg/hg{hg}/hg update $rev: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 3.922 s +- 0.045 s | 3.937 s +- 0.034 s | 100.4% m-u | | x | 3.673 s +- 0.028 s | 3.660 s +- 0.035 s | 99.6% m-u | x | | 3.959 s +- 0.073 s | 3.904 s +- 0.051 s | 98.6% m-u | x | x | 417.0 ms +- 13.0 ms | 406.2 ms +- 6.5 ms | 97.4% l-d-r | | | 534.3 ms +- 12.2 ms | 537.9 ms +- 8.8 ms | 100.7% l-d-r | | x | 10.671 s +- 0.275 s | 10.111 s +- 0.114 s | 94.8% <-- l-d-r | x | | 309.7 ms +- 3.5 ms | 307.9 ms +- 4.0 ms | 99.4% l-d-r | x | x | 1.837 s +- 0.014 s | 1.887 s +- 0.035 s | 102.7% Differential Revision: https://phab.mercurial-scm.org/D4371

treemanifest: use visitchildrenset when filtering a manifest to a matcher Differential Revision: https://phab.mercurial-scm.org/D4370

tests: stabilize test-no-symlink This goes with 89630d0b3e23.

shelve: use the internal phase when possible If the repository support it, use the internal phase for all changesets created by shelve.

treemanifest: avoid loading everything just to get their nodeid Differential Revision: https://phab.mercurial-scm.org/D4369