mercurial/upgrade.py

copies-rust: tokenize all paths into integer

Copy information for each changesets tend to affect a small new number of path.
However, each of these path might be handled a large number of time. Handling
HgPathBuf (aka `Vec<u8>`) is expensive. Handling integer is cheap.

With this patch we:
- turn any input path into an integer "token" early,
- do all the internal logic using such "token",
- turn "token" back into path right before returning a result.

This gives use a quite significant performance boost in our slower cases.

Repo            Case                                  Source-Rev   Dest-Rev      # of revisions  old time      new time      Difference    Factor    time per rev
---------------------------------------------------------------------------------------------------------------------------------------------------------------
pypy            x000_revs_x000_added_x_copies         8a3b5bfd266e 2c68e87c3efe :   6780 revs,   0.092828 s,   0.081225 s,  -0.011603 s, × 0.8750,    11 µs/rev
pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :  43645 revs,   0.711975 s,   0.586011 s,  -0.125964 s, × 0.8231,    13 µs/rev
pypy            x0000_revs_xx000_added_x000_copies    08ea3258278e d9fa043f30c0 :  11316 revs,   0.124505 s,   0.114173 s,  -0.010332 s, × 0.9170,    10 µs/rev
netbeans        x000_revs_x000_added_x000_copies      ff453e9fee32 411350406ec2 :   5750 revs,   0.072072 s,   0.061004 s,  -0.011068 s, × 0.8464,    10 µs/rev
netbeans        x0000_revs_xx000_added_x000_copies    588c2d1ced70 1aad62e59ddd :  66949 revs,   0.682732 s,   0.535874 s,  -0.146858 s, × 0.7849,     8 µs/rev
mozilla-central x00000_revs_x0000_added_x0000_copies  6832ae71433c 4c222a1d9a00 : 153721 revs,   1.935918 s,   1.781383 s,  -0.154535 s, × 0.9202,    11 µs/rev
mozilla-central x00000_revs_x00000_added_x000_copies  76caed42cf7c 1daa622bbe42 : 204976 revs,   2.827320 s,   2.603867 s,  -0.223453 s, × 0.9210,    12 µs/rev
mozilla-try     x0000_revs_xx000_added_x000_copies    89294cd501d9 7ccb2fc7ccb5 :  97052 revs,   3.243010 s,   1.529120 s,  -1.713890 s, × 0.4715,    15 µs/rev
mozilla-try     x00000_revs_x_added_0_copies          6a320851d377 1ebb79acd503 : 363753 revs,   5.693818 s,   4.842699 s,  -0.851119 s, × 0.8505,    13 µs/rev
mozilla-try     x00000_revs_x_added_x_copies          5173c4b6f97c 95d83ee7242d : 362229 revs,   5.677655 s,   4.761732 s,  -0.915923 s, × 0.8387,    13 µs/rev
mozilla-try     x00000_revs_x000_added_x_copies       9126823d0e9c ca82787bb23c : 359344 revs,   5.563370 s,   4.733912 s,  -0.829458 s, × 0.8509,    13 µs/rev
mozilla-try     x00000_revs_x0000_added_x0000_copies  8d3fafa80d4b eb884023b810 : 192665 revs,   2.864099 s,   2.593410 s,  -0.270689 s, × 0.9055,    13 µs/rev
mozilla-try     x00000_revs_x00000_added_x0000_copies 1b661134e2ca 1ae03d022d6d : 228985 revs, 113.297287 s,  41.041198 s, -72.256089 s, × 0.3622,   179 µs/rev
mozilla-try     x00000_revs_x00000_added_x000_copies  9b2a99adc05e 8e29777b48e6 : 382065 revs,  59.498652 s,  27.915689 s, -31.582963 s, × 0.4692,    73 µs/rev





Full timing comparison between this revision and the previous one:


Repo            Case                                  Source-Rev   Dest-Rev      # of revisions  old time      new time      Difference    Factor    time per rev
---------------------------------------------------------------------------------------------------------------------------------------------------------------
mercurial       x_revs_x_added_0_copies               ad6b123de1c7 39cfcef4f463 :      1 revs,   0.000042 s,   0.000042 s,  +0.000000 s, × 1.0000,    42 µs/rev
mercurial       x_revs_x_added_x_copies               2b1c78674230 0c1d10351869 :      6 revs,   0.000104 s,   0.000110 s,  +0.000006 s, × 1.0577,    18 µs/rev
mercurial       x000_revs_x000_added_x_copies         81f8ff2a9bf2 dd3267698d84 :   1032 revs,   0.004913 s,   0.004918 s,  +0.000005 s, × 1.0010,     4 µs/rev
pypy            x_revs_x_added_0_copies               aed021ee8ae8 099ed31b181b :      9 revs,   0.000191 s,   0.000195 s,  +0.000004 s, × 1.0209,    21 µs/rev
pypy            x_revs_x000_added_0_copies            4aa4e1f8e19a 359343b9ac0e :      1 revs,   0.000050 s,   0.000049 s,  -0.000001 s, × 0.9800,    49 µs/rev
pypy            x_revs_x_added_x_copies               ac52eb7bbbb0 72e022663155 :      7 revs,   0.000112 s,   0.000112 s,  +0.000000 s, × 1.0000,    16 µs/rev
pypy            x_revs_x00_added_x_copies             c3b14617fbd7 ace7255d9a26 :      1 revs,   0.000288 s,   0.000324 s,  +0.000036 s, × 1.1250,   324 µs/rev
pypy            x_revs_x000_added_x000_copies         df6f7a526b60 a83dc6a2d56f :      6 revs,   0.010411 s,   0.010611 s,  +0.000200 s, × 1.0192,  1768 µs/rev
pypy            x000_revs_xx00_added_0_copies         89a76aede314 2f22446ff07e :   4785 revs,   0.052852 s,   0.050835 s,  -0.002017 s, × 0.9618,    10 µs/rev
pypy            x000_revs_x000_added_x_copies         8a3b5bfd266e 2c68e87c3efe :   6780 revs,   0.092828 s,   0.081225 s,  -0.011603 s, × 0.8750,    11 µs/rev
pypy            x000_revs_x000_added_x000_copies      89a76aede314 7b3dda341c84 :   5441 revs,   0.063269 s,   0.061291 s,  -0.001978 s, × 0.9687,    11 µs/rev
pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :  43645 revs,   0.711975 s,   0.586011 s,  -0.125964 s, × 0.8231,    13 µs/rev
pypy            x0000_revs_xx000_added_0_copies       bf2c629d0071 4ffed77c095c :      2 revs,   0.012771 s,   0.012824 s,  +0.000053 s, × 1.0042,  6412 µs/rev
pypy            x0000_revs_xx000_added_x000_copies    08ea3258278e d9fa043f30c0 :  11316 revs,   0.124505 s,   0.114173 s,  -0.010332 s, × 0.9170,    10 µs/rev
netbeans        x_revs_x_added_0_copies               fb0955ffcbcd a01e9239f9e7 :      2 revs,   0.000082 s,   0.000085 s,  +0.000003 s, × 1.0366,    42 µs/rev
netbeans        x_revs_x000_added_0_copies            6f360122949f 20eb231cc7d0 :      2 revs,   0.000111 s,   0.000108 s,  -0.000003 s, × 0.9730,    54 µs/rev
netbeans        x_revs_x_added_x_copies               1ada3faf6fb6 5a39d12eecf4 :      3 revs,   0.000171 s,   0.000175 s,  +0.000004 s, × 1.0234,    58 µs/rev
netbeans        x_revs_x00_added_x_copies             35be93ba1e2c 9eec5e90c05f :      9 revs,   0.000708 s,   0.000719 s,  +0.000011 s, × 1.0155,    79 µs/rev
netbeans        x000_revs_xx00_added_0_copies         eac3045b4fdd 51d4ae7f1290 :   1421 revs,   0.010608 s,   0.010175 s,  -0.000433 s, × 0.9592,     7 µs/rev
netbeans        x000_revs_x000_added_x_copies         e2063d266acd 6081d72689dc :   1533 revs,   0.015635 s,   0.015569 s,  -0.000066 s, × 0.9958,    10 µs/rev
netbeans        x000_revs_x000_added_x000_copies      ff453e9fee32 411350406ec2 :   5750 revs,   0.072072 s,   0.061004 s,  -0.011068 s, × 0.8464,    10 µs/rev
netbeans        x0000_revs_xx000_added_x000_copies    588c2d1ced70 1aad62e59ddd :  66949 revs,   0.682732 s,   0.535874 s,  -0.146858 s, × 0.7849,     8 µs/rev
mozilla-central x_revs_x_added_0_copies               3697f962bb7b 7015fcdd43a2 :      2 revs,   0.000090 s,   0.000090 s,  +0.000000 s, × 1.0000,    45 µs/rev
mozilla-central x_revs_x000_added_0_copies            dd390860c6c9 40d0c5bed75d :      8 revs,   0.000210 s,   0.000281 s,  +0.000071 s, × 1.3381,    35 µs/rev
mozilla-central x_revs_x_added_x_copies               8d198483ae3b 14207ffc2b2f :      9 revs,   0.000182 s,   0.000187 s,  +0.000005 s, × 1.0275,    20 µs/rev
mozilla-central x_revs_x00_added_x_copies             98cbc58cc6bc 446a150332c3 :      7 revs,   0.000594 s,   0.000660 s,  +0.000066 s, × 1.1111,    94 µs/rev
mozilla-central x_revs_x000_added_x000_copies         3c684b4b8f68 0a5e72d1b479 :      3 revs,   0.003102 s,   0.003385 s,  +0.000283 s, × 1.0912,  1128 µs/rev
mozilla-central x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :      6 revs,   0.060234 s,   0.069812 s,  +0.009578 s, × 1.1590, 11635 µs/rev
mozilla-central x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   1593 revs,   0.006300 s,   0.006503 s,  +0.000203 s, × 1.0322,     4 µs/rev
mozilla-central x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :     41 revs,   0.004817 s,   0.004988 s,  +0.000171 s, × 1.0355,   121 µs/rev
mozilla-central x000_revs_x000_added_x000_copies      7c97034feb78 4407bd0c6330 :   7839 revs,   0.065451 s,   0.063963 s,  -0.001488 s, × 0.9773,     8 µs/rev
mozilla-central x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :    615 revs,   0.026282 s,   0.026225 s,  -0.000057 s, × 0.9978,    42 µs/rev
mozilla-central x0000_revs_xx000_added_x000_copies    f78c615a656c 96a38b690156 :  30263 revs,   0.206873 s,   0.201377 s,  -0.005496 s, × 0.9734,     6 µs/rev
mozilla-central x00000_revs_x0000_added_x0000_copies  6832ae71433c 4c222a1d9a00 : 153721 revs,   1.935918 s,   1.781383 s,  -0.154535 s, × 0.9202,    11 µs/rev
mozilla-central x00000_revs_x00000_added_x000_copies  76caed42cf7c 1daa622bbe42 : 204976 revs,   2.827320 s,   2.603867 s,  -0.223453 s, × 0.9210,    12 µs/rev
mozilla-try     x_revs_x_added_0_copies               aaf6dde0deb8 9790f499805a :      2 revs,   0.000842 s,   0.000845 s,  +0.000003 s, × 1.0036,   422 µs/rev
mozilla-try     x_revs_x000_added_0_copies            d8d0222927b4 5bb8ce8c7450 :      2 revs,   0.000870 s,   0.000862 s,  -0.000008 s, × 0.9908,   431 µs/rev
mozilla-try     x_revs_x_added_x_copies               092fcca11bdb 936255a0384a :      4 revs,   0.000165 s,   0.000161 s,  -0.000004 s, × 0.9758,    40 µs/rev
mozilla-try     x_revs_x00_added_x_copies             b53d2fadbdb5 017afae788ec :      2 revs,   0.001145 s,   0.001163 s,  +0.000018 s, × 1.0157,   581 µs/rev
mozilla-try     x_revs_x000_added_x000_copies         20408ad61ce5 6f0ee96e21ad :      1 revs,   0.026500 s,   0.032414 s,  +0.005914 s, × 1.2232, 32414 µs/rev
mozilla-try     x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :      6 revs,   0.059407 s,   0.070149 s,  +0.010742 s, × 1.1808, 11691 µs/rev
mozilla-try     x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   1593 revs,   0.006325 s,   0.006526 s,  +0.000201 s, × 1.0318,     4 µs/rev
mozilla-try     x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :     41 revs,   0.005171 s,   0.005187 s,  +0.000016 s, × 1.0031,   126 µs/rev
mozilla-try     x000_revs_x000_added_x000_copies      1346fd0130e4 4c65cbdabc1f :   6657 revs,   0.066837 s,   0.065047 s,  -0.001790 s, × 0.9732,     9 µs/rev
mozilla-try     x0000_revs_x_added_0_copies           63519bfd42ee a36a2a865d92 :  40314 revs,   0.314252 s,   0.301129 s,  -0.013123 s, × 0.9582,     7 µs/rev
mozilla-try     x0000_revs_x_added_x_copies           9fe69ff0762d bcabf2a78927 :  38690 revs,   0.304160 s,   0.280683 s,  -0.023477 s, × 0.9228,     7 µs/rev
mozilla-try     x0000_revs_xx000_added_x_copies       156f6e2674f2 4d0f2c178e66 :   8598 revs,   0.089223 s,   0.084897 s,  -0.004326 s, × 0.9515,     9 µs/rev
mozilla-try     x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :    615 revs,   0.026711 s,   0.026620 s,  -0.000091 s, × 0.9966,    43 µs/rev
mozilla-try     x0000_revs_xx000_added_x000_copies    89294cd501d9 7ccb2fc7ccb5 :  97052 revs,   3.243010 s,   1.529120 s,  -1.713890 s, × 0.4715,    15 µs/rev
mozilla-try     x0000_revs_x0000_added_x0000_copies   e928c65095ed e951f4ad123a :  52031 revs,   0.756500 s,   0.738709 s,  -0.017791 s, × 0.9765,    14 µs/rev
mozilla-try     x00000_revs_x_added_0_copies          6a320851d377 1ebb79acd503 : 363753 revs,   5.693818 s,   4.842699 s,  -0.851119 s, × 0.8505,    13 µs/rev
mozilla-try     x00000_revs_x00000_added_0_copies     dc8a3ca7010e d16fde900c9c :  34414 revs,   0.590904 s,   0.596946 s,  +0.006042 s, × 1.0102,    17 µs/rev
mozilla-try     x00000_revs_x_added_x_copies          5173c4b6f97c 95d83ee7242d : 362229 revs,   5.677655 s,   4.761732 s,  -0.915923 s, × 0.8387,    13 µs/rev
mozilla-try     x00000_revs_x000_added_x_copies       9126823d0e9c ca82787bb23c : 359344 revs,   5.563370 s,   4.733912 s,  -0.829458 s, × 0.8509,    13 µs/rev
mozilla-try     x00000_revs_x0000_added_x0000_copies  8d3fafa80d4b eb884023b810 : 192665 revs,   2.864099 s,   2.593410 s,  -0.270689 s, × 0.9055,    13 µs/rev
mozilla-try     x00000_revs_x00000_added_x0000_copies 1b661134e2ca 1ae03d022d6d : 228985 revs, 113.297287 s,  41.041198 s, -72.256089 s, × 0.3622,   179 µs/rev
mozilla-try     x00000_revs_x00000_added_x000_copies  9b2a99adc05e 8e29777b48e6 : 382065 revs,  59.498652 s,  27.915689 s, -31.582963 s, × 0.4692,    73 µs/rev


Full timing comparison between this revision and the filelog copy tracing.

Repo            Case                                  Source-Rev   Dest-Rev      # of revisions   filelog      sidedata      Difference    Factor    time per rev
---------------------------------------------------------------------------------------------------------------------------------------------------------------
mercurial       x_revs_x_added_0_copies               ad6b123de1c7 39cfcef4f463 :       1 revs,   0.000903 s,  0.000042 s,  -0.000861 s, × 0.0465,    41 µs/rev
mercurial       x_revs_x_added_x_copies               2b1c78674230 0c1d10351869 :       6 revs,   0.001861 s,  0.000110 s,  -0.001751 s, × 0.0591,    18 µs/rev
mercurial       x000_revs_x000_added_x_copies         81f8ff2a9bf2 dd3267698d84 :    1032 revs,   0.018577 s,  0.004918 s,  -0.013659 s, × 0.2647,     4 µs/rev
pypy            x_revs_x_added_0_copies               aed021ee8ae8 099ed31b181b :       9 revs,   0.001519 s,  0.000195 s,  -0.001324 s, × 0.1283,    21 µs/rev
pypy            x_revs_x000_added_0_copies            4aa4e1f8e19a 359343b9ac0e :       1 revs,   0.213855 s,  0.000049 s,  -0.350d73 s, × 0.0002,    48 µs/rev
pypy            x_revs_x_added_x_copies               ac52eb7bbbb0 72e022663155 :       7 revs,   0.017022 s,  0.000112 s,  -0.016910 s, × 0.0065,    15 µs/rev
pypy            x_revs_x00_added_x_copies             c3b14617fbd7 ace7255d9a26 :       1 revs,   0.019398 s,  0.000324 s,  -0.019074 s, × 0.0167,   323 µs/rev
pypy            x_revs_x000_added_x000_copies         df6f7a526b60 a83dc6a2d56f :       6 revs,   0.769467 s,  0.010611 s,  -0.758856 s, × 0.0137,  1768 µs/rev
pypy            x000_revs_xx00_added_0_copies         89a76aede314 2f22446ff07e :    4785 revs,   1.221952 s,  0.050835 s,  -1.171117 s, × 0.0416,    10 µs/rev
pypy            x000_revs_x000_added_x_copies         8a3b5bfd266e 2c68e87c3efe :    6780 revs,   1.304007 s,  0.081225 s,  -1.222782 s, × 0.0622,    11 µs/rev
pypy            x000_revs_x000_added_x000_copies      89a76aede314 7b3dda341c84 :    5441 revs,   1.686610 s,  0.061291 s,  -1.625319 s, × 0.0363,    11 µs/rev
pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :   43645 revs,   0.001107 s,  0.586011 s,  +0.584904 s, × 529.36,    13 µs/rev
pypy            x0000_revs_xx000_added_0_copies       bf2c629d0071 4ffed77c095c :       2 revs,   1.100760 s,  0.012824 s,  -1.087936 s, × 0.0116,  6408 µs/rev
pypy            x0000_revs_xx000_added_x000_copies    08ea3258278e d9fa043f30c0 :   11316 revs,   1.350547 s,  0.114173 s,  -1.236374 s, × 0.0845,    10 µs/rev
netbeans        x_revs_x_added_0_copies               fb0955ffcbcd a01e9239f9e7 :       2 revs,   0.027864 s,  0.000085 s,  -0.027779 s, × 0.0030,    42 µs/rev
netbeans        x_revs_x000_added_0_copies            6f360122949f 20eb231cc7d0 :       2 revs,   0.132479 s,  0.000108 s,  -0.132371 s, × 0.0008,    53 µs/rev
netbeans        x_revs_x_added_x_copies               1ada3faf6fb6 5a39d12eecf4 :       3 revs,   0.025405 s,  0.000175 s,  -0.025230 s, × 0.0068,    58 µs/rev
netbeans        x_revs_x00_added_x_copies             35be93ba1e2c 9eec5e90c05f :       9 revs,   0.053244 s,  0.000719 s,  -0.052525 s, × 0.0135,    79 µs/rev
netbeans        x000_revs_xx00_added_0_copies         eac3045b4fdd 51d4ae7f1290 :    1421 revs,   0.038017 s,  0.010175 s,  -0.027842 s, × 0.2676,     7 µs/rev
netbeans        x000_revs_x000_added_x_copies         e2063d266acd 6081d72689dc :    1533 revs,   0.198308 s,  0.015569 s,  -0.182739 s, × 0.0785,    10 µs/rev
netbeans        x000_revs_x000_added_x000_copies      ff453e9fee32 411350406ec2 :    5750 revs,   0.949749 s,  0.061004 s,  -0.888745 s, × 0.0642,    10 µs/rev
netbeans        x0000_revs_xx000_added_x000_copies    588c2d1ced70 1aad62e59ddd :   66949 revs,   3.932262 s,  0.535874 s,  -3.396388 s, × 0.1362,     8 µs/rev
mozilla-central x_revs_x_added_0_copies               3697f962bb7b 7015fcdd43a2 :       2 revs,   0.024490 s,  0.000090 s,  -0.024400 s, × 0.0036,    44 µs/rev
mozilla-central x_revs_x000_added_0_copies            dd390860c6c9 40d0c5bed75d :       8 revs,   0.143885 s,  0.000281 s,  -0.143604 s, × 0.0019,    35 µs/rev
mozilla-central x_revs_x_added_x_copies               8d198483ae3b 14207ffc2b2f :       9 revs,   0.025471 s,  0.000187 s,  -0.025284 s, × 0.0073,    20 µs/rev
mozilla-central x_revs_x00_added_x_copies             98cbc58cc6bc 446a150332c3 :       7 revs,   0.086013 s,  0.000660 s,  -0.085353 s, × 0.0076,    94 µs/rev
mozilla-central x_revs_x000_added_x000_copies         3c684b4b8f68 0a5e72d1b479 :       3 revs,   0.200726 s,  0.003385 s,  -0.197341 s, × 0.0168,  1127 µs/rev
mozilla-central x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :       6 revs,   2.224171 s,  0.069812 s,  -2.154359 s, × 0.0313, 11633 µs/rev
mozilla-central x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :    1593 revs,   0.090780 s,  0.006503 s,  -0.084277 s, × 0.0716,     4 µs/rev
mozilla-central x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :      41 revs,   0.764805 s,  0.004988 s,  -0.759817 s, × 0.0065,   121 µs/rev
mozilla-central x000_revs_x000_added_x000_copies      7c97034feb78 4407bd0c6330 :    7839 revs,   1.161405 s,  0.063963 s,  -1.097442 s, × 0.0550,     8 µs/rev
mozilla-central x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :     615 revs,   6.816186 s,  0.026225 s,  -6.789961 s, × 0.0038,    42 µs/rev
mozilla-central x0000_revs_xx000_added_x000_copies    f78c615a656c 96a38b690156 :   30263 revs,   3.374819 s,  0.201377 s,  -3.173442 s, × 0.0596,     6 µs/rev
mozilla-central x00000_revs_x0000_added_x0000_copies  6832ae71433c 4c222a1d9a00 :  153721 revs,  16.285469 s,  1.781383 s, -14.504086 s, × 0.1093,    11 µs/rev
mozilla-central x00000_revs_x00000_added_x000_copies  76caed42cf7c 1daa622bbe42 :  204976 revs,  21.207733 s,  2.603867 s, -18.603866 s, × 0.1227,    12 µs/rev
mozilla-try     x_revs_x_added_0_copies               aaf6dde0deb8 9790f499805a :       2 revs,   0.080843 s,  0.000845 s,  -0.079998 s, × 0.0104,   422 µs/rev
mozilla-try     x_revs_x000_added_0_copies            d8d0222927b4 5bb8ce8c7450 :       2 revs,   0.511068 s,  0.000862 s,  -0.510206 s, × 0.0016,   430 µs/rev
mozilla-try     x_revs_x_added_x_copies               092fcca11bdb 936255a0384a :       4 revs,   0.021573 s,  0.000161 s,  -0.021412 s, × 0.0074,    40 µs/rev
mozilla-try     x_revs_x00_added_x_copies             b53d2fadbdb5 017afae788ec :       2 revs,   0.227726 s,  0.001163 s,  -0.226563 s, × 0.0051,   581 µs/rev
mozilla-try     x_revs_x000_added_x000_copies         20408ad61ce5 6f0ee96e21ad :       1 revs,   1.120448 s,  0.032414 s,  -1.088034 s, × 0.0289, 32381 µs/rev
mozilla-try     x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :       6 revs,   2.241713 s,  0.070149 s,  -2.171564 s, × 0.0312, 11689 µs/rev
mozilla-try     x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :    1593 revs,   0.090633 s,  0.006526 s,  -0.084107 s, × 0.0720,     4 µs/rev
mozilla-try     x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :      41 revs,   0.770403 s,  0.005187 s,  -0.765216 s, × 0.0067,   126 µs/rev
mozilla-try     x000_revs_x000_added_x000_copies      1346fd0130e4 4c65cbdabc1f :    6657 revs,   1.184557 s,  0.065047 s,  -1.119510 s, × 0.0549,     9 µs/rev
mozilla-try     x0000_revs_x_added_0_copies           63519bfd42ee a36a2a865d92 :   40314 revs,   0.085790 s,  0.301129 s,  +0.215339 s, × 3.5100,     7 µs/rev
mozilla-try     x0000_revs_x_added_x_copies           9fe69ff0762d bcabf2a78927 :   38690 revs,   0.080616 s,  0.280683 s,  +0.200067 s, × 3.4817,     7 µs/rev
mozilla-try     x0000_revs_xx000_added_x_copies       156f6e2674f2 4d0f2c178e66 :    8598 revs,   7.712554 s,  0.084897 s,  -7.627657 s, × 0.0110,     9 µs/rev
mozilla-try     x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :     615 revs,   6.937294 s,  0.026620 s,  -6.910674 s, × 0.0038,    43 µs/rev
mozilla-try     x0000_revs_xx000_added_x000_copies    89294cd501d9 7ccb2fc7ccb5 :   97052 revs,   7.712313 s,  1.529120 s,  -6.183193 s, × 0.1982,    15 µs/rev
mozilla-try     x0000_revs_x0000_added_x0000_copies   e928c65095ed e951f4ad123a :   52031 revs,   9.966910 s,  0.738709 s,  -9.228201 s, × 0.0741,    14 µs/rev
mozilla-try     x00000_revs_x_added_0_copies          6a320851d377 1ebb79acd503 :  363753 revs,   0.090397 s,  4.842699 s,  +4.752302 s, × 53.571,    13 µs/rev
mozilla-try     x00000_revs_x00000_added_0_copies     dc8a3ca7010e d16fde900c9c :   34414 revs,  27.817167 s,  0.596946 s, -27.220221 s, × 0.0214,    17 µs/rev
mozilla-try     x00000_revs_x_added_x_copies          5173c4b6f97c 95d83ee7242d :  362229 revs,   0.091305 s,  4.761732 s,  +4.670427 s, × 52.151,    13 µs/rev
mozilla-try     x00000_revs_x000_added_x_copies       9126823d0e9c ca82787bb23c :  359344 revs,   0.231183 s,  4.733912 s,  +4.502729 s, × 20.476,    13 µs/rev
mozilla-try     x00000_revs_x0000_added_x0000_copies  8d3fafa80d4b eb884023b810 :  192665 revs,  19.830617 s,  2.593410 s, -17.237207 s, × 0.1307,    13 µs/rev
mozilla-try     x00000_revs_x00000_added_x0000_copies 1b661134e2ca 1ae03d022d6d :  228985 revs,  21.743873 s, 41.041198 s, +19.297325 s, × 1.8874,   179 µs/rev
mozilla-try     x00000_revs_x00000_added_x000_copies  9b2a99adc05e 8e29777b48e6 :  382065 revs,  25.935037 s, 27.915689 s,  +1.980652 s, × 1.0763,    73 µs/rev

Differential Revision: https://phab.mercurial-scm.org/D9493

# upgrade.py - functions for in place upgrade of Mercurial repository
#
# Copyright (c) 2016-present, Gregory Szorc
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

from __future__ import absolute_import

from .i18n import _
from . import (
    error,
    hg,
    localrepo,
    pycompat,
)

from .upgrade_utils import (
    actions as upgrade_actions,
    engine as upgrade_engine,
)

allformatvariant = upgrade_actions.allformatvariant

# search without '-' to support older form on newer client.
#
# We don't enforce backward compatibility for debug command so this
# might eventually be dropped. However, having to use two different
# forms in script when comparing result is anoying enough to add
# backward compatibility for a while.
legacy_opts_map = {
    b'redeltaparent': b're-delta-parent',
    b'redeltamultibase': b're-delta-multibase',
    b'redeltaall': b're-delta-all',
    b'redeltafulladd': b're-delta-fulladd',
}


def upgraderepo(
    ui,
    repo,
    run=False,
    optimize=None,
    backup=True,
    manifest=None,
    changelog=None,
    filelogs=None,
):
    """Upgrade a repository in place."""
    if optimize is None:
        optimize = []
    optimize = {legacy_opts_map.get(o, o) for o in optimize}
    repo = repo.unfiltered()

    revlogs = set(upgrade_engine.UPGRADE_ALL_REVLOGS)
    specentries = (
        (upgrade_engine.UPGRADE_CHANGELOG, changelog),
        (upgrade_engine.UPGRADE_MANIFEST, manifest),
        (upgrade_engine.UPGRADE_FILELOGS, filelogs),
    )
    specified = [(y, x) for (y, x) in specentries if x is not None]
    if specified:
        # we have some limitation on revlogs to be recloned
        if any(x for y, x in specified):
            revlogs = set()
            for upgrade, enabled in specified:
                if enabled:
                    revlogs.add(upgrade)
        else:
            # none are enabled
            for upgrade, __ in specified:
                revlogs.discard(upgrade)

    # Ensure the repository can be upgraded.
    upgrade_actions.check_source_requirements(repo)

    default_options = localrepo.defaultcreateopts(repo.ui)
    newreqs = localrepo.newreporequirements(repo.ui, default_options)
    newreqs.update(upgrade_actions.preservedrequirements(repo))

    upgrade_actions.check_requirements_changes(repo, newreqs)

    # Find and validate all improvements that can be made.
    alloptimizations = upgrade_actions.findoptimizations(repo)

    # Apply and Validate arguments.
    optimizations = []
    for o in alloptimizations:
        if o.name in optimize:
            optimizations.append(o)
            optimize.discard(o.name)

    if optimize:  # anything left is unknown
        raise error.Abort(
            _(b'unknown optimization action requested: %s')
            % b', '.join(sorted(optimize)),
            hint=_(b'run without arguments to see valid optimizations'),
        )

    deficiencies = upgrade_actions.finddeficiencies(repo)
    actions = upgrade_actions.determineactions(
        repo, deficiencies, repo.requirements, newreqs
    )
    actions.extend(
        o
        for o in sorted(optimizations)
        # determineactions could have added optimisation
        if o not in actions
    )

    removedreqs = repo.requirements - newreqs
    addedreqs = newreqs - repo.requirements

    if revlogs != upgrade_engine.UPGRADE_ALL_REVLOGS:
        incompatible = upgrade_actions.RECLONES_REQUIREMENTS & (
            removedreqs | addedreqs
        )
        if incompatible:
            msg = _(
                b'ignoring revlogs selection flags, format requirements '
                b'change: %s\n'
            )
            ui.warn(msg % b', '.join(sorted(incompatible)))
            revlogs = upgrade_engine.UPGRADE_ALL_REVLOGS

    def write_labeled(l, label):
        first = True
        for r in sorted(l):
            if not first:
                ui.write(b', ')
            ui.write(r, label=label)
            first = False

    def printrequirements():
        ui.write(_(b'requirements\n'))
        ui.write(_(b'   preserved: '))
        write_labeled(
            newreqs & repo.requirements, "upgrade-repo.requirement.preserved"
        )
        ui.write((b'\n'))
        removed = repo.requirements - newreqs
        if repo.requirements - newreqs:
            ui.write(_(b'   removed: '))
            write_labeled(removed, "upgrade-repo.requirement.removed")
            ui.write((b'\n'))
        added = newreqs - repo.requirements
        if added:
            ui.write(_(b'   added: '))
            write_labeled(added, "upgrade-repo.requirement.added")
            ui.write((b'\n'))
        ui.write(b'\n')

    def printoptimisations():
        optimisations = [
            a for a in actions if a.type == upgrade_actions.OPTIMISATION
        ]
        optimisations.sort(key=lambda a: a.name)
        if optimisations:
            ui.write(_(b'optimisations: '))
            write_labeled(
                [a.name for a in optimisations],
                "upgrade-repo.optimisation.performed",
            )
            ui.write(b'\n\n')

    def printupgradeactions():
        for a in actions:
            ui.status(b'%s\n   %s\n\n' % (a.name, a.upgrademessage))

    def print_affected_revlogs():
        if not revlogs:
            ui.write((b'no revlogs to process\n'))
        else:
            ui.write((b'processed revlogs:\n'))
            for r in sorted(revlogs):
                ui.write((b'  - %s\n' % r))
        ui.write((b'\n'))

    upgrade_op = upgrade_actions.UpgradeOperation(
        newreqs,
        [a.name for a in actions],
        revlogs,
    )

    if not run:
        fromconfig = []
        onlydefault = []

        for d in deficiencies:
            if d.fromconfig(repo):
                fromconfig.append(d)
            elif d.default:
                onlydefault.append(d)

        if fromconfig or onlydefault:

            if fromconfig:
                ui.status(
                    _(
                        b'repository lacks features recommended by '
                        b'current config options:\n\n'
                    )
                )
                for i in fromconfig:
                    ui.status(b'%s\n   %s\n\n' % (i.name, i.description))

            if onlydefault:
                ui.status(
                    _(
                        b'repository lacks features used by the default '
                        b'config options:\n\n'
                    )
                )
                for i in onlydefault:
                    ui.status(b'%s\n   %s\n\n' % (i.name, i.description))

            ui.status(b'\n')
        else:
            ui.status(
                _(
                    b'(no feature deficiencies found in existing '
                    b'repository)\n'
                )
            )

        ui.status(
            _(
                b'performing an upgrade with "--run" will make the following '
                b'changes:\n\n'
            )
        )

        printrequirements()
        printoptimisations()
        printupgradeactions()
        print_affected_revlogs()

        unusedoptimize = [i for i in alloptimizations if i not in actions]

        if unusedoptimize:
            ui.status(
                _(
                    b'additional optimizations are available by specifying '
                    b'"--optimize <name>":\n\n'
                )
            )
            for i in unusedoptimize:
                ui.status(_(b'%s\n   %s\n\n') % (i.name, i.description))
        return

    # Else we're in the run=true case.
    ui.write(_(b'upgrade will perform the following actions:\n\n'))
    printrequirements()
    printoptimisations()
    printupgradeactions()
    print_affected_revlogs()

    ui.status(_(b'beginning upgrade...\n'))
    with repo.wlock(), repo.lock():
        ui.status(_(b'repository locked and read-only\n'))
        # Our strategy for upgrading the repository is to create a new,
        # temporary repository, write data to it, then do a swap of the
        # data. There are less heavyweight ways to do this, but it is easier
        # to create a new repo object than to instantiate all the components
        # (like the store) separately.
        tmppath = pycompat.mkdtemp(prefix=b'upgrade.', dir=repo.path)
        backuppath = None
        try:
            ui.status(
                _(
                    b'creating temporary repository to stage migrated '
                    b'data: %s\n'
                )
                % tmppath
            )

            # clone ui without using ui.copy because repo.ui is protected
            repoui = repo.ui.__class__(repo.ui)
            dstrepo = hg.repository(repoui, path=tmppath, create=True)

            with dstrepo.wlock(), dstrepo.lock():
                backuppath = upgrade_engine.upgrade(
                    ui, repo, dstrepo, upgrade_op
                )
            if not (backup or backuppath is None):
                ui.status(
                    _(b'removing old repository content%s\n') % backuppath
                )
                repo.vfs.rmtree(backuppath, forcibly=True)
                backuppath = None

        finally:
            ui.status(_(b'removing temporary repository %s\n') % tmppath)
            repo.vfs.rmtree(tmppath, forcibly=True)

            if backuppath and not ui.quiet:
                ui.warn(
                    _(b'copy of old repository backed up at %s\n') % backuppath
                )
                ui.warn(
                    _(
                        b'the old repository will not be deleted; remove '
                        b'it to free up disk space once the upgraded '
                        b'repository is verified\n'
                    )
                )

            if upgrade_actions.sharesafe.name in addedreqs:
                ui.warn(
                    _(
                        b'repository upgraded to share safe mode, existing'
                        b' shares will still work in old non-safe mode. '
                        b'Re-share existing shares to use them in safe mode'
                        b' New shares will be created in safe mode.\n'
                    )
                )
            if upgrade_actions.sharesafe.name in removedreqs:
                ui.warn(
                    _(
                        b'repository downgraded to not use share safe mode, '
                        b'existing shares will not work and needs to'
                        b' be reshared.\n'
                    )
                )