exchange: extract a function to sort obsolete markers Follows up on changeset 01e8eefd9434, several other occurrences of sorted() need to be fixed. The _sortedmarkers() handles sorting obsmarkers with a None value as last element on Python 3.

py3: encode underlying error message during parse error of %include

convert: convert os.devnull to bytes before trying to join it with other bytes Together with the previous commit relating to emailparser this gets test-convert-tla.t passing under py3. Differential Revision: https://phab.mercurial-scm.org/D7062

rust-dirstate-status: add call to rust-fast path for `dirstate.status` The reasoning behind this patch is explained in the first patch of the series. Differential Revision: https://phab.mercurial-scm.org/D7060

rust-dirstate-status: rust-cpython bindings for `dirstate.status` The ref-sharing mechanism has improved, but its ergonomics still left a bit to be desired, as expected. Differential Revision: https://phab.mercurial-scm.org/D7059

rust-refsharing: add missing lifetime parameter in ref_sharing Differential Revision: https://phab.mercurial-scm.org/D7110

rust-dirstate-status: add first Rust implementation of `dirstate.status` Note: This patch also added the rayon crate as a Cargo dependency. It will help us immensely in making Rust code parallel and easy to maintain. It is a stable, well-known, and supported crate maintained by people on the Rust team. The current `dirstate.status` method has grown over the years through bug reports and new features to the point where it got too big and too complex. This series does not yet improve the logic, but adds a Rust fast-path to speed up certain cases. Tested on mozilla-try-2019-02-18 with zstd compression: - `hg diff` on an empty working copy: - c: 1.64(+-)0.04s - rust+c before this change: 2.84(+-)0.1s - rust+c: 849(+-)40ms - `hg commit` when creating a file: - c: 5.960s - rust+c before this change: 5.828s - rust+c: 4.668s - `hg commit` when updating a file: - c: 4.866s - rust+c before this change: 4.371s - rust+c: 3.855s - `hg status -mard` - c: 1.82(+-)0.04s - rust+c before this change: 2.64(+-)0.1s - rust+c: 896(+-)30ms The numbers are clear: the current Rust `dirstatemap` implementation is super slow, its performance needs to be addressed. This will be done in a future series, immediately after this one, with the goal of getting Rust to be at least to the speed of the Python + C implementation in all cases before the 5.2 freeze. At worse, we gate dirstatemap to only be used in those cases. Cases where the fast-path is not executed: - for commands that need ignore support (`status`, for example) - if subrepos are found (should not be hard to add, but winter is coming) - any other matcher than an `alwaysmatcher`, like patterns, etc. - with extensions like `sparse` and `fsmonitor` The next step after this is to rethink the logic to be closer to Jane Street's Valentin Gatien-Baron's Rust fast-path which does a lot less work when possible. Differential Revision: https://phab.mercurial-scm.org/D7058

share: unmark --relative as EXPERIMENTAL Differential Revision: https://phab.mercurial-scm.org/D7001

rust-cpython: removed now useless py_set() conversion In rust-cpython 0.3.0, HashSets implement the appropriate ToPythonObject, we can therefore get rid of this hacky conversion. There still remains an inefficiency in `MissingAncestors.bases()`: we have to clone, because `to_py_object()` requires full ownership. However: - the only use case outside of unit tests used to be from `setdiscovery.partialdiscovery` which is now fully implemented in Rust. - it's not worse than what `py_set()` used to do Differential Revision: https://phab.mercurial-scm.org/D7120

tests: stabilize test-eol-update on Windows These appeared in the recent eol fix series. I'm not sure about the identity mismatch warning- it seems like a general timing issue, because it happened on the first run but not when run with `--loop`. The buildbot also flagged this. Differential Revision: https://phab.mercurial-scm.org/D7112

tests: opt into python3 when testing virtualenv installation Otherwise it complains that Mercurial requires '~=2.7'. Since the existing linux py3 test doesn't flag this, I'm assuming that virtualenv simply isn't installed on that system. Differential Revision: https://phab.mercurial-scm.org/D7111

grep: put --diff in the first line of usage Differential Revision: https://phab.mercurial-scm.org/D7003

ui: option to preserve the progress bar Some interface like the fast-import format of git are meant for pipe processing. It can be still desirable to have progress bars in this case, but clearing the bar after each output operation is not desirable. Differential Revision: https://phab.mercurial-scm.org/D6843

phabricator: update hgmatcher to cope with the new data format The new conduit format can't be matched by the existing matcher since it shifts all the data into an urlencoded string of JSON, the order of which isn't stable between runs. Instead detect JSON values of params and load them into python dicts, which python will then naturally deep-equal compare. Differential Revision: https://phab.mercurial-scm.org/D7055

phabricator: treat non-utf-8 text files as binary as phabricator requires Phabricator can't cope with text files that are not UTF-8, so requires them to be submitted as binary files instead. This has the unfortunate effect of making them practically unreviewable in Phabricator since it will only display the separate versions of the file in other views, not a diff. `phabread`ing such submissions are similar, since it will just output the binary patch, but `hg import` copes with it fine and `hg diff` afterwards will show the actual changes. It is still a marked improvement over trying to submit them as text, which just leads to corruption (Phabricator will either output ? or HTML entities for non-UTF-8 characters, depending on context). Running decode on the whole file like this seems slightly unfortunate, but I'm not aware of a better way. Needs to be done to p1() version as well to detect conversions to UTF-8. Differential Revision: https://phab.mercurial-scm.org/D7054

phabricator: switch to the creatediff endpoint This lets the extension submit binary files, as well as set branch info so that it is exposed in the Phabricator interface. Differential Revision: https://phab.mercurial-scm.org/D7053

phabricator: change conduit data format to match arcanist The existing form isn't handled well by Phabricator for certain data types. Differential Revision: https://phab.mercurial-scm.org/D7052

phabricator: remove tests and all recordings The next commit is going to change the format of conduit API requests so none of the VCR recordings will match and all the tests will fail. Differential Revision: https://phab.mercurial-scm.org/D7051

phabricator: convert phabhunk and phabchange keys to bytes when finalising I thought I had included this code already but I'd missed it out. One of the disadvantages of attrs is that all the keys are unicode strings on py3, but we need them to be byte strings. Differential Revision: https://phab.mercurial-scm.org/D7066

phabricator: add addadded function This is the most complicated part of the new code, and is responsible for adding all added files, whether brand new, copied, or moved. This also includes creating the phabchanges for the original files that have been moved or copied from, since they might need their awayPaths and type updating if multiple copies are involved. Differential Revision: https://phab.mercurial-scm.org/D7050

sidedatacopies: directly fetch copies information from sidedata When using the sidedata mode, we don't need a complicated and expensive `context` object. Instead we directly fetch copies information from the sidedata (through a changelogrevision object). More optimisations coming. revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29 filelog: ! wall 3.679613 comb 3.680000 user 3.580000 sys 0.100000 (median of 3) base: ! wall 8.884369 comb 8.880000 user 8.850000 sys 0.030000 (median of 3) before: ! wall 4.681985 comb 4.680000 user 4.640000 sys 0.040000 (median of 3) after: ! wall 3.955894 comb 3.950000 user 3.940000 sys 0.010000 (median of 3) revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2 filelog: ! wall 0.003357 comb 0.010000 user 0.010000 sys 0.000000 (median of 781) base: ! wall 12.398524 comb 12.400000 user 12.330000 sys 0.070000 (median of 3) before: ! wall 6.459592 comb 6.470000 user 6.390000 sys 0.080000 (median of 3) after: ! wall 5.505774 comb 5.500000 user 5.410000 sys 0.090000 (median of 3) revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0 filelog: ! wall 2.754687 comb 2.760000 user 2.650000 sys 0.110000 (median of 4) base: ! wall 1.423166 comb 1.420000 user 1.400000 sys 0.020000 (median of 8) before: ! wall 0.961048 comb 0.960000 user 0.940000 sys 0.020000 (median of 11) after: ! wall 0.882950 comb 0.880000 user 0.880000 sys 0.000000 (median of 11) revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f filelog: ! wall 1.552293 comb 1.550000 user 1.510000 sys 0.040000 (median of 6 base: ! wall 0.022662 comb 0.020000 user 0.020000 sys 0.000000 (median of 128) before: ! wall 0.021649 comb 0.020000 user 0.020000 sys 0.000000 (median of 135) after: ! wall 0.020951 comb 0.020000 user 0.020000 sys 0.000000 (median of 141) revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63 filelog: ! wall 1.500983 comb 1.500000 user 1.420000 sys 0.080000 (median of 7) base: ! wall 0.006956 comb 0.010000 user 0.010000 sys 0.000000 (median of 392) before: ! wall 0.004022 comb 0.000000 user 0.000000 sys 0.000000 (median of 735) after: ! wall 0.003988 comb 0.000000 user 0.000000 sys 0.000000 (median of 736) revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e filelog: ! wall 0.011745 comb 0.020000 user 0.020000 sys 0.000000 (median of 250) base: ! wall 0.000156 comb 0.000000 user 0.000000 sys 0.000000 (median of 17180) before: ! wall 0.000118 comb 0.000000 user 0.000000 sys 0.000000 (median of 19170) after: ! wall 0.000097 comb 0.000000 user 0.000000 sys 0.000000 (median of 27276) revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe filelog: ! wall 3.228230 comb 3.230000 user 3.110000 sys 0.120000 (median of 4) base: ! wall 0.997640 comb 1.000000 user 0.980000 sys 0.020000 (median of 10) before: ! wall 0.679500 comb 0.680000 user 0.680000 sys 0.000000 (median of 15) after: ! wall 0.596779 comb 0.600000 user 0.600000 sys 0.000000 (median of 17) revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d filelog: ! wall 1.052501 comb 1.060000 user 1.040000 sys 0.020000 (median of 10 base: ! wall 0.214519 comb 0.220000 user 0.220000 sys 0.000000 (median of 45) before: ! wall 0.149675 comb 0.150000 user 0.150000 sys 0.000000 (median of 66) after: ! wall 0.130786 comb 0.130000 user 0.130000 sys 0.000000 (median of 75) Differential Revision: https://phab.mercurial-scm.org/D7072

copies: use an unfiltered repository for the changeset centric algorithm Since the algorithm work form heads to ancestors, we don't need to check filtering for anything but the two entries. Using an unfiltered version is noticeably more efficient. Some timing on the pypy repo: revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29 before: ! wall 2.717861 comb 2.720000 user 2.700000 sys 0.020000 (median of 10) after: ! wall 2.582204 comb 2.580000 user 2.560000 sys 0.020000 (median of 10) revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2 before: ! wall 4.003146 comb 4.010000 user 3.970000 sys 0.040000 (median of 10) after: ! wall 3.814613 comb 3.810000 user 3.760000 sys 0.050000 (median of 10) revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0 before: ! wall 0.704204 comb 0.700000 user 0.700000 sys 0.000000 (median of 13) after: ! wall 0.657387 comb 0.650000 user 0.640000 sys 0.010000 (best of 14) revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f before: ! wall 0.013493 comb 0.020000 user 0.020000 sys 0.000000 (median of 219) after: ! wall 0.013523 comb 0.020000 user 0.020000 sys 0.000000 (median of 218) revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63 before: ! wall 0.003017 comb 0.000000 user 0.000000 sys 0.000000 (median of 985) after: ! wall 0.002876 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000) revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e before: ! wall 0.000073 comb 0.000000 user 0.000000 sys 0.000000 (median of 12672) after: ! wall 0.000082 comb 0.000000 user 0.000000 sys 0.000000 (median of 11456) revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe before: ! wall 0.478061 comb 0.470000 user 0.470000 sys 0.000000 (median of 19) after: ! wall 0.452420 comb 0.450000 user 0.450000 sys 0.000000 (median of 21) revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d before: ! wall 0.116015 comb 0.110000 user 0.110000 sys 0.000000 (median of 84) after: ! wall 0.109153 comb 0.100000 user 0.100000 sys 0.000000 (median of 90) Differential Revision: https://phab.mercurial-scm.org/D7123

copies: extract data extraction into a `revinfo` function The function is build once at the beginning of the algorithm and used fetch appropriate information for each revision. This abstracts some implementation details from the main algorithm and will help us to access the data more efficiently in future changesets. Differential Revision: https://phab.mercurial-scm.org/D7070

copies: avoid instancing more changectx to access parent revisions We just need to know the revision numbers of the parents, creating full context is needlessly expensive. This provide a small, but noticeable performance boost. revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29 before: ! wall 2.885636 comb 2.900000 user 2.870000 sys 0.030000 (median of 10) after: ! wall 2.702270 comb 2.710000 user 2.690000 sys 0.020000 (median of 10) revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2 before: ! wall 4.298271 comb 4.290000 user 4.240000 sys 0.050000 (median of 10) after: ! wall 3.976610 comb 3.970000 user 3.920000 sys 0.050000 (median of 10) revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0 before: ! wall 0.773397 comb 0.770000 user 0.770000 sys 0.000000 (median of 11) after: ! wall 0.701634 comb 0.700000 user 0.700000 sys 0.000000 (median of 13) revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f before: ! wall 0.013585 comb 0.010000 user 0.010000 sys 0.000000 (median of 217) after: ! wall 0.013550 comb 0.010000 user 0.010000 sys 0.000000 (median of 218) revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63 before: ! wall 0.003202 comb 0.000000 user 0.000000 sys 0.000000 (median of 929) after: ! wall 0.002993 comb 0.010000 user 0.010000 sys 0.000000 (median of 992) revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e before: ! wall 0.000077 comb 0.000000 user 0.000000 sys 0.000000 (median of 12060) after: ! wall 0.000072 comb 0.000000 user 0.000000 sys 0.000000 (median of 12804) revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe before: ! wall 0.510614 comb 0.500000 user 0.500000 sys 0.000000 (median of 18) after: ! wall 0.473681 comb 0.470000 user 0.470000 sys 0.000000 (median of 20) revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d before: ! wall 0.126552 comb 0.130000 user 0.130000 sys 0.000000 (median of 77) after: ! wall 0.115240 comb 0.110000 user 0.110000 sys 0.000000 (median of 85) Differential Revision: https://phab.mercurial-scm.org/D7122

copies: get copies information directly from _copies Performance measurement does not show any significant performance movement. This is not surprising since p1copies() code is `self._copies[0]`. Differential Revision: https://phab.mercurial-scm.org/D7121

copies: simplify the handling of merges Instead of stacking copies for both parent on the head, we move copies outside of the heap into a dedicated dictionary. The two side of merge can we merged sooner, making the algorithm simpler. This simplicity reflect in the heap structure and speed up the execution for copies involving a large amount of merges. Here are timing for perfpathcopies of multiple revision pairs. - filelog: timing using filelog (with the introrev condition dropped) - base: this series base - before: the parent of this changeset - after: this changeset revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29 filelog: ! wall 3.679613 comb 3.680000 user 3.580000 sys 0.100000 (median of 3) base: ! wall 8.884369 comb 8.880000 user 8.850000 sys 0.030000 (median of 3) before: ! wall 8.443747 comb 8.420000 user 8.410000 sys 0.010000 (median of 3) after: ! wall 4.697917 comb 4.690000 user 4.660000 sys 0.030000 (median of 3) revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2 filelog: ! wall 0.003357 comb 0.010000 user 0.010000 sys 0.000000 (median of 781) base: ! wall 12.398524 comb 12.400000 user 12.330000 sys 0.070000 (median of 3) before: ! wall 10.852593 comb 10.850000 user 10.800000 sys 0.050000 (median of 3) after: ! wall 6.750832 comb 6.750000 user 6.640000 sys 0.110000 (median of 3) revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0 filelog: ! wall 2.754687 comb 2.760000 user 2.650000 sys 0.110000 (median of 4) base: ! wall 1.423166 comb 1.420000 user 1.400000 sys 0.020000 (median of 8) before: ! wall 1.068041 comb 1.060000 user 1.050000 sys 0.010000 (median of 10) after: ! wall 1.045916 comb 1.050000 user 1.040000 sys 0.010000 (median of 10) revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f filelog: ! wall 1.552293 comb 1.550000 user 1.510000 sys 0.040000 (median of 6 base: ! wall 0.022662 comb 0.020000 user 0.020000 sys 0.000000 (median of 128) before: ! wall 0.021111 comb 0.020000 user 0.020000 sys 0.000000 (median of 139) after: ! wall 0.021577 comb 0.020000 user 0.020000 sys 0.000000 (median of 138) revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63 filelog: ! wall 1.500983 comb 1.500000 user 1.420000 sys 0.080000 (median of 7) base: ! wall 0.006956 comb 0.010000 user 0.010000 sys 0.000000 (median of 392) before: ! wall 0.004356 comb 0.010000 user 0.010000 sys 0.000000 (median of 675) after: ! wall 0.004329 comb 0.000000 user 0.000000 sys 0.000000 (median of 682) revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e filelog: ! wall 0.011745 comb 0.020000 user 0.020000 sys 0.000000 (median of 250) base: ! wall 0.000156 comb 0.000000 user 0.000000 sys 0.000000 (median of 17180) before: ! wall 0.000100 comb 0.000000 user 0.000000 sys 0.000000 (median of 26912) after: ! wall 0.000105 comb 0.000000 user 0.000000 sys 0.000000 (median of 25689) revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe filelog: ! wall 3.228230 comb 3.230000 user 3.110000 sys 0.120000 (median of 4) base: ! wall 0.997640 comb 1.000000 user 0.980000 sys 0.020000 (median of 10) before: ! wall 0.778291 comb 0.780000 user 0.780000 sys 0.000000 (median of 13) after: ! wall 0.706594 comb 0.710000 user 0.710000 sys 0.000000 (median of 15) revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d filelog: ! wall 1.052501 comb 1.060000 user 1.040000 sys 0.020000 (median of 10 base: ! wall 0.214519 comb 0.220000 user 0.220000 sys 0.000000 (median of 45) before: ! wall 0.160804 comb 0.160000 user 0.160000 sys 0.000000 (median of 62) after: ! wall 0.163736 comb 0.160000 user 0.160000 sys 0.000000 (median of 60) Differential Revision: https://phab.mercurial-scm.org/D7069

rust-utils: introduce a debug util to print the python stack trace Differential Revision: https://phab.mercurial-scm.org/D7057

rust-cross-platform: remove `unimplemented!` to get compile-time errors We should get a compile-time error that there is missing functionality for the targeted platform instead of the program breaking at runtime. Differential Revision: https://phab.mercurial-scm.org/D7107

widening: pass in matchers instead of patterns This matches how it's done for the non-ellipsis case. The oldmatch is not used yet, but it should be used eventually when widening no longer resends manifests and files the client already has. Differential Revision: https://phab.mercurial-scm.org/D7099

widening: trust user to give full "known" set The new narrow_widen wire protocol command is supposed to return data for exactly the revisions that the client requested (in order for it to be strip-free). So we should not add ancestors the client didn't ask for. Differential Revision: https://phab.mercurial-scm.org/D7098

widening: remove "depth" argument since it's always None Differential Revision: https://phab.mercurial-scm.org/D7097

widening: remove unused "heads" argument Differential Revision: https://phab.mercurial-scm.org/D7096

widening: remove pointless code for second changegroup The caller passes in common==heads, so we will never find any nodes to visit in this code and there will therefore never be a second changegroup emitted. Differential Revision: https://phab.mercurial-scm.org/D7095

widening: remove always-true condition in widening code The function is now specifically about widening, so we don't need to check if we're widening. Differential Revision: https://phab.mercurial-scm.org/D7094

narrow: drop server support for widening using the getbundle command (BC) The client still supports widening using the getbundle, which we (Google) still depend on a for a little while more (we've started the migration to the new narrow_widen command, but we're not done yet). Differential Revision: https://phab.mercurial-scm.org/D7093

widening: duplicate generateellipsesbundle2() for widening The widening and the non-widening code are quite different. It will be clearer to have them as sepearate functions. To start with, I've just copied it exactly, so it's clearer over the next few patches how they're different. The new function should gradually become more similar to bundle2.widen_bundle(), and should perhaps eventually be merged with that function. However, I've left it in narrowbundle2.py for now since it still depends on constants like _KILLNODESIGNAL there. Differential Revision: https://phab.mercurial-scm.org/D7092

tests: show graph log before and after widening in more cases I'm about to make some changes to which revisions get sent during widening. Some more tests will make it clearer what changes. I've also switched to graph log so we easily notice if the graph shape changes. Differential Revision: https://phab.mercurial-scm.org/D7091

tracing: add a couple of trace points on obsolete and repoview I was seeing weird stalls in some fast commands, and these trace events help explain them. Differential Revision: https://phab.mercurial-scm.org/D6550

dirstate: add some traces on listdir calls I'm starting to spend some time trying to make `hg status` do less work, and knowing what's happening in here and what's expensive is an important start. Differential Revision: https://phab.mercurial-scm.org/D7109

cleanup: hgdemandimport.tracing accepts strings, not bytes This does mean that the resulting traces will have some b'' goo in them on Python 3, but I think that's a worthwile price to pay since it doesn't break any tests. Differential Revision: https://phab.mercurial-scm.org/D7108

tests: fix formatting test after move of black config Differential Revision: https://phab.mercurial-scm.org/D7102

rust-dependencies: pin the dependencies of hg-core to prevent breakage A new version of the `rand` crate broke two discovery tests. We should upgrade our dependencies periodically manually. Differential Revision: https://phab.mercurial-scm.org/D7115

automation: install Python 3.8.0 Python 3.8 has been released. Let's install it instead of the RC / development version. Differential Revision: https://phab.mercurial-scm.org/D7114

automation: schedule an EC2Launch run on next boot Without this, launching EC2 instances constructed from the AMI won't go through the normal EC2 instance launch machinery. This missing machinery does important things like set up network routes to use the instance metadata service and process any UserData. Since EC2Launch now runs on subsequent boots and UserData is processed, we needed to make setting of UserData conditional on bootstrapping mode. Differential Revision: https://phab.mercurial-scm.org/D7113

dirs: give formatting oversight to clang-format Differential Revision: https://phab.mercurial-scm.org/D7104

dirs: tag a struct as not being formattable Differential Revision: https://phab.mercurial-scm.org/D7103

sidedatacopies: only store an entry if it has values This will make for a shorter storage and help use to write faster code in simple case. This change already provided a speed boost for copy tracing. For example here is combined time of running copies tracing on a 6989 pairs of revision on the pypy repos: before: 771s after: 631s - 18% This also has a very positive impact on changelog size. For example here are the number for the `00changelog.d` file of pypy. before: 30449712 Bytes after: 24973718 Bytes - 18% This give an overall quite acceptable overhead for storing copies into the changelog: filelog-only: 23370586 sidedata: 24973718 Bytes + 7% Differential Revision: https://phab.mercurial-scm.org/D7068

sidedatacopies: only read from copies when in this mode If we know we expect data from sidedata, we read them from sidedata and nothing else. This avoid looking into extra for revision without sidedata entries. Such revision will be introduced in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D7067

dirs: remove mutable string optimization at all As far as I can see, the optimization trick has been dead since 42e89b87ca79 "dirs: speed up by storing number of direct children per dir". After 42e89b87ca79, the key variable is cleared to NULL at each iteration.

py3: prevent comparison with None in curses histedit (issue6196)

fix: match patterns relative to root I was surprised fixer patterns (used to determine which fixers to run) are applies to the parent directory, not the repo root directory. Danny Hooper (the author of the extension) seemed to agree that it's better to apply them to the repo root, so that's what this patch does. Differential Revision: https://phab.mercurial-scm.org/D7101

tests: add test showing that fixer patterns are currently relative to $PWD Differential Revision: https://phab.mercurial-scm.org/D7100

black: also ignore grey.py Differential Revision: https://phab.mercurial-scm.org/D7090

black: use multiline exclude definition Our excludes list is pretty complicated, so let's give ourselves some readability. Black treats multiline exclude definitions as verbse regular expressions, so we can split this up and make it easier to consume. Differential Revision: https://phab.mercurial-scm.org/D7089

black: move remaining config knobs into toml file Differential Revision: https://phab.mercurial-scm.org/D7088

black: rename pyproject.toml to black.toml Black won't read this automatically (you'll have to specify --config), but having a pyproject.toml *at all* puts pip in PEP 517/518 mode which breaks us for obscure reasons I don't understand. Rather than waste a ton of time fighting with pip, let's just do this. Differential Revision: https://phab.mercurial-scm.org/D7087

fix: remove a never-true check for unset pattern in Fixer.affects() We don't create an instance of a Fixer if the pattern is None, so we don't need to check that in affects(). Differential Revision: https://phab.mercurial-scm.org/D7086

fix: warn when a fixer doesn't have a configured command It seems we currently produce an empty command line and then decide to not run it, but it seems better to skip that part too. Differential Revision: https://phab.mercurial-scm.org/D7085

fix: make Fixer initialization more explicit for clarity I found it quite confusing that Fixer accessed fields that seemed like they didn't exist. Differential Revision: https://phab.mercurial-scm.org/D7084

fix: don't pass in default value when looking up config The defaults are now registered correctly so we don't need to pass in the default value when we look up a config value. Differential Revision: https://phab.mercurial-scm.org/D7083