Fri, 05 Apr 2024 11:17:25 +0200 phases: more compact error handling in analyzeremotephases
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 05 Apr 2024 11:17:25 +0200] rev 51578
phases: more compact error handling in analyzeremotephases using an intermediate variable result in more readable code, so let us use it.
Tue, 09 Apr 2024 02:54:12 +0200 push: rework the computation of fallbackheads to be correct
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 09 Apr 2024 02:54:12 +0200] rev 51577
push: rework the computation of fallbackheads to be correct The previous computation tried to be smart but ended up being wrong. This was caught by phase movement test while reworking the phase discovery logic to be faster. The previous logic was failing to catch case where the pushed set was not based on a common heads (i.e. when the discovery seemed to have "over discovered" content, outside the pushed set) In the following graph, `e` is a common head and we `hg push -r f`. We need to detect `c` as a fallback heads and we previous failed to do so:: e | d f |/ c | b | a The performance impact of the change seems minimal. On the most impacted repository at hand (mozilla-try), the slowdown seems mostly mixed in the overall noise `hg push` but seems to be in the hundred of milliseconds order of magnitude. When using rust, we seems to be a bit faster, probably because we leverage more accelaratd internals. I added a couple of performance related common for further investigation later on.
Fri, 05 Apr 2024 11:05:54 +0200 revset: stop serializing node when using "%ln"
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 05 Apr 2024 11:05:54 +0200] rev 51576
revset: stop serializing node when using "%ln" Turning hundred of thousand of node from node to hex and back can be slow… what about we stop doing it? In many case were we are using node id we should be using revision id. However this is not a good reason to have a stupidly slow implementation of "%ln". This caught my attention again because the phase discovery during push make an extensive use of "%ln" or huge set. In absolute, that phase discovery probably should use "%ld" and need to improves its algorithmic complexity, but improving "%ln" seems simple and long overdue. This greatly speeds up `hg push` on repository with many drafts. Here are some relevant poulpe benchmarks: ### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog # benchmark.name = hg.command.push # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.explicit-rev = all-out-heads # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = default ## benchmark.variants.revs = any-1-extra-rev before: 44.235070 after: 20.416329 (-53.85%, -23.82) ## benchmark.variants.revs = any-100-extra-rev before: 49.234697 after: 26.519829 (-46.14%, -22.71) ### benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 ## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog before: 10.138396 after: 7.750458 (-23.55%, -2.39) ## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog before: 1.263859 after: 0.700229 (-44.60%, -0.56) ## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog before: 399.484481 after: 346.5089 (-13.26%, -52.98) ## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog before: 4.540080 after: 3.401700 (-25.07%, -1.14) ## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog before: 2.975765 after: 1.870798 (-37.13%, -1.10)
Tue, 09 Apr 2024 14:41:48 +0200 bundlespec: drop unused _bundlespecvariants dictionary
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 09 Apr 2024 14:41:48 +0200] rev 51575
bundlespec: drop unused _bundlespecvariants dictionary Why do we have a `_bundlespecvariants`?
Tue, 09 Apr 2024 14:37:24 +0200 bundlespec: type the _bundlespeccontentopts dictionary
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 09 Apr 2024 14:37:24 +0200] rev 51574
bundlespec: type the _bundlespeccontentopts dictionary If only we had a tool to detect the kind of stupid error we just fixed… ho wait.
Tue, 09 Apr 2024 14:36:01 +0200 bundlespec: fix the "streamv2" and "streamv3-exp" variant
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 09 Apr 2024 14:36:01 +0200] rev 51573
bundlespec: fix the "streamv2" and "streamv3-exp" variant In c4aab3661f25, we broken this feature by adding unicode instead of bytes to the dictionary. On the other hand, this feature was never tested, so augment the tests to tests this.
Thu, 04 Apr 2024 14:15:32 +0100 wireprotoserver: ensure that output stream gets flushed on exception stable
Arseniy Alekseyev <aalekseyev@janestreet.com> [Thu, 04 Apr 2024 14:15:32 +0100] rev 51572
wireprotoserver: ensure that output stream gets flushed on exception Previously flush was happening due to Python finalizer being run on `BufferedWriter`. With upgrade to Python 3.11 this started randomly failing. My guess is that the finalizer on the raw `FileIO` object may be running before the finalizer of `BufferedWriter` has a chance to run. At any rate, since we're not relying on finalizers in the happy case we should also not rely on them in case of exception.
Mon, 15 Apr 2024 16:33:37 +0100 match: strengthen visit_children_set invariant, Recursive means "all files" stable
Arseniy Alekseyev <aalekseyev@janestreet.com> [Mon, 15 Apr 2024 16:33:37 +0100] rev 51571
match: strengthen visit_children_set invariant, Recursive means "all files" My previous interpretation of "Recursive" was too relaxed: I thought it instructed the caller to do something like this: > you can stop calling `visit_children_set` because you'll need to descend into > every directory recursively, but you should still check every file if it > matches or not Whereas the real instruction seems to be: > I guarantee that everything in this subtree matches, you can stop > querying the matcher for all files and dirs altogether. The evidence to support this: - the test actually passes with the stronger invariant, revealing no exceptions from this rule - the implementation of `visit_children_set` for `DifferenceMatcher` clearly relies on this requirement, so it must hold for that not to lead to bugs.
Fri, 12 Apr 2024 16:09:45 +0100 match: fix the rust-side bug in visit_children_set for rootfilesin matchers stable
Arseniy Alekseyev <aalekseyev@janestreet.com> [Fri, 12 Apr 2024 16:09:45 +0100] rev 51570
match: fix the rust-side bug in visit_children_set for rootfilesin matchers The fix is checked by `test_pattern_matcher_visit_children_set` test, which is what caught the bug in the first place, but also by an end-to-end test that I made for this purpose. Accept the new results of Cargo tests Many of these were already annotated with "FIXME", which is a good sign.
Fri, 12 Apr 2024 15:39:21 +0100 match: fix the "visitdir" method on "rootfilesin" matchers stable
Arseniy Alekseyev <aalekseyev@janestreet.com> [Fri, 12 Apr 2024 15:39:21 +0100] rev 51569
match: fix the "visitdir" method on "rootfilesin" matchers This fixes just the Python side, the fix for the rust side will follow shortly.
(0) -30000 -10000 -3000 -1000 -300 -100 -10 +10 +100 +300 tip