Sat, 12 Nov 2022 00:18:41 +0100 emitrevision: consider ancestors revision to emit as available base
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Nov 2022 00:18:41 +0100] rev 49672
emitrevision: consider ancestors revision to emit as available base This should make more delta base valid. This notably affects: * case where we skipped some parent with empty delta to directly delta against an ancestors * case where an intermediate snapshots is stored. This change means we could sent largish intermediate snapshots over the wire. However this is actually a sub goal here. Sending snapshots over the wire means the client have a high odd of simply storing the pre-computed delta instead of doing a lengthy process that will… end up doing the same intermediate snapshot. In addition the overall size of snapshot (or any level) is "only" some or the overall delta size. (0.17% for my mercurial clone, 20% for my clone of Mozilla try). So Sending them other the wire is unlikely to change large impact on the bandwidth used. If we decide that minimising the bandwidth is an explicit goal, we should introduce new logic to filter-out snapshot as delta. The current code has no notion explicite of snapshot so far, they just tended to fall into the wobbly filtering options. In some cases, this patch can yield large improvement to the bundling time: ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-bundle # benchmark.variants.revs = last-100000 before: 68.787066 seconds after: 47.552677 seconds (-30.87%) That translate to large improvement to the pull time : ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = pull # benchmark.variants.issue6528 = disabled # benchmark.variants.revs = last-100000 before: 142.186625 seconds after: 75.897745 seconds (-46.62%) No significant negative impact have been observed.
Wed, 09 Nov 2022 13:54:15 -0500 sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Nov 2022 13:54:15 -0500] rev 49671
sqlitestore: add an `ancestors` method We will need it during bundling. The implementation mirror the one in revlog.
Thu, 24 Nov 2022 04:04:19 +0100 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 24 Nov 2022 04:04:19 +0100] rev 49670
emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Falling back to `prev` does not yield any real value on modern storage and result in pathological changes to be created on the other side. Doing a delta against a parent will likely be smaller (helping the network) and will be safer to apply on the client (helping future pulls by Triggering intermediate snapshop where they will be needed by later deltas).
Mon, 28 Nov 2022 16:27:23 +0100 emitrevision: simplify the fallback to computed delta
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 16:27:23 +0100] rev 49669
emitrevision: simplify the fallback to computed delta Not using the stored delta, or having a full snapshot on disk behave the same ways, so lets use the same code path for that, this is simpler, and it update will be simpler.
Mon, 28 Nov 2022 15:59:52 +0100 emitrevision: also check the parents in the availability closure
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:59:52 +0100] rev 49668
emitrevision: also check the parents in the availability closure One of the point of having a closure is to gather the logic in it. So we gather the logic. The `parents[:]` part is a bit ugly but will be replaced by better code soon anyway.
Mon, 28 Nov 2022 15:48:51 +0100 emitrevision: add a small closure to check if a base is usable
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:48:51 +0100] rev 49667
emitrevision: add a small closure to check if a base is usable We will make more use of this and make it more complex too.
Mon, 17 Oct 2022 16:26:16 +0200 reuse-delta-base: improves some documentation
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 Oct 2022 16:26:16 +0200] rev 49666
reuse-delta-base: improves some documentation The current code got me a bit confused initially. So a bit more documentation around it cannot hurt.
(0) -30000 -10000 -3000 -1000 -300 -100 -30 -10 -7 +7 +10 +30 +100 +300 +1000 tip