Fri, 02 Dec 2022 03:51:27 +0100 path: clarify document of `get_clone_path`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 03:51:27 +0100] rev 49850
path: clarify document of `get_clone_path` This return a url as `bytes`, not a `path` object.
Fri, 02 Dec 2022 06:52:27 +0100 path: pass `path` to `peer` in `hg perf::discovery`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:52:27 +0100] rev 49849
path: pass `path` to `peer` in `hg perf::discovery` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:49:39 +0100 path: pass `path` to `peer` in remotefilelog's tests
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:49:39 +0100] rev 49848
path: pass `path` to `peer` in remotefilelog's tests We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:48:17 +0100 path: pass `path` to `peer` in `hg fastannotate`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:48:17 +0100] rev 49847
path: pass `path` to `peer` in `hg fastannotate` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:45:46 +0100 path: pass `path` to `peer` in infinitepush
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:45:46 +0100] rev 49846
path: pass `path` to `peer` in infinitepush We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:42:17 +0100 path: pass `path` to `peer` in largefiles
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:42:17 +0100] rev 49845
path: pass `path` to `peer` in largefiles We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:38:03 +0100 path: pass `path` to `peer` in narrow
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:38:03 +0100] rev 49844
path: pass `path` to `peer` in narrow We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:37:15 +0100 path: pass `path` to `peer` in `hg fetch`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:37:15 +0100] rev 49843
path: pass `path` to `peer` in `hg fetch` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:33:50 +0100 path: use `get_unique_pull_path_obj` in `hg relink`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:33:50 +0100] rev 49842
path: use `get_unique_pull_path_obj` in `hg relink` This is not really needed, but that help removing caller of the older function.
Fri, 02 Dec 2022 06:31:19 +0100 path: pass `path` to `peer` in `hg transplant`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:31:19 +0100] rev 49841
path: pass `path` to `peer` in `hg transplant` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:29:11 +0100 path: pass `path` to `peer` in `hg debugbackupbundle`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:29:11 +0100] rev 49840
path: pass `path` to `peer` in `hg debugbackupbundle` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:24:52 +0100 path: pass `path` to `peer` in `hg debugssl`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:24:52 +0100] rev 49839
path: pass `path` to `peer` in `hg debugssl` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 06:21:08 +0100 path: pass `path` to `peer` in `hg debugdiscovery`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 06:21:08 +0100] rev 49838
path: pass `path` to `peer` in `hg debugdiscovery` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 05:11:53 +0100 path: pass `path` to `peer` in `remote(...)` revset
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 05:11:53 +0100] rev 49837
path: pass `path` to `peer` in `remote(...)` revset We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 05:10:05 +0100 path: pass `path` to `peer` in `hg summary`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 05:10:05 +0100] rev 49836
path: pass `path` to `peer` in `hg summary` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 04:31:08 +0100 path: pass `path` to `peer` in `hg identify`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 04:31:08 +0100] rev 49835
path: pass `path` to `peer` in `hg identify` We directly use the `path` object to build the `peer` object.
Fri, 02 Dec 2022 03:50:28 +0100 path: introduce a `get_unique_pull_path_obj` function
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 03:50:28 +0100] rev 49834
path: introduce a `get_unique_pull_path_obj` function Unlike the previous one, `get_unique_pull_path`, this function return the `path` object, opening more option for the caller. note that this highlight we don't actually need the `repo` argument to `get_pull_paths`, however changing the API would be annoying for third party extensions.
Fri, 02 Dec 2022 01:55:05 +0100 path: simplify the `get_unique_pull_path` function
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 01:55:05 +0100] rev 49833
path: simplify the `get_unique_pull_path` function Simply delegate the search to `get_pull_paths` and check how many we got.
Fri, 02 Dec 2022 01:41:27 +0100 path: remove outdated documentation point from `get_unique_push_path`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 01:41:27 +0100] rev 49832
path: remove outdated documentation point from `get_unique_push_path` This is no longer true.
Thu, 01 Dec 2022 18:41:59 +0100 path: pass `path` to `peer` in `hg pull`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 18:41:59 +0100] rev 49831
path: pass `path` to `peer` in `hg pull` We directly use the `path` object to build the `peer` object. We don't use it for sub-repositories yet, which will have to be fixed at some point.
Thu, 01 Dec 2022 18:19:08 +0100 path: pass `path` to `peer` in `hg incoming`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 18:19:08 +0100] rev 49830
path: pass `path` to `peer` in `hg incoming` We directly use the `path` object to build the `peer` object. We don't use it for sub-repositories yet, which will have to be fixed at some point.
Thu, 01 Dec 2022 17:55:17 +0100 path: pass `path` to `peer` in `hg incoming` bookmark logic
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 17:55:17 +0100] rev 49829
path: pass `path` to `peer` in `hg incoming` bookmark logic We directly use the `path` object to build the `peer` object. Touching this code highlighted that we never honor the branches' information of when doing bookmark level incoming checks.
Thu, 01 Dec 2022 16:58:22 +0100 path: remove outdated documentation point from `get_unique_pull_path`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 16:58:22 +0100] rev 49828
path: remove outdated documentation point from `get_unique_pull_path` This is no longer true.
Thu, 01 Dec 2022 16:53:22 +0100 path: update `get_unique_pull_path` to point out it returns a url
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 16:53:22 +0100] rev 49827
path: update `get_unique_pull_path` to point out it returns a url Unlike other functions in this module it returns a url as `bytes` and not a `path` object. Let us point it out in the doc.
Fri, 02 Dec 2022 02:03:49 +0100 changelog-v2: fix the docket `struct`
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 02:03:49 +0100] rev 49826
changelog-v2: fix the docket `struct` The previous definition used `L` which is actually 4 bytes, while the documentation and intend was to use `Q`, i.e. 8 bytes.
Thu, 01 Dec 2022 02:26:34 +0100 path: pass `path` to `peer` in infinite push
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:26:34 +0100] rev 49825
path: pass `path` to `peer` in infinite push We directly use the `path` object to build the `peer` object.
Thu, 01 Dec 2022 02:21:18 +0100 path: pass `path` to `peer` in `hg histedit`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:21:18 +0100] rev 49824
path: pass `path` to `peer` in `hg histedit` We directly use the `path` object to build the `peer` object.
Thu, 01 Dec 2022 02:14:40 +0100 path: pass `path` to `peer` in the `outgoing` revset
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:14:40 +0100] rev 49823
path: pass `path` to `peer` in the `outgoing` revset We directly use the `path` object to build the `peer` object.
Thu, 01 Dec 2022 02:11:21 +0100 path: pass `path` to `peer` in `hg summary`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:11:21 +0100] rev 49822
path: pass `path` to `peer` in `hg summary` We directly use the `path` object to build the `peer` object.
Thu, 01 Dec 2022 02:09:43 +0100 path: pass `path` to `peer` in `hg outgoing`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:09:43 +0100] rev 49821
path: pass `path` to `peer` in `hg outgoing` We directly use the `path` object to build the `peer` object.
Thu, 01 Dec 2022 01:57:14 +0100 path: pass `path` to `peer` in `hg bundle`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:57:14 +0100] rev 49820
path: pass `path` to `peer` in `hg bundle` We directly use the `path` object to build the `peer` object.
Wed, 30 Nov 2022 19:43:26 +0100 path: have `peer` constructor accept a `path` object
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 19:43:26 +0100] rev 49819
path: have `peer` constructor accept a `path` object We don't do anything fancy with it yet, but this is an important step towards having the peers aware of their "source" and associated configurations.
Thu, 01 Dec 2022 01:46:46 +0100 path: deprecated the `pushloc` attribute
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:46:46 +0100] rev 49818
path: deprecated the `pushloc` attribute We want to make sure people with use the full featured path "variant".
Thu, 01 Dec 2022 01:41:34 +0100 path: update logic in `perf` to use the push variant when available
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:41:34 +0100] rev 49817
path: update logic in `perf` to use the push variant when available The command seems currently broken, but at least it won't be broken by us !
Thu, 01 Dec 2022 01:38:33 +0100 path: directly use the push_variant in `infinitepush`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:38:33 +0100] rev 49816
path: directly use the push_variant in `infinitepush` We don't need any extra processing now.
Thu, 01 Dec 2022 01:38:07 +0100 path: directly use the push_variant in `hg histedit` outgoing logic
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:38:07 +0100] rev 49815
path: directly use the push_variant in `hg histedit` outgoing logic We don't need any extra processing now.
Thu, 01 Dec 2022 01:37:41 +0100 path: directly use the push_variant in the `outgoing` revset
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:37:41 +0100] rev 49814
path: directly use the push_variant in the `outgoing` revset We don't need any extra processing now.
Thu, 01 Dec 2022 01:37:10 +0100 path: directly use the push_variant in outgoing internals
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:37:10 +0100] rev 49813
path: directly use the push_variant in outgoing internals We don't need any extra processing now.
Thu, 01 Dec 2022 01:35:17 +0100 path: directly use the push_variant in `hg summary`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:35:17 +0100] rev 49812
path: directly use the push_variant in `hg summary` We don't need any extra processing now.
Thu, 01 Dec 2022 01:34:58 +0100 path: directly use the push_variant in `hg outgoing`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:34:58 +0100] rev 49811
path: directly use the push_variant in `hg outgoing` We don't need any extra processing now.
Thu, 01 Dec 2022 01:34:26 +0100 path: directly use the push_variant in `hg push`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:34:26 +0100] rev 49810
path: directly use the push_variant in `hg push` We don't need any extra processing now.
Thu, 01 Dec 2022 01:33:27 +0100 path: have `get_push_paths` directly return the push variants
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:33:27 +0100] rev 49809
path: have `get_push_paths` directly return the push variants So the function directly returns usable paths, that should help the callers!
Thu, 01 Dec 2022 01:32:24 +0100 path: add a method to retrieve a "push variant" of a path
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:32:24 +0100] rev 49808
path: add a method to retrieve a "push variant" of a path This gets you the same path, but using the `pushurl` as destination. This opens the way for a lot of different improvements, the one which interests us is having `peer` objects aware of the `path` they came from.
Thu, 01 Dec 2022 01:27:47 +0100 path: move the url parsing and related attribute setting to a method
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:27:47 +0100] rev 49807
path: move the url parsing and related attribute setting to a method This will make it simpler to reuse this logic in the next changeset.
Tue, 29 Nov 2022 22:22:18 +0100 peer-or-repo: remove the now unused function
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:22:18 +0100] rev 49806
peer-or-repo: remove the now unused function We do not need it anymore.
Tue, 29 Nov 2022 22:21:19 +0100 peer-or-repo: build a repo directly in the `repo` function
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:21:19 +0100] rev 49805
peer-or-repo: build a repo directly in the `repo` function We skip the ambiguous _peerorrepo function to explicitly build a repo within the dedicated function. The peer scheme are therefore no longer considered to build the object.
Tue, 29 Nov 2022 22:04:23 +0100 peer-or-repo: build a peer directly in the `peer` function
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:04:23 +0100] rev 49804
peer-or-repo: build a peer directly in the `peer` function We skip the ambiguous _peerorrepo function to explicitly build a peer within the dedicated function. This mean explicitly getting a peer from an explicitly create repository when necessary.
Tue, 29 Nov 2022 22:03:10 +0100 static-http: have `statichttprepo.instance` return a peer object
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:03:10 +0100] rev 49803
static-http: have `statichttprepo.instance` return a peer object It previously returned a statichttprepo object which could not be used for anything but creating a peer. So lets put it into the peer bucket with a peer behavior.
Wed, 30 Nov 2022 12:22:02 +0100 scheme: move the drive letter checking in its own function
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 12:22:02 +0100] rev 49802
scheme: move the drive letter checking in its own function This help the readability of the main function as is was taking much more room than the main logic.
Tue, 29 Nov 2022 21:48:08 +0100 peer-or-repo: split the scheme between repo and peer
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 21:48:08 +0100] rev 49801
peer-or-repo: split the scheme between repo and peer Some of the scheme will always produce a peer and some will always produce a repository. So lets use different mapping to reduce the ambiguity.
Wed, 30 Nov 2022 13:55:15 +0100 peer-or-repo: stop relying on AttributeError in `islocal`
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 13:55:15 +0100] rev 49800
peer-or-repo: stop relying on AttributeError in `islocal` This will confused pytypes in a future changeset.
Tue, 29 Nov 2022 21:42:08 +0100 repo-or-peer: deprecate calling `islocal` on non-path object
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 21:42:08 +0100] rev 49799
repo-or-peer: deprecate calling `islocal` on non-path object There object have a `.local()` method and should use it.
Wed, 30 Nov 2022 11:12:48 +0100 share: stop using 'islocal' with repo instance
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 11:12:48 +0100] rev 49798
share: stop using 'islocal' with repo instance Having this level of polymorphism of the `islocal` function is weird, and we already have a way to know if the repo is local from the object itself. We are about to deprecate passing a non-bytes object to `islocal`, so clean this up beforehand. We might want to clean up this function too in the future, however this is another adventure.
Tue, 29 Nov 2022 19:54:55 +0100 peer-or-repo: make sure object in "scheme" have a `instance` object
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 19:54:55 +0100] rev 49797
peer-or-repo: make sure object in "scheme" have a `instance` object The previous form of having heterogeneous object in the dictionnary makes things more complicated than they needed to be. I am not super happy about the current (especially around 'islocal', that most item do not have), but this is already much better.
Tue, 29 Nov 2022 18:30:54 +0100 peer-or-repo: move the object setup in its own function
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 18:30:54 +0100] rev 49796
peer-or-repo: move the object setup in its own function The `_peerorrepo` function is problematic, because it can build different types of object (repository and peer). This make it hard to adjust the arguments to the type of object we needs. So this patch start a series of change to create peer and repo without going through a common function. We move the part of the function doing object setup it its own function to make it simpler to reuse in others contexts.
Sun, 06 Nov 2022 17:53:17 -0500 delta-find: use a single snapshot cache when applying a group to an object
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 17:53:17 -0500] rev 49795
delta-find: use a single snapshot cache when applying a group to an object This will avoid walking the revlog over and over again in some situations. The difference is hard to show in our current benchmark suite, as the gain is lower than their overall instability.
Sun, 06 Nov 2022 17:55:55 -0500 delta-find: make sure we only use newer full snapshot as candidate
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 17:55:55 -0500] rev 49794
delta-find: make sure we only use newer full snapshot as candidate The current code does not needs to protect against this, as there are no older snapshot in the current cache. However as we are getting ready to reuse this cache from one revision to another, we need the code to protect itself about what's coming.
Sun, 06 Nov 2022 17:55:46 -0500 delta-find: use sets instead of list in the snapshot cache
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 17:55:46 -0500] rev 49793
delta-find: use sets instead of list in the snapshot cache This seems more appropriate.
Sun, 06 Nov 2022 16:56:23 -0500 delta-find: use a smarter object for snapshot caching
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 16:56:23 -0500] rev 49792
delta-find: use a smarter object for snapshot caching This open the way for a longer lived cache.
Mon, 07 Nov 2022 22:12:59 -0500 find-delta: pass the cache-delta usage policy alongside the cache-delta
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 22:12:59 -0500] rev 49791
find-delta: pass the cache-delta usage policy alongside the cache-delta The idea is to give higher level code more control to what will happens with the cache delta passed. This should help with controling how we treat delta's from different sources. The final goal of this change is to allow for server modes where the client can blindly accept any server delta without regards to any local constraints. This will be implemented in later changesets.
Mon, 28 Nov 2022 18:58:35 +0100 find-delta: move most of the debug-find-delta code in the debug module
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 18:58:35 +0100] rev 49790
find-delta: move most of the debug-find-delta code in the debug module Lets us that module more. It will help us to keep revlog implementation details close to each other.
Mon, 07 Nov 2022 20:02:32 -0500 find-delta: minor preparatory change
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 20:02:32 -0500] rev 49789
find-delta: minor preparatory change We are about to add more item in the cachedelta object, so lets access the item by index instead of doing a full extensions.
Mon, 07 Nov 2022 17:57:28 -0500 find-delta: rename _isgooddeltainfo
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 17:57:28 -0500] rev 49788
find-delta: rename _isgooddeltainfo Lets move to a more readable name now that we are allowed to. This cannot hurt.
Mon, 07 Nov 2022 18:06:17 -0500 test-revlog-raw: drop the overwrite of dead code
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 18:06:17 -0500] rev 49787
test-revlog-raw: drop the overwrite of dead code The revlog class no longer have a _isgooddeltainfo method for a long time. So overwriting it does not get us anything. The test have been wrapping the right code since then anyway.
Sat, 12 Nov 2022 00:18:41 +0100 emitrevision: consider ancestors revision to emit as available base
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Nov 2022 00:18:41 +0100] rev 49786
emitrevision: consider ancestors revision to emit as available base This should make more delta base valid. This notably affects: * case where we skipped some parent with empty delta to directly delta against an ancestors * case where an intermediate snapshots is stored. This change means we could sent largish intermediate snapshots over the wire. However this is actually a sub goal here. Sending snapshots over the wire means the client have a high odd of simply storing the pre-computed delta instead of doing a lengthy process that will… end up doing the same intermediate snapshot. In addition the overall size of snapshot (or any level) is "only" some or the overall delta size. (0.17% for my mercurial clone, 20% for my clone of Mozilla try). So Sending them other the wire is unlikely to change large impact on the bandwidth used. If we decide that minimising the bandwidth is an explicit goal, we should introduce new logic to filter-out snapshot as delta. The current code has no notion explicite of snapshot so far, they just tended to fall into the wobbly filtering options. In some cases, this patch can yield large improvement to the bundling time: ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-bundle # benchmark.variants.revs = last-100000 before: 68.787066 seconds after: 47.552677 seconds (-30.87%) That translate to large improvement to the pull time : ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = pull # benchmark.variants.issue6528 = disabled # benchmark.variants.revs = last-100000 before: 142.186625 seconds after: 75.897745 seconds (-46.62%) No significant negative impact have been observed.
Wed, 09 Nov 2022 13:54:15 -0500 sqlitestore: add an `ancestors` method
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Nov 2022 13:54:15 -0500] rev 49785
sqlitestore: add an `ancestors` method We will need it during bundling. The implementation mirror the one in revlog.
Thu, 24 Nov 2022 04:04:19 +0100 emitrevision: if we need to compute a delta on the fly, try p1 or p2 first
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 24 Nov 2022 04:04:19 +0100] rev 49784
emitrevision: if we need to compute a delta on the fly, try p1 or p2 first Falling back to `prev` does not yield any real value on modern storage and result in pathological changes to be created on the other side. Doing a delta against a parent will likely be smaller (helping the network) and will be safer to apply on the client (helping future pulls by Triggering intermediate snapshop where they will be needed by later deltas).
Mon, 28 Nov 2022 16:27:23 +0100 emitrevision: simplify the fallback to computed delta
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 16:27:23 +0100] rev 49783
emitrevision: simplify the fallback to computed delta Not using the stored delta, or having a full snapshot on disk behave the same ways, so lets use the same code path for that, this is simpler, and it update will be simpler.
Mon, 28 Nov 2022 15:59:52 +0100 emitrevision: also check the parents in the availability closure
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:59:52 +0100] rev 49782
emitrevision: also check the parents in the availability closure One of the point of having a closure is to gather the logic in it. So we gather the logic. The `parents[:]` part is a bit ugly but will be replaced by better code soon anyway.
Mon, 28 Nov 2022 15:48:51 +0100 emitrevision: add a small closure to check if a base is usable
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:48:51 +0100] rev 49781
emitrevision: add a small closure to check if a base is usable We will make more use of this and make it more complex too.
Mon, 17 Oct 2022 16:26:16 +0200 reuse-delta-base: improves some documentation
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 Oct 2022 16:26:16 +0200] rev 49780
reuse-delta-base: improves some documentation The current code got me a bit confused initially. So a bit more documentation around it cannot hurt.
Fri, 25 Nov 2022 15:14:40 +0100 branching: merge stable into default
Raphaël Gomès <rgomes@octobus.net> [Fri, 25 Nov 2022 15:14:40 +0100] rev 49779
branching: merge stable into default
Tue, 22 Nov 2022 12:44:22 +0100 changelog-v2: add a configuration to disable rank computation
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 22 Nov 2022 12:44:22 +0100] rev 49778
changelog-v2: add a configuration to disable rank computation We encountered a graph where rank computation was pathologically slow. We add an option to disable this computation while this is getting fixed. Disabling the rank computation should allow for testing other changelog-v2 features undisturbed (like changeset-based copy tracing). I am purposely not adding a test for the new non-default code path, as this is a temporary work around of an experimental feature.
Mon, 21 Nov 2022 15:04:19 +0100 debugrevlog: display total stored information
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 21 Nov 2022 15:04:19 +0100] rev 49777
debugrevlog: display total stored information This is an interesting statistis, so let's display it.
Mon, 07 Nov 2022 14:38:52 -0500 debug-revlog: details about non-ancestors delta-bases
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 14:38:52 -0500] rev 49776
debug-revlog: details about non-ancestors delta-bases Deltas against a base that is not an ancestor of the revision that owns this delta are notable. For example, they introduce complexity during the bundling process as the base might not exist on the unbundling side. We detect them in `hg debugrevlog` and print information about them.
Mon, 07 Nov 2022 14:24:52 -0500 debug-revlog: move the code in revlogutils module
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 14:24:52 -0500] rev 49775
debug-revlog: move the code in revlogutils module We have a module dedicated to debug code, let us use it.
Mon, 07 Nov 2022 14:13:59 -0500 debug-revlog: move the --dump code in `revlogutils` module
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 14:13:59 -0500] rev 49774
debug-revlog: move the --dump code in `revlogutils` module We have a module dedicated to debug code, let us use it.
Wed, 23 Nov 2022 19:08:27 +0100 delta-find: set the default candidate chunk size to 10
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 23 Nov 2022 19:08:27 +0100] rev 49773
delta-find: set the default candidate chunk size to 10 I ran performance and storage tests on repositories of various sizes and shapes for the following values of the config : 5, 10, 20, 50, 100, no-chunking The performance tests do not show any statistical impact on computation times for large pushes and pulls. For searching for an individual delta, this can provide a significant performance improvement with a minor degradation of space-quality on the result. (see data at the end of the commit). For overall store size, the change : - does not have any impact on many small repositories, - has an observable, but very negligible impact on most larger repositories. - One private repository we use for testing sees a small increase in size (1%) in the narrower version. We will try to get more numbers on a larger version of that repository to make sure nothing pathological happens. We pick "10" as the limit as "5" seems a bit more risky. There are room to improve the current code, by using more aggressive filtering and better (i.e any) sorting of the candidates. However this is already a large improvement for pathological cases, with little impact in the common situations. The initial motivation for this change is to fix performance of delta computation for a file where the previous code ended up testing 20 000 possible candidate-bases in one go, which is… slow. This affected about ½ of the file revisions leading to atrocious performance, especially during some push/pull operations. Details about individual delta finding timing: ---------------------------------------------- The vast majority of benchmark cases are unchanged but the three below. The first two do not see any impact on the final delta. The last one sees a change in delta-size that is negligible compared to the full text size. ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-a (revision 756096) ∞: 5.844783 5: 4.473523 (-23.46%) 10: 4.970053 (-14.97%) 20: 5.770386 (-1.27%) 50 5.821358 100: 5.834887 MANIFESTLOG: rev = 756096: (no-limit) delta-base = 301840 search-rounds = 6 try-count = 60 delta-type = snapshot snap-depth = 7 delta-size = 179 MANIFESTLOG: rev=756096: (limit = 10) delta-base=301840 search-rounds=9 try-count=51 delta-type=snapshot snap-depth=7 delta-size=179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-d (revision 754060) ∞: 5.017663 5: 3.655931 (-27.14%) 10: 4.095436 (-18.38%) 20: 4.828949 (-3.76%) 50 4.987574 100: 4.994889 MANIFESTLOG: rev=754060: (no limit) delta-base=301840 search-rounds=5 try-count=53 delta-type=snapshot snap-depth=7 delta-size = 179 MANIFESTLOG: rev=754060: (limite = 10) delta-base=301840 search-rounds=8 try-count=45 delta-type=snapshot snap-depth=7 delta-size = 179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # bin-env-vars.hg.flavor = rust # benchmark.variants.rev = manifest-snapshot-many-tries-e (revision 693368) ∞: 4.869282 5: 2.039732 (-58.11%) 10: 2.413537 (-50.43%) 20: 4.449639 (-8.62%) 50 4.865863 100: 4.882649 MANIFESTLOG: rev=693368: delta-base=693336 search-rounds=6 try-count=53 delta-type=snapshot snap-depth=6 full-test-size=131065 delta-size=199 MANIFESTLOG: rev=693368: delta-base=278023 search-rounds=5 try-count=21 delta-type=snapshot snap-depth=4 full-test-size=131065 delta-size=278 Raw data for store size (in bytes) for various chunk size value below: ---------------------------------------------------------------------- 440 134 384 5 pypy/.hg/store/ 440 134 384 10 pypy/.hg/store/ 440 134 384 20 pypy/.hg/store/ 440 134 384 50 pypy/.hg/store/ 440 134 384 100 pypy/.hg/store/ 440 134 384 ... pypy/.hg/store/ 666 987 471 5 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 10 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 20 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 50 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 100 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 ... netbsd-xsrc-2022-11-15/.hg/store/ 852 844 884 5 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 10 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 20 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 50 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 100 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 ... netbsd-pkgsrc-2022-11-15/.hg/store/ 1 504 227 981 5 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 871 10 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 20 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 50 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 100 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 ... netbeans-2018-08-01-sparse-zstd/.hg/store/ 3 875 801 068 5 netbsd-src-2022-11-15/.hg/store/ 3 875 696 767 10 netbsd-src-2022-11-15/.hg/store/ 3 875 696 757 20 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 50 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 100 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 ... netbsd-src-2022-11-15/.hg/store/ 4 531 441 314 5 mozilla-central/.hg/store/ 4 531 435 157 10 mozilla-central/.hg/store/ 4 531 432 045 20 mozilla-central/.hg/store/ 4 531 429 119 50 mozilla-central/.hg/store/ 4 531 429 119 100 mozilla-central/.hg/store/ 4 531 429 119 ... mozilla-central/.hg/store/ 4 875 861 390 5 mozilla-unified/.hg/store/ 4 875 855 155 10 mozilla-unified/.hg/store/ 4 875 852 027 20 mozilla-unified/.hg/store/ 4 875 848 851 50 mozilla-unified/.hg/store/ 4 875 848 851 100 mozilla-unified/.hg/store/ 4 875 848 851 ... mozilla-unified/.hg/store/ 11 498 764 601 5 mozilla-try/.hg/store/ 11 497 968 858 10 mozilla-try/.hg/store/ 11 497 958 730 20 mozilla-try/.hg/store/ 11 497 927 156 50 mozilla-try/.hg/store/ 11 497 925 963 100 mozilla-try/.hg/store/ 11 497 923 428 ... mozilla-try/.hg/store/ 10 047 914 031 5 private-repo 9 969 132 101 10 private-repo 9 944 745 015 20 private-repo 9 939 756 703 50 private-repo 9 939 833 016 100 private-repo 9 939 822 035 ... private-repo
Sun, 06 Nov 2022 14:47:17 -0500 delta-find: add a way to control the number of bases tested at the same time
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 14:47:17 -0500] rev 49772
delta-find: add a way to control the number of bases tested at the same time See inline comment for details. The feature is currently disabled, but should be enabled by default to mitigate some existing pathological cases. Also see the next changeset for details.
Wed, 23 Nov 2022 21:11:46 -0500 setup: include vendored 3rd party type stubs
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 21:11:46 -0500] rev 49771
setup: include vendored 3rd party type stubs While pytype may not support PEP 561, PyCharm does, so having the stubs available means it can determine `foo = attr.ib(type=int)` means `foo` is an int. This only applies when using Mercurial as a library, like with TortoiseHg development- PyCharm is already smart enough to use the *.pyi files in the Mercurial source tree when hacking on Mercurial itself. I left the mercurial.cext stubs out because it seems very low level, that 3rd parties shouldn't be using directly.
Wed, 23 Nov 2022 20:59:53 -0500 ci: bump pytype to 2022.11.18
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:59:53 -0500] rev 49770
ci: bump pytype to 2022.11.18 No particular reason, other than the current build is fairly old. It flagged a few more things (that weren't errors based on the logic around them), but OTOH, some of the pyi stubs it generates are less specific.
Wed, 23 Nov 2022 20:56:22 -0500 ci: run the script to add vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:56:22 -0500] rev 49769
ci: run the script to add vendored type stubs to typeshed Since CI runs from docker images, it doesn't matter that this mucks with the typeshed bundled with pytype.
Wed, 23 Nov 2022 20:50:39 -0500 contrib: add a script for adding vendored type stubs to typeshed
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:50:39 -0500] rev 49768
contrib: add a script for adding vendored type stubs to typeshed I really hate this, but pytype doesn't support PEP 561 and doesn't seem to have the equivalent of `MYPYPATH` to point to custom stubs. Ignoring the vendored stubs isn't necessarily harmful, but pytype has been choking on the vendored attr package after pytype 2022.03.29 with errors like this: File "/mnt/c/Users/Matt/hg/mercurial/linelog.py", line 52, in __iter__: Built-in function iter was called with the wrong arguments [wrong-arg-types] Expected: (collection: bytearray) Actually passed: (collection: mercurial.thirdparty.attr._make._CountingAttr) File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 143, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types] Expected: (obj: Sized) Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr) Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__ File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 144, in pack: No attribute 'rfind' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error] File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 146, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types] Expected: (obj: Sized) Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr) Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__ File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 152, in pack: No attribute 'v2_data' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error] File "/mnt/c/Users/Matt/hg/mercurial/util.py", line 2817, in go: unsupported operand type(s) for /: 'count: mercurial.thirdparty.attr._make._CountingAttr' and 'float: float' [unsupported-operands] No attribute '__truediv__' on 'count: mercurial.thirdparty.attr._make._CountingAttr' or '__rtruediv__' on 'float: float' Called from (traceback): line 2981, in __bytes__ This is essentially the same hack we've been using in TortoiseHg to add the vendored PyQt5 stubs. What I don't understand is pytype *still* generates *.pyi files under .pytype/pyi/mercurial/thirdparty/attr, even when the package is explicitly ignored in the pytype command line args. But it avoids the errors, which means we aren't stuck on pytype==2022.03.29. https://github.com/google/pytype/issues/151
Wed, 23 Nov 2022 20:23:26 -0500 contrib: update check-pytype.sh to list stubs that caused pytype to crash
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:23:26 -0500] rev 49767
contrib: update check-pytype.sh to list stubs that caused pytype to crash The same logic is in the TortoiseHg tests for running pytype, and it's useful to know if a new version of pytype is better or worse.
Wed, 23 Nov 2022 16:11:20 -0500 typing: add py.typed to mercurial.cext for PEP 561 support
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 16:11:20 -0500] rev 49766
typing: add py.typed to mercurial.cext for PEP 561 support Unfortunately, pytype doesn't support this yet. But it was included with the attr package, so we might as well do it here for consistency. Unlike the attr package, these type hints are only partial, so they are marked as such[1] (but who knows if it matters, given these are C extensions, so no local source code to scan). [1] https://peps.python.org/pep-0561/#partial-stub-packages
Wed, 23 Nov 2022 15:50:20 -0500 typing: add missing signature for mercurial.cext.parsers.parse_index2()
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 15:50:20 -0500] rev 49765
typing: add missing signature for mercurial.cext.parsers.parse_index2() Flagged by pytype 2022.11.18 when the cext stubs are made visible to it. There are very likely other signatures that are missing, but this is enough to keep it happy for now.
Wed, 23 Nov 2022 11:22:22 -0500 typing: minor tweaks to allow updating to pytype 2022.11.18
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 11:22:22 -0500] rev 49764
typing: minor tweaks to allow updating to pytype 2022.11.18
Sun, 20 Nov 2022 22:54:43 -0500 typing: add type hints to mercurial/help.py
Matt Harbison <matt_harbison@yahoo.com> [Sun, 20 Nov 2022 22:54:43 -0500] rev 49763
typing: add type hints to mercurial/help.py Was hoping to find more issues like f09bc2ed9100, but it may be that nothing checks the args to that operation. In any event, the work is done and pytype doesn't do a very good job inferring the types. A few of th emore complicated things like the command table are left untyped, because they come from modules that aren't typed yet.
Sun, 20 Nov 2022 23:09:12 -0500 configitems: add a default value for "merge-tools.xxx.regappend"
Matt Harbison <matt_harbison@yahoo.com> [Sun, 20 Nov 2022 23:09:12 -0500] rev 49762
configitems: add a default value for "merge-tools.xxx.regappend" When trying to figure out how `hg help -v` took the Set interpolation path in f09bc2ed9100, I turned on devel warnings and noticed this (unrelated) warning: devel-warn: specifying a mismatched default value for a registered config item: 'merge-tools.beyondcompare4.regappend' '' at: c:\Users\Matt\hg\mercurial\filemerge.py:46 (_toolstr) The previous default value for this config was `None`, but that slightly complicates the code at the only site it is used, referenced above.
Mon, 21 Nov 2022 15:04:42 -0500 attr: vendor 22.1.0
Matt Harbison <matt_harbison@yahoo.com> [Mon, 21 Nov 2022 15:04:42 -0500] rev 49761
attr: vendor 22.1.0 The previous version was 5 years old, and pytype 2022.06.30 started complaining about various uses (e.g. seeing `mercurial.thirdparty.attr._make._CountingAttr` instead of `bytearray`). Hopefully this helps. Additionally, this has official python 3.11 support. The `attrs` package is left out, because it is simply a bunch of *.pyi stubs and `from attr.X import *`, and that's not how they've been used up to this point. We'd probably need to customize those anyway to `from mercurial.thirdparty.attr import *`.
Mon, 21 Nov 2022 16:18:28 -0500 tests: update test-util.py for modern attrs package
Matt Harbison <matt_harbison@yahoo.com> [Mon, 21 Nov 2022 16:18:28 -0500] rev 49760
tests: update test-util.py for modern attrs package When updating to 22.1.0, this test started failing: Traceback (most recent call last): File "/tmp/mercurial-ci/tests/test-util.py", line 53, in <module> _start_default = (util.timedcmstats.start.default, 'factory') AttributeError: type object 'timedcmstats' has no attribute 'start' Poking around in `hg debugshell`, the attribute is indeed missing, but looks to be attached to `__attrs_attrs__` in both the currently vendored and the modern version of attrs. The old attrs packages will print the same for both accesses, so fingers crossed... >>> print((util.timedcmstats.start.default, 'factory')) (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory') >>> print((util.timedcmstats.__attrs_attrs__.start.default, 'factory')) (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory')
Tue, 15 Nov 2022 01:52:46 +0100 rhg: upgrade the remainder of the dependencies
Raphaël Gomès <rgomes@octobus.net> [Tue, 15 Nov 2022 01:52:46 +0100] rev 49759
rhg: upgrade the remainder of the dependencies These are painless, so they are all grouped in this changeset.
Tue, 15 Nov 2022 00:02:43 +0100 rhg: upgrade `clap` dependency
Raphaël Gomès <rgomes@octobus.net> [Tue, 15 Nov 2022 00:02:43 +0100] rev 49758
rhg: upgrade `clap` dependency This one is the worst one to upgrade since v2 -> v4 broke a ton of API, which thankfully seems saner now. Contrary to what was done in the `hg-core/src/examples/nodemap` rewrite, we're not switching from the "builder" pattern to the "derive" pattern, since that would imply a much larger diff. It can be done incrementally.
Mon, 14 Nov 2022 17:18:56 +0100 hg-cpython: upgrade dependencies
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 17:18:56 +0100] rev 49757
hg-cpython: upgrade dependencies `hg-cpython` has no BC breaking dependencies, we can group them all in this changeset.
Mon, 14 Nov 2022 17:17:04 +0100 hg-core: upgrade all remaining dependencies
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 17:17:04 +0100] rev 49756
hg-core: upgrade all remaining dependencies Finally, these dependencies do not require any code changes, so they are all grouped in the same changeset.
Mon, 14 Nov 2022 17:14:20 +0100 hg-core: upgrade `clap` dependency
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 17:14:20 +0100] rev 49755
hg-core: upgrade `clap` dependency Upgrading is a finally possible now that we're supporting a more recent version of Rust. This required changes in the `nodemap` example.
(0) -30000 -10000 -3000 -1000 -300 -100 -96 +96 +100 +300 +1000 tip