Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 03:50:28 +0100] rev 49722
path: introduce a `get_unique_pull_path_obj` function
Unlike the previous one, `get_unique_pull_path`, this function return the `path`
object, opening more option for the caller.
note that this highlight we don't actually need the `repo` argument to
`get_pull_paths`, however changing the API would be annoying for third party
extensions.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 01:55:05 +0100] rev 49721
path: simplify the `get_unique_pull_path` function
Simply delegate the search to `get_pull_paths` and check how many we got.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 01:41:27 +0100] rev 49720
path: remove outdated documentation point from `get_unique_push_path`
This is no longer true.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 18:41:59 +0100] rev 49719
path: pass `path` to `peer` in `hg pull`
We directly use the `path` object to build the `peer` object.
We don't use it for sub-repositories yet, which will have to be fixed at some
point.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 18:19:08 +0100] rev 49718
path: pass `path` to `peer` in `hg incoming`
We directly use the `path` object to build the `peer` object.
We don't use it for sub-repositories yet, which will have to be fixed at some
point.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 17:55:17 +0100] rev 49717
path: pass `path` to `peer` in `hg incoming` bookmark logic
We directly use the `path` object to build the `peer` object.
Touching this code highlighted that we never honor the branches' information
of when doing bookmark level incoming checks.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 16:58:22 +0100] rev 49716
path: remove outdated documentation point from `get_unique_pull_path`
This is no longer true.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 16:53:22 +0100] rev 49715
path: update `get_unique_pull_path` to point out it returns a url
Unlike other functions in this module it returns a url as `bytes` and
not a `path` object. Let us point it out in the doc.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 02 Dec 2022 02:03:49 +0100] rev 49714
changelog-v2: fix the docket `struct`
The previous definition used `L` which is actually 4 bytes, while the
documentation and intend was to use `Q`, i.e. 8 bytes.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:26:34 +0100] rev 49713
path: pass `path` to `peer` in infinite push
We directly use the `path` object to build the `peer` object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:21:18 +0100] rev 49712
path: pass `path` to `peer` in `hg histedit`
We directly use the `path` object to build the `peer` object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:14:40 +0100] rev 49711
path: pass `path` to `peer` in the `outgoing` revset
We directly use the `path` object to build the `peer` object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:11:21 +0100] rev 49710
path: pass `path` to `peer` in `hg summary`
We directly use the `path` object to build the `peer` object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 02:09:43 +0100] rev 49709
path: pass `path` to `peer` in `hg outgoing`
We directly use the `path` object to build the `peer` object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:57:14 +0100] rev 49708
path: pass `path` to `peer` in `hg bundle`
We directly use the `path` object to build the `peer` object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 19:43:26 +0100] rev 49707
path: have `peer` constructor accept a `path` object
We don't do anything fancy with it yet, but this is an important step towards
having the peers aware of their "source" and associated configurations.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:46:46 +0100] rev 49706
path: deprecated the `pushloc` attribute
We want to make sure people with use the full featured path "variant".
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:41:34 +0100] rev 49705
path: update logic in `perf` to use the push variant when available
The command seems currently broken, but at least it won't be broken by us !
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:38:33 +0100] rev 49704
path: directly use the push_variant in `infinitepush`
We don't need any extra processing now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:38:07 +0100] rev 49703
path: directly use the push_variant in `hg histedit` outgoing logic
We don't need any extra processing now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:37:41 +0100] rev 49702
path: directly use the push_variant in the `outgoing` revset
We don't need any extra processing now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:37:10 +0100] rev 49701
path: directly use the push_variant in outgoing internals
We don't need any extra processing now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:35:17 +0100] rev 49700
path: directly use the push_variant in `hg summary`
We don't need any extra processing now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:34:58 +0100] rev 49699
path: directly use the push_variant in `hg outgoing`
We don't need any extra processing now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:34:26 +0100] rev 49698
path: directly use the push_variant in `hg push`
We don't need any extra processing now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:33:27 +0100] rev 49697
path: have `get_push_paths` directly return the push variants
So the function directly returns usable paths, that should help the callers!
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:32:24 +0100] rev 49696
path: add a method to retrieve a "push variant" of a path
This gets you the same path, but using the `pushurl` as destination.
This opens the way for a lot of different improvements, the one which
interests us is having `peer` objects aware of the `path` they came from.
Anton Shestakov <av6@dwimlabs.net> [Thu, 01 Dec 2022 18:01:24 +0400] rev 49695
tests: use an all too familiar executable in test-run-tests.t (
issue6661)
true(1) sometimes lives in /usr/bin/, and so this test fails on such systems.
We also can't use which(1), because it's apparently not POSIX and Debian
complains about it [1]. We also cannot really create a script and chmod +x it,
because this is a symlink case, execbit case is slightly below. So let's use
something that we know is executable, but not hg itself.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 01 Dec 2022 01:27:47 +0100] rev 49694
path: move the url parsing and related attribute setting to a method
This will make it simpler to reuse this logic in the next changeset.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:22:18 +0100] rev 49693
peer-or-repo: remove the now unused function
We do not need it anymore.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:21:19 +0100] rev 49692
peer-or-repo: build a repo directly in the `repo` function
We skip the ambiguous _peerorrepo function to explicitly build a repo within
the dedicated function. The peer scheme are therefore no longer considered to
build the object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:04:23 +0100] rev 49691
peer-or-repo: build a peer directly in the `peer` function
We skip the ambiguous _peerorrepo function to explicitly build a peer within
the dedicated function. This mean explicitly getting a peer from an explicitly
create repository when necessary.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 22:03:10 +0100] rev 49690
static-http: have `statichttprepo.instance` return a peer object
It previously returned a statichttprepo object which could not be used for
anything but creating a peer. So lets put it into the peer bucket with a peer
behavior.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 12:22:02 +0100] rev 49689
scheme: move the drive letter checking in its own function
This help the readability of the main function as is was taking much more room
than the main logic.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 21:48:08 +0100] rev 49688
peer-or-repo: split the scheme between repo and peer
Some of the scheme will always produce a peer and some will always produce a
repository. So lets use different mapping to reduce the ambiguity.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 13:55:15 +0100] rev 49687
peer-or-repo: stop relying on AttributeError in `islocal`
This will confused pytypes in a future changeset.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 21:42:08 +0100] rev 49686
repo-or-peer: deprecate calling `islocal` on non-path object
There object have a `.local()` method and should use it.
Anton Shestakov <av6@dwimlabs.net> [Thu, 01 Dec 2022 15:27:11 +0400] rev 49685
tests: add the missing space to test-hghave.t (
issue6762)
log() in run-tests.py add a space to the end of all messages that it prints,
and I missed this fact in my previous patch.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 30 Nov 2022 11:12:48 +0100] rev 49684
share: stop using 'islocal' with repo instance
Having this level of polymorphism of the `islocal` function is weird, and we
already have a way to know if the repo is local from the object itself.
We are about to deprecate passing a non-bytes object to `islocal`, so clean this
up beforehand.
We might want to clean up this function too in the future, however this is
another adventure.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 19:54:55 +0100] rev 49683
peer-or-repo: make sure object in "scheme" have a `instance` object
The previous form of having heterogeneous object in the dictionnary makes things
more complicated than they needed to be. I am not super happy about the current
(especially around 'islocal', that most item do not have), but this is already
much better.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 29 Nov 2022 18:30:54 +0100] rev 49682
peer-or-repo: move the object setup in its own function
The `_peerorrepo` function is problematic, because it can build different types
of object (repository and peer). This make it hard to adjust the arguments to
the type of object we needs. So this patch start a series of change to create
peer and repo without going through a common function.
We move the part of the function doing object setup it its own function to make
it simpler to reuse in others contexts.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 17:53:17 -0500] rev 49681
delta-find: use a single snapshot cache when applying a group to an object
This will avoid walking the revlog over and over again in some situations.
The difference is hard to show in our current benchmark suite, as the gain is
lower than their overall instability.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 17:55:55 -0500] rev 49680
delta-find: make sure we only use newer full snapshot as candidate
The current code does not needs to protect against this, as there are no older
snapshot in the current cache. However as we are getting ready to reuse this
cache from one revision to another, we need the code to protect itself about
what's coming.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 17:55:46 -0500] rev 49679
delta-find: use sets instead of list in the snapshot cache
This seems more appropriate.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 16:56:23 -0500] rev 49678
delta-find: use a smarter object for snapshot caching
This open the way for a longer lived cache.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 22:12:59 -0500] rev 49677
find-delta: pass the cache-delta usage policy alongside the cache-delta
The idea is to give higher level code more control to what will happens with
the cache delta passed. This should help with controling how we treat delta's
from different sources.
The final goal of this change is to allow for server modes where the client can
blindly accept any server delta without regards to any local constraints. This
will be implemented in later changesets.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 18:58:35 +0100] rev 49676
find-delta: move most of the debug-find-delta code in the debug module
Lets us that module more. It will help us to keep revlog implementation details
close to each other.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 20:02:32 -0500] rev 49675
find-delta: minor preparatory change
We are about to add more item in the cachedelta object, so lets access the item by index instead of doing a full extensions.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 17:57:28 -0500] rev 49674
find-delta: rename _isgooddeltainfo
Lets move to a more readable name now that we are allowed to. This cannot hurt.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 18:06:17 -0500] rev 49673
test-revlog-raw: drop the overwrite of dead code
The revlog class no longer have a _isgooddeltainfo method for a long time. So
overwriting it does not get us anything. The test have been wrapping the right
code since then anyway.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 12 Nov 2022 00:18:41 +0100] rev 49672
emitrevision: consider ancestors revision to emit as available base
This should make more delta base valid. This notably affects:
* case where we skipped some parent with empty delta to directly delta against
an ancestors
* case where an intermediate snapshots is stored.
This change means we could sent largish intermediate snapshots over the wire.
However this is actually a sub goal here. Sending snapshots over the wire means
the client have a high odd of simply storing the pre-computed delta instead of
doing a lengthy process that will… end up doing the same intermediate snapshot.
In addition the overall size of snapshot (or any level) is "only" some or the
overall delta size. (0.17% for my mercurial clone, 20% for my clone of Mozilla
try). So Sending them other the wire is unlikely to change large impact on the
bandwidth used.
If we decide that minimising the bandwidth is an explicit goal, we should
introduce new logic to filter-out snapshot as delta. The current code has no
notion explicite of snapshot so far, they just tended to fall into the wobbly
filtering options.
In some cases, this patch can yield large improvement to the bundling time:
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-bundle
# benchmark.variants.revs = last-100000
before: 68.787066 seconds
after: 47.552677 seconds (-30.87%)
That translate to large improvement to the pull time :
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = pull
# benchmark.variants.
issue6528 = disabled
# benchmark.variants.revs = last-100000
before: 142.186625 seconds
after: 75.897745 seconds (-46.62%)
No significant negative impact have been observed.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 09 Nov 2022 13:54:15 -0500] rev 49671
sqlitestore: add an `ancestors` method
We will need it during bundling.
The implementation mirror the one in revlog.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 24 Nov 2022 04:04:19 +0100] rev 49670
emitrevision: if we need to compute a delta on the fly, try p1 or p2 first
Falling back to `prev` does not yield any real value on modern storage and
result in pathological changes to be created on the other side. Doing a delta
against a parent will likely be smaller (helping the network) and will be safer
to apply on the client (helping future pulls by Triggering intermediate
snapshop where they will be needed by later deltas).
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 16:27:23 +0100] rev 49669
emitrevision: simplify the fallback to computed delta
Not using the stored delta, or having a full snapshot on disk behave the same
ways, so lets use the same code path for that, this is simpler, and it update
will be simpler.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:59:52 +0100] rev 49668
emitrevision: also check the parents in the availability closure
One of the point of having a closure is to gather the logic in it. So we gather
the logic.
The `parents[:]` part is a bit ugly but will be replaced by better code soon
anyway.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 28 Nov 2022 15:48:51 +0100] rev 49667
emitrevision: add a small closure to check if a base is usable
We will make more use of this and make it more complex too.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 Oct 2022 16:26:16 +0200] rev 49666
reuse-delta-base: improves some documentation
The current code got me a bit confused initially. So a bit more documentation
around it cannot hurt.
Anton Shestakov <av6@dwimlabs.net> [Mon, 28 Nov 2022 16:10:30 +0400] rev 49665
tests: expect the message from
1baf0fffd82f in test-hghave.t (
issue6762)
Couldn't reproduce locally, but apparently this message may occasionally pop up
when running this test.
Raphaël Gomès <rgomes@octobus.net> [Fri, 25 Nov 2022 15:14:40 +0100] rev 49664
branching: merge stable into default
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 22 Nov 2022 12:44:22 +0100] rev 49663
changelog-v2: add a configuration to disable rank computation
We encountered a graph where rank computation was pathologically slow. We add an
option to disable this computation while this is getting fixed.
Disabling the rank computation should allow for testing other changelog-v2
features undisturbed (like changeset-based copy tracing).
I am purposely not adding a test for the new non-default code path, as this is
a temporary work around of an experimental feature.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 21 Nov 2022 15:04:19 +0100] rev 49662
debugrevlog: display total stored information
This is an interesting statistis, so let's display it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 14:38:52 -0500] rev 49661
debug-revlog: details about non-ancestors delta-bases
Deltas against a base that is not an ancestor of the revision that owns this
delta are notable.
For example, they introduce complexity during the bundling process as the base
might not exist on the unbundling side.
We detect them in `hg debugrevlog` and print information about them.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 14:24:52 -0500] rev 49660
debug-revlog: move the code in revlogutils module
We have a module dedicated to debug code, let us use it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 07 Nov 2022 14:13:59 -0500] rev 49659
debug-revlog: move the --dump code in `revlogutils` module
We have a module dedicated to debug code, let us use it.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 23 Nov 2022 19:08:27 +0100] rev 49658
delta-find: set the default candidate chunk size to 10
I ran performance and storage tests on repositories of various sizes and shapes
for the following values of the config : 5, 10, 20, 50, 100, no-chunking
The performance tests do not show any statistical impact on computation
times for large pushes and pulls.
For searching for an individual delta, this can provide a significant
performance improvement with a minor degradation of space-quality on the
result. (see data at the end of the commit).
For overall store size, the change :
- does not have any impact on many small repositories,
- has an observable, but very negligible impact on most larger repositories.
- One private repository we use for testing sees a small increase in size
(1%) in the narrower version.
We will try to get more numbers on a larger version of that repository to
make sure nothing pathological happens.
We pick "10" as the limit as "5" seems a bit more risky.
There are room to improve the current code, by using more aggressive filtering
and better (i.e any) sorting of the candidates. However this is already a large
improvement for pathological cases, with little impact in the common
situations.
The initial motivation for this change is to fix performance of delta
computation for a file where the previous code ended up testing 20 000 possible
candidate-bases in one go, which is… slow. This affected about ½ of the file
revisions leading to atrocious performance, especially during some push/pull
operations.
Details about individual delta finding timing:
----------------------------------------------
The vast majority of benchmark cases are unchanged but the three below. The first
two do not see any impact on the final delta. The last one sees a change in
delta-size that is negligible compared to the full text size.
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-delta-find
# benchmark.variants.rev = manifest-snapshot-many-tries-a (revision 756096)
∞: 5.844783
5: 4.473523 (-23.46%)
10: 4.970053 (-14.97%)
20: 5.770386 (-1.27%)
50 5.821358
100: 5.834887
MANIFESTLOG: rev = 756096: (no-limit)
delta-base = 301840
search-rounds = 6
try-count = 60
delta-type = snapshot
snap-depth = 7
delta-size = 179
MANIFESTLOG: rev=756096: (limit = 10)
delta-base=301840
search-rounds=9
try-count=51
delta-type=snapshot
snap-depth=7
delta-size=179
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-delta-find
# benchmark.variants.rev = manifest-snapshot-many-tries-d (revision 754060)
∞: 5.017663
5: 3.655931 (-27.14%)
10: 4.095436 (-18.38%)
20: 4.828949 (-3.76%)
50 4.987574
100: 4.994889
MANIFESTLOG: rev=754060: (no limit)
delta-base=301840
search-rounds=5
try-count=53
delta-type=snapshot
snap-depth=7
delta-size = 179
MANIFESTLOG: rev=754060: (limite = 10)
delta-base=301840
search-rounds=8
try-count=45
delta-type=snapshot
snap-depth=7
delta-size = 179
### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
# benchmark.name = perf-delta-find
# bin-env-vars.hg.flavor = rust
# benchmark.variants.rev = manifest-snapshot-many-tries-e (revision 693368)
∞: 4.869282
5: 2.039732 (-58.11%)
10: 2.413537 (-50.43%)
20: 4.449639 (-8.62%)
50 4.865863
100: 4.882649
MANIFESTLOG: rev=693368:
delta-base=693336
search-rounds=6
try-count=53
delta-type=snapshot
snap-depth=6
full-test-size=131065
delta-size=199
MANIFESTLOG: rev=693368:
delta-base=278023
search-rounds=5
try-count=21
delta-type=snapshot
snap-depth=4
full-test-size=131065
delta-size=278
Raw data for store size (in bytes) for various chunk size value below:
----------------------------------------------------------------------
440 134 384 5 pypy/.hg/store/
440 134 384 10 pypy/.hg/store/
440 134 384 20 pypy/.hg/store/
440 134 384 50 pypy/.hg/store/
440 134 384 100 pypy/.hg/store/
440 134 384 ... pypy/.hg/store/
666 987 471 5 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 10 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 20 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 50 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 100 netbsd-xsrc-2022-11-15/.hg/store/
666 987 471 ... netbsd-xsrc-2022-11-15/.hg/store/
852 844 884 5 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 10 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 20 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 50 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 100 netbsd-pkgsrc-2022-11-15/.hg/store/
852 844 884 ... netbsd-pkgsrc-2022-11-15/.hg/store/
1 504 227 981 5 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 871 10 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 20 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 50 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 100 netbeans-2018-08-01-sparse-zstd/.hg/store/
1 504 227 813 ... netbeans-2018-08-01-sparse-zstd/.hg/store/
3 875 801 068 5 netbsd-src-2022-11-15/.hg/store/
3 875 696 767 10 netbsd-src-2022-11-15/.hg/store/
3 875 696 757 20 netbsd-src-2022-11-15/.hg/store/
3 875 696 653 50 netbsd-src-2022-11-15/.hg/store/
3 875 696 653 100 netbsd-src-2022-11-15/.hg/store/
3 875 696 653 ... netbsd-src-2022-11-15/.hg/store/
4 531 441 314 5 mozilla-central/.hg/store/
4 531 435 157 10 mozilla-central/.hg/store/
4 531 432 045 20 mozilla-central/.hg/store/
4 531 429 119 50 mozilla-central/.hg/store/
4 531 429 119 100 mozilla-central/.hg/store/
4 531 429 119 ... mozilla-central/.hg/store/
4 875 861 390 5 mozilla-unified/.hg/store/
4 875 855 155 10 mozilla-unified/.hg/store/
4 875 852 027 20 mozilla-unified/.hg/store/
4 875 848 851 50 mozilla-unified/.hg/store/
4 875 848 851 100 mozilla-unified/.hg/store/
4 875 848 851 ... mozilla-unified/.hg/store/
11 498 764 601 5 mozilla-try/.hg/store/
11 497 968 858 10 mozilla-try/.hg/store/
11 497 958 730 20 mozilla-try/.hg/store/
11 497 927 156 50 mozilla-try/.hg/store/
11 497 925 963 100 mozilla-try/.hg/store/
11 497 923 428 ... mozilla-try/.hg/store/
10 047 914 031 5 private-repo
9 969 132 101 10 private-repo
9 944 745 015 20 private-repo
9 939 756 703 50 private-repo
9 939 833 016 100 private-repo
9 939 822 035 ... private-repo
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 14:47:17 -0500] rev 49657
delta-find: add a way to control the number of bases tested at the same time
See inline comment for details.
The feature is currently disabled, but should be enabled by default to mitigate
some existing pathological cases.
Also see the next changeset for details.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 24 Nov 2022 10:34:34 +0100] rev 49656
test: adjust test-push-race.t timeout's to overall test timeout
The generic `tests/testlib/wait-on-file` mechanism scale its timeout with the
value of `HGTEST_TIMEOUT`, the `delaypush.py` in `test-push-race.t` is not doing
this, and we have been seeing more and more timeout from loaded CI worker
lately.
Adding this timeout scaling should help with that.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 21:11:46 -0500] rev 49655
setup: include vendored 3rd party type stubs
While pytype may not support PEP 561, PyCharm does, so having the stubs
available means it can determine `foo = attr.ib(type=int)` means `foo` is an
int. This only applies when using Mercurial as a library, like with TortoiseHg
development- PyCharm is already smart enough to use the *.pyi files in the
Mercurial source tree when hacking on Mercurial itself.
I left the mercurial.cext stubs out because it seems very low level, that 3rd
parties shouldn't be using directly.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:59:53 -0500] rev 49654
ci: bump pytype to 2022.11.18
No particular reason, other than the current build is fairly old. It flagged a
few more things (that weren't errors based on the logic around them), but OTOH,
some of the pyi stubs it generates are less specific.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:56:22 -0500] rev 49653
ci: run the script to add vendored type stubs to typeshed
Since CI runs from docker images, it doesn't matter that this mucks with the
typeshed bundled with pytype.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:50:39 -0500] rev 49652
contrib: add a script for adding vendored type stubs to typeshed
I really hate this, but pytype doesn't support PEP 561 and doesn't seem to have
the equivalent of `MYPYPATH` to point to custom stubs. Ignoring the vendored
stubs isn't necessarily harmful, but pytype has been choking on the vendored
attr package after pytype 2022.03.29 with errors like this:
File "/mnt/c/Users/Matt/hg/mercurial/linelog.py", line 52, in __iter__: Built-in function iter was called with the wrong arguments [wrong-arg-types]
Expected: (collection: bytearray)
Actually passed: (collection: mercurial.thirdparty.attr._make._CountingAttr)
File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 143, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types]
Expected: (obj: Sized)
Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr)
Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__
File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 144, in pack: No attribute 'rfind' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error]
File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 146, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types]
Expected: (obj: Sized)
Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr)
Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__
File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 152, in pack: No attribute 'v2_data' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error]
File "/mnt/c/Users/Matt/hg/mercurial/util.py", line 2817, in go: unsupported operand type(s) for /: 'count: mercurial.thirdparty.attr._make._CountingAttr' and 'float: float' [unsupported-operands]
No attribute '__truediv__' on 'count: mercurial.thirdparty.attr._make._CountingAttr' or '__rtruediv__' on 'float: float'
Called from (traceback):
line 2981, in __bytes__
This is essentially the same hack we've been using in TortoiseHg to add the
vendored PyQt5 stubs. What I don't understand is pytype *still* generates *.pyi
files under .pytype/pyi/mercurial/thirdparty/attr, even when the package is
explicitly ignored in the pytype command line args. But it avoids the errors,
which means we aren't stuck on pytype==2022.03.29.
https://github.com/google/pytype/issues/151
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 20:23:26 -0500] rev 49651
contrib: update check-pytype.sh to list stubs that caused pytype to crash
The same logic is in the TortoiseHg tests for running pytype, and it's useful to
know if a new version of pytype is better or worse.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 16:11:20 -0500] rev 49650
typing: add py.typed to mercurial.cext for PEP 561 support
Unfortunately, pytype doesn't support this yet. But it was included with the
attr package, so we might as well do it here for consistency. Unlike the attr
package, these type hints are only partial, so they are marked as such[1] (but
who knows if it matters, given these are C extensions, so no local source code
to scan).
[1] https://peps.python.org/pep-0561/#partial-stub-packages
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 15:50:20 -0500] rev 49649
typing: add missing signature for mercurial.cext.parsers.parse_index2()
Flagged by pytype 2022.11.18 when the cext stubs are made visible to it. There
are very likely other signatures that are missing, but this is enough to keep it
happy for now.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 23 Nov 2022 11:22:22 -0500] rev 49648
typing: minor tweaks to allow updating to pytype 2022.11.18
Raphaël Gomès <rgomes@octobus.net> [Wed, 23 Nov 2022 14:42:11 +0100] rev 49647
python-compat: adapt to Python 3.11 BC breakage with `random.sample`
As per https://docs.python.org/3/whatsnew/3.11.html#porting-to-python-3-11:
"The population parameter of `random.sample()` must be a sequence, and
automatic conversion of sets to lists is no longer supported.
Also, if the sample size is larger than the population size,
a `ValueError` is raised"
Matt Harbison <matt_harbison@yahoo.com> [Sun, 20 Nov 2022 22:54:43 -0500] rev 49646
typing: add type hints to mercurial/help.py
Was hoping to find more issues like
f09bc2ed9100, but it may be that nothing
checks the args to that operation. In any event, the work is done and pytype
doesn't do a very good job inferring the types. A few of th emore complicated
things like the command table are left untyped, because they come from modules
that aren't typed yet.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 22 Nov 2022 11:55:26 -0500] rev 49645
match: make the FLAG_RE pattern a raw string
PyCharm was complaining about invalid escape sequences since this was added
recently in
3eda36e9b3d6.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 20 Nov 2022 23:09:12 -0500] rev 49644
configitems: add a default value for "merge-tools.xxx.regappend"
When trying to figure out how `hg help -v` took the Set interpolation path in
f09bc2ed9100, I turned on devel warnings and noticed this (unrelated) warning:
devel-warn: specifying a mismatched default value for a registered config
item: 'merge-tools.beyondcompare4.regappend' ''
at: c:\Users\Matt\hg\mercurial\filemerge.py:46 (_toolstr)
The previous default value for this config was `None`, but that slightly
complicates the code at the only site it is used, referenced above.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 21 Nov 2022 15:04:42 -0500] rev 49643
attr: vendor 22.1.0
The previous version was 5 years old, and pytype 2022.06.30 started complaining
about various uses (e.g. seeing `mercurial.thirdparty.attr._make._CountingAttr`
instead of `bytearray`). Hopefully this helps. Additionally, this has official
python 3.11 support.
The `attrs` package is left out, because it is simply a bunch of *.pyi stubs and
`from attr.X import *`, and that's not how they've been used up to this point.
We'd probably need to customize those anyway to
`from mercurial.thirdparty.attr import *`.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 21 Nov 2022 16:18:28 -0500] rev 49642
tests: update test-util.py for modern attrs package
When updating to 22.1.0, this test started failing:
Traceback (most recent call last):
File "/tmp/mercurial-ci/tests/test-util.py", line 53, in <module>
_start_default = (util.timedcmstats.start.default, 'factory')
AttributeError: type object 'timedcmstats' has no attribute 'start'
Poking around in `hg debugshell`, the attribute is indeed missing, but looks to
be attached to `__attrs_attrs__` in both the currently vendored and the modern
version of attrs. The old attrs packages will print the same for both accesses,
so fingers crossed...
>>> print((util.timedcmstats.start.default, 'factory'))
(Factory(factory=<function timedcmstats.<lambda> at 0x
000001EFDF0F21F0>, takes_self=False), 'factory')
>>> print((util.timedcmstats.__attrs_attrs__.start.default, 'factory'))
(Factory(factory=<function timedcmstats.<lambda> at 0x
000001EFDF0F21F0>, takes_self=False), 'factory')
Raphaël Gomès <rgomes@octobus.net> [Tue, 15 Nov 2022 01:52:46 +0100] rev 49641
rhg: upgrade the remainder of the dependencies
These are painless, so they are all grouped in this changeset.
Raphaël Gomès <rgomes@octobus.net> [Tue, 15 Nov 2022 00:02:43 +0100] rev 49640
rhg: upgrade `clap` dependency
This one is the worst one to upgrade since v2 -> v4 broke a ton of API,
which thankfully seems saner now.
Contrary to what was done in the `hg-core/src/examples/nodemap` rewrite,
we're not switching from the "builder" pattern to the "derive" pattern,
since that would imply a much larger diff. It can be done incrementally.
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 17:18:56 +0100] rev 49639
hg-cpython: upgrade dependencies
`hg-cpython` has no BC breaking dependencies, we can group them all
in this changeset.
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 17:17:04 +0100] rev 49638
hg-core: upgrade all remaining dependencies
Finally, these dependencies do not require any code changes, so they are
all grouped in the same changeset.
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 17:14:20 +0100] rev 49637
hg-core: upgrade `clap` dependency
Upgrading is a finally possible now that we're supporting a more recent
version of Rust.
This required changes in the `nodemap` example.
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 16:35:57 +0100] rev 49636
hg-core: upgrade `zstd` dependency
Now that we support a newer version of Rust, we can update this dependency
to get all the latest bugfixes and improvements.
A slight API adjustment was needed.
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:43:05 +0100] rev 49635
hg-core: make use of `strip_suffix` now that we're using Rust 1.45+
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:34:51 +0100] rev 49634
rust: use `matches!` macro now that we're using Rust 1.42+
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:31:49 +0100] rev 49633
hg-core: remove unneeded util now that we support Rust 1.42+
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:29:58 +0100] rev 49632
hg-core: remove unneeded trait now that we support Rust 1.52+
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:20:48 +0100] rev 49631
rust: remove newly redundant `use` statements with the 2021 edition prelude
https://doc.rust-lang.org/edition-guide/rust-2021/prelude.html
Raphaël Gomès <rgomes@octobus.net> [Mon, 14 Nov 2022 15:19:27 +0100] rev 49630
rust: move all crates in the main workspace to edition 2021
We've changed our minimum Rust version to 1.61.0 in the previous patch,
and edition 2021 predates that version.
Raphaël Gomès <rgomes@octobus.net> [Thu, 20 Oct 2022 12:26:57 +0200] rev 49629
rust: upgrade supported Rust toolchain version
A few months ago¹, a decision was made to move the Rust toolchain target to
whatever Debian Testing was tracking. I didn't have the bandwidth to act on
it until now.
This is starting to be even more problematic than before, now that edition 2021
is out.
The CI has been updated to track the current Debian testing version, 1.61.0.
[1] https://lists.mercurial-scm.org/pipermail/mercurial-packaging/2022-April/000338.html
Matt Harbison <matt_harbison@yahoo.com> [Sun, 20 Nov 2022 15:55:27 -0500] rev 49628
help: fix a py3 error interpolating Set into b'%s'
I can't reproduce it, but a coworker hit this with `hg help -v` with 6.2.3:
...
File "mercurial\help.pyc", line 865, in helplist
TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'set'
I can confirm that the original expression fails in `hg debugshell`, and the new
one works. The second instance was found by searching for "%s", but PyCharm
detects a lot of variables as Any type, so I have no idea if there are other
lurking problems.
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 20:40:47 +0100] rev 49627
branching: merge stable into default
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 16:14:20 +0100] rev 49626
Added signature for changeset
c890d8b8bc59
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 16:14:08 +0100] rev 49625
Added tag 6.3.1 for changeset
c890d8b8bc59
Raphaël Gomès <rgomes@octobus.net> [Sat, 19 Nov 2022 16:00:39 +0100] rev 49624
relnotes: add 6.3.1
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 19 Nov 2022 16:43:02 +0100] rev 49623
tests: fix test-sparse-revlog
This one is not covered by the CIbecause I requires an expensive artifact to be
cached. So it goes out of think on regular basis (we should fix that…)
The test ouput was affected by
e706bb41fdb3 as we filtering now happens sooner,
removing for the output.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 19 Nov 2022 01:35:01 +0100] rev 49622
memory-usage: fix `hg log --follow --rev R F` space complexity
When running `hg log --follow --rev REVS FILES`, the log code will walk the
history of all FILES starting from the file revisions that exists in each REVS.
Before doing so, it looks if the files actually exists in the target revisions.
To do so, it opens the manifest of each revision in REVS to look up if we find
the associated items in FILES.
Before this changeset this was done in a way that created a changectx for
each target revision, keeping them in memory while we look into each file.
If the set of REVS is large, this means keeping the manifest for each entry in
REVS in memory. That can be large… if REV is in the form `::X`, this can quickly
become huge and saturate the memory. We have seen usage allocating 2GB per
second until memory runs out.
So this changeset invert the two loop so that only one revision is kept in
memory during the operation. This solve the memory explosion issue.
Arseniy Alekseyev <aalekseyev@janestreet.com> [Fri, 18 Nov 2022 13:47:29 +0000] rev 49621
tests: run many tests in $TESTTMP/repo instead of $TESTTMP
This is useful so we can store other files in $TESTTMP
(in particular tests that use docket files (nodemap, dirstate-v2) keep
file uids in $TESTTMP/UID)
Arseniy Alekseyev <aalekseyev@janestreet.com> [Fri, 18 Nov 2022 13:52:18 +0000] rev 49620
tests: fix the detection of dirstate-v2 in hghave.py
Arseniy Alekseyev <aalekseyev@janestreet.com> [Thu, 17 Nov 2022 14:37:43 +0000] rev 49619
dirstate-v2: do not put the dirstate data file in a transaction,
since the transaction reverts the store, while the dirstate is stored separately
Matt Harbison <matt_harbison@yahoo.com> [Fri, 18 Nov 2022 13:43:03 -0500] rev 49618
commit: properly consider file include and exclude options when closing branch
It looks like this is meant to prevent adding another commit that does nothing
but close a branch on top of a commit that already closed the branch. The
matcher building functions want `Dict[bytes, Any]`, not `Dict[str, Any]`, which
was found by adding type hints to the matcher related methods in scmutil.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 18 Nov 2022 14:03:56 -0500] rev 49617
tests: demonstrate a bug blocking a redundant branch close
Arseniy Alekseyev <aalekseyev@janestreet.com> [Thu, 17 Nov 2022 16:31:52 +0000] rev 49616
tests: stop creating temporary files in TESTDIR
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 15:03:31 -0500] rev 49615
delta-find: adjust the moment when we mark something as "tested"
In a coming change, not all elements of `group` might get tested. So we need to
have more control about when a revision is actually added to the `tested` set.
So we move to a more verbose (and more fragile) version.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 12:53:57 -0500] rev 49614
delta-find: rename a variable for clarity
the index in the delta-chain is also the snapshot depth. So we rename the
variable for clarity.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 12:53:03 -0500] rev 49613
delta-find: small documentation update
This is not a 1-1 mapping, but a 1-n mapping. Lets make the associated comment clearer.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 12:51:50 -0500] rev 49612
delta-find: move pre-filtering with other pre-filtering logic
This is more consistent and will help use to be in a clean state before dealing
with the "too large group" issue.
As a side effect, the debug output now skip some useless cases, making it more useful.
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 06 Nov 2022 13:46:08 -0500] rev 49611
delta-find: expand a function definition and call before extendin it
This make the next changeset more compact.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 17 Oct 2022 22:19:43 +0200] rev 49610
debug: add an option to display statistic about a unbundling operation
This will helps a lot to understand how the bundling decision have actually
impacted pull/unbundle on the other side.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Nov 2022 16:25:23 +0100] rev 49609
debug: add an option to display statistic about a bundling operation
This will helps a lot to understand how the bundling decision might impact
pull/unbundle on the other side.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Nov 2022 18:08:56 +0100] rev 49608
delta-find: add debug information about reuse of cached data
This will help us to understand the behavior of find-delta during a pull.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 08 Nov 2022 18:05:19 -0500] rev 49607
cffi: fix a bytes vs str issue on macOS when listing directories
This code hasn't been touched in recent years, and the other implementation
return bytes for the filename, so I assume this is a holdover from the py2 days.
I was unable to test it on mac though, because the `_osutil` import failed.
Jason R. Coombs <jaraco@jaraco.com> [Wed, 02 Nov 2022 12:54:12 -0400] rev 49606
packaging: refresh dependency hashes (
issue6750)
Also, add some documentation to the `.in` files.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Nov 2022 15:39:10 +0100] rev 49605
matcher: do not prepend '.*' to pattern using ^ after flags
Since the previous commit (fixing wider issue), the code generated strange
regex. This is now fixed and tested.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Nov 2022 16:38:42 +0100] rev 49604
matcher: fix the issue with regex inline-flag in rust oo
Same problem same solution.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Nov 2022 13:05:01 +0100] rev 49603
matcher: fix issues regex flag contained in pattern (
issue6759)
Python 3.11 is now enforcing that flag must be at the beginning of the regex
This creates a serious regression for people using Python 3.11 with an hgignore
using flag in a "relre" pattern.
We now detect any flags in such pattern and "prepend" our ".*" pattern after them.
In addition, we now insert the flag in the regexp to only affect the pattern we
are rewriting. Otherwise, the regex built from the combined pattern would these
flags in the middle of it anyway.
As a side effect of this last change, we fix a bug… before this change regex
flag in a pattern would affect all combined patterns. That was bad and is not
longer the case.
The Rust code needs to be updated to fix that very bug, but we will do it in
another changeset.