Raphaël Gomès <rgomes@octobus.net> [Mon, 06 Nov 2023 15:38:15 +0100] rev 51116
Added tag 6.5.3 for changeset
c083d9776cb2
Raphaël Gomès <rgomes@octobus.net> [Mon, 06 Nov 2023 15:32:30 +0100] rev 51115
relnotes: add 6.5.3
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 14 Oct 2023 03:24:13 +0200] rev 51114
revlog: avoid opening and closing the file for each cloned revision
The previous code was flushing files after each new revision, slowing things
down.
For exemple, with this change, the evolve repository can run
`hg debugupgraderepo --run --optimize re-delta-parent` in about 3.4s instead of
4.5 seconds.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 13 Oct 2023 23:21:46 +0200] rev 51113
censor: accept censored revision during upgrade
They can simply be passed by as censored.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 13 Oct 2023 22:40:10 +0200] rev 51112
censor: show that censored revision prevent repository upgrade
This is not great.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 31 Oct 2023 22:42:46 -0700] rev 51111
smartset: don't ignore hidden revs when intersecting
This fixes the bug I demonstrated in the previous commit, but I'm not sure at
all if it's the right way of doing it.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 31 Oct 2023 22:33:45 -0700] rev 51110
tests: demonstrate crash in `unstable()` with internal-phase orphans
Raphaël Gomès <rgomes@octobus.net> [Wed, 18 Oct 2023 14:50:14 +0200] rev 51109
rust-matchers: fix quadratic complexity in `FileMatcher`
Concretely, this command:
```
$ echo hg up -r <nodeid>; time hg revert dir1 dir2 -r <othernode> --debug
hg up -r <nodeid>
real 0m14.690s
user 0m14.766s
sys 0m5.430s
```
was much slower despite using 16 cores before this change.
The approach taken here is the same one used in match.py, in exactmatcher.
This changeset was originally written by Valentin Gatien-Baron in a private
repository.
I have redacted the commit message and did a minor clean up of the code.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 27 Oct 2023 08:54:41 +0200] rev 51108
revlog: add a small cache of unfiltered chunk
This can provides a massive boost to the reading of multiple revision and the
computation of a valid delta chain.
This greatly help operation like `hg log --patch`, delta computation (helping
pull/unbundle), linkrev adjustment (helping copy tracing).
A first round of benchmark for `hg log --patch --limit 1000` shows improvement
in the 10-20% range on "small" repository like pypy or mercurial and large
improvements (about 33%) for more complex ones like netbeans and mozilla's.
These speeds up are consistent with the improvement to `hg pull` (from a server
sending poor deltas) I saw benchmarking this last year. Further benchmark will
be run during the freeze.
I added some configuration in the experimental space to be able to further test
the effect of various tuning for now. This feature should fit well in the
"usage/resource profile" configuration that we should land next cycle.
When it does not provides a benefit the overhead of the cache seem to be around
2%, a small price for the big improvement. In addition I believe we could shave
most of this overhead with a more efficent lru implementation.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 27 Oct 2023 02:57:09 +0200] rev 51107
revlog: minor refactor in the chunk gather process
We will introduce some caching in this method in the next changeset, we make
some of the most "disruptive" change first as touching this could break (and
maybe did during the development process).
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 24 Oct 2023 11:08:49 +0200] rev 51106
changelog-delay: move the delay/divert logic inside the (inner) revlog
Instead of hacking throught the vfs/opener, we implement the delay/divert logic
inside the `_InnerRevlog` and `randomaccessfile` object. This will allow to an
alternative implementation of the `_InnerRevlog` that does not need to use Python details.
As a result, the new implementation can use the transaction less agressively
and avoid some extra output since no data had been written yet. That seems like
a good side effect.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 26 Oct 2023 05:37:37 +0200] rev 51105
revlog: add a `canonical_index_file` attribute on inner revlog
This is currently the same and the index_file but it will become more complex
when we start doing delayed write.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 26 Oct 2023 03:41:58 +0200] rev 51104
changelog-delay: move "delayed" check to a more official API
To avoid reaching inside the inner object in the future, we needs some official
API. We put one of such API early to reduce the size of the final diff.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 26 Oct 2023 03:29:46 +0200] rev 51103
changelog-delay: move the appender class next to randomaccessfile
We want to move the delay/divert logic at the revlog level (to have all IO
related logic in the _InnerRevlog) we start with small piece that are easy to
move on their own.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 20 Oct 2023 12:13:33 +0200] rev 51102
changelog-delay: adds some check around delaying and diverting write
Theses assert shows we never call delay or divert if the revlog hold file handle
on the revlog's file.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Oct 2023 23:14:20 +0200] rev 51101
revlog: consolidate cache invalidation within the inner objet
The invalidation of data that belong to the inner object can and should be
delegated to the inner object.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 24 Oct 2023 11:06:04 +0200] rev 51100
shelve: drop some weird manually stripping before transaction abort
I cannot find a case where this is actually useful, so lets triggering strip in
the middle of uncommitted transaction.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 05:19:55 +0200] rev 51099
revlog: move entry writing in the inner object
This is the second big piece of API we want to live on the inner object. With
this the inner object have freedom it needs to implement things internally.
(except for the delayed write feature of the changelog, that will be migrated in coming changesets)
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 04:11:39 +0200] rev 51098
revlog: move `sidedata` in the inner object
Same object as rawtext, but for sidedata.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 03:52:31 +0200] rev 51097
revlog: move the `rawtext` method on the inner object
This is one of the current core goal of the inner object: having a "simple" API
for some of the core revlog action, like getting a raw text, with the inner
object able to do that part internally, on its own.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 03:33:59 +0200] rev 51096
revlog: move the_revisioncache on the inner object
The goal for this inner object is to compute that things we cache, it make more
sense to have the inner object handle it directly.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 03:07:39 +0200] rev 51095
revlog: move the `deltachain` method on the inner object
This is a necessary step before being able to move more logic around restoring
a revision content there.
For now, we do a simple patch for the perf extension logic, when the
implementation of the inner object changes, we will likely need some evolution
of the API. However this is true of many things in the perf extension. So we
will see this later.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 03:00:58 +0200] rev 51094
revlog: move the `_chunks` method on the inner object
This is a necessary step before being able to move more logic around restoring
a revision content there.
For now, we do a simple patch for the perf extension logic, when the
implementation of the inner object changes, we will likely need some evolution
of the API. However this is true of many things in the perf extension. So we
will see this later.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Oct 2023 04:40:40 +0200] rev 51093
revlog: add a couple more of useful method on the inner object
This will be needed for the next changeset.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 02:57:05 +0200] rev 51092
revlog: move the `_chunk` method on the inner object
This is a necessary step before being able to move more logic around restoring
a revision content there.
For now, we do a simple patch for the perf extension logic, when the
implementation of the inner object changes, we will likely need some evolution
of the API. However this is true of many things in the perf extension. So we
will see this later.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Oct 2023 02:13:18 +0200] rev 51091
revlog: move the compression/decompression logic on the inner object
This is a necessary step before being able to move more logic around restoring
a revision content there.
For now, we do a simple patch for the perf extension logic, when the
implementation of the inner object changes, we will likely need some evolution
of the API. However this is true of many things in the perf extension. So we
will see this later.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 23 Oct 2023 14:27:07 +0200] rev 51090
revlog: move the splitting-inline-revlog logic inside the inner object
This is another large IO block that we need to move within the inner object if
we want's it to be self sufficient.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 25 Oct 2023 01:02:47 +0200] rev 51089
revlog: synchronise the various attribute holding the index filename
The segmentfile and the attribute need to be synchronized, let's enforce that.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 19 Oct 2023 01:50:07 +0200] rev 51088
revlog: drop reference to docket in the inline-splitting code
revlog with a docket do not use inline revlog and do not need to split them. So
we can remove some code handling docket there.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 24 Oct 2023 17:03:27 +0200] rev 51087
revlog: move _getsegmentforrevs on the internal object
See inline documentation for details.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 17 Oct 2023 06:02:33 +0200] rev 51086
revlog: create a iteration of a _InnerRevlog object within the revlog
The goal of this object is to isolate a sub-API that can be implemented by a
compiled object (e.g. Rust). So the boundary of this object will be arbitrary
depending of what can we easily implemented in the Compiled code.
For now, we start simple, and move the code that manage the IO objects in the
inner object. More will come in the coming changesets.
Note: the object definition could live in the different module to thin the
`revlog.py` file, however there are other better candidate for extraction first
and I have enought patch stacked on top of the this one for the split in this
patch not to be worth it. So I leave this to future me.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 17 Oct 2023 05:17:02 +0200] rev 51085
revlog: drop the unused `_chunkcache` attribute
Apparently, some time ago, the chunk cache moved in the randomaccessfile object.