Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 04 May 2021 00:02:48 +0200] rev 47244
revlog: simplify the replace_sidedata_info code
We already know how to serialize and de-serialize and enty. So lets just do that
and modify the entry tuple directly.
This avoid having to duplicated binary operation in complicated code.
Differential Revision: https://phab.mercurial-scm.org/D10641
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 23:59:15 +0200] rev 47243
revlog: use `rev` instead of `i` in replace_sidedata_info
This is clearer.
Differential Revision: https://phab.mercurial-scm.org/D10640
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:35 +0200] rev 47242
revlogv2: also keep track for the size of the "data" file
This is useful to make sure we always start writing at the right location,
without effort.
Differential Revision: https://phab.mercurial-scm.org/D10632
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:25 +0200] rev 47241
revlogv2: track pending write in the docket and expose it to hooks
The docket is now able to write pending data. We could have used a distinct
intermediate files, however keeping everything in the same file will make it
simpler to keep track of the various involved files if necessary.
However it might prove more complicated for streaming clone. This will be dealt
with later.
Note that we lifted the stderr redirection in the test since we no longer suffer
from "unkown working directory parent" message.
Differential Revision: https://phab.mercurial-scm.org/D10631
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:14 +0200] rev 47240
revlog: move the `trypending` logic from the `changelog` to the `revlog`
We move the -reading- logic for the pending's '.a' suffixed index within the
revlog class. This is motivated by the fact the logic could be simpler and
cleaner if directly handled by the revlog docket. Before we can do so, we need
to teach the revlog code about reading "pending" changes. To be honest, we
already needed some special casing of the `.a` postfix, so this does not adds
much complexity.
The logic around -writing- the special '00changelog.i.a' remains in the
`changelog` class. Note that the revlog-v2 logic no longer use this logic.
The only remaining user of the `postfix` argument is the `censor` logic. We
could probably also make the revlog full aware of it (most of the code is
already implemented in revlog anyway) and get rid of the `postfix` argument and
logic. However this is an adventure for another time.
Since we have more information, we add more, paranoid, Programming error in case
we detect such "pending reader" trying to do a read (which does not happens
anyways).
Differential Revision: https://phab.mercurial-scm.org/D10630
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:35:02 +0200] rev 47239
revlogv2: delay the update of the changelog docket to transaction end
This prevent external reader to see the transaction content before it is
commited. However this also prevent the hooks to see the transaction content.
We will fix this in later changesets.
We have to temporarily suppress the error output of the command ran during the
transaction as they sometimes get confused about unknown working directory and
sometimes issue message on std-err in unspecified order.
Differential Revision: https://phab.mercurial-scm.org/D10629
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:52 +0200] rev 47238
revlogv2: track current index size in the docket
This help use to fix transaction safety on repos. See next changesets for details.
Differential Revision: https://phab.mercurial-scm.org/D10628
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:41 +0200] rev 47237
revlogv2: also test transactionality of revlog v2
Without surprise, this is broken :-)
Now that it is tested, we can fix it.
Differential Revision: https://phab.mercurial-scm.org/D10627
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:31 +0200] rev 47236
revlog: add a new test file focussed on testing transactionally issue
What we test here is currently also covered by `test-hooks.t`. However having
our own test file allow for more focussed/deep testing and to cover more variant
when relevant. So we create a new test file.
Differential Revision: https://phab.mercurial-scm.org/D10626
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:21 +0200] rev 47235
revlogv2: store version information in the docket only
Having it duplicated in the index was both useless and a risk of discrepancy.
Differential Revision: https://phab.mercurial-scm.org/D10625
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:11 +0200] rev 47234
revlogv2: introduce a very basic docket file
This is the first stone toward using a docket file in revlogv2. Right now the
docket is very basic and only store the version number (which is -also- stored
into the index file…) and the other files have fixed name. This new
implementation break transactionally… but they are no test checking
transactionally for revlogv2… So I take this as an opportunity to start small.
They are no usage of revlogv2 outside of tests anyway.
The docket keeps the `.i` naming used by previous version index to preserve a
unique entry point. We could decide to use a different name and look it up
first, or to fully rework this in a future "store" version. However that does
not seems necessary right now.
We will re-introduces transactionality (and associated testing…) in a later
changesets.
A long list of TODOs have been added to the relevant comment.
Differential Revision: https://phab.mercurial-scm.org/D10624
Raphaël Gomès <rgomes@octobus.net> [Mon, 17 May 2021 15:05:24 +0200] rev 47233
branching: merge stable into default
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:34:01 +0200] rev 47232
revlogv2: mark revlogv2 as requires a full upgrade
This was overlooked previously.
Differential Revision: https://phab.mercurial-scm.org/D10623
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:46 +0200] rev 47231
revlog: unify flag processing when loading index
The new code use a simple declaration to do centralised processing. This is
clearer, shorter and less error prone. This will be especially useful as we plan
to add a fourth format: changelog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10622
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:35 +0200] rev 47230
revlog: unify checks for supported flag
The new code use a simple declaration to do centralised checking. This is
clearer, shorter and less error prone. This will be especially useful as we plan
to add a fourth format: changelog-v2.
Differential Revision: https://phab.mercurial-scm.org/D10621
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:24 +0200] rev 47229
side-data: drop the associated config and requirements
This is no longer and extensions of revlog v1, but a part of revlog v2. We no
longer needs independent config and requirement management for that.
Differential Revision: https://phab.mercurial-scm.org/D10620
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:14 +0200] rev 47228
revlog: fix capitalisation of an error
We don't start error message with capital letters.
Differential Revision: https://phab.mercurial-scm.org/D10619
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:30:03 +0200] rev 47227
test: no longer directly use the sidedata config and requirements
They are on their way out, so we simply use revlogv2 in these cases.
Some test does not requires explicite request for the revlogv2 format since the
upgrade to a side-data-capable format is impled by
`exp-use-copies-side-data-changeset`.
Differential Revision: https://phab.mercurial-scm.org/D10618
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:52 +0200] rev 47226
sidedata: add a 'side-data' repository feature and use it
Most code don't really care how sidedata support is added, but it needs to know
if it is present. To achieve this. we use the `repo.features` attributes with a
new dedicated features".
Having such centralised information is more robust and will help us to remove the
sidedata requirements in a later changesets.
Differential Revision: https://phab.mercurial-scm.org/D10617
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:41 +0200] rev 47225
statichttp: add the missing `features` attribute
This make statichttp closer to the interface it is "supposed" to follow.
Differential Revision: https://phab.mercurial-scm.org/D10616
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:30 +0200] rev 47224
sidedata: use revlogv2 requirement in the test helper
We are about to drop the requirement dedicated to sidedata (since revlogv2 will
support them unconditionally and previous version will not.).
To prepare this more we adapt the test code to not use that requirements.
Differential Revision: https://phab.mercurial-scm.org/D10615
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:19 +0200] rev 47223
upgrade: display sidedata before upgrade
This is a small gratuitous change that help making the test simpler to
understand on its own.
Differential Revision: https://phab.mercurial-scm.org/D10614
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:29:09 +0200] rev 47222
revlog: determine sidedata support based on the revlog version
Revlog version "2" support sidedata, previous version does not. So lets make the
initialization simpler.
Differential Revision: https://phab.mercurial-scm.org/D10613
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:58 +0200] rev 47221
requirements: no longer drop `generaldelta` requirement with revlogv2
A repository could use a mix of revlogv1 and revlogv2, making the requirements
still necessary. Overall we should move away from the "requirements" file being
used a way to configure the repository and stick to it "what do you need to
access this repository". However this is a wider work for another time.
In addition the logic we just dropped was confusing the `hg debugformat`
command, breaking the upgrade code and inconsistent (eg: `sparse-revlog` is also
implied by `revlogv2`).
Finally, multiple other config option would imply the use of the `revlogv2`
requirements, without drop the `generaldelta` one, leading to more
inconsistency.
Differential Revision: https://phab.mercurial-scm.org/D10612
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:47 +0200] rev 47220
config: drop the `format.exp-revlogv2.2` option
It is oddly named and not used at all. The one used by the code is still
`experimental.revlogv2`. So we drop that one option for consistency. We move
associate documentation to the actual option.
Differential Revision: https://phab.mercurial-scm.org/D10611
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:36 +0200] rev 47219
revlog: only use the `_indexfp` method for read operation
This will avoid "other" code to not overlook `_writing` usage. We introduces
private method dedicated to writing to make use the right option are always
used.
Differential Revision: https://phab.mercurial-scm.org/D10610
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:26 +0200] rev 47218
revlog: use `_writing` in `rewrite_sidedata`
Special cases aren't special enough to break the rules.
Differential Revision: https://phab.mercurial-scm.org/D10609
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 03 May 2021 12:28:15 +0200] rev 47217
revlog: open files in 'r+' instead of 'a+'
The code doing actual writing is already doing the necessary seeking, so we
could safely use 'r+'. This make the file objecs usable in more situation, like
updating the sidedata information during pull.
revlog: forcibly move the file cursor at the right location before writing
This is a paranoid change in case the changelog computation moved the cursors
under our feets.
This is not known to happens right now.
Differential Revision: https://phab.mercurial-scm.org/D10608