Gregory Szorc <gregory.szorc@gmail.com> [Fri, 03 Aug 2018 10:01:58 -0700] rev 38900
changegroup: control reordering via constructor argument
cg2packer.__init__ exists just to override self._reorder. Let's
parameterize this behavior via an argument to the parent's
__init__.
The logic for self._reorder is kinda wonky. None is used as a
special value and the value should be None in some situations.
It is probably worth rewriting this logic to make behavior more
explicit. This will likely happen as part of future work to
control the delta generation process that I have planned.
Differential Revision: https://phab.mercurial-scm.org/D4082
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 03 Aug 2018 09:44:30 -0700] rev 38899
changegroup: consolidate tree manifests sending into cg1packer
Previously, we overrode a method to control how manifests were
serialized. This method was redefined on cg3packer to send tree
manifests.
This commit moves the tree manifests sending variation to cg1packer
and teaches the cgpacker constructor to control which version to
use.
After these changes, cg3packer was empty. So it has been removed.
Differential Revision: https://phab.mercurial-scm.org/D4081
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 18:04:51 -0700] rev 38898
changegroup: pass end of manifests marker into constructor
cg3 inserts a custom marker in the stream once all manifests
have been transferred. This is currently abstracted out by
overriding a method.
Let's pass the end of manifests marker in as an argument to avoid
the extra method.
Differential Revision: https://phab.mercurial-scm.org/D4080
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 17:44:56 -0700] rev 38897
changegroup: pass function to build delta header into constructor
Previously, the delta header struct format was defined on each
class and each class had a separate function for building the
delta header.
We replace both of these with an argument to __init__ containing
a callable that can format a delta header given a revisiondelta
instance.
Differential Revision: https://phab.mercurial-scm.org/D4079
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 17:52:21 -0700] rev 38896
changegroup: make delta header struct formatters actual structs
Why we weren't using compiled Struct instances, I don't know. They
make code simpler. In theory they are faster. Although I don't
believe I was able to measure any meaningful change. That could be
because this code is often dominated by compression, deltafication,
and function call overhead.
Differential Revision: https://phab.mercurial-scm.org/D4078
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 17:33:23 -0700] rev 38895
changegroup: pass version into constructor
Currently, the version is an attribute on each class. Passing
the argument into the constructor gets us one step closer to
eliminating cg2packer and cg3packer.
Differential Revision: https://phab.mercurial-scm.org/D4077
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 17:29:53 -0700] rev 38894
changegroup: define functions for creating changegroup packers
Currently, we have 3 classes for changegroup generation. Each class
handles a specific changegroup format. And each subsequent version's
class inherits from the previous one.
The interface for the classes is not very well defined and a lot of
version-specific behavior is behind overloaded functions. This
approach adds complexity and makes changegroup generation difficult
to reason about.
Upcoming commits will be consolidating these 3 classes so differences
between changegroup versions and changegroup generation are controlled
by parameters to a single constructor / type rather than by
overriding class attributes via inheritance.
We begin this process by building dedicated functions for creating
each changegroup packer instance. Currently they just call the
constructor on the appropriate class. This will soon change.
Differential Revision: https://phab.mercurial-scm.org/D4076
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 03 Aug 2018 10:05:26 -0700] rev 38893
changegroup: capture revision delta in a data structure
The current changegroup generation code is tightly coupled to
the revlog API. This tight coupling makes it difficult to implement
alternate storage backends without requiring a large surface area
of the revlog API to be exposed. This is not desirable.
In order to support changegroup generation with non-revlog storage,
we'll need to abstract the concept of delta generation.
This commit is the first step down that road. We introduce a
data structure for representing a delta in a changegroup.
The API still leaves a lot to be desired. But at least we now
have separation between data and actions performed on it.
As part of this, we tweak behavior slightly: we no longer
concatenate the delta prefix with the metadata header. Instead,
we track and emit the prefix as a separate chunk. This shouldn't
have any meaningful impact since all the chunks just get sent to
the wire, the compressor, etc.
Because we're introducing a new object, this does add some
overhead to changegroup execution. `hg perfchangegroupchangelog`
on my clone of the Mercurial repo (~40,000 visible revisions in
the changelog) slows down a bit:
! wall 1.268600 comb 1.270000 user 1.270000 sys 0.000000 (best of 8)
! wall 1.419479 comb 1.410000 user 1.410000 sys 0.000000 (best of 8)
With for `hg bundle -t none-v2 -a /dev/null`:
before: real 6.610 secs (user 6.460+0.000 sys 0.140+0.000)
after: real 7.210 secs (user 7.060+0.000 sys 0.140+0.000)
I plan to claw back this regression in future commits. And I may
even do away with this data structure once the refactor is complete.
For now, it makes things easier to comprehend.
Differential Revision: https://phab.mercurial-scm.org/D4075
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 16:36:40 -0700] rev 38892
changegroup: inline ellipsisdata()
There's only one caller of it. I don't think it needs to exist as
a standalone function.
Differential Revision: https://phab.mercurial-scm.org/D4074
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 17:05:11 -0700] rev 38891
changegroup: rename "revlog" variables
"revlog" shadows the module import. But more importantly, changegroup
generation should be storage agnostic and not assume the existence
of revlogs. Let's rename the thing providing revision storage to
"store" to reflect this ideal property.
Differential Revision: https://phab.mercurial-scm.org/D4073
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 14:15:10 -0700] rev 38890
changegroup: move generate() modifications from narrow
Narrow had a custom version of generate() that was essentially a copy
of generate() with inline additions to facilitate ellipses serving.
This commit inlines those modifications into generate().
Differential Revision: https://phab.mercurial-scm.org/D4067
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 12:18:35 -0700] rev 38889
changegroup: move generatefiles() from narrow
The code is a bit ugly in that it overrides the linknodes
function that is passed in as a function. I'd like to think
that the caller of generatefiles() would pass in the appropriate
function. We can clean this up later.
Differential Revision: https://phab.mercurial-scm.org/D4066
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 12:12:12 -0700] rev 38888
changegroup: move _sortgroup() from narrow
Differential Revision: https://phab.mercurial-scm.org/D4065
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 02 Aug 2018 09:52:01 -0700] rev 38887
changegroup: move close() from narrow
More of the same.
Differential Revision: https://phab.mercurial-scm.org/D4064