changegroup: control reordering via constructor argument cg2packer.__init__ exists just to override self._reorder. Let's parameterize this behavior via an argument to the parent's __init__. The logic for self._reorder is kinda wonky. None is used as a special value and the value should be None in some situations. It is probably worth rewriting this logic to make behavior more explicit. This will likely happen as part of future work to control the delta generation process that I have planned. Differential Revision: https://phab.mercurial-scm.org/D4082

changegroup: consolidate tree manifests sending into cg1packer Previously, we overrode a method to control how manifests were serialized. This method was redefined on cg3packer to send tree manifests. This commit moves the tree manifests sending variation to cg1packer and teaches the cgpacker constructor to control which version to use. After these changes, cg3packer was empty. So it has been removed. Differential Revision: https://phab.mercurial-scm.org/D4081

changegroup: pass end of manifests marker into constructor cg3 inserts a custom marker in the stream once all manifests have been transferred. This is currently abstracted out by overriding a method. Let's pass the end of manifests marker in as an argument to avoid the extra method. Differential Revision: https://phab.mercurial-scm.org/D4080

changegroup: pass function to build delta header into constructor Previously, the delta header struct format was defined on each class and each class had a separate function for building the delta header. We replace both of these with an argument to __init__ containing a callable that can format a delta header given a revisiondelta instance. Differential Revision: https://phab.mercurial-scm.org/D4079

changegroup: make delta header struct formatters actual structs Why we weren't using compiled Struct instances, I don't know. They make code simpler. In theory they are faster. Although I don't believe I was able to measure any meaningful change. That could be because this code is often dominated by compression, deltafication, and function call overhead. Differential Revision: https://phab.mercurial-scm.org/D4078

changegroup: pass version into constructor Currently, the version is an attribute on each class. Passing the argument into the constructor gets us one step closer to eliminating cg2packer and cg3packer. Differential Revision: https://phab.mercurial-scm.org/D4077

changegroup: define functions for creating changegroup packers Currently, we have 3 classes for changegroup generation. Each class handles a specific changegroup format. And each subsequent version's class inherits from the previous one. The interface for the classes is not very well defined and a lot of version-specific behavior is behind overloaded functions. This approach adds complexity and makes changegroup generation difficult to reason about. Upcoming commits will be consolidating these 3 classes so differences between changegroup versions and changegroup generation are controlled by parameters to a single constructor / type rather than by overriding class attributes via inheritance. We begin this process by building dedicated functions for creating each changegroup packer instance. Currently they just call the constructor on the appropriate class. This will soon change. Differential Revision: https://phab.mercurial-scm.org/D4076

changegroup: capture revision delta in a data structure The current changegroup generation code is tightly coupled to the revlog API. This tight coupling makes it difficult to implement alternate storage backends without requiring a large surface area of the revlog API to be exposed. This is not desirable. In order to support changegroup generation with non-revlog storage, we'll need to abstract the concept of delta generation. This commit is the first step down that road. We introduce a data structure for representing a delta in a changegroup. The API still leaves a lot to be desired. But at least we now have separation between data and actions performed on it. As part of this, we tweak behavior slightly: we no longer concatenate the delta prefix with the metadata header. Instead, we track and emit the prefix as a separate chunk. This shouldn't have any meaningful impact since all the chunks just get sent to the wire, the compressor, etc. Because we're introducing a new object, this does add some overhead to changegroup execution. `hg perfchangegroupchangelog` on my clone of the Mercurial repo (~40,000 visible revisions in the changelog) slows down a bit: ! wall 1.268600 comb 1.270000 user 1.270000 sys 0.000000 (best of 8) ! wall 1.419479 comb 1.410000 user 1.410000 sys 0.000000 (best of 8) With for `hg bundle -t none-v2 -a /dev/null`: before: real 6.610 secs (user 6.460+0.000 sys 0.140+0.000) after: real 7.210 secs (user 7.060+0.000 sys 0.140+0.000) I plan to claw back this regression in future commits. And I may even do away with this data structure once the refactor is complete. For now, it makes things easier to comprehend. Differential Revision: https://phab.mercurial-scm.org/D4075

changegroup: inline ellipsisdata() There's only one caller of it. I don't think it needs to exist as a standalone function. Differential Revision: https://phab.mercurial-scm.org/D4074

changegroup: rename "revlog" variables "revlog" shadows the module import. But more importantly, changegroup generation should be storage agnostic and not assume the existence of revlogs. Let's rename the thing providing revision storage to "store" to reflect this ideal property. Differential Revision: https://phab.mercurial-scm.org/D4073

changegroup: move generate() modifications from narrow Narrow had a custom version of generate() that was essentially a copy of generate() with inline additions to facilitate ellipses serving. This commit inlines those modifications into generate(). Differential Revision: https://phab.mercurial-scm.org/D4067

changegroup: move generatefiles() from narrow The code is a bit ugly in that it overrides the linknodes function that is passed in as a function. I'd like to think that the caller of generatefiles() would pass in the appropriate function. We can clean this up later. Differential Revision: https://phab.mercurial-scm.org/D4066

changegroup: move _sortgroup() from narrow Differential Revision: https://phab.mercurial-scm.org/D4065

changegroup: move close() from narrow More of the same. Differential Revision: https://phab.mercurial-scm.org/D4064