Gregory Szorc <gregory.szorc@gmail.com> [Wed, 03 Oct 2018 13:57:42 -0700] rev 40179
exchangev2: use filesdata
filesdata is a more efficient mechanism for bulk fetching files data for a
range of changesets. Let's use it in exchangev2.
With this change, a client performing a full clone of mozilla-unified
transmits substantially fewer bytes across the wire:
before: 139,124,863 bytes sent
after: 20,522,499 bytes sent
The bulk of the remaining bytes is likely the transfer of ~1M nodes for
changesets and manifests. We can eliminate this by making requests in
terms of node ranges instead of explicit node lists...
Differential Revision: https://phab.mercurial-scm.org/D4982
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 03 Oct 2018 12:54:39 -0700] rev 40178
wireprotov2: define and implement "filesdata" command
Previously, the only way to access file revision data was the
"filedata" command. This command is useful to have. But, it only
allowed resolving revision data for a single file. This meant that
clients needed to send 1 command for each tracked path they were
seeking data on. Furthermore, those commands would need to enumerate
the exact file nodes they wanted data for.
This approach meant that clients were sending a lot of data to
remotes in order to request file data. e.g. if there were 1M
file revisions, we'd need at least 20,000,000 bytes just to encode
file nodes! Many clients on the internet don't have that kind of
upload capacity.
In order to limit the amount of data that clients must send, we'll
need more efficient ways to request repository data.
This commit defines and implements a new "filesdata" command. This
command allows the retrieval of data for multiple files by specifying
changeset revisions and optional file patterns. The command figures
out what file revisions are "relevant" and sends them in bulk.
The logic around choosing which file revisions to send in the case of
haveparents not being set is overly simple and will over-send files. We
will need more smarts here eventually. (Specifically, the client will
need to tell the server which revisions it knows about.) This work
is deferred until a later time.
Differential Revision: https://phab.mercurial-scm.org/D4981
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 02 Oct 2018 10:31:36 -0700] rev 40177
wireprotov2: extract file object emission to own function
An upcoming commit will introduce another caller.
Differential Revision: https://phab.mercurial-scm.org/D4980