Matt Mackall <mpm@selenic.com> [Thu, 21 May 2015 14:30:22 -0500] rev 25241
tests: fix test-clone on vfat
Martin von Zweigbergk <martinvonz@google.com> [Thu, 21 May 2015 11:34:40 -0700] rev 25240
wireproto: remove unused 'store' import
Durham Goode <durham@fb.com> [Sat, 16 May 2015 16:16:18 -0700] rev 25239
match: allow unioning arbitrary match functions
A future patch will be allowing nested matchers. To support that, let's refactor
_buildmatch to build a list of matchers then return True if any match.
We were already doing that for filesets + regex patterns, but this way will be
more generic.
Durham Goode <durham@fb.com> [Sat, 16 May 2015 16:12:00 -0700] rev 25238
match: add root to _buildmatch
A future patch will make _buildmatch able to expand relative include patterns.
Doing so will require knowing the root of the repo, so let's go ahead and pass
it in.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 21 May 2015 10:41:06 -0700] rev 25237
localrepo: extract stream clone application into reusable function
The existing stream_in method assumes a streaming clone is applied via
the wire protocol. Previous patches have enabled streaming clone data to
be produced and consumed outside the context of the wire protocol.
However, the consuming part was incomplete because it didn't deal with
things like updating the branch caches or writing out a requirements
file.
This patch finishes the separation of stream clone handling from the
wire protocol. After this patch, it is possible to consume stream clones
from arbitrary sources, including files. Mozilla plans to leverage this
to serve pre-generated stream clone files to consumers, drastically
reducing the wall and CPU time required to clone large repositories.
This will enable clones to be nearly as fast as `tar`.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 21 May 2015 10:27:45 -0700] rev 25236
exchange: move code for consuming streaming clone into exchange
For reasons outlined in the previous commit, we want to make the code
for consuming "stream bundles" reusable. This patch extracts the code
into a standalone function.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 21 May 2015 10:27:22 -0700] rev 25235
exchange: move code for generating a streaming clone into exchange
Streaming clones are fast because they are essentially tar files.
On mozilla-central, a streaming clone only consumes ~55s CPU time
on clients as opposed to ~340s CPU time for a regular clone or gzip
bundle unbundle.
Mozilla is deploying static file "lookaside" support to our Mercurial
server. Static bundles are pre-generated and uploaded to S3. When a
clone is performed, the static file is fetched, applied, and then an
incremental pull is performed. Unfortunately, on an ideal network
connection this still takes as much wall and CPU time as a regular
clone (although it does save significant server resources).
We like the client-side wall time wins of streaming clones. But we want
to leverage S3-based pre-generated files for serving the bulk of clone
data.
This patch moves the code for producing a "stream bundle" into its
own standalone function, away from the wire protocol. This will enable
stream bundle files to be produced outside the context of the wire
protocol.
A bikeshed on whether exchange is the best module for this function
might be warranted. I selected exchange instead of changegroup because
"stream bundles" aren't changegroups (yet).
Martin von Zweigbergk <martinvonz@google.com> [Tue, 19 May 2015 10:13:43 -0700] rev 25234
dirstate: avoid match.files() in walk()