Thu, 21 May 2015 10:27:45 -0700 exchange: move code for consuming streaming clone into exchange
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 21 May 2015 10:27:45 -0700] rev 25236
exchange: move code for consuming streaming clone into exchange For reasons outlined in the previous commit, we want to make the code for consuming "stream bundles" reusable. This patch extracts the code into a standalone function.
Thu, 21 May 2015 10:27:22 -0700 exchange: move code for generating a streaming clone into exchange
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 21 May 2015 10:27:22 -0700] rev 25235
exchange: move code for generating a streaming clone into exchange Streaming clones are fast because they are essentially tar files. On mozilla-central, a streaming clone only consumes ~55s CPU time on clients as opposed to ~340s CPU time for a regular clone or gzip bundle unbundle. Mozilla is deploying static file "lookaside" support to our Mercurial server. Static bundles are pre-generated and uploaded to S3. When a clone is performed, the static file is fetched, applied, and then an incremental pull is performed. Unfortunately, on an ideal network connection this still takes as much wall and CPU time as a regular clone (although it does save significant server resources). We like the client-side wall time wins of streaming clones. But we want to leverage S3-based pre-generated files for serving the bulk of clone data. This patch moves the code for producing a "stream bundle" into its own standalone function, away from the wire protocol. This will enable stream bundle files to be produced outside the context of the wire protocol. A bikeshed on whether exchange is the best module for this function might be warranted. I selected exchange instead of changegroup because "stream bundles" aren't changegroups (yet).
(0) -10000 -3000 -1000 -300 -100 -30 -10 -2 +2 +10 +30 +100 +300 +1000 +3000 +10000 tip