Mercurial > hg
comparison mercurial/wireproto.py @ 35787:a84dbc87dae9
exchange: send bundle2 stream clones uncompressed
Stream clones don't compress well. And compression undermines
a point of stream clones which is to trade significant CPU
reductions by increasing size.
Building upon our introduction of metadata to communicate bundle
information back to callers of exchange.getbundlechunks(), we add
an attribute to the bundler that communicates whether the bundle is
best left uncompressed. We return this attribute as part of the bundle
metadata. And the wire protocol honors it when determining whether
to compress the wire protocol response.
The added test demonstrates that the raw result from the wire
protocol is not compressed. It also demonstrates that the server
will serve stream responses when the feature isn't enabled. We'll
address that in another commit.
The effect of this change is that server-side CPU usage for bundle2
stream clones is significantly reduced by removing zstd compression.
For the mozilla-unified repository:
before: 37.69 user 8.01 system
after: 27.38 user 7.34 system
Assuming things are CPU bound, that ~10s reduction would translate to
faster clones on the client. zstd can decompress at >1 GB/s. So the
overhead from decompression on the client is small in the grand scheme
of things. But if zlib compression were being used, the overhead would
be much greater.
Differential Revision: https://phab.mercurial-scm.org/D1926
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Mon, 22 Jan 2018 12:12:29 -0800 |
parents | ba15580e53d5 |
children | d9e71cce3b2f 742ce6fbc109 |
comparison
equal
deleted
inserted
replaced
35786:8144f1b07e21 | 35787:a84dbc87dae9 |
---|---|
860 if proto.name == 'http': | 860 if proto.name == 'http': |
861 return ooberror(bundle2required) | 861 return ooberror(bundle2required) |
862 raise error.Abort(bundle2requiredmain, | 862 raise error.Abort(bundle2requiredmain, |
863 hint=bundle2requiredhint) | 863 hint=bundle2requiredhint) |
864 | 864 |
865 preferuncompressed = False | 865 prefercompressed = True |
866 | 866 |
867 try: | 867 try: |
868 if repo.ui.configbool('server', 'disablefullbundle'): | 868 if repo.ui.configbool('server', 'disablefullbundle'): |
869 # Check to see if this is a full clone. | 869 # Check to see if this is a full clone. |
870 clheads = set(repo.changelog.heads()) | 870 clheads = set(repo.changelog.heads()) |
877 _('server has pull-based clones disabled'), | 877 _('server has pull-based clones disabled'), |
878 hint=_('remove --pull if specified or upgrade Mercurial')) | 878 hint=_('remove --pull if specified or upgrade Mercurial')) |
879 | 879 |
880 info, chunks = exchange.getbundlechunks(repo, 'serve', | 880 info, chunks = exchange.getbundlechunks(repo, 'serve', |
881 **pycompat.strkwargs(opts)) | 881 **pycompat.strkwargs(opts)) |
882 prefercompressed = info.get('prefercompressed', True) | |
882 except error.Abort as exc: | 883 except error.Abort as exc: |
883 # cleanly forward Abort error to the client | 884 # cleanly forward Abort error to the client |
884 if not exchange.bundle2requested(opts.get('bundlecaps')): | 885 if not exchange.bundle2requested(opts.get('bundlecaps')): |
885 if proto.name == 'http': | 886 if proto.name == 'http': |
886 return ooberror(str(exc) + '\n') | 887 return ooberror(str(exc) + '\n') |
892 if exc.hint is not None: | 893 if exc.hint is not None: |
893 advargs.append(('hint', exc.hint)) | 894 advargs.append(('hint', exc.hint)) |
894 bundler.addpart(bundle2.bundlepart('error:abort', | 895 bundler.addpart(bundle2.bundlepart('error:abort', |
895 manargs, advargs)) | 896 manargs, advargs)) |
896 chunks = bundler.getchunks() | 897 chunks = bundler.getchunks() |
897 preferuncompressed = True | 898 prefercompressed = False |
898 | 899 |
899 return streamres(gen=chunks, prefer_uncompressed=preferuncompressed) | 900 return streamres(gen=chunks, prefer_uncompressed=not prefercompressed) |
900 | 901 |
901 @wireprotocommand('heads') | 902 @wireprotocommand('heads') |
902 def heads(repo, proto): | 903 def heads(repo, proto): |
903 h = repo.heads() | 904 h = repo.heads() |
904 return encodelist(h) + "\n" | 905 return encodelist(h) + "\n" |