Mercurial > hg
comparison mercurial/hgweb/protocol.py @ 30466:2add671bf55b
wireproto: perform chunking and compression at protocol layer (API)
Currently, the "streamres" response type is populated with a generator
of chunks with compression possibly already applied. This puts the onus
on commands to perform chunking and compression. Architecturally, I
think this is the wrong place to perform this work. I think commands
should say "here is the data" and the protocol layer should take care
of encoding the final bytes to put on the wire.
Additionally, upcoming commits will improve wire protocol support for
compression. Having a central place for performing compression in the
protocol transport layer will be easier than having to deal with
compression at the commands layer.
This commit refactors the "streamres" response type to accept either
a generator or an object with "read." Additionally, the type now
accepts a flag indicating whether the response is a "version 1
compressible" response. This basically identifies all commands
currently performing compression. I could have used a special type
for this, but a flag works just as well. The argument name
foreshadows the introduction of wire protocol changes, hence the "v1."
The code for chunking and compressing has been moved to the output
generation function for each protocol transport. Some code has been
inlined, resulting in the deletion of now unused methods.
| author | Gregory Szorc <gregory.szorc@gmail.com> |
|---|---|
| date | Sun, 20 Nov 2016 13:50:45 -0800 |
| parents | 038547a14d85 |
| children | b3a9ef3d30e8 |
comparison
equal
deleted
inserted
replaced
| 30465:40a1871eea5e | 30466:2add671bf55b |
|---|---|
| 71 def restore(self): | 71 def restore(self): |
| 72 val = self.ui.fout.getvalue() | 72 val = self.ui.fout.getvalue() |
| 73 self.ui.ferr, self.ui.fout = self.oldio | 73 self.ui.ferr, self.ui.fout = self.oldio |
| 74 return val | 74 return val |
| 75 | 75 |
| 76 def groupchunks(self, fh): | |
| 77 def getchunks(): | |
| 78 while True: | |
| 79 chunk = fh.read(32768) | |
| 80 if not chunk: | |
| 81 break | |
| 82 yield chunk | |
| 83 | |
| 84 return self.compresschunks(getchunks()) | |
| 85 | |
| 86 def compresschunks(self, chunks): | 76 def compresschunks(self, chunks): |
| 87 # Don't allow untrusted settings because disabling compression or | 77 # Don't allow untrusted settings because disabling compression or |
| 88 # setting a very high compression level could lead to flooding | 78 # setting a very high compression level could lead to flooding |
| 89 # the server's network or CPU. | 79 # the server's network or CPU. |
| 90 opts = {'level': self.ui.configint('server', 'zliblevel', -1)} | 80 opts = {'level': self.ui.configint('server', 'zliblevel', -1)} |
| 104 rsp = wireproto.dispatch(repo, p, cmd) | 94 rsp = wireproto.dispatch(repo, p, cmd) |
| 105 if isinstance(rsp, str): | 95 if isinstance(rsp, str): |
| 106 req.respond(HTTP_OK, HGTYPE, body=rsp) | 96 req.respond(HTTP_OK, HGTYPE, body=rsp) |
| 107 return [] | 97 return [] |
| 108 elif isinstance(rsp, wireproto.streamres): | 98 elif isinstance(rsp, wireproto.streamres): |
| 99 if rsp.reader: | |
| 100 gen = iter(lambda: rsp.reader.read(32768), '') | |
| 101 else: | |
| 102 gen = rsp.gen | |
| 103 | |
| 104 if rsp.v1compressible: | |
| 105 gen = p.compresschunks(gen) | |
| 106 | |
| 109 req.respond(HTTP_OK, HGTYPE) | 107 req.respond(HTTP_OK, HGTYPE) |
| 110 return rsp.gen | 108 return gen |
| 111 elif isinstance(rsp, wireproto.pushres): | 109 elif isinstance(rsp, wireproto.pushres): |
| 112 val = p.restore() | 110 val = p.restore() |
| 113 rsp = '%d\n%s' % (rsp.res, val) | 111 rsp = '%d\n%s' % (rsp.res, val) |
| 114 req.respond(HTTP_OK, HGTYPE, body=rsp) | 112 req.respond(HTTP_OK, HGTYPE, body=rsp) |
| 115 return [] | 113 return [] |