Gregory Szorc <gregory.szorc@gmail.com> [Sun, 20 Nov 2016 13:55:53 -0800] rev 30465
httppeer: use compression engine API for decompressing responses
In preparation for supporting multiple compression formats on the
wire protocol, we need all users of the wire protocol to use
compression engine APIs.
This commit ports the HTTP wire protocol client to use the
compression engine API.
The code for handling the HTTPException is a bit hacky. Essentially,
HTTPException could be thrown by any read() from the socket. However,
as part of porting the API, we no longer have a generator wrapping
the socket and we don't have a single place where we can trap the
exception. We solve this by introducing a proxy class that intercepts
read() and converts the exception appropriately.
In the future, we could introduce a new compression engine API that
supports emitting a generator of decompressed chunks. This would
eliminate the need for the proxy class. As I said when I introduced
the decompressorreader() API, I'm not fond of it and would support
transitioning to something better. This can be done as a follow-up,
preferably once all the code is using the compression engine API and
we have a better idea of the API needs of all the consumers.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 19 Nov 2016 18:31:40 -0800] rev 30464
httppeer: do decompression inside _callstream
The current HTTP transport protocol only compresses certain command
responses and requires calls to that command to call
"_callcompressable," which zlib decompresses the response
transparently.
Upcoming changes will enable *any* response to be compressed with
varying compression formats. In order to handle this better, this
commit moves the decompression bits to the main function performing
the HTTP request. We introduce an underscore-prefixed argument to
denote this behavior so it doesn't conflict with a named argument
to a command.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 19 Nov 2016 17:11:12 -0800] rev 30463
keepalive: reorder header precedence
There are 3 sources of headers used by this function:
* The default headers defined by the URL opener
* Headers that are copied on redirects
* Headers that aren't copied on redirects
Previously, we applied the default headers from the URL
opener last. This feels wrong to me as those headers are
the most low level and something built on top of the URL
opener may wish to override them. So, this commit changes
the order to apply them with the least precedence.
While I was here, I removed a Python version test that is
no longer necessary.