Gregory Szorc <gregory.szorc@gmail.com> [Mon, 08 Oct 2018 17:00:16 -0700] rev 40133
wireprotov2: send protocol settings frame from client
Now that we have client and server reactor support for protocol
settings and encoding frames, we can start to send them out over
the wire!
This commit teaches the client reactor to send out a protocol
settings frame when needed. The httpv2 peer has been taught to
gather a list of supported content encoders and to advertise them
through the client reactor.
Because the client is now sending new frame types by default, this
constitutes a compatibility break in the framing protocol. The
media type version has been bumped accordingly. This will ensure
existing clients won't attempt to send the new frames to old
servers not supporting this explicit media type. I'm not bothering
with the BC annotation because everything wireprotov2 is highly
experimental and nobody should be running a server yet.
Differential Revision: https://phab.mercurial-scm.org/D4922
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 08 Oct 2018 17:10:59 -0700] rev 40132
wireprotov2: define and use stream encoders
Now that we have basic support for defining stream encoding, it is
time to start doing something with it.
We define various classes implementing stream encoders/decoders for
the defined encoding profiles. This is relatively straightforward.
We teach the inputstream and outputstream classes how to encode,
decode, and flush data.
We then teach the clientreactor how to filter received data through
the inputstream decoder.
One of the features of the framing format is that streams can span
requests. This is a differentiating feature from say HTTP/2, which
associates streams with requests. By allowing streams to span requests,
we can reuse compression context data across requests/responses. But
in order to do this, we need a mechanism to "flush" the encoder at
logical boundaries so that receivers receive all data where it is
expected. And a "flush" event is distinct from a "finish" event from
the perspective of certain compressors because a "flush" will retain
compression context state whereas a "finish" operation will not. This
is why encoders have both a flush() and a finish() and each uses
specific flushing semantics on the underlying compressor.
The added tests verify various behavior of decoders via clientreactor.
These tests do test some compression behavior via use of outputstream.
But for all intents and purposes, server reactor support for encoding
is not yet implemented.
Differential Revision: https://phab.mercurial-scm.org/D4921
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Oct 2018 17:39:16 -0700] rev 40131
wireprotov2: establish dedicated classes for input and output streams
Streams are unidirectional. As part of implementing encoding/decoding
support, it became clear that it didn't make sense for a generic
"stream" class to hold functionality related to both encoding and
decoding. So we create new classes to represent the flavor of
stream.
Differential Revision: https://phab.mercurial-scm.org/D4920
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Oct 2018 17:17:57 -0700] rev 40130
wireprotov2: pass ui into clientreactor and serverreactor
This will allow us to use config options to influence compression
settings.
Differential Revision: https://phab.mercurial-scm.org/D4919
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Oct 2018 16:44:21 -0700] rev 40129
wireprotov2: handle stream encoding settings frames
Like what we just did for the server reactor, we teach the client
reactor to handle stream encoding settings frames. The code is
very similar.
We define a method on the stream class to handle processing the data
within the decoded frames. However, it doesn't yet do anything useful.
Differential Revision: https://phab.mercurial-scm.org/D4918
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Oct 2018 15:43:21 -0700] rev 40128
wireprotov2: document client reactor actions
We should document these so consumers have an easier life.
Differential Revision: https://phab.mercurial-scm.org/D4917
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Oct 2018 16:26:45 -0700] rev 40127
wireprotov2: handle sender protocol settings frames
We teach the server reactor to handle the optional sender protocol
settings frames, which can only be sent at the beginning of frame
exchange.
Right now, we simply decode the data and record the sender protocol
settings on the server reactor instance: we don't yet do anything
meaningful with the data.
Differential Revision: https://phab.mercurial-scm.org/D4916
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Oct 2018 14:05:16 -0700] rev 40126
wireprotov2: update stream encoding specification
The encoding of data within streams in the frame-based protocol is
not yet defined or implemented. This means that all data in wire
protocol version 2 is currently being sent out raw, without
compression. That's obviously not ideal.
This commit formalizes the beginnings of stream encoding support
in the protocol.
I suspect we'll change behavior substantially in the future. My goal
is to get something landed so we can use compression. We can build
out more robust support later.
Because the frame type ID changed, this is strictly BC. But existing
code wasn't using the frame. I'll bump the framing protocol version
later once code is introduced to use the new frame.
Differential Revision: https://phab.mercurial-scm.org/D4915
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 04 Oct 2018 15:08:42 -0700] rev 40125
cborutil: cast bytearray to bytes
This code didn't like passing in bytearray instances. Let's cast
bytearray to bytes so it works.
Differential Revision: https://phab.mercurial-scm.org/D4914
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 08 Oct 2018 17:06:24 -0700] rev 40124
tests: disable zstd in test
This makes the test pass in pure installs.
Differential Revision: https://phab.mercurial-scm.org/D4913
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 08 Oct 2018 17:20:41 -0700] rev 40123
wireprotov2: remove "compression" from capabilities response
This is not used. And future commits will change how this mechanism
works. Let's remove it.
As a bonus, this fixes some test failures on pure installs (due to
zstd references).
Differential Revision: https://phab.mercurial-scm.org/D4912
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 08 Oct 2018 16:27:40 -0700] rev 40122
zstandard: vendor python-zstandard 0.10.1
This was just released.
The upstream source distribution from PyPI was extracted. Unwanted
files were removed.
The clang-format ignore list was updated to reflect the new source
of files.
setup.py was updated to pass a new argument to python-zstandard's
function for returning an Extension instance. Upstream had to change
to use relative paths because Python 3.7's packaging doesn't
seem to like absolute paths when defining sources, includes, etc.
The default relative path calculation is relative to setup_zstd.py
which is different from the directory of Mercurial's setup.py.
The project contains a vendored copy of zstandard 1.3.6. The old
version was 1.3.4.
The API should be backwards compatible and nothing in core should
need adjusted. However, there is a new "chunker" API that we
may find useful in places where we want to emit compressed chunks
of a fixed size.
There are a pair of bug fixes in 0.10.0 with regards to
compressobj() and decompressobj() when block flushing is used. I
actually found these bugs when introducing these APIs in Mercurial!
But existing Mercurial code is not affected because we don't
perform block flushing.
# no-check-commit because 3rd party code has different style guidelines
Differential Revision: https://phab.mercurial-scm.org/D4911
Yuya Nishihara <yuya@tcha.org> [Tue, 25 Sep 2018 20:55:03 +0900] rev 40121
rust-chg: install signal handlers to forward signals to server
I use sync::Once as a synchronization primitive because it's quite easy
to use, and is good enough to prevent data race in these C functions.
Yuya Nishihara <yuya@tcha.org> [Mon, 24 Sep 2018 22:19:49 +0900] rev 40120
rust-chg: remove SIGCHLD handler which won't work in oxidized chg
Since pager is managed by the Rust part, the C code doesn't know the pager
pid. I could make the Rust part teach the pid to C, but still installing
SIGCHLD handler seems horrible idea since we no longer use handcrafted
low-level process management functions.
Instead, I'm thinking of adding async handler to send SIGPIPE at the exit
of the pager.
Yuya Nishihara <yuya@tcha.org> [Mon, 24 Sep 2018 22:04:57 +0900] rev 40119
rust-chg: extract signal handlers from chg/procutil.c
abortmsgerrno() and debugmsg() are removed, and the public interface instead
returns success/error status. Since signal handlers can't propagate errors,
the result of kill() is just ignored.
Yuya Nishihara <yuya@tcha.org> [Sun, 07 Oct 2018 23:19:49 +0900] rev 40118
help: document about "version" template keywords