fuzz: try setting PYTHONNOUSERSITE=1 to avoid loading site-packages Right now the fuzzer is crashing trying to look up the home dir for uid 0, which is breaking in the fuzz environment. Differential Revision: https://phab.mercurial-scm.org/D4936

perf: extract result formatting in its own function This will make it easier to reuse in another changesets.

perf: extract the timing of a section in a context manager This makes it easier to reuse it in other (future) part of the code that requires their own time management.

pref: support negative indexing in perfrevlogrevisions This is useful to check for a consistent final slice in multiple different repositories.

perf: accept formatter option for perfmanifest

perf: fix -T json The previous code was mixing formatting and data, breaking `-T json` with unexpected data. We fix the issue and add a test to prevent future regression.

formatter: more details on assertion failure This is useful when the assertion fails.

wireprotov2: raise ProgrammingError on unknown action Suggested by @durin42 in review of D4923. Differential Revision: https://phab.mercurial-scm.org/D4935

wireprotov2: send content encoded frames from server Now that we have support for negotiating encodings and configuring an encoder, we can start sending content encoded frames from the server. This commit teaches the wireprotov2 server code to send content encoded frames. On the mozilla-unified repository with zstd enabled peers, this change reduces the total amount of data transferred from server to client drastically: befor: 7,190,995,812 bytes after: 1,605,508,691 bytes Differential Revision: https://phab.mercurial-scm.org/D4927

wireprotov2: raise exception in objects() if future has been resolved Differential Revision: https://phab.mercurial-scm.org/D4926

wireprotov2: don't emit empty frames Staring at logs revealed the presence of empty frames that should have contained payload. Let's stop that from happening. Differential Revision: https://phab.mercurial-scm.org/D4925

wireprotov2: remove functions for creating response frames from bytes All code in the actual server uses oncommandresponsereadyobjects(). Test code was ported to that method. This resulted in a handful of subtle test changes. Differential Revision: https://phab.mercurial-scm.org/D4924

wireprotov2: handle noop action This action can be returned from the client reactor. We should handle it. Differential Revision: https://phab.mercurial-scm.org/D4923

wireprotov2: send protocol settings frame from client Now that we have client and server reactor support for protocol settings and encoding frames, we can start to send them out over the wire! This commit teaches the client reactor to send out a protocol settings frame when needed. The httpv2 peer has been taught to gather a list of supported content encoders and to advertise them through the client reactor. Because the client is now sending new frame types by default, this constitutes a compatibility break in the framing protocol. The media type version has been bumped accordingly. This will ensure existing clients won't attempt to send the new frames to old servers not supporting this explicit media type. I'm not bothering with the BC annotation because everything wireprotov2 is highly experimental and nobody should be running a server yet. Differential Revision: https://phab.mercurial-scm.org/D4922

wireprotov2: define and use stream encoders Now that we have basic support for defining stream encoding, it is time to start doing something with it. We define various classes implementing stream encoders/decoders for the defined encoding profiles. This is relatively straightforward. We teach the inputstream and outputstream classes how to encode, decode, and flush data. We then teach the clientreactor how to filter received data through the inputstream decoder. One of the features of the framing format is that streams can span requests. This is a differentiating feature from say HTTP/2, which associates streams with requests. By allowing streams to span requests, we can reuse compression context data across requests/responses. But in order to do this, we need a mechanism to "flush" the encoder at logical boundaries so that receivers receive all data where it is expected. And a "flush" event is distinct from a "finish" event from the perspective of certain compressors because a "flush" will retain compression context state whereas a "finish" operation will not. This is why encoders have both a flush() and a finish() and each uses specific flushing semantics on the underlying compressor. The added tests verify various behavior of decoders via clientreactor. These tests do test some compression behavior via use of outputstream. But for all intents and purposes, server reactor support for encoding is not yet implemented. Differential Revision: https://phab.mercurial-scm.org/D4921

lfs: avoid a potential variable reference before assignment error in cmdserver A coworker hit this once yesterday when pulling in thg (a retry worked), and then I hit it with strip after a pull. I had a difficult time recreating a test for this (at least one of the tricks was to not use '-R', which seems to cause reposetup() to be called for each command), so I'm not sure how large of a window there actually is for this. Calling reposetup() *after* the requirement is added will skip the hook entirely. The other issue I had was adding a couple `ui.status()` lines around the check that installs the hook. On Windows, the cmdserver process ballooned to 1.6GB and hung. Changing that to `ui.warn()` avoided the hang. It also hung on macOS, but without the large memory usage.

wireprotov2: establish dedicated classes for input and output streams Streams are unidirectional. As part of implementing encoding/decoding support, it became clear that it didn't make sense for a generic "stream" class to hold functionality related to both encoding and decoding. So we create new classes to represent the flavor of stream. Differential Revision: https://phab.mercurial-scm.org/D4920

wireprotov2: pass ui into clientreactor and serverreactor This will allow us to use config options to influence compression settings. Differential Revision: https://phab.mercurial-scm.org/D4919

wireprotov2: handle stream encoding settings frames Like what we just did for the server reactor, we teach the client reactor to handle stream encoding settings frames. The code is very similar. We define a method on the stream class to handle processing the data within the decoded frames. However, it doesn't yet do anything useful. Differential Revision: https://phab.mercurial-scm.org/D4918

wireprotov2: document client reactor actions We should document these so consumers have an easier life. Differential Revision: https://phab.mercurial-scm.org/D4917

wireprotov2: handle sender protocol settings frames We teach the server reactor to handle the optional sender protocol settings frames, which can only be sent at the beginning of frame exchange. Right now, we simply decode the data and record the sender protocol settings on the server reactor instance: we don't yet do anything meaningful with the data. Differential Revision: https://phab.mercurial-scm.org/D4916

wireprotov2: update stream encoding specification The encoding of data within streams in the frame-based protocol is not yet defined or implemented. This means that all data in wire protocol version 2 is currently being sent out raw, without compression. That's obviously not ideal. This commit formalizes the beginnings of stream encoding support in the protocol. I suspect we'll change behavior substantially in the future. My goal is to get something landed so we can use compression. We can build out more robust support later. Because the frame type ID changed, this is strictly BC. But existing code wasn't using the frame. I'll bump the framing protocol version later once code is introduced to use the new frame. Differential Revision: https://phab.mercurial-scm.org/D4915

cborutil: cast bytearray to bytes This code didn't like passing in bytearray instances. Let's cast bytearray to bytes so it works. Differential Revision: https://phab.mercurial-scm.org/D4914

tests: disable zstd in test This makes the test pass in pure installs. Differential Revision: https://phab.mercurial-scm.org/D4913