comparison mercurial/bundle2.py @ 23009:90f86ad3d4ff

bundle2: change header size and make them signed (new format)

We are changing all integers that denote the size of a chunk to read to int32.
There are two main motivations for that.

First, we change everything to the same width (32 bits) to make it possible for
a reasonably agnostic actor to forward a bundle2 without any extra processing.
With this change, this could be achieved by just reading int32s and forwarding
chunks of the size read. A bit a smartness would be logic to detect the end of
stream but nothing too complicated.

Second, we need some capacity to transmit special information during the bundle
processing. For example we would like to be able to raise an exception while a
part is being read if this exception happend while this part was generated.
Having signed integer let us use negative numbers to trigger special events
during the parsing of the bundle.

The format is renamed for B2X to B2Y because this breaks binary
compatibility. The B2X format support is dropped. It was experimental to
allow this kind of things. All elements not directly related to the binary
format remain flagged "b2x" because they are still compatible.

23008:d3137827016a

stream level parameters
------------------------

Binary format is as follow

:params size: (16 bits integer)

  The total number of Bytes used by the parameters

:params value: arbitrary number of Bytes

Payload part
------------------------

Binary format is as follow

:header size: (16 bits inter)

  The total number of Bytes used by the part headers. When the header is empty
  (size = 0) this is interpreted as the end of stream marker.

:header:

:payload:

    payload is a series of `<chunksize><chunkdata>`.

    `chunksize` is a 32 bits integer, `chunkdata` are plain bytes (as much as
    `chunksize` says)` The payload part is concluded by a zero size chunk.

    The current implementation always produces either zero or one chunk.
    This is an implementation limitation that will ultimately be lifted.

Bundle processing
============================

Each part is processed in order using a "part handler". Handler are registered

from i18n import _

_pack = struct.pack
_unpack = struct.unpack

_magicstring = 'HG2X'

_fstreamparamsize = '>H'
_fpartheadersize = '>H'
_fparttypesize = '>B'
_fpartid = '>I'
_fpayloadsize = '>I'
_fpartparamcount = '>BB'

preferedchunksize = 4096

def _makefpartparamsizes(nbparams):

        if header is None:
            header = self._readexact(4)
            magic, version = header[0:2], header[2:4]
            if magic != 'HG':
                raise util.Abort(_('not a Mercurial bundle'))
            if version != '2X':
                raise util.Abort(_('unknown bundle version %s') % version)
        self.ui.debug('start processing of %s stream\n' % header)

    @util.propertycache
    def params(self):

            data = self._payloadstream.read(size)
        if size is None or len(data) < size:
            self.consumed = True
        return data

capabilities = {'HG2X': (),
                'b2x:listkeys': (),
                'b2x:pushkey': (),
                'b2x:changegroup': (),
               }

23009:90f86ad3d4ff

stream level parameters
------------------------

Binary format is as follow

:params size: int32

  The total number of Bytes used by the parameters

:params value: arbitrary number of Bytes

Payload part
------------------------

Binary format is as follow

:header size: int32

  The total number of Bytes used by the part headers. When the header is empty
  (size = 0) this is interpreted as the end of stream marker.

:header:

:payload:

    payload is a series of `<chunksize><chunkdata>`.

    `chunksize` is an int32, `chunkdata` are plain bytes (as much as
    `chunksize` says)` The payload part is concluded by a zero size chunk.

    The current implementation always produces either zero or one chunk.
    This is an implementation limitation that will ultimately be lifted.

    `chunksize` can be negative to trigger special case processing. No such
    processing is in place yet.

Bundle processing
============================

Each part is processed in order using a "part handler". Handler are registered

from i18n import _

_pack = struct.pack
_unpack = struct.unpack

_magicstring = 'HG2Y'

_fstreamparamsize = '>i'
_fpartheadersize = '>i'
_fparttypesize = '>B'
_fpartid = '>I'
_fpayloadsize = '>i'
_fpartparamcount = '>BB'

preferedchunksize = 4096

def _makefpartparamsizes(nbparams):

        if header is None:
            header = self._readexact(4)
            magic, version = header[0:2], header[2:4]
            if magic != 'HG':
                raise util.Abort(_('not a Mercurial bundle'))
            if version != '2Y':
                raise util.Abort(_('unknown bundle version %s') % version)
        self.ui.debug('start processing of %s stream\n' % header)

    @util.propertycache
    def params(self):

            data = self._payloadstream.read(size)
        if size is None or len(data) < size:
            self.consumed = True
        return data

capabilities = {'HG2Y': (),
                'b2x:listkeys': (),
                'b2x:pushkey': (),
                'b2x:changegroup': (),
               }