comparison tests/test-bundle2.t @ 20876:ddd56f3eb786

bundle2: support for bundling and unbundling payload

We add the ability to bundle and unbundle a payload in parts. The payload is the
actual binary data of the part. It is used to convey all the applicative data.
For now we stick to very simple implementation with all the data fit in single
chunk. This open the door to some bundle2 testing usage. This will be improved before
bundle2 get used for real. We need to be able to stream the payload in multiple
part to exchange any changegroup efficiently. This simple version will do for
now.

Bundling and unbundling are done in the same changeset because the test for
parts is less modular. However, the result is not too complex.

20875:cc62c9d6887a

  > from mercurial import cmdutil
  > from mercurial import util
  > from mercurial import bundle2
  > cmdtable = {}
  > command = cmdutil.command(cmdtable)
  > 
  > @command('bundle2',
  >          [('', 'param', [], 'stream level parameter'),
  >           ('', 'parts', False, 'include some arbitrary parts to the bundle'),],
  >          '[OUTPUTFILE]')

  >     if opts['parts']:
  >        part = bundle2.part('test:empty')
  >        bundler.addpart(part)
  >        # add a second one to make sure we handle multiple parts
  >        part = bundle2.part('test:empty')
  >        bundler.addpart(part)
  > 
  >     if path is None:
  >        file = sys.stdout
  >     else:

  >             ui.write('    %s\n' % value)
  >     parts = list(unbundler)
  >     ui.write('parts count:   %i\n' % len(parts))
  >     for p in parts:
  >         ui.write('  :%s:\n' % p.type)
  > EOF
  $ cat >> $HGRCPATH << EOF
  > [extensions]
  > bundle2=$TESTTMP/bundle2.py
  > EOF

  start emission of HG20 stream
  bundle parameter: 
  start of parts
  bundle part: "test:empty"
  bundle part: "test:empty"
  end of bundle

  $ cat ../parts.hg2
  HG20\x00\x00\x00\r (esc)
  test:empty\x00\x00\x00\x00\x00\x00\x00\r (esc)
  test:empty\x00\x00\x00\x00\x00\x00\x00\x00 (no-eol) (esc)


  $ hg unbundle2 < ../parts.hg2
  options count: 0
  parts count:   2
    :test:empty:
    :test:empty:

  $ hg unbundle2 --debug < ../parts.hg2
  start processing of HG20 stream
  reading bundle2 stream parameters
  options count: 0

  payload chunk size: 0
  part header size: 13
  part type: "test:empty"
  part parameters: 0
  payload chunk size: 0
  part header size: 0
  end of bundle2 stream
  parts count:   2
    :test:empty:
    :test:empty:

20876:ddd56f3eb786

  > from mercurial import cmdutil
  > from mercurial import util
  > from mercurial import bundle2
  > cmdtable = {}
  > command = cmdutil.command(cmdtable)
  > 
  > ELEPHANTSSONG = """Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko
  > Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko
  > Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko."""
  > assert len(ELEPHANTSSONG) == 178 # future test say 178 bytes, trust it.
  > 
  > @command('bundle2',
  >          [('', 'param', [], 'stream level parameter'),
  >           ('', 'parts', False, 'include some arbitrary parts to the bundle'),],
  >          '[OUTPUTFILE]')

  >     if opts['parts']:
  >        part = bundle2.part('test:empty')
  >        bundler.addpart(part)
  >        # add a second one to make sure we handle multiple parts
  >        part = bundle2.part('test:empty')
  >        bundler.addpart(part)
  >        part = bundle2.part('test:song', data=ELEPHANTSSONG)
  >        bundler.addpart(part)
  > 
  >     if path is None:
  >        file = sys.stdout
  >     else:

  >             ui.write('    %s\n' % value)
  >     parts = list(unbundler)
  >     ui.write('parts count:   %i\n' % len(parts))
  >     for p in parts:
  >         ui.write('  :%s:\n' % p.type)
  >         ui.write('    payload: %i bytes\n' % len(p.data))
  > EOF
  $ cat >> $HGRCPATH << EOF
  > [extensions]
  > bundle2=$TESTTMP/bundle2.py
  > EOF

  start emission of HG20 stream
  bundle parameter: 
  start of parts
  bundle part: "test:empty"
  bundle part: "test:empty"
  bundle part: "test:song"
  end of bundle

  $ cat ../parts.hg2
  HG20\x00\x00\x00\r (esc)
  test:empty\x00\x00\x00\x00\x00\x00\x00\r (esc)
  test:empty\x00\x00\x00\x00\x00\x00\x00\x0c	test:song\x00\x00\x00\x00\x00\xb2Patali Dirapata, Cromda Cromda Ripalo, Pata Pata, Ko Ko Ko (esc)
  Bokoro Dipoulito, Rondi Rondi Pepino, Pata Pata, Ko Ko Ko
  Emana Karassoli, Loucra Loucra Ponponto, Pata Pata, Ko Ko Ko.\x00\x00\x00\x00\x00\x00 (no-eol) (esc)


  $ hg unbundle2 < ../parts.hg2
  options count: 0
  parts count:   3
    :test:empty:
      payload: 0 bytes
    :test:empty:
      payload: 0 bytes
    :test:song:
      payload: 178 bytes

  $ hg unbundle2 --debug < ../parts.hg2
  start processing of HG20 stream
  reading bundle2 stream parameters
  options count: 0

  payload chunk size: 0
  part header size: 13
  part type: "test:empty"
  part parameters: 0
  payload chunk size: 0
  part header size: 12
  part type: "test:song"
  part parameters: 0
  payload chunk size: 178
  payload chunk size: 0
  part header size: 0
  end of bundle2 stream
  parts count:   3
    :test:empty:
      payload: 0 bytes
    :test:empty:
      payload: 0 bytes
    :test:song:
      payload: 178 bytes