view tests/test-convert-bzr-114.t @ 39560:84bf6ded9317

wireprotoframing: buffer emitted data to reduce frame count

An upcoming commit introduces a wire protocol command that can emit
hundreds of thousands of small objects. Without a buffering layer,
we would emit a single, small frame for every object. Performance
profiling revealed this to be a source of significant overhead for
both client and server.

This commit introduces a very crude buffering layer so that we emit
fewer, bigger frames in such a scenario. This code will likely get
rewritten in the future to be part of the streams API, as we'll
need a similar strategy for compressing data. I don't want to think
about it too much at the moment though.

server
before: user 32.500+0.000 sys 1.160+0.000
after:  user 20.230+0.010 sys 0.180+0.000

client
before: user 133.400+0.000 sys 93.120+0.000
after:  user  68.370+0.000 sys 32.950+0.000

This appears to indicate we have significant overhead in the frame
processing code on both client and server. It might be worth profiling
that at some point...

Differential Revision: https://phab.mercurial-scm.org/D4473

#require bzr bzr114

  $ . "$TESTDIR/bzr-definitions"

The file/directory replacement can only be reproduced on
bzr >= 1.4. Merge it back in test-convert-bzr-directories once
this version becomes mainstream.
replace file with dir

  $ mkdir test-replace-file-with-dir
  $ cd test-replace-file-with-dir
  $ bzr init -q source
  $ cd source
  $ echo d > d
  $ bzr add -q d
  $ bzr commit -q -m 'add d file'
  $ rm d
  $ mkdir d
  $ bzr add -q d
  $ bzr commit -q -m 'replace with d dir'
  $ echo a > d/a
  $ bzr add -q d/a
  $ bzr commit -q -m 'add d/a'
  $ cd ..
  $ hg convert source source-hg
  initializing destination source-hg repository
  scanning source...
  sorting...
  converting...
  2 add d file
  1 replace with d dir
  0 add d/a
  $ manifest source-hg tip
  % manifest of tip
  644   d/a
  $ cd source-hg
  $ hg update
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ cd ../..