bundle2: inline changegroup.readexactly()
Profiling reveals this loop is pretty tight. Literally any
function call elimination can make a big difference.
This commit inlines the relatively trivial changegroup.readexactly()
method inside the loop.
The results with `hg perfbundleread` on a bundle of the Firefox repo
speak for themselves:
! read(8k)
! wall 0.679730 comb 0.680000 user 0.140000 sys 0.540000 (best of 15)
! read(16k)
! wall 0.577228 comb 0.570000 user 0.080000 sys 0.490000 (best of 17)
! read(32k)
! wall 0.516060 comb 0.520000 user 0.040000 sys 0.480000 (best of 20)
! read(128k)
! wall 0.496378 comb 0.490000 user 0.010000 sys 0.480000 (best of 20)
! bundle2 iterparts()
! wall 3.460903 comb 3.460000 user 2.760000 sys 0.700000 (best of 3)
! wall 3.056811 comb 3.050000 user 2.340000 sys 0.710000 (best of 4)
! bundle2 iterparts() seekable
! wall 4.312722 comb 4.310000 user 3.480000 sys 0.830000 (best of 3)
! wall 4.007676 comb 4.000000 user 3.170000 sys 0.830000 (best of 3)
! bundle2 part seek()
! wall 6.754764 comb 6.740000 user 3.970000 sys 2.770000 (best of 3)
! wall 6.267110 comb 6.250000 user 3.480000 sys 2.770000 (best of 3)
! bundle2 part read(8k)
! wall 3.668004 comb 3.660000 user 2.960000 sys 0.700000 (best of 3)
! wall 3.404164 comb 3.400000 user 2.650000 sys 0.750000 (best of 3)
! bundle2 part read(16k)
! wall 3.489196 comb 3.480000 user 2.750000 sys 0.730000 (best of 3)
! wall 3.197972 comb 3.200000 user 2.490000 sys 0.710000 (best of 4)
! bundle2 part read(32k)
! wall 3.388569 comb 3.380000 user 2.640000 sys 0.740000 (best of 3)
! wall 3.060557 comb 3.060000 user 2.340000 sys 0.720000 (best of 4)
! bundle2 part read(128k)
! wall 3.276415 comb 3.270000 user 2.560000 sys 0.710000 (best of 4)
! wall 2.952209 comb 2.950000 user 2.230000 sys 0.720000 (best of 4)
Differential Revision: https://phab.mercurial-scm.org/D1392
# Create server
$ hg init server
$ cd server
$ cat >> .hg/hgrc << EOF
> [extensions]
> extension=$TESTDIR/flagprocessorext.py
> EOF
$ cd ../
# Clone server and enable extensions
$ hg clone -q server client
$ cd client
$ cat >> .hg/hgrc << EOF
> [extensions]
> extension=$TESTDIR/flagprocessorext.py
> EOF
# Commit file that will trigger the noop extension
$ echo '[NOOP]' > noop
$ hg commit -Aqm "noop"
# Commit file that will trigger the base64 extension
$ echo '[BASE64]' > base64
$ hg commit -Aqm 'base64'
# Commit file that will trigger the gzip extension
$ echo '[GZIP]' > gzip
$ hg commit -Aqm 'gzip'
# Commit file that will trigger noop and base64
$ echo '[NOOP][BASE64]' > noop-base64
$ hg commit -Aqm 'noop+base64'
# Commit file that will trigger noop and gzip
$ echo '[NOOP][GZIP]' > noop-gzip
$ hg commit -Aqm 'noop+gzip'
# Commit file that will trigger base64 and gzip
$ echo '[BASE64][GZIP]' > base64-gzip
$ hg commit -Aqm 'base64+gzip'
# Commit file that will trigger base64, gzip and noop
$ echo '[BASE64][GZIP][NOOP]' > base64-gzip-noop
$ hg commit -Aqm 'base64+gzip+noop'
# TEST: ensure the revision data is consistent
$ hg cat noop
[NOOP]
$ hg debugdata noop 0
[NOOP]
$ hg cat -r . base64
[BASE64]
$ hg debugdata base64 0
W0JBU0U2NF0K (no-eol)
$ hg cat -r . gzip
[GZIP]
$ hg debugdata gzip 0
x\x9c\x8bv\x8f\xf2\x0c\x88\xe5\x02\x00\x08\xc8\x01\xfd (no-eol) (esc)
$ hg cat -r . noop-base64
[NOOP][BASE64]
$ hg debugdata noop-base64 0
W05PT1BdW0JBU0U2NF0K (no-eol)
$ hg cat -r . noop-gzip
[NOOP][GZIP]
$ hg debugdata noop-gzip 0
x\x9c\x8b\xf6\xf3\xf7\x0f\x88\x8dv\x8f\xf2\x0c\x88\xe5\x02\x00\x1dH\x03\xf1 (no-eol) (esc)
$ hg cat -r . base64-gzip
[BASE64][GZIP]
$ hg debugdata base64-gzip 0
eJyLdnIMdjUziY12j/IMiOUCACLBBDo= (no-eol)
$ hg cat -r . base64-gzip-noop
[BASE64][GZIP][NOOP]
$ hg debugdata base64-gzip-noop 0
eJyLdnIMdjUziY12j/IMiI328/cPiOUCAESjBi4= (no-eol)
# Push to the server
$ hg push
pushing to $TESTTMP/server (glob)
searching for changes
adding changesets
adding manifests
adding file changes
added 7 changesets with 7 changes to 7 files
# Initialize new client (not cloning) and setup extension
$ cd ..
$ hg init client2
$ cd client2
$ cat >> .hg/hgrc << EOF
> [paths]
> default = $TESTTMP/server
> [extensions]
> extension=$TESTDIR/flagprocessorext.py
> EOF
# Pull from server and update to latest revision
$ hg pull default
pulling from $TESTTMP/server (glob)
requesting all changes
adding changesets
adding manifests
adding file changes
added 7 changesets with 7 changes to 7 files
new changesets 07b1b9442c5b:6e48f4215d24
(run 'hg update' to get a working copy)
$ hg update
7 files updated, 0 files merged, 0 files removed, 0 files unresolved
# TEST: ensure the revision data is consistent
$ hg cat noop
[NOOP]
$ hg debugdata noop 0
[NOOP]
$ hg cat -r . base64
[BASE64]
$ hg debugdata base64 0
W0JBU0U2NF0K (no-eol)
$ hg cat -r . gzip
[GZIP]
$ hg debugdata gzip 0
x\x9c\x8bv\x8f\xf2\x0c\x88\xe5\x02\x00\x08\xc8\x01\xfd (no-eol) (esc)
$ hg cat -r . noop-base64
[NOOP][BASE64]
$ hg debugdata noop-base64 0
W05PT1BdW0JBU0U2NF0K (no-eol)
$ hg cat -r . noop-gzip
[NOOP][GZIP]
$ hg debugdata noop-gzip 0
x\x9c\x8b\xf6\xf3\xf7\x0f\x88\x8dv\x8f\xf2\x0c\x88\xe5\x02\x00\x1dH\x03\xf1 (no-eol) (esc)
$ hg cat -r . base64-gzip
[BASE64][GZIP]
$ hg debugdata base64-gzip 0
eJyLdnIMdjUziY12j/IMiOUCACLBBDo= (no-eol)
$ hg cat -r . base64-gzip-noop
[BASE64][GZIP][NOOP]
$ hg debugdata base64-gzip-noop 0
eJyLdnIMdjUziY12j/IMiI328/cPiOUCAESjBi4= (no-eol)
# TEST: ensure a missing processor is handled
$ echo '[FAIL][BASE64][GZIP][NOOP]' > fail-base64-gzip-noop
$ hg commit -Aqm 'fail+base64+gzip+noop'
abort: missing processor for flag '0x1'!
[255]
$ rm fail-base64-gzip-noop
# TEST: ensure we cannot register several flag processors on the same flag
$ cat >> .hg/hgrc << EOF
> [extensions]
> extension=$TESTDIR/flagprocessorext.py
> duplicate=$TESTDIR/flagprocessorext.py
> EOF
$ hg debugrebuilddirstate
Traceback (most recent call last):
File "*/mercurial/extensions.py", line *, in _runextsetup (glob)
extsetup(ui)
File "*/tests/flagprocessorext.py", line *, in extsetup (glob)
validatehash,
File "*/mercurial/revlog.py", line *, in addflagprocessor (glob)
raise error.Abort(msg)
Abort: cannot register multiple processors on flag '0x8'.
*** failed to set up extension duplicate: cannot register multiple processors on flag '0x8'.
$ hg st 2>&1 | egrep 'cannot register multiple processors|flagprocessorext'
File "*/tests/flagprocessorext.py", line *, in extsetup (glob)
Abort: cannot register multiple processors on flag '0x8'.
*** failed to set up extension duplicate: cannot register multiple processors on flag '0x8'.
File "*/tests/flagprocessorext.py", line *, in b64decode (glob)
$ cd ..
# TEST: bundle repo
$ hg init bundletest
$ cd bundletest
$ cat >> .hg/hgrc << EOF
> [extensions]
> flagprocessor=$TESTDIR/flagprocessorext.py
> EOF
$ for i in 0 single two three 4; do
> echo '[BASE64]a-bit-longer-'$i > base64
> hg commit -m base64-$i -A base64
> done
$ hg update 2 -q
$ echo '[BASE64]a-bit-longer-branching' > base64
$ hg commit -q -m branching
$ hg bundle --base 1 bundle.hg
4 changesets found
$ hg --config extensions.strip= strip -r 2 --no-backup --force -q
$ hg -R bundle.hg log --stat -T '{rev} {desc}\n' base64
5 branching
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
4 base64-4
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
3 base64-three
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
2 base64-two
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
1 base64-single
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
0 base64-0
base64 | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
$ hg bundle -R bundle.hg --base 1 bundle-again.hg -q
$ hg -R bundle-again.hg log --stat -T '{rev} {desc}\n' base64
5 branching
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
4 base64-4
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
3 base64-three
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
2 base64-two
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
1 base64-single
base64 | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
0 base64-0
base64 | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
$ rm bundle.hg bundle-again.hg
# TEST: hg status
$ hg status
$ hg diff