view tests/test-debug-revlog-stats.t @ 51108:0250e45040f1

revlog: add a small cache of unfiltered chunk

This can provides a massive boost to the reading of multiple revision and the
computation of a valid delta chain.

This greatly help operation like `hg log --patch`, delta computation (helping
pull/unbundle), linkrev adjustment (helping copy tracing).

A first round of benchmark for `hg log --patch --limit 1000` shows improvement
in the 10-20% range on "small" repository like pypy or mercurial and large
improvements (about 33%) for more complex ones like netbeans and mozilla's.

These speeds up are consistent with the improvement to `hg pull` (from a server
sending poor deltas) I saw benchmarking this last year. Further benchmark will
be run during the freeze.

I added some configuration in the experimental space to be able to further test
the effect of various tuning for now. This feature should fit well in the
"usage/resource profile" configuration that we should land next cycle.

When it does not provides a benefit the overhead of the cache seem to be around
2%, a small price for the big improvement. In addition I believe we could shave
most of this overhead with a more efficent lru implementation.

Force revlog max inline value to be smaller than default

  $ mkdir $TESTTMP/ext
  $ cat << EOF > $TESTTMP/ext/small_inline.py
  > from mercurial import revlog
  > revlog._maxinline = 8
  > EOF

  $ cat << EOF >> $HGRCPATH
  > [extensions]
  > small_inline=$TESTTMP/ext/small_inline.py
  > EOF

  $ hg init repo
  $ cd repo

Try on an empty repository

  $ hg debug-revlog-stats
  rev-count   data-size inl type      target 

  $ mkdir folder
  $ touch a b folder/c folder/d
  $ hg commit -Aqm 0
  $ echo "text" > a
  $ hg rm b
  $ echo "longer string" > folder/d
  $ hg commit -Aqm 1

Differences in data size observed with pure is due to different compression
algorithms

  $ hg debug-revlog-stats
  rev-count   data-size inl type      target 
          2         138 no  changelog  (no-pure !)
          2         137 no  changelog  (pure !)
          2         177 no  manifest   (no-pure !)
          2         168 no  manifest   (pure !)
          2           6 yes file      a
          1           0 yes file      b
          1           0 yes file      folder/c
          2          15 no  file      folder/d

Test 'changelog' command argument

  $ hg debug-revlog-stats -c
  rev-count   data-size inl type      target 
          2         138 no  changelog  (no-pure !)
          2         137 no  changelog  (pure !)

Test 'manifest' command argument

  $ hg debug-revlog-stats -m
  rev-count   data-size inl type      target 
          2         177 no  manifest   (no-pure !)
          2         168 no  manifest   (pure !)

Test 'file' command argument

  $ hg debug-revlog-stats -f
  rev-count   data-size inl type      target 
          2           6 yes file      a
          1           0 yes file      b
          1           0 yes file      folder/c
          2          15 no  file      folder/d

Test multiple command arguments

  $ hg debug-revlog-stats -cm
  rev-count   data-size inl type      target 
          2         138 no  changelog  (no-pure !)
          2         137 no  changelog  (pure !)
          2         177 no  manifest   (no-pure !)
          2         168 no  manifest   (pure !)