diff tests/test-revlog-mmapindex.t @ 34296:3c9691728237

revlog: add option to mmap revlog index

Following on from Jun Wu's patch last October[1], we have found that using mmap
for the revlog index in repos with large revlogs gives a noticable performance
improvment (~110ms on each hg invocation), particularly for commands that don't
touch the index very much.

This changeset adds this as an option, activated by a new experimental config
option so that it can be enabled on a per-repo basis. The configuration option
specifies an index size threshold at which Mercurial will switch to using mmap
to access the index.

If the configuration option is not specified, the default remains to load the
full file, which seems to be the best option for smaller repos.

Some initial performance numbers for average of 5 invocations of `hg log -l 5`
for different cache states:

| Repo: | HG | FB |
|---|---|---|
| Index size: | 2.3MB | much bigger |
| read (warm): | 237ms | 432ms |
| mmap (warm): | 227ms | 321ms |
|   | (-3%) | (-26%) |
| read (cold): | 397ms | 696ms |
| mmap (cold): | 410ms | 888ms |
|   | (+3%) | (+28%) |

[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-October/088737.html

Test Plan:
`hg log --config experimental.mmapindex=true`

Differential Revision: https://phab.mercurial-scm.org/D477

--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tests/test-revlog-mmapindex.t	Wed Sep 13 17:26:26 2017 +0000
@@ -0,0 +1,54 @@
+create verbosemmap.py
+  $ cat << EOF > verbosemmap.py
+  > # extension to make util.mmapread verbose
+  > 
+  > from __future__ import absolute_import
+  > 
+  > from mercurial import (
+  >     extensions,
+  >     util,
+  > )
+  > 
+  > def mmapread(orig, fp):
+  >     print "mmapping %s" % fp.name
+  >     return orig(fp)
+  > 
+  > def extsetup(ui):
+  >     extensions.wrapfunction(util, 'mmapread', mmapread)
+  > EOF
+
+setting up base repo
+  $ hg init a
+  $ cd a
+  $ touch a
+  $ hg add a
+  $ hg commit -qm base
+  $ for i in `$TESTDIR/seq.py 1 100` ; do
+  > echo $i > a
+  > hg commit -qm $i
+  > done
+
+set up verbosemmap extension
+  $ cat << EOF >> $HGRCPATH
+  > [extensions]
+  > verbosemmap=$TESTTMP/verbosemmap.py
+  > EOF
+
+mmap index which is now more than 4k long
+  $ hg log -l 5 -T '{rev}\n' --config experimental.mmapindexthreshold=4k
+  mmapping $TESTTMP/a/.hg/store/00changelog.i (glob)
+  100
+  99
+  98
+  97
+  96
+
+do not mmap index which is still less than 32k
+  $ hg log -l 5 -T '{rev}\n' --config experimental.mmapindexthreshold=32k
+  100
+  99
+  98
+  97
+  96
+
+  $ cd ..