tests/test-parseindex

findrenames: Optimise "addremove -s100" by matching files by their SHA1 hashes.

We speed up 'findrenames' for the usecase when a user specifies they
want a similarity of 100% by matching files by their exact SHA1 hash
value. This reduces the number of comparisons required to find exact
matches from O(n^2) to O(n).

While it would be nice if we could just use mercurial's pre-calculated
SHA1 hash for existing files, this hash includes the file's ancestor
information making it unsuitable for our purposes. Instead, we calculate
the hash of old content from scratch.

The following benchmarks were taken on the current head of crew:

addremove 100% similarity:
  rm -rf *; hg up -C; mv tests tests.new
  hg --time addremove -s100 --dry-run

  before:  real 176.350 secs (user 128.890+0.000 sys 47.430+0.000)
  after:   real   2.130 secs (user   1.890+0.000 sys  0.240+0.000)

addremove 75% similarity:
  rm -rf *; hg up -C; mv tests tests.new; \
      for i in tests.new/*; do echo x >> $i; done
  hg --time addremove -s75  --dry-run

  before: real 264.560 secs (user 215.130+0.000 sys 49.410+0.000)
  after:  real 218.710 secs (user 172.790+0.000 sys 45.870+0.000)

#!/bin/sh
#
# revlog.parseindex must be able to parse the index file even if
# an index entry is split between two 64k blocks.  The ideal test
# would be to create an index file with inline data where
# 64k < size < 64k + 64 (64k is the size of the read buffer, 64 is
# the size of an index entry) and with an index entry starting right
# before the 64k block boundary, and try to read it.
#
# We approximate that by reducing the read buffer to 1 byte.
#

hg init a
cd a
echo abc > foo
hg add foo
hg commit -m 'add foo' -d '1000000 0'

echo >> foo
hg commit -m 'change foo' -d '1000001 0'
hg log -r 0:

cat >> test.py << EOF
from mercurial import changelog, util
from mercurial.node import *

class singlebyteread(object):
    def __init__(self, real):
        self.real = real

    def read(self, size=-1):
        if size == 65536:
            size = 1
        return self.real.read(size)

    def __getattr__(self, key):
        return getattr(self.real, key)

def opener(*args):
    o = util.opener(*args)
    def wrapper(*a):
        f = o(*a)
        return singlebyteread(f)
    return wrapper

cl = changelog.changelog(opener('.hg/store'))
print len(cl), 'revisions:'
for r in cl:
    print short(cl.node(r))
EOF

python test.py