view tests/test-parseindex.t @ 24725:ee751d47cf2c

vfs: add walk

To eliminate "path prefix" (= "the root of vfs") part from "dirpath"
yielded by "os.walk()" correctly, "path prefix" should have "os.sep"
at the end of own string, but it isn't easy to ensure it, because:

  - examination by "path.endswith(os.sep)" isn't portable

    Some problematic encodings use 0x5c (= "os.sep" on Windows) as the
    tail byte of some multi-byte characters.

  - "os.path.join(path, '')" isn't portable

    With Python 2.7.9, this invocation doesn't add "os.sep" at the end
    of UNC path (see issue4557 for detail).

Python 2.7.9 changed also behavior of "os.path.normpath()" (see *) and
"os.path.splitdrive()" for UNC path.

    vfs root        normpath       splitdrive          os.sep required
    =============== ============== =================== ============
    z:\             z:\            z: + \              no
    z:\foo          z:\foo         z: + \foo           yes
    z:\foo\         z:\foo         z: + \foo           yes

    [before Python 2.7.9]
    \\foo\bar       \\foo\bar      '' + \\foo\bar      yes
    \\foo\bar\      \\foo\bar  (*) '' + \\foo\bar      yes
    \\foo\bar\baz   \\foo\bar\baz  '' + \\foo\bar\baz  yes
    \\foo\bar\baz\  \\foo\bar\baz  '' + \\foo\bar\baz  yes

    [Python 2.7.9]
    \\foo\bar       \\foo\bar      \\foo\bar + ''      yes
    \\foo\bar\      \\foo\bar\ (*) \\foo\bar + \       no
    \\foo\bar\baz   \\foo\bar\baz  \\foo\bar + \baz    yes
    \\foo\bar\baz\  \\foo\bar\baz  \\foo\bar + \baz    yes

If it is ensured that "normpath()"-ed vfs root is passed to
"splitdrive()", adding "os.sep" is required only when "path" part of
"splitdrive()" result isn't "os.sep" itself. This is just what
"pathutil.nameasprefix()" examines.

This patch applies "os.path.normpath()" on "self.join(None)"
explicitly, because it isn't ensured that vfs root is already
normalized: vfs itself is constructed with "realpath=False" (= avoid
normalizing in "vfs.__init__()") in many code paths.

This normalization should be much cheaper than subsequent file I/O for
directory traversal.

revlog.parseindex must be able to parse the index file even if
an index entry is split between two 64k blocks.  The ideal test
would be to create an index file with inline data where
64k < size < 64k + 64 (64k is the size of the read buffer, 64 is
the size of an index entry) and with an index entry starting right
before the 64k block boundary, and try to read it.
We approximate that by reducing the read buffer to 1 byte.

  $ hg init a
  $ cd a
  $ echo abc > foo
  $ hg add foo
  $ hg commit -m 'add foo'
  $ echo >> foo
  $ hg commit -m 'change foo'
  $ hg log -r 0:
  changeset:   0:7c31755bf9b5
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     add foo
  
  changeset:   1:26333235a41c
  tag:         tip
  user:        test
  date:        Thu Jan 01 00:00:00 1970 +0000
  summary:     change foo
  
  $ cat >> test.py << EOF
  > from mercurial import changelog, scmutil
  > from mercurial.node import *
  > 
  > class singlebyteread(object):
  >     def __init__(self, real):
  >         self.real = real
  > 
  >     def read(self, size=-1):
  >         if size == 65536:
  >             size = 1
  >         return self.real.read(size)
  > 
  >     def __getattr__(self, key):
  >         return getattr(self.real, key)
  > 
  > def opener(*args):
  >     o = scmutil.opener(*args)
  >     def wrapper(*a):
  >         f = o(*a)
  >         return singlebyteread(f)
  >     return wrapper
  > 
  > cl = changelog.changelog(opener('.hg/store'))
  > print len(cl), 'revisions:'
  > for r in cl:
  >     print short(cl.node(r))
  > EOF
  $ python test.py
  2 revisions:
  7c31755bf9b5
  26333235a41c

  $ cd ..