view tests/test-parseindex @ 12252:4481f8a93c7a stable

convert/darcs: handle non-ASCII metadata in darcs changelog (issue2354)

Given a commit author or message with non-ASCII characters in a darcs
repo, convert would raise a UnicodeEncodeError when adding changesets
to the hg changelog.

This happened because etree returns back unicode objects for any text
it can't encode into ASCII. convert was passing these objects to
changelog.add(), which would then attempt encoding.fromlocal() on
them.

This patch ensures converter_source.recode() is called on each piece
of commit data returned by etree.

(Also note that darcs is currently encoding agnostic and will print
out whatever is in a patch's metadata byte-for-byte, even in the XML
changelog.)

#!/bin/sh
#
# revlog.parseindex must be able to parse the index file even if
# an index entry is split between two 64k blocks.  The ideal test
# would be to create an index file with inline data where
# 64k < size < 64k + 64 (64k is the size of the read buffer, 64 is
# the size of an index entry) and with an index entry starting right
# before the 64k block boundary, and try to read it.
#
# We approximate that by reducing the read buffer to 1 byte.
#

hg init a
cd a
echo abc > foo
hg add foo
hg commit -m 'add foo' -d '1000000 0'

echo >> foo
hg commit -m 'change foo' -d '1000001 0'
hg log -r 0:

cat >> test.py << EOF
from mercurial import changelog, util
from mercurial.node import *

class singlebyteread(object):
    def __init__(self, real):
        self.real = real

    def read(self, size=-1):
        if size == 65536:
            size = 1
        return self.real.read(size)

    def __getattr__(self, key):
        return getattr(self.real, key)

def opener(*args):
    o = util.opener(*args)
    def wrapper(*a):
        f = o(*a)
        return singlebyteread(f)
    return wrapper

cl = changelog.changelog(opener('.hg/store'))
print len(cl), 'revisions:'
for r in cl:
    print short(cl.node(r))
EOF

python test.py