ui: fix progress debug log format strings to work on Python 3

tests: update `f` helper script to work on Python 3

tests: fix run-tests XML reporting on Python 3 cdatasafe wants to work in terms of bytes, but of course we have a unicode. Easy to work around, especially since we know we'll get utf-8 at the end.

tests: convert bisect support regex to binary The bisection feature of run-tests still fails tests with this because bisect itself doesn't work yet. We'll get there.

tests: decode bytes path to hg command before hitting with shellquote

tests: fix test-is-whitelisted check in run-tests Again, look for bytes in a set of bytes objects.

tests: fix run-tests "slow test" check Look for bytes in a set of bytes objects.

tests: fix run-tests default values in Test constructor As far as I can tell, default values are evaluated far earlier on Python 3.6 than 2.7, meaning we've got to be more careful about when we read the defaults dictionary for the kwarg defaults. Sigh. With this change, test-run-tests.t is significantly less of a mess under 3.6.

tests: add support for listing tests to run in a file This will make it easier for us to run the tests in the python3 whitelist in buildbot.

python3: remove test-update-dest.t from the whitelist I've poked at this on and off several times, and I can't figure out what regressed. Let's kick this out of the whitelist for now so that we can get the *rest* of our progress covered by the buildbots.

bundle2: move part processing to a separate function Now that the part processing loop is tiny, let's move it to a separate function. This will allow extensions to completely replace the part processing logic, without having to replace the overall bundle processing logic or the stream maintenance logic. This will be useful for the infinitepush extension, so it can completely take over receiving a bundle and rerouting it to a side store. This will also make it easier to upstream the infinitepush functionality later. Differential Revision: https://phab.mercurial-scm.org/D709

bundle2: remove unnecessary try finally This is no longer needed. Differential Revision: https://phab.mercurial-scm.org/D708

bundle2: move handler validation out of processpart As part of refactoring bundle part processing let's move handler validation to its own function. Differential Revision: https://phab.mercurial-scm.org/D707

bundle2: move processpart stream maintenance into part iterator The processpart function also did some stream maintenance, so let's move it to the part iterator as well, as part of moving all part iteration logic into the class. There is one place processpart is called outside of the normal loop, so we manually handle the seek there. The now-empty try/finally will be removed in a later patch, for ease of review. Differential Revision: https://phab.mercurial-scm.org/D706

merge with stable

py3: convert system strings to bytes in doctest of formatter.py

py3: rewrite stdout hack of doctest by using ui.pushbuffer() We can't use pycompat.stdout here because the doctest runner replaces sys.stdout with a string buffer.

py3: use bytes os.sep in doctest of pathutil.py

py3: fix doctests in patch.py to be compatible with Python 3 We were lucky that parsepatch() could concatenate a character slice as if it were a list of chunks.

py3: stop using bytes[n] in patch.py

error: move patch.PatchError so it can easily implement __bytes__ (API)

templater: extract shortest() logic from template function It can be useful for extensions to be able to produce the shortest unambiguous hash (including the in-tree "show" extension). That logic is currently inside the shortest() template function. Let's move it out of the templater. I've put it on revlog since it's closely related to revlog._partialmatch. We may also want a convenience method on context, but I'll leave that for a later patch. Differential Revision: https://phab.mercurial-scm.org/D724

bundle2: raise a more helpful error if building a bundle part header fails I've tripped on this several times now, and am tired of debugging. Now the header parts are part of the error message when the ''.join() fails, which makes debugging obvious.

hgwebdir: read 'web.template' untrusted The 'hgweb_mod.py' version of this read it untrusted. For consistency we align the two versions of this code.

configitems: register the 'web.templates' config

configitems: register the 'web.style' config

configitems: register the 'web.stripes' config

configitems: register the 'web.refreshinterval' config

configitems: register the 'web.prefix' config

configitems: register the 'web.port' config

configitems: register the 'web.ipv6' config

configitems: register the 'web.errorlog' config

configitems: register the 'web.encoding' config

configitems: register the 'web.description' config

configitems: register the 'web.descend' config

configitems: register the 'web.deny_read' config

configitems: register the 'web.csp' config

configitems: register the 'web.collapse' config

configitems: register the 'web.certificate' config

configitems: register the 'web.cacerts' config

configitems: register the 'web.baseurl' config

configitems: register the 'web.allow_read' config

configitems: register the 'web.allow_archive' config

configitems: register the 'web.address' config

configitems: register the 'web.accesslog' config

web: use '_unset' default value for proxy config method This special value is needed to make sure registered default value are taken in account.

tests: use int() instead of long() in test-pathencode.py

tests: add xrange alias for test-pathencode.py

python3: another 3 tests whitelisted

repair: reliably obtain bytestr of node ids

bundles: turn nbchanges int into a bytestr using pycompat.bytestr Fixes some python 3 failures.

exchange: hit opargs with pycompat.strkwargs before **-ing it Fixes Python 3 problems.

revlog: update signature of dummy addgroup() in bundlerepo and unionrepo Per c8b6ed51386b, 2f5c45fe3a3b and 00e3f909907f.

py3: use 'surrogatepass' error handler to process U+DCxx transparently It's disallowed by default on Python 3. https://docs.python.org/3/library/codecs.html#error-handlers

py3: don't pass bytes to array.array()

py3: wrap bytes in encoding.from/toutf8b() with bytestr

py3: iterate bytes as a byte string in store.lowerencode()

py3: use bytechr() in store._buildlowerencodefun()

store: give name to lowerencode function lambda function isn't easy to track in traceback.

py3: iterate bytes as a byte string in dagparser.py