py3: more passing tests (ten this time) Differential Revision: https://phab.mercurial-scm.org/D2670

util: fix unsafe url abort with bytestr() on url Differential Revision: https://phab.mercurial-scm.org/D2669

xdiff: fix builds on Windows This works on my ancient Fedora system too, without warnings. There are, however, warnings about various 64 to 32 bit conversions on Windows that need to be examined.

py3: use bytes() instead of str() Differential Revision: https://phab.mercurial-scm.org/D2665

py3: use b"%d" instead of str() to convert integers to bytes Differential Revision: https://phab.mercurial-scm.org/D2664

py3: use util.forcebytestr to convert testedwith value to bytes Bad extensions can put anything in testedwith so we should use util.forcebytestr here. Differential Revision: https://phab.mercurial-scm.org/D2663

py3: use pycompat.bytestr instead of str Differential Revision: https://phab.mercurial-scm.org/D2648

setdiscovery: avoid a Yoda condition Differential Revision: https://phab.mercurial-scm.org/D2646

setdiscovery: remove unnecessary sample size limiting Both _takequicksample() and _takefullsample() already limit their result to the request size, so there's no need to let the caller do that again. Differential Revision: https://phab.mercurial-scm.org/D2645

setdiscovery: remove initialsamplesize from a condition It seems more direct to compare the actual sample size. That way we can change the sample taken earlier in the code without breaking the condition. Differential Revision: https://phab.mercurial-scm.org/D2644

setdiscovery: back out changeset 5cfdf6137af8 (issue5809) As explained in the bug report, this commit caused a performance regression. The problem occurs when the local repo has very many heads. Before 5cfdf6137af8, we used to get the remote's list of heads and if these heads mostly overlapped with the local repo's heads, we would mark these common heads as common, which would greatly reduce the size of the set of undecided nodes. Note that a similar problem existed before 5cfdf6137af8: If the local repo had very many heads and the server just had a few (or many heads from a disjoint set), we would do the same kind of slow discovery as we would with 5cfdf6137af8 in the case where local and remote repos share a large set of common nodes. For now, we just back out 5cfdf6137af8. We should improve the discovery in the "local has many heads, remote has few heads" case, but let's do that after backing this out. Differential Revision: https://phab.mercurial-scm.org/D2643

hgweb: fix up trailing slash detection on Python 3 Fixes a couple of hgweb tests. Differential Revision: https://phab.mercurial-scm.org/D2661

hgweb: convert req.form to bytes for all keys and values This is just going to be a lot cleaner for our internals. Differential Revision: https://phab.mercurial-scm.org/D2660

templater: show repr of string we're rejecting I feel like this should make it a little easier to hunt down problems. Differential Revision: https://phab.mercurial-scm.org/D2659

webutil: some %d instead of %s love on ints Differential Revision: https://phab.mercurial-scm.org/D2658

py3: whitelist three more cases Differential Revision: https://phab.mercurial-scm.org/D2657

archival: our filenames are bytes, not strs Differential Revision: https://phab.mercurial-scm.org/D2656

archival: tar file modes need to be sysstrs Differential Revision: https://phab.mercurial-scm.org/D2655

archival: fsdecode paths before passing to tar or zip objects Both of these traffic in unicodes for filenames on Python 3, and inspection of the tarfile module shows that it uses the filesystem encoding, so fsdecode is the right choice. Differential Revision: https://phab.mercurial-scm.org/D2654

py3: add b'' prefixes in tests/test-minirst.py # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D2653

py3: make sure __repr__ returns a str # skip-blame because just r'' prefix Differential Revision: https://phab.mercurial-scm.org/D2652

py3: make sure regular expressions are bytes # skip-blame because just b'' prefix Differential Revision: https://phab.mercurial-scm.org/D2651

py3: use bytes instead of str to make sure we use bytes internally Differential Revision: https://phab.mercurial-scm.org/D2650

py3: use util.forcebytestr instead of str to convert error messages Differential Revision: https://phab.mercurial-scm.org/D2649

lock: block signal interrupt while making a lock file On Windows where symlink isn't supported, util.makelock() could leave an empty file if interrupted immediately after os.open(). This empty lock never dies as it has no process id recorded. ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL) # an interrupt may occur here os.write(ld, info) os.close(ld) This was a long-standing bug of TortoiseHg which runs a command-server and kills it by CTRL_C_EVENT, reported by random Windows users. https://bitbucket.org/tortoisehg/thg/issues/4873/#comment-43591129 At first, I tried to fix makelock() to clean up a stale lock file, which turned out to be hard because any instructions may be interrupted by a signal. ld = None try: # CALL_FUNCTION # os.open(...) # an interrupt may occur here # STORE_FAST # ld = ... ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL) os.write(ld, info) ... return True except: if ld: ... os.unlink(pathname) return False So I decided to block signals by temporarily replacing the signal handlers so makelcok() and held = 1 will never be interrupted. Many thanks to Fernando Najera for investigating the issue.

fuzz: add some more docs about building/running fuzzers Differential Revision: https://phab.mercurial-scm.org/D2635

util: also silence py3 warnings from codec module Fixes warnings like this: + mercurial/util.py:2446: DeprecationWarning: invalid escape sequence '\d' + return codecs.escape_decode(s)[0] Differential Revision: https://phab.mercurial-scm.org/D2642

test-annotate: set stdin and stdout to binary to get CR unmodified

test-annotate: rewrite sed with some python I hope this will fix the test failure seen on FreeBSD and Windows.

docs: small fixes for profiling.nested and the overall description - profiling.nested defaults to 0, not 5 - profiling is not always done with lsprof Differential Revision: https://phab.mercurial-scm.org/D2641