py3: byte-stringify test-config.t and test-config-env.py

py3: use startswith() instead of slicing to detect leading whitespace

archival: use py3 friendly replacements for chr() and long() Differential Revision: https://phab.mercurial-scm.org/D2673

archival: ensure file mode for gzipfile is sysstr Differential Revision: https://phab.mercurial-scm.org/D2672

archival: fix a missing r'' on a kwargs check # skip-blame just an r prefix Differential Revision: https://phab.mercurial-scm.org/D2671

py3: more passing tests (ten this time) Differential Revision: https://phab.mercurial-scm.org/D2670

util: fix unsafe url abort with bytestr() on url Differential Revision: https://phab.mercurial-scm.org/D2669

xdiff: fix builds on Windows This works on my ancient Fedora system too, without warnings. There are, however, warnings about various 64 to 32 bit conversions on Windows that need to be examined.

py3: use bytes() instead of str() Differential Revision: https://phab.mercurial-scm.org/D2665

py3: use b"%d" instead of str() to convert integers to bytes Differential Revision: https://phab.mercurial-scm.org/D2664

py3: use util.forcebytestr to convert testedwith value to bytes Bad extensions can put anything in testedwith so we should use util.forcebytestr here. Differential Revision: https://phab.mercurial-scm.org/D2663

py3: use pycompat.bytestr instead of str Differential Revision: https://phab.mercurial-scm.org/D2648

setdiscovery: avoid a Yoda condition Differential Revision: https://phab.mercurial-scm.org/D2646

setdiscovery: remove unnecessary sample size limiting Both _takequicksample() and _takefullsample() already limit their result to the request size, so there's no need to let the caller do that again. Differential Revision: https://phab.mercurial-scm.org/D2645

setdiscovery: remove initialsamplesize from a condition It seems more direct to compare the actual sample size. That way we can change the sample taken earlier in the code without breaking the condition. Differential Revision: https://phab.mercurial-scm.org/D2644

setdiscovery: back out changeset 5cfdf6137af8 (issue5809) As explained in the bug report, this commit caused a performance regression. The problem occurs when the local repo has very many heads. Before 5cfdf6137af8, we used to get the remote's list of heads and if these heads mostly overlapped with the local repo's heads, we would mark these common heads as common, which would greatly reduce the size of the set of undecided nodes. Note that a similar problem existed before 5cfdf6137af8: If the local repo had very many heads and the server just had a few (or many heads from a disjoint set), we would do the same kind of slow discovery as we would with 5cfdf6137af8 in the case where local and remote repos share a large set of common nodes. For now, we just back out 5cfdf6137af8. We should improve the discovery in the "local has many heads, remote has few heads" case, but let's do that after backing this out. Differential Revision: https://phab.mercurial-scm.org/D2643

hgweb: fix up trailing slash detection on Python 3 Fixes a couple of hgweb tests. Differential Revision: https://phab.mercurial-scm.org/D2661

hgweb: convert req.form to bytes for all keys and values This is just going to be a lot cleaner for our internals. Differential Revision: https://phab.mercurial-scm.org/D2660

templater: show repr of string we're rejecting I feel like this should make it a little easier to hunt down problems. Differential Revision: https://phab.mercurial-scm.org/D2659

webutil: some %d instead of %s love on ints Differential Revision: https://phab.mercurial-scm.org/D2658

py3: whitelist three more cases Differential Revision: https://phab.mercurial-scm.org/D2657

archival: our filenames are bytes, not strs Differential Revision: https://phab.mercurial-scm.org/D2656

archival: tar file modes need to be sysstrs Differential Revision: https://phab.mercurial-scm.org/D2655

archival: fsdecode paths before passing to tar or zip objects Both of these traffic in unicodes for filenames on Python 3, and inspection of the tarfile module shows that it uses the filesystem encoding, so fsdecode is the right choice. Differential Revision: https://phab.mercurial-scm.org/D2654

py3: add b'' prefixes in tests/test-minirst.py # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D2653

py3: make sure __repr__ returns a str # skip-blame because just r'' prefix Differential Revision: https://phab.mercurial-scm.org/D2652

py3: make sure regular expressions are bytes # skip-blame because just b'' prefix Differential Revision: https://phab.mercurial-scm.org/D2651

py3: use bytes instead of str to make sure we use bytes internally Differential Revision: https://phab.mercurial-scm.org/D2650

py3: use util.forcebytestr instead of str to convert error messages Differential Revision: https://phab.mercurial-scm.org/D2649

lock: block signal interrupt while making a lock file On Windows where symlink isn't supported, util.makelock() could leave an empty file if interrupted immediately after os.open(). This empty lock never dies as it has no process id recorded. ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL) # an interrupt may occur here os.write(ld, info) os.close(ld) This was a long-standing bug of TortoiseHg which runs a command-server and kills it by CTRL_C_EVENT, reported by random Windows users. https://bitbucket.org/tortoisehg/thg/issues/4873/#comment-43591129 At first, I tried to fix makelock() to clean up a stale lock file, which turned out to be hard because any instructions may be interrupted by a signal. ld = None try: # CALL_FUNCTION # os.open(...) # an interrupt may occur here # STORE_FAST # ld = ... ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL) os.write(ld, info) ... return True except: if ld: ... os.unlink(pathname) return False So I decided to block signals by temporarily replacing the signal handlers so makelcok() and held = 1 will never be interrupted. Many thanks to Fernando Najera for investigating the issue.

fuzz: add some more docs about building/running fuzzers Differential Revision: https://phab.mercurial-scm.org/D2635

util: also silence py3 warnings from codec module Fixes warnings like this: + mercurial/util.py:2446: DeprecationWarning: invalid escape sequence '\d' + return codecs.escape_decode(s)[0] Differential Revision: https://phab.mercurial-scm.org/D2642

test-annotate: set stdin and stdout to binary to get CR unmodified

test-annotate: rewrite sed with some python I hope this will fix the test failure seen on FreeBSD and Windows.

docs: small fixes for profiling.nested and the overall description - profiling.nested defaults to 0, not 5 - profiling is not always done with lsprof Differential Revision: https://phab.mercurial-scm.org/D2641

scmutil: fix oversight in b76248e51605c6 where I forgot to use msg Thanks to Yuya for spotting my mistake. Differential Revision: https://phab.mercurial-scm.org/D2636

hghave: remove unused "as ex" in exception block I overlooked this when removing a debug print in another change. Differential Revision: https://phab.mercurial-scm.org/D2637

tests: port test-log to Python 3 Required some porting to >>> inline Python instead of using heredocs into $PYTHON. Differential Revision: https://phab.mercurial-scm.org/D2621

py3: make gettext domain a system string

templater: fix position of terminator character in error message Since a template expression starts after the '{' character, the expression should be considered to end immediately before the terminator '{'.

merge with stable

test-subrepo: glob away an unstable hash This is the instability mentioned at the beginning of the series. I don't like hiding it, but I don't want to sit on a fix for a user reported problem while trying to figure this out. The instability seems related to the cset with a .hgsub with a remote URL. (There's very little existing remote URL subrepo testing.)

subrepo: activate clone pooling to enable sharing with remote URLs This is the easiest way to ensure that repositories with remote subrepo references can share the subrepos, consistent with how local subrepos can be shared.

subrepo: don't attempt to share remote sources (issue5793) Untangling _abssource() to resolve the new subrepo relative to the shared parent's share path, and then either sharing from there (if it exists), or cloning to that location and then sharing, is probably more than should be attempted on stable. Absolute subrepo references are discouraged, so for now, this resumes the behavior prior to 68e0bcb90357 of cloning the absolute subrepo locally.

test-subrepo: demonstrate problems with subrepo sharing and absolute paths This affects remote paths in .hgsub, as well as clone pooling from a remote source. For reasons unknown, there are stability issues with the relative-path.t tests. If run as a single test, it is stable. If run with --loop, or with -jX for X>1, the hash of the parent repo changes. I'm seeing this on both Windows and Fedora 26. I added an `hg log --debug`, and the manifest hash changes, but I have no idea why.

revsetlang: add a hint for more useful parse errors This logic is largely based on the similar logic added to template error messages in D2608 and D2609, but with a few tweaks based on how revsets actually work. Differential Revision: https://phab.mercurial-scm.org/D2619

setup: ignore extension load failures when finding working hg Previously, `make local` would fail if any extension was not properly loading. Differential Revision: https://phab.mercurial-scm.org/D2589

profile: colorize output on Windows Previously, the ANSI codes were printed to the screen, throwing off the alignment. We could probably do this unconditionally, but it's kind of a hack, so I figured I'd limit the scope.