convert: fix two %r output formats with pycompat.bytestr() wrapping Differential Revision: https://phab.mercurial-scm.org/D2529

convert: use our shlex wrapper in filemap to avoid Python 3 tracebacks Differential Revision: https://phab.mercurial-scm.org/D2527

convcmd: use our shlex wrapper to avoid Python 3 tracebacks Differential Revision: https://phab.mercurial-scm.org/D2526

convert: add some utility code for working with shlex on Python 3 This could have gone in pycompat, but it's only needed in convert, so I figured it made more sense here. It's got py3 in the name and checks pycompat.ispy3, so we'll find it whenever we decide to drop Python 2 support in 20x6. # no-check-commit because of required foo_bar naming on the proxy class Differential Revision: https://phab.mercurial-scm.org/D2525

pycompat: add support for encoding argument to our wrapper This only works on Python 3, but I'm about to need it for a regrettable hack in the convert code. Differential Revision: https://phab.mercurial-scm.org/D2524

py3: port tests/test-wireproto.py to Python 3 Differential Revision: https://phab.mercurial-scm.org/D2534

py3: use pycompat.bytestr() to convert None to bytes Differential Revision: https://phab.mercurial-scm.org/D2533

webcommands: use explicit integer division for Python 3 compat Differential Revision: https://phab.mercurial-scm.org/D2532

templatefilters: stop using str as a variable name str() is an inbuilt function. Differential Revision: https://phab.mercurial-scm.org/D2531

py3: fix string slicing in util.parsetimezone()

py3: replace type 'str' by 'bytes' in templater.py

py3: don't crash when re-raising encoding error

py3: mark all string literals in test-command-template.t as bytes # skip-blame because just b'' prefixes

py3: byte-stringify ValueError of unescapestr() to reraise as ParseError

py3: fix type of string literals in templater.tokenize() # skip-blame because just b'' prefixes

py3: drop b'' from error message generated by templater.runmember()

py3: fix join(), min(), and max() template functions over string It's silly to split a string into characters and concatenate them, but that should work and test-command-template.t has one. min() and max() had the same issue on Python 3, so fixed too.

py3: use startswith() to check existence of trailing '\n' in .hgtags file

py3: silence return value of file.write() in test-command-template.t

error: fix isinstnace check to use bytes instead of str Differential Revision: https://phab.mercurial-scm.org/D2528

py3: whitelist another nine passing tests I now see 347 tests passing[0] and 336 failing, so we're past halfway there. Hooray! 0: A few tests appear to regress in small ways (doctest output changes, for example) on Python 3.7.0a1, which is what I'm now using to test. That said, I'm pleased to report no major regressions. Differential Revision: https://phab.mercurial-scm.org/D2439

util: use pycompat.bytestr() on repr() in date parse abort Avoids the b'' prefix. Differential Revision: https://phab.mercurial-scm.org/D2438

py3: whitelist 14 new tests passing 12 new tests passed because of parent changeset and 2 were passing before that too. Differential Revision: https://phab.mercurial-scm.org/D2523

py3: slice over bytes to prevent getting ascii values This fixed reading of mergestate files and fixes 14 tests on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2522

wireprotoserver: move SSHV1 and SSHV2 constants to wireprototypes To avoid a cycle between modules in an upcoming commit. Differential Revision: https://phab.mercurial-scm.org/D2482

wireproto: use named arguments for commandentry We'll be adding more arguments in upcoming commits. Using named arguments will make the code easier to read. Differential Revision: https://phab.mercurial-scm.org/D2481

debugcommands: support for triggering push protocol The mechanism for pushing to a remote is a bit more complicated than other commands. On SSH, we wait for a positive reply from the server before we start sending the bundle payload. This commit adds a mechanism to the "command" action in `hg debugwireproto` to trigger the "push protocol" and to specify a file whose contents should be submitted as the command payload. With this new feature, we implement a handful of tests for the "unbundle" command. We try to cover various server failures and hook/output scenarios so protocol behavior is as comprehensively tested as possible. Even with so much test output, we only cover bundle1 with Python hooks. There's still a lot of test coverage that needs to be implemented. But this is certainly a good start. Because there are so many new tests, we split these tests into their own test file. In order to make output deterministic, we need to disable the doublepipe primitive. We add an option to `hg debugwireproto` to do that. Because something in the bowels of the peer does a read of stderr, we still capture read I/O from stderr. So there is test coverage of what the server emits. The tests around I/O capture some wonkiness. For example, interleaved ui.write() and ui.write_err() calls are emitted in order. However, (presumably due to buffering), print() to sys.stdout and sys.stderr aren't in order. We currently only test bundle1 because bundle2 is substantially harder to test because it is more complicated (the server responds with a stream containing a bundle2 instead of a frame). Differential Revision: https://phab.mercurial-scm.org/D2471

sshpeer: support not reading and forwarding stderr The "doublepipe" primitive as used by sshpeer will automatically read from stderr and forward output to the local ui. This poses problems for deterministic testing because reads may not be consistent. For example, the server may not be done sending all output to stderr and the client will perform different numbers of read operations or will read from stderr and stdout at different times. To make tests deterministic, we'll need to disable the "doublepipe" primitive and perform stderr I/O explicitly. We add an argument to the sshpeer constructor to disable the use of the doublepipe. Differential Revision: https://phab.mercurial-scm.org/D2467

tests: add wire protocol tests for pushkey Let's get the wire format of some pushkey requests in test-ssh-proto.t. Differential Revision: https://phab.mercurial-scm.org/D2466

debugcommands: support for sending "batch" requests Let's teach `hg debugwireproto` to send "batch" requests. The easiest way to implement this was as a pair of instructions to begin and end a batched operation. Otherwise, we would have to reinvent the parsing wheel or factor out the parsing code. To prove it works, we add a batched request to test-ssh-proto.t. Differential Revision: https://phab.mercurial-scm.org/D2408