wireproto: port heads command to wire protocol v2 After much thought and consideration, wire protocol version 2's commands will be defined in different functions from the existing commands. This will make it easier to implement these commands because it won't require shoehorning things like response formatting and argument declaration into the same APIs. For example, wire protocol version 1 requires that commands declare a fixed and ordered list of argument names. It isn't really possible to insert new arguments or have optional arguments without breaking backwards compatibility. Wire protocol version 2, however, uses CBOR maps for passing arguments. So arguments a) can be optional b) can be added without BC c) can be strongly typed. This commit starts our trek towards reimplementing the wire protocol for version 2 with the heads command. It is pretty similar to the existing heads command. One added feature is it can be told to operate on only public phase changesets. This is useful for making discovery faster when a repo has tens of thousands of draft phase heads (such as Mozilla's "try" repository). The HTTPv2 server-side protocol has had its `getargs()` implementation updated to reflect that arguments are a map and not a list. Differential Revision: https://phab.mercurial-scm.org/D3179

largefiles: wrap heads command handler more directly extensions.wrapfunction() is a more robust method for wrapping a function, since it allows multiple wrappers. While we're here, wrap the function registered with the command instead of installing a new command handler. Differential Revision: https://phab.mercurial-scm.org/D3178

wireproto: crude support for version 2 HTTP peer As part of implementing the server-side bits of the wire protocol command handlers for version 2, we want a way to easily test those commands. Currently, we use the "httprequest" action of `hg debugwireproto`. But this requires explicitly specifying the HTTP request headers, low-level frame details, and the data structure to encode with CBOR. That's a lot of boilerplate and a lot of it can change as the wire protocol evolves. `hg debugwireproto` has a mechanism to issue commands via the peer interface. That is *much* easier to use and we prefer to test with that going forward. This commit implements enough parts of the peer API to send basic requests via the HTTP version 2 transport. The peer code is super hacky. Again, the goal is to facilitate server testing, not robustly implement a client. The client code will receive love at a later time. Differential Revision: https://phab.mercurial-scm.org/D3177

tests: extract wire protocol shell helpers to standalone file This will make it easier for other tests to get up and running without the boilerplate. Differential Revision: https://phab.mercurial-scm.org/D3176

addbranchrevs: no longer accept revset as "revs" (API) The only caller was removed in 0b4692b9646d (bundle: avoid repo.lookup() for converting revnum to nodeid, 2018-04-02). Differential Revision: https://phab.mercurial-scm.org/D3192

clone: avoid using repo.lookup() with binary nodeid The code in hg.clone() is a bit of a mess, but it seems like the "checkout" variable is always a binary nodeid (tests pass when run with "assert len(checkout) == 20" before the repo.lookup()). repo.lookup() will soon work only with string inputs, so we need remove this use. Differential Revision: https://phab.mercurial-scm.org/D3191

merge: avoid unnecessary conversion from binary nodeid to binary nodeid "node" is already a binary nodeid here, so there's no need to convert it. repo.lookup() will soon lose support for binary nodeids as input. Differential Revision: https://phab.mercurial-scm.org/D3190

tests: stabilize test-push-http.t for Windows See 594dd384803c.

tests: add a substitution pattern for HTTP error log dates This will make tests with error log content less of a nuisance. I didn't roll it into to $LOGDATE$ because the regex got long and line wrapped. It didn't seem worth the obscurity.

wireproto: convert python literal to object without using unsafe eval() Follows up cc5a040fe150. At this point, I don't think we need a real eval(). If we want to support a set literal, maybe we can vendor ast.literal_eval(), which is relatively simple function.

tests: quote variable passed to shell test command

py3: system-stringify repr(frame) That's the Py3 requirement.

wireproto: show unknown id and flags in repr(frame) Perhaps we'll want it for debugging.

wireproto: fix repr(frame) to not crash by unknown type id Follows up 5ef2da00e935.

py3: use s.startswith() instead of s[n] while parsing patches I know 'bytes[n] in bytes' magically works, but I'm tired of finding which one breaks the tests.

py3: do not try to byte-stringify None in cmdutil.tryimportone() It's a debug message, so just use '' instead.

py3: work around weird handling of bytes/unicode in decode_header() Basically decode_header() works as follows, and on Python 3, email headers ARE UNICODE. def decode_header(header): if not ecre.search(header): # ecre is unicode regexp return [(header, None)] # so header is unicode string ... decode header into [(bytes_data, unicode_charset_name)] return collapsed

py3: use system string to access email headers

py3: fix string issues of email message in test-import.t - payload can be bytes - headers must be unicode on Python 3 - need to call msg.as_bytes() on Python 3, but msg.as_string() on Python 2, where bytes(msg) magic works

py3: use lower-cased module 'email.message' in test-import.t

py3: drop b'' from error message of fancyopts

procutil: drop unused 'newlines' option from popen*() (API) It's unlikely for us to use the universal_newlines option.

procutil: make explainexit() simply return a message (API) Almost all callers want it.

procutil: do not convert return code of signal exit to positive number (API) The docstring states that "codes from kill are negative", and it doesn't make sense to make exit/signal code ambiguous. .. api:: ``hook.hook()`` and ``hook.runhooks()`` may return a negative integer to denote that the process was killed by signal.

procutil: fix error message of tempfile filter First, we need to use procutil.system() to get an exit code compatible with explainexit(). Second, explainexit() returns (msg, code) tuple.

procutil: unify platform.explainexit() Since 4368f582c806 "use subprocess instead of os.system", posix.explainexit() is the superset of Windows implementation.

procutil: rewrite popen() as a subprocess.Popen wrapper (issue4746) (API) os.popen() of Python 3 is not the popen() we want. First, it doesn't accept command in bytes. Second, a returned stream is always wrapped by TextIO. So we have to reimplement our popen(). Fortunately, this fixes the bug 4746 since ours returns an exit code compatible with explainexit(). .. api:: ``procutil.popen()`` no longer supports text mode I/O.

procutil: always popen() in binary mode On Python 3, non-binary stream is useless. Let's convert line ending by ourselves. Note that we don't need fromnativeeol() in patch._externalpatch() since any whitespace characters are rstrip()-ed.

py3: byte-stringify test-import.t Still the test doesn't pass. # skip-blame because just adding some b''

py3: convert parsed message items to bytes in patch.extract() Appears that BytesParser() parses bytes into unicode, sigh.

py3: silence warning about deprecation of imp module Well, we could fix that, but we aren't yet to reach the state caring about deprecation on the Python 3 line. So let's silence it for now to fix tons of "minor" Py2/3 incompatibilities by relying on our test suite.

tests: add another full hex node in plain text Should have been part of D3168. Differential Revision: https://phab.mercurial-scm.org/D3186