convert: fix a file descriptor leak test-check-code flagged this after I changed this line for something unrelated.

merge with stable

revlog: catch more specific exception in shortest() Since revlog._partialmatch() catches RevlogError coming from cext and re-raises AmbiguousPrefixLookupError, catching RevlogError here seems less correct. Differential Revision: https://phab.mercurial-scm.org/D4735

py3: update missing module list in test-check-py3-compat.t for Windows

py3: add b'' prefixes to wire protocol test Otherwise the CBOR encoder fails on Python 3 due to lacking support for encoding str/unicode. # skip-blame just some b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D4734

py3: use pycompat.strkwargs() Otherwise we get an error attempting to dispatch a command with arguments because we're passing a dict with bytes keys. Differential Revision: https://phab.mercurial-scm.org/D4732

py3: ensure _start_response() is called with system string This was preventing HTTP 500's from being sent in Python 3. Differential Revision: https://phab.mercurial-scm.org/D4730

py3: convert arguments, cwd and env to native strings when spawning subprocess This keeps Windows happy.

py3: apply byteskwargs to contrib/perf

py3: un-byteify strings around os.system() and os.devnull in contrib/perf

py3: work around the lack of sys.maxint in contrib/perf

py3: switch contrib/perf from xrange to pycompat.xrange

py3: handle sysstr conversion around get/set attr in contrib/perf

py3: proxy posixfile objects to re-add a useful 'name' attribute on Windows This file object is used in the vfs layer, so there are many errors like this: ... File "mercurial\localrepo.py", line 2569, in savecommitmessage return self.pathto(fp.name[len(self.root) + 1:]) TypeError: 'int' object is not subscriptable It looks like the 'name' value is actually the fileno() value, and the documentation says the name parameter to PyFile_FromFd() is ignored. [1] I tried just assigning the attribute after osutil.posixfile() returns, but that crashes saying that it's read-only. [1] https://docs.python.org/3.6/c-api/file.html

py3: don't use os.getcwdb() on Windows to avoid DeprecationWarnings See also ac32685011a3.

py3: rename pycompat.getcwd() to encoding.getcwd() (API) We need to avoid os.getcwdb() on Windows to avoid DeprecationWarnings, and we need encoding.strtolocal() to encode the result of os.getcwd().

py3: whitelist two more passing tests Caught by the ratchet, but initially only the non-legacy path of test-clone-uncompressed.t was passing. That required the preceding patch. Differential Revision: https://phab.mercurial-scm.org/D4729

keepalive: be more careful about self._rbuf when calling super impls In Python 3, HTTPResponse implements read() in terms of readinto(), which was calling back into our readinto(), which duplicates self._rbuf if it's not empty. Before calling into super's read(), ensure self._rbuf is empty. Inheritance is bad, and undocumented self-use of your public API is one of many reasons. Differential Revision: https://phab.mercurial-scm.org/D4728

wireprotov2: teach changesetdata to fetch ancestors until depth For shallow clone, it is useful to specify a starting node and tell the server to send up to N ancestors from that starting point. This enables the server to perform the DAG walk without the client having to discover the base/stop node(s) first. This commit implements support for said queries on the changesetdata command. Differential Revision: https://phab.mercurial-scm.org/D4621

wireprotov2: allow multiple fields to follow revision maps The *data wire protocol commands emit a series of CBOR values. Because revision/delta data may be large, their data is emitted outside the map as a top-level bytestring value. Before this commit, we'd emit a single optional bytestring value after the revision descriptor map. This got the job done. But it was limiting in that we could only send a single field. And, it required the consumer to know that the presence of a key in the map implied the existence of a following bytestring value. This commit changes the encoding strategy so top-level bytestring values in the stream are explicitly denoted in a "fieldsfollowing" key. This key contains an array defining what fields that follow and the expected size of each field. By defining things this way, we can easily send N bytestring values without any ambiguity about their order. In addition, clients only need to know how to parse ``fieldsfollowing`` to know if extra values are present. Because this breaks backwards compatibility, we've bumped the version number of the wire protocol version 2 API endpoint. Differential Revision: https://phab.mercurial-scm.org/D4620

wireprotov2: advertise set of valid values for requestable fields changesetdata, manifestdata, and filedata all allow the caller to specify what data fields to request. Data fields are extensible and may evolve over time. In order to prevent clients from making requests for fields that are not available, the client needs to know what fields are available. This commit teaches the server to declare a set of "valid values" for wire protocol command arguments. That set of values is exposed in the command's capabilities descriptor. The changesetdata, manifestdata, and filedata commands all declare their set of available "fields." Differential Revision: https://phab.mercurial-scm.org/D4619

wireprotov2: expose rich arguments metadata Now that we internally store rich metadata about arguments, it makes sense to make that metadata available to the client. This will allow clients to validate outgoing command requests before they are sent over the wire. Strictly speaking, we should bump the wire protocol version for this change since it is backwards incompatible. But no client-side code touches the arguments map and I don't want to incur the work. Differential Revision: https://phab.mercurial-scm.org/D4618

wireprotov2: advertise recognized path filter prefixes While the wire protocol doesn't yet support it, we'll eventually have commands that accept narrow patterns to specify the set of files relevant to a command. For security and performance reasons, only specific filter types are allowed. This commit teaches the server to advertise the set of allowed filter types. By doing so, clients can e.g. validate user-specified patterns against the server's abilities without having to send a command to retrieve data. Having the data in the capabilities data structure will also serve as a check against unwanted BC. Differential Revision: https://phab.mercurial-scm.org/D4616

wireprotov2: declare command arguments richly Previously, we declared command arguments with an example of their value. After this commit, we declare command arguments as a dict of metadata. This allows us to define the value type, whether the argument is required, and provide default values. This in turn allows us to have nice things, such as less boilerplate code in individual commands for validating input and assigning default values. It should also make command behavior more consistent as a result. Test output changed slightly because I realized that the "fields" argument wasn't being consistently defined as a set. Oops! Other test output changed because of slight differences in code performing type validation. Differential Revision: https://phab.mercurial-scm.org/D4615

annotate: pass in wdir rev and node to formatter (BC) This is a part of the unification series. The 'ff..' hash is preferred over None as it is a valid revision specifier. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Sanity_check_output

annotate: pass around full hex node until formatting plain output In short, this patch moves h[:12] from hexfn() to formathex() so that formathex() can test if h is the wdirhex or not. This helps switching the wdir value to wdirrev/wdirhex. See the next patch.

log: fill in pseudo rev and node as wdir() manifest identifiers While we'll never support such identifiers to look up the manifest, this behavior seems more consistent.

log: have changesetformatter fill in wdir() rev and node (BC) This matches the behavior of the default template keywords. wdir() support is still experimental so we can change the output.

hgweb: use scmutil.binnode() to translate None to wdir hash (issue5988) I left some of ctx.node() calls unchanged as they seemed unlikely to be workingctx, or passed to diff functions where None is the default value. Note that a None revision can also cause a similar problem, but I'm not sure if we can simply bulk-replace ctx.rev() with scmutil.intrev(ctx) as there's large hole between tip revision and wdir revision. If such pair were passed in to xrange() for example, we would waste CPU time.

hgweb: register web.static to the config table Otherwise we would got a develwarn.