copyright: update to 2019 Differential Revision: https://phab.mercurial-scm.org/D5779

patch: handle 0 context lines (diff.unified=0) when parsing patches Previously, if there were no context lines, we would just keep updating the ranges and the hunk, but not actually storing the hunk (just overwriting it each time). Thus a diff like this: $ hg diff --config diff.unified=0 diff --git a/bar b/bar --- a/bar +++ b/bar @@ -1,0 +2,1 @@ 1 +change1 @@ -3,0 +5,1 @@ 3 +change2 would come out of the parser like this (change1 is lost): bar: @@ -3,0 +5,1 @@ 3 +change2 This had some really weird side effects for things like commit --interactive, split, etc. Differential Revision: https://phab.mercurial-scm.org/D5743

py3: pass str into RuntimeError() to prevent b'' in output # skip-blame as just r'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5740

py3: use '%d' instead of '%s' for integers Differential Revision: https://phab.mercurial-scm.org/D5738

tests: port test-hgweb-auth.py to Python 3 Differential Revision: https://phab.mercurial-scm.org/D5736

url: convert some variables back to bytes Differential Revision: https://phab.mercurial-scm.org/D5735

url: add some defensive asserts on expected incoming types Our type handling is a nightmare here, and we're loading passwords to do network IO, so we can afford to be potentially-slow but pedantic here. Differential Revision: https://phab.mercurial-scm.org/D5734

changegroup: don't try to build changelog chunks if not required When we extend a narrow clone without ellipsis, we don't download changelog information because that's already present with the client. However we still try to build that chunk stream. Building that chunk stream involves calling a lookup function and store.emitrevisions() API. The lookup function is called len(cl) number of times. On large repositories, where len(cl) is in millions, calling that lookup function is not a good idea. Also it's not required to use the store.emitrevisons() API because we already have nodes present which we can use. This patch short-circuits state building logic if we are processing a non-ellipsis case and changelog is not required. This saves up ~20 seconds on our internal repo for a single extend call. Differential Revision: https://phab.mercurial-scm.org/D5733

revlog: make sure we never use sparserevlog without general delta (issue6056) We are getting user report where the delta code tries to use `sparse-revlog` logic on repository where `generaldelta` is disabled. This can't work so we ensure the two booleans have a consistent value. Creating this kind of repository is not expected to be possible the current bug report point at a clonebundle related bug that is still to be properly isolated (Yuya Nishihara seems to a have done it). Corrupting a repository to reproduce the issue is possible. A test using this method is included in this fix.

sparserevlog: document the config option This was overlooked when this graduated from experimental.

changegroup: initialize the state variable a bit earlier This will make the next patch much easier. Differential Revision: https://phab.mercurial-scm.org/D5732

tests: conditionalize test output on Python 3.7 Python 3.7 changed behavior of urllib.parse.quote() from RFC 2396 to RFC 3986 and ~ is now in the set of reserved characters and isn't escaped. We conditioanlize test output accordingly. Differential Revision: https://phab.mercurial-scm.org/D5717

hghave: add pyXY features for Python version numbers This will allow us to sniff for Python >= versions in tests. Differential Revision: https://phab.mercurial-scm.org/D5088

py3: whitelist couple more passing tests found by buildbot Differential Revision: https://phab.mercurial-scm.org/D5731

keepalive: implement _close_conn() so closes are known Keepalives were not working on Python 3 because http.client.HTTPResponse was refactored to call _close_conn() instead of close(). Our custom close() is what returns inactive connections to the available state. We better support Python 3 by implementing a _close_conn(). Differential Revision: https://phab.mercurial-scm.org/D5720

lfs: explicitly add the Content-Length header when uploading blobs, for py3 This was the reason for test-lfs-test-server.t#git-server complaining about an "invalid byte in chunk length". For some reason if this isn't explicitly added, py3.7.1 is adding `transfer-encoding: chunked` as well as `Content-length: x`. Wireshark flagged this as malformed. However, if this is set, it doesn't bother with `transfer-encoding`. Before this patch with py3: PUT /objects/31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b HTTP/1.1 Accept-Encoding: identity Content-length: 12 accept: application/vnd.git-lfs content-type: application/octet-stream host: localhost:20062 transfer-encoding: chunked user-agent: git-lfs/2.3.4 (Mercurial 4.9rc0+149-7eb7637e34bf) Before this patch with py27: PUT /objects/31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b HTTP/1.1 Accept-Encoding: identity accept: application/vnd.git-lfs content-type: application/octet-stream content-length: 12 host: localhost:20062 user-agent: git-lfs/2.3.4 (Mercurial 4.9rc0+149-7eb7637e34bf+20190128) With this patch and py3, the content is the same as the py27 example. RFC2616 says to ignore `Content-Length` if `Transfer-Encoding` is present, so maybe there's nothing to do in the hg-server side (though I'm not sure which it is using if presented both). Maybe chunked encoding is better to do? If someone knows how to suppress the `Content-Length`, we can try that instead.

perf: support looking up multiple revisions The nodemap code has optimisations around the number of lookup we actually made. As a result, being able to specify multiple revisions to look up is important when measuring performances. One can now specify full revspecs with the --rev arguments.

perf: add a no-lookup variant to perfindex It is useful to check how long it takes to create a index object without doing anything with it. We add a new flag dedicated to that.

perf: add some documentation to perfindex It seems useful to document how the arguments can affect the benchmark.

perf: move cache clearing in the `setup` step of `perfheads` The cache clearing is pretty fast, but this seems more "correct".

revlog: document cext oddities in terms of object/caches This clarify why we just call clearcaches on a single object.

perf: document perfheads

py3: stabilize the output of lfs commandserver tests The print() statements were being output at the very end, so write to the same stdout sink as runcommand, and explicitly flush.

py3: conditionalize some LFS test output I'm not sure why the one stackframe is py2 only, but that seems harmless. The remaining failure is LfsCorruptionError printing the fully qualified name, as well as b'' around its message.

lfs: strip the response headers from the Batch API before printing For reasons unknown, py3 is adding an extra '\n' before the headers print out. This makes the output the same as py2.

py3: force hgweb.server error log to internally write unicode Otherwise, there's a lot of py2/py3 divergence in the LFS tests because of the "HG error" lines picking up a b'' prefix. wsgicgi.py uses procutil.stderr, so I assume the input was meant to be bytes.

py3: byteify the decoded JSON responses upon receipt in the LFS blobstore It got too confusing juggling r'' vs b'' across several functions.

hgweb: ensure Content-Length and Content-Type are not promoted to HTTP_ on py3 In stabilizing test-lfs-serve-access.t for py3, the server started asserting on blob upload: Environment should not have the key: HTTP_CONTENT_LENGTH (use CONTENT_LENGTH instead) It could be avoided by explicitly setting the Content-Length header on the client side. I didn't go back to py2, but printing the original header here in py37 revealed 'Content-length' when sent to the error log.

py3: raw stringify various JSON and HTTP headers in the LFS blobstore module This is (almost?) entirely from Augie's work. I'm a bit surprised that the JSON data is being encoded with ASCII via `pycompat.bytesurl()`- I would have thought UTF-8.

py3: byteify the LFS blobstore module This is almost entirely b'' prefixing, with a couple of exceptions forced to bytes. Much of this is also borrowed from Augie's code. There's an HTTPError.read() that I flagged that I assume needs to be converted to bytes, but I can't find confirmation. Handling the deserialized JSON object over several functions made r'' vs b'' accesses confusing, so this assumes that the JSON object will be converted to bytes immediately. That will be done in the following commits, so it's not buried in these trivial changes.