wireproto: move command registration types to wireprototypes These are shared across wire protocol implementations. wireprototypes is our module for common code. Differential Revision: https://phab.mercurial-scm.org/D3396

wireproto: make version 2 @wireprotocommand an independent function Previously, the code for this decorator was shared between version 1 and version 2 commands. Very few parts of the function were identical. So I don't think sharing is justified. wireprotov2server now has its own @wireprotocommand decorator function. Because the decorator is no longer shared, code for configuring the transport policy has been removed. i.e. commands must have separate implementations for each wire protocol version. Differential Revision: https://phab.mercurial-scm.org/D3395

wireproto: don't pass transportpolicy argument The default is version 1 only. So we don't need to pass this argument when declaring commands. Differential Revision: https://phab.mercurial-scm.org/D3394

forget: rename --confirm to --interactive Differential Revision: https://phab.mercurial-scm.org/D3405

notify: add maxdiffstat option to truncate long file lists Large scale changes like a new GCC version can easily result in 1MB+ emails due to diffstat alone. The new maxdiffstat option truncates the list similar to what maxdiff already provides for the diffs. Differential Revision: https://phab.mercurial-scm.org/D3402

log: fix crash on empty revision with --copies switch If a revset is empty, .max() raises ValueError. I don't see any reason to recompute the revs, so I made it reuse the one returned by logcmdutil.getrevs(). If no revs specified by command line, the endrev will be smartset.spanset(repo) + 1, which is basically the same as len(repo), the default of getrenamedfn(). If --follow specified, revs.max() points to the working parent, which seems more correct.

revsetlang: do not pass in non-bytes to parse() Since parse() isn't a simple function, we shouldn't expect it would raise TypeError or ValueError for invalid inputs. Before, TypeError was raised at 'if pos != len(spec)', which was quite late to report an error. This patch also makes tokenize() detect invalid object before converting it to a py3-safe bytes. Spotted while adding the 'revset(...)' hack to _parsewith().

fix: use templater to substitute values in command string bytes.format() isn't supported on Python 3. Luckily, our template syntax is similar so we can reuse it. We need a hack to disable \-escapes as '\' is a directory separator on Windows.

logcmdutil: pass formatter to jsonchangeset as argument And rename the class. Now we can reuse it for 'log -Tcbor' (and '-Tpickle' if we want.)

logcmdutil: rewrite jsonchangeset printer to be backed by jsonformatter This is a bit slower than the original implementation, but I don't think that would actually matter. It's still faster than full templating. $ hg log -Tjson -r0:5000 --time > /dev/null (orig) time: real 1.550 secs (user 1.500+0.000 sys 0.040+0.000) (new) time: real 1.810 secs (user 1.740+0.000 sys 0.070+0.000) cf. $ hg log -Tdefault -r0:5000 --time > /dev/null time: real 4.980 secs (user 4.850+0.000 sys 0.130+0.000) $ hg log -r0:5000 --time > /dev/null time: real 2.340 secs (user 2.220+0.000 sys 0.100+0.000) $ hg log -r0:5000 -q --time > /dev/null time: real 0.750 secs (user 0.670+0.000 sys 0.070+0.000) The test output changes because keys are sorted alphabetically.

logcmdutil: remove unused attribute 'cache' from jsonchangeset printer Perhaps it's a copy-pasta of changeset_templater at bd15932846a4.

formatter: convert timestamp to int Spotted while porting jsonchangeset printer to formatter. A timestamp may be float type, but its meaningful part is just int.

bundle: introduce per-engine compression level If experimental.bundlecomplevel.$engine is set, prefer it over the generic experimental.bundlecomplevel. Given that compression levels have widely different meanings across engines, this allows much saner configuration. Differential Revision: https://phab.mercurial-scm.org/D3393

bundlespec: drop externalnames flag Always provide the human readable version of compression and version. Add the translated wire format name in the new wirecompression and wireversion fields. Differential Revision: https://phab.mercurial-scm.org/D3392

revlog: make shortest() take a full binary nodeid (API) Follow-up to 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14). Differential Revision: https://phab.mercurial-scm.org/D3403

lfs: add the 'Authorization' property to the Batch API response, if present The client copies all of these properties under 'header' to the HTTP Headers of the subsequent GET or PUT request that it performs. That allows the Basic HTTP authentication used to authorize the Batch API request to also authorize the upload/download action. There's likely further work to do here. There's an 'authenticated' boolean key in the Batch API response that can be set, and there is an 'LFS-Authenticate' header that is used instead of 'WWW-Authenticate'[1]. (We likely need to support both, since some hosting solutions are likely to only respond with the latter.) In any event, this works with SCM Manager, so there is real world benefit. I'm limiting the headers returned to 'Basic', because that's all the lfs spec calls out. In practice, I've seen gitbucket emit custom header content[2]. [1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md#response-errors [2] https://github.com/gitbucket/gitbucket/blob/35655f33c7713f08515ed640ece0948acd6d6168/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala#L119

lfs: enable the final download count status message At this point, I think all of the core commands are prefetching, except grep and verify. Verify will need some special handling, in case the revlogs are corrupt. Grep has an issue that still needs to be debugged, but we probably need to give the behavior some thought too- it would be a shame to have to download everything in order to search. I think the benefit of having this info for all commands outweighs extra printing in a command that is arguably not well behaved in this context anyway.

diff: invoke the file prefetch hook By invoking it this deep within the command, we pick up both subrepo and hgweb support, as well as --patch support for commands that implement logopts.

export: invoke the file prefetch hook cmdutil.exportfile() is only called by shelve, mq and patchbomb. Those are unlikely to mix with lfs, but it may as well be invoked there for completeness.

scmutil: teach the file prefetch hook to handle multiple commits The remainder of the commands that need prefetch deal with multiple revisions. I initially coded this as a separate hook, but then it needed a list of files to handle `diff` and `grep`, so it didn't seem worth keeping them separate. Not every matcher will emit bad file messages (some are built from a list of files that are known to exist). But it seems better to filter this in one place than to push this on either each caller or each hook implementation.

tests: arrange for a server in wireproto-command-capabilities.t to be killed The stray servers were piling up after the test harness exited. On Windows, this means the *.pyd files can't be rebuilt, which is why the build warning count dropped to 1 recently.

revset: skip legacy lookup for revspec wrapped in 'revset(...)' Currently, multiple labels can take forms that can be confused with revset (eg: "rev(0)" is a valid tag). Since we look up for tags before evaluating revset, this means a tag can shadow a valid revset at any time. We now enforce the strict revset parsing when wrapped with 'revset(...)'. For now, This only work on a whole revspec (but can be used within the revset without effect). This might change in the future if we improve the implementation. The feature is undocumented for now, keeping it in the experimental namespace. In case a better approach to achieve the same goal is found. The syntax looks like a revset but is not implemented as such for now. Since the goal is to avoid some preprocessing that happens before revset parsing, we cannot simply implement it as a revset predicate. There was other approaches discussed over the mailing-list but they were less convincing. Having a configuration flag to disable legacy lookup have been considered but discarded. There are too many common uses of ambiguous identifier (eg: '+', '-' or '..') to have the legacy lookup mechanism turned off. In addition, the approach can control the parsing of each revset, making it more flexible. For example, a revset used as the value of an existing configuration option (eg: pushrev) could enforce its resolution as a revset (by using the prefix) while user inputs would still use the legacy lookup. In addition of offering a way to unambiguously input a revset, this prefix allow skipping the name lookup providing a significant speedup in some case.

revset: add more test to show current behaviors with label looking like revset There are very few constraints on what character can be put into tags and other labels. We add more tests showing some of extreme cases that user can currently use.

commands: use command executor interface Differential Revision: https://phab.mercurial-scm.org/D3391

exchange: use command executor interface for calling listkeys So the requests are compatible with version 2 peers. Differential Revision: https://phab.mercurial-scm.org/D3390

forget: add --confirm option Also added confirmopts in cmdutil.py Differential Revision: https://phab.mercurial-scm.org/D2934

largefiles: opts appears to already be bytes in this instance test-largefiles.t now passes. Differential Revision: https://phab.mercurial-scm.org/D3364

tests: port inline extensions in test-hook.t to py3 This test *almost* passes now, but some import errors print very differently in ways that seem at least somewhat important. Differential Revision: https://phab.mercurial-scm.org/D3363

tests: manually print list in test-hook.t Changes the output a bit, but not in an important way. Differential Revision: https://phab.mercurial-scm.org/D3362

hook: also use pprint on lists for stable output on py2/3 Differential Revision: https://phab.mercurial-scm.org/D3361

hook: use stringutil.pprint instead of reinventing it Differential Revision: https://phab.mercurial-scm.org/D3360

stringutil: make b prefixes on string output optional I need this to preserve some behavior in hook.py. Differential Revision: https://phab.mercurial-scm.org/D3359

stringutil: teach pprint how to format None Differential Revision: https://phab.mercurial-scm.org/D3358

tests: update inline extensions in test-bundle2-exchange.t to py3 The test doesn't pass for what superficially look like good reasons. We'll need to come back to it later. # skip-blame because it's b prefixes and a couple of b'N' instead of str(N) Differential Revision: https://phab.mercurial-scm.org/D3357

wsgicgi: un-do some prior porting work that is now wrong The Python 3 WSGI behavior is that the environ dict should be full of unicodes. We previously tried Too Hard here, so we unwind that bit of porting. Also add some bytesurl() encodes on status and headers. test-clone-cgi.t now passes. Differential Revision: https://phab.mercurial-scm.org/D3356

hgweb_mod: inform hgweb class about paths actually being bytes Differential Revision: https://phab.mercurial-scm.org/D3355

hgweb: inform hgweb.hgweb() entrypoint that paths should be bytes Differential Revision: https://phab.mercurial-scm.org/D3354

tests: port inline cgi script in test-largefiles.t to python 3 Differential Revision: https://phab.mercurial-scm.org/D3353

lfcommands: use %d on known-int in format string Differential Revision: https://phab.mercurial-scm.org/D3352

py3: fix test-shelve.t on Python 3 We get a better error message out of the abort here, which is fine. Differential Revision: https://phab.mercurial-scm.org/D3351

tests: fix up a couple of minor bytes inconsistencies in run-tests.py Only in the automatic bisection code, so fortunately nothing major is amiss. Fixes test-run-tests.t under Python 3. Differential Revision: https://phab.mercurial-scm.org/D3350

tests: make sure test-run-tests.t actually runs run-tests.py under Python 3 I'm fairly certain it hasn't been until now. Mercifully, there doesn't appear to be any ninja breakage. Differential Revision: https://phab.mercurial-scm.org/D3349

py3: another three passing Differential Revision: https://phab.mercurial-scm.org/D3348

httppeer: work around API differences on urllib Request objects Since this is only a problem in httppeer, I'd rather keep this a local-to-the-module kludge rather than pile more on pycompat. We'll still find it easily to clean up later because it checks pycompat.ispy3. Differential Revision: https://phab.mercurial-scm.org/D3347

httppeer: no matter what Python 3 might think, http headers are bytes Differential Revision: https://phab.mercurial-scm.org/D3346

httppeer: fix debug prints to work on Python 3 Differential Revision: https://phab.mercurial-scm.org/D3345

url: some bytes/str cleanup where we interface with stdlib funcs Differential Revision: https://phab.mercurial-scm.org/D3344

hgweb: these strings should be sysstrs, not bytes Differential Revision: https://phab.mercurial-scm.org/D3343