test-lfs: add tests to force server error path coverage The tests are somewhat fragile in that the extension that forces the errors is counting how many times some of the functions are being called, so it depends heavily on the content of the repo. Maybe we can do something clever like load an extension on the client, and have it send over instructions in the HTTP header how to fail. (I'm trying to avoid killing and restarting the server, because Windows seems to have issues with doing that a lot.) But I'd rather fix issues than polish tests before the freeze.

keepalive: add ** overlooked in 83250442dc81 Caught by Yuya in D3326. Differential Revision: https://phab.mercurial-scm.org/D3372

test-check-commit: don't run hg per commit We aren't stress testing CPU. $ time ./run-tests.py -l test-check-commit.t --timeout 600 (orig) 162.59s user 17.98s system 101% cpu 2:58.55 total (new) 5.85s user 0.99s system 98% cpu 6.939 total

templates: make divergentnodes in whyunstable keyword be a hybrid list

templates: add whyunstable template keyword

commands: drop spurious r'' on dry_run in forget Fixes test-add.t on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3365

tests: port inline extension in test-http-bundle1.t to py3 # skip-blame just b prefixes Differential Revision: https://phab.mercurial-scm.org/D3366

tests: add all missing b prefixes in reactor tests Both of these tests now pass on Python 3. # skip-blame just b prefixes. So many b prefixes. Differential Revision: https://phab.mercurial-scm.org/D3369

stringutil: ast.literal_eval needs a unicode on py3 Fortunately for us, this is really only used internally, so we can be lazy about the encoding here. test-wireproto-framing.py now passes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D3368

scmutil: introduce shortesthexnodeidprefix() We have scmutil.resolvehexnodeidprefix() for resolving a prefix to a full nodeid, so it makes sense to have the inverse method next to it. For now it just delegates to changelog.shortest(), but it will soon also make sure it's called on the unfiltered repo, to match resolvehexnodeidprefix(). Note that the change in show.py also makes it so the conversion from revnum to nodeid is done on the filtered repo, but that should be inconsequential since the revs are all from the filtered repo anyway. Differential Revision: https://phab.mercurial-scm.org/D3370

scmutil: use resolvehexnodeidprefix() from revsymbol() I should have copied this from changectx.__init__ into in 35b34202dd3b (context: handle partial nodeids in revsymbol(), 2018-04-08). Differential Revision: https://phab.mercurial-scm.org/D3310

scmutil: rename resolvepartialhexnodeid() to resolvehexnodeidprefix() I'm going to wrap revlog.shortest() in scmutil. I plan on calling it shortesthexnodeidprefix(). resolvehexnodeidprefix() matches that better. Also, "prefix" carries more information than "partial". Differential Revision: https://phab.mercurial-scm.org/D3309

scmutil: document that isrevsymbol() raises on ambiguous node prefix Differential Revision: https://phab.mercurial-scm.org/D3308

context: set stack level for deprecation warning This patch makes the deprecation warning print the caller of repo.__getitem__. That's not quite correct, since there could also be other callers of changectx.__init__, but it seems like the most helpful stack level in practice. Differential Revision: https://phab.mercurial-scm.org/D3307

util: set correct stack level on deprecation warnings Without this patch, you'll get something like this: <path>/mercurial/util.py:3784: DeprecationWarning: 'util.hgexecutable' is deprecated, use 'utils.procutil.hgexecutable' (but on one line) Differential Revision: https://phab.mercurial-scm.org/D3331

revset: pass in lookup function instead of repo (API) And document that it's only for legacy lookup. If we have a repo, we're likely to do more things where that shouldn't be done.

revset: drop support for posttreebuilthook() (API) AFAIK, the only user was the directaccess extension, which is in core now.

py3: make values bytes before passing into server.runservice() The values of opts dict still needed to be converted to bytes. Differential Revision: https://phab.mercurial-scm.org/D3330

py3: two more passing tests Differential Revision: https://phab.mercurial-scm.org/D3338

localrepo: add some overlooked strkwargs love for py3 Differential Revision: https://phab.mercurial-scm.org/D3337

py3: paper over differences in future exception handling It looks like Python 3's futures library lacks set_exception_info entirely. We'll just give up and use set_exception in that case. # no-check-commit because the underbar naming is just saner here Differential Revision: https://phab.mercurial-scm.org/D3336

py3: add b'' prefixes to make values bytes # skip-blame beacuse just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3329

py3: use b"%d" instead of str() to convert int to bytes While I was here, I added 'and None' to suppress return values of .write() calls. Differential Revision: https://phab.mercurial-scm.org/D3328

py3: add b'' prefixes to make values bytes # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3327

py3: use str variables to check keys in request header The values in header are of str type. Differential Revision: https://phab.mercurial-scm.org/D3326

py3: make sure curses.tigetstr() first argument is a str Differential Revision: https://phab.mercurial-scm.org/D3325

py3: use stringutil.forcebytestr() instead of str() We need to convert errors to bytes using stringutil.forcebytestr() Differential Revision: https://phab.mercurial-scm.org/D3324

py3: iterate over a copy of dict while changing it Differential Revision: https://phab.mercurial-scm.org/D3323

py3: use urllib.parse.unquote_plus instead of urllib.unquote_plus The later is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D3322

py3: add b'' prefixes to tests/test-status-inprocess.py # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3321

lock: don't use 'file' as a variable name Differential Revision: https://phab.mercurial-scm.org/D3320

py3: use b"%d" instead of str() to convert integers to bytes Differential Revision: https://phab.mercurial-scm.org/D3319

wireproto: expose repository formats via capabilities Servers need to expose their set of repository storage requirements in order to facilitate streaming clones (clients need to know if they are capable of reading the raw storage files that the server exposes). Differential Revision: https://phab.mercurial-scm.org/D3335

wireprotoframing: record when new stream is encountered Without this, we choke after receiving the 2nd frame in a stream. Not sure how we made it this far without finding this bug. Differential Revision: https://phab.mercurial-scm.org/D3334

wireprotoframing: use value passed into function Oops. Differential Revision: https://phab.mercurial-scm.org/D3333

httppeer: handle error response from client reactor With this in place, we're now seeing useful errors when running tests with the new wire protocol enabled! Differential Revision: https://phab.mercurial-scm.org/D3332

wireproto: add media type to version 2 capabilities response This is useful to advertise because servers reject unsupported media types. A client may wish to speak multiple media types and choose the one the server supports. I doubt we'll ever use multiple media types or negotiation in core. But during the course of developing this protocol, I may end up making extensions that backport and forward port protocol support as needed to support Mercurial deploys in the wild. e.g. I may deploy support for an older protocol on a server so old clients can continue using it. It's worth pursuing changing the SSH protocol's upgrade mechanism to support multiple media types as well... Differential Revision: https://phab.mercurial-scm.org/D3299

debugcommands: use command executor for invoking commands Now that all peers support the command executor interface, we can use it in place of peer._call() (which isn't part of the public API). Tests changed because we are now returning the decoded response instead of the raw bytes. The raw bytes over the wire are already logged. So we're not losing any test coverage. In fact, we're adding test coverage because we're testing decoding of those command responses as well. Differential Revision: https://phab.mercurial-scm.org/D3298

httppeer: implement command executor for version 2 peer Now that we have a new API for issuing commands which is compatible with wire protocol version 2, we can start using it with wire protocol version 2. This commit replaces our hacky implementation of _call() with something a bit more robust based on the new command executor interface. We now have proper support for issuing multiple commands per HTTP request. Each HTTP request maintains its own client reactor. The implementation is similar to the one in the legacy wire protocol. We use a ThreadPoolExecutor for spinning up a thread to read the HTTP response in the background. This allows responses to resolve in any order. While not implemented on the server yet, a client could use concurrent.futures.as_completed() with a collection of futures and handle responses as they arrive from the server. The return value from issued commands is still a simple list of raw or decoded CBOR data. This is still super hacky. We will want a rich data type for representing command responses. But at least this commit gets us one step closer to a proper peer implementation. Differential Revision: https://phab.mercurial-scm.org/D3297

repository: remove ipeercommands from ipeerbase AFAICT all callers in core have moved to the commandexecutor interface for invoking wire protocol commands. Or at least they aren't using the named methods on ipeercommands to invoke them. This means we can drop ipeercommands from the ipeerbase interface. As far as interface based programming goes, it is now illegal to call an ipeercommands method for issuing wire protocol commands. However, the methods are still there, so they will still work. At some point we will want to break that API... Differential Revision: https://phab.mercurial-scm.org/D3318

wireproto: properly call clonebundles command We should not be using _call() to make wire protocol calls because it isn't part of the peer API. But clonebundles wasn't part of the supported commands in the peer API! So this commit defines that command in the commands interface, implements it, and teaches the one caller in core to call it using the command executor interface. Differential Revision: https://phab.mercurial-scm.org/D3317

exchange: use command executor for getbundle The code consuming the bundle has been moved to inside the context manager, as that is supposed to be part of the API. (Although it doesn't matter for version 1 peers.) Differential Revision: https://phab.mercurial-scm.org/D3316

exchange: use command executor for pushkey Differential Revision: https://phab.mercurial-scm.org/D3315

wireproto: use command executor for unbundle This also required unifying the name of the argument because the new API always passes arguments by keyword. I decided to change implementations to "bundle" instead of the interface to "cg" because "bundle" is more appropriate in a modern world. Differential Revision: https://phab.mercurial-scm.org/D3314

debugcommands: perform handshake when obtaining httpv2 peer If we obtain an httpv2peer directly, the instance doesn't have an API descriptor and therefore doesn't know about the remote's commands, feature support, etc. This doesn't matter now. But when we implement the peer so it consults the API descriptor as part of sending commands, it will. So we change the logic for obtaining an http version 2 peer to go through makepeer() so the peer will perform the handshake and pass the API descriptor to the httpv2peer instance. Tests changed because we now perform a ?cmd=capabilities when obtaining version 2 peers. The Content-Length header is globbed because compression info will lack zstandard for pure builds. Differential Revision: https://phab.mercurial-scm.org/D3296

wireproto: rename HTTPV2 so it less like HTTP/2 Per review suggestion on D3230 from Augie. Differential Revision: https://phab.mercurial-scm.org/D3295

bundlerepo: use command executor for wire protocol commands Differential Revision: https://phab.mercurial-scm.org/D3294

bundlerepo: rename "other" to "peer" Differential Revision: https://phab.mercurial-scm.org/D3293