.arcconfig
author Gregory Szorc <gregory.szorc@gmail.com>
Fri, 13 Apr 2018 11:02:34 -0700
changeset 37631 2f626233859b
parent 33583 b2c27d84f05c
child 44128 ff396501e841
permissions -rw-r--r--
wireproto: implement batching on peer executor interface This is a bit more complicated than non-batch requests because we need to buffer sends until the last request arrives *and* we need to support resolving futures as data arrives from the remote. In a classical concurrent.futures executor model, the future "starts" as soon as it is submitted. However, we have nothing to start until the last command is submitted. If we did nothing, calling result() would deadlock, since the future hasn't "started." So in the case where we queue the command, we return a special future type whose result() will trigger sendcommands(). This eliminates the deadlock potential. It also serves as a check against callers who may be calling result() prematurely, as it will prevent any subsequent callcommands() from working. This behavior is slightly annoying and a bit restrictive. But it's the world that half duplex connections forces on us. In order to support streaming responses, we were previously using a generator. But with a futures-based API, we're using futures and not generators. So in order to get streaming, we need a background thread to read data from the server. The approach taken in this patch is to leverage the ThreadPoolExecutor from concurrent.futures for managing a background thread. We create an executor and future that resolves when all response data is processed (or an error occurs). When exiting the context manager, we wait on that background reading before returning. I was hoping we could manually spin up a threading.Thread and this would be simple. But I ran into a few deadlocks when implementing. After looking at the source code to concurrent.futures, I figured it would just be easier to use a ThreadPoolExecutor than implement all the code needed to manually manage a thread. To prove this works, a use of the batch API in discovery has been updated. Differential Revision: https://phab.mercurial-scm.org/D3269
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
33583
b2c27d84f05c phabricator: include the suggested arc config in the repo
Alex Gaynor <agaynor@mozilla.com>
parents:
diff changeset
     1
{
b2c27d84f05c phabricator: include the suggested arc config in the repo
Alex Gaynor <agaynor@mozilla.com>
parents:
diff changeset
     2
    "conduit_uri": "https://phab.mercurial-scm.org/api",
b2c27d84f05c phabricator: include the suggested arc config in the repo
Alex Gaynor <agaynor@mozilla.com>
parents:
diff changeset
     3
    "repository.callsign": "HG",
b2c27d84f05c phabricator: include the suggested arc config in the repo
Alex Gaynor <agaynor@mozilla.com>
parents:
diff changeset
     4
    "arc.land.onto.default": "@",
b2c27d84f05c phabricator: include the suggested arc config in the repo
Alex Gaynor <agaynor@mozilla.com>
parents:
diff changeset
     5
    "base": "hg:.^"
b2c27d84f05c phabricator: include the suggested arc config in the repo
Alex Gaynor <agaynor@mozilla.com>
parents:
diff changeset
     6
}