wireproto: define permissions-based routing of HTTPv2 wire protocol Now that we have a scaffolding for serving version 2 of the HTTP protocol, let's start implementing it. A good place to start is URL routing and basic request processing semantics. We can focus on content types, capabilities detect, etc later. Version 2 of the HTTP wire protocol encodes the needed permissions of the request in the URL path. The reasons for this are documented in the added documentation. In short, a) it makes it really easy and fail proof for server administrators to implement path-based authentication and b) it will enable clients to realize very early in a server exchange that authentication will be required to complete the operation. This latter point avoids all kinds of complexity and problems, like dealing with Expect: 100-continue and clients finding out later during `hg push` that they need to provide authentication. This will avoid the current badness where clients send a full bundle, get an HTTP 403, provide authentication, then retransmit the bundle. In order to implement command checking, we needed to implement a protocol handler for the new wire protocol. Our handler is just small enough to run the code we've implemented. Tests for the defined functionality have been added. I very much want to refactor the permissions checking code and define a better response format. But this can be done later. Nothing is covered by backwards compatibility at this point. Differential Revision: https://phab.mercurial-scm.org/D2836

wireproto: support /api/* URL space for exposing APIs I will soon be introducing a new version of the HTTP wire protocol. One of the things I want to change with it is the URL routing. I want to rely on URL paths to define endpoints rather than the "cmd" query string argument. That should be pretty straightforward. I was thinking about what URL space to reserve for the new protocol. We /could/ put everything at a top-level path. e.g. /wireproto/* or /http-v2-wireproto/*. However, these constrain us a bit because they assume there will only be 1 API: version 2 of the HTTP wire protocol. I think there is room to grow multiple APIs. For example, there may someday be a proper JSON API to query or even manipulate the repository. And I don't think we should have to create a new top-level URL space for each API nor should we attempt to shoehorn each future API into the same shared URL space: that would just be too chaotic. This commits reserves the /api/* URL space for all our future API needs. Essentially, all requests to /api/* get routed to a new WSGI handler. By default, it 404's the entire URL space unless the "api server" feature is enabled. When enabled, requests to "/api" list available APIs. URLs of the form /api/<name>/* are reserved for a particular named API. Behavior within each API is left up to that API. So, we can grow new APIs easily without worrying about URL space conflicts. APIs can be registered by adding entries to a global dict. This allows extensions to provide their own APIs should they choose to do so. This is probably a premature feature. But IMO the code is easier to read if we're not dealing with API-specific behavior like config option querying inline. To prove it works, we implement a very basic API for version 2 of the HTTP wire protocol. It does nothing of value except facilitate testing of the /api/* URL space. We currently emit plain text responses for all /api/* endpoints. There's definitely room to look at Accept and other request headers to vary the response format. But we have to start somewhere. Differential Revision: https://phab.mercurial-scm.org/D2834

url: support suppressing Accept header Sending this header automatically could interfere with future testing and client behavior. Let's add a knob to disable the behavior. We don't have a control for User-Agent because urllib will send it if we don't set something. I don't feel like hacking into the bowels of urllib to figure out how to suppress that. UA shouldn't be used for anything meaningful. So it shouldn't pose any problems beyond non-determinism (since the header has the Mercurial version in it). Differential Revision: https://phab.mercurial-scm.org/D2843

util: don't log low-level I/O calls for HTTP peer `hg debugwireproto` is useful for testing HTTP interactions. Possibly more useful than `get-with-headers.py`. But one thing that makes it annoying for mid-level tests is that it logs all API calls, such as readline(). This makes output - especially headers - overly verbose. This commit teaches our file and socket observers to not log API calls on functions dealing with data. We change the behavior of `hg debugwireproto` to enable this mode by default. --debug can be added to restore the previous behavior. As the test changes demonstrate, this makes tests much easier to read. As a bonus, it also removes some required (glob) over lengths in system call results. One thing that's lacking is knowing which side sent data. But we can fix this in a follow-up once it becomes a problem. Differential Revision: https://phab.mercurial-scm.org/D2842

rebase: rename conclude[memory]node() to commit[memory]node() The functions do little more than commit at this point. Differential Revision: https://phab.mercurial-scm.org/D2924

rebase: pass in "user" instead of "ctx" to conclude[memory]node() This was the only remaining part of the context object that was needed. Differential Revision: https://phab.mercurial-scm.org/D2923

rebase: look up default date outside of conclude[memory]node() Differential Revision: https://phab.mercurial-scm.org/D2922

rebase: move config override out of conclude[memory]node() Differential Revision: https://phab.mercurial-scm.org/D2921

rebase: pass in entire "overrides" dict to conclude[memory]node() As with previous patches, this was done the same way in both functions, so let's make the caller do it instead. Differential Revision: https://phab.mercurial-scm.org/D2920

rebase: pass in "keepbranch" to conclude[memory]node() Both functions calculated the same "keepbranch" value from the "keepbranches" we passed in, so let's make the caller do it instead. Differential Revision: https://phab.mercurial-scm.org/D2919