hgweb: use a multidict for holding query string parameters My intention with refactoring the WSGI code was to make it easier to read. I initially wanted to vendor and use WebOb, because it seems to be a pretty reasonable abstraction layer for WSGI. However, it isn't using relative imports and I didn't want to deal with the hassle of patching it. But that doesn't mean we can't use good ideas from WebOb. WebOb has a "multidict" data structure for holding parsed query string and POST form data. It quacks like a dict but allows you to store multiple values for each key. It offers mechanisms to return just one value, all values, or return 1 value asserting that only 1 value is set. I quite like its API. This commit implements a read-only "multidict" in the spirit of WebOb's multidict. We replace the query string attributes of our parsed request with an instance of it. Differential Revision: https://phab.mercurial-scm.org/D2776

hgweb: create dedicated type for WSGI responses We have refactored the request side of WSGI processing into a dedicated type. Now let's do the same thing for the response side. We invent a ``wsgiresponse`` type. It takes an instance of a request (for consulation) and the WSGI application's "start_response" handler. The type basically allows setting the HTTP status line, response headers, and the response body. The WSGI application calls sendresponse() to start sending output. Output is emitted as a generator to be fed through the WSGI application. According to PEP 3333, this is the preferred way for output to be transmitted. (Our legacy ``wsgirequest`` exposed a write() to send data. We do not wish to support this API because it isn't recommended by PEP 3333.) The wire protocol code has been ported to use the new API. Differential Revision: https://phab.mercurial-scm.org/D2775

tests: add test for a wire protocol request to wrong base URL We have code that validates that wire protocol commands (which are specified via query string) must occur at the base URL of a repo. But we have no test coverage for this behavior. Let's add some. Differential Revision: https://phab.mercurial-scm.org/D2778

hgweb: remove support for short query string based aliases (BC) Form data exposed by hgweb is post-processed to expand certain shortcuts. For example, URLs with "?cs=@" is essentially expanded to "?cmd=changeset&node=@". And the URL router treats this the same as "/changeset/@". These shortcuts were initially added in 2005 in 34cb3957d875 and 964baa35faf8. They have rarely been touched in the last decade (just moving code around a bit). We have almost no test coverage of this feature. AFAICT no templates reference URLs of this form. I even looked at the initial version of paper and coal from ~2008 and they use the "/command/params" URL form and not these shortcuts. Furthermore, I couldn't even get some shortcuts to work! For example, "?sl=@" attempts to do a revision search instead of showing shortlog starting at revision @. Maybe I'm just doing it wrong? Because this is ancient, mostly untested code, there is a migration path to something better, and because anyone passionate enough to preserve URLs can install URL redirects, let's nuke the feature. .. bc:: Query string shorts in hgweb like ``?cs=@`` have been removed. Use URLs of the form ``/:cmd`` instead. Differential Revision: https://phab.mercurial-scm.org/D2773

hgweb: remove support for POST form data (BC) Previously, we called out to cgi.parse(), which for POST requests parsed multipart/form-data and application/x-www-form-urlencoded Content-Type requests for form data, combined it with query string parameters, returned a union of the values. As far as I know, nothing in Mercurial actually uses this mechanism to submit data to the HTTP server. The wire protocol has its own mechanism for passing parameters. And the web interface only does GET requests. Removing support for parsing POST data doesn't break any tests. Another reason to not like this feature is that cgi.parse() may modify the QUERY_STRING environment variable as a side-effect. In addition, it merges both POST data and the query string into one data structure. This prevents consumers from knowing whether a variable came from the query string or POST data. That can matter for some operations. I suspect we use cgi.parse() because back when this code was initially implemented, it was the function that was readily available. In other words, I don't think there was conscious choice to support POST data: we just got it because cgi.parse() supported it. Since nothing uses the feature and it is untested, let's remove support for parsing POST form data. We can add it back in easily enough if we need it in the future. .. bc:: Hgweb no longer reads form data in POST requests from multipart/form-data and application/x-www-form-urlencoded requests. Arguments should be specified as URL path components or in the query string in the URL instead. Differential Revision: https://phab.mercurial-scm.org/D2774

hgweb: expose input stream on parsed WSGI request object Our next step towards moving away from wsgirequest to our newer, friendlier parsedrequest type is input stream access. This commit exposes the input stream on the instance. Consumers in the HTTP protocol server switch to it. Because there were very few consumers of the input stream, we stopped storing a reference to the input stream on wsgirequest directly. All access now goes through parsedrequest. However, wsgirequest still may read from this stream as part of cgi.parse(). So we still need to create the stream from wsgirequest. Differential Revision: https://phab.mercurial-scm.org/D2771