subrepo: check return code for git push (issue4223)

localrepo: add unbundle support Localrepo now supports the unbundle method of pushing changegroups. We plan to use the unbundle call for bundle2 so it is important that all peers supports it. The `peer.unbundle` and `peer.addchangegroup` code path have small difference so cause some test output changes. None of those changes seems problematic.

unbundle: extract the core logic in another function The `exchange` module now contains an `unbundle` function that holds the core unbundle logic. The wire protocol keeps its own unbundle function. It enforces wireprotocol-specific logic and then calls the extracted function. This aims at implementing unbundle for localrepo.

unbundle: extract checkheads in its own function We are going to refactor the unbundle function to have it working on a local repository too. Having this function extracted will ease the process. In the case of non-matching heads, the function now directly raises an exception. The top level of the function is catching it.

phase: apply publishing enforcement for "serve" source When a changegroup is added by a push on a publishing server, we ensure they are added as public. This is used to enforce publishing on server when the client is not aware of phases. It also prevents race conditions where a reader could see the changesets as draft before they get turned public by the client. Finally, this save rounds trip as the client does not need additional request to turn them public. However, this logic was only enforced when the changegroup was from a "push" source. And "push" is used for local pushes only. Wireprotocol push uses "serve" as source since Mercurial 1.9. We now enforce this logic for both "push" and "serve" sources. One could note that this logic was mainly useful during wireprotocol exchanges. So this code is finally put into good use, 9 versions after its introduction.

merge with stable

http: reuse authentication info after the first failed request (issue3567) [This was applied in 181108726ea5 but backed out again in af02783dea65 because of Python 2.4 issues. This edition and test-http.t works with Python 2.4.] Context: mercurial access to repository server with http access, and this server is protected by basic auth. Before patch: * mercurial try an anonymous access to server, server return 401 response and mercurial resend request with login / password information After patch: * mercurial try an anonymous access to server, server return 401 response. For all next requests, mercurial keep in memory this information (this server need basic auth information). This patch reduce the number of http access against mercurial server. Example, before patch: 10.10.168.170 - - [25/Oct/2013:15:44:51 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:44:52 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 200 147 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:00 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:01 +0200] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 200 147 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:03 +0200] "GET /hg/testagt?cmd=batch HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:04 +0200] "GET /hg/testagt?cmd=batch HTTP/1.1" 200 42 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:06 +0200] "GET /hg/testagt?cmd=getbundle HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:07 +0200] "GET /hg/testagt?cmd=getbundle HTTP/1.1" 200 61184 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:09 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:10 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 15 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:12 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [25/Oct/2013:15:45:12 +0200] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 - "-" "mercurial/proto-1.0" Example after patch: 10.10.168.170 - - [28/Oct/2013:11:49:14 +0100] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 401 260 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:15 +0100] "GET /hg/testagt?cmd=capabilities HTTP/1.1" 200 147 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:17 +0100] "GET /hg/testagt?cmd=batch HTTP/1.1" 200 42 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:19 +0100] "GET /hg/testagt?cmd=getbundle HTTP/1.1" 200 61184 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:22 +0100] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 15 "-" "mercurial/proto-1.0" 10.10.168.170 - - [28/Oct/2013:11:49:24 +0100] "GET /hg/testagt?cmd=listkeys HTTP/1.1" 200 - "-" "mercurial/proto-1.0" In this last example, you can see only one 401 response.

bundle2: use discard to remove bundle2 cap caps.remove('bundle2') was throwing an exception if bundle2 wasn't present in the capabilities. This was causing test-static-http.t to hang. Let's just use discard, so we don't get an exception.

statichttp: respect localrepo _restrictcapabilities The static http repository was doing his own filtering of capability ignoring the filtering done in the local repo main class. This led to static http using the current draft of bundle2. We now apply both.

tests: make unshelve tests more tricky - don't depend on size change One reason shelve and largefiles doesn't work could be rapidly changing standin files. Prove that shelve in general doesn't have problems with that.

shelve: introduce secret option for using fixed date for temporary commit Using a fixed date makes hashes stable and makes debugging simpler. The date and hashes of this changeset are normally not exposed.

mq: repo['.'] is not a wctx, repo[None] is The parameters passed to subrepo.submerge are confusing anyway.

shelve: repo['.'] is not a wctx but a pctx Don't confuse hackers!

revlog: deal with chunk ranges over 2G on Windows (issue4215) Python uses a C long (32 bits on Windows 64) rather than an ssize_t in read(), and thus has a 2G size limit. Work around this by falling back to reading one chunk at a time on overflow. This approximately doubles our headroom until we run back into the size limit on single reads.

exchange: pass bundlecaps through to changegroup The bundlecaps passed to exchange.getbundle were being dropped completely. We should pass them on through to the changegroup. This affected the remotefilelog extension, since it relies on those bundlecaps.

bundle2: allow pulling changegroups using bundle2 This changeset refactors the pull code to use a bundle2 when available. We keep bundle2 disabled by default. The current code is not ready for prime time. Ultimately we'll want to unify the API of `bunde10` and `bundle20` to have less different code. But for now, testing the bundle2 exchange flow is an higher priority.

bundle2: add an exchange.getbundle function This function can return a `HG10` or `HG20` bundle. It uses the `bundlecaps` parameters to decides which one to return. This is a distinct function from `changegroup.getbundle` for two reasons. First the APIs of `bundle10` and `bundle20` are not compatible yet. The two functions may be reunited in the future. Second `exchange.getbundle` will grow parameters for all kinds of data (phases, obsmarkers, ...) so it's better to keep the changegroup generation in its own function for now. This function will be used it in the next changesets.

localpeer: propagate bundlecaps in getbundle call Best arguments are the ones in use...

bundle2: make it possible have a global transaction for the unbundling We use the `gettransaction` method approach already used for pull. We need this because we do not know beforehand if the bundle needs a transaction to be created. And (1) we do not want to create a transaction for nothing. (2) Some bundle2 bundles may be read-only and do not require any lock or transaction to be held.

tests: fix test output typo

bundle2: first crude version of bundling changeset with bundle2 The current changegroup format is put in a "changegroup" part and processed by an appropriate handlers. This is not production ready code, but let us start smoke testing.

bundle2: record processing results in the bundleoperation object Part handlers can now add records to the `bundleoperation` object. This can be used to help other parts or to let the caller of the unbundling process react to the results.

bundle2: introduce a bundleoperation object This object will hold all data and state gathered through the processing of a bundle. This will allow: - each handler to be aware of the things unbundled so far - the caller to retrieve data about the execution - bear useful object and logic (like repo, transaction) - bear possible bundle2 reply triggered by the unbundling. For now the object is very simple but it will grow at the same time as the bundle2 implementation.

bundle2: feed a unbundle20 to the `processbundle` function The unbundle can comes from multiple sources. (on disk file, peer, etc) and (ultimately) of multiple type (bundle10, bundle20). The `processbundle` is no longer in charge of creating the bundle.