Merge stable with security patch.

Added signature for changeset 77eaf9539499

Added tag 4.1.3 for changeset 77eaf9539499

dispatch: protect against malicious 'hg serve --stdio' invocations (sec) Some shared-ssh installations assume that 'hg serve --stdio' is a safe command to run for minimally trusted users. Unfortunately, the messy implementation of argument parsing here meant that trying to access a repo named '--debugger' would give the user a pdb prompt, thereby sidestepping any hoped-for sandboxing. Serving repositories over HTTP(S) is unaffected. We're not currently hardening any subcommands other than 'serve'. If your service exposes other commands to users with arbitrary repository names, it is imperative that you defend against repository names of '--debugger' and anything starting with '--config'. The read-only mode of hg-ssh stopped working because it provided its hook configuration to "hg serve --stdio" via --config parameter. This is banned for security reasons now. This patch switches it to directly call ui.setconfig(). If your custom hosting infrastructure relies on passing --config to "hg serve --stdio", you'll need to find a different way to get that configuration into Mercurial, either by using ui.setconfig() as hg-ssh does in this patch, or by placing an hgrc file someplace where Mercurial will read it. mitrandir@fb.com provided some extra fixes for the dispatch code and for hg-ssh in places that I overlooked.

progress: retry ferr.flush() and .write() on EINTR (issue5532) See the inline comment how this could mitigate the issue. I couldn't reproduce the exact problem on my Linux machine, but there are at least two people who got EINTR in progress.py, and it seems file_write() of Python 2 is fundamentally broken [1]. Let's make something in on 4.2. [1]: https://hg.python.org/cpython/file/v2.7.13/Objects/fileobject.c#l1850

progress: extract stubs to restart ferr.flush() and .write() on EINTR

filemerge: optionally strip quotes from merge marker template (BC) For consistency with the other template options. Quotes are necessary if you want to preserve leading/trailing whitespace, which would be stripped by config parser.

commit: optionally strip quotes from commit template (BC) For consistency with the other template options.

graphlog: optionally strip quotes from graphnode template (BC) For consistency with the other template options.

dispatch: ignore further SIGPIPE while handling KeyboardInterrupt I got the following error by running "hg log" and quitting the pager immediately. Any output here may trigger another SIGPIPE, so only thing we can do is to swallow the exception and exit with an error status. Traceback (most recent call last): File "./hg", line 45, in <module> mercurial.dispatch.run() File "mercurial/dispatch.py", line 83, in run status = (dispatch(req) or 0) & 255 File "mercurial/dispatch.py", line 167, in dispatch req.ui.warn(_("interrupted!\n")) File "mercurial/ui.py", line 1224, in warn self.write_err(*msg, **opts) File "mercurial/ui.py", line 790, in write_err self._write_err(*msgs, **opts) File "mercurial/ui.py", line 798, in _write_err self.ferr.write(a) File "mercurial/ui.py", line 129, in _catchterm raise error.SignalInterrupt mercurial.error.SignalInterrupt Perhaps this wasn't visible before de5c9d0e02ea because the original stderr handle was restored very late.

worker: print traceback for uncaught exception unconditionally This is what a Python interpreter would do if there were no os._exit().

worker: propagate exit code to main process Follows up 86cd09bc13ba.

dispatch: print traceback in scmutil.callcatch() if --traceback specified Otherwise, traceback wouldn't be printed for a known exception occurred in worker processes.

dispatch: mark callcatch() as a private function

templatefilters: fix crash by string formatting of '{x|splitlines}' Before, it crashed because mapping['templ'] was missing. As it didn't support the legacy list template from the beginning, we can simply use hybridlist().

templatekw: factor out showdict() helper Make it less cryptic for common cases.

templatekw: have showlist() take mapping dict with no **kwargs expansion (API) See the previous commit for why. splitlines() does not pass a mapping dict, which would probably mean the legacy template didn't work from the beginning.

templatekw: change _showlist() to take mapping dict with no **kwargs expansion There was a risk that a template keyword could conflict with an argument name (e.g. 'name', 'values', 'plural', etc.) Let's make it less magical.

templatekw: rename 'args' to 'mapping' in showlist() The name 'args' provides no information. Call it 'mapping' as in templater.py.

templatekw: eliminate unnecessary temporary variable 'names' from _showlist() Replace 'names' with the optional argument 'plural'.

color: update the help with the new default The default is now "auto" we update the help to match reality.

upgrade: register all format variants in a list Now that all known format variants exists outside of the function, we can gather them in a lists. This build a single entry point other code can use (current target: extensions). The repository upgrade code is updated to simply use entries from this list. As a side effect this will also allow extensions to register their own format variants, to do this "properly" we should introduce a "registrar" for this category of object. However I prefer to keep this series simple, and that will be adventure for future time.

upgrade: move descriptions and selection logic in individual classes Our goal here is to get top level definition for all the format variants. Having them defined outside of the function enabled other users of that logic. They are two keys components of a format variant: 1) the name and various descriptions of its effect, 2) the code that checks if the repo is using this variant and if the config enables it. That second items make us pick a class-based approach, since different variants requires different code (even if in practice, many can reuse the same logic). Each variants define its own class that is then used like a singleton. The class-based approach also clarify the definitions part a bit since each are simple assignment in an indented block. The 'fromdefault' and 'fromconfig' are respectively replaced by a class attribute and a method to be called at the one place where "fromconfig" matters. Overall, they are many viable approach for this, but this is the one I picked.

upgrade: introduce a 'formatvariant' class The 'deficiency' type has multiple specificities. We create a dedicated class to host them. More logic will be added incrementally in future changesets.

upgrade: implement '__hash__' on 'improvement' class The pythonomicon request its implementation.

upgrade: implement '__ne__' on 'improvement' class The pythonomicon request its implementation.

color: also enable by default on windows I've not found anything related to color + windows on the bug tracker. So I'm suggesting we get bolder and turn it on for windows too in the release candidate. We can always backout this changeset if we find serious issue on windows.

color: turn on by default (but for windows) Color support is all in core for a couple of months. I've browsed the bug tracker without finding any blocker bug. So I'm moving forward and enable color on by default before '4.2-rc'. In the worse case, having it on in the release candidate will help us to find blocker bug and we can turn it off for the final release. I remember people talking about issue with Windows during the freeze so I'm keeping it off by default on that OS. We could do various cleaning of the color used and the label issued. However the label are probably already in our backward compatibility envelope since the color extensions has been around since for ever and I do not think the color choice themself should be considered BC. So I think we should rather gives color to all user sooner than later. A couple of test needs to be updated to avoid having color related control code spoil the tested output.

pager: stop using the color extension in tests The feature is in core so let us use the core config knob directly.

bundle2: ignore errors seeking a bundle after an exception (issue4784) Many have seen a "stream ended unexpectedly" error. This message is raised from changegroup.readexactly() when a read(n) operation fails to return exactly N bytes. I believe most occurrences of this error in the wild stem from the code changed in this patch. Before, if bundle2's part applicator raised an Exception when processing/applying parts, the exception handler would attempt to iterate the remaining parts. If I/O during this iteration failed, it would likely raise the "stream ended unexpectedly" exception. The problem with this approach is that if we already encountered an I/O error iterating the bundle2 data during application, then any further I/O would almost certainly fail. If the original stream were closed, changegroup.readexactly() would obtain an empty string, triggering "stream ended unexpectedly" with "got 0." This is the error message that users would see. What's worse is that the original I/O related exception would be lost since a new exception would be raised. This made debugging the actual I/O failure effectively impossible. This patch changes the exception handler for bundle2 application to ignore errors when seeking the underlying stream. When the underlying error is I/O related, the seek should fail fast and the original exception will be re-raised. The output changes in test-http-bad-server.t demonstrate this. When the underlying error is not I/O related and the stream can be seeked, the old behavior is preserved.