Added tag 4.2.3 for changeset 943c91326b23

ssh: unban the use of pipe character in user@host:port string This vulnerability was fixed by the previous patch and there were more ways to exploit than using '|shellcmd'. So it doesn't make sense to reject only pipe character. Test cases are updated to actually try to exploit the bug. As the SSH bridge of git/svn subrepos are not managed by our code, the tests for non-hg subrepos are just removed. This may be folded into the original patches.

ssh: quote parameters using shellquote (SEC) This patch uses shellquote to quote ssh parameters more strictly to avoid shell injection.

subrepo: add tests for git rogue ssh urls (SEC) 'ssh://' has an exploit that will pass the url blindly to the ssh command, allowing a malicious person to have a subrepo with '-oProxyCommand' which could run arbitrary code on a user's machine. In addition, at least on Windows, a pipe '|' is able to execute arbitrary commands. When this happens, let's throw a big abort into the user's face so that they can inspect what's going on.

subrepo: add tests for svn rogue ssh urls (SEC) 'ssh://' has an exploit that will pass the url blindly to the ssh command, allowing a malicious person to have a subrepo with '-oProxyCommand' which could run arbitrary code on a user's machine. In addition, at least on Windows, a pipe '|' is able to execute arbitrary commands. When this happens, let's throw a big abort into the user's face so that they can inspect what's going on.

subrepo: add tests for hg rogue ssh urls (SEC) 'ssh://' has an exploit that will pass the url blindly to the ssh command, allowing a malicious person to have a subrepo with '-oProxyCommand' which could run arbitrary code on a user's machine. In addition, at least on Windows, a pipe '|' is able to execute arbitrary commands. When this happens, let's throw a big abort into the user's face so that they can inspect what's going on.

push: add tests for unsafe ssh url (SEC)

pull: add tests for unsafe ssh url (SEC)

clone: add tests for unsafe ssh url (SEC)

sshpeer: check for safe ssh url (SEC) Checking in the sshpeer for a rogue ssh:// urls seems like the right place to do it (instead of whack-a-mole with pull, clone, push, etc).

ssh: ban any username@host or host that starts with - (SEC) This paranoia probably isn't required, but it can't hurt either.

util: add utility method to check for bad ssh urls (SEC) Our use of SSH has an exploit that will parse the first part of an url blindly as a hostname. Prior to this set of security patches, a url with '-oProxyCommand' could run arbitrary code on a user's machine. In addition, at least on Windows, a pipe '|' can be abused to execute arbitrary commands in a similar fashion. We defend against this by checking ssh:// URLs and looking for a hostname that starts with a - or contains a |. When this happens, let's throw a big abort into the user's face so that they can inspect what's going on.

pathauditor: disable cache of audited paths by default (issue5628) The initial attempt was to discard cache when appropriate, but it appears to be error prone. We had to carefully inspect all places where audit() is called e.g. without actually updating filesystem, before removing files and directories, etc. So, this patch disables the cache of audited paths by default, and enables it only for the following cases: - short-lived auditor objects - repo.vfs, repo.svfs, and repo.cachevfs, which are managed directories and considered sort of append-only (a file/directory would never be replaced with a symlink) There would be more cacheable vfs objects (e.g. mq.queue.opener), but I decided not to inspect all of them in this patch. We can make them cached later. Benchmark result: - using old clone of http://selenic.com/repo/linux-2.6/ (38319 files) - on tmpfs - run HGRCPATH=/dev/null hg up -q --time tip && hg up -q null - try 4 times and take the last three results original: real 7.480 secs (user 1.140+22.760 sys 0.150+1.690) real 8.010 secs (user 1.070+22.280 sys 0.170+2.120) real 7.470 secs (user 1.120+22.390 sys 0.120+1.910) clearcache (the other series): real 7.680 secs (user 1.120+23.420 sys 0.140+1.970) real 7.670 secs (user 1.110+23.620 sys 0.130+1.810) real 7.740 secs (user 1.090+23.510 sys 0.160+1.940) enable cache only for vfs and svfs (this series): real 8.730 secs (user 1.500+25.190 sys 0.260+2.260) real 8.750 secs (user 1.490+25.170 sys 0.250+2.340) real 9.010 secs (user 1.680+25.340 sys 0.280+2.540) remove cache function at all (for reference): real 9.620 secs (user 1.440+27.120 sys 0.250+2.980) real 9.420 secs (user 1.400+26.940 sys 0.320+3.130) real 9.760 secs (user 1.530+27.270 sys 0.250+2.970)

tests: show cache of audited paths is never invalidated

Added signature for changeset 5544af862286

Added tag 4.3 for changeset 5544af862286

chg: define _GNU_SOURCE to allow CentOS 5 compilation Without this flag, compilation fails with: hgclient.c: In function 'hgc_open': hgclient.c:466: error: 'O_DIRECTORY' undeclared (first use in this function) hgclient.c:466: error: (Each undeclared identifier is reported only once hgclient.c:466: error: for each function it appears in.) Differential Revision: https://phab.mercurial-scm.org/D260

centos5: use vault urls CentOS 5 has reached end-of-life. A working build for it is still possible using 'vault.centos.org'. Differential Revision: https://phab.mercurial-scm.org/D261

ui: make sure buffer is flushed before waiting for user input (issue5587) Without this patch on Windows 'hg ci -i' hangs waiting for user input and "examine changes to 'file'? [Ynesfdaq?]" is never displayed (at least if the diff is sufficiently small). When Ctrl+C is pressed, this prompt becomes visible, which suggests that the buffer just wasn't flushed. I've never seen this happening on Linux, but this looks harmless enough to not platform-gate it.

util: fix sortdict.update() to call __setitem__() on PyPy (issue5639) It appears that overriding __setitem__() doesn't work as documented on PyPy. Let's patch it as before bd0fd3ff9916. https://docs.python.org/2/library/collections.html#ordereddict-examples-and-recipes The issue was ui.configitems() wasn't ordered correctly, so the pull command was wrapped in different order.

test-revlog-raw: close file handles explicitly (issue5644)

tests: clear __pycache__ for PyPy compatibility (issue5638) (issue5642)

test-extension: glob module name in error message for PyPy

rebase: sort collapsed revisions in commit message (issue5643) This fixes the test instability.

i18n: use actual filename, in which function is defined, for hg.pot Before this patch, source filename for msgid in hg.pot file becomes incorrect, if a function is defined in file A, but detected in dict in file B, For example, almost all debug* commands are defined in debugcommands.py, but hggettext detects them in "table" of commands.py. Therefore, docstring fragments of debug* commands are marked as "defined in commands.py" in hg.pot file. This is serious problem for translation, because the cost to find out original location of texts increases very much.

ui: enable pager always for explicit --pager=on (issue5580) Before this patch, explicit --pager=on is unintentionally ignored by any disabling factor, even if priority of it is less than --pager=on (e.g. "[ui] paginate = off").

README: make paragraphs before example code end with "::" for reST syntax After this patch, README as reST text can be rendered into HTML more appropriately. This is useful for hosting servers, which render README file as a part of index HTML of its directory (for example, Bitbucket).

commit: don't let failed commit with --addremove update dirstate (issue5645) Differential Revision: https://phab.mercurial-scm.org/D204

tests: demonstrate that failed "hg ci -A" updates dirstate (issue5645) Differential Revision: https://phab.mercurial-scm.org/D203

tests: fix test-flagprocessor.t flakiness The test for duplicate flag processors depended on the timestamps being set in the dirstate to work. If the time between the the previous failed commit (which would set the timestamp, due to bug 5645) and the attempted commit with the duplicate flag processors was small enough, it would fail. The failure was caused by a call to commands.status() early in the commit process. If the dirstate did not have the timestamp set, it would need to fetch the file content to compare with. Since two flag processors had been registered, it would attempted to base64 decode the contents twice, which would of course fail. This patch adds a "hg debugrebuilddirstate" to make it deterministic and also replaces the test case's "hg commit" by simply "hg status", since that will trigger reading of the contents and thereby use of the flag processors as noted above. Differential Revision: https://phab.mercurial-scm.org/D202