revlog: avoid hard-coded hash sizes Differential Revision: https://phab.mercurial-scm.org/D8683

test: restore expected output in test-persistent-nodemap.t The merge 78cafd48b9b2 had a small error and wrongly dropped the output updated from one side of the merge. This changeset re-install it. Differential Revision: https://phab.mercurial-scm.org/D8678

heptapod-ci: build rhg before tests run Differential Revision: https://phab.mercurial-scm.org/D8682

rhg: add a limited `rhg root` subcommand Clap has been choosen for argument parsing for the following reasons: - it's a wildly used and maintained crate - it can deal with OS encoding making it suitable for any encoding - it supports nonambiguous prefix matching as already available in hg - it will soon allow for structopts-style declarative pattern instead of the currently used builder pattern Differential Revision: https://phab.mercurial-scm.org/D8613

rhg: add RootCommand using hg-core FindRoot operation to prepare `hg root` The println macro is not used to avoid string usage. Dealing only with bytes allows us to be compatible with any encoding and not just UTF8. Later on, format macro will be made to have more readable code. Differential Revision: https://phab.mercurial-scm.org/D8612

archival: abort if compression method is unavailable `tarfile.CompressionError` is documented to be the "exception for unavailable compression methods". Also, make tests conditional on whether the lzma module is available or not.

demandimport: ignore `lzma` module for demandimport This makes importing the module fail if the `_lzma` module is not present. This makes e.g. tarfile correctly recognize if LZMA support is not present. It changes the exception File "/usr/lib/python3.6/tarfile.py", line 1694, in xzopen fileobj = lzma.LZMAFile(fileobj or name, mode, preset=preset) AttributeError: module 'lzma' has no attribute 'LZMAFile' to the more correct exception File "/usr/lib/python3.6/tarfile.py", line 1692, in xzopen raise CompressionError("lzma module is not available") tarfile.CompressionError: lzma module is not available Also, it prevents that the error "abort: No module named '_lzma'!" is shown when a development warning is to be shown. The reason why that happened is that for showing the warning, we get information about the stack frames from the inspect module, which accesses the `__file__` attribute of all modules in `sys.modules` to build some cache, causing all modules (including `lzma`) to be imported.

merge with stable

procutil: make mercurial.utils.procutil.stderr unbuffered For most Mercurial code, it doesn’t make a difference, as the ui object flushes stderr explicitly (after the change, we could get rid of the explicit flush). One example where it makes a observable difference is mercurial.util.timed(). Without the patch, the time is not immediately shown on Python 3. With the patch, it’s shown immediately on all Python versions and platforms.

tests: generalize common test case code in test-stdio.py

procutil: make stdout line-buffered on Windows if connected to TTY Windows doesn’t support line buffering. Previously, we worked around that by setting the stream unbuffered. Instead, we can use our own line buffering we already use on Python 3.

procutil: split import and reassignment Besides making the code clearer, it will reduce the diff in the next patch.

procutil: move assignments closer to reassignments Doing reassignments is an anti-pattern IMHO, but I see how it makes sense here. When first looking at this code after jumping here with ctags, I missed the fact that stdout was reassigned. To make the code clearer, the assignments should be as close as possible to the reassignments.

procutil: factor out conditional creation of LineBufferedWrapper At the same time, document the logic and generalize it to work on all Python versions.

procutil: define LineBufferedWrapper on all Python versions There’s nothing Python 3-only about LineBufferedWrapper. In the future, we may want to use it on Windows, to work around missing line-buffering support.

tests: add tests for buffering behavior of mercurial.utils.procutil.stdout

cleanup: use slightly more meaningful name for temporary variable Not that it makes a big difference, but using `p` instead of `x` is clearer to me.

cleanup: use any() instead of checking truthiness of temporary list It was not immediately obvious to me, when first seeing this, why a list was created. It needed a second look to understand that the purpose was to check whether the condition is true for any of the parents. Using any() for that is clearer.

chg: suppress OSError in _restoreio() and add some logging (issue6330) According to issue6330, running chg on heavy loaded systems can lead to following error: ``` Traceback (most recent call last): File "path-to-hg/mercurial/commandserver.py", line 650, in _acceptnewconnection self._runworker(conn) File "path-to-hg/mercurial/commandserver.py", line 701, in _runworker prereposetups=[self._reposetup], File "path-to-hg/mercurial/commandserver.py", line 470, in _serverequest sv.cleanup() File "path-to-hg/mercurial/chgserver.py", line 381, in cleanup self._restoreio() File "path-to-hg/mercurial/chgserver.py", line 444, in _restoreio os.dup2(fd, fp.fileno()) OSError: [Errno 16] Device or resource busy ``` [man dup2] indicates that, on Linux, EBUSY comes from a race condition between open() and dup2(). However it's not clear why open() race occurred for newfd=stdin/out/err. We suppress the OSError in _restoreio() since the forked worker process will finish anyway and add some logging. Thanks to Mitchell Plamann for a detailed bug description and Yuya Nishihara for suggesting the fix.

py3: fix crash when server address is 0.0.0.0 (issue6362) `socket.getfqdn()` assumes that the name is passed as `str` on Python 3 and always returns `str` in this case. Mercurial passed `bytes` (but still expected a `str` result), which worked by chance in many cases, except for e.g. b'0.0.0.0', which was returned unchanged, breaking later code. Instead of calling `socket.getfqdn()`, we can also use `self.server_name` from the base `HTTPServer` class, which already stores the FQDN of the locally-bound socket name (see `BaseHTTPServer.py` in the Python 2 stdlib and `http/server.py` in the Python 3 stdlib).

ui: fix Python 2.7 support for ui.timestamp-output Differential Revision: https://phab.mercurial-scm.org/D8675

merge with stable

run-tests: find python binary on Python 3 (issue6361) Return strings from _findprogram as all callers expect unicode strings. Previously the check in _usecorrectpython agains sysexecutable was always false on Python 3. Differential Revision: https://phab.mercurial-scm.org/D8674

tests: ignore possible diagnostics from gpg 2.2 Differential Revision: https://phab.mercurial-scm.org/D8672

Added signature for changeset 0ea9c86fac89

Added tag 5.4.2 for changeset 0ea9c86fac89

convert: handle percent-encoded bytes in file URLs like Subversion 75b59d221aa3 added most of the code that gets removed by this patch. It helped making progress on Python 3, but the reasoning was wrong in many ways. I tried to retract it while it was queued, but it was too late. Back then, I was asssuming that what happened on Python 2 (preserving bytes) is correct and my Python 3 change is a hack. However it turned out that Subversion interprets percent-encoded bytes as UTF-8. Accepting the same format as Subversion is a good idea. Consistency with urlreq.pathname2url() (as described in the removed comment) doesn’t matter because that function is only used for passing paths to urllib. This is not a backwards-incompatible change because before 5c0d5b48e58c, non-ASCII filenames didn’t work at all on Python 2. When the locale encoding is ISO-8859-15, `svn` accepts `file:///tmp/a%E2%82%AC` for `/tmp/a€`. Before this patch, this was the case for this extension on Python 3, but not on Python 2. This patch makes it work like with `svn` on both Python 2 and Python 3.

convert: add docstring on convert.subversion.geturl() The function is unusual for a bytes-handling function in Mercurial because it can’t handle arbitrary bytes. Therefore we should document this fact. Pointed out by Yuya Nishihara while reviewing e3b19004087a.

ui: add option to timestamp status and diagnostic messages Differential Revision: https://phab.mercurial-scm.org/D8640

tests: use path inside test dir This will make the diff for the next patch less noisy.

convert: convert URLs to UTF-8 for Subversion Preamble: for comprehension, note that the `path` of geturl() would better be called `path_or_url` (the argument of the call of getsvn() is called `url`). For HTTP(S) URLs, the changes don’t make a difference, as they are restricted to ASCII. For file URLs, the reasoning is the same as for paths: we have to roundtrip with what Subversion is doing. When the locale encoding is ISO-8859-15, trying to convert a SVN repo `file:///tmp/a€` failed before like this: file:///tmp/a%A4 does not look like a Subversion repository to libsvn version 1.14.0 Decoding the path using the locale encoding can fail. In this case, we have to bail out, as Subversion won’t be able to do anything useful with the path.

convert: correctly convert paths to UTF-8 for Subversion The previous code using encoding.tolocal() only worked by chance in these situations: * The string is ASCII: The fast path was triggered and the string was returned unmodified. * The local encoding is UTF-8: The source and target encoding is the same. * The string is not valid UTF-8 and the native encoding is ISO-8859-1: If the string doesn’t decode using UTF-8, ISO-8859-1 is tried as a fallback. During `hg convert`, the local encoding is always UTF-8. The irony is that in this case, encoding.tolocal() behaves like what someone would expect the reverse function, encoding.fromlocal(), to do. When the locale encoding is ISO-8859-15, trying to convert a SVN repo `/tmp/a€` failed before like this: file:///tmp/a%C2%A4 does not look like a Subversion repository to libsvn version 1.14.0 The correct URL is `file:///tmp/a%E2%82%AC`. Unlike previously (with the ISO-8859-1 fallback), decoding the path using the locale encoding can fail. In this case, we have to bail out, as Subversion won’t be able to do anything useful with the path.