merge with stable

hgweb: add absolute urls for archives in json-summary

tests: allow bz2 archives in test-hgweb-json.t Only testing a specific type because list items seem to be in arbitrary order.

keepalive: switch from thread to threading module The thread module in py3 is renamed to _thread, but we can use the high level threading module instead.

py3: conditionalize httplib import The httplib library is renamed to http.client in python 3. So the import is conditionalized and a test is added in check-code to warn to use util.httplib

Added signature for changeset 26a5d605b868

Added tag 3.8.4 for changeset 26a5d605b868

sslutil: synchronize hostname matching logic with CPython sslutil contains its own hostname matching logic. CPython has code for the same intent. However, it is only available to Python 2.7.9+ (or distributions that have backported 2.7.9's ssl module improvements). This patch effectively imports CPython's hostname matching code from its ssl.py into sslutil.py. The hostname matching code itself is pretty similar. However, the DNS name matching code is much more robust and spec conformant. As the test changes show, this changes some behavior around wildcard handling and IDNA matching. The new behavior allows wildcards in the middle of words (e.g. 'f*.com' matches 'foo.com') This is spec compliant according to RFC 6125 Section 6.5.3 item 3. There is one test where the matcher is more strict. Before, '*.a.com' matched '.a.com'. Now it doesn't match. Strictly speaking this is a security vulnerability.

tests: import CPython's hostname matching tests CPython has a more comprehensive test suite for it's built-in hostname matching functionality. This patch adds its tests so we can improve our hostname matching functionality. Many of the tests have different results from CPython. These will be addressed in a subsequent commit.

i18n-pt_BR: synchronized with dd9175ca81dc

sslutil: emit warning when no CA certificates loaded If no CA certificates are loaded, that is almost certainly a/the reason certificate verification fails when connecting to a server. The modern ssl module in Python 2.7.9+ provides an API to access the list of loaded CA certificates. This patch emits a warning on modern Python when certificate verification fails and there are no loaded CA certificates. There is no way to detect the number of loaded CA certificates unless the modern ssl module is present. Hence the differences in test output depending on whether modern ssl is available. It's worth noting that a test which specifies a CA file still renders this warning. That is because the certificate it is loading is a x509 client certificate and not a CA certificate. This test could be updated if anyone is so inclined.

tests: test case where default ca certs not available I'm not a fan of TLS tests not testing both branches of a possible configuration. While we have test coverage of the inability to validate a cert later in this file, I insist that we add this branch so our testing of security code is extra comprehensive.

sslutil: don't load default certificates when they aren't relevant Before, we would call SSLContext.load_default_certs() when certificate verification wasn't being used. Since SSLContext.verify_mode == ssl.CERT_NONE, this would ideally no-op. However, there is a slim chance the loading of system certs could cause a failure. Furthermore, this behavior interfered with a future patch that aims to provide a more helpful error message when we're unable to load CAs. The lack of test fallout is hopefully a sign that our security code and tests are in a relatively good state.

sslutil: display a better error message when CA file loading fails Before, sslcontext.load_verify_locations() would raise a ssl.SSLError which would be caught further up the stack and converted to a urlerror. By that time, we lost track of what actually errored. Trapping the error here gives users a slightly more actionable error message. The behavior between Python <2.7.9 and Python 2.7.9+ differs. This is because our fake SSLContext class installed on <2.7.9 doesn't actually do anything during load_verify_locations: it defers actions until wrap_socket() time. Unfortunately, a number of errors can occur at wrap_socket() time and we're unable to ascertain what the root cause is. But that shouldn't stop us from providing better error messages to people running a modern and secure Python version.

tests: add test for empty CA certs file smf reported that an environment with no loaded CA certs resulted in a weird error. I'd like to detect this a bit better so we can display an actionable error message. The actual error being globbed over in this patch is "unknown error" with a ssl.c line number. That isn't useful at all.

internals: move the bitmanipulation routines into its own file This is to allow more flexibility with the C sources -- now the bitmanipulation routines can be safely imported without importing Python.h

journal: new experimental extension Records bookmark locations and shows you where bookmarks were located in the past. This is the first in a planned series of locations to be recorded; a future patch will add working copy (dirstate) tracking, and remote bookmarks will be supported as well, so the journal storage format should be fairly generic to support those use-cases.

httpclient: update to 54868ef054d2 of httpplus As of that revision, httpplus fully supports Python 3, including mimicing all the subtle behavior changes around headers in Python 3's http.client.

revset: check invalid function syntax "func-name"() explicitly Before the error was caught at func() as an unknown identifier, and the optimizer failed to detect the syntax error. This patch introduces getsymbol() helper to ensure that a string is not allowed as a function name.

chg: silence warning of unused parameter 'sig'

hgweb: reindent atom/changelogentry.tmpl It was mixing tabs and spaces, and not in a good way. Indent style of other atom entries seems to be 1 space per level, so let's apply it here as well.

hgweb: remove unused/nonexistent filelogentry from atom/map filelog in atom style uses changelogentry to show commits, and atom/filelogentry.tmpl doesn't even exist.

hgweb: remove unnecessary CDATA markup in atom/branchentry.tmpl Since content is of type "text" (and is already escaped), using a CDATA section is not required. Looks like this was just an artifact of copying things from rss style in add2f9ddcfb5, because other entries in atom style don't use CDATA in such places.

hgweb: reindent rss/changelogentry.tmpl It was mixing tabs and spaces, and not in a good way. Indent style of other rss entries seems to be 4 spaces per level, so let's apply it here as well.

hgweb: remove a couple of superfluous spaces in rss style

py3: add tests in check-code to load modules from util.py The conditionalize imports are added in util.py and now we import modules from there. So adding tests so that someone in future can use that.

py3: conditionalize SocketServer import The SocketServer is renamed to socketserver in python 3

py3: conditionalize xmlrpclib import The xmlrpclib library is renamed to xmlrpc.client in python 3

py3: conditionalize the urlparse import The urlparse library is renamed to urllib.parse in python 3

py3: update tests/test-check-py3-compat.t The lower part of the test runs with python 3 and hence remain unchanged.