commandserver: backport handling of forking server from chgserver This is common between chg and vanilla forking server, so move it to commandserver and unify handle(). It would be debatable whether we really need gc.collect() or not, but that is beyond the scope of this series. Maybe we can remove gc.collect() once all resource deallocations are switched to context manager.

commandserver: promote .cleanup() hook from chgserver This allows us to unify _requesthandler.handle().

commandserver: extract method to create commandserver instance per request This is a step toward merging chgserver._requesthandler with commandserver's.

error: make hintable exceptions reject unknown keyword arguments (API) Previously they would accept any typos of the hint keyword.

error: make HintException a mix-in class not derived from BaseException (API) HintException is unrelated to the hierarchy of errors. It is an implementation detail whether a class inherits from HintException or not, a sort of "private inheritance" in C++. New Hint isn't an exception class, which prevents catching error by its type: try: dosomething() except error.Hint: pass Unfortunately, this passes on PyPy 5.3.1, but not on Python 2, and raises more detailed TypeError on Python 3.

sslutil: move context options flags to _hostsettings Again, moving configuration determination to a single location.

sslutil: move protocol determination to _hostsettings Most of the logic for configuring TLS is now in this function. Let's move protocol determination code there as well.

share: don't recreate the source repo each time Previously, every time you asked for the source repo of a shared working copy it would recreate the repo object, which required calling reposetup. With certain extension enabled, this can be quite expensive, and it can happen many times (for instance, share attaches a post transaction hook to update bookmarks that triggers this). The fix is to just cache the repo object instead of constantly recreating it.

setup: prepare for future cffi modules by adding placeholder in setup

journal: add support for seaching by pattern If a pattern is used, include the entry name in the output, to make it clear what name was matched.

journal: add share extension support Rather than put everything into one journal file, split entries up in *shared* and *local* entries. Working copy changes are local to a specific working copy, so should remain local only. Other entries are shared with the source if so configured when the share was created. When unsharing, any shared journale entries are copied across.

journal: add dirstate tracking Note that now the default action for `hg journal` is to list the working copy history, not all bookmarks. In its place is the `--all` switch which lists all name changes recorded, including the name for which the change was recorded on each line. Locking is switched to using a dedicated lock to avoid issues with the dirstate being written during wlock unlocking (you can't re-lock during that process).

merge with stable

sslutil: try to find CA certficates in well-known locations Many Linux distros and other Nixen have CA certificates in well-defined locations. Rather than potentially fail to load any CA certificates at all (which will always result in a certificate verification failure), we scan for paths to known CA certificate files and load one if seen. Because a proper Mercurial install will have the path to the CA certificate file defined at install time, we print a warning that the install isn't proper and provide a URL with instructions to correct things. We only perform path-based fallback on Pythons that don't know how to call into OpenSSL to load the default verify locations. This is because we trust that Python/OpenSSL is properly configured and knows better than Mercurial. So this new code effectively only runs on Python <2.7.9 (technically Pythons without the modern ssl module).

sslutil: issue warning when unable to load certificates on OS X Previously, failure to load system certificates on OS X would lead to a certificate verify failure and that's it. We now print a warning message with a URL that will contain information on how to configure certificates on OS X. As the inline comment states, there is room to improve here. I think we could try harder to detect Homebrew and MacPorts installed certificate files, for example. It's worth noting that Homebrew's openssl package uses `security find-certificate -a -p` during package installation to export the system keychain root CAs to etc/openssl/cert.pem. This is something we could consider adding to setup.py. We could also encourage packagers to do this. For now, I'd just like to get this warning (which matches Windows behavior) landed. We should have time to improve things before release.

revert: don't backup if no files reverted in interactive mode (issue4793) When reverting interactively, we always backup files before prompting the user to find out if they actually want to revert them. This can create spurious *.orig files if a user enters an interactive revert session and then doesn't revert any files. Instead, we should only backup files that are actually being touched.

perf: define command annotation locally for Mercurial earlier than 3.1 Before this patch, using cmdutil.command() for "@command" annotation prevents perf.py from being loaded by Mercurial earlier than 1.9 (or 2daa5179e73f), because cmdutil.command() isn't available in such Mercurial, even though there are some code paths for Mercurial earlier than 1.9. For example, setting "_prereadsize" attribute in perfindex() and perfnodelookup() is effective only with hg earlier than 1.8 (or 61c9bc3da402). In addition to it, "norepo" option of command annotation has been available since 3.1 (or 75a96326cecb), and this is another blocker for loading perf.py with earlier Mercurial. ============ ============ ====== command of hg version cmdutil norepo ============ ============ ====== 3.1 or later o o 1.9 or later o x earlier x x ============ ============ ====== This patch defines "command()" for annotation locally as below: - define wrapper of existing cmdutil.command(), if cmdutil.command() doesn't support "norepo" (for Mercurial earlier than 3.1) - define full command() locally with minimum function, if cmdutil.command() isn't available at runtime (for Mercurial earlier than 1.9) This patch also defines parsealiases() locally without examining whether it is available or not, because it is small enough to define locally.

perf: avoid using formatteropts for Mercurial earlier than 3.2 Before this patch, referring commands.formatteropts prevents perf.py from being loaded by Mercurial earlier than 3.2 (or 7a7eed5176a4), because it isn't available in such Mercurial, even though formatting itself has been available since 2.2 (or ae5f92e154d3). In addition to it, there are some code paths for Mercurial earlier than 3.2. For example, setting "_prereadsize" attribute in perfindex() and perfnodelookup() is effective only with hg earlier than 1.8 (or 61c9bc3da402). This patch uses empty option list as formatteropts, if it isn't available in commands module at runtime. Disabling -T/--template option for earlier Mercurial should be reasonable, because: - since 427e80a18ef8, -T/--template for formatter has been available - since 7a7eed5176a4, commands.formatteropts has been available - the latter revision is direct child of the former

perf: use locally defined revlog option list for Mercurial earlier than 3.7 Before this patch, referring commands.debugrevlogopts prevents perf.py from being loaded by Mercurial earlier than 3.7 (or 5606f7d0d063), because it isn't available in such Mercurial, even though cmdutil.openrevlog(), a user of these options, has been available since 1.9 (or a79fea6b3e77). In addition to it, there are some code paths for Mercurial earlier than 3.7. For example, setting "_prereadsize" attribute in perfindex() and perfnodelookup() is effective only with hg earlier than 1.8 (or 61c9bc3da402). But just "using locally defined revlog option list" might cause unexpected behavior at runtime. If --dir option is specified to cmdutil.openrevlog() of Mercurial earlier than 3.5 (or 49c583ca48c4), it is silently ignored without any warning or so. ============ ============ ===== =============== debugrevlogopts hg version openrevlog() --dir of commands ============ ============ ===== =============== 3.7 or later o o o 3.5 or later o o x 1.9 or later o x x earlier x x x ============ ============ ===== =============== Therefore, this patch does: - use locally defined option list, if commands.debugrevlogopts isn't available (for Mercurial earlier than 3.7) - wrap cmdutil.openrevlog(), if it is ambiguous whether cmdutil.openrevlog() can recognize --dir option correctly (for Mercurial earlier than 3.5) This wrapper function aborts execution, if: - --dir option is specified, and - localrepository doesn't have "dirlog" attribute, which indicates that localrepository has a function for '--dir' BTW, extensions.wrapfunction() has been available since 1.1 (or 0ab5f21c390b), and this seems old enough for "historical portability" of perf.py, which has been available since 1.1 (or eb240755386d).

perf: define util.safehasattr forcibly for Mercurial earlier than 1.9.3 Before this patch, using util.safehasattr() prevents perf.py from being loaded by Mercurial earlier than 1.9.3 (or 94b200a11cf7), because util.safehasattr() isn't available in such Mercurial, even though there are some code paths for Mercurial earlier than 1.9.3. For example, setting "_prereadsize" attribute in perfindex() and perfnodelookup() is effective only with Mercurial earlier than 1.8 (or 61c9bc3da402). This patch is a preparation for using util.safehasattr() safely in subsequent patches. This patch defines util.safehasattr() forcibly without examining whether it is available or not, because: - examining existence of "safehasattr" safely itself needs similar logic - safehasattr() is small enough to define locally

perf: add historical portability policy for future reference

tests: check ETag format in test-hgweb-commands

hgweb: emit a valid, weak ETag Previously, ETag headers from hgweb weren't correctly formed, because rfc2616 (section 14, header definitions) requires double quotes around the content of the header. str(web.mtime) didn't do that. Additionally, strong ETags signify that the resource representations are byte-for-byte identical. That is, they can be reconstructed from byte ranges if client so wishes. Considering ETags for all hgweb pages is just mtime of 00changelog.i and doesn't consider of e.g. .hg/hgrc with description, contact and other fields, it's clearly shouldn't be strong. The W/ prefix marks it as weak, which still allows caching the whole served file/page, but doesn't allow byte-range requests.

policy: add cffi policy for PyPy This adds cffi policy in the case where we don't want to use C modules, but instead we're happy to rely on cffi (bundled with pypy)

sslutil: handle default CA certificate loading on Windows See the inline comment for what's going on here. There is magic built into the "ssl" module that ships with modern CPython that knows how to load the system CA certificates on Windows. Since we're not shipping a CA bundle with Mercurial, if we're running on legacy CPython there's nothing we can do to load CAs on Windows, so it makes sense to print a warning. I don't anticipate many people will see this warning because the official (presumed popular) Mercurial distributions on Windows bundle Python and should be distributing a modern Python capable of loading system CA certs.

sslutil: expand _defaultcacerts docstring to note calling assumptions We should document this so future message additions don't seem out of place.

sslutil: document the Apple OpenSSL cert trick This is sort of documented in _plainapplypython()'s docstring. But it helps to be explicit in security code.

sslutil: use certificates provided by certifi if available The "certifi" Python package provides a distribution of the Mozilla trusted CA certificates as a Python package. If it is present, we assume the user intends it to be used and we use it to provide the default CA certificates when certificates are otherwise not configured. It's worth noting that this behavior roughly matches the popular "requests" package, which also attempts to use "certifi" if present.

py3: make files use absolute_import and print_function This patch includes addition of absolute_import and print_function to the files where they are missing. The modern importing conventions are also followed.

sslutil: don't attempt to find default CA certs file when told not to Before, devel.disableloaddefaultcerts only impacted the loading of default certs via SSLContext. After this patch, the config option also prevents sslutil._defaultcacerts() from being called. This config option is meant to be used by tests to force no CA certs to be loaded. Future patches will enable _defaultcacerts() to have success more often. Without this change we can't reliably test the failure to load CA certs. (This patch also likely fixes test failures on some OS X configurations.)