Mercurial: Changelog

pyoxidizer: use Python 3.9 (BC) Newer versions of PyOxidizer use Python 3.9 by default. We previously pinned the version to 3.8 to facilitate porting to a new PyOxidizer version and diffing results. Now that the porting work is complete, let's bump Python to Python 3.9. This will effectively change our Windows Inno and WiX Python 3 installers from Python 3.8 to 3.9. .. bc:: Windows .msi and .exe installers now use Python 3.9 instead of Python 3.8. Differential Revision: https://phab.mercurial-scm.org/D10689

packaging: use PyOxidizer for producing WiX MSI installer We recently taught our in-tree PyOxidizer configuration file to produce MSI installers with WiX using PyOxidizer's built-in support for doing so. This commit changes our WiX + PyOxidizer installer generation code to use this functionality. After this change, all the Python packaging code is doing is the following: * Building HTML documentation * Making gettext available to the build process. * Munging CLI arguments to variables for the `pyoxidizer` execution. * Invoking `pyoxidizer build`. * Copying the produced `.msi` to the `dist/` directory. Applying this stack on stable and rebuilding the 5.8 MSI installer produced the following differences from the official 5.8 installer: * .exe and .pyd files aren't byte identical (this is expected). * Various .dist-info/ directories have different names due to older versions of PyOxidizer being buggy and not properly normalizing package names. (The new behavior is correct.) * Various *.dist-info/RECORD files are different due to content divergence of files (this is expected). * The python38.dll differs due to newer PyOxidizer shipping a newer version of Python 3.8. * We now ship python3.dll because PyOxidizer now includes this file by default. * The vcruntime140.dll differs because newer PyOxidizer installs a newer version. We also now ship a vcruntime140_1.dll because newer versions of the redistributable ship 2 files now. The WiX GUIDs and IDs of installed files have likely changed as a result of PyOxidizer's different mechanism for generating those identifiers. This means that an upgrade install of the MSI will replace files instead of doing an incremental update. This is likely harmless and we've incurred this kind of breakage before. As far as I can tell, the new PyOxidizer-built MSI is functionally equivalent to the old method. Once we drop support for Python 2.7 MSI installers, we can delete the WiX code from the repository. This commit temporarily drops support for extra `.wxs` files. We raise an exception instead of silently not using them, which I think is appropriate. We should be able to add support back in by injecting state into pyoxidizer.bzl via `--var`. I just didn't want to expend cognitive load to think about the solution as part of this series. Differential Revision: https://phab.mercurial-scm.org/D10688

packaging: extract invocation of pyoxidizer to own function I'll be refactoring how the WiX installer creation calls into pyoxidizer and will need a lower level function for facilitating that. The new `run_pyoxidizer()` builds our execution environment (with gettext available) and invokes `pyoxidizer`. Differential Revision: https://phab.mercurial-scm.org/D10687

packaging: rename run_pyoxidizer() I'm going to split this function up in a future commit and I'll want the name "run_pyoxidizer()" for a lower-level function for invoking `pyoxidizer`. Differential Revision: https://phab.mercurial-scm.org/D10686

packaging: move documentation HTML building to own function This is part of some light refactoring to enable us to use PyOxidizer for WiX MSI installer generation. Differential Revision: https://phab.mercurial-scm.org/D10685

pyoxidizer: support code signing Newer versions of PyOxidizer feature built-in support for code signing. You simply declare a code signer in the Starlark configuration file, activate it for automatic signing, and PyOxidizer will add code signatures to signable files as it encounters them. This commit teaches our Starlark configuration file to enable automatic code signing. But only on Windows for the moment, as our immediate goal is to overhaul the Windows packaging. The feature is opt-in: you must pass variables to PyOxidizer's build context via `pyoxidizer build --var` or `pyoxidizer build --var-env` to activate code signing. Differential Revision: https://phab.mercurial-scm.org/D10684

pyoxidizer: support producing MSI installers Newer versions of PyOxidizer have support for building WiX MSI installers "natively." Essentially, you can script the definition of your WiX installer via Starlark and PyOxidizer can invoke WiX tools to produce the installer. This commit teaches our PyOxidizer config file to produce MSI installers similarly to how `contrib/packaging/packging.py wix` would do it. We had to make a very minor change to `mercurial.wxs` to reflect different paths depending on who builds. This is because when PyOxidizer builds WiX installers, it does so from an isolated directory, not Mercurial's source directory. We simply copy the files into the build environment so they are accessible. After this change, running `pyoxidizer build msi` produces a nearly identical install layout to what the previous method produces. When I applied this series on top of the 5.8 tag, here is the list of differences and explanations: * docs/*.html files are missing from the new installer because the Python build environment doesn't have docutils. * .pyd and .exe files differ, likely because I'm using a different Visual Studio toolchain on my local computer than the official build environment. * Various .dist-info/ directories have different names. This is because older versions of PyOxidizer had buggy behavior and weren't properly normalizing package names in .dist-info/ directories. e.g. we went from `cached-property-1.5.2.dist-info` to `cached_property-1.5.2.dist-info`. * Translations (.mo files) may be missing if gettext isn't in %Path%. This is because the packaging.py code installs gettext and ensures it can be found. * Some *.dist-info/RECORD files vary due to SHA-256 content digest divergence due to build environment differences. (This should be harmless.) * The new install layout ships a python3.dll because newer versions of PyOxidizer ship this file. * The new install layout has a different vcruntime140.dll and also a vcruntime140_1.dll because newer versions of PyOxidizer ship a newer version of the Visual C++ Redistributable Runtime. The new PyOxidizer functionality is not yet integrated with packaging.py. This will come in a subsequent commit. So for now, the new functionality introduced here is unused. Differential Revision: https://phab.mercurial-scm.org/D10683