view contrib/packaging/packaging.py @ 45095:8e04607023e5

procutil: ensure that procutil.std{out,err}.write() writes all bytes

Python 3 offers different kind of streams and it’s not guaranteed for all of
them that calling write() writes all bytes.

When Python is started in unbuffered mode, sys.std{out,err}.buffer are
instances of io.FileIO, whose write() can write less bytes for
platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could
write less if interrupted by a signal; when writing to Windows consoles, it’s
limited to 32767 bytes to avoid the "not enough space" error). This can lead to
silent loss of data, both when using sys.std{out,err}.buffer (which may in fact
not be a buffered stream) and when using the text streams sys.std{out,err}
(I’ve created a CPython bug report for that:
https://bugs.python.org/issue41221).

Python may fix the problem at some point. For now, we implement our own wrapper
for procutil.std{out,err} that calls the raw stream’s write() method until all
bytes have been written. We don’t use sys.std{out,err} for larger writes, so I
think it’s not worth the effort to patch them.

#!/usr/bin/env python3
#
# packaging.py - Mercurial packaging functionality
#
# Copyright 2019 Gregory Szorc <[e-mail]>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.

import os
import pathlib
import subprocess
import sys
import venv


HERE = pathlib.Path(os.path.abspath(__file__)).parent
REQUIREMENTS_TXT = HERE / "requirements.txt"
SOURCE_DIR = HERE.parent.parent
VENV = SOURCE_DIR / "build" / "venv-packaging"


def bootstrap():
    venv_created = not VENV.exists()

    VENV.parent.mkdir(exist_ok=True)

    venv.create(VENV, with_pip=True)

    if os.name == "nt":
        venv_bin = VENV / "Scripts"
        pip = venv_bin / "pip.exe"
        python = venv_bin / "python.exe"
    else:
        venv_bin = VENV / "bin"
        pip = venv_bin / "pip"
        python = venv_bin / "python"

    args = [
        str(pip),
        "install",
        "-r",
        str(REQUIREMENTS_TXT),
        "--disable-pip-version-check",
    ]

    if not venv_created:
        args.append("-q")

    subprocess.run(args, check=True)

    os.environ["HGPACKAGING_BOOTSTRAPPED"] = "1"
    os.environ["PATH"] = "%s%s%s" % (venv_bin, os.pathsep, os.environ["PATH"])

    subprocess.run([str(python), __file__] + sys.argv[1:], check=True)


def run():
    import hgpackaging.cli as cli

    # Need to strip off main Python executable.
    cli.main()


if __name__ == "__main__":
    try:
        if "HGPACKAGING_BOOTSTRAPPED" not in os.environ:
            bootstrap()
        else:
            run()
    except subprocess.CalledProcessError as e:
        sys.exit(e.returncode)
    except KeyboardInterrupt:
        sys.exit(1)