view doc/docchecker @ 52156:8766d47edfd1 stable

ci: add a runner for Windows 10

This is currently only manually invoked, and allows for failure because we only
have a single runner that takes over 2h for a full run, and there are a handful
of flakey tests, plus 3 known failing tests.

The system being used here is running MSYS, Python, Visual Studio, etc, as
installed by `install-windows-dependencies.ps1`.  This script installs
everything to a specific directory instead of using the defaults, so we adjust
the MinGW shell path to compensate.  Additionally, the script doesn't install
the launcher `py.exe`.  It is possible to adjust the script to install it, but
it's an option to an existing python install (instead of a standalone installer),
and I've had the whole python install fail and rollback when requested to install
the launcher if it detects a newer one is already installed.  In short, it is
a point of failure for a feature we don't (yet?) need.

Unlike other systems where the intepreter name includes the version, everything
here is `python.exe`, so they can't all exist on `PATH` and let the script
choose the desired one.  (The `py.exe` launcher would accomplish, using the
registry instead of `PATH`, but that wouldn't allow for venv installs.)  Because
of this, switch to the absolute path of the python interpreter to be used (in
this case a venv created from the py39 install, which is old, but what both
pyoxidizer and TortoiseHg currently use).

The `RUNTEST_ARGS` hardcodes `-j8` because this system has 4 cores, and
therefore runs 4 parallel tests by default.  However on Windows, using more
parallel tests than cores results in better performance for whatever reason.  I
don't have an optimal value yet (ideally the runner itself can make the
adjustment on Windows), but this results in saving ~15m on a full run that
otherwise takes ~2.5h.  I'm also not concerned about how it would affect other
Windows machines, because we don't have any at this point, and I have no idea
when we can get more.

As far as system setup goes, the CI is run by a dedicated user that lacks admin
rights.  The install script was run by an admin user, and then the standard user
was configured to use it.  If I set this up again, I'd probably give the
dedicated user admin rights to run the install script, and reset to standard
user rights when done.  The python intepreter failed in weird ways when run by
the standard user until it was manually reinstalled by the standard user:

    Fatal Python error: init_fs_encoding: failed to get the Python codec of the
        filesystem encoding

Additionally, changing the environment through the Windows UI prompts to
escalate to an admin user, and then setting the user level environment variables
like `TEMP` and `PATH` (to try to avoid exceeding the 260 character path limit)
didn't actually change the user's environment.  (Likely it changed the admin
user's environment, but I didn't confirm that.)  I ended up having to use the
registry editor for the standard user to make those changes.

#!/usr/bin/env python3
#
# docchecker - look for problematic markup
#
# Copyright 2016 timeless <[e-mail]> and others
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.


import os
import re
import sys

try:
    import msvcrt

    msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
    msvcrt.setmode(sys.stderr.fileno(), os.O_BINARY)
except ImportError:
    pass

stdout = getattr(sys.stdout, 'buffer', sys.stdout)

leadingline = re.compile(br'(^\s*)(\S.*)$')

checks = [
    (
        br""":hg:`[^`]*'[^`]*`""",
        b"""warning: please avoid nesting ' in :hg:`...`""",
    ),
    (br'\w:hg:`', b'warning: please have a space before :hg:'),
    (
        br"""(?:[^a-z][^'.])hg ([^,;"`]*'(?!hg)){2}""",
        b'''warning: please use " instead of ' for hg ... "..."''',
    ),
]


def check(line):
    messages = []
    for match, msg in checks:
        if re.search(match, line):
            messages.append(msg)
    if messages:
        stdout.write(b'%s\n' % line)
        for msg in messages:
            stdout.write(b'%s\n' % msg)


def work(file):
    (llead, lline) = (b'', b'')

    for line in file:
        # this section unwraps lines
        match = leadingline.match(line)
        if not match:
            check(lline)
            (llead, lline) = (b'', b'')
            continue

        lead, line = match.group(1), match.group(2)
        if lead == llead:
            if lline != b'':
                lline += b' ' + line
            else:
                lline = line
        else:
            check(lline)
            (llead, lline) = (lead, line)
    check(lline)


def main():
    for f in sys.argv[1:]:
        try:
            with open(f, 'rb') as file:
                work(file)
        except BaseException as e:
            sys.stdout.write(r"failed to process %s: %s\n" % (f, e))


main()