comparison mercurial/chgserver.py @ 44261:04a3ae7aba14

chg: force-set LC_CTYPE on server start to actual value from the environment

Python 3.7+ will "coerce" the LC_CTYPE variable in many instances, and this can
cause issues with chg being able to start up. D7550 attempted to fix this, but a
combination of a misreading of the way that python3.7 does the coercion and an
untested state (LC_CTYPE being set to an invalid value) meant that this was
still not quite working.

This change will cause differences between chg and hg: hg will have the LC_CTYPE
environment variable coerced, while chg will not. This is unlikely to cause any
detectable behavior differences in what Mercurial itself outputs, but it does
have two known effects:

-   When using hg, the coerced LC_CTYPE will be passed to subprocesses, even
    non-python ones. Using chg will remove the coercion, and this will not
    happen. This is arguably more correct behavior on chg's part.
-   On macOS, if you set your region to Brazil but your language to English,
    this isn't representable in locale strings, so macOS sets LC_CTYPE=UTF-8. If
    this value is passed along when ssh'ing to a non-macOS machine, some
    functions (such as locale.setlocale()) may raise an exception due to an
    unsupported locale setting. This is most easily encountered when doing an
    interactive commit/split/etc. when using ui.interface=curses.

Differential Revision: https://phab.mercurial-scm.org/D8039

44260:216fc4633800

            newenv = dict(s.split(b'=', 1) for s in l)
        except ValueError:
            raise ValueError(b'unexpected value in setenv request')
        self.ui.log(b'chgserver', b'setenv: %r\n', sorted(newenv.keys()))

        # Python3 has some logic to "coerce" the C locale to a UTF-8 capable
        # one, and it sets LC_CTYPE in the environment to C.UTF-8 if none of
        # 'LC_CTYPE', 'LC_ALL' or 'LANG' are set (to any value). This can be
        # disabled with PYTHONCOERCECLOCALE=0 in the environment.
        #
        # When fromui is called via _inithashstate, python has already set
        # this, so that's in the environment right when we start up the hg
        # process. Then chg will call us and tell us to set the environment to
        # the one it has; this might NOT have LC_CTYPE, so we'll need to
        # carry-forward the LC_CTYPE that was coerced in these situations.
        #
        # If this is not handled, we will fail config+env validation and fail
        # to start chg. If this is just ignored instead of carried forward, we
        # may have different behavior between chg and non-chg.
        if pycompat.ispy3:
            # Rename for wordwrapping purposes
            oldenv = encoding.environ
            if not any(
                e.get(b'PYTHONCOERCECLOCALE') == b'0' for e in [oldenv, newenv]
            ):
                keys = [b'LC_CTYPE', b'LC_ALL', b'LANG']
                old_keys = [k for k, v in oldenv.items() if k in keys and v]
                new_keys = [k for k, v in newenv.items() if k in keys and v]
                # If the user's environment (from chg) doesn't have ANY of the
                # keys that python looks for, and the environment (from
                # initialization) has ONLY LC_CTYPE and it's set to C.UTF-8,
                # carry it forward.
                if (
                    not new_keys
                    and old_keys == [b'LC_CTYPE']
                    and oldenv[b'LC_CTYPE'] == b'C.UTF-8'
                ):
                    newenv[b'LC_CTYPE'] = oldenv[b'LC_CTYPE']

        encoding.environ.clear()
        encoding.environ.update(newenv)

    capabilities = commandserver.server.capabilities.copy()
    capabilities.update(

    # demandimport or detecting chg client started by chg client. When executed
    # here, CHGINTERNALMARK is no longer useful and hence dropped to make
    # environ cleaner.
    if b'CHGINTERNALMARK' in encoding.environ:
        del encoding.environ[b'CHGINTERNALMARK']

    if repo:
        # one chgserver can serve multiple repos. drop repo information
        ui.setconfig(b'bundle', b'mainreporoot', b'', b'repo')
    h = chgunixservicehandler(ui)

44261:04a3ae7aba14

            newenv = dict(s.split(b'=', 1) for s in l)
        except ValueError:
            raise ValueError(b'unexpected value in setenv request')
        self.ui.log(b'chgserver', b'setenv: %r\n', sorted(newenv.keys()))

        encoding.environ.clear()
        encoding.environ.update(newenv)

    capabilities = commandserver.server.capabilities.copy()
    capabilities.update(

    # demandimport or detecting chg client started by chg client. When executed
    # here, CHGINTERNALMARK is no longer useful and hence dropped to make
    # environ cleaner.
    if b'CHGINTERNALMARK' in encoding.environ:
        del encoding.environ[b'CHGINTERNALMARK']
    # Python3.7+ "coerces" the LC_CTYPE environment variable to a UTF-8 one if
    # it thinks the current value is "C". This breaks the hash computation and
    # causes chg to restart loop.
    if b'CHGORIG_LC_CTYPE' in encoding.environ:
        encoding.environ[b'LC_CTYPE'] = encoding.environ[b'CHGORIG_LC_CTYPE']
        del encoding.environ[b'CHGORIG_LC_CTYPE']
    elif b'CHG_CLEAR_LC_CTYPE' in encoding.environ:
        if b'LC_CTYPE' in encoding.environ:
            del encoding.environ[b'LC_CTYPE']
        del encoding.environ[b'CHG_CLEAR_LC_CTYPE']

    if repo:
        # one chgserver can serve multiple repos. drop repo information
        ui.setconfig(b'bundle', b'mainreporoot', b'', b'repo')
    h = chgunixservicehandler(ui)