hgext/logtoprocess.py
author timeless <timeless@mozdev.org>
Mon, 16 May 2016 21:30:53 +0000
changeset 29216 ead25aa27a43
parent 28901 a368da441b32
child 29463 4cf0542dcbe7
permissions -rw-r--r--
py3: convert to next() function next(..) was introduced in py2.6 and .next() is not available in py3 https://docs.python.org/2/library/functions.html#next
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     1
# logtoprocess.py - send ui.log() data to a subprocess
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     2
#
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     3
# Copyright 2016 Facebook, Inc.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     4
#
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     7
"""Send ui.log() data to a subprocess (EXPERIMENTAL)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     8
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     9
This extension lets you specify a shell command per ui.log() event,
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    10
sending all remaining arguments to as environment variables to that command.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    11
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    12
Each positional argument to the method results in a `MSG[N]` key in the
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    13
environment, starting at 1 (so `MSG1`, `MSG2`, etc.). Each keyword argument
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    14
is set as a `OPT_UPPERCASE_KEY` variable (so the key is uppercased, and
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    15
prefixed with `OPT_`). The original event name is passed in the `EVENT`
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    16
environment variable, and the process ID of mercurial is given in `HGPID`.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    17
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    18
So given a call `ui.log('foo', 'bar', 'baz', spam='eggs'), a script configured
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    19
for the `foo` event can expect an environment with `MSG1=bar`, `MSG2=baz`, and
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    20
`OPT_SPAM=eggs`.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    21
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    22
Scripts are configured in the `[logtoprocess]` section, each key an event name.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    23
For example::
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    24
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    25
  [logtoprocess]
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    26
  commandexception = echo "$MSG2$MSG3" > /var/log/mercurial_exceptions.log
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    27
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    28
would log the warning message and traceback of any failed command dispatch.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    29
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    30
Scripts are run asychronously as detached daemon processes; mercurial will
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    31
not ensure that they exit cleanly.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    32
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    33
"""
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    34
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    35
from __future__ import absolute_import
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    36
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    37
import itertools
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    38
import os
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    39
import platform
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    40
import subprocess
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    41
import sys
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    42
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    43
# Note for extension authors: ONLY specify testedwith = 'internal' for
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    44
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    45
# be specifying the version(s) of Mercurial they are tested with, or
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    46
# leave the attribute unspecified.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    47
testedwith = 'internal'
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    48
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    49
def uisetup(ui):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    50
    if platform.system() == 'Windows':
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    51
        # no fork on Windows, but we can create a detached process
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    52
        # https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863.aspx
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    53
        # No stdlib constant exists for this value
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    54
        DETACHED_PROCESS = 0x00000008
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    55
        _creationflags = DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    56
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    57
        def runshellcommand(script, env):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    58
            # we can't use close_fds *and* redirect stdin. I'm not sure that we
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    59
            # need to because the detached process has no console connection.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    60
            subprocess.Popen(
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    61
                script, shell=True, env=env, close_fds=True,
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    62
                creationflags=_creationflags)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    63
    else:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    64
        def runshellcommand(script, env):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    65
            # double-fork to completely detach from the parent process
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    66
            # based on http://code.activestate.com/recipes/278731
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    67
            pid = os.fork()
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    68
            if pid:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    69
                # parent
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    70
                return
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    71
            # subprocess.Popen() forks again, all we need to add is
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    72
            # flag the new process as a new session.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    73
            if sys.version_info < (3, 2):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    74
                newsession = {'preexec_fn': os.setsid}
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    75
            else:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    76
                newsession = {'start_new_session': True}
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    77
            try:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    78
                # connect stdin to devnull to make sure the subprocess can't
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    79
                # muck up that stream for mercurial.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    80
                subprocess.Popen(
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    81
                    script, shell=True, stdin=open(os.devnull, 'r'), env=env,
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    82
                    close_fds=True, **newsession)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    83
            finally:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    84
                # mission accomplished, this child needs to exit and not
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    85
                # continue the hg process here.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    86
                os._exit(0)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    87
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    88
    class logtoprocessui(ui.__class__):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    89
        def log(self, event, *msg, **opts):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    90
            """Map log events to external commands
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    91
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    92
            Arguments are passed on as environment variables.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    93
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    94
            """
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    95
            script = ui.config('logtoprocess', event)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    96
            if script:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    97
                if msg:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    98
                    # try to format the log message given the remaining
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    99
                    # arguments
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   100
                    try:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   101
                        # Python string formatting with % either uses a
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   102
                        # dictionary *or* tuple, but not both. If we have
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   103
                        # keyword options, assume we need a mapping.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   104
                        formatted = msg[0] % (opts or msg[1:])
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   105
                    except (TypeError, KeyError):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   106
                        # Failed to apply the arguments, ignore
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   107
                        formatted = msg[0]
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   108
                    messages = (formatted,) + msg[1:]
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   109
                else:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   110
                    messages = msg
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   111
                # positional arguments are listed as MSG[N] keys in the
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   112
                # environment
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   113
                msgpairs = (
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   114
                    ('MSG{0:d}'.format(i), str(m))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   115
                    for i, m in enumerate(messages, 1))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   116
                # keyword arguments get prefixed with OPT_ and uppercased
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   117
                optpairs = (
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   118
                    ('OPT_{0}'.format(key.upper()), str(value))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   119
                    for key, value in opts.iteritems())
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   120
                env = dict(itertools.chain(os.environ.items(),
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   121
                                           msgpairs, optpairs),
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   122
                           EVENT=event, HGPID=str(os.getpid()))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   123
                # Connect stdin to /dev/null to prevent child processes messing
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   124
                # with mercurial's stdin.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   125
                runshellcommand(script, env)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   126
            return super(logtoprocessui, self).log(event, *msg, **opts)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   127
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   128
    # Replace the class for this instance and all clones created from it:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   129
    ui.__class__ = logtoprocessui