hgext/logtoprocess.py
author Martin von Zweigbergk <martinvonz@google.com>
Sun, 14 Jan 2018 00:02:40 -0800
changeset 35710 5cd60b0587a8
parent 35086 52790352dd05
child 39831 c31ce080eb75
permissions -rw-r--r--
evolution: make reporting of new unstable changesets optional This makes it possible to turn off the reporting of unstable changesets (e.g. "1 new orphan changesets"), just in case it's too slow for some users. Anyone who's been using the evolve extension has already been accepting the cost of the reporting, so this is just for the few users who have turned on obsmarkers but not been using the evolve extension (as I believe Facebook has). Differential Revision: https://phab.mercurial-scm.org/D1868
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     1
# logtoprocess.py - send ui.log() data to a subprocess
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     2
#
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     3
# Copyright 2016 Facebook, Inc.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     4
#
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
31606
8e7feaad2d8d logtoprocess: use lowercase for docstring title
Jun Wu <quark@fb.com>
parents: 30643
diff changeset
     7
"""send ui.log() data to a subprocess (EXPERIMENTAL)
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     8
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     9
This extension lets you specify a shell command per ui.log() event,
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    10
sending all remaining arguments to as environment variables to that command.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    11
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    12
Each positional argument to the method results in a `MSG[N]` key in the
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    13
environment, starting at 1 (so `MSG1`, `MSG2`, etc.). Each keyword argument
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    14
is set as a `OPT_UPPERCASE_KEY` variable (so the key is uppercased, and
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    15
prefixed with `OPT_`). The original event name is passed in the `EVENT`
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    16
environment variable, and the process ID of mercurial is given in `HGPID`.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    17
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    18
So given a call `ui.log('foo', 'bar', 'baz', spam='eggs'), a script configured
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    19
for the `foo` event can expect an environment with `MSG1=bar`, `MSG2=baz`, and
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    20
`OPT_SPAM=eggs`.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    21
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    22
Scripts are configured in the `[logtoprocess]` section, each key an event name.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    23
For example::
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    24
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    25
  [logtoprocess]
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    26
  commandexception = echo "$MSG2$MSG3" > /var/log/mercurial_exceptions.log
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    27
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    28
would log the warning message and traceback of any failed command dispatch.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    29
30342
318a24b52eeb spelling: fixes of non-dictionary words
Mads Kiilerich <madski@unity3d.com>
parents: 29852
diff changeset
    30
Scripts are run asynchronously as detached daemon processes; mercurial will
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    31
not ensure that they exit cleanly.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    32
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    33
"""
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    34
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    35
from __future__ import absolute_import
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    36
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    37
import itertools
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    38
import os
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    39
import subprocess
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    40
import sys
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    41
34640
68ed3b4f86ef logtoprocess: do not use platform.system()
Jun Wu <quark@fb.com>
parents: 31606
diff changeset
    42
from mercurial import (
68ed3b4f86ef logtoprocess: do not use platform.system()
Jun Wu <quark@fb.com>
parents: 31606
diff changeset
    43
    encoding,
68ed3b4f86ef logtoprocess: do not use platform.system()
Jun Wu <quark@fb.com>
parents: 31606
diff changeset
    44
    pycompat,
68ed3b4f86ef logtoprocess: do not use platform.system()
Jun Wu <quark@fb.com>
parents: 31606
diff changeset
    45
)
30643
1c5cbf28f007 py3: replace os.environ with encoding.environ (part 5 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30342
diff changeset
    46
29852
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29463
diff changeset
    47
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    48
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    49
# be specifying the version(s) of Mercurial they are tested with, or
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    50
# leave the attribute unspecified.
29852
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29463
diff changeset
    51
testedwith = 'ships-with-hg-core'
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    52
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    53
def uisetup(ui):
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34640
diff changeset
    54
    if pycompat.iswindows:
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    55
        # no fork on Windows, but we can create a detached process
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    56
        # https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863.aspx
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    57
        # No stdlib constant exists for this value
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    58
        DETACHED_PROCESS = 0x00000008
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    59
        _creationflags = DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    60
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    61
        def runshellcommand(script, env):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    62
            # we can't use close_fds *and* redirect stdin. I'm not sure that we
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    63
            # need to because the detached process has no console connection.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    64
            subprocess.Popen(
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    65
                script, shell=True, env=env, close_fds=True,
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    66
                creationflags=_creationflags)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    67
    else:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    68
        def runshellcommand(script, env):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    69
            # double-fork to completely detach from the parent process
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    70
            # based on http://code.activestate.com/recipes/278731
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    71
            pid = os.fork()
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    72
            if pid:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    73
                # parent
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    74
                return
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    75
            # subprocess.Popen() forks again, all we need to add is
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    76
            # flag the new process as a new session.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    77
            if sys.version_info < (3, 2):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    78
                newsession = {'preexec_fn': os.setsid}
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    79
            else:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    80
                newsession = {'start_new_session': True}
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    81
            try:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    82
                # connect stdin to devnull to make sure the subprocess can't
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    83
                # muck up that stream for mercurial.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    84
                subprocess.Popen(
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    85
                    script, shell=True, stdin=open(os.devnull, 'r'), env=env,
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    86
                    close_fds=True, **newsession)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    87
            finally:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    88
                # mission accomplished, this child needs to exit and not
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    89
                # continue the hg process here.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    90
                os._exit(0)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    91
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    92
    class logtoprocessui(ui.__class__):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    93
        def log(self, event, *msg, **opts):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    94
            """Map log events to external commands
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    95
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    96
            Arguments are passed on as environment variables.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    97
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    98
            """
29463
4cf0542dcbe7 logtoprocess: do not leak the ui object in uisetup
Jun Wu <quark@fb.com>
parents: 28901
diff changeset
    99
            script = self.config('logtoprocess', event)
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   100
            if script:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   101
                if msg:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   102
                    # try to format the log message given the remaining
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   103
                    # arguments
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   104
                    try:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   105
                        # Python string formatting with % either uses a
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   106
                        # dictionary *or* tuple, but not both. If we have
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   107
                        # keyword options, assume we need a mapping.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   108
                        formatted = msg[0] % (opts or msg[1:])
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   109
                    except (TypeError, KeyError):
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   110
                        # Failed to apply the arguments, ignore
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   111
                        formatted = msg[0]
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   112
                    messages = (formatted,) + msg[1:]
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   113
                else:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   114
                    messages = msg
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   115
                # positional arguments are listed as MSG[N] keys in the
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   116
                # environment
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   117
                msgpairs = (
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   118
                    ('MSG{0:d}'.format(i), str(m))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   119
                    for i, m in enumerate(messages, 1))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   120
                # keyword arguments get prefixed with OPT_ and uppercased
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   121
                optpairs = (
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   122
                    ('OPT_{0}'.format(key.upper()), str(value))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   123
                    for key, value in opts.iteritems())
30643
1c5cbf28f007 py3: replace os.environ with encoding.environ (part 5 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30342
diff changeset
   124
                env = dict(itertools.chain(encoding.environ.items(),
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   125
                                           msgpairs, optpairs),
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   126
                           EVENT=event, HGPID=str(os.getpid()))
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   127
                runshellcommand(script, env)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   128
            return super(logtoprocessui, self).log(event, *msg, **opts)
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   129
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   130
    # Replace the class for this instance and all clones created from it:
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
   131
    ui.__class__ = logtoprocessui