Mercurial > hg
annotate hgext/logtoprocess.py @ 45068:8cd18aba5e6c
tests: proof test-stdio.py against buffer fill-up
With the previous code, it could in theory happen that the pipe / PTY buffer of
the child stdout / stderr fills up and the process never finishes.
To prevent that, we read all of the stream before waiting for the end of the
process. To ensure that the stream reaches EOF when the child finishes, we must
close the parent "copy" of the child stdout / stderr.
author | Manuel Jacob <me@manueljacob.de> |
---|---|
date | Tue, 07 Jul 2020 11:06:37 +0200 |
parents | 7c0b8652fd8c |
children | 6000f5b25c9b |
rev | line source |
---|---|
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
1 # logtoprocess.py - send ui.log() data to a subprocess |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
2 # |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
3 # Copyright 2016 Facebook, Inc. |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
4 # |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
5 # This software may be used and distributed according to the terms of the |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
6 # GNU General Public License version 2 or any later version. |
31601
8e7feaad2d8d
logtoprocess: use lowercase for docstring title
Jun Wu <quark@fb.com>
parents:
30638
diff
changeset
|
7 """send ui.log() data to a subprocess (EXPERIMENTAL) |
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
8 |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
9 This extension lets you specify a shell command per ui.log() event, |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
10 sending all remaining arguments to as environment variables to that command. |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
11 |
40622
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
12 Positional arguments construct a log message, which is passed in the `MSG1` |
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
13 environment variables. Each keyword argument is set as a `OPT_UPPERCASE_KEY` |
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
14 variable (so the key is uppercased, and prefixed with `OPT_`). The original |
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
15 event name is passed in the `EVENT` environment variable, and the process ID |
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
16 of mercurial is given in `HGPID`. |
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
17 |
40622
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
18 So given a call `ui.log('foo', 'bar %s\n', 'baz', spam='eggs'), a script |
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
19 configured for the `foo` event can expect an environment with `MSG1=bar baz`, |
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
20 and `OPT_SPAM=eggs`. |
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
21 |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
22 Scripts are configured in the `[logtoprocess]` section, each key an event name. |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
23 For example:: |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
24 |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
25 [logtoprocess] |
40622
d2c997b8001f
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents:
40620
diff
changeset
|
26 commandexception = echo "$MSG1" > /var/log/mercurial_exceptions.log |
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
27 |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
28 would log the warning message and traceback of any failed command dispatch. |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
29 |
30332
318a24b52eeb
spelling: fixes of non-dictionary words
Mads Kiilerich <madski@unity3d.com>
parents:
29841
diff
changeset
|
30 Scripts are run asynchronously as detached daemon processes; mercurial will |
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
31 not ensure that they exit cleanly. |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
32 |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
33 """ |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
34 |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
35 from __future__ import absolute_import |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
36 |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
37 import os |
30638
1c5cbf28f007
py3: replace os.environ with encoding.environ (part 5 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents:
30332
diff
changeset
|
38 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
40761
diff
changeset
|
39 from mercurial.utils import procutil |
39826
c31ce080eb75
py3: convert arguments, cwd and env to native strings when spawning subprocess
Matt Harbison <matt_harbison@yahoo.com>
parents:
35063
diff
changeset
|
40 |
29841
d5883fd055c6
extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents:
29463
diff
changeset
|
41 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for |
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
42 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
43 # be specifying the version(s) of Mercurial they are tested with, or |
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
44 # leave the attribute unspecified. |
43077
687b865b95ad
formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents:
43076
diff
changeset
|
45 testedwith = b'ships-with-hg-core' |
28901
a368da441b32
logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff
changeset
|
46 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
40761
diff
changeset
|
47 |
40677
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
48 class processlogger(object): |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
49 """Map log events to external commands |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
50 |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
51 Arguments are passed on as environment variables. |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
52 """ |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
53 |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
54 def __init__(self, ui): |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
55 self._scripts = dict(ui.configitems(b'logtoprocess')) |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
56 |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
57 def tracked(self, event): |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
58 return bool(self._scripts.get(event)) |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
59 |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
60 def log(self, ui, event, msg, opts): |
40730
55b053af7196
ui: manage logger instances and event filtering by core ui
Yuya Nishihara <yuya@tcha.org>
parents:
40677
diff
changeset
|
61 script = self._scripts[event] |
44417
7c0b8652fd8c
logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
43077
diff
changeset
|
62 maxmsg = 100000 |
7c0b8652fd8c
logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
43077
diff
changeset
|
63 if len(msg) > maxmsg: |
7c0b8652fd8c
logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
43077
diff
changeset
|
64 # Each env var has a 128KiB limit on linux. msg can be long, in |
7c0b8652fd8c
logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
43077
diff
changeset
|
65 # particular for command event, where it's the full command line. |
7c0b8652fd8c
logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
43077
diff
changeset
|
66 # Prefer truncating the message than raising "Argument list too |
7c0b8652fd8c
logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
43077
diff
changeset
|
67 # long" error. |
7c0b8652fd8c
logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents:
43077
diff
changeset
|
68 msg = msg[:maxmsg] + b' (truncated)' |
40677
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
69 env = { |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
70 b'EVENT': event, |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
71 b'HGPID': os.getpid(), |
40760
ffd574c144d2
ui: pass in formatted message to logger.log()
Yuya Nishihara <yuya@tcha.org>
parents:
40730
diff
changeset
|
72 b'MSG1': msg, |
40677
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
73 } |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
74 # keyword arguments get prefixed with OPT_ and uppercased |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
40761
diff
changeset
|
75 env.update( |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
40761
diff
changeset
|
76 (b'OPT_%s' % key.upper(), value) for key, value in opts.items() |
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
40761
diff
changeset
|
77 ) |
40677
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
78 fullenv = procutil.shellenviron(env) |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
79 procutil.runbgcommand(script, fullenv, shell=True) |
2b859742ea15
logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents:
40622
diff
changeset
|
80 |
43076
2372284d9457
formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents:
40761
diff
changeset
|
81 |
40730
55b053af7196
ui: manage logger instances and event filtering by core ui
Yuya Nishihara <yuya@tcha.org>
parents:
40677
diff
changeset
|
82 def uipopulate(ui): |
55b053af7196
ui: manage logger instances and event filtering by core ui
Yuya Nishihara <yuya@tcha.org>
parents:
40677
diff
changeset
|
83 ui.setlogger(b'logtoprocess', processlogger(ui)) |