annotate tests/test-worker.t @ 44116:b5aaa09be18c

py3: suppress unraisable exceptions in test-worker.t Python 3.8 calls sys.unraisablehook when an unraisable exception is encountered. The default behavior is to print a warning. test-worker.t was triggering this hook due to a race between a newly forked process exiting and that process's _os.register_at_fork handlers running. I was seeing the stdlib's random module in the stack re-seeding itself. Although there could be other after-fork handlers in the mix. This commit defines sys.unraisablehook to effectively no-op. This suppresses the warning and makes test output on Python 3.8 consistent with prior versions. test-worker.t now passes on Python 3.8. Differential Revision: https://phab.mercurial-scm.org/D7949
author Gregory Szorc <gregory.szorc@gmail.com>
date Sat, 18 Jan 2020 11:13:01 -0800
parents 40bf3d7ecc42
children 12491abf93bd
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
1 Test UI worker interaction
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
2
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
3 $ cat > t.py <<EOF
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
4 > from __future__ import absolute_import, print_function
44116
b5aaa09be18c py3: suppress unraisable exceptions in test-worker.t
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43402
diff changeset
5 > import sys
32168
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32167
diff changeset
6 > import time
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
7 > from mercurial import (
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
8 > error,
32376
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 32168
diff changeset
9 > registrar,
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
10 > ui as uimod,
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
11 > worker,
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
12 > )
44116
b5aaa09be18c py3: suppress unraisable exceptions in test-worker.t
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43402
diff changeset
13 > sys.unraisablehook = lambda x: None
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
14 > def abort(ui, args):
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
15 > if args[0] == 0:
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
16 > # by first worker for test stability
36220
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
17 > raise error.Abort(b'known exception')
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
18 > return runme(ui, [])
32043
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
19 > def exc(ui, args):
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
20 > if args[0] == 0:
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
21 > # by first worker for test stability
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
22 > raise Exception('unknown exception')
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
23 > return runme(ui, [])
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
24 > def runme(ui, args):
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
25 > for arg in args:
36220
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
26 > ui.status(b'run\n')
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
27 > yield 1, arg
32168
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32167
diff changeset
28 > time.sleep(0.1) # easier to trigger killworkers code path
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
29 > functable = {
36220
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
30 > b'abort': abort,
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
31 > b'exc': exc,
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
32 > b'runme': runme,
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
33 > }
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
34 > cmdtable = {}
32376
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 32168
diff changeset
35 > command = registrar.command(cmdtable)
36220
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
36 > @command(b'test', [], b'hg test [COST] [FUNC]')
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
37 > def t(ui, repo, cost=1.0, func=b'runme'):
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
38 > cost = float(cost)
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
39 > func = functable[func]
36220
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
40 > ui.status(b'start\n')
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
41 > runs = worker.worker(ui, cost, func, (ui,), range(8))
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
42 > for n, i in runs:
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
43 > pass
36220
4f0439981a8a py3: add b'' prefixes in test-worker.t
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33109
diff changeset
44 > ui.status(b'done\n')
31701
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
45 > EOF
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
46 $ abspath=`pwd`/t.py
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
47 $ hg init
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
48
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
49 Run tests with worker enable by forcing a heigh cost
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
50
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
51 $ hg --config "extensions.t=$abspath" test 100000.0
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
52 start
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
53 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
54 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
55 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
56 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
57 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
58 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
59 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
60 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
61 done
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
62
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
63 Run tests without worker by forcing a low cost
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
64
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
65 $ hg --config "extensions.t=$abspath" test 0.0000001
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
66 start
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
67 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
68 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
69 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
70 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
71 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
72 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
73 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
74 run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
75 done
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
76
32061
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
77 #if no-windows
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
78
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
79 Known exception should be caught, but printed if --traceback is enabled
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
80
32168
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32167
diff changeset
81 $ hg --config "extensions.t=$abspath" --config 'worker.numcpus=8' \
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32167
diff changeset
82 > test 100000.0 abort 2>&1
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
83 start
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
84 abort: known exception
32042
8f8ad0139b8b worker: propagate exit code to main process
Yuya Nishihara <yuya@tcha.org>
parents: 32041
diff changeset
85 [255]
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31701
diff changeset
86
32168
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32167
diff changeset
87 $ hg --config "extensions.t=$abspath" --config 'worker.numcpus=8' \
41535
bad59bbd9bec py3: add alternate output on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36220
diff changeset
88 > test 100000.0 abort --traceback 2>&1 | egrep '(SystemExit|Abort)'
bad59bbd9bec py3: add alternate output on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36220
diff changeset
89 raise error.Abort(b'known exception')
43402
40bf3d7ecc42 py3: add a __str__ method to Abort
Denis Laxalde <denis@laxalde.org>
parents: 41535
diff changeset
90 mercurial.error.Abort: known exception (py3 !)
41535
bad59bbd9bec py3: add alternate output on Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36220
diff changeset
91 Abort: known exception (no-py3 !)
32167
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32166
diff changeset
92 SystemExit: 255
32043
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
93
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
94 Traceback must be printed for unknown exceptions
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
95
32168
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32167
diff changeset
96 $ hg --config "extensions.t=$abspath" --config 'worker.numcpus=8' \
32167
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32166
diff changeset
97 > test 100000.0 exc 2>&1 | grep '^Exception'
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32166
diff changeset
98 Exception: unknown exception
32061
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
99
32166
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
100 Workers should not do cleanups in all cases
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
101
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
102 $ cat > $TESTTMP/detectcleanup.py <<EOF
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
103 > from __future__ import absolute_import
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
104 > import atexit
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
105 > import os
44116
b5aaa09be18c py3: suppress unraisable exceptions in test-worker.t
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43402
diff changeset
106 > import sys
32166
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
107 > import time
44116
b5aaa09be18c py3: suppress unraisable exceptions in test-worker.t
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43402
diff changeset
108 > sys.unraisablehook = lambda x: None
32166
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
109 > oldfork = os.fork
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
110 > count = 0
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
111 > parentpid = os.getpid()
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
112 > def delayedfork():
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
113 > global count
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
114 > count += 1
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
115 > pid = oldfork()
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
116 > # make it easier to test SIGTERM hitting other workers when they have
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
117 > # not set up error handling yet.
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
118 > if count > 1 and pid == 0:
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
119 > time.sleep(0.1)
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
120 > return pid
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
121 > os.fork = delayedfork
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
122 > def cleanup():
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
123 > if os.getpid() != parentpid:
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
124 > os.write(1, 'should never happen\n')
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
125 > atexit.register(cleanup)
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
126 > EOF
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
127
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
128 $ hg --config "extensions.t=$abspath" --config worker.numcpus=8 --config \
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
129 > "extensions.d=$TESTTMP/detectcleanup.py" test 100000 abort
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
130 start
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
131 abort: known exception
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
132 [255]
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
133
32061
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
134 #endif