tests/test-worker.t
author Martin von Zweigbergk <martinvonz@google.com>
Fri, 25 Aug 2017 15:50:07 -0700
changeset 33938 9180f8f593f3
parent 33097 fce4ed2912bb
child 36182 4f0439981a8a
permissions -rw-r--r--
revlog: abort on attempt to write null revision My repo got corrupted yesterday by something that ended up writing the null revision to the revlog (nullid hash, not nullrev index, of course). We use many extensions internally (narrowhg, remotefilelog, evolve, internal extensions) and treemanifests are on. The null revision was written to the changelog, the root manifest log, and one subdirectory manifest log. I have no idea exactly why the null revision was written, but it seems cheap enough to check that we should fail instead of corrupting the repo. Differential Revision: https://phab.mercurial-scm.org/D522
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
     1
Test UI worker interaction
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
     2
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
     3
  $ cat > t.py <<EOF
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
     4
  > from __future__ import absolute_import, print_function
32114
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32113
diff changeset
     5
  > import time
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
     6
  > from mercurial import (
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
     7
  >     error,
32337
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 32114
diff changeset
     8
  >     registrar,
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
     9
  >     ui as uimod,
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    10
  >     worker,
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    11
  > )
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    12
  > def abort(ui, args):
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    13
  >     if args[0] == 0:
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    14
  >         # by first worker for test stability
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    15
  >         raise error.Abort('known exception')
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    16
  >     return runme(ui, [])
32043
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    17
  > def exc(ui, args):
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    18
  >     if args[0] == 0:
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    19
  >         # by first worker for test stability
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    20
  >         raise Exception('unknown exception')
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    21
  >     return runme(ui, [])
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    22
  > def runme(ui, args):
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    23
  >     for arg in args:
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    24
  >         ui.status('run\n')
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    25
  >         yield 1, arg
32114
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32113
diff changeset
    26
  >     time.sleep(0.1) # easier to trigger killworkers code path
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    27
  > functable = {
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    28
  >     'abort': abort,
32043
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    29
  >     'exc': exc,
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    30
  >     'runme': runme,
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    31
  > }
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    32
  > cmdtable = {}
32337
46ba2cdda476 registrar: move cmdutil.command to registrar module (API)
Yuya Nishihara <yuya@tcha.org>
parents: 32114
diff changeset
    33
  > command = registrar.command(cmdtable)
33097
fce4ed2912bb py3: make sure commands name are bytes in tests
Pulkit Goyal <7895pulkit@gmail.com>
parents: 32337
diff changeset
    34
  > @command(b'test', [], 'hg test [COST] [FUNC]')
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    35
  > def t(ui, repo, cost=1.0, func='runme'):
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    36
  >     cost = float(cost)
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    37
  >     func = functable[func]
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    38
  >     ui.status('start\n')
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    39
  >     runs = worker.worker(ui, cost, func, (ui,), range(8))
31696
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    40
  >     for n, i in runs:
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    41
  >         pass
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    42
  >     ui.status('done\n')
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    43
  > EOF
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    44
  $ abspath=`pwd`/t.py
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    45
  $ hg init
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    46
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    47
Run tests with worker enable by forcing a heigh cost
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    48
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    49
  $ hg --config "extensions.t=$abspath" test 100000.0
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    50
  start
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    51
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    52
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    53
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    54
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    55
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    56
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    57
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    58
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    59
  done
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    60
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    61
Run tests without worker by forcing a low cost
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    62
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    63
  $ hg --config "extensions.t=$abspath" test 0.0000001
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    64
  start
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    65
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    66
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    67
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    68
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    69
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    70
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    71
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    72
  run
9d3d56aa1a9f worker: flush ui buffers before running the worker
David Soria Parra <davidsp@fb.com>
parents:
diff changeset
    73
  done
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    74
32061
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
    75
#if no-windows
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
    76
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    77
Known exception should be caught, but printed if --traceback is enabled
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    78
32114
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32113
diff changeset
    79
  $ hg --config "extensions.t=$abspath" --config 'worker.numcpus=8' \
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32113
diff changeset
    80
  > test 100000.0 abort 2>&1
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    81
  start
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    82
  abort: known exception
32042
8f8ad0139b8b worker: propagate exit code to main process
Yuya Nishihara <yuya@tcha.org>
parents: 32041
diff changeset
    83
  [255]
32041
38963a53ab0d dispatch: print traceback in scmutil.callcatch() if --traceback specified
Yuya Nishihara <yuya@tcha.org>
parents: 31696
diff changeset
    84
32114
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32113
diff changeset
    85
  $ hg --config "extensions.t=$abspath" --config 'worker.numcpus=8' \
32113
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32112
diff changeset
    86
  > test 100000.0 abort --traceback 2>&1 | egrep '^(SystemExit|Abort)'
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32112
diff changeset
    87
  Abort: known exception
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32112
diff changeset
    88
  SystemExit: 255
32043
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    89
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    90
Traceback must be printed for unknown exceptions
b844d0d367e2 worker: print traceback for uncaught exception unconditionally
Yuya Nishihara <yuya@tcha.org>
parents: 32042
diff changeset
    91
32114
44a98a2ea431 test-worker: exercise more about "killworkers" situation
Jun Wu <quark@fb.com>
parents: 32113
diff changeset
    92
  $ hg --config "extensions.t=$abspath" --config 'worker.numcpus=8' \
32113
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32112
diff changeset
    93
  > test 100000.0 exc 2>&1 | grep '^Exception'
9f0c055eebae test-worker: capture tracebacks more reliably
Jun Wu <quark@fb.com>
parents: 32112
diff changeset
    94
  Exception: unknown exception
32061
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
    95
32112
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
    96
Workers should not do cleanups in all cases
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
    97
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
    98
  $ cat > $TESTTMP/detectcleanup.py <<EOF
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
    99
  > from __future__ import absolute_import
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   100
  > import atexit
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   101
  > import os
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   102
  > import time
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   103
  > oldfork = os.fork
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   104
  > count = 0
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   105
  > parentpid = os.getpid()
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   106
  > def delayedfork():
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   107
  >     global count
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   108
  >     count += 1
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   109
  >     pid = oldfork()
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   110
  >     # make it easier to test SIGTERM hitting other workers when they have
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   111
  >     # not set up error handling yet.
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   112
  >     if count > 1 and pid == 0:
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   113
  >         time.sleep(0.1)
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   114
  >     return pid
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   115
  > os.fork = delayedfork
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   116
  > def cleanup():
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   117
  >     if os.getpid() != parentpid:
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   118
  >         os.write(1, 'should never happen\n')
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   119
  > atexit.register(cleanup)
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   120
  > EOF
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   121
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   122
  $ hg --config "extensions.t=$abspath" --config worker.numcpus=8 --config \
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   123
  > "extensions.d=$TESTTMP/detectcleanup.py" test 100000 abort
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   124
  start
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   125
  abort: known exception
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   126
  [255]
31763785094b worker: rewrite error handling so os._exit covers all cases
Jun Wu <quark@fb.com>
parents: 32061
diff changeset
   127
32061
6e0368b6e0bb test-worker: disable tests of forked workers on Windows
Yuya Nishihara <yuya@tcha.org>
parents: 32043
diff changeset
   128
#endif