annotate tests/run-tests.py @ 33343:305935be8d5c

test-status-rev: conditionalize output instead of tests
author Matt Harbison <matt_harbison@yahoo.com>
date Sat, 08 Jul 2017 18:46:43 -0400
parents cf826b9e9ea4
children e80041832eec
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
1 #!/usr/bin/env python
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
2 #
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
3 # run-tests.py - Run a set of tests on Mercurial
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
4 #
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
5 # Copyright 2006 Matt Mackall <mpm@selenic.com>
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
6 #
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8161
diff changeset
7 # This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10154
diff changeset
8 # GNU General Public License version 2 or any later version.
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
9
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
10 # Modifying this script is tricky because it has many modes:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
11 # - serial (default) vs parallel (-jN, N > 1)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
12 # - no coverage (default) vs coverage (-c, -C, -s)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
13 # - temp install (default) vs specific hg script (--with-hg, --local)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
14 # - tests are a mix of shell scripts and Python scripts
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
15 #
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
16 # If you change this script, it is recommended that you ensure you
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
17 # haven't broken it by running it in various modes with a representative
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
18 # sample of test scripts. For example:
8843
eb7b247a98ea kill trailing whitespace
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 8724
diff changeset
19 #
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
20 # 1) serial, no coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
21 # ./run-tests.py test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
22 # 2) serial, no coverage, local hg:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
23 # ./run-tests.py --local test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
24 # 3) serial, coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
25 # ./run-tests.py -c test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
26 # 4) serial, coverage, local hg:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
27 # ./run-tests.py -c --local test-s* # unsupported
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
28 # 5) parallel, no coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
29 # ./run-tests.py -j2 test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
30 # 6) parallel, no coverage, local hg:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
31 # ./run-tests.py -j2 --local test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
32 # 7) parallel, coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
33 # ./run-tests.py -j2 -c test-s* # currently broken
9899
be574a37a8ae run-tests: give each child its own tmp dir (issue1911)
Greg Ward <greg@gerg.ca>
parents: 9707
diff changeset
34 # 8) parallel, coverage, local install:
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
35 # ./run-tests.py -j2 -c --local test-s* # unsupported (and broken)
9899
be574a37a8ae run-tests: give each child its own tmp dir (issue1911)
Greg Ward <greg@gerg.ca>
parents: 9707
diff changeset
36 # 9) parallel, custom tmp dir:
be574a37a8ae run-tests: give each child its own tmp dir (issue1911)
Greg Ward <greg@gerg.ca>
parents: 9707
diff changeset
37 # ./run-tests.py -j2 --tmpdir /tmp/myhgtests
26158
342ab95a1f4b run-tests: use $HGTEST_RUN_TESTS_PURE
timeless@mozdev.org
parents: 26109
diff changeset
38 # 10) parallel, pure, tests that call run-tests:
342ab95a1f4b run-tests: use $HGTEST_RUN_TESTS_PURE
timeless@mozdev.org
parents: 26109
diff changeset
39 # ./run-tests.py --pure `grep -l run-tests.py *.t`
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
40 #
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
41 # (You could use any subset of the tests: test-s* happens to match
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
42 # enough that it's worth doing parallel runs, few enough that it
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
43 # completes fairly quickly, includes both shell and Python scripts, and
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
44 # includes some scripts that run daemon processes.)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
45
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
46 from __future__ import absolute_import, print_function
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
47
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
48 import difflib
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
49 import distutils.version as version
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
50 import errno
28126
562a073a2a1b tests: load json with no fallback
Yuya Nishihara <yuya@tcha.org>
parents: 28099
diff changeset
51 import json
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
52 import optparse
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
53 import os
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
54 import random
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
55 import re
10905
13a1b2fb7ef2 pylint, pyflakes: remove unused or duplicate imports
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 10903
diff changeset
56 import shutil
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
57 import signal
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
58 import socket
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
59 import subprocess
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
60 import sys
32302
8627cf4de929 run-tests: drop fallback for sysconfig for pre-py2.7
Martin von Zweigbergk <martinvonz@google.com>
parents: 31950
diff changeset
61 import sysconfig
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
62 import tempfile
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
63 import threading
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
64 import time
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
65 import unittest
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
66 import xml.dom.minidom as minidom
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
67
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
68 try:
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
69 import Queue as queue
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
70 except ImportError:
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
71 import queue
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
72
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
73 try:
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
74 import shlex
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
75 shellquote = shlex.quote
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
76 except (ImportError, AttributeError):
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
77 import pipes
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
78 shellquote = pipes.quote
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
79
29282
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
80 if os.environ.get('RTUNICODEPEDANTRY', False):
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
81 try:
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
82 reload(sys)
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
83 sys.setdefaultencoding("undefined")
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
84 except NameError:
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
85 pass
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
86
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
87 origenviron = os.environ.copy()
25037
570141fa38f9 run-tests: introduce a name for os.environb
Augie Fackler <augie@google.com>
parents: 25036
diff changeset
88 osenvironb = getattr(os, 'environb', os.environ)
14019
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
89 processlock = threading.Lock()
19413
a4de0d3dc35a run-tests: lock popen wait/poll
Brendan Cully <brendan@kublai.com>
parents: 19407
diff changeset
90
25160
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
91 if sys.version_info > (3, 5, 0):
25157
a8d22895a575 run-tests: introduce PYTHON3 boolean constant (issue4668)
Augie Fackler <augie@google.com>
parents: 25156
diff changeset
92 PYTHON3 = True
25033
2bdd9e442bcc run-tests: work around the rename of xrange to range
Augie Fackler <augie@google.com>
parents: 25031
diff changeset
93 xrange = range # we use xrange in one place, and we'd rather not use range
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
94 def _bytespath(p):
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
95 return p.encode('utf-8')
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
96
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
97 def _strpath(p):
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
98 return p.decode('utf-8')
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
99
25160
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
100 elif sys.version_info >= (3, 0, 0):
32353
4bffe2421f34 run-tests: remove references to Python 2.6
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32328
diff changeset
101 print('%s is only supported on Python 3.5+ and 2.7, not %s' %
25160
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
102 (sys.argv[0], '.'.join(str(v) for v in sys.version_info[:3])))
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
103 sys.exit(70) # EX_SOFTWARE from `man 3 sysexit`
25157
a8d22895a575 run-tests: introduce PYTHON3 boolean constant (issue4668)
Augie Fackler <augie@google.com>
parents: 25156
diff changeset
104 else:
a8d22895a575 run-tests: introduce PYTHON3 boolean constant (issue4668)
Augie Fackler <augie@google.com>
parents: 25156
diff changeset
105 PYTHON3 = False
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
106
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
107 # In python 2.x, path operations are generally done using
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
108 # bytestrings by default, so we don't have to do any extra
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
109 # fiddling there. We define the wrapper functions anyway just to
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
110 # help keep code consistent between platforms.
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
111 def _bytespath(p):
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
112 return p
25033
2bdd9e442bcc run-tests: work around the rename of xrange to range
Augie Fackler <augie@google.com>
parents: 25031
diff changeset
113
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
114 _strpath = _bytespath
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
115
25177
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
116 # For Windows support
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
117 wifexited = getattr(os, "WIFEXITED", lambda x: False)
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
118
30984
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
119 # Whether to use IPv6
31002
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
120 def checksocketfamily(name, port=20058):
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
121 """return true if we can listen on localhost using family=name
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
122
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
123 name should be either 'AF_INET', or 'AF_INET6'.
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
124 port being used is okay - EADDRINUSE is considered as successful.
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
125 """
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
126 family = getattr(socket, name, None)
30984
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
127 if family is None:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
128 return False
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
129 try:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
130 s = socket.socket(family, socket.SOCK_STREAM)
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
131 s.bind(('localhost', port))
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
132 s.close()
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
133 return True
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
134 except socket.error as exc:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
135 if exc.errno == errno.EADDRINUSE:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
136 return True
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
137 elif exc.errno in (errno.EADDRNOTAVAIL, errno.EPROTONOSUPPORT):
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
138 return False
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
139 else:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
140 raise
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
141 else:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
142 return False
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
143
31011
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
144 # useipv6 will be set by parseargs
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
145 useipv6 = None
30984
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
146
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
147 def checkportisavailable(port):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
148 """return true if a port seems free to bind on localhost"""
30985
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
149 if useipv6:
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
150 family = socket.AF_INET6
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
151 else:
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
152 family = socket.AF_INET
30987
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
153 try:
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
154 s = socket.socket(family, socket.SOCK_STREAM)
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
155 s.bind(('localhost', port))
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
156 s.close()
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
157 return True
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
158 except socket.error as exc:
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
159 if exc.errno not in (errno.EADDRINUSE, errno.EADDRNOTAVAIL,
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
160 errno.EPROTONOSUPPORT):
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
161 raise
30886
2aaa8bfc7bd9 runtests: check ports on IPv6 address
Jun Wu <quark@fb.com>
parents: 30716
diff changeset
162 return False
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
163
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
164 closefds = os.name == 'posix'
19262
7864e8f274fe run-tests: add env dict to isolate test environment
Matt Mackall <mpm@selenic.com>
parents: 19252
diff changeset
165 def Popen4(cmd, wd, timeout, env=None):
14019
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
166 processlock.acquire()
19262
7864e8f274fe run-tests: add env dict to isolate test environment
Matt Mackall <mpm@selenic.com>
parents: 19252
diff changeset
167 p = subprocess.Popen(cmd, shell=True, bufsize=-1, cwd=wd, env=env,
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
168 close_fds=closefds,
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
169 stdin=subprocess.PIPE, stdout=subprocess.PIPE,
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
170 stderr=subprocess.STDOUT)
14019
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
171 processlock.release()
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
172
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
173 p.fromchild = p.stdout
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
174 p.tochild = p.stdin
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
175 p.childerr = p.stderr
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
176
14337
439ed4721a6d run-tests: ignore timeout when Popen.terminate is unavailable
Patrick Mezard <pmezard@gmail.com>
parents: 14336
diff changeset
177 p.timeout = False
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
178 if timeout:
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
179 def t():
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
180 start = time.time()
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
181 while time.time() - start < timeout and p.returncode is None:
16346
48692b416fbc tests: shorten post-test sleeps
Matt Mackall <mpm@selenic.com>
parents: 15942
diff changeset
182 time.sleep(.1)
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
183 p.timeout = True
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
184 if p.returncode is None:
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
185 terminate(p)
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
186 threading.Thread(target=t).start()
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
187
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
188 return p
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
189
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
190 PYTHON = _bytespath(sys.executable.replace('\\', '/'))
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
191 IMPL_PATH = b'PYTHONPATH'
10758
2ed667a9dfcb tests: adapt the test runner to work with jython
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10750
diff changeset
192 if 'java' in sys.platform:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
193 IMPL_PATH = b'JYTHONPATH'
4881
c51c9bc4579d Add hghave utility and run-tests.py support.
Patrick Mezard <pmezard@gmail.com>
parents: 4880
diff changeset
194
6366
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
195 defaults = {
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
196 'jobs': ('HGTEST_JOBS', 1),
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
197 'timeout': ('HGTEST_TIMEOUT', 180),
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
198 'slowtimeout': ('HGTEST_SLOWTIMEOUT', 500),
6366
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
199 'port': ('HGTEST_PORT', 20059),
15941
af289d6cd422 tests: let run-tests.py default to use 'sh' in $PATH instead of '/bin/sh'
Mads Kiilerich <mads@kiilerich.com>
parents: 15940
diff changeset
200 'shell': ('HGTEST_SHELL', 'sh'),
6366
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
201 }
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
202
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
203 def canonpath(path):
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
204 return os.path.realpath(os.path.expanduser(path))
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
205
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
206 def parselistfiles(files, listtype, warn=True):
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
207 entries = dict()
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
208 for filename in files:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
209 try:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
210 path = os.path.expanduser(os.path.expandvars(filename))
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
211 f = open(path, "rb")
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
212 except IOError as err:
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
213 if err.errno != errno.ENOENT:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
214 raise
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
215 if warn:
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
216 print("warning: no such %s file: %s" % (listtype, filename))
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
217 continue
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
218
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
219 for line in f.readlines():
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
220 line = line.split(b'#', 1)[0].strip()
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
221 if line:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
222 entries[line] = filename
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
223
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
224 f.close()
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
225 return entries
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
226
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
227 def parsettestcases(path):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
228 """read a .t test file, return a set of test case names
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
229
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
230 If path does not exist, return an empty set.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
231 """
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
232 cases = set()
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
233 try:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
234 with open(path, 'rb') as f:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
235 for l in f:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
236 if l.startswith(b'#testcases '):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
237 cases.update(l[11:].split())
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
238 except IOError as ex:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
239 if ex.errno != errno.ENOENT:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
240 raise
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
241 return cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
242
21008
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
243 def getparser():
21383
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
244 """Obtain the OptionParser used by the CLI."""
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
245 parser = optparse.OptionParser("%prog [options] [tests]")
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
246
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
247 # keep these sorted
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
248 parser.add_option("--blacklist", action="append",
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
249 help="skip tests listed in the specified blacklist file")
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
250 parser.add_option("--whitelist", action="append",
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
251 help="always run tests listed in the specified whitelist file")
20821
3d1d16b19e7d tests: add run-tests --changed option for running tests changed in revisions
Mads Kiilerich <madski@unity3d.com>
parents: 20793
diff changeset
252 parser.add_option("--changed", type="string",
3d1d16b19e7d tests: add run-tests --changed option for running tests changed in revisions
Mads Kiilerich <madski@unity3d.com>
parents: 20793
diff changeset
253 help="run tests that are changed in parent rev or working directory")
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
254 parser.add_option("-C", "--annotate", action="store_true",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
255 help="output files annotated with coverage")
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
256 parser.add_option("-c", "--cover", action="store_true",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
257 help="print a test coverage report")
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
258 parser.add_option("-d", "--debug", action="store_true",
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
259 help="debug mode: write output of test scripts to console"
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21023
diff changeset
260 " rather than capturing and diffing it (disables timeout)")
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
261 parser.add_option("-f", "--first", action="store_true",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
262 help="exit on the first test failure")
15859
44a371823f83 tests: add htmlcov option
Markus Zapke-Gründemann <info@keimlink.de>
parents: 15858
diff changeset
263 parser.add_option("-H", "--htmlcov", action="store_true",
44a371823f83 tests: add htmlcov option
Markus Zapke-Gründemann <info@keimlink.de>
parents: 15858
diff changeset
264 help="create an HTML report of the coverage of the files")
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
265 parser.add_option("-i", "--interactive", action="store_true",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
266 help="prompt to accept changed output")
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
267 parser.add_option("-j", "--jobs", type="int",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
268 help="number of jobs to run in parallel"
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
269 " (default: $%s or %d)" % defaults['jobs'])
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
270 parser.add_option("--keep-tmpdir", action="store_true",
9706
f8b4df4b033d run-tests: make --tmpdir option more useful.
Greg Ward <greg-hg@gerg.ca>
parents: 9582
diff changeset
271 help="keep temporary directory after running tests")
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
272 parser.add_option("-k", "--keywords",
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
273 help="run tests matching keywords")
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
274 parser.add_option("--list-tests", action="store_true",
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
275 help="list tests instead of running them")
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
276 parser.add_option("-l", "--local", action="store_true",
29583
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
277 help="shortcut for --with-hg=<testdir>/../hg, "
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
278 "and --with-chg=<testdir>/../contrib/chg/chg if --chg is set")
19283
8300adf9ca33 run-tests: add --loop support
Matt Mackall <mpm@selenic.com>
parents: 19282
diff changeset
279 parser.add_option("--loop", action="store_true",
8300adf9ca33 run-tests: add --loop support
Matt Mackall <mpm@selenic.com>
parents: 19282
diff changeset
280 help="loop tests repeatedly")
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
281 parser.add_option("--runs-per-test", type="int", dest="runs_per_test",
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
282 help="run each test N times (default=1)", default=1)
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
283 parser.add_option("-n", "--nodiff", action="store_true",
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
284 help="skip showing test changes")
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
285 parser.add_option("--outputdir", type="string",
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
286 help="directory to write error logs to (default=test directory)")
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
287 parser.add_option("-p", "--port", type="int",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
288 help="port on which servers should listen"
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
289 " (default: $%s or %d)" % defaults['port'])
17966
ae20cde050c2 run-tests: add a --compiler option
Bryan O'Sullivan <bryano@fb.com>
parents: 17965
diff changeset
290 parser.add_option("--compiler", type="string",
ae20cde050c2 run-tests: add a --compiler option
Bryan O'Sullivan <bryano@fb.com>
parents: 17965
diff changeset
291 help="compiler to build with")
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
292 parser.add_option("--pure", action="store_true",
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
293 help="use pure Python code instead of C extensions")
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
294 parser.add_option("-R", "--restart", action="store_true",
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
295 help="restart at last error")
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
296 parser.add_option("-r", "--retest", action="store_true",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
297 help="retest failed tests")
9580
25858f9e65e8 run-tests: add --noskips option
Matt Mackall <mpm@selenic.com>
parents: 9408
diff changeset
298 parser.add_option("-S", "--noskips", action="store_true",
25858f9e65e8 run-tests: add --noskips option
Matt Mackall <mpm@selenic.com>
parents: 9408
diff changeset
299 help="don't report skip tests verbosely")
14202
b68a41420397 run-tests: add --shell command line flag
Martin Geisler <mg@lazybytes.net>
parents: 14201
diff changeset
300 parser.add_option("--shell", type="string",
b68a41420397 run-tests: add --shell command line flag
Martin Geisler <mg@lazybytes.net>
parents: 14201
diff changeset
301 help="shell to use (default: $%s or %s)" % defaults['shell'])
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
302 parser.add_option("-t", "--timeout", type="int",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
303 help="kill errant tests after TIMEOUT seconds"
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
304 " (default: $%s or %d)" % defaults['timeout'])
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
305 parser.add_option("--slowtimeout", type="int",
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
306 help="kill errant slow tests after SLOWTIMEOUT seconds"
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
307 " (default: $%s or %d)" % defaults['slowtimeout'])
17921
4ac9cf3d810c run-tests: add --time option to log times for each test
Siddharth Agarwal <sid0@fb.com>
parents: 17920
diff changeset
308 parser.add_option("--time", action="store_true",
4ac9cf3d810c run-tests: add --time option to log times for each test
Siddharth Agarwal <sid0@fb.com>
parents: 17920
diff changeset
309 help="time how long each test takes")
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
310 parser.add_option("--json", action="store_true",
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
311 help="store test result data in 'report.json' file")
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
312 parser.add_option("--tmpdir", type="string",
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
313 help="run tests in the given temporary directory"
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
314 " (implies --keep-tmpdir)")
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
315 parser.add_option("-v", "--verbose", action="store_true",
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
316 help="output verbose messages")
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
317 parser.add_option("--xunit", type="string",
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
318 help="record xunit results at specified path")
11040
8f951ed6c63c run-tests: add --view switch to use external diff viewer
Matt Mackall <mpm@selenic.com>
parents: 11039
diff changeset
319 parser.add_option("--view", type="string",
8f951ed6c63c run-tests: add --view switch to use external diff viewer
Matt Mackall <mpm@selenic.com>
parents: 11039
diff changeset
320 help="external diff viewer")
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
321 parser.add_option("--with-hg", type="string",
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
322 metavar="HG",
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
323 help="test using specified hg script rather than a "
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
324 "temporary installation")
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
325 parser.add_option("--chg", action="store_true",
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
326 help="install and use chg wrapper in place of hg")
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
327 parser.add_option("--with-chg", metavar="CHG",
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
328 help="use specified chg wrapper in place of hg")
31011
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
329 parser.add_option("--ipv6", action="store_true",
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
330 help="prefer IPv6 to IPv4 for network related tests")
9028
bea567ae3ff6 tests: add -3 switch to run-tests.py
Alejandro Santos <alejolp@alejolp.com>
parents: 8943
diff changeset
331 parser.add_option("-3", "--py3k-warnings", action="store_true",
32353
4bffe2421f34 run-tests: remove references to Python 2.6
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32328
diff changeset
332 help="enable Py3k warnings on Python 2.7+")
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
333 # This option should be deleted once test-check-py3-compat.t and other
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
334 # Python 3 tests run with Python 3.
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
335 parser.add_option("--with-python3", metavar="PYTHON3",
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
336 help="Python 3 interpreter (if running under Python 2)"
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
337 " (TEMPORARY)")
14134
8468ec1109d1 run-tests: add flag to provide extra hgrc options for test runs
Augie Fackler <durin42@gmail.com>
parents: 14062
diff changeset
338 parser.add_option('--extra-config-opt', action="append",
8468ec1109d1 run-tests: add flag to provide extra hgrc options for test runs
Augie Fackler <durin42@gmail.com>
parents: 14062
diff changeset
339 help='set the given config opt in the test hgrc')
19057
3d265e0822d3 run-tests: introduce --random for running tests in random error
Mads Kiilerich <madski@unity3d.com>
parents: 18788
diff changeset
340 parser.add_option('--random', action="store_true",
3d265e0822d3 run-tests: introduce --random for running tests in random error
Mads Kiilerich <madski@unity3d.com>
parents: 18788
diff changeset
341 help='run tests in random order')
25107
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
342 parser.add_option('--profile-runner', action='store_true',
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
343 help='run statprof on run-tests')
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
344 parser.add_option('--allow-slow-tests', action='store_true',
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
345 help='allow extremely slow tests')
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
346 parser.add_option('--showchannels', action='store_true',
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
347 help='show scheduling channels')
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
348 parser.add_option('--known-good-rev', type="string",
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
349 metavar="known_good_rev",
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
350 help=("Automatically bisect any failures using this "
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
351 "revision as a known-good revision."))
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
352
14201
57e04ded3da4 run-tests: use type of default to convert environment variable
Martin Geisler <mg@lazybytes.net>
parents: 14192
diff changeset
353 for option, (envvar, default) in defaults.items():
57e04ded3da4 run-tests: use type of default to convert environment variable
Martin Geisler <mg@lazybytes.net>
parents: 14192
diff changeset
354 defaults[option] = type(default)(os.environ.get(envvar, default))
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
355 parser.set_defaults(**defaults)
21008
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
356
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
357 return parser
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
358
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
359 def parseargs(args, parser):
21383
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
360 """Parse arguments with our OptionParser and validate results."""
21006
723e41ad59b4 run-tests: Pass arguments into argument parser
Gregory Szorc <gregory.szorc@gmail.com>
parents: 20821
diff changeset
361 (options, args) = parser.parse_args(args)
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
362
10758
2ed667a9dfcb tests: adapt the test runner to work with jython
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10750
diff changeset
363 # jython is always pure
10766
afbcea270bb8 run-tests: force to test pure on pypy as well
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10765
diff changeset
364 if 'java' in sys.platform or '__pypy__' in sys.modules:
10765
fd31a3237498 Fix run-tests.py -jX after 2ed667a9dfcb
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10758
diff changeset
365 options.pure = True
10758
2ed667a9dfcb tests: adapt the test runner to work with jython
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10750
diff changeset
366
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
367 if options.with_hg:
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
368 options.with_hg = canonpath(_bytespath(options.with_hg))
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
369 if not (os.path.isfile(options.with_hg) and
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
370 os.access(options.with_hg, os.X_OK)):
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
371 parser.error('--with-hg must specify an executable hg script')
32328
531e6a57abd2 run-tests: allow hg executable to be hg.exe
Kostia Balytskyi <ikostia@fb.com>
parents: 32316
diff changeset
372 if os.path.basename(options.with_hg) not in [b'hg', b'hg.exe']:
14359
ad5c68a0db6a run-tests: print a newline after all warnings
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14340
diff changeset
373 sys.stderr.write('warning: --with-hg should specify an hg script\n')
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
374 if options.local:
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
375 testdir = os.path.dirname(_bytespath(canonpath(sys.argv[0])))
29582
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
376 reporootdir = os.path.dirname(testdir)
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
377 pathandattrs = [(b'hg', 'with_hg')]
29583
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
378 if options.chg:
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
379 pathandattrs.append((b'contrib/chg/chg', 'with_chg'))
29582
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
380 for relpath, attr in pathandattrs:
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
381 binpath = os.path.join(reporootdir, relpath)
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
382 if os.name != 'nt' and not os.access(binpath, os.X_OK):
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
383 parser.error('--local specified, but %r not found or '
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
384 'not executable' % binpath)
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
385 setattr(options, attr, binpath)
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
386
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
387 if (options.chg or options.with_chg) and os.name == 'nt':
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
388 parser.error('chg does not work on %s' % os.name)
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
389 if options.with_chg:
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
390 options.chg = False # no installation to temporary location
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
391 options.with_chg = canonpath(_bytespath(options.with_chg))
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
392 if not (os.path.isfile(options.with_chg) and
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
393 os.access(options.with_chg, os.X_OK)):
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
394 parser.error('--with-chg must specify a chg executable')
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
395 if options.chg and options.with_hg:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
396 # chg shares installation location with hg
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
397 parser.error('--chg does not work when --with-hg is specified '
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
398 '(use --with-chg instead)')
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
399
31011
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
400 global useipv6
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
401 if options.ipv6:
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
402 useipv6 = checksocketfamily('AF_INET6')
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
403 else:
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
404 # only use IPv6 if IPv4 is unavailable and IPv6 is available
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
405 useipv6 = ((not checksocketfamily('AF_INET'))
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
406 and checksocketfamily('AF_INET6'))
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
407
15859
44a371823f83 tests: add htmlcov option
Markus Zapke-Gründemann <info@keimlink.de>
parents: 15858
diff changeset
408 options.anycoverage = options.cover or options.annotate or options.htmlcov
10648
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
409 if options.anycoverage:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
410 try:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
411 import coverage
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
412 covver = version.StrictVersion(coverage.__version__).version
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
413 if covver < (3, 3):
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
414 parser.error('coverage options require coverage 3.3 or later')
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
415 except ImportError:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
416 parser.error('coverage options now require the coverage package')
8095
f5428d4ffd97 run-tests: reduce global variables set by parse_args().
Greg Ward <greg-hg@gerg.ca>
parents: 8094
diff changeset
417
10648
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
418 if options.anycoverage and options.local:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
419 # this needs some path mangling somewhere, I guess
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
420 parser.error("sorry, coverage options do not work when --local "
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
421 "is specified")
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
422
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
423 if options.anycoverage and options.with_hg:
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
424 parser.error("sorry, coverage options do not work when --with-hg "
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
425 "is specified")
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
426
19250
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
427 global verbose
8095
f5428d4ffd97 run-tests: reduce global variables set by parse_args().
Greg Ward <greg-hg@gerg.ca>
parents: 8094
diff changeset
428 if options.verbose:
19279
3868c9ab4bf8 run-tests: drop options.child and users
Matt Mackall <mpm@selenic.com>
parents: 19278
diff changeset
429 verbose = ''
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
430
9394
31203db1b2ac run-tests: expand --tmpdir and create it if needed
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 9031
diff changeset
431 if options.tmpdir:
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
432 options.tmpdir = canonpath(options.tmpdir)
9394
31203db1b2ac run-tests: expand --tmpdir and create it if needed
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 9031
diff changeset
433
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
434 if options.jobs < 1:
9408
70bf7f853adc run-tests: standardize on --foo instead of -f/--foo
Martin Geisler <mg@lazybytes.net>
parents: 9407
diff changeset
435 parser.error('--jobs must be positive')
9707
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
436 if options.interactive and options.debug:
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
437 parser.error("-i/--interactive and -d/--debug are incompatible")
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
438 if options.debug:
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
439 if options.timeout != defaults['timeout']:
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
440 sys.stderr.write(
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
441 'warning: --timeout option ignored with --debug\n')
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
442 if options.slowtimeout != defaults['slowtimeout']:
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
443 sys.stderr.write(
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
444 'warning: --slowtimeout option ignored with --debug\n')
9707
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
445 options.timeout = 0
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
446 options.slowtimeout = 0
9028
bea567ae3ff6 tests: add -3 switch to run-tests.py
Alejandro Santos <alejolp@alejolp.com>
parents: 8943
diff changeset
447 if options.py3k_warnings:
25158
3906b9783cd9 run-tests: prefer PYTHON3 constant to many version_info checks (issue4668)
Augie Fackler <augie@google.com>
parents: 25157
diff changeset
448 if PYTHON3:
3906b9783cd9 run-tests: prefer PYTHON3 constant to many version_info checks (issue4668)
Augie Fackler <augie@google.com>
parents: 25157
diff changeset
449 parser.error(
32353
4bffe2421f34 run-tests: remove references to Python 2.6
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32328
diff changeset
450 '--py3k-warnings can only be used on Python 2.7')
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
451 if options.with_python3:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
452 if PYTHON3:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
453 parser.error('--with-python3 cannot be used when executing with '
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
454 'Python 3')
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
455
28645
c226e9b69bac run-tests: use canonpath for with-python3
timeless <timeless@mozdev.org>
parents: 28644
diff changeset
456 options.with_python3 = canonpath(options.with_python3)
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
457 # Verify Python3 executable is acceptable.
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
458 proc = subprocess.Popen([options.with_python3, b'--version'],
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
459 stdout=subprocess.PIPE,
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
460 stderr=subprocess.STDOUT)
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
461 out, _err = proc.communicate()
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
462 ret = proc.wait()
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
463 if ret != 0:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
464 parser.error('could not determine version of python 3')
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
465 if not out.startswith('Python '):
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
466 parser.error('unexpected output from python3 --version: %s' %
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
467 out)
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
468 vers = version.LooseVersion(out[len('Python '):])
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
469 if vers < version.LooseVersion('3.5.0'):
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
470 parser.error('--with-python3 version must be 3.5.0 or greater; '
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
471 'got %s' % out)
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
472
9959
b37b060d84c7 run-tests: add a "--blacklist target" option to skip predefined test lists
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 9958
diff changeset
473 if options.blacklist:
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
474 options.blacklist = parselistfiles(options.blacklist, 'blacklist')
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
475 if options.whitelist:
19279
3868c9ab4bf8 run-tests: drop options.child and users
Matt Mackall <mpm@selenic.com>
parents: 19278
diff changeset
476 options.whitelisted = parselistfiles(options.whitelist, 'whitelist')
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
477 else:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
478 options.whitelisted = {}
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
479
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
480 if options.showchannels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
481 options.nodiff = True
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
482
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
483 return (options, args)
5384
e3a0c092b4e2 Allow tests to run in parallel.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5383
diff changeset
484
5800
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
485 def rename(src, dst):
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
486 """Like os.rename(), trade atomicity and opened files friendliness
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
487 for existing destination support.
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
488 """
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
489 shutil.copy(src, dst)
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
490 os.remove(src)
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
491
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
492 _unified_diff = difflib.unified_diff
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
493 if PYTHON3:
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
494 import functools
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
495 _unified_diff = functools.partial(difflib.diff_bytes, difflib.unified_diff)
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
496
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
497 def getdiff(expected, output, ref, err):
21022
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
498 servefail = False
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
499 lines = []
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
500 for line in _unified_diff(expected, output, ref, err):
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
501 if line.startswith(b'+++') or line.startswith(b'---'):
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
502 line = line.replace(b'\\', b'/')
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
503 if line.endswith(b' \n'):
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
504 line = line[:-2] + b'\n'
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
505 lines.append(line)
21022
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
506 if not servefail and line.startswith(
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
507 b'+ abort: child process failed to start'):
21022
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
508 servefail = True
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
509
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
510 return servefail, lines
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
511
19250
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
512 verbose = False
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
513 def vlog(*msg):
21535
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
514 """Log only when in verbose mode."""
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
515 if verbose is False:
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
516 return
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
517
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
518 return log(*msg)
19250
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
519
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
520 # Bytes that break XML even in a CDATA block: control characters 0-31
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
521 # sans \t, \n and \r
25051
9c28f3236677 run-tests: do cdata escaping using bytes instead of str
Augie Fackler <augie@google.com>
parents: 25050
diff changeset
522 CDATA_EVIL = re.compile(br"[\000-\010\013\014\016-\037]")
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
523
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
524 # Match feature conditionalized output lines in the form, capturing the feature
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
525 # list in group 2, and the preceeding line output in group 1:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
526 #
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
527 # output..output (feature !)\n
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
528 optline = re.compile(b'(.+) \((.+?) !\)\n$')
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
529
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
530 def cdatasafe(data):
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
531 """Make a string safe to include in a CDATA block.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
532
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
533 Certain control characters are illegal in a CDATA block, and
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
534 there's no way to include a ]]> in a CDATA either. This function
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
535 replaces illegal bytes with ? and adds a space between the ]] so
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
536 that it won't break the CDATA block.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
537 """
25051
9c28f3236677 run-tests: do cdata escaping using bytes instead of str
Augie Fackler <augie@google.com>
parents: 25050
diff changeset
538 return CDATA_EVIL.sub(b'?', data).replace(b']]>', b'] ]>')
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
539
19251
6857f53456f2 run-tests: add a log function
Matt Mackall <mpm@selenic.com>
parents: 19250
diff changeset
540 def log(*msg):
21535
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
541 """Log something to stdout.
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
542
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
543 Arguments are strings to print.
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
544 """
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
545 with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
546 if verbose:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
547 print(verbose, end=' ')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
548 for m in msg:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
549 print(m, end=' ')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
550 print()
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
551 sys.stdout.flush()
19251
6857f53456f2 run-tests: add a log function
Matt Mackall <mpm@selenic.com>
parents: 19250
diff changeset
552
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
553 def terminate(proc):
32303
dc131b9772f2 run-tests: drop fallback for proc.terminate() for pre-py2.6
Martin von Zweigbergk <martinvonz@google.com>
parents: 32302
diff changeset
554 """Terminate subprocess"""
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
555 vlog('# Terminating process %d' % proc.pid)
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
556 try:
32303
dc131b9772f2 run-tests: drop fallback for proc.terminate() for pre-py2.6
Martin von Zweigbergk <martinvonz@google.com>
parents: 32302
diff changeset
557 proc.terminate()
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
558 except OSError:
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
559 pass
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
560
19263
062c0a0a5549 run-tests: use env dict to kill daemons
Matt Mackall <mpm@selenic.com>
parents: 19262
diff changeset
561 def killdaemons(pidfile):
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
562 import killdaemons as killmod
19263
062c0a0a5549 run-tests: use env dict to kill daemons
Matt Mackall <mpm@selenic.com>
parents: 19262
diff changeset
563 return killmod.killdaemons(pidfile, tryhard=False, remove=True,
17464
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 16906
diff changeset
564 logfn=vlog)
10336
bc9a3bb267fa run-tests: kill daemons on ^C with -j.
Brendan Cully <brendan@kublai.com>
parents: 10300
diff changeset
565
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
566 class Test(unittest.TestCase):
21309
0b123e6a318c run-tests: allow Test.run() to run multiple times
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21308
diff changeset
567 """Encapsulates a single, runnable test.
0b123e6a318c run-tests: allow Test.run() to run multiple times
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21308
diff changeset
568
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
569 While this class conforms to the unittest.TestCase API, it differs in that
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
570 instances need to be instantiated manually. (Typically, unittest.TestCase
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
571 classes are instantiated automatically by scanning modules.)
21309
0b123e6a318c run-tests: allow Test.run() to run multiple times
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21308
diff changeset
572 """
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
573
21380
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
574 # Status code reserved for skipped tests (used by hghave).
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
575 SKIPPED_STATUS = 80
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
576
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
577 def __init__(self, path, outputdir, tmpdir, keeptmpdir=False,
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
578 debug=False,
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
579 timeout=defaults['timeout'],
21516
1e275c09242e run-tests: move py3kwarnings to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21515
diff changeset
580 startport=defaults['port'], extraconfigopts=None,
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
581 py3kwarnings=False, shell=None, hgcommand=None,
30986
f07ca071a058 runtests: set web.ipv6 if we use IPv6
Jun Wu <quark@fb.com>
parents: 30985
diff changeset
582 slowtimeout=defaults['slowtimeout'], usechg=False,
f07ca071a058 runtests: set web.ipv6 if we use IPv6
Jun Wu <quark@fb.com>
parents: 30985
diff changeset
583 useipv6=False):
21502
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
584 """Create a test from parameters.
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
585
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
586 path is the full path to the file defining the test.
21338
3cd2d2de4060 run-tests: move computation of test paths into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21337
diff changeset
587
21504
888a5dfe1569 run-tests: pass temp dir into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21502
diff changeset
588 tmpdir is the main temporary directory to use for this test.
21505
3a1881dbf860 run-tests: pass abort into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21504
diff changeset
589
21509
d21d53ee0d7a run-tests: factor options.keep_tmpdir into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21508
diff changeset
590 keeptmpdir determines whether to keep the test's temporary directory
d21d53ee0d7a run-tests: factor options.keep_tmpdir into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21508
diff changeset
591 after execution. It defaults to removal (False).
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
592
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
593 debug mode will make the test execute verbosely, with unfiltered
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
594 output.
21511
3ec3e81a4110 run-tests: move diff options into arguments of Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21510
diff changeset
595
21513
acfd19f3e79c run-tests: move timeout into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21512
diff changeset
596 timeout controls the maximum run time of the test. It is ignored when
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
597 debug is True. See slowtimeout for tests with #require slow.
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
598
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
599 slowtimeout overrides timeout if the test has #require slow.
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
600
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
601 startport controls the starting port number to use for this test. Each
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
602 test will reserve 3 port numbers for execution. It is the caller's
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
603 responsibility to allocate a non-overlapping port range to Test
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
604 instances.
21515
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
605
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
606 extraconfigopts is an iterable of extra hgrc config options. Values
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
607 must have the form "key=value" (something understood by hgrc). Values
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
608 of the form "foo.key=value" will result in "[foo] key=value".
21516
1e275c09242e run-tests: move py3kwarnings to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21515
diff changeset
609
1e275c09242e run-tests: move py3kwarnings to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21515
diff changeset
610 py3kwarnings enables Py3k warnings.
21517
af7d3a5c330b run-tests: move shell to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21516
diff changeset
611
af7d3a5c330b run-tests: move shell to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21516
diff changeset
612 shell is the shell to execute tests in.
21502
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
613 """
25039
8505eb1bafb1 run-tests: be more judicious about bytes vs string on test attrs
Augie Fackler <augie@google.com>
parents: 25038
diff changeset
614 self.path = path
8505eb1bafb1 run-tests: be more judicious about bytes vs string on test attrs
Augie Fackler <augie@google.com>
parents: 25038
diff changeset
615 self.bname = os.path.basename(path)
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
616 self.name = _strpath(self.bname)
21502
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
617 self._testdir = os.path.dirname(path)
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
618 self._outputdir = outputdir
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
619 self._tmpname = os.path.basename(path)
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
620 self.errpath = os.path.join(self._outputdir, b'%s.err' % self.bname)
21435
f376f56a354e run-tests: rename Test._test to Test.name
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21434
diff changeset
621
25039
8505eb1bafb1 run-tests: be more judicious about bytes vs string on test attrs
Augie Fackler <augie@google.com>
parents: 25038
diff changeset
622 self._threadtmp = tmpdir
21509
d21d53ee0d7a run-tests: factor options.keep_tmpdir into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21508
diff changeset
623 self._keeptmpdir = keeptmpdir
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
624 self._debug = debug
21513
acfd19f3e79c run-tests: move timeout into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21512
diff changeset
625 self._timeout = timeout
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
626 self._slowtimeout = slowtimeout
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
627 self._startport = startport
21515
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
628 self._extraconfigopts = extraconfigopts or []
21516
1e275c09242e run-tests: move py3kwarnings to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21515
diff changeset
629 self._py3kwarnings = py3kwarnings
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
630 self._shell = _bytespath(shell)
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
631 self._hgcommand = hgcommand or b'hg'
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
632 self._usechg = usechg
31003
225f574e0645 runtests: export HGIPV6 to hint test scripts whether to use IPv6
Jun Wu <quark@fb.com>
parents: 31002
diff changeset
633 self._useipv6 = useipv6
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
634
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
635 self._aborted = False
21319
44c96e2bab20 run-tests: kill daemons during Test.cleanup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21318
diff changeset
636 self._daemonpids = []
21447
f8c5b8a288c5 run-tests: keep track of test execution state in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21446
diff changeset
637 self._finished = None
21449
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
638 self._ret = None
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
639 self._out = None
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
640 self._skipped = None
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
641 self._testtmp = None
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
642 self._chgsockdir = None
21447
f8c5b8a288c5 run-tests: keep track of test execution state in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21446
diff changeset
643
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
644 self._refout = self.readrefout()
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
645
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
646 def readrefout(self):
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
647 """read reference output"""
21318
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
648 # If we're not in --debug mode and reference output file exists,
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
649 # check test output against it.
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
650 if self._debug:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
651 return None # to match "out is None"
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
652 elif os.path.exists(self.refpath):
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
653 with open(self.refpath, 'rb') as f:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
654 return f.read().splitlines(True)
21318
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
655 else:
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
656 return []
21318
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
657
24965
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
658 # needed to get base class __repr__ running
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
659 @property
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
660 def _testMethodName(self):
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
661 return self.name
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
662
21463
c908ff887589 run-tests: implement Test.__str__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21462
diff changeset
663 def __str__(self):
c908ff887589 run-tests: implement Test.__str__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21462
diff changeset
664 return self.name
c908ff887589 run-tests: implement Test.__str__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21462
diff changeset
665
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
666 def shortDescription(self):
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
667 return self.name
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
668
21446
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
669 def setUp(self):
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
670 """Tasks to perform before run()."""
21447
f8c5b8a288c5 run-tests: keep track of test execution state in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21446
diff changeset
671 self._finished = False
21449
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
672 self._ret = None
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
673 self._out = None
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
674 self._skipped = None
21446
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
675
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
676 try:
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
677 os.mkdir(self._threadtmp)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
678 except OSError as e:
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
679 if e.errno != errno.EEXIST:
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
680 raise
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
681
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
682 name = self._tmpname
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
683 self._testtmp = os.path.join(self._threadtmp, name)
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
684 os.mkdir(self._testtmp)
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
685
21457
12dd94e32102 run-tests: move errpath deletion to setUp()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21456
diff changeset
686 # Remove any previous output files.
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
687 if os.path.exists(self.errpath):
24332
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
688 try:
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
689 os.remove(self.errpath)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
690 except OSError as e:
24332
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
691 # We might have raced another test to clean up a .err
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
692 # file, so ignore ENOENT when removing a previous .err
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
693 # file.
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
694 if e.errno != errno.ENOENT:
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
695 raise
21457
12dd94e32102 run-tests: move errpath deletion to setUp()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21456
diff changeset
696
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
697 if self._usechg:
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
698 self._chgsockdir = os.path.join(self._threadtmp,
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
699 b'%s.chgsock' % name)
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
700 os.mkdir(self._chgsockdir)
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
701
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
702 def run(self, result):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
703 """Run this test and report results against a TestResult instance."""
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
704 # This function is extremely similar to unittest.TestCase.run(). Once
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
705 # we require Python 2.7 (or at least its version of unittest), this
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
706 # function can largely go away.
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
707 self._result = result
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
708 result.startTest(self)
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
709 try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
710 try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
711 self.setUp()
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
712 except (KeyboardInterrupt, SystemExit):
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
713 self._aborted = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
714 raise
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
715 except Exception:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
716 result.addError(self, sys.exc_info())
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
717 return
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
718
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
719 success = False
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
720 try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
721 self.runTest()
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
722 except KeyboardInterrupt:
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
723 self._aborted = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
724 raise
32932
240183a04429 tests: use unittest.SkipTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32907
diff changeset
725 except unittest.SkipTest as e:
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
726 result.addSkip(self, str(e))
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
727 # The base class will have already counted this as a
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
728 # test we "ran", but we want to exclude skipped tests
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
729 # from those we count towards those run.
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
730 result.testsRun -= 1
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
731 except self.failureException as e:
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
732 # This differs from unittest in that we don't capture
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
733 # the stack trace. This is for historical reasons and
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
734 # this decision could be revisited in the future,
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
735 # especially for PythonTest instances.
21753
b7baef94a333 run-tests: checks behaviour of test on failure while testing
anuraggoel <anurag.dsps@gmail.com>
parents: 21740
diff changeset
736 if result.addFailure(self, str(e)):
b7baef94a333 run-tests: checks behaviour of test on failure while testing
anuraggoel <anurag.dsps@gmail.com>
parents: 21740
diff changeset
737 success = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
738 except Exception:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
739 result.addError(self, sys.exc_info())
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
740 else:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
741 success = True
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
742
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
743 try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
744 self.tearDown()
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
745 except (KeyboardInterrupt, SystemExit):
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
746 self._aborted = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
747 raise
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
748 except Exception:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
749 result.addError(self, sys.exc_info())
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
750 success = False
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
751
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
752 if success:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
753 result.addSuccess(self)
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
754 finally:
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
755 result.stopTest(self, interrupted=self._aborted)
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
756
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
757 def runTest(self):
21383
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
758 """Run this test instance.
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
759
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
760 This will return a tuple describing the result of the test.
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
761 """
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
762 env = self._getenv()
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
763 self._genrestoreenv(env)
21319
44c96e2bab20 run-tests: kill daemons during Test.cleanup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21318
diff changeset
764 self._daemonpids.append(env['DAEMON_PIDS'])
21382
4b8ffe3abdd2 run-tests: move createhgrc into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21381
diff changeset
765 self._createhgrc(env['HGRCPATH'])
21300
a2774731a51a run-tests: move createhgrc() call into Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21299
diff changeset
766
21435
f376f56a354e run-tests: rename Test._test to Test.name
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21434
diff changeset
767 vlog('# Test', self.name)
21337
79930515604f run-tests: move logging of test start into Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21336
diff changeset
768
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
769 ret, out = self._run(env)
21500
130cc0d7bfde run-tests: don't trap exceptions in Test.runTest()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21499
diff changeset
770 self._finished = True
130cc0d7bfde run-tests: don't trap exceptions in Test.runTest()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21499
diff changeset
771 self._ret = ret
130cc0d7bfde run-tests: don't trap exceptions in Test.runTest()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21499
diff changeset
772 self._out = out
21305
d7a7825ff2cf run-tests: capture execution results in a TestResult class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21304
diff changeset
773
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
774 def describe(ret):
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
775 if ret < 0:
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
776 return 'killed by signal: %d' % -ret
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
777 return 'returned error code %d' % ret
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
778
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
779 self._skipped = False
21336
45ab0668d1b2 run-tests: remove remaining uses of TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21335
diff changeset
780
21380
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
781 if ret == self.SKIPPED_STATUS:
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
782 if out is None: # Debug mode, nothing to parse.
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
783 missing = ['unknown']
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
784 failed = None
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
785 else:
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
786 missing, failed = TTest.parsehghaveoutput(out)
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
787
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
788 if not missing:
22292
102f0e926668 run-tests: report skipped tests as "skipped" - they might still be "relevant"
Mads Kiilerich <madski@unity3d.com>
parents: 22165
diff changeset
789 missing = ['skipped']
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
790
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
791 if failed:
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
792 self.fail('hg have failed checking for %s' % failed[-1])
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
793 else:
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
794 self._skipped = True
32932
240183a04429 tests: use unittest.SkipTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32907
diff changeset
795 raise unittest.SkipTest(missing[-1])
21325
0e66eb57e42a run-tests: generate timeout result in Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21324
diff changeset
796 elif ret == 'timeout':
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
797 self.fail('timed out')
21522
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
798 elif ret is False:
32934
6123a5267119 tests: remove WarnTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32933
diff changeset
799 self.fail('no result code from test')
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
800 elif out != self._refout:
21614
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
801 # Diff generation may rely on written .err file.
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
802 if (ret != 0 or out != self._refout) and not self._skipped \
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
803 and not self._debug:
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
804 f = open(self.errpath, 'wb')
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
805 for line in out:
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
806 f.write(line)
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
807 f.close()
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
808
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
809 # The result object handles diff calculation for us.
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
810 if self._result.addOutputMismatch(self, ret, out, self._refout):
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
811 # change was accepted, skip failing
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
812 return
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
813
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
814 if ret:
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
815 msg = 'output changed and ' + describe(ret)
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
816 else:
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
817 msg = 'output changed'
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
818
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
819 self.fail(msg)
21327
206814c9072a run-tests: move remaining result processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21326
diff changeset
820 elif ret:
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
821 self.fail(describe(ret))
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
822
21446
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
823 def tearDown(self):
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
824 """Tasks to perform after run()."""
21456
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
825 for entry in self._daemonpids:
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
826 killdaemons(entry)
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
827 self._daemonpids = []
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
828
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
829 if self._keeptmpdir:
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
830 log('\nKeeping testtmp dir: %s\nKeeping threadtmp dir: %s' %
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
831 (self._testtmp.decode('utf-8'),
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
832 self._threadtmp.decode('utf-8')))
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
833 else:
21461
a46a91989d57 run-tests: ignore failures from rmtree
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21460
diff changeset
834 shutil.rmtree(self._testtmp, True)
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
835 shutil.rmtree(self._threadtmp, True)
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
836
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
837 if self._usechg:
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
838 # chgservers will stop automatically after they find the socket
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
839 # files are deleted
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
840 shutil.rmtree(self._chgsockdir, True)
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
841
21455
0f0bace82149 run-tests: move err file writing to tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21454
diff changeset
842 if (self._ret != 0 or self._out != self._refout) and not self._skipped \
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
843 and not self._debug and self._out:
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
844 f = open(self.errpath, 'wb')
21455
0f0bace82149 run-tests: move err file writing to tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21454
diff changeset
845 for line in self._out:
0f0bace82149 run-tests: move err file writing to tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21454
diff changeset
846 f.write(line)
0f0bace82149 run-tests: move err file writing to tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21454
diff changeset
847 f.close()
0f0bace82149 run-tests: move err file writing to tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21454
diff changeset
848
24926
3fe1e07f1a32 run-test: include test name in the return vlog
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
849 vlog("# Ret was:", self._ret, '(%s)' % self.name)
21452
1517c0461b75 run-tests: move some functionality to Test.tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21451
diff changeset
850
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
851 def _run(self, env):
21339
de25e968b4d8 run-tests: refactor runone() into gettest() and scheduletests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21338
diff changeset
852 # This should be implemented in child classes to run tests.
32932
240183a04429 tests: use unittest.SkipTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32907
diff changeset
853 raise unittest.SkipTest('unknown test type')
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
854
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
855 def abort(self):
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
856 """Terminate execution of this test."""
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
857 self._aborted = True
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
858
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
859 def _portmap(self, i):
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
860 offset = b'' if i == 0 else b'%d' % i
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
861 return (br':%d\b' % (self._startport + i), b':$HGPORT%s' % offset)
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
862
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
863 def _getreplacements(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
864 """Obtain a mapping of text replacements to apply to test output.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
865
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
866 Test output needs to be normalized so it can be compared to expected
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
867 output. This function defines how some of that normalization will
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
868 occur.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
869 """
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
870 r = [
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
871 # This list should be parallel to defineport in _getenv
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
872 self._portmap(0),
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
873 self._portmap(1),
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
874 self._portmap(2),
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
875 (br'(?m)^(saved backup bundle to .*\.hg)( \(glob\))?$',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
876 br'\1 (glob)'),
31008
636cf3f7620d tests: use LOCALIP
Jun Wu <quark@fb.com>
parents: 31006
diff changeset
877 (br'([^0-9])%s' % re.escape(self._localip()), br'\1$LOCALIP'),
31741
728d37353e1e run-tests: auto-replace 'TXNID' output
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31673
diff changeset
878 (br'\bHG_TXNID=TXN:[a-f0-9]{40}\b', br'HG_TXNID=TXN:$ID$'),
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
879 ]
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
880 r.append((self._escapepath(self._testtmp), b'$TESTTMP'))
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
881
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
882 return r
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
883
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
884 def _escapepath(self, p):
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
885 if os.name == 'nt':
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
886 return (
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
887 (b''.join(c.isalpha() and b'[%s%s]' % (c.lower(), c.upper()) or
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
888 c in b'/\\' and br'[/\\]' or c.isdigit() and c or b'\\' + c
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
889 for c in p))
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
890 )
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
891 else:
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
892 return re.escape(p)
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
893
31006
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
894 def _localip(self):
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
895 if self._useipv6:
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
896 return b'::1'
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
897 else:
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
898 return b'127.0.0.1'
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
899
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
900 def _genrestoreenv(self, testenv):
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
901 """Generate a script that can be used by tests to restore the original
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
902 environment."""
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
903 # Put the restoreenv script inside self._threadtmp
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
904 scriptpath = os.path.join(self._threadtmp, b'restoreenv.sh')
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
905 testenv['HGTEST_RESTOREENV'] = scriptpath
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
906
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
907 # Only restore environment variable names that the shell allows
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
908 # us to export.
33140
f458a6701983 tests: fix variable name regular expression in _genrestoreenv()
Adam Simpkins <simpkins@fb.com>
parents: 33126
diff changeset
909 name_regex = re.compile('^[a-zA-Z][a-zA-Z0-9_]*$')
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
910
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
911 # Do not restore these variables; otherwise tests would fail.
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
912 reqnames = {'PYTHON', 'TESTDIR', 'TESTTMP'}
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
913
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
914 with open(scriptpath, 'w') as envf:
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
915 for name, value in origenviron.items():
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
916 if not name_regex.match(name):
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
917 # Skip environment variables with unusual names not
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
918 # allowed by most shells.
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
919 continue
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
920 if name in reqnames:
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
921 continue
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
922 envf.write('%s=%s\n' % (name, shellquote(value)))
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
923
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
924 for name in testenv:
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
925 if name in origenviron or name in reqnames:
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
926 continue
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
927 envf.write('unset %s\n' % (name,))
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
928
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
929 def _getenv(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
930 """Obtain environment variables to use during test execution."""
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
931 def defineport(i):
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
932 offset = '' if i == 0 else '%s' % i
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
933 env["HGPORT%s" % offset] = '%s' % (self._startport + i)
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
934 env = os.environ.copy()
32302
8627cf4de929 run-tests: drop fallback for sysconfig for pre-py2.7
Martin von Zweigbergk <martinvonz@google.com>
parents: 31950
diff changeset
935 env['PYTHONUSERBASE'] = sysconfig.get_config_var('userbase')
31950
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31829
diff changeset
936 env['HGEMITWARNINGS'] = '1'
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
937 env['TESTTMP'] = self._testtmp
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
938 env['HOME'] = self._testtmp
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
939 # This number should match portneeded in _getport
28170
bc010fcd836b run-tests: stop allocating HGPORT3+HGPORT4
timeless <timeless@mozdev.org>
parents: 28169
diff changeset
940 for port in xrange(3):
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
941 # This list should be parallel to _portmap in _getreplacements
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
942 defineport(port)
25034
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
943 env["HGRCPATH"] = os.path.join(self._threadtmp, b'.hgrc')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
944 env["DAEMON_PIDS"] = os.path.join(self._threadtmp, b'daemon.pids')
23347
49cdf51cbc6c run-tests: include quotes in the HGEDITOR value when storing sys.executable
Matt Harbison <matt_harbison@yahoo.com>
parents: 23263
diff changeset
945 env["HGEDITOR"] = ('"' + sys.executable + '"'
49cdf51cbc6c run-tests: include quotes in the HGEDITOR value when storing sys.executable
Matt Harbison <matt_harbison@yahoo.com>
parents: 23263
diff changeset
946 + ' -c "import sys; sys.exit(0)"')
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
947 env["HGMERGE"] = "internal:merge"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
948 env["HGUSER"] = "test"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
949 env["HGENCODING"] = "ascii"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
950 env["HGENCODINGMODE"] = "strict"
31003
225f574e0645 runtests: export HGIPV6 to hint test scripts whether to use IPv6
Jun Wu <quark@fb.com>
parents: 31002
diff changeset
951 env['HGIPV6'] = str(int(self._useipv6))
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
952
31006
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
953 # LOCALIP could be ::1 or 127.0.0.1. Useful for tests that require raw
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
954 # IP addresses.
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
955 env['LOCALIP'] = self._localip()
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
956
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
957 # Reset some environment variables to well-known values so that
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
958 # the tests produce repeatable output.
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
959 env['LANG'] = env['LC_ALL'] = env['LANGUAGE'] = 'C'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
960 env['TZ'] = 'GMT'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
961 env["EMAIL"] = "Foo Bar <foo.bar@example.com>"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
962 env['COLUMNS'] = '80'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
963 env['TERM'] = 'xterm'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
964
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
965 for k in ('HG HGPROF CDPATH GREP_OPTIONS http_proxy no_proxy ' +
31635
632e07e72a22 runtests: unset editor and pager related environment variables
Jun Wu <quark@fb.com>
parents: 31227
diff changeset
966 'HGPLAIN HGPLAINEXCEPT EDITOR VISUAL PAGER ' +
30716
3de9df6ee5bf run-tests: unset CHGDEBUG
Jun Wu <quark@fb.com>
parents: 30420
diff changeset
967 'NO_PROXY CHGDEBUG').split():
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
968 if k in env:
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
969 del env[k]
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
970
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
971 # unset env related to hooks
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
972 for k in env.keys():
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
973 if k.startswith('HG_'):
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
974 del env[k]
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
975
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
976 if self._usechg:
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
977 env['CHGSOCKNAME'] = os.path.join(self._chgsockdir, b'server')
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
978
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
979 return env
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
980
21382
4b8ffe3abdd2 run-tests: move createhgrc into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21381
diff changeset
981 def _createhgrc(self, path):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
982 """Create an hgrc file for this test."""
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
983 hgrc = open(path, 'wb')
25034
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
984 hgrc.write(b'[ui]\n')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
985 hgrc.write(b'slash = True\n')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
986 hgrc.write(b'interactive = False\n')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
987 hgrc.write(b'mergemarkers = detailed\n')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
988 hgrc.write(b'promptecho = True\n')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
989 hgrc.write(b'[defaults]\n')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
990 hgrc.write(b'[devel]\n')
25290
8f88f768e24c devel: rename 'all' to 'all-warnings' (BC)
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25262
diff changeset
991 hgrc.write(b'all-warnings = true\n')
32410
f94d7e3e46a0 devel: activate default-date in tests
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 32353
diff changeset
992 hgrc.write(b'default-date = 0 0\n')
25034
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
993 hgrc.write(b'[largefiles]\n')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
994 hgrc.write(b'usercache = %s\n' %
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
995 (os.path.join(self._testtmp, b'.cache/largefiles')))
31009
161ab32b44a1 runtests: set web.address to localhost
Jun Wu <quark@fb.com>
parents: 31008
diff changeset
996 hgrc.write(b'[web]\n')
161ab32b44a1 runtests: set web.address to localhost
Jun Wu <quark@fb.com>
parents: 31008
diff changeset
997 hgrc.write(b'address = localhost\n')
31227
ac47df82bdba tests: make run-tests.py run on Python 3 again
Yuya Nishihara <yuya@tcha.org>
parents: 31048
diff changeset
998 hgrc.write(b'ipv6 = %s\n' % str(self._useipv6).encode('ascii'))
23388
42ed0780ec4b run-tests: set a default largefiles usercache in the default hgrc file
Matt Harbison <matt_harbison@yahoo.com>
parents: 23352
diff changeset
999
21515
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
1000 for opt in self._extraconfigopts:
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
1001 section, key = opt.split('.', 1)
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
1002 assert '=' in key, ('extra config opt %s must '
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
1003 'have an = for assignment' % opt)
25034
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
1004 hgrc.write(b'[%s]\n%s\n' % (section, key))
21382
4b8ffe3abdd2 run-tests: move createhgrc into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21381
diff changeset
1005 hgrc.close()
4b8ffe3abdd2 run-tests: move createhgrc into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21381
diff changeset
1006
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
1007 def fail(self, msg):
21522
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
1008 # unittest differentiates between errored and failed.
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
1009 # Failed is denoted by AssertionError (by default at least).
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
1010 raise AssertionError(msg)
21323
a7c677e2f6ae run-tests: move fail() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21322
diff changeset
1011
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
1012 def _runcommand(self, cmd, env, normalizenewlines=False):
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1013 """Run command in a sub-process, capturing the output (stdout and
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1014 stderr).
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1015
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1016 Return a tuple (exitcode, output). output is None in debug mode.
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1017 """
24509
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
1018 if self._debug:
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
1019 proc = subprocess.Popen(cmd, shell=True, cwd=self._testtmp,
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
1020 env=env)
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1021 ret = proc.wait()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1022 return (ret, None)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1023
24509
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
1024 proc = Popen4(cmd, self._testtmp, self._timeout, env)
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1025 def cleanup():
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1026 terminate(proc)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1027 ret = proc.wait()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1028 if ret == 0:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1029 ret = signal.SIGTERM << 8
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1030 killdaemons(env['DAEMON_PIDS'])
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1031 return ret
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1032
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1033 output = ''
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1034 proc.tochild.close()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1035
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1036 try:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1037 output = proc.fromchild.read()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1038 except KeyboardInterrupt:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1039 vlog('# Handling keyboard interrupt')
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1040 cleanup()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1041 raise
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1042
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1043 ret = proc.wait()
25177
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
1044 if wifexited(ret):
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1045 ret = os.WEXITSTATUS(ret)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1046
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1047 if proc.timeout:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1048 ret = 'timeout'
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1049
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1050 if ret:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1051 killdaemons(env['DAEMON_PIDS'])
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1052
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
1053 for s, r in self._getreplacements():
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1054 output = re.sub(s, r, output)
24510
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
1055
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
1056 if normalizenewlines:
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
1057 output = output.replace('\r\n', '\n')
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
1058
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1059 return ret, output.splitlines(True)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
1060
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
1061 class PythonTest(Test):
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
1062 """A Python-based test."""
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
1063
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
1064 @property
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
1065 def refpath(self):
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
1066 return os.path.join(self._testdir, b'%s.out' % self.bname)
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
1067
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
1068 def _run(self, env):
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
1069 py3kswitch = self._py3kwarnings and b' -3' or b''
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
1070 cmd = b'%s%s "%s"' % (PYTHON, py3kswitch, self.path)
21311
f9a7018a35ff run-tests: roll pytest() into PythonTest._run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21310
diff changeset
1071 vlog("# Running", cmd)
24510
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
1072 normalizenewlines = os.name == 'nt'
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
1073 result = self._runcommand(cmd, env,
24510
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
1074 normalizenewlines=normalizenewlines)
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1075 if self._aborted:
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1076 raise KeyboardInterrupt()
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1077
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1078 return result
21311
f9a7018a35ff run-tests: roll pytest() into PythonTest._run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21310
diff changeset
1079
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1080 # Some glob patterns apply only in some circumstances, so the script
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1081 # might want to remove (glob) annotations that otherwise should be
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1082 # retained.
23352
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
1083 checkcodeglobpats = [
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1084 # On Windows it looks like \ doesn't require a (glob), but we know
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1085 # better.
25059
0e0f1068b878 run-tests: fix checking a line to see if it needs globbing
Augie Fackler <augie@google.com>
parents: 25058
diff changeset
1086 re.compile(br'^pushing to \$TESTTMP/.*[^)]$'),
0e0f1068b878 run-tests: fix checking a line to see if it needs globbing
Augie Fackler <augie@google.com>
parents: 25058
diff changeset
1087 re.compile(br'^moving \S+/.*[^)]$'),
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1088 re.compile(br'^pulling from \$TESTTMP/.*[^)]$'),
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1089 # Not all platforms have 127.0.0.1 as loopback (though most do),
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
1090 # so we always glob that too.
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 31635
diff changeset
1091 re.compile(br'.*\$LOCALIP.*$'),
23352
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
1092 ]
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
1093
25036
61fc2cdbc57c run-tests: work around chr() producing unicode in Python 3
Augie Fackler <augie@google.com>
parents: 25035
diff changeset
1094 bchr = chr
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
1095 if PYTHON3:
25036
61fc2cdbc57c run-tests: work around chr() producing unicode in Python 3
Augie Fackler <augie@google.com>
parents: 25035
diff changeset
1096 bchr = lambda x: bytes([x])
61fc2cdbc57c run-tests: work around chr() producing unicode in Python 3
Augie Fackler <augie@google.com>
parents: 25035
diff changeset
1097
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
1098 class TTest(Test):
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
1099 """A "t test" is a test backed by a .t file."""
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
1100
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
1101 SKIPPED_PREFIX = b'skipped: '
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
1102 FAILED_PREFIX = b'hghave check failed: '
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1103 NEEDESCAPE = re.compile(br'[\x00-\x08\x0b-\x1f\x7f-\xff]').search
21384
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1104
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1105 ESCAPESUB = re.compile(br'[\x00-\x08\x0b-\x1f\\\x7f-\xff]').sub
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1106 ESCAPEMAP = dict((bchr(i), br'\x%02x' % i) for i in range(256))
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1107 ESCAPEMAP.update({b'\\': b'\\\\', b'\r': br'\r'})
21381
9aa5784992d4 run-tests: move SKIPPED_PREFIX and FAILED_PREFIX into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21380
diff changeset
1108
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1109 def __init__(self, path, *args, **kwds):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1110 # accept an extra "case" parameter
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1111 case = None
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1112 if 'case' in kwds:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1113 case = kwds.pop('case')
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1114 self._case = case
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1115 self._allcases = parsettestcases(path)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1116 super(TTest, self).__init__(path, *args, **kwds)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1117 if case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1118 self.name = '%s (case %s)' % (self.name, _strpath(case))
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1119 self.errpath = b'%s.%s.err' % (self.errpath[:-4], case)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1120 self._tmpname += b'-%s' % case
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1121
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
1122 @property
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
1123 def refpath(self):
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1124 return os.path.join(self._testdir, self.bname)
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
1125
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
1126 def _run(self, env):
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
1127 f = open(self.path, 'rb')
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1128 lines = f.readlines()
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1129 f.close()
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1130
32981
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
1131 # .t file is both reference output and the test input, keep reference
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
1132 # output updated with the the test input. This avoids some race
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
1133 # conditions where the reference output does not match the actual test.
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
1134 if self._refout is not None:
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
1135 self._refout = lines
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
1136
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
1137 salt, script, after, expected = self._parsetest(lines)
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1138
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1139 # Write out the generated script.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1140 fname = b'%s.sh' % self._testtmp
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
1141 f = open(fname, 'wb')
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1142 for l in script:
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1143 f.write(l)
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1144 f.close()
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1145
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1146 cmd = b'%s "%s"' % (self._shell, fname)
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1147 vlog("# Running", cmd)
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1148
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
1149 exitcode, output = self._runcommand(cmd, env)
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1150
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1151 if self._aborted:
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1152 raise KeyboardInterrupt()
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1153
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1154 # Do not merge output if skipped. Return hghave message instead.
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1155 # Similarly, with --debug, output is None.
21380
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
1156 if exitcode == self.SKIPPED_STATUS or output is None:
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1157 return exitcode, output
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
1158
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1159 return self._processoutput(exitcode, output, salt, after, expected)
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
1160
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
1161 def _hghave(self, reqs):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1162 # TODO do something smarter when all other uses of hghave are gone.
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25388
diff changeset
1163 runtestdir = os.path.abspath(os.path.dirname(_bytespath(__file__)))
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25388
diff changeset
1164 tdir = runtestdir.replace(b'\\', b'/')
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1165 proc = Popen4(b'%s -c "%s/hghave %s"' %
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1166 (self._shell, tdir, b' '.join(reqs)),
23933
769027075e21 run-tests.py: execute hghave with same env vars as ones for actual tests
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23859
diff changeset
1167 self._testtmp, 0, self._getenv())
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1168 stdout, stderr = proc.communicate()
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1169 ret = proc.wait()
25177
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
1170 if wifexited(ret):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1171 ret = os.WEXITSTATUS(ret)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1172 if ret == 2:
28699
5cc59dbd199f py3: convert hghave output to text
timeless <timeless@mozdev.org>
parents: 28698
diff changeset
1173 print(stdout.decode('utf-8'))
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1174 sys.exit(1)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1175
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
1176 if ret != 0:
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27396
diff changeset
1177 return False, stdout
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
1178
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
1179 if 'slow' in reqs:
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
1180 self._timeout = self._slowtimeout
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27396
diff changeset
1181 return True, None
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1182
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1183 def _iftest(self, args):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1184 # implements "#if"
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1185 reqs = []
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1186 for arg in args:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1187 if arg.startswith(b'no-') and arg[3:] in self._allcases:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1188 if arg[3:] == self._case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1189 return False
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1190 elif arg in self._allcases:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1191 if arg != self._case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1192 return False
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1193 else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1194 reqs.append(arg)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1195 return self._hghave(reqs)[0]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1196
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
1197 def _parsetest(self, lines):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1198 # We generate a shell script which outputs unique markers to line
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1199 # up script results with our source. These markers include input
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1200 # line number and the last return code.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1201 salt = b"SALT%d" % time.time()
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1202 def addsalt(line, inpython):
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1203 if inpython:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1204 script.append(b'%s %d 0\n' % (salt, line))
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1205 else:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1206 script.append(b'echo %s %d $?\n' % (salt, line))
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1207
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1208 script = []
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1209
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1210 # After we run the shell script, we re-unify the script output
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1211 # with non-active parts of the source, with synchronization by our
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1212 # SALT line number markers. The after table contains the non-active
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1213 # components, ordered by line number.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1214 after = {}
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1215
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1216 # Expected shell script output.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1217 expected = {}
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1218
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1219 pos = prepos = -1
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1220
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1221 # True or False when in a true or false conditional section
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1222 skipping = None
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1223
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1224 # We keep track of whether or not we're in a Python block so we
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1225 # can generate the surrounding doctest magic.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1226 inpython = False
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1227
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
1228 if self._debug:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
1229 script.append(b'set -x\n')
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
1230 if self._hgcommand != b'hg':
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
1231 script.append(b'alias hg="%s"\n' % self._hgcommand)
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1232 if os.getenv('MSYSTEM'):
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
1233 script.append(b'alias pwd="pwd -W"\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1234
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
1235 n = 0
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1236 for n, l in enumerate(lines):
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1237 if not l.endswith(b'\n'):
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1238 l += b'\n'
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1239 if l.startswith(b'#require'):
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
1240 lsplit = l.split()
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1241 if len(lsplit) < 2 or lsplit[0] != b'#require':
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
1242 after.setdefault(pos, []).append(' !!! invalid #require\n')
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27396
diff changeset
1243 haveresult, message = self._hghave(lsplit[1:])
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27396
diff changeset
1244 if not haveresult:
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27396
diff changeset
1245 script = [b'echo "%s"\nexit 80\n' % message]
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
1246 break
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
1247 after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1248 elif l.startswith(b'#if'):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1249 lsplit = l.split()
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1250 if len(lsplit) < 2 or lsplit[0] != b'#if':
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1251 after.setdefault(pos, []).append(' !!! invalid #if\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1252 if skipping is not None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1253 after.setdefault(pos, []).append(' !!! nested #if\n')
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
1254 skipping = not self._iftest(lsplit[1:])
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1255 after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1256 elif l.startswith(b'#else'):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1257 if skipping is None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1258 after.setdefault(pos, []).append(' !!! missing #if\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1259 skipping = not skipping
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1260 after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1261 elif l.startswith(b'#endif'):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1262 if skipping is None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1263 after.setdefault(pos, []).append(' !!! missing #if\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1264 skipping = None
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1265 after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1266 elif skipping:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1267 after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1268 elif l.startswith(b' >>> '): # python inlines
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1269 after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1270 prepos = pos
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1271 pos = n
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1272 if not inpython:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1273 # We've just entered a Python block. Add the header.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1274 inpython = True
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1275 addsalt(prepos, False) # Make sure we report the exit code.
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
1276 script.append(b'%s -m heredoctest <<EOF\n' % PYTHON)
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1277 addsalt(n, True)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1278 script.append(l[2:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1279 elif l.startswith(b' ... '): # python inlines
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1280 after.setdefault(prepos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1281 script.append(l[2:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1282 elif l.startswith(b' $ '): # commands
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1283 if inpython:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
1284 script.append(b'EOF\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1285 inpython = False
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1286 after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1287 prepos = pos
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1288 pos = n
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1289 addsalt(n, False)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1290 cmd = l[4:].split()
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
1291 if len(cmd) == 2 and cmd[0] == b'cd':
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
1292 l = b' $ cd %s || exit 1\n' % cmd[1]
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1293 script.append(l[4:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1294 elif l.startswith(b' > '): # continuations
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1295 after.setdefault(prepos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1296 script.append(l[4:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
1297 elif l.startswith(b' '): # results
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1298 # Queue up a list of expected results.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1299 expected.setdefault(pos, []).append(l[2:])
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1300 else:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1301 if inpython:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
1302 script.append(b'EOF\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1303 inpython = False
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1304 # Non-command/result. Queue up for merged output.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1305 after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1306
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1307 if inpython:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
1308 script.append(b'EOF\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1309 if skipping is not None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1310 after.setdefault(pos, []).append(' !!! missing #endif\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1311 addsalt(n + 1, False)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1313 return salt, script, after, expected
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1314
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1315 def _processoutput(self, exitcode, output, salt, after, expected):
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1316 # Merge the script output back into a unified test.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1317 warnonly = 1 # 1: not yet; 2: yes; 3: for sure not
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1318 if exitcode != 0:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1319 warnonly = 3
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1320
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1321 pos = -1
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1322 postout = []
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1323 for l in output:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1324 lout, lcmd = l, None
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1325 if salt in l:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1326 lout, lcmd = l.split(salt, 1)
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1327
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
1328 while lout:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1329 if not lout.endswith(b'\n'):
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1330 lout += b' (no-eol)\n'
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1331
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1332 # Find the expected output at the current position.
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1333 els = [None]
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1334 if expected.get(pos, None):
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1335 els = expected[pos]
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1336
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1337 i = 0
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
1338 optional = []
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1339 while i < len(els):
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1340 el = els[i]
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1341
28568
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1342 r = TTest.linematch(el, lout)
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1343 if isinstance(r, str):
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1344 if r == '+glob':
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1345 lout = el[:-1] + ' (glob)\n'
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1346 r = '' # Warn only this line.
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1347 elif r == '-glob':
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1348 lout = ''.join(el.rsplit(' (glob)', 1))
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1349 r = '' # Warn only this line.
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1350 elif r == "retry":
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1351 postout.append(b' ' + el)
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1352 els.pop(i)
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1353 break
28568
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1354 else:
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1355 log('\ninfo, unknown linematch result: %r\n' % r)
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
1356 r = False
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1357 if r:
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1358 els.pop(i)
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1359 break
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1360 if el:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1361 if el.endswith(b" (?)\n"):
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1362 optional.append(i)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1363 else:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1364 m = optline.match(el)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1365 if m:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1366 conditions = [c for c in m.group(2).split(' ')]
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1367
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1368 if self._hghave(conditions)[0]:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1369 lout = el
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1370 else:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1371 optional.append(i)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1372
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1373 i += 1
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1374
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1375 if r:
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1376 if r == "retry":
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
1377 continue
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
1378 # clean up any optional leftovers
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
1379 for i in optional:
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
1380 postout.append(b' ' + els[i])
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
1381 for i in reversed(optional):
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
1382 del els[i]
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1383 postout.append(b' ' + el)
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1384 else:
21384
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1385 if self.NEEDESCAPE(lout):
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1386 lout = TTest._stringescape(b'%s (esc)\n' %
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1387 lout.rstrip(b'\n'))
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1388 postout.append(b' ' + lout) # Let diff deal with it.
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1389 if r != '': # If line failed.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1390 warnonly = 3 # for sure not
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1391 elif warnonly == 1: # Is "not yet" and line is warn only.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1392 warnonly = 2 # Yes do warn.
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
1393 break
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
1394 else:
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
1395 # clean up any optional leftovers
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
1396 while expected.get(pos, None):
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
1397 el = expected[pos].pop(0)
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1398 if el:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1399 if (not optline.match(el)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1400 and not el.endswith(b" (?)\n")):
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1401 break
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
1402 postout.append(b' ' + el)
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1403
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1404 if lcmd:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1405 # Add on last return code.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1406 ret = int(lcmd.split()[1])
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1407 if ret != 0:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1408 postout.append(b' [%d]\n' % ret)
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1409 if pos in after:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1410 # Merge in non-active test bits.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1411 postout += after.pop(pos)
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1412 pos = int(lcmd.split()[0])
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1413
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1414 if pos in after:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1415 postout += after.pop(pos)
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1416
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1417 if warnonly == 2:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1418 exitcode = False # Set exitcode to warned.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1419
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
1420 return exitcode, postout
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
1421
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
1422 @staticmethod
21316
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1423 def rematch(el, l):
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1424 try:
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1425 # use \Z to ensure that the regex matches to the end of the string
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1426 if os.name == 'nt':
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1427 return re.match(el + br'\r?\n\Z', l)
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1428 return re.match(el + br'\n\Z', l)
21316
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1429 except re.error:
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1430 # el is an invalid regex
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1431 return False
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1432
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
1433 @staticmethod
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1434 def globmatch(el, l):
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1435 # The only supported special characters are * and ? plus / which also
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1436 # matches \ on windows. Escaping of these characters is supported.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1437 if el + b'\n' == l:
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1438 if os.altsep:
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1439 # matching on "/" is not needed for this line
23352
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
1440 for pat in checkcodeglobpats:
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
1441 if pat.match(el):
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
1442 return True
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1443 return b'-glob'
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1444 return True
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 31635
diff changeset
1445 el = el.replace(b'$LOCALIP', b'*')
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1446 i, n = 0, len(el)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1447 res = b''
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1448 while i < n:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1449 c = el[i:i + 1]
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1450 i += 1
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1451 if c == b'\\' and i < n and el[i:i + 1] in b'*?\\/':
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1452 res += el[i - 1:i + 1]
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1453 i += 1
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1454 elif c == b'*':
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1455 res += b'.*'
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1456 elif c == b'?':
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1457 res += b'.'
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1458 elif c == b'/' and os.altsep:
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1459 res += b'[/\\\\]'
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1460 else:
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1461 res += re.escape(c)
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1462 return TTest.rematch(res, l)
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1463
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
1464 @staticmethod
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
1465 def linematch(el, l):
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
1466 retry = False
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
1467 if el == l: # perfect match (fast)
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
1468 return True
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
1469 if el:
26612
a38924f7680c run-tests: add b-prefix on two strings to fix python3 support
Augie Fackler <augie@google.com>
parents: 26422
diff changeset
1470 if el.endswith(b" (?)\n"):
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
1471 retry = "retry"
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
1472 el = el[:-5] + b"\n"
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1473 else:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1474 m = optline.match(el)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1475 if m:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1476 el = m.group(1) + b"\n"
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1477 retry = "retry"
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
1478
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1479 if el.endswith(b" (esc)\n"):
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
1480 if PYTHON3:
25047
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
1481 el = el[:-7].decode('unicode_escape') + '\n'
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
1482 el = el.encode('utf-8')
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
1483 else:
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
1484 el = el[:-7].decode('string-escape') + '\n'
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1485 if el == l or os.name == 'nt' and el[:-1] + b'\r\n' == l:
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
1486 return True
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1487 if el.endswith(b" (re)\n"):
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
1488 return TTest.rematch(el[:-6], l) or retry
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1489 if el.endswith(b" (glob)\n"):
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23388
diff changeset
1490 # ignore '(glob)' added to l by 'replacements'
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1491 if l.endswith(b" (glob)\n"):
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1492 l = l[:-8] + b"\n"
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31741
diff changeset
1493 return TTest.globmatch(el[:-8], l) or retry
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1494 if os.altsep and l.replace(b'\\', b'/') == el:
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
1495 return b'+glob'
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
1496 return retry
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
1497
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1498 @staticmethod
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1499 def parsehghaveoutput(lines):
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1500 '''Parse hghave log lines.
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1501
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1502 Return tuple of lists (missing, failed):
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1503 * the missing/unknown features
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1504 * the features for which existence check failed'''
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1505 missing = []
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1506 failed = []
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1507 for line in lines:
21381
9aa5784992d4 run-tests: move SKIPPED_PREFIX and FAILED_PREFIX into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21380
diff changeset
1508 if line.startswith(TTest.SKIPPED_PREFIX):
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1509 line = line.splitlines()[0]
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
1510 missing.append(line[len(TTest.SKIPPED_PREFIX):].decode('utf-8'))
21381
9aa5784992d4 run-tests: move SKIPPED_PREFIX and FAILED_PREFIX into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21380
diff changeset
1511 elif line.startswith(TTest.FAILED_PREFIX):
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1512 line = line.splitlines()[0]
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
1513 failed.append(line[len(TTest.FAILED_PREFIX):].decode('utf-8'))
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1514
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1515 return missing, failed
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
1516
21384
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1517 @staticmethod
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1518 def _escapef(m):
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1519 return TTest.ESCAPEMAP[m.group(0)]
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1520
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1521 @staticmethod
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1522 def _stringescape(s):
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1523 return TTest.ESCAPESUB(TTest._escapef, s)
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
1524
22104
70bdf59d27b6 run-tests: attempt to fix iolock handling
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
1525 iolock = threading.RLock()
14000
636a6f5aa2cd run-tests: add locking on results struct
Matt Mackall <mpm@selenic.com>
parents: 13999
diff changeset
1526
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1527 class TestResult(unittest._TextTestResult):
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1528 """Holds results when executing via unittest."""
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1529 # Don't worry too much about accessing the non-public _TextTestResult.
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1530 # It is relatively common in Python testing tools.
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1531 def __init__(self, options, *args, **kwargs):
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1532 super(TestResult, self).__init__(*args, **kwargs)
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1533
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1534 self._options = options
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1535
21430
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1536 # unittest.TestResult didn't have skipped until 2.7. We need to
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1537 # polyfill it.
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1538 self.skipped = []
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1539
21431
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1540 # We have a custom "ignored" result that isn't present in any Python
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1541 # unittest implementation. It is very similar to skipped. It may make
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1542 # sense to map it into skip some day.
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1543 self.ignored = []
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1544
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1545 self.times = []
27637
b502138f5faa cleanup: remove superfluous space after space after equals (python)
timeless <timeless@mozdev.org>
parents: 27636
diff changeset
1546 self._firststarttime = None
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1547 # Data stored for the benefit of generating xunit reports.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1548 self.successes = []
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1549 self.faildata = {}
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1550
21462
8a4ef661f08d run-tests: make failure reporting in unittest mode equivalent to default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21461
diff changeset
1551 def addFailure(self, test, reason):
8a4ef661f08d run-tests: make failure reporting in unittest mode equivalent to default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21461
diff changeset
1552 self.failures.append((test, reason))
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1553
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1554 if self._options.first:
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1555 self.stop()
21735
5ee547fdb0be run-tests: produce error on running a failing test
anuraggoel <anurag.dsps@gmail.com>
parents: 21733
diff changeset
1556 else:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1557 with iolock:
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
1558 if reason == "timed out":
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
1559 self.stream.write('t')
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
1560 else:
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
1561 if not self._options.nodiff:
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
1562 self.stream.write('\nERROR: %s output changed\n' % test)
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
1563 self.stream.write('!')
21754
7e14d026c4c4 run-tests: fixes the '--interactive' option error
anuraggoel <anurag.dsps@gmail.com>
parents: 21753
diff changeset
1564
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1565 self.stream.flush()
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1566
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1567 def addSuccess(self, test):
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1568 with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1569 super(TestResult, self).addSuccess(test)
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1570 self.successes.append(test)
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1571
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1572 def addError(self, test, err):
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1573 super(TestResult, self).addError(test, err)
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1574 if self._options.first:
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1575 self.stop()
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1576
21430
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1577 # Polyfill.
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1578 def addSkip(self, test, reason):
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1579 self.skipped.append((test, reason))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1580 with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1581 if self.showAll:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1582 self.stream.writeln('skipped %s' % reason)
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1583 else:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1584 self.stream.write('s')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1585 self.stream.flush()
21430
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
1586
21431
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1587 def addIgnore(self, test, reason):
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1588 self.ignored.append((test, reason))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1589 with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1590 if self.showAll:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1591 self.stream.writeln('ignored %s' % reason)
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
1592 else:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1593 if reason not in ('not retesting', "doesn't match keyword"):
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1594 self.stream.write('i')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1595 else:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1596 self.testsRun += 1
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1597 self.stream.flush()
21431
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
1598
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
1599 def addOutputMismatch(self, test, ret, got, expected):
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
1600 """Record a mismatch in test output for a particular test."""
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
1601 if self.shouldStop:
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
1602 # don't print, some other test case already failed and
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
1603 # printed, we're just stale and probably failed due to our
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
1604 # temp dir getting cleaned up.
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
1605 return
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
1606
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
1607 accepted = False
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1608 lines = []
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
1609
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1610 with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1611 if self._options.nodiff:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1612 pass
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1613 elif self._options.view:
25056
e5f6c6ec21b8 run-tests: be more paranoid about os.system using bytes
Augie Fackler <augie@google.com>
parents: 25055
diff changeset
1614 v = self._options.view
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
1615 if PYTHON3:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
1616 v = _bytespath(v)
25056
e5f6c6ec21b8 run-tests: be more paranoid about os.system using bytes
Augie Fackler <augie@google.com>
parents: 25055
diff changeset
1617 os.system(b"%s %s %s" %
e5f6c6ec21b8 run-tests: be more paranoid about os.system using bytes
Augie Fackler <augie@google.com>
parents: 25055
diff changeset
1618 (v, test.refpath, test.errpath))
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
1619 else:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1620 servefail, lines = getdiff(expected, got,
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1621 test.refpath, test.errpath)
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1622 if servefail:
32943
750c3b1bb8a3 tests: removed ReportedTest exception
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32942
diff changeset
1623 raise test.failureException(
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1624 'server failed to start (HGPORT=%s)' % test._startport)
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1625 else:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1626 self.stream.write('\n')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1627 for line in lines:
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
1628 if PYTHON3:
25053
4f2c74ef8128 run-tests: write bytes to the binary buffer on sys.{stdout,stderr}
Augie Fackler <augie@google.com>
parents: 25052
diff changeset
1629 self.stream.flush()
4f2c74ef8128 run-tests: write bytes to the binary buffer on sys.{stdout,stderr}
Augie Fackler <augie@google.com>
parents: 25052
diff changeset
1630 self.stream.buffer.write(line)
4f2c74ef8128 run-tests: write bytes to the binary buffer on sys.{stdout,stderr}
Augie Fackler <augie@google.com>
parents: 25052
diff changeset
1631 self.stream.buffer.flush()
4f2c74ef8128 run-tests: write bytes to the binary buffer on sys.{stdout,stderr}
Augie Fackler <augie@google.com>
parents: 25052
diff changeset
1632 else:
4f2c74ef8128 run-tests: write bytes to the binary buffer on sys.{stdout,stderr}
Augie Fackler <augie@google.com>
parents: 25052
diff changeset
1633 self.stream.write(line)
4f2c74ef8128 run-tests: write bytes to the binary buffer on sys.{stdout,stderr}
Augie Fackler <augie@google.com>
parents: 25052
diff changeset
1634 self.stream.flush()
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
1635
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1636 # handle interactive prompt without releasing iolock
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1637 if self._options.interactive:
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1638 if test.readrefout() != expected:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1639 self.stream.write(
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1640 'Reference output has changed (run again to prompt '
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1641 'changes)')
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1642 else:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1643 self.stream.write('Accept this change? [n] ')
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1644 answer = sys.stdin.readline().strip()
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1645 if answer.lower() in ('y', 'yes'):
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32981
diff changeset
1646 if test.path.endswith(b'.t'):
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1647 rename(test.errpath, test.path)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1648 else:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1649 rename(test.errpath, '%s.out' % test.path)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
1650 accepted = True
28127
807bc140e915 run-tests: remove useless "failed" flag from addOutputMismatch()
Yuya Nishihara <yuya@tcha.org>
parents: 28126
diff changeset
1651 if not accepted:
25052
c4217a046b62 run-tests: record faildata using bytes instead of str
Augie Fackler <augie@google.com>
parents: 25051
diff changeset
1652 self.faildata[test.name] = b''.join(lines)
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
1653
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
1654 return accepted
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
1655
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1656 def startTest(self, test):
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1657 super(TestResult, self).startTest(test)
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1658
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
1659 # os.times module computes the user time and system time spent by
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
1660 # child's processes along with real elapsed time taken by a process.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
1661 # This module has one limitation. It can only work for Linux user
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
1662 # and not for Windows.
24331
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
1663 test.started = os.times()
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
1664 if self._firststarttime is None: # thread racy but irrelevant
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
1665 self._firststarttime = test.started[4]
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1666
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1667 def stopTest(self, test, interrupted=False):
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1668 super(TestResult, self).stopTest(test)
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1669
24331
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
1670 test.stopped = os.times()
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
1671
24331
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
1672 starttime = test.started
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
1673 endtime = test.stopped
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
1674 origin = self._firststarttime
24984
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
1675 self.times.append((test.name,
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
1676 endtime[2] - starttime[2], # user space CPU time
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
1677 endtime[3] - starttime[3], # sys space CPU time
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
1678 endtime[4] - starttime[4], # real time
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
1679 starttime[4] - origin, # start date in run context
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
1680 endtime[4] - origin, # end date in run context
24984
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
1681 ))
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
1682
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1683 if interrupted:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1684 with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1685 self.stream.writeln('INTERRUPTED: %s (after %d seconds)' % (
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1686 test.name, self.times[-1][3]))
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
1687
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1688 class TestSuite(unittest.TestSuite):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
1689 """Custom unittest TestSuite that knows how to execute Mercurial tests."""
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
1690
21533
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
1691 def __init__(self, testdir, jobs=1, whitelist=None, blacklist=None,
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
1692 retest=False, keywords=None, loop=False, runs_per_test=1,
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1693 loadtest=None, showchannels=False,
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1694 *args, **kwargs):
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
1695 """Create a new instance that can run tests with a configuration.
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1696
21533
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
1697 testdir specifies the directory where tests are executed from. This
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
1698 is typically the ``tests`` directory from Mercurial's source
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
1699 repository.
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
1700
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
1701 jobs specifies the number of jobs to run concurrently. Each test
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
1702 executes on its own thread. Tests actually spawn new processes, so
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
1703 state mutation should not be an issue.
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1704
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1705 If there is only one job, it will use the main thread.
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1706
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1707 whitelist and blacklist denote tests that have been whitelisted and
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1708 blacklisted, respectively. These arguments don't belong in TestSuite.
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1709 Instead, whitelist and blacklist should be handled by the thing that
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1710 populates the TestSuite with tests. They are present to preserve
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1711 backwards compatible behavior which reports skipped tests as part
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1712 of the results.
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
1713
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
1714 retest denotes whether to retest failed tests. This arguably belongs
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
1715 outside of TestSuite.
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
1716
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
1717 keywords denotes key words that will be used to filter which tests
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
1718 to execute. This arguably belongs outside of TestSuite.
21532
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
1719
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
1720 loop denotes whether to loop over tests forever.
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
1721 """
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1722 super(TestSuite, self).__init__(*args, **kwargs)
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1723
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
1724 self._jobs = jobs
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1725 self._whitelist = whitelist
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1726 self._blacklist = blacklist
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
1727 self._retest = retest
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
1728 self._keywords = keywords
21532
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
1729 self._loop = loop
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
1730 self._runs_per_test = runs_per_test
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
1731 self._loadtest = loadtest
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1732 self._showchannels = showchannels
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1733
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1734 def run(self, result):
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1735 # We have a number of filters that need to be applied. We do this
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1736 # here instead of inside Test because it makes the running logic for
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1737 # Test simpler.
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1738 tests = []
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
1739 num_tests = [0]
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1740 for test in self._tests:
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
1741 def get():
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
1742 num_tests[0] += 1
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
1743 if getattr(test, 'should_reload', False):
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
1744 return self._loadtest(test, num_tests[0])
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
1745 return test
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1746 if not os.path.exists(test.path):
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1747 result.addSkip(test, "Doesn't exist")
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1748 continue
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1749
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
1750 if not (self._whitelist and test.name in self._whitelist):
25055
d79258e30499 run-tests: blacklist entries are bytes, use bname to check blacklisting
Augie Fackler <augie@google.com>
parents: 25053
diff changeset
1751 if self._blacklist and test.bname in self._blacklist:
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1752 result.addSkip(test, 'blacklisted')
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1753 continue
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1754
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
1755 if self._retest and not os.path.exists(test.errpath):
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1756 result.addIgnore(test, 'not retesting')
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1757 continue
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1758
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
1759 if self._keywords:
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
1760 f = open(test.path, 'rb')
25048
6b8ddf896463 run-tests: refer to test.bname when sniffing for keywords
Augie Fackler <augie@google.com>
parents: 25047
diff changeset
1761 t = f.read().lower() + test.bname.lower()
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1762 f.close()
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1763 ignored = False
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
1764 for k in self._keywords.lower().split():
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1765 if k not in t:
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1766 result.addIgnore(test, "doesn't match keyword")
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1767 ignored = True
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1768 break
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1769
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1770 if ignored:
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
1771 continue
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
1772 for _ in xrange(self._runs_per_test):
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
1773 tests.append(get())
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1774
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1775 runtests = list(tests)
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1776 done = queue.Queue()
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1777 running = 0
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1778
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1779 channels = [""] * self._jobs
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1780
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1781 def job(test, result):
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1782 for n, v in enumerate(channels):
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1783 if not v:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1784 channel = n
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1785 break
32621
c59451e11cbf tests: make run-tests fail early if no channel is found
Augie Fackler <augie@google.com>
parents: 32414
diff changeset
1786 else:
c59451e11cbf tests: make run-tests fail early if no channel is found
Augie Fackler <augie@google.com>
parents: 32414
diff changeset
1787 raise ValueError('Could not find output channel')
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1788 channels[channel] = "=" + test.name[5:].split(".")[0]
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1789 try:
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1790 test(result)
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1791 done.put(None)
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1792 except KeyboardInterrupt:
27933
a6833e464b07 run-tests: "fix" race condition in race condition fix
Bryan O'Sullivan <bryano@fb.com>
parents: 27927
diff changeset
1793 pass
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1794 except: # re-raises
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1795 done.put(('!', test, 'run-test raised an error, see traceback'))
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1796 raise
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
1797 finally:
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
1798 try:
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
1799 channels[channel] = ''
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
1800 except IndexError:
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
1801 pass
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1802
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1803 def stat():
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1804 count = 0
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1805 while channels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1806 d = '\n%03s ' % count
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1807 for n, v in enumerate(channels):
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1808 if v:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1809 d += v[0]
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1810 channels[n] = v[1:] or '.'
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1811 else:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1812 d += ' '
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1813 d += ' '
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1814 with iolock:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1815 sys.stdout.write(d + ' ')
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1816 sys.stdout.flush()
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1817 for x in xrange(10):
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1818 if channels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1819 time.sleep(.1)
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1820 count += 1
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1821
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24506
diff changeset
1822 stoppedearly = False
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24506
diff changeset
1823
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1824 if self._showchannels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1825 statthread = threading.Thread(target=stat, name="stat")
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1826 statthread.start()
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1827
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1828 try:
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1829 while tests or running:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1830 if not done.empty() or running == self._jobs or not tests:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1831 try:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1832 done.get(True, 1)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1833 running -= 1
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1834 if result and result.shouldStop:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1835 stoppedearly = True
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1836 break
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1837 except queue.Empty:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1838 continue
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1839 if tests and not running == self._jobs:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1840 test = tests.pop(0)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1841 if self._loop:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1842 if getattr(test, 'should_reload', False):
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1843 num_tests[0] += 1
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1844 tests.append(
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
1845 self._loadtest(test, num_tests[0]))
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1846 else:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1847 tests.append(test)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1848 if self._jobs == 1:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1849 job(test, result)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1850 else:
27689
50e621fe0362 run-tests: skip threading for a single test
timeless <timeless@mozdev.org>
parents: 27686
diff changeset
1851 t = threading.Thread(target=job, name=test.name,
50e621fe0362 run-tests: skip threading for a single test
timeless <timeless@mozdev.org>
parents: 27686
diff changeset
1852 args=(test, result))
50e621fe0362 run-tests: skip threading for a single test
timeless <timeless@mozdev.org>
parents: 27686
diff changeset
1853 t.start()
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1854 running += 1
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24506
diff changeset
1855
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1856 # If we stop early we still need to wait on started tests to
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1857 # finish. Otherwise, there is a race between the test completing
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1858 # and the test's cleanup code running. This could result in the
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1859 # test reporting incorrect.
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1860 if stoppedearly:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1861 while running:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1862 try:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1863 done.get(True, 1)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1864 running -= 1
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1865 except queue.Empty:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
1866 continue
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
1867 except KeyboardInterrupt:
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1868 for test in runtests:
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
1869 test.abort()
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1870
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1871 channels = []
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
1872
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1873 return result
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
1874
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1875 # Save the most recent 5 wall-clock runtimes of each test to a
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1876 # human-readable text file named .testtimes. Tests are sorted
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1877 # alphabetically, while times for each test are listed from oldest to
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1878 # newest.
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1879
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
1880 def loadtimes(outputdir):
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1881 times = []
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1882 try:
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
1883 with open(os.path.join(outputdir, b'.testtimes-')) as fp:
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1884 for line in fp:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1885 ts = line.split()
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1886 times.append((ts[0], [float(t) for t in ts[1:]]))
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1887 except IOError as err:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1888 if err.errno != errno.ENOENT:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1889 raise
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1890 return times
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1891
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
1892 def savetimes(outputdir, result):
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
1893 saved = dict(loadtimes(outputdir))
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1894 maxruns = 5
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1895 skipped = set([str(t[0]) for t in result.skipped])
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1896 for tdata in result.times:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1897 test, real = tdata[0], tdata[3]
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1898 if test not in skipped:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1899 ts = saved.setdefault(test, [])
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1900 ts.append(real)
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1901 ts[:] = ts[-maxruns:]
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1902
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
1903 fd, tmpname = tempfile.mkstemp(prefix=b'.testtimes',
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
1904 dir=outputdir, text=True)
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1905 with os.fdopen(fd, 'w') as fp:
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
1906 for name, ts in sorted(saved.items()):
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1907 fp.write('%s %s\n' % (name, ' '.join(['%.3f' % (t,) for t in ts])))
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
1908 timepath = os.path.join(outputdir, b'.testtimes')
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1909 try:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1910 os.unlink(timepath)
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1911 except OSError:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1912 pass
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1913 try:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1914 os.rename(tmpname, timepath)
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1915 except OSError:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1916 pass
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
1917
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1918 class TextTestRunner(unittest.TextTestRunner):
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1919 """Custom unittest test runner that uses appropriate settings."""
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
1920
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1921 def __init__(self, runner, *args, **kwargs):
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1922 super(TextTestRunner, self).__init__(*args, **kwargs)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1923
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1924 self._runner = runner
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1925
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1926 def listtests(self, test):
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1927 result = TestResult(self._runner.options, self.stream,
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1928 self.descriptions, 0)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1929 test = sorted(test, key=lambda t: t.name)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1930 for t in test:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1931 print(t.name)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1932 result.addSuccess(t)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1933
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1934 if self._runner.options.xunit:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1935 with open(self._runner.options.xunit, "wb") as xuf:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1936 self._writexunit(result, xuf)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1937
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1938 if self._runner.options.json:
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
1939 jsonpath = os.path.join(self._runner._outputdir, b'report.json')
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1940 with open(jsonpath, 'w') as fp:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1941 self._writejson(result, fp)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1942
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1943 return result
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
1944
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1945 def run(self, test):
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1946 result = TestResult(self._runner.options, self.stream,
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
1947 self.descriptions, self.verbosity)
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1948
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1949 test(result)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1950
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1951 failed = len(result.failures)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1952 skipped = len(result.skipped)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1953 ignored = len(result.ignored)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1954
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1955 with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1956 self.stream.writeln('')
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1957
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1958 if not self._runner.options.noskips:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1959 for test, msg in result.skipped:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1960 self.stream.writeln('Skipped %s: %s' % (test.name, msg))
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1961 for test, msg in result.failures:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1962 self.stream.writeln('Failed %s: %s' % (test.name, msg))
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1963 for test, msg in result.errors:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1964 self.stream.writeln('Errored %s: %s' % (test.name, msg))
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1965
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1966 if self._runner.options.xunit:
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
1967 with open(self._runner.options.xunit, "wb") as xuf:
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
1968 self._writexunit(result, xuf)
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
1969
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1970 if self._runner.options.json:
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
1971 jsonpath = os.path.join(self._runner._outputdir, b'report.json')
27773
bf45edfa9d90 run-tests: use a context manager for file I/O
Bryan O'Sullivan <bryano@fb.com>
parents: 27689
diff changeset
1972 with open(jsonpath, 'w') as fp:
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
1973 self._writejson(result, fp)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
1974
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
1975 self._runner._checkhglib('Tested')
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
1976
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
1977 savetimes(self._runner._outputdir, result)
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1978
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1979 if failed and self._runner.options.known_good_rev:
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1980 def nooutput(args):
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1981 p = subprocess.Popen(args, stderr=subprocess.STDOUT,
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1982 stdout=subprocess.PIPE)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1983 p.stdout.read()
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1984 p.wait()
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1985 for test, msg in result.failures:
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1986 nooutput(['hg', 'bisect', '--reset']),
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1987 nooutput(['hg', 'bisect', '--bad', '.'])
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1988 nooutput(['hg', 'bisect', '--good',
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1989 self._runner.options.known_good_rev])
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1990 # TODO: we probably need to forward some options
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1991 # that alter hg's behavior inside the tests.
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1992 rtc = '%s %s %s' % (sys.executable, sys.argv[0], test)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1993 sub = subprocess.Popen(['hg', 'bisect', '--command', rtc],
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1994 stderr=subprocess.STDOUT,
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1995 stdout=subprocess.PIPE)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1996 data = sub.stdout.read()
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1997 sub.wait()
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1998 m = re.search(
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
1999 (r'\nThe first (?P<goodbad>bad|good) revision '
28823
c5565fc8848d run-tests: fix broken regular expression
Augie Fackler <augie@google.com>
parents: 28812
diff changeset
2000 r'is:\nchangeset: +\d+:(?P<node>[a-f0-9]+)\n.*\n'
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2001 r'summary: +(?P<summary>[^\n]+)\n'),
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2002 data, (re.MULTILINE | re.DOTALL))
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2003 if m is None:
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2004 self.stream.writeln(
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2005 'Failed to identify failure point for %s' % test)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2006 continue
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2007 dat = m.groupdict()
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2008 verb = 'broken' if dat['goodbad'] == 'bad' else 'fixed'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2009 self.stream.writeln(
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2010 '%s %s by %s (%s)' % (
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
2011 test, verb, dat['node'], dat['summary']))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
2012 self.stream.writeln(
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32934
diff changeset
2013 '# Ran %d tests, %d skipped, %d failed.'
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32934
diff changeset
2014 % (result.testsRun, skipped + ignored, failed))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
2015 if failed:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
2016 self.stream.writeln('python hash seed: %s' %
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
2017 os.environ['PYTHONHASHSEED'])
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
2018 if self._runner.options.time:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
2019 self.printtimes(result.times)
32907
bd77ac2bd23a run-tests: explicitly flush test runner output for Windows stability
Matt Harbison <matt_harbison@yahoo.com>
parents: 32853
diff changeset
2020 self.stream.flush()
22104
70bdf59d27b6 run-tests: attempt to fix iolock handling
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
2021
21613
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
2022 return result
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
2023
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
2024 def printtimes(self, times):
22104
70bdf59d27b6 run-tests: attempt to fix iolock handling
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
2025 # iolock held by run
21494
dcefc4091c86 run-tests: move outputtimes() into unittest runner class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21493
diff changeset
2026 self.stream.writeln('# Producing time report')
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
2027 times.sort(key=lambda t: (t[3]))
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
2028 cols = '%7.3f %7.3f %7.3f %7.3f %7.3f %s'
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
2029 self.stream.writeln('%-7s %-7s %-7s %-7s %-7s %s' %
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
2030 ('start', 'end', 'cuser', 'csys', 'real', 'Test'))
24982
5c15f7e0f52b run-tests: stop explicit expansion of time data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24981
diff changeset
2031 for tdata in times:
5c15f7e0f52b run-tests: stop explicit expansion of time data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24981
diff changeset
2032 test = tdata[0]
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
2033 cuser, csys, real, start, end = tdata[1:6]
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
2034 self.stream.writeln(cols % (start, end, cuser, csys, real, test))
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
2035
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2036 @staticmethod
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2037 def _writexunit(result, outf):
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2038 # See http://llg.cubic.org/docs/junit/ for a reference.
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2039 timesd = dict((t[0], t[3]) for t in result.times)
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2040 doc = minidom.Document()
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2041 s = doc.createElement('testsuite')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2042 s.setAttribute('name', 'run-tests')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2043 s.setAttribute('tests', str(result.testsRun))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2044 s.setAttribute('errors', "0") # TODO
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2045 s.setAttribute('failures', str(len(result.failures)))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2046 s.setAttribute('skipped', str(len(result.skipped) +
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2047 len(result.ignored)))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2048 doc.appendChild(s)
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2049 for tc in result.successes:
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2050 t = doc.createElement('testcase')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2051 t.setAttribute('name', tc.name)
32702
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
2052 tctime = timesd.get(tc.name)
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
2053 if tctime is not None:
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
2054 t.setAttribute('time', '%.3f' % tctime)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2055 s.appendChild(t)
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2056 for tc, err in sorted(result.faildata.items()):
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2057 t = doc.createElement('testcase')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2058 t.setAttribute('name', tc)
32702
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
2059 tctime = timesd.get(tc)
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
2060 if tctime is not None:
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
2061 t.setAttribute('time', '%.3f' % tctime)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2062 # createCDATASection expects a unicode or it will
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2063 # convert using default conversion rules, which will
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2064 # fail if string isn't ASCII.
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2065 err = cdatasafe(err).decode('utf-8', 'replace')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2066 cd = doc.createCDATASection(err)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2067 # Use 'failure' here instead of 'error' to match errors = 0,
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2068 # failures = len(result.failures) in the testsuite element.
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2069 failelem = doc.createElement('failure')
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2070 failelem.setAttribute('message', 'output changed')
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2071 failelem.setAttribute('type', 'output-mismatch')
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2072 failelem.appendChild(cd)
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
2073 t.appendChild(failelem)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2074 s.appendChild(t)
32715
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2075 for tc, message in result.skipped:
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2076 # According to the schema, 'skipped' has no attributes. So store
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2077 # the skip message as a text node instead.
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2078 t = doc.createElement('testcase')
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2079 t.setAttribute('name', tc.name)
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2080 message = cdatasafe(message).decode('utf-8', 'replace')
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2081 cd = doc.createCDATASection(message)
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2082 skipelem = doc.createElement('skipped')
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2083 skipelem.appendChild(cd)
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2084 t.appendChild(skipelem)
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
2085 s.appendChild(t)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2086 outf.write(doc.toprettyxml(indent=' ', encoding='utf-8'))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
2087
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2088 @staticmethod
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2089 def _writejson(result, outf):
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2090 timesd = {}
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2091 for tdata in result.times:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2092 test = tdata[0]
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2093 timesd[test] = tdata[1:]
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2094
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2095 outcome = {}
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2096 groups = [('success', ((tc, None)
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2097 for tc in result.successes)),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2098 ('failure', result.failures),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2099 ('skip', result.skipped)]
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2100 for res, testcases in groups:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2101 for tc, __ in testcases:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2102 if tc.name in timesd:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2103 diff = result.faildata.get(tc.name, b'')
32853
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
2104 try:
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
2105 diff = diff.decode('unicode_escape')
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
2106 except UnicodeDecodeError as e:
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
2107 diff = '%r decoding diff, sorry' % e
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2108 tres = {'result': res,
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2109 'time': ('%0.3f' % timesd[tc.name][2]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2110 'cuser': ('%0.3f' % timesd[tc.name][0]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2111 'csys': ('%0.3f' % timesd[tc.name][1]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2112 'start': ('%0.3f' % timesd[tc.name][3]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2113 'end': ('%0.3f' % timesd[tc.name][4]),
32853
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
2114 'diff': diff,
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2115 }
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2116 else:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2117 # blacklisted test
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2118 tres = {'result': res}
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2119
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2120 outcome[tc.name] = tres
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2121 jsonout = json.dumps(outcome, sort_keys=True, indent=4,
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2122 separators=(',', ': '))
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2123 outf.writelines(("testreport =", jsonout))
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
2124
21340
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
2125 class TestRunner(object):
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
2126 """Holds context for executing tests.
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
2127
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
2128 Tests rely on a lot of state. This object holds it for them.
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
2129 """
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2130
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2131 # Programs required to run tests.
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2132 REQUIREDTOOLS = [
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2133 b'diff',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2134 b'grep',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2135 b'unzip',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2136 b'gunzip',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2137 b'bunzip2',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2138 b'sed',
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2139 ]
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2140
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2141 # Maps file extensions to test class.
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2142 TESTTYPES = [
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2143 (b'.py', PythonTest),
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2144 (b'.t', TTest),
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2145 ]
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2146
21341
cb88d4a04f58 run-tests: move TESTDIR out of a global
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21340
diff changeset
2147 def __init__(self):
21348
b3399154505f run-tests: add options to runner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21347
diff changeset
2148 self.options = None
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
2149 self._hgroot = None
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2150 self._testdir = None
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
2151 self._outputdir = None
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2152 self._hgtmp = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2153 self._installdir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2154 self._bindir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2155 self._tmpbinddir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2156 self._pythondir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2157 self._coveragefile = None
21352
39fd89fbbc3c run-tests: move createdfiles out of a global and into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21351
diff changeset
2158 self._createdfiles = []
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
2159 self._hgcommand = None
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2160 self._hgpath = None
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2161 self._portoffset = 0
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2162 self._ports = {}
21340
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
2163
21376
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
2164 def run(self, args, parser=None):
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2165 """Run the test suite."""
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2166 oldmask = os.umask(0o22)
21375
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
2167 try:
21376
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
2168 parser = parser or getparser()
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
2169 options, args = parseargs(args, parser)
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2170 # positional arguments are paths to test files to run, so
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2171 # we make sure they're all bytestrings
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2172 args = [_bytespath(a) for a in args]
21376
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
2173 self.options = options
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
2174
21375
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
2175 self._checktools()
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2176 testdescs = self.findtests(args)
25107
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2177 if options.profile_runner:
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2178 import statprof
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2179 statprof.start()
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2180 result = self._run(testdescs)
25107
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2181 if options.profile_runner:
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2182 statprof.stop()
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2183 statprof.display()
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2184 return result
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
2185
21375
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
2186 finally:
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
2187 os.umask(oldmask)
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2188
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2189 def _run(self, testdescs):
21372
3a44787e50e2 run-tests: move test shuffling and sorting into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21371
diff changeset
2190 if self.options.random:
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2191 random.shuffle(testdescs)
21372
3a44787e50e2 run-tests: move test shuffling and sorting into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21371
diff changeset
2192 else:
3a44787e50e2 run-tests: move test shuffling and sorting into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21371
diff changeset
2193 # keywords for slow tests
25067
f52c5701925a run-tests: allow different extra weight for slow tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25060
diff changeset
2194 slow = {b'svn': 10,
27394
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2195 b'cvs': 10,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2196 b'hghave': 10,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2197 b'largefiles-update': 10,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2198 b'run-tests': 10,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2199 b'corruption': 10,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2200 b'race': 10,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2201 b'i18n': 10,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2202 b'check': 100,
b4d7743e174a run-tests: add more scheduling weight hints
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
2203 b'gendoc': 100,
27101
61fbf5dc12b2 test-contrib-perf: add smoke tests for perf.py
timeless <timeless@mozdev.org>
parents: 26612
diff changeset
2204 b'contrib-perf': 200,
25067
f52c5701925a run-tests: allow different extra weight for slow tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25060
diff changeset
2205 }
27635
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2206 perf = {}
21372
3a44787e50e2 run-tests: move test shuffling and sorting into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21371
diff changeset
2207 def sortkey(f):
3a44787e50e2 run-tests: move test shuffling and sorting into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21371
diff changeset
2208 # run largest tests first, as they tend to take the longest
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2209 f = f['path']
21372
3a44787e50e2 run-tests: move test shuffling and sorting into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21371
diff changeset
2210 try:
27635
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2211 return perf[f]
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2212 except KeyError:
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2213 try:
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2214 val = -os.stat(f).st_size
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2215 except OSError as e:
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2216 if e.errno != errno.ENOENT:
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2217 raise
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2218 perf[f] = -1e9 # file does not exist, tell early
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2219 return -1e9
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2220 for kw, mul in slow.items():
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2221 if kw in f:
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2222 val *= mul
28096
62ef44273678 run-tests: do not compare bytes with str while ordering tests
Yuya Nishihara <yuya@tcha.org>
parents: 28055
diff changeset
2223 if f.endswith(b'.py'):
27636
21b363bd87dc tests: make a stab at approximating wall-clock times
Bryan O'Sullivan <bos@serpentine.com>
parents: 27635
diff changeset
2224 val /= 10.0
21b363bd87dc tests: make a stab at approximating wall-clock times
Bryan O'Sullivan <bos@serpentine.com>
parents: 27635
diff changeset
2225 perf[f] = val / 1000.0
27635
e9f099ede405 tests: only stat a test file if we don't already know its "cost"
Bryan O'Sullivan <bos@serpentine.com>
parents: 27634
diff changeset
2226 return perf[f]
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2227 testdescs.sort(key=sortkey)
21372
3a44787e50e2 run-tests: move test shuffling and sorting into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21371
diff changeset
2228
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2229 self._testdir = osenvironb[b'TESTDIR'] = getattr(
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2230 os, 'getcwdb', os.getcwd)()
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
2231 if self.options.outputdir:
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
2232 self._outputdir = canonpath(_bytespath(self.options.outputdir))
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
2233 else:
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
2234 self._outputdir = self._testdir
21371
a10ba7870c2d run-tests: assign testdir in TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21370
diff changeset
2235
21370
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
2236 if 'PYTHONHASHSEED' not in os.environ:
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
2237 # use a random python hash seed all the time
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
2238 # we do the randomness ourself to know what seed is used
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
2239 os.environ['PYTHONHASHSEED'] = str(random.getrandbits(32))
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
2240
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2241 if self.options.tmpdir:
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2242 self.options.keep_tmpdir = True
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2243 tmpdir = _bytespath(self.options.tmpdir)
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2244 if os.path.exists(tmpdir):
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2245 # Meaning of tmpdir has changed since 1.3: we used to create
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2246 # HGTMP inside tmpdir; now HGTMP is tmpdir. So fail if
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2247 # tmpdir already exists.
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2248 print("error: temp dir %r already exists" % tmpdir)
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2249 return 1
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2250
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2251 # Automatically removing tmpdir sounds convenient, but could
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2252 # really annoy anyone in the habit of using "--tmpdir=/tmp"
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2253 # or "--tmpdir=$HOME".
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2254 #vlog("# Removing temp dir", tmpdir)
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2255 #shutil.rmtree(tmpdir)
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2256 os.makedirs(tmpdir)
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2257 else:
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2258 d = None
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2259 if os.name == 'nt':
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2260 # without this, we get the default temp dir location, but
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2261 # in all lowercase, which causes troubles with paths (issue3490)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2262 d = osenvironb.get(b'TMP', None)
25262
5a809deb88e6 run-tests: python3.5 now supports mkdtemp using bytes for paths
Augie Fackler <raf@durin42.com>
parents: 25261
diff changeset
2263 tmpdir = tempfile.mkdtemp(b'', b'hgtests.', d)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2264
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2265 self._hgtmp = osenvironb[b'HGTMP'] = (
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2266 os.path.realpath(tmpdir))
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
2267
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2268 if self.options.with_hg:
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2269 self._installdir = None
25042
201823c50610 run-tests: work around with_hg being bytes or string depending on provenance
Augie Fackler <augie@google.com>
parents: 25041
diff changeset
2270 whg = self.options.with_hg
201823c50610 run-tests: work around with_hg being bytes or string depending on provenance
Augie Fackler <augie@google.com>
parents: 25041
diff changeset
2271 self._bindir = os.path.dirname(os.path.realpath(whg))
201823c50610 run-tests: work around with_hg being bytes or string depending on provenance
Augie Fackler <augie@google.com>
parents: 25041
diff changeset
2272 assert isinstance(self._bindir, bytes)
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
2273 self._hgcommand = os.path.basename(whg)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2274 self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2275 os.makedirs(self._tmpbindir)
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2276
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2277 # This looks redundant with how Python initializes sys.path from
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2278 # the location of the script being executed. Needed because the
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2279 # "hg" specified by --with-hg is not the only Python script
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2280 # executed in the test suite that needs to import 'mercurial'
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2281 # ... which means it's not really redundant at all.
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2282 self._pythondir = self._bindir
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2283 else:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2284 self._installdir = os.path.join(self._hgtmp, b"install")
28098
d7b0e81b84c2 run-tests: drop redundant assignment to BINDIR
Yuya Nishihara <yuya@tcha.org>
parents: 28097
diff changeset
2285 self._bindir = os.path.join(self._installdir, b"bin")
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
2286 self._hgcommand = b'hg'
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2287 self._tmpbindir = self._bindir
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2288 self._pythondir = os.path.join(self._installdir, b"lib", b"python")
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2289
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
2290 # set CHGHG, then replace "hg" command by "chg"
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
2291 chgbindir = self._bindir
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2292 if self.options.chg or self.options.with_chg:
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
2293 osenvironb[b'CHGHG'] = os.path.join(self._bindir, self._hgcommand)
28880
f74eed3115fd hghave: add "chg" flag to skip tests that can't be compatible with chg
Yuya Nishihara <yuya@tcha.org>
parents: 28829
diff changeset
2294 else:
f74eed3115fd hghave: add "chg" flag to skip tests that can't be compatible with chg
Yuya Nishihara <yuya@tcha.org>
parents: 28829
diff changeset
2295 osenvironb.pop(b'CHGHG', None) # drop flag for hghave
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2296 if self.options.chg:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2297 self._hgcommand = b'chg'
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2298 elif self.options.with_chg:
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
2299 chgbindir = os.path.dirname(os.path.realpath(self.options.with_chg))
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
2300 self._hgcommand = os.path.basename(self.options.with_chg)
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
2301
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2302 osenvironb[b"BINDIR"] = self._bindir
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
2303 osenvironb[b"PYTHON"] = PYTHON
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2304
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
2305 if self.options.with_python3:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
2306 osenvironb[b'PYTHON3'] = self.options.with_python3
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
2307
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2308 fileb = _bytespath(__file__)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2309 runtestdir = os.path.abspath(os.path.dirname(fileb))
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
2310 osenvironb[b'RUNTESTDIR'] = runtestdir
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
2311 if PYTHON3:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2312 sepb = _bytespath(os.pathsep)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2313 else:
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2314 sepb = os.pathsep
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2315 path = [self._bindir, runtestdir] + osenvironb[b"PATH"].split(sepb)
24742
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
2316 if os.path.islink(__file__):
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
2317 # test helper will likely be at the end of the symlink
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2318 realfile = os.path.realpath(fileb)
24742
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
2319 realdir = os.path.abspath(os.path.dirname(realfile))
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
2320 path.insert(2, realdir)
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
2321 if chgbindir != self._bindir:
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
2322 path.insert(1, chgbindir)
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
2323 if self._testdir != runtestdir:
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
2324 path = [self._testdir] + path
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2325 if self._tmpbindir != self._bindir:
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2326 path = [self._tmpbindir] + path
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2327 osenvironb[b"PATH"] = sepb.join(path)
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
2328
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2329 # Include TESTDIR in PYTHONPATH so that out-of-tree extensions
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2330 # can run .../tests/run-tests.py test-foo where test-foo
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2331 # adds an extension to HGRC. Also include run-test.py directory to
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2332 # import modules like heredoctest.
23859
661b246bf1c4 run-tests: include testdir in $PATH so tests easily can use helper tools
Mads Kiilerich <madski@unity3d.com>
parents: 23728
diff changeset
2333 pypath = [self._pythondir, self._testdir, runtestdir]
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2334 # We have to augment PYTHONPATH, rather than simply replacing
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2335 # it, in case external libraries are only available via current
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2336 # PYTHONPATH. (In particular, the Subversion bindings on OS X
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2337 # are in /opt/subversion.)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2338 oldpypath = osenvironb.get(IMPL_PATH)
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2339 if oldpypath:
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2340 pypath.append(oldpypath)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2341 osenvironb[IMPL_PATH] = sepb.join(pypath)
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2342
23935
d64dd1252386 run-tests.py: inherit --pure option from outer run-tests.py execution
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23933
diff changeset
2343 if self.options.pure:
d64dd1252386 run-tests.py: inherit --pure option from outer run-tests.py execution
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23933
diff changeset
2344 os.environ["HGTEST_RUN_TESTS_PURE"] = "--pure"
28905
c969c72d6cbc run-tests: set HGMODULEPOLICY for --pure
timeless <timeless@mozdev.org>
parents: 28880
diff changeset
2345 os.environ["HGMODULEPOLICY"] = "py"
23935
d64dd1252386 run-tests.py: inherit --pure option from outer run-tests.py execution
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23933
diff changeset
2346
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
2347 if self.options.allow_slow_tests:
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
2348 os.environ["HGTEST_SLOW"] = "slow"
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
2349 elif 'HGTEST_SLOW' in os.environ:
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
2350 del os.environ['HGTEST_SLOW']
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
2351
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2352 self._coveragefile = os.path.join(self._testdir, b'.coverage')
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
2353
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2354 vlog("# Using TESTDIR", self._testdir)
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
2355 vlog("# Using RUNTESTDIR", osenvironb[b'RUNTESTDIR'])
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2356 vlog("# Using HGTMP", self._hgtmp)
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2357 vlog("# Using PATH", os.environ["PATH"])
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2358 vlog("# Using", IMPL_PATH, osenvironb[IMPL_PATH])
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
2359 vlog("# Writing to directory", self._outputdir)
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2360
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2361 try:
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2362 return self._runtests(testdescs) or 0
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2363 finally:
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2364 time.sleep(.1)
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2365 self._cleanup()
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2366
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2367 def findtests(self, args):
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2368 """Finds possible test files from arguments.
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2369
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2370 If you wish to inject custom tests into the test harness, this would
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2371 be a good function to monkeypatch or override in a derived class.
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2372 """
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2373 if not args:
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2374 if self.options.changed:
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2375 proc = Popen4('hg st --rev "%s" -man0 .' %
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2376 self.options.changed, None, 0)
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2377 stdout, stderr = proc.communicate()
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2378 args = stdout.strip(b'\0').split(b'\0')
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2379 else:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2380 args = os.listdir(b'.')
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2381
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2382 tests = []
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2383 for t in args:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2384 if not (os.path.basename(t).startswith(b'test-')
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2385 and (t.endswith(b'.py') or t.endswith(b'.t'))):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2386 continue
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2387 if t.endswith(b'.t'):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2388 # .t file may contain multiple test cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2389 cases = sorted(parsettestcases(t))
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2390 if cases:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2391 tests += [{'path': t, 'case': c} for c in sorted(cases)]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2392 else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2393 tests.append({'path': t})
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2394 else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2395 tests.append({'path': t})
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2396 return tests
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
2397
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2398 def _runtests(self, testdescs):
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2399 def _reloadtest(test, i):
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2400 # convert a test back to its description dict
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2401 desc = {'path': test.path}
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2402 case = getattr(test, '_case', None)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2403 if case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2404 desc['case'] = case
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2405 return self._gettest(desc, i)
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2406
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2407 try:
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2408 if self.options.restart:
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2409 orig = list(testdescs)
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2410 while testdescs:
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2411 desc = testdescs[0]
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32719
diff changeset
2412 # desc['path'] is a relative path
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2413 if 'case' in desc:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2414 errpath = b'%s.%s.err' % (desc['path'], desc['case'])
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2415 else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2416 errpath = b'%s.err' % desc['path']
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32719
diff changeset
2417 errpath = os.path.join(self._outputdir, errpath)
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2418 if os.path.exists(errpath):
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2419 break
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2420 testdescs.pop(0)
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2421 if not testdescs:
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2422 print("running all tests")
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2423 testdescs = orig
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2424
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2425 tests = [self._gettest(d, i) for i, d in enumerate(testdescs)]
21437
d9532be2fc4d run-tests: pass Test instances into TestRunner._executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21436
diff changeset
2426
21458
c42219733f30 run-tests: don't print results in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21457
diff changeset
2427 failed = False
25050
28526bb5b3b5 run-tests: make sure keyword(s) are in bytes and not str
Augie Fackler <augie@google.com>
parents: 25049
diff changeset
2428 kws = self.options.keywords
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
2429 if kws is not None and PYTHON3:
25050
28526bb5b3b5 run-tests: make sure keyword(s) are in bytes and not str
Augie Fackler <augie@google.com>
parents: 25049
diff changeset
2430 kws = kws.encode('utf-8')
21458
c42219733f30 run-tests: don't print results in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21457
diff changeset
2431
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2432 suite = TestSuite(self._testdir,
21533
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
2433 jobs=self.options.jobs,
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
2434 whitelist=self.options.whitelisted,
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
2435 blacklist=self.options.blacklist,
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
2436 retest=self.options.retest,
25050
28526bb5b3b5 run-tests: make sure keyword(s) are in bytes and not str
Augie Fackler <augie@google.com>
parents: 25049
diff changeset
2437 keywords=kws,
21532
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
2438 loop=self.options.loop,
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
2439 runs_per_test=self.options.runs_per_test,
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
2440 showchannels=self.options.showchannels,
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2441 tests=tests, loadtest=_reloadtest)
21464
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
2442 verbosity = 1
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
2443 if self.options.verbose:
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
2444 verbosity = 2
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
2445 runner = TextTestRunner(self, verbosity=verbosity)
32703
9d1d3199382e run-tests: install hg after computing tests to run
Siddharth Agarwal <sid0@fb.com>
parents: 32702
diff changeset
2446
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2447 if self.options.list_tests:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2448 result = runner.listtests(suite)
32703
9d1d3199382e run-tests: install hg after computing tests to run
Siddharth Agarwal <sid0@fb.com>
parents: 32702
diff changeset
2449 else:
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2450 if self._installdir:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2451 self._installhg()
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2452 self._checkhglib("Testing")
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2453 else:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2454 self._usecorrectpython()
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2455 if self.options.chg:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2456 assert self._installdir
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2457 self._installchg()
32703
9d1d3199382e run-tests: install hg after computing tests to run
Siddharth Agarwal <sid0@fb.com>
parents: 32702
diff changeset
2458
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
2459 result = runner.run(suite)
21613
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
2460
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
2461 if result.failures:
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
2462 failed = True
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2463
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2464 if self.options.anycoverage:
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
2465 self._outputcoverage()
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2466 except KeyboardInterrupt:
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2467 failed = True
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2468 print("\ninterrupted!")
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2469
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2470 if failed:
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2471 return 1
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
2472
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2473 def _getport(self, count):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2474 port = self._ports.get(count) # do we have a cached entry?
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2475 if port is None:
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2476 portneeded = 3
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2477 # above 100 tries we just give up and let test reports failure
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2478 for tries in xrange(100):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2479 allfree = True
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
2480 port = self.options.port + self._portoffset
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2481 for idx in xrange(portneeded):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2482 if not checkportisavailable(port + idx):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2483 allfree = False
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2484 break
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2485 self._portoffset += portneeded
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2486 if allfree:
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2487 break
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2488 self._ports[count] = port
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2489 return port
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2490
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2491 def _gettest(self, testdesc, count):
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2492 """Obtain a Test by looking at its filename.
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2493
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2494 Returns a Test instance. The Test may not be runnable if it doesn't
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2495 map to a known type.
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2496 """
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
2497 path = testdesc['path']
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2498 lctest = path.lower()
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2499 testcls = Test
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2500
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
2501 for ext, cls in self.TESTTYPES:
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2502 if lctest.endswith(ext):
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2503 testcls = cls
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2504 break
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2505
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
2506 refpath = os.path.join(self._testdir, path)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2507 tmpdir = os.path.join(self._hgtmp, b'child%d' % count)
21504
888a5dfe1569 run-tests: pass temp dir into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21502
diff changeset
2508
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2509 # extra keyword parameters. 'case' is used by .t tests
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2510 kwds = dict((k, testdesc[k]) for k in ['case'] if k in testdesc)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2511
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
2512 t = testcls(refpath, self._outputdir, tmpdir,
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
2513 keeptmpdir=self.options.keep_tmpdir,
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
2514 debug=self.options.debug,
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
2515 timeout=self.options.timeout,
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
2516 startport=self._getport(count),
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
2517 extraconfigopts=self.options.extra_config_opt,
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
2518 py3kwarnings=self.options.py3k_warnings,
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
2519 shell=self.options.shell,
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
2520 hgcommand=self._hgcommand,
30986
f07ca071a058 runtests: set web.ipv6 if we use IPv6
Jun Wu <quark@fb.com>
parents: 30985
diff changeset
2521 usechg=bool(self.options.with_chg or self.options.chg),
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
2522 useipv6=useipv6, **kwds)
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
2523 t.should_reload = True
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
2524 return t
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
2525
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
2526 def _cleanup(self):
21350
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2527 """Clean up state from this test invocation."""
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2528 if self.options.keep_tmpdir:
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2529 return
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2530
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2531 vlog("# Cleaning up HGTMP", self._hgtmp)
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2532 shutil.rmtree(self._hgtmp, True)
21352
39fd89fbbc3c run-tests: move createdfiles out of a global and into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21351
diff changeset
2533 for f in self._createdfiles:
21350
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2534 try:
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2535 os.remove(f)
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2536 except OSError:
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2537 pass
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
2538
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
2539 def _usecorrectpython(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2540 """Configure the environment to use the appropriate Python in tests."""
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2541 # Tests must use the same interpreter as us or bad things will happen.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
2542 pyexename = sys.platform == 'win32' and b'python.exe' or b'python'
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2543 if getattr(os, 'symlink', None):
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2544 vlog("# Making python executable in test path a symlink to '%s'" %
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2545 sys.executable)
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2546 mypython = os.path.join(self._tmpbindir, pyexename)
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2547 try:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2548 if os.readlink(mypython) == sys.executable:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2549 return
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2550 os.unlink(mypython)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2551 except OSError as err:
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2552 if err.errno != errno.ENOENT:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2553 raise
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2554 if self._findprogram(pyexename) != sys.executable:
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2555 try:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2556 os.symlink(sys.executable, mypython)
21352
39fd89fbbc3c run-tests: move createdfiles out of a global and into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21351
diff changeset
2557 self._createdfiles.append(mypython)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2558 except OSError as err:
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2559 # child processes may race, which is harmless
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2560 if err.errno != errno.EEXIST:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2561 raise
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2562 else:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2563 exedir, exename = os.path.split(sys.executable)
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2564 vlog("# Modifying search path to find %s as %s in '%s'" %
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2565 (exename, pyexename, exedir))
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2566 path = os.environ['PATH'].split(os.pathsep)
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2567 while exedir in path:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2568 path.remove(exedir)
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2569 os.environ['PATH'] = os.pathsep.join([exedir] + path)
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2570 if not self._findprogram(pyexename):
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2571 print("WARNING: Cannot find %s in search path" % pyexename)
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
2572
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
2573 def _installhg(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2574 """Install hg into the test environment.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2575
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2576 This will also configure hg with the appropriate testing settings.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2577 """
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2578 vlog("# Performing temporary installation of HG")
28829
65fb87479792 run-tests: move install.err into test area
timeless <timeless@mozdev.org>
parents: 28823
diff changeset
2579 installerrs = os.path.join(self._hgtmp, b"install.err")
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2580 compiler = ''
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2581 if self.options.compiler:
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2582 compiler = '--compiler ' + self.options.compiler
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 24075
diff changeset
2583 if self.options.pure:
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2584 pure = b"--pure"
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 24075
diff changeset
2585 else:
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2586 pure = b""
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2587
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2588 # Run installer in hg root
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2589 script = os.path.realpath(sys.argv[0])
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2590 exe = sys.executable
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
2591 if PYTHON3:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2592 compiler = _bytespath(compiler)
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2593 script = _bytespath(script)
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2594 exe = _bytespath(exe)
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2595 hgroot = os.path.dirname(os.path.dirname(script))
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
2596 self._hgroot = hgroot
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2597 os.chdir(hgroot)
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2598 nohome = b'--home=""'
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2599 if os.name == 'nt':
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2600 # The --home="" trick works only on OS where os.sep == '/'
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2601 # because of a distutils convert_path() fast-path. Avoid it at
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2602 # least on Windows for now, deal with .pydistutils.cfg bugs
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2603 # when they happen.
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2604 nohome = b''
28397
98a1a9547bb1 run-tests: remove 2to3 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28317
diff changeset
2605 cmd = (b'%(exe)s setup.py %(pure)s clean --all'
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2606 b' build %(compiler)s --build-base="%(base)s"'
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2607 b' install --force --prefix="%(prefix)s"'
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2608 b' --install-lib="%(libdir)s"'
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2609 b' --install-scripts="%(bindir)s" %(nohome)s >%(logfile)s 2>&1'
28397
98a1a9547bb1 run-tests: remove 2to3 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28317
diff changeset
2610 % {b'exe': exe, b'pure': pure,
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2611 b'compiler': compiler,
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2612 b'base': os.path.join(self._hgtmp, b"build"),
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2613 b'prefix': self._installdir, b'libdir': self._pythondir,
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2614 b'bindir': self._bindir,
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2615 b'nohome': nohome, b'logfile': installerrs})
24075
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2616
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2617 # setuptools requires install directories to exist.
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2618 def makedirs(p):
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2619 try:
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2620 os.makedirs(p)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2621 except OSError as e:
24075
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2622 if e.errno != errno.EEXIST:
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2623 raise
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2624 makedirs(self._pythondir)
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2625 makedirs(self._bindir)
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
2626
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2627 vlog("# Running", cmd)
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2628 if os.system(cmd) == 0:
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2629 if not self.options.verbose:
26087
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
2630 try:
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
2631 os.remove(installerrs)
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
2632 except OSError as e:
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
2633 if e.errno != errno.ENOENT:
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
2634 raise
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2635 else:
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
2636 f = open(installerrs, 'rb')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2637 for line in f:
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
2638 if PYTHON3:
25040
6b8c48cfb85e run-tests: write bytes to sys.stdout.buffer in python 3
Augie Fackler <augie@google.com>
parents: 25039
diff changeset
2639 sys.stdout.buffer.write(line)
6b8c48cfb85e run-tests: write bytes to sys.stdout.buffer in python 3
Augie Fackler <augie@google.com>
parents: 25039
diff changeset
2640 else:
6b8c48cfb85e run-tests: write bytes to sys.stdout.buffer in python 3
Augie Fackler <augie@google.com>
parents: 25039
diff changeset
2641 sys.stdout.write(line)
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2642 f.close()
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2643 sys.exit(1)
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2644 os.chdir(self._testdir)
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2645
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
2646 self._usecorrectpython()
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2647
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2648 if self.options.py3k_warnings and not self.options.anycoverage:
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2649 vlog("# Updating hg command to enable Py3k Warnings switch")
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
2650 f = open(os.path.join(self._bindir, 'hg'), 'rb')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2651 lines = [line.rstrip() for line in f]
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2652 lines[0] += ' -3'
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2653 f.close()
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
2654 f = open(os.path.join(self._bindir, 'hg'), 'wb')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2655 for line in lines:
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2656 f.write(line + '\n')
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2657 f.close()
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2658
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2659 hgbat = os.path.join(self._bindir, b'hg.bat')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2660 if os.path.isfile(hgbat):
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2661 # hg.bat expects to be put in bin/scripts while run-tests.py
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2662 # installation layout put it in bin/ directly. Fix it
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2663 f = open(hgbat, 'rb')
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2664 data = f.read()
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2665 f.close()
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2666 if b'"%~dp0..\python" "%~dp0hg" %*' in data:
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2667 data = data.replace(b'"%~dp0..\python" "%~dp0hg" %*',
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2668 b'"%~dp0python" "%~dp0hg" %*')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2669 f = open(hgbat, 'wb')
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2670 f.write(data)
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2671 f.close()
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2672 else:
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2673 print('WARNING: cannot fix hg.bat reference to python.exe')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2674
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2675 if self.options.anycoverage:
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2676 custom = os.path.join(self._testdir, 'sitecustomize.py')
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2677 target = os.path.join(self._pythondir, 'sitecustomize.py')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2678 vlog('# Installing coverage trigger to %s' % target)
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2679 shutil.copyfile(custom, target)
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
2680 rc = os.path.join(self._testdir, '.coveragerc')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2681 vlog('# Installing coverage rc to %s' % rc)
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2682 os.environ['COVERAGE_PROCESS_START'] = rc
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2683 covdir = os.path.join(self._installdir, '..', 'coverage')
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2684 try:
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2685 os.mkdir(covdir)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
2686 except OSError as e:
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2687 if e.errno != errno.EEXIST:
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2688 raise
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2689
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2690 os.environ['COVERAGE_DIR'] = covdir
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
2691
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
2692 def _checkhglib(self, verb):
21354
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
2693 """Ensure that the 'mercurial' package imported by python is
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
2694 the one we expect it to be. If not, print a warning to stderr."""
21733
9ad11d5bcc2f run-tests: don't check for the mercurial library used when using --with-hg
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21614
diff changeset
2695 if ((self._bindir == self._pythondir) and
9ad11d5bcc2f run-tests: don't check for the mercurial library used when using --with-hg
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21614
diff changeset
2696 (self._bindir != self._tmpbindir)):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
2697 # The pythondir has been inferred from --with-hg flag.
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
2698 # We cannot expect anything sensible here.
21733
9ad11d5bcc2f run-tests: don't check for the mercurial library used when using --with-hg
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21614
diff changeset
2699 return
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
2700 expecthg = os.path.join(self._pythondir, b'mercurial')
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2701 actualhg = self._gethgpath()
21354
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
2702 if os.path.abspath(actualhg) != os.path.abspath(expecthg):
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
2703 sys.stderr.write('warning: %s with unexpected mercurial lib: %s\n'
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
2704 ' (expected %s)\n'
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
2705 % (verb, actualhg, expecthg))
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2706 def _gethgpath(self):
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2707 """Return the path to the mercurial package that is actually found by
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2708 the current Python interpreter."""
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2709 if self._hgpath is not None:
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2710 return self._hgpath
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2711
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
2712 cmd = b'%s -c "import mercurial; print (mercurial.__path__[0])"'
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
2713 cmd = cmd % PYTHON
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
2714 if PYTHON3:
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
2715 cmd = _strpath(cmd)
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
2716 pipe = os.popen(cmd)
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2717 try:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2718 self._hgpath = _bytespath(pipe.read().strip())
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2719 finally:
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2720 pipe.close()
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2721
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
2722 return self._hgpath
21354
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
2723
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2724 def _installchg(self):
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2725 """Install chg into the test environment"""
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2726 vlog('# Performing temporary installation of CHG')
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2727 assert os.path.dirname(self._bindir) == self._installdir
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2728 assert self._hgroot, 'must be called after _installhg()'
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2729 cmd = (b'"%(make)s" clean install PREFIX="%(prefix)s"'
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2730 % {b'make': 'make', # TODO: switch by option or environment?
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2731 b'prefix': self._installdir})
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2732 cwd = os.path.join(self._hgroot, b'contrib', b'chg')
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2733 vlog("# Running", cmd)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2734 proc = subprocess.Popen(cmd, shell=True, cwd=cwd,
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2735 stdin=subprocess.PIPE, stdout=subprocess.PIPE,
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2736 stderr=subprocess.STDOUT)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2737 out, _err = proc.communicate()
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2738 if proc.returncode != 0:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2739 if PYTHON3:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2740 sys.stdout.buffer.write(out)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2741 else:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2742 sys.stdout.write(out)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2743 sys.exit(1)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
2744
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
2745 def _outputcoverage(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2746 """Produce code coverage output."""
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
2747 import coverage
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
2748 coverage = coverage.coverage
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
2749
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2750 vlog('# Producing coverage report')
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2751 # chdir is the easiest way to get short, relative paths in the
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2752 # output.
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
2753 os.chdir(self._hgroot)
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2754 covdir = os.path.join(self._installdir, '..', 'coverage')
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2755 cov = coverage(data_file=os.path.join(covdir, 'cov'))
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
2756
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
2757 # Map install directory paths back to source directory.
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
2758 cov.config.paths['srcdir'] = ['.', self._pythondir]
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
2759
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
2760 cov.combine()
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
2761
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2762 omit = [os.path.join(x, '*') for x in [self._bindir, self._testdir]]
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2763 cov.report(ignore_errors=True, omit=omit)
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2764
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
2765 if self.options.htmlcov:
32719
74680ed89a29 run-tests: output coverage to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
2766 htmldir = os.path.join(self._outputdir, 'htmlcov')
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2767 cov.html_report(directory=htmldir, omit=omit)
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
2768 if self.options.annotate:
32719
74680ed89a29 run-tests: output coverage to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
2769 adir = os.path.join(self._outputdir, 'annotated')
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
2770 if not os.path.isdir(adir):
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
2771 os.mkdir(adir)
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
2772 cov.annotate(directory=adir, omit=omit)
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
2773
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2774 def _findprogram(self, program):
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2775 """Search PATH for a executable program"""
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2776 dpb = _bytespath(os.defpath)
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
2777 sepb = _bytespath(os.pathsep)
25038
66da89457c47 run-tests: fix _findprogram to reliably return bytes
Augie Fackler <augie@google.com>
parents: 25037
diff changeset
2778 for p in osenvironb.get(b'PATH', dpb).split(sepb):
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2779 name = os.path.join(p, program)
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2780 if os.name == 'nt' or os.access(name, os.X_OK):
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2781 return name
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2782 return None
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2783
21374
592b3d2616d7 run-tests: move checktools into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21373
diff changeset
2784 def _checktools(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
2785 """Ensure tools required to run tests are present."""
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2786 for p in self.REQUIREDTOOLS:
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2787 if os.name == 'nt' and not p.endswith('.exe'):
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2788 p += '.exe'
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2789 found = self._findprogram(p)
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2790 if found:
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2791 vlog("# Found prerequisite", p, "at", found)
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2792 else:
28698
c756b6791760 py3: convert prereq bytes to string in run-tests
timeless <timeless@mozdev.org>
parents: 28645
diff changeset
2793 print("WARNING: Did not find prerequisite tool: %s " %
c756b6791760 py3: convert prereq bytes to string in run-tests
timeless <timeless@mozdev.org>
parents: 28645
diff changeset
2794 p.decode("utf-8"))
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
2795
13347
ce07defe7d9f run-tests: loadable as module
Simon Heimberg <simohe@besonet.ch>
parents: 13031
diff changeset
2796 if __name__ == '__main__':
21377
71081f7f9e52 run-tests: eliminate main()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21376
diff changeset
2797 runner = TestRunner()
22120
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2798
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2799 try:
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2800 import msvcrt
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2801 msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2802 msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2803 msvcrt.setmode(sys.stderr.fileno(), os.O_BINARY)
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2804 except ImportError:
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2805 pass
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
2806
21377
71081f7f9e52 run-tests: eliminate main()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21376
diff changeset
2807 sys.exit(runner.run(sys.argv[1:]))