tests/run-tests.py
author Yuya Nishihara <yuya@tcha.org>
Fri, 18 May 2018 21:32:05 +0900
branchstable
changeset 38039 5b831053d9b6
parent 37741 700aaa19de63
child 38222 507bdc40bb17
permissions -rwxr-xr-x
hgweb: do not try to replace signal handlers while locking According to the issue 5889, mod_wsgi issues a warning on signal.signal() call, and we wouldn't want to see it in error log. The problem addressed by d77c3b023393 could potentially occur in web session, but that would be less likely than in user processes.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
     1
#!/usr/bin/env python
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
     2
#
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
     3
# run-tests.py - Run a set of tests on Mercurial
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
     4
#
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
     5
# Copyright 2006 Matt Mackall <mpm@selenic.com>
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
     6
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8161
diff changeset
     7
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 10154
diff changeset
     8
# GNU General Public License version 2 or any later version.
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
     9
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    10
# Modifying this script is tricky because it has many modes:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    11
#   - serial (default) vs parallel (-jN, N > 1)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    12
#   - no coverage (default) vs coverage (-c, -C, -s)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    13
#   - temp install (default) vs specific hg script (--with-hg, --local)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    14
#   - tests are a mix of shell scripts and Python scripts
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    15
#
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    16
# If you change this script, it is recommended that you ensure you
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    17
# haven't broken it by running it in various modes with a representative
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    18
# sample of test scripts.  For example:
8843
eb7b247a98ea kill trailing whitespace
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 8724
diff changeset
    19
#
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    20
#  1) serial, no coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    21
#      ./run-tests.py test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    22
#  2) serial, no coverage, local hg:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    23
#      ./run-tests.py --local test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    24
#  3) serial, coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    25
#      ./run-tests.py -c test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    26
#  4) serial, coverage, local hg:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    27
#      ./run-tests.py -c --local test-s*      # unsupported
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    28
#  5) parallel, no coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    29
#      ./run-tests.py -j2 test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    30
#  6) parallel, no coverage, local hg:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    31
#      ./run-tests.py -j2 --local test-s*
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    32
#  7) parallel, coverage, temp install:
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    33
#      ./run-tests.py -j2 -c test-s*          # currently broken
9899
be574a37a8ae run-tests: give each child its own tmp dir (issue1911)
Greg Ward <greg@gerg.ca>
parents: 9707
diff changeset
    34
#  8) parallel, coverage, local install:
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    35
#      ./run-tests.py -j2 -c --local test-s*  # unsupported (and broken)
9899
be574a37a8ae run-tests: give each child its own tmp dir (issue1911)
Greg Ward <greg@gerg.ca>
parents: 9707
diff changeset
    36
#  9) parallel, custom tmp dir:
be574a37a8ae run-tests: give each child its own tmp dir (issue1911)
Greg Ward <greg@gerg.ca>
parents: 9707
diff changeset
    37
#      ./run-tests.py -j2 --tmpdir /tmp/myhgtests
26158
342ab95a1f4b run-tests: use $HGTEST_RUN_TESTS_PURE
timeless@mozdev.org
parents: 26109
diff changeset
    38
#  10) parallel, pure, tests that call run-tests:
342ab95a1f4b run-tests: use $HGTEST_RUN_TESTS_PURE
timeless@mozdev.org
parents: 26109
diff changeset
    39
#      ./run-tests.py --pure `grep -l run-tests.py *.t`
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    40
#
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    41
# (You could use any subset of the tests: test-s* happens to match
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    42
# enough that it's worth doing parallel runs, few enough that it
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    43
# completes fairly quickly, includes both shell and Python scripts, and
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    44
# includes some scripts that run daemon processes.)
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
    45
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    46
from __future__ import absolute_import, print_function
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
    47
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
    48
import argparse
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
    49
import collections
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
    50
import difflib
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    51
import distutils.version as version
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
    52
import errno
28126
562a073a2a1b tests: load json with no fallback
Yuya Nishihara <yuya@tcha.org>
parents: 28099
diff changeset
    53
import json
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
    54
import os
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    55
import random
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    56
import re
10905
13a1b2fb7ef2 pylint, pyflakes: remove unused or duplicate imports
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 10903
diff changeset
    57
import shutil
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
    58
import signal
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
    59
import socket
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    60
import subprocess
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
    61
import sys
32302
8627cf4de929 run-tests: drop fallback for sysconfig for pre-py2.7
Martin von Zweigbergk <martinvonz@google.com>
parents: 31950
diff changeset
    62
import sysconfig
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
    63
import tempfile
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    64
import threading
2571
83cfd95eafb5 tests: add timeouts, make run-tests.py clean up dead daemon processes
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2570
diff changeset
    65
import time
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    66
import unittest
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    67
import xml.dom.minidom as minidom
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
    68
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
    69
try:
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
    70
    import Queue as queue
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
    71
except ImportError:
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
    72
    import queue
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
    73
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
    74
try:
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
    75
    import shlex
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
    76
    shellquote = shlex.quote
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
    77
except (ImportError, AttributeError):
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
    78
    import pipes
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
    79
    shellquote = pipes.quote
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
    80
29282
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
    81
if os.environ.get('RTUNICODEPEDANTRY', False):
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
    82
    try:
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
    83
        reload(sys)
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
    84
        sys.setdefaultencoding("undefined")
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
    85
    except NameError:
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
    86
        pass
89822d7a9d5f run-tests: add support for RTUNICODEPEDANTRY environment variable
timeless <timeless@mozdev.org>
parents: 29246
diff changeset
    87
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
    88
origenviron = os.environ.copy()
25037
570141fa38f9 run-tests: introduce a name for os.environb
Augie Fackler <augie@google.com>
parents: 25036
diff changeset
    89
osenvironb = getattr(os, 'environb', os.environ)
14019
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
    90
processlock = threading.Lock()
19413
a4de0d3dc35a run-tests: lock popen wait/poll
Brendan Cully <brendan@kublai.com>
parents: 19407
diff changeset
    91
33552
754569f5e999 run-tests: make sure to check if pygments is installed before using it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33500
diff changeset
    92
pygmentspresent = False
33500
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
    93
# ANSI color is unsupported prior to Windows 10
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
    94
if os.name != 'nt':
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
    95
    try: # is pygments installed
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
    96
        import pygments
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
    97
        import pygments.lexers as lexers
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
    98
        import pygments.lexer as lexer
33500
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
    99
        import pygments.formatters as formatters
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   100
        import pygments.token as token
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   101
        import pygments.style as style
33552
754569f5e999 run-tests: make sure to check if pygments is installed before using it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33500
diff changeset
   102
        pygmentspresent = True
33580
20436925e080 run-tests: pre instantiate pygments objects
Jun Wu <quark@fb.com>
parents: 33568
diff changeset
   103
        difflexer = lexers.DiffLexer()
20436925e080 run-tests: pre instantiate pygments objects
Jun Wu <quark@fb.com>
parents: 33568
diff changeset
   104
        terminal256formatter = formatters.Terminal256Formatter()
33500
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
   105
    except ImportError:
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
   106
        pass
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33203
diff changeset
   107
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   108
if pygmentspresent:
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   109
    class TestRunnerStyle(style.Style):
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   110
        default_style = ""
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   111
        skipped = token.string_to_tokentype("Token.Generic.Skipped")
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   112
        failed = token.string_to_tokentype("Token.Generic.Failed")
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   113
        skippedname = token.string_to_tokentype("Token.Generic.SName")
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   114
        failedname = token.string_to_tokentype("Token.Generic.FName")
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   115
        styles = {
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   116
            skipped:         '#e5e5e5',
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   117
            skippedname:     '#00ffff',
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   118
            failed:          '#7f0000',
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   119
            failedname:      '#ff0000',
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   120
        }
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   121
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   122
    class TestRunnerLexer(lexer.RegexLexer):
35848
8a7140ec4c89 testrunner: on error, color the "(case xxx)" part the same as filename
Martin von Zweigbergk <martinvonz@google.com>
parents: 35823
diff changeset
   123
        testpattern = r'[\w-]+\.(t|py)( \(case [\w-]+\))?'
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   124
        tokens = {
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   125
            'root': [
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   126
                (r'^Skipped', token.Generic.Skipped, 'skipped'),
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   127
                (r'^Failed ', token.Generic.Failed, 'failed'),
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   128
                (r'^ERROR: ', token.Generic.Failed, 'failed'),
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   129
            ],
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   130
            'skipped': [
35848
8a7140ec4c89 testrunner: on error, color the "(case xxx)" part the same as filename
Martin von Zweigbergk <martinvonz@google.com>
parents: 35823
diff changeset
   131
                (testpattern, token.Generic.SName),
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   132
                (r':.*', token.Generic.Skipped),
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   133
            ],
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   134
            'failed': [
35848
8a7140ec4c89 testrunner: on error, color the "(case xxx)" part the same as filename
Martin von Zweigbergk <martinvonz@google.com>
parents: 35823
diff changeset
   135
                (testpattern, token.Generic.FName),
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   136
                (r'(:| ).*', token.Generic.Failed),
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   137
            ]
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   138
        }
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33710
diff changeset
   139
33866
4e8a46c25fac run-tests: pre instantiate pygments objects
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33814
diff changeset
   140
    runnerformatter = formatters.Terminal256Formatter(style=TestRunnerStyle)
4e8a46c25fac run-tests: pre instantiate pygments objects
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33814
diff changeset
   141
    runnerlexer = TestRunnerLexer()
4e8a46c25fac run-tests: pre instantiate pygments objects
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33814
diff changeset
   142
25160
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
   143
if sys.version_info > (3, 5, 0):
25157
a8d22895a575 run-tests: introduce PYTHON3 boolean constant (issue4668)
Augie Fackler <augie@google.com>
parents: 25156
diff changeset
   144
    PYTHON3 = True
25033
2bdd9e442bcc run-tests: work around the rename of xrange to range
Augie Fackler <augie@google.com>
parents: 25031
diff changeset
   145
    xrange = range # we use xrange in one place, and we'd rather not use range
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
   146
    def _bytespath(p):
33710
2e43c5cd57a7 tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
   147
        if p is None:
2e43c5cd57a7 tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
   148
            return p
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
   149
        return p.encode('utf-8')
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   150
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   151
    def _strpath(p):
33710
2e43c5cd57a7 tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
   152
        if p is None:
2e43c5cd57a7 tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
   153
            return p
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   154
        return p.decode('utf-8')
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   155
25160
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
   156
elif sys.version_info >= (3, 0, 0):
32353
4bffe2421f34 run-tests: remove references to Python 2.6
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32328
diff changeset
   157
    print('%s is only supported on Python 3.5+ and 2.7, not %s' %
25160
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
   158
          (sys.argv[0], '.'.join(str(v) for v in sys.version_info[:3])))
fefc72523491 run-tests: insist that if people use Python 3, they use 3.5.x
Augie Fackler <augie@google.com>
parents: 25159
diff changeset
   159
    sys.exit(70) # EX_SOFTWARE from `man 3 sysexit`
25157
a8d22895a575 run-tests: introduce PYTHON3 boolean constant (issue4668)
Augie Fackler <augie@google.com>
parents: 25156
diff changeset
   160
else:
a8d22895a575 run-tests: introduce PYTHON3 boolean constant (issue4668)
Augie Fackler <augie@google.com>
parents: 25156
diff changeset
   161
    PYTHON3 = False
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   163
    # In python 2.x, path operations are generally done using
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   164
    # bytestrings by default, so we don't have to do any extra
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   165
    # fiddling there. We define the wrapper functions anyway just to
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   166
    # help keep code consistent between platforms.
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
   167
    def _bytespath(p):
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
   168
        return p
25033
2bdd9e442bcc run-tests: work around the rename of xrange to range
Augie Fackler <augie@google.com>
parents: 25031
diff changeset
   169
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   170
    _strpath = _bytespath
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   171
25177
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
   172
# For Windows support
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
   173
wifexited = getattr(os, "WIFEXITED", lambda x: False)
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
   174
30984
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   175
# Whether to use IPv6
31002
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
   176
def checksocketfamily(name, port=20058):
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
   177
    """return true if we can listen on localhost using family=name
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
   178
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
   179
    name should be either 'AF_INET', or 'AF_INET6'.
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
   180
    port being used is okay - EADDRINUSE is considered as successful.
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
   181
    """
a489ee9b2852 runtests: prefer IPv4 to IPv6
Jun Wu <quark@fb.com>
parents: 30987
diff changeset
   182
    family = getattr(socket, name, None)
30984
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   183
    if family is None:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   184
        return False
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   185
    try:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   186
        s = socket.socket(family, socket.SOCK_STREAM)
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   187
        s.bind(('localhost', port))
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   188
        s.close()
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   189
        return True
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   190
    except socket.error as exc:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   191
        if exc.errno == errno.EADDRINUSE:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   192
            return True
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   193
        elif exc.errno in (errno.EADDRNOTAVAIL, errno.EPROTONOSUPPORT):
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   194
            return False
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   195
        else:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   196
            raise
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   197
    else:
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   198
        return False
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   199
31011
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   200
# useipv6 will be set by parseargs
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   201
useipv6 = None
30984
15f9084a9a0c runtests: add a function to test if IPv6 is available
Jun Wu <quark@fb.com>
parents: 30896
diff changeset
   202
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
   203
def checkportisavailable(port):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
   204
    """return true if a port seems free to bind on localhost"""
30985
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
   205
    if useipv6:
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
   206
        family = socket.AF_INET6
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
   207
    else:
1f803482844a runtests: checkportisavailable should only check one family
Jun Wu <quark@fb.com>
parents: 30984
diff changeset
   208
        family = socket.AF_INET
30987
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   209
    try:
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   210
        s = socket.socket(family, socket.SOCK_STREAM)
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   211
        s.bind(('localhost', port))
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   212
        s.close()
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   213
        return True
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   214
    except socket.error as exc:
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   215
        if exc.errno not in (errno.EADDRINUSE, errno.EADDRNOTAVAIL,
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   216
                             errno.EPROTONOSUPPORT):
1ee685defe80 runtests: unindent an "if True" block
Jun Wu <quark@fb.com>
parents: 30986
diff changeset
   217
            raise
30886
2aaa8bfc7bd9 runtests: check ports on IPv6 address
Jun Wu <quark@fb.com>
parents: 30716
diff changeset
   218
    return False
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
   219
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   220
closefds = os.name == 'posix'
19262
7864e8f274fe run-tests: add env dict to isolate test environment
Matt Mackall <mpm@selenic.com>
parents: 19252
diff changeset
   221
def Popen4(cmd, wd, timeout, env=None):
14019
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
   222
    processlock.acquire()
19262
7864e8f274fe run-tests: add env dict to isolate test environment
Matt Mackall <mpm@selenic.com>
parents: 19252
diff changeset
   223
    p = subprocess.Popen(cmd, shell=True, bufsize=-1, cwd=wd, env=env,
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   224
                         close_fds=closefds,
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   225
                         stdin=subprocess.PIPE, stdout=subprocess.PIPE,
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   226
                         stderr=subprocess.STDOUT)
14019
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
   227
    processlock.release()
fbbd5f91d5e1 run-tests: do chdir for tests under a lock for thread safety
Matt Mackall <mpm@selenic.com>
parents: 14018
diff changeset
   228
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   229
    p.fromchild = p.stdout
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   230
    p.tochild = p.stdin
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   231
    p.childerr = p.stderr
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   232
14337
439ed4721a6d run-tests: ignore timeout when Popen.terminate is unavailable
Patrick Mezard <pmezard@gmail.com>
parents: 14336
diff changeset
   233
    p.timeout = False
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   234
    if timeout:
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   235
        def t():
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   236
            start = time.time()
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   237
            while time.time() - start < timeout and p.returncode is None:
16346
48692b416fbc tests: shorten post-test sleeps
Matt Mackall <mpm@selenic.com>
parents: 15942
diff changeset
   238
                time.sleep(.1)
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   239
            p.timeout = True
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   240
            if p.returncode is None:
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
   241
                terminate(p)
14001
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   242
        threading.Thread(target=t).start()
9c4da6ab4e5a run-tests: switch timeout handling from alarm to helper thread
Matt Mackall <mpm@selenic.com>
parents: 14000
diff changeset
   243
8280
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   244
    return p
0b02d98d44d0 util: always use subprocess
Martin Geisler <mg@lazybytes.net>
parents: 8258
diff changeset
   245
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
   246
PYTHON = _bytespath(sys.executable.replace('\\', '/'))
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
   247
IMPL_PATH = b'PYTHONPATH'
10758
2ed667a9dfcb tests: adapt the test runner to work with jython
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10750
diff changeset
   248
if 'java' in sys.platform:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
   249
    IMPL_PATH = b'JYTHONPATH'
4881
c51c9bc4579d Add hghave utility and run-tests.py support.
Patrick Mezard <pmezard@gmail.com>
parents: 4880
diff changeset
   250
6366
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
   251
defaults = {
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
   252
    'jobs': ('HGTEST_JOBS', 1),
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
   253
    'timeout': ('HGTEST_TIMEOUT', 180),
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   254
    'slowtimeout': ('HGTEST_SLOWTIMEOUT', 500),
6366
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
   255
    'port': ('HGTEST_PORT', 20059),
15941
af289d6cd422 tests: let run-tests.py default to use 'sh' in $PATH instead of '/bin/sh'
Mads Kiilerich <mads@kiilerich.com>
parents: 15940
diff changeset
   256
    'shell': ('HGTEST_SHELL', 'sh'),
6366
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
   257
}
07c3cd695b48 run-tests.py: Allow environment variables to set jobs/timeout/port.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 6343
diff changeset
   258
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
   259
def canonpath(path):
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
   260
    return os.path.realpath(os.path.expanduser(path))
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
   261
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   262
def parselistfiles(files, listtype, warn=True):
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   263
    entries = dict()
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   264
    for filename in files:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   265
        try:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   266
            path = os.path.expanduser(os.path.expandvars(filename))
21916
792ebd7dc5f6 run-tests: write out scripts in binary mode
Augie Fackler <raf@durin42.com>
parents: 21763
diff changeset
   267
            f = open(path, "rb")
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
   268
        except IOError as err:
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   269
            if err.errno != errno.ENOENT:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   270
                raise
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   271
            if warn:
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
   272
                print("warning: no such %s file: %s" % (listtype, filename))
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   273
            continue
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   274
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   275
        for line in f.readlines():
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
   276
            line = line.split(b'#', 1)[0].strip()
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   277
            if line:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   278
                entries[line] = filename
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   279
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   280
        f.close()
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   281
    return entries
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   282
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   283
def parsettestcases(path):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   284
    """read a .t test file, return a set of test case names
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   285
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   286
    If path does not exist, return an empty set.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   287
    """
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   288
    cases = set()
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   289
    try:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   290
        with open(path, 'rb') as f:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   291
            for l in f:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   292
                if l.startswith(b'#testcases '):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   293
                    cases.update(l[11:].split())
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   294
    except IOError as ex:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   295
        if ex.errno != errno.ENOENT:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   296
            raise
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   297
    return cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   298
21008
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
   299
def getparser():
21383
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
   300
    """Obtain the OptionParser used by the CLI."""
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
   301
    parser = argparse.ArgumentParser(usage='%(prog)s [options] [tests]')
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
   302
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   303
    selection = parser.add_argument_group('Test Selection')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   304
    selection.add_argument('--allow-slow-tests', action='store_true',
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   305
        help='allow extremely slow tests')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   306
    selection.add_argument("--blacklist", action="append",
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
   307
        help="skip tests listed in the specified blacklist file")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   308
    selection.add_argument("--changed",
20821
3d1d16b19e7d tests: add run-tests --changed option for running tests changed in revisions
Mads Kiilerich <madski@unity3d.com>
parents: 20793
diff changeset
   309
        help="run tests that are changed in parent rev or working directory")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   310
    selection.add_argument("-k", "--keywords",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   311
        help="run tests matching keywords")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   312
    selection.add_argument("-r", "--retest", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   313
        help = "retest failed tests")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   314
    selection.add_argument("--test-list", action="append",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   315
        help="read tests to run from the specified file")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   316
    selection.add_argument("--whitelist", action="append",
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   317
        help="always run tests listed in the specified whitelist file")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   318
    selection.add_argument('tests', metavar='TESTS', nargs='*',
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   319
                        help='Tests to run')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   320
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   321
    harness = parser.add_argument_group('Test Harness Behavior')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   322
    harness.add_argument('--bisect-repo',
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   323
                        metavar='bisect_repo',
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   324
                        help=("Path of a repo to bisect. Use together with "
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   325
                              "--known-good-rev"))
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   326
    harness.add_argument("-d", "--debug", action="store_true",
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
   327
        help="debug mode: write output of test scripts to console"
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 21023
diff changeset
   328
             " rather than capturing and diffing it (disables timeout)")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   329
    harness.add_argument("-f", "--first", action="store_true",
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   330
        help="exit on the first test failure")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   331
    harness.add_argument("-i", "--interactive", action="store_true",
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   332
        help="prompt to accept changed output")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   333
    harness.add_argument("-j", "--jobs", type=int,
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   334
        help="number of jobs to run in parallel"
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   335
             " (default: $%s or %d)" % defaults['jobs'])
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   336
    harness.add_argument("--keep-tmpdir", action="store_true",
9706
f8b4df4b033d run-tests: make --tmpdir option more useful.
Greg Ward <greg-hg@gerg.ca>
parents: 9582
diff changeset
   337
        help="keep temporary directory after running tests")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   338
    harness.add_argument('--known-good-rev',
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   339
                        metavar="known_good_rev",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   340
                        help=("Automatically bisect any failures using this "
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   341
                              "revision as a known-good revision."))
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   342
    harness.add_argument("--list-tests", action="store_true",
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
   343
        help="list tests instead of running them")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   344
    harness.add_argument("--loop", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   345
        help="loop tests repeatedly")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   346
    harness.add_argument('--random', action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   347
        help='run tests in random order')
36665
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
   348
    harness.add_argument('--order-by-runtime', action="store_true",
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
   349
        help='run slowest tests first, according to .testtimes')
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   350
    harness.add_argument("-p", "--port", type=int,
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   351
        help="port on which servers should listen"
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   352
             " (default: $%s or %d)" % defaults['port'])
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   353
    harness.add_argument('--profile-runner', action='store_true',
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   354
                        help='run statprof on run-tests')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   355
    harness.add_argument("-R", "--restart", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   356
        help="restart at last error")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   357
    harness.add_argument("--runs-per-test", type=int, dest="runs_per_test",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   358
        help="run each test N times (default=1)", default=1)
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   359
    harness.add_argument("--shell",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   360
        help="shell to use (default: $%s or %s)" % defaults['shell'])
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   361
    harness.add_argument('--showchannels', action='store_true',
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   362
                        help='show scheduling channels')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   363
    harness.add_argument("--slowtimeout", type=int,
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   364
        help="kill errant slow tests after SLOWTIMEOUT seconds"
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   365
             " (default: $%s or %d)" % defaults['slowtimeout'])
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   366
    harness.add_argument("-t", "--timeout", type=int,
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   367
        help="kill errant tests after TIMEOUT seconds"
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   368
             " (default: $%s or %d)" % defaults['timeout'])
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   369
    harness.add_argument("--tmpdir",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   370
        help="run tests in the given temporary directory"
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   371
             " (implies --keep-tmpdir)")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   372
    harness.add_argument("-v", "--verbose", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   373
        help="output verbose messages")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   374
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   375
    hgconf = parser.add_argument_group('Mercurial Configuration')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   376
    hgconf.add_argument("--chg", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   377
        help="install and use chg wrapper in place of hg")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   378
    hgconf.add_argument("--compiler",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   379
        help="compiler to build with")
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
   380
    hgconf.add_argument('--extra-config-opt', action="append", default=[],
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   381
        help='set the given config opt in the test hgrc')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   382
    hgconf.add_argument("-l", "--local", action="store_true",
29583
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
   383
        help="shortcut for --with-hg=<testdir>/../hg, "
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
   384
             "and --with-chg=<testdir>/../contrib/chg/chg if --chg is set")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   385
    hgconf.add_argument("--ipv6", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   386
        help="prefer IPv6 to IPv4 for network related tests")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   387
    hgconf.add_argument("--pure", action="store_true",
11039
cf0a309f6c4d run-tests: sort options
Matt Mackall <mpm@selenic.com>
parents: 11038
diff changeset
   388
        help="use pure Python code instead of C extensions")
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   389
    hgconf.add_argument("-3", "--py3k-warnings", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   390
        help="enable Py3k warnings on Python 2.7+")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   391
    hgconf.add_argument("--with-chg", metavar="CHG",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   392
        help="use specified chg wrapper in place of hg")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   393
    hgconf.add_argument("--with-hg",
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   394
        metavar="HG",
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   395
        help="test using specified hg script rather than a "
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   396
             "temporary installation")
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   397
    # This option should be deleted once test-check-py3-compat.t and other
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   398
    # Python 3 tests run with Python 3.
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   399
    hgconf.add_argument("--with-python3", metavar="PYTHON3",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   400
        help="Python 3 interpreter (if running under Python 2)"
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   401
             " (TEMPORARY)")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   402
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   403
    reporting = parser.add_argument_group('Results Reporting')
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   404
    reporting.add_argument("-C", "--annotate", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   405
        help="output files annotated with coverage")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   406
    reporting.add_argument("--color", choices=["always", "auto", "never"],
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   407
        default=os.environ.get('HGRUNTESTSCOLOR', 'auto'),
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   408
        help="colorisation: always|auto|never (default: auto)")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   409
    reporting.add_argument("-c", "--cover", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   410
        help="print a test coverage report")
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
   411
    reporting.add_argument('--exceptions', action='store_true',
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
   412
        help='log all exceptions and generate an exception report')
35188
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   413
    reporting.add_argument("-H", "--htmlcov", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   414
        help="create an HTML report of the coverage of the files")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   415
    reporting.add_argument("--json", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   416
        help="store test result data in 'report.json' file")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   417
    reporting.add_argument("--outputdir",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   418
        help="directory to write error logs to (default=test directory)")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   419
    reporting.add_argument("-n", "--nodiff", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   420
        help="skip showing test changes")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   421
    reporting.add_argument("-S", "--noskips", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   422
        help="don't report skip tests verbosely")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   423
    reporting.add_argument("--time", action="store_true",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   424
        help="time how long each test takes")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   425
    reporting.add_argument("--view",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   426
        help="external diff viewer")
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   427
    reporting.add_argument("--xunit",
d997b82152e8 run-tests: organize options into argument groups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
   428
        help="record xunit results at specified path")
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
   429
14201
57e04ded3da4 run-tests: use type of default to convert environment variable
Martin Geisler <mg@lazybytes.net>
parents: 14192
diff changeset
   430
    for option, (envvar, default) in defaults.items():
57e04ded3da4 run-tests: use type of default to convert environment variable
Martin Geisler <mg@lazybytes.net>
parents: 14192
diff changeset
   431
        defaults[option] = type(default)(os.environ.get(envvar, default))
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   432
    parser.set_defaults(**defaults)
21008
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
   433
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
   434
    return parser
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
   435
c1dd04be3d9a run-tests: allow option parser to be extended
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21007
diff changeset
   436
def parseargs(args, parser):
21383
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
   437
    """Parse arguments with our OptionParser and validate results."""
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
   438
    options = parser.parse_args(args)
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   439
10758
2ed667a9dfcb tests: adapt the test runner to work with jython
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10750
diff changeset
   440
    # jython is always pure
10766
afbcea270bb8 run-tests: force to test pure on pypy as well
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10765
diff changeset
   441
    if 'java' in sys.platform or '__pypy__' in sys.modules:
10765
fd31a3237498 Fix run-tests.py -jX after 2ed667a9dfcb
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10758
diff changeset
   442
        options.pure = True
10758
2ed667a9dfcb tests: adapt the test runner to work with jython
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 10750
diff changeset
   443
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   444
    if options.with_hg:
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
   445
        options.with_hg = canonpath(_bytespath(options.with_hg))
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   446
        if not (os.path.isfile(options.with_hg) and
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   447
                os.access(options.with_hg, os.X_OK)):
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   448
            parser.error('--with-hg must specify an executable hg script')
32328
531e6a57abd2 run-tests: allow hg executable to be hg.exe
Kostia Balytskyi <ikostia@fb.com>
parents: 32316
diff changeset
   449
        if os.path.basename(options.with_hg) not in [b'hg', b'hg.exe']:
14359
ad5c68a0db6a run-tests: print a newline after all warnings
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14340
diff changeset
   450
            sys.stderr.write('warning: --with-hg should specify an hg script\n')
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   451
    if options.local:
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
   452
        testdir = os.path.dirname(_bytespath(canonpath(sys.argv[0])))
29582
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   453
        reporootdir = os.path.dirname(testdir)
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   454
        pathandattrs = [(b'hg', 'with_hg')]
29583
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
   455
        if options.chg:
6ce870dba6fa run-tests: make --local set --with-chg if --chg is used
Jun Wu <quark@fb.com>
parents: 29582
diff changeset
   456
            pathandattrs.append((b'contrib/chg/chg', 'with_chg'))
29582
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   457
        for relpath, attr in pathandattrs:
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   458
            binpath = os.path.join(reporootdir, relpath)
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   459
            if os.name != 'nt' and not os.access(binpath, os.X_OK):
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   460
                parser.error('--local specified, but %r not found or '
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   461
                             'not executable' % binpath)
ee96d95e81a4 run-tests: allow --local to set multiple attributes
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   462
            setattr(options, attr, binpath)
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   463
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
   464
    if (options.chg or options.with_chg) and os.name == 'nt':
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
   465
        parser.error('chg does not work on %s' % os.name)
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
   466
    if options.with_chg:
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
   467
        options.chg = False  # no installation to temporary location
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
   468
        options.with_chg = canonpath(_bytespath(options.with_chg))
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
   469
        if not (os.path.isfile(options.with_chg) and
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
   470
                os.access(options.with_chg, os.X_OK)):
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
   471
            parser.error('--with-chg must specify a chg executable')
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
   472
    if options.chg and options.with_hg:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
   473
        # chg shares installation location with hg
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
   474
        parser.error('--chg does not work when --with-hg is specified '
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
   475
                     '(use --with-chg instead)')
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
   476
33567
d9677e2ed16a run-tests: warn if --color=always and no pygments installed
Martin von Zweigbergk <martinvonz@google.com>
parents: 33566
diff changeset
   477
    if options.color == 'always' and not pygmentspresent:
d9677e2ed16a run-tests: warn if --color=always and no pygments installed
Martin von Zweigbergk <martinvonz@google.com>
parents: 33566
diff changeset
   478
        sys.stderr.write('warning: --color=always ignored because '
d9677e2ed16a run-tests: warn if --color=always and no pygments installed
Martin von Zweigbergk <martinvonz@google.com>
parents: 33566
diff changeset
   479
                         'pygments is not installed\n')
d9677e2ed16a run-tests: warn if --color=always and no pygments installed
Martin von Zweigbergk <martinvonz@google.com>
parents: 33566
diff changeset
   480
34041
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 34040
diff changeset
   481
    if options.bisect_repo and not options.known_good_rev:
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 34040
diff changeset
   482
        parser.error("--bisect-repo cannot be used without --known-good-rev")
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 34040
diff changeset
   483
31011
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   484
    global useipv6
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   485
    if options.ipv6:
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   486
        useipv6 = checksocketfamily('AF_INET6')
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   487
    else:
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   488
        # only use IPv6 if IPv4 is unavailable and IPv6 is available
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   489
        useipv6 = ((not checksocketfamily('AF_INET'))
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   490
                   and checksocketfamily('AF_INET6'))
01eebb65a61d runtests: add an IPv6 command line flag
Jun Wu <quark@fb.com>
parents: 31010
diff changeset
   491
15859
44a371823f83 tests: add htmlcov option
Markus Zapke-Gründemann <info@keimlink.de>
parents: 15858
diff changeset
   492
    options.anycoverage = options.cover or options.annotate or options.htmlcov
10648
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   493
    if options.anycoverage:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   494
        try:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   495
            import coverage
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   496
            covver = version.StrictVersion(coverage.__version__).version
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   497
            if covver < (3, 3):
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   498
                parser.error('coverage options require coverage 3.3 or later')
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   499
        except ImportError:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   500
            parser.error('coverage options now require the coverage package')
8095
f5428d4ffd97 run-tests: reduce global variables set by parse_args().
Greg Ward <greg-hg@gerg.ca>
parents: 8094
diff changeset
   501
10648
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   502
    if options.anycoverage and options.local:
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   503
        # this needs some path mangling somewhere, I guess
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   504
        parser.error("sorry, coverage options do not work when --local "
58128004cca1 tests: use external coverage, mandate newer version
Dirkjan Ochtman <djc.ochtman@kentyde.com>
parents: 10413
diff changeset
   505
                     "is specified")
8674
0941ee76489e run-tests: redefine --with-hg so it takes the 'hg' script to run.
Greg Ward <greg-hg@gerg.ca>
parents: 8673
diff changeset
   506
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
   507
    if options.anycoverage and options.with_hg:
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
   508
        parser.error("sorry, coverage options do not work when --with-hg "
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
   509
                     "is specified")
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
   510
19250
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
   511
    global verbose
8095
f5428d4ffd97 run-tests: reduce global variables set by parse_args().
Greg Ward <greg-hg@gerg.ca>
parents: 8094
diff changeset
   512
    if options.verbose:
19279
3868c9ab4bf8 run-tests: drop options.child and users
Matt Mackall <mpm@selenic.com>
parents: 19278
diff changeset
   513
        verbose = ''
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   514
9394
31203db1b2ac run-tests: expand --tmpdir and create it if needed
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 9031
diff changeset
   515
    if options.tmpdir:
28644
2e50eb6304bd run-tests: add canonpath function
timeless <timeless@mozdev.org>
parents: 28620
diff changeset
   516
        options.tmpdir = canonpath(options.tmpdir)
9394
31203db1b2ac run-tests: expand --tmpdir and create it if needed
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 9031
diff changeset
   517
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   518
    if options.jobs < 1:
9408
70bf7f853adc run-tests: standardize on --foo instead of -f/--foo
Martin Geisler <mg@lazybytes.net>
parents: 9407
diff changeset
   519
        parser.error('--jobs must be positive')
9707
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
   520
    if options.interactive and options.debug:
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
   521
        parser.error("-i/--interactive and -d/--debug are incompatible")
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
   522
    if options.debug:
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
   523
        if options.timeout != defaults['timeout']:
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
   524
            sys.stderr.write(
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
   525
                'warning: --timeout option ignored with --debug\n')
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   526
        if options.slowtimeout != defaults['slowtimeout']:
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   527
            sys.stderr.write(
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   528
                'warning: --slowtimeout option ignored with --debug\n')
9707
38deec407f8d run-tests: add "debug" mode: don't capture child output, just show it.
Greg Ward <greg-hg@gerg.ca>
parents: 9706
diff changeset
   529
        options.timeout = 0
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   530
        options.slowtimeout = 0
9028
bea567ae3ff6 tests: add -3 switch to run-tests.py
Alejandro Santos <alejolp@alejolp.com>
parents: 8943
diff changeset
   531
    if options.py3k_warnings:
25158
3906b9783cd9 run-tests: prefer PYTHON3 constant to many version_info checks (issue4668)
Augie Fackler <augie@google.com>
parents: 25157
diff changeset
   532
        if PYTHON3:
3906b9783cd9 run-tests: prefer PYTHON3 constant to many version_info checks (issue4668)
Augie Fackler <augie@google.com>
parents: 25157
diff changeset
   533
            parser.error(
32353
4bffe2421f34 run-tests: remove references to Python 2.6
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32328
diff changeset
   534
                '--py3k-warnings can only be used on Python 2.7')
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   535
    if options.with_python3:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   536
        if PYTHON3:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   537
            parser.error('--with-python3 cannot be used when executing with '
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   538
                         'Python 3')
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   539
28645
c226e9b69bac run-tests: use canonpath for with-python3
timeless <timeless@mozdev.org>
parents: 28644
diff changeset
   540
        options.with_python3 = canonpath(options.with_python3)
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   541
        # Verify Python3 executable is acceptable.
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   542
        proc = subprocess.Popen([options.with_python3, b'--version'],
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   543
                                stdout=subprocess.PIPE,
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   544
                                stderr=subprocess.STDOUT)
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   545
        out, _err = proc.communicate()
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   546
        ret = proc.wait()
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   547
        if ret != 0:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   548
            parser.error('could not determine version of python 3')
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   549
        if not out.startswith('Python '):
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   550
            parser.error('unexpected output from python3 --version: %s' %
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   551
                         out)
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   552
        vers = version.LooseVersion(out[len('Python '):])
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   553
        if vers < version.LooseVersion('3.5.0'):
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   554
            parser.error('--with-python3 version must be 3.5.0 or greater; '
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   555
                         'got %s' % out)
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
   556
9959
b37b060d84c7 run-tests: add a "--blacklist target" option to skip predefined test lists
Nicolas Dumazet <nicdumz.commits@gmail.com>
parents: 9958
diff changeset
   557
    if options.blacklist:
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   558
        options.blacklist = parselistfiles(options.blacklist, 'blacklist')
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   559
    if options.whitelist:
19279
3868c9ab4bf8 run-tests: drop options.child and users
Matt Mackall <mpm@selenic.com>
parents: 19278
diff changeset
   560
        options.whitelisted = parselistfiles(options.whitelist, 'whitelist')
14493
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   561
    else:
5cc7905bccc9 run-tests: allow whitelisting tests that should always run
Augie Fackler <durin42@gmail.com>
parents: 14454
diff changeset
   562
        options.whitelisted = {}
8091
e85cc856d2e1 run-tests: factor out parse_args(). Clarify use of globals a bit.
Greg Ward <greg-hg@gerg.ca>
parents: 8060
diff changeset
   563
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
   564
    if options.showchannels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
   565
        options.nodiff = True
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
   566
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
   567
    return options
5384
e3a0c092b4e2 Allow tests to run in parallel.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5383
diff changeset
   568
5800
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
   569
def rename(src, dst):
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
   570
    """Like os.rename(), trade atomicity and opened files friendliness
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
   571
    for existing destination support.
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
   572
    """
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
   573
    shutil.copy(src, dst)
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
   574
    os.remove(src)
2f597243e1d7 Make run-tests.py --interactive work on Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5524
diff changeset
   575
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   576
_unified_diff = difflib.unified_diff
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
   577
if PYTHON3:
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   578
    import functools
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   579
    _unified_diff = functools.partial(difflib.diff_bytes, difflib.unified_diff)
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   580
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   581
def getdiff(expected, output, ref, err):
21022
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
   582
    servefail = False
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   583
    lines = []
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   584
    for line in _unified_diff(expected, output, ref, err):
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   585
        if line.startswith(b'+++') or line.startswith(b'---'):
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   586
            line = line.replace(b'\\', b'/')
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   587
            if line.endswith(b' \n'):
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   588
                line = line[:-2] + b'\n'
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   589
        lines.append(line)
21022
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
   590
        if not servefail and line.startswith(
25045
8a425c2eef5d run-tests: use difflib.diff_bytes on Python 3
Augie Fackler <augie@google.com>
parents: 25044
diff changeset
   591
                             b'+  abort: child process failed to start'):
21022
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
   592
            servefail = True
52e9e63f1495 run-tests: test result shows when a failed test could not start a server
Simon Heimberg <simohe@besonet.ch>
parents: 21009
diff changeset
   593
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   594
    return servefail, lines
2110
25a8d116ab6a Add a pure python version of run-tests.
Stephen Darnell <stephen@darnell.plus.com>
parents:
diff changeset
   595
19250
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
   596
verbose = False
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
   597
def vlog(*msg):
21535
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   598
    """Log only when in verbose mode."""
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   599
    if verbose is False:
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   600
        return
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   601
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   602
    return log(*msg)
19250
5fa946330970 run-tests: make vlog a proper function
Matt Mackall <mpm@selenic.com>
parents: 19249
diff changeset
   603
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   604
# Bytes that break XML even in a CDATA block: control characters 0-31
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   605
# sans \t, \n and \r
25051
9c28f3236677 run-tests: do cdata escaping using bytes instead of str
Augie Fackler <augie@google.com>
parents: 25050
diff changeset
   606
CDATA_EVIL = re.compile(br"[\000-\010\013\014\016-\037]")
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   607
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   608
# Match feature conditionalized output lines in the form, capturing the feature
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   609
# list in group 2, and the preceeding line output in group 1:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   610
#
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   611
#   output..output (feature !)\n
34444
df78b1a24094 test-logtoprocess: make it compatible with chg
Jun Wu <quark@fb.com>
parents: 34269
diff changeset
   612
optline = re.compile(b'(.*) \((.+?) !\)\n$')
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   613
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   614
def cdatasafe(data):
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   615
    """Make a string safe to include in a CDATA block.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   616
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   617
    Certain control characters are illegal in a CDATA block, and
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   618
    there's no way to include a ]]> in a CDATA either. This function
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   619
    replaces illegal bytes with ? and adds a space between the ]] so
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   620
    that it won't break the CDATA block.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   621
    """
25051
9c28f3236677 run-tests: do cdata escaping using bytes instead of str
Augie Fackler <augie@google.com>
parents: 25050
diff changeset
   622
    return CDATA_EVIL.sub(b'?', data).replace(b']]>', b'] ]>')
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   623
19251
6857f53456f2 run-tests: add a log function
Matt Mackall <mpm@selenic.com>
parents: 19250
diff changeset
   624
def log(*msg):
21535
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   625
    """Log something to stdout.
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   626
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   627
    Arguments are strings to print.
ab7e224bc089 run-tests: avoid duplicate code in vlog()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21534
diff changeset
   628
    """
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
   629
    with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
   630
        if verbose:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
   631
            print(verbose, end=' ')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
   632
        for m in msg:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
   633
            print(m, end=' ')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
   634
        print()
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
   635
        sys.stdout.flush()
19251
6857f53456f2 run-tests: add a log function
Matt Mackall <mpm@selenic.com>
parents: 19250
diff changeset
   636
33930
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   637
def highlightdiff(line, color):
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   638
    if not color:
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   639
        return line
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   640
    assert pygmentspresent
33931
7cab956794e4 run-tests: pass unicode to Pygments
Yuya Nishihara <yuya@tcha.org>
parents: 33930
diff changeset
   641
    return pygments.highlight(line.decode('latin1'), difflexer,
7cab956794e4 run-tests: pass unicode to Pygments
Yuya Nishihara <yuya@tcha.org>
parents: 33930
diff changeset
   642
                              terminal256formatter).encode('latin1')
33930
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   643
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   644
def highlightmsg(msg, color):
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   645
    if not color:
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   646
        return msg
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   647
    assert pygmentspresent
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   648
    return pygments.highlight(msg, runnerlexer, runnerformatter)
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
   649
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
   650
def terminate(proc):
32303
dc131b9772f2 run-tests: drop fallback for proc.terminate() for pre-py2.6
Martin von Zweigbergk <martinvonz@google.com>
parents: 32302
diff changeset
   651
    """Terminate subprocess"""
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
   652
    vlog('# Terminating process %d' % proc.pid)
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
   653
    try:
32303
dc131b9772f2 run-tests: drop fallback for proc.terminate() for pre-py2.6
Martin von Zweigbergk <martinvonz@google.com>
parents: 32302
diff changeset
   654
        proc.terminate()
14821
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
   655
    except OSError:
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
   656
        pass
2017495bd552 run-tests: fallback to SIGTERM if subprocess.Popen does not have terminate()
Thomas Arendsen Hein <thomas@intevation.de>
parents: 14598
diff changeset
   657
19263
062c0a0a5549 run-tests: use env dict to kill daemons
Matt Mackall <mpm@selenic.com>
parents: 19262
diff changeset
   658
def killdaemons(pidfile):
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
   659
    import killdaemons as killmod
19263
062c0a0a5549 run-tests: use env dict to kill daemons
Matt Mackall <mpm@selenic.com>
parents: 19262
diff changeset
   660
    return killmod.killdaemons(pidfile, tryhard=False, remove=True,
17464
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 16906
diff changeset
   661
                               logfn=vlog)
10336
bc9a3bb267fa run-tests: kill daemons on ^C with -j.
Brendan Cully <brendan@kublai.com>
parents: 10300
diff changeset
   662
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   663
class Test(unittest.TestCase):
21309
0b123e6a318c run-tests: allow Test.run() to run multiple times
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21308
diff changeset
   664
    """Encapsulates a single, runnable test.
0b123e6a318c run-tests: allow Test.run() to run multiple times
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21308
diff changeset
   665
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   666
    While this class conforms to the unittest.TestCase API, it differs in that
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   667
    instances need to be instantiated manually. (Typically, unittest.TestCase
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   668
    classes are instantiated automatically by scanning modules.)
21309
0b123e6a318c run-tests: allow Test.run() to run multiple times
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21308
diff changeset
   669
    """
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
   670
21380
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
   671
    # Status code reserved for skipped tests (used by hghave).
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
   672
    SKIPPED_STATUS = 80
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
   673
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   674
    def __init__(self, path, outputdir, tmpdir, keeptmpdir=False,
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   675
                 debug=False,
35541
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   676
                 first=False,
34264
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   677
                 timeout=None,
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   678
                 startport=None, extraconfigopts=None,
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
   679
                 py3kwarnings=False, shell=None, hgcommand=None,
34264
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   680
                 slowtimeout=None, usechg=False,
30986
f07ca071a058 runtests: set web.ipv6 if we use IPv6
Jun Wu <quark@fb.com>
parents: 30985
diff changeset
   681
                 useipv6=False):
21502
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
   682
        """Create a test from parameters.
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
   683
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
   684
        path is the full path to the file defining the test.
21338
3cd2d2de4060 run-tests: move computation of test paths into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21337
diff changeset
   685
21504
888a5dfe1569 run-tests: pass temp dir into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21502
diff changeset
   686
        tmpdir is the main temporary directory to use for this test.
21505
3a1881dbf860 run-tests: pass abort into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21504
diff changeset
   687
21509
d21d53ee0d7a run-tests: factor options.keep_tmpdir into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21508
diff changeset
   688
        keeptmpdir determines whether to keep the test's temporary directory
d21d53ee0d7a run-tests: factor options.keep_tmpdir into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21508
diff changeset
   689
        after execution. It defaults to removal (False).
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
   690
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
   691
        debug mode will make the test execute verbosely, with unfiltered
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
   692
        output.
21511
3ec3e81a4110 run-tests: move diff options into arguments of Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21510
diff changeset
   693
21513
acfd19f3e79c run-tests: move timeout into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21512
diff changeset
   694
        timeout controls the maximum run time of the test. It is ignored when
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   695
        debug is True. See slowtimeout for tests with #require slow.
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   696
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   697
        slowtimeout overrides timeout if the test has #require slow.
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
   698
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
   699
        startport controls the starting port number to use for this test. Each
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
   700
        test will reserve 3 port numbers for execution. It is the caller's
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
   701
        responsibility to allocate a non-overlapping port range to Test
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
   702
        instances.
21515
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
   703
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
   704
        extraconfigopts is an iterable of extra hgrc config options. Values
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
   705
        must have the form "key=value" (something understood by hgrc). Values
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
   706
        of the form "foo.key=value" will result in "[foo] key=value".
21516
1e275c09242e run-tests: move py3kwarnings to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21515
diff changeset
   707
1e275c09242e run-tests: move py3kwarnings to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21515
diff changeset
   708
        py3kwarnings enables Py3k warnings.
21517
af7d3a5c330b run-tests: move shell to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21516
diff changeset
   709
af7d3a5c330b run-tests: move shell to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21516
diff changeset
   710
        shell is the shell to execute tests in.
21502
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
   711
        """
34264
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   712
        if timeout is None:
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   713
            timeout = defaults['timeout']
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   714
        if startport is None:
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   715
            startport = defaults['port']
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   716
        if slowtimeout is None:
8999851a503f tests: fix run-tests default values in Test constructor
Augie Fackler <augie@google.com>
parents: 34263
diff changeset
   717
            slowtimeout = defaults['slowtimeout']
25039
8505eb1bafb1 run-tests: be more judicious about bytes vs string on test attrs
Augie Fackler <augie@google.com>
parents: 25038
diff changeset
   718
        self.path = path
8505eb1bafb1 run-tests: be more judicious about bytes vs string on test attrs
Augie Fackler <augie@google.com>
parents: 25038
diff changeset
   719
        self.bname = os.path.basename(path)
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
   720
        self.name = _strpath(self.bname)
21502
f8515564d617 run-tests: pass a full test path into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21501
diff changeset
   721
        self._testdir = os.path.dirname(path)
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   722
        self._outputdir = outputdir
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   723
        self._tmpname = os.path.basename(path)
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   724
        self.errpath = os.path.join(self._outputdir, b'%s.err' % self.bname)
21435
f376f56a354e run-tests: rename Test._test to Test.name
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21434
diff changeset
   725
25039
8505eb1bafb1 run-tests: be more judicious about bytes vs string on test attrs
Augie Fackler <augie@google.com>
parents: 25038
diff changeset
   726
        self._threadtmp = tmpdir
21509
d21d53ee0d7a run-tests: factor options.keep_tmpdir into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21508
diff changeset
   727
        self._keeptmpdir = keeptmpdir
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
   728
        self._debug = debug
35541
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   729
        self._first = first
21513
acfd19f3e79c run-tests: move timeout into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21512
diff changeset
   730
        self._timeout = timeout
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
   731
        self._slowtimeout = slowtimeout
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
   732
        self._startport = startport
21515
9978ff48b1e8 run-tests: move extra config options to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21514
diff changeset
   733
        self._extraconfigopts = extraconfigopts or []
21516
1e275c09242e run-tests: move py3kwarnings to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21515
diff changeset
   734
        self._py3kwarnings = py3kwarnings
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
   735
        self._shell = _bytespath(shell)
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
   736
        self._hgcommand = hgcommand or b'hg'
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   737
        self._usechg = usechg
31003
225f574e0645 runtests: export HGIPV6 to hint test scripts whether to use IPv6
Jun Wu <quark@fb.com>
parents: 31002
diff changeset
   738
        self._useipv6 = useipv6
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   739
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   740
        self._aborted = False
21319
44c96e2bab20 run-tests: kill daemons during Test.cleanup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21318
diff changeset
   741
        self._daemonpids = []
21447
f8c5b8a288c5 run-tests: keep track of test execution state in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21446
diff changeset
   742
        self._finished = None
21449
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
   743
        self._ret = None
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
   744
        self._out = None
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
   745
        self._skipped = None
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
   746
        self._testtmp = None
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   747
        self._chgsockdir = None
21447
f8c5b8a288c5 run-tests: keep track of test execution state in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21446
diff changeset
   748
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   749
        self._refout = self.readrefout()
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   750
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   751
    def readrefout(self):
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   752
        """read reference output"""
21318
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
   753
        # If we're not in --debug mode and reference output file exists,
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
   754
        # check test output against it.
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   755
        if self._debug:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   756
            return None # to match "out is None"
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   757
        elif os.path.exists(self.refpath):
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   758
            with open(self.refpath, 'rb') as f:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   759
                return f.read().splitlines(True)
21318
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
   760
        else:
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
   761
            return []
21318
6b3d66e4d3be run-tests: capture reference output in Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21317
diff changeset
   762
24965
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
   763
    # needed to get base class __repr__ running
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
   764
    @property
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
   765
    def _testMethodName(self):
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
   766
        return self.name
cecbe207cebd run-tests: implement Test._testMethodName
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24926
diff changeset
   767
21463
c908ff887589 run-tests: implement Test.__str__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21462
diff changeset
   768
    def __str__(self):
c908ff887589 run-tests: implement Test.__str__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21462
diff changeset
   769
        return self.name
c908ff887589 run-tests: implement Test.__str__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21462
diff changeset
   770
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   771
    def shortDescription(self):
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   772
        return self.name
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   773
21446
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
   774
    def setUp(self):
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
   775
        """Tasks to perform before run()."""
21447
f8c5b8a288c5 run-tests: keep track of test execution state in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21446
diff changeset
   776
        self._finished = False
21449
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
   777
        self._ret = None
aedf18bcde11 run-tests: store test return code and output in Test instance
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21448
diff changeset
   778
        self._out = None
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
   779
        self._skipped = None
21446
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
   780
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   781
        try:
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   782
            os.mkdir(self._threadtmp)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
   783
        except OSError as e:
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   784
            if e.errno != errno.EEXIST:
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   785
                raise
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   786
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
   787
        name = self._tmpname
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   788
        self._testtmp = os.path.join(self._threadtmp, name)
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
   789
        os.mkdir(self._testtmp)
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
   790
21457
12dd94e32102 run-tests: move errpath deletion to setUp()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21456
diff changeset
   791
        # Remove any previous output files.
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
   792
        if os.path.exists(self.errpath):
24332
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
   793
            try:
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
   794
                os.remove(self.errpath)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
   795
            except OSError as e:
24332
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
   796
                # We might have raced another test to clean up a .err
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
   797
                # file, so ignore ENOENT when removing a previous .err
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
   798
                # file.
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
   799
                if e.errno != errno.ENOENT:
9612b96730d7 run-tests: ignore ENOENT failures when removing old .err results
Augie Fackler <augie@google.com>
parents: 24331
diff changeset
   800
                    raise
21457
12dd94e32102 run-tests: move errpath deletion to setUp()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21456
diff changeset
   801
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   802
        if self._usechg:
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   803
            self._chgsockdir = os.path.join(self._threadtmp,
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   804
                                            b'%s.chgsock' % name)
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   805
            os.mkdir(self._chgsockdir)
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   806
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   807
    def run(self, result):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   808
        """Run this test and report results against a TestResult instance."""
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   809
        # This function is extremely similar to unittest.TestCase.run(). Once
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   810
        # we require Python 2.7 (or at least its version of unittest), this
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   811
        # function can largely go away.
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   812
        self._result = result
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   813
        result.startTest(self)
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   814
        try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   815
            try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   816
                self.setUp()
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   817
            except (KeyboardInterrupt, SystemExit):
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   818
                self._aborted = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   819
                raise
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   820
            except Exception:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   821
                result.addError(self, sys.exc_info())
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   822
                return
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   823
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   824
            success = False
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   825
            try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   826
                self.runTest()
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   827
            except KeyboardInterrupt:
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   828
                self._aborted = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   829
                raise
32932
240183a04429 tests: use unittest.SkipTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32907
diff changeset
   830
            except unittest.SkipTest as e:
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   831
                result.addSkip(self, str(e))
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
   832
                # The base class will have already counted this as a
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
   833
                # test we "ran", but we want to exclude skipped tests
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
   834
                # from those we count towards those run.
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
   835
                result.testsRun -= 1
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
   836
            except self.failureException as e:
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   837
                # This differs from unittest in that we don't capture
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   838
                # the stack trace. This is for historical reasons and
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
   839
                # this decision could be revisited in the future,
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   840
                # especially for PythonTest instances.
21753
b7baef94a333 run-tests: checks behaviour of test on failure while testing
anuraggoel <anurag.dsps@gmail.com>
parents: 21740
diff changeset
   841
                if result.addFailure(self, str(e)):
b7baef94a333 run-tests: checks behaviour of test on failure while testing
anuraggoel <anurag.dsps@gmail.com>
parents: 21740
diff changeset
   842
                    success = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   843
            except Exception:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   844
                result.addError(self, sys.exc_info())
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   845
            else:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   846
                success = True
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   847
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   848
            try:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   849
                self.tearDown()
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   850
            except (KeyboardInterrupt, SystemExit):
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   851
                self._aborted = True
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   852
                raise
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   853
            except Exception:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   854
                result.addError(self, sys.exc_info())
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   855
                success = False
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   856
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   857
            if success:
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   858
                result.addSuccess(self)
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   859
        finally:
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   860
            result.stopTest(self, interrupted=self._aborted)
21488
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   861
feb8ad2d57ee run-tests: merge MercurialTest into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21464
diff changeset
   862
    def runTest(self):
21383
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
   863
        """Run this test instance.
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
   864
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
   865
        This will return a tuple describing the result of the test.
772ed56e2519 run-tests: add some docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21382
diff changeset
   866
        """
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
   867
        env = self._getenv()
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
   868
        self._genrestoreenv(env)
21319
44c96e2bab20 run-tests: kill daemons during Test.cleanup()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21318
diff changeset
   869
        self._daemonpids.append(env['DAEMON_PIDS'])
21382
4b8ffe3abdd2 run-tests: move createhgrc into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21381
diff changeset
   870
        self._createhgrc(env['HGRCPATH'])
21300
a2774731a51a run-tests: move createhgrc() call into Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21299
diff changeset
   871
21435
f376f56a354e run-tests: rename Test._test to Test.name
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21434
diff changeset
   872
        vlog('# Test', self.name)
21337
79930515604f run-tests: move logging of test start into Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21336
diff changeset
   873
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
   874
        ret, out = self._run(env)
21500
130cc0d7bfde run-tests: don't trap exceptions in Test.runTest()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21499
diff changeset
   875
        self._finished = True
130cc0d7bfde run-tests: don't trap exceptions in Test.runTest()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21499
diff changeset
   876
        self._ret = ret
130cc0d7bfde run-tests: don't trap exceptions in Test.runTest()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21499
diff changeset
   877
        self._out = out
21305
d7a7825ff2cf run-tests: capture execution results in a TestResult class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21304
diff changeset
   878
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   879
        def describe(ret):
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   880
            if ret < 0:
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   881
                return 'killed by signal: %d' % -ret
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   882
            return 'returned error code %d' % ret
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   883
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
   884
        self._skipped = False
21336
45ab0668d1b2 run-tests: remove remaining uses of TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21335
diff changeset
   885
21380
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
   886
        if ret == self.SKIPPED_STATUS:
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   887
            if out is None: # Debug mode, nothing to parse.
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   888
                missing = ['unknown']
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   889
                failed = None
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   890
            else:
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
   891
                missing, failed = TTest.parsehghaveoutput(out)
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   892
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   893
            if not missing:
22292
102f0e926668 run-tests: report skipped tests as "skipped" - they might still be "relevant"
Mads Kiilerich <madski@unity3d.com>
parents: 22165
diff changeset
   894
                missing = ['skipped']
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   895
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   896
            if failed:
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
   897
                self.fail('hg have failed checking for %s' % failed[-1])
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   898
            else:
21453
aaf52b78327e run-tests: store skipped state on Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21452
diff changeset
   899
                self._skipped = True
32932
240183a04429 tests: use unittest.SkipTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32907
diff changeset
   900
                raise unittest.SkipTest(missing[-1])
21325
0e66eb57e42a run-tests: generate timeout result in Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21324
diff changeset
   901
        elif ret == 'timeout':
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
   902
            self.fail('timed out')
21522
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
   903
        elif ret is False:
32934
6123a5267119 tests: remove WarnTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32933
diff changeset
   904
            self.fail('no result code from test')
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   905
        elif out != self._refout:
21614
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
   906
            # Diff generation may rely on written .err file.
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
   907
            if (ret != 0 or out != self._refout) and not self._skipped \
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
   908
                and not self._debug:
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
   909
                with open(self.errpath, 'wb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
   910
                    for line in out:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
   911
                        f.write(line)
21614
609a642dff61 run-tests: write .err files earlier
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21613
diff changeset
   912
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   913
            # The result object handles diff calculation for us.
35541
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   914
            with firstlock:
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   915
                if self._result.addOutputMismatch(self, ret, out, self._refout):
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   916
                    # change was accepted, skip failing
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   917
                    return
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   918
                if self._first:
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   919
                    global firsterror
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
   920
                    firsterror = True
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   921
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   922
            if ret:
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   923
                msg = 'output changed and ' + describe(ret)
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   924
            else:
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
   925
                msg = 'output changed'
21326
04e04766065f run-tests: move output difference processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21325
diff changeset
   926
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
   927
            self.fail(msg)
21327
206814c9072a run-tests: move remaining result processing to Test.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21326
diff changeset
   928
        elif ret:
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
   929
            self.fail(describe(ret))
21324
6454ddaee991 run-tests: add skip processing to Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21323
diff changeset
   930
21446
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
   931
    def tearDown(self):
9a3b4f795f62 run-tests: support setUp() and tearDown() in TestCase wrapper
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21445
diff changeset
   932
        """Tasks to perform after run()."""
21456
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
   933
        for entry in self._daemonpids:
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
   934
            killdaemons(entry)
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
   935
        self._daemonpids = []
a06a4142ad1f run-tests: kill daemons during tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21455
diff changeset
   936
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   937
        if self._keeptmpdir:
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   938
            log('\nKeeping testtmp dir: %s\nKeeping threadtmp dir: %s' %
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
   939
                (self._testtmp.decode('utf-8'),
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
   940
                 self._threadtmp.decode('utf-8')))
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   941
        else:
21461
a46a91989d57 run-tests: ignore failures from rmtree
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21460
diff changeset
   942
            shutil.rmtree(self._testtmp, True)
21497
798c81e32b5e run-tests: refactor temporary directories in Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21496
diff changeset
   943
            shutil.rmtree(self._threadtmp, True)
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
   944
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   945
        if self._usechg:
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   946
            # chgservers will stop automatically after they find the socket
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   947
            # files are deleted
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   948
            shutil.rmtree(self._chgsockdir, True)
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
   949
21455
0f0bace82149 run-tests: move err file writing to tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21454
diff changeset
   950
        if (self._ret != 0 or self._out != self._refout) and not self._skipped \
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
   951
            and not self._debug and self._out:
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
   952
            with open(self.errpath, 'wb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
   953
                for line in self._out:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
   954
                    f.write(line)
21455
0f0bace82149 run-tests: move err file writing to tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21454
diff changeset
   955
24926
3fe1e07f1a32 run-test: include test name in the return vlog
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   956
        vlog("# Ret was:", self._ret, '(%s)' % self.name)
21452
1517c0461b75 run-tests: move some functionality to Test.tearDown()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21451
diff changeset
   957
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
   958
    def _run(self, env):
21339
de25e968b4d8 run-tests: refactor runone() into gettest() and scheduletests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21338
diff changeset
   959
        # This should be implemented in child classes to run tests.
32932
240183a04429 tests: use unittest.SkipTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32907
diff changeset
   960
        raise unittest.SkipTest('unknown test type')
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
   961
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   962
    def abort(self):
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   963
        """Terminate execution of this test."""
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   964
        self._aborted = True
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
   965
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
   966
    def _portmap(self, i):
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
   967
        offset = b'' if i == 0 else b'%d' % i
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
   968
        return (br':%d\b' % (self._startport + i), b':$HGPORT%s' % offset)
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
   969
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
   970
    def _getreplacements(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   971
        """Obtain a mapping of text replacements to apply to test output.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   972
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   973
        Test output needs to be normalized so it can be compared to expected
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   974
        output. This function defines how some of that normalization will
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   975
        occur.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
   976
        """
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
   977
        r = [
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
   978
            # This list should be parallel to defineport in _getenv
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
   979
            self._portmap(0),
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
   980
            self._portmap(1),
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
   981
            self._portmap(2),
31008
636cf3f7620d tests: use LOCALIP
Jun Wu <quark@fb.com>
parents: 31006
diff changeset
   982
            (br'([^0-9])%s' % re.escape(self._localip()), br'\1$LOCALIP'),
31741
728d37353e1e run-tests: auto-replace 'TXNID' output
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31673
diff changeset
   983
            (br'\bHG_TXNID=TXN:[a-f0-9]{40}\b', br'HG_TXNID=TXN:$ID$'),
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
   984
            ]
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
   985
        r.append((self._escapepath(self._testtmp), b'$TESTTMP'))
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
   986
35194
779c6cf2967b run-tests: avoid calculating _testdir again
Martin von Zweigbergk <martinvonz@google.com>
parents: 35193
diff changeset
   987
        replacementfile = os.path.join(self._testdir, b'common-pattern.py')
35069
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 35066
diff changeset
   988
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 35066
diff changeset
   989
        if os.path.exists(replacementfile):
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 35066
diff changeset
   990
            data = {}
35092
1ac4c0887de4 run-test: drop 'execfile' usage for 'common-pattern.py' file
Boris Feld <boris.feld@octobus.net>
parents: 35091
diff changeset
   991
            with open(replacementfile, mode='rb') as source:
1ac4c0887de4 run-test: drop 'execfile' usage for 'common-pattern.py' file
Boris Feld <boris.feld@octobus.net>
parents: 35091
diff changeset
   992
                # the intermediate 'compile' step help with debugging
1ac4c0887de4 run-test: drop 'execfile' usage for 'common-pattern.py' file
Boris Feld <boris.feld@octobus.net>
parents: 35091
diff changeset
   993
                code = compile(source.read(), replacementfile, 'exec')
1ac4c0887de4 run-test: drop 'execfile' usage for 'common-pattern.py' file
Boris Feld <boris.feld@octobus.net>
parents: 35091
diff changeset
   994
                exec(code, data)
35991
04304b779df1 tests: raise a better error when patterns are wrongly formatted
Boris Feld <boris.feld@octobus.net>
parents: 35855
diff changeset
   995
                for value in data.get('substitutions', ()):
04304b779df1 tests: raise a better error when patterns are wrongly formatted
Boris Feld <boris.feld@octobus.net>
parents: 35855
diff changeset
   996
                    if len(value) != 2:
04304b779df1 tests: raise a better error when patterns are wrongly formatted
Boris Feld <boris.feld@octobus.net>
parents: 35855
diff changeset
   997
                        msg = 'malformatted substitution in %s: %r'
04304b779df1 tests: raise a better error when patterns are wrongly formatted
Boris Feld <boris.feld@octobus.net>
parents: 35855
diff changeset
   998
                        msg %= (replacementfile, value)
04304b779df1 tests: raise a better error when patterns are wrongly formatted
Boris Feld <boris.feld@octobus.net>
parents: 35855
diff changeset
   999
                        raise ValueError(msg)
04304b779df1 tests: raise a better error when patterns are wrongly formatted
Boris Feld <boris.feld@octobus.net>
parents: 35855
diff changeset
  1000
                    r.append(value)
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
  1001
        return r
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
  1002
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
  1003
    def _escapepath(self, p):
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
  1004
        if os.name == 'nt':
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
  1005
            return (
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1006
                (b''.join(c.isalpha() and b'[%s%s]' % (c.lower(), c.upper()) or
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1007
                    c in b'/\\' and br'[/\\]' or c.isdigit() and c or b'\\' + c
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
  1008
                    for c in p))
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
  1009
            )
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
  1010
        else:
28055
92e8e3f20a6f run-tests: factor out _escapepath
timeless <timeless@mozdev.org>
parents: 28037
diff changeset
  1011
            return re.escape(p)
21298
ba4750352180 run-tests: move replacements generation into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21297
diff changeset
  1012
31006
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1013
    def _localip(self):
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1014
        if self._useipv6:
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1015
            return b'::1'
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1016
        else:
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1017
            return b'127.0.0.1'
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1018
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1019
    def _genrestoreenv(self, testenv):
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1020
        """Generate a script that can be used by tests to restore the original
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1021
        environment."""
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1022
        # Put the restoreenv script inside self._threadtmp
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1023
        scriptpath = os.path.join(self._threadtmp, b'restoreenv.sh')
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1024
        testenv['HGTEST_RESTOREENV'] = scriptpath
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1025
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1026
        # Only restore environment variable names that the shell allows
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1027
        # us to export.
33140
f458a6701983 tests: fix variable name regular expression in _genrestoreenv()
Adam Simpkins <simpkins@fb.com>
parents: 33126
diff changeset
  1028
        name_regex = re.compile('^[a-zA-Z][a-zA-Z0-9_]*$')
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1029
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
  1030
        # Do not restore these variables; otherwise tests would fail.
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
  1031
        reqnames = {'PYTHON', 'TESTDIR', 'TESTTMP'}
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
  1032
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1033
        with open(scriptpath, 'w') as envf:
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
  1034
            for name, value in origenviron.items():
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1035
                if not name_regex.match(name):
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1036
                    # Skip environment variables with unusual names not
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1037
                    # allowed by most shells.
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1038
                    continue
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
  1039
                if name in reqnames:
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
  1040
                    continue
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1041
                envf.write('%s=%s\n' % (name, shellquote(value)))
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1042
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1043
            for name in testenv:
33203
cf826b9e9ea4 tests: actually restore the original environment before running syshg
Yuya Nishihara <yuya@tcha.org>
parents: 33140
diff changeset
  1044
                if name in origenviron or name in reqnames:
33126
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1045
                    continue
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1046
                envf.write('unset %s\n' % (name,))
98e2c78e309c tests: more completely restore the environment in syshgenv
Adam Simpkins <simpkins@fb.com>
parents: 33115
diff changeset
  1047
21514
59fe123dbb00 run-tests: refactor port number declaration
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21513
diff changeset
  1048
    def _getenv(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  1049
        """Obtain environment variables to use during test execution."""
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
  1050
        def defineport(i):
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
  1051
            offset = '' if i == 0 else '%s' % i
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
  1052
            env["HGPORT%s" % offset] = '%s' % (self._startport + i)
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1053
        env = os.environ.copy()
35540
f948c5b3f5c9 run-tests: avoid set PYTHONUSERBASE environment variable to None
Mihai Popescu <mihai@unity3d.com>
parents: 34842
diff changeset
  1054
        env['PYTHONUSERBASE'] = sysconfig.get_config_var('userbase') or ''
31950
cc70c6dbac30 util: add a way to issue deprecation warning without a UI object
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 31829
diff changeset
  1055
        env['HGEMITWARNINGS'] = '1'
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
  1056
        env['TESTTMP'] = self._testtmp
36037
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  1057
        env['TESTNAME'] = self.name
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
  1058
        env['HOME'] = self._testtmp
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
  1059
        # This number should match portneeded in _getport
28170
bc010fcd836b run-tests: stop allocating HGPORT3+HGPORT4
timeless <timeless@mozdev.org>
parents: 28169
diff changeset
  1060
        for port in xrange(3):
28169
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
  1061
            # This list should be parallel to _portmap in _getreplacements
1b07331f5900 run-tests: refactor port allocation into functions
timeless <timeless@mozdev.org>
parents: 28143
diff changeset
  1062
            defineport(port)
25034
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
  1063
        env["HGRCPATH"] = os.path.join(self._threadtmp, b'.hgrc')
af6e6a0781d7 run-tests: use bytes explicitly for tmpdir and hgrc construction
Augie Fackler <augie@google.com>
parents: 25033
diff changeset
  1064
        env["DAEMON_PIDS"] = os.path.join(self._threadtmp, b'daemon.pids')
23347
49cdf51cbc6c run-tests: include quotes in the HGEDITOR value when storing sys.executable
Matt Harbison <matt_harbison@yahoo.com>
parents: 23263
diff changeset
  1065
        env["HGEDITOR"] = ('"' + sys.executable + '"'
49cdf51cbc6c run-tests: include quotes in the HGEDITOR value when storing sys.executable
Matt Harbison <matt_harbison@yahoo.com>
parents: 23263
diff changeset
  1066
                           + ' -c "import sys; sys.exit(0)"')
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1067
        env["HGMERGE"] = "internal:merge"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1068
        env["HGUSER"]   = "test"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1069
        env["HGENCODING"] = "ascii"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1070
        env["HGENCODINGMODE"] = "strict"
31003
225f574e0645 runtests: export HGIPV6 to hint test scripts whether to use IPv6
Jun Wu <quark@fb.com>
parents: 31002
diff changeset
  1071
        env['HGIPV6'] = str(int(self._useipv6))
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1072
37342
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1073
        extraextensions = []
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1074
        for opt in self._extraconfigopts:
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1075
            section, key = opt.encode('utf-8').split(b'.', 1)
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1076
            if section != 'extensions':
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1077
                continue
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1078
            name = key.split(b'=', 1)[0]
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1079
            extraextensions.append(name)
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1080
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1081
        if extraextensions:
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1082
            env['HGTESTEXTRAEXTENSIONS'] = b' '.join(extraextensions)
4e6a6d0dccee tests: conditionalize tests based on presence of custom extensions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37009
diff changeset
  1083
31006
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1084
        # LOCALIP could be ::1 or 127.0.0.1. Useful for tests that require raw
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1085
        # IP addresses.
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1086
        env['LOCALIP'] = self._localip()
d4916aebf3d0 runtests: export LOCALIP
Jun Wu <quark@fb.com>
parents: 31003
diff changeset
  1087
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1088
        # Reset some environment variables to well-known values so that
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1089
        # the tests produce repeatable output.
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1090
        env['LANG'] = env['LC_ALL'] = env['LANGUAGE'] = 'C'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1091
        env['TZ'] = 'GMT'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1092
        env["EMAIL"] = "Foo Bar <foo.bar@example.com>"
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1093
        env['COLUMNS'] = '80'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1094
        env['TERM'] = 'xterm'
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1095
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1096
        for k in ('HG HGPROF CDPATH GREP_OPTIONS http_proxy no_proxy ' +
31635
632e07e72a22 runtests: unset editor and pager related environment variables
Jun Wu <quark@fb.com>
parents: 31227
diff changeset
  1097
                  'HGPLAIN HGPLAINEXCEPT EDITOR VISUAL PAGER ' +
30716
3de9df6ee5bf run-tests: unset CHGDEBUG
Jun Wu <quark@fb.com>
parents: 30420
diff changeset
  1098
                  'NO_PROXY CHGDEBUG').split():
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1099
            if k in env:
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1100
                del env[k]
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1101
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1102
        # unset env related to hooks
36522
c3df20906689 tests: fix run-tests environment cleanup on Python 3
Augie Fackler <augie@google.com>
parents: 36462
diff changeset
  1103
        for k in list(env):
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1104
            if k.startswith('HG_'):
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1105
                del env[k]
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1106
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
  1107
        if self._usechg:
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
  1108
            env['CHGSOCKNAME'] = os.path.join(self._chgsockdir, b'server')
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
  1109
21299
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1110
        return env
7861de61583b run-tests: move createenv() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21298
diff changeset
  1111
21382
4b8ffe3abdd2 run-tests: move createhgrc into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21381
diff changeset
  1112
    def _createhgrc(self, path):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  1113
        """Create an hgrc file for this test."""
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1114
        with open(path, 'wb') as hgrc:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1115
            hgrc.write(b'[ui]\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1116
            hgrc.write(b'slash = True\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1117
            hgrc.write(b'interactive = False\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1118
            hgrc.write(b'mergemarkers = detailed\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1119
            hgrc.write(b'promptecho = True\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1120
            hgrc.write(b'[defaults]\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1121
            hgrc.write(b'[devel]\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1122
            hgrc.write(b'all-warnings = true\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1123
            hgrc.write(b'default-date = 0 0\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1124
            hgrc.write(b'[largefiles]\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1125
            hgrc.write(b'usercache = %s\n' %
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1126
                       (os.path.join(self._testtmp, b'.cache/largefiles')))
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1127
            hgrc.write(b'[lfs]\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1128
            hgrc.write(b'usercache = %s\n' %
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1129
                       (os.path.join(self._testtmp, b'.cache/lfs')))
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1130
            hgrc.write(b'[web]\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1131
            hgrc.write(b'address = localhost\n')
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1132
            hgrc.write(b'ipv6 = %s\n' % str(self._useipv6).encode('ascii'))
37009
5890e5872f36 hgweb: allow defining Server response header for HTTP server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
  1133
            hgrc.write(b'server-header = testing stub value\n')
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1134
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1135
            for opt in self._extraconfigopts:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1136
                section, key = opt.encode('utf-8').split(b'.', 1)
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1137
                assert b'=' in key, ('extra config opt %s must '
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1138
                                     'have an = for assignment' % opt)
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1139
                hgrc.write(b'[%s]\n%s\n' % (section, key))
21382
4b8ffe3abdd2 run-tests: move createhgrc into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21381
diff changeset
  1140
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
  1141
    def fail(self, msg):
21522
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
  1142
        # unittest differentiates between errored and failed.
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
  1143
        # Failed is denoted by AssertionError (by default at least).
eeaec308ad5f run-tests: raise WarnTest outside of Test.fail()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21521
diff changeset
  1144
        raise AssertionError(msg)
21323
a7c677e2f6ae run-tests: move fail() into Test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21322
diff changeset
  1145
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
  1146
    def _runcommand(self, cmd, env, normalizenewlines=False):
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1147
        """Run command in a sub-process, capturing the output (stdout and
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1148
        stderr).
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1149
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1150
        Return a tuple (exitcode, output). output is None in debug mode.
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1151
        """
24509
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
  1152
        if self._debug:
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
  1153
            proc = subprocess.Popen(cmd, shell=True, cwd=self._testtmp,
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
  1154
                                    env=env)
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1155
            ret = proc.wait()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1156
            return (ret, None)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1157
24509
27092bb70293 run-tests: remove arguments from Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24508
diff changeset
  1158
        proc = Popen4(cmd, self._testtmp, self._timeout, env)
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1159
        def cleanup():
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1160
            terminate(proc)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1161
            ret = proc.wait()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1162
            if ret == 0:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1163
                ret = signal.SIGTERM << 8
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1164
            killdaemons(env['DAEMON_PIDS'])
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1165
            return ret
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1166
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1167
        output = ''
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1168
        proc.tochild.close()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1169
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1170
        try:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1171
            output = proc.fromchild.read()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1172
        except KeyboardInterrupt:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1173
            vlog('# Handling keyboard interrupt')
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1174
            cleanup()
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1175
            raise
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1176
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1177
        ret = proc.wait()
25177
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
  1178
        if wifexited(ret):
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1179
            ret = os.WEXITSTATUS(ret)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1180
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1181
        if proc.timeout:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1182
            ret = 'timeout'
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1183
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1184
        if ret:
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1185
            killdaemons(env['DAEMON_PIDS'])
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1186
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
  1187
        for s, r in self._getreplacements():
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1188
            output = re.sub(s, r, output)
24510
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
  1189
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
  1190
        if normalizenewlines:
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
  1191
            output = output.replace('\r\n', '\n')
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
  1192
24508
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1193
        return ret, output.splitlines(True)
fbe2fb71a6e6 run-tests: move run into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1194
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
  1195
class PythonTest(Test):
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
  1196
    """A Python-based test."""
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
  1197
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
  1198
    @property
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
  1199
    def refpath(self):
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  1200
        return os.path.join(self._testdir, b'%s.out' % self.bname)
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
  1201
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
  1202
    def _run(self, env):
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  1203
        py3kswitch = self._py3kwarnings and b' -3' or b''
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  1204
        cmd = b'%s%s "%s"' % (PYTHON, py3kswitch, self.path)
21311
f9a7018a35ff run-tests: roll pytest() into PythonTest._run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21310
diff changeset
  1205
        vlog("# Running", cmd)
24510
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
  1206
        normalizenewlines = os.name == 'nt'
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
  1207
        result = self._runcommand(cmd, env,
24510
8d6fd0b8f622 run-tests: separate newline normalization from replacements
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24509
diff changeset
  1208
                                  normalizenewlines=normalizenewlines)
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1209
        if self._aborted:
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1210
            raise KeyboardInterrupt()
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1211
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1212
        return result
21311
f9a7018a35ff run-tests: roll pytest() into PythonTest._run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21310
diff changeset
  1213
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1214
# Some glob patterns apply only in some circumstances, so the script
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1215
# might want to remove (glob) annotations that otherwise should be
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1216
# retained.
23352
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
  1217
checkcodeglobpats = [
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1218
    # On Windows it looks like \ doesn't require a (glob), but we know
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1219
    # better.
25059
0e0f1068b878 run-tests: fix checking a line to see if it needs globbing
Augie Fackler <augie@google.com>
parents: 25058
diff changeset
  1220
    re.compile(br'^pushing to \$TESTTMP/.*[^)]$'),
0e0f1068b878 run-tests: fix checking a line to see if it needs globbing
Augie Fackler <augie@google.com>
parents: 25058
diff changeset
  1221
    re.compile(br'^moving \S+/.*[^)]$'),
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1222
    re.compile(br'^pulling from \$TESTTMP/.*[^)]$'),
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1223
    # Not all platforms have 127.0.0.1 as loopback (though most do),
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29485
diff changeset
  1224
    # so we always glob that too.
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 31635
diff changeset
  1225
    re.compile(br'.*\$LOCALIP.*$'),
23352
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
  1226
]
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
  1227
25036
61fc2cdbc57c run-tests: work around chr() producing unicode in Python 3
Augie Fackler <augie@google.com>
parents: 25035
diff changeset
  1228
bchr = chr
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
  1229
if PYTHON3:
25036
61fc2cdbc57c run-tests: work around chr() producing unicode in Python 3
Augie Fackler <augie@google.com>
parents: 25035
diff changeset
  1230
    bchr = lambda x: bytes([x])
61fc2cdbc57c run-tests: work around chr() producing unicode in Python 3
Augie Fackler <augie@google.com>
parents: 25035
diff changeset
  1231
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
  1232
class TTest(Test):
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
  1233
    """A "t test" is a test backed by a .t file."""
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
  1234
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
  1235
    SKIPPED_PREFIX = b'skipped: '
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
  1236
    FAILED_PREFIX = b'hghave check failed: '
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1237
    NEEDESCAPE = re.compile(br'[\x00-\x08\x0b-\x1f\x7f-\xff]').search
21384
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1238
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1239
    ESCAPESUB = re.compile(br'[\x00-\x08\x0b-\x1f\\\x7f-\xff]').sub
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1240
    ESCAPEMAP = dict((bchr(i), br'\x%02x' % i) for i in range(256))
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1241
    ESCAPEMAP.update({b'\\': b'\\\\', b'\r': br'\r'})
21381
9aa5784992d4 run-tests: move SKIPPED_PREFIX and FAILED_PREFIX into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21380
diff changeset
  1242
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1243
    def __init__(self, path, *args, **kwds):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1244
        # accept an extra "case" parameter
35193
f12747de13ea run-tests: simplify by using dict.pop() with default
Martin von Zweigbergk <martinvonz@google.com>
parents: 35190
diff changeset
  1245
        case = kwds.pop('case', None)
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1246
        self._case = case
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1247
        self._allcases = parsettestcases(path)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1248
        super(TTest, self).__init__(path, *args, **kwds)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1249
        if case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1250
            self.name = '%s (case %s)' % (self.name, _strpath(case))
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1251
            self.errpath = b'%s.%s.err' % (self.errpath[:-4], case)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1252
            self._tmpname += b'-%s' % case
36462
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1253
        self._have = {}
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1254
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
  1255
    @property
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
  1256
    def refpath(self):
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1257
        return os.path.join(self._testdir, self.bname)
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
  1258
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
  1259
    def _run(self, env):
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1260
        with open(self.path, 'rb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1261
            lines = f.readlines()
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1262
32981
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
  1263
        # .t file is both reference output and the test input, keep reference
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
  1264
        # output updated with the the test input. This avoids some race
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
  1265
        # conditions where the reference output does not match the actual test.
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
  1266
        if self._refout is not None:
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
  1267
            self._refout = lines
02bca6dc5f41 run-tests: update .t reference output after reading the test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
  1268
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
  1269
        salt, script, after, expected = self._parsetest(lines)
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1270
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1271
        # Write out the generated script.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1272
        fname = b'%s.sh' % self._testtmp
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1273
        with open(fname, 'wb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1274
            for l in script:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1275
                f.write(l)
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1276
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1277
        cmd = b'%s "%s"' % (self._shell, fname)
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1278
        vlog("# Running", cmd)
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1279
24516
62fb03e0d990 run-tests: obtain replacements inside Test._runcommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24510
diff changeset
  1280
        exitcode, output = self._runcommand(cmd, env)
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1281
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1282
        if self._aborted:
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1283
            raise KeyboardInterrupt()
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1284
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1285
        # Do not merge output if skipped. Return hghave message instead.
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1286
        # Similarly, with --debug, output is None.
21380
de6ea36ca9f7 run-tests: move SKIPPED_STATUS into Test class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21379
diff changeset
  1287
        if exitcode == self.SKIPPED_STATUS or output is None:
21313
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1288
            return exitcode, output
a2bd02a3b6d2 run-tests: move t test execution from tsttest() to TTest.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21312
diff changeset
  1289
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1290
        return self._processoutput(exitcode, output, salt, after, expected)
21296
cd8776030833 run-tests: create classes for representing tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21231
diff changeset
  1291
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
  1292
    def _hghave(self, reqs):
36462
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1293
        allreqs = b' '.join(reqs)
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1294
        if allreqs in self._have:
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1295
            return self._have.get(allreqs)
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1296
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1297
        # TODO do something smarter when all other uses of hghave are gone.
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25388
diff changeset
  1298
        runtestdir = os.path.abspath(os.path.dirname(_bytespath(__file__)))
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25388
diff changeset
  1299
        tdir = runtestdir.replace(b'\\', b'/')
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1300
        proc = Popen4(b'%s -c "%s/hghave %s"' %
36462
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1301
                      (self._shell, tdir, allreqs),
23933
769027075e21 run-tests.py: execute hghave with same env vars as ones for actual tests
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23859
diff changeset
  1302
                      self._testtmp, 0, self._getenv())
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1303
        stdout, stderr = proc.communicate()
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1304
        ret = proc.wait()
25177
c3459555318e run-tests: resurrect the wifexited polyfill (backout 6ab5a1c9ea3c)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25162
diff changeset
  1305
        if wifexited(ret):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1306
            ret = os.WEXITSTATUS(ret)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1307
        if ret == 2:
28699
5cc59dbd199f py3: convert hghave output to text
timeless <timeless@mozdev.org>
parents: 28698
diff changeset
  1308
            print(stdout.decode('utf-8'))
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1309
            sys.exit(1)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1310
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
  1311
        if ret != 0:
36462
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1312
            self._have[allreqs] = (False, stdout)
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27396
diff changeset
  1313
            return False, stdout
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
  1314
34265
df1c290df034 tests: fix run-tests "slow test" check
Augie Fackler <augie@google.com>
parents: 34264
diff changeset
  1315
        if b'slow' in reqs:
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27101
diff changeset
  1316
            self._timeout = self._slowtimeout
36462
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1317
5c1cea8a3e60 run-tests: cache hghave results
Matt Harbison <matt_harbison@yahoo.com>
parents: 36461
diff changeset
  1318
        self._have[allreqs] = (True, None)
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27396
diff changeset
  1319
        return True, None
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1320
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1321
    def _iftest(self, args):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1322
        # implements "#if"
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1323
        reqs = []
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1324
        for arg in args:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1325
            if arg.startswith(b'no-') and arg[3:] in self._allcases:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1326
                if arg[3:] == self._case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1327
                    return False
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1328
            elif arg in self._allcases:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1329
                if arg != self._case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1330
                    return False
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1331
            else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1332
                reqs.append(arg)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1333
        return self._hghave(reqs)[0]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1334
21454
046587aa1c8a run-tests: refactor testtmp
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21453
diff changeset
  1335
    def _parsetest(self, lines):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1336
        # We generate a shell script which outputs unique markers to line
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1337
        # up script results with our source. These markers include input
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1338
        # line number and the last return code.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1339
        salt = b"SALT%d" % time.time()
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1340
        def addsalt(line, inpython):
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1341
            if inpython:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1342
                script.append(b'%s %d 0\n' % (salt, line))
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1343
            else:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1344
                script.append(b'echo %s %d $?\n' % (salt, line))
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1345
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1346
        script = []
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1347
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1348
        # After we run the shell script, we re-unify the script output
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1349
        # with non-active parts of the source, with synchronization by our
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1350
        # SALT line number markers. The after table contains the non-active
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1351
        # components, ordered by line number.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1352
        after = {}
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1353
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1354
        # Expected shell script output.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1355
        expected = {}
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1356
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1357
        pos = prepos = -1
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1358
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1359
        # True or False when in a true or false conditional section
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1360
        skipping = None
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1361
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1362
        # We keep track of whether or not we're in a Python block so we
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1363
        # can generate the surrounding doctest magic.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1364
        inpython = False
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1365
21510
97127c4ce460 run-tests: move debug into an argument to Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21509
diff changeset
  1366
        if self._debug:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
  1367
            script.append(b'set -x\n')
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
  1368
        if self._hgcommand != b'hg':
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
  1369
            script.append(b'alias hg="%s"\n' % self._hgcommand)
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1370
        if os.getenv('MSYSTEM'):
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
  1371
            script.append(b'alias pwd="pwd -W"\n')
35536
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35489
diff changeset
  1372
        if self._case:
35823
4be991331a46 tests: get run-tests to reliably hand shellquote a string and not a bytes
Augie Fackler <augie@google.com>
parents: 35751
diff changeset
  1373
            if isinstance(self._case, str):
4be991331a46 tests: get run-tests to reliably hand shellquote a string and not a bytes
Augie Fackler <augie@google.com>
parents: 35751
diff changeset
  1374
                quoted = shellquote(self._case)
4be991331a46 tests: get run-tests to reliably hand shellquote a string and not a bytes
Augie Fackler <augie@google.com>
parents: 35751
diff changeset
  1375
            else:
4be991331a46 tests: get run-tests to reliably hand shellquote a string and not a bytes
Augie Fackler <augie@google.com>
parents: 35751
diff changeset
  1376
                quoted = shellquote(self._case.decode('utf8')).encode('utf8')
4be991331a46 tests: get run-tests to reliably hand shellquote a string and not a bytes
Augie Fackler <augie@google.com>
parents: 35751
diff changeset
  1377
            script.append(b'TESTCASE=%s\n' % quoted)
35536
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35489
diff changeset
  1378
            script.append(b'export TESTCASE\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1379
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
  1380
        n = 0
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1381
        for n, l in enumerate(lines):
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1382
            if not l.endswith(b'\n'):
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1383
                l += b'\n'
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1384
            if l.startswith(b'#require'):
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
  1385
                lsplit = l.split()
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1386
                if len(lsplit) < 2 or lsplit[0] != b'#require':
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
  1387
                    after.setdefault(pos, []).append('  !!! invalid #require\n')
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36665
diff changeset
  1388
                if not skipping:
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36665
diff changeset
  1389
                    haveresult, message = self._hghave(lsplit[1:])
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36665
diff changeset
  1390
                    if not haveresult:
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36665
diff changeset
  1391
                        script = [b'echo "%s"\nexit 80\n' % message]
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36665
diff changeset
  1392
                        break
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
  1393
                after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1394
            elif l.startswith(b'#if'):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1395
                lsplit = l.split()
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1396
                if len(lsplit) < 2 or lsplit[0] != b'#if':
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1397
                    after.setdefault(pos, []).append('  !!! invalid #if\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1398
                if skipping is not None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1399
                    after.setdefault(pos, []).append('  !!! nested #if\n')
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  1400
                skipping = not self._iftest(lsplit[1:])
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1401
                after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1402
            elif l.startswith(b'#else'):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1403
                if skipping is None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1404
                    after.setdefault(pos, []).append('  !!! missing #if\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1405
                skipping = not skipping
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1406
                after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1407
            elif l.startswith(b'#endif'):
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1408
                if skipping is None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1409
                    after.setdefault(pos, []).append('  !!! missing #if\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1410
                skipping = None
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1411
                after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1412
            elif skipping:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1413
                after.setdefault(pos, []).append(l)
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1414
            elif l.startswith(b'  >>> '): # python inlines
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1415
                after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1416
                prepos = pos
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1417
                pos = n
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1418
                if not inpython:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1419
                    # We've just entered a Python block. Add the header.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1420
                    inpython = True
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1421
                    addsalt(prepos, False) # Make sure we report the exit code.
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  1422
                    script.append(b'%s -m heredoctest <<EOF\n' % PYTHON)
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1423
                addsalt(n, True)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1424
                script.append(l[2:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1425
            elif l.startswith(b'  ... '): # python inlines
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1426
                after.setdefault(prepos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1427
                script.append(l[2:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1428
            elif l.startswith(b'  $ '): # commands
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1429
                if inpython:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
  1430
                    script.append(b'EOF\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1431
                    inpython = False
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1432
                after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1433
                prepos = pos
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1434
                pos = n
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1435
                addsalt(n, False)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1436
                cmd = l[4:].split()
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
  1437
                if len(cmd) == 2 and cmd[0] == b'cd':
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
  1438
                    l = b'  $ cd %s || exit 1\n' % cmd[1]
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1439
                script.append(l[4:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1440
            elif l.startswith(b'  > '): # continuations
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1441
                after.setdefault(prepos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1442
                script.append(l[4:])
25035
1203ca7005fa run-tests: use bytes when constructing shell script
Augie Fackler <augie@google.com>
parents: 25034
diff changeset
  1443
            elif l.startswith(b'  '): # results
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1444
                # Queue up a list of expected results.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1445
                expected.setdefault(pos, []).append(l[2:])
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1446
            else:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1447
                if inpython:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
  1448
                    script.append(b'EOF\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1449
                    inpython = False
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1450
                # Non-command/result. Queue up for merged output.
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1451
                after.setdefault(pos, []).append(l)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1452
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1453
        if inpython:
25060
29e54fe22a3f run-tests: make sure all script lines are bytes
Augie Fackler <augie@google.com>
parents: 25059
diff changeset
  1454
            script.append(b'EOF\n')
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1455
        if skipping is not None:
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1456
            after.setdefault(pos, []).append('  !!! missing #endif\n')
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1457
        addsalt(n + 1, False)
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1458
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1459
        return salt, script, after, expected
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1460
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1461
    def _processoutput(self, exitcode, output, salt, after, expected):
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1462
        # Merge the script output back into a unified test.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1463
        warnonly = 1 # 1: not yet; 2: yes; 3: for sure not
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1464
        if exitcode != 0:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1465
            warnonly = 3
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1466
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1467
        pos = -1
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1468
        postout = []
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1469
        for l in output:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1470
            lout, lcmd = l, None
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1471
            if salt in l:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1472
                lout, lcmd = l.split(salt, 1)
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1473
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
  1474
            while lout:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1475
                if not lout.endswith(b'\n'):
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1476
                    lout += b' (no-eol)\n'
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1477
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1478
                # Find the expected output at the current position.
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1479
                els = [None]
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1480
                if expected.get(pos, None):
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1481
                    els = expected[pos]
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1482
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1483
                i = 0
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
  1484
                optional = []
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1485
                while i < len(els):
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1486
                    el = els[i]
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1487
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1488
                    r = self.linematch(el, lout)
28568
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1489
                    if isinstance(r, str):
35382
dfae14354660 run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents: 35381
diff changeset
  1490
                        if r == '-glob':
28568
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1491
                            lout = ''.join(el.rsplit(' (glob)', 1))
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1492
                            r = '' # Warn only this line.
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1493
                        elif r == "retry":
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1494
                            postout.append(b'  ' + el)
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1495
                            els.pop(i)
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1496
                            break
28568
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1497
                        else:
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1498
                            log('\ninfo, unknown linematch result: %r\n' % r)
4a908089fe29 run-tests: indent _processoutput to aid readability for next patch
timeless <timeless@mozdev.org>
parents: 28455
diff changeset
  1499
                            r = False
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1500
                    if r:
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1501
                        els.pop(i)
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1502
                        break
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1503
                    if el:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1504
                        if el.endswith(b" (?)\n"):
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1505
                            optional.append(i)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1506
                        else:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1507
                            m = optline.match(el)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1508
                            if m:
33710
2e43c5cd57a7 tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
  1509
                                conditions = [
2e43c5cd57a7 tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
  1510
                                    c for c in m.group(2).split(b' ')]
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1511
33934
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33931
diff changeset
  1512
                                if not self._iftest(conditions):
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1513
                                    optional.append(i)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1514
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1515
                    i += 1
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1516
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1517
                if r:
28569
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1518
                    if r == "retry":
1ad0ddf8cccc run-tests: teach _processoutput to handle multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28568
diff changeset
  1519
                        continue
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
  1520
                    # clean up any optional leftovers
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
  1521
                    for i in optional:
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
  1522
                        postout.append(b'  ' + els[i])
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
  1523
                    for i in reversed(optional):
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28699
diff changeset
  1524
                        del els[i]
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1525
                    postout.append(b'  ' + el)
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1526
                else:
21384
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1527
                    if self.NEEDESCAPE(lout):
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1528
                        lout = TTest._stringescape(b'%s (esc)\n' %
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1529
                                                   lout.rstrip(b'\n'))
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1530
                    postout.append(b'  ' + lout) # Let diff deal with it.
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1531
                    if r != '': # If line failed.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1532
                        warnonly = 3 # for sure not
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1533
                    elif warnonly == 1: # Is "not yet" and line is warn only.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1534
                        warnonly = 2 # Yes do warn.
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
  1535
                break
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
  1536
            else:
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
  1537
                # clean up any optional leftovers
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
  1538
                while expected.get(pos, None):
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
  1539
                    el = expected[pos].pop(0)
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1540
                    if el:
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1541
                        if not el.endswith(b" (?)\n"):
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1542
                            m = optline.match(el)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1543
                            if m:
33871
9e8b01d0654f tests: fixed a bytes/unicode confusion in the test runner
Alex Gaynor <alex.gaynor@gmail.com>
parents: 33870
diff changeset
  1544
                                conditions = [c for c in m.group(2).split(b' ')]
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1545
33934
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33931
diff changeset
  1546
                                if self._iftest(conditions):
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1547
                                    # Don't append as optional line
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1548
                                    continue
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1549
                            else:
33696
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
  1550
                                continue
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28284
diff changeset
  1551
                    postout.append(b'  ' + el)
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1552
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1553
            if lcmd:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1554
                # Add on last return code.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1555
                ret = int(lcmd.split()[1])
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1556
                if ret != 0:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1557
                    postout.append(b'  [%d]\n' % ret)
21314
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1558
                if pos in after:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1559
                    # Merge in non-active test bits.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1560
                    postout += after.pop(pos)
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1561
                pos = int(lcmd.split()[0])
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1562
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1563
        if pos in after:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1564
            postout += after.pop(pos)
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1565
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1566
        if warnonly == 2:
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1567
            exitcode = False # Set exitcode to warned.
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1568
76d7967d8f3d run-tests: finish moving tsttest() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21313
diff changeset
  1569
        return exitcode, postout
21312
986b8a58a6d3 run-tests: move t test parsing into its own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21311
diff changeset
  1570
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
  1571
    @staticmethod
21316
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1572
    def rematch(el, l):
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1573
        try:
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
  1574
            el = b'(?:' + el + b')'
21316
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1575
            # use \Z to ensure that the regex matches to the end of the string
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1576
            if os.name == 'nt':
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1577
                return re.match(el + br'\r?\n\Z', l)
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1578
            return re.match(el + br'\n\Z', l)
21316
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1579
        except re.error:
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1580
            # el is an invalid regex
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1581
            return False
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1582
ab9bf8a5e573 run-tests: make rematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21315
diff changeset
  1583
    @staticmethod
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1584
    def globmatch(el, l):
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1585
        # The only supported special characters are * and ? plus / which also
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1586
        # matches \ on windows. Escaping of these characters is supported.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1587
        if el + b'\n' == l:
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1588
            if os.altsep:
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1589
                # matching on "/" is not needed for this line
23352
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
  1590
                for pat in checkcodeglobpats:
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
  1591
                    if pat.match(el):
5bd04faaa3ee run-tests: don't warn on unnecessary globs mandated by check-code.py
Matt Harbison <matt_harbison@yahoo.com>
parents: 23347
diff changeset
  1592
                        return True
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1593
                return b'-glob'
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1594
            return True
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 31635
diff changeset
  1595
        el = el.replace(b'$LOCALIP', b'*')
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1596
        i, n = 0, len(el)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1597
        res = b''
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1598
        while i < n:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1599
            c = el[i:i + 1]
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1600
            i += 1
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1601
            if c == b'\\' and i < n and el[i:i + 1] in b'*?\\/':
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1602
                res += el[i - 1:i + 1]
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1603
                i += 1
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1604
            elif c == b'*':
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1605
                res += b'.*'
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1606
            elif c == b'?':
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1607
                res += b'.'
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1608
            elif c == b'/' and os.altsep:
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1609
                res += b'[/\\\\]'
21317
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1610
            else:
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1611
                res += re.escape(c)
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1612
        return TTest.rematch(res, l)
58a599784a0c run-tests: make globmatch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21316
diff changeset
  1613
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1614
    def linematch(self, el, l):
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
  1615
        retry = False
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
  1616
        if el == l: # perfect match (fast)
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
  1617
            return True
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
  1618
        if el:
26612
a38924f7680c run-tests: add b-prefix on two strings to fix python3 support
Augie Fackler <augie@google.com>
parents: 26422
diff changeset
  1619
            if el.endswith(b" (?)\n"):
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
  1620
                retry = "retry"
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
  1621
                el = el[:-5] + b"\n"
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1622
            else:
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1623
                m = optline.match(el)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1624
                if m:
33710
2e43c5cd57a7 tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents: 33696
diff changeset
  1625
                    conditions = [c for c in m.group(2).split(b' ')]
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1626
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1627
                    el = m.group(1) + b"\n"
33934
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33931
diff changeset
  1628
                    if not self._iftest(conditions):
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33592
diff changeset
  1629
                        retry = "retry"    # Not required by listed features
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
  1630
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1631
            if el.endswith(b" (esc)\n"):
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
  1632
                if PYTHON3:
25047
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
  1633
                    el = el[:-7].decode('unicode_escape') + '\n'
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
  1634
                    el = el.encode('utf-8')
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
  1635
                else:
8c7e938c8146 run-tests: string-escape no longer exists in python 3, use unicode_escape
Augie Fackler <augie@google.com>
parents: 25046
diff changeset
  1636
                    el = el[:-7].decode('string-escape') + '\n'
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1637
            if el == l or os.name == 'nt' and el[:-1] + b'\r\n' == l:
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
  1638
                return True
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1639
            if el.endswith(b" (re)\n"):
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
  1640
                return TTest.rematch(el[:-6], l) or retry
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1641
            if el.endswith(b" (glob)\n"):
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23388
diff changeset
  1642
                # ignore '(glob)' added to l by 'replacements'
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1643
                if l.endswith(b" (glob)\n"):
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  1644
                    l = l[:-8] + b"\n"
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31741
diff changeset
  1645
                return TTest.globmatch(el[:-8], l) or retry
35381
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 35280
diff changeset
  1646
            if os.altsep:
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 35280
diff changeset
  1647
                _l = l.replace(b'\\', b'/')
14fd435763ee run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents: 35280
diff changeset
  1648
                if el == _l or os.name == 'nt' and el[:-1] + b'\r\n' == _l:
35382
dfae14354660 run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents: 35381
diff changeset
  1649
                    return True
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25290
diff changeset
  1650
        return retry
21315
56610da39b48 run-tests: make linematch a static method of TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21314
diff changeset
  1651
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1652
    @staticmethod
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1653
    def parsehghaveoutput(lines):
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1654
        '''Parse hghave log lines.
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1655
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1656
        Return tuple of lists (missing, failed):
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1657
          * the missing/unknown features
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1658
          * the features for which existence check failed'''
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1659
        missing = []
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1660
        failed = []
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1661
        for line in lines:
21381
9aa5784992d4 run-tests: move SKIPPED_PREFIX and FAILED_PREFIX into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21380
diff changeset
  1662
            if line.startswith(TTest.SKIPPED_PREFIX):
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1663
                line = line.splitlines()[0]
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
  1664
                missing.append(line[len(TTest.SKIPPED_PREFIX):].decode('utf-8'))
21381
9aa5784992d4 run-tests: move SKIPPED_PREFIX and FAILED_PREFIX into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21380
diff changeset
  1665
            elif line.startswith(TTest.FAILED_PREFIX):
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1666
                line = line.splitlines()[0]
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
  1667
                failed.append(line[len(TTest.FAILED_PREFIX):].decode('utf-8'))
21379
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1668
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1669
        return missing, failed
ab1a95270a50 run-tests: move parsehghaveoutput() into TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21378
diff changeset
  1670
21384
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1671
    @staticmethod
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1672
    def _escapef(m):
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1673
        return TTest.ESCAPEMAP[m.group(0)]
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1674
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1675
    @staticmethod
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1676
    def _stringescape(s):
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1677
        return TTest.ESCAPESUB(TTest._escapef, s)
a36cc85a5b7b run-tests: move string escaping to TTest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21383
diff changeset
  1678
22104
70bdf59d27b6 run-tests: attempt to fix iolock handling
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
  1679
iolock = threading.RLock()
35541
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
  1680
firstlock = threading.RLock()
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
  1681
firsterror = False
14000
636a6f5aa2cd run-tests: add locking on results struct
Matt Mackall <mpm@selenic.com>
parents: 13999
diff changeset
  1682
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  1683
class TestResult(unittest._TextTestResult):
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  1684
    """Holds results when executing via unittest."""
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  1685
    # Don't worry too much about accessing the non-public _TextTestResult.
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  1686
    # It is relatively common in Python testing tools.
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1687
    def __init__(self, options, *args, **kwargs):
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  1688
        super(TestResult, self).__init__(*args, **kwargs)
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  1689
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1690
        self._options = options
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1691
21430
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1692
        # unittest.TestResult didn't have skipped until 2.7. We need to
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1693
        # polyfill it.
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1694
        self.skipped = []
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1695
21431
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1696
        # We have a custom "ignored" result that isn't present in any Python
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1697
        # unittest implementation. It is very similar to skipped. It may make
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1698
        # sense to map it into skip some day.
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1699
        self.ignored = []
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1700
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1701
        self.times = []
27637
b502138f5faa cleanup: remove superfluous space after space after equals (python)
timeless <timeless@mozdev.org>
parents: 27636
diff changeset
  1702
        self._firststarttime = None
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1703
        # Data stored for the benefit of generating xunit reports.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1704
        self.successes = []
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1705
        self.faildata = {}
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1706
33565
0982d900dccb run-tests: pass color option via test case object , not global var
Martin von Zweigbergk <martinvonz@google.com>
parents: 33561
diff changeset
  1707
        if options.color == 'auto':
33568
a2c35146596b run-tests: remove unnecessary 'with_color' variable
Martin von Zweigbergk <martinvonz@google.com>
parents: 33567
diff changeset
  1708
            self.color = pygmentspresent and self.stream.isatty()
33565
0982d900dccb run-tests: pass color option via test case object , not global var
Martin von Zweigbergk <martinvonz@google.com>
parents: 33561
diff changeset
  1709
        elif options.color == 'never':
0982d900dccb run-tests: pass color option via test case object , not global var
Martin von Zweigbergk <martinvonz@google.com>
parents: 33561
diff changeset
  1710
            self.color = False
0982d900dccb run-tests: pass color option via test case object , not global var
Martin von Zweigbergk <martinvonz@google.com>
parents: 33561
diff changeset
  1711
        else: # 'always', for testing purposes
0982d900dccb run-tests: pass color option via test case object , not global var
Martin von Zweigbergk <martinvonz@google.com>
parents: 33561
diff changeset
  1712
            self.color = pygmentspresent
33561
2893face0af5 run-tests: check if stream is a tty before using color
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33552
diff changeset
  1713
21462
8a4ef661f08d run-tests: make failure reporting in unittest mode equivalent to default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21461
diff changeset
  1714
    def addFailure(self, test, reason):
8a4ef661f08d run-tests: make failure reporting in unittest mode equivalent to default
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21461
diff changeset
  1715
        self.failures.append((test, reason))
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1716
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1717
        if self._options.first:
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1718
            self.stop()
21735
5ee547fdb0be run-tests: produce error on running a failing test
anuraggoel <anurag.dsps@gmail.com>
parents: 21733
diff changeset
  1719
        else:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1720
            with iolock:
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
  1721
                if reason == "timed out":
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
  1722
                    self.stream.write('t')
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
  1723
                else:
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
  1724
                    if not self._options.nodiff:
34842
8bce3e51b101 run-tests: move newline out of colorized message
Martin von Zweigbergk <martinvonz@google.com>
parents: 34804
diff changeset
  1725
                        self.stream.write('\n')
8bce3e51b101 run-tests: move newline out of colorized message
Martin von Zweigbergk <martinvonz@google.com>
parents: 34804
diff changeset
  1726
                        # Exclude the '\n' from highlighting to lex correctly
8bce3e51b101 run-tests: move newline out of colorized message
Martin von Zweigbergk <martinvonz@google.com>
parents: 34804
diff changeset
  1727
                        formatted = 'ERROR: %s output changed\n' % test
33930
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
  1728
                        self.stream.write(highlightmsg(formatted, self.color))
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
  1729
                    self.stream.write('!')
21754
7e14d026c4c4 run-tests: fixes the '--interactive' option error
anuraggoel <anurag.dsps@gmail.com>
parents: 21753
diff changeset
  1730
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1731
                self.stream.flush()
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1732
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1733
    def addSuccess(self, test):
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1734
        with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1735
            super(TestResult, self).addSuccess(test)
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1736
        self.successes.append(test)
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1737
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1738
    def addError(self, test, err):
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1739
        super(TestResult, self).addError(test, err)
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1740
        if self._options.first:
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1741
            self.stop()
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  1742
21430
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1743
    # Polyfill.
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1744
    def addSkip(self, test, reason):
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1745
        self.skipped.append((test, reason))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1746
        with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1747
            if self.showAll:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1748
                self.stream.writeln('skipped %s' % reason)
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1749
            else:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1750
                self.stream.write('s')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1751
                self.stream.flush()
21430
cf2992656bf8 run-tests: teach unittest about skipped tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21429
diff changeset
  1752
21431
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1753
    def addIgnore(self, test, reason):
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1754
        self.ignored.append((test, reason))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1755
        with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1756
            if self.showAll:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1757
                self.stream.writeln('ignored %s' % reason)
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21993
diff changeset
  1758
            else:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1759
                if reason not in ('not retesting', "doesn't match keyword"):
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1760
                    self.stream.write('i')
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1761
                else:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1762
                    self.testsRun += 1
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1763
                self.stream.flush()
21431
0f12bc8aed80 run-tests: teach unittest about ignored tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21430
diff changeset
  1764
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
  1765
    def addOutputMismatch(self, test, ret, got, expected):
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
  1766
        """Record a mismatch in test output for a particular test."""
35541
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
  1767
        if self.shouldStop or firsterror:
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
  1768
            # don't print, some other test case already failed and
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
  1769
            # printed, we're just stale and probably failed due to our
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
  1770
            # temp dir getting cleaned up.
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22486
diff changeset
  1771
            return
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
  1772
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
  1773
        accepted = False
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1774
        lines = []
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
  1775
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1776
        with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1777
            if self._options.nodiff:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1778
                pass
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1779
            elif self._options.view:
25056
e5f6c6ec21b8 run-tests: be more paranoid about os.system using bytes
Augie Fackler <augie@google.com>
parents: 25055
diff changeset
  1780
                v = self._options.view
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
  1781
                if PYTHON3:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  1782
                    v = _bytespath(v)
25056
e5f6c6ec21b8 run-tests: be more paranoid about os.system using bytes
Augie Fackler <augie@google.com>
parents: 25055
diff changeset
  1783
                os.system(b"%s %s %s" %
e5f6c6ec21b8 run-tests: be more paranoid about os.system using bytes
Augie Fackler <augie@google.com>
parents: 25055
diff changeset
  1784
                          (v, test.refpath, test.errpath))
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
  1785
            else:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1786
                servefail, lines = getdiff(expected, got,
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1787
                                           test.refpath, test.errpath)
36438
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1788
                self.stream.write('\n')
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1789
                for line in lines:
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1790
                    line = highlightdiff(line, self.color)
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1791
                    if PYTHON3:
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1792
                        self.stream.flush()
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1793
                        self.stream.buffer.write(line)
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1794
                        self.stream.buffer.flush()
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1795
                    else:
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1796
                        self.stream.write(line)
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36037
diff changeset
  1797
                        self.stream.flush()
21521
855f092c96e5 run-tests: move diff generation into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21520
diff changeset
  1798
36461
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
  1799
                if servefail:
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
  1800
                    raise test.failureException(
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
  1801
                        'server failed to start (HGPORT=%s)' % test._startport)
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
  1802
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1803
            # handle interactive prompt without releasing iolock
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1804
            if self._options.interactive:
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1805
                if test.readrefout() != expected:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1806
                    self.stream.write(
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1807
                        'Reference output has changed (run again to prompt '
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1808
                        'changes)')
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1809
                else:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1810
                    self.stream.write('Accept this change? [n] ')
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1811
                    answer = sys.stdin.readline().strip()
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1812
                    if answer.lower() in ('y', 'yes'):
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32981
diff changeset
  1813
                        if test.path.endswith(b'.t'):
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1814
                            rename(test.errpath, test.path)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1815
                        else:
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1816
                            rename(test.errpath, '%s.out' % test.path)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32943
diff changeset
  1817
                        accepted = True
28127
807bc140e915 run-tests: remove useless "failed" flag from addOutputMismatch()
Yuya Nishihara <yuya@tcha.org>
parents: 28126
diff changeset
  1818
            if not accepted:
25052
c4217a046b62 run-tests: record faildata using bytes instead of str
Augie Fackler <augie@google.com>
parents: 25051
diff changeset
  1819
                self.faildata[test.name] = b''.join(lines)
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
  1820
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21754
diff changeset
  1821
        return accepted
21523
9fb6f328576a run-tests: move interactive test acceptance into TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21522
diff changeset
  1822
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1823
    def startTest(self, test):
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1824
        super(TestResult, self).startTest(test)
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1825
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
  1826
        # os.times module computes the user time and system time spent by
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
  1827
        # child's processes along with real elapsed time taken by a process.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
  1828
        # This module has one limitation. It can only work for Linux user
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
  1829
        # and not for Windows.
24331
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
  1830
        test.started = os.times()
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
  1831
        if self._firststarttime is None: # thread racy but irrelevant
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
  1832
            self._firststarttime = test.started[4]
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1833
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1834
    def stopTest(self, test, interrupted=False):
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1835
        super(TestResult, self).stopTest(test)
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1836
24331
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
  1837
        test.stopped = os.times()
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
  1838
24331
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
  1839
        starttime = test.started
d3bdd8c7174f run-tests: stop storing start/stop times in a dict by test name
Augie Fackler <augie@google.com>
parents: 24330
diff changeset
  1840
        endtime = test.stopped
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
  1841
        origin = self._firststarttime
24984
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
  1842
        self.times.append((test.name,
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
  1843
                           endtime[2] - starttime[2], # user space CPU time
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
  1844
                           endtime[3] - starttime[3], # sys  space CPU time
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
  1845
                           endtime[4] - starttime[4], # real time
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
  1846
                           starttime[4] - origin, # start date in run context
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25068
diff changeset
  1847
                           endtime[4] - origin, # end date in run context
24984
5195322b9f80 run-tests: spread and document the content of time tuple
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24983
diff changeset
  1848
                           ))
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
  1849
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1850
        if interrupted:
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1851
            with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1852
                self.stream.writeln('INTERRUPTED: %s (after %d seconds)' % (
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  1853
                    test.name, self.times[-1][3]))
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  1854
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  1855
class TestSuite(unittest.TestSuite):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
  1856
    """Custom unittest TestSuite that knows how to execute Mercurial tests."""
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
  1857
21533
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
  1858
    def __init__(self, testdir, jobs=1, whitelist=None, blacklist=None,
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
  1859
                 retest=False, keywords=None, loop=False, runs_per_test=1,
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1860
                 loadtest=None, showchannels=False,
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1861
                 *args, **kwargs):
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
  1862
        """Create a new instance that can run tests with a configuration.
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  1863
21533
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
  1864
        testdir specifies the directory where tests are executed from. This
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
  1865
        is typically the ``tests`` directory from Mercurial's source
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
  1866
        repository.
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
  1867
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
  1868
        jobs specifies the number of jobs to run concurrently. Each test
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
  1869
        executes on its own thread. Tests actually spawn new processes, so
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
  1870
        state mutation should not be an issue.
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1871
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  1872
        If there is only one job, it will use the main thread.
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  1873
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1874
        whitelist and blacklist denote tests that have been whitelisted and
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1875
        blacklisted, respectively. These arguments don't belong in TestSuite.
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1876
        Instead, whitelist and blacklist should be handled by the thing that
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1877
        populates the TestSuite with tests. They are present to preserve
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1878
        backwards compatible behavior which reports skipped tests as part
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1879
        of the results.
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
  1880
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
  1881
        retest denotes whether to retest failed tests. This arguably belongs
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
  1882
        outside of TestSuite.
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
  1883
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
  1884
        keywords denotes key words that will be used to filter which tests
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
  1885
        to execute. This arguably belongs outside of TestSuite.
21532
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
  1886
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
  1887
        loop denotes whether to loop over tests forever.
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
  1888
        """
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  1889
        super(TestSuite, self).__init__(*args, **kwargs)
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  1890
21528
32b9bbca2052 run-tests: pass jobs into TestSuite constructor
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21523
diff changeset
  1891
        self._jobs = jobs
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1892
        self._whitelist = whitelist
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  1893
        self._blacklist = blacklist
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
  1894
        self._retest = retest
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
  1895
        self._keywords = keywords
21532
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
  1896
        self._loop = loop
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
  1897
        self._runs_per_test = runs_per_test
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  1898
        self._loadtest = loadtest
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1899
        self._showchannels = showchannels
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  1900
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  1901
    def run(self, result):
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1902
        # We have a number of filters that need to be applied. We do this
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1903
        # here instead of inside Test because it makes the running logic for
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1904
        # Test simpler.
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1905
        tests = []
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  1906
        num_tests = [0]
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1907
        for test in self._tests:
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  1908
            def get():
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  1909
                num_tests[0] += 1
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  1910
                if getattr(test, 'should_reload', False):
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  1911
                    return self._loadtest(test, num_tests[0])
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  1912
                return test
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1913
            if not os.path.exists(test.path):
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1914
                result.addSkip(test, "Doesn't exist")
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1915
                continue
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1916
34266
cd2ee4db8f95 tests: fix test-is-whitelisted check in run-tests
Augie Fackler <augie@google.com>
parents: 34265
diff changeset
  1917
            if not (self._whitelist and test.bname in self._whitelist):
25055
d79258e30499 run-tests: blacklist entries are bytes, use bname to check blacklisting
Augie Fackler <augie@google.com>
parents: 25053
diff changeset
  1918
                if self._blacklist and test.bname in self._blacklist:
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1919
                    result.addSkip(test, 'blacklisted')
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1920
                    continue
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1921
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
  1922
                if self._retest and not os.path.exists(test.errpath):
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1923
                    result.addIgnore(test, 'not retesting')
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1924
                    continue
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1925
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
  1926
                if self._keywords:
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1927
                    with open(test.path, 'rb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  1928
                        t = f.read().lower() + test.bname.lower()
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1929
                    ignored = False
21531
7fcda22acc43 run-tests: make keywords a named argument to TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21530
diff changeset
  1930
                    for k in self._keywords.lower().split():
21507
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1931
                        if k not in t:
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1932
                            result.addIgnore(test, "doesn't match keyword")
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1933
                            ignored = True
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1934
                            break
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1935
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1936
                    if ignored:
d839e4820da7 run-tests: move test filtering into TestSuite.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21506
diff changeset
  1937
                        continue
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
  1938
            for _ in xrange(self._runs_per_test):
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  1939
                tests.append(get())
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1940
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  1941
        runtests = list(tests)
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1942
        done = queue.Queue()
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1943
        running = 0
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1944
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1945
        channels = [""] * self._jobs
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1946
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1947
        def job(test, result):
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1948
            for n, v in enumerate(channels):
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1949
                if not v:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1950
                    channel = n
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1951
                    break
32621
c59451e11cbf tests: make run-tests fail early if no channel is found
Augie Fackler <augie@google.com>
parents: 32414
diff changeset
  1952
            else:
c59451e11cbf tests: make run-tests fail early if no channel is found
Augie Fackler <augie@google.com>
parents: 32414
diff changeset
  1953
                raise ValueError('Could not find output channel')
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1954
            channels[channel] = "=" + test.name[5:].split(".")[0]
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1955
            try:
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1956
                test(result)
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1957
                done.put(None)
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1958
            except KeyboardInterrupt:
27933
a6833e464b07 run-tests: "fix" race condition in race condition fix
Bryan O'Sullivan <bryano@fb.com>
parents: 27927
diff changeset
  1959
                pass
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1960
            except: # re-raises
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1961
                done.put(('!', test, 'run-test raised an error, see traceback'))
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1962
                raise
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
  1963
            finally:
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
  1964
                try:
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
  1965
                    channels[channel] = ''
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
  1966
                except IndexError:
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32621
diff changeset
  1967
                    pass
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1968
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1969
        def stat():
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1970
            count = 0
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1971
            while channels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1972
                d = '\n%03s  ' % count
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1973
                for n, v in enumerate(channels):
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1974
                    if v:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1975
                        d += v[0]
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1976
                        channels[n] = v[1:] or '.'
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1977
                    else:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1978
                        d += ' '
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1979
                    d += ' '
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1980
                with iolock:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1981
                    sys.stdout.write(d + '  ')
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1982
                    sys.stdout.flush()
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1983
                for x in xrange(10):
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1984
                    if channels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1985
                        time.sleep(.1)
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1986
                count += 1
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1987
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24506
diff changeset
  1988
        stoppedearly = False
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24506
diff changeset
  1989
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1990
        if self._showchannels:
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1991
            statthread = threading.Thread(target=stat, name="stat")
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1992
            statthread.start()
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  1993
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  1994
        try:
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  1995
            while tests or running:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  1996
                if not done.empty() or running == self._jobs or not tests:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  1997
                    try:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  1998
                        done.get(True, 1)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  1999
                        running -= 1
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2000
                        if result and result.shouldStop:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2001
                            stoppedearly = True
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2002
                            break
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2003
                    except queue.Empty:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2004
                        continue
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2005
                if tests and not running == self._jobs:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2006
                    test = tests.pop(0)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2007
                    if self._loop:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2008
                        if getattr(test, 'should_reload', False):
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2009
                            num_tests[0] += 1
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2010
                            tests.append(
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2011
                                self._loadtest(test, num_tests[0]))
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2012
                        else:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2013
                            tests.append(test)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2014
                    if self._jobs == 1:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2015
                        job(test, result)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2016
                    else:
27689
50e621fe0362 run-tests: skip threading for a single test
timeless <timeless@mozdev.org>
parents: 27686
diff changeset
  2017
                        t = threading.Thread(target=job, name=test.name,
50e621fe0362 run-tests: skip threading for a single test
timeless <timeless@mozdev.org>
parents: 27686
diff changeset
  2018
                                             args=(test, result))
50e621fe0362 run-tests: skip threading for a single test
timeless <timeless@mozdev.org>
parents: 27686
diff changeset
  2019
                        t.start()
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2020
                    running += 1
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24506
diff changeset
  2021
27880
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2022
            # If we stop early we still need to wait on started tests to
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2023
            # finish. Otherwise, there is a race between the test completing
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2024
            # and the test's cleanup code running. This could result in the
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2025
            # test reporting incorrect.
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2026
            if stoppedearly:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2027
                while running:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2028
                    try:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2029
                        done.get(True, 1)
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2030
                        running -= 1
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2031
                    except queue.Empty:
b04df9ce1fb0 run-tests: skip threading for a single test (issue5040)
timeless <timeless@mozdev.org>
parents: 27777
diff changeset
  2032
                        continue
21496
f145914e698d run-tests: move _executetests into TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21495
diff changeset
  2033
        except KeyboardInterrupt:
21520
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  2034
            for test in runtests:
fa6ba4372678 run-tests: remove global abort flag
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21518
diff changeset
  2035
                test.abort()
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  2036
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  2037
        channels = []
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  2038
21439
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  2039
        return result
2e22954b97e3 run-tests: define a custom TestSuite that uses _executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21438
diff changeset
  2040
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2041
# Save the most recent 5 wall-clock runtimes of each test to a
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2042
# human-readable text file named .testtimes. Tests are sorted
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2043
# alphabetically, while times for each test are listed from oldest to
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2044
# newest.
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2045
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
  2046
def loadtimes(outputdir):
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2047
    times = []
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2048
    try:
35855
69d7fcd91696 testrunner: fix updating of .testtimes file
Martin von Zweigbergk <martinvonz@google.com>
parents: 35854
diff changeset
  2049
        with open(os.path.join(outputdir, b'.testtimes')) as fp:
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2050
            for line in fp:
35854
568917059243 testrunner: make reading of test times work with #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 35848
diff changeset
  2051
                m = re.match('(.*?) ([0-9. ]+)', line)
568917059243 testrunner: make reading of test times work with #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 35848
diff changeset
  2052
                times.append((m.group(1),
568917059243 testrunner: make reading of test times work with #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 35848
diff changeset
  2053
                              [float(t) for t in m.group(2).split()]))
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2054
    except IOError as err:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2055
        if err.errno != errno.ENOENT:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2056
            raise
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2057
    return times
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2058
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
  2059
def savetimes(outputdir, result):
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
  2060
    saved = dict(loadtimes(outputdir))
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2061
    maxruns = 5
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2062
    skipped = set([str(t[0]) for t in result.skipped])
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2063
    for tdata in result.times:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2064
        test, real = tdata[0], tdata[3]
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2065
        if test not in skipped:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2066
            ts = saved.setdefault(test, [])
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2067
            ts.append(real)
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2068
            ts[:] = ts[-maxruns:]
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2069
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
  2070
    fd, tmpname = tempfile.mkstemp(prefix=b'.testtimes',
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
  2071
                                   dir=outputdir, text=True)
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2072
    with os.fdopen(fd, 'w') as fp:
28284
0fe00bdb2f4f run-tests: fix Python 3 incompatibilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28180
diff changeset
  2073
        for name, ts in sorted(saved.items()):
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2074
            fp.write('%s %s\n' % (name, ' '.join(['%.3f' % (t,) for t in ts])))
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
  2075
    timepath = os.path.join(outputdir, b'.testtimes')
27634
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2076
    try:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2077
        os.unlink(timepath)
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2078
    except OSError:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2079
        pass
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2080
    try:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2081
        os.rename(tmpname, timepath)
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2082
    except OSError:
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2083
        pass
a1eff44c432b tests: write recent run times to a file named tests/.testtimes
Bryan O'Sullivan <bos@serpentine.com>
parents: 27602
diff changeset
  2084
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  2085
class TextTestRunner(unittest.TextTestRunner):
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  2086
    """Custom unittest test runner that uses appropriate settings."""
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  2087
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2088
    def __init__(self, runner, *args, **kwargs):
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2089
        super(TextTestRunner, self).__init__(*args, **kwargs)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2090
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2091
        self._runner = runner
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2092
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2093
    def listtests(self, test):
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2094
        result = TestResult(self._runner.options, self.stream,
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2095
                            self.descriptions, 0)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2096
        test = sorted(test, key=lambda t: t.name)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2097
        for t in test:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2098
            print(t.name)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2099
            result.addSuccess(t)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2100
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2101
        if self._runner.options.xunit:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2102
            with open(self._runner.options.xunit, "wb") as xuf:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2103
                self._writexunit(result, xuf)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2104
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2105
        if self._runner.options.json:
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  2106
            jsonpath = os.path.join(self._runner._outputdir, b'report.json')
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2107
            with open(jsonpath, 'w') as fp:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2108
                self._writejson(result, fp)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2109
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2110
        return result
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2111
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2112
    def run(self, test):
21460
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  2113
        result = TestResult(self._runner.options, self.stream,
df580990507e run-tests: abort tests after first failure in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21459
diff changeset
  2114
                            self.descriptions, self.verbosity)
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2115
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2116
        test(result)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2117
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2118
        failed = len(result.failures)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2119
        skipped = len(result.skipped)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2120
        ignored = len(result.ignored)
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2121
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2122
        with iolock:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2123
            self.stream.writeln('')
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2124
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2125
            if not self._runner.options.noskips:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2126
                for test, msg in result.skipped:
33929
5d2ce90c71f1 run-tests: include "\n" in formatted message instead of calling writeln()
Yuya Nishihara <yuya@tcha.org>
parents: 33871
diff changeset
  2127
                    formatted = 'Skipped %s: %s\n' % (test.name, msg)
33930
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
  2128
                    self.stream.write(highlightmsg(formatted, result.color))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2129
            for test, msg in result.failures:
33929
5d2ce90c71f1 run-tests: include "\n" in formatted message instead of calling writeln()
Yuya Nishihara <yuya@tcha.org>
parents: 33871
diff changeset
  2130
                formatted = 'Failed %s: %s\n' % (test.name, msg)
33930
f5d4bb8e944d run-tests: factor out highlight functions
Yuya Nishihara <yuya@tcha.org>
parents: 33929
diff changeset
  2131
                self.stream.write(highlightmsg(formatted, result.color))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2132
            for test, msg in result.errors:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2133
                self.stream.writeln('Errored %s: %s' % (test.name, msg))
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2134
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2135
            if self._runner.options.xunit:
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2136
                with open(self._runner.options.xunit, "wb") as xuf:
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2137
                    self._writexunit(result, xuf)
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  2138
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2139
            if self._runner.options.json:
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  2140
                jsonpath = os.path.join(self._runner._outputdir, b'report.json')
27773
bf45edfa9d90 run-tests: use a context manager for file I/O
Bryan O'Sullivan <bryano@fb.com>
parents: 27689
diff changeset
  2141
                with open(jsonpath, 'w') as fp:
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2142
                    self._writejson(result, fp)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  2143
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2144
            self._runner._checkhglib('Tested')
21459
d5945324b130 run-tests: print compatible output from TextTestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21458
diff changeset
  2145
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
  2146
            savetimes(self._runner._outputdir, result)
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
  2147
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28582
diff changeset
  2148
            if failed and self._runner.options.known_good_rev:
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2149
                self._bisecttests(t for t, m in result.failures)
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2150
            self.stream.writeln(
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32934
diff changeset
  2151
                '# Ran %d tests, %d skipped, %d failed.'
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32934
diff changeset
  2152
                % (result.testsRun, skipped + ignored, failed))
25046
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2153
            if failed:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2154
                self.stream.writeln('python hash seed: %s' %
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2155
                    os.environ['PYTHONHASHSEED'])
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2156
            if self._runner.options.time:
40b4308d5653 run-tests: switch all uses of iolock.acquire() to a context manager
Augie Fackler <augie@google.com>
parents: 25045
diff changeset
  2157
                self.printtimes(result.times)
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2158
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2159
            if self._runner.options.exceptions:
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2160
                exceptions = aggregateexceptions(
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2161
                    os.path.join(self._runner._outputdir, b'exceptions'))
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2162
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2163
                self.stream.writeln('Exceptions Report:')
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2164
                self.stream.writeln('%d total from %d frames' %
36037
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2165
                                    (exceptions['total'],
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2166
                                     len(exceptions['exceptioncounts'])))
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2167
                combined = exceptions['combined']
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2168
                for key in sorted(combined, key=combined.get, reverse=True):
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2169
                    frame, line, exc = key
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2170
                    totalcount, testcount, leastcount, leasttest = combined[key]
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2171
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2172
                    self.stream.writeln('%d (%d tests)\t%s: %s (%s - %d total)'
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2173
                                        % (totalcount,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2174
                                           testcount,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2175
                                           frame, exc,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  2176
                                           leasttest, leastcount))
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2177
32907
bd77ac2bd23a run-tests: explicitly flush test runner output for Windows stability
Matt Harbison <matt_harbison@yahoo.com>
parents: 32853
diff changeset
  2178
            self.stream.flush()
22104
70bdf59d27b6 run-tests: attempt to fix iolock handling
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
  2179
21613
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
  2180
        return result
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
  2181
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2182
    def _bisecttests(self, tests):
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2183
        bisectcmd = ['hg', 'bisect']
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2184
        bisectrepo = self._runner.options.bisect_repo
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2185
        if bisectrepo:
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2186
            bisectcmd.extend(['-R', os.path.abspath(bisectrepo)])
34803
d817bf1fc675 run-tests: extract Popen logic to a single method
Jun Wu <quark@fb.com>
parents: 34802
diff changeset
  2187
        def pread(args):
34804
149109c96904 run-tests: set HGPLAIN=1 when bisecting
Jun Wu <quark@fb.com>
parents: 34803
diff changeset
  2188
            env = os.environ.copy()
149109c96904 run-tests: set HGPLAIN=1 when bisecting
Jun Wu <quark@fb.com>
parents: 34803
diff changeset
  2189
            env['HGPLAIN'] = '1'
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2190
            p = subprocess.Popen(args, stderr=subprocess.STDOUT,
34804
149109c96904 run-tests: set HGPLAIN=1 when bisecting
Jun Wu <quark@fb.com>
parents: 34803
diff changeset
  2191
                                 stdout=subprocess.PIPE, env=env)
34803
d817bf1fc675 run-tests: extract Popen logic to a single method
Jun Wu <quark@fb.com>
parents: 34802
diff changeset
  2192
            data = p.stdout.read()
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2193
            p.wait()
34803
d817bf1fc675 run-tests: extract Popen logic to a single method
Jun Wu <quark@fb.com>
parents: 34802
diff changeset
  2194
            return data
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2195
        for test in tests:
34803
d817bf1fc675 run-tests: extract Popen logic to a single method
Jun Wu <quark@fb.com>
parents: 34802
diff changeset
  2196
            pread(bisectcmd + ['--reset']),
d817bf1fc675 run-tests: extract Popen logic to a single method
Jun Wu <quark@fb.com>
parents: 34802
diff changeset
  2197
            pread(bisectcmd + ['--bad', '.'])
d817bf1fc675 run-tests: extract Popen logic to a single method
Jun Wu <quark@fb.com>
parents: 34802
diff changeset
  2198
            pread(bisectcmd + ['--good', self._runner.options.known_good_rev])
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2199
            # TODO: we probably need to forward more options
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2200
            # that alter hg's behavior inside the tests.
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2201
            opts = ''
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2202
            withhg = self._runner.options.with_hg
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2203
            if withhg:
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2204
                opts += ' --with-hg=%s ' % shellquote(_strpath(withhg))
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2205
            rtc = '%s %s %s %s' % (sys.executable, sys.argv[0], opts,
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2206
                                   test)
34803
d817bf1fc675 run-tests: extract Popen logic to a single method
Jun Wu <quark@fb.com>
parents: 34802
diff changeset
  2207
            data = pread(bisectcmd + ['--command', rtc])
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2208
            m = re.search(
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2209
                (br'\nThe first (?P<goodbad>bad|good) revision '
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2210
                 br'is:\nchangeset: +\d+:(?P<node>[a-f0-9]+)\n.*\n'
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2211
                 br'summary: +(?P<summary>[^\n]+)\n'),
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2212
                data, (re.MULTILINE | re.DOTALL))
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2213
            if m is None:
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2214
                self.stream.writeln(
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2215
                    'Failed to identify failure point for %s' % test)
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2216
                continue
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2217
            dat = m.groupdict()
37741
700aaa19de63 tests: fix up a couple of minor bytes inconsistencies in run-tests.py
Augie Fackler <augie@google.com>
parents: 37342
diff changeset
  2218
            verb = 'broken' if dat['goodbad'] == b'bad' else 'fixed'
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2219
            self.stream.writeln(
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2220
                '%s %s by %s (%s)' % (
37741
700aaa19de63 tests: fix up a couple of minor bytes inconsistencies in run-tests.py
Augie Fackler <augie@google.com>
parents: 37342
diff changeset
  2221
                    test, verb, dat['node'].decode('ascii'),
700aaa19de63 tests: fix up a couple of minor bytes inconsistencies in run-tests.py
Augie Fackler <augie@google.com>
parents: 37342
diff changeset
  2222
                    dat['summary'].decode('utf8', 'ignore')))
34802
9c7548eb7d1c run-tests: move bisect logic to a separate method
Jun Wu <quark@fb.com>
parents: 34444
diff changeset
  2223
21495
b162bdc78bef run-tests: capture execution times in TestResult
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21494
diff changeset
  2224
    def printtimes(self, times):
22104
70bdf59d27b6 run-tests: attempt to fix iolock handling
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
  2225
        # iolock held by run
21494
dcefc4091c86 run-tests: move outputtimes() into unittest runner class
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21493
diff changeset
  2226
        self.stream.writeln('# Producing time report')
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21919
diff changeset
  2227
        times.sort(key=lambda t: (t[3]))
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
  2228
        cols = '%7.3f %7.3f %7.3f %7.3f %7.3f   %s'
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
  2229
        self.stream.writeln('%-7s %-7s %-7s %-7s %-7s   %s' %
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
  2230
                            ('start', 'end', 'cuser', 'csys', 'real', 'Test'))
24982
5c15f7e0f52b run-tests: stop explicit expansion of time data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24981
diff changeset
  2231
        for tdata in times:
5c15f7e0f52b run-tests: stop explicit expansion of time data
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24981
diff changeset
  2232
            test = tdata[0]
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
  2233
            cuser, csys, real, start, end = tdata[1:6]
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
  2234
            self.stream.writeln(cols % (start, end, cuser, csys, real, test))
21429
203ed3cf6c81 run-tests: define custom result and runner classes for unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21428
diff changeset
  2235
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2236
    @staticmethod
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2237
    def _writexunit(result, outf):
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2238
        # See http://llg.cubic.org/docs/junit/ for a reference.
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2239
        timesd = dict((t[0], t[3]) for t in result.times)
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2240
        doc = minidom.Document()
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2241
        s = doc.createElement('testsuite')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2242
        s.setAttribute('name', 'run-tests')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2243
        s.setAttribute('tests', str(result.testsRun))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2244
        s.setAttribute('errors', "0") # TODO
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2245
        s.setAttribute('failures', str(len(result.failures)))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2246
        s.setAttribute('skipped', str(len(result.skipped) +
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2247
                                      len(result.ignored)))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2248
        doc.appendChild(s)
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2249
        for tc in result.successes:
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2250
            t = doc.createElement('testcase')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2251
            t.setAttribute('name', tc.name)
32702
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
  2252
            tctime = timesd.get(tc.name)
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
  2253
            if tctime is not None:
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
  2254
                t.setAttribute('time', '%.3f' % tctime)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2255
            s.appendChild(t)
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2256
        for tc, err in sorted(result.faildata.items()):
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2257
            t = doc.createElement('testcase')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2258
            t.setAttribute('name', tc)
32702
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
  2259
            tctime = timesd.get(tc)
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
  2260
            if tctime is not None:
d0b9c36851f5 run-tests: make time field optional for xunit report
Siddharth Agarwal <sid0@fb.com>
parents: 32701
diff changeset
  2261
                t.setAttribute('time', '%.3f' % tctime)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2262
            # createCDATASection expects a unicode or it will
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2263
            # convert using default conversion rules, which will
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2264
            # fail if string isn't ASCII.
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2265
            err = cdatasafe(err).decode('utf-8', 'replace')
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2266
            cd = doc.createCDATASection(err)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2267
            # Use 'failure' here instead of 'error' to match errors = 0,
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2268
            # failures = len(result.failures) in the testsuite element.
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2269
            failelem = doc.createElement('failure')
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2270
            failelem.setAttribute('message', 'output changed')
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2271
            failelem.setAttribute('type', 'output-mismatch')
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2272
            failelem.appendChild(cd)
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32711
diff changeset
  2273
            t.appendChild(failelem)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2274
            s.appendChild(t)
32715
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2275
        for tc, message in result.skipped:
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2276
            # According to the schema, 'skipped' has no attributes. So store
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2277
            # the skip message as a text node instead.
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2278
            t = doc.createElement('testcase')
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2279
            t.setAttribute('name', tc.name)
34269
20f547806a4d tests: fix run-tests XML reporting on Python 3
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
  2280
            binmessage = message.encode('utf-8')
20f547806a4d tests: fix run-tests XML reporting on Python 3
Augie Fackler <augie@google.com>
parents: 34268
diff changeset
  2281
            message = cdatasafe(binmessage).decode('utf-8', 'replace')
32715
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2282
            cd = doc.createCDATASection(message)
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2283
            skipelem = doc.createElement('skipped')
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2284
            skipelem.appendChild(cd)
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2285
            t.appendChild(skipelem)
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  2286
            s.appendChild(t)
32700
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2287
        outf.write(doc.toprettyxml(indent='  ', encoding='utf-8'))
3afe258fb0fe run-tests: factor out xunit write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
  2288
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2289
    @staticmethod
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2290
    def _writejson(result, outf):
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2291
        timesd = {}
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2292
        for tdata in result.times:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2293
            test = tdata[0]
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2294
            timesd[test] = tdata[1:]
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2295
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2296
        outcome = {}
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2297
        groups = [('success', ((tc, None)
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2298
                   for tc in result.successes)),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2299
                  ('failure', result.failures),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2300
                  ('skip', result.skipped)]
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2301
        for res, testcases in groups:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2302
            for tc, __ in testcases:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2303
                if tc.name in timesd:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2304
                    diff = result.faildata.get(tc.name, b'')
32853
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  2305
                    try:
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  2306
                        diff = diff.decode('unicode_escape')
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  2307
                    except UnicodeDecodeError as e:
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  2308
                        diff = '%r decoding diff, sorry' % e
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2309
                    tres = {'result': res,
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2310
                            'time': ('%0.3f' % timesd[tc.name][2]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2311
                            'cuser': ('%0.3f' % timesd[tc.name][0]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2312
                            'csys': ('%0.3f' % timesd[tc.name][1]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2313
                            'start': ('%0.3f' % timesd[tc.name][3]),
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2314
                            'end': ('%0.3f' % timesd[tc.name][4]),
32853
3139a7a1e7d4 tests: try and fail more gracefully with broken unicode escapes
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  2315
                            'diff': diff,
32701
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2316
                            }
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2317
                else:
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2318
                    # blacklisted test
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2319
                    tres = {'result': res}
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2320
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2321
                outcome[tc.name] = tres
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2322
        jsonout = json.dumps(outcome, sort_keys=True, indent=4,
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2323
                             separators=(',', ': '))
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2324
        outf.writelines(("testreport =", jsonout))
60c921ff4104 run-tests: factor out json write code into another method
Siddharth Agarwal <sid0@fb.com>
parents: 32700
diff changeset
  2325
36665
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2326
def sorttests(testdescs, previoustimes, shuffle=False):
35489
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2327
    """Do an in-place sort of tests."""
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2328
    if shuffle:
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2329
        random.shuffle(testdescs)
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2330
        return
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2331
36665
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2332
    if previoustimes:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2333
        def sortkey(f):
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2334
            f = f['path']
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2335
            if f in previoustimes:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2336
                # Use most recent time as estimate
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2337
                return -previoustimes[f][-1]
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2338
            else:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2339
                # Default to a rather arbitrary value of 1 second for new tests
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2340
                return -1.0
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2341
    else:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2342
        # keywords for slow tests
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2343
        slow = {b'svn': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2344
                b'cvs': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2345
                b'hghave': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2346
                b'largefiles-update': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2347
                b'run-tests': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2348
                b'corruption': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2349
                b'race': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2350
                b'i18n': 10,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2351
                b'check': 100,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2352
                b'gendoc': 100,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2353
                b'contrib-perf': 200,
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2354
                }
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2355
        perf = {}
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2356
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2357
        def sortkey(f):
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2358
            # run largest tests first, as they tend to take the longest
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2359
            f = f['path']
35489
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2360
            try:
36665
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2361
                return perf[f]
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2362
            except KeyError:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2363
                try:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2364
                    val = -os.stat(f).st_size
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2365
                except OSError as e:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2366
                    if e.errno != errno.ENOENT:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2367
                        raise
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2368
                    perf[f] = -1e9  # file does not exist, tell early
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2369
                    return -1e9
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2370
                for kw, mul in slow.items():
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2371
                    if kw in f:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2372
                        val *= mul
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2373
                if f.endswith(b'.py'):
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2374
                    val /= 10.0
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2375
                perf[f] = val / 1000.0
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2376
                return perf[f]
35489
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2377
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2378
    testdescs.sort(key=sortkey)
212a6e9aecb0 run-tests: extract sorting of tests to own function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35488
diff changeset
  2379
21340
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
  2380
class TestRunner(object):
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
  2381
    """Holds context for executing tests.
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
  2382
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
  2383
    Tests rely on a lot of state. This object holds it for them.
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
  2384
    """
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2385
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2386
    # Programs required to run tests.
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  2387
    REQUIREDTOOLS = [
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2388
        b'diff',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2389
        b'grep',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2390
        b'unzip',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2391
        b'gunzip',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2392
        b'bunzip2',
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2393
        b'sed',
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  2394
    ]
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  2395
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2396
    # Maps file extensions to test class.
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2397
    TESTTYPES = [
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2398
        (b'.py', PythonTest),
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2399
        (b'.t', TTest),
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2400
    ]
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2401
21341
cb88d4a04f58 run-tests: move TESTDIR out of a global
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21340
diff changeset
  2402
    def __init__(self):
21348
b3399154505f run-tests: add options to runner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21347
diff changeset
  2403
        self.options = None
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
  2404
        self._hgroot = None
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2405
        self._testdir = None
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
  2406
        self._outputdir = None
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2407
        self._hgtmp = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2408
        self._installdir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2409
        self._bindir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2410
        self._tmpbinddir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2411
        self._pythondir = None
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2412
        self._coveragefile = None
21352
39fd89fbbc3c run-tests: move createdfiles out of a global and into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21351
diff changeset
  2413
        self._createdfiles = []
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
  2414
        self._hgcommand = None
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2415
        self._hgpath = None
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2416
        self._portoffset = 0
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2417
        self._ports = {}
21340
fda36de1cb0e run-tests: establish a class to hold testing state
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21339
diff changeset
  2418
21376
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
  2419
    def run(self, args, parser=None):
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2420
        """Run the test suite."""
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2421
        oldmask = os.umask(0o22)
21375
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
  2422
        try:
21376
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
  2423
            parser = parser or getparser()
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
  2424
            options = parseargs(args, parser)
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
  2425
            tests = [_bytespath(a) for a in options.tests]
34263
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  2426
            if options.test_list is not None:
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  2427
                for listfile in options.test_list:
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  2428
                    with open(listfile, 'rb') as f:
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
  2429
                        tests.extend(t for t in f.read().splitlines() if t)
21376
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
  2430
            self.options = options
e4366bc08879 run-tests: move option parser logic to TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21375
diff changeset
  2431
21375
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
  2432
            self._checktools()
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
  2433
            testdescs = self.findtests(tests)
25107
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2434
            if options.profile_runner:
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2435
                import statprof
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2436
                statprof.start()
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2437
            result = self._run(testdescs)
25107
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2438
            if options.profile_runner:
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2439
                statprof.stop()
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2440
                statprof.display()
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2441
            return result
149cc7663ac8 run-tests: add a --profile-runner option
Augie Fackler <augie@google.com>
parents: 25098
diff changeset
  2442
21375
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
  2443
        finally:
bd70dcb91af6 run-tests: move umask into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21374
diff changeset
  2444
            os.umask(oldmask)
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2445
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2446
    def _run(self, testdescs):
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2447
        self._testdir = osenvironb[b'TESTDIR'] = getattr(
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2448
            os, 'getcwdb', os.getcwd)()
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34842
diff changeset
  2449
        # assume all tests in same folder for now
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34842
diff changeset
  2450
        if testdescs:
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34842
diff changeset
  2451
            pathname = os.path.dirname(testdescs[0]['path'])
35066
57d56f603f70 run-tests: fix TESTDIR if testdescs are absolute paths
Kyle Lippincott <spectral@google.com>
parents: 34969
diff changeset
  2452
            if pathname:
57d56f603f70 run-tests: fix TESTDIR if testdescs are absolute paths
Kyle Lippincott <spectral@google.com>
parents: 34969
diff changeset
  2453
                osenvironb[b'TESTDIR'] = os.path.join(osenvironb[b'TESTDIR'],
57d56f603f70 run-tests: fix TESTDIR if testdescs are absolute paths
Kyle Lippincott <spectral@google.com>
parents: 34969
diff changeset
  2454
                                                      pathname)
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
  2455
        if self.options.outputdir:
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
  2456
            self._outputdir = canonpath(_bytespath(self.options.outputdir))
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
  2457
        else:
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
  2458
            self._outputdir = self._testdir
35097
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35092
diff changeset
  2459
            if testdescs and pathname:
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35092
diff changeset
  2460
                self._outputdir = os.path.join(self._outputdir, pathname)
36665
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2461
        previoustimes = {}
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2462
        if self.options.order_by_runtime:
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2463
            previoustimes = dict(loadtimes(self._outputdir))
6276cbc704a6 testrunner: add option to sort tests by previous run time
Martin von Zweigbergk <martinvonz@google.com>
parents: 36522
diff changeset
  2464
        sorttests(testdescs, previoustimes, shuffle=self.options.random)
21371
a10ba7870c2d run-tests: assign testdir in TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21370
diff changeset
  2465
21370
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
  2466
        if 'PYTHONHASHSEED' not in os.environ:
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
  2467
            # use a random python hash seed all the time
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
  2468
            # we do the randomness ourself to know what seed is used
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
  2469
            os.environ['PYTHONHASHSEED'] = str(random.getrandbits(32))
97475f27bebe run-tests: move hash seed logic to TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21369
diff changeset
  2470
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2471
        if self.options.tmpdir:
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2472
            self.options.keep_tmpdir = True
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  2473
            tmpdir = _bytespath(self.options.tmpdir)
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2474
            if os.path.exists(tmpdir):
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2475
                # Meaning of tmpdir has changed since 1.3: we used to create
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2476
                # HGTMP inside tmpdir; now HGTMP is tmpdir.  So fail if
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2477
                # tmpdir already exists.
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2478
                print("error: temp dir %r already exists" % tmpdir)
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2479
                return 1
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2480
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2481
            os.makedirs(tmpdir)
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2482
        else:
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2483
            d = None
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2484
            if os.name == 'nt':
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2485
                # without this, we get the default temp dir location, but
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2486
                # in all lowercase, which causes troubles with paths (issue3490)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2487
                d = osenvironb.get(b'TMP', None)
25262
5a809deb88e6 run-tests: python3.5 now supports mkdtemp using bytes for paths
Augie Fackler <raf@durin42.com>
parents: 25261
diff changeset
  2488
            tmpdir = tempfile.mkdtemp(b'', b'hgtests.', d)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2489
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2490
        self._hgtmp = osenvironb[b'HGTMP'] = (
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2491
            os.path.realpath(tmpdir))
21369
1d0aa8bccc87 run-tests: move tmpdir calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21368
diff changeset
  2492
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
  2493
        if self.options.with_hg:
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2494
            self._installdir = None
25042
201823c50610 run-tests: work around with_hg being bytes or string depending on provenance
Augie Fackler <augie@google.com>
parents: 25041
diff changeset
  2495
            whg = self.options.with_hg
201823c50610 run-tests: work around with_hg being bytes or string depending on provenance
Augie Fackler <augie@google.com>
parents: 25041
diff changeset
  2496
            self._bindir = os.path.dirname(os.path.realpath(whg))
201823c50610 run-tests: work around with_hg being bytes or string depending on provenance
Augie Fackler <augie@google.com>
parents: 25041
diff changeset
  2497
            assert isinstance(self._bindir, bytes)
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
  2498
            self._hgcommand = os.path.basename(whg)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2499
            self._tmpbindir = os.path.join(self._hgtmp, b'install', b'bin')
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2500
            os.makedirs(self._tmpbindir)
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
  2501
35569
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2502
            normbin = os.path.normpath(os.path.abspath(whg))
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2503
            normbin = normbin.replace(os.sep.encode('ascii'), b'/')
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2504
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2505
            # Other Python scripts in the test harness need to
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2506
            # `import mercurial`. If `hg` is a Python script, we assume
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2507
            # the Mercurial modules are relative to its path and tell the tests
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2508
            # to load Python modules from its directory.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2509
            with open(whg, 'rb') as fh:
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2510
                initial = fh.read(1024)
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2511
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2512
            if re.match(b'#!.*python', initial):
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2513
                self._pythondir = self._bindir
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2514
            # If it looks like our in-repo Rust binary, use the source root.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2515
            # This is a bit hacky. But rhg is still not supported outside the
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2516
            # source directory. So until it is, do the simple thing.
35600
31acf6619f08 run-tests: fix regular expression for path test
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35569
diff changeset
  2517
            elif re.search(b'/rust/target/[^/]+/hg', normbin):
35569
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2518
                self._pythondir = os.path.dirname(self._testdir)
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2519
            # Fall back to the legacy behavior.
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2520
            else:
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2521
                self._pythondir = self._bindir
964212780daf rust: implementation of `hg`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35536
diff changeset
  2522
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
  2523
        else:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2524
            self._installdir = os.path.join(self._hgtmp, b"install")
28098
d7b0e81b84c2 run-tests: drop redundant assignment to BINDIR
Yuya Nishihara <yuya@tcha.org>
parents: 28097
diff changeset
  2525
            self._bindir = os.path.join(self._installdir, b"bin")
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
  2526
            self._hgcommand = b'hg'
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2527
            self._tmpbindir = self._bindir
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2528
            self._pythondir = os.path.join(self._installdir, b"lib", b"python")
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
  2529
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
  2530
        # set CHGHG, then replace "hg" command by "chg"
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
  2531
        chgbindir = self._bindir
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2532
        if self.options.chg or self.options.with_chg:
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
  2533
            osenvironb[b'CHGHG'] = os.path.join(self._bindir, self._hgcommand)
28880
f74eed3115fd hghave: add "chg" flag to skip tests that can't be compatible with chg
Yuya Nishihara <yuya@tcha.org>
parents: 28829
diff changeset
  2534
        else:
f74eed3115fd hghave: add "chg" flag to skip tests that can't be compatible with chg
Yuya Nishihara <yuya@tcha.org>
parents: 28829
diff changeset
  2535
            osenvironb.pop(b'CHGHG', None)  # drop flag for hghave
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2536
        if self.options.chg:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2537
            self._hgcommand = b'chg'
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2538
        elif self.options.with_chg:
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
  2539
            chgbindir = os.path.dirname(os.path.realpath(self.options.with_chg))
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
  2540
            self._hgcommand = os.path.basename(self.options.with_chg)
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
  2541
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2542
        osenvironb[b"BINDIR"] = self._bindir
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  2543
        osenvironb[b"PYTHON"] = PYTHON
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
  2544
28582
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
  2545
        if self.options.with_python3:
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
  2546
            osenvironb[b'PYTHON3'] = self.options.with_python3
cdbc25306696 run-tests: add --with-python3 to define a Python 3 interpreter
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28569
diff changeset
  2547
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  2548
        fileb = _bytespath(__file__)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2549
        runtestdir = os.path.abspath(os.path.dirname(fileb))
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  2550
        osenvironb[b'RUNTESTDIR'] = runtestdir
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
  2551
        if PYTHON3:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  2552
            sepb = _bytespath(os.pathsep)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2553
        else:
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2554
            sepb = os.pathsep
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2555
        path = [self._bindir, runtestdir] + osenvironb[b"PATH"].split(sepb)
24742
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
  2556
        if os.path.islink(__file__):
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
  2557
            # test helper will likely be at the end of the symlink
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2558
            realfile = os.path.realpath(fileb)
24742
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
  2559
            realdir = os.path.abspath(os.path.dirname(realfile))
39ee0444e27c run-tests: also follow symlink when update PATH with 'run-tests.py' dir
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24516
diff changeset
  2560
            path.insert(2, realdir)
28142
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
  2561
        if chgbindir != self._bindir:
85e28a46c7f1 run-tests: add --with-chg option to run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28127
diff changeset
  2562
            path.insert(1, chgbindir)
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  2563
        if self._testdir != runtestdir:
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  2564
            path = [self._testdir] + path
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2565
        if self._tmpbindir != self._bindir:
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2566
            path = [self._tmpbindir] + path
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2567
        osenvironb[b"PATH"] = sepb.join(path)
21368
a884548f5421 run-tests: move more path calculations into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21367
diff changeset
  2568
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2569
        # Include TESTDIR in PYTHONPATH so that out-of-tree extensions
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2570
        # can run .../tests/run-tests.py test-foo where test-foo
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2571
        # adds an extension to HGRC. Also include run-test.py directory to
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2572
        # import modules like heredoctest.
23859
661b246bf1c4 run-tests: include testdir in $PATH so tests easily can use helper tools
Mads Kiilerich <madski@unity3d.com>
parents: 23728
diff changeset
  2573
        pypath = [self._pythondir, self._testdir, runtestdir]
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2574
        # We have to augment PYTHONPATH, rather than simply replacing
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2575
        # it, in case external libraries are only available via current
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2576
        # PYTHONPATH.  (In particular, the Subversion bindings on OS X
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2577
        # are in /opt/subversion.)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2578
        oldpypath = osenvironb.get(IMPL_PATH)
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2579
        if oldpypath:
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2580
            pypath.append(oldpypath)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2581
        osenvironb[IMPL_PATH] = sepb.join(pypath)
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2582
23935
d64dd1252386 run-tests.py: inherit --pure option from outer run-tests.py execution
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23933
diff changeset
  2583
        if self.options.pure:
d64dd1252386 run-tests.py: inherit --pure option from outer run-tests.py execution
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23933
diff changeset
  2584
            os.environ["HGTEST_RUN_TESTS_PURE"] = "--pure"
28905
c969c72d6cbc run-tests: set HGMODULEPOLICY for --pure
timeless <timeless@mozdev.org>
parents: 28880
diff changeset
  2585
            os.environ["HGMODULEPOLICY"] = "py"
23935
d64dd1252386 run-tests.py: inherit --pure option from outer run-tests.py execution
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23933
diff changeset
  2586
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
  2587
        if self.options.allow_slow_tests:
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
  2588
            os.environ["HGTEST_SLOW"] = "slow"
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
  2589
        elif 'HGTEST_SLOW' in os.environ:
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
  2590
            del os.environ['HGTEST_SLOW']
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 26087
diff changeset
  2591
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2592
        self._coveragefile = os.path.join(self._testdir, b'.coverage')
21367
522e3d24a461 run-tests: move pypath manipulation into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21366
diff changeset
  2593
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2594
        if self.options.exceptions:
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2595
            exceptionsdir = os.path.join(self._outputdir, b'exceptions')
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2596
            try:
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2597
                os.makedirs(exceptionsdir)
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2598
            except OSError as e:
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2599
                if e.errno != errno.EEXIST:
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2600
                    raise
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2601
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2602
            # Remove all existing exception reports.
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2603
            for f in os.listdir(exceptionsdir):
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2604
                os.unlink(os.path.join(exceptionsdir, f))
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2605
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2606
            osenvironb[b'HGEXCEPTIONSDIR'] = exceptionsdir
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2607
            logexceptions = os.path.join(self._testdir, b'logexceptions.py')
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2608
            self.options.extra_config_opt.append(
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2609
                'extensions.logexceptions=%s' % logexceptions.decode('utf-8'))
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  2610
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2611
        vlog("# Using TESTDIR", self._testdir)
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  2612
        vlog("# Using RUNTESTDIR", osenvironb[b'RUNTESTDIR'])
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2613
        vlog("# Using HGTMP", self._hgtmp)
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2614
        vlog("# Using PATH", os.environ["PATH"])
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2615
        vlog("# Using", IMPL_PATH, osenvironb[IMPL_PATH])
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
  2616
        vlog("# Writing to directory", self._outputdir)
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2617
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2618
        try:
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2619
            return self._runtests(testdescs) or 0
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2620
        finally:
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2621
            time.sleep(.1)
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2622
            self._cleanup()
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2623
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2624
    def findtests(self, args):
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2625
        """Finds possible test files from arguments.
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2626
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2627
        If you wish to inject custom tests into the test harness, this would
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2628
        be a good function to monkeypatch or override in a derived class.
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2629
        """
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2630
        if not args:
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2631
            if self.options.changed:
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2632
                proc = Popen4('hg st --rev "%s" -man0 .' %
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2633
                              self.options.changed, None, 0)
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2634
                stdout, stderr = proc.communicate()
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2635
                args = stdout.strip(b'\0').split(b'\0')
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2636
            else:
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2637
                args = os.listdir(b'.')
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2638
34969
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2639
        expanded_args = []
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2640
        for arg in args:
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2641
            if os.path.isdir(arg):
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2642
                if not arg.endswith(b'/'):
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2643
                    arg += b'/'
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2644
                expanded_args.extend([arg + a for a in os.listdir(arg)])
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2645
            else:
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2646
                expanded_args.append(arg)
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2647
        args = expanded_args
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34966
diff changeset
  2648
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2649
        tests = []
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2650
        for t in args:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2651
            if not (os.path.basename(t).startswith(b'test-')
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2652
                    and (t.endswith(b'.py') or t.endswith(b'.t'))):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2653
                continue
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2654
            if t.endswith(b'.t'):
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2655
                # .t file may contain multiple test cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2656
                cases = sorted(parsettestcases(t))
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2657
                if cases:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2658
                    tests += [{'path': t, 'case': c} for c in sorted(cases)]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2659
                else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2660
                    tests.append({'path': t})
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2661
            else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2662
                tests.append({'path': t})
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2663
        return tests
21363
00e5f5b9fc90 run-tests: move test discovery logic into a function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21362
diff changeset
  2664
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2665
    def _runtests(self, testdescs):
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2666
        def _reloadtest(test, i):
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2667
            # convert a test back to its description dict
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2668
            desc = {'path': test.path}
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2669
            case = getattr(test, '_case', None)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2670
            if case:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2671
                desc['case'] = case
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2672
            return self._gettest(desc, i)
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2673
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2674
        try:
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2675
            if self.options.restart:
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2676
                orig = list(testdescs)
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2677
                while testdescs:
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2678
                    desc = testdescs[0]
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32719
diff changeset
  2679
                    # desc['path'] is a relative path
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2680
                    if 'case' in desc:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2681
                        errpath = b'%s.%s.err' % (desc['path'], desc['case'])
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2682
                    else:
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2683
                        errpath = b'%s.err' % desc['path']
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32719
diff changeset
  2684
                    errpath = os.path.join(self._outputdir, errpath)
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2685
                    if os.path.exists(errpath):
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2686
                        break
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2687
                    testdescs.pop(0)
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2688
                if not testdescs:
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2689
                    print("running all tests")
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2690
                    testdescs = orig
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2691
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2692
            tests = [self._gettest(d, i) for i, d in enumerate(testdescs)]
21437
d9532be2fc4d run-tests: pass Test instances into TestRunner._executetests()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21436
diff changeset
  2693
21458
c42219733f30 run-tests: don't print results in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21457
diff changeset
  2694
            failed = False
25050
28526bb5b3b5 run-tests: make sure keyword(s) are in bytes and not str
Augie Fackler <augie@google.com>
parents: 25049
diff changeset
  2695
            kws = self.options.keywords
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
  2696
            if kws is not None and PYTHON3:
25050
28526bb5b3b5 run-tests: make sure keyword(s) are in bytes and not str
Augie Fackler <augie@google.com>
parents: 25049
diff changeset
  2697
                kws = kws.encode('utf-8')
21458
c42219733f30 run-tests: don't print results in unittest mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21457
diff changeset
  2698
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2699
            suite = TestSuite(self._testdir,
21533
aecac8059c00 run-tests: make testdir an argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21532
diff changeset
  2700
                              jobs=self.options.jobs,
21529
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  2701
                              whitelist=self.options.whitelisted,
117e027390ab run-tests: move whitelist and blacklist to named arguments of TestSuite
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21528
diff changeset
  2702
                              blacklist=self.options.blacklist,
21530
78289625e986 run-tests: make retest a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21529
diff changeset
  2703
                              retest=self.options.retest,
25050
28526bb5b3b5 run-tests: make sure keyword(s) are in bytes and not str
Augie Fackler <augie@google.com>
parents: 25049
diff changeset
  2704
                              keywords=kws,
21532
9d2ba7e2324d run-tests: move loop to a named argument of TestSuite.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21531
diff changeset
  2705
                              loop=self.options.loop,
24329
e7ca4d4b10e1 run-tests: add --runs-per-test flag
Augie Fackler <augie@google.com>
parents: 24306
diff changeset
  2706
                              runs_per_test=self.options.runs_per_test,
27396
64c584070fc7 run-tests: show scheduling with --showchannels
Matt Mackall <mpm@selenic.com>
parents: 27394
diff changeset
  2707
                              showchannels=self.options.showchannels,
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2708
                              tests=tests, loadtest=_reloadtest)
21464
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
  2709
            verbosity = 1
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
  2710
            if self.options.verbose:
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
  2711
                verbosity = 2
d19164a018a1 run-tests: execute tests via unittest
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21463
diff changeset
  2712
            runner = TextTestRunner(self, verbosity=verbosity)
32703
9d1d3199382e run-tests: install hg after computing tests to run
Siddharth Agarwal <sid0@fb.com>
parents: 32702
diff changeset
  2713
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2714
            if self.options.list_tests:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2715
                result = runner.listtests(suite)
32703
9d1d3199382e run-tests: install hg after computing tests to run
Siddharth Agarwal <sid0@fb.com>
parents: 32702
diff changeset
  2716
            else:
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2717
                if self._installdir:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2718
                    self._installhg()
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2719
                    self._checkhglib("Testing")
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2720
                else:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2721
                    self._usecorrectpython()
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2722
                if self.options.chg:
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2723
                    assert self._installdir
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2724
                    self._installchg()
32703
9d1d3199382e run-tests: install hg after computing tests to run
Siddharth Agarwal <sid0@fb.com>
parents: 32702
diff changeset
  2725
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32703
diff changeset
  2726
                result = runner.run(suite)
21613
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
  2727
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
  2728
            if result.failures:
b3213b9fafed run-tests: exit with non-0 exit code when tests fail or warn
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21539
diff changeset
  2729
                failed = True
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2730
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2731
            if self.options.anycoverage:
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
  2732
                self._outputcoverage()
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2733
        except KeyboardInterrupt:
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2734
            failed = True
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2735
            print("\ninterrupted!")
21360
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2736
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2737
        if failed:
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2738
            return 1
becce297ae0c run-tests: move runtests() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21359
diff changeset
  2739
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2740
    def _getport(self, count):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2741
        port = self._ports.get(count) # do we have a cached entry?
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2742
        if port is None:
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2743
            portneeded = 3
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2744
            # above 100 tries we just give up and let test reports failure
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2745
            for tries in xrange(100):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2746
                allfree = True
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
  2747
                port = self.options.port + self._portoffset
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2748
                for idx in xrange(portneeded):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2749
                    if not checkportisavailable(port + idx):
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2750
                        allfree = False
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2751
                        break
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2752
                self._portoffset += portneeded
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2753
                if allfree:
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2754
                    break
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2755
            self._ports[count] = port
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2756
        return port
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2757
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2758
    def _gettest(self, testdesc, count):
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2759
        """Obtain a Test by looking at its filename.
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2760
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2761
        Returns a Test instance. The Test may not be runnable if it doesn't
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2762
        map to a known type.
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2763
        """
32311
a77a75a428f7 run-tests: rename test description dict to testdesc
Jun Wu <quark@fb.com>
parents: 32310
diff changeset
  2764
        path = testdesc['path']
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2765
        lctest = path.lower()
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2766
        testcls = Test
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2767
21501
98a0c58ee200 run-tests: factor refpath into Test classes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21500
diff changeset
  2768
        for ext, cls in self.TESTTYPES:
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2769
            if lctest.endswith(ext):
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2770
                testcls = cls
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2771
                break
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2772
32310
b96be0098624 run-tests: change test identity from a path to a dict
Jun Wu <quark@fb.com>
parents: 32303
diff changeset
  2773
        refpath = os.path.join(self._testdir, path)
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2774
        tmpdir = os.path.join(self._hgtmp, b'child%d' % count)
21504
888a5dfe1569 run-tests: pass temp dir into Test.__init__
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21502
diff changeset
  2775
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2776
        # extra keyword parameters. 'case' is used by .t tests
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2777
        kwds = dict((k, testdesc[k]) for k in ['case'] if k in testdesc)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2778
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
  2779
        t = testcls(refpath, self._outputdir, tmpdir,
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  2780
                    keeptmpdir=self.options.keep_tmpdir,
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  2781
                    debug=self.options.debug,
35541
87676e8ee056 test-run-tests: stabilize the test (issue5735)
Jun Wu <quark@fb.com>
parents: 35540
diff changeset
  2782
                    first=self.options.first,
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  2783
                    timeout=self.options.timeout,
24967
00790cc2b753 run-test: ensure the test ports are available before launching test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24965
diff changeset
  2784
                    startport=self._getport(count),
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  2785
                    extraconfigopts=self.options.extra_config_opt,
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  2786
                    py3kwarnings=self.options.py3k_warnings,
28099
a5f0c0aab2bb run-tests: allow to specify executable of any name by --with-hg
Yuya Nishihara <yuya@tcha.org>
parents: 28098
diff changeset
  2787
                    shell=self.options.shell,
28620
759d167f75cf run-tests: use different chg socket directories for different tests
Jun Wu <quark@fb.com>
parents: 28596
diff changeset
  2788
                    hgcommand=self._hgcommand,
30986
f07ca071a058 runtests: set web.ipv6 if we use IPv6
Jun Wu <quark@fb.com>
parents: 30985
diff changeset
  2789
                    usechg=bool(self.options.with_chg or self.options.chg),
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 32311
diff changeset
  2790
                    useipv6=useipv6, **kwds)
24330
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  2791
        t.should_reload = True
799bc18e14d1 run-tests: avoid running the same test instance concurrently
Augie Fackler <augie@google.com>
parents: 24329
diff changeset
  2792
        return t
21357
4c4f64b8df3c run-tests: move gettest() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21356
diff changeset
  2793
21366
5047248536c5 run-tests: establish TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21365
diff changeset
  2794
    def _cleanup(self):
21350
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2795
        """Clean up state from this test invocation."""
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2796
        if self.options.keep_tmpdir:
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2797
            return
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2798
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2799
        vlog("# Cleaning up HGTMP", self._hgtmp)
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2800
        shutil.rmtree(self._hgtmp, True)
21352
39fd89fbbc3c run-tests: move createdfiles out of a global and into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21351
diff changeset
  2801
        for f in self._createdfiles:
21350
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2802
            try:
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2803
                os.remove(f)
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2804
            except OSError:
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2805
                pass
dfcef61f5bd4 run-tests: move cleanup() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21349
diff changeset
  2806
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
  2807
    def _usecorrectpython(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2808
        """Configure the environment to use the appropriate Python in tests."""
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2809
        # Tests must use the same interpreter as us or bad things will happen.
25041
09c71e3da704 run-tests: even more bytestring annotations for Python 3
Augie Fackler <augie@google.com>
parents: 25040
diff changeset
  2810
        pyexename = sys.platform == 'win32' and b'python.exe' or b'python'
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2811
        if getattr(os, 'symlink', None):
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2812
            vlog("# Making python executable in test path a symlink to '%s'" %
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2813
                 sys.executable)
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2814
            mypython = os.path.join(self._tmpbindir, pyexename)
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2815
            try:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2816
                if os.readlink(mypython) == sys.executable:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2817
                    return
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2818
                os.unlink(mypython)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2819
            except OSError as err:
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2820
                if err.errno != errno.ENOENT:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2821
                    raise
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  2822
            if self._findprogram(pyexename) != sys.executable:
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2823
                try:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2824
                    os.symlink(sys.executable, mypython)
21352
39fd89fbbc3c run-tests: move createdfiles out of a global and into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21351
diff changeset
  2825
                    self._createdfiles.append(mypython)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2826
                except OSError as err:
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2827
                    # child processes may race, which is harmless
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2828
                    if err.errno != errno.EEXIST:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2829
                        raise
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2830
        else:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2831
            exedir, exename = os.path.split(sys.executable)
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2832
            vlog("# Modifying search path to find %s as %s in '%s'" %
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2833
                 (exename, pyexename, exedir))
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2834
            path = os.environ['PATH'].split(os.pathsep)
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2835
            while exedir in path:
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2836
                path.remove(exedir)
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2837
            os.environ['PATH'] = os.pathsep.join([exedir] + path)
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  2838
            if not self._findprogram(pyexename):
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2839
                print("WARNING: Cannot find %s in search path" % pyexename)
21351
fe5647506565 run-tests: move usecorrectpython() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21350
diff changeset
  2840
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
  2841
    def _installhg(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2842
        """Install hg into the test environment.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2843
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2844
        This will also configure hg with the appropriate testing settings.
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  2845
        """
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2846
        vlog("# Performing temporary installation of HG")
28829
65fb87479792 run-tests: move install.err into test area
timeless <timeless@mozdev.org>
parents: 28823
diff changeset
  2847
        installerrs = os.path.join(self._hgtmp, b"install.err")
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2848
        compiler = ''
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2849
        if self.options.compiler:
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2850
            compiler = '--compiler ' + self.options.compiler
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 24075
diff changeset
  2851
        if self.options.pure:
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2852
            pure = b"--pure"
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 24075
diff changeset
  2853
        else:
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2854
            pure = b""
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2855
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2856
        # Run installer in hg root
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2857
        script = os.path.realpath(sys.argv[0])
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2858
        exe = sys.executable
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
  2859
        if PYTHON3:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  2860
            compiler = _bytespath(compiler)
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  2861
            script = _bytespath(script)
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  2862
            exe = _bytespath(exe)
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2863
        hgroot = os.path.dirname(os.path.dirname(script))
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
  2864
        self._hgroot = hgroot
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2865
        os.chdir(hgroot)
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2866
        nohome = b'--home=""'
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2867
        if os.name == 'nt':
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2868
            # The --home="" trick works only on OS where os.sep == '/'
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2869
            # because of a distutils convert_path() fast-path. Avoid it at
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2870
            # least on Windows for now, deal with .pydistutils.cfg bugs
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2871
            # when they happen.
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2872
            nohome = b''
28397
98a1a9547bb1 run-tests: remove 2to3 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28317
diff changeset
  2873
        cmd = (b'%(exe)s setup.py %(pure)s clean --all'
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2874
               b' build %(compiler)s --build-base="%(base)s"'
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2875
               b' install --force --prefix="%(prefix)s"'
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2876
               b' --install-lib="%(libdir)s"'
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2877
               b' --install-scripts="%(bindir)s" %(nohome)s >%(logfile)s 2>&1'
28397
98a1a9547bb1 run-tests: remove 2to3 support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 28317
diff changeset
  2878
               % {b'exe': exe, b'pure': pure,
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2879
                  b'compiler': compiler,
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2880
                  b'base': os.path.join(self._hgtmp, b"build"),
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2881
                  b'prefix': self._installdir, b'libdir': self._pythondir,
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2882
                  b'bindir': self._bindir,
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2883
                  b'nohome': nohome, b'logfile': installerrs})
24075
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2884
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2885
        # setuptools requires install directories to exist.
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2886
        def makedirs(p):
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2887
            try:
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2888
                os.makedirs(p)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2889
            except OSError as e:
24075
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2890
                if e.errno != errno.EEXIST:
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2891
                    raise
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2892
        makedirs(self._pythondir)
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2893
        makedirs(self._bindir)
4bf484276787 run-tests: ensure install directories exist
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24074
diff changeset
  2894
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2895
        vlog("# Running", cmd)
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2896
        if os.system(cmd) == 0:
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2897
            if not self.options.verbose:
26087
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  2898
                try:
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  2899
                    os.remove(installerrs)
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  2900
                except OSError as e:
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  2901
                    if e.errno != errno.ENOENT:
06cd67a5044f run-tests: ignore failed removal of nonexistent installerrs
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  2902
                        raise
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2903
        else:
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2904
            with open(installerrs, 'rb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2905
                for line in f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2906
                    if PYTHON3:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2907
                        sys.stdout.buffer.write(line)
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2908
                    else:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2909
                        sys.stdout.write(line)
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2910
            sys.exit(1)
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2911
        os.chdir(self._testdir)
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2912
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
  2913
        self._usecorrectpython()
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2914
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2915
        if self.options.py3k_warnings and not self.options.anycoverage:
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2916
            vlog("# Updating hg command to enable Py3k Warnings switch")
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2917
            with open(os.path.join(self._bindir, 'hg'), 'rb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2918
                lines = [line.rstrip() for line in f]
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2919
                lines[0] += ' -3'
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2920
            with open(os.path.join(self._bindir, 'hg'), 'wb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2921
                for line in lines:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2922
                    f.write(line + '\n')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2923
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2924
        hgbat = os.path.join(self._bindir, b'hg.bat')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2925
        if os.path.isfile(hgbat):
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2926
            # hg.bat expects to be put in bin/scripts while run-tests.py
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2927
            # installation layout put it in bin/ directly. Fix it
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2928
            with open(hgbat, 'rb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2929
                data = f.read()
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2930
            if b'"%~dp0..\python" "%~dp0hg" %*' in data:
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2931
                data = data.replace(b'"%~dp0..\python" "%~dp0hg" %*',
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2932
                                    b'"%~dp0python" "%~dp0hg" %*')
35450
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2933
                with open(hgbat, 'wb') as f:
e31773898197 run-tests: use context managers for file descriptors
Matt Harbison <matt_harbison@yahoo.com>
parents: 35449
diff changeset
  2934
                    f.write(data)
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2935
            else:
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2936
                print('WARNING: cannot fix hg.bat reference to python.exe')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2937
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2938
        if self.options.anycoverage:
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2939
            custom = os.path.join(self._testdir, 'sitecustomize.py')
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2940
            target = os.path.join(self._pythondir, 'sitecustomize.py')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2941
            vlog('# Installing coverage trigger to %s' % target)
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2942
            shutil.copyfile(custom, target)
21534
3ece55d16044 run-tests: make attributes of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21533
diff changeset
  2943
            rc = os.path.join(self._testdir, '.coveragerc')
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2944
            vlog('# Installing coverage rc to %s' % rc)
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2945
            os.environ['COVERAGE_PROCESS_START'] = rc
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  2946
            covdir = os.path.join(self._installdir, '..', 'coverage')
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  2947
            try:
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  2948
                os.mkdir(covdir)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 24984
diff changeset
  2949
            except OSError as e:
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  2950
                if e.errno != errno.EEXIST:
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  2951
                    raise
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  2952
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  2953
            os.environ['COVERAGE_DIR'] = covdir
21353
a42a5195a182 run-tests: move installhg() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21352
diff changeset
  2954
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
  2955
    def _checkhglib(self, verb):
21354
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
  2956
        """Ensure that the 'mercurial' package imported by python is
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
  2957
        the one we expect it to be.  If not, print a warning to stderr."""
21733
9ad11d5bcc2f run-tests: don't check for the mercurial library used when using --with-hg
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21614
diff changeset
  2958
        if ((self._bindir == self._pythondir) and
9ad11d5bcc2f run-tests: don't check for the mercurial library used when using --with-hg
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21614
diff changeset
  2959
            (self._bindir != self._tmpbindir)):
23139
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
  2960
            # The pythondir has been inferred from --with-hg flag.
e53f6b72a0e4 spelling: fixes from proofreading of spell checker issues
Mads Kiilerich <madski@unity3d.com>
parents: 23077
diff changeset
  2961
            # We cannot expect anything sensible here.
21733
9ad11d5bcc2f run-tests: don't check for the mercurial library used when using --with-hg
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21614
diff changeset
  2962
            return
25044
9de94acfde8a run-tests: fix installation of hg by bytesifying more constants
Augie Fackler <augie@google.com>
parents: 25042
diff changeset
  2963
        expecthg = os.path.join(self._pythondir, b'mercurial')
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2964
        actualhg = self._gethgpath()
21354
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
  2965
        if os.path.abspath(actualhg) != os.path.abspath(expecthg):
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
  2966
            sys.stderr.write('warning: %s with unexpected mercurial lib: %s\n'
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
  2967
                             '         (expected %s)\n'
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
  2968
                             % (verb, actualhg, expecthg))
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2969
    def _gethgpath(self):
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2970
        """Return the path to the mercurial package that is actually found by
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2971
        the current Python interpreter."""
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2972
        if self._hgpath is not None:
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2973
            return self._hgpath
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2974
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  2975
        cmd = b'%s -c "import mercurial; print (mercurial.__path__[0])"'
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  2976
        cmd = cmd % PYTHON
25159
138dc8381049 run-tests: move all open-coded sys.version_info checks to PYTHON3 (issue4668)
Augie Fackler <augie@google.com>
parents: 25158
diff changeset
  2977
        if PYTHON3:
25162
153b9c5235c2 run-tests: replace open-coded .decode()s on paths with a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25161
diff changeset
  2978
            cmd = _strpath(cmd)
25058
caa2043cc322 run-tests: unblock running python tests in python 3
Augie Fackler <augie@google.com>
parents: 25057
diff changeset
  2979
        pipe = os.popen(cmd)
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2980
        try:
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  2981
            self._hgpath = _bytespath(pipe.read().strip())
21385
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2982
        finally:
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2983
            pipe.close()
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2984
28414e5ac9ec run-tests: move _gethgpath() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21384
diff changeset
  2985
        return self._hgpath
21354
29629ef43d39 run-tests: move checkhglib into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21353
diff changeset
  2986
28143
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2987
    def _installchg(self):
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2988
        """Install chg into the test environment"""
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2989
        vlog('# Performing temporary installation of CHG')
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2990
        assert os.path.dirname(self._bindir) == self._installdir
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2991
        assert self._hgroot, 'must be called after _installhg()'
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2992
        cmd = (b'"%(make)s" clean install PREFIX="%(prefix)s"'
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2993
               % {b'make': 'make',  # TODO: switch by option or environment?
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2994
                  b'prefix': self._installdir})
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2995
        cwd = os.path.join(self._hgroot, b'contrib', b'chg')
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2996
        vlog("# Running", cmd)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2997
        proc = subprocess.Popen(cmd, shell=True, cwd=cwd,
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2998
                                stdin=subprocess.PIPE, stdout=subprocess.PIPE,
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  2999
                                stderr=subprocess.STDOUT)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3000
        out, _err = proc.communicate()
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3001
        if proc.returncode != 0:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3002
            if PYTHON3:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3003
                sys.stdout.buffer.write(out)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3004
            else:
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3005
                sys.stdout.write(out)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3006
            sys.exit(1)
c754996fd41f run-tests: add --chg option to install and run tests using chg
Yuya Nishihara <yuya@tcha.org>
parents: 28142
diff changeset
  3007
21378
f7ac3c63d844 run-tests: make some methods of TestRunner internal
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21377
diff changeset
  3008
    def _outputcoverage(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  3009
        """Produce code coverage output."""
29485
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
  3010
        import coverage
6a98f9408a50 py3: make files use absolute_import and print_function
Pulkit Goyal <7895pulkit@gmail.com>
parents: 29282
diff changeset
  3011
        coverage = coverage.coverage
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
  3012
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3013
        vlog('# Producing coverage report')
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3014
        # chdir is the easiest way to get short, relative paths in the
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3015
        # output.
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
  3016
        os.chdir(self._hgroot)
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  3017
        covdir = os.path.join(self._installdir, '..', 'coverage')
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  3018
        cov = coverage(data_file=os.path.join(covdir, 'cov'))
24506
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
  3019
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
  3020
        # Map install directory paths back to source directory.
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
  3021
        cov.config.paths['srcdir'] = ['.', self._pythondir]
60bbb4079c28 run-tests: report code coverage from source directory
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24505
diff changeset
  3022
24505
031947baf4d0 run-tests: collect aggregate code coverage
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24504
diff changeset
  3023
        cov.combine()
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
  3024
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3025
        omit = [os.path.join(x, '*') for x in [self._bindir, self._testdir]]
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3026
        cov.report(ignore_errors=True, omit=omit)
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3027
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
  3028
        if self.options.htmlcov:
32719
74680ed89a29 run-tests: output coverage to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  3029
            htmldir = os.path.join(self._outputdir, 'htmlcov')
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3030
            cov.html_report(directory=htmldir, omit=omit)
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
  3031
        if self.options.annotate:
32719
74680ed89a29 run-tests: output coverage to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  3032
            adir = os.path.join(self._outputdir, 'annotated')
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
  3033
            if not os.path.isdir(adir):
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
  3034
                os.mkdir(adir)
24504
7046ecabd9a8 run-tests: obtain code coverage via Python API
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
  3035
            cov.annotate(directory=adir, omit=omit)
21356
f96d7dfd8cb5 run-tests: move outputcoverage() into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21355
diff changeset
  3036
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3037
    def _findprogram(self, program):
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3038
        """Search PATH for a executable program"""
25161
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  3039
        dpb = _bytespath(os.defpath)
4d30467d944e run-tests: move unicode-to-bytes operations on paths to a helper (issue4667)
Augie Fackler <augie@google.com>
parents: 25160
diff changeset
  3040
        sepb = _bytespath(os.pathsep)
25038
66da89457c47 run-tests: fix _findprogram to reliably return bytes
Augie Fackler <augie@google.com>
parents: 25037
diff changeset
  3041
        for p in osenvironb.get(b'PATH', dpb).split(sepb):
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3042
            name = os.path.join(p, program)
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3043
            if os.name == 'nt' or os.access(name, os.X_OK):
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3044
                return name
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3045
        return None
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3046
21374
592b3d2616d7 run-tests: move checktools into TestRunner.run()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21373
diff changeset
  3047
    def _checktools(self):
21536
92a6b16c9186 run-tests: add docstrings
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21535
diff changeset
  3048
        """Ensure tools required to run tests are present."""
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3049
        for p in self.REQUIREDTOOLS:
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3050
            if os.name == 'nt' and not p.endswith('.exe'):
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3051
                p += '.exe'
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3052
            found = self._findprogram(p)
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3053
            if found:
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3054
                vlog("# Found prerequisite", p, "at", found)
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3055
            else:
28698
c756b6791760 py3: convert prereq bytes to string in run-tests
timeless <timeless@mozdev.org>
parents: 28645
diff changeset
  3056
                print("WARNING: Did not find prerequisite tool: %s " %
c756b6791760 py3: convert prereq bytes to string in run-tests
timeless <timeless@mozdev.org>
parents: 28645
diff changeset
  3057
                      p.decode("utf-8"))
21365
10cf9054d941 run-tests: move program searching into TestRunner
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21364
diff changeset
  3058
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3059
def aggregateexceptions(path):
36037
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3060
    exceptioncounts = collections.Counter()
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3061
    testsbyfailure = collections.defaultdict(set)
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3062
    failuresbytest = collections.defaultdict(set)
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3063
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3064
    for f in os.listdir(path):
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3065
        with open(os.path.join(path, f), 'rb') as fh:
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3066
            data = fh.read().split(b'\0')
36037
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3067
            if len(data) != 5:
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3068
                continue
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3069
36037
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3070
            exc, mainframe, hgframe, hgline, testname = data
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3071
            exc = exc.decode('utf-8')
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3072
            mainframe = mainframe.decode('utf-8')
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3073
            hgframe = hgframe.decode('utf-8')
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3074
            hgline = hgline.decode('utf-8')
36037
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3075
            testname = testname.decode('utf-8')
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3076
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3077
            key = (hgframe, hgline, exc)
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3078
            exceptioncounts[key] += 1
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3079
            testsbyfailure[key].add(testname)
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3080
            failuresbytest[testname].add(key)
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3081
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3082
    # Find test having fewest failures for each failure.
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3083
    leastfailing = {}
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3084
    for key, tests in testsbyfailure.items():
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3085
        fewesttest = None
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3086
        fewestcount = 99999999
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3087
        for test in sorted(tests):
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3088
            if len(failuresbytest[test]) < fewestcount:
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3089
                fewesttest = test
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3090
                fewestcount = len(failuresbytest[test])
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3091
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3092
        leastfailing[key] = (fewestcount, fewesttest)
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3093
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3094
    # Create a combined counter so we can sort by total occurrences and
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3095
    # impacted tests.
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3096
    combined = {}
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3097
    for key in exceptioncounts:
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3098
        combined[key] = (exceptioncounts[key],
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3099
                         len(testsbyfailure[key]),
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3100
                         leastfailing[key][0],
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3101
                         leastfailing[key][1])
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3102
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3103
    return {
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3104
        'exceptioncounts': exceptioncounts,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3105
        'total': sum(exceptioncounts.values()),
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3106
        'combined': combined,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3107
        'leastfailing': leastfailing,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3108
        'byfailure': testsbyfailure,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3109
        'bytest': failuresbytest,
8de90e006c78 run-tests: report tests that exception occurred in
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35991
diff changeset
  3110
    }
35190
bd8875b6473c run-tests: mechanism to report exceptions during test execution
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35189
diff changeset
  3111
13347
ce07defe7d9f run-tests: loadable as module
Simon Heimberg <simohe@besonet.ch>
parents: 13031
diff changeset
  3112
if __name__ == '__main__':
21377
71081f7f9e52 run-tests: eliminate main()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21376
diff changeset
  3113
    runner = TestRunner()
22120
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3114
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3115
    try:
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3116
        import msvcrt
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3117
        msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3118
        msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3119
        msvcrt.setmode(sys.stderr.fileno(), os.O_BINARY)
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3120
    except ImportError:
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3121
        pass
68a7ef4311ce run-tests: self-test on Windows needs binary streams
Matt Mackall <mpm@selenic.com>
parents: 21919
diff changeset
  3122
21377
71081f7f9e52 run-tests: eliminate main()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 21376
diff changeset
  3123
    sys.exit(runner.run(sys.argv[1:]))