view tests/silenttestrunner.py @ 42386:15d5a2de44aa

tests: make run-tests exit non-zero if there are "errors"

Previously, if there was an error such as a broken .t file that caused
run-tests.py to encounter an exception during parsing, the test would be
considered in an "errored" state, which is separate from "failed".

The check for whether to exit non-zero or not was based entirely on whether
there were any tests in a "failed" state, so if there was only an error,
run-tests would exit with 0. Our test infrastructure would then consider the
test as passing, causing us to have some tests with false negatives that have
gone undetected for a few weeks now.

Differential Revision: https://phab.mercurial-scm.org/D6452

from __future__ import absolute_import, print_function
import os
import sys
import unittest

def main(modulename):
    '''run the tests found in module, printing nothing when all tests pass'''
    module = sys.modules[modulename]
    suite = unittest.defaultTestLoader.loadTestsFromModule(module)
    results = unittest.TestResult()
    suite.run(results)
    if results.errors or results.failures:
        for tc, exc in results.errors:
            print('ERROR:', tc)
            print()
            sys.stdout.write(exc)
        for tc, exc in results.failures:
            print('FAIL:', tc)
            print()
            sys.stdout.write(exc)
        sys.exit(1)

if os.environ.get('SILENT_BE_NOISY'):
    main = unittest.main