view tests/.balto.toml @ 42386:15d5a2de44aa

tests: make run-tests exit non-zero if there are "errors"

Previously, if there was an error such as a broken .t file that caused
run-tests.py to encounter an exception during parsing, the test would be
considered in an "errored" state, which is separate from "failed".

The check for whether to exit non-zero or not was based entirely on whether
there were any tests in a "failed" state, so if there was only an error,
run-tests would exit with 0. Our test infrastructure would then consider the
test as passing, causing us to have some tests with false negatives that have
gone undetected for a few weeks now.

Differential Revision: https://phab.mercurial-scm.org/D6452

# Balto (https://bitbucket.org/lothiraldan/balto/src) is a test orchestrator
# which is compatible with all test runner that can emit the LITF
# (https://github.com/lothiraldan/litf) test format.

# The plugin for the Mercurial test runner is mercurial-litf
# (https://pypi.org/project/mercurial-litf/). Make sure to follow the
# instruction and configuration instructions here:
# https://bitbucket.org/lothiraldan/mercurial_litf/src/default/

# You can launch Balto with `balto /path/to/mercurial/tests/`

name = "Mercurial Test Suite"
tool = "mercurial"