tests/test-run-tests.t
author Gregory Szorc <gregory.szorc@gmail.com>
Mon, 17 Sep 2018 15:57:32 -0700
changeset 39677 d27fde3e023e
parent 38824 3086a8627b29
child 39707 5abc47d4ca6b
permissions -rw-r--r--
transaction: make file a private attribute This holds a file handle for the journal file. This file handle should not be touched outside the journal class and doesn't belong on the public interface. Differential Revision: https://phab.mercurial-scm.org/D4625
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
This file tests the behavior of run-tests.py itself.
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     3
Avoid interference from actual test env:
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     4
29220
054bcdad6042 tests: refactor run-tests helpers
timeless <timeless@mozdev.org>
parents: 29199
diff changeset
     5
  $ . "$TESTDIR/helper-runtests.sh"
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     6
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
     7
Smoke test with install
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
============
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
     9
  $ $PYTHON $TESTDIR/run-tests.py $HGTEST_RUN_TESTS_PURE -l
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    10
  
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    11
  # Ran 0 tests, 0 skipped, 0 failed.
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    12
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    13
Define a helper to avoid the install step
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    14
=============
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    15
  $ rt()
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    16
  > {
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
    17
  >     $PYTHON $TESTDIR/run-tests.py --with-hg=`which hg` "$@"
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    18
  > }
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    19
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    20
error paths
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    21
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    22
#if symlink
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    23
  $ ln -s `which true` hg
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
    24
  $ $PYTHON $TESTDIR/run-tests.py --with-hg=./hg
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    25
  warning: --with-hg should specify an hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    26
  
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    27
  # Ran 0 tests, 0 skipped, 0 failed.
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    28
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    29
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    30
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    31
#if execbit
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    32
  $ touch hg
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
    33
  $ $PYTHON $TESTDIR/run-tests.py --with-hg=./hg
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
    34
  usage: run-tests.py [options] [tests]
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    35
  run-tests.py: error: --with-hg must specify an executable hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    36
  [2]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    37
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    38
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    39
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    40
Features for testing optional lines
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    41
===================================
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    42
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    43
  $ cat > hghaveaddon.py <<EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    44
  > import hghave
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    45
  > @hghave.check("custom", "custom hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    46
  > def has_custom():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    47
  >     return True
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    48
  > @hghave.check("missing", "missing hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    49
  > def has_missing():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    50
  >     return False
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    51
  > EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    52
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    53
an empty test
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    54
=======================
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    55
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    56
  $ touch test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    57
  $ rt
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    58
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    59
  # Ran 1 tests, 0 skipped, 0 failed.
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    60
  $ rm test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    61
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    62
a succesful test
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    63
=======================
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    64
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    65
  $ cat > test-success.t << EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    66
  >   $ echo babar
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    67
  >   babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    68
  >   $ echo xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    69
  >   dont_print (?)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    70
  >   nothing[42]line (re) (?)
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28619
diff changeset
    71
  >   never*happens (glob) (?)
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    72
  >   more_nothing (?)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    73
  >   xyzzy
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
    74
  >   nor this (?)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    75
  >   $ printf 'abc\ndef\nxyz\n'
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    76
  >   123 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    77
  >   abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    78
  >   def (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    79
  >   456 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    80
  >   xyz
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    81
  >   $ printf 'zyx\nwvu\ntsr\n'
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    82
  >   abc (?)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    83
  >   zyx (custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    84
  >   wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    85
  >   no_print (no-custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    86
  >   tsr (no-missing !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    87
  >   missing (missing !)
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    88
  > EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    89
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    90
  $ rt
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    91
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    92
  # Ran 1 tests, 0 skipped, 0 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    93
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    94
failing test
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    95
==================
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    96
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    97
test churn with globs
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    98
  $ cat > test-failure.t <<EOF
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
    99
  >   $ echo "bar-baz"; echo "bar-bad"; echo foo
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   100
  >   bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   101
  >   bar*baz (glob)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   102
  >   | fo (re)
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   103
  > EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   104
  $ rt test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   105
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   106
  --- $TESTTMP/test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   107
  +++ $TESTTMP/test-failure.t.err
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   108
  @@ -1,4 +1,4 @@
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   109
     $ echo "bar-baz"; echo "bar-bad"; echo foo
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   110
  +  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   111
     bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   112
  -  bar*baz (glob)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   113
  -  | fo (re)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   114
  +  foo
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   115
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   116
  ERROR: test-failure.t output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   117
  !
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   118
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   119
  # Ran 1 tests, 0 skipped, 1 failed.
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   120
  python hash seed: * (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   121
  [1]
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   122
38553
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   123
test how multiple globs gets matched with lines in output
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   124
  $ cat > test-failure-globs.t <<EOF
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   125
  >   $ echo "context"; echo "context"; \
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   126
  >     echo "key: 1"; echo "value: not a"; \
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   127
  >     echo "key: 2"; echo "value: not b"; \
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   128
  >     echo "key: 3"; echo "value: c"; \
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   129
  >     echo "key: 4"; echo "value: d"
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   130
  >   context
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   131
  >   context
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   132
  >   key: 1
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   133
  >   value: a
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   134
  >   key: 2
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   135
  >   value: b
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   136
  >   key: 3
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   137
  >   value: * (glob)
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   138
  >   key: 4
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   139
  >   value: * (glob)
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   140
  > EOF
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   141
  $ rt test-failure-globs.t
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   142
  
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   143
  --- $TESTTMP/test-failure-globs.t
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   144
  +++ $TESTTMP/test-failure-globs.t.err
38554
f83600efa1ca tests: don't allow reodering of glob/re lines across non-glob/re lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38553
diff changeset
   145
  @@ -2,9 +2,9 @@
38553
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   146
     context
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   147
     context
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   148
     key: 1
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   149
  -  value: a
38554
f83600efa1ca tests: don't allow reodering of glob/re lines across non-glob/re lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38553
diff changeset
   150
  +  value: not a
38553
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   151
     key: 2
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   152
  -  value: b
38554
f83600efa1ca tests: don't allow reodering of glob/re lines across non-glob/re lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38553
diff changeset
   153
  +  value: not b
38553
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   154
     key: 3
38554
f83600efa1ca tests: don't allow reodering of glob/re lines across non-glob/re lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38553
diff changeset
   155
     value: * (glob)
38553
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   156
     key: 4
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   157
  
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   158
  ERROR: test-failure-globs.t output changed
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   159
  !
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   160
  Failed test-failure-globs.t: output changed
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   161
  # Ran 1 tests, 0 skipped, 1 failed.
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   162
  python hash seed: * (glob)
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   163
  [1]
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   164
  $ rm test-failure-globs.t
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   165
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   166
test diff colorisation
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   167
33552
754569f5e999 run-tests: make sure to check if pygments is installed before using it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33500
diff changeset
   168
#if no-windows pygments
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   169
  $ rt test-failure.t --color always
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   170
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   171
  \x1b[38;5;124m--- $TESTTMP/test-failure.t\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   172
  \x1b[38;5;34m+++ $TESTTMP/test-failure.t.err\x1b[39m (esc)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   173
  \x1b[38;5;90;01m@@ -1,4 +1,4 @@\x1b[39;00m (esc)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   174
     $ echo "bar-baz"; echo "bar-bad"; echo foo
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   175
  \x1b[38;5;34m+  bar*baz (glob)\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   176
     bar*bad (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   177
  \x1b[38;5;124m-  bar*baz (glob)\x1b[39m (esc)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   178
  \x1b[38;5;124m-  | fo (re)\x1b[39m (esc)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   179
  \x1b[38;5;34m+  foo\x1b[39m (esc)
34842
8bce3e51b101 run-tests: move newline out of colorized message
Martin von Zweigbergk <martinvonz@google.com>
parents: 34263
diff changeset
   180
  
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33696
diff changeset
   181
  \x1b[38;5;88mERROR: \x1b[39m\x1b[38;5;9mtest-failure.t\x1b[39m\x1b[38;5;88m output changed\x1b[39m (esc)
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   182
  !
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33696
diff changeset
   183
  \x1b[38;5;88mFailed \x1b[39m\x1b[38;5;9mtest-failure.t\x1b[39m\x1b[38;5;88m: output changed\x1b[39m (esc)
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   184
  # Ran 1 tests, 0 skipped, 1 failed.
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   185
  python hash seed: * (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   186
  [1]
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   187
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   188
  $ rt test-failure.t 2> tmp.log
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   189
  [1]
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   190
  $ cat tmp.log
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   191
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   192
  --- $TESTTMP/test-failure.t
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   193
  +++ $TESTTMP/test-failure.t.err
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   194
  @@ -1,4 +1,4 @@
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   195
     $ echo "bar-baz"; echo "bar-bad"; echo foo
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   196
  +  bar*baz (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   197
     bar*bad (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   198
  -  bar*baz (glob)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   199
  -  | fo (re)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   200
  +  foo
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   201
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   202
  ERROR: test-failure.t output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   203
  !
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   204
  Failed test-failure.t: output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   205
  # Ran 1 tests, 0 skipped, 1 failed.
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   206
  python hash seed: * (glob)
33500
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
   207
#endif
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   208
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   209
  $ cat > test-failure.t << EOF
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   210
  >   $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   211
  >   should go away (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   212
  >   $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   213
  >   should stay (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   214
  > 
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   215
  > Should remove first line, not second or third
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   216
  >   $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   217
  >   baz*foo (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   218
  >   foobar*foo (glob) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   219
  >   te*ting (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   220
  > 
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   221
  > Should keep first two lines, remove third and last
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   222
  >   $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   223
  >   test.ng (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   224
  >   foo.ar (re) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   225
  >   b.r (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   226
  >   missing (?)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   227
  >   awol (true !)
33696
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   228
  > 
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   229
  > The "missing" line should stay, even though awol is dropped
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   230
  >   $ echo 'testing'
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   231
  >   test.ng (re) (true !)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   232
  >   foo.ar (?)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   233
  >   awol
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   234
  >   missing (?)
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   235
  > EOF
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   236
  $ rt test-failure.t
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   237
  
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   238
  --- $TESTTMP/test-failure.t
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   239
  +++ $TESTTMP/test-failure.t.err
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   240
  @@ -1,11 +1,9 @@
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   241
     $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   242
  -  should go away (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   243
     $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   244
     should stay (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   245
   
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   246
   Should remove first line, not second or third
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   247
     $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   248
  -  baz*foo (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   249
     foobar*foo (glob) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   250
     te*ting (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   251
   
33696
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   252
     foo.ar (re) (false !)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   253
     missing (?)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   254
  @@ -13,13 +11,10 @@
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   255
     $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   256
     test.ng (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   257
     foo.ar (re) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   258
  -  b.r (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   259
     missing (?)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   260
  -  awol (true !)
33696
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   261
   
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   262
   The "missing" line should stay, even though awol is dropped
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   263
     $ echo 'testing'
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   264
     test.ng (re) (true !)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   265
     foo.ar (?)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   266
  -  awol
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   267
     missing (?)
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   268
  
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   269
  ERROR: test-failure.t output changed
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   270
  !
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   271
  Failed test-failure.t: output changed
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   272
  # Ran 1 tests, 0 skipped, 1 failed.
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   273
  python hash seed: * (glob)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   274
  [1]
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   275
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   276
basic failing test
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   277
  $ cat > test-failure.t << EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   278
  >   $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   279
  >   rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   280
  > This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   281
  > this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   282
  > pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   283
  > pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   284
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   285
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   286
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   287
  > pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   288
  > EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   289
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   290
  >>> fh = open('test-failure-unicode.t', 'wb')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   291
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   292
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   293
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   294
  $ rt
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   295
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   296
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   297
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   298
  @@ -1,5 +1,5 @@
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   299
     $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   300
  -  rataxes
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   301
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   302
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   303
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   304
   pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   305
  
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   306
  ERROR: test-failure.t output changed
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   307
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   308
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   309
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   310
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   311
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   312
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   313
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   314
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   315
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   316
  !
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   317
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   318
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   319
  # Ran 3 tests, 0 skipped, 2 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   320
  python hash seed: * (glob)
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   321
  [1]
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   322
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   323
test --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   324
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   325
  $ rt --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   326
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   327
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   328
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   329
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   330
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   331
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   332
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   333
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   334
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   335
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   336
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   337
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   338
  !.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   339
  --- $TESTTMP/test-failure-unicode.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   340
  +++ $TESTTMP/output/test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   341
  @@ -1,2 +1,2 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   342
     $ echo babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   343
  -  l\xce\xb5\xce\xb5t (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   344
  +  babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   345
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   346
  ERROR: test-failure-unicode.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   347
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   348
  Failed test-failure.t: output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   349
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   350
  # Ran 3 tests, 0 skipped, 2 failed.
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   351
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   352
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   353
  $ ls -a output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   354
  .
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   355
  ..
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
   356
  .testtimes
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   357
  test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   358
  test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   359
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   360
test --xunit support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   361
  $ rt --xunit=xunit.xml
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   362
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   363
  --- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   364
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   365
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   366
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   367
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   368
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   369
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   370
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   371
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   372
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   373
  ERROR: test-failure.t output changed
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   374
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   375
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   376
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   377
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   378
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   379
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   380
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   381
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   382
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   383
  !
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   384
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   385
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   386
  # Ran 3 tests, 0 skipped, 2 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   387
  python hash seed: * (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   388
  [1]
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   389
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   390
  <?xml version="1.0" encoding="utf-8"?>
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   391
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   392
    <testcase name="test-success.t" time="*"/> (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   393
    <testcase name="test-failure-unicode.t" time="*"> (glob)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   394
      <failure message="output changed" type="output-mismatch">
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   395
  <![CDATA[--- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   396
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   397
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   398
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   399
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   400
  +  babar\xce\xb1 (esc)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   401
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   402
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   403
    <testcase name="test-failure.t" time="*"> (glob)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   404
      <failure message="output changed" type="output-mismatch">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   405
  <![CDATA[--- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   406
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   407
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   408
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   409
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   410
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   411
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   412
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   413
   pad pad pad pad............................................................
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   414
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   415
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   416
  </testsuite>
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   417
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   418
  $ cat .testtimes
35855
69d7fcd91696 testrunner: fix updating of .testtimes file
Martin von Zweigbergk <martinvonz@google.com>
parents: 35536
diff changeset
   419
  test-empty.t * (glob)
38553
3fb46734508d tests: add test showing puzzling test output with (glob) lines
Martin von Zweigbergk <martinvonz@google.com>
parents: 38290
diff changeset
   420
  test-failure-globs.t * (glob)
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   421
  test-failure-unicode.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   422
  test-failure.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   423
  test-success.t * (glob)
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   424
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   425
  $ rt --list-tests
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   426
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   427
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   428
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   429
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   430
  $ rt --list-tests --json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   431
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   432
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   433
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   434
  $ cat report.json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   435
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   436
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   437
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   438
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   439
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   440
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   441
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   442
      "test-success.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   443
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   444
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   445
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   446
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   447
  $ rt --list-tests --xunit=xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   448
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   449
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   450
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   451
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   452
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   453
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   454
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   455
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   456
    <testcase name="test-success.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   457
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   458
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   459
  $ rt --list-tests test-failure* --json --xunit=xunit.xml --outputdir output
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   460
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   461
  test-failure.t
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   462
  $ cat output/report.json
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   463
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   464
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   465
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   466
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   467
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   468
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   469
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   470
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   471
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   472
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   473
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   474
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   475
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   476
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   477
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   478
  $ rm test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   479
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   480
test for --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   481
====================
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   482
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   483
  $ rt --retest
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   484
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   485
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   486
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   487
  @@ -1,5 +1,5 @@
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   488
     $ echo babar
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   489
  -  rataxes
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   490
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   491
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   492
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   493
   pad pad pad pad............................................................
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   494
  
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   495
  ERROR: test-failure.t output changed
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   496
  !
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   497
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   498
  # Ran 2 tests, 1 skipped, 1 failed.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   499
  python hash seed: * (glob)
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   500
  [1]
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   501
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   502
--retest works with --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   503
  $ rm -r output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   504
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   505
  $ mv test-failure.t.err output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   506
  $ rt --retest --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   507
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   508
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   509
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   510
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   511
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   512
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   513
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   514
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   515
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   516
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   517
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   518
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   519
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   520
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   521
  # Ran 2 tests, 1 skipped, 1 failed.
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   522
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   523
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   524
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   525
Selecting Tests To Run
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   526
======================
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   527
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   528
successful
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   529
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   530
  $ rt test-success.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   531
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   532
  # Ran 1 tests, 0 skipped, 0 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   533
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   534
success w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   535
  $ rt -k xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   536
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   537
  # Ran 2 tests, 1 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   538
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   539
failed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   540
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   541
  $ rt test-failure.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   542
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   543
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   544
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   545
  @@ -1,5 +1,5 @@
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   546
     $ echo babar
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   547
  -  rataxes
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   548
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   549
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   550
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   551
   pad pad pad pad............................................................
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   552
  
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   553
  ERROR: test-failure.t output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   554
  !
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   555
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   556
  # Ran 1 tests, 0 skipped, 1 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   557
  python hash seed: * (glob)
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   558
  [1]
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   559
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   560
failure w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   561
  $ rt -k rataxes
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   562
  
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   563
  --- $TESTTMP/test-failure.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   564
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   565
  @@ -1,5 +1,5 @@
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   566
     $ echo babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   567
  -  rataxes
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   568
  +  babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   569
   This is a noop statement so that
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   570
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   571
   pad pad pad pad............................................................
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   572
  
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   573
  ERROR: test-failure.t output changed
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   574
  !
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   575
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   576
  # Ran 2 tests, 1 skipped, 1 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   577
  python hash seed: * (glob)
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   578
  [1]
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   579
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   580
Verify that when a process fails to start we show a useful message
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   581
==================================================================
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   582
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   583
  $ cat > test-serve-fail.t <<EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   584
  >   $ echo 'abort: child process failed to start blah'
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   585
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   586
  $ rt test-serve-fail.t
36461
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
   587
  
36438
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   588
  --- $TESTTMP/test-serve-fail.t
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   589
  +++ $TESTTMP/test-serve-fail.t.err
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   590
  @@ -1* +1,2 @@ (glob)
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   591
     $ echo 'abort: child process failed to start blah'
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   592
  +  abort: child process failed to start blah
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   593
  
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   594
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   595
  !
36461
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
   596
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   597
  # Ran 1 tests, 0 skipped, 1 failed.
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   598
  python hash seed: * (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   599
  [1]
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   600
  $ rm test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   601
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   602
Verify that we can try other ports
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   603
===================================
37434
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   604
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   605
Extensions aren't inherited by the invoked run-tests.py. An extension
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   606
introducing a repository requirement could cause this to fail. So we force
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   607
HGRCPATH to get a clean environment.
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   608
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   609
  $ HGRCPATH= hg init inuse
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   610
  $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   611
  $ cat blocks.pid >> $DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   612
  $ cat > test-serve-inuse.t <<EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   613
  >   $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   614
  >   $ cat hg.pid >> \$DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   615
  > EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   616
  $ rt test-serve-inuse.t
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   617
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   618
  # Ran 1 tests, 0 skipped, 0 failed.
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   619
  $ rm test-serve-inuse.t
29221
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   620
  $ killdaemons.py $DAEMON_PIDS
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   621
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   622
Running In Debug Mode
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   623
======================
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   624
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   625
  $ rt --debug 2>&1 | grep -v pwd
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   626
  + echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   627
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   628
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   629
  babar
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   630
  + echo *SALT* 10 0 (glob)
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   631
  *SALT* 10 0 (glob)
27509
6ff90d618d72 test-run-tests: glob away a --debug run difference on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 27395
diff changeset
   632
  *+ echo *SALT* 0 0 (glob)
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   633
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   634
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   635
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   636
  + echo *SALT* 2 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   637
  *SALT* 2 0 (glob)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   638
  + echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   639
  xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   640
  + echo *SALT* 9 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   641
  *SALT* 9 0 (glob)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   642
  + printf *abc\ndef\nxyz\n* (glob)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   643
  abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   644
  def
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   645
  xyz
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   646
  + echo *SALT* 15 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   647
  *SALT* 15 0 (glob)
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   648
  + printf *zyx\nwvu\ntsr\n* (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   649
  zyx
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   650
  wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   651
  tsr
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   652
  + echo *SALT* 22 0 (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   653
  *SALT* 22 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   654
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   655
  # Ran 2 tests, 0 skipped, 0 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   656
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   657
Parallel runs
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   658
==============
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   659
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   660
(duplicate the failing test to get predictable output)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   661
  $ cp test-failure.t test-failure-copy.t
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   662
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   663
  $ rt --jobs 2 test-failure*.t -n
23107
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   664
  !!
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   665
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   666
  Failed test-failure*.t: output changed (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   667
  # Ran 2 tests, 0 skipped, 2 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   668
  python hash seed: * (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   669
  [1]
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   670
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   671
failures in parallel with --first should only print one failure
34894
b63a7d839c33 tests: de-flake test-run-tests.t's "--jobs=2 --first" test
Martin von Zweigbergk <martinvonz@google.com>
parents: 34842
diff changeset
   672
  $ rt --jobs 2 --first test-failure*.t
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   673
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   674
  --- $TESTTMP/test-failure*.t (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   675
  +++ $TESTTMP/test-failure*.t.err (glob)
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   676
  @@ -1,5 +1,5 @@
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   677
     $ echo babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   678
  -  rataxes
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   679
  +  babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   680
   This is a noop statement so that
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   681
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   682
   pad pad pad pad............................................................
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   683
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   684
  Failed test-failure*.t: output changed (glob)
34894
b63a7d839c33 tests: de-flake test-run-tests.t's "--jobs=2 --first" test
Martin von Zweigbergk <martinvonz@google.com>
parents: 34842
diff changeset
   685
  Failed test-failure*.t: output changed (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   686
  # Ran 2 tests, 0 skipped, 2 failed.
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   687
  python hash seed: * (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   688
  [1]
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   689
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   690
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   691
(delete the duplicated test file)
34894
b63a7d839c33 tests: de-flake test-run-tests.t's "--jobs=2 --first" test
Martin von Zweigbergk <martinvonz@google.com>
parents: 34842
diff changeset
   692
  $ rm test-failure-copy.t
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   693
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   694
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   695
Interactive run
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   696
===============
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   697
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   698
(backup the failing test)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   699
  $ cp test-failure.t backup
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   700
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   701
Refuse the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   702
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   703
  $ echo 'n' | rt -i
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   704
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   705
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   706
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   707
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   708
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   709
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   710
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   711
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   712
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   713
   pad pad pad pad............................................................
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   714
  Accept this change? [n] 
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   715
  ERROR: test-failure.t output changed
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   716
  !.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   717
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   718
  # Ran 2 tests, 0 skipped, 1 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   719
  python hash seed: * (glob)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   720
  [1]
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   721
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   722
  $ cat test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   723
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   724
    rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   725
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   726
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   727
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   728
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   729
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   730
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   731
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   732
  pad pad pad pad............................................................
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   733
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   734
Interactive with custom view
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   735
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   736
  $ echo 'n' | rt -i --view echo
35393
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   737
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   738
  Accept this change? [n]* (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   739
  ERROR: test-failure.t output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   740
  !.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   741
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   742
  # Ran 2 tests, 0 skipped, 1 failed.
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   743
  python hash seed: * (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   744
  [1]
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   745
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   746
View the fix
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   747
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   748
  $ echo 'y' | rt --view echo
35393
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   749
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   750
  
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   751
  ERROR: test-failure.t output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   752
  !.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   753
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   754
  # Ran 2 tests, 0 skipped, 1 failed.
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   755
  python hash seed: * (glob)
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   756
  [1]
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   757
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   758
Accept the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   759
35416
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   760
  $ cat >> test-failure.t <<EOF
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   761
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   762
  >   saved backup bundle to \$TESTTMP/foo.hg
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   763
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   764
  >   saved backup bundle to $TESTTMP\\foo.hg
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   765
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   766
  >   saved backup bundle to \$TESTTMP/*.hg (glob)
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   767
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   768
  $ echo 'y' | rt -i 2>&1
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   769
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   770
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   771
  +++ $TESTTMP/test-failure.t.err
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   772
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   773
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   774
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   775
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   776
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   777
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   778
   pad pad pad pad............................................................
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   779
  @@ -11,6 +11,6 @@
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   780
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   781
     saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   782
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   783
  -  saved backup bundle to $TESTTMP\foo.hg
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   784
  +  saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   785
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   786
     saved backup bundle to $TESTTMP/*.hg (glob)
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   787
  Accept this change? [n] ..
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   788
  # Ran 2 tests, 0 skipped, 0 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   789
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   790
  $ sed -e 's,(glob)$,&<,g' test-failure.t
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   791
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   792
    babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   793
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   794
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   795
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   796
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   797
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   798
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   799
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   800
  pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   801
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   802
    saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   803
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   804
    saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   805
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   806
    saved backup bundle to $TESTTMP/*.hg (glob)<
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   807
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   808
Race condition - test file was modified when test is running
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   809
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   810
  $ TESTRACEDIR=`pwd`
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   811
  $ export TESTRACEDIR
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   812
  $ cat > test-race.t <<EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   813
  >   $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   814
  >   $ echo "# a new line" >> $TESTRACEDIR/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   815
  > EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   816
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   817
  $ rt -i test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   818
  
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   819
  --- $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   820
  +++ $TESTTMP/test-race.t.err
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   821
  @@ -1,2 +1,3 @@
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   822
     $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   823
  +  1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   824
     $ echo "# a new line" >> $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   825
  Reference output has changed (run again to prompt changes)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   826
  ERROR: test-race.t output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   827
  !
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   828
  Failed test-race.t: output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   829
  # Ran 1 tests, 0 skipped, 1 failed.
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   830
  python hash seed: * (glob)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   831
  [1]
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   832
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   833
  $ rm test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   834
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   835
When "#testcases" is used in .t files
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   836
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   837
  $ cat >> test-cases.t <<EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   838
  > #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   839
  > #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   840
  >   $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   841
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   842
  > #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   843
  >   $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   844
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   845
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   846
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   847
  $ cat <<EOF | rt -i test-cases.t 2>&1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   848
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   849
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   850
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   851
  
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   852
  --- $TESTTMP/test-cases.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
   853
  +++ $TESTTMP/test-cases.t#a.err
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   854
  @@ -1,6 +1,7 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   855
   #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   856
   #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   857
     $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   858
  +  1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   859
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   860
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   861
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   862
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   863
  --- $TESTTMP/test-cases.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
   864
  +++ $TESTTMP/test-cases.t#b.err
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   865
  @@ -5,4 +5,5 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   866
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   867
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   868
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   869
  +  2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   870
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   871
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   872
  # Ran 2 tests, 0 skipped, 0 failed.
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   873
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   874
  $ cat test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   875
  #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   876
  #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   877
    $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   878
    1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   879
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   880
  #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   881
    $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   882
    2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   883
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   884
33934
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   885
  $ cat >> test-cases.t <<'EOF'
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   886
  > #if a
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   887
  >   $ NAME=A
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   888
  > #else
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   889
  >   $ NAME=B
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   890
  > #endif
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   891
  >   $ echo $NAME
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   892
  >   A (a !)
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   893
  >   B (b !)
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   894
  > EOF
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   895
  $ rt test-cases.t
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   896
  ..
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   897
  # Ran 2 tests, 0 skipped, 0 failed.
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   898
38824
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   899
When using multiple dimensions of "#testcases" in .t files
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   900
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   901
  $ cat > test-cases.t <<'EOF'
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   902
  > #testcases a b
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   903
  > #testcases c d
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   904
  > #if a d
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   905
  >   $ echo $TESTCASE
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   906
  >   a#d
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   907
  > #endif
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   908
  > #if b c
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   909
  >   $ echo yes
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   910
  >   no
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   911
  > #endif
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   912
  > EOF
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   913
  $ rt test-cases.t
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   914
  ..
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   915
  --- $TESTTMP/test-cases.t
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   916
  +++ $TESTTMP/test-cases.t#b#c.err
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   917
  @@ -6,5 +6,5 @@
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   918
   #endif
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   919
   #if b c
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   920
     $ echo yes
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   921
  -  no
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   922
  +  yes
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   923
   #endif
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   924
  
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   925
  ERROR: test-cases.t#b#c output changed
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   926
  !.
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   927
  Failed test-cases.t#b#c: output changed
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   928
  # Ran 4 tests, 0 skipped, 1 failed.
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   929
  python hash seed: * (glob)
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   930
  [1]
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   931
3086a8627b29 testrunner: allow multiple #testcases
Martin von Zweigbergk <martinvonz@google.com>
parents: 38823
diff changeset
   932
  $ rm test-cases.t#b#c.err
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   933
  $ rm test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   934
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   935
(reinstall)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   936
  $ mv backup test-failure.t
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   937
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   938
No Diff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   939
===============
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   940
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   941
  $ rt --nodiff
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   942
  !.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   943
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   944
  # Ran 2 tests, 0 skipped, 1 failed.
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   945
  python hash seed: * (glob)
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   946
  [1]
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   947
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   948
test --tmpdir support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   949
  $ rt --tmpdir=$TESTTMP/keep test-success.t
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   950
  
35393
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   951
  Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   952
  Keeping threadtmp dir: $TESTTMP/keep/child1 
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   953
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   954
  # Ran 1 tests, 0 skipped, 0 failed.
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   955
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   956
timeouts
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   957
========
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   958
  $ cat > test-timeout.t <<EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   959
  >   $ sleep 2
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   960
  >   $ echo pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   961
  >   pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   962
  > EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   963
  > echo '#require slow' > test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   964
  > cat test-timeout.t >> test-slow-timeout.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   965
  $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   966
  st
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
   967
  Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests)
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   968
  Failed test-timeout.t: timed out
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   969
  # Ran 1 tests, 1 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   970
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   971
  [1]
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   972
  $ rt --timeout=1 --slowtimeout=3 \
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   973
  > test-timeout.t test-slow-timeout.t --allow-slow-tests
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   974
  .t
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   975
  Failed test-timeout.t: timed out
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   976
  # Ran 2 tests, 0 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   977
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   978
  [1]
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   979
  $ rm test-timeout.t test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   980
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   981
test for --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   982
==================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   983
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   984
  $ rt test-success.t --time
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   985
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   986
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   987
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   988
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   989
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   990
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   991
test for --time with --job enabled
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   992
====================================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   993
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   994
  $ rt test-success.t --time --jobs 2
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   995
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   996
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   997
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   998
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   999
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1000
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1001
Skips
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1002
================
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1003
  $ cat > test-skip.t <<EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1004
  >   $ echo xyzzy
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1005
  > #if true
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
  1006
  > #require false
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1007
  > #end
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1008
  > EOF
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1009
  $ cat > test-noskip.t <<EOF
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1010
  > #if false
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1011
  > #require false
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1012
  > #endif
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1013
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1014
  $ rt --nodiff
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1015
  !.s.
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1016
  Skipped test-skip.t: missing feature: nail clipper
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1017
  Failed test-failure.t: output changed
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1018
  # Ran 3 tests, 1 skipped, 1 failed.
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1019
  python hash seed: * (glob)
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1020
  [1]
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
  1021
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
  1022
  $ rm test-noskip.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1023
  $ rt --keyword xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
  1024
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1025
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1026
  # Ran 2 tests, 2 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
  1027
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1028
Skips with xml
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1029
  $ rt --keyword xyzzy \
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1030
  >  --xunit=xunit.xml
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
  1031
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1032
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1033
  # Ran 2 tests, 2 skipped, 0 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1034
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1035
  <?xml version="1.0" encoding="utf-8"?>
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
  1036
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1037
    <testcase name="test-success.t" time="*"/> (glob)
32715
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  1038
    <testcase name="test-skip.t">
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  1039
      <skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  1040
  <![CDATA[missing feature: nail clipper]]>    </skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
  1041
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1042
  </testsuite>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
  1043
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
  1044
Missing skips or blacklisted skips don't count as executed:
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
  1045
  $ echo test-failure.t > blacklist
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1046
  $ rt --blacklist=blacklist --json\
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
  1047
  >   test-failure.t test-bogus.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
  1048
  ss
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
  1049
  Skipped test-bogus.t: Doesn't exist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
  1050
  Skipped test-failure.t: blacklisted
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1051
  # Ran 0 tests, 2 skipped, 0 failed.
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1052
  $ cat report.json
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1053
  testreport ={
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1054
      "test-bogus.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1055
          "result": "skip"
29199
daff05dcd184 run-tests: handle json.dumps divergence
timeless <timeless@mozdev.org>
parents: 29173
diff changeset
  1056
      },
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1057
      "test-failure.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1058
          "result": "skip"
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1059
      }
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
  1060
  } (no-eol)
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1061
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1062
Whitelist trumps blacklist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1063
  $ echo test-failure.t > whitelist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1064
  $ rt --blacklist=blacklist --whitelist=whitelist --json\
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1065
  >   test-failure.t test-bogus.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1066
  s
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1067
  --- $TESTTMP/test-failure.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1068
  +++ $TESTTMP/test-failure.t.err
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1069
  @@ -1,5 +1,5 @@
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1070
     $ echo babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1071
  -  rataxes
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1072
  +  babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1073
   This is a noop statement so that
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1074
   this test is still more bytes than success.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1075
   pad pad pad pad............................................................
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1076
  
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1077
  ERROR: test-failure.t output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1078
  !
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1079
  Skipped test-bogus.t: Doesn't exist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1080
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1081
  # Ran 1 tests, 1 skipped, 1 failed.
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1082
  python hash seed: * (glob)
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1083
  [1]
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1084
34263
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1085
Ensure that --test-list causes only the tests listed in that file to
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1086
be executed.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1087
  $ echo test-success.t >> onlytest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1088
  $ rt --test-list=onlytest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1089
  .
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1090
  # Ran 1 tests, 0 skipped, 0 failed.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1091
  $ echo test-bogus.t >> anothertest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1092
  $ rt --test-list=onlytest --test-list=anothertest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1093
  s.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1094
  Skipped test-bogus.t: Doesn't exist
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1095
  # Ran 1 tests, 1 skipped, 0 failed.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1096
  $ rm onlytest anothertest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1097
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1098
test for --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1099
==================
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1100
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1101
  $ rt --json
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1102
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1103
  --- $TESTTMP/test-failure.t
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1104
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1105
  @@ -1,5 +1,5 @@
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1106
     $ echo babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1107
  -  rataxes
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1108
  +  babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1109
   This is a noop statement so that
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1110
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1111
   pad pad pad pad............................................................
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1112
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1113
  ERROR: test-failure.t output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1114
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1115
  Skipped test-skip.t: missing feature: nail clipper
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1116
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1117
  # Ran 2 tests, 1 skipped, 1 failed.
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1118
  python hash seed: * (glob)
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1119
  [1]
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1120
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1121
  $ cat report.json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1122
  testreport ={
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1123
      "test-failure.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1124
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1125
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1126
          "diff": "---.+\+\+\+.+", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1127
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1128
          "result": "failure", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1129
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1130
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1131
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1132
      "test-skip.t": {
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1133
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1134
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1135
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1136
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1137
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1138
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1139
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1140
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1141
      "test-success.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1142
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1143
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1144
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1145
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1146
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1147
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1148
          "time": "\s*[\d\.]{4,5}" (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1149
      }
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1150
  } (no-eol)
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1151
--json with --outputdir
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1152
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1153
  $ rm report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1154
  $ rm -r output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1155
  $ mkdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1156
  $ rt --json --outputdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1157
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1158
  --- $TESTTMP/test-failure.t
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1159
  +++ $TESTTMP/output/test-failure.t.err
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1160
  @@ -1,5 +1,5 @@
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1161
     $ echo babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1162
  -  rataxes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1163
  +  babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1164
   This is a noop statement so that
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1165
   this test is still more bytes than success.
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1166
   pad pad pad pad............................................................
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1167
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1168
  ERROR: test-failure.t output changed
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1169
  !.s
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1170
  Skipped test-skip.t: missing feature: nail clipper
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1171
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1172
  # Ran 2 tests, 1 skipped, 1 failed.
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1173
  python hash seed: * (glob)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1174
  [1]
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1175
  $ f report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1176
  report.json: file not found
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1177
  $ cat output/report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1178
  testreport ={
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1179
      "test-failure.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1180
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1181
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1182
          "diff": "---.+\+\+\+.+", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1183
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1184
          "result": "failure", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1185
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1186
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1187
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1188
      "test-skip.t": {
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1189
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1190
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1191
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1192
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1193
          "result": "skip", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1194
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1195
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1196
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1197
      "test-success.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1198
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1199
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1200
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1201
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1202
          "result": "success", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1203
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1204
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1205
      }
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1206
  } (no-eol)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1207
  $ ls -a output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1208
  .
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1209
  ..
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1210
  .testtimes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1211
  report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1212
  test-failure.t.err
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
  1213
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1214
Test that failed test accepted through interactive are properly reported:
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1215
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1216
  $ cp test-failure.t backup
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1217
  $ echo y | rt --json -i
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1218
  
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1219
  --- $TESTTMP/test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1220
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1221
  @@ -1,5 +1,5 @@
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1222
     $ echo babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1223
  -  rataxes
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1224
  +  babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1225
   This is a noop statement so that
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1226
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1227
   pad pad pad pad............................................................
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1228
  Accept this change? [n] ..s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1229
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1230
  # Ran 2 tests, 1 skipped, 0 failed.
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1231
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1232
  $ cat report.json
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1233
  testreport ={
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1234
      "test-failure.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1235
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1236
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1237
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1238
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1239
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1240
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1241
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1242
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1243
      "test-skip.t": {
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1244
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1245
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1246
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1247
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1248
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1249
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1250
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1251
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1252
      "test-success.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1253
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1254
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1255
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1256
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1257
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1258
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1259
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1260
      }
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1261
  } (no-eol)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1262
  $ mv backup test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1263
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1264
backslash on end of line with glob matching is handled properly
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1265
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1266
  $ cat > test-glob-backslash.t << EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1267
  >   $ echo 'foo bar \\'
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1268
  >   foo * \ (glob)
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1269
  > EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1270
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1271
  $ rt test-glob-backslash.t
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1272
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1273
  # Ran 1 tests, 0 skipped, 0 failed.
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1274
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1275
  $ rm -f test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1276
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1277
Test globbing of local IP addresses
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1278
  $ echo 172.16.18.1
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1279
  $LOCALIP (glob)
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1280
  $ echo dead:beef::1
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1281
  $LOCALIP (glob)
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1282
38616
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1283
Add support for external test formatter
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1284
=======================================
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1285
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1286
  $ CUSTOM_TEST_RESULT=basic_test_result $PYTHON $TESTDIR/run-tests.py --with-hg=`which hg` "$@" test-success.t test-failure.t
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1287
  
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1288
  # Ran 2 tests, 0 skipped, 0 failed.
38621
f4a214300957 run-tests: add missing life-cycle methods on the example custom test result
Boris Feld <boris.feld@octobus.net>
parents: 38616
diff changeset
  1289
  ON_START! <__main__.TestSuite tests=[<__main__.TTest testMethod=test-failure.t>, <__main__.TTest testMethod=test-success.t>]>
38616
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1290
  FAILURE! test-failure.t output changed
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1291
  SUCCESS! test-success.t
38621
f4a214300957 run-tests: add missing life-cycle methods on the example custom test result
Boris Feld <boris.feld@octobus.net>
parents: 38616
diff changeset
  1292
  ON_END!
38616
c44ae5997869 run-tests: add support for external test result
Boris Feld <boris.feld@octobus.net>
parents: 38554
diff changeset
  1293
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1294
Test reusability for third party tools
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1295
======================================
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1296
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1297
  $ mkdir "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1298
  $ cd "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1299
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1300
test that `run-tests.py` can execute hghave, even if it runs not in
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1301
Mercurial source tree.
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1302
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1303
  $ cat > test-hghave.t <<EOF
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1304
  > #require true
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1305
  >   $ echo foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1306
  >   foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1307
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1308
  $ rt test-hghave.t
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1309
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1310
  # Ran 1 tests, 0 skipped, 0 failed.
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1311
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1312
test that RUNTESTDIR refers the directory, in which `run-tests.py` now
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1313
running is placed.
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1314
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1315
  $ cat > test-runtestdir.t <<EOF
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1316
  > - $TESTDIR, in which test-run-tests.t is placed
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1317
  > - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1318
  > - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1319
  > 
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1320
  > #if windows
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1321
  >   $ test "\$TESTDIR" = "$TESTTMP\anothertests"
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1322
  > #else
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1323
  >   $ test "\$TESTDIR" = "$TESTTMP"/anothertests
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1324
  > #endif
35387
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1325
  > If this prints a path, that means RUNTESTDIR didn't equal
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1326
  > TESTDIR as it should have.
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1327
  >   $ test "\$RUNTESTDIR" = "$TESTDIR" || echo "\$RUNTESTDIR"
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1328
  > This should print the start of check-code. If this passes but the
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1329
  > previous check failed, that means we found a copy of check-code at whatever
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1330
  > RUNTESTSDIR ended up containing, even though it doesn't match TESTDIR.
32937
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  1331
  >   $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py | sed 's@.!.*python@#!USRBINENVPY@'
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  1332
  >   #!USRBINENVPY
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1333
  >   #
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1334
  >   # check-code - a style and portability checker for Mercurial
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1335
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1336
  $ rt test-runtestdir.t
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1337
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1338
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1339
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1340
#if execbit
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1341
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1342
test that TESTDIR is referred in PATH
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1343
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1344
  $ cat > custom-command.sh <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1345
  > #!/bin/sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1346
  > echo "hello world"
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1347
  > EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1348
  $ chmod +x custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1349
  $ cat > test-testdir-path.t <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1350
  >   $ custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1351
  >   hello world
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1352
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1353
  $ rt test-testdir-path.t
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1354
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1355
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1356
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1357
#endif
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1358
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1359
test support for --allow-slow-tests
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1360
  $ cat > test-very-slow-test.t <<EOF
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1361
  > #require slow
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1362
  >   $ echo pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1363
  >   pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1364
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1365
  $ rt test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1366
  s
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
  1367
  Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1368
  # Ran 0 tests, 1 skipped, 0 failed.
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1369
  $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1370
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1371
  # Ran 1 tests, 0 skipped, 0 failed.
28180
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1372
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1373
support for running a test outside the current directory
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1374
  $ mkdir nonlocal
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1375
  $ cat > nonlocal/test-is-not-here.t << EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1376
  >   $ echo pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1377
  >   pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1378
  > EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1379
  $ rt nonlocal/test-is-not-here.t
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1380
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1381
  # Ran 1 tests, 0 skipped, 0 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1382
34969
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1383
support for automatically discovering test if arg is a folder
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1384
  $ mkdir tmp && cd tmp
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1385
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1386
  $ cat > test-uno.t << EOF
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1387
  >   $ echo line
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1388
  >   line
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1389
  > EOF
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1390
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1391
  $ cp test-uno.t test-dos.t
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1392
  $ cd ..
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1393
  $ cp -R tmp tmpp
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1394
  $ cp tmp/test-uno.t test-solo.t
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1395
35241
2a99c5bfea47 test-run-tests: do not rebuild hg in the test
Jun Wu <quark@fb.com>
parents: 35189
diff changeset
  1396
  $ rt tmp/ test-solo.t tmpp
34969
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1397
  .....
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1398
  # Ran 5 tests, 0 skipped, 0 failed.
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1399
  $ rm -rf tmp tmpp
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1400
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1401
support for running run-tests.py from another directory
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1402
  $ mkdir tmp && cd tmp
35097
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1403
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1404
  $ cat > useful-file.sh << EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1405
  > important command
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1406
  > EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1407
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1408
  $ cat > test-folder.t << EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1409
  >   $ cat \$TESTDIR/useful-file.sh
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1410
  >   important command
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1411
  > EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1412
35097
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1413
  $ cat > test-folder-fail.t << EOF
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1414
  >   $ cat \$TESTDIR/useful-file.sh
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1415
  >   important commando
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1416
  > EOF
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1417
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1418
  $ cd ..
35241
2a99c5bfea47 test-run-tests: do not rebuild hg in the test
Jun Wu <quark@fb.com>
parents: 35189
diff changeset
  1419
  $ rt tmp/test-*.t
35097
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1420
  
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1421
  --- $TESTTMP/anothertests/tmp/test-folder-fail.t
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1422
  +++ $TESTTMP/anothertests/tmp/test-folder-fail.t.err
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1423
  @@ -1,2 +1,2 @@
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1424
     $ cat $TESTDIR/useful-file.sh
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1425
  -  important commando
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1426
  +  important command
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1427
  
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1428
  ERROR: test-folder-fail.t output changed
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1429
  !.
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1430
  Failed test-folder-fail.t: output changed
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1431
  # Ran 2 tests, 0 skipped, 1 failed.
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1432
  python hash seed: * (glob)
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1433
  [1]
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1434
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1435
support for bisecting failed tests automatically
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1436
  $ hg init bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1437
  $ cd bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1438
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1439
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1440
  >   pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1441
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1442
  $ hg add test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1443
  $ hg ci -m 'good'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1444
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1445
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1446
  >   fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1447
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1448
  $ hg ci -m 'bad'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1449
  $ rt --known-good-rev=0 test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1450
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1451
  --- $TESTTMP/anothertests/bisect/test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1452
  +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1453
  @@ -1,4 +1,4 @@
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1454
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1455
     pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1456
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1457
  -  fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1458
  +  pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1459
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1460
  ERROR: test-bisect.t output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1461
  !
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1462
  Failed test-bisect.t: output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1463
  test-bisect.t broken by 72cbf122d116 (bad)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1464
  # Ran 1 tests, 0 skipped, 1 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1465
  python hash seed: * (glob)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1466
  [1]
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1467
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1468
  $ cd ..
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1469
34041
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1470
support bisecting a separate repo
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1471
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1472
  $ hg init bisect-dependent
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1473
  $ cd bisect-dependent
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1474
  $ cat > test-bisect-dependent.t <<EOF
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1475
  >   $ tail -1 \$TESTDIR/../bisect/test-bisect.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1476
  >     pass
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1477
  > EOF
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1478
  $ hg commit -Am dependent test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1479
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1480
  $ rt --known-good-rev=0 test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1481
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1482
  --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1483
  +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1484
  @@ -1,2 +1,2 @@
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1485
     $ tail -1 $TESTDIR/../bisect/test-bisect.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1486
  -    pass
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1487
  +    fail
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1488
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1489
  ERROR: test-bisect-dependent.t output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1490
  !
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1491
  Failed test-bisect-dependent.t: output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1492
  Failed to identify failure point for test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1493
  # Ran 1 tests, 0 skipped, 1 failed.
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1494
  python hash seed: * (glob)
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1495
  [1]
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1496
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1497
  $ rt --bisect-repo=../test-bisect test-bisect-dependent.t
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
  1498
  usage: run-tests.py [options] [tests]
34041
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1499
  run-tests.py: error: --bisect-repo cannot be used without --known-good-rev
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1500
  [2]
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1501
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1502
  $ rt --known-good-rev=0 --bisect-repo=../bisect test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1503
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1504
  --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1505
  +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1506
  @@ -1,2 +1,2 @@
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1507
     $ tail -1 $TESTDIR/../bisect/test-bisect.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1508
  -    pass
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1509
  +    fail
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1510
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1511
  ERROR: test-bisect-dependent.t output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1512
  !
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1513
  Failed test-bisect-dependent.t: output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1514
  test-bisect-dependent.t broken by 72cbf122d116 (bad)
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1515
  # Ran 1 tests, 0 skipped, 1 failed.
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1516
  python hash seed: * (glob)
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1517
  [1]
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1518
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1519
  $ cd ..
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1520
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1521
Test a broken #if statement doesn't break run-tests threading.
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1522
==============================================================
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1523
  $ mkdir broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1524
  $ cd broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1525
  $ cat > test-broken.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1526
  > true
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1527
  > #if notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1528
  >   $ false
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1529
  > #endif
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1530
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1531
  $ for f in 1 2 3 4 ; do
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1532
  > cat > test-works-$f.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1533
  > This is test case $f
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1534
  >   $ sleep 1
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1535
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1536
  > done
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1537
  $ rt -j 2
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1538
  ....
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1539
  # Ran 5 tests, 0 skipped, 0 failed.
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1540
  skipped: unknown feature: notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1541
  
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1542
  $ cd ..
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1543
  $ rm -rf broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1544
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1545
Test cases in .t files
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1546
======================
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1547
  $ mkdir cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1548
  $ cd cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1549
  $ cat > test-cases-abc.t <<'EOF'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1550
  > #testcases A B C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1551
  >   $ V=B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1552
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1553
  >   $ V=A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1554
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1555
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1556
  >   $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1557
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1558
  >   $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1559
  >   C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1560
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1561
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1562
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1563
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1564
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1565
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1566
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1567
  > #if no-C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1568
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1569
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1570
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1571
  >   $ [ $V = D ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1572
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1573
  > EOF
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1574
  $ rt
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1575
  .
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1576
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1577
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1578
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1579
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1580
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1581
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1582
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1583
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1584
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1585
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1586
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1587
  
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1588
  ERROR: test-cases-abc.t#B output changed
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1589
  !.
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1590
  Failed test-cases-abc.t#B: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1591
  # Ran 3 tests, 0 skipped, 1 failed.
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1592
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1593
  [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1594
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1595
--restart works
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1596
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1597
  $ rt --restart
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1598
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1599
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1600
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1601
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1602
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1603
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1604
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1605
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1606
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1607
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1608
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1609
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1610
  
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1611
  ERROR: test-cases-abc.t#B output changed
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1612
  !.
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1613
  Failed test-cases-abc.t#B: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1614
  # Ran 2 tests, 0 skipped, 1 failed.
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1615
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1616
  [1]
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1617
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1618
--restart works with outputdir
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1619
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1620
  $ mkdir output
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1621
  $ mv test-cases-abc.t#B.err output
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1622
  $ rt --restart --outputdir output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1623
  
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1624
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1625
  +++ $TESTTMP/anothertests/cases/output/test-cases-abc.t#B.err
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1626
  @@ -7,7 +7,7 @@
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1627
     $ V=C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1628
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1629
     $ echo $V | sed 's/A/C/'
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1630
  -  C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1631
  +  B
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1632
   #if C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1633
     $ [ $V = C ]
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1634
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1635
  
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1636
  ERROR: test-cases-abc.t#B output changed
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1637
  !.
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1638
  Failed test-cases-abc.t#B: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1639
  # Ran 2 tests, 0 skipped, 1 failed.
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1640
  python hash seed: * (glob)
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1641
  [1]
35069
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1642
35536
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1643
Test TESTCASE variable
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1644
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1645
  $ cat > test-cases-ab.t <<'EOF'
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1646
  >   $ dostuff() {
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1647
  >   >   echo "In case $TESTCASE"
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1648
  >   > }
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1649
  > #testcases A B
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1650
  > #if A
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1651
  >   $ dostuff
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1652
  >   In case A
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1653
  > #endif
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1654
  > #if B
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1655
  >   $ dostuff
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1656
  >   In case B
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1657
  > #endif
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1658
  > EOF
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1659
  $ rt test-cases-ab.t
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1660
  ..
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1661
  # Ran 2 tests, 0 skipped, 0 failed.
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1662
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1663
Support running a specific test case
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1664
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1665
  $ rt "test-cases-abc.t#B"
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1666
  
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1667
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1668
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1669
  @@ -7,7 +7,7 @@
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1670
     $ V=C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1671
   #endif
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1672
     $ echo $V | sed 's/A/C/'
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1673
  -  C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1674
  +  B
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1675
   #if C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1676
     $ [ $V = C ]
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1677
   #endif
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1678
  
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1679
  ERROR: test-cases-abc.t#B output changed
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1680
  !
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1681
  Failed test-cases-abc.t#B: output changed
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1682
  # Ran 1 tests, 0 skipped, 1 failed.
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1683
  python hash seed: * (glob)
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1684
  [1]
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1685
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1686
Support running multiple test cases in the same file
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1687
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1688
  $ rt test-cases-abc.t#B test-cases-abc.t#C
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1689
  
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1690
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1691
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1692
  @@ -7,7 +7,7 @@
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1693
     $ V=C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1694
   #endif
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1695
     $ echo $V | sed 's/A/C/'
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1696
  -  C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1697
  +  B
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1698
   #if C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1699
     $ [ $V = C ]
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1700
   #endif
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1701
  
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1702
  ERROR: test-cases-abc.t#B output changed
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1703
  !.
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1704
  Failed test-cases-abc.t#B: output changed
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1705
  # Ran 2 tests, 0 skipped, 1 failed.
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1706
  python hash seed: * (glob)
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1707
  [1]
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1708
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1709
Support ignoring invalid test cases
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1710
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1711
  $ rt test-cases-abc.t#B test-cases-abc.t#D
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1712
  
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1713
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1714
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1715
  @@ -7,7 +7,7 @@
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1716
     $ V=C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1717
   #endif
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1718
     $ echo $V | sed 's/A/C/'
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1719
  -  C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1720
  +  B
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1721
   #if C
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1722
     $ [ $V = C ]
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1723
   #endif
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1724
  
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1725
  ERROR: test-cases-abc.t#B output changed
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1726
  !
38223
b865bba56db1 run-tests: update the test case name format
Boris Feld <boris.feld@octobus.net>
parents: 38222
diff changeset
  1727
  Failed test-cases-abc.t#B: output changed
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1728
  # Ran 1 tests, 0 skipped, 1 failed.
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1729
  python hash seed: * (glob)
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1730
  [1]
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1731
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1732
Support running complex test cases names
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1733
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1734
  $ cat > test-cases-advanced-cases.t <<'EOF'
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1735
  > #testcases simple case-with-dashes casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1736
  >   $ echo $TESTCASE
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1737
  >   simple
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1738
  > EOF
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1739
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1740
  $ cat test-cases-advanced-cases.t
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1741
  #testcases simple case-with-dashes casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1742
    $ echo $TESTCASE
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1743
    simple
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1744
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1745
  $ rt test-cases-advanced-cases.t
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1746
  
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1747
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1748
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1749
  @@ -1,3 +1,3 @@
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1750
   #testcases simple case-with-dashes casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1751
     $ echo $TESTCASE
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1752
  -  simple
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1753
  +  case-with-dashes
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1754
  
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1755
  ERROR: test-cases-advanced-cases.t#case-with-dashes output changed
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1756
  !
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1757
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1758
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#casewith_-.chars.err
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1759
  @@ -1,3 +1,3 @@
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1760
   #testcases simple case-with-dashes casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1761
     $ echo $TESTCASE
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1762
  -  simple
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1763
  +  casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1764
  
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1765
  ERROR: test-cases-advanced-cases.t#casewith_-.chars output changed
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1766
  !.
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1767
  Failed test-cases-advanced-cases.t#case-with-dashes: output changed
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1768
  Failed test-cases-advanced-cases.t#casewith_-.chars: output changed
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1769
  # Ran 3 tests, 0 skipped, 2 failed.
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1770
  python hash seed: * (glob)
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1771
  [1]
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1772
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1773
  $ rt "test-cases-advanced-cases.t#case-with-dashes"
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1774
  
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1775
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1776
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1777
  @@ -1,3 +1,3 @@
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1778
   #testcases simple case-with-dashes casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1779
     $ echo $TESTCASE
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1780
  -  simple
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1781
  +  case-with-dashes
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1782
  
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1783
  ERROR: test-cases-advanced-cases.t#case-with-dashes output changed
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1784
  !
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1785
  Failed test-cases-advanced-cases.t#case-with-dashes: output changed
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1786
  # Ran 1 tests, 0 skipped, 1 failed.
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1787
  python hash seed: * (glob)
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1788
  [1]
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1789
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1790
  $ rt "test-cases-advanced-cases.t#casewith_-.chars"
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1791
  
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1792
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
38823
a9a4b0fc1e5d testrunner: use "#" for "test cases" suffix in .err filename too
Martin von Zweigbergk <martinvonz@google.com>
parents: 38621
diff changeset
  1793
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#casewith_-.chars.err
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1794
  @@ -1,3 +1,3 @@
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1795
   #testcases simple case-with-dashes casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1796
     $ echo $TESTCASE
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1797
  -  simple
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1798
  +  casewith_-.chars
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1799
  
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1800
  ERROR: test-cases-advanced-cases.t#casewith_-.chars output changed
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1801
  !
38290
b5651ae53127 run-tests: restrict the test cases allowed characters
Boris Feld <boris.feld@octobus.net>
parents: 38245
diff changeset
  1802
  Failed test-cases-advanced-cases.t#casewith_-.chars: output changed
38245
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1803
  # Ran 1 tests, 0 skipped, 1 failed.
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1804
  python hash seed: * (glob)
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1805
  [1]
f2e3196a34f9 run-tests: follow-up on the test-case format
Boris Feld <boris.feld@octobus.net>
parents: 38223
diff changeset
  1806
35069
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1807
Test automatic pattern replacement
38222
507bdc40bb17 run-tests: add support for running specific test cases
Boris Feld <boris.feld@octobus.net>
parents: 37846
diff changeset
  1808
==================================
35069
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1809
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1810
  $ cat << EOF >> common-pattern.py
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1811
  > substitutions = [
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1812
  >     (br'foo-(.*)\\b',
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1813
  >      br'\$XXX=\\1\$'),
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1814
  >     (br'bar\\n',
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1815
  >      br'\$YYY$\\n'),
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1816
  > ]
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1817
  > EOF
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1818
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1819
  $ cat << EOF >> test-substitution.t
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1820
  >   $ echo foo-12
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1821
  >   \$XXX=12$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1822
  >   $ echo foo-42
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1823
  >   \$XXX=42$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1824
  >   $ echo bar prior
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1825
  >   bar prior
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1826
  >   $ echo lastbar
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1827
  >   last\$YYY$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1828
  >   $ echo foo-bar foo-baz
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1829
  > EOF
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1830
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1831
  $ rt test-substitution.t
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1832
  
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1833
  --- $TESTTMP/anothertests/cases/test-substitution.t
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1834
  +++ $TESTTMP/anothertests/cases/test-substitution.t.err
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1835
  @@ -7,3 +7,4 @@
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1836
     $ echo lastbar
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1837
     last$YYY$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1838
     $ echo foo-bar foo-baz
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1839
  +  $XXX=bar foo-baz$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1840
  
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1841
  ERROR: test-substitution.t output changed
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1842
  !
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1843
  Failed test-substitution.t: output changed
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1844
  # Ran 1 tests, 0 skipped, 1 failed.
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1845
  python hash seed: * (glob)
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1846
  [1]
35189
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1847
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1848
--extra-config-opt works
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1849
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1850
  $ cat << EOF >> test-config-opt.t
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1851
  >   $ hg init test-config-opt
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1852
  >   $ hg -R test-config-opt purge
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1853
  > EOF
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1854
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1855
  $ rt --extra-config-opt extensions.purge= test-config-opt.t
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1856
  .
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1857
  # Ran 1 tests, 0 skipped, 0 failed.