tests/test-run-tests.t
author Jun Wu <quark@fb.com>
Wed, 21 Jun 2017 01:12:31 -0700
changeset 33000 573baab2a797
parent 32998 8dc62c97a665
child 33420 e80041832eec
permissions -rw-r--r--
run-tests: fix -i when "#testcases" is used in .t test The "#testcases" feature introduced by 7340465bd788 has issues with "-i" because "-i" uses "test.name.endswith('.t')" to test if a test is .t or not. test.name could now be something like "test-foo.t (caseA)" so the above endswith test is no longer valid. This patch changes the test to use "self.path" which won't have the issue.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
This file tests the behavior of run-tests.py itself.
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     3
Avoid interference from actual test env:
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     4
29220
054bcdad6042 tests: refactor run-tests helpers
timeless <timeless@mozdev.org>
parents: 29199
diff changeset
     5
  $ . "$TESTDIR/helper-runtests.sh"
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     6
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
     7
Smoke test with install
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
============
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     9
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    10
  $ run-tests.py $HGTEST_RUN_TESTS_PURE -l
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    11
  
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
    12
  # Ran 0 tests, 0 skipped, 0 failed.
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    13
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    14
Define a helper to avoid the install step
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    15
=============
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    16
  $ rt()
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    17
  > {
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    18
  >     run-tests.py --with-hg=`which hg` "$@"
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    19
  > }
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    20
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    21
error paths
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    22
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    23
#if symlink
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    24
  $ ln -s `which true` hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    25
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    26
  warning: --with-hg should specify an hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    27
  
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
    28
  # Ran 0 tests, 0 skipped, 0 failed.
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    29
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    30
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    31
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    32
#if execbit
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    33
  $ touch hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    34
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    35
  Usage: run-tests.py [options] [tests]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    36
  
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    37
  run-tests.py: error: --with-hg must specify an executable hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    38
  [2]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    39
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    40
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    41
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    42
Features for testing optional lines
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    43
===================================
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    44
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    45
  $ cat > hghaveaddon.py <<EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    46
  > import hghave
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    47
  > @hghave.check("custom", "custom hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    48
  > def has_custom():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    49
  >     return True
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    50
  > @hghave.check("missing", "missing hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    51
  > def has_missing():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    52
  >     return False
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    53
  > EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    54
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    55
an empty test
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    56
=======================
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    57
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    58
  $ touch test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    59
  $ rt
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    60
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
    61
  # Ran 1 tests, 0 skipped, 0 failed.
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    62
  $ rm test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    63
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    64
a succesful test
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    65
=======================
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    66
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    67
  $ cat > test-success.t << EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    68
  >   $ echo babar
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    69
  >   babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    70
  >   $ echo xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31678
diff changeset
    71
  >   dont_print (?)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31678
diff changeset
    72
  >   nothing[42]line (re) (?)
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28619
diff changeset
    73
  >   never*happens (glob) (?)
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31678
diff changeset
    74
  >   more_nothing (?)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    75
  >   xyzzy
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
    76
  >   nor this (?)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    77
  >   $ printf 'abc\ndef\nxyz\n'
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    78
  >   123 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    79
  >   abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    80
  >   def (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    81
  >   456 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    82
  >   xyz
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    83
  >   $ printf 'zyx\nwvu\ntsr\n'
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    84
  >   abc (?)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    85
  >   zyx (custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    86
  >   wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    87
  >   no_print (no-custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    88
  >   tsr (no-missing !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    89
  >   missing (missing !)
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    90
  > EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    91
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    92
  $ rt
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    93
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
    94
  # Ran 1 tests, 0 skipped, 0 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    95
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    96
failing test
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    97
==================
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    98
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    99
test churn with globs
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   100
  $ cat > test-failure.t <<EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   101
  >   $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   102
  >   bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   103
  >   bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   104
  > EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   105
  $ rt test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   106
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   107
  --- $TESTTMP/test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   108
  +++ $TESTTMP/test-failure.t.err
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   109
  @@ -1,3 +1,3 @@
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   110
     $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   111
  +  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   112
     bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   113
  -  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   114
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   115
  ERROR: test-failure.t output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   116
  !
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   117
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   118
  # Ran 1 tests, 0 skipped, 1 failed.
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   119
  python hash seed: * (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   120
  [1]
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   121
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   122
basic failing test
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   123
  $ cat > test-failure.t << EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   124
  >   $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   125
  >   rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   126
  > This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   127
  > this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   128
  > pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   129
  > pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   130
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   131
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   132
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   133
  > pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   134
  > EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   135
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   136
  >>> fh = open('test-failure-unicode.t', 'wb')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   137
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   138
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   139
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   140
  $ rt
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   141
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   142
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   143
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   144
  @@ -1,5 +1,5 @@
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   145
     $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   146
  -  rataxes
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   147
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   148
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   149
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   150
   pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   151
  
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   152
  ERROR: test-failure.t output changed
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   153
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   154
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   155
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   156
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   157
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   158
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   159
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   160
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   161
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   162
  !
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   163
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   164
  Failed test-failure-unicode.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   165
  # Ran 3 tests, 0 skipped, 2 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   166
  python hash seed: * (glob)
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   167
  [1]
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   168
32736
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   169
test --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   170
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   171
  $ rt --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   172
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   173
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   174
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   175
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   176
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   177
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   178
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   179
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   180
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   181
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   182
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   183
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   184
  !.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   185
  --- $TESTTMP/test-failure-unicode.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   186
  +++ $TESTTMP/output/test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   187
  @@ -1,2 +1,2 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   188
     $ echo babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   189
  -  l\xce\xb5\xce\xb5t (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   190
  +  babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   191
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   192
  ERROR: test-failure-unicode.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   193
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   194
  Failed test-failure.t: output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   195
  Failed test-failure-unicode.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   196
  # Ran 3 tests, 0 skipped, 2 failed.
32736
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   197
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   198
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   199
  $ ls -a output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   200
  .
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   201
  ..
32737
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32736
diff changeset
   202
  .testtimes
32736
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   203
  test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   204
  test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   205
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   206
test --xunit support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   207
  $ rt --xunit=xunit.xml
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   208
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   209
  --- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   210
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   211
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   212
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   213
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   214
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   215
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   216
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   217
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   218
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   219
  ERROR: test-failure.t output changed
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   220
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   221
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   222
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   223
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   224
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   225
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   226
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   227
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   228
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   229
  !
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   230
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   231
  Failed test-failure-unicode.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   232
  # Ran 3 tests, 0 skipped, 2 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   233
  python hash seed: * (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   234
  [1]
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   235
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   236
  <?xml version="1.0" encoding="utf-8"?>
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   237
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   238
    <testcase name="test-success.t" time="*"/> (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   239
    <testcase name="test-failure-unicode.t" time="*"> (glob)
32734
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32724
diff changeset
   240
      <failure message="output changed" type="output-mismatch">
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   241
  <![CDATA[--- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   242
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   243
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   244
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   245
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   246
  +  babar\xce\xb1 (esc)
32734
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32724
diff changeset
   247
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32724
diff changeset
   248
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   249
    <testcase name="test-failure.t" time="*"> (glob)
32734
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32724
diff changeset
   250
      <failure message="output changed" type="output-mismatch">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   251
  <![CDATA[--- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   252
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   253
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   254
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   255
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   256
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   257
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   258
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   259
   pad pad pad pad............................................................
32734
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32724
diff changeset
   260
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32724
diff changeset
   261
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   262
  </testsuite>
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   263
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   264
  $ cat .testtimes
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   265
  test-failure-unicode.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   266
  test-failure.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   267
  test-success.t * (glob)
32724
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   268
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   269
  $ rt --list-tests
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   270
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   271
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   272
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   273
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   274
  $ rt --list-tests --json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   275
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   276
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   277
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   278
  $ cat report.json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   279
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   280
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   281
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   282
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   283
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   284
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   285
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   286
      "test-success.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   287
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   288
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   289
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   290
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   291
  $ rt --list-tests --xunit=xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   292
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   293
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   294
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   295
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   296
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   297
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   298
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   299
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   300
    <testcase name="test-success.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   301
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   302
32738
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   303
  $ rt --list-tests test-failure* --json --xunit=xunit.xml --outputdir output
32724
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   304
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   305
  test-failure.t
32738
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   306
  $ cat output/report.json
32724
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   307
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   308
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   309
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   310
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   311
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   312
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   313
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   314
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   315
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   316
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   317
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   318
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   319
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   320
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32645
diff changeset
   321
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   322
  $ rm test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   323
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   324
test for --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   325
====================
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   326
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   327
  $ rt --retest
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   328
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   329
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   330
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   331
  @@ -1,5 +1,5 @@
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   332
     $ echo babar
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   333
  -  rataxes
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   334
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   335
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   336
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   337
   pad pad pad pad............................................................
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   338
  
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   339
  ERROR: test-failure.t output changed
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   340
  !
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   341
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   342
  # Ran 2 tests, 1 skipped, 1 failed.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   343
  python hash seed: * (glob)
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   344
  [1]
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   345
32736
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   346
--retest works with --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   347
  $ rm -r output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   348
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   349
  $ mv test-failure.t.err output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   350
  $ rt --retest --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   351
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   352
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   353
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   354
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   355
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   356
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   357
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   358
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   359
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   360
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   361
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   362
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   363
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   364
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   365
  # Ran 2 tests, 1 skipped, 1 failed.
32736
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   366
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   367
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32735
diff changeset
   368
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   369
Selecting Tests To Run
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   370
======================
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   371
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   372
successful
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   373
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   374
  $ rt test-success.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   375
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   376
  # Ran 1 tests, 0 skipped, 0 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   377
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   378
success w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   379
  $ rt -k xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   380
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   381
  # Ran 2 tests, 1 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   382
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   383
failed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   384
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   385
  $ rt test-failure.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   386
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   387
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   388
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   389
  @@ -1,5 +1,5 @@
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   390
     $ echo babar
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   391
  -  rataxes
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   392
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   393
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   394
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   395
   pad pad pad pad............................................................
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   396
  
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   397
  ERROR: test-failure.t output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   398
  !
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   399
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   400
  # Ran 1 tests, 0 skipped, 1 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   401
  python hash seed: * (glob)
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   402
  [1]
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   403
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   404
failure w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   405
  $ rt -k rataxes
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   406
  
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   407
  --- $TESTTMP/test-failure.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   408
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   409
  @@ -1,5 +1,5 @@
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   410
     $ echo babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   411
  -  rataxes
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   412
  +  babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   413
   This is a noop statement so that
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   414
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   415
   pad pad pad pad............................................................
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   416
  
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   417
  ERROR: test-failure.t output changed
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   418
  !
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   419
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   420
  # Ran 2 tests, 1 skipped, 1 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   421
  python hash seed: * (glob)
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   422
  [1]
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   423
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   424
Verify that when a process fails to start we show a useful message
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   425
==================================================================
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   426
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   427
  $ cat > test-serve-fail.t <<EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   428
  >   $ echo 'abort: child process failed to start blah'
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   429
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   430
  $ rt test-serve-fail.t
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   431
  
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   432
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   433
  !
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   434
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   435
  # Ran 1 tests, 0 skipped, 1 failed.
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   436
  python hash seed: * (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   437
  [1]
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   438
  $ rm test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   439
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   440
Verify that we can try other ports
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   441
===================================
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   442
  $ hg init inuse
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   443
  $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   444
  $ cat blocks.pid >> $DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   445
  $ cat > test-serve-inuse.t <<EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   446
  >   $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   447
  >   $ cat hg.pid >> \$DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   448
  > EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   449
  $ rt test-serve-inuse.t
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   450
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   451
  # Ran 1 tests, 0 skipped, 0 failed.
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   452
  $ rm test-serve-inuse.t
29221
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   453
  $ killdaemons.py $DAEMON_PIDS
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   454
  $ rm $DAEMON_PIDS
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   455
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   456
Running In Debug Mode
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   457
======================
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   458
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   459
  $ rt --debug 2>&1 | grep -v pwd
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   460
  + echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   461
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   462
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   463
  babar
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   464
  + echo *SALT* 10 0 (glob)
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   465
  *SALT* 10 0 (glob)
27509
6ff90d618d72 test-run-tests: glob away a --debug run difference on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 27395
diff changeset
   466
  *+ echo *SALT* 0 0 (glob)
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   467
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   468
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   469
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   470
  + echo *SALT* 2 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   471
  *SALT* 2 0 (glob)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   472
  + echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   473
  xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31678
diff changeset
   474
  + echo *SALT* 9 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31678
diff changeset
   475
  *SALT* 9 0 (glob)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   476
  + printf *abc\ndef\nxyz\n* (glob)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   477
  abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   478
  def
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   479
  xyz
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31678
diff changeset
   480
  + echo *SALT* 15 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31678
diff changeset
   481
  *SALT* 15 0 (glob)
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   482
  + printf *zyx\nwvu\ntsr\n* (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   483
  zyx
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   484
  wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   485
  tsr
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   486
  + echo *SALT* 22 0 (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   487
  *SALT* 22 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   488
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   489
  # Ran 2 tests, 0 skipped, 0 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   490
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   491
Parallel runs
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   492
==============
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   493
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   494
(duplicate the failing test to get predictable output)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   495
  $ cp test-failure.t test-failure-copy.t
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   496
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   497
  $ rt --jobs 2 test-failure*.t -n
23107
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   498
  !!
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   499
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   500
  Failed test-failure*.t: output changed (glob)
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   501
  # Ran 2 tests, 0 skipped, 2 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   502
  python hash seed: * (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   503
  [1]
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   504
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   505
failures in parallel with --first should only print one failure
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   506
  >>> f = open('test-nothing.t', 'w')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   507
  >>> f.write('foo\n' * 1024) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   508
  >>> f.write('  $ sleep 1') and None
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   509
  $ rt --jobs 2 --first
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   510
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   511
  --- $TESTTMP/test-failure*.t (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   512
  +++ $TESTTMP/test-failure*.t.err (glob)
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   513
  @@ -1,5 +1,5 @@
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   514
     $ echo babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   515
  -  rataxes
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   516
  +  babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   517
   This is a noop statement so that
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   518
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   519
   pad pad pad pad............................................................
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   520
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   521
  Failed test-failure*.t: output changed (glob)
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   522
  Failed test-nothing.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   523
  # Ran 2 tests, 0 skipped, 2 failed.
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   524
  python hash seed: * (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   525
  [1]
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   526
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   527
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   528
(delete the duplicated test file)
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   529
  $ rm test-failure-copy.t test-nothing.t
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   530
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   531
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   532
Interactive run
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   533
===============
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   534
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   535
(backup the failing test)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   536
  $ cp test-failure.t backup
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   537
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   538
Refuse the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   539
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   540
  $ echo 'n' | rt -i
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   541
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   542
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   543
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   544
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   545
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   546
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   547
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   548
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   549
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   550
   pad pad pad pad............................................................
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   551
  Accept this change? [n] 
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   552
  ERROR: test-failure.t output changed
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   553
  !.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   554
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   555
  # Ran 2 tests, 0 skipped, 1 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   556
  python hash seed: * (glob)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   557
  [1]
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   558
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   559
  $ cat test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   560
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   561
    rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   562
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   563
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   564
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   565
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   566
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   567
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   568
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   569
  pad pad pad pad............................................................
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   570
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   571
Interactive with custom view
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   572
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   573
  $ echo 'n' | rt -i --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   574
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   575
  Accept this change? [n]* (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   576
  ERROR: test-failure.t output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   577
  !.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   578
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   579
  # Ran 2 tests, 0 skipped, 1 failed.
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   580
  python hash seed: * (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   581
  [1]
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   582
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   583
View the fix
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   584
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   585
  $ echo 'y' | rt --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   586
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   587
  
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   588
  ERROR: test-failure.t output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   589
  !.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   590
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   591
  # Ran 2 tests, 0 skipped, 1 failed.
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   592
  python hash seed: * (glob)
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   593
  [1]
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   594
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   595
Accept the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   596
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   597
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   598
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   599
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   600
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg (glob)" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   601
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   602
  $ echo "  saved backup bundle to \$TESTTMP/*.hg (glob)" >> test-failure.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   603
  $ echo 'y' | rt -i 2>&1
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   604
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   605
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   606
  +++ $TESTTMP/test-failure.t.err
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   607
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   608
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   609
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   610
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   611
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   612
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   613
   pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   614
  @@ -9,7 +9,7 @@
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   615
   pad pad pad pad............................................................
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   616
   pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   617
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   618
  -  saved backup bundle to $TESTTMP/foo.hg
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   619
  +  saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   620
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   621
     saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   622
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   623
  Accept this change? [n] ..
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   624
  # Ran 2 tests, 0 skipped, 0 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   625
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   626
  $ sed -e 's,(glob)$,&<,g' test-failure.t
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   627
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   628
    babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   629
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   630
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   631
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   632
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   633
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   634
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   635
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   636
  pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   637
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   638
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   639
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   640
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   641
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   642
    saved backup bundle to $TESTTMP/*.hg (glob)<
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   643
32998
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   644
Race condition - test file was modified when test is running
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   645
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   646
  $ TESTRACEDIR=`pwd`
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   647
  $ export TESTRACEDIR
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   648
  $ cat > test-race.t <<EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   649
  >   $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   650
  >   $ echo "# a new line" >> $TESTRACEDIR/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   651
  > EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   652
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   653
  $ rt -i test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   654
  
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   655
  --- $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   656
  +++ $TESTTMP/test-race.t.err
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   657
  @@ -1,2 +1,3 @@
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   658
     $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   659
  +  1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   660
     $ echo "# a new line" >> $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   661
  Reference output has changed (run again to prompt changes)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   662
  ERROR: test-race.t output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   663
  !
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   664
  Failed test-race.t: output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   665
  # Ran 1 tests, 0 skipped, 1 failed.
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   666
  python hash seed: * (glob)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   667
  [1]
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   668
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   669
  $ rm test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32960
diff changeset
   670
33000
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   671
When "#testcases" is used in .t files
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   672
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   673
  $ cat >> test-cases.t <<EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   674
  > #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   675
  > #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   676
  >   $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   677
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   678
  > #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   679
  >   $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   680
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   681
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   682
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   683
  $ cat <<EOF | rt -i test-cases.t 2>&1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   684
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   685
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   686
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   687
  
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   688
  --- $TESTTMP/test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   689
  +++ $TESTTMP/test-cases.t.a.err
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   690
  @@ -1,6 +1,7 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   691
   #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   692
   #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   693
     $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   694
  +  1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   695
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   696
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   697
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   698
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   699
  --- $TESTTMP/test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   700
  +++ $TESTTMP/test-cases.t.b.err
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   701
  @@ -5,4 +5,5 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   702
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   703
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   704
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   705
  +  2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   706
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   707
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   708
  # Ran 2 tests, 0 skipped, 0 failed.
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   709
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   710
  $ cat test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   711
  #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   712
  #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   713
    $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   714
    1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   715
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   716
  #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   717
    $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   718
    2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   719
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   720
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   721
  $ rm test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32998
diff changeset
   722
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   723
(reinstall)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   724
  $ mv backup test-failure.t
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   725
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   726
No Diff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   727
===============
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   728
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   729
  $ rt --nodiff
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   730
  !.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   731
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   732
  # Ran 2 tests, 0 skipped, 1 failed.
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   733
  python hash seed: * (glob)
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   734
  [1]
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   735
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   736
test --tmpdir support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   737
  $ rt --tmpdir=$TESTTMP/keep test-success.t
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   738
  
26432
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   739
  Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t (glob)
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   740
  Keeping threadtmp dir: $TESTTMP/keep/child1  (glob)
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   741
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   742
  # Ran 1 tests, 0 skipped, 0 failed.
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   743
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   744
timeouts
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   745
========
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   746
  $ cat > test-timeout.t <<EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   747
  >   $ sleep 2
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   748
  >   $ echo pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   749
  >   pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   750
  > EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   751
  > echo '#require slow' > test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   752
  > cat test-timeout.t >> test-slow-timeout.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   753
  $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   754
  st
32507
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32356
diff changeset
   755
  Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests)
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   756
  Failed test-timeout.t: timed out
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   757
  # Ran 1 tests, 1 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   758
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   759
  [1]
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   760
  $ rt --timeout=1 --slowtimeout=3 \
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   761
  > test-timeout.t test-slow-timeout.t --allow-slow-tests
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   762
  .t
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   763
  Failed test-timeout.t: timed out
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   764
  # Ran 2 tests, 0 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   765
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   766
  [1]
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   767
  $ rm test-timeout.t test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   768
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   769
test for --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   770
==================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   771
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   772
  $ rt test-success.t --time
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   773
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   774
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   775
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   776
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   777
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   778
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   779
test for --time with --job enabled
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   780
====================================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   781
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   782
  $ rt test-success.t --time --jobs 2
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   783
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   784
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   785
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   786
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   787
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   788
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   789
Skips
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   790
================
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   791
  $ cat > test-skip.t <<EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   792
  >   $ echo xyzzy
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
   793
  > #require false
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   794
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   795
  $ rt --nodiff
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   796
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   797
  Skipped test-skip.t: missing feature: nail clipper
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   798
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   799
  # Ran 2 tests, 1 skipped, 1 failed.
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   800
  python hash seed: * (glob)
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   801
  [1]
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   802
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   803
  $ rt --keyword xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   804
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   805
  Skipped test-skip.t: missing feature: nail clipper
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   806
  # Ran 2 tests, 2 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   807
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   808
Skips with xml
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   809
  $ rt --keyword xyzzy \
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   810
  >  --xunit=xunit.xml
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   811
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   812
  Skipped test-skip.t: missing feature: nail clipper
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   813
  # Ran 2 tests, 2 skipped, 0 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   814
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   815
  <?xml version="1.0" encoding="utf-8"?>
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   816
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   817
    <testcase name="test-success.t" time="*"/> (glob)
32735
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32734
diff changeset
   818
    <testcase name="test-skip.t">
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32734
diff changeset
   819
      <skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32734
diff changeset
   820
  <![CDATA[missing feature: nail clipper]]>    </skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32734
diff changeset
   821
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   822
  </testsuite>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   823
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   824
Missing skips or blacklisted skips don't count as executed:
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   825
  $ echo test-failure.t > blacklist
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   826
  $ rt --blacklist=blacklist --json\
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   827
  >   test-failure.t test-bogus.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   828
  ss
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   829
  Skipped test-bogus.t: Doesn't exist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   830
  Skipped test-failure.t: blacklisted
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   831
  # Ran 0 tests, 2 skipped, 0 failed.
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   832
  $ cat report.json
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   833
  testreport ={
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   834
      "test-bogus.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   835
          "result": "skip"
29199
daff05dcd184 run-tests: handle json.dumps divergence
timeless <timeless@mozdev.org>
parents: 29173
diff changeset
   836
      },
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   837
      "test-failure.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   838
          "result": "skip"
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   839
      }
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   840
  } (no-eol)
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   841
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   842
Whitelist trumps blacklist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   843
  $ echo test-failure.t > whitelist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   844
  $ rt --blacklist=blacklist --whitelist=whitelist --json\
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   845
  >   test-failure.t test-bogus.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   846
  s
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   847
  --- $TESTTMP/test-failure.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   848
  +++ $TESTTMP/test-failure.t.err
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   849
  @@ -1,5 +1,5 @@
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   850
     $ echo babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   851
  -  rataxes
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   852
  +  babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   853
   This is a noop statement so that
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   854
   this test is still more bytes than success.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   855
   pad pad pad pad............................................................
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   856
  
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   857
  ERROR: test-failure.t output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   858
  !
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   859
  Skipped test-bogus.t: Doesn't exist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   860
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   861
  # Ran 1 tests, 1 skipped, 1 failed.
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   862
  python hash seed: * (glob)
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   863
  [1]
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   864
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   865
test for --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   866
==================
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   867
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   868
  $ rt --json
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   869
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   870
  --- $TESTTMP/test-failure.t
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   871
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   872
  @@ -1,5 +1,5 @@
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   873
     $ echo babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   874
  -  rataxes
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   875
  +  babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   876
   This is a noop statement so that
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   877
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   878
   pad pad pad pad............................................................
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   879
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   880
  ERROR: test-failure.t output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   881
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   882
  Skipped test-skip.t: missing feature: nail clipper
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   883
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   884
  # Ran 2 tests, 1 skipped, 1 failed.
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   885
  python hash seed: * (glob)
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   886
  [1]
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   887
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   888
  $ cat report.json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   889
  testreport ={
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   890
      "test-failure.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   891
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   892
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   893
          "diff": "---.+\+\+\+.+", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   894
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   895
          "result": "failure", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   896
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   897
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   898
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   899
      "test-skip.t": {
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   900
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   901
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   902
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   903
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   904
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   905
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   906
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   907
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   908
      "test-success.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   909
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   910
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   911
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   912
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   913
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   914
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   915
          "time": "\s*[\d\.]{4,5}" (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   916
      }
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   917
  } (no-eol)
32738
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   918
--json with --outputdir
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   919
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   920
  $ rm report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   921
  $ rm -r output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   922
  $ mkdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   923
  $ rt --json --outputdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   924
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   925
  --- $TESTTMP/test-failure.t
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   926
  +++ $TESTTMP/output/test-failure.t.err
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   927
  @@ -1,5 +1,5 @@
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   928
     $ echo babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   929
  -  rataxes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   930
  +  babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   931
   This is a noop statement so that
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   932
   this test is still more bytes than success.
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   933
   pad pad pad pad............................................................
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   934
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   935
  ERROR: test-failure.t output changed
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   936
  !.s
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   937
  Skipped test-skip.t: missing feature: nail clipper
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   938
  Failed test-failure.t: output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   939
  # Ran 2 tests, 1 skipped, 1 failed.
32738
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   940
  python hash seed: * (glob)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   941
  [1]
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   942
  $ f report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   943
  report.json: file not found
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   944
  $ cat output/report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   945
  testreport ={
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   946
      "test-failure.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   947
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   948
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   949
          "diff": "---.+\+\+\+.+", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   950
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   951
          "result": "failure", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   952
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   953
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   954
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   955
      "test-skip.t": {
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   956
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   957
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   958
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   959
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   960
          "result": "skip", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   961
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   962
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   963
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   964
      "test-success.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   965
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   966
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   967
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   968
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   969
          "result": "success", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   970
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   971
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   972
      }
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   973
  } (no-eol)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   974
  $ ls -a output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   975
  .
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   976
  ..
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   977
  .testtimes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   978
  report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32737
diff changeset
   979
  test-failure.t.err
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
   980
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   981
Test that failed test accepted through interactive are properly reported:
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   982
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   983
  $ cp test-failure.t backup
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   984
  $ echo y | rt --json -i
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   985
  
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   986
  --- $TESTTMP/test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   987
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   988
  @@ -1,5 +1,5 @@
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   989
     $ echo babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   990
  -  rataxes
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   991
  +  babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   992
   This is a noop statement so that
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   993
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   994
   pad pad pad pad............................................................
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   995
  Accept this change? [n] ..s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   996
  Skipped test-skip.t: missing feature: nail clipper
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
   997
  # Ran 2 tests, 1 skipped, 0 failed.
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   998
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   999
  $ cat report.json
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1000
  testreport ={
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1001
      "test-failure.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1002
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1003
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1004
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1005
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1006
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1007
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1008
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1009
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1010
      "test-skip.t": {
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1011
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1012
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1013
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1014
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1015
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1016
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1017
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1018
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1019
      "test-success.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1020
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1021
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1022
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1023
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1024
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1025
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1026
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1027
      }
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1028
  } (no-eol)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1029
  $ mv backup test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1030
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1031
backslash on end of line with glob matching is handled properly
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1032
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1033
  $ cat > test-glob-backslash.t << EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1034
  >   $ echo 'foo bar \\'
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1035
  >   foo * \ (glob)
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1036
  > EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1037
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1038
  $ rt test-glob-backslash.t
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1039
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1040
  # Ran 1 tests, 0 skipped, 0 failed.
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1041
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1042
  $ rm -f test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1043
31678
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1044
Test globbing of local IP addresses
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1045
  $ echo 172.16.18.1
31678
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1046
  $LOCALIP (glob)
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1047
  $ echo dead:beef::1
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1048
  $LOCALIP (glob)
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1049
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1050
Test reusability for third party tools
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1051
======================================
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1052
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1053
  $ mkdir "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1054
  $ cd "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1055
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1056
test that `run-tests.py` can execute hghave, even if it runs not in
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1057
Mercurial source tree.
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1058
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1059
  $ cat > test-hghave.t <<EOF
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1060
  > #require true
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1061
  >   $ echo foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1062
  >   foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1063
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1064
  $ rt test-hghave.t
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1065
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1066
  # Ran 1 tests, 0 skipped, 0 failed.
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1067
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1068
test that RUNTESTDIR refers the directory, in which `run-tests.py` now
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1069
running is placed.
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1070
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1071
  $ cat > test-runtestdir.t <<EOF
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1072
  > - $TESTDIR, in which test-run-tests.t is placed
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1073
  > - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1074
  > - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1075
  > 
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1076
  > #if windows
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1077
  >   $ test "\$TESTDIR" = "$TESTTMP\anothertests"
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1078
  > #else
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1079
  >   $ test "\$TESTDIR" = "$TESTTMP"/anothertests
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1080
  > #endif
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1081
  >   $ test "\$RUNTESTDIR" = "$TESTDIR"
32955
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32740
diff changeset
  1082
  >   $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py | sed 's@.!.*python@#!USRBINENVPY@'
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32740
diff changeset
  1083
  >   #!USRBINENVPY
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1084
  >   #
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1085
  >   # check-code - a style and portability checker for Mercurial
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1086
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1087
  $ rt test-runtestdir.t
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1088
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1089
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1090
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1091
#if execbit
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1092
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1093
test that TESTDIR is referred in PATH
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1094
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1095
  $ cat > custom-command.sh <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1096
  > #!/bin/sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1097
  > echo "hello world"
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1098
  > EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1099
  $ chmod +x custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1100
  $ cat > test-testdir-path.t <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1101
  >   $ custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1102
  >   hello world
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1103
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1104
  $ rt test-testdir-path.t
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1105
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1106
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1107
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1108
#endif
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1110
test support for --allow-slow-tests
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1111
  $ cat > test-very-slow-test.t <<EOF
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1112
  > #require slow
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1113
  >   $ echo pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1114
  >   pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1115
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1116
  $ rt test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1117
  s
32507
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32356
diff changeset
  1118
  Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests)
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1119
  # Ran 0 tests, 1 skipped, 0 failed.
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1120
  $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1121
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1122
  # Ran 1 tests, 0 skipped, 0 failed.
28180
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1123
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1124
support for running a test outside the current directory
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1125
  $ mkdir nonlocal
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1126
  $ cat > nonlocal/test-is-not-here.t << EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1127
  >   $ echo pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1128
  >   pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1129
  > EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1130
  $ rt nonlocal/test-is-not-here.t
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1131
  .
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1132
  # Ran 1 tests, 0 skipped, 0 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1133
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1134
support for bisecting failed tests automatically
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1135
  $ hg init bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1136
  $ cd bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1137
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1138
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1139
  >   pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1140
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1141
  $ hg add test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1142
  $ hg ci -m 'good'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1143
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1144
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1145
  >   fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1146
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1147
  $ hg ci -m 'bad'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1148
  $ rt --known-good-rev=0 test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1149
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1150
  --- $TESTTMP/anothertests/bisect/test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1151
  +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1152
  @@ -1,4 +1,4 @@
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1153
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1154
     pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1155
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1156
  -  fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1157
  +  pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1158
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1159
  ERROR: test-bisect.t output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1160
  !
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1161
  Failed test-bisect.t: output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1162
  test-bisect.t broken by 72cbf122d116 (bad)
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1163
  # Ran 1 tests, 0 skipped, 1 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1164
  python hash seed: * (glob)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1165
  [1]
32356
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1166
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1167
  $ cd ..
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1168
32645
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1169
Test a broken #if statement doesn't break run-tests threading.
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1170
==============================================================
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1171
  $ mkdir broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1172
  $ cd broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1173
  $ cat > test-broken.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1174
  > true
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1175
  > #if notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1176
  >   $ false
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1177
  > #endif
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1178
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1179
  $ for f in 1 2 3 4 ; do
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1180
  > cat > test-works-$f.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1181
  > This is test case $f
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1182
  >   $ sleep 1
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1183
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1184
  > done
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1185
  $ rt -j 2
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1186
  ....
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1187
  # Ran 5 tests, 0 skipped, 0 failed.
32645
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1188
  skipped: unknown feature: notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1189
  
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1190
  $ cd ..
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1191
  $ rm -rf broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32507
diff changeset
  1192
32356
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1193
Test cases in .t files
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1194
======================
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1195
  $ mkdir cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1196
  $ cd cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1197
  $ cat > test-cases-abc.t <<'EOF'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1198
  > #testcases A B C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1199
  >   $ V=B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1200
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1201
  >   $ V=A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1202
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1203
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1204
  >   $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1205
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1206
  >   $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1207
  >   C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1208
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1209
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1210
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1211
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1212
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1213
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1214
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1215
  > #if no-C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1216
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1217
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1218
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1219
  >   $ [ $V = D ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1220
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1221
  > EOF
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1222
  $ rt
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1223
  .
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1224
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1225
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1226
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1227
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1228
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1229
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1230
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1231
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1232
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1233
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1234
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1235
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1236
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1237
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1238
  Failed test-cases-abc.t (case B): output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1239
  # Ran 3 tests, 0 skipped, 1 failed.
32356
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1240
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1241
  [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1242
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1243
--restart works
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1244
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1245
  $ rt --restart
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1246
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1247
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1248
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1249
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1250
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1251
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1252
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1253
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1254
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1255
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1256
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1257
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1258
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1259
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1260
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1261
  Failed test-cases-abc.t (case B): output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1262
  # Ran 2 tests, 0 skipped, 1 failed.
32356
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1263
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1264
  [1]
32740
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1265
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1266
--restart works with outputdir
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1267
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1268
  $ mkdir output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1269
  $ mv test-cases-abc.t.B.err output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1270
  $ rt --restart --outputdir output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1271
  
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1272
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1273
  +++ $TESTTMP/anothertests/cases/output/test-cases-abc.t.B.err
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1274
  @@ -7,7 +7,7 @@
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1275
     $ V=C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1276
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1277
     $ echo $V | sed 's/A/C/'
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1278
  -  C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1279
  +  B
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1280
   #if C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1281
     $ [ $V = C ]
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1282
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1283
  
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1284
  ERROR: test-cases-abc.t (case B) output changed
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1285
  !.
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1286
  Failed test-cases-abc.t (case B): output changed
32960
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32955
diff changeset
  1287
  # Ran 2 tests, 0 skipped, 1 failed.
32740
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1288
  python hash seed: * (glob)
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32738
diff changeset
  1289
  [1]