view tests/test-run-tests.t @ 46582:b0a3ca02d17a

copies-rust: implement PartialEqual manually

Now that we know that each (dest, rev) pair has at most a unique CopySource, we
can simplify comparison a lot.

This "simple" step buy a good share of the previous slowdown back in some case:

Repo            Case                                  Source-Rev   Dest-Rev      # of revisions  old time    new time      Difference    Factor    time per rev
---------------------------------------------------------------------------------------------------------------------------------------------------------------
mozilla-try     x00000_revs_x00000_added_x000_copies  9b2a99adc05e 8e29777b48e6 : 382065 revs,  43.304637 s,  34.443661 s,  -8.860976 s, × 0.7954,    90 µs/rev


Full benchmark:

Repo            Case                                  Source-Rev   Dest-Rev      # of revisions  old time    new time      Difference    Factor    time per rev
---------------------------------------------------------------------------------------------------------------------------------------------------------------
mercurial       x_revs_x_added_0_copies               ad6b123de1c7 39cfcef4f463 :      1 revs,   0.000043 s,   0.000043 s,  +0.000000 s, × 1.0000,    43 µs/rev
mercurial       x_revs_x_added_x_copies               2b1c78674230 0c1d10351869 :      6 revs,   0.000114 s,   0.000117 s,  +0.000003 s, × 1.0263,    19 µs/rev
mercurial       x000_revs_x000_added_x_copies         81f8ff2a9bf2 dd3267698d84 :   1032 revs,   0.004937 s,   0.004892 s,  -0.000045 s, × 0.9909,     4 µs/rev
pypy            x_revs_x_added_0_copies               aed021ee8ae8 099ed31b181b :      9 revs,   0.000339 s,   0.000196 s,  -0.000143 s, × 0.5782,    21 µs/rev
pypy            x_revs_x000_added_0_copies            4aa4e1f8e19a 359343b9ac0e :      1 revs,   0.000049 s,   0.000050 s,  +0.000001 s, × 1.0204,    50 µs/rev
pypy            x_revs_x_added_x_copies               ac52eb7bbbb0 72e022663155 :      7 revs,   0.000202 s,   0.000117 s,  -0.000085 s, × 0.5792,    16 µs/rev
pypy            x_revs_x00_added_x_copies             c3b14617fbd7 ace7255d9a26 :      1 revs,   0.000409 s,   0.6f1f4a s,  -0.000087 s, × 0.7873,   322 µs/rev
pypy            x_revs_x000_added_x000_copies         df6f7a526b60 a83dc6a2d56f :      6 revs,   0.011984 s,   0.011949 s,  -0.000035 s, × 0.9971,  1991 µs/rev
pypy            x000_revs_xx00_added_0_copies         89a76aede314 2f22446ff07e :   4785 revs,   0.050820 s,   0.050802 s,  -0.000018 s, × 0.9996,    10 µs/rev
pypy            x000_revs_x000_added_x_copies         8a3b5bfd266e 2c68e87c3efe :   6780 revs,   0.087953 s,   0.088090 s,  +0.000137 s, × 1.0016,    12 µs/rev
pypy            x000_revs_x000_added_x000_copies      89a76aede314 7b3dda341c84 :   5441 revs,   0.062902 s,   0.062079 s,  -0.000823 s, × 0.9869,    11 µs/rev
pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :  43645 revs,   0.679234 s,   0.635337 s,  -0.043897 s, × 0.9354,    14 µs/rev
pypy            x0000_revs_xx000_added_0_copies       bf2c629d0071 4ffed77c095c :      2 revs,   0.013095 s,   0.013262 s,  +0.000167 s, × 1.0128,  6631 µs/rev
pypy            x0000_revs_xx000_added_x000_copies    08ea3258278e d9fa043f30c0 :  11316 revs,   0.120910 s,   0.120085 s,  -0.000825 s, × 0.9932,    10 µs/rev
netbeans        x_revs_x_added_0_copies               fb0955ffcbcd a01e9239f9e7 :      2 revs,   0.000087 s,   0.000085 s,  -0.000002 s, × 0.9770,    42 µs/rev
netbeans        x_revs_x000_added_0_copies            6f360122949f 20eb231cc7d0 :      2 revs,   0.000107 s,   0.000110 s,  +0.000003 s, × 1.0280,    55 µs/rev
netbeans        x_revs_x_added_x_copies               1ada3faf6fb6 5a39d12eecf4 :      3 revs,   0.000186 s,   0.000177 s,  -0.000009 s, × 0.9516,    59 µs/rev
netbeans        x_revs_x00_added_x_copies             35be93ba1e2c 9eec5e90c05f :      9 revs,   0.000754 s,   0.000743 s,  -0.000011 s, × 0.9854,    82 µs/rev
netbeans        x000_revs_xx00_added_0_copies         eac3045b4fdd 51d4ae7f1290 :   1421 revs,   0.010443 s,   0.010168 s,  -0.000275 s, × 0.9737,     7 µs/rev
netbeans        x000_revs_x000_added_x_copies         e2063d266acd 6081d72689dc :   1533 revs,   0.015697 s,   0.015946 s,  +0.000249 s, × 1.0159,    10 µs/rev
netbeans        x000_revs_x000_added_x000_copies      ff453e9fee32 411350406ec2 :   5750 revs,   0.063528 s,   0.062712 s,  -0.000816 s, × 0.9872,    10 µs/rev
netbeans        x0000_revs_xx000_added_x000_copies    588c2d1ced70 1aad62e59ddd :  66949 revs,   0.545515 s,   0.523832 s,  -0.021683 s, × 0.9603,     7 µs/rev
mozilla-central x_revs_x_added_0_copies               3697f962bb7b 7015fcdd43a2 :      2 revs,   0.000089 s,   0.000090 s,  +0.000001 s, × 1.0112,    45 µs/rev
mozilla-central x_revs_x000_added_0_copies            dd390860c6c9 40d0c5bed75d :      8 revs,   0.000265 s,   0.000264 s,  -0.000001 s, × 0.9962,    33 µs/rev
mozilla-central x_revs_x_added_x_copies               8d198483ae3b 14207ffc2b2f :      9 revs,   0.000381 s,   0.000187 s,  -0.000194 s, × 0.4908,    20 µs/rev
mozilla-central x_revs_x00_added_x_copies             98cbc58cc6bc 446a150332c3 :      7 revs,   0.000672 s,   0.000665 s,  -0.000007 s, × 0.9896,    95 µs/rev
mozilla-central x_revs_x000_added_x000_copies         3c684b4b8f68 0a5e72d1b479 :      3 revs,   0.003497 s,   0.003556 s,  +0.000059 s, × 1.0169,  1185 µs/rev
mozilla-central x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :      6 revs,   0.073204 s,   0.071345 s,  -0.001859 s, × 0.9746, 11890 µs/rev
mozilla-central x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   1593 revs,   0.006482 s,   0.006551 s,  +0.000069 s, × 1.0106,     4 µs/rev
mozilla-central x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :     41 revs,   0.005066 s,   0.005078 s,  +0.000012 s, × 1.0024,   123 µs/rev
mozilla-central x000_revs_x000_added_x000_copies      7c97034feb78 4407bd0c6330 :   7839 revs,   0.065707 s,   0.065823 s,  +0.000116 s, × 1.0018,     8 µs/rev
mozilla-central x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :    615 revs,   0.026800 s,   0.027050 s,  +0.000250 s, × 1.0093,    43 µs/rev
mozilla-central x0000_revs_xx000_added_x000_copies    f78c615a656c 96a38b690156 :  30263 revs,   0.203856 s,   0.202443 s,  -0.001413 s, × 0.9931,     6 µs/rev
mozilla-central x00000_revs_x0000_added_x0000_copies  6832ae71433c 4c222a1d9a00 : 153721 revs,   1.293394 s,   1.261583 s,  -0.031811 s, × 0.9754,     8 µs/rev
mozilla-central x00000_revs_x00000_added_x000_copies  76caed42cf7c 1daa622bbe42 : 204976 revs,   1.698239 s,   1.643869 s,  -0.054370 s, × 0.9680,     8 µs/rev
mozilla-try     x_revs_x_added_0_copies               aaf6dde0deb8 9790f499805a :      2 revs,   0.000875 s,   0.000868 s,  -0.000007 s, × 0.9920,   434 µs/rev
mozilla-try     x_revs_x000_added_0_copies            d8d0222927b4 5bb8ce8c7450 :      2 revs,   0.000891 s,   0.000887 s,  -0.000004 s, × 0.9955,   443 µs/rev
mozilla-try     x_revs_x_added_x_copies               092fcca11bdb 936255a0384a :      4 revs,   0.000292 s,   0.000168 s,  -0.000124 s, × 0.5753,    42 µs/rev
mozilla-try     x_revs_x00_added_x_copies             b53d2fadbdb5 017afae788ec :      2 revs,   0.003939 s,   0.001160 s,  -0.002779 s, × 0.2945,   580 µs/rev
mozilla-try     x_revs_x000_added_x000_copies         20408ad61ce5 6f0ee96e21ad :      1 revs,   0.033027 s,   0.033016 s,  -0.000011 s, × 0.9997, 33016 µs/rev
mozilla-try     x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :      6 revs,   0.073703 s,   0.073312 s,  -0.39ae31 s, × 0.9947, 12218 µs/rev
mozilla-try     x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   1593 revs,   0.006469 s,   0.006485 s,  +0.000016 s, × 1.0025,     4 µs/rev
mozilla-try     x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :     41 revs,   0.005278 s,   0.005494 s,  +0.000216 s, × 1.0409,   134 µs/rev
mozilla-try     x000_revs_x000_added_x000_copies      1346fd0130e4 4c65cbdabc1f :   6657 revs,   0.064995 s,   0.064879 s,  -0.000116 s, × 0.9982,     9 µs/rev
mozilla-try     x0000_revs_x_added_0_copies           63519bfd42ee a36a2a865d92 :  40314 revs,   0.301041 s,   0.301469 s,  +0.000428 s, × 1.0014,     7 µs/rev
mozilla-try     x0000_revs_x_added_x_copies           9fe69ff0762d bcabf2a78927 :  38690 revs,   0.285575 s,   0.297113 s,  +0.011538 s, × 1.0404,     7 µs/rev
mozilla-try     x0000_revs_xx000_added_x_copies       156f6e2674f2 4d0f2c178e66 :   8598 revs,   0.085597 s,   0.085890 s,  +0.000293 s, × 1.0034,     9 µs/rev
mozilla-try     x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :    615 revs,   0.027118 s,   0.027718 s,  +0.000600 s, × 1.0221,    45 µs/rev
mozilla-try     x0000_revs_xx000_added_x000_copies    89294cd501d9 7ccb2fc7ccb5 :  97052 revs,   2.119204 s,   2.048949 s,  -0.070255 s, × 0.9668,    21 µs/rev
mozilla-try     x0000_revs_x0000_added_x0000_copies   e928c65095ed e951f4ad123a :  52031 revs,   0.701479 s,   0.685924 s,  -0.015555 s, × 0.9778,    13 µs/rev
mozilla-try     x00000_revs_x_added_0_copies          6a320851d377 1ebb79acd503 : 363753 revs,   4.482399 s,   4.482891 s,  +0.000492 s, × 1.0001,    12 µs/rev
mozilla-try     x00000_revs_x00000_added_0_copies     dc8a3ca7010e d16fde900c9c :  34414 revs,   0.574082 s,   0.577633 s,  +0.003551 s, × 1.0062,    16 µs/rev
mozilla-try     x00000_revs_x_added_x_copies          5173c4b6f97c 95d83ee7242d : 362229 revs,   4.480366 s,   4.397816 s,  -0.082550 s, × 0.9816,    12 µs/rev
mozilla-try     x00000_revs_x000_added_x_copies       9126823d0e9c ca82787bb23c : 359344 revs,   4.369070 s,   4.370538 s,  +0.001468 s, × 1.0003,    12 µs/rev
mozilla-try     x00000_revs_x0000_added_x0000_copies  8d3fafa80d4b eb884023b810 : 192665 revs,   1.592506 s,   1.570439 s,  -0.022067 s, × 0.9861,     8 µs/rev
mozilla-try     x00000_revs_x00000_added_x0000_copies 1b661134e2ca 1ae03d022d6d : 228985 revs,  87.824489 s,  88.388512 s,  +0.564023 s, × 1.0064,   386 µs/rev
mozilla-try     x00000_revs_x00000_added_x000_copies  9b2a99adc05e 8e29777b48e6 : 382065 revs,  43.304637 s,  34.443661 s,  -8.860976 s, × 0.7954,    90 µs/rev
private                                                                         : 459513 revs,  33.853687 s,  27.370148 s,  -6.483539 s, × 0.8085,    59 µs/rev

Differential Revision: https://phab.mercurial-scm.org/D9653

This file tests the behavior of run-tests.py itself.

Avoid interference from actual test env:

  $ . "$TESTDIR/helper-runtests.sh"

Smoke test with install
============
  $ "$PYTHON" $TESTDIR/run-tests.py $HGTEST_RUN_TESTS_PURE -l
  running 0 tests using 0 parallel processes 
  
  # Ran 0 tests, 0 skipped, 0 failed.

Define a helper to avoid the install step
=============
  $ rt()
  > {
  >     "$PYTHON" $TESTDIR/run-tests.py --with-hg=`which hg` -j1 "$@"
  > }

error paths

#if symlink
  $ ln -s `which true` hg
  $ "$PYTHON" $TESTDIR/run-tests.py --with-hg=./hg
  warning: --with-hg should specify an hg script
  running 0 tests using 0 parallel processes 
  
  # Ran 0 tests, 0 skipped, 0 failed.
  $ rm hg
#endif

#if execbit
  $ touch hg
  $ "$PYTHON" $TESTDIR/run-tests.py --with-hg=./hg
  usage: run-tests.py [options] [tests]
  run-tests.py: error: --with-hg must specify an executable hg script
  [2]
  $ rm hg
#endif

Features for testing optional lines
===================================

  $ cat > hghaveaddon.py <<EOF
  > import hghave
  > @hghave.check("custom", "custom hghave feature")
  > def has_custom():
  >     return True
  > @hghave.check("missing", "missing hghave feature")
  > def has_missing():
  >     return False
  > EOF

an empty test
=======================

  $ touch test-empty.t
  $ rt
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.
  $ rm test-empty.t

a succesful test
=======================

  $ cat > test-success.t << EOF
  >   $ echo babar
  >   babar
  >   $ echo xyzzy
  >   dont_print (?)
  >   nothing[42]line (re) (?)
  >   never*happens (glob) (?)
  >   more_nothing (?)
  >   xyzzy
  >   nor this (?)
  >   $ printf 'abc\ndef\nxyz\n'
  >   123 (?)
  >   abc
  >   def (?)
  >   456 (?)
  >   xyz
  >   $ printf 'zyx\nwvu\ntsr\n'
  >   abc (?)
  >   zyx (custom !)
  >   wvu
  >   no_print (no-custom !)
  >   tsr (no-missing !)
  >   missing (missing !)
  > EOF

  $ rt
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

failing test
==================

test churn with globs
  $ cat > test-failure.t <<EOF
  >   $ echo "bar-baz"; echo "bar-bad"; echo foo
  >   bar*bad (glob)
  >   bar*baz (glob)
  >   | fo (re)
  > EOF
  $ rt test-failure.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,4 +1,4 @@
     $ echo "bar-baz"; echo "bar-bad"; echo foo
  +  bar*baz (glob)
     bar*bad (glob)
  -  bar*baz (glob)
  -  | fo (re)
  +  foo
  
  ERROR: test-failure.t output changed
  !
  Failed test-failure.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

test how multiple globs gets matched with lines in output
  $ cat > test-failure-globs.t <<EOF
  >   $ echo "context"; echo "context"; \
  >     echo "key: 1"; echo "value: not a"; \
  >     echo "key: 2"; echo "value: not b"; \
  >     echo "key: 3"; echo "value: c"; \
  >     echo "key: 4"; echo "value: d"
  >   context
  >   context
  >   key: 1
  >   value: a
  >   key: 2
  >   value: b
  >   key: 3
  >   value: * (glob)
  >   key: 4
  >   value: * (glob)
  > EOF
  $ rt test-failure-globs.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure-globs.t
  +++ $TESTTMP/test-failure-globs.t.err
  @@ -2,9 +2,9 @@
     context
     context
     key: 1
  -  value: a
  +  value: not a
     key: 2
  -  value: b
  +  value: not b
     key: 3
     value: * (glob)
     key: 4
  
  ERROR: test-failure-globs.t output changed
  !
  Failed test-failure-globs.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]
  $ rm test-failure-globs.t

test diff colorisation

#if no-windows pygments
  $ rt test-failure.t --color always
  running 1 tests using 1 parallel processes 
  
  \x1b[38;5;124m--- $TESTTMP/test-failure.t\x1b[39m (esc)
  \x1b[38;5;34m+++ $TESTTMP/test-failure.t.err\x1b[39m (esc)
  \x1b[38;5;90;01m@@ -1,4 +1,4 @@\x1b[39;00m (esc)
     $ echo "bar-baz"; echo "bar-bad"; echo foo
  \x1b[38;5;34m+  bar*baz (glob)\x1b[39m (esc)
     bar*bad (glob)
  \x1b[38;5;124m-  bar*baz (glob)\x1b[39m (esc)
  \x1b[38;5;124m-  | fo (re)\x1b[39m (esc)
  \x1b[38;5;34m+  foo\x1b[39m (esc)
  
  \x1b[38;5;88mERROR: \x1b[39m\x1b[38;5;9mtest-failure.t\x1b[39m\x1b[38;5;88m output changed\x1b[39m (esc)
  !
  \x1b[38;5;88mFailed \x1b[39m\x1b[38;5;9mtest-failure.t\x1b[39m\x1b[38;5;88m: output changed\x1b[39m (esc)
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ rt test-failure.t 2> tmp.log
  running 1 tests using 1 parallel processes 
  [1]
  $ cat tmp.log
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,4 +1,4 @@
     $ echo "bar-baz"; echo "bar-bad"; echo foo
  +  bar*baz (glob)
     bar*bad (glob)
  -  bar*baz (glob)
  -  | fo (re)
  +  foo
  
  ERROR: test-failure.t output changed
  !
  Failed test-failure.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
#endif

  $ cat > test-failure.t << EOF
  >   $ true
  >   should go away (true !)
  >   $ true
  >   should stay (false !)
  > 
  > Should remove first line, not second or third
  >   $ echo 'testing'
  >   baz*foo (glob) (true !)
  >   foobar*foo (glob) (false !)
  >   te*ting (glob) (true !)
  > 
  > Should keep first two lines, remove third and last
  >   $ echo 'testing'
  >   test.ng (re) (true !)
  >   foo.ar (re) (false !)
  >   b.r (re) (true !)
  >   missing (?)
  >   awol (true !)
  > 
  > The "missing" line should stay, even though awol is dropped
  >   $ echo 'testing'
  >   test.ng (re) (true !)
  >   foo.ar (?)
  >   awol
  >   missing (?)
  > EOF
  $ rt test-failure.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,11 +1,9 @@
     $ true
  -  should go away (true !)
     $ true
     should stay (false !)
   
   Should remove first line, not second or third
     $ echo 'testing'
  -  baz*foo (glob) (true !)
     foobar*foo (glob) (false !)
     te*ting (glob) (true !)
   
     foo.ar (re) (false !)
     missing (?)
  @@ -13,13 +11,10 @@
     $ echo 'testing'
     test.ng (re) (true !)
     foo.ar (re) (false !)
  -  b.r (re) (true !)
     missing (?)
  -  awol (true !)
   
   The "missing" line should stay, even though awol is dropped
     $ echo 'testing'
     test.ng (re) (true !)
     foo.ar (?)
  -  awol
     missing (?)
  
  ERROR: test-failure.t output changed
  !
  Failed test-failure.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

basic failing test
  $ cat > test-failure.t << EOF
  >   $ echo babar
  >   rataxes
  > This is a noop statement so that
  > this test is still more bytes than success.
  > pad pad pad pad............................................................
  > pad pad pad pad............................................................
  > pad pad pad pad............................................................
  > pad pad pad pad............................................................
  > pad pad pad pad............................................................
  > pad pad pad pad............................................................
  > EOF

  >>> fh = open('test-failure-unicode.t', 'wb')
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None

  $ rt
  running 3 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !.
  --- $TESTTMP/test-failure-unicode.t
  +++ $TESTTMP/test-failure-unicode.t.err
  @@ -1,2 +1,2 @@
     $ echo babar\xce\xb1 (esc)
  -  l\xce\xb5\xce\xb5t (esc)
  +  babar\xce\xb1 (esc)
  
  ERROR: test-failure-unicode.t output changed
  !
  Failed test-failure-unicode.t: output changed
  Failed test-failure.t: output changed
  # Ran 3 tests, 0 skipped, 2 failed.
  python hash seed: * (glob)
  [1]

test --outputdir
  $ mkdir output
  $ rt --outputdir output
  running 3 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/output/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !.
  --- $TESTTMP/test-failure-unicode.t
  +++ $TESTTMP/output/test-failure-unicode.t.err
  @@ -1,2 +1,2 @@
     $ echo babar\xce\xb1 (esc)
  -  l\xce\xb5\xce\xb5t (esc)
  +  babar\xce\xb1 (esc)
  
  ERROR: test-failure-unicode.t output changed
  !
  Failed test-failure-unicode.t: output changed
  Failed test-failure.t: output changed
  # Ran 3 tests, 0 skipped, 2 failed.
  python hash seed: * (glob)
  [1]
  $ ls -a output
  .
  ..
  .testtimes
  test-failure-unicode.t.err
  test-failure.t.err

test --xunit support
  $ rt --xunit=xunit.xml
  running 3 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !.
  --- $TESTTMP/test-failure-unicode.t
  +++ $TESTTMP/test-failure-unicode.t.err
  @@ -1,2 +1,2 @@
     $ echo babar\xce\xb1 (esc)
  -  l\xce\xb5\xce\xb5t (esc)
  +  babar\xce\xb1 (esc)
  
  ERROR: test-failure-unicode.t output changed
  !
  Failed test-failure-unicode.t: output changed
  Failed test-failure.t: output changed
  # Ran 3 tests, 0 skipped, 2 failed.
  python hash seed: * (glob)
  [1]
  $ cat xunit.xml
  <?xml version="1.0" encoding="utf-8"?>
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
    <testcase name="test-success.t" time="*"/> (glob)
    <testcase name="test-failure-unicode.t" time="*"> (glob)
      <failure message="output changed" type="output-mismatch"><![CDATA[--- $TESTTMP/test-failure-unicode.t (py38 !)
      <failure message="output changed" type="output-mismatch"> (no-py38 !)
  <![CDATA[--- $TESTTMP/test-failure-unicode.t (no-py38 !)
  +++ $TESTTMP/test-failure-unicode.t.err
  @@ -1,2 +1,2 @@
     $ echo babar\xce\xb1 (esc)
  -  l\xce\xb5\xce\xb5t (esc)
  +  babar\xce\xb1 (esc)
  ]]></failure> (py38 !)
  ]]>    </failure> (no-py38 !)
    </testcase>
    <testcase name="test-failure.t" time="*"> (glob)
      <failure message="output changed" type="output-mismatch"><![CDATA[--- $TESTTMP/test-failure.t (py38 !)
      <failure message="output changed" type="output-mismatch"> (no-py38 !)
  <![CDATA[--- $TESTTMP/test-failure.t (no-py38 !)
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  ]]></failure> (py38 !)
  ]]>    </failure> (no-py38 !)
    </testcase>
  </testsuite>

  $ cat .testtimes
  test-empty.t * (glob)
  test-failure-globs.t * (glob)
  test-failure-unicode.t * (glob)
  test-failure.t * (glob)
  test-success.t * (glob)

  $ rt --list-tests
  test-failure-unicode.t
  test-failure.t
  test-success.t

  $ rt --list-tests --json
  test-failure-unicode.t
  test-failure.t
  test-success.t
  $ cat report.json
  testreport ={
      "test-failure-unicode.t": {
          "result": "success"
      },
      "test-failure.t": {
          "result": "success"
      },
      "test-success.t": {
          "result": "success"
      }
  } (no-eol)

  $ rt --list-tests --xunit=xunit.xml
  test-failure-unicode.t
  test-failure.t
  test-success.t
  $ cat xunit.xml
  <?xml version="1.0" encoding="utf-8"?>
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
    <testcase name="test-failure-unicode.t"/>
    <testcase name="test-failure.t"/>
    <testcase name="test-success.t"/>
  </testsuite>

  $ rt --list-tests test-failure* --json --xunit=xunit.xml --outputdir output
  test-failure-unicode.t
  test-failure.t
  $ cat output/report.json
  testreport ={
      "test-failure-unicode.t": {
          "result": "success"
      },
      "test-failure.t": {
          "result": "success"
      }
  } (no-eol)
  $ cat xunit.xml
  <?xml version="1.0" encoding="utf-8"?>
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
    <testcase name="test-failure-unicode.t"/>
    <testcase name="test-failure.t"/>
  </testsuite>

  $ rm test-failure-unicode.t

test for --retest
====================

  $ rt --retest
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !
  Failed test-failure.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

--retest works with --outputdir
  $ rm -r output
  $ mkdir output
  $ mv test-failure.t.err output
  $ rt --retest --outputdir output
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/output/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !
  Failed test-failure.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Selecting Tests To Run
======================

successful

  $ rt test-success.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

success w/ keyword
  $ rt -k xyzzy
  running 2 tests using 1 parallel processes 
  .
  # Ran 2 tests, 1 skipped, 0 failed.

failed

  $ rt test-failure.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !
  Failed test-failure.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

failure w/ keyword
  $ rt -k rataxes
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !
  Failed test-failure.t: output changed
  # Ran 2 tests, 1 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Verify that when a process fails to start we show a useful message
==================================================================

  $ cat > test-serve-fail.t <<EOF
  >   $ echo 'abort: child process failed to start blah'
  > EOF
  $ rt test-serve-fail.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-serve-fail.t
  +++ $TESTTMP/test-serve-fail.t.err
  @@ -1* +1,2 @@ (glob)
     $ echo 'abort: child process failed to start blah'
  +  abort: child process failed to start blah
  
  ERROR: test-serve-fail.t output changed
  !
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]
  $ rm test-serve-fail.t

Verify that we can try other ports
===================================

Extensions aren't inherited by the invoked run-tests.py. An extension
introducing a repository requirement could cause this to fail. So we force
HGRCPATH to get a clean environment.

  $ HGRCPATH= hg init inuse
  $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
  $ cat blocks.pid >> $DAEMON_PIDS
  $ cat > test-serve-inuse.t <<EOF
  >   $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
  >   $ cat hg.pid >> \$DAEMON_PIDS
  > EOF
  $ rt test-serve-inuse.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.
  $ rm test-serve-inuse.t
  $ killdaemons.py $DAEMON_PIDS

Running In Debug Mode
======================

  $ rt --debug 2>&1 | grep -v pwd
  running 2 tests using 1 parallel processes 
  + alias hg=hg.exe (windows !)
  + echo *SALT* 0 0 (glob)
  *SALT* 0 0 (glob)
  + echo babar
  babar
  + echo *SALT* 10 0 (glob)
  *SALT* 10 0 (glob)
  .+ alias hg=hg.exe (windows !)
  *+ echo *SALT* 0 0 (glob)
  *SALT* 0 0 (glob)
  + echo babar
  babar
  + echo *SALT* 2 0 (glob)
  *SALT* 2 0 (glob)
  + echo xyzzy
  xyzzy
  + echo *SALT* 9 0 (glob)
  *SALT* 9 0 (glob)
  + printf *abc\ndef\nxyz\n* (glob)
  abc
  def
  xyz
  + echo *SALT* 15 0 (glob)
  *SALT* 15 0 (glob)
  + printf *zyx\nwvu\ntsr\n* (glob)
  zyx
  wvu
  tsr
  + echo *SALT* 22 0 (glob)
  *SALT* 22 0 (glob)
  .
  # Ran 2 tests, 0 skipped, 0 failed.

Parallel runs
==============

(duplicate the failing test to get predictable output)
  $ cp test-failure.t test-failure-copy.t

  $ rt --jobs 2 test-failure*.t -n
  running 2 tests using 2 parallel processes 
  !!
  Failed test-failure*.t: output changed (glob)
  Failed test-failure*.t: output changed (glob)
  # Ran 2 tests, 0 skipped, 2 failed.
  python hash seed: * (glob)
  [1]

failures in parallel with --first should only print one failure
  $ rt --jobs 2 --first test-failure*.t
  running 2 tests using 2 parallel processes 
  
  --- $TESTTMP/test-failure*.t (glob)
  +++ $TESTTMP/test-failure*.t.err (glob)
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  Failed test-failure*.t: output changed (glob)
  Failed test-failure*.t: output changed (glob)
  # Ran 2 tests, 0 skipped, 2 failed.
  python hash seed: * (glob)
  [1]


(delete the duplicated test file)
  $ rm test-failure-copy.t

multiple runs per test should be parallelized

  $ rt --jobs 2 --runs-per-test 2 test-success.t
  running 2 tests using 2 parallel processes 
  ..
  # Ran 2 tests, 0 skipped, 0 failed.

Interactive run
===============

(backup the failing test)
  $ cp test-failure.t backup

Refuse the fix

  $ echo 'n' | rt -i
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  Accept this change? [y/N] 
  ERROR: test-failure.t output changed
  !.
  Failed test-failure.t: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ cat test-failure.t
    $ echo babar
    rataxes
  This is a noop statement so that
  this test is still more bytes than success.
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................

Interactive with custom view

  $ echo 'n' | rt -i --view echo
  running 2 tests using 1 parallel processes 
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
  Accept this change? [y/N]* (glob)
  ERROR: test-failure.t output changed
  !.
  Failed test-failure.t: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

View the fix

  $ echo 'y' | rt --view echo
  running 2 tests using 1 parallel processes 
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
  
  ERROR: test-failure.t output changed
  !.
  Failed test-failure.t: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Accept the fix

  $ cat >> test-failure.t <<EOF
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
  >   saved backup bundle to \$TESTTMP/foo.hg
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
  >   saved backup bundle to $TESTTMP\\foo.hg
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
  >   saved backup bundle to \$TESTTMP/*.hg (glob)
  > EOF
  $ echo 'y' | rt -i 2>&1
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  @@ -11,6 +11,6 @@
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
     saved backup bundle to $TESTTMP/foo.hg
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
  -  saved backup bundle to $TESTTMP\foo.hg
  +  saved backup bundle to $TESTTMP/foo.hg
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
     saved backup bundle to $TESTTMP/*.hg (glob)
  Accept this change? [y/N] ..
  # Ran 2 tests, 0 skipped, 0 failed.

  $ sed -e 's,(glob)$,&<,g' test-failure.t
    $ echo babar
    babar
  This is a noop statement so that
  this test is still more bytes than success.
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................
  pad pad pad pad............................................................
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
    saved backup bundle to $TESTTMP/foo.hg
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
    saved backup bundle to $TESTTMP/foo.hg
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
    saved backup bundle to $TESTTMP/*.hg (glob)<

  $ rm test-failure.t

Race condition - test file was modified when test is running

  $ TESTRACEDIR=`pwd`
  $ export TESTRACEDIR
  $ cat > test-race.t <<EOF
  >   $ echo 1
  >   $ echo "# a new line" >> $TESTRACEDIR/test-race.t
  > EOF

  $ rt -i test-race.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-race.t
  +++ $TESTTMP/test-race.t.err
  @@ -1,2 +1,3 @@
     $ echo 1
  +  1
     $ echo "# a new line" >> $TESTTMP/test-race.t
  Reference output has changed (run again to prompt changes)
  ERROR: test-race.t output changed
  !
  Failed test-race.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ rm test-race.t

When "#testcases" is used in .t files

  $ cat >> test-cases.t <<EOF
  > #testcases a b
  > #if a
  >   $ echo 1
  > #endif
  > #if b
  >   $ echo 2
  > #endif
  > EOF

  $ cat <<EOF | rt -i test-cases.t 2>&1
  > y
  > y
  > EOF
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/test-cases.t
  +++ $TESTTMP/test-cases.t#a.err
  @@ -1,6 +1,7 @@
   #testcases a b
   #if a
     $ echo 1
  +  1
   #endif
   #if b
     $ echo 2
  Accept this change? [y/N] .
  --- $TESTTMP/test-cases.t
  +++ $TESTTMP/test-cases.t#b.err
  @@ -5,4 +5,5 @@
   #endif
   #if b
     $ echo 2
  +  2
   #endif
  Accept this change? [y/N] .
  # Ran 2 tests, 0 skipped, 0 failed.

  $ cat test-cases.t
  #testcases a b
  #if a
    $ echo 1
    1
  #endif
  #if b
    $ echo 2
    2
  #endif

  $ cat >> test-cases.t <<'EOF'
  > #if a
  >   $ NAME=A
  > #else
  >   $ NAME=B
  > #endif
  >   $ echo $NAME
  >   A (a !)
  >   B (b !)
  > EOF
  $ rt test-cases.t
  running 2 tests using 1 parallel processes 
  ..
  # Ran 2 tests, 0 skipped, 0 failed.

When using multiple dimensions of "#testcases" in .t files

  $ cat > test-cases.t <<'EOF'
  > #testcases a b
  > #testcases c d
  > #if a d
  >   $ echo $TESTCASE
  >   a#d
  > #endif
  > #if b c
  >   $ echo yes
  >   no
  > #endif
  > EOF
  $ rt test-cases.t
  running 4 tests using 1 parallel processes 
  ..
  --- $TESTTMP/test-cases.t
  +++ $TESTTMP/test-cases.t#b#c.err
  @@ -6,5 +6,5 @@
   #endif
   #if b c
     $ echo yes
  -  no
  +  yes
   #endif
  
  ERROR: test-cases.t#b#c output changed
  !.
  Failed test-cases.t#b#c: output changed
  # Ran 4 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ rt --retest
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/test-cases.t
  +++ $TESTTMP/test-cases.t#b#c.err
  @@ -6,5 +6,5 @@
   #endif
   #if b c
     $ echo yes
  -  no
  +  yes
   #endif
  
  ERROR: test-cases.t#b#c output changed
  !
  Failed test-cases.t#b#c: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]
  $ rm test-cases.t#b#c.err
  $ rm test-cases.t

(reinstall)
  $ mv backup test-failure.t

No Diff
===============

  $ rt --nodiff
  running 2 tests using 1 parallel processes 
  !.
  Failed test-failure.t: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

test --tmpdir support
  $ rt --tmpdir=$TESTTMP/keep test-success.t
  running 1 tests using 1 parallel processes 
  
  Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t
  Keeping threadtmp dir: $TESTTMP/keep/child1 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

timeouts
========
  $ cat > test-timeout.t <<EOF
  >   $ sleep 2
  >   $ echo pass
  >   pass
  > EOF
  > echo '#require slow' > test-slow-timeout.t
  > cat test-timeout.t >> test-slow-timeout.t
  $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
  running 2 tests using 1 parallel processes 
  st
  Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests)
  Failed test-timeout.t: timed out
  # Ran 1 tests, 1 skipped, 1 failed.
  python hash seed: * (glob)
  [1]
  $ rt --timeout=1 --slowtimeout=3 \
  > test-timeout.t test-slow-timeout.t --allow-slow-tests
  running 2 tests using 1 parallel processes 
  .t
  Failed test-timeout.t: timed out
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]
  $ rm test-timeout.t test-slow-timeout.t

test for --time
==================

  $ rt test-success.t --time
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.
  # Producing time report
  start   end     cuser   csys    real      Test
  \s*[\d\.]{5,8} \s*[\d\.]{5,8} \s*[\d\.]{5,8} \s*[\d\.]{5,8} \s*[\d\.]{5,8}   test-success.t (re)

test for --time with --job enabled
====================================

  $ rt test-success.t --time --jobs 2
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.
  # Producing time report
  start   end     cuser   csys    real      Test
  \s*[\d\.]{5,8} \s*[\d\.]{5,8} \s*[\d\.]{5,8} \s*[\d\.]{5,8} \s*[\d\.]{5,8}   test-success.t (re)

Skips
================
  $ cat > test-skip.t <<EOF
  >   $ echo xyzzy
  > #if true
  > #require false
  > #end
  > EOF
  $ cat > test-noskip.t <<EOF
  > #if false
  > #require false
  > #endif
  > EOF
  $ rt --nodiff
  running 4 tests using 1 parallel processes 
  !.s.
  Skipped test-skip.t: missing feature: nail clipper
  Failed test-failure.t: output changed
  # Ran 3 tests, 1 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ rm test-noskip.t
  $ rt --keyword xyzzy
  running 3 tests using 1 parallel processes 
  .s
  Skipped test-skip.t: missing feature: nail clipper
  # Ran 2 tests, 2 skipped, 0 failed.

Skips with xml
  $ rt --keyword xyzzy \
  >  --xunit=xunit.xml
  running 3 tests using 1 parallel processes 
  .s
  Skipped test-skip.t: missing feature: nail clipper
  # Ran 2 tests, 2 skipped, 0 failed.
  $ cat xunit.xml
  <?xml version="1.0" encoding="utf-8"?>
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
    <testcase name="test-success.t" time="*"/> (glob)
    <testcase name="test-skip.t">
      <skipped><![CDATA[missing feature: nail clipper]]></skipped> (py38 !)
      <skipped> (no-py38 !)
  <![CDATA[missing feature: nail clipper]]>    </skipped> (no-py38 !)
    </testcase>
  </testsuite>

Missing skips or blacklisted skips don't count as executed:
  $ echo test-failure.t > blacklist
  $ rt --blacklist=blacklist --json\
  >   test-failure.t test-bogus.t
  running 2 tests using 1 parallel processes 
  ss
  Skipped test-bogus.t: Doesn't exist
  Skipped test-failure.t: blacklisted
  # Ran 0 tests, 2 skipped, 0 failed.
  $ cat report.json
  testreport ={
      "test-bogus.t": {
          "result": "skip"
      },
      "test-failure.t": {
          "result": "skip"
      }
  } (no-eol)

Whitelist trumps blacklist
  $ echo test-failure.t > whitelist
  $ rt --blacklist=blacklist --whitelist=whitelist --json\
  >   test-failure.t test-bogus.t
  running 2 tests using 1 parallel processes 
  s
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !
  Skipped test-bogus.t: Doesn't exist
  Failed test-failure.t: output changed
  # Ran 1 tests, 1 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Ensure that --test-list causes only the tests listed in that file to
be executed.
  $ echo test-success.t >> onlytest
  $ rt --test-list=onlytest
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.
  $ echo test-bogus.t >> anothertest
  $ rt --test-list=onlytest --test-list=anothertest
  running 2 tests using 1 parallel processes 
  s.
  Skipped test-bogus.t: Doesn't exist
  # Ran 1 tests, 1 skipped, 0 failed.
  $ rm onlytest anothertest

test for --json
==================

  $ rt --json
  running 3 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !.s
  Skipped test-skip.t: missing feature: nail clipper
  Failed test-failure.t: output changed
  # Ran 2 tests, 1 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ cat report.json
  testreport ={
      "test-failure.t": [\{] (re)
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "---.+\+\+\+.+", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "failure", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }, ? (re)
      "test-skip.t": {
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "skip", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }, ? (re)
      "test-success.t": [\{] (re)
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "success", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }
  } (no-eol)
--json with --outputdir

  $ rm report.json
  $ rm -r output
  $ mkdir output
  $ rt --json --outputdir output
  running 3 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/output/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  
  ERROR: test-failure.t output changed
  !.s
  Skipped test-skip.t: missing feature: nail clipper
  Failed test-failure.t: output changed
  # Ran 2 tests, 1 skipped, 1 failed.
  python hash seed: * (glob)
  [1]
  $ f report.json
  report.json: file not found
  $ cat output/report.json
  testreport ={
      "test-failure.t": [\{] (re)
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "---.+\+\+\+.+", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "failure", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }, ? (re)
      "test-skip.t": {
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "skip", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }, ? (re)
      "test-success.t": [\{] (re)
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "success", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }
  } (no-eol)
  $ ls -a output
  .
  ..
  .testtimes
  report.json
  test-failure.t.err

Test that failed test accepted through interactive are properly reported:

  $ cp test-failure.t backup
  $ echo y | rt --json -i
  running 3 tests using 1 parallel processes 
  
  --- $TESTTMP/test-failure.t
  +++ $TESTTMP/test-failure.t.err
  @@ -1,5 +1,5 @@
     $ echo babar
  -  rataxes
  +  babar
   This is a noop statement so that
   this test is still more bytes than success.
   pad pad pad pad............................................................
  Accept this change? [y/N] ..s
  Skipped test-skip.t: missing feature: nail clipper
  # Ran 2 tests, 1 skipped, 0 failed.

  $ cat report.json
  testreport ={
      "test-failure.t": [\{] (re)
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "success", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }, ? (re)
      "test-skip.t": {
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "skip", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }, ? (re)
      "test-success.t": [\{] (re)
          "csys": "\s*\d+\.\d{3,4}", ? (re)
          "cuser": "\s*\d+\.\d{3,4}", ? (re)
          "diff": "", ? (re)
          "end": "\s*\d+\.\d{3,4}", ? (re)
          "result": "success", ? (re)
          "start": "\s*\d+\.\d{3,4}", ? (re)
          "time": "\s*\d+\.\d{3,4}" (re)
      }
  } (no-eol)
  $ mv backup test-failure.t

backslash on end of line with glob matching is handled properly

  $ cat > test-glob-backslash.t << EOF
  >   $ echo 'foo bar \\'
  >   foo * \ (glob)
  > EOF

  $ rt test-glob-backslash.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

  $ rm -f test-glob-backslash.t

Test globbing of local IP addresses
  $ echo 172.16.18.1
  $LOCALIP (glob)
  $ echo dead:beef::1
  $LOCALIP (glob)

Add support for external test formatter
=======================================

  $ CUSTOM_TEST_RESULT=basic_test_result "$PYTHON" $TESTDIR/run-tests.py --with-hg=`which hg` -j1 "$@" test-success.t test-failure.t
  running 2 tests using 1 parallel processes 
  
  # Ran 2 tests, 0 skipped, 0 failed.
  ON_START! <__main__.TestSuite tests=[<__main__.TTest testMethod=test-failure.t>, <__main__.TTest testMethod=test-success.t>]>
  FAILURE! test-failure.t output changed
  SUCCESS! test-success.t
  ON_END!

Test reusability for third party tools
======================================

  $ mkdir "$TESTTMP"/anothertests
  $ cd "$TESTTMP"/anothertests

test that `run-tests.py` can execute hghave, even if it runs not in
Mercurial source tree.

  $ cat > test-hghave.t <<EOF
  > #require true
  >   $ echo foo
  >   foo
  > EOF
  $ rt test-hghave.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

test that RUNTESTDIR refers the directory, in which `run-tests.py` now
running is placed.

  $ cat > test-runtestdir.t <<EOF
  > - $TESTDIR, in which test-run-tests.t is placed
  > - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
  > - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
  > 
  > #if windows
  >   $ test "\$TESTDIR" = "$TESTTMP\anothertests"
  > #else
  >   $ test "\$TESTDIR" = "$TESTTMP"/anothertests
  > #endif
  > If this prints a path, that means RUNTESTDIR didn't equal
  > TESTDIR as it should have.
  >   $ test "\$RUNTESTDIR" = "$TESTDIR" || echo "\$RUNTESTDIR"
  > This should print the start of check-code. If this passes but the
  > previous check failed, that means we found a copy of check-code at whatever
  > RUNTESTSDIR ended up containing, even though it doesn't match TESTDIR.
  >   $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py | sed 's@.!.*python3@#!USRBINENVPY@'
  >   #!USRBINENVPY
  >   #
  >   # check-code - a style and portability checker for Mercurial
  > EOF
  $ rt test-runtestdir.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

#if execbit

test that TESTDIR is referred in PATH

  $ cat > custom-command.sh <<EOF
  > #!/bin/sh
  > echo "hello world"
  > EOF
  $ chmod +x custom-command.sh
  $ cat > test-testdir-path.t <<EOF
  >   $ custom-command.sh
  >   hello world
  > EOF
  $ rt test-testdir-path.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

#endif

test support for --allow-slow-tests
  $ cat > test-very-slow-test.t <<EOF
  > #require slow
  >   $ echo pass
  >   pass
  > EOF
  $ rt test-very-slow-test.t
  running 1 tests using 1 parallel processes 
  s
  Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests)
  # Ran 0 tests, 1 skipped, 0 failed.
  $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

support for running a test outside the current directory
  $ mkdir nonlocal
  $ cat > nonlocal/test-is-not-here.t << EOF
  >   $ echo pass
  >   pass
  > EOF
  $ rt nonlocal/test-is-not-here.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

support for automatically discovering test if arg is a folder
  $ mkdir tmp && cd tmp

  $ cat > test-uno.t << EOF
  >   $ echo line
  >   line
  > EOF

  $ cp test-uno.t test-dos.t
  $ cd ..
  $ cp -R tmp tmpp
  $ cp tmp/test-uno.t test-solo.t

  $ rt tmp/ test-solo.t tmpp
  running 5 tests using 1 parallel processes 
  .....
  # Ran 5 tests, 0 skipped, 0 failed.
  $ rm -rf tmp tmpp

support for running run-tests.py from another directory
  $ mkdir tmp && cd tmp

  $ cat > useful-file.sh << EOF
  > important command
  > EOF

  $ cat > test-folder.t << EOF
  >   $ cat \$TESTDIR/useful-file.sh
  >   important command
  > EOF

  $ cat > test-folder-fail.t << EOF
  >   $ cat \$TESTDIR/useful-file.sh
  >   important commando
  > EOF

  $ cd ..
  $ rt tmp/test-*.t
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/tmp/test-folder-fail.t
  +++ $TESTTMP/anothertests/tmp/test-folder-fail.t.err
  @@ -1,2 +1,2 @@
     $ cat $TESTDIR/useful-file.sh
  -  important commando
  +  important command
  
  ERROR: test-folder-fail.t output changed
  !.
  Failed test-folder-fail.t: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

support for bisecting failed tests automatically
  $ hg init bisect
  $ cd bisect
  $ cat >> test-bisect.t <<EOF
  >   $ echo pass
  >   pass
  > EOF
  $ hg add test-bisect.t
  $ hg ci -m 'good'
  $ cat >> test-bisect.t <<EOF
  >   $ echo pass
  >   fail
  > EOF
  $ hg ci -m 'bad'
  $ rt --known-good-rev=0 test-bisect.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/bisect/test-bisect.t
  +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
  @@ -1,4 +1,4 @@
     $ echo pass
     pass
     $ echo pass
  -  fail
  +  pass
  
  ERROR: test-bisect.t output changed
  !
  Failed test-bisect.t: output changed
  test-bisect.t broken by 72cbf122d116 (bad)
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ cd ..

support bisecting a separate repo

  $ hg init bisect-dependent
  $ cd bisect-dependent
  $ cat > test-bisect-dependent.t <<EOF
  >   $ tail -1 \$TESTDIR/../bisect/test-bisect.t
  >     pass
  > EOF
  $ hg commit -Am dependent test-bisect-dependent.t

  $ rt --known-good-rev=0 test-bisect-dependent.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
  +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
  @@ -1,2 +1,2 @@
     $ tail -1 $TESTDIR/../bisect/test-bisect.t
  -    pass
  +    fail
  
  ERROR: test-bisect-dependent.t output changed
  !
  Failed test-bisect-dependent.t: output changed
  Failed to identify failure point for test-bisect-dependent.t
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ rt --bisect-repo=../test-bisect test-bisect-dependent.t
  usage: run-tests.py [options] [tests]
  run-tests.py: error: --bisect-repo cannot be used without --known-good-rev
  [2]

  $ rt --known-good-rev=0 --bisect-repo=../bisect test-bisect-dependent.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
  +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
  @@ -1,2 +1,2 @@
     $ tail -1 $TESTDIR/../bisect/test-bisect.t
  -    pass
  +    fail
  
  ERROR: test-bisect-dependent.t output changed
  !
  Failed test-bisect-dependent.t: output changed
  test-bisect-dependent.t broken by 72cbf122d116 (bad)
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ cd ..

Test a broken #if statement doesn't break run-tests threading.
==============================================================
  $ mkdir broken
  $ cd broken
  $ cat > test-broken.t <<EOF
  > true
  > #if notarealhghavefeature
  >   $ false
  > #endif
  > EOF
  $ for f in 1 2 3 4 ; do
  > cat > test-works-$f.t <<EOF
  > This is test case $f
  >   $ sleep 1
  > EOF
  > done
  $ rt -j 2
  running 5 tests using 2 parallel processes 
  ....
  # Ran 5 tests, 0 skipped, 0 failed.
  skipped: unknown feature: notarealhghavefeature
  
  $ cd ..
  $ rm -rf broken

Test cases in .t files
======================
  $ mkdir cases
  $ cd cases
  $ cat > test-cases-abc.t <<'EOF'
  > #testcases A B C
  >   $ V=B
  > #if A
  >   $ V=A
  > #endif
  > #if C
  >   $ V=C
  > #endif
  >   $ echo $V | sed 's/A/C/'
  >   C
  > #if C
  >   $ [ $V = C ]
  > #endif
  > #if A
  >   $ [ $V = C ]
  >   [1]
  > #endif
  > #if no-C
  >   $ [ $V = C ]
  >   [1]
  > #endif
  >   $ [ $V = D ]
  >   [1]
  > EOF
  $ rt
  running 3 tests using 1 parallel processes 
  .
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
  @@ -7,7 +7,7 @@
     $ V=C
   #endif
     $ echo $V | sed 's/A/C/'
  -  C
  +  B
   #if C
     $ [ $V = C ]
   #endif
  
  ERROR: test-cases-abc.t#B output changed
  !.
  Failed test-cases-abc.t#B: output changed
  # Ran 3 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

--restart works

  $ rt --restart
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
  @@ -7,7 +7,7 @@
     $ V=C
   #endif
     $ echo $V | sed 's/A/C/'
  -  C
  +  B
   #if C
     $ [ $V = C ]
   #endif
  
  ERROR: test-cases-abc.t#B output changed
  !.
  Failed test-cases-abc.t#B: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

--restart works with outputdir

  $ mkdir output
  $ mv test-cases-abc.t#B.err output
  $ rt --restart --outputdir output
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
  +++ $TESTTMP/anothertests/cases/output/test-cases-abc.t#B.err
  @@ -7,7 +7,7 @@
     $ V=C
   #endif
     $ echo $V | sed 's/A/C/'
  -  C
  +  B
   #if C
     $ [ $V = C ]
   #endif
  
  ERROR: test-cases-abc.t#B output changed
  !.
  Failed test-cases-abc.t#B: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Test TESTCASE variable

  $ cat > test-cases-ab.t <<'EOF'
  >   $ dostuff() {
  >   >   echo "In case $TESTCASE"
  >   > }
  > #testcases A B
  > #if A
  >   $ dostuff
  >   In case A
  > #endif
  > #if B
  >   $ dostuff
  >   In case B
  > #endif
  > EOF
  $ rt test-cases-ab.t
  running 2 tests using 1 parallel processes 
  ..
  # Ran 2 tests, 0 skipped, 0 failed.

Support running a specific test case

  $ rt "test-cases-abc.t#B"
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
  @@ -7,7 +7,7 @@
     $ V=C
   #endif
     $ echo $V | sed 's/A/C/'
  -  C
  +  B
   #if C
     $ [ $V = C ]
   #endif
  
  ERROR: test-cases-abc.t#B output changed
  !
  Failed test-cases-abc.t#B: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Support running multiple test cases in the same file

  $ rt test-cases-abc.t#B test-cases-abc.t#C
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
  @@ -7,7 +7,7 @@
     $ V=C
   #endif
     $ echo $V | sed 's/A/C/'
  -  C
  +  B
   #if C
     $ [ $V = C ]
   #endif
  
  ERROR: test-cases-abc.t#B output changed
  !.
  Failed test-cases-abc.t#B: output changed
  # Ran 2 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Support ignoring invalid test cases

  $ rt test-cases-abc.t#B test-cases-abc.t#D
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
  @@ -7,7 +7,7 @@
     $ V=C
   #endif
     $ echo $V | sed 's/A/C/'
  -  C
  +  B
   #if C
     $ [ $V = C ]
   #endif
  
  ERROR: test-cases-abc.t#B output changed
  !
  Failed test-cases-abc.t#B: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Support running complex test cases names

  $ cat > test-cases-advanced-cases.t <<'EOF'
  > #testcases simple case-with-dashes casewith_-.chars
  >   $ echo $TESTCASE
  >   simple
  > EOF

  $ cat test-cases-advanced-cases.t
  #testcases simple case-with-dashes casewith_-.chars
    $ echo $TESTCASE
    simple

  $ rt test-cases-advanced-cases.t
  running 3 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err
  @@ -1,3 +1,3 @@
   #testcases simple case-with-dashes casewith_-.chars
     $ echo $TESTCASE
  -  simple
  +  case-with-dashes
  
  ERROR: test-cases-advanced-cases.t#case-with-dashes output changed
  !
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#casewith_-.chars.err
  @@ -1,3 +1,3 @@
   #testcases simple case-with-dashes casewith_-.chars
     $ echo $TESTCASE
  -  simple
  +  casewith_-.chars
  
  ERROR: test-cases-advanced-cases.t#casewith_-.chars output changed
  !.
  Failed test-cases-advanced-cases.t#case-with-dashes: output changed
  Failed test-cases-advanced-cases.t#casewith_-.chars: output changed
  # Ran 3 tests, 0 skipped, 2 failed.
  python hash seed: * (glob)
  [1]

  $ rt "test-cases-advanced-cases.t#case-with-dashes"
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err
  @@ -1,3 +1,3 @@
   #testcases simple case-with-dashes casewith_-.chars
     $ echo $TESTCASE
  -  simple
  +  case-with-dashes
  
  ERROR: test-cases-advanced-cases.t#case-with-dashes output changed
  !
  Failed test-cases-advanced-cases.t#case-with-dashes: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

  $ rt "test-cases-advanced-cases.t#casewith_-.chars"
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
  +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#casewith_-.chars.err
  @@ -1,3 +1,3 @@
   #testcases simple case-with-dashes casewith_-.chars
     $ echo $TESTCASE
  -  simple
  +  casewith_-.chars
  
  ERROR: test-cases-advanced-cases.t#casewith_-.chars output changed
  !
  Failed test-cases-advanced-cases.t#casewith_-.chars: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

Test automatic pattern replacement
==================================

  $ cat << EOF >> common-pattern.py
  > substitutions = [
  >     (br'foo-(.*)\\b',
  >      br'\$XXX=\\1\$'),
  >     (br'bar\\n',
  >      br'\$YYY$\\n'),
  > ]
  > EOF

  $ cat << EOF >> test-substitution.t
  >   $ echo foo-12
  >   \$XXX=12$
  >   $ echo foo-42
  >   \$XXX=42$
  >   $ echo bar prior
  >   bar prior
  >   $ echo lastbar
  >   last\$YYY$
  >   $ echo foo-bar foo-baz
  > EOF

  $ rt test-substitution.t
  running 1 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-substitution.t
  +++ $TESTTMP/anothertests/cases/test-substitution.t.err
  @@ -7,3 +7,4 @@
     $ echo lastbar
     last$YYY$
     $ echo foo-bar foo-baz
  +  $XXX=bar foo-baz$
  
  ERROR: test-substitution.t output changed
  !
  Failed test-substitution.t: output changed
  # Ran 1 tests, 0 skipped, 1 failed.
  python hash seed: * (glob)
  [1]

--extra-config-opt works

  $ cat << EOF >> test-config-opt.t
  >   $ hg init test-config-opt
  >   $ hg -R test-config-opt purge
  >   $ echo "HGTESTEXTRAEXTENSIONS: \$HGTESTEXTRAEXTENSIONS"
  >   HGTESTEXTRAEXTENSIONS: purge
  > EOF

  $ rt --extra-config-opt extensions.purge= \
  >    --extra-config-opt not.an.extension=True test-config-opt.t
  running 1 tests using 1 parallel processes 
  .
  # Ran 1 tests, 0 skipped, 0 failed.

Test conditional output matching
================================

  $ cat << EOF >> test-conditional-matching.t
  > #testcases foo bar
  >   $ echo richtig
  >   richtig (true !)
  >   $ echo falsch
  >   falsch (false !)
  > #if foo
  >   $ echo arthur
  >   arthur (bar !)
  > #endif
  >   $ echo celeste
  >   celeste (foo !)
  >   $ echo zephir
  >   zephir (bar !)
  > EOF

  $ rt test-conditional-matching.t
  running 2 tests using 1 parallel processes 
  
  --- $TESTTMP/anothertests/cases/test-conditional-matching.t
  +++ $TESTTMP/anothertests/cases/test-conditional-matching.t#bar.err
  @@ -3,11 +3,13 @@
     richtig (true !)
     $ echo falsch
     falsch (false !)
  +  falsch
   #if foo
     $ echo arthur
     arthur \(bar !\) (re)
   #endif
     $ echo celeste
     celeste \(foo !\) (re)
  +  celeste
     $ echo zephir
     zephir \(bar !\) (re)
  
  ERROR: test-conditional-matching.t#bar output changed
  !
  --- $TESTTMP/anothertests/cases/test-conditional-matching.t
  +++ $TESTTMP/anothertests/cases/test-conditional-matching.t#foo.err
  @@ -3,11 +3,14 @@
     richtig (true !)
     $ echo falsch
     falsch (false !)
  +  falsch
   #if foo
     $ echo arthur
     arthur \(bar !\) (re)
  +  arthur
   #endif
     $ echo celeste
     celeste \(foo !\) (re)
     $ echo zephir
     zephir \(bar !\) (re)
  +  zephir
  
  ERROR: test-conditional-matching.t#foo output changed
  !
  Failed test-conditional-matching.t#bar: output changed
  Failed test-conditional-matching.t#foo: output changed
  # Ran 2 tests, 0 skipped, 2 failed.
  python hash seed: * (glob)
  [1]