Mercurial > hg
annotate tests/test-run-tests.py @ 38554:f83600efa1ca
tests: don't allow reodering of glob/re lines across non-glob/re lines
As shown in the test case added in the previous patch, it can be
really hard to interpret diffs from the test runner if there are
multiple lines that would match a given glob or regular expression. It
looks like this has been the case since 1ad0ddf8cccc (run-tests: teach
_processoutput to handle multiple lines of churn, 2016-03-17). It
seems like the point of that was to preserve the "(glob)" annotation
on lines even if they got moved. This patch tries to preserve that but
only allows the lines to be moved past other glob/re lines.
Differential Revision: https://phab.mercurial-scm.org/D3881
author | Martin von Zweigbergk <martinvonz@google.com> |
---|---|
date | Mon, 02 Jul 2018 11:14:13 -0700 |
parents | dfae14354660 |
children | aaad36b88298 |
rev | line source |
---|---|
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
1 """test line matching with some failing examples and some which warn |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
2 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
3 run-test.t only checks positive matches and can not see warnings |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
4 (both by design) |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
5 """ |
28917
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
6 from __future__ import absolute_import, print_function |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
7 |
28917
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
8 import doctest |
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
9 import os |
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
10 import re |
20284
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
11 # this is hack to make sure no escape characters are inserted into the output |
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
12 if 'TERM' in os.environ: |
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
13 del os.environ['TERM'] |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
14 run_tests = __import__('run-tests') |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
15 |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
16 def prn(ex): |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
17 m = ex.args[0] |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
18 if isinstance(m, str): |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
19 print(m) |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
20 else: |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
21 print(m.decode('utf-8')) |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
22 |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
23 def lm(expected, output): |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
24 r"""check if output matches expected |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
25 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
26 does it generally work? |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
27 >>> lm(b'H*e (glob)\n', b'Here\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
28 True |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
29 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
30 fail on bad test data |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
31 >>> try: lm(b'a\n',b'a') |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
32 ... except AssertionError as ex: print(ex) |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
33 missing newline |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
34 >>> try: lm(b'single backslash\n', b'single \backslash\n') |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
35 ... except AssertionError as ex: prn(ex) |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
36 single backslash or unknown char |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
37 """ |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
38 assert (expected.endswith(b'\n') |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
39 and output.endswith(b'\n')), 'missing newline' |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
40 assert not re.search(br'[^ \w\\/\r\n()*?]', expected + output), \ |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
41 b'single backslash or unknown char' |
33710
2e43c5cd57a7
tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents:
33695
diff
changeset
|
42 test = run_tests.TTest(b'test-run-test.t', b'.', b'.') |
38554
f83600efa1ca
tests: don't allow reodering of glob/re lines across non-glob/re lines
Martin von Zweigbergk <martinvonz@google.com>
parents:
35382
diff
changeset
|
43 match, exact = test.linematch(expected, output) |
20273
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
44 if isinstance(match, str): |
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
45 return 'special: ' + match |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
46 elif isinstance(match, bytes): |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
47 return 'special: ' + match.decode('utf-8') |
20273
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
48 else: |
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
49 return bool(match) # do not return match object |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
50 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
51 def wintests(): |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
52 r"""test matching like running on windows |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
53 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
54 enable windows matching on any os |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
55 >>> _osaltsep = os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
56 >>> os.altsep = True |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
57 >>> _osname = os.name |
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
58 >>> os.name = 'nt' |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
59 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
60 valid match on windows |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
61 >>> lm(b'g/a*/d (glob)\n', b'g\\abc/d\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
62 True |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
63 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
64 direct matching, glob unnecessary |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
65 >>> lm(b'g/b (glob)\n', b'g/b\n') |
20274
7a259dfe24f7
run-tests: print more information on unnecessary glob matching
Simon Heimberg <simohe@besonet.ch>
parents:
20273
diff
changeset
|
66 'special: -glob' |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
67 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
68 missing glob |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
69 >>> lm(b'/g/c/d/fg\n', b'\\g\\c\\d/fg\n') |
35382
dfae14354660
run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents:
35381
diff
changeset
|
70 True |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
71 >>> lm(b'/g/c/d/fg\n', b'\\g\\c\\d\\fg\r\n') |
35382
dfae14354660
run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents:
35381
diff
changeset
|
72 True |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
73 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
74 restore os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
75 >>> os.altsep = _osaltsep |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
76 >>> os.name = _osname |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
77 """ |
20284
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
78 pass |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
79 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
80 def otherostests(): |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
81 r"""test matching like running on non-windows os |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
82 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
83 disable windows matching on any os |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
84 >>> _osaltsep = os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
85 >>> os.altsep = False |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
86 >>> _osname = os.name |
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
87 >>> os.name = 'nt' |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
88 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
89 backslash does not match slash |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
90 >>> lm(b'h/a* (glob)\n', b'h\\ab\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
91 False |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
92 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
93 direct matching glob can not be recognized |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
94 >>> lm(b'h/b (glob)\n', b'h/b\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
95 True |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
96 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
97 missing glob can not not be recognized |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
98 >>> lm(b'/h/c/df/g/\n', b'\\h/c\\df/g\\\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
99 False |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
100 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
101 restore os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
102 >>> os.altsep = _osaltsep |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
103 >>> os.name = _osname |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
104 """ |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
105 pass |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
106 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
107 if __name__ == '__main__': |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
108 doctest.testmod() |