annotate tests/test-run-tests.py @ 20274:7a259dfe24f7

run-tests: print more information on unnecessary glob matching Extend the message with the test name and the approximate line number. (The line number is the one of the command producing the output.) Finding the line to fix is easier now. old message: ...... Info, unnecessary glob: at a/b/c (glob) .. new message: ...... Info, unnecessary glob in test-example.t (after line 9): at a/b/c (glob) .. The test result is still pass as before.
author Simon Heimberg <simohe@besonet.ch>
date Thu, 16 Jan 2014 12:08:57 +0100
parents d9d6cbbeef0d
children e1e6ddaef299
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
1 """test line matching with some failing examples and some which warn
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
2
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
3 run-test.t only checks positive matches and can not see warnings
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
4 (both by design)
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
5 """
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
6
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
7
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
8 import doctest, os, re
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
9 run_tests = __import__('run-tests')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
10
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
11 def lm(expected, output):
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
12 r"""check if output matches expected
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
13
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
14 does it generally work?
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
15 >>> lm('H*e (glob)\n', 'Here\n')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
16 True
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
17
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
18 fail on bad test data
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
19 >>> try: lm('a\n','a')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
20 ... except AssertionError, ex: print ex
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
21 missing newline
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
22 >>> try: lm('single backslash\n', 'single \backslash\n')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
23 ... except AssertionError, ex: print ex
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
24 single backslash or unknown char
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
25 """
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
26 assert expected.endswith('\n') and output.endswith('\n'), 'missing newline'
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
27 assert not re.search(r'[^ \w\\/\r\n()*?]', expected + output), \
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
28 'single backslash or unknown char'
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
29 match = run_tests.linematch(expected, output)
20273
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
30 if isinstance(match, str):
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
31 return 'special: ' + match
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
32 else:
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
33 return bool(match) # do not return match object
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
34
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
35 def wintests():
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
36 r"""test matching like running on windows
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
37
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
38 enable windows matching on any os
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
39 >>> _osaltsep = os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
40 >>> os.altsep = True
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
41
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
42 valid match on windows
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
43 >>> lm('g/a*/d (glob)\n', 'g\\abc/d\n')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
44 True
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
45
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
46 direct matching, glob unnecessary
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
47 >>> lm('g/b (glob)\n', 'g/b\n')
20274
7a259dfe24f7 run-tests: print more information on unnecessary glob matching
Simon Heimberg <simohe@besonet.ch>
parents: 20273
diff changeset
48 'special: -glob'
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
49
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
50 missing glob
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
51 >>> lm('/g/c/d/fg\n', '\\g\\c\\d/fg\n')
20273
d9d6cbbeef0d run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents: 20271
diff changeset
52 'special: +glob'
20271
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
53
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
54 restore os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
55 >>> os.altsep = _osaltsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
56 """
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
57 os.altsep # for pyflakes, because it does not see os in the doctest
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
58
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
59 def otherostests():
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
60 r"""test matching like running on non-windows os
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
61
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
62 disable windows matching on any os
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
63 >>> _osaltsep = os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
64 >>> os.altsep = False
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
65
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
66 backslash does not match slash
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
67 >>> lm('h/a* (glob)\n', 'h\\ab\n')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
68 False
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
69
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
70 direct matching glob can not be recognized
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
71 >>> lm('h/b (glob)\n', 'h/b\n')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
72 True
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
73
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
74 missing glob can not not be recognized
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
75 >>> lm('/h/c/df/g/\n', '\\h/c\\df/g\\\n')
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
76 False
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
77
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
78 restore os.altsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
79 >>> os.altsep = _osaltsep
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
80 """
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
81 pass
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
82
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
83 if __name__ == '__main__':
4453d08a616a tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff changeset
84 doctest.testmod()