Mercurial > hg
annotate tests/test-run-tests.py @ 39561:d06834e0f48e
wireprotov2peer: stream decoded responses
Previously, wire protocol version 2 would buffer all response data.
Only once all data was received did we CBOR decode it and resolve
the future associated with the command. This was obviously not
desirable. In future commits that introduce large response payloads,
this caused significant memory bloat and slowed down client
operations due to waiting on the server.
This commit refactors the response handling code so that response
data can be streamed.
Command response objects now contain a buffered CBOR decoder. As
new data arrives, it is fed into the decoder. Decoded objects are
made available to the generator as they are decoded.
Because there is a separate thread processing incoming frames and
feeding data into the response object, there is the potential for
race conditions when mutating response objects. So a lock has been
added to guard access to critical state variables.
Because the generator emitting decoded objects needs to wait on
those objects to become available, we've added an Event for the
generator to wait on so it doesn't busy loop. This does mean
there is the potential for deadlocks. And I'm pretty sure they can
occur in some scenarios. We already have a handful of TODOs around
this. But I've added some more. Fixing this will likely require
moving the background thread receiving frames into clienthandler.
We likely would have done this anyway when implementing the client
bits for the SSH transport.
Test output changes because the initial CBOR map holding the overall
response state is now always handled internally by the response
object.
Differential Revision: https://phab.mercurial-scm.org/D4474
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Wed, 29 Aug 2018 15:17:11 -0700 |
parents | f83600efa1ca |
children | aaad36b88298 |
rev | line source |
---|---|
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
1 """test line matching with some failing examples and some which warn |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
2 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
3 run-test.t only checks positive matches and can not see warnings |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
4 (both by design) |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
5 """ |
28917
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
6 from __future__ import absolute_import, print_function |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
7 |
28917
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
8 import doctest |
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
9 import os |
f798ffe7cb08
tests: make test-run-tests use absolute_import
Pulkit Goyal <7895pulkit@gmail.com>
parents:
25061
diff
changeset
|
10 import re |
20284
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
11 # this is hack to make sure no escape characters are inserted into the output |
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
12 if 'TERM' in os.environ: |
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
13 del os.environ['TERM'] |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
14 run_tests = __import__('run-tests') |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
15 |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
16 def prn(ex): |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
17 m = ex.args[0] |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
18 if isinstance(m, str): |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
19 print(m) |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
20 else: |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
21 print(m.decode('utf-8')) |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
22 |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
23 def lm(expected, output): |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
24 r"""check if output matches expected |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
25 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
26 does it generally work? |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
27 >>> lm(b'H*e (glob)\n', b'Here\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
28 True |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
29 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
30 fail on bad test data |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
31 >>> try: lm(b'a\n',b'a') |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
32 ... except AssertionError as ex: print(ex) |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
33 missing newline |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
34 >>> try: lm(b'single backslash\n', b'single \backslash\n') |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
35 ... except AssertionError as ex: prn(ex) |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
36 single backslash or unknown char |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
37 """ |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
38 assert (expected.endswith(b'\n') |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
39 and output.endswith(b'\n')), 'missing newline' |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
40 assert not re.search(br'[^ \w\\/\r\n()*?]', expected + output), \ |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
41 b'single backslash or unknown char' |
33710
2e43c5cd57a7
tests: fix up test-run-tests failures on Python 3.6
Augie Fackler <augie@google.com>
parents:
33695
diff
changeset
|
42 test = run_tests.TTest(b'test-run-test.t', b'.', b'.') |
38554
f83600efa1ca
tests: don't allow reodering of glob/re lines across non-glob/re lines
Martin von Zweigbergk <martinvonz@google.com>
parents:
35382
diff
changeset
|
43 match, exact = test.linematch(expected, output) |
20273
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
44 if isinstance(match, str): |
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
45 return 'special: ' + match |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
46 elif isinstance(match, bytes): |
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
47 return 'special: ' + match.decode('utf-8') |
20273
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
48 else: |
d9d6cbbeef0d
run-tests: suggest to append glob when only path sep does not match
Simon Heimberg <simohe@besonet.ch>
parents:
20271
diff
changeset
|
49 return bool(match) # do not return match object |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
50 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
51 def wintests(): |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
52 r"""test matching like running on windows |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
53 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
54 enable windows matching on any os |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
55 >>> _osaltsep = os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
56 >>> os.altsep = True |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
57 >>> _osname = os.name |
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
58 >>> os.name = 'nt' |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
59 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
60 valid match on windows |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
61 >>> lm(b'g/a*/d (glob)\n', b'g\\abc/d\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
62 True |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
63 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
64 direct matching, glob unnecessary |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
65 >>> lm(b'g/b (glob)\n', b'g/b\n') |
20274
7a259dfe24f7
run-tests: print more information on unnecessary glob matching
Simon Heimberg <simohe@besonet.ch>
parents:
20273
diff
changeset
|
66 'special: -glob' |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
67 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
68 missing glob |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
69 >>> lm(b'/g/c/d/fg\n', b'\\g\\c\\d/fg\n') |
35382
dfae14354660
run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents:
35381
diff
changeset
|
70 True |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
71 >>> lm(b'/g/c/d/fg\n', b'\\g\\c\\d\\fg\r\n') |
35382
dfae14354660
run-tests: accept '\' vs '/' path differences without '(glob)'
Matt Harbison <matt_harbison@yahoo.com>
parents:
35381
diff
changeset
|
72 True |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
73 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
74 restore os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
75 >>> os.altsep = _osaltsep |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
76 >>> os.name = _osname |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
77 """ |
20284
e1e6ddaef299
tests: fix test-run-tests.py on OS X
Simon Heimberg <simohe@besonet.ch>
parents:
20274
diff
changeset
|
78 pass |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
79 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
80 def otherostests(): |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
81 r"""test matching like running on non-windows os |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
82 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
83 disable windows matching on any os |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
84 >>> _osaltsep = os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
85 >>> os.altsep = False |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
86 >>> _osname = os.name |
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
87 >>> os.name = 'nt' |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
88 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
89 backslash does not match slash |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
90 >>> lm(b'h/a* (glob)\n', b'h\\ab\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
91 False |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
92 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
93 direct matching glob can not be recognized |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
94 >>> lm(b'h/b (glob)\n', b'h/b\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
95 True |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
96 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
97 missing glob can not not be recognized |
25061
625dd917f04f
test-run-tests: fix for Python 3.5
Augie Fackler <augie@google.com>
parents:
21315
diff
changeset
|
98 >>> lm(b'/h/c/df/g/\n', b'\\h/c\\df/g\\\n') |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
99 False |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
100 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
101 restore os.altsep |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
102 >>> os.altsep = _osaltsep |
35381
14fd435763ee
run-tests: suggest a (glob) for os.path.sep mismatches with '\r\n' EOL too
Matt Harbison <matt_harbison@yahoo.com>
parents:
33710
diff
changeset
|
103 >>> os.name = _osname |
20271
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
104 """ |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
105 pass |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
106 |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
107 if __name__ == '__main__': |
4453d08a616a
tests: new test for line matching functions in run-tests
Simon Heimberg <simohe@besonet.ch>
parents:
diff
changeset
|
108 doctest.testmod() |