annotate tests/hypothesishelpers.py @ 48642:009e86022a9d

test-http-bad-server: use the new pattern-reading for a test-case This test case is now less sensitive to change of unrelated bits of the client/server exchange. Since this introduce some churn in the output, we do it independently for each test cases. This patch is the last of such changes, for both sent and recv cases. Differential Revision: https://phab.mercurial-scm.org/D12073
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Fri, 21 Jan 2022 19:57:47 +0100
parents 89a2afe31e82
children 6000f5b25c9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
1 # Helper module to use the Hypothesis tool in tests
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
2 #
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
3 # Copyright 2015 David R. MacIver
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
4 #
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
5 # For details see http://hypothesis.readthedocs.org
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
6
28728
8699c89f3ae9 py3: use print_function in hypothesishelpers.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 28727
diff changeset
7 from __future__ import absolute_import, print_function
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
8 import os
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
9 import sys
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
10 import traceback
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
11
27998
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
12 try:
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
13 # hypothesis 2.x
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
14 from hypothesis.configuration import set_hypothesis_home_dir
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
15 from hypothesis import settings
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
16 except ImportError:
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
17 # hypothesis 1.x
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
18 from hypothesis.settings import set_hypothesis_home_dir
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
19 from hypothesis import Settings as settings
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
20 import hypothesis.strategies as st
27998
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
21 from hypothesis import given
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
22
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
23 # hypothesis store data regarding generate example and code
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
24 set_hypothesis_home_dir(os.path.join(os.getenv('TESTTMP'), ".hypothesis"))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
25
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
26
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
27 def check(*args, **kwargs):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
28 """decorator to make a function a hypothesis test
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
29
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
30 Decorated function are run immediately (to be used doctest style)"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
31
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
32 def accept(f):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
33 # Workaround for https://github.com/DRMacIver/hypothesis/issues/206
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
34 # Fixed in version 1.13 (released 2015 october 29th)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
35 f.__module__ = '__anon__'
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
36 try:
27999
b30b40804a3f tests: pass settings of hypothesis by with statement
Yuya Nishihara <yuya@tcha.org>
parents: 27998
diff changeset
37 with settings(max_examples=2000):
b30b40804a3f tests: pass settings of hypothesis by with statement
Yuya Nishihara <yuya@tcha.org>
parents: 27998
diff changeset
38 given(*args, **kwargs)(f)()
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
39 except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
40 traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
41 sys.exit(1)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
42
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
43 return accept
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
44
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
45
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
46 def roundtrips(data, decode, encode):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43076
diff changeset
47 """helper to tests function that must do proper encode/decode roundtripping"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
48
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
49 @given(data)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
50 def testroundtrips(value):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
51 encoded = encode(value)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
52 decoded = decode(encoded)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
53 if decoded != value:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
54 raise ValueError(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
55 "Round trip failed: %s(%r) -> %s(%r) -> %r"
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
56 % (encode.__name__, value, decode.__name__, encoded, decoded)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
57 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
58
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
59 try:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
60 testroundtrips()
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
61 except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
62 # heredoc swallow traceback, we work around it
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
63 traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
64 raise
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
65 print("Round trip OK")
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
66
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
67
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
68 # strategy for generating bytestring that might be an issue for Mercurial
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
69 bytestrings = (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
70 st.builds(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
71 lambda s, e: s.encode(e),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
72 st.text(),
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43076
diff changeset
73 st.sampled_from(
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43076
diff changeset
74 [
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43076
diff changeset
75 'utf-8',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43076
diff changeset
76 'utf-16',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43076
diff changeset
77 ]
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43076
diff changeset
78 ),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
79 )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 28728
diff changeset
80 ) | st.binary()