annotate tests/hypothesishelpers.py @ 35782:9d249f3de730

wireproto: don't compress errors from getbundle() Errors should be small. There's no real need to compress them. Truth be told, there's no good reason to not compress them either. But leaving them uncompressed makes it easier to test failures by looking at the raw HTTP response. This makes it easier for us to write tests. It may make it easier for people writing their own clients. Differential Revision: https://phab.mercurial-scm.org/D1922
author Gregory Szorc <gregory.szorc@gmail.com>
date Sat, 20 Jan 2018 15:43:02 -0800
parents 8699c89f3ae9
children 2372284d9457
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
1 # Helper module to use the Hypothesis tool in tests
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
2 #
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
3 # Copyright 2015 David R. MacIver
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
4 #
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
5 # For details see http://hypothesis.readthedocs.org
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
6
28728
8699c89f3ae9 py3: use print_function in hypothesishelpers.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 28727
diff changeset
7 from __future__ import absolute_import, print_function
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
8 import os
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
9 import sys
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
10 import traceback
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
11
27998
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
12 try:
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
13 # hypothesis 2.x
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
14 from hypothesis.configuration import set_hypothesis_home_dir
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
15 from hypothesis import settings
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
16 except ImportError:
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
17 # hypothesis 1.x
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
18 from hypothesis.settings import set_hypothesis_home_dir
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
19 from hypothesis import Settings as settings
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
20 import hypothesis.strategies as st
27998
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
21 from hypothesis import given
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
22
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
23 # hypothesis store data regarding generate example and code
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
24 set_hypothesis_home_dir(os.path.join(
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
25 os.getenv('TESTTMP'), ".hypothesis"
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
26 ))
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
27
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
28 def check(*args, **kwargs):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
29 """decorator to make a function a hypothesis test
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
30
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
31 Decorated function are run immediately (to be used doctest style)"""
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
32 def accept(f):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
33 # Workaround for https://github.com/DRMacIver/hypothesis/issues/206
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
34 # Fixed in version 1.13 (released 2015 october 29th)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
35 f.__module__ = '__anon__'
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
36 try:
27999
b30b40804a3f tests: pass settings of hypothesis by with statement
Yuya Nishihara <yuya@tcha.org>
parents: 27998
diff changeset
37 with settings(max_examples=2000):
b30b40804a3f tests: pass settings of hypothesis by with statement
Yuya Nishihara <yuya@tcha.org>
parents: 27998
diff changeset
38 given(*args, **kwargs)(f)()
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
39 except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
40 traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
41 sys.exit(1)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
42 return accept
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
43
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
44
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
45 def roundtrips(data, decode, encode):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
46 """helper to tests function that must do proper encode/decode roundtripping
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
47 """
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
48 @given(data)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
49 def testroundtrips(value):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
50 encoded = encode(value)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
51 decoded = decode(encoded)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
52 if decoded != value:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
53 raise ValueError(
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
54 "Round trip failed: %s(%r) -> %s(%r) -> %r" % (
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
55 encode.__name__, value, decode.__name__, encoded,
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
56 decoded
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
57 ))
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
58 try:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
59 testroundtrips()
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
60 except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
61 # heredoc swallow traceback, we work around it
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
62 traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
63 raise
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
64 print("Round trip OK")
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
65
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
66
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
67 # strategy for generating bytestring that might be an issue for Mercurial
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
68 bytestrings = (
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
69 st.builds(lambda s, e: s.encode(e), st.text(), st.sampled_from([
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
70 'utf-8', 'utf-16',
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
71 ]))) | st.binary()