annotate tests/hypothesishelpers.py @ 26866:0f75461a9698

test: enforce non-general delta in 'test-generaldelta.t' If general delta becomes the default, we need to be explicit about what we want to be tested.
author Pierre-Yves David <pierre-yves.david@fb.com>
date Sun, 18 Oct 2015 18:49:59 +0200
parents 0f76c64f5cc3
children 84513a4fcc3a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
1 # Helper module to use the Hypothesis tool in tests
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
2 #
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
3 # Copyright 2015 David R. MacIver
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
4 #
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
5 # For details see http://hypothesis.readthedocs.org
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
6
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
7 import os
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
8 import sys
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
9 import traceback
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
10
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
11 from hypothesis.settings import set_hypothesis_home_dir
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
12 import hypothesis.strategies as st
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
13 from hypothesis import given, Settings
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
14
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
15 # hypothesis store data regarding generate example and code
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
16 set_hypothesis_home_dir(os.path.join(
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
17 os.getenv('TESTTMP'), ".hypothesis"
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
18 ))
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
19
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
20 def check(*args, **kwargs):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
21 """decorator to make a function a hypothesis test
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
22
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
23 Decorated function are run immediately (to be used doctest style)"""
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
24 def accept(f):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
25 # Workaround for https://github.com/DRMacIver/hypothesis/issues/206
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
26 # Fixed in version 1.13 (released 2015 october 29th)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
27 f.__module__ = '__anon__'
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
28 try:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
29 given(*args, settings=Settings(max_examples=2000), **kwargs)(f)()
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
30 except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
31 traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
32 sys.exit(1)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
33 return accept
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
34
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
35
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
36 def roundtrips(data, decode, encode):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
37 """helper to tests function that must do proper encode/decode roundtripping
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
38 """
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
39 @given(data)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
40 def testroundtrips(value):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
41 encoded = encode(value)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
42 decoded = decode(encoded)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
43 if decoded != value:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
44 raise ValueError(
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
45 "Round trip failed: %s(%r) -> %s(%r) -> %r" % (
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
46 encode.__name__, value, decode.__name__, encoded,
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
47 decoded
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
48 ))
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
49 try:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
50 testroundtrips()
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
51 except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
52 # heredoc swallow traceback, we work around it
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
53 traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
54 raise
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
55 print("Round trip OK")
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
56
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
57
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
58 # strategy for generating bytestring that might be an issue for Mercurial
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
59 bytestrings = (
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
60 st.builds(lambda s, e: s.encode(e), st.text(), st.sampled_from([
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
61 'utf-8', 'utf-16',
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
62 ]))) | st.binary()