tests/hypothesishelpers.py
author Gregory Szorc <gregory.szorc@gmail.com>
Thu, 20 Sep 2018 12:57:23 -0700
changeset 39814 d059cb669632
parent 28728 8699c89f3ae9
child 43076 2372284d9457
permissions -rw-r--r--
wireprotov2: allow multiple fields to follow revision maps The *data wire protocol commands emit a series of CBOR values. Because revision/delta data may be large, their data is emitted outside the map as a top-level bytestring value. Before this commit, we'd emit a single optional bytestring value after the revision descriptor map. This got the job done. But it was limiting in that we could only send a single field. And, it required the consumer to know that the presence of a key in the map implied the existence of a following bytestring value. This commit changes the encoding strategy so top-level bytestring values in the stream are explicitly denoted in a "fieldsfollowing" key. This key contains an array defining what fields that follow and the expected size of each field. By defining things this way, we can easily send N bytestring values without any ambiguity about their order. In addition, clients only need to know how to parse ``fieldsfollowing`` to know if extra values are present. Because this breaks backwards compatibility, we've bumped the version number of the wire protocol version 2 API endpoint. Differential Revision: https://phab.mercurial-scm.org/D4620
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     1
# Helper module to use the Hypothesis tool in tests
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     2
#
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     3
# Copyright 2015 David R. MacIver
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     4
#
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     5
# For details see http://hypothesis.readthedocs.org
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     6
28728
8699c89f3ae9 py3: use print_function in hypothesishelpers.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 28727
diff changeset
     7
from __future__ import absolute_import, print_function
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     8
import os
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
     9
import sys
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    10
import traceback
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    11
27998
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    12
try:
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    13
    # hypothesis 2.x
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    14
    from hypothesis.configuration import set_hypothesis_home_dir
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    15
    from hypothesis import settings
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    16
except ImportError:
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    17
    # hypothesis 1.x
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    18
    from hypothesis.settings import set_hypothesis_home_dir
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    19
    from hypothesis import Settings as settings
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    20
import hypothesis.strategies as st
27998
84513a4fcc3a tests: adjust for code move in Hypothesis 2.0.0
Yuya Nishihara <yuya@tcha.org>
parents: 26842
diff changeset
    21
from hypothesis import given
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    22
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    23
# hypothesis store data regarding generate example and code
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    24
set_hypothesis_home_dir(os.path.join(
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    25
    os.getenv('TESTTMP'), ".hypothesis"
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    26
))
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    27
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    28
def check(*args, **kwargs):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    29
    """decorator to make a function a hypothesis test
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    30
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    31
    Decorated function are run immediately (to be used doctest style)"""
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    32
    def accept(f):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    33
        # Workaround for https://github.com/DRMacIver/hypothesis/issues/206
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    34
        # Fixed in version 1.13 (released 2015 october 29th)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    35
        f.__module__ = '__anon__'
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    36
        try:
27999
b30b40804a3f tests: pass settings of hypothesis by with statement
Yuya Nishihara <yuya@tcha.org>
parents: 27998
diff changeset
    37
            with settings(max_examples=2000):
b30b40804a3f tests: pass settings of hypothesis by with statement
Yuya Nishihara <yuya@tcha.org>
parents: 27998
diff changeset
    38
                given(*args, **kwargs)(f)()
26842
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    39
        except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    40
            traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    41
            sys.exit(1)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    42
    return accept
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    43
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    44
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    45
def roundtrips(data, decode, encode):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    46
    """helper to tests function that must do proper encode/decode roundtripping
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    47
    """
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    48
    @given(data)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    49
    def testroundtrips(value):
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    50
        encoded = encode(value)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    51
        decoded = decode(encoded)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    52
        if decoded != value:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    53
            raise ValueError(
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    54
                "Round trip failed: %s(%r) -> %s(%r) -> %r" % (
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    55
                    encode.__name__, value, decode.__name__, encoded,
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    56
                    decoded
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    57
                ))
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    58
    try:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    59
        testroundtrips()
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    60
    except Exception:
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    61
        # heredoc swallow traceback, we work around it
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    62
        traceback.print_exc(file=sys.stdout)
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    63
        raise
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    64
    print("Round trip OK")
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    65
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    66
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    67
# strategy for generating bytestring that might be an issue for Mercurial
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    68
bytestrings = (
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    69
    st.builds(lambda s, e: s.encode(e), st.text(), st.sampled_from([
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    70
        'utf-8', 'utf-16',
0f76c64f5cc3 testing: add hypothesis fuzz testing
David R. MacIver <david@drmaciver.com>
parents:
diff changeset
    71
    ]))) | st.binary()