tests/test-fuzz-targets.t
author Matt Harbison <matt_harbison@yahoo.com>
Mon, 21 Aug 2023 16:53:16 -0400
changeset 50946 4135e7271b9e
parent 47063 1d075b857c90
permissions -rw-r--r--
debugdata: migrate `opts` to native kwargs
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
     1
#require test-repo py3
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
     2
35670
2b9e2415f5b5 contrib: add some basic scaffolding for some fuzz test targets
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
  $ cd $TESTDIR/../contrib/fuzz
43815
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
     4
  $ OUT=$TESTTMP ; export OUT
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
     5
40726
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     6
which(1) could exit nonzero, but that's fine because we'll still end
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     7
up without a valid executable, so we don't need to check $? here.
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     8
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
     9
  $ if which gmake >/dev/null 2>&1; then
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    10
  >     MAKE=gmake
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    11
  > else
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    12
  >     MAKE=make
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    13
  > fi
6c01fad8de32 tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 38246
diff changeset
    14
40727
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    15
  $ havefuzz() {
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    16
  >     cat > $TESTTMP/dummy.cc <<EOF
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    17
  > #include <stdlib.h>
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    18
  > #include <stdint.h>
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    19
  > int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { return 0; }
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    20
  > int main(int argc, char **argv) {
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    21
  >     const char data[] = "asdf";
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    22
  >     return LLVMFuzzerTestOneInput((const uint8_t *)data, 4);
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    23
  > }
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    24
  > EOF
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    25
  >     $CXX $TESTTMP/dummy.cc -o $TESTTMP/dummy \
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    26
  >        -fsanitize=fuzzer-no-link,address || return 1
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    27
  > }
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    28
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    29
Try to find a python3-config that's next to our sys.executable. If
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    30
that doesn't work, fall back to looking for a global python3-config
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    31
and hope that works out for the best.
47063
1d075b857c90 tests: ensure `$PYTHON` is quoted for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 46257
diff changeset
    32
  $ PYBIN=`"$PYTHON" -c 'import sys, os; print(os.path.dirname(sys.executable))'`
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    33
  $ if [ -x "$PYBIN/python3-config" ] ; then
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    34
  >   PYTHON_CONFIG="$PYBIN/python3-config"
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    35
  > else
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    36
  >   PYTHON_CONFIG="`which python3-config`"
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    37
  > fi
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    38
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    39
#if clang-libfuzzer
40727
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    40
  $ CXX=clang++ havefuzz || exit 80
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    41
  $ $MAKE -s clean all PYTHON_CONFIG="$PYTHON_CONFIG"
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    42
#endif
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    43
#if no-clang-libfuzzer clang-6.0
40727
d9fc51f77cc5 tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40726
diff changeset
    44
  $ CXX=clang++-6.0 havefuzz || exit 80
46257
b918494198f7 fuzz: try and ensure fuzzer tests run against the right python-config
Augie Fackler <augie@google.com>
parents: 43815
diff changeset
    45
  $ $MAKE -s clean all CC=clang-6.0 CXX=clang++-6.0 PYTHON_CONFIG="$PYTHON_CONFIG"
38236
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    46
#endif
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    47
#if no-clang-libfuzzer no-clang-6.0
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    48
  $ exit 80
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    49
#endif
a6347ae6168d test-fuzz-targets: look for clang-6.0 binary as well
Yuya Nishihara <yuya@tcha.org>
parents: 38235
diff changeset
    50
43815
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    51
  $ cd $TESTTMP
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    52
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    53
Run each fuzzer using dummy.cc as a fake input, to make sure it runs
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    54
at all. In the future we should instead unpack the corpus for each
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    55
fuzzer and use that instead.
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    56
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    57
  $ for fuzzer in `ls *_fuzzer | sort` ; do
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    58
  >   echo run $fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    59
  >   ./$fuzzer dummy.cc > /dev/null 2>&1 
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    60
  > done
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    61
  run bdiff_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    62
  run dirs_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    63
  run dirstate_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    64
  run fm1readmarkers_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    65
  run fncache_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    66
  run jsonescapeu8fast_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    67
  run manifest_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    68
  run mpatch_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    69
  run revlog_fuzzer...
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    70
  run xdiff_fuzzer...
42948
39cab871e880 tests: clean up built binaries after running test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40727
diff changeset
    71
39cab871e880 tests: clean up built binaries after running test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40727
diff changeset
    72
Clean up.
43815
19da643dc10c tests: finally fix up test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 42948
diff changeset
    73
  $ cd $TESTDIR/../contrib/fuzz
42948
39cab871e880 tests: clean up built binaries after running test-fuzz-targets.t
Augie Fackler <augie@google.com>
parents: 40727
diff changeset
    74
  $ $MAKE -s clean