contrib/fuzz/pyutil.cc
author Matt Harbison <matt_harbison@yahoo.com>
Thu, 24 Oct 2024 22:47:31 -0400
changeset 52127 fd200f5bcaea
parent 44961 ee5f27d7b9fb
permissions -rw-r--r--
wireprototypes: make `baseprotocolhandler` methods abstract The documentation says it's an abstract base class, so let's enforce it. The `typing.Protocol` class is already an ABC, but it only prevents instantiation if there are abstract attrs that are missing. For example, from `hg debugshell`: >>> from mercurial import wireprototypes >>> x = wireprototypes.baseprotocolhandler() Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: Can't instantiate abstract class baseprotocolhandler with abstract method name >>> class fake(wireprototypes.baseprotocolhandler): ... pass ... >>> x = fake() Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: Can't instantiate abstract class fake with abstract method name That's great, but it doesn't protect against calling non-abstract methods at runtime, rather it depends on the protocol type hint being added to method signatures or class attrs, and then running a type checker to notice when an instance is assigned that doesn't conform to the protocol. We don't widely use type hints yet, and do have a lot of class hierarchy in the repository area, which could lead to surprises like this: >>> class fake(wireprototypes.baseprotocolhandler): ... @property ... def name(self) -> bytes: ... return b'name' ... >>> z = fake() >>> z.client() >>> print(z.client()) None Oops. That was supposed to return `bytes`. So not only is a bad/unexpected value returned, but it's one that violates the type hints (since the base client() method will be annotated to return bytes). With this change, we get: >>> from mercurial import wireprototypes >>> class fake(wireprototypes.baseprotocolhandler): ... @property ... def name(self) -> bytes: ... return b'name' ... >>> x = fake() Traceback (most recent call last): File "<console>", line 1, in <module> TypeError: Can't instantiate abstract class fake with abstract methods addcapabilities, checkperm, client, getargs, getpayload, getprotocaps, mayberedirectstdio So this looks like a reasonable safety harness to me, and lets us catch problems by running the standard tests while the type hints are being added, and pytype is improved. We should probably do this for all Protocol class methods that don't supply a method implementation.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
#include "pyutil.h"
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
43807
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41013
diff changeset
     3
#include <iostream>
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
#include <string>
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
namespace contrib
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
{
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
     9
#if PY_MAJOR_VERSION >= 3
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    10
#define HG_FUZZER_PY3 1
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    11
PyMODINIT_FUNC PyInit_parsers(void);
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    12
#else
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    13
PyMODINIT_FUNC initparsers(void);
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    14
#endif
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    15
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    16
static char cpypath[8192] = "\0";
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
static PyObject *mainmod;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
static PyObject *globals;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
void initpy(const char *cselfpath)
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
{
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    23
#ifdef HG_FUZZER_PY3
44961
ee5f27d7b9fb pyutil: this has taken so long to fix, I'm using 3.8 now
Augie Fackler <augie@google.com>
parents: 43859
diff changeset
    24
	const std::string subdir = "/sanpy/lib/python3.8";
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    25
#else
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
	const std::string subdir = "/sanpy/lib/python2.7";
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    27
#endif
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    28
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    29
	/* HACK ALERT: we need a full Python installation built without
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    30
	   pymalloc and with ASAN, so we dump one in
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
	   $OUT/sanpy/lib/python2.7. This helps us wire that up. */
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
	std::string selfpath(cselfpath);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
	std::string pypath;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
	auto pos = selfpath.rfind("/");
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    35
	if (pos == std::string::npos) {
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
		char wd[8192];
43807
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41013
diff changeset
    37
		if (!getcwd(wd, 8192)) {
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41013
diff changeset
    38
			std::cerr << "Failed to call getcwd: errno " << errno
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41013
diff changeset
    39
			          << std::endl;
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41013
diff changeset
    40
			exit(1);
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41013
diff changeset
    41
		}
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    42
		pypath = std::string(wd) + subdir;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    43
	} else {
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    44
		pypath = selfpath.substr(0, pos) + subdir;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    45
	}
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    46
	strncpy(cpypath, pypath.c_str(), pypath.size());
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    47
	setenv("PYTHONPATH", cpypath, 1);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    48
	setenv("PYTHONNOUSERSITE", "1", 1);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    49
	/* prevent Python from looking up users in the fuzz environment */
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    50
	setenv("PYTHONUSERBASE", cpypath, 1);
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    51
#ifdef HG_FUZZER_PY3
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    52
	std::wstring wcpypath(pypath.begin(), pypath.end());
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    53
	Py_SetPythonHome(wcpypath.c_str());
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    54
#else
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    55
	Py_SetPythonHome(cpypath);
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    56
#endif
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    57
	Py_InitializeEx(0);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    58
	mainmod = PyImport_AddModule("__main__");
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    59
	globals = PyModule_GetDict(mainmod);
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    60
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    61
#ifdef HG_FUZZER_PY3
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    62
	PyObject *mod = PyInit_parsers();
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    63
#else
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    64
	initparsers();
43859
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    65
	PyObject *mod = PyImport_ImportModule("parsers");
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    66
#endif
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    67
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43807
diff changeset
    68
	PyDict_SetItemString(globals, "parsers", mod);
41013
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    69
}
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    70
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    71
PyObject *pyglobals()
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    72
{
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    73
	return globals;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    74
}
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    75
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    76
} // namespace contrib