contrib/fuzz/pyutil.cc
author Matt Harbison <matt_harbison@yahoo.com>
Tue, 08 Dec 2020 12:43:18 -0500
changeset 46098 1b5e0d0bdb05
parent 44996 ee5f27d7b9fb
permissions -rw-r--r--
hghave: update the check for virtualenv This started as `hghave --test-features` failing on Windows in `test-hghave.t`. IDK how this worked, as neither my Linux nor Windows machines have the old attribute with virtualenv 20.2.2, even on py2. I think this was noticed recently because 357d8415aa27 mentioned an AttributeError, and mitigated by making this py2 only. But as mentioned, this is also a problem on py2 (where the failure was observed). When I got this working by removing the attribute reference, the command in the test failed because the `--no-site-package` argument was removed some time ago. Therefore, this backs out 357d8415aa27 and references a known good attribute (which was done to suppress the warning about an unused import) that also ensures the command does not need the argument. Since there appears to be (minor) broken stuff on py3, manually apply the `no-py3` guard that was backed out of the check itself. Differential Revision: https://phab.mercurial-scm.org/D9547
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
#include "pyutil.h"
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
43825
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41023
diff changeset
     3
#include <iostream>
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
#include <string>
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
namespace contrib
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
{
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
     9
#if PY_MAJOR_VERSION >= 3
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    10
#define HG_FUZZER_PY3 1
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    11
PyMODINIT_FUNC PyInit_parsers(void);
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    12
#else
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    13
PyMODINIT_FUNC initparsers(void);
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    14
#endif
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    15
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    16
static char cpypath[8192] = "\0";
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
static PyObject *mainmod;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
static PyObject *globals;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
void initpy(const char *cselfpath)
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
{
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    23
#ifdef HG_FUZZER_PY3
44996
ee5f27d7b9fb pyutil: this has taken so long to fix, I'm using 3.8 now
Augie Fackler <augie@google.com>
parents: 43870
diff changeset
    24
	const std::string subdir = "/sanpy/lib/python3.8";
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    25
#else
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
	const std::string subdir = "/sanpy/lib/python2.7";
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    27
#endif
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    28
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    29
	/* HACK ALERT: we need a full Python installation built without
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    30
	   pymalloc and with ASAN, so we dump one in
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
	   $OUT/sanpy/lib/python2.7. This helps us wire that up. */
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
	std::string selfpath(cselfpath);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
	std::string pypath;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
	auto pos = selfpath.rfind("/");
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    35
	if (pos == std::string::npos) {
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
		char wd[8192];
43825
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41023
diff changeset
    37
		if (!getcwd(wd, 8192)) {
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41023
diff changeset
    38
			std::cerr << "Failed to call getcwd: errno " << errno
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41023
diff changeset
    39
			          << std::endl;
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41023
diff changeset
    40
			exit(1);
c78f8f0720cc fuzz: fix an unused result on getcwd() in pyutil
Augie Fackler <augie@google.com>
parents: 41023
diff changeset
    41
		}
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    42
		pypath = std::string(wd) + subdir;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    43
	} else {
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    44
		pypath = selfpath.substr(0, pos) + subdir;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    45
	}
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    46
	strncpy(cpypath, pypath.c_str(), pypath.size());
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    47
	setenv("PYTHONPATH", cpypath, 1);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    48
	setenv("PYTHONNOUSERSITE", "1", 1);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    49
	/* prevent Python from looking up users in the fuzz environment */
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    50
	setenv("PYTHONUSERBASE", cpypath, 1);
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    51
#ifdef HG_FUZZER_PY3
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    52
	std::wstring wcpypath(pypath.begin(), pypath.end());
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    53
	Py_SetPythonHome(wcpypath.c_str());
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    54
#else
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    55
	Py_SetPythonHome(cpypath);
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    56
#endif
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    57
	Py_InitializeEx(0);
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    58
	mainmod = PyImport_AddModule("__main__");
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    59
	globals = PyModule_GetDict(mainmod);
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    60
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    61
#ifdef HG_FUZZER_PY3
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    62
	PyObject *mod = PyInit_parsers();
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    63
#else
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    64
	initparsers();
43870
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    65
	PyObject *mod = PyImport_ImportModule("parsers");
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    66
#endif
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    67
8766728dbce6 fuzz: add support for fuzzing under either Python 2 or 3
Augie Fackler <augie@google.com>
parents: 43825
diff changeset
    68
	PyDict_SetItemString(globals, "parsers", mod);
41023
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    69
}
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    70
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    71
PyObject *pyglobals()
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    72
{
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    73
	return globals;
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    74
}
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    75
ef103c96ed33 fuzz: extract Python initialization to utility package
Augie Fackler <augie@google.com>
parents:
diff changeset
    76
} // namespace contrib