import-checker: use testparseutil.embedded() to centralize detection logic
This patch fixes issues of embedded() in import-checker.py below, too.
- overlook (or mis-detect) the end of inline script in doctest style
- overlook inline script in doctest style at the end of file
(and ignore invalid un-closed heredoc at the end of file, too)
- overlook code fragment in styles below
- "python <<EOF" (heredoc should be "cat > file <<EOF" style)
- "cat > foobar.py << ANYLIMIT" (limit mark should be "EOF")
- "cat << EOF > foobar.py" (filename should be placed before limit mark)
- "cat >> foobar.py << EOF" (appending is ignored)
tests: use NO_CHECK_EOF as heredoc limit mark to omit checking code fragments
This patch uses NO_CHECK_EOF as heredoc limit mark instead of EOF, in
order to avoid checking all python code fragments in
test-contrib-check-code.t, because almost all of them has
un-recommended implementations intentionally.
contrib: add an utility module to parse test scripts
This patch centralizes the logic to pick up code fragments embedded in
*.t script, in order to:
- apply checking with patterns in check-code.py on such embedded
code fragments
Now, check-code.py completely ignores embedded code
fragments. I'll post another patch series to check them.
- replace similar code path in contrib/import-checker.py
Current import-checker.py has problems below. Fixing each of them
is a little difficult, because parsing logic and pattern strings
are tightly coupled.
- overlook (or mis-detect) the end of inline script in doctest
style
8a8dd6e4a97a fixed a part of this issue, but not enough.
- it overlooks inline script in doctest style at the end of file
(and ignores invalid un-closed heredoc at the end of file, too)
- it overlooks code fragment in styles below
- "python <<EOF" (heredoc should be "cat > file <<EOF" style)
- "cat > foobar.py << ANYLIMIT" (limit mark should be "EOF")
- "cat << EOF > foobar.py" (filename should be placed before limit mark)
- "cat >> foobar.py << EOF" (appending is ignored)
- it is not extensible for other than python code fragments
(e.g. shell script, hgrc file, and so on)
This new module can detect python code fragments in styles below:
- inline script in doctest style (starting by " >>> " line)
- python invocation with heredoc script ("python <<EOF")
- python script in heredoc style (redirected into ".py" file)
As an example of extensibility of new module, this patch also contains
implementation to pick up code fragment below. This will be useful to
add additional restriction for them, for example.
- shell script in heredoc style (redirected into ".sh" file)
- hgrc configuration in heredoc style (redirected into hgrc or $HGRCPATH)
tests: use environment variable indirectly
Using environment variable directly in heredoc python code will cause
syntax error at checking module importation by import-checker.py
strictly, because "$varname" is invalid in Python syntax. "$varname"
becomes valid after environment variable substitution by shell at
writing text into file.
Current import-checker.py overlooks code fragment changed in this
patch, because of a restriction below for a line starting code
fragment.
- filename must be specified before limit mark
NG: cat <<EOF > FILE.py
OK: cat > FILE.py <<EOF
import-checker.py itself is fixed in subsequent patch.
tests: import multiple modules separately
Current import-checker.py overlooks code fragment changed in this
patch, because of restrictions below for a line starting code
fragment.
- filename must be specified before limit mark
NG: cat <<EOF > FILE.py
OK: cat > FILE.py <<EOF
- limit mark must not be quoted
NG: cat > FILE.py <<'EOF'
OK: cat > FILE.py <<EOF
import-checker.py itself is fixed in subsequent patch.
fuzz: allow manifest fuzzer to detect leaks
Huzzah!
Differential Revision: https://phab.mercurial-scm.org/D4907
fuzzers: init Python in LLVMFuzzerInitialize and intentionally leak it
This sidesteps leaks (or "leaks", I'm not sure) in CPython, and lets
our fuzzer work.
Differential Revision: https://phab.mercurial-scm.org/D4906
revlog: if the module is initialized more than once, don't leak nullentry
Caught (annoyingly) by the manifest fuzzer.
Differential Revision: https://phab.mercurial-scm.org/D4905