packaging: builddeb's cleanup needs to expand PWD, safely Single quotes would not expand the variable.

packaging: blindly factor out trap's cleanup function in builddeb This commit blindly extracts builddeb's trap routine in a dedicated function. While doing so, I think two bugs are exposed, which will be addressed in the next commits: - single quoting around '$CLEANUP' will always evaluate to the literal '$CLEANUP' regardless of the variable's value. The "if" will always be true. - the removal operation will not expand $PWD (and a variable expansion would need double quotes, anyways.

packaging: print full path to the packages when builddeb finishes successfully

packaging: print more specific error messages when builddeb fails

cmdutil: sort unresolved paths I noticed that `hg status` was printing unresolved paths in a non-deterministic order. This patch fixes that. I'm not sure if the sorting should be done in merge.mergestate.unresolved() instead. Either way fixes the presentation issue. Differential Revision: https://phab.mercurial-scm.org/D4929

fuzz: report error if Python code raised exception I think that's what we wanted to do, given the most of the code block is surrounded by try-except. 'lazymanifest(mdata)' is moved to the try block as it can fail.

revlog: explicitly initialize static variables I know .bss section is zero-filled, but explicit initialization should be better as we rely on that.

tests: do not change sys.path, it can break loading cext.parsers When running this tests with run-tests, the prefix would resolve mercurial.cext to the source tree and the attempt to load mercurial.cext.parsers would therefore fail since it doesn't exist in it. With the regular search path from run-tests, it is picked up from the temporary prefix correctly. Differential Revision: https://phab.mercurial-scm.org/D4910

tests: deal with differences in tic from ncurses and NetBSD Differential Revision: https://phab.mercurial-scm.org/D4909

closehead: fix close-head -r listification Differential Revision: https://phab.mercurial-scm.org/D4908

import-checker: use testparseutil.embedded() to centralize detection logic This patch fixes issues of embedded() in import-checker.py below, too. - overlook (or mis-detect) the end of inline script in doctest style - overlook inline script in doctest style at the end of file (and ignore invalid un-closed heredoc at the end of file, too) - overlook code fragment in styles below - "python <<EOF" (heredoc should be "cat > file <<EOF" style) - "cat > foobar.py << ANYLIMIT" (limit mark should be "EOF") - "cat << EOF > foobar.py" (filename should be placed before limit mark) - "cat >> foobar.py << EOF" (appending is ignored)

tests: use NO_CHECK_EOF as heredoc limit mark to omit checking code fragments This patch uses NO_CHECK_EOF as heredoc limit mark instead of EOF, in order to avoid checking all python code fragments in test-contrib-check-code.t, because almost all of them has un-recommended implementations intentionally.

contrib: add an utility module to parse test scripts This patch centralizes the logic to pick up code fragments embedded in *.t script, in order to: - apply checking with patterns in check-code.py on such embedded code fragments Now, check-code.py completely ignores embedded code fragments. I'll post another patch series to check them. - replace similar code path in contrib/import-checker.py Current import-checker.py has problems below. Fixing each of them is a little difficult, because parsing logic and pattern strings are tightly coupled. - overlook (or mis-detect) the end of inline script in doctest style 8a8dd6e4a97a fixed a part of this issue, but not enough. - it overlooks inline script in doctest style at the end of file (and ignores invalid un-closed heredoc at the end of file, too) - it overlooks code fragment in styles below - "python <<EOF" (heredoc should be "cat > file <<EOF" style) - "cat > foobar.py << ANYLIMIT" (limit mark should be "EOF") - "cat << EOF > foobar.py" (filename should be placed before limit mark) - "cat >> foobar.py << EOF" (appending is ignored) - it is not extensible for other than python code fragments (e.g. shell script, hgrc file, and so on) This new module can detect python code fragments in styles below: - inline script in doctest style (starting by " >>> " line) - python invocation with heredoc script ("python <<EOF") - python script in heredoc style (redirected into ".py" file) As an example of extensibility of new module, this patch also contains implementation to pick up code fragment below. This will be useful to add additional restriction for them, for example. - shell script in heredoc style (redirected into ".sh" file) - hgrc configuration in heredoc style (redirected into hgrc or $HGRCPATH)

tests: use environment variable indirectly Using environment variable directly in heredoc python code will cause syntax error at checking module importation by import-checker.py strictly, because "$varname" is invalid in Python syntax. "$varname" becomes valid after environment variable substitution by shell at writing text into file. Current import-checker.py overlooks code fragment changed in this patch, because of a restriction below for a line starting code fragment. - filename must be specified before limit mark NG: cat <<EOF > FILE.py OK: cat > FILE.py <<EOF import-checker.py itself is fixed in subsequent patch.

tests: import multiple modules separately Current import-checker.py overlooks code fragment changed in this patch, because of restrictions below for a line starting code fragment. - filename must be specified before limit mark NG: cat <<EOF > FILE.py OK: cat > FILE.py <<EOF - limit mark must not be quoted NG: cat > FILE.py <<'EOF' OK: cat > FILE.py <<EOF import-checker.py itself is fixed in subsequent patch.

fuzz: allow manifest fuzzer to detect leaks Huzzah! Differential Revision: https://phab.mercurial-scm.org/D4907

fuzzers: init Python in LLVMFuzzerInitialize and intentionally leak it This sidesteps leaks (or "leaks", I'm not sure) in CPython, and lets our fuzzer work. Differential Revision: https://phab.mercurial-scm.org/D4906

revlog: if the module is initialized more than once, don't leak nullentry Caught (annoyingly) by the manifest fuzzer. Differential Revision: https://phab.mercurial-scm.org/D4905

narrow: move remaining narrow-limited dirstate walks to core In most places we now filter at a higher level (the context object), but there are few places that relied on the dirstate walk to be filtered by the narrowspec. The important cases are those used by `hg add` and `hg addremove`. This patch updates them to pass in a matcher instead of relying on the dirstate to do the filtering. The dirstate filtering is also dropped in narrowdirstate.py. Not always filtering in the dirstate should be useful for a future `hg status --include-outside-narrow` option. These places now end up doing an unrestricted dirstate walk after this patch: * debugfileset * perfwalk * sparse (but restricted to sparse config) * largefiles I'll let anyone who cares about these cases adapt them to work with narrow if necessary. Differential Revision: https://phab.mercurial-scm.org/D4901

narrow: allow repo.narrowmatch(match) to include exact matches from "match" Differential Revision: https://phab.mercurial-scm.org/D4900

narrow: filter files by narrowspec in ctx.matches() This has no effect yet because 1) for committed changes, ctx.matches() just calls ctx.walk(), which we updated in the previous patch, and 2) for the working copy, the filtering is also done in the overridden dirstate.walk() in narrowdirstate. Differential Revision: https://phab.mercurial-scm.org/D4899

narrow: only walk files within narrowspec also for committed revisions Narrow has been walking only paths matching the narrowspec when walking the working copy. We have not done the same filtering when walking committed revisions (e.g. "hg files -r "), which seems a little odd. Let's make it consistent. Differential Revision: https://phab.mercurial-scm.org/D4898

status: intersect matcher with narrow matcher instead of filtering afterwards I seem to have done a very naive move of the code from the narrow extension into core in e411774a2e0f (narrow: move status-filtering to core and to ctx, 2018-08-02). It seems obvious that a better way is to intersect the matchers. Note that this means that when requesting status for the working directory in a narrow repo, we now pass the narrow matcher (possibly intersected with a user-provided matcher) into _buildstatus() and then into dirstate.status() and dirstate.walk(), which will the intersect it again with the narrow matcher. That's functionally fine, but wasteful. I hope to later remove the dirstate wrapping that adds the second layer of matcher intersection. Differential Revision: https://phab.mercurial-scm.org/D4897

localrepo: allow narrowmatch() to accept matcher to intersect with It's pretty common that we need to intersect a matcher we already have (usually from the user) with the narrow matcher. Let's make repo.narrowmatch() take an optional matcher to intersect with. Differential Revision: https://phab.mercurial-scm.org/D4896

obsolete: fix ValueError when stored note contains ':' char (issue5783) The newer version of `amend -n 'Some some'` accepts containing ':' char. The information contained in this note 'Testing::Obstore' gives ValueError, because we are trying to store more than 2 values in key and value. Differential Revision: https://phab.mercurial-scm.org/D4883 Differential Revision: https://phab.mercurial-scm.org/D4882

narrow: update TODO.rst now that we share format with sparse The narrowspec format was unified with the sparse format in f64ebe7d2259 (narrowspec: use sparse.parseconfig() to parse narrowspec file (BC), 2018-08-03). Differential Revision: https://phab.mercurial-scm.org/D4904

narrow: update TODO.rst now that we filter status in ctx The comment referred to was addressed in e411774a2e0f (narrow: move status-filtering to core and to ctx, 2018-08-02). I also think 84092edd5c88 (narrow: drop unnecessary overrides of patch, 2018-09-28) suggests that it was the right thing to do. Differential Revision: https://phab.mercurial-scm.org/D4903

narrow: update TODO.rst now that the narrowspec is in .hg/store We no longer have the unfortunate wrappostshare() and unsharenarrowspec() since 576eef1ab43d (narrow: move .hg/narrowspec to .hg/store/narrowspec (BC), 2018-08-02). Differential Revision: https://phab.mercurial-scm.org/D4902

py3: add 8 new passing tests to whitelist found by buildbot We are getting close! Differential Revision: https://phab.mercurial-scm.org/D4893

py3: use '%f' for floats instead of '%s' I remember Yuya saying we need to use bytestr() or '%r' because '%s' was clever. Not sure it applies to this or not. Differential Revision: https://phab.mercurial-scm.org/D4894