py3: fix test-newcgi.t Differential Revision: https://phab.mercurial-scm.org/D5683

py3: fix up test-encoding-align.t for python3 I'm not super thrilled with how this reads, but the original didn't thrill me either. Differential Revision: https://phab.mercurial-scm.org/D5682

py3: these two casefolding tests pass for me on my Mac I assume the buildbot didn't catch them because it's on a case-sensitive filesystem. Differential Revision: https://phab.mercurial-scm.org/D5681

py3: port test-bugzilla.t to Python 3 Some assorted fixes required in the extension itself, all around bytes/str issues. Differential Revision: https://phab.mercurial-scm.org/D5680

uncommit: set dirstateparents from within _fixdirstate() It's now done in the same way for uncommit and unamend, so we can share the code. Differential Revision: https://phab.mercurial-scm.org/D5662

unamend: fix unamending of renamed rename Before this patch, we were passing in the result of a status call for a different set of commits than what we calculated copies for, which seemed suspicous to me. The rewritten version gets status and copy information from the same sets of commits. Differential Revision: https://phab.mercurial-scm.org/D5661

uncommit: mark old node obsolete after updating dirstate The next patch will start doing more things with the context object for the old node and that ran into problems without this change. Regardless of that, I think it seems better to first update to the new node and then mark the old node obsolete. Differential Revision: https://phab.mercurial-scm.org/D5660

unamend: import "copies" module as "copiesmod" to avoid shadowing Differential Revision: https://phab.mercurial-scm.org/D5659

tests: add more tests of uncommit/unamend with copies The unamend tests show suboptimal behavior. Differential Revision: https://phab.mercurial-scm.org/D5658

tests: clean up after each test in test-unamend.t Differential Revision: https://phab.mercurial-scm.org/D5657

cleanup: use clang-tidy to add missing {} around one-line statements I find this easier to read. Cleanup performed like this: hg files 'set:(**.c or **.cc or **.h) and not "listfile:contrib/clang-format-ignorelist"' | while read f ; do clang-tidy -fix -checks=readability-braces-around-statements $f -- $(python-config --cflags) -Imercurial/cext -Imercurial done make format-c I had to revert chg/chg.c as it's got a construct that seems to confuse clang-tidy, so I'll work on that file later if this change is acceptable. I only tackle files that are under clang-format's authority because otherwise I'd have to do a bunch of manual formatting. A few files didn't get edited because clang-tidy couldn't find some headers. Again, I'll figure that out later assuming this change is accepted. No check-code rule added for now because writing the regex sounds hard. In a perfect world I guess we could write a test that uses clang-tidy on these files, but I think clang-tidy is pretty rarely installed. :/ Differential Revision: https://phab.mercurial-scm.org/D5675

py3: port test-batching.py to python3 I used byteify-strings.py on this file, then manually added the various pycompat and bprint bits as needed. Differential Revision: https://phab.mercurial-scm.org/D5678

py3: fix missing b prefixes in test-arbitraryfilectx.t Test passes on Python 3. # skip-blame b prefixes Differential Revision: https://phab.mercurial-scm.org/D5679

py3: add missing b prefixes in test-acl.t There's one failure left in the test, which looks like a real problem around executing hooks. I have not investigated further. # skip-blame as just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5677

py3: buildbot found more passing tests, thanks to indygreg for most of them Differential Revision: https://phab.mercurial-scm.org/D5676

tests: drop a duplicate definition of a constant Differential Revision: https://phab.mercurial-scm.org/D5663

tests: support URL quoting on Python 3 We could use mercurial.urllibcompat, but meh. This makes things easier to read. The test still fails on Python 3 for some reason. But at least we no longer have an exception. Differential Revision: https://phab.mercurial-scm.org/D5669

tests: write directly to stdout to avoid b'' prefixes This enables the test to pass on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5668

tests: handle string escaping/encoding on Python 3 This code was failing on Python 3 for a few reasons: 1) sys.argv is str and str doesn't have a .decode() 2) the "string_escape" encoding was renamed to "unicode_escape" It is wonky casting to bytes to str to bytes. But this is test code, so meh. I don't believe we exercise any code paths in these tests where the arguments aren't ascii. Differential Revision: https://phab.mercurial-scm.org/D5667

convert: use raw strings for XML strings Due to the source transformer, we were passing bytes into the XML APIs. This results in not finding elements and doing compares against mismatched types. Use raw string literals so we use str everywhere. Differential Revision: https://phab.mercurial-scm.org/D5664

tests: normalize XML values to bytes This fixes some type mismatches. The encoding shouldn't matter for what this script is used for. UTF-8 seems reasonable, especially since I'm pretty sure SVN will emit UTF-8 encoded XML. Differential Revision: https://phab.mercurial-scm.org/D5665

hgweb: ensure template mapping keys are bytes Before, str keys were being added in Python 3 because named arguments to dict() use native str for keys. This caused the templater to fail to find the keys since it was looking for bytes versions. This makes a handful of tests pass on Python 3. We may want to consider having the templater validate that keys in mapping dicts are bytes. But I'm unsure whether this is appropriate and won't be doing this. Differential Revision: https://phab.mercurial-scm.org/D5666

remotefilelog: use %d to format an int Differential Revision: https://phab.mercurial-scm.org/D5656

tests: use bytes for file I/O Otherwise we get various type mismatches. Differential Revision: https://phab.mercurial-scm.org/D5655

tests: make filenames bytes for Python 3 I also snuck a %s -> %d in there to appease Python 3. Differential Revision: https://phab.mercurial-scm.org/D5654

tests: use assertEqual() This avoids a deprecation warning under at least Python 3.7. Differential Revision: https://phab.mercurial-scm.org/D5653

tests: add setsockopt() output on Python 3 Python 3 appears to call setsockopt() where Python 2 did not. Differential Revision: https://phab.mercurial-scm.org/D5652

hgweb: don't use raw string for session vars This r'' is leaking into the templating layer and causing an assertion failure. The r'' was added in d1fccbd50fcd (October 2017). Similar code in hgweb_mod.py was also changed in that changeset. hgweb_mod.py was updated in ec46415ed826 (March 2018) to use webutil.sessionvars(), which doesn't use raw strings. Differential Revision: https://phab.mercurial-scm.org/D5651

tests: normalize to bytes in test-install.t directory() was returning str and hgdirectory() was returning bytes. This made the set compare fail. Let's normalize the types on Python 3 so the test passes. Differential Revision: https://phab.mercurial-scm.org/D5650

config: reject str sections and keys on Python 3 Otherwise we could end up with a dict having both the str and bytes versions of a particular config item. This may cause some tests to regress. I haven't checked. But it is better behavior to fail fast. We could just as easily change this to normalize the values. But I like catching all non-compliant call sites first. Differential Revision: https://phab.mercurial-scm.org/D5649

tests: add size and hash for Python 3 test-archive.t now passes on Python 3.7.1. Differential Revision: https://phab.mercurial-scm.org/D5648

httppeer: use %s for formatting This prevents a b'' from appearing in Python 3 output. We keep the single quotes for backwards compatibility. Differential Revision: https://phab.mercurial-scm.org/D5647

tests: add b'' to notcapable This fixes some exceptions in a few tests. # skip-blame just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5646

tests: add b'' to test-missing-capability.t The test now passes on Python 3. # skip-blame just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5645

fuzz: exercise more of the lazymanifest code Differential Revision: https://phab.mercurial-scm.org/D5643

fuzz: restrict manifest input size Again, let's keep the fuzzer from getting excited about huge inputs. Differential Revision: https://phab.mercurial-scm.org/D5642

fuzz: exercise more of the revlog API I noticed in the coverage report that we didn't have much coverage in revlog.py. Let's try and get some of the more interesting bits tested by the fuzzer. I ran this locally for a few minutes to verify that I appear to be calling the various functions in reasonable ways. Differential Revision: https://phab.mercurial-scm.org/D5641

fuzz: don't allow enormous revlog inputs either I'm about to make the fuzzer do more, and without this it was getting enthusiastic about large (and therefore slow) inputs that I don't think buy us much. Differential Revision: https://phab.mercurial-scm.org/D5640

merge with stable

Added signature for changeset 4ea21df312ec

Added tag 4.9.1 for changeset 4ea21df312ec

record: prevent commits that don't pick up dirty subrepo changes (issue6102) This path covers interactive mode for commit, amend, and shelve, as well as the deprecated record extension. Since shelf creation uses commit without -S in the non-interactive case, aborting here should be OK. (I didn't check what happens to non interactive shelve creation if `ui.commitsubrepos=True` is set.) subrepoutil.precommit() will abort on a dirty subrepo if the config option isn't set, but the hint recommends using --subrepos to commit. Since only the commit command currently supports that option, the error has to be raised here to omit the hint. Doing the check before asking about all of the hunks in the MQ test seems like an improvement on its own. There's probably an additional check on this path that can be removed.

test-https: add some more known failure messages of client certs (issue6030) I don't think the exact error message is important here. On Debian sid, ECONNRESET is raised, and "[SSL] tlsv13 alert certificate required" on NetBSD.

test-https: turn off system OpenSSL configuration This mostly fixes the test failure on Debian sid where TLS 1.0 and 1.1 are disabled by default. https://sources.debian.org/patches/openssl/1.1.1a-1/Set-systemwide-default-settings-for-libssl-users.patch/ $OPENSSL_CONF could be set by run-tests.py, but the other tests should work without a "legacy" TLS, so I decided to not.

rebase: add missing dashes in help text

test: stabilize bundle generation for test-sparse-revlog.t To reduce the instability in the bundle binary content, we force it to contains delta against p1 in all cases. In the previous changeset, we already stabilized the processing of the bundle. So we don't see any output change in the test itself.

test: don't trust delta bases from the bundle in test-sparse-revlog.t The point of the test is to check the strategy sparse-revlog uses to pick delta base. If we trust the bases used in the bundle, we no longer fully test this logic. In order to force this computation we have to use the side effect of a legacy format configuration `format.generaldelta`. The lack of a more official way to do so will be fixed in a later changeset.

test: update test-sparse-revlog.t output This test is skipped unless a large artefact is pre-build. It seems like nobody ran it in a while. Changeset 3764330f76a6 changed the expected output but nobody noticed. This changeset focus on the first and simpler step: putting the expected output back to what one would get by running this test. However this test changes highlight a couple of deeper issues: 1) Even if the revision content did not changed, a change in the delta contained in the bundle affected the delta stored in the final revlog, changing the test result. Since we are testing the delta computation strategy with sparse, we should not blindly reuse the delta-base from the bundled delta. 2) A change in the format of the repository used to generate the bundle changed the delta stored in the bundle. We should get a more stable output to avoid future instabilities of this test. 3) The test is it not run by CI or developer. We'll try to address all this issues in the coming changesets.

tests: add a missing abort exit code on Windows in test-audit-subrepo.t

subrepo: add test for Windows relative-ish path with drive letter Matt Harbison pointed out that Windows had some weird path syntax. Fortunately it's rejected appropriately by pathauditor, so we're safe. Let's test the behavior as we have a special handling for Windows drive letters. This patch includes a basic example. Maybe we'll need to extend the test case further, but writing such tests on Linux isn't easy.

tests: fix regression tests failing on CentOS 7 Differential Revision: https://phab.mercurial-scm.org/D5877

packaging: modify rc detection to work with X.Yrc instead of X.Y-rc rc detection on CentOS failed without this change, resulting in upgrades from 4.9rc to 4.9 not working (4.9rc was considered more recent than 4.9). Differential Revision: https://phab.mercurial-scm.org/D5876

subrepo: avoid false unsafe path detection on Windows Subrepo paths are not normalized for the OS, so what was happening in the subsequent root path check was: root -> $TESTTMP\issue1852a\sub/repo util.expandpath(...) -> $TESTTMP\issue1852a\sub/repo os.path.realpath(...) -> $TESTTMP\issue1852a\sub\repo

Added signature for changeset 83377b4b4ae0

Added tag 4.9 for changeset 83377b4b4ae0

subrepo: reject potentially unsafe subrepo paths (BC) (SEC) In addition to the previous patch, this prohibits '~', '$nonexistent', etc. for any subrepo types. I think this is safer, and real-world subrepos wouldn't use such (local) paths.

subrepo: prohibit variable expansion on creation of hg subrepo (SEC) It's probably wrong to expand path at localrepo.*repository() layer, but fixing the layering issue would require careful inspection of call paths. So, this patch adds add a validation to the subrepo constructor. os.path.realpath(util.expandpath(root)) is what vfsmod.vfs() would do.

subrepo: extend path auditing test to include more weird patterns (SEC) While reviewing patches for the issue 5739, "$foo in repository path expanded", I realized that subrepo paths can also be cheated. This patch includes various subrepo paths which are potentially unsafe. Since an expanded subrepo path isn't audited, this bug allows symlink check bypass. As a result, a malicious subrepository could be checked out to a sub tree of e.g. $HOME directory. The good news is that the destination directory must be empty or nonexistent, so the existing ~/.bashrc wouldn't be overwritten. See the last part of the tests for details.

copyright: update to 2019 Differential Revision: https://phab.mercurial-scm.org/D5779

revlog: make sure we never use sparserevlog without general delta (issue6056) We are getting user report where the delta code tries to use `sparse-revlog` logic on repository where `generaldelta` is disabled. This can't work so we ensure the two booleans have a consistent value. Creating this kind of repository is not expected to be possible the current bug report point at a clonebundle related bug that is still to be properly isolated (Yuya Nishihara seems to a have done it). Corrupting a repository to reproduce the issue is possible. A test using this method is included in this fix.