contrib: make check-code.py check code fragments embedded in test scripts

contrib: add line offset information to file check function of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. This information will be useful to show correct line number in an actual file for errors detected in code fragments embedded in *.t test scripts.

contrib: change return value of file checking function of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. After this patch, caller of _checkfiledata() can count number of errors in each code fragments of an actual file. This will be useful to share --per-file limitation by all embedded code fragments.

contrib: factor out actual error check for file data of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Newly added _checkfiledata() will be useful to apply checks on code fragments embedded in *.t test scripts.

contrib: refactor preparation logic for patterns of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Before this patch, preparation logic in _preparepats() of check-code.py is not reusable. It can handle only module global list "checks". This patch splits preparation logic into small internal functions, and add the loop to invoke them, in order to increase reusability of the logic. "c[-2]" is equivalent to "c[3]" for "checks". This patch uses the former, because it will be more reusable for subsequent patch than the latter.

contrib: split pypats list in check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Ideally, all patterns in "pypats" before this patch should be applied on not only normal *.py files but also code fragments embedded in *.t test scripts. But fixing test scripts for some patterns requires many changes, and has less profit than effort. Therefore, this patch splits pypats list into two below: - commonpypats, which are applied on all (= including code fragments embedded in *.t test scripts) *.py files - pypats, which are applied only on normal *.py

tests: remove useless line wrapping in a code fragment embedded in test script This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. This revision avoids error of check-code.py below. Use () to wrap long lines in Python, not \

storage: introduce a `revlog.reuse-external-delta` config This option goes a bit further and provides a way to get the same behavior as the `re-delta-all` optimisation from `hg debugupgraderepo`. The effect of the option is a bit hard to test as we do not have multiple diff algorithm at hand. However, we at least make sure the code path run.

storage: introduce a `revlog.reuse-external-delta-parent` config As pointed in c59987ab96b4, we had no simple way to get a client to not blindly reuse the delta parent from a bundle. Instead one had to rely on a side effect of the deprecated `format.generaldelta` configuration. We introduce an explicit `revlog.reuse-external-delta-parent` configuration option (default to True) to control this behavior. If the option is not set, `format.generaldelta` still control this behavior. To test the new option, we convert a couple of place where `generaldelta` have been used for its side effects.

merge with stable

py3: convert return values of inspect.getabsfile() to bytes It's weird that python docs for inspect does not mention getabsfile(). Differential Revision: https://phab.mercurial-scm.org/D6033

py3: add two new passing tests found by buildbot Differential Revision: https://phab.mercurial-scm.org/D6032

py3: make contrib/debugshell.py work with Python 3 I changed default mercurial installation of my personal laptop to one installed with python 3.7. debugshell is one of the extension which I have enabled and it was failing. This patch makes debugshell works with Python 3. I found that chg does not work with python 3. Differential Revision: https://phab.mercurial-scm.org/D6031

absorb: let scmutil.cleanupnodes() take care of setting phase Differential Revision: https://phab.mercurial-scm.org/D6036

absorb: use scmutil.cleanupnodes() also when obsmarkers are disabled A side-effect of this is the new warning in the test case, but the warning is valid, so I don't think that's a problem. Differential Revision: https://phab.mercurial-scm.org/D6035

absorb: use scmutil.cleanupnodes() so operation gets set The operation is useful for e.g. `hg obslog` output. Differential Revision: https://phab.mercurial-scm.org/D6034

test: follow-up on 1c4d6ab2ecb8, stabilize test-remotefilelog-bgprefetch.t In 1c4d6ab2ecb8, I've added some sleeps after about half of the calls to `hg debugwaitonrepack` but it seems the other calls are also flaky as catched by our CI: https://ci.octobus.net/job/MercurialPy2/299/console. Add sleeps after all calls in order to be safe.

tests: add more wildcards to test-extdiff.t This test has been flakey recently on Windows, but it appears to be the same scenario as 88d4477ac4f6.

branchmap: prevent reading the file twice through different iterators Otherwise, test-static-http.t breaks. Differential Revision: https://phab.mercurial-scm.org/D6028

remotefilelog: fix format str, blobsize isn't always a #, can be "(missing)" Differential Revision: https://phab.mercurial-scm.org/D6025

remotefilelog: do not specify an explicit version for repack Differential Revision: https://phab.mercurial-scm.org/D6024

test: generate the `test-sparse-revlog` artifact when slow-test is allowed The `test-sparse-revlog.t` logic requires a large bundle to be generated. This bundle can be reused from one run to the next but its initial generation is slow. With this patch, if the bundle is missing and slow tests are permitted, the bundle will be generated during the test run. This should ensure that CI run this test.

runtest: also update slow test timeout during `#if` clauses For a `#if slow` test to be useful, we need the test timeout to be increased. Without this, the slow section would likely be killed before it finish.

runtest: move slow timeout process earlier in the `_hghave` method Before this changesets, early returns might prevent this logic to apply. It seem safer to run it sooner.

runtest: extract the logic that update timeout for slow tests We want to use the "slow" test logic not only for "#require" clauses but also in "#if" clauses. For this to be useful we need the logic to bump the timeout in at least two spots. The first step it to factor it out.

tests: increase timeout for slow test Test case `test-sparse-revlog.t` need some artifact (a bundle) build before it can run. The artifact is expensive to build, but can be reused from one run to the other. We are about to update that test to make the artifact building automatic if `--allow-slow-tests` is passed. However, we need a bump the timeout a bit to make sure the artifact building as time to finish. We could maybe teach run-tests.py how to directly handle such artifacts. However since there is only one of them for now, this seems premature. There are also some room to speed up the bundle creation for test-sparse-revlog.t

merge with stable

rewriting: add an option for rewrite commands to use the archived phase Using the archived phase for cleanup provide the same effect than stripping, but in a faster, append-only way. We keep the feature experimental for now until it gets a bit more testing.

strip: introduce a soft strip option This is the first user-accessible way to use the archived phase introduced in 4.8. This implements a feature discussed during the Stockholm sprint, using the archived phase for hiding changesets. The archived phase behaves exactly as stripping: changesets are no longer visible, but pulling/unbundling them will make then reappear. The only notable difference is that unlike hard stripping, soft stripping does not affect obsmarkers. The next changeset will make use of the archived phase for history rewriting command. However, having a way to manually trigger the feature first seems a necessary step before exposing users to this phase; there is a way to un-archived changesets (unbundling), so there must be a way to archive them again. Adding a flag to strip is a good way to provide access to the feature without taking a too big risk on the final UI we want. The flag is experimental so it won't be exposed by default. Using the archived phase is faster and less traumatic for the repository than actually stripping changesets.

branchcache: move loading of branch names and nodes into it's own function This will help me in implementing lazy loading of the branchcache in upcoming patches. Differential Revision: https://phab.mercurial-scm.org/D6023

rebase: add missing dashes in help text

test: stabilize bundle generation for test-sparse-revlog.t To reduce the instability in the bundle binary content, we force it to contains delta against p1 in all cases. In the previous changeset, we already stabilized the processing of the bundle. So we don't see any output change in the test itself.