server: allow customizing the default repo filter hgweb has the (undocument) configuration option web.view that allows restricting visible revisions to immutable. This is useful for serving the same storage as publishing and non-publishing repo. Add the new server.view option to serve the same purpose by changing the default behavior of `getdispatchrepo`. Drop the hard-coded 'served' filter in the batch handler of v1 of the wire proto, this is a left-over from the days before `getdispatchrepo` existed. Differential Revision: https://phab.mercurial-scm.org/D5946

py3: convert filtername to str if it's None I have not called pycompat.bytestr() and rather converted the value there because I am starting to get concerned about the function call overhead of all this bytes to str or vice versa convert functions. Differential Revision: https://phab.mercurial-scm.org/D6042

walkfilerevs: rename filerevgen() to filerevs() since it's not a generator Differential Revision: https://phab.mercurial-scm.org/D6053

global: use raw string for setlocale() argument Otherwise Python 2 will coerce a unicode to str, which fails on HGUNICODEPEDANTRY=1. Differential Revision: https://phab.mercurial-scm.org/D6052

encoding: use raw strings for encoding arguments This prevents the internals of Python from coercing a unicode to str on Python 2 and makes tests run with HGUNICODEPEDANTRY=1 a lot happier. Differential Revision: https://phab.mercurial-scm.org/D6051

revsetlang: use sysbytes() instead of blind encode() Otherwise we will call str.encode() on Python 2, which is wrong. sysbytes() does encode('utf-8') on Python 3. But the source is guaranteed ASCII, so it shouldn't matter. With this change, `hg` now runs with `HGUNICODEPEDANTRY=1` set. However, several tests are failing. Differential Revision: https://phab.mercurial-scm.org/D6050

global: use raw strings for namedtuple() Otherwise Python 2 will attempt to coerce unicode to str, which we don't want. Differential Revision: https://phab.mercurial-scm.org/D6049

attr: don't attempt to .encode() a str on Python 2 Otherwise it coerces automatically. Differential Revision: https://phab.mercurial-scm.org/D6048

procutil: use a raw string for module name Otherwise Python 2 will coerce unicode to str. Differential Revision: https://phab.mercurial-scm.org/D6047

global: use raw strings for __slots__ Otherwise Python 2 will coerce unicode to str at module load time. We don't like automatic string coercions. Differential Revision: https://phab.mercurial-scm.org/D6046

absorb: run cleanupnodes() within transaction also when not using obsmarkers scmutil.cleanupnodes() schedules stripping to be done after the current transaction, so we can safely run it within the transaction. This also means that the phase will be updated within the transaction, which I believe means that the initial (possibly incorrect) phase will not visible. Differential Revision: https://phab.mercurial-scm.org/D6037

py3: pass a str to getpass.getpass() Otherwise I see `password: b''` on the prompt. # skip-blame because just prefix Differential Revision: https://phab.mercurial-scm.org/D6041

py3: port things from chgserver.py I have installed mercurial on my personal laptop using Python 3.7. I also have hg aliased to chg. Before this commit, `hg version` didn't work. After this patch, things are better and now chg works. Differential Revision: https://phab.mercurial-scm.org/D6040

py3: make sure return value of posix.groupname() is bytes Differential Revision: https://phab.mercurial-scm.org/D6039

contrib: make check-code.py check code fragments embedded in test scripts

contrib: add line offset information to file check function of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. This information will be useful to show correct line number in an actual file for errors detected in code fragments embedded in *.t test scripts.

contrib: change return value of file checking function of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. After this patch, caller of _checkfiledata() can count number of errors in each code fragments of an actual file. This will be useful to share --per-file limitation by all embedded code fragments.

contrib: factor out actual error check for file data of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Newly added _checkfiledata() will be useful to apply checks on code fragments embedded in *.t test scripts.

contrib: refactor preparation logic for patterns of check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Before this patch, preparation logic in _preparepats() of check-code.py is not reusable. It can handle only module global list "checks". This patch splits preparation logic into small internal functions, and add the loop to invoke them, in order to increase reusability of the logic. "c[-2]" is equivalent to "c[3]" for "checks". This patch uses the former, because it will be more reusable for subsequent patch than the latter.

contrib: split pypats list in check-code.py This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. Ideally, all patterns in "pypats" before this patch should be applied on not only normal *.py files but also code fragments embedded in *.t test scripts. But fixing test scripts for some patterns requires many changes, and has less profit than effort. Therefore, this patch splits pypats list into two below: - commonpypats, which are applied on all (= including code fragments embedded in *.t test scripts) *.py files - pypats, which are applied only on normal *.py

tests: remove useless line wrapping in a code fragment embedded in test script This is a part of preparation to apply checking with check-code.py on code fragments embedded in *.t test scripts. This revision avoids error of check-code.py below. Use () to wrap long lines in Python, not \

storage: introduce a `revlog.reuse-external-delta` config This option goes a bit further and provides a way to get the same behavior as the `re-delta-all` optimisation from `hg debugupgraderepo`. The effect of the option is a bit hard to test as we do not have multiple diff algorithm at hand. However, we at least make sure the code path run.

storage: introduce a `revlog.reuse-external-delta-parent` config As pointed in c59987ab96b4, we had no simple way to get a client to not blindly reuse the delta parent from a bundle. Instead one had to rely on a side effect of the deprecated `format.generaldelta` configuration. We introduce an explicit `revlog.reuse-external-delta-parent` configuration option (default to True) to control this behavior. If the option is not set, `format.generaldelta` still control this behavior. To test the new option, we convert a couple of place where `generaldelta` have been used for its side effects.

merge with stable

py3: convert return values of inspect.getabsfile() to bytes It's weird that python docs for inspect does not mention getabsfile(). Differential Revision: https://phab.mercurial-scm.org/D6033

py3: add two new passing tests found by buildbot Differential Revision: https://phab.mercurial-scm.org/D6032

py3: make contrib/debugshell.py work with Python 3 I changed default mercurial installation of my personal laptop to one installed with python 3.7. debugshell is one of the extension which I have enabled and it was failing. This patch makes debugshell works with Python 3. I found that chg does not work with python 3. Differential Revision: https://phab.mercurial-scm.org/D6031

absorb: let scmutil.cleanupnodes() take care of setting phase Differential Revision: https://phab.mercurial-scm.org/D6036

absorb: use scmutil.cleanupnodes() also when obsmarkers are disabled A side-effect of this is the new warning in the test case, but the warning is valid, so I don't think that's a problem. Differential Revision: https://phab.mercurial-scm.org/D6035

absorb: use scmutil.cleanupnodes() so operation gets set The operation is useful for e.g. `hg obslog` output. Differential Revision: https://phab.mercurial-scm.org/D6034

test: follow-up on 1c4d6ab2ecb8, stabilize test-remotefilelog-bgprefetch.t In 1c4d6ab2ecb8, I've added some sleeps after about half of the calls to `hg debugwaitonrepack` but it seems the other calls are also flaky as catched by our CI: https://ci.octobus.net/job/MercurialPy2/299/console. Add sleeps after all calls in order to be safe.

tests: add more wildcards to test-extdiff.t This test has been flakey recently on Windows, but it appears to be the same scenario as 88d4477ac4f6.

branchmap: prevent reading the file twice through different iterators Otherwise, test-static-http.t breaks. Differential Revision: https://phab.mercurial-scm.org/D6028

remotefilelog: fix format str, blobsize isn't always a #, can be "(missing)" Differential Revision: https://phab.mercurial-scm.org/D6025

remotefilelog: do not specify an explicit version for repack Differential Revision: https://phab.mercurial-scm.org/D6024

test: generate the `test-sparse-revlog` artifact when slow-test is allowed The `test-sparse-revlog.t` logic requires a large bundle to be generated. This bundle can be reused from one run to the next but its initial generation is slow. With this patch, if the bundle is missing and slow tests are permitted, the bundle will be generated during the test run. This should ensure that CI run this test.

runtest: also update slow test timeout during `#if` clauses For a `#if slow` test to be useful, we need the test timeout to be increased. Without this, the slow section would likely be killed before it finish.

runtest: move slow timeout process earlier in the `_hghave` method Before this changesets, early returns might prevent this logic to apply. It seem safer to run it sooner.

runtest: extract the logic that update timeout for slow tests We want to use the "slow" test logic not only for "#require" clauses but also in "#if" clauses. For this to be useful we need the logic to bump the timeout in at least two spots. The first step it to factor it out.

tests: increase timeout for slow test Test case `test-sparse-revlog.t` need some artifact (a bundle) build before it can run. The artifact is expensive to build, but can be reused from one run to the other. We are about to update that test to make the artifact building automatic if `--allow-slow-tests` is passed. However, we need a bump the timeout a bit to make sure the artifact building as time to finish. We could maybe teach run-tests.py how to directly handle such artifacts. However since there is only one of them for now, this seems premature. There are also some room to speed up the bundle creation for test-sparse-revlog.t

merge with stable

rewriting: add an option for rewrite commands to use the archived phase Using the archived phase for cleanup provide the same effect than stripping, but in a faster, append-only way. We keep the feature experimental for now until it gets a bit more testing.

strip: introduce a soft strip option This is the first user-accessible way to use the archived phase introduced in 4.8. This implements a feature discussed during the Stockholm sprint, using the archived phase for hiding changesets. The archived phase behaves exactly as stripping: changesets are no longer visible, but pulling/unbundling them will make then reappear. The only notable difference is that unlike hard stripping, soft stripping does not affect obsmarkers. The next changeset will make use of the archived phase for history rewriting command. However, having a way to manually trigger the feature first seems a necessary step before exposing users to this phase; there is a way to un-archived changesets (unbundling), so there must be a way to archive them again. Adding a flag to strip is a good way to provide access to the feature without taking a too big risk on the final UI we want. The flag is experimental so it won't be exposed by default. Using the archived phase is faster and less traumatic for the repository than actually stripping changesets.

branchcache: move loading of branch names and nodes into it's own function This will help me in implementing lazy loading of the branchcache in upcoming patches. Differential Revision: https://phab.mercurial-scm.org/D6023

rebase: add missing dashes in help text

test: stabilize bundle generation for test-sparse-revlog.t To reduce the instability in the bundle binary content, we force it to contains delta against p1 in all cases. In the previous changeset, we already stabilized the processing of the bundle. So we don't see any output change in the test itself.

test: don't trust delta bases from the bundle in test-sparse-revlog.t The point of the test is to check the strategy sparse-revlog uses to pick delta base. If we trust the bases used in the bundle, we no longer fully test this logic. In order to force this computation we have to use the side effect of a legacy format configuration `format.generaldelta`. The lack of a more official way to do so will be fixed in a later changeset.

test: update test-sparse-revlog.t output This test is skipped unless a large artefact is pre-build. It seems like nobody ran it in a while. Changeset 3764330f76a6 changed the expected output but nobody noticed. This changeset focus on the first and simpler step: putting the expected output back to what one would get by running this test. However this test changes highlight a couple of deeper issues: 1) Even if the revision content did not changed, a change in the delta contained in the bundle affected the delta stored in the final revlog, changing the test result. Since we are testing the delta computation strategy with sparse, we should not blindly reuse the delta-base from the bundled delta. 2) A change in the format of the repository used to generate the bundle changed the delta stored in the bundle. We should get a more stable output to avoid future instabilities of this test. 3) The test is it not run by CI or developer. We'll try to address all this issues in the coming changesets.

bundle2: don't send "shared" requirement when cloning from a share Differential Revision: https://phab.mercurial-scm.org/D5985

tests: demonstrate failure when cloning from a share via bundle2 Differential Revision: https://phab.mercurial-scm.org/D5984

mq: disable qrecord during histedit (issue5981) qrecord during histedit may lead to deadlock-like situations. qpop will throw an error on called during histedit even after qrecord-ing those changes. This patch makes qrecord to abort on histedit. Differential Revision: https://phab.mercurial-scm.org/D5997

committablectx: move status-related methods closer together The modified()/added()/removed()/deleted() clearly belong very close to status(). I separated them in committablectx by the new p[12]copies() methods. This brings the close again. Sorry about the churn. Differential Revision: https://phab.mercurial-scm.org/D5996

tests: add test for hg-test-mode emacs code This is just coverage for the compilation-mode support, but that was enough of a hassle that I wanted to have it covered somehow. Test methodology is _extremely_ cargo-culted from the test for compilation-mode in emacs, so I still have no idea what I'm doing. Differential Revision: https://phab.mercurial-scm.org/D6003

hghave: add check for GNU emacs Differential Revision: https://phab.mercurial-scm.org/D6002

contrib: also linkify tracebacks in compilation output when using hg-test-mode Differential Revision: https://phab.mercurial-scm.org/D6001

contrib: add compilation-mode linking for our test output These regular expressions will cause compilation-mode buffers in emacs to link to source when there are check-code errors in the output of a .t test. In the true tradition of this file, I also have no idea what I'm doing. Differential Revision: https://phab.mercurial-scm.org/D6000

diff: make sure we output stat even when --git is not passed (issue4037) (BC) Before this patch, `hg diff --stat` will give an empty output. It will not show the stat information. I debugged and found that the underlying code does not return the diff header and due to that, other code paths fails to parse that as a diff. I looked into why we don't return diff headers in quiet mode and found the behavior is from 8f8bb77d560e70bcc95577e4dfa877df18d876ab which does not have any mention about why it is done. We also show the diff headers in git, so I think it's fine showing diff header in normal diff in quiet mode. Differential Revision: https://phab.mercurial-scm.org/D6007

tests: add test to demonstrate issue4037 `hg diff --stat -q --config diff.git=0` does not output anything whereas it should print the stat. This is a quiet old bug dating to 2013 and looking at code I think it exists it since 2005 or when --stat was introduced. The next patch will fix the bug. Differential Revision: https://phab.mercurial-scm.org/D6006

grep: reuse getrenamedfn() from scmutil My motivation is to reduce uses of filectx.renamed(). Reusing scmutil.getrenamedfn() also means that we get some caching of copy information per file and revision. I don't think that matters for `hg grep` (I doubt it speeds up significantly, and I doubt it wastes significant memory), but I'm not sure. Differential Revision: https://phab.mercurial-scm.org/D6022

templatekw: move getrenamedfn() to scmutil (API) The function is already used by `hg log` (for following renames, not for templates), so it seems it does not belong in templatekw. Differential Revision: https://phab.mercurial-scm.org/D6021