repoview: fix conditional around unserved changesets The conditional could lead to wrong computation since we have more unserved changesets than just the "secret" phase. In addition, now that we have efficient caching of phased changesets, we don't need the conditional anymore.

branch: return early if we find an open named branch apart from default The current code builds a list of all the open named branches except default and then bool that. This is mostly fine until you get a repo which has thousands of named branches. Differential Revision: https://phab.mercurial-scm.org/D6211

branchcache: don't verify closed nodes in _branchtip() We only do membership testing there. Differential Revision: https://phab.mercurial-scm.org/D6210

branchcache: don't verify closed nodes in iteropen() We expect that the nodes passed to iteropen() will be verified. We are only testing for membership in closed nodes set, so we don't need to verify the whole closed nodes set. This will speed up calculating branchheads() when there are lot of closed nodes related to other branches. Differential Revision: https://phab.mercurial-scm.org/D6209

branchcache: lazily validate nodes from the branchmap On my personal hg-repository with 365 entries in .hg/cache/branch2, following are the numbers for perfbranchmapload. Before this patch: ! wall 0.000866 comb 0.000000 user 0.000000 sys 0.000000 (best of 2680) ! wall 0.001525 comb 0.000000 user 0.000000 sys 0.000000 (max of 2680) ! wall 0.001107 comb 0.001097 user 0.001086 sys 0.000011 (avg of 2680) ! wall 0.001104 comb 0.000000 user 0.000000 sys 0.000000 (median of 2680) With this patch: ! wall 0.000530 comb 0.000000 user 0.000000 sys 0.000000 (best of 4240) ! wall 0.001078 comb 0.000000 user 0.000000 sys 0.000000 (max of 4240) ! wall 0.000696 comb 0.000693 user 0.000677 sys 0.000017 (avg of 4240) ! wall 0.000690 comb 0.000000 user 0.000000 sys 0.000000 (median of 4240) On our internal repository with ~20k entries in branchcache, I see improvement from 0.125 sec to 0.066 sec which is 47% speed up. The above are the numbers of perfbranchmapload which shows how much time we saved by not validating the nodes. But we need to validate some nodes. Following are timings of some mercurial operations which have speed up because of this lazy validation of nodes: No-op `hg update` on our internal repository (Avg on 4 runs): Before: 0.540 secs After: 0.430 secs Setting a branch name which already exists without --force (Avg of 4 runs): Before: 0.510 secs After: 0.250 secs I ran the ASV performance suite and was unable to see any improvements except there was improvement of perfdirstatewrite() on netbeans which I think was not related. I looked into the commit code, the command which I am trying to speedup, it looks like it uses revbranchcache to update the branchcache. Differential Revision: https://phab.mercurial-scm.org/D6208

branchcache: add functions to validate changelog nodes This patch adds functions to validate closed nodes, validate nodes for a certain branch and for all the branches. These functions will be used in upcoming patches. Differential Revision: https://phab.mercurial-scm.org/D6207

readline: provide styled prompt to readline (issue6070) Differential Revision: https://phab.mercurial-scm.org/D6168

copies: move comment about implementation of mergecopies() to end When you start reading about mergecopies(), you want to know what it is, not that there are different implementations depending on config, so this patch moves that comment to the end. By the way, we don't seem to define what "copytracing" is. I'm just leaving it that way because I don't know what it is myself. It seems to be referred to only on mergecopies() (and not in pathcopies(), for example), so maybe "copytracing" is supposed to be exactly what mergecopies() does? Differential Revision: https://phab.mercurial-scm.org/D6235

remotefilelog: return expected type from copies overrides copies._computeforwardmissing() and copies._computenonoverlap() return sets, so the overrides should also do that. Differential Revision: https://phab.mercurial-scm.org/D6234

changelog: extract a _string_unescape() to mirror _string_escape() We use our own _string_escape() to encode the "extras" field. Then we use codecs.escape_decode() to escape it. But there's also a little workaround for dealing with escaped text that looks like octal numbers since the fix for https://bz.mercurial-scm.org/show_bug.cgi?id=3156. This patch extracts the call to codecs.escape_decode() along with the fix for octal numbers and puts it in a _string_unescape(). It also updates the test to check for the octal-number case from the aforementioned bug. As you may have suspected, I want to be able to reuse this new function later. Differential Revision: https://phab.mercurial-scm.org/D6184

copies: extract function for deciding whether to use changeset-centric algos We'll eventually have a "experimental.copies.read-from=changeset-only" option too and I don't want to spread the logic for determining if we should use changeset-centric of filelog-centric algorithms. Differential Revision: https://phab.mercurial-scm.org/D6163

getrenamedfn: get copy data from context object if configured The function returned from getrenamedfn() calls filelog.renamed(). That won't work when storing copy metadata in the changeset. I've just switched to a simple implementation here. We may or may not need to optimize it later, possibly by optimizing the callers. No more tests fail with "--extra-config-opt experimental.copies.read-from=compatibility)" than they did before this patch. Differential Revision: https://phab.mercurial-scm.org/D6162

branchmap: implement __contains__() We have good occurences of `if branch in branchmap()` in our code. If __contains__() is not implemented then it will use __iter__() to find whether the element exists or not which is not good. I am bit confused that whether I should move existing callers to hasbranch() or this patch is a good way. Differential Revision: https://phab.mercurial-scm.org/D6206

branchmap: prevent using __getitem__() in branchheads() branchheads() can directly use self._entries instead. Differential Revision: https://phab.mercurial-scm.org/D6205

branchmap: dynamically resolve type of branchcache class This is required to support subclassing. Thanks to Yuya for suggesting this in D6151. Differential Revision: https://phab.mercurial-scm.org/D6204

copies: print list of divergent renames in sorted order The current order is not obvious to the user (and I may change it in a later patch). Lexicographical order seems like the obvious choice here. Differential Revision: https://phab.mercurial-scm.org/D6227

import: suppress accept: header irccloud and others may perform content negotation and reject (406) if the accept header does not match the mime type of the object

rebase: fix bug that prevented dry-run rebases from printing failures As far as I can tell it should be fine to unconditionally skip _prepareabortorcontinue if we're in the process of raising an Abort here. Differential Revision: https://phab.mercurial-scm.org/D6226

rebase: demonstrate bug in dry-run mode which causes cycles to not be reported Differential Revision: https://phab.mercurial-scm.org/D6225

test: minor cleanup to test-server-view.t While looking into adding error output in this test, I did some cleanup.

repoview: improve documentation for `repo.filtered` method I am sitting next to Joerg Sonnenberger and we are discussion his experience with repoview. This first effect of this discussion is this documentation clarification.

revset: short docstring for checkstatus This is where all the action happens for the status-related revsets, and a little documentation doesn't hurt.

discovery: stop direct use of attribute of partialdiscovery Instead of accessing `undecided` directly for ui display purposes, we introduce a `stats()` method that could be extended in the future with more interesting information. This is in preparation for a forthcoming Rust version of this object. Indeed, attributes and furthermore properties are a bit complicated for classes in native code. We could go further and rename `undecided` to mark it private, but `_undecided` is already taken as support for `_undecided` lazyness.

overlayworkingctx: remove misleading trailing slash from directory pattern The paths passed into the matcher are normalized (this applies to include patterns and regular patterns, and to both glob kind and path kind), so the regex for input "foo/" ended up being "foo(?:/|$)". Once we have a (recursive) pattern kind only for directories, we could switch to that here and remove the "mfiles[0] == path" check. Until then, let's at least make it not misleading. Differential Revision: https://phab.mercurial-scm.org/D6224

overlayworkingctx: fix file/dir audit to be repo-relative Before this patch, test-rebase-inmemory.t would stop erroring out about the conflict if you added a "cd a" before line 252. That was because a glob matcher (which are relative) was unintentionally used. That happened because the matcher was given "include" patterns (not regular patterns), and "include" patterns are always glob by default (i.e. unless you write them including the kind prefix). IOW, the "default='path'" argument passed to ctx.match() was ignored. Differential Revision: https://phab.mercurial-scm.org/D6223

messages: replace some instances of "folder" by "directory" I'm pretty sure this is our preferred term. Differential Revision: https://phab.mercurial-scm.org/D6222

match: fix re2 compability broken in 2e2699af5649 When using re2, we call test_match() instead of match() on the compiled regex object. While match() returns a matcher object or None, test_match() returns True or False. So since 2e2699af5649 running test on a machine with a re2 install fails in many places. Instead we make the code a bit more general and everything goes back to normal.

py3: add b'' prefixes to new doctests in match.py # skip-blame as just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D6221

py3: add one new passing test found by buildbot Differential Revision: https://phab.mercurial-scm.org/D6220

cext: cast s# arguments of Py_BuildValue() to Py_ssize_t The doc doesn't state that "s#" of Py_BuildValue() is controlled by PY_SSIZE_T_CLEAN (unlike the one for PyArg_ParseTuple()), but actually it's switched to Py_ssize_t. https://docs.python.org/2/c-api/arg.html#c.Py_BuildValue https://github.com/python/cpython/blob/2.7/Python/modsupport.c#L432 Follow up for b01bbb8ff1f2 and 896b19d12c08.

remotefilelog: correctly reject wdir filenodes This fixes `hg grep -r 'wdir()'` when remotefilelog is enabled and the working directory contains uncommitted modifications. Differential Revision: https://phab.mercurial-scm.org/D6217

remotefilelog: add tests of `hg grep -r 'wdir()'` This demonstrates how remotefilelog breaks grepping dirtied working directories. A future change will introduce a fix. Differential Revision: https://phab.mercurial-scm.org/D6216

config: read configs from directories in lexicographical order Mercurial currently reads the .rc files specified in HGRCPATH (and the system-default paths) in directory order, which is unspecified. My team at work maintains a set of .rc files. So far there has been no overlap between them, so we had not noticed this behavior. However, we would now like to release some common .rc files and then have another one per plaform with platform-specific overrides. It would be nice if we can determine the load order by choosing names carefully. This patch enables that by loading the .rc files in lexicographical order. Before this patch, the added test case would consistently say "30" on my file system (whatever I have -- some Linux FS). Differential Revision: https://phab.mercurial-scm.org/D6193

remotefilelog: fix crash on `hg addremove` of added-but-deleted file If you `hg add` a file and then delete it from disk, and then run `hg addremove`, the file ends up in the "removed" set that gets passed to the findrenames() override. We then crash because the file is not in the working copy parent. This patch fixes that. Differential Revision: https://phab.mercurial-scm.org/D6194

packaging: ensure that --python is an absolute path when building on Windows For whatever reason, even though only python2 is on PATH, passing `python.exe` causes the later check that it's not py3 to bail out.

packaging: don't crash building wix with python3.6 and earlier `capture_output` was added in 3.7. I was tempted to just check and abort in build.py, since Windows doesn't have the Linux problem where some distros only ship an older python. But this is in a library that could be used elsewhere in the future.

chistedit: add basic colours to diff view This isn't complete, and it would be nice to show the exact same colours that `hg diff` would show. That goal is too lofty, so this just shows some basic colours, on the premise that a little is better than nothing.

chistedit: use default curses colours Terminals will define default colours (for example, white text on black background), but curses doesn't obey those default colours unless told to do so. Calling `curses.use_default_colors` makes curses obey the default terminal colours. One of the most obvious effects is that this allows transparency on terminals that support it. This also brings chistedit closer in appearance to crecord, which also uses default colours. The call may error out if the terminal doesn't support colors, but as far as I can tell, everything still works. If we need a more careful handling of lack of colours, blame me for not doing it now.

match: let regex match function return a boolean Match function for regex pattern kind is built through _buildregexmatch() and _buildmatch() using _rematcher() that returns a re.match function, which either returns a match object or None. This does not conform to Mercurial's matcher interface for __call__() or exact(), which are expected to return a boolean value. We fix this by building a lambda around _rematcher() in _buildregexmatch(). Accordingly, we update doctest examples to remove bool() calls that are now useless.

match: make arguments of _expandsets() optional Arguments 'ctx', 'listsubrepos' and 'badfn' are optional in function body.

match: make _donormalize's auditor and warn arguments optional Argument 'warn' is actually non-required, since there's a 'if warn:' check before usage. Argument 'auditor' is passed to pathutil.canonpath(), in which it is optional.

match: add doctest examples in match() Make the docstring raw, as it now includes escape characters.

match: complete documentation of match() parameters

match: add doctest examples for patkind()

match: add a docstring with doctest examples to patternmatcher Doctest examples aim at illustrating how __call__() and exact() are different, depending on the pattern kind.

match: add doctest examples for exactmatcher Make the docstring raw, since it now includes escape characters.

localrepo: don't allow lookup of working directory revision It seems that repo.lookup(), which is what supports the "lookup" wire protocol command, should not allow the working copy revision input. This fixes both the pull test and the convert test I just added. Differential Revision: https://phab.mercurial-scm.org/D6215

tests: demonstrate broken pull of "ffffffffffff" revision Differential Revision: https://phab.mercurial-scm.org/D6214

tests: demonstrate broken `hg convert` if "ffffffffffff" is in description Differential Revision: https://phab.mercurial-scm.org/D6213

tests: add test of for hash reference translation by `hg convert` The convert extension translates commit references in the commit message. We didn't have any explicit testing of this before, so let's add a test. Differential Revision: https://phab.mercurial-scm.org/D6212

py3: write out hgextindex as bytes in setup.py I hit this trying to build the py2exe target using python3, just to see what would happen. After commenting out `py2exe.Distribution` in setup.py and pointing to a local copy of py2exe that supports python3[1], it complained that `out` was bytes, not str. [1] https://github.com/albertosottile/py2exe/releases/tag/v0.9.3.0

setup: fix a possible NameError on rust build File "setup.py", line 975, in rustbuild "command: %r, environment: %r" % (self.rustsrcdir, cmd, env)) NameError: global name 'cmd' is not defined

crecord: new keys g & G to navigate to the top and bottom respectively This patch introduces two new keys 'g' and 'G' that helps to navigate to the top and bottom of the file/hunk/line respectively. This is inline with the shortcuts used in man, less, more and such tools that makes it convenient to navigate swiftly. 'g' or HOME navigates to the top most file in the ncurses window. 'G' or END navigates to the bottom most file/hunk/line depending on the whether the fold is active or not. If the bottom most file is folded, it navigates to that file and stops there. If the bottom most file is unfolded, it navigates to the bottom most hunk in that file and stops there. If the bottom most hunk is unfolded, it navigates to the bottom most line in that hunk. Differential Revision: https://phab.mercurial-scm.org/D6178

chistedit: properly show verbose diffs I'm not sure if that ever worked and it's an internal API breakage, but `"verbose": True` is not correctly parsed, as most of these options are parsed by diffopts, whereas verbose is a global option. Setting the UI to verbose instead does work and does show a verbose patch, with full commit message. It also shows all files, which unfortunately are a bit hard to read on a single line in the default verbose template. Thus, we also change the default template to use the status template, which shows one file per line as well as its modification state.

interactive: do not prompt about files given in command line For commit and revert commands with --interactive and explicit files given in the command line, we now skip the invite to "examine changes to <file> ? [Ynesfdaq?]". The reason for this is that, if <file> is specified by the user, asking for confirmation is redundant. In patch.filterpatch(), we now use an optional "match" argument to conditionally call the prompt() function when entering a new "header" item. We use .exact() method to compare with files from the "header" in order to only consider (rel)path patterns. Add tests with glob patterns for commit and revert, to make sure we still ask to examine files in these cases.

zstandard: vendor python-zstandard 0.11 The upstream source distribution from PyPI was extracted. Unwanted files were removed. The clang-format ignore list was updated to reflect the new source of files. The project contains a vendored copy of zstandard 1.3.8. The old version was 1.3.6. This should result in some minor performance wins. test-check-py3-compat.t was updated to reflect now-passing tests on Python 3.8. Some HTTP tests were updated to reflect new zstd compression output. # no-check-commit because 3rd party code has different style guidelines Differential Revision: https://phab.mercurial-scm.org/D6199