rebase: clear merge state when aborting in-memory merge on dirty working copy Differential Revision: https://phab.mercurial-scm.org/D9509

tests: show that in-memory rebase leaves state when working copy is dirty When in-memory rebase falls back to on-disk rebase, it checks if the working copy is dirty. If it is, it aborts the rebase. However, it leaves the rebase state on disk. I broke it in feffeb18d412 (rebase: teach in-memory rebase to not restart with on-disk rebase on conflict, 2020-09-18). Differential Revision: https://phab.mercurial-scm.org/D9508

extensions: avoid a crash when the version isn't properly byteified on py3 We already force bytestr on the `testedwith` and `buglink` attributes in dispatch.py when generating a bug report with a similar comment about not every extension being ported to py3. We should do the same here, so the function can be properly typed. Differential Revision: https://phab.mercurial-scm.org/D9433

ui: ensure `getpass()` returns bytes Previously, this could return either bytes or str. I'm not sure which direction we should go in, but since the input is bytes, I guess bytes makes sense as output. `debuguigetpass` crashed because it assumed bytes would be returned, `sslcontext.load_cert_chain()` is happy with bytes or str if the type info in PyCharm is correct, and `smtplib.SMTP.login()` wants str. I couldn't figure out how to test this, because the test stalls for input with `echo test | hg debuguigetpass --config ui.interactive=1`, likely because it drains stdin before prompting. The custom input reading with `ui.nontty=1` does not. I'm also a bit concerned with all of this encoding/decoding. The existing code in the mail module uses `encoding.strfromlocal()`, but the username and password are ascii encoded/decoded in `mercurial.url.passwordmgr.find_user_password()` with `pycompat.{str,bytes}url()`. I'm not sure if this inconsistency could cause subtle compatability issues. Differential Revision: https://phab.mercurial-scm.org/D9375

packaging: regenerate the Windows requirements manifest on Windows SecretStorage is a Linux package, and the other stuff removed is a dependency of it. I assume this was last generated on Linux, and noticed this trying to add another package and regenerating on Windows. Differential Revision: https://phab.mercurial-scm.org/D9404

pyoxidizer: point to the py3 requirements instead of py2 on Windows Differential Revision: https://phab.mercurial-scm.org/D9406

extensions: gracefully warn when doing min version check with no local version After doing a `make clean`, I started getting cryptic failures to import extensions with the `minimumhgversion` attribute on py3: *** failed to import extension evolve: '>' not supported between instances of 'int' and 'NoneType' *** failed to import extension topic: '>' not supported between instances of 'int' and 'NoneType' This now handles the `(None, None)` tuple before comparing, and disables the extension with the same friendly message as in py2. Differential Revision: https://phab.mercurial-scm.org/D9363

diff: do not concatenate immutable bytes while building a/b bodies (issue6445) Use bytearray instead. I don't know what's changed since Python 2, but bytes concatenation is 100x slow on Python 3. % python2.7 -m timeit -s "s = b''" "for i in range(10000): s += b'line'" 1000 loops, best of 3: 321 usec per loop % python3.9 -m timeit -s "s = b''" "for i in range(10000): s += b'line'" 5 loops, best of 5: 39.2 msec per loop Benchmark using tailwind.css (measuring the fast path, a is empty): % HGRCPATH=/dev/null python2.7 ./hg log -R /tmp/issue6445 -p --time \ --color=always --config diff.word-diff=true >/dev/null (prev) time: real 1.580 secs (user 1.560+0.000 sys 0.020+0.000) (this) time: real 1.610 secs (user 1.570+0.000 sys 0.030+0.000) % HGRCPATH=/dev/null python3.9 ./hg log -R /tmp/issue6445 -p --time \ --color=always --config diff.word-diff=true >/dev/null (prev) time: real 114.500 secs (user 114.460+0.000 sys 0.030+0.000) (this) time: real 2.180 secs (user 2.140+0.000 sys 0.040+0.000) Benchmark using random tabular text data (not the fast path): % dd if=/dev/urandom bs=1k count=1000 | hexdump -v -e '16/1 "%3u," "\n"' > ttf % hg ci -ma % dd if=/dev/urandom bs=1k count=1000 | hexdump -v -e '16/1 "%3u," "\n"' > ttf % hg ci -mb % HGRCPATH=/dev/null python2.7 ./hg log -R /tmp/issue6445 -p --time \ --color=always --config diff.word-diff=true >/dev/null (prev) time: real 3.240 secs (user 3.040+0.000 sys 0.200+0.000 (this) time: real 3.230 secs (user 3.070+0.000 sys 0.160+0.000) % HGRCPATH=/dev/null python3.9 ./hg log -R /tmp/issue6445 -p --time \ --color=always --config diff.word-diff=true >/dev/null (prev) time: real 44.130 secs (user 43.850+0.000 sys 0.270+0.000) (this) time: real 4.170 secs (user 3.850+0.000 sys 0.310+0.000)

procutil: use rapply(tonativestr, ...) to preserve lists when they come in This was broken when script was a list instead of a string. I caught this with an internal extension at Google, and I'm not really sure why it wasn't caught in any kind of CI. Differential Revision: https://phab.mercurial-scm.org/D9471

statprof: separate functions and "line", assume 4 digit line numbers Previously, the profile output looked like this (I've removed many lines that are mostly inconsequential): ``` | 100.0% 0.02s hg: <module> line 43: dispatch.run() | 100.0% 0.02s dispatch.py: run line 115: status = dispatch(req) | 100.0% 0.02s dispatch.py: _runcatchfunc line 432: return _dispatch(req) \ 50.0% 0.01s dispatch.py: _dispatch line 1228: return runcommand( | 50.0% 0.01s dispatch.py: runcommand line 883: ret = _runcommand(ui, optio... | 50.0% 0.01s dispatch.py: _runcommand line 1240: return cmdfunc() | 50.0% 0.01s localrepo.py: __getitem__ line 1670: quick_access = self._quick_... | 50.0% 0.01s localrepo.py: _quick_access_changeidline 1650: return self._quick_access_c... | 50.0% 0.01s localrepo.py: __get__ line 179: return getattr(unfi, self.n... | 50.0% 0.01s util.py: __get__ line 1747: result = self.func(obj) | 50.0% 0.01s localrepo.py: _quick_access_changeid_wcline 1611: cl = self.unfiltered().chan... | 50.0% 0.01s localrepo.py: __get__ line 110: return super(_basefilecache... | 50.0% 0.01s util.py: __getattribute__line 245: self.__spec__.loader.exec_m... | 50.0% 0.01s <frozen importlib._bootstrap_external>: exec_moduleline 783: | 50.0% 0.01s <frozen importlib._bootstrap>: _call_with_frames_removedline 219: | 50.0% 0.01s changelog.py: <module> line 376: class changelog(revlog.revl... | 50.0% 0.01s util.py: __getattribute__line 245: self.__spec__.loader.exec_m... | 50.0% 0.01s <frozen importlib._bootstrap_external>: exec_moduleline 779: | 50.0% 0.01s <frozen importlib._bootstrap_external>: get_codeline 868: | 50.0% 0.01s <frozen importlib._bootstrap_external>: path_statsline 1012: | 50.0% 0.01s <frozen importlib._bootstrap_external>: _path_statline 87: ``` This has a few problems, though I'm only addressing some of them. 1. If the stuff before "line ###" is long, there's no separation between the function name and the "line" string. 2. If the stuff before "line ###" is really long, there's excessive separation between the "line" string and the line number. 3. We frequently have 4-digit line numbers, the code on the right wasn't dynamically indented and ended up quite messy looking. To solve these problems, I've added a ", " prefix before "line" iff it would otherwise not have any separation such as spaces. I've added a 'max' so that we never use a negative width (which is the cause of problem #2 above), and I've added a default assumption of 4 digit line numbers (but again using a 'max' so this shouldn't cause problems if we go beyond that. With these changes, it now looks like this: ``` | 100.0% 0.02s hg: <module> line 43: dispatch.run() | 100.0% 0.02s dispatch.py: run line 115: status = dispatch(req) | 100.0% 0.02s dispatch.py: _runcatchfunc line 432: return _dispatch(req) \ 50.0% 0.01s dispatch.py: _dispatch line 1228: return runcommand( | 50.0% 0.01s dispatch.py: runcommand line 883: ret = _runcommand(ui, optio... | 50.0% 0.01s dispatch.py: _runcommand line 1240: return cmdfunc() | 50.0% 0.01s localrepo.py: __getitem__ line 1670: quick_access = self._quick_... | 50.0% 0.01s localrepo.py: _quick_access_changeid, line 1650: return self._quick_access_c... | 50.0% 0.01s localrepo.py: __get__ line 179: return getattr(unfi, self.n... | 50.0% 0.01s util.py: __get__ line 1747: result = self.func(obj) | 50.0% 0.01s localrepo.py: _quick_access_changeid_wc, line 1611: cl = self.unfiltered().chan... | 50.0% 0.01s localrepo.py: __get__ line 110: return super(_basefilecache... | 50.0% 0.01s util.py: __getattribute__, line 245: self.__spec__.loader.exec_m... | 50.0% 0.01s <frozen importlib._bootstrap_external>: exec_module, line 783: | 50.0% 0.01s <frozen importlib._bootstrap>: _call_with_frames_removed, line 219: | 50.0% 0.01s changelog.py: <module> line 376: class changelog(revlog.revl... | 50.0% 0.01s util.py: __getattribute__, line 245: self.__spec__.loader.exec_m... | 50.0% 0.01s <frozen importlib._bootstrap_external>: exec_module, line 779: | 50.0% 0.01s <frozen importlib._bootstrap_external>: get_code, line 868: | 50.0% 0.01s <frozen importlib._bootstrap_external>: path_stats, line 1012: | 50.0% 0.01s <frozen importlib._bootstrap_external>: _path_stat, line 87: ``` Differential Revision: https://phab.mercurial-scm.org/D9511

statprof: fix off-by-one-line error in output martinvonz claims they thought that this was intentional, but couldn't remember the reasoning for it. I can't understand why it would be preferable, and I didn't see anything in the comments in the file about why this would be useful, so I'm hopefully not breaking anything by "fixing" it. ### Old output ``` | 100.0% 0.01s dispatch.py: run line 43: dispatch.run() | 100.0% 0.01s dispatch.py: dispatch line 115: status = dispatch(req) | 100.0% 0.01s dispatch.py: _runcatch line 266: ret = _runcatch(req) or 0 | 100.0% 0.01s dispatch.py: _callcatch line 442: return _callcatch(ui, _runc... | 100.0% 0.01s scmutil.py: callcatch line 451: return scmutil.callcatch(ui... | 100.0% 0.01s dispatch.py: _runcatchfunc line 155: return func() | 100.0% 0.01s dispatch.py: _dispatch line 432: return _dispatch(req) | 100.0% 0.01s hg.py: repository line 1179: repo = hg.repository( | 100.0% 0.01s hg.py: _peerorrepo line 221: peer = _peerorrepo( | 100.0% 0.01s util.py: __getattribute__ line 188: obj = _peerlookup(path).ins... | 100.0% 0.01s localrepo.py: makelocalrepositoryline 3227: return makelocalrepository(... | 100.0% 0.01s localrepo.py: __init__ line 683: return cls( | 100.0% 0.01s util.py: __getattribute__ line 1262: self._extrafilterid = repov... | 100.0% 0.01s <frozen importlib._bootstrap_external>: exec_moduleline 245: self.__spec__.loader.exec_m... | 100.0% 0.01s <frozen importlib._bootstrap_external>: get_codeline 779: | 100.0% 0.01s <frozen importlib._bootstrap_external>: path_statsline 868: | 100.0% 0.01s <frozen importlib._bootstrap_external>: _path_statline 1012: ``` ### New output ``` | 100.0% 0.01s hg: <module> line 43: dispatch.run() | 100.0% 0.01s dispatch.py: run line 115: status = dispatch(req) | 100.0% 0.01s dispatch.py: dispatch line 266: ret = _runcatch(req) or 0 | 100.0% 0.01s dispatch.py: _runcatch line 442: return _callcatch(ui, _runc... | 100.0% 0.01s dispatch.py: _callcatch line 451: return scmutil.callcatch(ui... | 100.0% 0.01s scmutil.py: callcatch line 155: return func() | 100.0% 0.01s dispatch.py: _runcatchfunc line 432: return _dispatch(req) | 100.0% 0.01s dispatch.py: _dispatch line 1179: repo = hg.repository( | 100.0% 0.01s hg.py: repository line 221: peer = _peerorrepo( | 100.0% 0.01s hg.py: _peerorrepo line 188: obj = _peerlookup(path).ins... | 100.0% 0.01s localrepo.py: instance line 3227: return makelocalrepository(... | 100.0% 0.01s localrepo.py: makelocalrepositoryline 683: return cls( | 100.0% 0.01s localrepo.py: __init__ line 1262: self._extrafilterid = repov... | 100.0% 0.01s util.py: __getattribute__ line 245: self.__spec__.loader.exec_m... | 100.0% 0.01s <frozen importlib._bootstrap_external>: exec_moduleline 779: | 100.0% 0.01s <frozen importlib._bootstrap_external>: get_codeline 868: | 100.0% 0.01s <frozen importlib._bootstrap_external>: path_statsline 1012: | 100.0% 0.01s <frozen importlib._bootstrap_external>: _path_statline 87: ``` Differential Revision: https://phab.mercurial-scm.org/D9510

phab-refresh: do not error out when the stack is empty Actually, empty stack is easier to get than I expected (and harmless). When a stack is publish, heptapod will detect the even, empty the stack (triggering CI) and "forgetting" the branch afterward. So we stop returning as error in this case. Differential Revision: https://phab.mercurial-scm.org/D9513

run-tests: allow some slack about 'waiting on lock' message It is common to run the tests on very loaded machine when concurrent run might take a bit longer. Such message are usually harmless, but anoying as they break the tests. Test that explicitly depends on this value have been adjusted. This make them more robust anyway. A fun case was `test-clone-pull-corruption.t` which, without the previous changeset introducing extra flushing, ended use having a line 31 (`pulling from ../source`) changing order because the warning message was no longer flushing stdin before using stderr (stderr being invisible in the test). Differential Revision: https://phab.mercurial-scm.org/D9507

pull: flush stdin after the `pull from` message That message can end up being flushed after some stderr message in some case, leading to confusing output. Differential Revision: https://phab.mercurial-scm.org/D9506

tests: explicitly skip the lock warning in some remotefilelog tests The output was conditional zed, so lets official skip it instead. Differential Revision: https://phab.mercurial-scm.org/D9505

tests: use the right python when running dummyssh for narrow some plateform no longer have a `python` executable. Differential Revision: https://phab.mercurial-scm.org/D9504

copies: avoid materializing a full directory map during copy tracing Materializing a full copy of every directory in a treemanifest repo can be quite expensive, even with a narrow matcher. For flat manifest repos, this should be equivalent - it will still materialize (and cache) a dict of all of the dirs inside of the manifest object, we just don't get a copy of it. In a repo I have here, this brings the time for a simple rebase from 11.197s to 4.609s. Differential Revision: https://phab.mercurial-scm.org/D9503

rebase: clear merge state when aborting in-memory merge on dirty working copy Differential Revision: https://phab.mercurial-scm.org/D9509

tests: show that in-memory rebase leaves state when working copy is dirty When in-memory rebase falls back to on-disk rebase, it checks if the working copy is dirty. If it is, it aborts the rebase. However, it leaves the rebase state on disk. I broke it in feffeb18d412 (rebase: teach in-memory rebase to not restart with on-disk rebase on conflict, 2020-09-18). Differential Revision: https://phab.mercurial-scm.org/D9508

helptext: document share safe functionality in `hg help config -v` If share safe functionality is enabled, we read `.hg/hgrc' of shared source. Differential Revision: https://phab.mercurial-scm.org/D9413

helptext: mention in `hg help config` that `.hg/hgrc-not-shared` is consulted Recently we added `.hg/hgrc-not-shared` which is a config file which won't be shared with shares when share-safe mode is enabled. Irrespective of whether we are using share-safe or not, we consult this file now. Let's document that. Differential Revision: https://phab.mercurial-scm.org/D9412

share: add documentation about share-safe mode in `hg help -e share` Differential Revision: https://phab.mercurial-scm.org/D9411

helptext: update first hg version when share-safe will be released I authored the patch which added the helptext before 5.6 release hoping that my patches will make it. However they didn't before the release and were pushed after the release only. Differential Revision: https://phab.mercurial-scm.org/D9410

share: show warning if share is outdated while source supports share-safe Previous patches in the series and some which are already committed implements share safe functionality where config and requirements will be shared too. Rolling this feature has a problem that existing shares may never upgrade as they will never learn about the new config. To help the transition, we show a warning message if the share source supports share-safe mechanism. This provides the source repo ability to upgrade and pass on the message to shares that you should reshare and upgrade too. Differential Revision: https://phab.mercurial-scm.org/D9369

upgrade: add support to downgrade share safe mode In previous patch we added support to upgrade current repository to use share safe mode. This patch adds support to downgrade to remove share-safe mode. Differential Revision: https://phab.mercurial-scm.org/D9358

upgrade: add support for experimental safe share mode Recently we introduce the share-safe functionality which makes shares share requirements and config of share source. This patch adds support to `debugupgraderepo` command to upgrade repository to share-safe mode when `format.exp-share-safe` config is enabled. Differential Revision: https://phab.mercurial-scm.org/D9144

chgserver: catch RepoError while loading configuration Recent share safe work introduced functionality to read share source config file on dispatch. This can result in RepoError while reading config file as the shared source might not be present. `test-share.t#safe` was failing with chg earlier because of this. Differential Revision: https://phab.mercurial-scm.org/D9462

registrar: clarify the documentation about some byte strings being required I *thought* these needed to be byte strings, but didn't remember and had to search out examples. Differential Revision: https://phab.mercurial-scm.org/D9489

match: skip walking up the directory hierarchy if the number of pats are small Previously, we would receive a path like abc/def/ghi and "walk up" the directory hierarchy, checking abc/def, abc, and `b''` to see if they were in the set of prefixes that this matcher covered. We did this indiscriminately - we generated all of these paths even if the set of prefixes the matcher covered was completely empty, which is the case for a lot of repos at my company (the narrow matcher we use is usually non-recursive). This brings the time for a rebase in one of my repos from 12.20s to 10.87s. In this particular repo, this is entirely due to the `len(prefix_set) == 0` check, as I do not have any recursive patterns in the narrowspec. Differential Revision: https://phab.mercurial-scm.org/D9488

relnotes: document better memory use for unbundle Differential Revision: https://phab.mercurial-scm.org/D9481