black: merge config into main pyproject.toml now that we have it This means that naive contributors who just run `black` on a source file will get reasonable behavior as long as they have a recent black. Yay! Differential Revision: https://phab.mercurial-scm.org/D9834

pyproject: add config file This will tell pip et al to call our setup.py for the majority of packaging concerns, but also gives us a place to put standard config stuff like black. Differential Revision: https://phab.mercurial-scm.org/D9833

upgrade: update only requirements if we can Upgrade operations which involves just upgrading requirements earlier used to go through whole revlog cloning business. Now we just upgrade the requirement and skip the cloning part. Differential Revision: https://phab.mercurial-scm.org/D9775

engine: add `if True` to prepare for next patch This will help making next patch easier to read. Differential Revision: https://phab.mercurial-scm.org/D9774

test: unquiet few tests to demonstrate changes in upcoming patches Upcoming patches will skip revlog cloning for share-safe upgrades. Differential Revision: https://phab.mercurial-scm.org/D9773

upgrade: mark sharesafe improvement as only touching requirements Upgrading the repository to use share safe functionality only touches requirements. Differential Revision: https://phab.mercurial-scm.org/D9772

actions: calculate what all parts does the operation touches To make upgrade work less in certain situations, we need to teach it to find out what all parts of repository current operation is touching. This patch starts doing that. In next patch we will be setting values in improvement objects. For now, we assume everything touches everything. Differential Revision: https://phab.mercurial-scm.org/D9771

upgrade: don't create store backup if `--no-backup` is passed If the user explicitly mentioned that they don't need backup, then let's not create it. Differential Revision: https://phab.mercurial-scm.org/D9770

cext: add Python 3.10 support * Replace "Py_TYPE(obj) = type;" with "Py_SET_TYPE(obj, type);" * Add pythoncapi_compat.h header file to get Py_SET_TYPE() on Python 2.7-3.8. Header file added to mercurial/ and contrib/python-zstandard/zstd/common/. In Python 3.10, Py_TYPE(obj) must not longer be used as an l-value. pythoncapi_compat.h comes from: https://github.com/pythoncapi/pythoncapi_compat Differential Revision: https://phab.mercurial-scm.org/D9825

revlog: decouple caching from addrevision callback for addgroup For changesets, it is useful to cache the content as it will almost always be processed afterwards. For manifests on the other hand, the content is often not used directly as there is a fast path for deltas. Explicitly disable the cache in exchangev2's manifest handling for that reason. Differential Revision: https://phab.mercurial-scm.org/D9847

branchmap: update rev-branch-cache incrementally Historically, the revision to branch mapping cache was updated on demand and shared via bundle2 to avoid the cost of rebuilding on first use. Use the new `register_changeset` callback and update rbc directly on every change. Make the transfer of the bundle part redundant, but keep it for the moment to avoid the test churn. Over all, "hg unbundle" for large bundles is less than 1.8% slower for different larger repositories and that seems to a reasonable trade off. Differential Revision: https://phab.mercurial-scm.org/D9781

repository: introduce register_changeset callback The new callback is called whenever a changeset is added to the repository (commit, unbundle or exchange). Since the bulk operations already parse the changeset (readfiles or full changesetrevision), always use the latter to avoid redundant lookups. The first consumer of the new interface needs to look at extra. Differential Revision: https://phab.mercurial-scm.org/D9780

changelog: move branchinfo to changelogrevision The function parses the extra dictionary after looking up the changelogrevision. To avoid duplicated look up, it is better to provide it as property of changelogrevision instead. Keep the function for a release cycle as at least the topic extension depends on it. Differential Revision: https://phab.mercurial-scm.org/D9779

rust: lower compile error on non-linux platforms to a warning As discussed on D9671#146704 this lowers the error to a warning. Differential Revision: https://phab.mercurial-scm.org/D9852

purge: move extension into core mercurial The motivation is simple: it's nicer to avoid gating basic functionality. To reduce the risk of people shooting themselves in the feet, `--confirm` is now the default, unless the extensions is loaded.. For review of the body of the purge command, use this instead of what hg/phabricator will show (the block of code is modified, not just moved): Differential Revision: https://phab.mercurial-scm.org/D9820

purge: also deal with directory with --confirm Getting the exact number is simple (you have to project the result of previous deletion to detect super-directory with only about-to-be-deleted content). So we use a vaguer message. Differential Revision: https://phab.mercurial-scm.org/D9819

purge: add a --confirm option The options provide a prompt to the user before permanent deletion are made. The prompt is currently not aware of directory deletion. I'll fix this in the next changesets. Differential Revision: https://phab.mercurial-scm.org/D9818

filemerge: add a hacktastic version of internal:merge3 for merge diffs This is a version of merge3 that always reports success, so that conflict markers get preserved without us having to implement conflict storage for in-memory merge. Credit to martinvonz for the idea. The only planned consumer of this "merge tool" is my upcoming merge-diffs functionality, though I suspect it could be useful in other ways. Differential Revision: https://phab.mercurial-scm.org/D8515

rebase: add a config knob for forcing in-memory rebasing Use it in a test case where we know the rebase should proceed without incident in-memory, so we can see tracebacks rather than fallbacks. This makes it easier to try and debug when things are broken. Differential Revision: https://phab.mercurial-scm.org/D8623

cleanup: use mergestate.unresolvedcount() instead of bool(list(unresolved())) This avoids some pointless copying. Differential Revision: https://phab.mercurial-scm.org/D8566

reverse-branch-cache: switch to doubling allocating scheme In preperation for updating the reverse-branch-cache incrementally whenever a new changeset comes in, avoid bad performance on resize with Python 3.7 (and likely other 3.x versions). Differential Revision: https://phab.mercurial-scm.org/D9778

discovery: add config options to control sample size This get useful for running analysis of the current algorithm. Differential Revision: https://phab.mercurial-scm.org/D9813

exchangev2: avoid second look-up by node Accessing the revlog by node is slightly more expensive than by revision, so look up the revision first and use it afterwards. Differential Revision: https://phab.mercurial-scm.org/D9831

commit: look-up new revision once Look-up by node is slightly more expensive, so since it is necessary more than once, do it explicitly. Differential Revision: https://phab.mercurial-scm.org/D9830

resolve: also detect new :mergediff conflict markers The conflict markers created by `:mergediff` were not detected as conflicts, which affects both `commands.resolve.mark-check` and `mergetools.<tool>.check`. This patch fixes that. The new regex it uses for finding conflict markers is less restrictive because it `:mergediff` doesn't follow the `<<<<<<<` and `>>>>>>>` lines by a space (and a description). Hopefully lines like that don't give too many false positives. We can add back the space and make `:mergediff` add trailing spaces if it turns out to be a problem. OTOH, there will always be some false positives and we have ways of overriding the checks already. This patch can go onto the default or stable branch, depending on how much we care about an experimental feature. Differential Revision: https://phab.mercurial-scm.org/D9835

simplemerge: delete unused find_unconflicted() The function has been unused ever since it was introduced in 465b9ea02868 (Import 3-way merge code from bzr, 2007-04-16). Differential Revision: https://phab.mercurial-scm.org/D9832

perf: use the `perf--` prefix for perf command This is the one command namespace where they should not be any ambiguity about command that should be in it. The perf extensions is only adding performance related command. so this is a good ground to start putting dash folding to the tests. Differential Revision: https://phab.mercurial-scm.org/D9516

merge with stable

share: move share safe functionality out of experimental The share-safe functionality is complete and all configuration options are implemented. The behavior is well discussed on mailing list and in reviews. Let's unmark this as experimental to solve a chichen and egg issue. Differential Revision: https://phab.mercurial-scm.org/D9823

share: rename share-safe warning config Config introduced in previous patch was `share.source-safe-mismatch`. Let's rename the warn as `share.source-safe-mismatch.warn`. While we are here, made sure we have different configs for upgrade and downgrade. Differential Revision: https://phab.mercurial-scm.org/D9786

share: rework config options to be much clearer and easier Recently I implemented various boolean configs which control how to behave when there is a share-safe mismatch between source and share repository. Mismatch means that source supports share-safe where as share does not or vice versa. However, while discussion and documentation we realized that it's too complicated and there are some combinations of values which makes no sense. We decided to introduce a config option with 4 possible values which makes controlling and understanding things easier. The config option `share.safe-mismatch.source-{not-}safe` can have following 4 values: * abort (default): error out if there is mismatch * allow: allow to work with respecting share source configuration * {up|down}grade-abort: try to {up|down}grade, if it fails, abort * {up|down}grade-allow: try to {up|down}grade, if it fails, continue in allow mode I am not sure if I can explain 3 config options which I deleted right now in just 5 lines which is a sign of how complex they became. No test changes demonstrate that functionality is same, only names have changed. Differential Revision: https://phab.mercurial-scm.org/D9785

typing: disable processing of mercurial/upgrade_utils/actions.py There was a recent regression in pytype that causes processing to stall and consume memory, and it doesn't sound like it will be fixed any time soon. Differential Revision: https://phab.mercurial-scm.org/D10273

re2: feed unicode string to re2 module when necessary My previous test were using the `pyre2` Python project, that wrap the Google RE2 library in python as a `re2` module and accept bytes as input. However the `fb-re2` Python project is also offering a wrapping of the Google RE2 library in python as a `re2` module ans accept only unicode on python3. So we detect this situation and convert thing to unicode when necessary. Hooray… We should consider using a rust wrapping for regexp handling. We needs regexps in Rust anyway and this give use more control with less variants and more sanity. Differential Revision: https://phab.mercurial-scm.org/D10284

typing: rewrite a conditional assignment to unconfuse pytype Otherwise, pytype and PyCharm flags it: File "/mnt/c/Users/Matt/hg/mercurial/localrepo.py", line 2903, in wlock: No attribute 'held' on _weakref.ReferenceType[nothing] [attribute-error] In Optional[Union[Any, _weakref.ReferenceType[nothing]]] File "/mnt/c/Users/Matt/hg/mercurial/localrepo.py", line 2904, in wlock: No attribute 'lock' on _weakref.ReferenceType[nothing] [attribute-error] In Optional[Union[Any, _weakref.ReferenceType[nothing]]] Differential Revision: https://phab.mercurial-scm.org/D10215

typing: add assertions to localrepo.py to appease pytype Differential Revision: https://phab.mercurial-scm.org/D10214

obsutil: maintain a homogenous list when computing successors In practice, this wasn't a problem because of an internal check for an empty list, but it avoids the following pytype error: File "/mnt/c/Users/Matt/hg/mercurial/obsutil.py", line 823, in successorsandmarkers: No attribute 'markers' on List[nothing] [attribute-error] In Union[Any, List[nothing], _succs] Differential Revision: https://phab.mercurial-scm.org/D10213

typing: add an assertion to verify.py to appease pytype Differential Revision: https://phab.mercurial-scm.org/D10212

typing: add an assertion to mercurial/hgweb/webcommands.py to help pytype I have no idea why this is needed, given the check immediately above. But without it, I'm getting this: File "/mnt/c/Users/Matt/hg/mercurial/hgweb/webcommands.py", line 1164, in filelog: Missing parameter 'toline' in call to function mercurial.dagop.blockdescendants [missing-parameter] Expected: (fctx, fromline, toline) Actually passed: (fctx, fromline) File "/mnt/c/Users/Matt/hg/mercurial/hgweb/webcommands.py", line 1166, in filelog: Missing parameter 'toline' in call to function mercurial.dagop.blockancestors [missing-parameter] Expected: (fctx, fromline, toline, followfirst) Actually passed: (fctx, fromline) Differential Revision: https://phab.mercurial-scm.org/D10211

crecord: null out the curses attribute with `None` on failure to import Pytype got really confused that this could be `Union[module, bool]`, and spewed about tons of attributes that are not available on `bool`. Differential Revision: https://phab.mercurial-scm.org/D10210

typing: fix directives mangled by black Differential Revision: https://phab.mercurial-scm.org/D10209

typing: disable module attribute warnings for properly conditionalized code Differential Revision: https://phab.mercurial-scm.org/D10208

typing: disable import error warnings that are already handled I'm assuming that the cffi ones are handled somewhere, but the others definitely are. Differential Revision: https://phab.mercurial-scm.org/D10207

typing: disable a few errors when accessing Windows specific attributes This is already guarded with conditionals. The entire win32 module is conditionally loaded. File "/mnt/c/Users/Matt/hg/mercurial/utils/procutil.py", line 162, in <module>: No attribute 'winstdout' on module 'mercurial.posix' [module-attr] File "/mnt/c/Users/Matt/hg/mercurial/utils/procutil.py", line 163, in <module>: No attribute 'winstdout' on module 'mercurial.posix' [module-attr] File "/mnt/c/Users/Matt/hg/mercurial/utils/procutil.py", line 174, in <module>: No attribute 'winstdout' on module 'mercurial.posix' [module-attr] File "/mnt/c/Users/Matt/hg/mercurial/utils/procutil.py", line 175, in <module>: No attribute 'winstdout' on module 'mercurial.posix' [module-attr] Differential Revision: https://phab.mercurial-scm.org/D10206

typing: disable a few errors calling py3.7+ functions in resourceutil.py There's a graceful fallback when these methods are unavailable. Differential Revision: https://phab.mercurial-scm.org/D10205

formatting: fix redundant parentheses These were introduced by 0d055849d5f9d682ef931d2566b760d5c6bf7e52. Differential Revision: https://phab.mercurial-scm.org/D10229

wsgicgi: switch the default PATH_INFO back to str It looks like everything here is meant to be str, and regressed in 687b865b95ad. Pytype didn't flag this, but is really confused about the range of possible types for the dictionary values, and flagged missing `lower()` on various types in this area. Differential Revision: https://phab.mercurial-scm.org/D10204

crecord: convert an instance of bytes to str Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/crecord.py", line 1154, in printstring: Function _CursesWindow.addstr was called with the wrong arguments [wrong-arg-types] Expected: (self, str: str, ...) Actually passed: (self, str: bytes, ...) Differential Revision: https://phab.mercurial-scm.org/D10203

crecord: change str literals to byte literals Differential Revision: https://phab.mercurial-scm.org/D10202

bisect: replace a missing method on the list primitive Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/commands.py", line 1095, in bisect: No attribute 'last' on list [attribute-error] In list Differential Revision: https://phab.mercurial-scm.org/D10201

typing: disable a module-attr warning in the worker module's py2 code Differential Revision: https://phab.mercurial-scm.org/D10183

wireprotoserver: convert ErrorResponse to bytes Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/wireprotoserver.py", line 236, in handlewsgirequest: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: mercurial.hgweb.common.ErrorResponse) The following methods aren't implemented on mercurial.hgweb.common.ErrorResponse: __iter__ File "/mnt/c/Users/Matt/hg/mercurial/wireprotoserver.py", line 239, in handlewsgirequest: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: mercurial.hgweb.common.ErrorResponse) The following methods aren't implemented on mercurial.hgweb.common.ErrorResponse: __iter__ File "/mnt/c/Users/Matt/hg/mercurial/wireprotov2server.py", line 91, in handlehttpv2request: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: mercurial.hgweb.common.ErrorResponse) The following methods aren't implemented on mercurial.hgweb.common.ErrorResponse: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10182

verify: convert an exception to bytes before logging I'm not entirely sure why this code appears to be trying to convert twice, but it was flagged by pytype: File "/mnt/c/Users/Matt/hg/mercurial/verify.py", line 84, in _exc: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: Exception) The following methods aren't implemented on Exception: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10181

typing: add an assertion to the upgrade engine to help pytype Differential Revision: https://phab.mercurial-scm.org/D10180

subrepo: handle unexpected file types from git gracefully This was flagged by pytype because `tar.extractfile(...)` can return None if the entry is not a file or symlink. I don't think that git supports other types, but better safe than sorry. Differential Revision: https://phab.mercurial-scm.org/D10179

patch: convert a UI message to bytes when editing a patch Differential Revision: https://phab.mercurial-scm.org/D10178

merge: force an exception message to bytes before printing as a warning Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/merge.py", line 1346, in batchremove: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: str) Differential Revision: https://phab.mercurial-scm.org/D10177

hg: convert an exception to bytes in the repo creation exception handler Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/hg.py", line 77, in _local: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: Union[TypeError, ValueError]) Differential Revision: https://phab.mercurial-scm.org/D10176

typing: add an assertion instead of blacklisting mercurial/extensions.py Differential Revision: https://phab.mercurial-scm.org/D10175

debug: convert a few exceptions to bytes before wrapping in another error Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/debugcommands.py", line 2118, in debugmanifestfulltextcache: Function Abort.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, message: Union[bytearray, bytes, memoryview], ...) Actually passed: (self, message: mercurial.error.LookupError, ...) File "/mnt/c/Users/Matt/hg/mercurial/debugcommands.py", line 2453, in debugobsolete: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: ValueError) The following methods aren't implemented on ValueError: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10174

grep: convert an exception to bytes for a warning message Caught by pytype: File "/mnt/c/Users/Matt/hg/mercurial/commands.py", line 3457, in grep: Function _bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: re.error) The following methods aren't implemented on re.error: __iter__ Differential Revision: https://phab.mercurial-scm.org/D10173