path: explicitly declare the `pushrev` suboptions This will help documentation and discovery. Differential Revision: https://phab.mercurial-scm.org/D12436

path: explicitly declare the `multi-urls` suboptions This will help documentation and discovery. Differential Revision: https://phab.mercurial-scm.org/D12435

path: explicitly declare the `bookmarks.mode` suboptions This will help documentation and discovery. Differential Revision: https://phab.mercurial-scm.org/D12434

crecord: avoid duplicating lines when reverting noeol->eol change When reversing a patch that looks like this while using crecord: ``` @@ -301,4 +302,4 @@ zza zzb zzc zzd -zze \ No newline at end of file +zze ``` we would previously reverse the `-zze` line to be an add, encounter the "no newline" line and stop inspecting lines. This caused us to duplicate the line, producing `zzezze` (still without a newline). `break` is the correct action if we know there will be no lines afterwards, as would be the case in an eol -> noeol transition. It is incorrect if there are lines afterward, such as if both sides are missing the newline or if only the lhs is missing the newline. Differential Revision: https://phab.mercurial-scm.org/D12441

crecord: add test demonstrating issue when reverting noeol->eol change Differential Revision: https://phab.mercurial-scm.org/D12440

revert: ask user to confirm before tracking new file when interactive If interactively reverting from a commit with `hg revert -i -r`, we would unconditionally add files from that commit that are not already tracked in the working copy. We have prompts for adding back files removed in the working copy, but that's specific to such files and does not apply to adding files from another revision. Differential Revision: https://phab.mercurial-scm.org/D12416

revert: use a `continue` to reduce indentation I'm about to add more code in the block I'm modifying here. Differential Revision: https://phab.mercurial-scm.org/D12415

stringutil: try to avoid running `splitlines()` only to get first line It's wasteful to call `splitlines()` and only get the first line from it. However, Python doesn't seem to provide a built-in way of doing just one split based on the set of bytes used by `splitlines()`. As a workaround, we do an initial split on just LF and then call `splitlines()` on the result. Thanks to Joerg for this suggestion. I didn't bother to also split on CR, so users with old Mac editors (or repos created by such editors) will not get this performance improvement. Differential Revision: https://phab.mercurial-scm.org/D12413

logcmdutil: use new function for getting first line of string Differential Revision: https://phab.mercurial-scm.org/D12412

filemerge: use new function for getting first line of string Differential Revision: https://phab.mercurial-scm.org/D12411

absorb: use new function for getting first line of string Differential Revision: https://phab.mercurial-scm.org/D12410

extensions: use new function for getting first line of string Differential Revision: https://phab.mercurial-scm.org/D12409

bookmarks: use new function for getting first line of string Differential Revision: https://phab.mercurial-scm.org/D12408

help: use new function for getting first line of string Differential Revision: https://phab.mercurial-scm.org/D12407

histedit: remove an unnecessary default value of `b''` for commit message I don't think `ctx.description()` is ever anything falsy other than `b''`. I think the comment added in 4c4232e51167 (histedit: extract common summary code into method, 2016-05-27) is mistaken (I don't see any code following that pattern before the function was added). Differential Revision: https://phab.mercurial-scm.org/D12406

histedit: use new function for getting first line of a string This fixes a crash you can run into if you enter a commit message that's just a "newline-like" byte, like a form feed byte (`hg ci -m \x0f` in Fish). That bug is the motivation for this series. Differential Revision: https://phab.mercurial-scm.org/D12405

templates: extract function to `stringutil` for getting first line of text It's surprisingly hard to get the first line from a string, so let's have our own function in `stringutil` for it. Differential Revision: https://phab.mercurial-scm.org/D12404

templates: make `firstline` filter not keep '\v', '\f' and similar In b288b4bb8448 (hide some functions behind lambdas, so demandload is useful., 2006-02-28), `x.splitlines(1)[0]` was replaced by `x.splitlines(1)[0].rstrip('\r\n')`, i.e. stripping trailing '\r' and '\n'. Combined with the "truthy" `1` passed to `splitlines()` to get it to keep line endings, that results in e.g. trailing '\v' (Line Tabulation) and '\f' (Form Feed) being preserved. I can't see why one would want that, and I doubt that was the intention; I suspect the author just didn't think to instead remove the `1` argument. Perhaps they thought the 1 being passed there - added by themselves in a7e416bf3c1d (improve templating., 2006-02-27) - was to limit the number of splits to 1 (i.e. thinking about it as `maxsplit=1` rather than `keepends=1`). Differential Revision: https://phab.mercurial-scm.org/D12403

pytype: drop py3.6 support Pytype 2022.01.07 only supports 3.7+. Differential Revision: https://phab.mercurial-scm.org/D12400

perf-util: add a `compare-discovery-case` script This script run the same discovery case using multiple variants of the algorithm and report differences in behavior, especially regarding the numbers of roundtrip. Differential Revision: https://phab.mercurial-scm.org/D12399

discovery: also audit the number of queries done In addition to the number of roundtrip, we now also track the number of queries we perform, this is useful to assert the tradeoff between number of roundtrip and the number of queries. Differential Revision: https://phab.mercurial-scm.org/D12398

search-discovery-case: display more information about the interresting case We display information about the total number of revs and the common/missing numbers. This is useful to spot the interresting case. Differential Revision: https://phab.mercurial-scm.org/D12397

subsetmaker: rework the antichain generation to be usable Before this, antichain computation can run for 10s of hours without completion in sight. We use a more direct approach in the computation to keep the computation in complexity in check. With good result. We can now have a full antichain computation on mozilla-try in about one minute. Which is usable. Differential Revision: https://phab.mercurial-scm.org/D12396

subsetmaker: use SortedSet for the scratch variant This provides a massive speedup on wide repository with many heads. For example on mozilla-try, this move from un-usable slow to fairly instant. Differential Revision: https://phab.mercurial-scm.org/D12395

subsetmaker: stabilize the computation of `scratch` subset `heads` is set, order of the element are not deterministic and we need to stabilize that if we want to get reproducible results. Differential Revision: https://phab.mercurial-scm.org/D12394

revlog: recommit 49fd21f32695 with a fix for issue6528 `filelog.size` currently special cases two forms of metadata encoding: - copy data via the parent order as flag bit - censor data by peaking into the raw delta All other forms of metadata encoding including the empty metadata block are mishandled. In `basefilectx.cmp` the empty metadata block is explicitly checked to compensate for this. Restore 49fd21f32695, but disable it for filelog, so that the original flag bit use contines to work. Document all this mess for now in preparation of a proper rework. Differential Revision: https://phab.mercurial-scm.org/D11203

merge-lists: make it possible to specify pattern to match The `merge-lists` tool doesn't know anything about Python other than its regex that attempts to match import lines. Let's make it possible to pass in a custom regex so it's easy to use the tool for e.g. C/C++ `#include` lines or Rust `use` lines (given the limited). Differential Revision: https://phab.mercurial-scm.org/D12392

contrib: add a partial-merge tool for sorted lists (such as Python imports) This is a pretty naive tool that uses a regular expression for matching lines. It is based on a Google-internal tool that worked in a similar way. For now, the regular expression is hard-coded to attempt to match single-line Python imports. The only commit I've found in the hg core repo where the tool helped was commit 9cd6292abfdf. I think that's because we often use multiple imports per import statement. I think this tool is still a decent first step (especially once the regex is made configurable in the next patch). The merging should ideally use a proper Python parser and do the merge at the AST (or CST?) level, but that's significantly harder, especially if you want to preserve comments and whitespace. It's also less generic. Differential Revision: https://phab.mercurial-scm.org/D12380

branching: merge stable into default

merge: stable into default

tests: stop excluding the pycompat module from pyflakes I assume this was skipped because of all of the py2 stuff causing a lot of spew. The "unused" imports are left in place in case any 3rd party stuff is using it. I don't care about most of it, but TortoiseHg uses `io` and `queue`, so minimally I'd like to keep those. Differential Revision: https://phab.mercurial-scm.org/D12423

tests: drop some py2 specific pyflake failures Differential Revision: https://phab.mercurial-scm.org/D12422

util: drop a duplicate import This was already imported several lines above. Differential Revision: https://phab.mercurial-scm.org/D12421

pycompat: drop the pickle import I suspect this is what df56e6bd37f6 meant to eliminate. Differential Revision: https://phab.mercurial-scm.org/D12420

util: restore the util.pickle symbol This was accidently dropped in df56e6bd37f6, which started importing pickle directly. That commit explicitly says it will retain it for compatibility with external stuff though. The unused import in pycompat isn't flagged because that module is skipped. Just importing with a comment seemed cleaner than `import X as Y` and then assigning to a `pickle` variable, just to avoid the pyflakes warning. Differential Revision: https://phab.mercurial-scm.org/D12419

merge: stable into default

completion: install completers to conventional locations Installs the bash and zsh completers to the convential locations so they will automatically be picked up without user intervention. The zsh completer on Debian is still installed to vendor-completions to match their policy. bash: https://github.com/scop/bash-completion#faq zsh: https://github.com/zsh-users/zsh/blob/57305cf245853b8b30895b41a90142dffab97e38/INSTALL#L254 Debian zsh: https://salsa.debian.org/debian/zsh/-/blob/5086b5356abcef8849dc8a09902b7c55f01db3c0/debian/README.Debian#L73

pytype: disable a few errors about Windows specific module attributes These were flagged by pytype 2022.03.21. Differential Revision: https://phab.mercurial-scm.org/D12401

rhg: sort unsupported extensions in error message This caused some flakiness in test output, and is also just better for users. Differential Revision: https://phab.mercurial-scm.org/D12389

hgignore: ignore .testtimes in more location See the inline comment. Differential Revision: https://phab.mercurial-scm.org/D12393

merge: with stable

partial-merge: add support for `.args` config (`$local` etc.) It will be useful to be able to define custom command-line arguments per partial merge tool just like we have for regular merge tools. In particular, I expect the same binary to handle multiple languages, so it will be useful to be able to pass some argument indicating the language, or perhaps simply an argument defining a regex that's used for finding lines to merge as a sorted set. Differential Revision: https://phab.mercurial-scm.org/D12383

filemerge: add support for partial conflict resolution by external tool A common class of merge conflicts is in imports/#includes/etc. It's relatively easy to write a tool that can resolve these conflicts, perhaps by naively just unioning the statements and leaving any cleanup to other tools to do later [1]. Such specialized tools cannot generally resolve all conflicts in a file, of course. Let's therefore call them "partial merge tools". Note that the internal simplemerge algorithm is such a partial merge tool - one that only resolves trivial "conflicts" where one side is unchanged or both sides change in the same way. One can also imagine having smarter language-aware partial tools that merge the AST. It may be useful for such tools to interactively let the user resolve any conflicts it can't resolve itself. However, having the option of implementing it as a partial merge tool means that the developer doesn't *need* to create a UI for it. Instead, the user can resolve any remaining conflicts with their regular merge tool (e.g. `:merge3` or `meld). We don't currently have a way to let the user define such partial merge tools. That's what this patch addresses. It lets the user configure partial merge tools to run. Each tool can be configured to run only on files matching certain patterns (e.g. "*.py"). The tool takes three inputs (local, base, other) and resolves conflicts by updating these in place. For example, let's say the inputs are these: base: ``` import sys def main(): print('Hello') ``` local: ``` import os import sys def main(): print('Hi') ``` other: ``` import re import sys def main(): print('Howdy') ``` A partial merge tool could now resolve the conflicting imports by replacing the import statements in *all* files by the following snippet, while leaving the remainder of the files unchanged. ``` import os import re import sys ``` As a result, simplemerge and any regular merge tool that runs after the partial merge tool(s) will consider the imports to be non-conflicting and will only present the conflict in `main()` to the user. Differential Revision: https://phab.mercurial-scm.org/D12356

branching: merge stable into default

rhg: add support for ignoring all extensions Some workflows just want what `rhg` does and don't care about any extensions, this makes it easier. Differential Revision: https://phab.mercurial-scm.org/D12385

branching: merge stable into default

revlog: fix index_fast_rank (wip) As far as I can tell, rank is stored as a 32-bit big endian value, I'm not sure how grabbing the first byte can possibly work. I assume there's no test coverage here? cc @pacien Fixes: https://www.mercurial-scm.org/repo/hg/rev/e633e660158f Differential Revision: https://phab.mercurial-scm.org/D12376

tests: fix glob pattern for dynamic timer alignment The number of space characters varies depending on the number of digits of the timer, making some tests fail on slow machines in an unintended way: ```diff --- /build/mercurial-6.1/tests/test-merge-halt.t +++ /build/mercurial-6.1/tests/test-merge-halt.t.err @@ -210,6 +210,6 @@ merge halted after failed merge (see hg resolve) [240] $ hg shelve --list - default (* ago) changes to: foo (glob) + default (11s ago) changes to: foo $ hg unshelve --abort unshelve of 'default' aborted ERROR: test-merge-halt.t output changed ``` Differential Revision: https://phab.mercurial-scm.org/D12381

branching: merge stable into default

tests: fix formatting issue in run-tests.py after c194e93d1ebc Differential Revision: https://phab.mercurial-scm.org/D12375

import-checker: allow symbol imports from typing module As we add typing annotations, we'll want to use a lot of symbols from the `typing` module. Typing `typing` all the time will be annoying. Let's allow symbol imports from this module. While I was here, I changed some comments from "whitelist" to "allow list" as the former is non-inclusive terminology. Differential Revision: https://phab.mercurial-scm.org/D12365

pycompat: remove json.loads polyfill for Python 3.5 We no longer support Python 3.5 so this can be deleted. Differential Revision: https://phab.mercurial-scm.org/D12364

pycompat: remove check for Python >= 3.6 We dropped support for Python 3.5 so this is always true. Differential Revision: https://phab.mercurial-scm.org/D12363

hgdemandimport: delete check for Python 3.5 We dropped support for Python 3.5. So we no longer need to do this. Differential Revision: https://phab.mercurial-scm.org/D12362

hg: always import hgdemandimport The deleted if condition is always true now that we dropped Python 2 and 3.5. Differential Revision: https://phab.mercurial-scm.org/D12361

tests: support another error case when detecting ipv4/ipv6 support I encountered this on Linux in a VM environment with a rather strange networking setup (both on the host and in the VM). Differential Revision: https://phab.mercurial-scm.org/D12371

debugdiscovery: fix a typo in the help Differential Revision: https://phab.mercurial-scm.org/D12372

rust-hg-cypython: upgrade dependencies This upgrades all dependencies to their latest version. This is routinely done to keep-up. Differential Revision: https://phab.mercurial-scm.org/D12359

rust-hg-core: upgrade dependencies This upgrades all dependencies to their latest version, except `clap` and `zstd` whose latest versions do not support our minimum supported Rust version 1.48.0. Same as for `rhg`, it contains security fix for `regex` which does not affect us too much, but doesn't hurt, and the rest of the upgrades are there simply to keep up. Differential Revision: https://phab.mercurial-scm.org/D12358

rhg: upgrade dependencies This upgrades all dependencies to their latest version, except `clap`, which is upgraded to the last 2.x series version, since 3.x does not support our minimum supported Rust version of 1.48.0. This contains a security fix for `regex` which does not affect us too much, but doesn't hurt, and the rest of the upgrades are there simply to keep up. Differential Revision: https://phab.mercurial-scm.org/D12357

chgserver: remove Python 2 file descriptor logic Follows up 0bb28b7736bc "chgserver: remove Python 2 support code." On Python 2, we had to close newfp prior to restoring the original file description since "delete newfp" would otherwise close the file descriptor shared with the long-lived fp: in attachio(): newfp = os.fdopen(fp.fileno(), mode, bufsize) in _restoreio(): newfp.close() # temporarily close newfp.fileno() (= fp.fileno()) os.dup2(fd, fp.fileno()) # reopen fp.fileno() with original fd On the other hand, we shouldn't call newfp.close() on Python 3 since any function calls are proxied to the underlying file object by procutil.LineBufferedWrapper.

dirstate-tree: optimize HashMap lookups with raw_entry_mut This switches to using `HashMap` from the hashbrown crate, in order to use its `raw_entry_mut` method. The standard library’s `HashMap` is also based on this same crate, but `raw_entry_mut` is not yet stable there: https://github.com/rust-lang/rust/issues/56167 Using version 0.9 because 0.10 is yanked and 0.11 requires Rust 1.49 This replaces in `DirstateMap::get_or_insert_node` a call to `HashMap<K, V>::entry` with `K = WithBasename<Cow<'on_disk, HgPath>>`. `entry` takes and consumes an "owned" `key: K` parameter, in case a new entry ends up inserted. This key is converted by `to_cow` from a value that borrows the `'path` lifetime. When this function is called by `Dirstate::new_v1`, `'path` is in fact the same as `'on_disk` so `to_cow` can return an owned key that contains `Cow::Borrowed`. For other callers, `to_cow` needs to create a `Cow::Owned` and thus make a costly heap memory allocation. This is wasteful if this key was already present in the map. Even when inserting a new node this is typically the case for its ancestor nodes (assuming most directories have numerous descendants). Differential Revision: https://phab.mercurial-scm.org/D12317

heptapod-ci: remove useless mentions of Python 3 Now that we don't have Python 2 jobs, we can go with shorter names. Differential Revision: https://phab.mercurial-scm.org/D12354

revlog: remove unused `rank` variable from D12209 Differential Revision: https://phab.mercurial-scm.org/D12355

demandimport: delete demandimportpy2 module We no longer support Python 2. Differential Revision: https://phab.mercurial-scm.org/D12353

py3: use class X: instead of class X(object): The inheritance from object is implied in Python 3. So this should be equivalent. This change was generated via an automated search and replace. So there may have been some accidental changes. Differential Revision: https://phab.mercurial-scm.org/D12352

check-code: remove ban on old-style classes In Python 3, `class foo:` is equivalent to `class foo(object):`. So we can allow the former form now. Differential Revision: https://phab.mercurial-scm.org/D12351

tests: remove Python 2 support code Differential Revision: https://phab.mercurial-scm.org/D12350

tests: remove Python < 3 check We require Python 3 now. Differential Revision: https://phab.mercurial-scm.org/D12349

cborutil: remove Python 2 definition of _elementtointeger() Differential Revision: https://phab.mercurial-scm.org/D12348

py3: stop using util.iterfile() The Python 3 implementation is a no-op. So this is equivalent. We still keep util.iterfile() around for backwards API compatibility to help the Python 3 migration. It can be deleted in a future release. Differential Revision: https://phab.mercurial-scm.org/D12347

util: remove iterfile() variant for buggy EINTR handling The workaround for Python 2 is no longer needed. So we can delete some code. Differential Revision: https://phab.mercurial-scm.org/D12346

statprof: remove superfluous sys.version_info check Always true since we require Python 3 now. Differential Revision: https://phab.mercurial-scm.org/D12345

policy: remove Python 2.7 compatibility code Differential Revision: https://phab.mercurial-scm.org/D12344

lsprof: remove some Python 2.7 compatibility code Differential Revision: https://phab.mercurial-scm.org/D12343

url: remove Python 2.7 support code Differential Revision: https://phab.mercurial-scm.org/D12342

py3: replace pycompat.itervalues(x) with x.values() pycompat.itervalues(x) just calls x.values(). So this is equivalent. The rewrite was perfomed via an automated search and replace. Differential Revision: https://phab.mercurial-scm.org/D12341

py3: use str instead of pycompat.unicode pycompat.unicode is an alias to str. Differential Revision: https://phab.mercurial-scm.org/D12340

ui: use input() directly pycompat.rawinput() is an alias to input() Differential Revision: https://phab.mercurial-scm.org/D12339

py3: use int instead of pycompat.long pycompat.long is aliased to int. So this should have no change in functionality. Differential Revision: https://phab.mercurial-scm.org/D12338

verify: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12336

templatekw: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12335

templatefuncs: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12334

revlog: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12333

merge: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12332

manifest: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12331

localrepo: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12330

keepalive: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12329

help: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12328

dirstate: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12327

debugcommands: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12326

config: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12325

commands: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12324

remotefilelog: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12323

lfs: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12322

largefiles: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12321