filemerge: add support for partial conflict resolution by external tool A common class of merge conflicts is in imports/#includes/etc. It's relatively easy to write a tool that can resolve these conflicts, perhaps by naively just unioning the statements and leaving any cleanup to other tools to do later [1]. Such specialized tools cannot generally resolve all conflicts in a file, of course. Let's therefore call them "partial merge tools". Note that the internal simplemerge algorithm is such a partial merge tool - one that only resolves trivial "conflicts" where one side is unchanged or both sides change in the same way. One can also imagine having smarter language-aware partial tools that merge the AST. It may be useful for such tools to interactively let the user resolve any conflicts it can't resolve itself. However, having the option of implementing it as a partial merge tool means that the developer doesn't *need* to create a UI for it. Instead, the user can resolve any remaining conflicts with their regular merge tool (e.g. `:merge3` or `meld). We don't currently have a way to let the user define such partial merge tools. That's what this patch addresses. It lets the user configure partial merge tools to run. Each tool can be configured to run only on files matching certain patterns (e.g. "*.py"). The tool takes three inputs (local, base, other) and resolves conflicts by updating these in place. For example, let's say the inputs are these: base: ``` import sys def main(): print('Hello') ``` local: ``` import os import sys def main(): print('Hi') ``` other: ``` import re import sys def main(): print('Howdy') ``` A partial merge tool could now resolve the conflicting imports by replacing the import statements in *all* files by the following snippet, while leaving the remainder of the files unchanged. ``` import os import re import sys ``` As a result, simplemerge and any regular merge tool that runs after the partial merge tool(s) will consider the imports to be non-conflicting and will only present the conflict in `main()` to the user. Differential Revision: https://phab.mercurial-scm.org/D12356

pullbundle: fix file name in the help text It is pullbundles.manifest and not pullbundle.manifest. Differential Revision: https://phab.mercurial-scm.org/D12391

unamend: abort if commit was not created by `hg [un]amend` `hg unamend` can currently undo any kind of rewrite, as long as it has an obsmarker. However, that has quite unexpected results if you run it after e.g. `hg rebase` (expecting it to behave like a generic `hg undo` command), because it updates to the predecessor and leaves the old changes in the working copy. I think it's better to allow `hg unamend` only after `hg amend` (and after `hg unamend` because that's documented as being supported). Differential Revision: https://phab.mercurial-scm.org/D12390

branching: merge stable into default

test: use `wait-on-file` in `test-racy-mutations.t` The official utility scale its timeout with the run-tests.py one. So lets use it. Differential Revision: https://phab.mercurial-scm.org/D12382

amend: fix amend with copies in extras If copy information is stored only in the commit extras and not in filelogs, then they get lost on amend if the file wasn't also modified in the working copy. That's because we create `filectx` object from the old commit in those cases, and the `.copysource()` of such objects read only from the filelog. This patch fixes it by always creating a new `memfilectx` in these cases, passing the calculated copy information to it. Differential Revision: https://phab.mercurial-scm.org/D12387

tests: demonstrate that copy info in changeset gets lost on amend When copy information is stored in changesets, it gets lost on amend. We didn't notice that until now because our users at Google have the config set to `compatibility`, which means copy information is stored in both changeset and filelogs. Differential Revision: https://phab.mercurial-scm.org/D12386

ci: use the `v1.0` flavor of the docker images in the CI This new versioning will help us to maintain backward compatibility in the docker image. This will be useful to deal with mismatch between default/stable in version and the re-run CI on older changesets in the future. Once this changeset land on stable, we will have to merge it in default. Then we can start make backward incompatible changes in a new image version. Differential Revision: https://phab.mercurial-scm.org/D12388

rust-status: cap the number of concurrent threads to 16 During benchmarking it was determined that the use of more threads is very advantageous... until we use more than 16. This is most likely due to some resource contention (thrashing, etc.). Until we have time to figure out and fix the underlying cause, let's just cap at 16 threads. Differential Revision: https://phab.mercurial-scm.org/D12384

rhg: add support for ignoring all extensions Some workflows just want what `rhg` does and don't care about any extensions, this makes it easier. Differential Revision: https://phab.mercurial-scm.org/D12385

branching: merge stable into default

revlog: fix index_fast_rank (wip) As far as I can tell, rank is stored as a 32-bit big endian value, I'm not sure how grabbing the first byte can possibly work. I assume there's no test coverage here? cc @pacien Fixes: https://www.mercurial-scm.org/repo/hg/rev/e633e660158f Differential Revision: https://phab.mercurial-scm.org/D12376

tests: fix glob pattern for dynamic timer alignment The number of space characters varies depending on the number of digits of the timer, making some tests fail on slow machines in an unintended way: ```diff --- /build/mercurial-6.1/tests/test-merge-halt.t +++ /build/mercurial-6.1/tests/test-merge-halt.t.err @@ -210,6 +210,6 @@ merge halted after failed merge (see hg resolve) [240] $ hg shelve --list - default (* ago) changes to: foo (glob) + default (11s ago) changes to: foo $ hg unshelve --abort unshelve of 'default' aborted ERROR: test-merge-halt.t output changed ``` Differential Revision: https://phab.mercurial-scm.org/D12381

test: update test-clone-stream.t to pass on bigendian Fixes: a3cf460a6b1b ("stream-clone: also filter the requirement we put in the bundle 2") Differential Revision: https://phab.mercurial-scm.org/D12377

filemerge: when merge tool uses $output, don't leave markers in $local As explained in the previous patch, we incorrectly leave conflict markers in both `$local` and `$output` since D12190. I don't understand why it broke but the fix is simple and clear after all the recent refactoring. Differential Revision: https://phab.mercurial-scm.org/D12379

tests: demonstrate how conflict markers end up $local *and* $output When a merge tool is configured to keep conflict markers, they are supposed to be written to `$local` if `$output` is not mentioned in the tool's `merge-tools.<tool>.args` config, and in `$output` if it is mentioned. However, I broke the latter case in D12190. Differential Revision: https://phab.mercurial-scm.org/D12378

branching: merge stable into default

revlog: fix wrong type of rank_unknown variable We treat "rank" as an int everywhere, but declare rank_unknown as a char. On architectures where char is signed, that works out ok, but when char is unsigned, rank_unknown is 255 instead of -1. Differential Revision: https://phab.mercurial-scm.org/D12374

tests: fix formatting issue in run-tests.py after c194e93d1ebc Differential Revision: https://phab.mercurial-scm.org/D12375

rust-hg-core: use correct type for libc hostname buffer The type of libc::c_char is u8 on aarch64 rather than i8, which causes the use of a specifically-typed constant to fail. Differential Revision: https://phab.mercurial-scm.org/D12373

import-checker: allow symbol imports from typing module As we add typing annotations, we'll want to use a lot of symbols from the `typing` module. Typing `typing` all the time will be annoying. Let's allow symbol imports from this module. While I was here, I changed some comments from "whitelist" to "allow list" as the former is non-inclusive terminology. Differential Revision: https://phab.mercurial-scm.org/D12365

pycompat: remove json.loads polyfill for Python 3.5 We no longer support Python 3.5 so this can be deleted. Differential Revision: https://phab.mercurial-scm.org/D12364

pycompat: remove check for Python >= 3.6 We dropped support for Python 3.5 so this is always true. Differential Revision: https://phab.mercurial-scm.org/D12363

hgdemandimport: delete check for Python 3.5 We dropped support for Python 3.5. So we no longer need to do this. Differential Revision: https://phab.mercurial-scm.org/D12362

hg: always import hgdemandimport The deleted if condition is always true now that we dropped Python 2 and 3.5. Differential Revision: https://phab.mercurial-scm.org/D12361

tests: support another error case when detecting ipv4/ipv6 support I encountered this on Linux in a VM environment with a rather strange networking setup (both on the host and in the VM). Differential Revision: https://phab.mercurial-scm.org/D12371

debugdiscovery: fix a typo in the help Differential Revision: https://phab.mercurial-scm.org/D12372

rust-hg-cypython: upgrade dependencies This upgrades all dependencies to their latest version. This is routinely done to keep-up. Differential Revision: https://phab.mercurial-scm.org/D12359

rust-hg-core: upgrade dependencies This upgrades all dependencies to their latest version, except `clap` and `zstd` whose latest versions do not support our minimum supported Rust version 1.48.0. Same as for `rhg`, it contains security fix for `regex` which does not affect us too much, but doesn't hurt, and the rest of the upgrades are there simply to keep up. Differential Revision: https://phab.mercurial-scm.org/D12358

rhg: upgrade dependencies This upgrades all dependencies to their latest version, except `clap`, which is upgraded to the last 2.x series version, since 3.x does not support our minimum supported Rust version of 1.48.0. This contains a security fix for `regex` which does not affect us too much, but doesn't hurt, and the rest of the upgrades are there simply to keep up. Differential Revision: https://phab.mercurial-scm.org/D12357

chgserver: remove Python 2 file descriptor logic Follows up 0bb28b7736bc "chgserver: remove Python 2 support code." On Python 2, we had to close newfp prior to restoring the original file description since "delete newfp" would otherwise close the file descriptor shared with the long-lived fp: in attachio(): newfp = os.fdopen(fp.fileno(), mode, bufsize) in _restoreio(): newfp.close() # temporarily close newfp.fileno() (= fp.fileno()) os.dup2(fd, fp.fileno()) # reopen fp.fileno() with original fd On the other hand, we shouldn't call newfp.close() on Python 3 since any function calls are proxied to the underlying file object by procutil.LineBufferedWrapper.

dirstate-tree: optimize HashMap lookups with raw_entry_mut This switches to using `HashMap` from the hashbrown crate, in order to use its `raw_entry_mut` method. The standard library’s `HashMap` is also based on this same crate, but `raw_entry_mut` is not yet stable there: https://github.com/rust-lang/rust/issues/56167 Using version 0.9 because 0.10 is yanked and 0.11 requires Rust 1.49 This replaces in `DirstateMap::get_or_insert_node` a call to `HashMap<K, V>::entry` with `K = WithBasename<Cow<'on_disk, HgPath>>`. `entry` takes and consumes an "owned" `key: K` parameter, in case a new entry ends up inserted. This key is converted by `to_cow` from a value that borrows the `'path` lifetime. When this function is called by `Dirstate::new_v1`, `'path` is in fact the same as `'on_disk` so `to_cow` can return an owned key that contains `Cow::Borrowed`. For other callers, `to_cow` needs to create a `Cow::Owned` and thus make a costly heap memory allocation. This is wasteful if this key was already present in the map. Even when inserting a new node this is typically the case for its ancestor nodes (assuming most directories have numerous descendants). Differential Revision: https://phab.mercurial-scm.org/D12317

heptapod-ci: remove useless mentions of Python 3 Now that we don't have Python 2 jobs, we can go with shorter names. Differential Revision: https://phab.mercurial-scm.org/D12354

revlog: remove unused `rank` variable from D12209 Differential Revision: https://phab.mercurial-scm.org/D12355

demandimport: delete demandimportpy2 module We no longer support Python 2. Differential Revision: https://phab.mercurial-scm.org/D12353

py3: use class X: instead of class X(object): The inheritance from object is implied in Python 3. So this should be equivalent. This change was generated via an automated search and replace. So there may have been some accidental changes. Differential Revision: https://phab.mercurial-scm.org/D12352

check-code: remove ban on old-style classes In Python 3, `class foo:` is equivalent to `class foo(object):`. So we can allow the former form now. Differential Revision: https://phab.mercurial-scm.org/D12351

tests: remove Python 2 support code Differential Revision: https://phab.mercurial-scm.org/D12350

tests: remove Python < 3 check We require Python 3 now. Differential Revision: https://phab.mercurial-scm.org/D12349

cborutil: remove Python 2 definition of _elementtointeger() Differential Revision: https://phab.mercurial-scm.org/D12348

py3: stop using util.iterfile() The Python 3 implementation is a no-op. So this is equivalent. We still keep util.iterfile() around for backwards API compatibility to help the Python 3 migration. It can be deleted in a future release. Differential Revision: https://phab.mercurial-scm.org/D12347

util: remove iterfile() variant for buggy EINTR handling The workaround for Python 2 is no longer needed. So we can delete some code. Differential Revision: https://phab.mercurial-scm.org/D12346

statprof: remove superfluous sys.version_info check Always true since we require Python 3 now. Differential Revision: https://phab.mercurial-scm.org/D12345

policy: remove Python 2.7 compatibility code Differential Revision: https://phab.mercurial-scm.org/D12344

lsprof: remove some Python 2.7 compatibility code Differential Revision: https://phab.mercurial-scm.org/D12343

url: remove Python 2.7 support code Differential Revision: https://phab.mercurial-scm.org/D12342

py3: replace pycompat.itervalues(x) with x.values() pycompat.itervalues(x) just calls x.values(). So this is equivalent. The rewrite was perfomed via an automated search and replace. Differential Revision: https://phab.mercurial-scm.org/D12341

py3: use str instead of pycompat.unicode pycompat.unicode is an alias to str. Differential Revision: https://phab.mercurial-scm.org/D12340

ui: use input() directly pycompat.rawinput() is an alias to input() Differential Revision: https://phab.mercurial-scm.org/D12339

py3: use int instead of pycompat.long pycompat.long is aliased to int. So this should have no change in functionality. Differential Revision: https://phab.mercurial-scm.org/D12338

verify: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12336

templatekw: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12335

templatefuncs: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12334

revlog: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12333

merge: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12332

manifest: remove pycompat.iteritems() Differential Revision: https://phab.mercurial-scm.org/D12331