util: implement sortdict.insert() As flagged by pytype (reported via Matt Harbison, thanks). This was broken by bd0fd3ff9916 (util: rewrite sortdict using Python 2.7's OrderedDict, 2017-05-16). We actually call insert() on namespaces.py:100, but we clearly don't have test coverage of that an no users have reported it AFAIK. Differential Revision: https://phab.mercurial-scm.org/D7680

patch: make __repr__() return str Caught by pytype: line 969, in __repr__: Function bytes.join was called with the wrong arguments [wrong-arg-types] Expected: (self, iterable: Iterable[bytes]) Actually passed: (self, iterable: Iterator[str]) Differential Revision: https://phab.mercurial-scm.org/D7682

pytype: suppress warnings about no 'open_binary' on importlib.resources Fixes these pytype warnings: line 43, in <module>: No attribute 'open_binary' on module 'importlib.resources' [module-attr] line 47, in open_resource: No attribute 'open_binary' on module 'importlib.resources' [module-attr] For some reason, I can't upgrade from 3.6.8 in my WSL environment. Differential Revision: https://phab.mercurial-scm.org/D7681

windows: if username(uid=None) is loaded, just use getpass This is at least consistent with what we do on other platforms in the base case. I don't know enough about Windows to fill in other cases that might exist here, but this at least should be a start. Differential Revision: https://phab.mercurial-scm.org/D7679

transplant: use check_incompatible_arguments() Differential Revision: https://phab.mercurial-scm.org/D7663

bookmarks: use check_incompatible_arguments() for inactive+action Differential Revision: https://phab.mercurial-scm.org/D7662

bookmarks: use cmdutil.check_incompatible_arguments() for action+rev Differential Revision: https://phab.mercurial-scm.org/D7648

bookmarks: use cmdutil.check_at_most_one_arg() for action Differential Revision: https://phab.mercurial-scm.org/D7647

rebase: use cmdutil.check_at_most_one_arg() for action Here we also needed to know what the action was (if any), so I've updated the helper to return any specified option. Differential Revision: https://phab.mercurial-scm.org/D7640

releasenotes: extract helper for checking for incompatible arguments This patch extracts a new check_incompatible_arguments() function similar to check_at_most_one_arg(). The difference is that the new function is for checking for arguments that are disallowed together with some other argument but not mutually exclusive among themselves. Differential Revision: https://phab.mercurial-scm.org/D7639

fix: use cmdutil.check_at_most_one_arg() Differential Revision: https://phab.mercurial-scm.org/D7638

patchbomb: use cmdutil.check_at_most_one_arg() Differential Revision: https://phab.mercurial-scm.org/D7637

export: use cmdutil.check_at_most_one_arg() Differential Revision: https://phab.mercurial-scm.org/D7636

amend: use cmdutil.check_at_most_one_arg() Differential Revision: https://phab.mercurial-scm.org/D7635

commit: use cmdutil.check_at_most_one_arg() Differential Revision: https://phab.mercurial-scm.org/D7634

clone: extract helper for checking mutually exclusive args We have some duplicated code for aborting if the user provided mutually exclusive arguments. Extensions surely have more such code. We also have duplicated translations and inconsistent output in this area. This patch introduces a simpler helper for checking if more than one option among a given set was given on the command line. I've made the clone code call the function to show that it works. The function has no good way of checking arguments with hyphens in them. I'll add that later if necessary. The function still won't be applicable in all cases, but I think it's still better than nothing. Differential Revision: https://phab.mercurial-scm.org/D7633

dirstate: when calling rebuild(), avoid some N^2 codepaths I had a user repo with 200k files in it. Calling `hg debugrebuilddirstate` took tens of minutes (I didn't wait for it). In that situation, changedfiles==allfiles, and both are lists. This meant that we had to run an average of 100k comparisons, for each of 200k files, just to check whether a file needed to have normallookup called (it always did), or drop. While it's probably not a huge issue, in my very awkward synthetic benchmark I wrote (not using a benchmark library or anything), I was seeing some slowdowns for small-changedfiles and very-large-allfiles invocations, with an inflection somewhere around 10 items in changedfiles (regardless of the size of allfiles); above 10 items in changedfiles, the new code appears to always be faster. For the case of 50k files in changedfiles and the same items in allfiles, I'm seeing differences of 15s of just running comparisons vs. 0.003793s. I haven't bothered to run a comparison of 200k items in changedfiles and allfiles. :) Differential Revision: https://phab.mercurial-scm.org/D7665

rust-warnings: fix warnings in tests It turns out that I also missed those warnings inside tests. This should be the last of them. One day we will get rid of this interface anyway. Differential Revision: https://phab.mercurial-scm.org/D7678

relnotes: mention the merging of index and nodemap

crecord: remove toggleamend Previous commit removed its only calling site.

crecord: repurpose "a" key to toggle all selections (BC) I really don't like "a". I keep accidentally hitting it when I actually want "A", and then I'm suddenly in a state I don't want to be in. There's a big wall of text telling me that I've turned amend mode on or off (which one was I orginally in?), and this seems very useless. If I wanted to amend or not, I would have chosen that from the command-line, not change my mind after I've already started picking hunks apart. Furthermore, for most uses of the hunk selector (revert, uncommit, shelve/unshelve), this amend toggle doesn't make sense. It seems much better to repurpose this key to be a "weaker" version of "A". It toggles all selections. This is pretty harmless if hit accidentally, (can just hit "a" again to toggle everything and undo it), and has immediate visual feedback that something happened: all the x's and blank spaces get switched around. And unlike with amend, the current flipped state is also immediately visible without having to read a wall of text. I'm calling this a BC, however, because somewhere, someone out there has probably really fallen in love with the old use of "a" and will get angry that we took it away.

hgweb: fix error in docstring Despite the subtle semantic difference, this sentence really meant to say "overridden", not "overwritten".

crecord: rewrite help string to avoid mentioning "crecord" Despite its heritage, "crecord" is now mostly Mercurial-internal jargon. I find it better to call it "the curses hunk selector". Also slightly rewrote the part about which commands can use it. While I do believe that commit, shelve, and revert are the only commands in core that can use it, Evolve also adds at least amend and uncommit to the list.

update: add some tests for the status quo of morestatus on update conflicts Differential Revision: https://phab.mercurial-scm.org/D7666

typing: suppress a warning that mercurial.windows.checkosfilename is missing This function is used to check filename portability everwhere, so it isn't tucked into the windows.py module. I supposed the alternative is to move it and then alias it in `util`. I'm guessing it was done like this to save an import. Differential Revision: https://phab.mercurial-scm.org/D7677

typing: add an assertion to util.versiontuple Fixes the following warning: line 1177, in versiontuple: No attribute 'split' on None [attribute-error] In Optional[bytes] Differential Revision: https://phab.mercurial-scm.org/D7676

typing: add a couple of assertions to lrucachedict to help pytype Fixes the following warnings: line 1488, in pop: No attribute 'markempty' on None [attribute-error] In Optional[Union[Any, _lrucachenode, nothing]] line 1578, in popoldest: No attribute 'markempty' on None [attribute-error] In Optional[Union[Any, _lrucachenode, nothing]] Differential Revision: https://phab.mercurial-scm.org/D7674

util: rename a variable to avoid confusing pytype Fixes the following warning: line 1205, in f: No attribute 'append' on Dict[nothing, nothing] [attribute-error] In Union[Dict[nothing, nothing], list] Differential Revision: https://phab.mercurial-scm.org/D7673

pytype: suppress the import-error in util.py when importing re2 Differential Revision: https://phab.mercurial-scm.org/D7672

debugcommands: finish moving `extendeddateformats` from util to dateutil Caught by pytype. Differential Revision: https://phab.mercurial-scm.org/D7671

ui: convert exception data to bytes when printing chained exception info Caught by pytype. Differential Revision: https://phab.mercurial-scm.org/D7670

ui: use a context manager to handle file streams in edit() Differential Revision: https://phab.mercurial-scm.org/D7669

graft: reuse cmdutl.resolvecommitoptions() Differential Revision: https://phab.mercurial-scm.org/D7632

rebase: make rebase not crash if p1 == p2 != nullid While this shouldn't happen normally, some historical bugs can have caused this kind of commit to exist. Instead of crashing and having it be unobvious how to recover, let's try to continue on. Without this, we get an error like "ValueError: min() arg is an empty sequence" Differential Revision: https://phab.mercurial-scm.org/D7664

merge: add commands.merge.require-rev to require an argument to hg merge This is related to commands.rebase.requiredest, commands.update.requiredest, and commands.push.require-revs. Since it isn't really a "destination", I went with require-rev to be similar to push's require-revs. Differential Revision: https://phab.mercurial-scm.org/D7620

tests: make test-config-env.py a little less hacky Overriding os.path.isdir to always returns False is very confusing. Let's make it more explicit by specifically not returning any defaultrc/ configs. Differential Revision: https://phab.mercurial-scm.org/D7623

help: get helptext/ data from `resources` module if available For PyOxidizer, we need to read configs using the `resources` module. This patch makes it so we use that module if available (i.e. Python >= 3.7). It does that by adding a new `open_resource()` function to our `resourceutil` module. Tested by running `$PYTHON ./hg help pager` for each $PYTHON in {python2, python3.6, python3.7}. Differential Revision: https://phab.mercurial-scm.org/D7622

pycompat: allow pycompat.sysbytes() even if input already is bytes pycompat.sysstr() on py3 accepts an input that's already str (i.e. unicode). This patch makes it so pycompat.sysbytes() on py3 accepts an input that's already bytes. Allowing that makes it possible to do pycompat.sysbytes(fp.name) where fp.name is either bytes or unicode, as we'll get when fp can come from either open() or resources.open_binary(). Differential Revision: https://phab.mercurial-scm.org/D7621

merge with stable

rust-dirs: handle forgotten `Result`s In 1fe2e574616e I introduced a temporary bugfix to align Rust code with a new behavior from C/Python and forgot about a few `Result`s (cargo's compiler cache does not re-emit warnings on cached modules). This fixes it. For the record, I am still unsure that this behavior change is a good idea. Note: I was already quite unhappy with the setters and getters for the `DirstateMap` and, indirectly, `Dirs`, and this only further reinforces my feelings. I hope we can one day fix that situation at the type level; Georges Racinet and I were just talking about devising a POC for using the builder pattern in the context of FFI with Python, we'll see what comes out of it. Differential Revision: https://phab.mercurial-scm.org/D7609

merge with stable

status: outputting structured unfinished-operation information This adds a new item in the json/template output for morestatus and added item types to all entries. Differential Revision: https://phab.mercurial-scm.org/D7595

chg: fix chg to work with py3.7+ "coercing" the locale When the environment is empty (specifically: it doesn't contain LC_ALL, LC_CTYPE, or LANG), Python will "coerce" the locale environment variables to be a UTF-8 capable one. It sets LC_CTYPE in the environment, and this breaks chg, since chg operates by: - start hg, using whatever environment the user has when chg starts - hg stores a hash of this "original" environment, but python has already set LC_CTYPE even though the user doesn't have it in their environment - chg calls setenv over the commandserver. This clears the environment inside of hg and sets it to be exactly what the environment in chg is (without LC_CTYPE). - chg calls validate to ensure that the environment hg is using (after the setenv call) is the one that the chg process has - if not, it is assumed the user changed their environment and we should use a different server. This will *never* be true in this situation because LC_CTYPE was removed. Differential Revision: https://phab.mercurial-scm.org/D7550

fuzz: add support for fuzzing under either Python 2 or 3 This was more of a hairball than I hoped, but it appears to work. The hg-py3 branch of my oss-fuzz fork on github has the remaining changes to switch us to Python 3, but we may as well retain Python 2 fuzzing support for at least a little while. Differential Revision: https://phab.mercurial-scm.org/D7592

phabricator: color the status in the "phabstatus" view I couldn't figure out strikethrough for "abandoned" like I've see with word diff. Differential Revision: https://phab.mercurial-scm.org/D7608

phabricator: add the "Changes Planned" status name I noticed this when coloring the status output. But it looks like status names are also used in the pseudo revset language as well, and abort when parsing a name not in this list. Differential Revision: https://phab.mercurial-scm.org/D7607

fuzz: fix mpatch_corpus to not have an overridden __repr__ on py3 Differential Revision: https://phab.mercurial-scm.org/D7606

formatting: fix some recent formatting regressions Differential Revision: https://phab.mercurial-scm.org/D7605

cext-revlog: fixed __delitem__ for uninitialized nodetree This is a bug in a code path that's seldom used, because in practice (at least in the whole test suite), calls to `del index[i:j]` currently just don't happen before the nodetree has been initialized. However, in our current work to replace the nodetree by a Rust implementation, this is of course systematic. In `index_slice_del()`, if the slice start is smaller than `self->length`, the whole of `self->added` has to be cleared. Before this change, the clearing was done only by the call to `index_invalidate_added(self, 0)`, that happens only for initialized nodetrees. Hence the removal was effective only from `start` to `self->length`. The consequence is index corruption, with bogus results in subsequent calls, and in particular errors such as `ValueError("parent out of range")`, due to the fact that parents of entries in `self->added` are now just invalid. This is detected by the rebase tests, under conditions that the nodetree of revlog.c is never initialized. The provided specific test is more direct. Differential Revision: https://phab.mercurial-scm.org/D7603

filemerge: fix a missing attribute usage Flagged by both pytype and VSCode. Differential Revision: https://phab.mercurial-scm.org/D7465

filemerge: drop a default argument to appease pytype The function slices and takes the length of this argument without internally setting it if not provided. There was no bug here because both callers passed the argument. Differential Revision: https://phab.mercurial-scm.org/D7464

fuzz: add a seed corpus for the dirs fuzzer I was hoping to trigger an asan violation under Python 3 that some internal tests at Google found, but for some reason that's beyond me I can't seem to manage. Differential Revision: https://phab.mercurial-scm.org/D7600

fuzz: clean up production of seed corpora This was getting out of hand. Differential Revision: https://phab.mercurial-scm.org/D7599

status: add template/json data about whether a file has unresolved conflicts Differential Revision: https://phab.mercurial-scm.org/D7594

status: split morestatus data loading from display This is a small refactoring in preparation for adding more morestatus functionality (notably for templated/JSON output) - the goal is to use the data inside the status display loop, as well as output the overall state in a templatable/structured way. Differential Revision: https://phab.mercurial-scm.org/D7593

phabricator: add a "phabstatus" template keyword We add a "phabstatus" template keyword, returning an object with "url" and "status" keys. This is quite similar to "phabreview" template keyword, but it queries phabricator for each specified revision so it's going to be slow (as compared to the "phabstatus" show view from previous changeset). Differential Revision: https://phab.mercurial-scm.org/D7507

phabricator: add a "phabstatus" show view We add a "phabstatus" show view (called as "hg show phabstatus") which renders a dag with underway revisions associated with a differential revision and displays their status. The revisions shown is a subset of that shown by "work" view, only including revisions with known by Phabricator. Differential Revision: https://phab.mercurial-scm.org/D7506

dirs: fix out-of-bounds access in Py3 The hack for mutating Python's variable-length integers that was ported to py3 in cb3048746dae (dirs: port PyInt code to work on Python 3, 2016-10-08) was reading from ob_digit[1] instead of ob_digit[0] for some reason. Space for ob_digit[1] would only be allocated for integers larger than 30 bits, so we ended up writing to unallocated memory. Also, we would write an integer that's 2^30 times too large, so we would never free these integers. Found by AddressSanitizer. Differential Revision: https://phab.mercurial-scm.org/D7597

manifestcache: add some test involving shares Some issue around shares have been reported. I am adding tests to better cover the share case. The test show that the code behave as expected so far. Differential Revision: https://phab.mercurial-scm.org/D7602

fuzz: fix test-fuzz-targets.t to run with python3 Differential Revision: https://phab.mercurial-scm.org/D7601

py3: fix a bytes vs str issue in remotefilelog extension Differential Revision: https://phab.mercurial-scm.org/D7598

rust-matchers: add doctests for `AlwaysMatcher` Differential Revision: https://phab.mercurial-scm.org/D7527

rust-hg-path: add method to get part of a path relative to a prefix This will be used in the next patch in this series. Differential Revision: https://phab.mercurial-scm.org/D7526

rust-matchers: improve `Matcher` trait ergonomics `VisitChildrenSet` has no need to own the set, this will save allocations. The `file_set` return type change is motivated by both ergonomics and... being able to compile code. The `AlwaysMatcher` does not store a `file_set`, which requires it to return an owned `HashSet`, which in turn would change our return type to `Cow<&HgPath>` (lifetimes omitted). This is both un-ergonomic and troublesome for more complex lifetime issues (especially with the upcoming `FileMatcher` in the following patch). Differential Revision: https://phab.mercurial-scm.org/D7525