merge with stable

cmdutil: reimplement finddate() without using walkchangerevs() It's simpler and slightly faster maybe because a fewer Python ops would run. Unscientific benchmark: $ python -m timeit \ -s 'from mercurial import hg, ui, cmdutil; repo = hg.repository(ui.ui())' \ 'cmdutil.finddate(repo.ui, repo, "<2008-01-01")' (orig) 10 loops, best of 3: 1.45 sec per loop (new) 10 loops, best of 3: 1.25 sec per loop Now "hg churn" and "hg grep" are the only users of walkchangerevs(), which I want to refactor and fix bugs.

test-simple-update: add test for -d DATE option I couldn't find any tests for cmdutil.finddate() except for the abort case.

largefiles: walk history in ascending order while downloading all lfiles I don't think the order matters. Maybe it's purely because of the use of walkchangerevs(), which was originally designed for "hg log" command. Surprisingly, the number of objects fetched in test-largefiles.t has changed. According to the --verbose output, the order of the following fetches flipped and the latter got deduplicated. getting large3:eb7338044dc27f9bc59b8dd5a246b065ead7a9c4 found eb7338044dc27f9bc59b8dd5a246b065ead7a9c4 in store getting sub/large4:eb7338044dc27f9bc59b8dd5a246b065ead7a9c4 found eb7338044dc27f9bc59b8dd5a246b065ead7a9c4 in store

largefiles: replace use of walkchangerevs() with simple revset query Since it does nothing in prepare(), what we're doing is just walking revisions matching ".hglf" in reverse order.

largefiles: remove unused 'rev' parameter from downloadlfiles() It's no longer used since 83ead8cb0ff2 "largefiles: implement pull --all-largefiles as a special case of --lfrev."

resourceutil: document when we expect to take the importlib.resouces code path Differential Revision: https://phab.mercurial-scm.org/D9018

templater: fix reading of templates in frozen binaries with py3 < 3.7 When using a frozen binary with py3 < 3.7, there's no `importlib.resources` module, so we use the code path that reads the resources from the file system. That code path expects bytes for package name and resource name. Differential Revision: https://phab.mercurial-scm.org/D9008

histedit: cache description line Navigating the curses-based histedit interface can be pretty slow because it redraws everything whenever you make a change. This patch simply replaces `@property` by `@util.propertycache` on the `histeditrule.desc()` function so it's not re-calculated every time the screen needs to be re-rendered. I timed it on an example of 30 simple commits, where I moved the top commit down 25 steps and then up 25 steps after. Before this patch, that (the whole `hg histedit` invocation) took 11.6 s of CPU and after this patch it took 0.8 s). Differential Revision: https://phab.mercurial-scm.org/D9016

tests: make emacs test less strict It was failing like this for me: ``` @@ -2,7 +2,7 @@ $ emacs -q -no-site-file -batch -l $TESTDIR/../contrib/hg-test-mode.el \ > -f ert-run-tests-batch-and-exit Running 1 tests (*) (glob) - passed 1/1 hg-test-mode--compilation-mode-support + passed 1/1 hg-test-mode--compilation-mode-support (0.045732 sec) - Ran 1 tests, 1 results as expected (*) (glob) + Ran 1 tests, 1 results as expected, 0 unexpected (2020-09-11 16:49:56+0000, 0.046034 sec) ``` Differential Revision: https://phab.mercurial-scm.org/D9017

graphlog: use '%' only if there are *unresolved* conflicts In 14d0e89520a2, I made graphlog use '%' for the "other" context when there's an existing merge state. However, that has confused many people because it shows up even if all conflicts are already resolved, which makes it show up even after e.g. `hg update -m` with automatically resolved conflicts. This patch makes it so we show the '%' only if there still unresolved conflicts. This patch replaces my earlier attempt in D8930, where I decided to automatically clear the mergestate if there are no remaining conflicts. That had the problem that it wouldn't let the user re-resolve the conflicts using `hg resolve`. Note that an in-progress "proper" merge (one that will result in a commit with two parents, such as after `hg merge`) will already have two dirstate parents before the commit happens. That means that both sides of the merge will get drawn as '@' in the graph, since "is dirstate parent" takes precedence over "is involved in merge conflict". Differential Revision: https://phab.mercurial-scm.org/D9007

merge: show number of ancestors in bid merge debug notes A good number of times, we don't generate an action for a file in manifestmerge() which can result in bid merge doing the wrong thing. Mentioning the number of ancestors from which we are bidding will help spot such cases where we are not returning an action. Differential Revision: https://phab.mercurial-scm.org/D8967

debugmergestate: sort extras before printing Prevents flaky output. Differential Revision: https://phab.mercurial-scm.org/D8986

debugmergestate: show extras for files which are not in mergestate With recent changes and future changes, we will like to store extras information about files which were merged automatically. Hence, we will have merge extras for files which are not in the mergestate. Differential Revision: https://phab.mercurial-scm.org/D8942

tests: update lockdelay.py to handle the `wait` argument Spotted by a future change.

commit: pass mergestate into _filecommit() instead of re-reading it mergestate reading although cheap is not free. Let's read mergestate once on top and pass it into `_filecommit()`. In upcoming patches, we will be reading mergestate more in `_filecommit()`. Differential Revision: https://phab.mercurial-scm.org/D8984

py3: fix bytes and str mixup in run-tests Differential Revision: https://phab.mercurial-scm.org/D9009

hg-core: fix some `clippy` warnings Differential Revision: https://phab.mercurial-scm.org/D8957

rhg: print error message when argument parsing fails Differential Revision: https://phab.mercurial-scm.org/D8956

rhg: fix `clippy` warnings Differential Revision: https://phab.mercurial-scm.org/D8955

rhg: pass `ui` to `Command` `run` Allow implementation of `From<clap::ArgMatches> for Command` Differential Revision: https://phab.mercurial-scm.org/D8954

rhg: add harness tests for rhg files Differential Revision: https://phab.mercurial-scm.org/D8873

rhg: make output of `files` relative to the current directory and the root This matches the behavior of `hg files`. The util is added in `hg-core` instead of `rhg` because this operation could be useful for other external tools. (this was definitely not prompted by rust issue #50784, I swear) Differential Revision: https://phab.mercurial-scm.org/D8872

tags: take lock instead of wlock before writing hgtagsfnodes1 cache This cache is shared across stores and hence we should take store lock before writing to it. Otherwise there will be race where one share with wlock is writing to this cache and other share with wlock is trying to read it simultaneously. Differential Revision: https://phab.mercurial-scm.org/D9001

statichttprepo: use LockUnavailable() instead of Abort() for lock (API) If we try to get a lock on statichttprepo, we get `error.Abort()` instead of subclass of `error.LockError()`. The callers which catches `error.LockError` fails to catch this case as the correct error is not raised. Raising `error.LockUnavaible()` is same as what is done for wlock also.

localrepo: warn if we are writing to cache without a lock From quite sometime we have two types of cache, `cache` and `wcache`. The later one is a working copy cache and the first one is a store cache. Let's add a check for warning if we are missing store lock while writing to these caches. This is inspired from some tag cache breakage which is observed when multiple shares are in play. The interesting part is that although we are still taking wlock to write store caches at many places, but still the test pases. Differential Revision: https://phab.mercurial-scm.org/D9000

commit: fix a wrong argument name in documentation Differential Revision: https://phab.mercurial-scm.org/D8985

git: remove unrequired assignment of listignored and listunknown Caught by test-check-pyflakes.t

grep: fix hash(linestate) to not include linenum linestate.__eq__() just compares the line values whereas __hash__() does hash both self.line and self.linenum, which violates the rule. __hash__() was added at fb502719c75c, "python 2.6 compatibility: add __hash__ to classes that have __eq__" with no additional remarks, so this would probably be a simple mistake. The test output changed because difflib.SequenceMatcher() internally uses a dict. As you can see, the line "export" is unchanged at the revision 2, so the new output is correct.

test-grep: document that trivially moved lines might not be detected I'm not going to fix this. I just learned "grep --diff" does clever thing than a simple grep(patch.diff(context=0)), and I think it's better to document how things work.

revset: remove indirect indexing of status tuple Just use the attribute name.

rewriteutil: relax the sha1 hash references to handle future hash types Per discussion with nbjoerg in IRC. Differential Revision: https://phab.mercurial-scm.org/D8951

phabricator: update the hashes in commit messages as they get submitted Due to the sequence of actions (posting, amending the local commit, and then updating the Differential properties), the message in the commit is updated on the initial send but the message displayed in Phabricator is not. It should get updated there if the review is reposted. The data in the Commits tab is accurate for the first `phabsend`. Differential Revision: https://phab.mercurial-scm.org/D8950

rewriteutil: also consider pending obsoletes when updating hashes in messages Phabricator builds up the replacement commits and mapping in a single transaction, and then finalizes everything once the commits have been rewritten. That's too late when trying to update the messages for those commits. I'm a little concerned that this isn't a generic enough interface, since it doesn't mimic the list of list return of `obsutil.successorssets()`. But this is the type of mapping that phabricator maintains, and I don't think the methods that would be interested in calling this need to worry about split and divergence. We can fix that later if the need arises. Differential Revision: https://phab.mercurial-scm.org/D8949

rewriteutil: extract evolve code used to replace obsolete hashes in commits The evolve command uses it, but there are core things like `phabsend` and `rebase` that would also benefit. Differential Revision: https://phab.mercurial-scm.org/D8948

git: fix index handling of removed files during commit (issue6398) Other changes in this series also changed the behavior here in positive ways, but this was the final step that actually fixed things. Differential Revision: https://phab.mercurial-scm.org/D8999

git: make dirstate actually support listclean parameter As far as I can tell listignored and listunknown should already work. I'm vexed that there doesn't seem to be a way to get clean files out of the pygit2 status method, but this at least makes things work better. Differential Revision: https://phab.mercurial-scm.org/D8998

git: make dirstate status() respect matcher As with other changes in this stack, we appear to have been getting lucky in the past. An upcoming change behaved _very_ oddly without this fix. Differential Revision: https://phab.mercurial-scm.org/D8997

git: fix up dirstate use of index This was super-broken before, and somehow none of the existing attempts to use this code tripped on the defects here. Sigh. Differential Revision: https://phab.mercurial-scm.org/D8996

git: correctly handle "nothing changed" commits I guess localrepo.commit() actually returns an Optional[node], which is a bit of a surprise to me. Differential Revision: https://phab.mercurial-scm.org/D8995

localrepo: use functools.wraps() in unfilteredmethod decorator This makes it easier to figure out what function you're holding on to when doing printf-style debugging. Differential Revision: https://phab.mercurial-scm.org/D8994

git: actually copy treemanifest instances in .copy() (issue6398) The implementation here is so simple I honestly have no idea why I didn't do it at the time. Hopefully there's not some nuance past-me forgot to write down. Differential Revision: https://phab.mercurial-scm.org/D8993

log: fix crash and bad filematcher lookup by -fr'wdir()' PATH "-fr'wdir()' ADDED-PATH" is still wrong, which will be fixed in default branch.

hgdemandimport: bypass demandimport for _ast module (issue6407) This is broken on Python 3.9rc1, and while it sounds like there may be a fix in Python, we probably also should have this workaround in place in hg. See the bug for more details (including on bugs at redhat and b.p.o). Differential Revision: https://phab.mercurial-scm.org/D9004

git: restore basic functionality after b3040b6739ce We don't yet have a formal interface for the changelog, but it's mostly specified. Sadly, b3040b6739ce added a semi-private pseudo-enum that we need to cope with, so it's probably high time that someone (me?) attempts to define that interface to prevent future backsliding. Differential Revision: https://phab.mercurial-scm.org/D8992

git: convert tz offset to int (issue6359) Differential Revision: https://phab.mercurial-scm.org/D8991

repoview: pin revisions for `local` and `other` when a merge is active I've hit this a couple of times, where pulling with a dirty `wdir` obsoletes `p1` and updating to the successor results in merge conflicts. The problem was resolving them failed immediately, complaining that the old checkout was filtered. The change in `test-rebase-obsolete.t` is because there's an outstanding merge conflict in a rebase operation. The summary prompt to merge seems incorrect for this scenario, but that's an existing issue. Differential Revision: https://phab.mercurial-scm.org/D8980

merge with stable

extdiff: refactor cmdline and gui calculation login in a separate fn This is some good logic with handling of many config items and various ways of specifying it. I want to reuse in `diff --tool` and hence refatcored it in a separate function of it's own.

extdiff: reorder an if-else conditional This brings continue part on top and will help in next patch where we will like to introduce utility functions for rest of the code.

run-tests: extract logic to get errpath in a utility func Differential Revision: https://phab.mercurial-scm.org/D8979

run-tests: refactor filtering logic for --retest flag How I got to this: While re-running failed tests using --retest I noticed that the output: "running x tests using y parallel processes". was not actually correct, because x was the total number of tests present in the directory, but it should be the number of failed tests. Although it would run only the failed tests and later will say that remaining tests were skipped. Changes in test files reflect the fixed behaviour. This patch change and move the logic for filtering failed test for --retest option and make sure that we create instances of class Test only for the tests we need to run. As mentioned in the deleted text (in this patch itself) the logic for --retest should be outside of TestSuite. Differential Revision: https://phab.mercurial-scm.org/D8938

tests: add a --retest test to demonstrate a fix in next patch As you see above the added lines, there was only one test failed. So the output should be: "running 1 tests using 1 parallel processes" Next patch will be fixing this. Differential Revision: https://phab.mercurial-scm.org/D8964

tests: remove a sub-test since we are done with it in test-run-tests.t removing test-failure.t to make sure that in next patch when we re-run failed tests using --retest, it won't get considered unnecessarily. Differential Revision: https://phab.mercurial-scm.org/D8963

storageutil: allow modern hash sizes for fileids Differential Revision: https://phab.mercurial-scm.org/D8783

registrar: fix a documentation typo Differential Revision: https://phab.mercurial-scm.org/D8976

documentation: add `zstd` compression to the internal `revlogs` documentation Differential Revision: https://phab.mercurial-scm.org/D8953

extdiff: move single file handling inside `not per-file` conditional If `--per-file` is passed, we will diffing files one by one, hence there is no need to make this optimization around single file in that case. This optimization is only required when `--per-file` is not passed and we will be opening directory-diff for a single file. Differential Revision: https://phab.mercurial-scm.org/D8971

extdiff: remove dir2root and pass full path as dir2 in _runperfilediff() The only use of `dir2root` was to join with `dir2` to generate the path for other side of diff. Like in previous patch, `dir1a` and `dir1b` are full paths and no longer base names, hence we pass `dir2` as full path too and making `dir2root` unrequired. Differential Revision: https://phab.mercurial-scm.org/D8970

extdiff: pass full paths of `dir1a` and `dir1b` to `_runperfilediff()` Earlier we were passing basename instead of the fullpath and then joining with the tmproot. This is wrong because the user can choose `rev1a` as wdir and in those cases, tmproot should not be joined with the path. I am working on refactoring extdiff logic so that we can have options like `diff --tool` and encountered this. Although this patch in itself makes no difference, however in future when a new caller of `_runperfilediff()` will be added, it will be useful to directly pass on the full paths instead. Differential Revision: https://phab.mercurial-scm.org/D8969