tests: make HGCATAPULTSERVERPIPE imply HGTESTCATAPULTSERVERPIPE I had attempted to do this before, but missed this case. This makes it so that one can do the following to get catapult traces that include both the .t test name (and non-hg commands run by that .t test) *and* the hg-internal tracing, in one trace: HGCATAPULTSERVERPIPE=/tmp/catapult.pipe run-tests.py <args> Without this change, we need to specify both `HG{,TEST}CATAPULTSERVERPIPE`; if we specify just the TEST one, we only get the .t tests (no hg-internals), which is working as intended. If we specify the non-TEST one, we only get the hg-internals (not the rest of the .t test), which was not intended. If you want to restore the previous behavior (just hg internals, not the stuff from the .t tests), run like: HGTESTCATAPULTSERVERPIPE=/dev/null \ HGCATAPULTSERVERPIPE=/tmp/catapult.pipe \ run-tests.py <args> Differential Revision: https://phab.mercurial-scm.org/D5569

tests: add test for warning on histedit with tagged commits This is a follow-up patch to 7b7e081f8954 (D5494). Differential Revision: https://phab.mercurial-scm.org/D5552

extdiff: move external tool command line building into separate function

test: explicit a rebase source in test-rebase-collapse.t At the time of the rebase, the current revision is `null` so this rebase should fail. However, a current bug in formatspec makes it pass, we fix rebase call of the "affected" test to express its actual intent before we fix the actual bug in a later changeset.

tags: join string that's unnecessarily split across lines Differential Revision: https://phab.mercurial-scm.org/D5574

dispatch: add newline after ui.log "ui blocked ms" message Differential Revision: https://phab.mercurial-scm.org/D5573

remotefilelog: fix {file_copies} template keyword remotefilelog overrides templatekw.getrenamedfn(). I changed the return type of that function in ec37df9042f9 (templatekw: make getrenamed() return only filename, not nodeid, 2018-03-26). So we need to change remotefilelog too. The function is used by the {file_copies} template keyword and by `hg log --copies`. Differential Revision: https://phab.mercurial-scm.org/D5576

tests: demonstrate broken {file_copies} template with remotefilelog Differential Revision: https://phab.mercurial-scm.org/D5575

contrib: update window title when possible in perf-revlog-write-plot.py This is useful when comparing multiple graphs.

histedit: add newline after ui.log "# acttions to histedit" message Differential Revision: https://phab.mercurial-scm.org/D5572

rust-cpython: bindings for MissingAncestors The exposition is rather straightforward, except for the remove_ancestors_from() method, which forces us to an inefficient conversion between Python sets and Rust HashSets. Two alternatives are proposed in comments: - changing the inner API to "emit" the revision numbers to discard this would be a substantial change, and it would be better only in the cases where there are more to retain than to discard - mutating the Python set directly: this would force us to define an abstract `RevisionSet` trait, and implement it both for plain `HashSet` and for a struct enclosing a Python set with the GIL marker `Python<'p>`, also a non trivial effort. The main (and seemingly only) caller of this method being `mercurial.setdiscovery`, which is currently undergoing serious refactoring, it's not clear whether these improvements would be worth the effort right now, so we're leaving it as-is. Also, in `get_bases()` (will also be used by `setdiscovery`), we'd prefer to build a Python set directly, but we resort to returning a tuple, waiting to hear back from our PR onto rust-cpython about that Differential Revision: https://phab.mercurial-scm.org/D5550

rust-cpython: generalised conversion function Because `hg::ancestors::MissingAncestors` has methods taking some `HashSet<Revision>` besides `impl IntoIterator<Item = Revision>` as parameters we'll need the more generic `rev_pyiter_collect()` function to also build these Differential Revision: https://phab.mercurial-scm.org/D5549

rust-cpython: style consistency leftovers In particular, during review of `LazyAncestors` bindings, most `match` statements for error conversion have been replaced by higher level methods of `Result` and most personal insecurity comments have been removed. This makes it more systematic. Differential Revision: https://phab.mercurial-scm.org/D5548

rust-cpython: consistency in use of hg-core constructs While not everybody likes the `CoreLazy` and `CoreIterator` aliases, it's better not to mix them with direct references. Note: it's quite possible in the future that these would stop being exposed at the top of the `hg` crate Differential Revision: https://phab.mercurial-scm.org/D5547

rust-cpython: rustdoc improvements By default, `cargo doc` builds the documentation for public constructs only, so we make public those that can. Since `cindex` is not safe, we keep it private. Unfortunately, the macro syntax of rust-cpython doesn't allow us to document the classes directly, so we resort to do that at the module level. Differential Revision: https://phab.mercurial-scm.org/D5546

fuzz: stop building Python in the Makefile $OUT/sanpy is now built by the build.sh in oss-fuzz, which simplifies the setup here considerably. Differential Revision: https://phab.mercurial-scm.org/D5553

py3: add a test to whitelist caught by buildbot Differential Revision: https://phab.mercurial-scm.org/D5555

tests: de-flake test-narrow-share.t by making dirstate predictable test-narrow-share.t was sometimes (~0.5% on my machine) failing like this: @@ -61,7 +61,7 @@ A d3/g $ hg -R main debugdirstate --no-dates n 644 2 set d1/f - n 644 2 unset d3/f + n 644 2 set d3/f a 0 -1 unset d3/g n 644 2 set d5/f n 644 2 set d7/f The timestamp for d3/f would get set if it was determined at some point that it was clean. That check is usually done when the user runs `hg st`. We don't do that before the failure in the test case, but it happens at the end of the `hg clone` call. So if the file system's time happens to roll over after the clone's working copy has been written, but before its (final) dirstate has been written, we can end up with a set timestamp there. This patch makes it consistent by sleeping for 2 seconds so the timestamp gets reliably set. Differential Revision: https://phab.mercurial-scm.org/D5568

convert: don't drop commits that are empty in the source when using --filemap I ran into this when using `hg lfconvert --to-normal` (which uses the filemap class internally), and saw that commits with nothing but a branch change were dropped. We could put in an option that only lfconvert uses internally. But silently dropping anything other than a commit where all changes were excluded seems unintended. For example, there's a message in mercurial_sink.putcommit() if it drops an empty commit. (And the reason that isn't kicking in here is because lfconvert isn't passing --filemap, so the self.filemapmode conditional there is always False.) The naive change of `return not files` broke test-convert-filemap.t, so this is a little more elaborate than needed for converting from largefiles.

convert: add missing numcommits() override to hg sources Otherwise, the progressbar in converter.walktree() gets None as its total during the initial scan. (Though that seems harmless, and there are other foreign vcs sources without this.)

testrunner: make `-j100 --runs-per-test=100 test-foo.t` use 100 jobs It would use only 1 job since 1039404c5e1d (run-tests: print number of tests and parallel process count, 2018-10-13), but I'm pretty sure that was unintentional. Differential Revision: https://phab.mercurial-scm.org/D5557

narrow: use merge.ACTION_GET instead of duplicating 'g' constant As suggested by Yuya. Differential Revision: https://phab.mercurial-scm.org/D5512

narrow: drop explicit dirstate write The dirstate is written when the wlock is released, so we don't need to write it explicitly in updateworkingcopy(). I don't know why I put it there in the first place (tests pass without it even in the commit that introduced it). Differential Revision: https://phab.mercurial-scm.org/D5515

histedit: fix --continue and --abort when curses is enabled I overlooked this when I did the initial import. Differential Revision: https://phab.mercurial-scm.org/D5540

graft: abort if --date/user specified with --currentdate/currentuser (BC) They are conflicting options. Let's explicitly ban such combination. Note that this means you need "--no-currentdate --date" to override --currentdate specified in aliases/defaults.

amend: turn currentdate into tri-state bool so config option can be negated Otherwise, rewrite.update-timestap couldn't be overridden by --no-currentdate. FWIW, which do we like? --no-currentdate vs --no-current-date. If the latter is preferred, maybe we can add "graft --current-date" and deprecate "--currentdate".

discovery: improve partial discovery documentation The older documentation could be improved.

discovery: add a simple `addinfo` method The method can directly process a sample result. This makes the main code simpler to follow.

discovery: move missing tracking inside the partialdiscovery object This is the final set that we need to track to have a fully up to date information within the object.

discovery: add a `iscomplete` method to the `partialdiscovery` object The method is used by higher level logic to know if the initial discovery question has been answered.

discovery: update undecided from common directly within partialdiscovery The object knows when there are changes in the "common" set, so it is a good place to update the undecided set.

discovery: move undecided set on the partialdiscovery To initialize it, we need to know the discovery target. This commit only move the set on the `partialdiscovery` object, later changeset will take advantage of it.

histedit: use context manager for locks Differential Revision: https://phab.mercurial-scm.org/D5542

histedit: drop unused constructor arguments (API) All in-tree callers pass a repo and nothing else. Differential Revision: https://phab.mercurial-scm.org/D5541

phabricator: teach {phabreview} to work without --amend There's some possibility of this being wrong (e.g. if the url config was changed). But commit messages can be amended too, so that isn't bulletproof either. Having something seems better than just returning None.

phabricator: warn if unable to amend, instead of aborting after posting There was a divergence in behavior here between obsolete and strip based amending. I first noticed the abort when testing outside of the test harness, but then had trouble recreating it here after reverting the code changes. It turns out, strip based amend was successfully amending the public commit after it was posted! It looks like the protection is in the `commit --amend` command, not in the underlying code that it calls. I considered doing a preflight check and aborting. But the locks are only acquired at the end, if amending, and this is too large a section of code to be wrapped in a maybe-it's-held-or-not context manager for my tastes. Additionally, some people do post-push reviews, and amending is the default behavior, so they shouldn't see a misleading error message. The lack of a 'Differential Revision' entry in the commit message breaks a {phabreview} test, so it had to be partially conditionalized.

discovery: re-adjust a conditional wrongly changed In 71b0db4fa027 we updated this conditional to `<=`. As Yuya Nishihara pointed out, this was wrong.

cleanup: stop including thirdparty.cbor in builds Differential Revision: https://phab.mercurial-scm.org/D5524

tests: get access to thirdparty.cbor without requiring it to be installed This makes the test a noop when run from a tarball, but in return we can stop shipping a library we don't use. Differential Revision: https://phab.mercurial-scm.org/D5523

tests: add simplestorerepo to test-check-interfaces.py I'm not thrilled with this, but it'll help avoid future bitrot. Differential Revision: https://phab.mercurial-scm.org/D5521

simplestorerepo: migrate to in-hg CBOR code This is the only use of thirdparty.cbor outside of a test-* file, so it felt worthwhile to clean it up. Differential Revision: https://phab.mercurial-scm.org/D5520

simplestorerepo: minimal changes required to get this mostly working again I was going to change this code's use of CBOR to use our in-house CBOR code, but discovered it's been broken for a while. This messy change gets it back to a point where it mostly works, I think roughly as well as it ever did. Should we keep this and fix it up the rest of the way, or dump it in favor of the sqlite store? Would this be a good jumping-off point for some sort of union store that could facilitate a cleanup in remotefilelog? Differential Revision: https://phab.mercurial-scm.org/D5519

perfrevlogwrite: use progress helper on modern hg Differential Revision: https://phab.mercurial-scm.org/D5372

merge: make local file storage in the .hg/merge directory extensible This is similar to remotefilelog's 'getlocalkey' method, which must be overridden by systems which rely on full path names for access control purposes. Differential Revision: https://phab.mercurial-scm.org/D5534

context: schedule file prefetch before comparing for cleanliness When using a system like remotefilelog, we can occasionally run into scenarios where the local content cache does not have the data we need to perform these comparisons. These will be fetched on-demand, but individually; if the remotefilelog server isn't extremely low latency, the runtime of the command becomes dominated by the multiple getfile requests for individual files. By performing the prefetch, we can download these files in bulk, and save server resources and many network roundtrips. Differential Revision: https://phab.mercurial-scm.org/D5532

manifest: convert a recursive function to iterative one using stacks I am debugging a memory issue from yesterday where `hg update` goes upto taking 22GB of memory on our internal treemanifest repository. This is an interesting function and I saw memory consumption increasing while this function was running. It's sometimes hard to understand a recursive function and also the profile won't show you actual operations which took time, rather it will show you the function again and again in profile. I am yet to notice any memory consumption decrease with this patch, but I believe this will help like in making this a generator. Differential Revision: https://phab.mercurial-scm.org/D5413

obsutil: fix the issue5686 While traversing the obsolescence graph to find the successors sets of csets: In its 4th case (read comments of obsutil.successorssets to see all 4 cases) where we know successors sets of all direct successors of CURRENT, we were just missing a condition to filter out the case when a cset is pruned. And without this condition (that this patch added) it was making a whole successor set to [] just because of one pruned marker. For e.g:if following is the successors set of a cset A: A -> [a, b, c] if we prune c, we expect A's successors set to be [a, b] but you would get: A -> [] So this patch make sure that we calculate the right successorsset of csets considering the pruned cset (in split case). Differential Revision: https://phab.mercurial-scm.org/D5474

histedit: add user input to warning message on editing tagged commits This is a follow-up patch to D5489. Now, the user will be able to input yes/no(default) on the warning message. Initially, it was the sleep of 1s and histedit window opens. Changes were made as suggested by @mharbison72 and @yuja. Differential Revision: https://phab.mercurial-scm.org/D5494