tests: make python oneliner portable to python 3 in remotefilelog test Differential Revision: https://phab.mercurial-scm.org/D5602

remotefilelog: implement __bool__ as well as __nonzero__ for py3 Differential Revision: https://phab.mercurial-scm.org/D5601

remotefilelog: fix logging in retry decorator This still fails with an error about no exception being available to re-raise, but so it goes. Differential Revision: https://phab.mercurial-scm.org/D5600

basepack: avoid 'rbe' mode in Python 3 Differential Revision: https://phab.mercurial-scm.org/D5598

remotefilelog: do file IO in terms of bytes Differential Revision: https://phab.mercurial-scm.org/D5597

rust-cpython: using MissingAncestors from Python code As precedently done with LazyAncestors on cpython.rs, we test for the presence of the 'rustext' module. incrementalmissingrevs() has two callers within the Mercurial core: `setdiscovery.partialdiscovery` and the `only()` revset. This move shows a significant discovery performance improvement in cases where the baseline is slow: using perfdiscovery on the PyPy repos, prepared with `contrib/discovery-helper <repo> 50 100`, we get averaged medians of 403ms with the Rust version vs 742ms without (about 45% better). But there are still indications that performance can be worse in cases the baseline is fast, possibly due to the conversion from Python to Rust and back becoming the bottleneck. We could measure this on mozilla-central in cases were the delta is just a few changesets. This requires confirmation, but if that's the reason, then an upcoming `partialdiscovery` fully in Rust should solve the problem. Differential Revision: https://phab.mercurial-scm.org/D5551

rust: MissingAncestors.basesheads() This new API method on `MissingAncestors` leverages directly the Rust implementation for relative heads of a set, and also lowers the cost of returning the results to Python in the context of discovery. These interchange costs can probably be further reduced by implementing the `partialdiscovery` class in Rust, but that will be investigated in the 5.0 development cycle. Differential Revision: https://phab.mercurial-scm.org/D5584

discovery: using the new basesheads() Our ultimate goal is to switch eventually to a Rust implementation, but this move actually seems to increase the performance in a pure Python build. What follows is a quick measurement done on PyPy on repos prepared with `contrib/discovery-helper.sh 50 100`. Before: ! wall 0.894384 comb 0.890000 user 0.890000 sys 0.000000 (best of 11) ! wall 0.971199 comb 0.970000 user 0.950000 sys 0.020000 (max of 11) ! wall 0.927993 comb 0.925455 user 0.919091 sys 0.006364 (avg of 11) ! wall 0.921619 comb 0.920000 user 0.910000 sys 0.010000 (median of 11) After: ! wall 0.614278 comb 0.610000 user 0.610000 sys 0.000000 (best of 14) ! wall 0.789459 comb 0.790000 user 0.770000 sys 0.020000 (max of 14) ! wall 0.722765 comb 0.720000 user 0.715714 sys 0.004286 (avg of 14) ! wall 0.734448 comb 0.720000 user 0.720000 sys 0.000000 (median of 14) Differential Revision: https://phab.mercurial-scm.org/D5583

ancestor: incrementalmissingancestors.basesheads() This new method will avoid the need to access the `bases` attribute directly in `setdiscovery`, and to prefilter `nullrev` before passing it to the `heads()` revset. Being a method, it can transparently be reimplemented in a Rust (or any native) version. Differential Revision: https://phab.mercurial-scm.org/D5582

rust-cpython: set conversion for MissingAncestors.bases() Also I hope that the separate `py_set()` helper will help transition to proper `PySet` support in `rust-cpython` Took the opportunity to replace explict for loop with iteration and collect(). Differential Revision: https://phab.mercurial-scm.org/D5581

rust: dagop.headrevs() Rust counterparts This introduces two Rust implementations for `mercurial.dagop.headrevs`. The algorithm is identical to the Python version. Depending on the caller, one or the other could be the most practical, or the most performant, by minimizing the amount of memory copy and allocations. Differential Revision: https://phab.mercurial-scm.org/D5580

rust: factorized testing Graphs it will useful to use these outside of `ancestors`, too. Differential Revision: https://phab.mercurial-scm.org/D5579

rust-cpython: moved generic conversion fn out of ancestors module This will allow to use it easily from other submodules Differential Revision: https://phab.mercurial-scm.org/D5578

revset: transparently forward _intlist argument in all case We took a safe approach for the first take, we can get bolder now.

narrow: reuse narrowspec.updateworkingcopy() when narrowing Similar to the previous patch for widening, but here we also need to teach updateworkingcopy() to forcefully delete files that are not recorded in the dirstate as clean. That should be safe because the narrowing command (e.g. `hg tracked --removeinclude`) has already checked that the working copy is clean. Differential Revision: https://phab.mercurial-scm.org/D5511

narrow: reuse narrowspec.updateworkingcopy() when widening The widening of the working copy we do after widening a repo is practically the same as we do in a repo share after the store narrowspec has been changed in a different share. Let's reuse the code for that that we now have in the narrowspec module. Differential Revision: https://phab.mercurial-scm.org/D5510

narrow: move copytonarrowspec() out of setnarrowpats() I think it was a mistake to write the working copy's narrowspec every time the store narrowspec is written. This starts separating those actions. Differential Revision: https://phab.mercurial-scm.org/D5509

narrow: drop now-unnecessary reassignment of repo attributes Differential Revision: https://phab.mercurial-scm.org/D5507

packaging: allow running packaging with custom uid+gid for CentOS rpmbuild in CentOS 7 has a bug causing rpmbuild to fail with "Bad owner/group" if spec or source files are owned by a different user: https://github.com/rpm-software-management/rpm/issues/2 This makes it very annoying to try and build the CentOS RPMs on CentOS with Docker. As an alternative, this change makes it possible to do so, using an environment variable. Differential Revision: https://phab.mercurial-scm.org/D5571

hg-docker: fix Python 3.4 compatibility (for CentOS 7) I realize Mercurial is not targetting Python 3.4 compatibility, but without this change, it's not even possible to build it on CentOS 7 (and I assume the same is true for RHEL 7). Differential Revision: https://phab.mercurial-scm.org/D5570

copies: use node.nullrev instead of literal -1 Differential Revision: https://phab.mercurial-scm.org/D5593

copies: use node.wdirrev instead of inventing another constant for it Differential Revision: https://phab.mercurial-scm.org/D5592

narrow: extract repo property for store narrowmatcher When a repo lock is released, we try to persist the manifest cache. That involves getting the narrowmatcher for the manifestlog. That should not fail if the store and working copy narrowspecs are out of sync. Without this patch, the later patches in this series will fail because of that. Differential Revision: https://phab.mercurial-scm.org/D5508

narrow: copy store narrowspec to working copy immediately We no longer need to delay it until the end of the transaction since we now restore a backup if the transaction aborts. Differential Revision: https://phab.mercurial-scm.org/D5506

narrow: include working copy narrowspec in transaction journal Now that we have separate narrowspecs for the store and the working copy, we need to include both in the transaction journal. Differential Revision: https://phab.mercurial-scm.org/D5505

narrow: make dirstateguard back up and restore working copy narrowspec instead We used to have only one narrowspec for the store and the working copy, but now that we have one narrowspec for each, it seems clear that the dirstateguard was supposed to back up and restore the narrowspec associated with the working copy, not the one associated with the store. clearbackup() (for the store narrowspec) is not needed because the presence of the file in localrepository._journalfiles() takes care of that. Differential Revision: https://phab.mercurial-scm.org/D5504

narrow: include journal.narrowspec in transaction journal We had missed this file before, which led to it lying around after the transaction completed. Differential Revision: https://phab.mercurial-scm.org/D5556

progress: deprecate ui.progress() It is now just a weird wrapper for ui.makeprogress(). Differential Revision: https://phab.mercurial-scm.org/D5531

context: use scmutil.matchfiles instead of matchmod.match(exact=True) Differential Revision: https://phab.mercurial-scm.org/D5591

histedit: fix call to _getgoal() by adding a byteskwargs() wrapper I also added some b-prefixes while I was here because I got confused and it seems silly to not just add them since it clarifies the whole change. Differential Revision: https://phab.mercurial-scm.org/D5585

revset: introduce an API that avoids `formatspec` input serialization Instead of having the data fully serialized, the input can be directly inserted in the tree at a later stage. Just using it for simple "%ld" case provide a significant boost. For example here are the impact on a sample discovery run between two pypy repositories with arbitrary differences (using hg perfdiscovery). $ hg perfdiscovery before: ! wall 0.700435 comb 0.710000 user 0.700000 sys 0.010000 (median of 15) after: ! wall 0.501305 comb 0.510000 user 0.490000 sys 0.020000 (median of 20)

revset: detect integer list on parsing Right now, using "%ld" with `repo.revs("…%ld…", somerevs)` is very inefficient, all items in `somerevs` will be serialized to ascii and then reparsed as integers. If `somerevs` contains just an handful of entry this is fine, however, when you get to thousands or hundreds of thousands of revisions this becomes very slow. To avoid this serialization we need to first detect this situation. The code involved in the whole process is quite complex so we start simple and focus on some "simple" but widespread cases. So far we only detect the situation and don't do anything special about it. The singled out will be serialized in `formatspec` in the same way as before.

revert: extract "%ld" formatting in a _formatintlist function We'll have to reuse this logic in different places.

revset: extract parsing logic out of formatspec We want to be able to perform better handling of some input when running revset (eg: `repo.revs("%ld", somerevs)`). The first step is to be able to access some of the parsed content before it gets substituted. There are many possible different substitutions, we'll add support for them gradually. In this changeset we support none, we just split some logic in a sub function as a preparatory step.

revset: enforce "%d" to be interpreted as literal revision number (API) (BC) Before this change, `formatspec("%d", x)` results in `"%d" % int(x)`. This seems simple and correct until you consider `nullrev`. In revset, a direct "-1" symbol is equivalent to `tip` not `nullrev`. This is a subtle error that went undetected for a while. Wrapping the revision number inside 'rev()' remove the ambiguity, preserving nullrev value passed to formatspec. It got caught by the rebase code, were the following wrongly returned `[1]`: repo.revs("children(%d) and ancestors(%ld)", 0, [nullrev]) This is flagged as API, because `%d` can be used for non-revision integer argument of revset function. We probably need to introduce a new '%…' substitution to allow literal integer (maybe `%i`). However, the `%d` usage is currently widespread for revision number so it is important to fix this issue for `%d`. This choice is reinforced by the fact _intlist is implemented as revisions only. Restricting `%d` to revision only makes things more consistent. This bug can become especially tricky since `_intlist` recognize `nullrev` right. So `revs('%ld', [-1, 0])` → select `[nullrev, 0]` but `revs('%ld', [-1])` is simplified and treated as `%d` selecting `[tip]`. Another side effect is that "%d" of an unknown revision simply match nothing. It was previously raising and error. This is consistent with what "%ld" (and `_intlist`) is doing, so it seems like a good move.

revset: remove the last usage of "%d" for a non-revision entry In order to fix an important bug, we are about to narrow the semantic of "%d" in revset. This is one of the few exceptions that we need to get rid of before being able to fix the bug. See the later semantic narrowing changeset for full rationale on the semantic change.

revset: remove a rare usage of "%d" for a non-revision item In order to fix an important bug, we are about to narrow the semantic of "%d" in revset. This is one of the few exceptions that we need to get rid of before being able to fix the bug. See the later semantic narrowing changeset for full rationale on the semantic change.

merge with stable

tests: replace mockmakedate function in test-amend.t This is a follow up patch for D5554. This replaces mockmakedate function by `tests/mockmakedate.py`. Differential Revision: https://phab.mercurial-scm.org/D5577

histedit: add rewrite.update-timestamp support to fold and mess This adds the config option to update time to current in histedit fold and mess options. Setting rewrite.update-timestamp option to `True` will update the timestamp to current time. This also adds `tests/mockmakedate.py` for supplying testable values in case current time is invoked in the tests. Differential Revision: https://phab.mercurial-scm.org/D5554

histedit: remove trailing space from warning message

histedit: fix weird indent of i18n text

progress: specify updatebar() function by constructor argument This makes it easy for ui extensions to intercept progress messages. It also seems slightly nicer in that scmutil.progress doesn't touch ui internals.

progress: change _updatebar() to take parameters as arguments I want to move updatebar() back to ui. See the next patch for why.

archival: construct progress helper through ui.makeprogress() No idea why we didn't.

help: document rewrite.backup-bundle option

repair: move ui.history-editing-backup to [rewrite] section Since we have the "rewrite" section for general history-editing options, the backup option should be there.

revlog: use separate variables to track version flags It wasn't obvious to me that "versionflags" is used both to define the default version+features value for new revlogs and to track the value read from a revlog. We rename the former use and add explicit assignment of "versionflags" later to differentiate between the two. Differential Revision: https://phab.mercurial-scm.org/D5564

revlog: inline opener options logic into _loadindex() We always call _loadindex() during __init__. But we also call _loadindex() as part of censorrevision(). Before, when reloading the index during censorrevision(), we would lose the configured mmapindexthreshold setting from the opener. By inlining the logic in _loadindex(), we ensure that opener options are always respected when loading the index. Differential Revision: https://phab.mercurial-scm.org/D5563

revlog: store mmaplargeindex as an instance attribute This makes it more consistent with everything else. It also makes it possible to move the opener/features processing into _loadindex(). Differential Revision: https://phab.mercurial-scm.org/D5562

revlog: always enable generaldelta on version 2 revlogs This commit starts the process of diverging version 2 revlogs from version 1 revlogs. generaldelta is a useful feature and has been enabled by default for ages. I can't think of a good reason why the feature should be disabled. Yes, it is true changelogs today don't have generaldelta enabled. But that's because they don't have delta chains enabled, so generaldelta makes no sense there. This commit makes generaldelta always enabled on version 2 revlogs. As part of this, one-off code in changelog.py mucking with revlog.version had to be made conditional on the revlog version, as we don't want to change revlog feature flags on version 2 revlogs. The fact this code exists is horrible and stems from revlog options being shared by the opener. We probably want a better API here. But that can wait for another patch. Differential Revision: https://phab.mercurial-scm.org/D5561

revlog: rename v to versionflags Single letter variables are harder to read. Differential Revision: https://phab.mercurial-scm.org/D5560

revlog: always process opener options I'm not sure when ``opener.options`` would ever be explicitly set to None. It is definitely not possible to construct a repo this way because ``localrepo.resolvestorevfsoptions()`` always returns a dict and ``localrepo.makelocalrepository()`` always sets ``opener.options`` to this value. Because we always execute this code now, if options are empty we defaulted to creating version 0 revlogs. So we had to change the code slightly to fall back to the default revlog version and flags. As astute reader will note that it is not possible to create version 0 revlogs now. However, I don't think it was possible before, as this required ``opener.options`` being unset, which I don't think was possible. I suspect this means our test coverage for version 0 revlog repositories is possibly non-existent! Since I don't see a config option to disable revlog v1, I'm not even sure if we had a way to create new repos with version 0 revlogs! Who knows. Differential Revision: https://phab.mercurial-scm.org/D5559

internals: minor rewriting of revlogs documentation In preparation for formalizing revlog version 2. Differential Revision: https://phab.mercurial-scm.org/D5558

tests: make HGCATAPULTSERVERPIPE imply HGTESTCATAPULTSERVERPIPE I had attempted to do this before, but missed this case. This makes it so that one can do the following to get catapult traces that include both the .t test name (and non-hg commands run by that .t test) *and* the hg-internal tracing, in one trace: HGCATAPULTSERVERPIPE=/tmp/catapult.pipe run-tests.py <args> Without this change, we need to specify both `HG{,TEST}CATAPULTSERVERPIPE`; if we specify just the TEST one, we only get the .t tests (no hg-internals), which is working as intended. If we specify the non-TEST one, we only get the hg-internals (not the rest of the .t test), which was not intended. If you want to restore the previous behavior (just hg internals, not the stuff from the .t tests), run like: HGTESTCATAPULTSERVERPIPE=/dev/null \ HGCATAPULTSERVERPIPE=/tmp/catapult.pipe \ run-tests.py <args> Differential Revision: https://phab.mercurial-scm.org/D5569

tests: add test for warning on histedit with tagged commits This is a follow-up patch to 7b7e081f8954 (D5494). Differential Revision: https://phab.mercurial-scm.org/D5552

extdiff: move external tool command line building into separate function

test: explicit a rebase source in test-rebase-collapse.t At the time of the rebase, the current revision is `null` so this rebase should fail. However, a current bug in formatspec makes it pass, we fix rebase call of the "affected" test to express its actual intent before we fix the actual bug in a later changeset.

tags: join string that's unnecessarily split across lines Differential Revision: https://phab.mercurial-scm.org/D5574

dispatch: add newline after ui.log "ui blocked ms" message Differential Revision: https://phab.mercurial-scm.org/D5573

remotefilelog: fix {file_copies} template keyword remotefilelog overrides templatekw.getrenamedfn(). I changed the return type of that function in ec37df9042f9 (templatekw: make getrenamed() return only filename, not nodeid, 2018-03-26). So we need to change remotefilelog too. The function is used by the {file_copies} template keyword and by `hg log --copies`. Differential Revision: https://phab.mercurial-scm.org/D5576

tests: demonstrate broken {file_copies} template with remotefilelog Differential Revision: https://phab.mercurial-scm.org/D5575

contrib: update window title when possible in perf-revlog-write-plot.py This is useful when comparing multiple graphs.

histedit: add newline after ui.log "# acttions to histedit" message Differential Revision: https://phab.mercurial-scm.org/D5572

rust-cpython: bindings for MissingAncestors The exposition is rather straightforward, except for the remove_ancestors_from() method, which forces us to an inefficient conversion between Python sets and Rust HashSets. Two alternatives are proposed in comments: - changing the inner API to "emit" the revision numbers to discard this would be a substantial change, and it would be better only in the cases where there are more to retain than to discard - mutating the Python set directly: this would force us to define an abstract `RevisionSet` trait, and implement it both for plain `HashSet` and for a struct enclosing a Python set with the GIL marker `Python<'p>`, also a non trivial effort. The main (and seemingly only) caller of this method being `mercurial.setdiscovery`, which is currently undergoing serious refactoring, it's not clear whether these improvements would be worth the effort right now, so we're leaving it as-is. Also, in `get_bases()` (will also be used by `setdiscovery`), we'd prefer to build a Python set directly, but we resort to returning a tuple, waiting to hear back from our PR onto rust-cpython about that Differential Revision: https://phab.mercurial-scm.org/D5550

rust-cpython: generalised conversion function Because `hg::ancestors::MissingAncestors` has methods taking some `HashSet<Revision>` besides `impl IntoIterator<Item = Revision>` as parameters we'll need the more generic `rev_pyiter_collect()` function to also build these Differential Revision: https://phab.mercurial-scm.org/D5549

rust-cpython: style consistency leftovers In particular, during review of `LazyAncestors` bindings, most `match` statements for error conversion have been replaced by higher level methods of `Result` and most personal insecurity comments have been removed. This makes it more systematic. Differential Revision: https://phab.mercurial-scm.org/D5548

rust-cpython: consistency in use of hg-core constructs While not everybody likes the `CoreLazy` and `CoreIterator` aliases, it's better not to mix them with direct references. Note: it's quite possible in the future that these would stop being exposed at the top of the `hg` crate Differential Revision: https://phab.mercurial-scm.org/D5547

rust-cpython: rustdoc improvements By default, `cargo doc` builds the documentation for public constructs only, so we make public those that can. Since `cindex` is not safe, we keep it private. Unfortunately, the macro syntax of rust-cpython doesn't allow us to document the classes directly, so we resort to do that at the module level. Differential Revision: https://phab.mercurial-scm.org/D5546

fuzz: stop building Python in the Makefile $OUT/sanpy is now built by the build.sh in oss-fuzz, which simplifies the setup here considerably. Differential Revision: https://phab.mercurial-scm.org/D5553

py3: add a test to whitelist caught by buildbot Differential Revision: https://phab.mercurial-scm.org/D5555

tests: de-flake test-narrow-share.t by making dirstate predictable test-narrow-share.t was sometimes (~0.5% on my machine) failing like this: @@ -61,7 +61,7 @@ A d3/g $ hg -R main debugdirstate --no-dates n 644 2 set d1/f - n 644 2 unset d3/f + n 644 2 set d3/f a 0 -1 unset d3/g n 644 2 set d5/f n 644 2 set d7/f The timestamp for d3/f would get set if it was determined at some point that it was clean. That check is usually done when the user runs `hg st`. We don't do that before the failure in the test case, but it happens at the end of the `hg clone` call. So if the file system's time happens to roll over after the clone's working copy has been written, but before its (final) dirstate has been written, we can end up with a set timestamp there. This patch makes it consistent by sleeping for 2 seconds so the timestamp gets reliably set. Differential Revision: https://phab.mercurial-scm.org/D5568

convert: don't drop commits that are empty in the source when using --filemap I ran into this when using `hg lfconvert --to-normal` (which uses the filemap class internally), and saw that commits with nothing but a branch change were dropped. We could put in an option that only lfconvert uses internally. But silently dropping anything other than a commit where all changes were excluded seems unintended. For example, there's a message in mercurial_sink.putcommit() if it drops an empty commit. (And the reason that isn't kicking in here is because lfconvert isn't passing --filemap, so the self.filemapmode conditional there is always False.) The naive change of `return not files` broke test-convert-filemap.t, so this is a little more elaborate than needed for converting from largefiles.

convert: add missing numcommits() override to hg sources Otherwise, the progressbar in converter.walktree() gets None as its total during the initial scan. (Though that seems harmless, and there are other foreign vcs sources without this.)

testrunner: make `-j100 --runs-per-test=100 test-foo.t` use 100 jobs It would use only 1 job since 1039404c5e1d (run-tests: print number of tests and parallel process count, 2018-10-13), but I'm pretty sure that was unintentional. Differential Revision: https://phab.mercurial-scm.org/D5557

narrow: use merge.ACTION_GET instead of duplicating 'g' constant As suggested by Yuya. Differential Revision: https://phab.mercurial-scm.org/D5512

narrow: drop explicit dirstate write The dirstate is written when the wlock is released, so we don't need to write it explicitly in updateworkingcopy(). I don't know why I put it there in the first place (tests pass without it even in the commit that introduced it). Differential Revision: https://phab.mercurial-scm.org/D5515

histedit: fix --continue and --abort when curses is enabled I overlooked this when I did the initial import. Differential Revision: https://phab.mercurial-scm.org/D5540

graft: abort if --date/user specified with --currentdate/currentuser (BC) They are conflicting options. Let's explicitly ban such combination. Note that this means you need "--no-currentdate --date" to override --currentdate specified in aliases/defaults.

amend: turn currentdate into tri-state bool so config option can be negated Otherwise, rewrite.update-timestap couldn't be overridden by --no-currentdate. FWIW, which do we like? --no-currentdate vs --no-current-date. If the latter is preferred, maybe we can add "graft --current-date" and deprecate "--currentdate".

discovery: improve partial discovery documentation The older documentation could be improved.

discovery: add a simple `addinfo` method The method can directly process a sample result. This makes the main code simpler to follow.

discovery: move missing tracking inside the partialdiscovery object This is the final set that we need to track to have a fully up to date information within the object.

discovery: add a `iscomplete` method to the `partialdiscovery` object The method is used by higher level logic to know if the initial discovery question has been answered.

discovery: update undecided from common directly within partialdiscovery The object knows when there are changes in the "common" set, so it is a good place to update the undecided set.

discovery: move undecided set on the partialdiscovery To initialize it, we need to know the discovery target. This commit only move the set on the `partialdiscovery` object, later changeset will take advantage of it.

histedit: use context manager for locks Differential Revision: https://phab.mercurial-scm.org/D5542

histedit: drop unused constructor arguments (API) All in-tree callers pass a repo and nothing else. Differential Revision: https://phab.mercurial-scm.org/D5541

phabricator: teach {phabreview} to work without --amend There's some possibility of this being wrong (e.g. if the url config was changed). But commit messages can be amended too, so that isn't bulletproof either. Having something seems better than just returning None.

phabricator: warn if unable to amend, instead of aborting after posting There was a divergence in behavior here between obsolete and strip based amending. I first noticed the abort when testing outside of the test harness, but then had trouble recreating it here after reverting the code changes. It turns out, strip based amend was successfully amending the public commit after it was posted! It looks like the protection is in the `commit --amend` command, not in the underlying code that it calls. I considered doing a preflight check and aborting. But the locks are only acquired at the end, if amending, and this is too large a section of code to be wrapped in a maybe-it's-held-or-not context manager for my tastes. Additionally, some people do post-push reviews, and amending is the default behavior, so they shouldn't see a misleading error message. The lack of a 'Differential Revision' entry in the commit message breaks a {phabreview} test, so it had to be partially conditionalized.

discovery: re-adjust a conditional wrongly changed In 71b0db4fa027 we updated this conditional to `<=`. As Yuya Nishihara pointed out, this was wrong.

cleanup: stop including thirdparty.cbor in builds Differential Revision: https://phab.mercurial-scm.org/D5524

tests: get access to thirdparty.cbor without requiring it to be installed This makes the test a noop when run from a tarball, but in return we can stop shipping a library we don't use. Differential Revision: https://phab.mercurial-scm.org/D5523

tests: add simplestorerepo to test-check-interfaces.py I'm not thrilled with this, but it'll help avoid future bitrot. Differential Revision: https://phab.mercurial-scm.org/D5521

simplestorerepo: migrate to in-hg CBOR code This is the only use of thirdparty.cbor outside of a test-* file, so it felt worthwhile to clean it up. Differential Revision: https://phab.mercurial-scm.org/D5520

simplestorerepo: minimal changes required to get this mostly working again I was going to change this code's use of CBOR to use our in-house CBOR code, but discovered it's been broken for a while. This messy change gets it back to a point where it mostly works, I think roughly as well as it ever did. Should we keep this and fix it up the rest of the way, or dump it in favor of the sqlite store? Would this be a good jumping-off point for some sort of union store that could facilitate a cleanup in remotefilelog? Differential Revision: https://phab.mercurial-scm.org/D5519

perfrevlogwrite: use progress helper on modern hg Differential Revision: https://phab.mercurial-scm.org/D5372

merge: make local file storage in the .hg/merge directory extensible This is similar to remotefilelog's 'getlocalkey' method, which must be overridden by systems which rely on full path names for access control purposes. Differential Revision: https://phab.mercurial-scm.org/D5534

context: schedule file prefetch before comparing for cleanliness When using a system like remotefilelog, we can occasionally run into scenarios where the local content cache does not have the data we need to perform these comparisons. These will be fetched on-demand, but individually; if the remotefilelog server isn't extremely low latency, the runtime of the command becomes dominated by the multiple getfile requests for individual files. By performing the prefetch, we can download these files in bulk, and save server resources and many network roundtrips. Differential Revision: https://phab.mercurial-scm.org/D5532

manifest: convert a recursive function to iterative one using stacks I am debugging a memory issue from yesterday where `hg update` goes upto taking 22GB of memory on our internal treemanifest repository. This is an interesting function and I saw memory consumption increasing while this function was running. It's sometimes hard to understand a recursive function and also the profile won't show you actual operations which took time, rather it will show you the function again and again in profile. I am yet to notice any memory consumption decrease with this patch, but I believe this will help like in making this a generator. Differential Revision: https://phab.mercurial-scm.org/D5413

obsutil: fix the issue5686 While traversing the obsolescence graph to find the successors sets of csets: In its 4th case (read comments of obsutil.successorssets to see all 4 cases) where we know successors sets of all direct successors of CURRENT, we were just missing a condition to filter out the case when a cset is pruned. And without this condition (that this patch added) it was making a whole successor set to [] just because of one pruned marker. For e.g:if following is the successors set of a cset A: A -> [a, b, c] if we prune c, we expect A's successors set to be [a, b] but you would get: A -> [] So this patch make sure that we calculate the right successorsset of csets considering the pruned cset (in split case). Differential Revision: https://phab.mercurial-scm.org/D5474

histedit: add user input to warning message on editing tagged commits This is a follow-up patch to D5489. Now, the user will be able to input yes/no(default) on the warning message. Initially, it was the sleep of 1s and histedit window opens. Changes were made as suggested by @mharbison72 and @yuja. Differential Revision: https://phab.mercurial-scm.org/D5494

sparse: fix debugrebuilddirsate when narrow extension is enabled dirstate wrapping in narrow extension converts the manifest object to a list. So let's assume we get a list of files in sparse extension. Differential Revision: https://phab.mercurial-scm.org/D5481

tests: show that debugrebuilddirstate is broken with narrow+sparse This adds a test to show that debugrebuildirstate is broken when both narrow and sparse extensions are enabled. Differential Revision: https://phab.mercurial-scm.org/D5480

sparse: don't enable on clone if it was a narrow clone When both sparse and narrow extensions are enabled and we do a narrow clone, sparse enables itself because it reads --include flag and thinks that user is trying to do a sparse clone. This patch changes that behavior, and now if both extensions are enabled and user passes `--narrow`, then the includes and excludes won't be considered as part of sparse profile. Differential Revision: https://phab.mercurial-scm.org/D5479

tests: add tests for narrow clone when both sparse and narrow are enabled This patch adds tests for doing a narrow clone when client has both sparse and narrow enabled. As shown by this patch, we can see that the narrow clone has sparse enabled which we does not want here. Upcoming patch will fix this. Also while writing tests, I realized we don't have a flag to list the current sparse profile. Differential Revision: https://phab.mercurial-scm.org/D5478

histedit: crashing with a more useful error message on empty defaultrev Before this, `hg --config histedit.defaultrev= histedit` would crash with File "destutil.py", line 385, in desthistedit if revs: UnboundLocalError: local variable 'revs' referenced before assignment Differential Revision: https://phab.mercurial-scm.org/D5543

progress: check what type of progress bar to use only once per topic This seems to have sped up `hg perfprogress` from 1.78 s to 1.41 s. Differential Revision: https://phab.mercurial-scm.org/D5530

progress: split up _print() method in bar-updating and debug-printing I just thought this was clearer, but it turned out to also simplify the next patch. This seems to have sped up `hg perfprogress` from 1.85 s to 1.78 s. Differential Revision: https://phab.mercurial-scm.org/D5529

progress: move cached debug flag from progress.progbar to scmutil.progress It's simpler this way. One possible drawback (and a possisble advantage) is that we now check the debug flag once per topic, so processes that generate new topics all the time will still check the flag frequently. Differential Revision: https://phab.mercurial-scm.org/D5528

progress: write ui.progress() in terms of ui.makeprogress() I think ui.makeprogress() should be the preferred interface and we should deprecate ui.progress(). All in-core callers already use ui.makeprogress(). Moving the logic to the scmutil.progress() will let us make further improvements. This seems to have sped up `hg perfprogress` from 1.92 s to 1.85 s, perhaps because we now skip the indirection of updating the progress bar via ui.progress(). Differential Revision: https://phab.mercurial-scm.org/D5527

testrunner: avoid capturing a regex group we don't care about Differential Revision: https://phab.mercurial-scm.org/D5536

remotefilelog: add newlines to ui.log() invocations The comment at the top of ui.log() says that the message should be a newline-terminated string. When using the blackbox logger, if the string does not end in a newline, the appearance in the file gets quite messy. This sometimes leaves a string with just the newline as the message, these are logged by the blackbox logger just fine. I don't know what other loggers do when logging structured data and a message that is just the newline. Differential Revision: https://phab.mercurial-scm.org/D5533

xdiff: don't attempt to use fuzzer inputs larger than 100k This is the recommended approach from [0], and limiting the input was suggested in https://github.com/google/oss-fuzz/issues/2076 when discussing our broken coverage build. 0: https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md#custom-libfuzzer-options-for-clusterfuzz Differential Revision: https://phab.mercurial-scm.org/D5525

tests: support passing testcase after .t paths that have path separators This probably could have been implemented by changing the regex above this bit of code, but I wasn't sure if it would end up handling various OSes correctly, so I decided to go with this version instead. Previously: $ tests/run-tests.py tests/test-ssh.t -l running 2 tests using 2 parallel processes .. # Ran 2 tests, 0 skipped, 0 failed. $ tests/run-tests.py tests/test-ssh.t#sshv1 -l running 0 tests using 0 parallel processes # Ran 0 tests, 0 skipped, 0 failed. Now: $ tests/run-tests.py tests/test-ssh.t -l running 2 tests using 2 parallel processes .. # Ran 2 tests, 0 skipped, 0 failed. $ tests/run-tests.py tests/test-ssh.t#sshv1 -l running 1 tests using 1 parallel processes . # Ran 1 tests, 0 skipped, 0 failed. Differential Revision: https://phab.mercurial-scm.org/D5535

progress: document progress.debug config option I think it was not spotted by test-check-config.t that we had not documented it because no caller refers to the ui object simply as "ui" (it was either "self.ui" or just "self"). Differential Revision: https://phab.mercurial-scm.org/D5526

tests: migrate test-wireproto-serverreactor.py to our internal CBOR This leaves the only client of thirdparty.cbor as test-cbor.py, which appears to be testing the behavior of cborutil against cbor. We should figure out some appropriate test strings and drop thirdparty.cbor, but that's a mission for another day. Differential Revision: https://phab.mercurial-scm.org/D5522

state: update comment about use of CBOR We use our internal cbor library, not the vendored one in thirdparty. Differential Revision: https://phab.mercurial-scm.org/D5518

rust-ancestors: adjust branches and inline comments per previous change Now the top-level "if" can be read as both_visit|revs_visit|bases_visit.

rust-ancestors: remove unreachable conditions from missing_ancestors()