tests: add test to show widening is broken without ellipsis This patch adds a test to show that widening a narrow clone is broken if ellipsis is disabled. I don't think I can add cases to existing test-narrow-widen.t and check-in a failing version of that. So I created a copy. Once the test is fixed, we can merge this new test file back into the original one using testcases. Also, this is just testing treemanifest case because having two cases and both or are failing with different outputs is a bit hard to manage. This is important because upcoming patches will try to fix the broken part. Differential Revision: https://phab.mercurial-scm.org/D4382

stringutil: emit multiple chunks when pretty printing This avoids concatenating output inside pprintgen() itself. But the real reason for this is it will make it easier to add indentation, as we'll need to account for indentation when emitting each individual object in a collection. The verbosity of this code compared to the original is a bit unfortunate. But I suppose this is the price to pay for having nice things (streaming and indenting). We could probably abstract the "print a collection" bits into a generic function to avoid some duplication. But I'm not overly inclined to do this. Differential Revision: https://phab.mercurial-scm.org/D4398

stringutil: refactor core of pprint so it emits chunks This commit splits the core of pprint() to a new function that is a generator of chunks instead of a function returning a single value. This will make it possible to stream output without waiting for all data to be formatted first. And it will make it easier to implement support for indenting. Differential Revision: https://phab.mercurial-scm.org/D4397

contrib: import the relnotes script from the release-tools repo I figure this makes more sense to keep in the main repo, as it's a guide of sorts on how to use the releasenotes extension in the presence of commits that don't get relnotes annotations. Ported to Python 3, cleaned up some logic in a few places, but for the most part it's what we've been using for years. Differential Revision: https://phab.mercurial-scm.org/D4291

hghave: move from requiring the PYTHON3 env var to looking for `python3` We're starting to write new utility scripts in Python 3, and I think it's reasonable to expect `python3` to be something we should use if it's present. I wrote the regex to specifically ban Python 3 prior to Python 3.5 because we won't ever support that. I moved the only test that was using $PYTHON3 to just using `python3` directly. Differential Revision: https://phab.mercurial-scm.org/D4290

rename: return error status if any rename/copy failed Ever since 447ea621e50e (copy: propagate errors properly, 2007-12-06), we have returned an error status if the source file did not exist. That commit did not return error status for any other errors, and it's unclear if that was on purpose or not. It seems to me like we should return an error in the other cases to, so that's what this patch does. Differential Revision: https://phab.mercurial-scm.org/D4419

rename: emit hint about using --after consistently Both the code and the message for the case where the source file was missing was inconsistent with the other similar messages. Differential Revision: https://phab.mercurial-scm.org/D4418

rename: quote hg commands in warnings I think we usally use single quotes around hg commands in messages. Differential Revision: https://phab.mercurial-scm.org/D4417

zsh_completion: complete hg files Differential Revision: https://phab.mercurial-scm.org/D4361

zsh_completion: adjust usage of common options addremove, remove and serve accept -S/--subrepos, and purge actually doesn't. qnew accepts -I/--include and -X/--exclude. qrefresh accepts -D/--currentdate, -U/--currentuser, -d/--date and -u/--user. rebase accepts -n/--dry-run. Differential Revision: https://phab.mercurial-scm.org/D4360

resolve: make condition clearer, and able to handle a new flag Differential Revision: https://phab.mercurial-scm.org/D4377

shelve: drop mention of obsshelve file Core shelve extension does not use obsolescence, we clean up this unused file before adding a new info file.

shelve: rename method for data write/read There are no user of this code nor any code related to obsshelve in the extensions right now, so we rename the function before using them. Note: The shelvedfile looks a bit strange. We should probably refactor it, however, this is an adventure for another series.

shelve: move createcmd next to _docreatecmd It is simpler not to have to jump around the file all the time.

minirst: filter blocks by full path to section

minirst: mark getsections() as an internal helper

help: rewrite parsing of help topic to not drop section name with dots A subtopic is no longer lowercased since it should be considered a part of a topic path, not a section filter.

test-help: add tests showing that sections can't be filtered by dotted name

revlogdeltas: move finddeltainfo filtering inside _candidategroups Now all "simple" filtering logic is gathered in one place. The `finddeltainfo` method is more straightforward and the `_candidate_groups` function knows more about filtering revisions before we compute a delta against them.

revlogdeltas: split candidate groups selection from the filtering logic The group iteration has two main components: * walking candidates, the logic that we are about to extend to build intermediate snapshots, * Making sure we test diffs against interesting bases. No duplicated tests, skipping empty revisions, etc. We split `_candidategroups` to separate the two components. This achieves two goals: * It will be simpler to update the walking logic for intermediate snapshots, * We can gather the filtering logic from `finddeltainfo` into `_candidategroups` to centralize it.

revlogdeltas: pass revision number to _candidatesgroups The `_candidates_groups` logic works on revisions, so the nodes are currently just converted to revs at the top of the function. It seems cleaner to instead pass revision numbers to the function. Having this conversion out of the way will help for further cleanups.

revlogdeltas: extract _getcandidaterevs in a function The logic barely uses the object it is attached to. This is an important function that we will clean up in the coming changesets. Moving it at the top level helps us with that cleanup.

revlogdeltas: always return a delta info object in finddeltainfo Previously, the method returned `None` when a full snapshot was needed. The caller had to determine how to produce one itself. In practice, building a `_deltainfo` object for a full snapshot is simple. So we build it at the `finddeltainfo` level and always return a `_deltainfo` object. The caller can now simply process the `_deltainfo` return in all cases.

revlogdeltas: move special cases around raw revisions in finddeltainfo The method already contains logic for no-diff cases. Having everything in the same place is more consistent and unlocks other code improvements.

revlogdeltas: extra fulltext building in its own function The process of building a full text from a delta is complex enough to deserve isolation. The previous function mixed cache management and computation, we now have clearer boundaries and simpler functions/methods.

revlog: split functionality related to deltas computation in a new module The revlog module is getting big and this logic is getting more and more advanced. Moving it to `mercurial.revlogutils.deltas` split a lot off revlog.py and will help this logic to become less interleaved with revlog. The code is simply moved without modification (but for namespace changes). Refactoring and improvement will be made in later changesets.

revlog: split constants into a new `revlogutils.constants` module We want to split some logic out of the main revlog file (the delta computing logic). However, this logic needs access to multiple constants related to the revlog. So we move all revlog related constants into a new module that could be imported from multiple places. We don't copy the file (preserving blame history) because there are only a few moving lines. Also, copying the file would result in annoying merge conflicts with ongoing work from others contributors.

rebase: skip extinct revisions even if it has no successor in rebase set Differential Revision: https://phab.mercurial-scm.org/D4408

tests: add test showing that rebase of extinct commit with successor fails As the test case shows, attempting to rebase a commit that has a successor that is not in the rebase set and not in the destination currently fails because it "will cause divergences". However, it doesn't seem like there's any harm in skipping the extinct commit. I suspect this case missed simply because extinct revisions are usually hidden. Differential Revision: https://phab.mercurial-scm.org/D4407

tests: add log output after stripping for context for next test It's not obvious what the repo state will be after stripping some nodes and obsmarkers, so let's include the log output to make it easier to follow the test case that comes after. Differential Revision: https://phab.mercurial-scm.org/D4406

obsutil: make sure "addedmarkers" is not None in getobsoleted() Similar reasoning as in 52e6171ec822 (obsutil: make sure "addedrevs" is not None in getobsoleted(), 2018-08-25). Differential Revision: https://phab.mercurial-scm.org/D4411

revsetlang: fix position of '-' in spaceless 'a-b' expressions I don't know if there is an observable difference, but it seems like an obvious fix. Differential Revision: https://phab.mercurial-scm.org/D4410

tests: mark manifestfulltextcache as conditional on revlog store This file is currently specific to the revlog store. Other stores may not implement it. Mark it as such in test output. Differential Revision: https://phab.mercurial-scm.org/D4395

manifest: use fulltextcache instead of _fulltextcache fulltextcache is part of the documented interface and is aliased to _fulltextcache. Differential Revision: https://phab.mercurial-scm.org/D4394

manifest: rename manifestlog._revlog to _rootstore Because it may not be a revlog. Differential Revision: https://phab.mercurial-scm.org/D4393

manifest: use public API for obtaining storage object We should not be accessing private attributes on the manifestlog, since its behavior is governed by an interface. Differential Revision: https://phab.mercurial-scm.org/D4392

perf: add function for obtaining manifest revision This will use the modern API if available or fall back to the legacy direct revlog access case. Without this, the perf command won't work on all code bases. Differential Revision: https://phab.mercurial-scm.org/D4391

perf: use storage API for resolving manifest node lookup() isn't part of the storage API. And this code shouldn't be accessing manifestlog._revlog directly for the modern code base. So let's port it to the modern API. Note that the previous code was busted for cases where we needed to call lookup() because lookup() isn't exposed by manifestrevlog any more. This change is strictly BC breaking because we no longer support resolving partial nodes. But it is a perf* command and I don't think we should flag the change as such. Differential Revision: https://phab.mercurial-scm.org/D4390

manifest: change terminology for storage in context classes Storage may not be backed by revlogs. So it is wrong to refer to storage as "revlog" or use abbreviations like "rl." This commit replaces revlog terminology in manifest context classes and starts referring to things as "store" or "storage." Differential Revision: https://phab.mercurial-scm.org/D4389

manifest: don't go through revlog to access node symbols The revlog module re-exports some node.* symbols courtesy of importing them. Let's access the node.* symbols via our local import. Differential Revision: https://phab.mercurial-scm.org/D4388

manifest: make tree a public attribute changegroup generation accesses this attribute. We should make it public and expose it on the interface. Differential Revision: https://phab.mercurial-scm.org/D4387

manifest: proxy to revlog instance instead of inheriting Previously, manifestrevlog inherited revlog.revlog and therefore exposed all its APIs. This inevitably resulted in consumers calling low-level revlog APIs. As part of abstracting storage, we want to formalize the interface for manifest storage. The revlog API is much too large to define as the interface. Like we did for filelog, this commit divorces the manifest class from revlog so that we can standardize on a smaller API surface. The way I went about this commit was I broke the inheritance, ran tests, and added proxies until all tests passed. Like filelog, there are a handful of attributes that don't belong on the interface. And like filelog, we'll tease these out in the future. As part of this, we formalize an interface for manifest storage and add checks that manifestrevlog conforms to the interface. Adding proxies will introduce some overhead due to extra attribute lookups and function calls. On the mozilla-unified repository: $ hg verify before: real 627.220 secs (user 525.870+0.000 sys 18.800+0.000) after: real 628.930 secs (user 532.050+0.000 sys 18.320+0.000) $ hg serve (for a clone) before: user 223.580+0.000 sys 14.270+0.000 after: user 227.720+0.000 sys 13.920+0.000 $ hg clone before: user 506.390+0.000 sys 29.720+0.000 after: user 513.080+0.000 sys 28.280+0.000 There appears to be some overhead here. But it appears to be 1-2%. I think that is an appropriate price to pay for storage abstraction, which will eventually let us have much nicer things. If the overhead is noticed in other operations (whose CPU time isn't likely dwarfed by fulltext resolution) or if we want to cut down on the overhead, we could dynamically build up a type whose methods are effectively aliased to a revlog instance's. I'm inclined to punt on that problem for now. We may have to do it for the changelog. At which point it could be implemented in a generic way and ported to filelog and manifestrevlog easily enough I would think. .. api:: manifest.manifestrevlog no longer inherits from revlog The manifestrevlog class now wraps a revlog instance instead of inheriting from revlog. Various attributes and methods on instances are no longer available. Differential Revision: https://phab.mercurial-scm.org/D4386

pager: do not enable when TERM=dumb Before this, running hg in emacs' M-x shell started `less`, which would complain about the terminal not being fully functional. Differential Revision: https://phab.mercurial-scm.org/D4376

localrepo: do not cache auditor/nofsauditor which would make reference cycle Before, self.auditor and self.nofsauditor held self through self._checknested, and the following code couldn't free a repo by ref-counting. def main(): repo = localrepo.localrepository(uimod.ui(), '../testrepos/hello') main() With this change, the cache of the nofsauditor is limited to a single match session. I think that's okay as the nofsauditor doesn't do any filesystem access. Alternatively, maybe we can remove the callback from nofsauditor since it isn't used unless realfs=True, but I have no idea whether it is a bug or not.

minirst: remove unused function decorateblocks() It was added at ee6988aea74e "minirst: add decorateblocks search helper", but it's been unused since then.

minirst: make format() simply return a formatted text It's a source of bugs to change the type of the return value conditionally.

help: reorder section filtering flow to not format help text twice

help: inline minirst.format() I'm going to make minirst.format() always returns a formatted text, not text | (text, pruned).

minirst: extract function that formats parsed blocks as plain text

minirst: unindent "if True" block in filtersections()

minirst: extract function that filters parsed blocks by section name I'll move some bits from minirst.format() to caller to make the function interface simpler.

bookmarks: adjust exception type so present(bookmark(.)) works as expected

revset: expand bookmark(.) to the active bookmark We do that in several places. I'm not pretty sure if "literal:." should be expanded or not, so it's disabled for now.

pycompat: remove membershiprange which is no longer used Backed out changeset 45e05d39d9ce

transaction: remember original len(repo) instead of tracking added revs (API) It's silly to keep updating xrange(len(changelog), len(changelog) + 1) for each added revision. Instead, let's simply remember the first revision to be added. The test output slightly changed as the branch cache is also warmed up by stream clone, which seems more consistent. .. api:: ``tr.changes['revs']`` is replaced by ``tr.changes['origrepolen']`` which is the first revision number to be added.

obsutil: make sure "addedrevs" is not None in getobsoleted() If it were None, "rev in addedrevs" would crash. So tr.changes["revs"] should be a mandatory parameter here.

phases: enforce internal phase support We should not use the internal phase for repository without the requirement. Otherwise, older clients could have a look at the repository and see the internal changesets. For now, we introduce a low-level Programming error, more UI friendly error will be introduced later.

phases: add a repository requirement about internal phase For internal changeset to be properly hidden, the client version needs to support it. So we introduce a new repository requirement that will make sure clients touching a repository that uses internal phase supports the feature.

phases: add an internal phases The phase is meant to be used for internal commit that are a byproduct of command operation (eg:shelve). This changeset focus on getting the most important feature in, more advanced API is to be introduced in later changesets. The phase approach to handle internal has multiple advantages: * simple to implement, reuse optimized code, * fits well with existing phases. We don't want internal changeset to be exchanged or served. * easy to extend to for lifecycle handling. More thinking about internal changeset at https://www.mercurial-scm.org/wiki/InternalsPlan We add this new phases with a high number to leave some room to possible other phases. We also try out playing with the idea of "flag" for phases, each flag would convey a distinct meaning. We can drop the flag idea in the future (keeping the existing numbers). The flag property should still move in a monotonic direction (enabled -> disabled) or be immutable like the "internal" flag. To simplify the addition of this new phase, we introduce many placeholder phases. They are not meant to be used for now. Keeping `allphases` as a list ensure existing algorithm works fine. The full performance impact of adding that many hollow phases is unclear so far. The impact on phase computation is visible but not worrisome. Runnin `hg perfphase` in my mercurial development repository. Before: ! wall 0.001807 comb 0.000000 user 0.000000 sys 0.000000 (median of 1597) after: ! wall 0.001906 comb 0.000000 user 0.000000 sys 0.000000 (median of 1521)

stringutil: teach pprint() to recognize generators Otherwise they get serialized as e.g. <generator object X at 0x7f543d3d68c0> Differential Revision: https://phab.mercurial-scm.org/D4396