tests: add test showing that paths outside narrowspec are not added We had a test that shows that trying to explicitly add a file outside the narrowspec results in an error, but we didn't have a test that shows that paths from subdirectories outside the narrowspec are not added (on e.g. `hg add .`) but silently ignored. Differential Revision: https://phab.mercurial-scm.org/D4822

catapipe: fix bug introduced when switching to the timer api Differential Revision: https://phab.mercurial-scm.org/D4818

narrow: move the ellipses server capability to core This will be used in core logic to determining whether a server is ellipses enabled or not. And also this will ease moving narrow related things to core. Differential Revision: https://phab.mercurial-scm.org/D4809

narrow: move the wireprotocol narrow capability name to core We are trying to integrate the whole of narrow logic into core and it will be helpful for upcoming patches to have these capability names in core. The next patch will move the ellipses capability to core also. The exact motivation is to know whether we are cloning a ellipses repo or not and adding an ellipses repo requirement. Differential Revision: https://phab.mercurial-scm.org/D4808

help: document about "files" template keywords

formatter: remove experimental marker from -T option I think it's good enough to stabilize the feature, at least for the core and non-experimental commands.

grep: rename {line_number} to {lineno} as well (BC)

annotate: rename {line_number} to {lineno} (BC) I think {lineno} looks more like a common template keyword. It isn't called a {line} to avoid conflicts with the element name of {lines} and the {_|splitlines} filter. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary

resolve: rename {status} to {mergestatus} to not shadow change status (BC) This is a part of the name unification. I think it's somewhat useful to provide a combined view of change/merge statuses. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary

py3: whitelist one additional test Differential Revision: https://phab.mercurial-scm.org/D4817

logtoprocess: connect all fds to /dev/null to avoid bad interaction with pager We detected that pager is waiting for log-to-process script to finish, which is annoying when adding a script on commandfinish that does an HTTP push. There seems to be no workaround on the script side and it will make the behavior on Linux/MacOS closer to the Windows behavior. The drawback is that it makes the related tests more flaky as log-to-process outputs are now really asynchronous. If it's considered a BC change, another option would be to add a config option for this new behavior. I personally think that the different behavior between Windows and Linux is confusing and that it's a bug I would be fine with a new config option. Differential Revision: https://phab.mercurial-scm.org/D4816

logtoprocess: add a test to show pager and ltp bad interaction The next changeset will fix the bad interaction between the pager and log-to- process. Add a test first to make the next changeset easier to understand. Differential Revision: https://phab.mercurial-scm.org/D4815

py3: add one more passing test to whitelist caught by buildbot Thanks to Matt Harbison who fixed the remaining failures of this test. Differential Revision: https://phab.mercurial-scm.org/D4814

cleanupnodes: pass multiple predecessors to `createmarkers` directly

obsolete: allow multiple predecessors in createmarkers The logic for this change is similar to the change to `cleanupnodes` that we did earlier. Now that the rebase code is trying to record a fold, we need to actually record it in the markers. The first step is to have the markers creation API able to receive such fold data. To keep things sane, we restrict fold to on successors.

obsolete: preindent code in createmarkers This will help to make the next change clearer.

obsolete: explicitly pass relation items to effectflag computation To track folds, we are about to change the content of `rel`. To simplify this change, we update the `geteffectflag` function beforehand.

rebase: explicitly track collapses as fold A collapse is a large fold. It is now explicitly tracked (at the cleanupnodes API level).

rebase: use tuple as `replacement` keys Now that `cleanupnodes` support tuples as key, we update the rebase code to use them. No changes in the replacement tracked are introduced yet.

rebase: expand a long "one-liner" When a one-liner gets 3 lines longs, it lose its expressivity benefits. We expand it into a simple for loop. This makes future changes of the code in that area clearer.

cleanupnodes: drop special casing around prune markers (API) The `cleanupnodes` has logic to skip the creation of "prune" markers if the changeset is already obsolete. This feels strange and gets in the way of code changes to tracks folds. Now that callers no longer request such prune, we can drop this logic. In many cases, pruning through cleanupnodes should be replaced by internal phase usage.

rebase: don't try to prune obsolete changeset already in the destination With similar motivations to the previous changesets, we stop marking changeset from pruning when it is not the command intention. In this case, we still need to distinguish between the strip and the obsolete case.

histedit: don't cleanup nodes already disposed of If something else took care of these temporary nodes, we don't need to do anything about it. This less liberal usage of pruning through cleanup nodes will help us further cleanup on the road to explicitly tracks folds.

py3: avoid b'' output in test-hgweb-non-interactive.t I couldn't figure out how to get the list to print without b'' with pycompat.write(), without converted each element to str.

py3: convert print to a function call in a few tests I *think* this is the last of them.

py3: use util.forcebytestr() to convert push lock error to bytes

py3: byteify windows.shelltocmdexe() This makes test-doctest.py happy on Windows.

run-tests: flush output stream before prompting to accept changes With py3 on Windows, the prompt to accept changes in an --interactive run wasn't showing up until after the choice was read from stdin.

py3: use util.forcebytestr() to convert error messages to bytes This makes the python 3 buildbot green again. Differential Revision: https://phab.mercurial-scm.org/D4811

context: stop catching TypeError when converting hex nodeid to binary It has been a programming error to pass a 40-character string that is not a hex nodeid since 8b86acc7aa64 (context: drop support for looking up context by ambiguous changeid (API), 2018-04-28), so we can just let it raise a TypeError. Differential Revision: https://phab.mercurial-scm.org/D4807

context: stop catching and re-raising FilteredRepoLookupError FilteredRepoLookupError is only raised by changectx's constructor and the higher-level scmutil.revsymbol(), so there's no need to catch it in changectx's constructor. Differential Revision: https://phab.mercurial-scm.org/D4806

util: use a context manager in readlock()

py3: convert os.readlink() path to native strings on Windows Windows insisted that it needs to be str. I skipped the stuff in the posix module, and left `tests/f` and `run-tests.py` alone for now.

py3: suppress the output from .write() calls in the remaining tests

py3: conditionalize json float precision difference in test-debugcommands.t

py3: byteify extension in test-debugcommands.t

pullreport: rev duplicated and extinct into account If we already have some obsolete and hidden nodes locally and the server send them again to you, it seems useful to point it out instead of being silent about it.

pullreport: issue a message about "extinct" pulled changesets Changeset pulled from a remote repository while already obsolete locally can end up hidden after the pull. Hiding obsolete changesets is a good behavior but silently "skipping" some of the pulled content can get confusing. We now detect this situation and emit a message about it. The message is simple and the wording could be improved, however, we focus on the detection here. Evolution is still an experimental feature, so the output is open to changes. In particular, we could point out at the latest successors of the obsolete changesets, however, it can get tricky is there are many of them. So we delay these improvements to another adventure. Another easy improvement would be to merge this message with the previous line about the new nodes and their phases. This is a good example of cases where we can only transmit a limited amount of data to users by default. We need some sort of "transaction journal" we could point the user to.

pullreport: skip or rework some early return We are about to add more logic in this report. Before that, we need it to not quit so early.

pullreport: skip filtered revs instead of obsolete ones Obsolescence is closely related to visibility but still a distinct concept. We can receive changesets that are obsolete but visible (eg: when pulling orphans). Such changeset should be reported too. In addition, the filtering level can be anything, we should respect it.

pullreport: add a test to show misreporting of visible changeset The current code ignores all obsolete changesets including the visible one. We add a test showing this behavior before fixing the behavior.

repo: don't look up context for tip node if it's not needed We were doing repo['tip'].node() or similar in a few places where repo.changelog.tip() would be enough. Differential Revision: https://phab.mercurial-scm.org/D4781

repo: look up nullrev context by revnum, not symbolic name I think lookup of the 'null' symbol should be done via scmutil.revsymbol() and repo['null'] is only supported for historical reasons. However, repo[nullrev] is fine, so we can switch to that instead of switching to scmutil.revsymbol('null'). Differential Revision: https://phab.mercurial-scm.org/D4780

bundle: consistently put revnums in "base" collection The "base" collection contains revnums, except that it can also contain the "null" symbol. That's a little weird. Let's be consistent. Differential Revision: https://phab.mercurial-scm.org/D4779

py3: ensure printing to stdout uses str in test-hgweb-no-request-uri.t

scmutil: accept multiple predecessors in 'replacements' (API) This changeset makes 'cleanupnodes' accepts multiple predecessors as `replacements` keys. The same as it accepts multiple successors as `replacements` values. To avoid breaking all callers, the old and new ways are currently valid at the same time. We'll deprecate and drop the old way later. This change is the first step toward a better tracking of "fold" event in the evolution history. While working on the "rewind" command (in the evolve extension), we realized that first class tracking of folds are necessary. We already have good tracking of splits. When walking the evolution history from predecessors to successors, that makes for a clear distinction between having multiple successors because of the actual splitting of a changeset or content-divergences. The "rewind" command allows restoring older evolution of a stack of changesets. One of its mode walks the evolution history to automatically find appropriate predecessors. This means walking from successors to predecessors. In this case, we need to be able to make the same distinction between an actual fold and other cases. So we will have to track folds explicitly. This changesets only focus on making it possible to express fold at the `cleanupnodes` API level. The actual tracking will be implemented later.

scmutil: expand long "one-liner" When a one-liner gets 3 lines longs, it loses its expressivity benefits. We expand it into a simple for loop. This makes future changes of the code in that area clearer.

shelve: find shelvedctx from bundle even if they are already in the repo We use the new "duplicates" node tracking to find the tip of the bundle even if it already exists in the repository. Such logic is not supposed to be needed in theory. If the shelve was made using internal-phase, we already know its node. Otherwise, the bundle content should have been stripped. However, handling it makes the shelve code more robust and provide a good example of "revduplicates" usage.

shelve: return the shelved node as part of bundle application It make sense to have the function in charge of unbundling the shelved revision also return the node of that revision (when the data is in the bundle). This will help us to handle unnatural state where the unshelved change already exists in the repository.

changelog: keep track of duplicated node in the transaction adding them The transaction is already tracking the new nodes. We now tracks the "duplicates" in the same location.

revlog: add a callback "tracking" duplicate node addition If a changegroup contains node already added to the repository, they will be skipped. Skipping them is the right behavior (we don't need to store things twice), but it can hide some information to the code doing the unbundle (eg: shelve looking for the tip of the bundle). The first step to improve this situation is to add a low level callback. We do not need this tracking on all revlog, so actual tracking will be added in the next changeset.

logtoprocess: define $HG for children processes So they can compute the hg version for instance. Differential Revision: https://phab.mercurial-scm.org/D4768

py3: mask out None type when printing in `debuglocks` Apparently, %b doesn't allow None.

py3: ensure standard exceptions use `str` type strings in windows.py See also edaa40dc5fe5.

py3: replace a StandardError reference This doesn't exist on py3, and the standard way of handling this seems to be to catch both exceptions.

storageutil: extract revision number iteration This code is a bit quirky (and possibly buggy). It will likely be used by multiple storage backends. Let's extract it so it is reusable. Differential Revision: https://phab.mercurial-scm.org/D4757

storageutil: new function for extracting metadata-less content from text Other storage backends will want to do this. I'm not concerned about Python function call overhead because I expect self.revision() to dwarf the function call overhead time, since self.revision() requires multiple function calls and may involve decompression in the common case. Differential Revision: https://phab.mercurial-scm.org/D4756

storageutil: move _censoredtext() from revlog This seems like generic functionality we'll want to use from non-revlog storage backends. Differential Revision: https://phab.mercurial-scm.org/D4755

storageutil: move metadata parsing and packing from revlog (API) Parsing and writing of revision text metadata is likely identical across storage backends. Let's move the code out of revlog so we don't need to import the revlog module in order to use it. Differential Revision: https://phab.mercurial-scm.org/D4754

storageutil: new module for storage primitives (API) There will exist common code between storage backends. It would be nice to have a central place to put that code. This commit attempts to create that place by creating the "storageutil" module. The first thing we move is revlog.hash(), which is the function for computing the SHA-1 hash of revision fulltext and parents. Differential Revision: https://phab.mercurial-scm.org/D4753