treemanifests: skip extraneous check for item before calling _loadlazy Differential Revision: https://phab.mercurial-scm.org/D4844

treemanifests: make _loadchildrensetlazy just call _loadlazy This was suggested on the original review for _loadchildrensetlazy (D4370), it's taken me a while to get to it though. Differential Revision: https://phab.mercurial-scm.org/D4843

treemanifests: make _loadlazy tolerate item not on _lazydirs I'd like to clean up a few cases where we check for an item in _lazydirs before calling _loadlazy - this will remove an extraneous dict lookup and make it slightly more versatile. Differential Revision: https://phab.mercurial-scm.org/D4842

debugcommands: add a debugindexstats command Someone went through the trouble of recording these stats, so let's make them accessible. Differential Revision: https://phab.mercurial-scm.org/D4832

rust-chg: add main program It can at least connect to a *running* command server, and execute Mercurial command. No signal handling nor daemon management is implemented yet.

rust-chg: add interface to chdir the server

rust-chg: add Client extensions to run cHg-specific requests This just provides a nicer way to issue command-server requests.

rust-chg: port basic socket path handling from cHg of C This is basically modeled after setcmdserveropts() of chg.c.

rust-chg: add state machine to handle "runcommand" request with cHg extension This is modeled after tokio-hglib's RunCommand state to support the "S" channel message.

rust-chg: add callback to handle pager and shell command requests This could be inlined into the ChgRunCommand state to be introduced by the next patch, but it seemed good to separate any user interactions from the IPC code.

rust-chg: add low-level function to set pager fd blocking This is necessary because the server expects stdout/stderr to be blocking, whereas we'll use async library to spawn pager, which makes pipes unblocking.

rust-chg: add future that handles "attachio" request This is the sequence to send client-side stdio and pager stdin to the server.

rust-chg: add parser for request messages sent to "S" channel The data structure is documented at chgserver.py:channeledsystem().

rust-chg: add wrapper around C function

rust-chg: add function to send fds via domain socket As a beginning, I wrote some C. It's extracted from attachio() of contrib/chg/hgclient.c. Maybe it could be rewritten in Rust by using the libc (and/or nix) crates, but doing that wouldn't be trivial as the code depends on CMSG_*() macros. IMO, using C is better here.

rust-chg: update dependencies

rust-chg: add project skeleton This directory will host the reimplementation of cHg in Rust. It will use Tokio [1], and tokio-hglib [2] which I wrote for an oxidized CHg, no idea if there's such carbon bonding in nature btw. [1]: https://tokio.rs/ [2]: https://bitbucket.org/yuja/tokio-hglib/ The reasoning for depending on Tokio is that it will allow us to handle Unix signals in a safer way. Well, I believed that until I found a weird function, handlestopsignal(), in cHg codebase. It resends the same signal to the same process by temporarily masking the handler, which can't be inherently async. So the signal handlers will stay in C, which means there isn't actually much reason to write async code right now, other than I've already done most of the async stuff, and slightly easier pager handling. The reasoning for the rewrite is that it will eventually be possible to port server-side config validation back to the client side, which will reduce the complexity of the current daemon management. It will also encourage us to write frontend library (e.g. command line and config parsers) in Rust. The license is GPL2+ because it's likely to include derived work from the cHg of C. The rust/chg crate is excluded from the root workspace as it's unclear how the whole rust packages should be laid out. That can be revisited later.

narrow: move copies overrides to core The copies overrides seems to have been a little complicated just by not being in core. When moved to core, it becomes trivial (at least I think these overrides have the same effect). Differential Revision: https://phab.mercurial-scm.org/D4825

narrow: pass old includes and excludes to _widen() In future patches we will need to pass them in the widen wireprotocol command which we are building. Differential Revision: https://phab.mercurial-scm.org/D4812

narrow: check for servers' narrow support before doing anything (BC) Recently we introduced narrow capabilities for the server. So we can check whether a server has narrow clone support or not before doing anything. This is BC because new clients won't be able to extend from old narrow-enabled servers. I *think* narrow is not used much (maybe just inside Google), also it's experimental so I think we can change this. We will need to this someday anyway. The "doesn't" in error is changed to "does not" because I think that's we do in core. I also changed one more instance of the error message to use 'does not' for consistency. Differential Revision: https://phab.mercurial-scm.org/D4789

narrow: don't do the dirstate dance if ellipses is not enabled I believe we set dirstate parents to nullid before widening pull because in ellipses cases, the parent might be stripped off with a new changeset. However the second ds.setparents() call invalidate my assumption. I am not sure why we do this. So here is a patch. This patch also adds tests showing we break nothing in non-ellipses cases. Differential Revision: https://phab.mercurial-scm.org/D4788

narrow: pass 'narrow_widen' as source while generating changegroup Extensions inspect the `source` parameter and add some custom logic on the basis of value of that parameter. The parameter is also passed to hooks. So let's pass the value as 'narrow_widen' to differentiate widening from pull. Differential Revision: https://phab.mercurial-scm.org/D4787

narrow: factor out logic to create cg while widening into separate fn This patch takes out the logic which generates a changegroup for widening a narrow clone when ellipses are disabled. This is done because future patches will introduce a narrow_widen() wireprotocol command which will send a bundle2 with changegroup and will use this function. The new function for now returns just the changegroup for compatibility with existing code, but in future patches, when we establish a wireprotocol command and call this function from there, this will return the required bundle2. Differential Revision: https://phab.mercurial-scm.org/D4786

narrow: avoid looking up dirstate again when editing dirstate The narrow extension overrides the dirstate editing functions to restrict paths outside the narrowspec. These overrides access the dirstate from repo.dirstate instead of using the "self" reference passed to the overridden functions. I don't see a reason for this and it caused me problems with a later patch (it caused infinite recursion when I modified localrepo.dirstate()), so let's change it. Differential Revision: https://phab.mercurial-scm.org/D4829

repo: move unfiltered-repo optimization to workingctx localrepo.__getitem__ special-cased lookup of the working copy parent to avoid looking up obsmarkers. I think the reason for that code (which I once wrote myself) was to make `hg commit` not load obsmarkers, which it would otherwise do via ctx.p1() in localrepo.commitctx(). That had the somewhat unfortunate consequence of making lookup of an unrelated binary nodeid load the dirstate. Now that changectx's constructor is dumb, we can let workingctx._parents() have the opmtimization instead. This affects two tests, because they no longer end up loading the dirstate and their "warning: ignoring unknown working parent ..." messages therefore go way. Differential Revision: https://phab.mercurial-scm.org/D4828

context: move logic from changectx.__init__ to localrepo.__getitem__ (API) My motivation for this change was to make repo[node] not load the dirstate (more about that in the next patch), but I think it makes more sense this way too. For example, raising RepoLookupError seems to belong better in the repo lookup function (i.e. localrepo.__getitem__). This makes the changectx constructor very simple -- it just assigns the given repo, revnum, and nodeid to properties. Differential Revision: https://phab.mercurial-scm.org/D4827

context: reduce dependence of changectx constructor I want to change the constructor's signature and letting all creation of changectx instances go through the repo simplifies that. Differential Revision: https://phab.mercurial-scm.org/D4826

py3: use system strings for HTTP response header comparison res.headers is holding system strings. Attempting to compare against bytes always fails and we fail to print decoded CBOR responses on Python 3. After this change, various test-wireproto* tests are now properly printing CBOR response objects. Differential Revision: https://phab.mercurial-scm.org/D4835

py3: convert HTTP request headers to str The low-level request object ideally takes system strings for HTTP request headers and values. If we send in bytes, it works. But a duplicate header check fails and various tests emit duplicate user-agent headers. Differential Revision: https://phab.mercurial-scm.org/D4834

py3: use system strings in HTTP server code Previously the source transformer was converting some string literals to bytes and we were comparing a system native string to bytes and this was leading to sending the wrong HTTP response headers on Python 3. After this change, we now properly send Transfer-Encoding and Connection response headers on Python 3. Differential Revision: https://phab.mercurial-scm.org/D4833

py3: byteify test-storage.py The test now passed on Python 3. # skip-blame just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D4836

narrow: remove narrowpatch instead of narrowcopies 84092edd5c88 removed narrowpatch.py and wanted to drop narrowpatch from __init__.py but mistakenly removed narrowcopies. This will make buildbots green again. Differential Revision: https://phab.mercurial-scm.org/D4837

py3: suppress the output from .write() calls in more tests I missed these in 803b7569c9ea because I forgot about `...` lines.

py3: byteify test-lfs.t

lfs: explicitly name a key when sorting blob pointers This is needed to keep py3 happy. The other two instances of sorting already did this.

py3: byteify test-lock.py There's still a stacktrace about not being able to serialize _io.TextIOWrapper.

py3: byteify contrib/dumprevlog

py3: quote $PYTHON in test-merge-symlinks.t

copies: inline a variable that's used only once By inlining it, we also avoid calculating the value when "if of in seen" is false (probably not significant). Differential Revision: https://phab.mercurial-scm.org/D4831

bundlerepo: remove a variable alias "parentrepo" and "repo" were the same thing and I don't see much reason for it (unionrepo has similar structure and a similar alias but there are two repos there so at least it makes a little more sense there). Differential Revision: https://phab.mercurial-scm.org/D4830

merge with stable

narrow: move support for `hg verify` into core Differential Revision: https://phab.mercurial-scm.org/D4824

narrow: drop unnecessary overrides of patch The patch overrides became unnecessary when I made context.status() filter by the narrowspec in e411774a2e0f (narrow: move status-filtering to core and to ctx, 2018-08-02). Differential Revision: https://phab.mercurial-scm.org/D4823

tests: add test showing that paths outside narrowspec are not added We had a test that shows that trying to explicitly add a file outside the narrowspec results in an error, but we didn't have a test that shows that paths from subdirectories outside the narrowspec are not added (on e.g. `hg add .`) but silently ignored. Differential Revision: https://phab.mercurial-scm.org/D4822

catapipe: fix bug introduced when switching to the timer api Differential Revision: https://phab.mercurial-scm.org/D4818

narrow: move the ellipses server capability to core This will be used in core logic to determining whether a server is ellipses enabled or not. And also this will ease moving narrow related things to core. Differential Revision: https://phab.mercurial-scm.org/D4809

narrow: move the wireprotocol narrow capability name to core We are trying to integrate the whole of narrow logic into core and it will be helpful for upcoming patches to have these capability names in core. The next patch will move the ellipses capability to core also. The exact motivation is to know whether we are cloning a ellipses repo or not and adding an ellipses repo requirement. Differential Revision: https://phab.mercurial-scm.org/D4808

help: document about "files" template keywords

formatter: remove experimental marker from -T option I think it's good enough to stabilize the feature, at least for the core and non-experimental commands.

grep: rename {line_number} to {lineno} as well (BC)

annotate: rename {line_number} to {lineno} (BC) I think {lineno} looks more like a common template keyword. It isn't called a {line} to avoid conflicts with the element name of {lines} and the {_|splitlines} filter. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary

resolve: rename {status} to {mergestatus} to not shadow change status (BC) This is a part of the name unification. I think it's somewhat useful to provide a combined view of change/merge statuses. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary

py3: whitelist one additional test Differential Revision: https://phab.mercurial-scm.org/D4817

logtoprocess: connect all fds to /dev/null to avoid bad interaction with pager We detected that pager is waiting for log-to-process script to finish, which is annoying when adding a script on commandfinish that does an HTTP push. There seems to be no workaround on the script side and it will make the behavior on Linux/MacOS closer to the Windows behavior. The drawback is that it makes the related tests more flaky as log-to-process outputs are now really asynchronous. If it's considered a BC change, another option would be to add a config option for this new behavior. I personally think that the different behavior between Windows and Linux is confusing and that it's a bug I would be fine with a new config option. Differential Revision: https://phab.mercurial-scm.org/D4816

logtoprocess: add a test to show pager and ltp bad interaction The next changeset will fix the bad interaction between the pager and log-to- process. Add a test first to make the next changeset easier to understand. Differential Revision: https://phab.mercurial-scm.org/D4815

py3: add one more passing test to whitelist caught by buildbot Thanks to Matt Harbison who fixed the remaining failures of this test. Differential Revision: https://phab.mercurial-scm.org/D4814

cleanupnodes: pass multiple predecessors to `createmarkers` directly

obsolete: allow multiple predecessors in createmarkers The logic for this change is similar to the change to `cleanupnodes` that we did earlier. Now that the rebase code is trying to record a fold, we need to actually record it in the markers. The first step is to have the markers creation API able to receive such fold data. To keep things sane, we restrict fold to on successors.

obsolete: preindent code in createmarkers This will help to make the next change clearer.

obsolete: explicitly pass relation items to effectflag computation To track folds, we are about to change the content of `rel`. To simplify this change, we update the `geteffectflag` function beforehand.