revset: remove explicit sort() from unstable() The instability should be dealt with by baseset.__init__(), and the returned smartset is ordered by 'subset &' anyway.

revset: implement a simple 'foo#generations' expression The result of this expression is equivalent to 'foo#g[:]': all reachable ancestors and descendants of a revset foo. While not very useful functionality on its own, its implementation can be tested and revset.relations dict can be easily used by extensions to add other non-subscript relations. Differential Revision: https://phab.mercurial-scm.org/D8394

revset: rename generationsrel() to generationssubrel() This highlights that this is an implementation of 'x#y[z]' relation subscript operator as opposed to 'x#y' relation operator. Differential Revision: https://phab.mercurial-scm.org/D8393

manifest: leave a TODO where we may have more work for sha1 portability Differential Revision: https://phab.mercurial-scm.org/D8375

manifest: add test for long hashes Differential Revision: https://phab.mercurial-scm.org/D8373

manifest: remove a final 40-byte assumption from pure-python parser Differential Revision: https://phab.mercurial-scm.org/D8372

manifest: fix yet another 20-byte-hash assumption Differential Revision: https://phab.mercurial-scm.org/D8371

manifest: fix another pure-parsing 20-byte assumption Differential Revision: https://phab.mercurial-scm.org/D8370

manifest: teach treemanifest about long hashes Differential Revision: https://phab.mercurial-scm.org/D8369

manifest: start removing 40-byte hash restrictions from C code Differential Revision: https://phab.mercurial-scm.org/D8368

manifest: remove a 20-byte-hash assumption from pure manifest parsing This is almost certainly slower, but it shouldn't be by much. We need to do this for the eventual hash transition, no matter what that looks like. Differential Revision: https://phab.mercurial-scm.org/D8367

revset: mark `extinct()` experimental Differential Revision: https://phab.mercurial-scm.org/D8407

revset: mark `successors()` experimental Differential Revision: https://phab.mercurial-scm.org/D8406

revset: mark `obsolete()` experimental I assume not marking it was unintentional, since the obsolete concept it still experimental. Differential Revision: https://phab.mercurial-scm.org/D8405

nodemap: access the mmap opener option using bytes The key is bytes and need to be accessed s such. Differential Revision: https://phab.mercurial-scm.org/D8413

nodemap: rename a variable to prevent shawoding "_" This will make pyflakes happy with the next changeset. Differential Revision: https://phab.mercurial-scm.org/D8412

rust-chg: indent process_message() to prepare mass rewrite to futures-0.3 I'll start upgrading the codebase to modern async/await-based implementation, which cannot be done incrementally. This is the last non-breaking patch to prepare for the rewrite. Differential Revision: https://phab.mercurial-scm.org/D8403

rust-chg: move get_umask() call out of run() function run() will be an async function, but get_umask() isn't thread safe. Differential Revision: https://phab.mercurial-scm.org/D8402

rust-chg: leverage impl trait at argument position Differential Revision: https://phab.mercurial-scm.org/D8401

setup: link osutil.so to libsocket on Solaris/illumos (issue6299)

revset: import `unstable()` from the evolve extension Differential Revision: https://phab.mercurial-scm.org/D8404

fastexport: make a diagnostics message more localizable Differential Revision: https://phab.mercurial-scm.org/D8398

rust-chg: use "crate::" to import local modules I feel it's easier to follow to resolve modules from the crate root than relative path from self module. Differential Revision: https://phab.mercurial-scm.org/D8400

rust-chg: upgrade to 2018 edition and remove useless extern crates Differential Revision: https://phab.mercurial-scm.org/D8399

mergestate: store about files resolved in favour of other Committing a merge sometimes wrongly creates a new filenode where it can re-use an existing one. This happens because the commit code does it's own calculation and does not know what happened on merge. This starts storing information in mergestate about files which were automatically merged and the other/remote version of file was used. We need this information at commit to pick the filenode parent for the new commit. This issue was found by Pierre-Yves David and idea to store the relevant parts in mergestate is also suggested by him. Somethings which can be further investigated are: 1) refactoring of commit logic more to depend on this information 2) maybe a more generic solution? Differential Revision: https://phab.mercurial-scm.org/D8392

histedit: add test that an invalid editor doesn't bork client state We had a report of a situation like this borking a user at Google, but I can't reproduce it in a test. Let's at least backstop the issue with a test, so we don't accidentally introduce such a bug... Differential Revision: https://phab.mercurial-scm.org/D8391

rust-chg: silence warning about dated coding style Differential Revision: https://phab.mercurial-scm.org/D8384

rust-chg: update name of the server process This is a copy of updateprocname() of hgclient.c. At this point, rust-chg is basically functional. I did dogfooding for a couple of weeks in 2018. There are a few remaining tasks: a. loop detection by CHGINTERNALMARK b. forward unsupported commands (notably serve -d) to real hg c. better handling of early server exception d. modernize codebase (2018 edition, impl trait, async/await) For (d), we'll probably want to switch to async-std, but I'm thinking of upgrading to Tokio 0.2 as an intermediate step since process API isn't ported to async-std yet. I'm pretty sure future migration to async-std will be painless compared to the mass rewrite from futures-0.1 to 0.3. https://github.com/async-rs/async-std/issues/22 Differential Revision: https://phab.mercurial-scm.org/D8383

rust-chg: send client side umask to server This is equivalent to forwardumask() of hgclient.c. Differential Revision: https://phab.mercurial-scm.org/D8382

rust-chg: add config validation and process returned instructions This is the reimplementation of runinstructions() and main() in chg.c. In Rust version, we only pass in early arguments to the server as the locator doesn't know the full arguments. This should be fine since these arguments are just passed in to _earlyparseopts() and _parseconfig(), which means the server doesn't need full arguments. Another difference is the handling of the "exit <code>" instruction. In Rust version, we can simply reuse the connection instead of "exit(code)" as the command error isn't displayed yet. That's because the client-side stdio is not attached until the connection is validated. This behavior is cleaner than C, but it also means that the early server exception wouldn't be propagated to client because stderr isn't attached. So we might have to reconsider when to attach/detach the server stdio. Differential Revision: https://phab.mercurial-scm.org/D8381

rust-chg: collect server flags from command arguments This is the reimplementation of testsensitiveflag() and setcmdserverargs() of chg.c. Differential Revision: https://phab.mercurial-scm.org/D8380

rust-chg: add interface to run "validate" request Differential Revision: https://phab.mercurial-scm.org/D8379

rust-chg: add helper to parse instructions sent from server This is well structured version of runinstructions() of chg.c. Differential Revision: https://phab.mercurial-scm.org/D8378

tests: move verification closer to setup in test-copies-chain-merge.t I found this test case really hard to read because it requires scrolling back and forth between the setup and the verification. This patch moves them closer together. There was also duplicated descriptions for each test case (duplicated between setup and verification). They had already become out of sync. I picked the description from the verification, partly so the differences would be highlighted in this patch. Differential Revision: https://phab.mercurial-scm.org/D8377

tests: collect all branch creation in one place in test-copies-chain-merge.t I found this test case really hard to read because it requires scrolling back and forth between the setup and the verification. The next patch will move the verification close to the merge commits they test. This patch prepares for that by moving all branch creation first so they are separate from the merge commits (because many merge commits reuse the same branches). Differential Revision: https://phab.mercurial-scm.org/D8376

hgk: remove a "b" used on a kwargs expansion, the keys are strs Differential Revision: https://phab.mercurial-scm.org/D8385

rust-chg: send client-side environment variables to server This is also needed to run config validation. The "validate" request and its response handling will be implemented in the next batch. Differential Revision: https://phab.mercurial-scm.org/D8364

rust-chg: add helper to pack environment variables On my machine, "printenv | wc -c" says 3422. That's the only reason why the initial buffer capacity is set to 4kB. Differential Revision: https://phab.mercurial-scm.org/D8363

rust-chg: move set_current_dir() to Locator This is necessary to run config validation in proper environment. Differential Revision: https://phab.mercurial-scm.org/D8362

rust-chg: abort if server doesn't have required capabilities It's checked at Locator::connect() since we'll have to do more stuff in this function. Differential Revision: https://phab.mercurial-scm.org/D8361

rust-chg: spawn server process if not running This is the minimal reimplementation of gethgcmd(), execcmdserver(), retryconnectcmdserver(), and connectcmdserver() in chg.c. No config validation is implemented yet. And some Py3 workarounds would be missing as this is the code I wrote in 2018. Differential Revision: https://phab.mercurial-scm.org/D8360

rust-chg: update dependencies Differential Revision: https://phab.mercurial-scm.org/D8359

rust-chg: depend on released version of tokio-process Still it is futures-0.1 based. Differential Revision: https://phab.mercurial-scm.org/D8358

rust-chg: add brief comment about initial capacity of temp_sock_path() I don't know if it can be expressed as a compile-time constant, so it's a comment for now. About this series: This is quite old patches for rust-chg. I heard from Octobus people that there's a plan to do an experiment on merging hgcli + chg + some Rust?, so I decided to respin the rust-chg series. Maybe we'll rewrite the core to leverage the recent async/await functionality, but I want to first make my old patches in so the rust-chg can be a drop-in replacement for the chg of C. Compiler warnings will be removed later, and the codebase will be upgraded to the 2018 edition later. Differential Revision: https://phab.mercurial-scm.org/D8357

discovery: avoid wrongly saying there are nothing to pull We can get in a situation where a revision passed through `hg pull --rev REV` are available on the server, but not a descendant of the advertised server heads. For example the server could lying be during heads advertisement, to hide some pull request. Or obsolete/hidden content could be explicitly pulled. So in this case the lookup associated to `REV` returned successfully, but the normal discovery will find all advertised heads already known locally. This flip a special boolean `anyinc` that will prevent any fetch attempt, preventing `REV` to be pulled over. We add three line of code to detect this case and make sure a pull actually happens. My main target is to make some third party extensions happy (I expect the associated test to move upstream with the extension). However this fix already make some of the `infinitepush` test happier.

rebase: don't create merge when continuing rebase interrupted by old hg This fixes the bug described and demonstrated in the previous commit. It does so by practically undoing 8082a77cc3a2 (rebase: remove some redundant setting of dirstate parents, 2020-01-10). Differential Revision: https://phab.mercurial-scm.org/D8356

tests: demonstrate how continuing rebase after upgrade can result in merge If the user starts a rebase with an hg version before 9c9cfecd4600 (rebase: don't use rebased node as dirstate p2 (BC), 2020-01-10) and then runs into conflicts, they will be dropped out to the shell with the rebased node set as the dirstate's second parent. If they then upgrade to a later hg version, it will respect the dirstate's parents and will create a merge commit even if the user was rebasing a non-merge commit. Differential Revision: https://phab.mercurial-scm.org/D8355

manifest: also declare treemanifest as implementing imanifestdict It always has been intended to be, but it had a mismatched flags() method until the previous change. Differential Revision: https://phab.mercurial-scm.org/D8366

manifest: introduce new exception to signal unavailability of fastdelta() I've spent some time reflecting on this, and I think this is the best we can do in this API contract for now. This opens the door to adding treemanifest's implementation to the list of implementers of imanifestdict. Differential Revision: https://phab.mercurial-scm.org/D8365

copies-tests: remove spurious `]` in the template Fixing this typo happily impact all the tests output. Differential Revision: https://phab.mercurial-scm.org/D8258

merge with stable

debugcommands: fix typo in debuguigetpass

dagop: fix subsetparentswalker to set p1/p2 chains at merge revision The previous implementation was wrong because the '1'/'2' key would be appended at a fork revision. Since we traverse the graph from heads, a merge revision is actually a branching point, where the sort key must be generated.

dagop: simplify dict/set reuse condition in subsetparentswalker Prepares for fixing the calculation of p1/p2 sort keys. With this change, there will be one more copying on merge&fork case. I think the copying cost is negligible since we'll have to update each item in the dict on merge/fork.

extensions: refactor function for obtaining disabled extension help The way this worked before was hgext.__index__ was consulted. This file appears to only be present on some Windows distributions. This file contains a dict mapping extension name to its summary line, not its full docstring. The problem with this is that code in the help system was calling this function to resolve help text. If hgext.__index__ was present, only the summary line would be displayed. If not, the full extension help would be printed. This commit changes the function to not use hgext.__index__ such that it always returns the full extension help text. As a result of this change, test-extension.t and test-qrecord.t now pass when run from environments that have an hgext.__index__. Differential Revision: https://phab.mercurial-scm.org/D8344

tests: perform grep manually in test-doctest.py This test has been failing on Windows since 0af56d3ee24c introduced the `hg files` invocation. Specifically, Windows seems to be choking on special characters in the fileset pattern. I believe at least \n and > were causing issues. I attempted various incantations to make the Windows command line parser accept the fileset but couldn't get anything working. I declared bankruptcy and just reimplemented the grepping code in Python. After this change, the test now passes on Windows again. Differential Revision: https://phab.mercurial-scm.org/D8343

tests: prevent printing \r to stdout Like we've done in other recent commits, we need to change sys.stdout on Python 3 to not use os.linesep so output is consistent on Python 3 on Windows. With this change, test-notify.t now passes on Python 3 on Windows! Differential Revision: https://phab.mercurial-scm.org/D8342

tests: force \n newlines when writing to sys.stdout Without this, Python 3 on Windows inserts some \r that aren't present in the input, causing test-http-bad-server.t to fail. After this change, the test passes on Python 3 on Windows! Differential Revision: https://phab.mercurial-scm.org/D8341

dispatch: force \n for newlines on sys.std* streams (BC) The sys.std* streams behave differently on Python 3. On Python 3, these streams are an io.TextIOWrapper that wraps a binary buffer stored on a .buffer attribute. These TextIOWrapper instances normalize \n to os.linesep by default. On Windows, this means that \n is normalized to \r\n. So functions like print() which have an implicit end='\n' will actually emit \r\n for line endings. While most parts of Mercurial go through the ui.write() layer to print output, some code - notably in extensions and hooks - can use print(). If this code was using print() or otherwise writing to sys.std* on Windows, Mercurial would emit \r\n. In reality, pretty much everything on Windows reacts to \n just fine. Mercurial itself doesn't emit \r\n when going through the ui layer. Changing the sys.std* streams to not normalize line endings sounds like a scary change. But I think it is safe. It also makes Mercurial on Python 3 behave similarly to Python 2, which did not perform \r\n normalization in print() by default. .. bc:: sys.{stdout, stderr, stdin} now use \n line endings on Python 3 Differential Revision: https://phab.mercurial-scm.org/D8339

hook: move stdio redirection to context manager The old code was checking stdio redirection in a loop. This didn't make sense. The pattern is better expressed as a context manager IMO, so this commit refactors it to be one. Differential Revision: https://phab.mercurial-scm.org/D8338

pycompat: change argv conversion semantics Use of os.fsencode() to convert Python's sys.argv back to bytes was not correct because it isn't the logically inverse operation from what CPython was doing under the hood. This commit changes the logic for doing the str -> bytes conversion. This required a separate implementation for POSIX and Windows. The Windows behavior is arguably not ideal. The previous behavior on Windows was leading to failing tests, such as test-http-branchmap.t, which defines a utf-8 branch name via a command argument. Previously, Mercurial's argument parser looked to be receiving wchar_t bytes in some cases. After this commit, behavior on Windows is compatible with Python 2, where CPython did not implement `int wmain()` and Windows was performing a Unicode to ANSI conversion on the wchar_t native command line. Arguably better behavior on Windows would be for Mercurial to preserve the original Unicode sequence coming from Python and to wrap this in a bytes-like type so we can round trip safely. But, this would be new, backwards incompatible behavior. My goal for this commit was to converge Mercurial behavior on Python 3 on Windows to fix busted tests. And I believe I was successful, as this commit fixes 9 tests on my Windows machine and 14 tests in the AWS CI environment! Differential Revision: https://phab.mercurial-scm.org/D8337

Added signature for changeset 8fca7e8449a8

Added tag 5.3.2 for changeset 8fca7e8449a8

phabricator: extract logic to print the status when posting a commit This will make it easier to list each commit when folding. That makes the output less confusing because it matches the output of `--confirm` and the revisions listed on the command line. Differential Revision: https://phab.mercurial-scm.org/D8313

phabricator: extract the logic to amend diff properties to a function This will be needed on a separate code path when dealing with folding revisions. And since we know that will involve adding multiple local commmits to the diff properties instead of just one, restructure the logic slightly to allow it. Differential Revision: https://phab.mercurial-scm.org/D8312

phabricator: teach `getoldnodedrevmap()` to handle folded reviews The tricky part here is reasoning through all of the possible predecessor scenarios. In the typical case of submitting a folded range and then resubmitting it (also folded), filtering the list of commits for the diff stored on Phabricator through the local predecessor list for each single node will result in the typical 1:1 mapping to the old node. There are edge cases like using `hg fold` within the range prior to resubmitting, that will result in mapping to multiple old nodes. In that case, the first direct predecessor is needed for the base of the diff, and the last direct predecessor is needed for the head of the diff in order to make sure that the entire range is included in the diff content. And none of this matters for commits in the middle of the range, as they are never used. Fortunately the only crucial thing here is the `drev` number for each node. For these complicated cases where there are multiple old nodes, simply ignore them all. This will cause `createdifferentialrevision()` to generate a new diff (within the same Differential), and avoids complicating the code. Differential Revision: https://phab.mercurial-scm.org/D8311

phabricator: teach createdifferentialrevision() to allow a folded commit range No visible changes here, until an option to enable it is added to `phabsend`. Differential Revision: https://phab.mercurial-scm.org/D8310

phabricator: combine commit messages into the review when folding commits No visible changes here, until an option to enable it is added to `phabsend`. This combines the Differential fields like Arcanist does, rather than simply concatenating the text blocks. Aside from populating everything properly in the web interface, Phabricator fails the review create/update if repeated fields are seen as would happen with simple concatenation. On the flip side, now that the Summary and Test Plan fields can contain data from multiple commits, we can't just join these fields together to determine if an amend is needed. If that were to happen, every single commit in the folded range would get amended with the combined commit message, which seems clearly wrong. Aside from making a minor assumption about the content of the Differential Revision field (it seems they allow some minor variances with spacing), this means that for folded reviews, you can't post it, go to the web page add a missing Test Plan, and then get it added to the commit message by re-posting it. I don't think that's a big deal. Differential Revision: https://phab.mercurial-scm.org/D8309

phabricator: record all local commits used to create a Differential revision Arcanist records all of the commits that it squashes into a single review, and that info will be helpful when adding similar functionality. This info is used when submitting an updated review, so that the extension can recalculate the old diff and see if a new one is necessary, or if it is just a property update. It also shows on the `commits` tab in the `Revision Contents` section. When submitting in the usual 1:1 commit to review mode, the wire protocol is unchanged. The content of `hg:meta` is a bit odd, but such is the problem when folding several commits. The choice for the parent node is obvious, but the `node` value uses the tip commit because that seems more natural, and is used elsewhere to look up the previous diff when updating. The rest of the attributes follow from there. Differential Revision: https://phab.mercurial-scm.org/D8308

tests: use `f --hexdump` to print file content The inline print.py in this test wasn't fully compatible with Python 3 because it was reading from sys.stdin, which already normalized line endings since it operates in the realm of str on Python 3. To do this correctly, we'd need to read from sys.stdin.buffer on Python 3. This would entail conditional code. I felt this was too much effort. So I just replaced the custom script with `f`, which already knows how to do the right thing. test-mactext.t now passes on Python 3 on Windows. Differential Revision: https://phab.mercurial-scm.org/D8336

url: pass str to pathname2url This is needed to appease Python 3. This fixes test-extdata.t and test-url-download.t on Python 3 on Windows. Differential Revision: https://phab.mercurial-scm.org/D8335

tests: pass str to matchoutput() It accepts a str, not bytes. This fixes a failure in test-hghave.t on Windows. Why it wasn't failing on Linux, I don't know. I suspect the Windows process code in Python doesn't accept bytes and the POSIX code does? Differential Revision: https://phab.mercurial-scm.org/D8334

hgcli: customize for Mercurial Now that we have a shiny new PyOxidizer-based hgcli project, let's customize it for Mercurial! This commit replaces the auto-generated pyoxidizer.bzl with one that installs Mercurial from the local source repository. A README.md with build instructions has been added. The Cargo.toml file has been updated to reflect the proper license and reference the added README.md. In my Linux environment, running the test suite yields 27 failures. It's worth noting the run time of the test harness on Linux on my Ryzen 3950X: before: 378s wall; 9982s user; 1195s sys after: 353s wall; 8996s user; 958s sys % orig: 93.4 wall; 90.1 user; 80.2 sys While I haven't measured explicitly, I suspect the performance win is due to in-memory resource loading (which is known to be faster than Python's filesystem importer). Differential Revision: https://phab.mercurial-scm.org/D8351

hgcli: add stub PyOxidizer project Using commit c772a1379c3026314eda1c8ea244b86c0658951d of PyOxidizer, I ran `pyoxidizer init-rust-project hgcli` to create a stub Rust project. The only modifications I made from what that command produced are: * Update location of pyembed crate to PyOxidizer's Git repository. * Removed some trailing whitespace from pyoxidizer.bzl * Added auto-generated Cargo.lock file Subsequent commits will modify the stub project to Mercurial's needs. Differential Revision: https://phab.mercurial-scm.org/D8350

hgcli: remove legacy project This code is a logical precursor to PyOxidizer. It is now defunct. Differential Revision: https://phab.mercurial-scm.org/D8349

nodemap: automatically "vacuum" the persistent nodemap when too sparse We arbitrarily pick "10%" as the threshold. Differential Revision: https://phab.mercurial-scm.org/D8193

nodemap: display percentage of unused in `hg debugnodemap` This is useful to assess the density of the cache. Differential Revision: https://phab.mercurial-scm.org/D8192

nodemap: make sure on disk change get rolled back with the transaction In case of errors, we need to rollback the change made to the persistent nodemap. Differential Revision: https://phab.mercurial-scm.org/D8191

nodemap: test that concurrent process don't see the pending transaction We don't want other client to read uncommitted data, until the transaction is really committed. Differential Revision: https://phab.mercurial-scm.org/D8190

testlib: adjust wait-on-file timeout according to the global test timeout Lets assume that if test timeout have been set to be twice as long, it means local timeout should be twice as long too. I am not aware of any case were extending timeout for file based synchronisation was necessary, but the safety seems simple to implements. Differential Revision: https://phab.mercurial-scm.org/D8316

testlib: add a small scrip to help process to synchronise using file Creating and waiting for files is a robust way to synchronise two processes running concurrently. We already use this approach in various tests. I am adding a official script to do so before adding more usage of this. Differential Revision: https://phab.mercurial-scm.org/D8189

setup: work around old versions of distutils breaking setup.py I'm not really sure how to trigger this, but we saw it in our build environment for Windows at Google. This fixed it. Sigh.

chgserver: update the umask cache before each run posix.py uses a global variable to store the umask value resulting in caching of it when using chg. We need to update it before each command run as the umask can change between commands. This fixes test-inherit-mode.t with chg.

tests: handle new error string from FreeBSD for dns entry not resolving Differential Revision: https://phab.mercurial-scm.org/D8333

notify: optional mail threading based on obsmarker When notify.reply is set and a changeset has a predecessor in the repository, include In-Reply-To pointing to the message-id that would have been generated for the oldest predecessor. This allows mail threading like Phabricator for common cases like rebasing changes, but will be optimal for cases like folding. Differential Revision: https://phab.mercurial-scm.org/D8172

pathutil: document that dirs map type implies manifest/dirstate processing

git: pass a list to pathutil.dirs to indicate that it is a manifest The python implementation of pathutil.dirs just uses a for loop which happens to work the same on both dicts and lists. The rust implementation actually figures out which of the two types it is, and directs the execution to either dirstate or manifest processing.

git: implement basic bookmark activation This is very limited, but it allows 'hg update foo' when already on foo. The caching is based on bmstore's caching.

git: implement a basic checkconflict bookmark store method It is heavily based on bmstore's own checkconflict.

git: abort when attempting to set a branch Given the mapping we use (namely, a git head is a bookmark), it is better to error out with a hint.

git: remove obsolete todo item The changes in 02c47b74366c cleaned up the requirement check.

tests: don't run test-update-atomic.t on chg I am unable to find a good way to make `showwrites` extension in it to work with chg. Also putting the use of showwrites inside `if no-chg` will defeat the purpose of test, so I just made the test no-op on chg.

tests: don't run couple of tests related to extensions loading with chg The couple of files test extensions loading and debugging that. It sets `devel.debug.extensions` to True to do that. Either we have to restart chg if this config is set or just don't run these tests on chg. I tried the first way by adding the config option to chgserver.py list of config subsections but that does not seem to work.

tests: conditionalize test-phases.t output for chg IIUC chg loads the ProgrammingError class which leads to it directly referencing it in error output. This makes the test pass on chg.

chg: be stricter about checking invocation of `serve` command Few tests run serve command in form of `hg -R <repo> serve` which leads to chg thinking that it's not a serve command and failing tests. We become more stricter in checking for the serve command. This fixes test-server-view.t, test-remote-hidden.t, test-remotefilelog-http.t, test-phases-exchange.t, test-wireproto-content-redirects.t with chg.

chgserver: add merge-tools to sensitive config items Because this can change whether the ui is gui or not. This fixes test-extdiff.t with chg.

run-tests: add --chg-debug flag to show chg debug output This has helped me a lot in debugging chg failures in tests.

tests: update test-ssh.t output with --chg The output change was caused by d7304434390f5efca405744fa12a6585edae3d83.

tests: update test-devel-warnings.t output with chg The output change was caused by dc9901558e3c6a78bad3f6594b3888f95104c443.

chgserver: add fastannotate config section to sensitive list Depending on the config value of `fastannotate.modes`, the fastannotate extension can do different things in uisetup. Depending on value of `fastannotate.server`, it can register new wireprotocol capabilities. This fixes test-fastannotate-hg.t, test-fastannotate-protocol.t and test-fastannotate.t with chg.

histedit: add missing b prefix to a string If i18n is disabled (such as via HGPLAIN=1), `_()` doesn't convert from str to bytes, so this raises a TypeError on py3. Differential Revision: https://phab.mercurial-scm.org/D8354

phabricator: account for `basectx != ctx` when calculating renames No functional changes here because the two are the currently same, but they won't be with a `--fold` option. Differential Revision: https://phab.mercurial-scm.org/D8307

phabricator: add basectx arguments to file related `phabsend` utilities This is in support of a future `--fold` option, that allows rolling up several commits into a single review with a diff from the start to the end of the range. There are no functional changes yet- the original `ctx` is also passed as the new `basectx`, which represents the first commit in the review range (similar to `qbase` in MQ parlance). Other functions will need the range of commits, but these deal with status or the diffs, so they only need the end points. Differential Revision: https://phab.mercurial-scm.org/D8306

phabricator: eliminate a couple of duplicate filectx lookups Differential Revision: https://phab.mercurial-scm.org/D8305

tests: update test-debugcommands.t stack trace checks for python3.9 Python 3.9 prints a complete filepath in more cases, so we need to handle that in our test output. Since we don't really care *that* much about the specific path formatting here, just use globbing. Differential Revision: https://phab.mercurial-scm.org/D8329

tests: update test-archive.t expectations for python3.9 changes Sigh. Differential Revision: https://phab.mercurial-scm.org/D8328

setup: relax -Werror for declaration-after-statement on Python 3.9 It turns out Python 3.9 introduces such declarations in the headers, eg cpython/abstract.h:189:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement] so we have to be more relaxed when compiling for 3.9. Differential Revision: https://phab.mercurial-scm.org/D8327

py3: require values in changelog extras to be bytes I don't know what happened here because b436059c1cca (py3: use pycompat.bytestr() on extra values because it can be int, 2019-02-05) came about b44a47214122 (py3: use string for "close" value in commit extras, 2018-02-11). Whatever happened, we shouldn't need to convert the values to bytes now. It's better to not convert because that might cover up bugs where someone sets a unicode value in the extras and that works until the unicode value happens to contain non-ascii (at which point it will fail because `bytestr()` expects its argument to be ascii if it's unicode). Differential Revision: https://phab.mercurial-scm.org/D8332

py3: make setup.py's hgcommand() consistently return bytes Before this patch, it returned unicode when the command failed. That made e.g. `make local PYTHON=python3` fail on an obsolete commit. Differential Revision: https://phab.mercurial-scm.org/D8331

pvec: drop an unused `from __future__ import division` This module only uses the `//` operator (for integer division). Differential Revision: https://phab.mercurial-scm.org/D8326

py3: use integer division in histedit Histedit uses the `/` operator, which does type conversion to float in Python 3 instead of integer division likeon Python 2. Let's preserve the Python 2 behavior by importing and using the `//` operator. Differential Revision: https://phab.mercurial-scm.org/D8324

rust: update README to reflect use of `regex` crate Differential Revision: https://phab.mercurial-scm.org/D8325

rust: update all dependencies We do this periodically to say up to date. No major versions were crossed this time per se, but the `rand` is still in v0, and their 0.7x series broke three things: - Some distribution-related elements were moved to a separate crate, flashing a deprecation warning - The `LogNormal::new` associated function now returns a `Result` - Certain RNGs were updated to sample a `u32` instead of `usize` when their upper-bound is less than `u32::MAX` for better portability, which changed the output for 2 tests. Moreover, the recent use of the `regex` crate for ignore mechanisms prompted some benchmarking that revealed that `regex` was slower at compiling big regex than `Re2`. The author of `regex` was very quick to discover an optimization that yielded a 30% improvement. It's still slower than `Re2` in that regard, but less so in the 1.3.6 release. Differential Revision: https://phab.mercurial-scm.org/D8320

rust: update micro-timer dependency The new version uses a much more robust technique and should remove any existing risk of bad compiler error or performance hit. Differential Revision: https://phab.mercurial-scm.org/D8319

rust-status: only involve ignore mechanism when needed This prevents unnecessary fallbacks to Python, improving performance for `hg update` for instance. On Mozilla-Central a noop update goes from 1.6s down to 700ms. Differential Revision: https://phab.mercurial-scm.org/D8315

templater: fix cbor() filter to recursively convert smartset to list The previous attempt, e3e44e6e7245 "templater: fix cbor() filter to accept smartset", was incomplete since obj may be a collection containing a smartset. This works around the problem by converting smartsets recursively. Another option is to teach cborutil how to encode a smartset. That should be okay, but I hesitated to add "import smartset" to cborutil.py as the cborutil is pretty generic.

shelve: split up dounshelve() in unshelvecmd() and _dounshelve() I'd like to be able to override the new `_dounshelve()`, getting access to the name of the shelve to unshelve. `unshelvecmd()` seems to better match the existing `createcmd()`, `listcmd()` etc. Differential Revision: https://phab.mercurial-scm.org/D8322

darwin: use vim, not vi, to avoid data-loss inducing posix behavior Apple's version of vim, available at opensource.apple.com/release/macos-1015.html (for Catalina, but this behavior has been there for a while) has several tweaks from the version of vim from vim.org. Most of these tweaks appear to be for "Unix2003" compatibility. One of the tweaks is that if any ex command raises an error, the entire process will (when you exit, possibly minutes/hours later) also exit non-zero. Ex commands are things like `:foo`. Luckily, they only enabled this if vim was executed (via a symlink or copying the binary) as `vi` or `ex`. If you start it as `vim`, it doesn't have this behavior, so let's do that. To see this in action, run the following two commands on macOS: ``` $ vi -c ':unknown' -c ':qa' ; echo $? 1 $ vim -c ':unknown' -c ':qa' ; echo $? 0 ``` We don't want to start ignoring non-zero return types from the editor because that will mean you can't use `:cquit` to intentionally exit 1 (which, shows up as 2 if you combine an ex command error and a cquit, but only a 1 if you just use cquit, so we can't differentiate between the two statuses). Since we can't differentiate, we have to assume that all non-zero exit codes are intentional and an indication of the user's desire to not continue with whatever we're doing. If this was a complicated `hg split` or `hg histedit`, this is especially disastrous :( Differential Revision: https://phab.mercurial-scm.org/D8321

rust-matchers: use the `regex` crate Instead of falling back to Python when a code path with "ignore" functionality is reached and `Re2` is not installed, the default compilation (i.e. without the `with-re2` feature) will use the `regex` crate for all regular expressions business. As with the introduction of `Re2` in a previous series, this yields a big performance boost compared to the Python + C code in `status`, `diff`, `commit`, `update`, and maybe others. For now `Re2` looks to be faster at compiling the DFA (1.5ms vs 5ms for Netbeans' `.hgignore`) and a bit faster in actual use: (123ms vs 137ms for the parallel traversal of Netbeans' clean repo). I am in talks with the author of `regex` to see whether that performance difference is a bug, a "won't fix", or a tuning issue. The `regex` crate is already one of our dependencies and using this code does not require any additional work from the end-user than to use the Rust extensions. Differential Revision: https://phab.mercurial-scm.org/D8323

templater: add subsetparents(rev, revset) function Naming suggestions are welcome. And this could be flagged as an (ADVANCED) function since the primary use case is to draw a graph. This provides all data needed for drawing revisions graph filtered by revset, and allows us to implement a GUI graph viewer in some languages better than Python. A frontend grapher will be quite similar to our graphmod since subsetparents() just returns parent-child relations in the filtered sub graph. Frontend example: https://hg.sr.ht/~yuja/hgv/browse/default/core/hgchangesetgrapher.cpp However, the resulting graph will be simpler than the one "hg log -G" would generate because redundant edges are eliminated. This should be the same graph rendering strategy as TortoiseHg. This function could be implemented as a revset predicate, but that would mean the scanning state couldn't be cached and thus slow. Test cases are split to new file since test-template-functions.t is quite big and we'll need a new branchy repository anyway.

templater: remember cache key of evaluated revset This provides a hint for caching further computation result of the given revset. See the next patch for example.

templater: fix cbor() filter to accept smartset So the wrapper type can return a bare smartset.

templater: introduce wrapper for smartset (API) I want to add a template function which takes a revset as an argument: {somefunc(..., revset(...))} ^^^^^^^^^^^ evaluates to a revslist This wrapper will provide a method to get an underlying smartset. It should also be good for performance since count(revset(...)) will no longer have to fully consume the smartset for example, but that isn't the point of this change.

setup: build C extensions with -Werror=declaration-after-statement MSVC 2008 still needs declarations at the top of the scope. I added it to the 3rd party code too in case somebody vendors a new version with a problem- they'll get an early warning. Clang seems to ignore this (at least on 10.14 with Xcode 10), and gcc 7.4 will error out as desired on Ubuntu 18.04. Thanks to Yuya for remembering the name of the option. Differential Revision: https://phab.mercurial-scm.org/D8318

merge with stable

osutil: move declaration to top of the scope Otherwise the build would fail with -Werror=declaration-after-statement.

phabricator: remove *-argument from _getdrevs() It can't take more than one specs arguments per len(*specs).

revset: leverage internal _rev() function to implement rev() Now 'rev(n)' is identical to 'present(_rev(n))'.

revset: fix crash by repo.revs('%d', tip + 1) IndexError shouldn't be raised from a revset predicate. The error message is copied from scmutil.revsymbol().

revset: allow repo.revs('%d', wdirrev) Otherwise we can't write repo.revs('null:%d', subset.max()) to build a smartset covering the range {null .. tip} + {wdir} if subset includes wdir.

ui: use "procutil.shellsplit" to parse command A commandline containing a space ('"C:\\Program Files\\bar.exe" "..."') must not simply split at whitespace, instead quoting has to be taken into account. Use "shlex.split()" to parse it instead. This can improve the error message if we fail to launch a user configured editor which does not exist. Consider [ui] editor = "C:\Program Files\editor\editor.exe" where the path does not exist. "hg histedit" currently aborts with > Abort: edit failed: Program exited with status 1 here "Program" is not part of the message but the name of the program that failed (i.e. `basename("C:\\Program ")`). With this change the message instead reads > Abort: edit failed: C:\Program Files\editor\editor.exe exited with > status 1 which is also not ideal since infact "cmd.exe" exited with code 1, not the editor. But the real error message ("File not found") gets swallowed by `procutil` and including the correct path improves the error message nevertheless.

cext: move variable declaration to the top of the block for C89 support Not sure if we still care about C89 in general, but MSVC requires this style too. Differential Revision: https://phab.mercurial-scm.org/D8304

phabricator: allow multiple DREVSPEC args to phabread|phabimport|phabupdate Per Augie's request. Like regular revsets, the result is the union of all given specs. Unlike regular revsets, these don't resolve in parent -> child order, and should be specified as such on the command line. This change invalidated a previous test using an empty `hg phabread`, so it has been switched to `hg debugcallconduit` to preserve that coverage. Differential Revision: https://phab.mercurial-scm.org/D8233

phabricator: add a helper function to convert DREVSPECs to a DREV dict list Prep work for allowing multiple DREVSPECs to various commands, and properly validating the input. Differential Revision: https://phab.mercurial-scm.org/D8232

merge with stable

tests: avoid logging a commit with a Unicode character in test-phabricator.t The previous output was generated on Windows, and should have been wrapped in `HGENCODING=utf-8` like it is earlier in the test. It's simpler to just avoid it. I only noticed the output change when I got around to running it on a Mac. Differential Revision: https://phab.mercurial-scm.org/D8303

fix: mark -r as advanced See the previous patch for reasoning. I planned to even mark it deprecated, but someone (timeless?) on the #mercurial IRC channel said they sometimes wanted to use `-r` with its existing semantics. Differential Revision: https://phab.mercurial-scm.org/D8288

fix: add a -s option to format a revision and its descendants `hg fix -r abc123` will format that commit but not its descendants. That seems expected given the option name (`-r`), but it's very rarely what the user wants to do. The problem is that any descendants of that commit will not be formatted, leaving them as orphans that are hard to evolve. They are hard to evolve because the new parent will have formatting changes that the orphan doesn't have. I talked to Danny Hooper (who wrote most of the fix extension) about the problem and we agreed that deprecating `-r` in favor of a new `-s` argument (mimicing rebase's `-s`) would be a good way of reducing the risk that users end up with these hard-to-evolve orphans. So that's what this patch implements. Differential Revision: https://phab.mercurial-scm.org/D8287

fix: move handling of --all into getrevstofix() for consistency Differential Revision: https://phab.mercurial-scm.org/D8286

rust-status: add trace-level logging for Rust status fallback for debugging The added `log` crate is already a sub-dependency. Differential Revision: https://phab.mercurial-scm.org/D8300

hg: make _local() behave consistently on Python 3.8 (issue6287) Python 3.8 makes os.path.isfile quietly eat "path invalid" errors and return False instead of allowing the exception to propagate. Given that this is a change from 2018 (sigh) and it's mentioned in the release notes (double sigh) we're definitely too late to complain to Python about the behavior change, so open-code part of os.path.isfile() in this method so we can catch invalid-path errors and handle them appropriately. I confirmed that posixpath and ntpath both delegate to genericpath, which uses os.stat() under the covers. Differential Revision: https://phab.mercurial-scm.org/D8302

remotefilelog: add fake heads() method that allows viewing a file in hgweb As best I can discern, this is not going to hurt anything, but it'll cause a couple of options to exist in the UI that are nonsensical. That seems fine, given the nature of remotefilelog. Differential Revision: https://phab.mercurial-scm.org/D8299

tests: add test for remotefilelog interactions with hgweb It's not uncommon for hg users to rely on hgweb as a simple GUI and history browser (I do this all the time on Mercurial), but we lack any tests to ensure things keep working. At present, this merely demonstrates the "view contents of a single file" endpoint is broken. I'll fix that in a subsequent change. Differential Revision: https://phab.mercurial-scm.org/D8298

phabricator: remove duplicated byteskwargs conversion Fixes things on Python 3. Differential Revision: https://phab.mercurial-scm.org/D8301

chistedit: support histedit.summary-template in curses histedit plan Differential Revision: https://phab.mercurial-scm.org/D8296

vfs: fix typo in comment (remove extra "l") Differential Revision: https://phab.mercurial-scm.org/D8297

tests: make test-doctest.t automatically find files to run tests on Differential Revision: https://phab.mercurial-scm.org/D8294

tests: make test-doctest.t module list match reality Differential Revision: https://phab.mercurial-scm.org/D8280

tests: remove doctest in narrowspec, it is broken I think every item here is considered incorrect (if we fix doctest to run it), so let's just delete it. Differential Revision: https://phab.mercurial-scm.org/D8279

rust-core: add missing `Debug` traits Turns out you need them when trying to use `hg-core` as a library. Who knew. Differential Revision: https://phab.mercurial-scm.org/D8278

fix: refactor getrevstofix() to define revisions first, then validate them This refactoring makes it easier to add a new way of specifying revisions (I'm about to add a `--source`, which adds the specified revisions and their descendants). Differential Revision: https://phab.mercurial-scm.org/D8285

fix: disallow `hg fix --all --working-dir` `--all` implies `--working-dir`, so it's probably a mistake if the user uses both. Differential Revision: https://phab.mercurial-scm.org/D8284

byteify-string: resolve symlink before byteifying Otherwise the script turns symlinks into regular files.

tests: simplify test-fix-topology.t slightly by using a `(case !)` Differential Revision: https://phab.mercurial-scm.org/D8283

tests: fix rebase test broken by earlier cleanup It turns out I broke one of the rebase tests in dd2833e4d660 (tests: use drawdag in test-rebase-collapse.t, 2018-03-18) and didn't notice until now. Differential Revision: https://phab.mercurial-scm.org/D8277

rebase: accept multiple --base arguments (BC) Same reasoning as for the previous patch. Rebase has worked well with `--base 'A + B'` since 51e7c83e05ee (rebase: calculate ancestors for --base separately (issue5420), 2016-11-28). Differential Revision: https://phab.mercurial-scm.org/D8293

rebase: accept multiple --source arguments (BC) I think it's a little surprising that `hg rebase -s A -s B` rebases only `B::` and ignores `A`. That's because the `-s` flag is not a repeated flag. This patch makes it a repeated flag. Differential Revision: https://phab.mercurial-scm.org/D8292

rebase: mention -r argument in synopsis I'm guessing that we simply forgot to mention it there. I was wondering if maybe `-r` was experimental as long as evolution is, but that doesn't seem to be the case (`-r` is not marked experimental). Differential Revision: https://phab.mercurial-scm.org/D8295

rebase: remove unused defaults argument values from _definedestmap() The caller always provides values for these arguments. Differential Revision: https://phab.mercurial-scm.org/D8291

phabricator: add a config knob to create obsolete markers when importing This too can get a tristate switch if desired. But if so, that needs to happen by having evolve add the option. Differential Revision: https://phab.mercurial-scm.org/D8138

phabricator: add a config knob to import in the secret phase For my use anyway, it's better to have a config knob that's always set than to try to remember a switch. But we can add a tristate switch if desired. Differential Revision: https://phab.mercurial-scm.org/D8137

phabricator: add a `phabimport` command I've had `alias.phabimport = $hg phabread --stack $1 | $hg import --bypass -` for a while now, and I suspect others do too. That's limited though, in that it can't use the information on Phabricator to restore it in the original location, so I'm making it a first class command. This doesn't do anything ambitious like that- this is mostly a simplification of `hg import` to get the equivalent of the alias mentioned above. The `--bypass` option is hardcoded to be enabled and the message about amending rejects dropped (rejects aren't created with `--bypass`), because editing patches on Phabricator seems like an unusual workflow. This will need other options, like `--obsolete` and `--secret`. I think these would be more useful as config settings, as I imagine the workflows are pretty fixed depending on roles. Reviewers who don't queue patches probably never want `--obsolete`, but may need `--secret`. Reviewers who do will want the former, but not the latter. I left `--stack` as an option, but that should probably be a config knob too (or at least default to on)- if the point of this is to avoid rejects, it doesn't make sense to skip dependencies in most cases. Evolve is going to need a fix to its wrapping of `cmdutil.tryimportone()`, as it currently assumes `opts` has an `obsolete` key. It's worked around for now. Differential Revision: https://phab.mercurial-scm.org/D8136

rust: update the README with more up-to-date and thorough information Differential Revision: https://phab.mercurial-scm.org/D8274

transactions: convert changes['phases'] to list of ranges Consecutive revisions are often in the same phase, especially public revisions. This means that a dictionary keyed by the revision for the phase transitions is highly redundant. Build a list of (range, (old, new)) entries instead and aggressively merge ranges with the same transition. For the test case in issue5691, this reduces memory use by ~20MB. Differential Revision: https://phab.mercurial-scm.org/D8125

test-install: don't print Rust re2 bindings information if Rust is not in use Differential Revision: https://phab.mercurial-scm.org/D8273

tests: drop an extraneous (glob) from test-debugbackupbundle.t Since this was not needed, it ends up causing the test to end with an error saying that the output changed, but with no diff and a message at the end saying "no result code from test". Differential Revision: https://phab.mercurial-scm.org/D8269

pull: add `--confirm` flag to confirm before writing changes This introduces a new flag to pull command `--confirm` and also a config option named `pull.confirm` which if used will prompt user describing changes which are pulled and asking whether to accept them or not. Differential Revision: https://phab.mercurial-scm.org/D8200

scmutil: add option to register summary callbacks as transaction validators We have a list of summary callbacks which are run after the transaction is closed to show what has changed and what not. This patch makes it possible to register those callbacks as transaction validators so that we can show summary before committing the transaction and prompt user to accept the changes. The goal of this is to implement `pull --confirm`. Differential Revision: https://phab.mercurial-scm.org/D8199

transaction: add functionality to have multiple validators This will help us in adding more validators which can aggregate data from transaction and prompt user whether to commit the transaction or not. The current target is to use this to implement `pull --confirm`. Differential Revision: https://phab.mercurial-scm.org/D8198

hgit: make sure repository is local before checking for store type httppeer (and maybe others too) does not have a store attribute. This was causing `hg pull` being broken on a hg repository when the extension is enabled. localpeer.local() does returns a non-None value but I am not sure if it matters. Differential Revision: https://phab.mercurial-scm.org/D8217

hg-core: add function timing information This change makes use of the newly added logging infrastructure to trace the execution time of some important calls. This approach is very much complementary to using a profiler and will not guard against out-of-order execution or other kinds of compiler optimizations. That said, it is useful to get a rough high-level idea of where time is spent. Differential Revision: https://phab.mercurial-scm.org/D8253

rust: add logging utils This change adds the `log` crate, the community-approved logging facade backed by Rust core developers as well as the logging-consumer crate `simple_logger` to build a foundation for logging from Rust. Using this setup allows us to choose how to log depending on the way `hg-core` is used: if it's within the context of `hg-cpython`, we might not want to use it the same way as with a direct cli for example. Differential Revision: https://phab.mercurial-scm.org/D8252

rust-status: traverse working directory in parallel Using `rayon` for this task ensures that we are using the same work-stealing threadpool for everything. This change introduces `crossbeam` as an explicit dependency, although it is already a dependency of `rayon`. It provides better structures for multi-threaded tasks than the stdlib. Differential Revision: https://phab.mercurial-scm.org/D8251

rust-status: wrap `stat_dmap_entries` to ease profiling Differential Revision: https://phab.mercurial-scm.org/D8250

rust-status: refactor handling of unknown files Differential Revision: https://phab.mercurial-scm.org/D8249

rust-status: move to recursive traversal to prepare for parallel traversal I have looked into traversing the working directory in parallel either by a recursive or an iterative algorithm. The recursive approach won quite decisively both in terms of performance and code readability. You can look at my experiment here: https://heptapod.octobus.net/Alphare/rayon-recursive-traversal The chance of a stack overflow happening because the directories get too nested seems slim. This change does not yet do anything in parallel. Differential Revision: https://phab.mercurial-scm.org/D8215

rust-pathauditor: use interior mutability for use in multi-threaded contexts The usual recommendation for using `RwLock` or `Mutex` is that if there are about as many write as there are reads, use `Mutex`, and if there are more reads than writes, use `RwLock`. If after the main bottleneck (i.e. parallel traversal) is removed this shows up on profiles, we should investigate using the `parking_lot` since we don't need a poisoning API, or maybe move to different types of caches entirely. Differential Revision: https://phab.mercurial-scm.org/D8213

rust-pathauditor: actually populate the `audited_dirs` cache I forgot this when rewriting this logic during the review process. Differential Revision: https://phab.mercurial-scm.org/D8212

rust-pathauditor: actually split Windows shortname aliases at `~` Right now, splitting with a max of 1 parts is useless. Differential Revision: https://phab.mercurial-scm.org/D8211

debuginstall: add entry about re2 Rust bindings when applicable Differential Revision: https://phab.mercurial-scm.org/D8226

rust-cpython: add `debug` module to expose debug information to Python This will be mostly used in `debuginstall`. Differential Revision: https://phab.mercurial-scm.org/D8225

rust-status: use bare hg status fastpath from Python This change also adds a test case for subrepos. Repeating the benchmark information from the `hg-core` commit: On the Netbeans repository: C: 840ms Rust+C: 556ms Mozilla Central with the one pattern that causes a fallback removed: C: 2.315s Rust+C: 1.700s Differential Revision: https://phab.mercurial-scm.org/D7931

rust-status: update rust-cpython bridge to account for the changes in core Differential Revision: https://phab.mercurial-scm.org/D7930

rust-status: add bare `hg status` support in hg-core A lot of performance remains to be gained, most notably by doing more things in parallel, but also by caching, not falling back to Python but switching to another regex engine, etc.. I have measured on multiple repositories that this change, when in combination with the next two patches, improve bare `hg status` performance, and has no observable impact when falling back (because it does so early). On the Netbeans repository: C: 840ms Rust+C: 556ms Mozilla Central with the one pattern that causes a fallback removed: C: 2.315s Rust+C: 1.700 s Differential Revision: https://phab.mercurial-scm.org/D7929

rust-status: add function for sequential traversal of the working directory This change also introduces helper structs to make things clearer. Differential Revision: https://phab.mercurial-scm.org/D7928

rust-status: add missing variants to `Dispatch` enum Differential Revision: https://phab.mercurial-scm.org/D8088

rust-status: rename `StatusResult` to `DirstateStatus` "Result" has a special meaning in the Rust world, this should be clearer. Differential Revision: https://phab.mercurial-scm.org/D8087

rust-status: refactor options into a `StatusOptions` struct Differential Revision: https://phab.mercurial-scm.org/D8086

rust-status: add util for listing a directory I debated moving it to utils, but it is not used anywhere else for now, and its skip behavior is pretty specific to status. Differential Revision: https://phab.mercurial-scm.org/D7927

rust-matchers: add `IgnoreMatcher` This is a big change but all of the pieces call each other, so it makes sense to have this all in one patch. Differential Revision: https://phab.mercurial-scm.org/D7925

rust-matchers: add `build_regex_match` function This function will be used to help build the upcoming `IncludeMatcher`. Differential Revision: https://phab.mercurial-scm.org/D7924

rust-matchers: add functions to get roots, dirs and parents from patterns These functions will be used to help build the upcoming `IncludeMatcher`. Differential Revision: https://phab.mercurial-scm.org/D7923

rust-matchers: add function to generate a regex matcher function This function will be used to help build the upcoming `IncludeMatcher`. While Re2 is still used and behind a feature flag, this function returns an error meant for fallback in the default case. Differential Revision: https://phab.mercurial-scm.org/D7922

run-tests: restrict Rust thread pool to 3 threads during tests Differential Revision: https://phab.mercurial-scm.org/D8270

nodemap: make sure the nodemap docket is updated after the changelog It is better to have a lagging nodemap than a nodemap pointing to node that does not exists yet. (strictly speaking, the order was already right, however we make it more explicit). Differential Revision: https://phab.mercurial-scm.org/D8188

nodemap: make sure hooks have access to an up-to-date version We make sure hooks can read persistent nodemap data and that they access something up-to-date with the pending transaction. Differential Revision: https://phab.mercurial-scm.org/D8187

nodemap: track the tip_node for validation Differential Revision: https://phab.mercurial-scm.org/D8184

nodemap: add a todo list for getting out of experimental This is all the requirement I can think off. More might be added as they emerge. The first ones are mostly simple technical matters that will be taken care of soon. The question about the "status" of the persistent nodemap and the revlogs that will use it requires more discussion and thinking. Differential Revision: https://phab.mercurial-scm.org/D8181

rust-nodemap: automatically use the rust index for persistent nodemap The persistent nodemap requires the rust index to be used to provides any gains. So we automatically enable it for revlog using the persistent nodemap. We keep it off for other revset because now that the rust revlog fully initialise the nodemap using it everywhere introduce a fairly significant regression (eg: hg diff moving from 0.8s to 2.3s on mozilla-try) Differential Revision: https://phab.mercurial-scm.org/D8164

nodemap: use data from the index in debugnodemap --dump-new It is better to run the actual code when generating a new nodemap cache from scratch. So we do. Differential Revision: https://phab.mercurial-scm.org/D8163

rust-nodemap: also clear Rust data in `clearcaches` Differential Revision: https://phab.mercurial-scm.org/D8161

rust-nodemap: add binding to `nodemap_update_data` Differential Revision: https://phab.mercurial-scm.org/D8160

rust-nodemap: add binding for `nodemap_data_incremental` Differential Revision: https://phab.mercurial-scm.org/D8159

rust-nodemap: add binding for `nodemap_data_all` Differential Revision: https://phab.mercurial-scm.org/D8158

rust-nodemap: use proper Index API instead of using the C API Differential Revision: https://phab.mercurial-scm.org/D8157

rust-nodemap: add utils for propagating errors This also updates the copyright notice Differential Revision: https://phab.mercurial-scm.org/D8156

rust-nodemap: add utils to create `Node`s from Python objects Differential Revision: https://phab.mercurial-scm.org/D8155

rust-index: add `append` method to cindex/Index This will be used by `MixedIndex` in a later patch. Differential Revision: https://phab.mercurial-scm.org/D8154

rust-index: moved constructor in separate impl block with the upcoming addition of the nodemap logic, it's going to become more complicated. Being in the separate impl block has the advantage to be subject to rustfmt, whereas code inside macro calls isn't. Differential Revision: https://phab.mercurial-scm.org/D8153

revlog: using two new functions in C capsule from Rust code We expose `index_length` and `index_node` in the C capsule, so that the Rust representation of the C index can implement the `RevlogIndex` trait. Because our `Node` is actually a one-field struct, we have to decorate it for direct FFI exchange with the C `char*` It would be a good thing to get a length from the C layer, but doing so right now would probably interfere with the upcoming changes that will happen there for the hash length. Differential Revision: https://phab.mercurial-scm.org/D8152

nodemap: refresh the persistent data on nodemap creation The logic to read the data and validate the docket are still in python, so we need to "help" whatever compiled code live in the index to refresh it. Otherwise clearing the cache could lead to an expensive full recomputation and disk update even when the persisted data are still valid. Differential Revision: https://phab.mercurial-scm.org/D8174

cext-index: propagate inline_scan error in `index_deref` Before this change, revlog index corruption could be silently ignored in some situation. Differential Revision: https://phab.mercurial-scm.org/D8276

heptapod-ci: fix test paths in the listing file Now what we run the test from the root, we need to list test name from the root. Differential Revision: https://phab.mercurial-scm.org/D8275

tests: check availability of pyflakes by trying to import pyflakes module Since e397c6d74652, we use the pyflakes module instead of the pyflakes executable. As was pointed out during the review, the hghave check can be rewritten to try to import the pyflakes module instead of spawning a new subprocess.

nodemap: deal with the "debugupdatecache" case using a "fake" transaction We are going to need more and more methods of the transaction. So lets change approach. The `hg debugupdatecache` case do not need a transaction, because has the repositories locked, but is not adding any "store" data to it. Differential Revision: https://phab.mercurial-scm.org/D8186

changelog: change the implementation of `_divertopener` During a transaction, new data to the changelog index are "diverted". The previous implementat ways pretty basic, we need something better to be able to use the changelog vfs fully during transaction. (eg: accessing the vfs options). Differential Revision: https://phab.mercurial-scm.org/D8185

nodemap: test that an outdated nodemap can catch up If the persistent nodemap does not contains all revision that the index contains, this is should be detected and dealt with. We add a test for this case. Differential Revision: https://phab.mercurial-scm.org/D8183

nodemap: document the docket attributes More documentation rarely hurts. Differential Revision: https://phab.mercurial-scm.org/D8182

git: key off `git` in .hg/requires rather than separate file Differential Revision: https://phab.mercurial-scm.org/D8265

git: correctly handle p1() on dirstate when underlying git repo is empty This shows up in my next change, which ends up making an empty git repo and then running hg. Differential Revision: https://phab.mercurial-scm.org/D8271

rust: clean remains of direct-ffi code Since b55bec1ea972, the cext entry poitn have been removed, so we drop the code for consistency.

heptapod-ci: use the new `--rust/--no-rust` flag to run the tests

run-tests: add option for running with and without Rust extensions This provide a simple and clear way to run the test with or without rust.

run-tests: rename "pure" variable to setup_opts There are more useful flag than just `--pure`. We clarify the variable name before adding more options.