rebase: mention -r argument in synopsis I'm guessing that we simply forgot to mention it there. I was wondering if maybe `-r` was experimental as long as evolution is, but that doesn't seem to be the case (`-r` is not marked experimental). Differential Revision: https://phab.mercurial-scm.org/D8295

rebase: remove unused defaults argument values from _definedestmap() The caller always provides values for these arguments. Differential Revision: https://phab.mercurial-scm.org/D8291

phabricator: add a config knob to create obsolete markers when importing This too can get a tristate switch if desired. But if so, that needs to happen by having evolve add the option. Differential Revision: https://phab.mercurial-scm.org/D8138

phabricator: add a config knob to import in the secret phase For my use anyway, it's better to have a config knob that's always set than to try to remember a switch. But we can add a tristate switch if desired. Differential Revision: https://phab.mercurial-scm.org/D8137

phabricator: add a `phabimport` command I've had `alias.phabimport = $hg phabread --stack $1 | $hg import --bypass -` for a while now, and I suspect others do too. That's limited though, in that it can't use the information on Phabricator to restore it in the original location, so I'm making it a first class command. This doesn't do anything ambitious like that- this is mostly a simplification of `hg import` to get the equivalent of the alias mentioned above. The `--bypass` option is hardcoded to be enabled and the message about amending rejects dropped (rejects aren't created with `--bypass`), because editing patches on Phabricator seems like an unusual workflow. This will need other options, like `--obsolete` and `--secret`. I think these would be more useful as config settings, as I imagine the workflows are pretty fixed depending on roles. Reviewers who don't queue patches probably never want `--obsolete`, but may need `--secret`. Reviewers who do will want the former, but not the latter. I left `--stack` as an option, but that should probably be a config knob too (or at least default to on)- if the point of this is to avoid rejects, it doesn't make sense to skip dependencies in most cases. Evolve is going to need a fix to its wrapping of `cmdutil.tryimportone()`, as it currently assumes `opts` has an `obsolete` key. It's worked around for now. Differential Revision: https://phab.mercurial-scm.org/D8136

rust: update the README with more up-to-date and thorough information Differential Revision: https://phab.mercurial-scm.org/D8274

transactions: convert changes['phases'] to list of ranges Consecutive revisions are often in the same phase, especially public revisions. This means that a dictionary keyed by the revision for the phase transitions is highly redundant. Build a list of (range, (old, new)) entries instead and aggressively merge ranges with the same transition. For the test case in issue5691, this reduces memory use by ~20MB. Differential Revision: https://phab.mercurial-scm.org/D8125

test-install: don't print Rust re2 bindings information if Rust is not in use Differential Revision: https://phab.mercurial-scm.org/D8273

tests: drop an extraneous (glob) from test-debugbackupbundle.t Since this was not needed, it ends up causing the test to end with an error saying that the output changed, but with no diff and a message at the end saying "no result code from test". Differential Revision: https://phab.mercurial-scm.org/D8269

pull: add `--confirm` flag to confirm before writing changes This introduces a new flag to pull command `--confirm` and also a config option named `pull.confirm` which if used will prompt user describing changes which are pulled and asking whether to accept them or not. Differential Revision: https://phab.mercurial-scm.org/D8200

scmutil: add option to register summary callbacks as transaction validators We have a list of summary callbacks which are run after the transaction is closed to show what has changed and what not. This patch makes it possible to register those callbacks as transaction validators so that we can show summary before committing the transaction and prompt user to accept the changes. The goal of this is to implement `pull --confirm`. Differential Revision: https://phab.mercurial-scm.org/D8199

transaction: add functionality to have multiple validators This will help us in adding more validators which can aggregate data from transaction and prompt user whether to commit the transaction or not. The current target is to use this to implement `pull --confirm`. Differential Revision: https://phab.mercurial-scm.org/D8198

hgit: make sure repository is local before checking for store type httppeer (and maybe others too) does not have a store attribute. This was causing `hg pull` being broken on a hg repository when the extension is enabled. localpeer.local() does returns a non-None value but I am not sure if it matters. Differential Revision: https://phab.mercurial-scm.org/D8217

hg-core: add function timing information This change makes use of the newly added logging infrastructure to trace the execution time of some important calls. This approach is very much complementary to using a profiler and will not guard against out-of-order execution or other kinds of compiler optimizations. That said, it is useful to get a rough high-level idea of where time is spent. Differential Revision: https://phab.mercurial-scm.org/D8253

rust: add logging utils This change adds the `log` crate, the community-approved logging facade backed by Rust core developers as well as the logging-consumer crate `simple_logger` to build a foundation for logging from Rust. Using this setup allows us to choose how to log depending on the way `hg-core` is used: if it's within the context of `hg-cpython`, we might not want to use it the same way as with a direct cli for example. Differential Revision: https://phab.mercurial-scm.org/D8252

rust-status: traverse working directory in parallel Using `rayon` for this task ensures that we are using the same work-stealing threadpool for everything. This change introduces `crossbeam` as an explicit dependency, although it is already a dependency of `rayon`. It provides better structures for multi-threaded tasks than the stdlib. Differential Revision: https://phab.mercurial-scm.org/D8251

rust-status: wrap `stat_dmap_entries` to ease profiling Differential Revision: https://phab.mercurial-scm.org/D8250

rust-status: refactor handling of unknown files Differential Revision: https://phab.mercurial-scm.org/D8249

rust-status: move to recursive traversal to prepare for parallel traversal I have looked into traversing the working directory in parallel either by a recursive or an iterative algorithm. The recursive approach won quite decisively both in terms of performance and code readability. You can look at my experiment here: https://heptapod.octobus.net/Alphare/rayon-recursive-traversal The chance of a stack overflow happening because the directories get too nested seems slim. This change does not yet do anything in parallel. Differential Revision: https://phab.mercurial-scm.org/D8215

rust-pathauditor: use interior mutability for use in multi-threaded contexts The usual recommendation for using `RwLock` or `Mutex` is that if there are about as many write as there are reads, use `Mutex`, and if there are more reads than writes, use `RwLock`. If after the main bottleneck (i.e. parallel traversal) is removed this shows up on profiles, we should investigate using the `parking_lot` since we don't need a poisoning API, or maybe move to different types of caches entirely. Differential Revision: https://phab.mercurial-scm.org/D8213

rust-pathauditor: actually populate the `audited_dirs` cache I forgot this when rewriting this logic during the review process. Differential Revision: https://phab.mercurial-scm.org/D8212

rust-pathauditor: actually split Windows shortname aliases at `~` Right now, splitting with a max of 1 parts is useless. Differential Revision: https://phab.mercurial-scm.org/D8211

debuginstall: add entry about re2 Rust bindings when applicable Differential Revision: https://phab.mercurial-scm.org/D8226

rust-cpython: add `debug` module to expose debug information to Python This will be mostly used in `debuginstall`. Differential Revision: https://phab.mercurial-scm.org/D8225

rust-status: use bare hg status fastpath from Python This change also adds a test case for subrepos. Repeating the benchmark information from the `hg-core` commit: On the Netbeans repository: C: 840ms Rust+C: 556ms Mozilla Central with the one pattern that causes a fallback removed: C: 2.315s Rust+C: 1.700s Differential Revision: https://phab.mercurial-scm.org/D7931

rust-status: update rust-cpython bridge to account for the changes in core Differential Revision: https://phab.mercurial-scm.org/D7930

rust-status: add bare `hg status` support in hg-core A lot of performance remains to be gained, most notably by doing more things in parallel, but also by caching, not falling back to Python but switching to another regex engine, etc.. I have measured on multiple repositories that this change, when in combination with the next two patches, improve bare `hg status` performance, and has no observable impact when falling back (because it does so early). On the Netbeans repository: C: 840ms Rust+C: 556ms Mozilla Central with the one pattern that causes a fallback removed: C: 2.315s Rust+C: 1.700 s Differential Revision: https://phab.mercurial-scm.org/D7929

rust-status: add function for sequential traversal of the working directory This change also introduces helper structs to make things clearer. Differential Revision: https://phab.mercurial-scm.org/D7928

rust-status: add missing variants to `Dispatch` enum Differential Revision: https://phab.mercurial-scm.org/D8088

rust-status: rename `StatusResult` to `DirstateStatus` "Result" has a special meaning in the Rust world, this should be clearer. Differential Revision: https://phab.mercurial-scm.org/D8087

rust-status: refactor options into a `StatusOptions` struct Differential Revision: https://phab.mercurial-scm.org/D8086

rust-status: add util for listing a directory I debated moving it to utils, but it is not used anywhere else for now, and its skip behavior is pretty specific to status. Differential Revision: https://phab.mercurial-scm.org/D7927

rust-matchers: add `IgnoreMatcher` This is a big change but all of the pieces call each other, so it makes sense to have this all in one patch. Differential Revision: https://phab.mercurial-scm.org/D7925

rust-matchers: add `build_regex_match` function This function will be used to help build the upcoming `IncludeMatcher`. Differential Revision: https://phab.mercurial-scm.org/D7924

rust-matchers: add functions to get roots, dirs and parents from patterns These functions will be used to help build the upcoming `IncludeMatcher`. Differential Revision: https://phab.mercurial-scm.org/D7923

rust-matchers: add function to generate a regex matcher function This function will be used to help build the upcoming `IncludeMatcher`. While Re2 is still used and behind a feature flag, this function returns an error meant for fallback in the default case. Differential Revision: https://phab.mercurial-scm.org/D7922

run-tests: restrict Rust thread pool to 3 threads during tests Differential Revision: https://phab.mercurial-scm.org/D8270

nodemap: make sure the nodemap docket is updated after the changelog It is better to have a lagging nodemap than a nodemap pointing to node that does not exists yet. (strictly speaking, the order was already right, however we make it more explicit). Differential Revision: https://phab.mercurial-scm.org/D8188

nodemap: make sure hooks have access to an up-to-date version We make sure hooks can read persistent nodemap data and that they access something up-to-date with the pending transaction. Differential Revision: https://phab.mercurial-scm.org/D8187

nodemap: track the tip_node for validation Differential Revision: https://phab.mercurial-scm.org/D8184

nodemap: add a todo list for getting out of experimental This is all the requirement I can think off. More might be added as they emerge. The first ones are mostly simple technical matters that will be taken care of soon. The question about the "status" of the persistent nodemap and the revlogs that will use it requires more discussion and thinking. Differential Revision: https://phab.mercurial-scm.org/D8181

rust-nodemap: automatically use the rust index for persistent nodemap The persistent nodemap requires the rust index to be used to provides any gains. So we automatically enable it for revlog using the persistent nodemap. We keep it off for other revset because now that the rust revlog fully initialise the nodemap using it everywhere introduce a fairly significant regression (eg: hg diff moving from 0.8s to 2.3s on mozilla-try) Differential Revision: https://phab.mercurial-scm.org/D8164

nodemap: use data from the index in debugnodemap --dump-new It is better to run the actual code when generating a new nodemap cache from scratch. So we do. Differential Revision: https://phab.mercurial-scm.org/D8163

rust-nodemap: also clear Rust data in `clearcaches` Differential Revision: https://phab.mercurial-scm.org/D8161

rust-nodemap: add binding to `nodemap_update_data` Differential Revision: https://phab.mercurial-scm.org/D8160

rust-nodemap: add binding for `nodemap_data_incremental` Differential Revision: https://phab.mercurial-scm.org/D8159

rust-nodemap: add binding for `nodemap_data_all` Differential Revision: https://phab.mercurial-scm.org/D8158

rust-nodemap: use proper Index API instead of using the C API Differential Revision: https://phab.mercurial-scm.org/D8157

rust-nodemap: add utils for propagating errors This also updates the copyright notice Differential Revision: https://phab.mercurial-scm.org/D8156

rust-nodemap: add utils to create `Node`s from Python objects Differential Revision: https://phab.mercurial-scm.org/D8155

rust-index: add `append` method to cindex/Index This will be used by `MixedIndex` in a later patch. Differential Revision: https://phab.mercurial-scm.org/D8154

rust-index: moved constructor in separate impl block with the upcoming addition of the nodemap logic, it's going to become more complicated. Being in the separate impl block has the advantage to be subject to rustfmt, whereas code inside macro calls isn't. Differential Revision: https://phab.mercurial-scm.org/D8153

revlog: using two new functions in C capsule from Rust code We expose `index_length` and `index_node` in the C capsule, so that the Rust representation of the C index can implement the `RevlogIndex` trait. Because our `Node` is actually a one-field struct, we have to decorate it for direct FFI exchange with the C `char*` It would be a good thing to get a length from the C layer, but doing so right now would probably interfere with the upcoming changes that will happen there for the hash length. Differential Revision: https://phab.mercurial-scm.org/D8152

nodemap: refresh the persistent data on nodemap creation The logic to read the data and validate the docket are still in python, so we need to "help" whatever compiled code live in the index to refresh it. Otherwise clearing the cache could lead to an expensive full recomputation and disk update even when the persisted data are still valid. Differential Revision: https://phab.mercurial-scm.org/D8174

tests: check availability of pyflakes by trying to import pyflakes module Since e397c6d74652, we use the pyflakes module instead of the pyflakes executable. As was pointed out during the review, the hghave check can be rewritten to try to import the pyflakes module instead of spawning a new subprocess.

nodemap: deal with the "debugupdatecache" case using a "fake" transaction We are going to need more and more methods of the transaction. So lets change approach. The `hg debugupdatecache` case do not need a transaction, because has the repositories locked, but is not adding any "store" data to it. Differential Revision: https://phab.mercurial-scm.org/D8186

changelog: change the implementation of `_divertopener` During a transaction, new data to the changelog index are "diverted". The previous implementat ways pretty basic, we need something better to be able to use the changelog vfs fully during transaction. (eg: accessing the vfs options). Differential Revision: https://phab.mercurial-scm.org/D8185

nodemap: test that an outdated nodemap can catch up If the persistent nodemap does not contains all revision that the index contains, this is should be detected and dealt with. We add a test for this case. Differential Revision: https://phab.mercurial-scm.org/D8183

nodemap: document the docket attributes More documentation rarely hurts. Differential Revision: https://phab.mercurial-scm.org/D8182

git: key off `git` in .hg/requires rather than separate file Differential Revision: https://phab.mercurial-scm.org/D8265