rust-matchers: add `Matcher` trait and implement `AlwaysMatcher` In our quest of a faster Mercurial, we have arrived at the point where we need to implement the matchers in Rust. This RFC mainly for the `Matcher` trait to see if the changes proposed feel fine to people with more experience on the matter. While the `AlwaysMatcher` implementation is here as a trivial example, it should be the first step towards matchers use in Rust as it is currently the only supported one. Notable changes: - `exact` is renamed to `exact_match` - enums for `visit*` methods with `Recursive` instead of `'all'`, etc. - a new `roots`, separate from `file_set` - no `bad`, `explicitdir` or `traversedir` functions as they can be passed to the high functions instead of the matchers Thanks to Martin for suggesting the last two (most important) changes and for reaching out to help a few weeks ago. Differential Revision: https://phab.mercurial-scm.org/D7178

merge with stable

perf: add a way to benchmark `dirstate.status` Getting more details about time spend in this specific internal bit is meaningful.

largefiles: delete obsolete and unused repo.push() The function was removed from localrepo in 4d52e6eb98ea (locarepo: remove the `push` method (API), 2014-09-25). Differential Revision: https://phab.mercurial-scm.org/D7174

censor: document that some commands simply ignore censored data I can't see a benefit for hg grep to ever error out early when it encounters censored data.

grep: warn on censored revisions instead of erroring out We need most of the grep logic to go through in case we encounter a censored revision, so we just return a None body for a censored node, and we stop just short of trying to record matches with the contents of that censored body. The other parts such as recording that the censored file has been considered at this revision needs to go into the proper dicts. I have also gotten weary of all the abbreviations, so while I did a small refactor to move the file-data-getting operation into a common function, I also expanded the abbreviations of the relevant variables within this little function. Hopefully some day this helps someone figure out what all the abbreviations mean. Although the censoring docs currently state that some commands error out or are ignored depending on the `censor.policy` config, I cannot see a benefit for grep to ever stop dead in its tracks when a censored revision is encountered. I will also amend the docs to indicate that some commands, such as grep, unconditionally ignore censored revisions.

py3: enable legacy fs encoding to fix filename compatibility on Windows This patch is untested. I just followed the instruction: https://docs.python.org/3/whatsnew/3.6.html#pep-529-change-windows-filesystem-encoding-to-utf-8

rust-cpython: run cargo fmt

rust-cpython: remove useless PyResult<> from leak_immutable() The caller should know if the shared data is mutably borrowed or not.

rust-cpython: remove useless PyRefMut wrapper

rust-cpython: drop manual management of mutably_borrowed RefCell::borrow() should guarantee there's no mutable borrow. As a follow up, maybe PySharedState can be a pure data structure + function. Most ref-sharing business has already been moved to PySharedRef* and PyLeaked*.

rust-cpython: leverage RefCell::borrow() to guarantee there's no mutable ref Since the underlying value can't be mutably borrowed by PyLeaked, we don't have to manage yet another mutably-borrowed state. We can just rely on the RefCell implementation. Maybe we can add try_leak_immutable(), but this patch doesn't in order to keep the patch series not too long.

rust-cpython: remove useless Option<$leaked> from py_shared_iterator We no longer need to carefully drop the iterator when it's consumed. Mutation is allowed even if the iterator exists. There's a minor behavior change: next(iter) may return/raise something other than StopIteration if it's called after the iterator has been fully consumed, and if the Rust object isn't a FusedIterator.

rust-cpython: allow mutation unless leaked reference is borrowed In other words, mutation is allowed while a Python iterator holding PyLeaked exists. The iterator will be invalidated instead. We still need a borrow_count to prevent mutation while leaked data is dereferenced in Rust world, but most leak_count business is superseded by the generation counter. decrease_leak_count(py, true) will be removed soon.

rust-cpython: add generation counter to leaked reference This counter increments on borrow_mut() to invalidate existing leaked references. This is modeled after the iterator invalidation in Python. The other checks will be adjusted by the subsequent patches.

rust-cpython: add stub wrapper that'll prevent leaked data from being mutated In order to allow mutation of PySharedRefCell value while PyLeaked reference exists, we need yet another "borrow" scope where mutation is prohibited. try_borrow<'a> and try_borrow_mut<'a> defines the "borrow" scope <'a>. The subsequent patches will implement leak counter based on this scope. PyLeakedRef<T> and PyLeakedRefMut<T> could be unified to PyLeakedRef<&T> and PyLeakedRef<&mut T> respectively, but I didn't do that since it seemed a bit weird that deref_mut() would return a mutable reference to an immutable reference.

rust-cpython: rename PyLeakedRef to PyLeaked This series will make PyLeaked* behave more like a Python iterator, which means mutation of the owner object will be allowed and the leaked reference (i.e. the iterator) will be invalidated instead. I'll add PyLeakedRef/PyLeakedRefMut structs which will represent a "borrowed" state, and prevent the underlying value from being mutably borrowed while the leaked reference is in use: let shared = self.inner_shared(py); let leaked = shared.leak_immutable(); { let leaked_ref: PyLeakedRef<_> = leaked.borrow(py); shared.borrow_mut(); // panics since the underlying value is borrowed } shared.borrow_mut(); // allowed The relation between PyLeaked* structs is quite similar to RefCell/Ref/RefMut, but the implementation can't be reused because the borrowing state will have to be shared across objects having no lifetime relation. PyLeaked isn't named as PyLeakedCell since it isn't actually a cell in that leaked.borrow_mut() will require &mut self.

fix: fix grammar/typos in hg help -e fix

progress: flush stderr after clearing On python3, ui.stderr is buffered, it seems, so we need to flush it to actually get the progress bar off the screen. This is important since ui.write() will call into progbar.clear() if it thinks there's a progress bar on the screen, with the intent that the next thing it outputs is at the beginning of the line (instead of at the end of the progress bar line). Without the flush, we buffer up the clearing of the screen, and we get some really weird/corrupt output. Differential Revision: https://phab.mercurial-scm.org/D7784

subrepo: fix a crash when archiving an svn or git subrepo Only hgsubrepos have a repository attribute. This is pretty hacky, but probably the best we can do on stable. Pushing the lfstatus check down into the wrapper for hgsubrepo (and dropping the check for lfstatus at the top of `hgsubrepoarchive()`) resulted in various test failures because: 1) hgsubrepoarchive isn't returning the number of files archived at the bottom, resulting in an error about += NoneType 2) These copypasta archive wrappers don't use progress bars 3) Largefiles are *not* currently archived when using extdiff (68822b7cdd01), but pushing this context manager down into the subrepo resulted in it apparently doing so (as evidenced by progress bars being dropped) The other uses of `lfstatus()` are not in the substate processing loop, so they shouldn't be an issue. I initially put testcases in this test for largefiles-{on,off}, and it flagged a bunch of exit code differences for `cat` and `diff`, so I backed that off. Differential Revision: https://phab.mercurial-scm.org/D7714

Added signature for changeset b4c82b704180

Added tag 5.2.2 for changeset b4c82b704180

py3: force bytestr conversion of "reason" in scmutil.callcatch() For instance, reason may be an InvalidURL as shown in added test.

rust-cpython: do not convert warning pattern to utf-8 bytes On Unix, both Rust Path and Mercurial expect a locale-dependent bytes, and we don't support Windows yet. Differential Revision: https://phab.mercurial-scm.org/D7614

rust-cpython: import utils::files::* function at module level IIRC, it's common in Rust to call functions with the module prefix. Differential Revision: https://phab.mercurial-scm.org/D7613

py3: send bytes from Rust-created warning patterns Python code expects bytes in both Python 2 and Python 3, so we should send bytes. Differential Revision: https://phab.mercurial-scm.org/D7612

py3: pass bytes to `configint` and `configbool` Both functions require bytes, even in Python 3. Differential Revision: https://phab.mercurial-scm.org/D7611

rust-threads: force Rayon to respect the worker count in config As per the inline comment, this is a workaround because Rust code does not yet know how to read config files. Differential Revision: https://phab.mercurial-scm.org/D7610

remotefilelog: remove the `ensurestart` usage The option does not seems to do what we want since it does not wait on the background process actually starting (eg: taking the lock). And the race have been dealt with another way anyway. This effectively back out changeset 2c74337e6483. Differential Revision: https://phab.mercurial-scm.org/D7590

remotefilelog: remove most of the debugwait* command in the tests Since we use a dedicated developer config, we do not need them anymore. Differential Revision: https://phab.mercurial-scm.org/D7589