repoview: wrap changelog class when filtering The class doesn't yet do anything. I'll move the filter-aware overrides from the changelog class over one by one to this class in coming patches. That will leave the changelog class simpler and will centralize more of the filtering logic to repoview. I could not measure any performance difference. Differential Revision: https://phab.mercurial-scm.org/D7236

repoview: extract a function for wrapping changelog I would like to clean up the changelog class by moving out knowledge of filtering. The filtering will instead be done in a class that wraps the changelog. This patch prepares for that. Differential Revision: https://phab.mercurial-scm.org/D7235

revlog: move tiprev() from changelog up to revlog This makes the changelog override simpler and it seems more consistent. Differential Revision: https://phab.mercurial-scm.org/D7250

tests: fix a few `(no-windows !)` conditionals that are really `(symlink !)` The CI for py3 is assuming that symlinks are possible (they are when running as Administrator or when Developer Mode is enabled), and these popped up. Differential Revision: https://phab.mercurial-scm.org/D7232

rust-matchers: add `Matcher` trait and implement `AlwaysMatcher` In our quest of a faster Mercurial, we have arrived at the point where we need to implement the matchers in Rust. This RFC mainly for the `Matcher` trait to see if the changes proposed feel fine to people with more experience on the matter. While the `AlwaysMatcher` implementation is here as a trivial example, it should be the first step towards matchers use in Rust as it is currently the only supported one. Notable changes: - `exact` is renamed to `exact_match` - enums for `visit*` methods with `Recursive` instead of `'all'`, etc. - a new `roots`, separate from `file_set` - no `bad`, `explicitdir` or `traversedir` functions as they can be passed to the high functions instead of the matchers Thanks to Martin for suggesting the last two (most important) changes and for reaching out to help a few weeks ago. Differential Revision: https://phab.mercurial-scm.org/D7178

merge with stable

perf: add a way to benchmark `dirstate.status` Getting more details about time spend in this specific internal bit is meaningful.

largefiles: delete obsolete and unused repo.push() The function was removed from localrepo in 4d52e6eb98ea (locarepo: remove the `push` method (API), 2014-09-25). Differential Revision: https://phab.mercurial-scm.org/D7174

censor: document that some commands simply ignore censored data I can't see a benefit for hg grep to ever error out early when it encounters censored data.

grep: warn on censored revisions instead of erroring out We need most of the grep logic to go through in case we encounter a censored revision, so we just return a None body for a censored node, and we stop just short of trying to record matches with the contents of that censored body. The other parts such as recording that the censored file has been considered at this revision needs to go into the proper dicts. I have also gotten weary of all the abbreviations, so while I did a small refactor to move the file-data-getting operation into a common function, I also expanded the abbreviations of the relevant variables within this little function. Hopefully some day this helps someone figure out what all the abbreviations mean. Although the censoring docs currently state that some commands error out or are ignored depending on the `censor.policy` config, I cannot see a benefit for grep to ever stop dead in its tracks when a censored revision is encountered. I will also amend the docs to indicate that some commands, such as grep, unconditionally ignore censored revisions.

py3: enable legacy fs encoding to fix filename compatibility on Windows This patch is untested. I just followed the instruction: https://docs.python.org/3/whatsnew/3.6.html#pep-529-change-windows-filesystem-encoding-to-utf-8

rust-cpython: run cargo fmt

rust-cpython: remove useless PyResult<> from leak_immutable() The caller should know if the shared data is mutably borrowed or not.

rust-cpython: remove useless PyRefMut wrapper

rust-cpython: drop manual management of mutably_borrowed RefCell::borrow() should guarantee there's no mutable borrow. As a follow up, maybe PySharedState can be a pure data structure + function. Most ref-sharing business has already been moved to PySharedRef* and PyLeaked*.

rust-cpython: leverage RefCell::borrow() to guarantee there's no mutable ref Since the underlying value can't be mutably borrowed by PyLeaked, we don't have to manage yet another mutably-borrowed state. We can just rely on the RefCell implementation. Maybe we can add try_leak_immutable(), but this patch doesn't in order to keep the patch series not too long.

rust-cpython: remove useless Option<$leaked> from py_shared_iterator We no longer need to carefully drop the iterator when it's consumed. Mutation is allowed even if the iterator exists. There's a minor behavior change: next(iter) may return/raise something other than StopIteration if it's called after the iterator has been fully consumed, and if the Rust object isn't a FusedIterator.

rust-cpython: allow mutation unless leaked reference is borrowed In other words, mutation is allowed while a Python iterator holding PyLeaked exists. The iterator will be invalidated instead. We still need a borrow_count to prevent mutation while leaked data is dereferenced in Rust world, but most leak_count business is superseded by the generation counter. decrease_leak_count(py, true) will be removed soon.

rust-cpython: add generation counter to leaked reference This counter increments on borrow_mut() to invalidate existing leaked references. This is modeled after the iterator invalidation in Python. The other checks will be adjusted by the subsequent patches.

rust-cpython: add stub wrapper that'll prevent leaked data from being mutated In order to allow mutation of PySharedRefCell value while PyLeaked reference exists, we need yet another "borrow" scope where mutation is prohibited. try_borrow<'a> and try_borrow_mut<'a> defines the "borrow" scope <'a>. The subsequent patches will implement leak counter based on this scope. PyLeakedRef<T> and PyLeakedRefMut<T> could be unified to PyLeakedRef<&T> and PyLeakedRef<&mut T> respectively, but I didn't do that since it seemed a bit weird that deref_mut() would return a mutable reference to an immutable reference.

rust-cpython: rename PyLeakedRef to PyLeaked This series will make PyLeaked* behave more like a Python iterator, which means mutation of the owner object will be allowed and the leaked reference (i.e. the iterator) will be invalidated instead. I'll add PyLeakedRef/PyLeakedRefMut structs which will represent a "borrowed" state, and prevent the underlying value from being mutably borrowed while the leaked reference is in use: let shared = self.inner_shared(py); let leaked = shared.leak_immutable(); { let leaked_ref: PyLeakedRef<_> = leaked.borrow(py); shared.borrow_mut(); // panics since the underlying value is borrowed } shared.borrow_mut(); // allowed The relation between PyLeaked* structs is quite similar to RefCell/Ref/RefMut, but the implementation can't be reused because the borrowing state will have to be shared across objects having no lifetime relation. PyLeaked isn't named as PyLeakedCell since it isn't actually a cell in that leaked.borrow_mut() will require &mut self.

fix: fix grammar/typos in hg help -e fix

progress: flush stderr after clearing On python3, ui.stderr is buffered, it seems, so we need to flush it to actually get the progress bar off the screen. This is important since ui.write() will call into progbar.clear() if it thinks there's a progress bar on the screen, with the intent that the next thing it outputs is at the beginning of the line (instead of at the end of the progress bar line). Without the flush, we buffer up the clearing of the screen, and we get some really weird/corrupt output. Differential Revision: https://phab.mercurial-scm.org/D7784

subrepo: fix a crash when archiving an svn or git subrepo Only hgsubrepos have a repository attribute. This is pretty hacky, but probably the best we can do on stable. Pushing the lfstatus check down into the wrapper for hgsubrepo (and dropping the check for lfstatus at the top of `hgsubrepoarchive()`) resulted in various test failures because: 1) hgsubrepoarchive isn't returning the number of files archived at the bottom, resulting in an error about += NoneType 2) These copypasta archive wrappers don't use progress bars 3) Largefiles are *not* currently archived when using extdiff (68822b7cdd01), but pushing this context manager down into the subrepo resulted in it apparently doing so (as evidenced by progress bars being dropped) The other uses of `lfstatus()` are not in the substate processing loop, so they shouldn't be an issue. I initially put testcases in this test for largefiles-{on,off}, and it flagged a bunch of exit code differences for `cat` and `diff`, so I backed that off. Differential Revision: https://phab.mercurial-scm.org/D7714

Added signature for changeset b4c82b704180

Added tag 5.2.2 for changeset b4c82b704180

py3: force bytestr conversion of "reason" in scmutil.callcatch() For instance, reason may be an InvalidURL as shown in added test.

rust-cpython: do not convert warning pattern to utf-8 bytes On Unix, both Rust Path and Mercurial expect a locale-dependent bytes, and we don't support Windows yet. Differential Revision: https://phab.mercurial-scm.org/D7614

rust-cpython: import utils::files::* function at module level IIRC, it's common in Rust to call functions with the module prefix. Differential Revision: https://phab.mercurial-scm.org/D7613

py3: send bytes from Rust-created warning patterns Python code expects bytes in both Python 2 and Python 3, so we should send bytes. Differential Revision: https://phab.mercurial-scm.org/D7612

py3: pass bytes to `configint` and `configbool` Both functions require bytes, even in Python 3. Differential Revision: https://phab.mercurial-scm.org/D7611

rust-threads: force Rayon to respect the worker count in config As per the inline comment, this is a workaround because Rust code does not yet know how to read config files. Differential Revision: https://phab.mercurial-scm.org/D7610

remotefilelog: remove the `ensurestart` usage The option does not seems to do what we want since it does not wait on the background process actually starting (eg: taking the lock). And the race have been dealt with another way anyway. This effectively back out changeset 2c74337e6483. Differential Revision: https://phab.mercurial-scm.org/D7590

remotefilelog: remove most of the debugwait* command in the tests Since we use a dedicated developer config, we do not need them anymore. Differential Revision: https://phab.mercurial-scm.org/D7589

remotefilelog: use the correct redirection in tests The test were using '2>%1' which just redirect stderr in the "%1" file. The intend was '2>&1' which redirect stderr on stdin. This impact tests because some commit were picking up this '%1' files. Differential Revision: https://phab.mercurial-scm.org/D7588

remotefilelog: remove most sleep in tests Now that we removed most of the races, we can remove the infamous sleeps. Differential Revision: https://phab.mercurial-scm.org/D7587

remotefilelog: have command wait for background task in the tests We leverage the feature introduced in the two previous commit to have the main command wait for the background task to be finished before returning. All test still pass and instability seems to have gone away. Most of the sleep and explicit wait are now useless and will be cleaned up in later changesets. Differential Revision: https://phab.mercurial-scm.org/D7586

remotefilelog: add a developer option to wait for background processes In order to block the main command on the subprocess exiting, we ensure the repo's ui object will call the subprocess.wait() method to ensure the top-level hg process doesn't exit until all background processes have also done so. Currently, in the tests, most operation spawning background process as followed by commands waiting for these operations to complete. However this waiting is racy. First because it seems like we can start waiting before the background operation actually start, in which case it is prematurely detected as "done". Second, because some commands may spawn multiple background operation for the same operation (eg: rebase can apparently trigger multiple prefetch). The current approach could be updated to maybe handle the first issue, but the second one will never be properly handled. In most case, we do not care that the bg process keep running after the command end. (Since we explicitly wait for them to end before doing anything else). So we add an option to wait on the background process before exiting the command. We'll put it in use in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D7585

procutil: add a option to not fully detach background process When writing test, it is useful to make code pretending to spawn a fully detached process while still actually waiting for it to finish. We add the option to `procutil.runbgcommand`, to do so. We will use this in remote-filelog tests. Differential Revision: https://phab.mercurial-scm.org/D7584

Added signature for changeset a50fecefa691

Added tag 5.2.1 for changeset a50fecefa691

windows: fix an issue causing registry config paths to be ignored on py3 `util.lookupreg()` returns bytes. Differential Revision: https://phab.mercurial-scm.org/D7532

tests: cover revision conversion logic in githelp tests There was no test involving actual conversion of option values when they contain a git revision name (to be converted as a hg one by hgext.githelp.convert()). Adding one. This test would fail on Python 3 without https://phab.mercurial-scm.org/D7537. Differential Revision: https://phab.mercurial-scm.org/D7540

pvec: fix a `str` type conditional for py3 Differential Revision: https://phab.mercurial-scm.org/D7538

githelp: fix a `str` type conditional for py3 Differential Revision: https://phab.mercurial-scm.org/D7537

histedit: fix an `isinstance(nodelist, str)` check for py3 Differential Revision: https://phab.mercurial-scm.org/D7536

win32mbcs: fix a `str` type conditional for py3 Differential Revision: https://phab.mercurial-scm.org/D7535

hgweb: fix a few `str` type conditional for py3 Differential Revision: https://phab.mercurial-scm.org/D7534

repair: fix an `isinstance(nodelist, str)` check for py3 All of the callers appear to pass a list, so this doesn't fix anything in core hg. But maybe out of tree extensions use this shortcut. Differential Revision: https://phab.mercurial-scm.org/D7533

log: map None rev to wdirrev when filtering revisions with --line-range When 'hg log -f --line-range <file>,<range>' is invoked with <range> containing uncommitted changes, the command crashes on Python 3 as follows: [...] File "/usr/lib/python3/dist-packages/mercurial/commands.py", line 4725, in log revs, differ = logcmdutil.getlinerangerevs(repo, revs, opts) File "/usr/lib/python3/dist-packages/mercurial/logcmdutil.py", line 933, in getlinerangerevs if rev not in userrevs: File "/usr/lib/python3/dist-packages/mercurial/smartset.py", line 969, in __contains__ if l < x: TypeError: '<' not supported between instances of 'int' and 'NoneType' The None value is because requested line range has uncommitted changes, so 'rev' is the working directory revision. This only occurs in Python 3 as Python 2 allows comparing None with int. As suggested by Yuya Nishihara, mapping None to node.wdirrev resolves the issue and also make the '--line-range' option properly work with -r 'wdir()'. We add extra tests for non-regression and to illustrate handling of 'wdir()'.

tests: check that 'log --line-range' follows uncommitted changes The reason we start walking revisions from the working directory (None revision) in logcmdutil.getlinerangerevs() is because we can follow uncommitted changes. Adding a test to illustrate this based on an uncommitted rename as there was none before. This helps understand the fix in next changeset.

test: don't put $BINDIR in $PATH for test-merge-tools.t We call $BINDIR/hg explicitly anyway, so don't need it in $PATH. This fixes failures when running the test --with-hg=/usr/bin/hg, where we pick up /usr/bin/false as merge tool when we expected not to find it.

webutil: add missing argument to join() Differential Revision: https://phab.mercurial-scm.org/D7516

singlehead: making config item a bool again with the use of `configsuboptions`, the main config item has become a string (unless it's just the default value). This makes it in particular hard to override in a cascade of HGRC files, as we do in Heptapod to re-allow multiple heads on specific repositories while the default behaviour is to forbid them. The added test case reflects that use-case

util: convert an exception to bytes when passing to Abort() I happened to notice this searching for how to convert an exception to bytes in the previous patch. I'm pretty sure I've got a bunch of other instances that use `pycompat.bytestr()` suppressed locally. Differential Revision: https://phab.mercurial-scm.org/D7467

patch: fix a str + bytes issue in an exception handler Flagged by pytype. Differential Revision: https://phab.mercurial-scm.org/D7466

py3: wrap a __func__ in sysbytes() before logging as bytes Differential Revision: https://phab.mercurial-scm.org/D7461

py3: make doc strings containing deprecated '\.' escape sequence raw strings Differential Revision: https://phab.mercurial-scm.org/D7462

shelve: add the missing `create` parameter to the bundlerepo constructor Caught by pytype. Differential Revision: https://phab.mercurial-scm.org/D7458

shelve: fix a missing variable in the exception handler for delete Caught by pytype. I haven't paid much attention to the progress of this extension, but I *think* this was the intent. Differential Revision: https://phab.mercurial-scm.org/D7457