peer: remove non iterating batcher (API) The last use of this API was removed in b6e71f8af5b8 in 2016. While not formally deprecated, as of the last commit the code is no longer explicitly tested. I think the new API has existed long enough for people to transition to it. I also have plans to more formalize the peer API and removing batch() makes that work easier. I'm not convinced the current client-side API around batching is great. But it's the best we have at the moment. .. api:: remove peer.batch() Replace with peer.iterbatch(). Differential Revision: https://phab.mercurial-scm.org/D320

wireproto: overhaul iterating batcher code (API) The remote batching code is difficult to read. Let's improve it. As part of the refactor, the future returned by method calls on batchiter() instances is now populated. However, you still need to consume the results() generator for the future to be set. But at least now we can stuff the future somewhere and not have to worry about aligning method call order with result order since you can use a future to hold the result. Also as part of the change, we now verify that @batchable generators yield exactly 2 values. In other words, we enforce their API. The non-iter batcher has been unused since b6e71f8af5b8. And to my surprise we had no explicit unit test coverage of it! test-batching.py has been overhauled to use the iterating batcher. Since the iterating batcher doesn't allow non-batchable method calls nor local calls, tests have been updated to reflect reality. The iterating batcher has been used for multiple releases apparently without major issue. So this shouldn't cause alarm. .. api:: @peer.batchable functions must now yield exactly 2 values Differential Revision: https://phab.mercurial-scm.org/D319

wireproto: remove support for local results in @batchable (API) @peer.batchable decorated generator functions have two forms: yield value, None and yield args, future yield value These forms have been present since the decorator was introduced. There are currently no in-repo consumers of the first form. So this commit removes support for it. Note that remoteiterbatcher.submit() asserts the 2nd form. And b6e71f8af5b8 removed the last user of remotebatcher, forcing everyone to remoteiterbatcher. So anything relying on this in the wild would have been broken since b6e71f8af5b8. .. api:: @peer.batchable can no longer emit local values Differential Revision: https://phab.mercurial-scm.org/D318

wireproto: properly implement batchable checking remoteiterbatcher (unlike remotebatcher) only supports batchable commands. This claim can be validated by comparing their implementations of submit() and noting how remoteiterbatcher assumes the invoked method has a "batchable" attribute, which is set by @peer.batchable. remoteiterbatcher has a custom __getitem__ that was trying to validate that only batchable methods are called. However, it was only validating that the called method exists, not that it is batchable. This wasn't a big deal since remoteiterbatcher.submit() would raise an AttributeError attempting to `mtd.batchable(...)`. Let's fix the check and convert it to ProgrammingError, which may not have been around when this was originally implemented. Differential Revision: https://phab.mercurial-scm.org/D317

largefiles: remove remotestore.batch() This method was added in 9e1616307c4c. AFAICT it didn't do anything at inception. If it did, there was no test coverage for it because changing it to raise doesn't fail any tests at that revision. b6e71f8af5b8 later refactored all remote.batch() calls to remote.iterbatch(). So if this was somehow used, it isn't called any more because there are no calls to .batch() remaining in the repo. I suspect the original patch author got confused by the distinction between the peer/remote interface and the largefiles store. The lf store is a gateway to a peer instance. It exposes additional lf-specific methods to execute against a peer. However, it is not a peer and doesn't need to implement batch() because peer itself does that. Differential Revision: https://phab.mercurial-scm.org/D316

histedit: check first changeset for verb "roll" or "fold" (issue5498) If someone changes "pick" to "roll" or "fold" for the first changeset in a histedit rule Mercurial could remove a wrong changeset if the phase is non-public. roll or fold for the first changeset should be invalid.

encoding: drop circular import by proxying through '<policy>.charencode' I decided not to split charencode.c to new C extension module because it would duplicate binary codes unnecessarily.

policy: reroute proxy modules internally This allows us to split encoding functions from pure.parsers without doing that for cext.parsers. See the next patch for why.

cext: modernize charencode.c to use Py_ssize_t

cext: factor out header for charencode.c This merges a part of util.h with the header which should exist for charencode.c.

cext: split character encoding functions to new compilation unit This extracts charencode.c from parsers.c, which seems big enough for me to hesitate to add new JSON functions. Still charencode.o is linked to parsers.so to avoid duplication of binary codes.

cext: move _dict_new_presized() to header Prepares for splitting encoding functions from parsers.c.

ui: restore behavior to ignore some I/O errors (issue5658) e9646ff34d55 and 1bfb9a63b98e refactored ui methods to no longer silently swallow some IOError instances. This is arguably the correct thing to do. However, it had the unfortunate side-effect of causing StdioError to bubble up to sensitive code like transaction aborts, leading to an uncaught exceptions and failures to e.g. roll back a transaction. This could occur when a remote HTTP or SSH client connection dropped. The new behavior is resulting in semi-frequent "abandonded transaction" errors on multiple high-volume repositories at Mozilla. This commit effectively reverts e9646ff34d55 and 1bfb9a63b98e to restore the old behavior. I agree with the principle that I/O errors shouldn't be ignored. That makes this change... unfortunate. However, our hands are tied for what to do on stable. I think the proper solution is for the ui's behavior to be configurable (possibly via a context manager). During critical sections like transaction rollback and abort, it should be possible to suppress errors. But this feature would not be appropriate on stable.

tests: test behavior of IOError during transactions (issue5658) ui._write(), ui._write_err(), and ui.flush() all trap IOError and re-raise as error.StdioError. If a caller doesn't catch StdioError when writing to stdio, it could bubble all the way to dispatch. This commit adds tests for I/O failures around various transaction operations. The most notable badness is during abort. Here, an uncaught StdioError will result in incomplete transaction rollback, requiring an `hg rollback` to recover. This can result in a client "corrupting" a remote repo via terminated HTTP and SSH socket.

log: mention ui.logtemplate in the help text

hg: avoid relying on errno numbers / descriptions A few tests hardcode errno numbers and/or descriptions in the output, causing test failures on platforms where these values are different. Differential Revision: https://phab.mercurial-scm.org/D362

hg: tolerate long vs. int in test-context.py The file times here can be longs instead of ints on some platforms, which will cause a test failure due to these printing with an L suffix; instead always format with %d which will produce the same output in either case. Differential Revision: https://phab.mercurial-scm.org/D361

configitems: register the 'notify.test' config

configitems: register the 'notify.template' config

configitems: register the 'notify.style' config

configitems: register the 'notify.strip' config

configitems: register the 'notify.sources' config

configitems: register the 'notify.merge' config

configitems: register the 'notify.mbox' config

configitems: register the 'notify.maxsubject' config

configitems: register the 'notify.maxdiff' config

configitems: register the 'notify.fromauthor' config

configitems: register the 'notify.domain' config

configitems: register the 'notify.diffstat' config

configitems: register the 'notify.config' config

merge with stable

merge with stable

obsutil: rename allprecursors into allpredecessors Use util.nouideprecwarn because obsstore doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D247

obsolete: rename precursor into predecessor in obsolete docstrings The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D246

obsstore: rename precursors into predecessors Use util.nouideprecwarn because obsstore doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D245

obsolete: rename _addprecursors into _addpredecessors Use util.nouideprecwarn because _addpredecessors doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D244

obsmarker: rename precnode into prednode Rename prednode (predecessors node) into precnode (precursors node) in markers class. Use util.nouideprecwarn because markers doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D243

context: rename troubled into isunstable As we changed the meaning of unstable between the old vocabulary and the new one, we can't reuse the unstable method name at the risk of breaking extensions calling unstable and getting a wrong result. Instead rename troubled into isunstable so extensions will continue to work. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D242

context: rename bumped into phasedivergent Rename bumped context method into phasedivergent and add a deprecation warning on bumped. Only update all callers to keep the patch straightforward. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D241

context: rename divergent into contentdivergent Rename divergent context method into contentdivergent and add a deprecation warning on divergent. Only update all callers to keep the patch straightforward. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D240

context: rename unstable into orphan Rename unstable context method into orphan and add a deprecation warning on unstable. Only update all callers to keep the patch straightforward. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D239

context: rename troubles into instabilities Rename troubles context method into instabilities. Copy the old troubles method and add a deprecation warning. This way extensions calling troubles will see the deprecation warning but will not break due to new return values. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D238

repair: move manifest strip to a separate function This moves manifest stripping to a separate function so implementations of the manifest that don't support stripping can replace this function with a no-op. I considered adding a strip api to the manifestlog, so other implementations could make it a no-op there, but it seems like strip might be unique to the revlog implementation, and therefore shouldn't be present on the generic api. Differential Revision: https://phab.mercurial-scm.org/D292

repair: refactor broken linkrev collection This refactors broken linkrev collection such that manifest collection is in a separate function. This allows extensions to replace the manifest collection with a non-revlog oriented version. I considered moving the collect changes function onto the manifestlog itself, so it would be behind the abstraction, but since the store we're building doesn't even have the concept of strip, embeding that concept in the manifestlog api seemed odd. Differential Revision: https://phab.mercurial-scm.org/D291

tests: update duplicateoptions test to use bytestrings everywhere Test now passes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D297

extensions: attempt to use non-deprecated inspect method Avoids some deprecation warnings when extension loading breaks. Differential Revision: https://phab.mercurial-scm.org/D295

tests: clean up many print statements to be print functions instead Differential Revision: https://phab.mercurial-scm.org/D294

tests: fix simple heredoc print statements to work on Py3 There are a handful left that require some more care. Differential Revision: https://phab.mercurial-scm.org/D293

obsolete: use bytes() instead of str() so the node is bytes on py3 I'm not sure this is right, since this should either be bytes or str to match what's going on in the revlog layer. Differential Revision: https://phab.mercurial-scm.org/D271

tests: fix test-notify.t to use $PYTHON In doing so, I consolidated the filtering logic. This resulted in some small test output changes, but I think the consistency throughout the test is worth it. Differential Revision: https://phab.mercurial-scm.org/D279

tests: make test-highlight code portable to python3 This is easier than trying to do some sort of check-code shenanigans. Differential Revision: https://phab.mercurial-scm.org/D278

dagparser: make print statement in doctest Py3 portable Differential Revision: https://phab.mercurial-scm.org/D277

i18n: fix check-translation.py to be less broken on Python 3 These are all simple one-argument print statements, so this syntax works the same way in 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D276

ui: refactor extractchoices so it doesn't break on Python 3 Differential Revision: https://phab.mercurial-scm.org/D275

obsutil: defend against succsmarkers() returning None I'm not sure if this is a realistic problem, but doing this avoids some pretty awful test failures on Python 3, and it looks like it should be harmless. Differential Revision: https://phab.mercurial-scm.org/D274

changegroup: more **kwargs Differential Revision: https://phab.mercurial-scm.org/D273

bundle2: convert ints to strings using pycompat.bytestring() Fixes some Python 3 regressions. We don't use %d here because the part id is actually an Optional[int]. It should always be initialized to a non-None value by the time this code executes, but we shouldn't blindly depend on that being the case. Differential Revision: https://phab.mercurial-scm.org/D272

tests: fix up test-run-tests failures on Python 3.6 Some of the recent work on run-tests.py didn't work on Python 3. This fixes the regressions. Differential Revision: https://phab.mercurial-scm.org/D270

template: rename troubles templatekw into instabilities Rename troubles template keyword into instabilities and add a deprecation warning on templatekw. Update default mapfile and test files to use the new template keyword. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D237

test-rebase: add a brute force test Rebase is becoming more complex and it looks like a good idea to try some brute force enumeration to cover cases that are hard to find manually. Using brute force to generate repos in different shapes and enumerating the rebase source and destination would generate too many cases that takes too long to compute. This patch limits the "brute force" to only the "rebase source" part. Repo and destination are still manual. The added test cases are crafted manually to reveal some behaviors that are not covered by other tests: - "revlog index out of range" crash - after rebase, p1 == p2, p2 != null - "nothing to merge" abort In the future we might want to add more tests here. For now I'm more interested in revealing interesting behaviors in a minified way. I tried some more complex cases but didn't find other interesting behaviors. Differential Revision: https://phab.mercurial-scm.org/D262

dirstate: simplify dirstate's __iter__ Probably also a performance win, but not measurable in perfdirstate. Differential Revision: https://phab.mercurial-scm.org/D269

repo: skip invalidation of changelog if it has 'delayed' changes (API) The changelog object can store recently added revisions in memory until the transaction is committed. We don't want to lose those changes even if repo.invalidate(clearfilecache=True), so let's skip the changelog when it has such 'delayed' changes. Differential Revision: https://phab.mercurial-scm.org/D152

httppeer: make several instance attributes internal (API) Peer instances are supposed to conform to a well-defined API so consumers can be agnostic about the underlying peer type. To reinforce this, this commit renames a handful of instance attributes on httpeer so they no longer have a "public" name. Differential Revision: https://phab.mercurial-scm.org/D268

httppeer: remove unused handler attribute The consumer of this attribute was removed by b6bd4ee6ed85 in 2008. Differential Revision: https://phab.mercurial-scm.org/D267