scmutil: use util.shellquote instead of %r Changes some output, but also resolves differences with Python 3. Differential Revision: https://phab.mercurial-scm.org/D301

context: fix troubled deprecation troubled has been renamed into isunstable but troubled was calling unstable instead. Fix the mistake. Differential Revision: https://phab.mercurial-scm.org/D384

test-pushvars: invoke shell script hook via `sh` for Windows Invoking *.sh on Windows leads to the "what program should open this?" prompt, which stalls the test and led to the recent series of exceptions on the Windows test machine as the runner times out.

test-sparse: drop unnecessary globs

exchange: simplify unbundle locking using context managers Differential Revision: https://phab.mercurial-scm.org/D393

util: add base class for transactional context managers We have at least three types with a close() and a release() method where the close() method is supposed to be called on success and the release() method is supposed to be called last, whether successful or not. Two of them (transaction and dirstateguard) already have identical implementations of __enter__ and __exit__. Let's extract a base class for this, so we reuse the code and so the third type (transactionmanager) can also be used as a context manager. Differential Revision: https://phab.mercurial-scm.org/D392

exchange: remove need for "locked" variable The transactionmanager() constructor just assigned a few variables and cannot fail, so it's safe to move it inside the earlier try/except. Differential Revision: https://phab.mercurial-scm.org/D391

exchange: drop now-unnecessary "local" from lock name variables Since fda0867cfe03 (exchange: drop support for lock-based unbundling (BC), 2017-08-06), there is no more remote locking. Differential Revision: https://phab.mercurial-scm.org/D390

exchange: remove 'locallocked' member from pushop object The variable has been used only within a single function since 94e2862dbcfb (push: elevate phase transaction to cover entire operation, 2014-11-21), so there's no need to keep it on the pushop object. Differential Revision: https://phab.mercurial-scm.org/D389

rebase: change "result would have 3 parent" error message (BC) The old error message "cannot use revision REV as base, result would have 3 parents" is confusing - why use REV as base? why add a new parent?. This patch changes it to "cannot move parent", which seems better. Differential Revision: https://phab.mercurial-scm.org/D342

drawdag: allow override file contents via comments This makes drawdag more flexible, and allow us to create non-clean merges in test cases. Differential Revision: https://phab.mercurial-scm.org/D385

phabricator: add --amend option to phabsend Previously `hg phabsend` was imitating `hg email` and won't mutate changesets. That works fine with reviewer-push workflow, reviewers run `phabread`, `import`. However, it does not work well with author-push workflow. Namely, the author needs to run extra commands to get the right commit message, and remove the local tag after push. This patch solves those issues by adding the `--amend` option, so local changesets will have the right commit message, and tags become unnecessary. Test Plan: Given the following DAG: o 17 o 16 | o 15 | @ 14 |/ o 13 o 12 Run `hg phabsend '(13::)-17' --amend`, check the new DAG looks like: o 21 | o 20 | @ 19 |/ o 18 | o 17 | x 16 | x 13 |/ o 12 And commit messages are updated to contain the `Differential Revision` lines. Use `phabread` to make sure Phabricator has the amended node recorded. Also check `phabsend .` followed by a `phabsend . --amend`, the commit message will be updated and the tag will be removed. Differential Revision: https://phab.mercurial-scm.org/D122

rebase: rewrite core algorithm (issue5578) (issue5630) "defineparents" is the core algorithm of rebase. The old code has too many tech debts (like outdated comments, confusing assertions, etc) and is very error-prone to be improved. This patch rewrites "defineparents" to make the code easier to reason about, and solve a bunch of issues, including: - Assertion error: no base found (demonstrated by D212, issue5578) - Asymmetric result (demonstrated by D211, "F with one parent") - Wrong new parent (demonstrated by D262, "C':A'A'") - "revlog index out of range" (demonstrated by D262, issue5630) - "nothing to merge" (demonstrated by D262) As a side effect, merge base decision has been made more solid - rebase now prints out explicitly what could go wrong when it cannot find a unique suitable merge base. .. fix:: Issue 5578, Issue 5630 Core rebase algorithm has been rewritten to be more robust. Differential Revision: https://phab.mercurial-scm.org/D21

phabricator: remove an unnecessary writediffproperties call This was introduced by D229. Thanks Yuya for finding it! Differential Revision: https://phab.mercurial-scm.org/D366

releasenotes: minor bug fix for index error while serializing Differential Revision: https://phab.mercurial-scm.org/D356

label: rename log.trouble into log.instability The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D259

label: rename trouble.X into instability.X The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D258

label: rename changeset.troubled into changeset.unstable The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D257

test: add a test for stabilization related labels Differential Revision: https://phab.mercurial-scm.org/D256

obsolete: rename bumped volatile set into phasedivergent volatile set The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D255

obsolete: rename divergent volatile set into contentdivergent volatile set The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D254

obsolete: rename unstable volatile set into orphan volatile set The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D253

revset: rename bumped into phasedivergent Don't touch bumped volatile set name, only the revset name. The volatile set name will be updated in a later patch. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D252

revset: remane divergent into contentdivergent Don't touch divergent volatile set name, only the revset name. The volatile set name will be updated in a later patch. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D251

revset: rename unstable into orphan Don't touch unstable volatile set name, only the revset name. The volatile set name will be updated in a later patch. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D250

test: update evolution config evolution* config has been rewritten in stabilization* in the previous patch, update tests file to use the new names. Differential Revision: https://phab.mercurial-scm.org/D249

config: rename evolution config into stabilization Use aliases for backward-compatibility. Though I'm not sure how to emit compatibility warnings with aliases. Test configuration are updated in the next patch. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D248

morestatus: move fb extension to core by plugging to `hg status --verbose` morestatus extension in fbext use to show more context about the state of the repo like the repository is in a unfinished merge state, or a rebase is going on, or histedit is going on, listing the files which need to be resolved and also suggesting ways to handle the situation. This patch moves the extension directly to core by plugging it into the --verbose flag of the status command. So now if you are in any unfinished state and you do hg status -v, it will show you details and help related to the state. The extension in fbext also shows context about unfinished update state which is not ported to core as that plug in hooks to update command which need to be tackled somewhat differently. The following configuration will turn the behaviour on by default [commands] status.verbose = 1 You can also skip considering some states like bisect as follows: [commands] status.skipstates=bisect This patch also adds test for the feature. .. feature:: ``hg status -v`` can now show unfinished state. For example, when in an unfinished rebase state, ``hg status -v`` might show:: # The repository is in an unfinished *rebase* state. # No unresolved merge conflicts. # To continue: hg rebase --continue # To abort: hg rebase --abort Differential Revision: https://phab.mercurial-scm.org/D219

bundle2: fix transaction availability detection Changeset 5fc4ddfbe626 introduce more complex logic around 'bundleoperation.gettransaction'. In that process it turns the old "attribute" into a proper method which breaks the code that detects the "transaction availability". The change was visible in 'test-acl.t', fixing this reverts the test changes. Differential Revision: https://phab.mercurial-scm.org/D303

fsmonitor: correct an error message Without the change, the error looks like: warning: Watchman unavailable: "watchman" executable not in PATH (%s), while executing [Errno 2] No such file or directory With the change, it now looks like: warning: Watchman unavailable: "watchman" executable not in PATH ([Errno 2] No such file or directory) Differential Revision: https://phab.mercurial-scm.org/D322

sshpeer: make instance attributes and methods internal Peer types are supposed to conform to a formal interface defined by peer.peerrepository and wireproto.wirepeer. Every "public" attribute on *peer types makes it harder to understand what attributes are part of the interface and what are instance specific. This commit converts a number of "public" instance attributes and methods on sshpeer to internal so they can't be confused to be part of the peer API. The URL-related instance attributes were introduced in 876333a295ff in 2005. AFAICT most of them aren't used and could potentially be removed. But I kept them around anyway. I also reorded some code to make things slightly easier to read. .. api:: Rename attributes on sshpeer to reflect peer API Differential Revision: https://phab.mercurial-scm.org/D331

peer: remove non iterating batcher (API) The last use of this API was removed in b6e71f8af5b8 in 2016. While not formally deprecated, as of the last commit the code is no longer explicitly tested. I think the new API has existed long enough for people to transition to it. I also have plans to more formalize the peer API and removing batch() makes that work easier. I'm not convinced the current client-side API around batching is great. But it's the best we have at the moment. .. api:: remove peer.batch() Replace with peer.iterbatch(). Differential Revision: https://phab.mercurial-scm.org/D320

wireproto: overhaul iterating batcher code (API) The remote batching code is difficult to read. Let's improve it. As part of the refactor, the future returned by method calls on batchiter() instances is now populated. However, you still need to consume the results() generator for the future to be set. But at least now we can stuff the future somewhere and not have to worry about aligning method call order with result order since you can use a future to hold the result. Also as part of the change, we now verify that @batchable generators yield exactly 2 values. In other words, we enforce their API. The non-iter batcher has been unused since b6e71f8af5b8. And to my surprise we had no explicit unit test coverage of it! test-batching.py has been overhauled to use the iterating batcher. Since the iterating batcher doesn't allow non-batchable method calls nor local calls, tests have been updated to reflect reality. The iterating batcher has been used for multiple releases apparently without major issue. So this shouldn't cause alarm. .. api:: @peer.batchable functions must now yield exactly 2 values Differential Revision: https://phab.mercurial-scm.org/D319

wireproto: remove support for local results in @batchable (API) @peer.batchable decorated generator functions have two forms: yield value, None and yield args, future yield value These forms have been present since the decorator was introduced. There are currently no in-repo consumers of the first form. So this commit removes support for it. Note that remoteiterbatcher.submit() asserts the 2nd form. And b6e71f8af5b8 removed the last user of remotebatcher, forcing everyone to remoteiterbatcher. So anything relying on this in the wild would have been broken since b6e71f8af5b8. .. api:: @peer.batchable can no longer emit local values Differential Revision: https://phab.mercurial-scm.org/D318

wireproto: properly implement batchable checking remoteiterbatcher (unlike remotebatcher) only supports batchable commands. This claim can be validated by comparing their implementations of submit() and noting how remoteiterbatcher assumes the invoked method has a "batchable" attribute, which is set by @peer.batchable. remoteiterbatcher has a custom __getitem__ that was trying to validate that only batchable methods are called. However, it was only validating that the called method exists, not that it is batchable. This wasn't a big deal since remoteiterbatcher.submit() would raise an AttributeError attempting to `mtd.batchable(...)`. Let's fix the check and convert it to ProgrammingError, which may not have been around when this was originally implemented. Differential Revision: https://phab.mercurial-scm.org/D317

largefiles: remove remotestore.batch() This method was added in 9e1616307c4c. AFAICT it didn't do anything at inception. If it did, there was no test coverage for it because changing it to raise doesn't fail any tests at that revision. b6e71f8af5b8 later refactored all remote.batch() calls to remote.iterbatch(). So if this was somehow used, it isn't called any more because there are no calls to .batch() remaining in the repo. I suspect the original patch author got confused by the distinction between the peer/remote interface and the largefiles store. The lf store is a gateway to a peer instance. It exposes additional lf-specific methods to execute against a peer. However, it is not a peer and doesn't need to implement batch() because peer itself does that. Differential Revision: https://phab.mercurial-scm.org/D316

histedit: check first changeset for verb "roll" or "fold" (issue5498) If someone changes "pick" to "roll" or "fold" for the first changeset in a histedit rule Mercurial could remove a wrong changeset if the phase is non-public. roll or fold for the first changeset should be invalid.

encoding: drop circular import by proxying through '<policy>.charencode' I decided not to split charencode.c to new C extension module because it would duplicate binary codes unnecessarily.

policy: reroute proxy modules internally This allows us to split encoding functions from pure.parsers without doing that for cext.parsers. See the next patch for why.

cext: modernize charencode.c to use Py_ssize_t

cext: factor out header for charencode.c This merges a part of util.h with the header which should exist for charencode.c.

cext: split character encoding functions to new compilation unit This extracts charencode.c from parsers.c, which seems big enough for me to hesitate to add new JSON functions. Still charencode.o is linked to parsers.so to avoid duplication of binary codes.

cext: move _dict_new_presized() to header Prepares for splitting encoding functions from parsers.c.

configitems: register the 'notify.test' config

configitems: register the 'notify.template' config

configitems: register the 'notify.style' config

configitems: register the 'notify.strip' config

configitems: register the 'notify.sources' config

configitems: register the 'notify.merge' config

configitems: register the 'notify.mbox' config

configitems: register the 'notify.maxsubject' config

configitems: register the 'notify.maxdiff' config

configitems: register the 'notify.fromauthor' config

configitems: register the 'notify.domain' config

configitems: register the 'notify.diffstat' config

configitems: register the 'notify.config' config

merge with stable

merge with stable

obsutil: rename allprecursors into allpredecessors Use util.nouideprecwarn because obsstore doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D247

obsolete: rename precursor into predecessor in obsolete docstrings The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D246

obsstore: rename precursors into predecessors Use util.nouideprecwarn because obsstore doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D245

obsolete: rename _addprecursors into _addpredecessors Use util.nouideprecwarn because _addpredecessors doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D244

obsmarker: rename precnode into prednode Rename prednode (predecessors node) into precnode (precursors node) in markers class. Use util.nouideprecwarn because markers doesn't have easy access to an ui object. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D243

context: rename troubled into isunstable As we changed the meaning of unstable between the old vocabulary and the new one, we can't reuse the unstable method name at the risk of breaking extensions calling unstable and getting a wrong result. Instead rename troubled into isunstable so extensions will continue to work. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D242

context: rename bumped into phasedivergent Rename bumped context method into phasedivergent and add a deprecation warning on bumped. Only update all callers to keep the patch straightforward. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D241

context: rename divergent into contentdivergent Rename divergent context method into contentdivergent and add a deprecation warning on divergent. Only update all callers to keep the patch straightforward. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D240

context: rename unstable into orphan Rename unstable context method into orphan and add a deprecation warning on unstable. Only update all callers to keep the patch straightforward. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D239

context: rename troubles into instabilities Rename troubles context method into instabilities. Copy the old troubles method and add a deprecation warning. This way extensions calling troubles will see the deprecation warning but will not break due to new return values. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D238

repair: move manifest strip to a separate function This moves manifest stripping to a separate function so implementations of the manifest that don't support stripping can replace this function with a no-op. I considered adding a strip api to the manifestlog, so other implementations could make it a no-op there, but it seems like strip might be unique to the revlog implementation, and therefore shouldn't be present on the generic api. Differential Revision: https://phab.mercurial-scm.org/D292

repair: refactor broken linkrev collection This refactors broken linkrev collection such that manifest collection is in a separate function. This allows extensions to replace the manifest collection with a non-revlog oriented version. I considered moving the collect changes function onto the manifestlog itself, so it would be behind the abstraction, but since the store we're building doesn't even have the concept of strip, embeding that concept in the manifestlog api seemed odd. Differential Revision: https://phab.mercurial-scm.org/D291

tests: update duplicateoptions test to use bytestrings everywhere Test now passes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D297

extensions: attempt to use non-deprecated inspect method Avoids some deprecation warnings when extension loading breaks. Differential Revision: https://phab.mercurial-scm.org/D295

tests: clean up many print statements to be print functions instead Differential Revision: https://phab.mercurial-scm.org/D294

tests: fix simple heredoc print statements to work on Py3 There are a handful left that require some more care. Differential Revision: https://phab.mercurial-scm.org/D293

obsolete: use bytes() instead of str() so the node is bytes on py3 I'm not sure this is right, since this should either be bytes or str to match what's going on in the revlog layer. Differential Revision: https://phab.mercurial-scm.org/D271

tests: fix test-notify.t to use $PYTHON In doing so, I consolidated the filtering logic. This resulted in some small test output changes, but I think the consistency throughout the test is worth it. Differential Revision: https://phab.mercurial-scm.org/D279

tests: make test-highlight code portable to python3 This is easier than trying to do some sort of check-code shenanigans. Differential Revision: https://phab.mercurial-scm.org/D278

dagparser: make print statement in doctest Py3 portable Differential Revision: https://phab.mercurial-scm.org/D277

i18n: fix check-translation.py to be less broken on Python 3 These are all simple one-argument print statements, so this syntax works the same way in 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D276

ui: refactor extractchoices so it doesn't break on Python 3 Differential Revision: https://phab.mercurial-scm.org/D275

obsutil: defend against succsmarkers() returning None I'm not sure if this is a realistic problem, but doing this avoids some pretty awful test failures on Python 3, and it looks like it should be harmless. Differential Revision: https://phab.mercurial-scm.org/D274

changegroup: more **kwargs Differential Revision: https://phab.mercurial-scm.org/D273

bundle2: convert ints to strings using pycompat.bytestring() Fixes some Python 3 regressions. We don't use %d here because the part id is actually an Optional[int]. It should always be initialized to a non-None value by the time this code executes, but we shouldn't blindly depend on that being the case. Differential Revision: https://phab.mercurial-scm.org/D272

tests: fix up test-run-tests failures on Python 3.6 Some of the recent work on run-tests.py didn't work on Python 3. This fixes the regressions. Differential Revision: https://phab.mercurial-scm.org/D270

template: rename troubles templatekw into instabilities Rename troubles template keyword into instabilities and add a deprecation warning on templatekw. Update default mapfile and test files to use the new template keyword. The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D237

test-rebase: add a brute force test Rebase is becoming more complex and it looks like a good idea to try some brute force enumeration to cover cases that are hard to find manually. Using brute force to generate repos in different shapes and enumerating the rebase source and destination would generate too many cases that takes too long to compute. This patch limits the "brute force" to only the "rebase source" part. Repo and destination are still manual. The added test cases are crafted manually to reveal some behaviors that are not covered by other tests: - "revlog index out of range" crash - after rebase, p1 == p2, p2 != null - "nothing to merge" abort In the future we might want to add more tests here. For now I'm more interested in revealing interesting behaviors in a minified way. I tried some more complex cases but didn't find other interesting behaviors. Differential Revision: https://phab.mercurial-scm.org/D262

dirstate: simplify dirstate's __iter__ Probably also a performance win, but not measurable in perfdirstate. Differential Revision: https://phab.mercurial-scm.org/D269

repo: skip invalidation of changelog if it has 'delayed' changes (API) The changelog object can store recently added revisions in memory until the transaction is committed. We don't want to lose those changes even if repo.invalidate(clearfilecache=True), so let's skip the changelog when it has such 'delayed' changes. Differential Revision: https://phab.mercurial-scm.org/D152

httppeer: make several instance attributes internal (API) Peer instances are supposed to conform to a well-defined API so consumers can be agnostic about the underlying peer type. To reinforce this, this commit renames a handful of instance attributes on httpeer so they no longer have a "public" name. Differential Revision: https://phab.mercurial-scm.org/D268

httppeer: remove unused handler attribute The consumer of this attribute was removed by b6bd4ee6ed85 in 2008. Differential Revision: https://phab.mercurial-scm.org/D267

localrepo: remove unused requirements attributes on localpeer (API) The previous changeset removed the last consumer of requirements. I'm not sure when supportedformats became unused. But I couldn't find any obvious instances where it is being used. It likely stems from peers being derived from repository instances several years ago and is a holdover from that day. Differential Revision: https://phab.mercurial-scm.org/D266

exchange: access requirements on repo instead of peer As part of formalizing the peer interface, I audited for attribute accesses for non-internal names to find API violations. This uncovered the code changed in this commit. localpeer.requirements is just an alias to the repo's requirements attribute. So, change the code to get the data from the source instead of relying on a one-off attribute in the localpeer type. Differential Revision: https://phab.mercurial-scm.org/D265

exchange: drop support for lock-based unbundling (BC) Locking over the wire protocol and the "addchangegroup" wire protocol command has been deprecated since e8c4f3d3df8c, which was first part of Mercurial 0.9.1. Support for handling these commands from sshserver was dropped in 9f6e0e7ef828 in 2015, effectively locking out pre 0.9.1 clients from new servers. However, client-side code for calling lock and addchangegroup is still present in exchange.py and the various peer classes to facilitate pushing to pre 0.9.1 servers. The lock-based pushing mechanism is extremely brittle. 0.9.1 was released in July 2006 and I highly doubt anyone is still running such an ancient version of Mercurial on a server. I'm about to refactor the peer API and I don't think it is worth keeping support for this ancient protocol feature. So, this commit removes client support for the lock-based pushing mechanism. This means modern clients will no longer be able to push to pre 0.9.1 servers. Differential Revision: https://phab.mercurial-scm.org/D264

http: add a test of actually pushing with httppostargs This was previously untested. Sigh. Differential Revision: https://phab.mercurial-scm.org/D230

filemerge: move decorator definition for internal merge tools to registrar This patch also adds extra loading entry for internal merge tools to extensions.py, for similarity to other decorators defined in registrar.py. This patch uses "internalmerge" for decorator class name, instead of original "internaltool", because the latter is too generic. BTW, after this patch, 4-spaces indentation is added to the 1st line of internal merge tool description docstring, and this may make already translated entries in *.po fuzzy. Even though this indentation is required for "definition list" in reST syntax, absence of it has been overlooked, because help.makeitemsdoc() forcibly inserts it at generation of online help. But this forcible insertion causes formatting issue (I'll send another patch series for this). Therefore, this additional indentation should be reasonable.

releasenotes: add similarity check function to compare incoming notes It is possible that the incoming note fragments have some similar content as the existing release notes. In case of a bug fix, we match for issueNNNN in the existing notes. For other general cases, it makes use of fuzzywuzzy library to get a similarity score. If the score is above a certain threshold, we ignore the fragment, otherwise add it. But the score might be misleading for small commit messages. So, it uses similarity function only if the length of string (in words) is above a certain value. The patch adds tests related to its usage. But it needs improvement in the sense of combining incoming notes. We can use interactive mode for adding notes. Maybe we can do this if similarity is under a certain range.