rebase: move obsoleted not rebased messages earlier (BC) A later patch will clean up those states. This patch moves the messages earlier. Marked as BC since the order of message has changed. Differential Revision: https://phab.mercurial-scm.org/D23

rebase: extract ctx description logic to a function The function will be used in a later patch. Differential Revision: https://phab.mercurial-scm.org/D22

python3: whitelist another *13* tests that now pass Differential Revision: https://phab.mercurial-scm.org/D302

py3: introduce a wrapper for __builtins__.{raw_,}input() In order to make this work, we have to wrap the io streams in a TextIOWrapper so that __builtins__.input() can do unicode IO on Python 3. We can't just restore the original (unicode) sys.std* because we might be running a cmdserver, and if we blindly restore sys.* to the original values then we end up breaking the cmdserver. Sadly, TextIOWrapper tries to close the underlying stream during its __del__, so we have to make a sublcass to prevent that. If you see errors like: TypeError: a bytes-like object is required, not 'str' On an input() or print() call on Python 3, the substitution of sys.std* is probably the root cause. A previous version of this change tried to put the bytesinput() method in pycompat - it turns out we need to do some encoding handling, so we have to be in a higher layer that's allowed to use mercurial.encoding.encoding. As a result, this is in util for now, with the TextIOWrapper subclass hiding in encoding.py. I'm not sure of a better place for the time being. Differential Revision: https://phab.mercurial-scm.org/D299

python3: whitelist four more passing tests Differential Revision: https://phab.mercurial-scm.org/D298

extensions: don't give AttributeError bytes message on Python 3 Differential Revision: https://phab.mercurial-scm.org/D353

extensions: if on py3 and propname is a bytestr, convert to sysstr Property names are unicodes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D296

pushvars: add a coreconfigitem for push.pushvars.server Differential Revision: https://phab.mercurial-scm.org/D359

phabricator: add phabupdate command to update status in batch Changing status (accept, etc) on the webpage is not very convenient - currently there is no way to accept (or abandon etc.) a stack using a single click or command. This patch adds a `phabupdate` command that could be used to change status in batch. It also supports `--comment` which will write a comment on the last revision, which is similar to what we do using emails. Differential Revision: https://phab.mercurial-scm.org/D127

phabricator: add status to revision query language This patch adds status words (ex. `abandoned`, `accepted`, `needsreview`, `needsrevision`, `closed`) to the revision query language so people can select revision in a more flexible way. Test Plan: Try something like `phabread ':2 & accepted'`, `phabread ':105 - closed` and make sure they have desired outputs. Differential Revision: https://phab.mercurial-scm.org/D126

phabricator: add a small language to query Differential Revisions Previously, `phabread` can only be used to read a single patch, or a single stack of patches. In the future, we want to have more complex queries like filtering with status (open, accepted, closed, etc), or maybe more complex like filtering by reviewers etc. The command line flag approach won't scale with that. Besides, we might want to have other commands to update Differential Revision status in batch, like accepting a stack using a single command. Therefore, this patch adds a small language. It has basic set operations: `&`, `+`, `-` and an ancestor operator to support `--stack`. Test Plan: Try querying this Phabricator instance: hg phabread 1+2 # 1, 2 hg phabread D2+D1 # 2, 1 hg phabread ':118-115+:2-1' # 114, 116, 117, 118, 2 hg phabread '((:118-(D115+117)))&:117' # 114, 116 hg phabread ':2&:117' --debug # differential.query is called only once Make sure the output is expected and prefetch works. Differential Revision: https://phab.mercurial-scm.org/D125

phabricator: change "readpatch" to be more flexible Previously, `readpatch` and `querydrev` take a same `params` and `stack` parameters. This patch changes `readpatch` so it takes the output of `querydrev`, not the input of `querydrev`. This makes the code more flexible and cleaner. Differential Revision: https://phab.mercurial-scm.org/D124

filemerge: extract `_picklabels` as a helper function This shortens `simplemerge()` and is a bit cleaner, IMO. Differential Revision: https://phab.mercurial-scm.org/D376

simplemerge: write merge result to the localctx, if passed Differential Revision: https://phab.mercurial-scm.org/D375

simplemerge: use contexts to read file data from, if passed Differential Revision: https://phab.mercurial-scm.org/D374

filemerge: pass contexts to simplemerge Otherwise, this should be a no-op. Differential Revision: https://phab.mercurial-scm.org/D373

simplemerge: add optional context parameters to simplemerge Rename the existing parameters for clarity. These will, in subsequent patches, allow callers to redirect reads (of the three sides of the merge) and writes (of the result) to the given contexts, instead of using the filesystem. While in most cases, the writes will go to a workingfilectx, this opens the door for it to be a memfilectx in the case of an in-memory merge. Repo will be necessary in a subsequent comit. Differential Revision: https://phab.mercurial-scm.org/D372

simplemerge: extract verifytext as a helper function This will be used in a subsequent commit. Differential Revision: https://phab.mercurial-scm.org/D371

commit: use context manager with dirstateguard When I wrote 5ac845ca059a (commit: don't let failed commit with --addremove update dirstate (issue5645), 2017-07-31), Durham's 609606d21765 (rebase: use one dirstateguard for when using rebase.singletransaction, 2017-07-20) had not yet landed, so I had to write it in the old-fashioned way. Now that Durham's patch is in, we can simplify by using a context manager. Differential Revision: https://phab.mercurial-scm.org/D406

commit: move dirstateguard creation out of try-block This is just a simple refactoring to make the next patch simpler. If the dirstateguard constructor raises an exception, the finally-block won't do anything anyway, so this is functionally equivalent (and there is no except-block). Differential Revision: https://phab.mercurial-scm.org/D405

copies: add more details to the documentation of mergecopies() This documentation is very helpful for any developer to understand what copytracing is and what the function does. Since this is the main function of doing copytracing, I have also included bits about copytracing in it. This additions are picked from a doc by Stash@Fb. So thanks to him. Differential Revision: https://phab.mercurial-scm.org/D409

httppeer: add support for httppostargs when we're sending a file This is probably only used in the 'unbundle' command, but the code ended up being cleaner to make it generic and treat *all* httppostargs with a non-args request body as though they were file-like in nature. It also means we get test coverage more or less for free. A previous version of this change didn't use io.BytesIO, and it was a lot more complicated. This also fixes a server-side bug, so anyone using httppostargs should update all of their servers to this revision or later *before* this gets to their clients, otherwise servers will hang trying to over-read the POST body. Differential Revision: https://phab.mercurial-scm.org/D231

i18n: get translation entries for description of each compression engines Now, hggettext can be applied safely on util.py, of which i18nfunctions contains appropriate objects related to each compression types.

i18n: ignore docstring for modules under mercurial Docstring of modules is needed only for "hg help -e EXTNAME". This is a preparation for applying hggettext on util.py, which has module docstring, but it isn't needed for translation.

i18n: use saved object to get actual function information if available To list up available compression types instead of ".. bundlecompressionmarker" in "hg help bundlespec" output, proxy object "docobject" is used, because: - current online help system requires that __doc__ of registered object (maybe, function) is already well formatted in reST syntax - bundletype() method of compressionengine classes is used to list up available compression types, but - __doc__ of bundletype() object (= "instancemethod") is read-only On the other hand, hggettext requires original function object, in order to get document location in source code. Therefore, description of each compression types isn't yet translatable. Even if translatable, translators should make much effort to determine location of original texts in source code. To get actual function information, this patch makes hggettext use function object saved as "_origfunc", if it is available. This patch also changes bundlecompressiontopics() side, in order to explain how these changes work easily. This patch is a part of preparations for making description of each compression types translatable.

i18n: ignore doctest part to avoid warning at "make update-pot" hggettext assumes that backslashes in docstring are always doubled in original source code, in order to find the location of original docstring out certainly. This assumption almost always works as expected. But doctest easily breaks it, because many of backslashes in doctests aren't doubled. This mismatching causes "unknown offset in ..." warning at "make update-pot". To avoid such warning, this patch ignores doctest part of docstring before finding the location of original docstring out. BTW, at this patch, only person() in templatefilters.py has doctest part, which causes "unknown offset ..." warning. Therefore, just making backslashes in that doctest doubled can avoid such warning, too. But forcing doctest writers to double backslashes in doctest isn't reasonable, IMHO.

largefiles: fix help text to avoid warning at "make update-pot" This change helps hggettext to find out help text in original source, because it assumes that backslash ('\') is doubled in docstring.

i18n: make hggettext use original docstring to compute offset Before this patch, hggettext uses __doc__ of each functions to compute offset of document text. But __doc__ of many functions is already modified by decorators in registrar (e.g. @templatekeyword adds ":NAME: " prefix to it), and hggettext can not find it out in original source. This causes many "unknown offset in ..." warning at "make update-pot", and leaving them might cause overlooking serious problems. This patch makes hggettext use original docstring, which decorators in registrar save into _origdoc, to compute offset. Even after this patch, there are still a few "unknown offset in ..." warning at "make update-pot" for specific reasons. These will be fixed later one by one.

run-tests: also color the summary messages (skipped, failed...)

crecord: fixes the formatting of the select status in the status line The status line in the crecord has the "space" status field which has variable length depending on the length of the status label in the language of choice. In English, the status labels are "space: deselect" and "space:select". The "deselect" label is 2 glyphs longer. This makes the terminal output jump around if the terminal width is just right so that the shorter label makes the status line 1 line long, and the longer label makes it 2 lines long. This patch formats the selected status into a fixed-width field. The field width is the maximum of the lengths of the two possible labels, to account for differing translations and label lengths. This should make the label behavior uniform across localizations. There does not seem to be a test for crecord, so I verified the change manually with a local build of 'hg'.

cext: move PyInt macros to charencode.c properly Python3 build was broken at e9996bd7203f.

py3: change encoding.localstr to a subclass of bytes, not str

bundle2: relax the condition to update transaction.hookargs This is just a micro optimization. If hookargs is empty, nothing should be necessary.

bundle2: raise ProgrammingError for invalid call of addhookargs() It should be hard error. Also fixed the error message as s/hooks/hookargs/.

merge: removed sorting in casefolding detection, for a slight performance win It was not required for the correctness of the algorithm. Differential Revision: https://phab.mercurial-scm.org/D30

tests: verify that peer instances only expose interface members Our abstract interfaces are more useful if we guarantee that implementations conform to certain rules. Namely, we want to ensure that objects implementing interfaces don't expose new public attributes that aren't part of the interface. That way, as long as consumers don't access "internal" attributes (those beginning with "_") then (in theory) objects implementing interfaces can be swapped out and everything will "just work." We add a test that enforces our "no public attributes not part of the abstract interface" rule. We /could/ implement "interface compliance detection" at run-time. However, that is littered with problems. The obvious solutions are custom __new__ and __init__ methods. These rely on derived types actually calling the parent's implementation, which is no sure bet. Furthermore, __new__ and __init__ will likely be called before instance-specific attributes are assigned. In other words, they won't detect public attributes set on self.__dict__. This means public attribute detection won't be robust. We could work around lack of robust self.__dict__ public attribute detection by having our interfaces implement a custom __getattribute__, __getattr__, and/or __setattr__. However, this incurs an undesirable run-time penalty. And, subclasses could override our custom method, bypassing the check. The most robust solution is a non-runtime test. So that's what this commit implements. We have a generic function for validating that an object only has public attributes defined by abstract classes. Then, we instantiate some peers and verify a newly constructed object plays by the rules. Differential Revision: https://phab.mercurial-scm.org/D339

wireproto: use new peer interface The wirepeer class provides concrete implementations of peer interface methods for calling wire protocol commands. It makes sense for this class to inherit from the peer abstract base class. So we change that. Since httppeer and sshpeer have already been converted to the new interface, peerrepository is no longer adding any value. So it has been removed. httppeer and sshpeer have been updated to reflect the loss of peerrepository and the inheritance of the abstract base class in wirepeer. The code changes in wirepeer are reordering of methods to group by interface. Some Python code in tests was updated to reflect changed APIs. .. api:: peer.peerrepository has been removed. Use repository.peer abstract base class to represent a peer repository. Differential Revision: https://phab.mercurial-scm.org/D338

httppeer: use peer interface This is similar to what we did to sshpeer. Quirks and all. Differential Revision: https://phab.mercurial-scm.org/D337

sshpeer: use peer interface We need the same @property conversion of ui like we did for localpeer. We renamed _capabilities() to capabilities() to satisfy the new naming requirement. However, since we're inheriting from wireproto.wirepeer which inherits from peer.peerrepository and provides its own code accessing _capabilities(), we need to keep the old alias around. This wonkiness will disappear once wirepeer is cleaned up in subsequent commits. We also implement methods for basepeer that are identical to the defaults in peer.peerrepository in preparation for the removal of peerrepository. Differential Revision: https://phab.mercurial-scm.org/D336

localrepo: use peer interfaces We now have a formal abstract base class for peers. Let's transition the peer classes in localrepo to it. As part of the transition, we reorder methods so they are grouped by interface and match the order they are defined in the interface. We also had to change self.ui from an instance attribute to a property to satisfy the @abstractproperty requirement. As part of this change, we uncover the first "bug" as part of enforcing interfaces: stream_out() wasn't implemented on localpeer! This isn't technically a bug since the repo isn't advertising the stream capability, so clients shouldn't be attempting to call it. But I don't think there's a good reason why this is the case. We implement a dummy method to satisfy the interface requriements. We can make localpeer instances streamable as a future enhancement. # no-check-commit Differential Revision: https://phab.mercurial-scm.org/D335

repository: implement generic capability methods on peer class These methods are part of the peer interface, are generic, and can be implemented in terms of other members of the peer interface. So we implement them on the peer base class as a convenience. The implementation is essentially copied from peer.py. The code in peer.py will eventually be deleted. Differential Revision: https://phab.mercurial-scm.org/D334

repository: formalize wire protocol interface There are a well-defined set of commands constituting the wire protocol. Interaction with these and methods for calling them in batches are exposed via methods on peer instances. Let's formalize support for these features in abstract classes. The command parts come from the existing wireproto.wirepeer class. The batch methods come from peer.peerrepository. Ample documentation has been added as part of defining the interfaces. # no-check-commit Differential Revision: https://phab.mercurial-scm.org/D333

repository: formalize peer interface with abstract base class There are various interfaces for interacting with repositories and peers. They form a contract for how one should interact with a repo or peer object. The contracts today aren't very well-defined or enforced. There have been several bugs over the years where peers or repo types have forgotten to implement certain methods. In addition, the inheritance of some classes is wonky. For example, localrepository doesn't inherit from an interface and the god-object nature of that class means the repository interface isn't well-defined. Other repository types inherit from localrepository then stub out methods that don't make sense (e.g. statichttprepository re-defining locking methods to fail fast). Not having well-defined interfaces makes implementing alternate storage backends, wire protocol transports, and repository types difficult because it isn't clear what exactly needs to be implemented. This patch starts the process of attempting to establish more order to the type system around repositories and peers. Our first patch starts with a problem space that already has a partial solution: peers. The peer.peerrepository class already somewhat defines a peer interface. But it is missing a few things and the total interface isn't well-defined because it is combined with wireproto.wirepeer. Our newly-established basepeer class uses the abc module to declare an abstract base class with the properties and methods that a generic peer must implement. We create a new class that inherits from it. This class will hold our other future abstract base classes / interfaces so we can expose a unified base class/interface. We don't yet use the new interface because subsequent additions will break existing code without some refactoring first. A new module (repository.py) was created to hold the interfaces. I could have put things in peer.py. However, I have plans to eventually add interfaces to define repository and storage types. These almost certainly require a new module. And I figured having all the interfaces live in one module makes sense. So I created repository.py to be that future home. Differential Revision: https://phab.mercurial-scm.org/D332

util: make nogc effective for CPython 279cd80059d4 made `util.nogc` a no-op. It was to optimize PyPy. But it slows down CPython if many objects (like 300k+) are created. For example, running `hg log -r .` without extensions in `hg-committed` with 14k+ obsmarkers have the following times: before | after hg | chg | hg | chg ----------------------------- 1.262 | 0.860 | 1.077 | 0.619 (seconds, best of 20 runs) Therefore let's re-enable nogc for CPython. Differential Revision: https://phab.mercurial-scm.org/D402

scmutil: use util.shellquote instead of %r Changes some output, but also resolves differences with Python 3. Differential Revision: https://phab.mercurial-scm.org/D301

context: fix troubled deprecation troubled has been renamed into isunstable but troubled was calling unstable instead. Fix the mistake. Differential Revision: https://phab.mercurial-scm.org/D384

test-pushvars: invoke shell script hook via `sh` for Windows Invoking *.sh on Windows leads to the "what program should open this?" prompt, which stalls the test and led to the recent series of exceptions on the Windows test machine as the runner times out.

test-sparse: drop unnecessary globs

exchange: simplify unbundle locking using context managers Differential Revision: https://phab.mercurial-scm.org/D393

util: add base class for transactional context managers We have at least three types with a close() and a release() method where the close() method is supposed to be called on success and the release() method is supposed to be called last, whether successful or not. Two of them (transaction and dirstateguard) already have identical implementations of __enter__ and __exit__. Let's extract a base class for this, so we reuse the code and so the third type (transactionmanager) can also be used as a context manager. Differential Revision: https://phab.mercurial-scm.org/D392

exchange: remove need for "locked" variable The transactionmanager() constructor just assigned a few variables and cannot fail, so it's safe to move it inside the earlier try/except. Differential Revision: https://phab.mercurial-scm.org/D391

exchange: drop now-unnecessary "local" from lock name variables Since fda0867cfe03 (exchange: drop support for lock-based unbundling (BC), 2017-08-06), there is no more remote locking. Differential Revision: https://phab.mercurial-scm.org/D390

exchange: remove 'locallocked' member from pushop object The variable has been used only within a single function since 94e2862dbcfb (push: elevate phase transaction to cover entire operation, 2014-11-21), so there's no need to keep it on the pushop object. Differential Revision: https://phab.mercurial-scm.org/D389

rebase: change "result would have 3 parent" error message (BC) The old error message "cannot use revision REV as base, result would have 3 parents" is confusing - why use REV as base? why add a new parent?. This patch changes it to "cannot move parent", which seems better. Differential Revision: https://phab.mercurial-scm.org/D342

drawdag: allow override file contents via comments This makes drawdag more flexible, and allow us to create non-clean merges in test cases. Differential Revision: https://phab.mercurial-scm.org/D385

phabricator: add --amend option to phabsend Previously `hg phabsend` was imitating `hg email` and won't mutate changesets. That works fine with reviewer-push workflow, reviewers run `phabread`, `import`. However, it does not work well with author-push workflow. Namely, the author needs to run extra commands to get the right commit message, and remove the local tag after push. This patch solves those issues by adding the `--amend` option, so local changesets will have the right commit message, and tags become unnecessary. Test Plan: Given the following DAG: o 17 o 16 | o 15 | @ 14 |/ o 13 o 12 Run `hg phabsend '(13::)-17' --amend`, check the new DAG looks like: o 21 | o 20 | @ 19 |/ o 18 | o 17 | x 16 | x 13 |/ o 12 And commit messages are updated to contain the `Differential Revision` lines. Use `phabread` to make sure Phabricator has the amended node recorded. Also check `phabsend .` followed by a `phabsend . --amend`, the commit message will be updated and the tag will be removed. Differential Revision: https://phab.mercurial-scm.org/D122

rebase: rewrite core algorithm (issue5578) (issue5630) "defineparents" is the core algorithm of rebase. The old code has too many tech debts (like outdated comments, confusing assertions, etc) and is very error-prone to be improved. This patch rewrites "defineparents" to make the code easier to reason about, and solve a bunch of issues, including: - Assertion error: no base found (demonstrated by D212, issue5578) - Asymmetric result (demonstrated by D211, "F with one parent") - Wrong new parent (demonstrated by D262, "C':A'A'") - "revlog index out of range" (demonstrated by D262, issue5630) - "nothing to merge" (demonstrated by D262) As a side effect, merge base decision has been made more solid - rebase now prints out explicitly what could go wrong when it cannot find a unique suitable merge base. .. fix:: Issue 5578, Issue 5630 Core rebase algorithm has been rewritten to be more robust. Differential Revision: https://phab.mercurial-scm.org/D21

phabricator: remove an unnecessary writediffproperties call This was introduced by D229. Thanks Yuya for finding it! Differential Revision: https://phab.mercurial-scm.org/D366

releasenotes: minor bug fix for index error while serializing Differential Revision: https://phab.mercurial-scm.org/D356

label: rename log.trouble into log.instability The renaming is done according to https://www.mercurial-scm.org/wiki/CEDVocabulary. Differential Revision: https://phab.mercurial-scm.org/D259