wireproto: function for testing if wire protocol command is available Currently, we perform simple membership testing for whether a wire command is available. In the future, not all wire protocol commands will be available on all transports. For example, a legacy transport may not support newer commands. In preparation of this, teach the protocol handlers to call into a function to determine if a wire protocol command is available. That function currently does membership testing like before, so behavior should be identical. In the case of the HTTP server, behavior is a bit wonkier. "cmd" is used by both the wire protocol and hgweb. We do want the protocol handler to handle requests for all commands that look like wire protocol commands, even if they aren't available. Otherwise, the fallback to hgweb would only confuse automated clients and make it easier for hgweb to accidentally implement a "cmd" that is identical to wire protocol commands (since they aren't centrally registered). Differential Revision: https://phab.mercurial-scm.org/D1999

wireproto: define and use types for wire protocol commands Wire protocol commands have historically been declared as 2-tuples in wireproto.commands. There are some additional features I'd like to implement that require going beyond 2-tuples. But because the 2-tuple API (both destructuring assignment and direct assignment into the dict) is used throughout the code base and in 3rd party extensions, we can't do a trivial type change. This commit creates a new "commandentry" type to represent declared wire protocol commands. It implements __getitem__ and __iter__ so it can quack like a 2-tuple. The @wireprotocommand decorator now creates "commandentry" instances. We also create a "commanddict" type to represent the dictionary of declared wire protocol commands. It inherits from "dict" but provides a custom __setitem__ to coerce passed 2-tuples to "commandentry" instances. wireproto.commands is now an instance of this type. Various callers in core rely on the new functionality. And tests pass. So I'm reasonably confident things will "just work" in 3rd party extensions as well. Differential Revision: https://phab.mercurial-scm.org/D1998

wireproto: improve docstring for @wireprotocommand I'm about to add more arguments and want them to be documented. Plus, good documentation is nice to have. Differential Revision: https://phab.mercurial-scm.org/D1997

wireproto: remove unnecessary exception trapping The `try..except error.Abort` was added in 8474be4412ca back in 2012. The intent was to ensure a failing pushkey hook didn't crash the server. Since that changeset, repo.pushkey() and the hooks mechanism is now much more robust about trapping errors itself. As such, we no longer need this try..except block. So it has been removed. Differential Revision: https://phab.mercurial-scm.org/D1996

sshpeer: implement peer for version 2 of wire protocol Since the protocol is now negotiated before we construct a peer instance, we can return the negotiated protocol from the handshake function and instantiate an appropriate peer class for the protocol. Version 2 of the SSH protocol is currently identical to version 1 post handshake. So our sshv2peer class just inherits from sshv1peer for the time being. This will obviously change over time. Differential Revision: https://phab.mercurial-scm.org/D2063

sshpeer: rename sshpeer class to sshv1peer (API) With the introduction of version 2 of the SSH wire protocol, we will need a new peer class to speak that protocol because it will be too difficult to shoehorn a single class to speak two protocols. We rename sshpeer.sshpeer to sshpeer.sshv1peer to reflect the fact that there will be multiple versions of the peer depending on the negotiated protocol. .. api:: sshpeer.sshpeer renamed to sshpeer.sshv1peer. Differential Revision: https://phab.mercurial-scm.org/D2062

sshpeer: initial definition and implementation of new SSH protocol The existing SSH protocol has several design flaws. Future commits will elaborate on these flaws as new features are introduced to combat these flaws. For now, hopefully you can take me for my word that a ground up rewrite of the SSH protocol is needed. This commit lays the foundation for a new SSH protocol by defining a mechanism to upgrade the SSH transport channel away from the default (version 1) protocol to something modern (which we'll call "version 2" for now). This upgrade process is detailed in the internals documentation for the wire protocol. The gist of it is the client sends a request line preceding the "hello" command/line which basically says "I'm requesting an upgrade: here's what I support." If the server recognizes that line, it processes the upgrade request and the transport channel is switched to use the new version of the protocol. If not, it sends an empty response, which is how all Mercurial SSH servers from the beginning of time reacted to unknown commands. The upgrade request is effectively ignored and the client continues to use the existing version of the protocol as if nothing happened. The new version of the SSH protocol is completely identical to version 1 aside from the upgrade dance and the bytes that follow. The immediate bytes that follow the protocol switch are defined to be a length framed "capabilities: " line containing the remote's advertised capabilities. In reality, this looks very similar to what the "hello" response would look like. But it will evolve quickly. The methodology by which the protocol will evolve is important. I'm not going to introduce the new protocol all at once. That would likely lead to endless bike shedding and forward progress would stall. Instead, I intend to tricle out new features and diversions from the existing protocol in small, incremental changes. To support the gradual evolution of the protocol, the on-the-wire advertised protocol name contains an "exp" to denote "experimental" and a 4 digit field to capture the sub-version of the protocol. Whenever we make a BC change to the wire protocol, we can increment this version and lock out all older clients because it will appear as a completely different protocol version. This means we can incur as many breaking changes as we want. We don't have to commit to supporting any one feature or idea for a long period of time. We can even evolve the handshake mechanism, because that is defined as being an implementation detail of the negotiated protocol version! Hopefully this lowers the barrier to accepting changes to the protocol and for experimenting with "radical" ideas during its development. In core, sshpeer received most of the attention. We haven't even implemented the server bits for the new protocol in core yet. Instead, we add very primitive support to our test server, mainly just to exercise the added code paths in sshpeer. Differential Revision: https://phab.mercurial-scm.org/D2061 # no-check-commit because of required foo_bar naming

internals: refactor wire protocol documentation Upcoming work will introduce a new version of the HTTP and SSH transports. The differences will be significant enough to consider them new transports. So, we now attach a version number to each transport. In addition, having the handshake documented after the transport and in a single shared section made it harder to follow the flow of the connection. The handshake documentation is now moved to the protocol section it describes. We now have a generic section about the purpose of the handshake, which was rewritten significantly. Differential Revision: https://phab.mercurial-scm.org/D2060

revlog: rename 'self.checkinlinesize' into '_enforceinlinesize' The name change has two motivations: 1) The function has no external caller, so we move it to protected space. 2) the function does more than checking it also split the data if we have more data than 'inline' supports.

revlog: add a _datareadfp context manager for data access needs The helper handles: 1) is there a file handle already open that we shall just reuse, 2) is the revlog inlined or not. Using a context manager for all read access will help setting up file pointer caching in later changesets.

revlog: use context manager for data file lifetime in checksize This is clearer, safer and more modern.

revlog: use context manager for index file lifetime in checkinlinesize This is clearer, safer and more modern.

revlog: use context manager for data file lifetime in checkinlinesize This is clearer, safer and more modern.

revlog: use context manager for index file life time in __init__ This is clearer, safer and more modern.

revlog: move index file opening in a method Having file operation centralized into a single spot help to factor common logic out (eg: special flag handling according to the mode). It is also the first step to simplify file handling during batch operation (eg: revlog cloning). However, that part does not seems to be a hotspot yet.

revlog: move datafile opening in a method Having file operation centralized into a single spot help to factor common logic out. It is also the first step to simplify file handling during batch operation (eg: revlog cloning). However, that part does not seems to be a hotspot yet.

parseindex: implement context manager method on the wrapper This is needed for incoming cleanups.

parseindex: also forward keyword argument in a debug wrapper Otherwise, it gets in the way of a coming refactoring.

dumprevlog: handle being passed a mode parameter This makes it closer to an actual file opener. Otherwise, it gets in the way of a coming refactoring.

changelog: add the missing 'closed' property on 'appender' object This mimic file object further.

changelog: implement context manager method for 'appender' object This object mimics a file and was missing the context manager API.

diff: improve ui.write performance when not coloring on Windows Differential Revision: https://phab.mercurial-scm.org/D2022

hgsh: enable clang-format Nothing looks awful, so we can just turn it on. Differential Revision: https://phab.mercurial-scm.org/D2059

chg: enable clang-format on all .c and .h files Nothing in here looks awful, so I think we may as well just do it. # skip-blame because it's just reformatting with no functionality change Differential Revision: https://phab.mercurial-scm.org/D2058

python3: whitelist an additional 11 tests I think these are mostly the result of Pulkit's recent work. Thanks! Differential Revision: https://phab.mercurial-scm.org/D2078

ui: add explicit path to write prompt text bypassing buffers The prompt= flag was added at e35d7f131483, when colorui had its own write() function. Since we've merged colorui to ui, we can simply call the unbuffered write() function.

ui: write prompt text in ui.prompt(), not in ui._readline() self.label() is replaced by label= option, which should make it clearer why we can't pass the text to raw_input(prompt).

logcmdutil: mark changesetprinter.showpatch() as private

logcmdutil: drop default arguments from changesetdisplayer/templater() calls

logcmdutil: make default parameters of changesetprinters consistent

mdiff: use str.startswith/endswith() instead of slicing

patch: unify check_binary and binary flags Follows up 079b27b5a869. If opts.text=True, check_binary is ignored, so we can just pass the binary flag to unidiff(). perfunidiff now takes any inputs as text files, which I think is a desired behavior.

py3: make sure we open the file in bytes mode # skip-blame because we are just adding b Differential Revision: https://phab.mercurial-scm.org/D2077

py3: use bytes instead of str Differential Revision: https://phab.mercurial-scm.org/D2076

py3: add b'' prefixes to string literals in test files # skip-blame because we are just adding b'' Differential Revision: https://phab.mercurial-scm.org/D2075

py3: use range instead of xrange xrange is not available on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2074

py3: use open() instead of file() file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2073

py3: use pycompat.bytestr to convert _b85chars to bytes The tranformer does append b'' to the value and make that a bytes but bytes in Python 3 returns the ascii value on getting characters using indexing. Characters of this string are queried using indexing multiple times in the file and to support that we use pycompat.bytestr which returns the bytechrs using indexing. Differential Revision: https://phab.mercurial-scm.org/D2072

py3: use pycompat.bytechr instead of chr Differential Revision: https://phab.mercurial-scm.org/D2071

py3: use pycompat.ziplist instead of zip zip returns a zip object instead of a list on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2070

patches: move assignment outside the conditional Having this movement in its own patch will make the next patch clearer.

sshpeer: remove support for connecting to <0.9.1 servers (BC) 197d10e157ce made this change for the HTTP peer. Let's do the same for the SSH peer. Test output changes as expected. A redundant test has been dropped. .. bc:: Support for connecting to Mercurial servers older than 0.9.1 has been removed. Differential Revision: https://phab.mercurial-scm.org/D2036

sshpeer: document the handshake mechanism The mechanism by which SSH peers establish connections with remotes is wonky and requires a bit of code archeology to understand. While it is already documented in `hg help internals.wireproto`, it helps to have documentation in the code as well. Differential Revision: https://phab.mercurial-scm.org/D2035

sshpeer: move handshake outside of sshpeer With the handshake now performed before a peer class is instantiated, we can now instantiate a different peer class depending on the results of the handshake. Our test extension had to change to cope with the new API. Because we now issue the command via raw I/O calls and don't call _callstream(), we no longer have to register the fake command. (_callstream() uses the command registration to see what args to send). Differential Revision: https://phab.mercurial-scm.org/D2034

sshpeer: inline I/O into _validaterepo() We want to move the handshake code out of the peer class so the peer factory function can perform the handshake and instantiate a proper class depending on the results. To make that refactor easier to read, we first inline I/O functionality into _validaterepo(). Test output for low-level protocol tests didn't change, thus hopefully demonstrating that this refactor didn't change any material behavior. Because we no longer call _callstream(), our test extension for monkeypatching the peer had to change its hook point. Differential Revision: https://phab.mercurial-scm.org/D2033

sshpeer: clean up API for sshpeer.__init__ (API) Our refactoring left the state of sshpeer.__init__ in a poor state. "create" was no longer used. Process/pipe arguments were passed poorly. "name" was really a URL. This commit cleans all that up. .. api:: sshpeer.sshpeer.__init__ now receives arguments describing an existing connection instead of creating a connection itself. Differential Revision: https://phab.mercurial-scm.org/D2032

sshpeer: establish SSH connection before class instantiation We want to move the handshake to before peers are created so we can instantiate a different peer class depending on the results of the handshake. This necessitates moving the SSH process invocation to outside the peer class. As part of the code move, some variables were renamed for clarity. util.popen4() returns stdin, stdout, and stderr in their typical file descriptor order. However, stdin and stdout were being mapped to "pipeo" and "pipei" respectively. "o" for "stdin" and "i" for "stdout" is a bit confusing. Although it does make sense for "output" and "input" from the perspective of the client. But in the context of the new function, it makes sense to refer to these as their file descriptor names. In addition, the last use of self._path disappeared, so we stop setting that attribute and we can delete the redundant URL parsing necessary to set it. Differential Revision: https://phab.mercurial-scm.org/D2031

sshpeer: remove frivolous call to _cleanup() _validaterepo() is called once during __init__. _cleanup() no-ops if the self._pipe* attributes aren't set. These attributes are set during _validaterepo(). So the call to _cleanup() isn't necessary. But just to be on the safe side, we add an assertion. Differential Revision: https://phab.mercurial-scm.org/D2030

sshpeer: extract pipe cleanup logic to own function So it can be used outside of instantiated classes. This is needed to support pipe creation before __init__ is called. Differential Revision: https://phab.mercurial-scm.org/D2029

sshpeer: move ssh command and repo creation logic out of __init__ It was easier to move both of these at once because repository creation requires various variables and I didn't want to add tons of arguments and code to __init__ that will soon be deleted anyway. We do add an extra argument so we can proxy values to the _validaterepo() call. But this is minimally invasive. Some callers of self._abort() were converted to just raise. Like before, the _abort() call wasn't necessary because self._pipe* aren't populated this early in the object's lifetime. As part of this, various private attributes derived from the parsed URL are no longer used. So we no longer set them. Differential Revision: https://phab.mercurial-scm.org/D2028

sshpeer: move URL validation out of sshpeer.__init__ We will soon have another SSH peer class to support the new version of the SSH protocol. However, we won't know which peer class to instantiate until we perform a handshake on an active connection. This means that we need to move connection establishment and handshake code out of sshpeer.__init__. This commit starts the process of migrating peer creation code out of sshpeer.__init__ into instance(), which is the API for creating peers. The moved code no longer calls _abort(). _abort() runs _cleanup() and raises. _cleanup() only performs actions on self._pipe*. These objects aren't instantiated until we actually connect to the peer. So _abort() was not necessary in the old code. To keep the API the same, __init__() now makes a redundant call to util.url(). This will be fixed in subsequent commits. Differential Revision: https://phab.mercurial-scm.org/D2027

tests: add low-level SSH protocol tests We don't really have good low-level tests for the behavior of the SSH wire protocol. This commit attempts to establish some. The added tests consist of a mixture of starting a server with `hg serve --stdio` and sending bytes to it and using `hg debugpeer` to go through the official client code. Having insight into what raw bytes are exchanged as well as what the peer does is useful. We also introduce a test extension for modifying the behavior of the SSH server and peer. For example, we change the server to not recognize the "hello" command, simulating behavior of <0.9.1 servers. These tests are generally useful to have. But the impetus for creating them now is they will be needed for verifying behavior of old clients and servers when a new SSH protocol is introduced. Differential Revision: https://phab.mercurial-scm.org/D2026 # no-check-commit because of serve_forever()

debugcommands: introduce debugpeer command `hg debugpeer <path>` will establish a connection to a peer repository and print information about it. If you add --debug, it will log low-level protocol request info. This will be useful for upcoming tests around protocol handshaking. Differential Revision: https://phab.mercurial-scm.org/D2025

sshpeer: make "instance" a function The API is that peer modules must provide an "instance" symbol that is callable to return a peer. Making "instance" a function instead of an alias to "sshpeer" makes it easier to monkeypatch the "sshpeer" type. It will also make it possible to turn instance() into a factory function of sorts that returns different types based on connection properties. Differential Revision: https://phab.mercurial-scm.org/D2024

lfs: deduplicate oids in the transfer Apparently, we can't rely on the server to deduplicate for us. Sadly, the pointer object isn't hashable, so it can't be reduced by converting it to a set. In order to be hashable, it needs to be immutable. I had a bunch of code to change it to composition and forward the readonly dict methods to a member dict. But the pointer is updated via __setitem__() when creating the pointer file. So it didn't see worth adding all of that code to the class.

cat: call the storage prefetch hook It's not important to call in the case of a single file, but maybe it's better to do so for consistency.

archive: call the storage prefetch hook

lfs: prefetch lfs blobs during revert The revert command oddly prints out what it will do before requesting the files to be prefetched. But the 'need to transfer' line indicates the blobs are being grouped.

cmdutil: convert _revertprefetch() to a generic stored file hook (API) This will be used by LFS to fetch required files in a group for multiple commands, prior to being accessed. That avoids the one-at-a-time fetch when the filelog wrapper goes to access it, and it is missing locally (which costs two round trips to the server.) The core command list that needs this is probably at least: - annotate - archive (which is also used by extdiff) - cat - diff - export - grep - verify (sadly) - anything that has the '{data}' template There are no core users of the revert prefetch hook, and never have been since it was introduced in 45e02cfad4bd for remotefilelog. Thanks to Yuya for figuring out a way to reliably trigger the deprecated warning. Unfortunately, it wanted to blame the caller of revert. Passing along an adjusted stack level seemed the least bad choice (although it still blames a core function). One thing to note is that the store lock isn't being held when this is called. I'm not at all familiar with remotefilelog or its locking requirements, so this may not be a big deal. Currently, LFS doesn't hold a lock when downloading files. Even though largefiles doesn't either, I'm starting to think it should, and maybe the .hg/store/lock isn't good enough to cover the globally shared cache. .. api:: The cmdutil._revertprefetch() hook point for prefetching stored files has been replaced by the command agnostic cmdutil._prefetchfiles(). The new function takes a list of files, instead of a list of lists of files.

lfs: prefetch lfs blobs when applying merge updates In addition to merge, this method ultimately gets called by many commands: - backout - bisect - clone - fetch - graft - import (without --bypass) - pull -u - rebase - strip - share - transplant - unbundle - update Additionally, it's also called by histedit, shelve, unshelve, and split, but it seems that the related blobs should always be available locally for these. For `hg update`, it happens after the normal argument checking and pre-update hook processing, and remote corruption is detected prior to manipulating the working directory. Other commands could use this treatment (archive, cat, revert, etc), but this covers so many of the frequently used bulk commands, it seems like a good starting point. Losing the verbose message that prints the file name before a corrupt blob aborts the command is a little sad, because there's no easy way to go from oid to file name. I'd like to change that message to list the file name so it looks cleaner and less cryptic, but the pointer object is nowhere near where it needs to be to do this. So punt on that for now.

lfs: factor out a method for extracting the pointer of a single file This will be useful for filesets, among other things, instead of traversing the whole context.

py3: add b'' to literals in check-config.py # skip-blame because we are just adding b'' Differential Revision: https://phab.mercurial-scm.org/D2046

check-config: specify the mode 'rb' to open the file Differential Revision: https://phab.mercurial-scm.org/D2045

py3: use open() instead of file() file() is not present in Python 3 Differential Revision: https://phab.mercurial-scm.org/D2044

py3: use pycompat.strkwargs() to convert kwargs' key to str Differential Revision: https://phab.mercurial-scm.org/D2043

py3: add __bytes__() for mq.patchheader and make sure __str__ returns str Before this patch, __str__() function for mq.patchheader class return bytes which is not str on Python 3. So let's move that logic to __bytes__() and for __str__() convert the return value of __bytes__() to str. Differential Revision: https://phab.mercurial-scm.org/D2042

py3: use "%d" to convert integer to bytes Differential Revision: https://phab.mercurial-scm.org/D2041

py3: use .startswith() instead of bytes[0] Differential Revision: https://phab.mercurial-scm.org/D2040

py3: slice on bytes to prevent getting the ascii values Differential Revision: https://phab.mercurial-scm.org/D2039

py3: add r'' to convert keys to keyword arguments to str # skip-blame as we are just adding r'' Differential Revision: https://phab.mercurial-scm.org/D2038

py3: add b'' to tweakdefaults config string The tweakdefaults config string is enclosed inside triple quotes and the transformer does not adds b'' there. # skip-blame because we are just adding b'' Differential Revision: https://phab.mercurial-scm.org/D2037

cmdutil: introduce deprecated aliases c8e2d6ed1f9e moved some objects used by Evolve and hence broke the latest Evolve revision. Next Evolve version will use the new objects when available but introduce deprecated aliases so users using older version of Evolve won't have a broken Evolve extension. Differential Revision: https://phab.mercurial-scm.org/D2023

makefile: remove Ubuntu Yakkety and Zesty docker targets Yakkety (16.10) was supported until 2017-07-20 and Zesty (17.04) was supported until 2018-01-13.

merge: use operation-provided labels (ex: dest/source) in several merge-tools Tools that did not use labels already, used only one, or used some label other than the exact strings of "local" or "other" were unmodified. Cases that used the label "base" were modified as well, if they were otherwise changed in this CL; "merged" was *not* changed. There are other possible changes we might want to make, but I didn't: - bcompare (linux and osx) uses the labels "parent1" and "parent2" instead of "local" and "other", so it was left alone, even though beyondcompare3 (windows) *was* changed. - araxis used the labels "Other", "Base", and "Local :$local", so it was also left alone. - UltraCompare didn't provide a label for 'local', just for 'base' and 'other', so it was left alone. Differential Revision: https://phab.mercurial-scm.org/D2012

filemerge: support passing labels to external merge tools This adds $labellocal, $labelother, and $labelbase to the replacement set for merge-tools.<tool>.args config variables, and to the environment as HG_MY_LABEL, HG_OTHER_LABEL, and HG_BASE_LABEL, respectively. We also add merge-tools.<tool>.mergemarkers and merge-tools.<tool>.mergemarkertemplate config variables as counterparts of the variables available in [ui]. We are intentionally *not* respecting ui.mergemarkers when calling out to external merge programs; too often the default template will be too wide to display comfortably in most GUIs. Differential Revision: https://phab.mercurial-scm.org/D2011

bookmarks: drop deprecated methods (API) The ProgrammingError prevents accidental usage of the dict base class methods. .. api:: The following deprecated methods have been removed from bookmarks: __setitem__(), __delitem__(), update(), and recordchange(). Use bookmarks.applychanges() instead.

archive: rewrite default metadata template as a multi-line bytes literal This fixes test-directaccess.t on Python 3.

py3: drop b'' from repr() of smartset cmdutil._maybebytestr() is moved to pycompat.

py3: always drop b'' prefix from repr() of bytestr Perhaps this is what we wanted for py2-3 compatibility.

py3: format revision number as '%d' in debugrevspec Now "hg debugrevspec" works on Python 3.

py3: build repr() of smartset as bytes then convert to str This isn't pretty, but we have no way to teach Python 3 that our __repr__() would like to return a byte string.

py3: factor out helpers to apply string conversion recursively

py3: replace "if ispy3" by pycompat.sysbytes() or util.forcebytestr()

py3: replace "if ispy3" by pycompat.bytestr()

py3: replace "if ispy3" by encoding.strtolocal()

localrepo: drop the deprecated walk() method (API) .. api:: The deprecated localrepo.walk() has been removed, and replaced by repo[node].walk().

dirstate: drop deprecated methods (API) .. api:: beginparentchange() and endparentchange() have been replaced by the parentchange context manager.

templatekw: drop the deprecated '{troubles}' keyword

obsutil: drop deprecated methods (API) .. api:: The following deprecated methods have been removed from obsutil: marker.precnode() and allprecursors(). Use marker.prednode() and allpredecessors() instead.

revset: drop deprecated evolution predicates

obsolete: drop deprecated methods (API) .. api:: The following deprecated methods have been removed from obsolete, with replacements: - _addprecursors() -> _addpredecessors() - obsstore.precursors -> obsstore.predecessors - allprecursors() -> obsutil.allprecursors() - allsuccessors() -> obsutil.allsuccessors() - marker() -> obsutil.marker - getmarkers() -> obsutil.getmarkers() - exclusivemarkers() -> obsutil.exclusivemarkers() - foreground() -> obsutil.foreground() - successorssets() -> obsutil.successorsset() - unstable() -> orphan() - bumped() -> phasedivergent() - divergent() -> contentdivergent()

context: drop deprecated methods (API) .. api:: The following deprecated methods have been removed from context, with replacements: - unstable() -> orphan() - bumped() -> phasedivergent() - divergent() -> contentdivergent() - troubled() -> isunstable() - troubles() -> instabilities()

ui: improve performance for multi-component writes It is more efficient to pass down one large string to the output streams than many small ones. For a ``hg diff`` test case, it improves time from 2m26s to 2m8s. Differential Revision: https://phab.mercurial-scm.org/D1939

cmdutil: drop aliases for logcmdutil functions (API) .. api:: Log-related utility functions has been renamed as follows: - cmdutil.loglimit -> logcmdutil.getlimit - cmdutil.diffordiffstat -> logcmdutil.diffordiffstat - cmdutil._changesetlabels -> logcmdutil.changesetlabels - cmdutil.changeset_printer -> logcmdutil.changesetprinter - cmdutil.jsonchangeset = logcmdutil.jsonchangeset - cmdutil.changeset_templater -> logcmdutil.changesettemplater - cmdutil.logtemplatespec -> logcmdutil.templatespec - cmdutil.makelogtemplater -> logcmdutil.maketemplater - cmdutil.show_changeset -> logcmdutil.changesetdisplayer - cmdutil.getlogrevs -> logcmdutil.getrevs - cmdutil.getloglinerangerevs -> logcmdutil.getlinerangerevs - cmdutil.displaygraph -> logcmdutil.displaygraph - cmdutil.graphlog -> logcmdutil.graphlog - cmdutil.checkunsupportedgraphflags -> logcmdutil.checkunsupportedgraphflags - cmdutil.graphrevs -> logcmdutil.graphrevs - cmdutil._makenofollowlogfilematcher -> logcmdutil._makenofollowfilematcher

logcmdutil: drop redundant "log" from function names (API) A few exceptions: - s/loglimit/getlimit/ to avoid name conflict - s/_logrevs/_initialrevs/ to clarify its functionality

logcmdutil: rename classes and functions to conform to our coding style (API) show_changeset is renamed to changesetdisplayer as its return value is called a displayer.

cmdutil: split functions of log-like commands to new module (API) cmdutil.py is painfully big and makes Emacs slow. Let's split log-related functions. % wc -l mercurial/cmdutil.py 4027 mercurial/cmdutil.py % wc -l mercurial/cmdutil.py mercurial/logcmdutil.py 3141 mercurial/cmdutil.py 933 mercurial/logcmdutil.py 4074 total

httppeer: remove support for connecting to <0.9.1 servers (BC) Previously, HTTP wire protocol clients would attempt a "capabilities" wire protocol command. If that failed, they would fall back to issuing a "between" command. The "capabilities" command was added in Mercurial 0.9.1 (released July 2006). The "between" command has been present for as long as the wire protocol has existed. So if the "between" command failed, it was safe to assume that the remote could not speak any version of the Mercurial wire protocol. The "between" fallback was added in 395a84f78736 in 2011. Before that changeset, Mercurial would *always* issue the "between" command and would issue "capabilities" if capabilities were requested. At that time, many connections would issue "capabilities" eventually, so it was decided to issue "capabilities" by default and fall back to "between" if that failed. This saved a round trip when connecting to modern servers while still preserving compatibility with legacy servers. Fast forward ~7 years. Mercurial servers supporting "capabilities" have been around for over a decade. If modern clients are connecting to <0.9.1 servers, they are getting a bad experience. They may even be getting bad data (an old server is vulnerable to numerous security issues and could have been p0wned, leading to a Mercurial repository serving backdoors or other badness). In addition, the fallback can harm experience for modern servers. If a client experiences an intermittent HTTP request failure (due to bad network, etc) and falls back to a "between" that works, it would assume an empty capability set and would attempt to communicate with the repository using a very ancient wire protocol. Auditing HTTP logs for hg.mozilla.org, I did find a handful of requests for the null range of the "between" command. However, requests can be days apart. And when I do see requests, they come in batches. Those batches seem to correlate to spikes of HTTP 500 or other server/network events. So I think these requests are fallbacks from failed "capabilities" requests and not from old clients. If you need even more evidence to discontinue support, apparently we have no test coverage for communicating with servers not supporting "capabilities." I know this because all tests pass with the "between" fallback removed. Finally, server-side support for <0.9.1 pushing (the "addchangegroup" wire protocol command along with locking-related commands) was dropped from the HTTP client in fda0867cfe03 in 2017 and the SSH client in 9f6e0e7ef828 in 2015. I think this all adds up to enough justification for removing client support for communicating with servers not supporting "capabilities." So this commit removes that fallback. Differential Revision: https://phab.mercurial-scm.org/D2001

internals: document when "hello" and "capabilities" commands were added Both were introduced in 0.9.1. "hello" made its entrance in b17eebc911ae, 144280f1578f, and a1cfe679192c to support SSH. "capabilities" was added in c660691fb45d to support HTTP. Differential Revision: https://phab.mercurial-scm.org/D2000

tests: make doctest py3-compatible again A parsed tree is replaced with parse(expr) because it sucks to add b'' to every string literal.

lfs: emit a status message to indicate how many blobs were uploaded Previously, there was a progress bar indicating the byte count, but then it disappeared once the transfer was done. Having that value stay on the screen seems useful. Downloads are done one at a time, so hold off on that until they can be coalesced, to avoid a series of lines being printed. (I don't have any great ideas on how to do that. It would be a shame to have to wrap a bunch of read commands to be able to do this.) I'm not sure if the 'lfs:' prefix is the right thing to do here. The others in the test are verbose/debug messages, so in the normal case, this is the only line that's prefixed.

lfs: drop an unused function parameter

discovery: don't reimplement all() Differential Revision: https://phab.mercurial-scm.org/D1993

contrib: fix dirstatenonnormalcheck to work in Python 3 This is a redo of D1963 that has the added benefit of not breaking Python 2. Oops. # skip-blame because this is bytes prefixes and a s/iteritems/items/ Differential Revision: https://phab.mercurial-scm.org/D1970

python3: whitelist another 24 passing tests Differential Revision: https://phab.mercurial-scm.org/D1911

mdiff: use slice instead of index on bytestr when checking single bytes This is portable to Python 3. Differential Revision: https://phab.mercurial-scm.org/D1992

obsutil: work around filter() being a generator in Python 3 Differential Revision: https://phab.mercurial-scm.org/D1910

cmdutil: add a kludge to make bytes repr() the same on 2 and 3 This fixes the output formatting problems I see in debugobsolete. I still am seeing some effectflag differences, which we'll need to tackle separately. I'm not in love with this approach. There might be something better we could do, and I'd love it if someone else wanted to take a run at this. Differential Revision: https://phab.mercurial-scm.org/D1909

wireprotoserver: make name part of protocol interface This is a required part of the interface. Abstract properties must be defined at type creation time. So we make name a @property. Differential Revision: https://phab.mercurial-scm.org/D1991

wireprotoserver: make abstractserverproto a proper abstract base class Plug in the abc module so we can have run-time validation of type conformance. Differential Revision: https://phab.mercurial-scm.org/D1990

wireprotoserver: make response handling attributes private The send* methods are specific to sshserver and aren't part of the common protocol interface. So rename them accordingly. The handlers dict is also specific to sshserver and is related to these methods. So give it the same treatment. Differential Revision: https://phab.mercurial-scm.org/D1989

wireprotoserver: make some instance attributes private sshserver attempts to conform to a well-defined interface. The instance attributes changed as part of this commit don't appear to be part of that interface. So prefix them with _ to mark them as private. Differential Revision: https://phab.mercurial-scm.org/D1988

wireprotoserver: remove sshserver.getarg() AFAICT the last consumer of this helper method was removed by d054cc5c7737 in 2010. Differential Revision: https://phab.mercurial-scm.org/D1987

wireprotoserver: remove lock references AFAICT sshserver.lock is unused. The last caller of it disappeared in 9f6e0e7ef828 ~18 months ago as part of removing code to support ancient wire protocol commands. Let's remove some dead code. Differential Revision: https://phab.mercurial-scm.org/D1986

wireprotoserver: remove support for do_<command> handlers (API) Old versions of wire protocol handlers relied on methods named do_<command> to handle wire protocol commands. The last definition of these methods on sshserver was removed by 9f6e0e7ef828 ~2 years ago. I think it's time to not support this mechanism for defining command handlers. .. api:: sshserver no longers looks for wire protocol command handlers in methods named do_<command>. Use @wireproto.wireprotocommand to declare wire protocol command handler functions. Differential Revision: https://phab.mercurial-scm.org/D1985

wireprotoserver: make attributes private These aren't part of the protocol interface. So they should be _ prefixed. Differential Revision: https://phab.mercurial-scm.org/D1984

wireprotoserver: remove unused response attribute I think the last use of this attribute was removed by 2f8adc60e013 in 2010. Differential Revision: https://phab.mercurial-scm.org/D1983

wireprotoserver: rename p to proto To aid readability. And to make it easier to search the code base for protocol instances. Differential Revision: https://phab.mercurial-scm.org/D1982

wireprotoserver: add some blank lines between methods Let's make this file slightly easier to read. Differential Revision: https://phab.mercurial-scm.org/D1981

tests: start a set of unit tests for mdiff.py, starting with splitnewlines I want to optimize splitnewlines, so writing tests seems prudent. Differential Revision: https://phab.mercurial-scm.org/D1972 # no-check-commit because of test_ funciton

perf: add a perfunidiff command for benchmarking unified diff speed Differential Revision: https://phab.mercurial-scm.org/D1971

wireprotoserver: move abstractserverproto class from wireproto Let's have the interface live next to things that define it. Differential Revision: https://phab.mercurial-scm.org/D1969

wireprotoserver: move sshserver into module (API) Let's welcome the SSH protocol handler to our new central home for protocol handlers. .. api:: Content from mercurial.sshserver has been moved into mercurial.wireprotoserver. Differential Revision: https://phab.mercurial-scm.org/D1968 # no-check-commit because we're moving a foo_bar function

wireprotoserver: don't import symbol from hgweb.common Importing hgweb.common requires importing hgweb. hgweb/__init__.py contains a bit of code and does imports of large parts of the hgweb.* module tree. All we need is a constant defining the integer status code for HTTP OK. So just redefine HTTP_OK in wireprotoserver.py and avoid the excessive imports. Differential Revision: https://phab.mercurial-scm.org/D1967

wireprotoserver: rename call to callhttp In the context of multiple handlers, call() is ambiguous. Differential Revision: https://phab.mercurial-scm.org/D1966

wireprotoserver: rename hgweb.protocol to wireprotoserver (API) The HTTP wire protocol server / response handler is currently defined in the hgweb sub-package. That only kind of makes sense. hgweb does contain most of the HTTP server code. However, hgweb is more tailored for providing HTTP server and WSGI scaffolding and serving hgweb requests. The wire protocol is kind of its own beast. In addition, the code for HTTP and SSH wire protocol handling is actually pretty small and it needs to stay in sync to ensure parity between the transport implementations. We rename mercurial/hgweb/protocol.py to mercurial/wireprotoserver.py. The new module will eventually become the home of the SSH handler as well. .. api:: Content from mercurial.hgweb.protocol has been moved to mercurial.wireprotoserver. Differential Revision: https://phab.mercurial-scm.org/D1965

testrunner: fix updating of .testtimes file We attempt to write the 5 most recent test timings to a file called .testtimes, but we read previous results from a file called .testtimes- (including the hyphen), so we ended up no more than a single time per test. Differential Revision: https://phab.mercurial-scm.org/D1961

testrunner: make reading of test times work with #testcases Due to a bug that will be fixed in the next patch, we never actually read back .testcases, so we didn't notice that it could not be parsed successfully when there are #testcases tests. The parsing failed on lines like "test-amend-subrepo.t (case obsstore-off) 32.420" because we used a simple string.split() call and expected all parts but the first to be floating point numbers (and "(case" isn't, for example). Fix by using a regex instead. Differential Revision: https://phab.mercurial-scm.org/D1960