test-merge-tools: stabilize for Windows This masks the Windows argument parsing insanity[1], so it needs a bit of explanation. (The security reference in the footnote is probably useful to keep in mind if we ever whitelist certain in-repo config settings.) 9037c29e9f53 introduced tests that were failing on Windows with an unbalanced double quote[2]. What ends up happening here is util.shellquote() is double quoting the file path, but the shell script is placing this ->": "<- right next to it. So cmd.exe gets launched with 'lb:base": ""c:\...\f~base.xyz"', which got interpreted as 'lb:base: "c:\...\f~base.xyz'. If the test is adjusted to quote like "lb:$labelbase: $base", then MSYS runs interference and strips the '\' characters. I was able to get the expected result by dropping the quotes from '": "', and changing the space to underscore. But since we need to glob away the C: part anyway, just glob away the quote and leave the test unchanged. [1] https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/ [2] https://buildbot.mercurial-scm.org/builders/Win7%20x86_64%20hg%20tests/builds/441/steps/run-tests.py%20%28python%202.7.13%29/logs/stdio

py3: replace file() with open() in test-convert-hg-source.t file() is not present in Python 3. It also makes sure we read and write in bytes mode on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2132

py3: replace file() with open() in test-encoding-align.t file() is not present in Python 3. This patch also makes sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2131

py3: replace file() with open() in test-encoding.t file() is not present in Python 3. This also makes sure we write things in bytes mode in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2130

py3: make sure we return str from __repr__ Differential Revision: https://phab.mercurial-scm.org/D2109

py3: check for bytes instead of str in hg.share() Differential Revision: https://phab.mercurial-scm.org/D2151

py3: port metaedit extension to Python 3 Without these changes, this extension is responsible for a test of test failures. Differential Revision: https://phab.mercurial-scm.org/D2150

py3: pass system string to email.message.Message.set_type() Python 3 insists the type is a str. Differential Revision: https://phab.mercurial-scm.org/D2149

py3: open temporary file in binary mode Otherwise things fail later when we write bytes to the handle. Differential Revision: https://phab.mercurial-scm.org/D2148

py3: use email parser that operates on bytes email.parser.Parser() operates on str in both Python 2 and 3. Python 3.2 introduced the email.parser.BytesParser(), which works like Parser except it accepts bytes. We implement the pycompat helper as a function so we lazily import the "email" module. Differential Revision: https://phab.mercurial-scm.org/D2147

py3: port ext-phase-report.py extension The custom module importer doesn't run on Python files in the tests directory. So we need the source to be compatible with both Python 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D2145

py3: more robustly cast UUID to bytes The UUID type only has __str__ implemented. So we need to cast to bytes on Python 3. We need an actual bytes instance here (bytestr won't do) because the re.escape() later iterates over characters and characters need to behave like ints, not bytes instances of length 1. Differential Revision: https://phab.mercurial-scm.org/D2144

py3: use bytes() to cast context instances __str__ and __bytes__ are both implemented on context types. However, __str__ behaves differently on Python 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D2143

py3: convert __doc__ to bytes Differential Revision: https://phab.mercurial-scm.org/D2142

py3: cast repr() result to bytes Differential Revision: https://phab.mercurial-scm.org/D2141

py3: fix file i/o in test-status.t Use raw string literal for open() mode and pass the proper data type to write(). Differential Revision: https://phab.mercurial-scm.org/D2140

py3: pass system string to socket.getservbyname Differential Revision: https://phab.mercurial-scm.org/D2139

run-tests: report tests that exception occurred in We now record the test that an exception occurred in. We put this information to use by aggregating the count of failures in each test. For each exception, the exception report now prints the total number of tests having that exception and the test with the least number of exceptions exhibiting that failure. The exception list is now sorted by (total count, tests impacted, count of failures in least failing test). This allows us to: * Assess how widespread a failure is. Some exceptions occur a lot in a few tests. Others occur over many tests. * Easily run a test exhibiting an exception without having to find a failure in test output. * Find and fix low hanging fruit (e.g. exceptions that are the only failure in a test). Here's an example of the new output: 199 (4 tests) /home/gps/src/hg/hgext/blackbox.py:191: %b requires a bytes-like object, or an object that implements __bytes__, not 'str' (test-devel-warnings.t - 1 total) 142 (19 tests) /home/gps/src/hg/hgext/mq.py:655: list indices must be integers or slices, not bytes (test-hardlinks.t - 1 total) 140 (20 tests) /home/gps/src/hg/mercurial/patch.py:296: string argument expected, got 'bytes' (test-audit-subrepo.t - 1 total) 101 (15 tests) /home/gps/src/hg/hgext/convert/convcmd.py:60: encode() argument 1 must be str, not bytes (test-convert-clonebranches.t - 1 total) 90 (2 tests) /home/gps/src/hg/hgext/mq.py:456: can't concat str to bytes (test-mq-qqueue.t - 1 total) 87 (2 tests) /home/gps/src/hg/mercurial/branchmap.py:380: %b requires a bytes-like object, or an object that implements __bytes__, not 'FileNotFoundError' (test-branches.t - 2 total) 85 (22 tests) /home/gps/src/hg/mercurial/sshpeer.py:223: cannot convert 'UUID' object to bytes (test-bundle2-pushback.t - 1 total) 1 (1 tests) /home/gps/src/hg/mercurial/formatter.py:254: %b requires a bytes-like object, or an object that implements __bytes__, not 'str' (test-debugextensions.t - 2 total) 1 (1 tests) /home/gps/src/hg/hgext/convert/convcmd.py:420: startswith first arg must be str or a tuple of str, not bytes (test-convert-authormap.t - 2 total) 1 (1 tests) /home/gps/src/hg/mercurial/revlog.py:797: '>=' not supported between instances of 'NoneType' and 'int' (test-unionrepo.t - 1 total) 1 (1 tests) /home/gps/src/hg/hgext/show.py:129: %b requires a bytes-like object, or an object that implements __bytes__, not 'str' (test-show.t - 1 total) Differential Revision: https://phab.mercurial-scm.org/D2138

py3: use raw string for file open mode Otherwise the module importer rewrites it to bytes on Python 3, which angers Python 3. Differential Revision: https://phab.mercurial-scm.org/D2137

py3: convert content-type to bytes This fixes the exception: TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'str' On the line: ui.debug('Content-Type: %s\n' % content_type) And it prevents additional exceptions that use the content_type variable later. Differential Revision: https://phab.mercurial-scm.org/D2136

gpg: print unknown key IDs in their entirety Shortening the key is nice in theory but it results in ambiguity which can be exploited. Therefore, when encountering an unknown key ID we should print the whole ID returned by gpg. This may or may not be the whole key, however it will match the user preference set in gpg configuration. Furthermore, the key ID shortening had a couple of issues: (1) it truncated the key ID (dropping the last digit and outputting only 15 hex digits) making it very hard to find the correct key on a key server (2) since only 15 digits were fed into shortkey(), it always emitted the ui.debug() warning

tests: fix check for --graph option in test-log-linerange.t Previous test about incompatibility of --line-range and --graph options was problematic because, even without --graph option, the command would have failed because specified path (dir/baz) did not exist as it got dropped in previous test statements. New test statements should work without --graph. This prepares for making --line-range compatible with --graph option.

py3: use "%d" instead of "%s" for integers Differential Revision: https://phab.mercurial-scm.org/D2135

py3: replace file() with open() in test-contrib.t file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2134

py3: replace file() with open() in test-eol.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2129

py3: replace file() with open() in test-extdiff.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2128

py3: replace file() with open() in test-fileset.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2127

py3: replace file() with open() in test-import-bypass.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2126

py3: replace file() with open() in test-import-eol.t file() is not present in Python 3. This patch also adds b'' prefixes to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2125

py3: replace file() with open() in test-import-git.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2124

py3: replace file() with open() in test-import.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2123

py3: replace file() with open() in test-mactext.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2122

py3: replace file() with open() in test-mq-missingfiles.t file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2121

py3: replace file() with open() in test-mq-qimport.t file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2120

py3: replace file() with open() in test-patch-offset.t file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2119

py3: replace file() with open() in test-relink.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2118

py3: replace file() with open() in test-subrepo-missing.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2117

py3: replace file() with open() in test-tag.t file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2116

py3: replace file() with open() in test-transplant.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2115

py3: replace file() with open() in test-walk.t file() is not present in Python 3. This patch also adds a b'' prefix to make sure we write bytes in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2114

py3: replace file() with open() in test-win32text.t This patch also adds b'' prefix to make sure we write bytes on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2113

py3: replace file() with open() in test-convert-p4-filetypes.t file() is not present in Python 3. While I was here, I added b'' prefix to make sure we write things correctly in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2112

py3: replace file() with open() in test-convert-mtn.t While I was here, I added b'' prefixes to make sure we write things correctly in bytes mode on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2111

py3: replace file() with open() in test-clone.t While I was here, I made sure we write things correctly in bytes mode. Differential Revision: https://phab.mercurial-scm.org/D2110

merge: cut import cycle at merge -> extensions This seems less bad because checking fsmonitor looks like a temporary hack, and cmdutil -> mergemod should be a valid dependency.

subrepo: split non-core functions to new module Resolves import cycle caused by subrepo -> cmdutil. Still we have another cycle, cmdutil -> context -> subrepo, but where I think importing context is wrong. Perhaps we'll need repo.makememctx().

diff: remove fp.write() wrapper which drops label argument It's no longer needed since we've split labeled write() from file-like write().

log: pack filematcher and hunksfilter into changesetdiffer object This is just a way of getting rid of clumsy makefilematcher/makehunksfilter arguments. There might be a better abstraction, but I don't think this is bad. This makes filematcher and hunksfilter available by default, but that should be fine.

log: drop dead code to concatenate --line-range patterns and pats It's disabled since 2e45bbd3db7b, and the current implementation is unlikely to be reused.

logcmdutil: create hunksfilter and filematcher even if no diff option given It's okay since 5fe6f946f111, "log: allow matchfn to be non-null even if both --patch/--stat are off."

logcmdutil: unindent diff generator of changesetprinter Prepares for the next few patches which will make matchfn and hunksfilterfn always available.

logcmdutil: hold makefilematcher/makehunksfilter() by changesetpriner (API) This merges self.matchfn and self.show(matchfn) into self._makefilematcher, and does the same for hunksfilter. Because changesetprinter seems to have too many optional arguments, makefilematcher() and makehunksfilter() will be packed into one object by later patch.

log: pass ctx to makefilematcher() and makehunksfilter() functions This isn't important, but seems more consistent as changesetprinter.show() takes a ctx, not a revision number.

tests: allow age to go up to triple digits in test-shelve.t We started to glob the age of shelved changes in 51934fc796c0, which says that sometimes tests run slow and `hg shelve --list` says "2s" instead of "1s". However in some instances, like [1], tests run so slow that the age goes up to double digits ("13s" in that case). When that happens, `hg shelve --list` output has less white spaces after the age, so let's glob the spaces too. We probably won't ever need to handle triple digits there, but I went ahead and left only 2 required white spaces in total. [1]: https://buildd.debian.org/status/fetch.php?pkg=mercurial&arch=sparc64&ver=4.5-1&stamp=1518360804&raw=0

lfs: teach the 'lfs()' fileset to handle removed files The callstatus setting is required to notice the removal of 'lfs.test' in rev 6 in the tests, even though this isn't directly calling mctx.status(). However, it's not needed to get the results in the tests for `hg status`, so I'm probably missing something.

lfs: teach '{lfs_files}' to handle removed files

lfs: allow a pointer to be extracted from a context that removes the file This is needed to let 'set:lfs()' and '{lfs_files}' work normally on removed files. Yuya suggested returning a null pointer for removed files, instead of the pointer from the parent. The first attempt at this was to return None for a non LFS file, and a (pointer, ctx) tuple to hold the pointer and context (or parent pointer and context for a removed file). But this complicated the callers, even the ones that didn't care about removed files. Instead, let's use {} to represent a removed pointer. This has the added convenience of being a useful representation in the template language, and only affects the callers that care about removed files (and only slightly). Since pointers are explicitly serialized with a call to a member function, there is no danger of writing these to disk.

rebase: make "successors" a set in _computeobsoletenotrebased() There's no apparent reason for this variable to be a list and this avoids converting it to a set when needed.

merge with stable

rebase: do not consider extincts for divergence detection (issue5782) Extinct obsolete changesets cannot cause divergence upon rebase. We compute these obsoletes without a non-obsolete successor (extincts) in _computeobsoletenotrebased() and then filter them out from the set of obsolete revisions to rebase before getting into _checkobsrebase() to check for divergence candidates.

rebase: eliminate node from successors early in _computeobsoletenotrebased()

rebase: add a test case for issue5782 Issue 5782 reports that rebase incorrectly aborts when trying to rebase an extinct revision (an obsolete revision with only obsolete successor). We add a test to demonstrate this: the first "hg rebase" command aborts with the divergence warning while, when allowing divergence, the rebase completes and does not actually produce any divergence.

test: glob the temporary directory out of temporary file path The temporary directory used by python might be outside of '/tmp/' (eg: /dev/shm/) so we glob that part out.

tests: raise a better error when patterns are wrongly formatted It is fairly easy to make mistakes when merging conflict in the pattern file. A common mistake is to forget adding an extra trailing comma changing the length of the tuple. We now detect such error and raise a better error message that helps to find it.

lfs: add a fileset for detecting lfs files This currently has the same limitation as {lfs_files}, namely it doesn't report removed files. We may want a dedicated 'lfs()' revset for efficiency, but combining this with the 'contains()' revset should be equivalent for now. Combining with 'set:added()' or 'set:modified()' inside 'files()' should be equivalent to a hypothetical lfs_adds() and lfs_modifies(). I wonder if there's a way to tweak the filesets to evaluate lazily, to close the efficiency gap. It would also be interesting to come up with a template filter for '{files}' that looked at the pattern to 'files()', and filtered appropriately. While passing a fileset as the pattern to `hg log` does filter '{files}', the set is evaluated against the working directory, so there's no way to list all non-lfs files above a certain size in all revisions, for example.

tests: stabilize ssh tests on Windows This seems like a somewhat common type of failure (double vs single quote), so I'm interested in ideas about how to avoid this. I doubt that we should automatically fall back from single quote to double quote, like with '/' vs '\'.

wireprotoserver: rename abstractserverproto and improve docstring The docstring isn't completely accurate for the current state of the world. But it does describe the direction future patches will be taking things. Differential Revision: https://phab.mercurial-scm.org/D2065

wireprotoserver: document and improve the httplib workaround This workaround dates all the way back to a42d27bc809d in 2008. The code is esoteric enough to warrant an inline explanation. So I've added one. At the time the code was written, the only wire protocol command that accepted an HTTP request body was "unbundle." In the years since, we've grown the ability to accept command arguments via HTTP POST requests. So, the code has been changed to apply the httplib workaround to all HTTP POST requests. While staring at this code, I realized that the HTTP response body in case of error is always the same. And, it appears to mimic the behavior of a failed call to the "unbundle" command. Since we can hit this code path on theoretically any protocol request (since self.check_perm accepts custom auth checking functions which may raise), I'm having a hard time believing that clients react well to an "unbundle" response payload on any wire protocol command. I wouldn't be surprised if our test coverage for this feature only covers HTTP POST calls to "unbundle." In other words, the experimental support for sending arguments via HTTP POST request bodies may result in badness on the client. Something to investigate another time perhaps... Differential Revision: https://phab.mercurial-scm.org/D2064

wireprotoserver: move error response handling out of hgweb The exception handler for ErrorResponse has more to do with the wire protocol than the generic HTTP server. Move the code so it lives alongside other wire protocol code. Differential Revision: https://phab.mercurial-scm.org/D2021

hgweb: move call to protocol handler outside of try..except The protocol handler doesn't raise ErrorResponse. So it doesn't need to be in this `try..except ErrorResponse` block. Differential Revision: https://phab.mercurial-scm.org/D2020

wireprotoserver: move protocol parsing and dispatch out of hgweb Previously, hgweb_mod had code for detecting if the request was for the wire protocol. It would then (eventually) call wireprotoserver.callhttp() to dispatch the request handling. Detection of wire protocol requests is not trivial. There's currently a big gotcha in the handling of the "cmd" request parameter, for example. Furthermore, in the near future we will have a second HTTP protocol handler. Its mechanism for calling commands will be a bit different. And we don't want the low-level logic for detecting protocol commands to live in hgweb. We establish a new function in wireprotoserver for detecting an HTTP protocol request and for giving the caller an easy-to-use mechanism for dispatching requests to it. Some wire protocol specific functionality still lives in hgweb. This will be addressed in subsequent commits. Differential Revision: https://phab.mercurial-scm.org/D2019

largefiles: register wire protocol commands with modern APIs The wireproto.wireprotocommand decorator is the preferred mechanism for registering wire protocol commands. In addition, wireproto.commands is no longer a 2-tuple and use of that 2-tuple API should be considered deprecated. This commit ports largefiles to use wireproto.wireprotocommand() and ports to the "commandentry" API. As part of this, the definition of the "lheads" wire protocol command is moved to the proper stanza. We stop short of actually using wireprotocommand as a decorator in order to minimize churn. We should ideally move wire protocol commands to the registrar mechanism. But that's for another changeset. Differential Revision: https://phab.mercurial-scm.org/D2018

wireproto: function for testing if wire protocol command is available Currently, we perform simple membership testing for whether a wire command is available. In the future, not all wire protocol commands will be available on all transports. For example, a legacy transport may not support newer commands. In preparation of this, teach the protocol handlers to call into a function to determine if a wire protocol command is available. That function currently does membership testing like before, so behavior should be identical. In the case of the HTTP server, behavior is a bit wonkier. "cmd" is used by both the wire protocol and hgweb. We do want the protocol handler to handle requests for all commands that look like wire protocol commands, even if they aren't available. Otherwise, the fallback to hgweb would only confuse automated clients and make it easier for hgweb to accidentally implement a "cmd" that is identical to wire protocol commands (since they aren't centrally registered). Differential Revision: https://phab.mercurial-scm.org/D1999

wireproto: define and use types for wire protocol commands Wire protocol commands have historically been declared as 2-tuples in wireproto.commands. There are some additional features I'd like to implement that require going beyond 2-tuples. But because the 2-tuple API (both destructuring assignment and direct assignment into the dict) is used throughout the code base and in 3rd party extensions, we can't do a trivial type change. This commit creates a new "commandentry" type to represent declared wire protocol commands. It implements __getitem__ and __iter__ so it can quack like a 2-tuple. The @wireprotocommand decorator now creates "commandentry" instances. We also create a "commanddict" type to represent the dictionary of declared wire protocol commands. It inherits from "dict" but provides a custom __setitem__ to coerce passed 2-tuples to "commandentry" instances. wireproto.commands is now an instance of this type. Various callers in core rely on the new functionality. And tests pass. So I'm reasonably confident things will "just work" in 3rd party extensions as well. Differential Revision: https://phab.mercurial-scm.org/D1998

wireproto: improve docstring for @wireprotocommand I'm about to add more arguments and want them to be documented. Plus, good documentation is nice to have. Differential Revision: https://phab.mercurial-scm.org/D1997

wireproto: remove unnecessary exception trapping The `try..except error.Abort` was added in 8474be4412ca back in 2012. The intent was to ensure a failing pushkey hook didn't crash the server. Since that changeset, repo.pushkey() and the hooks mechanism is now much more robust about trapping errors itself. As such, we no longer need this try..except block. So it has been removed. Differential Revision: https://phab.mercurial-scm.org/D1996

sshpeer: implement peer for version 2 of wire protocol Since the protocol is now negotiated before we construct a peer instance, we can return the negotiated protocol from the handshake function and instantiate an appropriate peer class for the protocol. Version 2 of the SSH protocol is currently identical to version 1 post handshake. So our sshv2peer class just inherits from sshv1peer for the time being. This will obviously change over time. Differential Revision: https://phab.mercurial-scm.org/D2063

sshpeer: rename sshpeer class to sshv1peer (API) With the introduction of version 2 of the SSH wire protocol, we will need a new peer class to speak that protocol because it will be too difficult to shoehorn a single class to speak two protocols. We rename sshpeer.sshpeer to sshpeer.sshv1peer to reflect the fact that there will be multiple versions of the peer depending on the negotiated protocol. .. api:: sshpeer.sshpeer renamed to sshpeer.sshv1peer. Differential Revision: https://phab.mercurial-scm.org/D2062

sshpeer: initial definition and implementation of new SSH protocol The existing SSH protocol has several design flaws. Future commits will elaborate on these flaws as new features are introduced to combat these flaws. For now, hopefully you can take me for my word that a ground up rewrite of the SSH protocol is needed. This commit lays the foundation for a new SSH protocol by defining a mechanism to upgrade the SSH transport channel away from the default (version 1) protocol to something modern (which we'll call "version 2" for now). This upgrade process is detailed in the internals documentation for the wire protocol. The gist of it is the client sends a request line preceding the "hello" command/line which basically says "I'm requesting an upgrade: here's what I support." If the server recognizes that line, it processes the upgrade request and the transport channel is switched to use the new version of the protocol. If not, it sends an empty response, which is how all Mercurial SSH servers from the beginning of time reacted to unknown commands. The upgrade request is effectively ignored and the client continues to use the existing version of the protocol as if nothing happened. The new version of the SSH protocol is completely identical to version 1 aside from the upgrade dance and the bytes that follow. The immediate bytes that follow the protocol switch are defined to be a length framed "capabilities: " line containing the remote's advertised capabilities. In reality, this looks very similar to what the "hello" response would look like. But it will evolve quickly. The methodology by which the protocol will evolve is important. I'm not going to introduce the new protocol all at once. That would likely lead to endless bike shedding and forward progress would stall. Instead, I intend to tricle out new features and diversions from the existing protocol in small, incremental changes. To support the gradual evolution of the protocol, the on-the-wire advertised protocol name contains an "exp" to denote "experimental" and a 4 digit field to capture the sub-version of the protocol. Whenever we make a BC change to the wire protocol, we can increment this version and lock out all older clients because it will appear as a completely different protocol version. This means we can incur as many breaking changes as we want. We don't have to commit to supporting any one feature or idea for a long period of time. We can even evolve the handshake mechanism, because that is defined as being an implementation detail of the negotiated protocol version! Hopefully this lowers the barrier to accepting changes to the protocol and for experimenting with "radical" ideas during its development. In core, sshpeer received most of the attention. We haven't even implemented the server bits for the new protocol in core yet. Instead, we add very primitive support to our test server, mainly just to exercise the added code paths in sshpeer. Differential Revision: https://phab.mercurial-scm.org/D2061 # no-check-commit because of required foo_bar naming

internals: refactor wire protocol documentation Upcoming work will introduce a new version of the HTTP and SSH transports. The differences will be significant enough to consider them new transports. So, we now attach a version number to each transport. In addition, having the handshake documented after the transport and in a single shared section made it harder to follow the flow of the connection. The handshake documentation is now moved to the protocol section it describes. We now have a generic section about the purpose of the handshake, which was rewritten significantly. Differential Revision: https://phab.mercurial-scm.org/D2060

revlog: rename 'self.checkinlinesize' into '_enforceinlinesize' The name change has two motivations: 1) The function has no external caller, so we move it to protected space. 2) the function does more than checking it also split the data if we have more data than 'inline' supports.

revlog: add a _datareadfp context manager for data access needs The helper handles: 1) is there a file handle already open that we shall just reuse, 2) is the revlog inlined or not. Using a context manager for all read access will help setting up file pointer caching in later changesets.

revlog: use context manager for data file lifetime in checksize This is clearer, safer and more modern.

revlog: use context manager for index file lifetime in checkinlinesize This is clearer, safer and more modern.

revlog: use context manager for data file lifetime in checkinlinesize This is clearer, safer and more modern.

revlog: use context manager for index file life time in __init__ This is clearer, safer and more modern.

revlog: move index file opening in a method Having file operation centralized into a single spot help to factor common logic out (eg: special flag handling according to the mode). It is also the first step to simplify file handling during batch operation (eg: revlog cloning). However, that part does not seems to be a hotspot yet.

revlog: move datafile opening in a method Having file operation centralized into a single spot help to factor common logic out. It is also the first step to simplify file handling during batch operation (eg: revlog cloning). However, that part does not seems to be a hotspot yet.

parseindex: implement context manager method on the wrapper This is needed for incoming cleanups.

parseindex: also forward keyword argument in a debug wrapper Otherwise, it gets in the way of a coming refactoring.

dumprevlog: handle being passed a mode parameter This makes it closer to an actual file opener. Otherwise, it gets in the way of a coming refactoring.

changelog: add the missing 'closed' property on 'appender' object This mimic file object further.

changelog: implement context manager method for 'appender' object This object mimics a file and was missing the context manager API.

diff: improve ui.write performance when not coloring on Windows Differential Revision: https://phab.mercurial-scm.org/D2022

hgsh: enable clang-format Nothing looks awful, so we can just turn it on. Differential Revision: https://phab.mercurial-scm.org/D2059

chg: enable clang-format on all .c and .h files Nothing in here looks awful, so I think we may as well just do it. # skip-blame because it's just reformatting with no functionality change Differential Revision: https://phab.mercurial-scm.org/D2058

python3: whitelist an additional 11 tests I think these are mostly the result of Pulkit's recent work. Thanks! Differential Revision: https://phab.mercurial-scm.org/D2078

ui: add explicit path to write prompt text bypassing buffers The prompt= flag was added at e35d7f131483, when colorui had its own write() function. Since we've merged colorui to ui, we can simply call the unbuffered write() function.

ui: write prompt text in ui.prompt(), not in ui._readline() self.label() is replaced by label= option, which should make it clearer why we can't pass the text to raw_input(prompt).

logcmdutil: mark changesetprinter.showpatch() as private

logcmdutil: drop default arguments from changesetdisplayer/templater() calls

logcmdutil: make default parameters of changesetprinters consistent

mdiff: use str.startswith/endswith() instead of slicing

patch: unify check_binary and binary flags Follows up 079b27b5a869. If opts.text=True, check_binary is ignored, so we can just pass the binary flag to unidiff(). perfunidiff now takes any inputs as text files, which I think is a desired behavior.

fileset: don't abort when running copied() on a revision with a removed file It looks like AND with any status-y fileset would trigger this, as added() and removed() also failed. The 4.5-rc revision is a convenient test case, but the merge isn't necessary.

py3: make sure we open the file in bytes mode # skip-blame because we are just adding b Differential Revision: https://phab.mercurial-scm.org/D2077

py3: use bytes instead of str Differential Revision: https://phab.mercurial-scm.org/D2076

py3: add b'' prefixes to string literals in test files # skip-blame because we are just adding b'' Differential Revision: https://phab.mercurial-scm.org/D2075

py3: use range instead of xrange xrange is not available on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2074

py3: use open() instead of file() file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2073

py3: use pycompat.bytestr to convert _b85chars to bytes The tranformer does append b'' to the value and make that a bytes but bytes in Python 3 returns the ascii value on getting characters using indexing. Characters of this string are queried using indexing multiple times in the file and to support that we use pycompat.bytestr which returns the bytechrs using indexing. Differential Revision: https://phab.mercurial-scm.org/D2072

py3: use pycompat.bytechr instead of chr Differential Revision: https://phab.mercurial-scm.org/D2071

py3: use pycompat.ziplist instead of zip zip returns a zip object instead of a list on Python 3. Differential Revision: https://phab.mercurial-scm.org/D2070

patches: move assignment outside the conditional Having this movement in its own patch will make the next patch clearer.

sshpeer: remove support for connecting to <0.9.1 servers (BC) 197d10e157ce made this change for the HTTP peer. Let's do the same for the SSH peer. Test output changes as expected. A redundant test has been dropped. .. bc:: Support for connecting to Mercurial servers older than 0.9.1 has been removed. Differential Revision: https://phab.mercurial-scm.org/D2036

sshpeer: document the handshake mechanism The mechanism by which SSH peers establish connections with remotes is wonky and requires a bit of code archeology to understand. While it is already documented in `hg help internals.wireproto`, it helps to have documentation in the code as well. Differential Revision: https://phab.mercurial-scm.org/D2035

sshpeer: move handshake outside of sshpeer With the handshake now performed before a peer class is instantiated, we can now instantiate a different peer class depending on the results of the handshake. Our test extension had to change to cope with the new API. Because we now issue the command via raw I/O calls and don't call _callstream(), we no longer have to register the fake command. (_callstream() uses the command registration to see what args to send). Differential Revision: https://phab.mercurial-scm.org/D2034

sshpeer: inline I/O into _validaterepo() We want to move the handshake code out of the peer class so the peer factory function can perform the handshake and instantiate a proper class depending on the results. To make that refactor easier to read, we first inline I/O functionality into _validaterepo(). Test output for low-level protocol tests didn't change, thus hopefully demonstrating that this refactor didn't change any material behavior. Because we no longer call _callstream(), our test extension for monkeypatching the peer had to change its hook point. Differential Revision: https://phab.mercurial-scm.org/D2033

sshpeer: clean up API for sshpeer.__init__ (API) Our refactoring left the state of sshpeer.__init__ in a poor state. "create" was no longer used. Process/pipe arguments were passed poorly. "name" was really a URL. This commit cleans all that up. .. api:: sshpeer.sshpeer.__init__ now receives arguments describing an existing connection instead of creating a connection itself. Differential Revision: https://phab.mercurial-scm.org/D2032

sshpeer: establish SSH connection before class instantiation We want to move the handshake to before peers are created so we can instantiate a different peer class depending on the results of the handshake. This necessitates moving the SSH process invocation to outside the peer class. As part of the code move, some variables were renamed for clarity. util.popen4() returns stdin, stdout, and stderr in their typical file descriptor order. However, stdin and stdout were being mapped to "pipeo" and "pipei" respectively. "o" for "stdin" and "i" for "stdout" is a bit confusing. Although it does make sense for "output" and "input" from the perspective of the client. But in the context of the new function, it makes sense to refer to these as their file descriptor names. In addition, the last use of self._path disappeared, so we stop setting that attribute and we can delete the redundant URL parsing necessary to set it. Differential Revision: https://phab.mercurial-scm.org/D2031