util: factor out shellsplit() function It turned out to be more than the simple posix=True|False flag, so let's introduce a platform function. I also made it py3 ready.

diff: do not split function name if character encoding is unknown Only ASCII characters can be split reliably at any byte positions, so let's just leave long multi-byte sequence long. It's probably less bad than putting an invalid byte sequence into a diff. This doesn't try to split the first ASCII slice from multi-byte sequence because a combining character may follow.

histedit: use repo.revs() instead of repo.set() where revisions are needed Follows up 72da480db4a5. This is just a micro optimization, but looks slightly nicer.

acl: replace bare getpass.getuser() by platform function Follows up dbadf28d4db0. bytestr() shouldn't be applied here because getuser() isn't guaranteed to be all in ASCII. This change means GetUserNameA() is used on Windows, but that's probably better than trying to get the current user name in UNIX way.

context: remove basectx.__int__ (API) basectx is the only type in the repo having __int__ implemented. This magic method can result in unexpected coercion. Furthermore, having it implemented is wrong for some contexts, since rev() may return None in some cases. Previous commits removed known cases in core where contexts are coerced to integers. So let's delete basectx.__int__. This commit is a bit dangerous. While the test suite passes, there are likely still some callers in core that rely on __int__ that don't have test coverage. An alternative would be to issue a deprecation warning and let this bake for a few releases. .. api:: context.basectx no longer implements __int__. Context instances will no longer cast to ints. Consumers should call ``ctx.rev()`` instead. Differential Revision: https://phab.mercurial-scm.org/D2433

patchbomb: resolve revs before evaluating %ld revset Weaning off of basectx.__rev__. Differential Revision: https://phab.mercurial-scm.org/D2432

histedit: resolve revs before evaluating %ld revset We want to stop relying on basectx.__int__. That means we can't use the %ld revset operator with an iterable of contexts. So we expand an iterable of contexts into a list of revs before calling into the revset. Perhaps it would be worthwhile to add a revset format operator that recognizes context instances so we can just pass contexts as revset arguments? Differential Revision: https://phab.mercurial-scm.org/D2431

split: use ctx.rev() instead of %d % ctx And rename a variable for clarity while we're here. This is part of our effort to wean off basectx.__int__. Differential Revision: https://phab.mercurial-scm.org/D2430

commands: use ctx.rev() instead of %d % ctx Weaning off basectx.__int__. Differential Revision: https://phab.mercurial-scm.org/D2429

cmdutil: use ctx.rev() instead of %d % ctx Weaning off basectx.__int__. Differential Revision: https://phab.mercurial-scm.org/D2428

rebase: use ctx.rev() instead of %d % ctx Weaning away from basectx.__int__. Differential Revision: https://phab.mercurial-scm.org/D2427

histedit: use ctx.rev() instead of %d % ctx Weaning off basectx.__int__. Differential Revision: https://phab.mercurial-scm.org/D2426

histedit: rename variables so they have "ctx" in them It is convention for context instances to end with "ctx." Until we have type annotations, this makes auditing much, much easier. Differential Revision: https://phab.mercurial-scm.org/D2425

cmdutil: use ctx.rev() instead of int(ctx) Continuing to wean off basectx.__int__. Differential Revision: https://phab.mercurial-scm.org/D2424

templatekw: use ctx.rev() instead of casting context to int basectx has __int__ implemented. Per mailing list discussion with Yuya, we don't like having this implemented because it is too much magic and in some cases rev() will return None, which isn't an int. So convert a `'%d' % ctx` to ctx.rev() instead. Differential Revision: https://phab.mercurial-scm.org/D2423

py3: fix handling of keyword arguments at more places The keys of keyword arguments of Python 3 should be str, which is why we need to prevent getting the b'' prefix added by the transformer or convert keys to str using pycompat.strkwargs() Differential Revision: https://phab.mercurial-scm.org/D2420

py3: use '%d' for integers instead of '%s' Differential Revision: https://phab.mercurial-scm.org/D2419

merge with stable

py3: use '//' for integer division in hgweb/common.py '/' on Python 3 does the float division. Differential Revision: https://phab.mercurial-scm.org/D2422

py3: use util.forcebytestr to convert error messages to bytes Differential Revision: https://phab.mercurial-scm.org/D2421

py3: add b'' prefixes to config options in test/badserverext.py Differential Revision: https://phab.mercurial-scm.org/D2418

py3: replace file() with open() file() is not present in Python 3. Differential Revision: https://phab.mercurial-scm.org/D2417

py3: make sure regexes are bytes # skip-blame because we added just b'' prefix Differential Revision: https://phab.mercurial-scm.org/D2416

py3: convert dict.items() to list explicitly On python 3, dict.items() returns a dict_items object which cannot be sorted. Differential Revision: https://phab.mercurial-scm.org/D2415

py3: whitelist another 8 passing tests Differential Revision: https://phab.mercurial-scm.org/D2414

py3: use '//' for integer divisions Differential Revision: https://phab.mercurial-scm.org/D2413

py3: use "%d" for integers instead of "%s" util.parsedate() returns a tuple of integers. Writing this patch, I wish we had some type hinting. Differential Revision: https://phab.mercurial-scm.org/D2412

py3: use util.forcebytestr instead of str to convert error messages Differential Revision: https://phab.mercurial-scm.org/D2411

py3: use pycompat.byteskwargs() to fix keyword arguments handling I missed these when I fixed keyword arguments handling in the whole file. Differential Revision: https://phab.mercurial-scm.org/D2410

setup: only allow Python 3 from a source checkout (issue5804) People are running `pip install Mercurial` with Python 3 and that is working because not everything performs a Python version compatibility check. Modern versions of pip do recognize the "python_requires" keyword (https://packaging.python.org/tutorials/distributing-packages/#python-requires) which we set if using setuptools. But this isn't set nor recognized everywhere. To prevent people from accidentally installing Mercurial with Python 3 until Python 3 is officially supported, have setup.py fail when run with Python 3. But don't fail if we're running from a source checkout, as we don't want to anger Mercurial developers hacking on Python 3 nor Mercurial's test automation running from source checkouts. People running setup.py from source checkouts could still fall through a Python 3 crack. But at least the `pip install Mercurial` attempt will get nipped in the bud.

tests: add HTTP POST and PUT support to the $LOGDATE$ substitution The lfs serving code uses both POST and PUT requests (and there's existing support for POST).

py3: make regex bytes in hgweb/webcommands.py # skip-blame because just b'' prefix

py3: replace types.NoneType with type(None) types.NoneType is not present in Python 3.

py3: add missing b'' in test-arbitraryfilectx.t # skip-blame as just b'' prefix

py3: pass ctx.rev() instead of ctx in range()

py3: add b'' prefixes in test-alias.t # skip-blame as it's just b'' prefixes

py3: add b'' prefixes in test-revset.t # skip-blame because it's just b''

py3: make sure we use bytes in generate-working-copy-states.py

py3: fix keyword arguments handling in hgext/acl.py # skip-blame because we added r'' prefixes

py3: use pycompat.bytestr to convert str returned by getpass.getuser to bytes

py3: add b'' prefixes in test-abort-checkin.t # skip-blame because we just added a b'' prefix.

py3: add b'' prefixes in test-dispatch.py # skip-blame because this is just adding b'' prefixes

cleanup: say goodbye to manifestv2 format This experiment was a bust: we'd hoped for smaller repository sizes, but things got larger. Google ended up rolling out tree manifests in a format that's compatible with the original manifest format, and I believe Facebook is doing the same. This code was never implemented as native speedups, so I'm pretty comfortable saying nobody is using the experimental feature. Let's rip it out. I noticed this code still kicking around because I was investigating a repo corruption issue for timeless. .. bc:: Support for the experimental manifestv2 format has been removed, as it was never completed and failed to meet expectations. Differential Revision: https://phab.mercurial-scm.org/D2393

wireproto: document the wonky push protocol for SSH It took me several minutes to figure out how the "unbundle" protocol worked. It turns out that the SSH protocol handler sends an empty reply that is interpreted as "OK to send" and only then does the client send the bundle payload. On top of that, the response is different depending on whether the operation was successful or not. I nearly pulled out my hair deciphering this. Differential Revision: https://phab.mercurial-scm.org/D2385

wireprototypes: move baseprotocolhandler from wireprotoserver This is needed to prevent a cycle in an upcoming commit. Differential Revision: https://phab.mercurial-scm.org/D2384

sshpeer: defer pipe buffering and stderr sidechannel binding The doublepipe and bufferedinputpipe types facilitate polling multiple pipes without blocking and for automatically forwarding output from the SSH server's stderr pipe to the ui as "remote: " output. This all happens automatically and callers don't need to worry about reading from multiple pipes. An upcoming change to version 2 of the SSH wire protocol will eliminate the use of stderr and move side-channel output into the "main" pipe. The SSH wire protocol will use a pair of unidirectional pipes - just like the HTTP protocol. In this future world, the doublepipe primitive isn't necessary because the stderr pipe won't be used. To prepare for eventually not using doublepipe, we delay the construction of this primitive from immediately after connection establishment to inside construction of the peer instance. The handshake occurs between these two events. So we had to teach the handshake code to read from stderr so any stderr output from the server is still attended to early in the connection lifetime. Differential Revision: https://phab.mercurial-scm.org/D2383

sshpeer: make pipe polling code more explicit "hasbuffer" is a property on our special bufferedinputpipe class. When reading this code, I thought it might have had something special to do properties on built-in types. But "hasbuffer" doesn't appear in the CPython code base for either 2.7 or 3.7, so the answer is no. Let's make the code more explicit about the fact that it deals with our special bufferedinputpipe type. Differential Revision: https://phab.mercurial-scm.org/D2382

tests: store protocol payload in files Upcoming changes to version 2 of the SSH protocol will introduce binary components to the protocol. It will be easier to eliminate trailing newlines and use binary in the tests if the protocol payload is being generated by Python. So use inline Python to write payloads to files and pipe those files to server processes instead of shell strings/variables. Differential Revision: https://phab.mercurial-scm.org/D2381

sshpeer: return framed file object when needed Currently, wireproto.wirepeer has a default implementation of _submitbatch() and sshv1peer has a very similar implementation. The main difference is that sshv1peer is aware of the total amount of bytes it can read whereas the default implementation reads the stream until no more data is returned. The default implementation works for HTTP, since there is a known end to HTTP responses (either Content-Length or 0 sized chunk). This commit teaches sshv1peer to use our just-introduced "cappedreader" class for wrapping a file object to limit the number of bytes that can be read. We do this by introducing an argument to specify whether the response is framed. If set, we returned a cappedreader instance instead of the raw pipe. _call() always has framed responses. So we set this argument unconditionally and then .read() the entirety of the result. Strictly speaking, we don't need to use cappedreader in this case and can inline frame decoding/read logic. But I like when things are consistent. The overhead should be negligible. _callstream() and _callcompressable() are special: whether framing is used depends on the specific command. So, we define a set of commands that have framed response. It currently only contains "batch." As a result of this change, the one-off implementation of _submitbatch() in sshv1peer can be removed since it is now safe to .read() the response's file object until end of stream. cappedreader takes care of not overrunning the frame. Differential Revision: https://phab.mercurial-scm.org/D2380

sshpeer: move logic for sending a request into a new function The **args being used to pass arbitrary command arguments is limiting because it makes it harder to control behavior of the function. We factor most of _callstream() into a new function that doesn't use **args. Differential Revision: https://phab.mercurial-scm.org/D2379

help: fix wording describing SSH requirements

graphlog: document what "_" and "*" mean Documenting "*" should've been a part of 9b3f95d9783d, but I somehow didn't notice that the symbols are explained in the command's help text.

sshpeer: rename _recv and _send to _readframed and _writeframed Because it is reading and writing a chunk of data with a well-defined size. "recv" and "send" make it sound like things are a direct proxy to the underlying pipe, which they aren't. Differential Revision: https://phab.mercurial-scm.org/D2378

util: add a file object proxy that can read at most N bytes Sometimes we have data of a known size within a stream. For performance reasons, we don't want to pre-read this data (we want to allow consumers to read on demand). For simplicitly reasons, we don't want callers to necessarily know their data is coming from within an outer stream and there is a limit to how much they should read. The class introduced by this commit provides a very simple proxy around an underlying file object that allows the consumer to .read() up to N bytes from the file object. Attempts to read past this many bytes results in a simulated EOF. Differential Revision: https://phab.mercurial-scm.org/D2377

patches: release the GIL while applying the patch This will allow multiple threads to apply patches at the same time.

perfbranchmap: allow to select the filter to benchmark Running the branchmap computation on all filter levels can be expensive. Narrowing the run to some specific filters can speed up benchmarking time when working only on a subset of filter levels.

perfbranchmap: display 'unfiltered' for unfiltered performance This is slightly clearer than "None" and will help with coming changes to select the filter level we want timing for.

py3: two more narrow tests passing Differential Revision: https://phab.mercurial-scm.org/D2390

narrowbundle2: more kwargs native string fixes This gets test-narrow.t to *almost* pass. Something appears to be borked in producing bundles, but only some of the time? I'm lost, but this change is at least a clear improvement. # skip-blame just more r prefixes on strings Differential Revision: https://phab.mercurial-scm.org/D2389

py3: whitelist another 11 passing tests This is most of narrow. There's still some buglets at the margins, but it's pretty good progress for not a lot of work. Differential Revision: https://phab.mercurial-scm.org/D2388