Yuya Nishihara <yuya@tcha.org> [Tue, 06 Feb 2018 22:55:02 +0900] rev 36048
merge: cut import cycle at merge -> extensions
This seems less bad because checking fsmonitor looks like a temporary hack,
and cmdutil -> mergemod should be a valid dependency.
Yuya Nishihara <yuya@tcha.org> [Tue, 06 Feb 2018 22:36:38 +0900] rev 36047
subrepo: split non-core functions to new module
Resolves import cycle caused by subrepo -> cmdutil. Still we have another
cycle, cmdutil -> context -> subrepo, but where I think importing context
is wrong. Perhaps we'll need repo.makememctx().
Yuya Nishihara <yuya@tcha.org> [Wed, 07 Feb 2018 23:22:53 +0900] rev 36046
diff: remove fp.write() wrapper which drops label argument
It's no longer needed since we've split labeled write() from file-like
write().
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 15:54:18 +0900] rev 36045
log: pack filematcher and hunksfilter into changesetdiffer object
This is just a way of getting rid of clumsy makefilematcher/makehunksfilter
arguments. There might be a better abstraction, but I don't think this is bad.
This makes filematcher and hunksfilter available by default, but that should
be fine.
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 15:50:24 +0900] rev 36044
log: drop dead code to concatenate --line-range patterns and pats
It's disabled since 2e45bbd3db7b, and the current implementation is unlikely
to be reused.
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 15:34:37 +0900] rev 36043
logcmdutil: create hunksfilter and filematcher even if no diff option given
It's okay since 5fe6f946f111, "log: allow matchfn to be non-null even if both
--patch/--stat are off."
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 14:37:04 +0900] rev 36042
logcmdutil: unindent diff generator of changesetprinter
Prepares for the next few patches which will make matchfn and hunksfilterfn
always available.
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 14:07:52 +0900] rev 36041
logcmdutil: hold makefilematcher/makehunksfilter() by changesetpriner (API)
This merges self.matchfn and self.show(matchfn) into self._makefilematcher,
and does the same for hunksfilter. Because changesetprinter seems to have
too many optional arguments, makefilematcher() and makehunksfilter() will
be packed into one object by later patch.
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 13:40:31 +0900] rev 36040
log: pass ctx to makefilematcher() and makehunksfilter() functions
This isn't important, but seems more consistent as changesetprinter.show()
takes a ctx, not a revision number.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 27 Jan 2018 19:02:33 -0500] rev 36039
lfs: teach the 'lfs()' fileset to handle removed files
The callstatus setting is required to notice the removal of 'lfs.test' in rev 6
in the tests, even though this isn't directly calling mctx.status(). However,
it's not needed to get the results in the tests for `hg status`, so I'm probably
missing something.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 27 Jan 2018 19:22:15 -0500] rev 36038
lfs: teach '{lfs_files}' to handle removed files
Matt Harbison <matt_harbison@yahoo.com> [Sat, 27 Jan 2018 18:56:24 -0500] rev 36037
lfs: allow a pointer to be extracted from a context that removes the file
This is needed to let 'set:lfs()' and '{lfs_files}' work normally on removed
files.
Yuya suggested returning a null pointer for removed files, instead of the
pointer from the parent. The first attempt at this was to return None for a non
LFS file, and a (pointer, ctx) tuple to hold the pointer and context (or parent
pointer and context for a removed file). But this complicated the callers, even
the ones that didn't care about removed files.
Instead, let's use {} to represent a removed pointer. This has the added
convenience of being a useful representation in the template language, and only
affects the callers that care about removed files (and only slightly). Since
pointers are explicitly serialized with a call to a member function, there is no
danger of writing these to disk.
Denis Laxalde <denis@laxalde.org> [Sat, 10 Feb 2018 19:33:19 +0100] rev 36036
rebase: make "successors" a set in _computeobsoletenotrebased()
There's no apparent reason for this variable to be a list and this
avoids converting it to a set when needed.
Yuya Nishihara <yuya@tcha.org> [Sat, 10 Feb 2018 21:14:41 +0900] rev 36035
merge with stable
Denis Laxalde <denis@laxalde.org> [Fri, 09 Feb 2018 22:49:20 +0100] rev 36034
rebase: do not consider extincts for divergence detection (issue5782)
Extinct obsolete changesets cannot cause divergence upon rebase. We
compute these obsoletes without a non-obsolete successor (extincts) in
_computeobsoletenotrebased() and then filter them out from the set of
obsolete revisions to rebase before getting into _checkobsrebase() to
check for divergence candidates.
Denis Laxalde <denis@laxalde.org> [Fri, 09 Feb 2018 21:45:16 +0100] rev 36033
rebase: eliminate node from successors early in _computeobsoletenotrebased()
Denis Laxalde <denis.laxalde@logilab.fr> [Wed, 07 Feb 2018 12:06:13 +0100] rev 36032
rebase: add a test case for issue5782
Issue 5782 reports that rebase incorrectly aborts when trying to rebase
an extinct revision (an obsolete revision with only obsolete successor).
We add a test to demonstrate this: the first "hg rebase" command aborts
with the divergence warning while, when allowing divergence, the rebase
completes and does not actually produce any divergence.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Feb 2018 13:18:17 +0100] rev 36031
test: glob the temporary directory out of temporary file path
The temporary directory used by python might be outside of '/tmp/' (eg:
/dev/shm/) so we glob that part out.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Feb 2018 12:48:12 +0100] rev 36030
tests: raise a better error when patterns are wrongly formatted
It is fairly easy to make mistakes when merging conflict in the pattern file.
A common mistake is to forget adding an extra trailing comma changing the
length of the tuple.
We now detect such error and raise a better error message that helps to find
it.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 27 Jan 2018 17:58:19 -0500] rev 36029
lfs: add a fileset for detecting lfs files
This currently has the same limitation as {lfs_files}, namely it doesn't report
removed files.
We may want a dedicated 'lfs()' revset for efficiency, but combining this with
the 'contains()' revset should be equivalent for now. Combining with
'set:added()' or 'set:modified()' inside 'files()' should be equivalent to a
hypothetical lfs_adds() and lfs_modifies(). I wonder if there's a way to tweak
the filesets to evaluate lazily, to close the efficiency gap.
It would also be interesting to come up with a template filter for '{files}'
that looked at the pattern to 'files()', and filtered appropriately. While
passing a fileset as the pattern to `hg log` does filter '{files}', the set is
evaluated against the working directory, so there's no way to list all non-lfs
files above a certain size in all revisions, for example.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 07 Feb 2018 23:42:48 -0500] rev 36028
tests: stabilize ssh tests on Windows
This seems like a somewhat common type of failure (double vs single quote), so
I'm interested in ideas about how to avoid this. I doubt that we should
automatically fall back from single quote to double quote, like with '/' vs '\'.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 01 Feb 2018 08:54:48 -0800] rev 36027
wireprotoserver: rename abstractserverproto and improve docstring
The docstring isn't completely accurate for the current state
of the world. But it does describe the direction future patches
will be taking things.
Differential Revision: https://phab.mercurial-scm.org/D2065
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 01 Feb 2018 16:11:54 -0800] rev 36026
wireprotoserver: document and improve the httplib workaround
This workaround dates all the way back to a42d27bc809d in 2008.
The code is esoteric enough to warrant an inline explanation.
So I've added one.
At the time the code was written, the only wire protocol command
that accepted an HTTP request body was "unbundle." In the years
since, we've grown the ability to accept command arguments via
HTTP POST requests. So, the code has been changed to apply the
httplib workaround to all HTTP POST requests.
While staring at this code, I realized that the HTTP response
body in case of error is always the same. And, it appears to
mimic the behavior of a failed call to the "unbundle" command.
Since we can hit this code path on theoretically any protocol
request (since self.check_perm accepts custom auth checking
functions which may raise), I'm having a hard time believing
that clients react well to an "unbundle" response payload on
any wire protocol command. I wouldn't be surprised if our test
coverage for this feature only covers HTTP POST calls to
"unbundle." In other words, the experimental support for sending
arguments via HTTP POST request bodies may result in badness on
the client. Something to investigate another time perhaps...
Differential Revision: https://phab.mercurial-scm.org/D2064
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 31 Jan 2018 17:34:45 -0800] rev 36025
wireprotoserver: move error response handling out of hgweb
The exception handler for ErrorResponse has more to do with the
wire protocol than the generic HTTP server. Move the code so it
lives alongside other wire protocol code.
Differential Revision: https://phab.mercurial-scm.org/D2021
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 31 Jan 2018 16:43:46 -0800] rev 36024
hgweb: move call to protocol handler outside of try..except
The protocol handler doesn't raise ErrorResponse. So it doesn't
need to be in this `try..except ErrorResponse` block.
Differential Revision: https://phab.mercurial-scm.org/D2020
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 31 Jan 2018 16:21:43 -0800] rev 36023
wireprotoserver: move protocol parsing and dispatch out of hgweb
Previously, hgweb_mod had code for detecting if the request
was for the wire protocol. It would then (eventually) call
wireprotoserver.callhttp() to dispatch the request handling.
Detection of wire protocol requests is not trivial. There's
currently a big gotcha in the handling of the "cmd" request
parameter, for example.
Furthermore, in the near future we will have a second HTTP
protocol handler. Its mechanism for calling commands will be
a bit different. And we don't want the low-level logic for
detecting protocol commands to live in hgweb.
We establish a new function in wireprotoserver for detecting
an HTTP protocol request and for giving the caller an easy-to-use
mechanism for dispatching requests to it.
Some wire protocol specific functionality still lives in hgweb.
This will be addressed in subsequent commits.
Differential Revision: https://phab.mercurial-scm.org/D2019
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 01 Feb 2018 18:48:52 -0800] rev 36022
largefiles: register wire protocol commands with modern APIs
The wireproto.wireprotocommand decorator is the preferred mechanism for
registering wire protocol commands. In addition, wireproto.commands
is no longer a 2-tuple and use of that 2-tuple API should be considered
deprecated.
This commit ports largefiles to use wireproto.wireprotocommand()
and ports to the "commandentry" API.
As part of this, the definition of the "lheads" wire protocol
command is moved to the proper stanza.
We stop short of actually using wireprotocommand as a decorator
in order to minimize churn. We should ideally move wire protocol
commands to the registrar mechanism. But that's for another
changeset.
Differential Revision: https://phab.mercurial-scm.org/D2018
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 30 Jan 2018 18:41:44 -0800] rev 36021
wireproto: function for testing if wire protocol command is available
Currently, we perform simple membership testing for whether a wire
command is available. In the future, not all wire protocol commands
will be available on all transports. For example, a legacy transport
may not support newer commands.
In preparation of this, teach the protocol handlers to call into a
function to determine if a wire protocol command is available. That
function currently does membership testing like before, so behavior
should be identical.
In the case of the HTTP server, behavior is a bit wonkier. "cmd" is
used by both the wire protocol and hgweb. We do want the protocol
handler to handle requests for all commands that look like wire
protocol commands, even if they aren't available. Otherwise, the
fallback to hgweb would only confuse automated clients and make it
easier for hgweb to accidentally implement a "cmd" that is identical
to wire protocol commands (since they aren't centrally registered).
Differential Revision: https://phab.mercurial-scm.org/D1999
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 31 Jan 2018 14:05:11 -0800] rev 36020
wireproto: define and use types for wire protocol commands
Wire protocol commands have historically been declared as
2-tuples in wireproto.commands. There are some additional features I'd
like to implement that require going beyond 2-tuples. But because
the 2-tuple API (both destructuring assignment and direct assignment
into the dict) is used throughout the code base and in 3rd party
extensions, we can't do a trivial type change.
This commit creates a new "commandentry" type to represent declared
wire protocol commands. It implements __getitem__ and __iter__ so
it can quack like a 2-tuple. The @wireprotocommand decorator now
creates "commandentry" instances.
We also create a "commanddict" type to represent the dictionary of
declared wire protocol commands. It inherits from "dict" but provides
a custom __setitem__ to coerce passed 2-tuples to "commandentry"
instances. wireproto.commands is now an instance of this type.
Various callers in core rely on the new functionality. And tests
pass. So I'm reasonably confident things will "just work" in 3rd
party extensions as well.
Differential Revision: https://phab.mercurial-scm.org/D1998
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 30 Jan 2018 15:21:59 -0800] rev 36019
wireproto: improve docstring for @wireprotocommand
I'm about to add more arguments and want them to be documented.
Plus, good documentation is nice to have.
Differential Revision: https://phab.mercurial-scm.org/D1997
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 31 Jan 2018 10:06:39 -0800] rev 36018
wireproto: remove unnecessary exception trapping
The `try..except error.Abort` was added in 8474be4412ca back in
2012. The intent was to ensure a failing pushkey hook didn't crash
the server.
Since that changeset, repo.pushkey() and the hooks mechanism is
now much more robust about trapping errors itself. As such, we no
longer need this try..except block. So it has been removed.
Differential Revision: https://phab.mercurial-scm.org/D1996
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 06 Feb 2018 11:31:25 -0800] rev 36017
sshpeer: implement peer for version 2 of wire protocol
Since the protocol is now negotiated before we construct a
peer instance, we can return the negotiated protocol from the
handshake function and instantiate an appropriate peer class
for the protocol.
Version 2 of the SSH protocol is currently identical to version
1 post handshake. So our sshv2peer class just inherits from
sshv1peer for the time being. This will obviously change
over time.
Differential Revision: https://phab.mercurial-scm.org/D2063
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 06 Feb 2018 10:57:56 -0800] rev 36016
sshpeer: rename sshpeer class to sshv1peer (API)
With the introduction of version 2 of the SSH wire protocol,
we will need a new peer class to speak that protocol because
it will be too difficult to shoehorn a single class to speak
two protocols. We rename sshpeer.sshpeer to sshpeer.sshv1peer
to reflect the fact that there will be multiple versions of
the peer depending on the negotiated protocol.
.. api::
sshpeer.sshpeer renamed to sshpeer.sshv1peer.
Differential Revision: https://phab.mercurial-scm.org/D2062
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 06 Feb 2018 11:08:36 -0800] rev 36015
sshpeer: initial definition and implementation of new SSH protocol
The existing SSH protocol has several design flaws. Future commits
will elaborate on these flaws as new features are introduced
to combat these flaws. For now, hopefully you can take me for my
word that a ground up rewrite of the SSH protocol is needed.
This commit lays the foundation for a new SSH protocol by defining
a mechanism to upgrade the SSH transport channel away from the
default (version 1) protocol to something modern (which we'll call
"version 2" for now).
This upgrade process is detailed in the internals documentation
for the wire protocol. The gist of it is the client sends a
request line preceding the "hello" command/line which basically
says "I'm requesting an upgrade: here's what I support." If the
server recognizes that line, it processes the upgrade request and
the transport channel is switched to use the new version of the
protocol. If not, it sends an empty response, which is how all
Mercurial SSH servers from the beginning of time reacted to unknown
commands. The upgrade request is effectively ignored and the client
continues to use the existing version of the protocol as if nothing
happened.
The new version of the SSH protocol is completely identical to
version 1 aside from the upgrade dance and the bytes that follow.
The immediate bytes that follow the protocol switch are defined to
be a length framed "capabilities: " line containing the remote's
advertised capabilities. In reality, this looks very similar to
what the "hello" response would look like. But it will evolve
quickly.
The methodology by which the protocol will evolve is important.
I'm not going to introduce the new protocol all at once. That would
likely lead to endless bike shedding and forward progress would
stall. Instead, I intend to tricle out new features and diversions
from the existing protocol in small, incremental changes.
To support the gradual evolution of the protocol, the on-the-wire
advertised protocol name contains an "exp" to denote "experimental"
and a 4 digit field to capture the sub-version of the protocol.
Whenever we make a BC change to the wire protocol, we can increment
this version and lock out all older clients because it will appear
as a completely different protocol version. This means we can incur
as many breaking changes as we want. We don't have to commit to
supporting any one feature or idea for a long period of time. We
can even evolve the handshake mechanism, because that is defined
as being an implementation detail of the negotiated protocol version!
Hopefully this lowers the barrier to accepting changes to the
protocol and for experimenting with "radical" ideas during its
development.
In core, sshpeer received most of the attention. We haven't even
implemented the server bits for the new protocol in core yet.
Instead, we add very primitive support to our test server, mainly
just to exercise the added code paths in sshpeer.
Differential Revision: https://phab.mercurial-scm.org/D2061
# no-check-commit because of required foo_bar naming
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 06 Feb 2018 10:51:15 -0800] rev 36014
internals: refactor wire protocol documentation
Upcoming work will introduce a new version of the HTTP and SSH
transports. The differences will be significant enough to consider
them new transports. So, we now attach a version number to each
transport.
In addition, having the handshake documented after the transport
and in a single shared section made it harder to follow the flow
of the connection. The handshake documentation is now moved to the
protocol section it describes. We now have a generic section about
the purpose of the handshake, which was rewritten significantly.
Differential Revision: https://phab.mercurial-scm.org/D2060
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 18:04:40 +0100] rev 36013
revlog: rename 'self.checkinlinesize' into '_enforceinlinesize'
The name change has two motivations:
1) The function has no external caller, so we move it to protected space.
2) the function does more than checking it also split the data if we have more
data than 'inline' supports.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 17:57:29 +0100] rev 36012
revlog: add a _datareadfp context manager for data access needs
The helper handles:
1) is there a file handle already open that we shall just reuse,
2) is the revlog inlined or not.
Using a context manager for all read access will help setting up file pointer
caching in later changesets.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 17:35:14 +0100] rev 36011
revlog: use context manager for data file lifetime in checksize
This is clearer, safer and more modern.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 17:34:57 +0100] rev 36010
revlog: use context manager for index file lifetime in checkinlinesize
This is clearer, safer and more modern.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 17:34:47 +0100] rev 36009
revlog: use context manager for data file lifetime in checkinlinesize
This is clearer, safer and more modern.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 17:34:19 +0100] rev 36008
revlog: use context manager for index file life time in __init__
This is clearer, safer and more modern.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 17:22:13 +0100] rev 36007
revlog: move index file opening in a method
Having file operation centralized into a single spot help to factor common
logic out (eg: special flag handling according to the mode).
It is also the first step to simplify file handling during batch operation
(eg: revlog cloning). However, that part does not seems to be a hotspot yet.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 17:03:56 +0100] rev 36006
revlog: move datafile opening in a method
Having file operation centralized into a single spot help to factor common
logic out.
It is also the first step to simplify file handling during batch operation
(eg: revlog cloning). However, that part does not seems to be a hotspot yet.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Feb 2018 12:25:11 +0100] rev 36005
parseindex: implement context manager method on the wrapper
This is needed for incoming cleanups.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Feb 2018 11:52:37 +0100] rev 36004
parseindex: also forward keyword argument in a debug wrapper
Otherwise, it gets in the way of a coming refactoring.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Feb 2018 11:51:39 +0100] rev 36003
dumprevlog: handle being passed a mode parameter
This makes it closer to an actual file opener. Otherwise, it gets in the way
of a coming refactoring.
Boris Feld <boris.feld@octobus.net> [Tue, 06 Feb 2018 11:08:41 +0100] rev 36002
changelog: add the missing 'closed' property on 'appender' object
This mimic file object further.
Boris Feld <boris.feld@octobus.net> [Mon, 05 Feb 2018 20:36:51 +0100] rev 36001
changelog: implement context manager method for 'appender' object
This object mimics a file and was missing the context manager API.
Joerg Sonnenberger <joerg@bec.de> [Sun, 04 Feb 2018 00:29:22 +0100] rev 36000
diff: improve ui.write performance when not coloring on Windows
Differential Revision: https://phab.mercurial-scm.org/D2022
Augie Fackler <augie@google.com> [Tue, 06 Feb 2018 05:25:36 -0500] rev 35999
hgsh: enable clang-format
Nothing looks awful, so we can just turn it on.
Differential Revision: https://phab.mercurial-scm.org/D2059
Augie Fackler <augie@google.com> [Tue, 06 Feb 2018 05:24:28 -0500] rev 35998
chg: enable clang-format on all .c and .h files
Nothing in here looks awful, so I think we may as well just do it.
# skip-blame because it's just reformatting with no functionality change
Differential Revision: https://phab.mercurial-scm.org/D2058
Augie Fackler <augie@google.com> [Wed, 07 Feb 2018 06:28:43 -0500] rev 35997
python3: whitelist an additional 11 tests
I think these are mostly the result of Pulkit's recent work. Thanks!
Differential Revision: https://phab.mercurial-scm.org/D2078
Yuya Nishihara <yuya@tcha.org> [Mon, 05 Feb 2018 20:48:51 +0900] rev 35996
ui: add explicit path to write prompt text bypassing buffers
The prompt= flag was added at e35d7f131483, when colorui had its own write()
function. Since we've merged colorui to ui, we can simply call the unbuffered
write() function.
Yuya Nishihara <yuya@tcha.org> [Mon, 05 Feb 2018 20:40:49 +0900] rev 35995
ui: write prompt text in ui.prompt(), not in ui._readline()
self.label() is replaced by label= option, which should make it clearer why
we can't pass the text to raw_input(prompt).
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 14:00:56 +0900] rev 35994
logcmdutil: mark changesetprinter.showpatch() as private
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 14:28:03 +0900] rev 35993
logcmdutil: drop default arguments from changesetdisplayer/templater() calls
Yuya Nishihara <yuya@tcha.org> [Sun, 21 Jan 2018 13:47:06 +0900] rev 35992
logcmdutil: make default parameters of changesetprinters consistent
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Feb 2018 10:33:14 +0900] rev 35991
mdiff: use str.startswith/endswith() instead of slicing
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Feb 2018 10:28:03 +0900] rev 35990
patch: unify check_binary and binary flags
Follows up 079b27b5a869. If opts.text=True, check_binary is ignored, so we
can just pass the binary flag to unidiff().
perfunidiff now takes any inputs as text files, which I think is a desired
behavior.
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:49:02 +0530] rev 35989
py3: make sure we open the file in bytes mode
# skip-blame because we are just adding b
Differential Revision: https://phab.mercurial-scm.org/D2077
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:47:03 +0530] rev 35988
py3: use bytes instead of str
Differential Revision: https://phab.mercurial-scm.org/D2076
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:34:51 +0530] rev 35987
py3: add b'' prefixes to string literals in test files
# skip-blame because we are just adding b''
Differential Revision: https://phab.mercurial-scm.org/D2075
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:21:06 +0530] rev 35986
py3: use range instead of xrange
xrange is not available on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D2074
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:19:19 +0530] rev 35985
py3: use open() instead of file()
file() is not present in Python 3.
Differential Revision: https://phab.mercurial-scm.org/D2073
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:17:19 +0530] rev 35984
py3: use pycompat.bytestr to convert _b85chars to bytes
The tranformer does append b'' to the value and make that a bytes but bytes in
Python 3 returns the ascii value on getting characters using indexing.
Characters of this string are queried using indexing multiple times in the file
and to support that we use pycompat.bytestr which returns the bytechrs using
indexing.
Differential Revision: https://phab.mercurial-scm.org/D2072
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:11:38 +0530] rev 35983
py3: use pycompat.bytechr instead of chr
Differential Revision: https://phab.mercurial-scm.org/D2071
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 07 Feb 2018 13:10:24 +0530] rev 35982
py3: use pycompat.ziplist instead of zip
zip returns a zip object instead of a list on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D2070
Boris Feld <boris.feld@octobus.net> [Tue, 06 Feb 2018 14:24:16 +0100] rev 35981
patches: move assignment outside the conditional
Having this movement in its own patch will make the next patch clearer.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 14:58:32 -0800] rev 35980
sshpeer: remove support for connecting to <0.9.1 servers (BC)
197d10e157ce made this change for the HTTP peer. Let's do the same
for the SSH peer.
Test output changes as expected. A redundant test has been dropped.
.. bc::
Support for connecting to Mercurial servers older than 0.9.1 has
been removed.
Differential Revision: https://phab.mercurial-scm.org/D2036
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 14:44:04 -0800] rev 35979
sshpeer: document the handshake mechanism
The mechanism by which SSH peers establish connections with remotes
is wonky and requires a bit of code archeology to understand. While
it is already documented in `hg help internals.wireproto`, it helps
to have documentation in the code as well.
Differential Revision: https://phab.mercurial-scm.org/D2035
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Feb 2018 09:14:32 -0800] rev 35978
sshpeer: move handshake outside of sshpeer
With the handshake now performed before a peer class is instantiated,
we can now instantiate a different peer class depending on the results
of the handshake.
Our test extension had to change to cope with the new API. Because
we now issue the command via raw I/O calls and don't call
_callstream(), we no longer have to register the fake command.
(_callstream() uses the command registration to see what args to
send).
Differential Revision: https://phab.mercurial-scm.org/D2034
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 14:10:56 -0800] rev 35977
sshpeer: inline I/O into _validaterepo()
We want to move the handshake code out of the peer class so the
peer factory function can perform the handshake and instantiate
a proper class depending on the results. To make that refactor
easier to read, we first inline I/O functionality into
_validaterepo().
Test output for low-level protocol tests didn't change, thus
hopefully demonstrating that this refactor didn't change any
material behavior.
Because we no longer call _callstream(), our test extension for
monkeypatching the peer had to change its hook point.
Differential Revision: https://phab.mercurial-scm.org/D2033
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Feb 2018 14:17:24 -0800] rev 35976
sshpeer: clean up API for sshpeer.__init__ (API)
Our refactoring left the state of sshpeer.__init__ in a poor
state. "create" was no longer used. Process/pipe arguments were
passed poorly. "name" was really a URL.
This commit cleans all that up.
.. api::
sshpeer.sshpeer.__init__ now receives arguments describing an
existing connection instead of creating a connection itself.
Differential Revision: https://phab.mercurial-scm.org/D2032
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Feb 2018 14:05:59 -0800] rev 35975
sshpeer: establish SSH connection before class instantiation
We want to move the handshake to before peers are created so
we can instantiate a different peer class depending on the
results of the handshake. This necessitates moving the SSH
process invocation to outside the peer class.
As part of the code move, some variables were renamed for
clarity. util.popen4() returns stdin, stdout, and stderr in
their typical file descriptor order. However, stdin and stdout
were being mapped to "pipeo" and "pipei" respectively. "o"
for "stdin" and "i" for "stdout" is a bit confusing. Although
it does make sense for "output" and "input" from the perspective
of the client. But in the context of the new function, it makes
sense to refer to these as their file descriptor names.
In addition, the last use of self._path disappeared, so we stop
setting that attribute and we can delete the redundant URL
parsing necessary to set it.
Differential Revision: https://phab.mercurial-scm.org/D2031
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 11:40:13 -0800] rev 35974
sshpeer: remove frivolous call to _cleanup()
_validaterepo() is called once during __init__. _cleanup()
no-ops if the self._pipe* attributes aren't set. These attributes
are set during _validaterepo(). So the call to _cleanup() isn't
necessary.
But just to be on the safe side, we add an assertion.
Differential Revision: https://phab.mercurial-scm.org/D2030
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 11:37:19 -0800] rev 35973
sshpeer: extract pipe cleanup logic to own function
So it can be used outside of instantiated classes. This is
needed to support pipe creation before __init__ is called.
Differential Revision: https://phab.mercurial-scm.org/D2029
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 19:23:40 -0800] rev 35972
sshpeer: move ssh command and repo creation logic out of __init__
It was easier to move both of these at once because repository
creation requires various variables and I didn't want to add
tons of arguments and code to __init__ that will soon be deleted
anyway. We do add an extra argument so we can proxy values to the
_validaterepo() call. But this is minimally invasive.
Some callers of self._abort() were converted to just raise. Like
before, the _abort() call wasn't necessary because self._pipe*
aren't populated this early in the object's lifetime.
As part of this, various private attributes derived from the parsed
URL are no longer used. So we no longer set them.
Differential Revision: https://phab.mercurial-scm.org/D2028
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 12:55:18 -0800] rev 35971
sshpeer: move URL validation out of sshpeer.__init__
We will soon have another SSH peer class to support the new version
of the SSH protocol. However, we won't know which peer class to
instantiate until we perform a handshake on an active connection.
This means that we need to move connection establishment and handshake
code out of sshpeer.__init__.
This commit starts the process of migrating peer creation code
out of sshpeer.__init__ into instance(), which is the API for
creating peers.
The moved code no longer calls _abort(). _abort() runs _cleanup() and
raises. _cleanup() only performs actions on self._pipe*. These objects
aren't instantiated until we actually connect to the peer. So _abort()
was not necessary in the old code.
To keep the API the same, __init__() now makes a redundant call to
util.url(). This will be fixed in subsequent commits.
Differential Revision: https://phab.mercurial-scm.org/D2027
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 14:02:41 -0800] rev 35970
tests: add low-level SSH protocol tests
We don't really have good low-level tests for the behavior of the SSH
wire protocol. This commit attempts to establish some.
The added tests consist of a mixture of starting a server
with `hg serve --stdio` and sending bytes to it and using
`hg debugpeer` to go through the official client code. Having
insight into what raw bytes are exchanged as well as what the peer
does is useful.
We also introduce a test extension for modifying the behavior of
the SSH server and peer. For example, we change the server to
not recognize the "hello" command, simulating behavior of <0.9.1
servers.
These tests are generally useful to have. But the impetus for creating
them now is they will be needed for verifying behavior of old clients
and servers when a new SSH protocol is introduced.
Differential Revision: https://phab.mercurial-scm.org/D2026
# no-check-commit because of serve_forever()
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 03 Feb 2018 12:01:01 -0800] rev 35969
debugcommands: introduce debugpeer command
`hg debugpeer <path>` will establish a connection to a peer repository
and print information about it.
If you add --debug, it will log low-level protocol request info. This
will be useful for upcoming tests around protocol handshaking.
Differential Revision: https://phab.mercurial-scm.org/D2025
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Feb 2018 12:47:37 -0800] rev 35968
sshpeer: make "instance" a function
The API is that peer modules must provide an "instance" symbol
that is callable to return a peer.
Making "instance" a function instead of an alias to "sshpeer"
makes it easier to monkeypatch the "sshpeer" type. It will also
make it possible to turn instance() into a factory function of
sorts that returns different types based on connection properties.
Differential Revision: https://phab.mercurial-scm.org/D2024
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 16:17:43 -0500] rev 35967
lfs: deduplicate oids in the transfer
Apparently, we can't rely on the server to deduplicate for us.
Sadly, the pointer object isn't hashable, so it can't be reduced by converting
it to a set. In order to be hashable, it needs to be immutable. I had a bunch
of code to change it to composition and forward the readonly dict methods to a
member dict. But the pointer is updated via __setitem__() when creating the
pointer file. So it didn't see worth adding all of that code to the class.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 15:26:49 -0500] rev 35966
cat: call the storage prefetch hook
It's not important to call in the case of a single file, but maybe it's better
to do so for consistency.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 14:31:32 -0500] rev 35965
archive: call the storage prefetch hook
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 00:33:28 -0500] rev 35964
lfs: prefetch lfs blobs during revert
The revert command oddly prints out what it will do before requesting the files
to be prefetched. But the 'need to transfer' line indicates the blobs are being
grouped.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 04 Feb 2018 14:14:28 -0500] rev 35963
cmdutil: convert _revertprefetch() to a generic stored file hook (API)
This will be used by LFS to fetch required files in a group for multiple
commands, prior to being accessed. That avoids the one-at-a-time fetch when the
filelog wrapper goes to access it, and it is missing locally (which costs two
round trips to the server.) The core command list that needs this is probably
at least:
- annotate
- archive (which is also used by extdiff)
- cat
- diff
- export
- grep
- verify (sadly)
- anything that has the '{data}' template
There are no core users of the revert prefetch hook, and never have been since
it was introduced in 45e02cfad4bd for remotefilelog. Thanks to Yuya for
figuring out a way to reliably trigger the deprecated warning. Unfortunately,
it wanted to blame the caller of revert. Passing along an adjusted stack level
seemed the least bad choice (although it still blames a core function).
One thing to note is that the store lock isn't being held when this is called.
I'm not at all familiar with remotefilelog or its locking requirements, so this
may not be a big deal. Currently, LFS doesn't hold a lock when downloading
files. Even though largefiles doesn't either, I'm starting to think it should,
and maybe the .hg/store/lock isn't good enough to cover the globally shared
cache.
.. api::
The cmdutil._revertprefetch() hook point for prefetching stored files has
been replaced by the command agnostic cmdutil._prefetchfiles(). The new
function takes a list of files, instead of a list of lists of files.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 03 Feb 2018 21:26:12 -0500] rev 35962
lfs: prefetch lfs blobs when applying merge updates
In addition to merge, this method ultimately gets called by many commands:
- backout
- bisect
- clone
- fetch
- graft
- import (without --bypass)
- pull -u
- rebase
- strip
- share
- transplant
- unbundle
- update
Additionally, it's also called by histedit, shelve, unshelve, and split, but it
seems that the related blobs should always be available locally for these.
For `hg update`, it happens after the normal argument checking and pre-update
hook processing, and remote corruption is detected prior to manipulating the
working directory. Other commands could use this treatment (archive, cat,
revert, etc), but this covers so many of the frequently used bulk commands, it
seems like a good starting point.
Losing the verbose message that prints the file name before a corrupt blob
aborts the command is a little sad, because there's no easy way to go from oid
to file name. I'd like to change that message to list the file name so it looks
cleaner and less cryptic, but the pointer object is nowhere near where it needs
to be to do this. So punt on that for now.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 27 Jan 2018 14:53:16 -0500] rev 35961
lfs: factor out a method for extracting the pointer of a single file
This will be useful for filesets, among other things, instead of traversing the
whole context.
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:24:02 +0530] rev 35960
py3: add b'' to literals in check-config.py
# skip-blame because we are just adding b''
Differential Revision: https://phab.mercurial-scm.org/D2046
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 29 Dec 2017 05:40:49 +0530] rev 35959
check-config: specify the mode 'rb' to open the file
Differential Revision: https://phab.mercurial-scm.org/D2045
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:12:36 +0530] rev 35958
py3: use open() instead of file()
file() is not present in Python 3
Differential Revision: https://phab.mercurial-scm.org/D2044
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:12:01 +0530] rev 35957
py3: use pycompat.strkwargs() to convert kwargs' key to str
Differential Revision: https://phab.mercurial-scm.org/D2043
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:10:33 +0530] rev 35956
py3: add __bytes__() for mq.patchheader and make sure __str__ returns str
Before this patch, __str__() function for mq.patchheader class return bytes
which is not str on Python 3. So let's move that logic to __bytes__() and for
__str__() convert the return value of __bytes__() to str.
Differential Revision: https://phab.mercurial-scm.org/D2042
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 13:01:35 +0530] rev 35955
py3: use "%d" to convert integer to bytes
Differential Revision: https://phab.mercurial-scm.org/D2041
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 12:59:57 +0530] rev 35954
py3: use .startswith() instead of bytes[0]
Differential Revision: https://phab.mercurial-scm.org/D2040
Pulkit Goyal <7895pulkit@gmail.com> [Mon, 05 Feb 2018 12:58:51 +0530] rev 35953
py3: slice on bytes to prevent getting the ascii values
Differential Revision: https://phab.mercurial-scm.org/D2039