Augie Fackler <augie@google.com> [Fri, 27 Apr 2018 00:14:23 -0400] rev 37921
tests: port test-minifileset.py to Python 3
# skip-blame all b prefixes, nothing interesting
Differential Revision: https://phab.mercurial-scm.org/D3472
Augie Fackler <augie@google.com> [Fri, 27 Apr 2018 00:13:56 -0400] rev 37920
minifileset: fix on Python 3
Found by porting test-minifileset.py to Python 3.
Differential Revision: https://phab.mercurial-scm.org/D3471
Augie Fackler <augie@google.com> [Fri, 27 Apr 2018 00:08:48 -0400] rev 37919
tests: port test-url.py to Python 3
# skip-blame it's all b prefixes and pycompat.bytestr() wrappers
Differential Revision: https://phab.mercurial-scm.org/D3470
Augie Fackler <augie@google.com> [Fri, 27 Apr 2018 00:08:21 -0400] rev 37918
tests: fix error case in test-url.py's doctest
This required some careful attention, so I wanted to split it out from
the uninteresting bits that'll be in the next change.
Differential Revision: https://phab.mercurial-scm.org/D3469
Augie Fackler <augie@google.com> [Fri, 27 Apr 2018 00:06:08 -0400] rev 37917
util: make util.url __repr__ consistent on Python 2 and 3
Differential Revision: https://phab.mercurial-scm.org/D3468
Augie Fackler <augie@google.com> [Fri, 27 Apr 2018 00:05:46 -0400] rev 37916
sslutil: fix some edge cases in Python 3 support
Detected by fixing up test-url.py on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D3467
Augie Fackler <augie@google.com> [Thu, 26 Apr 2018 23:33:27 -0400] rev 37915
tests: port test-hybridencode.py to unittest
This was done predominantly through keyboard macros in emacs, so it's
not always pretty, but it's enough to make the test easier to work
with. There's a ton of room for improvement in this file, but it would
be labor intensive and error-prone, so I went with the dumbest option
that could work, so as to avoid transcription errors.
Paranoia:
$ egrep '^ def test' tests/test-hybridencode.py | wc -l
44
$ egrep '^ def test' tests/test-hybridencode.py | sort | uniq | wc -l
so I'm pretty confident there aren't any shadowed test methods.
This fixes the test on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D3466
Augie Fackler <augie@google.com> [Thu, 26 Apr 2018 21:39:35 -0400] rev 37914
tests: make test-check-interfaces.py work on Python 3
# skip-blame just a bunch of bytes prefixes
Differential Revision: https://phab.mercurial-scm.org/D3465
Augie Fackler <augie@google.com> [Thu, 26 Apr 2018 21:38:49 -0400] rev 37913
scmutil: clean up bytes/string cache decorator mess on Python 3 again
The previous fix to this area worked, but was dropping bytes in
__dict__ on Python 3. This was causing subtle breakage in
test-check-interfaces.py, and probably other things too.
Fixed now.
Differential Revision: https://phab.mercurial-scm.org/D3464
Martin von Zweigbergk <martinvonz@google.com> [Mon, 07 May 2018 12:18:09 -0700] rev 37912
directaccess: use resolvehexnodeidprefix() instead of _partialmatch()
Same reasoning as previous commit: I want to make
resolvehexnodeidprefix() move complex and don't want to duplicate that
code in directaccess.
Differential Revision: https://phab.mercurial-scm.org/D3463
Martin von Zweigbergk <martinvonz@google.com> [Mon, 07 May 2018 14:32:55 -0700] rev 37911
revset: use resolvehexnodeidprefix() in id() predicate (BC)
We now have a public method for this purpose, so we don't need to
access the private revlog._partialmatch(). Also, I'll probably make
some changes to resolvehexnodeidprefix() later, and I want those to be
reflected by the id() predicate.
Note that this breaks a test case, because we now resolve the prefix
in the unfiltered repo and get an ambiguous lookup, which results in
no revision being added to the revset. The test case was already
documented to be broken even though it wasn't. It's important to note
that {shortest(node)} already uses the unfiltered repo, so we're not
going to break people who get the prefix from there.
I think we may not want to ever use shortest() in the filtered
repo. It seems unlikely to be enough of a win to matter much. For
example, in my hg repo, it would save me only 0.2 hex digits. In
another repo that only I modify, it saves a little more, but it's
still only 0.29 hex digits. It seems unlikely that people will prune
enough commits that only 1/16 of the commits are visible (which is
what it would take a to save a single hex digit). Instead, I'm working
on another approach: allow ambiguous matches to be disambiguated
within a user-specified revset. Whether or not that pans out, I hope
we're okay with this little change in behavior for now and we can
decide what to do about it later.
Differential Revision: https://phab.mercurial-scm.org/D3311
Martin von Zweigbergk <martinvonz@google.com> [Mon, 07 May 2018 14:32:43 -0700] rev 37910
revset: make id() an empty set for ambiguous nodeid (BC)
As Yuya pointed out in the review of D3311, id() (and rev()) does not
raise an error when the input is an unknown identifier, so it doesn't
make sense for it to do that when the input is ambiguous with a
filtered node. However, it turned out that it already does raise an
error when the input is ambiguous among the visible nodes. So let's
start by fixing that.
Differential Revision: https://phab.mercurial-scm.org/D3462
Martin von Zweigbergk <martinvonz@google.com> [Sat, 05 May 2018 00:16:43 -0700] rev 37909
shortest: don't keep checking for longer prefix if node doesn't exist (API)
If revlog.shortest() is called with an invalid nodeid, we keep
checking if longer and longer prefixes are valid. We call
revlog._partialmatch() for each prefix. That function will give us
None if the node doesn't exist (and a RevlogError if it's ambiguous),
so there's no need to keep checking.
This patch instead makes revlog.shortest() raise a LookupError is the
node does not exist, and updates the caller to handle it. Before this
patch, revlog.shortest() would return the full hexnode for nonexistent
nodeids. By the same reasoning as in 7b2955624777 (scmutil: make
shortesthexnodeidprefix() take a full binary nodeid, 2018-04-14), it's
not revlog.shortest() that should decide how to present nonexistent
nodeids, so that's now moved to the template function.
This should speed up cases where {shortest()} is applied to an invalid
nodeid, but I couldn't think of a reasonable case where that would
happen.
Differential Revision: https://phab.mercurial-scm.org/D3461
Martin von Zweigbergk <martinvonz@google.com> [Wed, 02 May 2018 22:56:10 -0700] rev 37908
shortest: extract function for checking if a prefix is a revnum
Much of isvalid() was about testing if a prefix is a valid revnum. I
want to reuse that soon, so let's move it out.
There is no significant slowdown from the function call overhead.
Differential Revision: https://phab.mercurial-scm.org/D3460
Martin von Zweigbergk <martinvonz@google.com> [Thu, 03 May 2018 10:12:47 -0700] rev 37907
shortest: rename "test" variable to "prefix"
Sorry if this is considered churn, but "prefix" just seems much
clearer to me.
Differential Revision: https://phab.mercurial-scm.org/D3459
Martin von Zweigbergk <martinvonz@google.com> [Wed, 02 May 2018 22:49:06 -0700] rev 37906
shortest: move some safe code out of exception block
The RevlogError and WdirUnsupported could be raised by
_partialmatch(), but not by the rest of isvalid(), so let's move the
rest out to make it clearer.
Differential Revision: https://phab.mercurial-scm.org/D3458
Martin von Zweigbergk <martinvonz@google.com> [Fri, 04 May 2018 22:04:44 -0700] rev 37905
revlog: don't say "not found" on internal error
If index_node() returned NULL, then index_find_node() and and
nt_partialmatch() used to return -2 to signal that the node was not
found. However, we were passing in a revnum to index_node() that we
knew should exist, so the only reason it could return NULL was due to
some internal error or perhaps out of memory. Let's not use "not
found" for these cases. I suppose we never noticed this because these
error never happen in practice.
I think there are more places where we should error out instead of
reporting that the node was not found, but the cases mentioned above
were all I cared about right now (because using the same error code
for all failures simplified some future patches).
Differential Revision: https://phab.mercurial-scm.org/D3457
Martin von Zweigbergk <martinvonz@google.com> [Fri, 04 May 2018 21:58:43 -0700] rev 37904
revlog: extract function for getting node from known-to-exist rev
Many of the calls to index_node() (which converts a rev to a nodeid)
are done with a rev that's know to exist. If the function fails,
there's something really wrong and we should just abort. This was done
in only one place. This patch starts by extracting that code to a
function that we can reuse in later patches.
Differential Revision: https://phab.mercurial-scm.org/D3456
Martin von Zweigbergk <martinvonz@google.com> [Mon, 07 May 2018 09:15:29 -0700] rev 37903
shortest: make {shortest("fffffffff")} work again
{shortest("fffffffff")} should shorten it to the shortest unambiguous
prefix for the working directory. It used to do that until I broke it
in 7b2955624777 (scmutil: make shortesthexnodeidprefix() take a full
binary nodeid, 2018-04-14), when we started returning the full hex
nodeid for any working directory prefix shorter than 40 hex
digits. This patch fixes it by catching WdirUnsupported
specifically.
Differential Revision: https://phab.mercurial-scm.org/D3455
Martin von Zweigbergk <martinvonz@google.com> [Sun, 29 Apr 2018 14:29:09 -0700] rev 37902
revlog: use radix tree also for matching keys shorter than 4 hex digits
I don't know what the reason for the 4-digit limit was, and I can't
think of any real disadvantages of using the radix tree also when the
requested minimum length is short. This speeds up `hg log -T
'{shortest(node,1)}\n'` from 2m16s to 4.5s by making that not fall
back to pure code.
Differential Revision: https://phab.mercurial-scm.org/D3453
Martin von Zweigbergk <martinvonz@google.com> [Fri, 06 Apr 2018 12:55:32 -0700] rev 37901
context: convert to hex for error message only for 20-byte changeid
Now that 20-byte strings unambiguously mean binary (or a bug), we can
specialize the conversion to hex for that case.
Differential Revision: https://phab.mercurial-scm.org/D3452
Martin von Zweigbergk <martinvonz@google.com> [Fri, 06 Apr 2018 12:59:17 -0700] rev 37900
context: clarify that only one attempt is made to interpret changeid
We can now tell what type of revision specifier we have just by
looking at it (we no longer attempt to interpret it in one way after
the other -- that's now in scmutil.revsymbol()). Let's clarify this in
the code by swithing to if/elif.
Differential Revision: https://phab.mercurial-scm.org/D3451
Martin von Zweigbergk <martinvonz@google.com> [Fri, 06 Apr 2018 12:45:08 -0700] rev 37899
context: only bother looking for broken dirstate for 20-byte changeid
If we fail to look up a changeid in changectx.__init__, we check if it
exactly matches any of the dirstate parents, and if it does, we print
a more specific message ("working directory has unknown parent '...'!"
instead of "unknown revision '...'"). The dirstate parents are always
20 bytes, so there's no need to check for a match when the given
changeid is not 20 bytes. (And now that all the other allowed forms of
changeid have been moved out of the constructor, there's no risk that
a changeid that did match a dirstate parent was actually a valid
bookmark.)
Differential Revision: https://phab.mercurial-scm.org/D3450
Martin von Zweigbergk <martinvonz@google.com> [Sat, 28 Apr 2018 23:16:41 -0700] rev 37898
context: drop support for looking up context by ambiguous changeid (API)
This removes support for using the changectx constructor (and thereby
repo[x]) for looking up contexts by a stringified int, a namespace key
(e.g. a bookmark), or a partial hex nodeid. This means that
e.g. repo[<hex nodeid>] will now fail even if a bookmark with the same
name exists (which is a good thing IMO). It also means that doing
repo[<non-existent node>] no longer ends up loading namespaces (which
was a surprising side-effect of creating of failing to create a
context object that I recently ran into while debugging something
unrelated to this series).
Differential Revision: https://phab.mercurial-scm.org/D3449
Martin von Zweigbergk <martinvonz@google.com> [Sat, 28 Apr 2018 23:54:07 -0700] rev 37897
tests: pass parent revision as integer to repo[x] in test-context-metadata.t
Support for looking up by stringified int is going away. It's already
deprecated, but I think I didn't notice this case because the test
case pipes through `grep`.
Differential Revision: https://phab.mercurial-scm.org/D3448
Yuya Nishihara <yuya@tcha.org> [Sat, 05 May 2018 11:42:42 +0900] rev 37896
bookmarks: cache reverse mapping (issue5868)
I chose a simpler implementation. If the initial cost of building reverse
mapping is significant, we'll have to move it under @propertycache.
The nodemap could be a dict of sets, but I think keeping a sorted list is
better since each node is likely to have zero/one bookmark.
Micro-benchmark with 1001 bookmarks and 1001 revisions:
$ for n in `seq 0 1000`; do touch $n; hg book book$n; hg ci -qAm$n; done
$ hg bookmarks --time > /dev/null
(orig) time: real 0.040 secs (user 0.050+0.000 sys 0.000+0.000)
(new) time: real 0.040 secs (user 0.040+0.000 sys 0.010+0.000)
$ hg log -T '{bookmarks}\n' --time > /dev/null
(orig) time: real 0.160 secs (user 0.160+0.000 sys 0.000+0.000)
(new) time: real 0.090 secs (user 0.100+0.000 sys 0.000+0.000)
Yuya Nishihara <yuya@tcha.org> [Sat, 05 May 2018 11:44:43 +0900] rev 37895
bookmarks: make argument names of _set/_del() more specific
Yuya Nishihara <yuya@tcha.org> [Sat, 05 May 2018 11:34:03 +0900] rev 37894
bookmarks: extract function that looks up bookmark names by node
Yuya Nishihara <yuya@tcha.org> [Sat, 05 May 2018 11:21:41 +0900] rev 37893
bookmarks: hide dict behind bmstore class
This should make it clearer that the bmstore doesn't expose all dict APIs.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 May 2018 19:00:03 -0700] rev 37892
tests: remove pid file by default
Previously, killdaemons.py would kill PIDs listed in a file then
leave the file lingering around.
If the PIDs are killed, then there's no point leaving the PID file
around. In the worst case, a later invocation of killdaemons.py
(run-tests.py invokes killdaemons.py after running a test) could kill
a separate process whose PID conflicted with a previously-killed
process. By removing the PID file, we eliminate this possibility.
Some tests were manually removing the PID file after calling
killdaemons.py. So we update these tests to not do this.
Differential Revision: https://phab.mercurial-scm.org/D3443
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Apr 2018 19:54:55 -0700] rev 37891
tests: remove #require killdaemons
The killdaemons hghave feature has returned True since it was
introduced in 448d0c452140. As such, "#require killdaemons" has
no effect and is superfluous. So we remove instances of it.
Differential Revision: https://phab.mercurial-scm.org/D3442
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 May 2018 18:35:16 -0700] rev 37890
pycompat: export queue module instead of symbols in module (API)
Previously, pycompat and util re-exported individual symbols from the
queue module. This had the side-effect of forcing the loading of the
queue module whenever pycompat/util was imported.
These symbols aren't used very often. So importing the module to
get a handle on the symbols is wasteful.
This commit changes pycompat so it no longer exports the individual
symbols in the queue module. Instead, we make the imported module
a "public" symbol. We drop the individual symbol aliases from the
util module. All consumers are updated to use pycompat.queue.* instead.
This change makes 300 invocations of `hg log -r. -T '{rev}\n'` a little
faster:
before: 18.44s
after: 17.87s
Differential Revision: https://phab.mercurial-scm.org/D3441
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 May 2018 18:41:51 -0700] rev 37889
demandimport: make module ignores a set (API)
The list of modules to ignore is used for membership testing. Yet
it is defined as a list.
Sets are more efficient for membership testing. So this commit converts
the module list to a set.
Since we took an API hit, I renamed the variable to further clarify
the change.
This appears to reduce the CPU time for running 300 invocations of
`hg log -r. -T '{rev}'` on my i7-6700K:
before: 18.64s
after: 18.44s
Differential Revision: https://phab.mercurial-scm.org/D3440
Yuya Nishihara <yuya@tcha.org> [Thu, 19 Apr 2018 20:33:43 +0900] rev 37888
cmdutil: remove unused 'confirmopts' constant
Follows up f10cb49951e1.
Yuya Nishihara <yuya@tcha.org> [Thu, 03 May 2018 15:08:16 +0900] rev 37887
formatter: ditch namedtuple in favor of attr
Yuya Nishihara <yuya@tcha.org> [Sun, 29 Apr 2018 15:52:01 +0900] rev 37886
log: consume --stat/patch options at constructor of changesetprinter
The variable name, self.diffopts, was confusing. Let's split it to two
booleans.
Yuya Nishihara <yuya@tcha.org> [Sun, 29 Apr 2018 15:44:17 +0900] rev 37885
log: cache diffopts instance
It appears that calling patch.diff*opts() repeatedly has some cost.
$ hg log -T '{rev}\n' -R mercurial --time > /dev/null
(orig) time: real 4.430 secs (user 4.370+0.000 sys 0.050+0.000)
(new) time: real 1.950 secs (user 1.880+0.000 sys 0.060+0.000)
'diffopts or {}' isn't necessary as patch.diff*opts() accepts opts=None.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 05 May 2018 18:06:45 -0700] rev 37884
merge with stable
Yuya Nishihara <yuya@tcha.org> [Sun, 01 Jul 2018 23:36:53 +0900] rev 37883
encoding: alias cp65001 to utf-8 on Windows
As far as I can tell, cp65001 is the Windows name for UTF-8. I don't know
how different it is from the UTF-8, but Python 3 appears to have introduced
new codec for cp65001, so the alias is enabled only for Python 2.
https://bugs.python.org/issue13216
This patch is untested, but hopefully fixes the following issue.
https://bitbucket.org/tortoisehg/thg/issues/5127/
Sune Foldager <cryo@cyanite.org> [Wed, 04 Jul 2018 14:19:13 +0200] rev 37882
windows: fix incorrect detection of broken pipe when writing to pager
Paging e.g. hg incoming on Windows and quitting the pager before the
output is consumed will print 'abort: Invalid argument'. This is
because the windows error 0xE8 (ERROR_NO_DATA) is mapped to EINVAL
even though it is documented as 'The pipe is being closed'.
Note that this fix assumes that Windows' last error code is still
valid in the exception handler. It works correctly in all my tests.
A simpler fix would be to just map EINVAL to EPIPE, like was done is
flush previously, but that would be less precise.
This error was not observed previously, when pager was an extension.
Augie Fackler <raf@durin42.com> [Tue, 03 Jul 2018 12:10:22 -0400] rev 37881
Added signature for changeset 0b63a6743010
Augie Fackler <raf@durin42.com> [Tue, 03 Jul 2018 12:10:21 -0400] rev 37880
Added tag 4.6.2 for changeset 0b63a6743010
Sune Foldager <cryo@cyanite.org> [Mon, 25 Jun 2018 16:36:14 +0200] rev 37879
procutil: use unbuffered stdout on Windows
Windows doesn't support line buffering, treating it as fully buffered. This
causes output of slow commands to stutter. We use unbuffered instead.
Yuya Nishihara <yuya@tcha.org> [Tue, 19 Jun 2018 22:45:52 +0900] rev 37878
merge: do not fill manifest of committed revision with pseudo node (issue5526)
Since a75d24539aba "convert: fix convert dropping p2 contents during filemap
merge", wctx is not always a committablectx because the convert extension
passes in repo[n] as wctx. If wctx is a committed changeset, its manifest
dict shouldn't be mutated reflecting to the working directory.
Yuya Nishihara <yuya@tcha.org> [Fri, 15 Jun 2018 22:16:58 +0900] rev 37877
manifest: fix possible SEGV caused by uninitialized lazymanifest fields
Before, uninitialized self->pydata would be passed to lazymanifest_dealloc()
on OOM, and Py_DECREF(self->pydata) would crash if we were unlucky.
It's still wrong to do malloc() thingy in tp_init because __init__() may be
called more than once [1], but I don't want to go a step further in stable
branch.
[1]: https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_new
"The tp_new function should ... do only as much further initialization as
is absolutely necessary. Initialization that can safely be ignored or
repeated should be placed in the tp_init handler."
Augie Fackler <augie@google.com> [Fri, 15 Jun 2018 10:14:32 -0400] rev 37876
tests: replace `echo -n` with `printf` per check-code
Differential Revision: https://phab.mercurial-scm.org/D3749
Jun Wu <quark@fb.com> [Thu, 14 Jun 2018 14:04:26 -0700] rev 37875
crecord: fix line number in hunk header (issue5917)
`@@ -1,1 +-1,0 @@` is not a valid patch hunk header.
Change it to `@@ -1,1 +0,0 @@`.
Differential Revision: https://phab.mercurial-scm.org/D3737
Josef 'Jeff' Sipek <jeffpc@josefsipek.net> [Wed, 13 Jun 2018 10:41:20 -0400] rev 37874
lazymanifest: don't crash when out of memory (issue5916)
self->lines can be NULL if we failed to allocate memory for it.
Josef 'Jeff' Sipek <jeffpc@josefsipek.net> [Wed, 13 Jun 2018 10:37:39 -0400] rev 37873
cext: stop worrying and love the free(NULL)
There is no need to check for a NULL pointer before calling free since
free(NULL) is defined by C standards as a no-op. Lots of software relies on
this behavior so it is completely safe to call even on the most obscure of
systems.
Augie Fackler <augie@google.com> [Sun, 20 May 2018 23:05:18 -0400] rev 37872
tests: fix test-patch.t on pickier /bin/sh implementations
This is a graft of 0b39edeff033 and f44306940c94 from default because
I'm tired of seeing the FreeBSD build be red on stable. See those
revisions for details on what's going on here.
Jun Wu <quark@fb.com> [Wed, 06 Jun 2018 12:53:26 -0700] rev 37871
chg: fix an undefined behavior about memcpy
Spot by Wez Furlong. `memcpy(x, NULL, 0)` is undefined according to [1].
[1]: https://stackoverflow.com/questions/5243012
Differential Revision: https://phab.mercurial-scm.org/D3698
Augie Fackler <raf@durin42.com> [Wed, 06 Jun 2018 13:28:49 -0400] rev 37870
Added signature for changeset 9c5ced5276d6
Augie Fackler <raf@durin42.com> [Wed, 06 Jun 2018 13:28:48 -0400] rev 37869
Added tag 4.6.1 for changeset 9c5ced5276d6
Augie Fackler <augie@google.com> [Mon, 30 Apr 2018 22:24:58 -0400] rev 37868
mpatch: avoid integer overflow in combine() (SEC)
All the callers of this function can handle a NULL return, so that
appears to be the "safe" way to report an error.
Augie Fackler <augie@google.com> [Mon, 30 Apr 2018 22:23:06 -0400] rev 37867
mpatch: avoid integer overflow in mpatch_decode (SEC)
Augie Fackler <augie@google.com> [Mon, 30 Apr 2018 22:20:13 -0400] rev 37866
mpatch: fix UB integer overflows in discard() (SEC)
Augie Fackler <augie@google.com> [Mon, 30 Apr 2018 22:15:11 -0400] rev 37865
mpatch: fix UB in int overflows in gather() (SEC)
Augie Fackler <augie@google.com> [Thu, 03 May 2018 12:54:20 -0400] rev 37864
mpatch: introduce a safesub() helper as well
Same reason as safeadd().
Augie Fackler <augie@google.com> [Mon, 30 Apr 2018 22:13:42 -0400] rev 37863
mpatch: introduce a safeadd() helper to work around UB int overflow
We're about to make extensive use of this. This change duplicates some
stdbool.h portability hacks from cext/util.h. We should probably clean
that up in the future, but we'll skip that for now in order to make
security backports easier.
Augie Fackler <augie@google.com> [Sat, 28 Apr 2018 10:09:12 -0400] rev 37862
mpatch: ensure fragment start isn't past the end of orig (SEC)
Caught by oss-fuzz fuzzer during development.
This defect is OVE-20180430-0004. A CVE has not been obtained as of
this writing.
Augie Fackler <augie@google.com> [Sat, 28 Apr 2018 02:04:56 -0400] rev 37861
mpatch: protect against underflow in mpatch_apply (SEC)
Also caught by oss-fuzz fuzzer during development.
This defect is OVE-20180430-0002. A CVE has not been obtained as of this writing.
Augie Fackler <augie@google.com> [Sat, 28 Apr 2018 00:42:16 -0400] rev 37860
mpatch: be more careful about parsing binary patch data (SEC)
It appears to have been possible to trivially walk off the end of an
allocated region with a malformed patch. Oops.
Caught when writing an mpatch fuzzer for oss-fuzz.
This defect is OVE-20180430-0001. A CVE has not been obtained as of
this writing.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 06 Jun 2018 09:14:33 -0700] rev 37859
zstandard: pull in bug fixes from upstream 0.9.1 release (issue5884)
This changeset contains the meaningful code changes from
python-zstandard's 0.9.1 release. The main fix is to restore
support for compiling with mingw.
Anton Shestakov <av6@dwimlabs.net> [Wed, 06 Jun 2018 21:19:42 +0800] rev 37858
templatefuncs: only render text portion of minirst.format() result
When "keep" argument is provided, the function returns (text, pruned), where
pruned is a list of sections from the original plain text that were pruned from
the rendered result. Let's not output it together with the rendered HTML.
Anton Shestakov <av6@dwimlabs.net> [Wed, 06 Jun 2018 21:15:26 +0800] rev 37857
tests: demonstrate that hgweb renders "pruned" that minirst.format() returns
Notice at the bottom of the help text there's "windows". It's a section that is
in the original help text, but was pruned (because hgweb didn't ask for it).
Matt Harbison <matt_harbison@yahoo.com> [Tue, 05 Jun 2018 23:49:54 -0400] rev 37856
rebase: avoid RevlogError when computing obsoletenotrebased (issue5907)
The key to reproducing this seems to be missing an obsolete node that is not an
ancestor of the destination.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 02 Jun 2018 13:44:44 -0400] rev 37855
rebase: prioritize indicating an interrupted rebase over update (issue5838)
This should also cover the transplant extension, and any other non clearable
states.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 02 Jun 2018 13:25:45 -0400] rev 37854
tests: demonstrate inconsistent messaging around interrupted rebases
Matt Harbison <matt_harbison@yahoo.com> [Thu, 31 May 2018 22:15:52 -0400] rev 37853
tests: adapt test-check-pylint to run on Windows
The line endings are explicitly converted because this was ending up with
'\r (no-eol) (esc)' lines, in addition to the usual '\r (esc)' lines. I've seen
the fakerc output on other recently installed systems though (10.13 and/or
Fedora 26). Unfortunately, the output here uses '\\' on Windows, so glob away
the whole path.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 31 May 2018 22:11:47 -0400] rev 37852
hghave: avoid a deadlock reading the child process's output
The output of `pylint` is voluminous enough that it fills the buffer on Windows,
and waited for the parent to read it out. But the parent was waiting on the
child to exit.
I'm not sure what the intent of setting `ret = -1` in the exception handler just
above this was...
Matt Harbison <matt_harbison@yahoo.com> [Thu, 31 May 2018 09:19:09 -0400] rev 37851
lfs: bypass wrapped functions when reposetup() hasn't been called (issue5902)
There are only a handful of methods that access repo attributes that are applied
in reposetup(). The `diff` test covers all of the commands that call
scmutil.prefetchfiles(). Along the way, I saw that adding files and upgrading
the repo format were also problems (also tested here).
I don't think running `hg serve` through the commandserver is sane, but I
conditionalized both the capabilities and the wsgirequest handler because it's
trivially correct. It doesn't look like there has ever been a caller of
candownload(), so there's no test for that path.
The upload case isn't testable, because uploadblobs() bails if there are no
pointers. The requirement should be added any time pointers are introduced, and
that would force the extension to be loaded specifically for the repo. This
covers `debuglfsupload`, the pre-push hook (which isn't set until the repo is
promoted to LFS), and uploadblobsfromrevs(), which can be called by other
extensions.
I think readfromstore() and writetostore() are only reachable as a flag
processor for revlog.REVIDX_EXTSTORED, and a requirement is added as soon as
that is seen, so I don't think those are a problem.
Yuya Nishihara <yuya@tcha.org> [Thu, 24 May 2018 21:54:31 +0900] rev 37850
help: correct signature of separate() template function
Without the dots, it looked as if separate() would take a list of arguments.
Yuya Nishihara <yuya@tcha.org> [Fri, 18 May 2018 21:32:05 +0900] rev 37849
hgweb: do not try to replace signal handlers while locking
According to the issue 5889, mod_wsgi issues a warning on signal.signal()
call, and we wouldn't want to see it in error log. The problem addressed
by d77c3b023393 could potentially occur in web session, but that would be
less likely than in user processes.
Yuya Nishihara <yuya@tcha.org> [Fri, 18 May 2018 21:24:06 +0900] rev 37848
lock: add internal config to not replace signal handlers while locking
signal.signal() is blocked in some WSGI environments, and a horrible warning
is sent to the server log. So we need a way to disable it, and I think
abusing ui.config is the simplest workaround.
Augie Fackler <augie@google.com> [Tue, 22 May 2018 21:51:20 -0400] rev 37847
merge with i18n
Wagner Bruna <wbruna@softwareexpress.com.br> [Fri, 04 May 2018 18:55:57 -0300] rev 37846
i18n-pt_BR: synchronized with 32a75a8a5b0f
Wagner Bruna <wbruna@softwareexpress.com.br> [Fri, 04 May 2018 18:55:29 -0300] rev 37845
i18n-ja: fix block indentation
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 01 May 2018 18:22:52 +0900] rev 37844
i18n-ja: synchronized with 32a75a8a5b0f
Boris Feld <boris.feld@octobus.net> [Mon, 21 May 2018 15:14:46 +0200] rev 37843
httppeer: declare 'dbg' at the function level
As we just saw in the previous changeset, having the variable defined into a
branch creates bug. This is a cheap to move it at the function level.
Boris Feld <boris.feld@octobus.net> [Fri, 04 May 2018 19:06:46 +0200] rev 37842
httppeer: properly gate debug usage behind debug flag check
The "dbg" local variable is only defined if the 'debugflag' is set to True.
However, it was used indiscriminately later in the function. We hide its usage
behind the 'debugflag' value to avoid raising a NameError.
Yuya Nishihara <yuya@tcha.org> [Tue, 15 May 2018 22:12:55 +0900] rev 37841
push: continue without locking on lock failure other than EEXIST (issue5882)
This code was added by 3f5e75c22585 "push: make locking of source optional
(issue3684)", but EACCES isn't the only error that could be triggered by
filesystem permission. I think catching LockUnavailable is more appropriate
than testing errno value by caller.
Julien Cristau <jcristau@debian.org> [Sat, 12 May 2018 22:29:28 +0200] rev 37840
bdiff: fix yet more fallout from xdiff long/int64 conversion (issue5885)
"l" in Py_BuildValue's format string means long, so passing int64_t
instead results in fireworks on 32bit architectures.
Differential Revision: https://phab.mercurial-scm.org/D3538
Yuya Nishihara <yuya@tcha.org> [Fri, 11 May 2018 20:10:22 +0900] rev 37839
revset: pass in lookup function to matchany() (issue5879)
Silly mistake in f83cb91b052e.
Yuya Nishihara <yuya@tcha.org> [Fri, 11 May 2018 20:08:30 +0900] rev 37838
test-hgweb: add test for foo-bar name lookup
This is broken since f83cb91b052e "revset: pass in lookup function instead
of repo (API)."
Boris Feld <boris.feld@octobus.net> [Tue, 08 May 2018 14:17:46 -0700] rev 37837
bundle2: mark the bundle2 part as advisory (issue5872)
It blocks old clients to read bundle including this part.
Differential Revision: https://phab.mercurial-scm.org/D3481
Boris Feld <boris.feld@octobus.net> [Tue, 08 May 2018 11:39:38 +0200] rev 37836
debugbundle: also display if a part is mandatory or advisory
Most parts are mandatory but when introducing new parts, they should be
advisory if included by default or old clients won't be able to process it.
Differential Revision: https://phab.mercurial-scm.org/D3480
Kevin Bullock <kbullock@ringworld.org> [Sat, 05 May 2018 18:03:01 -0500] rev 37835
Added signature for changeset 6614cac550ae
Kevin Bullock <kbullock@ringworld.org> [Sat, 05 May 2018 18:02:59 -0500] rev 37834
Added tag 4.6 for changeset 6614cac550ae
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 30 Nov 2017 21:19:46 -0500] rev 37833
filelog: don't crash on invalid copy metadata (issue5748)
"copy" and "copyrev" are both supposed to appear next to each other.
However, a user report demonstrated a crash that indicates that
something in the wild is producing "copy" without "copyrev"
(probably `hg convert`).
While we should definitely fix the source of the bad metadata,
the bad code causing the crash is already in the wild and who knows
how many repositories are impacted. So let's be more defensive
when accessing the file revision metadata.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Apr 2018 15:32:11 -0700] rev 37832
httppeer: detect redirect to URL without query string (issue5860)
197d10e157ce subtly changed the HTTP peer's handling of HTTP redirects.
Before that changeset, we instantiated an HTTP peer instance and
performed the capabilities lookup with that instance. The old code had
the following relevant properties:
1) The HTTP request layer would automatically follow HTTP redirects.
2) An encountered HTTP redirect would update a peer instance variable
pointing to the repo URL.
3) The peer would automagically perform a "capabilities" command
request if a caller requested capabilities but capabilities were
not yet defined.
The first HTTP request issued by a peer is for ?cmd=capabilities. If
the server responds with an HTTP redirect to a ?cmd=capabilities URL,
the HTTP request layer automatically followed it, retrieved a valid
capabilities response, and the peer's base URL was updated
automatically so subsequent requests used the proper URL. In other
words, things "just worked."
In the case where the server redirected to a URL without the
?cmd=capabilities query string, the HTTP request layer would follow
the redirect and likely encounter HTML. The peer's base URL would be
updated and the unexpected Content-Type would raise a RepoError. We
would catch RepoError and immediately call between() (testing the case
for pre 0.9.1 servers not supporting the "capabilities" command). e.g.
try:
inst._fetchcaps()
except error.RepoError:
inst.between([(nullid, nullid)])
between() would eventually call into _callstream(). And _callstream()
made a call to self.capable('httpheader'). capable() would call
self.capabilities(), which would see that no capabilities were set
(because HTML was returned for that request) and call the "capabilities"
command to fetch capabilities. Because the base URL had been updated
from the redirect, this 2nd "capabilities" command would succeed and
the client would immediately call "between," which would also succeed.
The legacy handshake succeeded. Only because "capabilities" was
successfully executed as a side effect did the peer recognize that it
was talking to a modern server. In other words, this all appeared to
work accidentally.
After 197d10e157ce, we stopped calling the "capabilities" command on
the peer instance. Instead, we made the request via a low-level opener,
detected the redirect as part of response handling code, and passed the
redirected URL into the constructed peer instance.
For cases where the redirected URL included the query string, this
"just worked." But for cases where the redirected URL stripped the query
string, we threw RepoError and because we removed the "between" handshake
fallback, we fell through to the "is a static HTTP repo" check and
performed an HTTP request for .hg/requires.
While 197d10e157ce was marked as backwards incompatible, the only
intended backwards incompatible behavior was not performing the
"between" fallback. It was not realized that the "between" command
had the side-effect of recovering from an errant redirect that
dropped the query string.
This commit restores the previous behavior and allows clients to
handle a redirect that drops the query string. In the case where
the request is redirected and the query string is dropped, we raise
a special case of RepoError. We then catch this special exception in
the handshake code and perform another "capabilities" request against
the redirected URL. If that works, all is well. Otherwise, we fall back
to the "is a static HTTP repo" check.
The new code is arguably better than before 197d10e157ce, as it is
explicit about the expected behavior and we avoid performing a
"between" request, saving a server round trip.
Differential Revision: https://phab.mercurial-scm.org/D3433
Yuya Nishihara <yuya@tcha.org> [Thu, 03 May 2018 14:43:25 +0900] rev 37831
hgweb: prevent triggering dummy href="#" handler
Follow up for the previous patch.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 02 May 2018 21:00:43 -0700] rev 37830
paper: add href="#" to links with click handlers
This restores the styling that was accidentally removed by the
previous change to these files.
Differential Revision: https://phab.mercurial-scm.org/D3438
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 02 May 2018 19:16:01 -0700] rev 37829
paper: don't register click handlers with inline javascript (issue5812)
The use of inline href="javascript:" undermines CSP policies that
don't allow inline javascript.
This commit changes the registering of the diffstat and line wrapping
toggle handlers to the the global DOMContentLoaded handler, thus
eliminating all inline javascript from the paper template.
Differential Revision: https://phab.mercurial-scm.org/D3437
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Apr 2018 17:28:59 -0700] rev 37828
hgweb: allow Content-Security-Policy header on 304 responses (issue5844)
A side-effect of 98baf8dea553 was that the Content-Security-Policy
header was set on all HTTP responses by default. This header wasn't
in our list of allowed headers for HTTP 304 responses. This would
trigger a ProgrammingError when a 304 response was issued via hgwebdir.
This commit adds Content-Security-Policy to the allow list of headers
for 304 responses so we no longer encounter the error.
Differential Revision: https://phab.mercurial-scm.org/D3436
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Apr 2018 17:22:20 -0700] rev 37827
hgweb: discard Content-Type header for 304 responses (issue5844)
A side-effect of 98baf8dea553 was that hgwebdir always sets a global
default for the Content-Type header. HTTP 304 responses don't allow
the Content-Type header. So a side-effect of this change was that
HTTP 304 responses served via hgwebdir resulted in a ProgrammingError
being raised.
This commit teaches our 304 response issuing code to drop the
Content-Type header.
Differential Revision: https://phab.mercurial-scm.org/D3435
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Apr 2018 17:08:56 -0700] rev 37826
tests: add tests demonstrating ISE for HTTP 304 responses with hgwebdir
There are two separate failures here. One for the Content-Type header.
Another for the Content-Security-Policy header.
Differential Revision: https://phab.mercurial-scm.org/D3434
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 27 Apr 2018 14:51:02 -0700] rev 37825
hgweb: guard against empty Content-Length header
Discussion in issue 5860 seems to indicate this can occur.
Differential Revision: https://phab.mercurial-scm.org/D3432
Yuya Nishihara <yuya@tcha.org> [Thu, 26 Apr 2018 21:10:56 +0900] rev 37824
test-push-http: do not clear pid file
It's okay now, but we'll end up leaking daemon processes if we add some
more.
Yuya Nishihara <yuya@tcha.org> [Thu, 26 Apr 2018 21:24:13 +0900] rev 37823
debugcolor: fix crash by empty styles (issue5856)
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 25 Apr 2018 14:51:20 -0700] rev 37822
tests: explicitly define compression engines for tests
The zstd compression engine requires C extensions and isn't present
in pure Python builds.
The compression engine list leaks into the server capabilities string.
Unless we're testing functionality specific to a compression format,
the set of compression formats supported by a server doesn't matter
much.
So this commit explicitly defines the server's compression engines for
some tests so behavior is consistent between pure and non-pure builds.
Differential Revision: https://phab.mercurial-scm.org/D3431
Augie Fackler <augie@google.com> [Wed, 25 Apr 2018 13:18:51 -0400] rev 37821
tests: update no-zstd branch of test-treediscovery.t as in 330ada7e8ea5
This side of the test got overlooked. We should probably consider
having a way to run some of our tests through a "no-zstd" case just
like we run some things through a "no-obsmarkers" case, but that's not
an appropriate thing for stable.
Differential Revision: https://phab.mercurial-scm.org/D3430
Augie Fackler <augie@google.com> [Wed, 25 Apr 2018 13:13:42 -0400] rev 37820
tests: glob away content-length changes relating to missing zstd bindings
This doesn't fix everything in these two tests around missing zstd: we
still get some changes in the CBOR payload in ways that I think we
probably shouldn't bother to glob around. Maybe we should just disable
zstd support in some of these lower-level wireproto tests?
Differential Revision: https://phab.mercurial-scm.org/D3429
Martin von Zweigbergk <martinvonz@google.com> [Wed, 25 Apr 2018 09:24:07 -0700] rev 37819
revlog: make pure version of _partialmatch() support 40-byte hex nodeids
Without this patch, test-histedit-arguments.t would fail when run with
--pure. It turned out to be because the pure version of
_partialmatch() does not support full 40-byte hex nodeids. When
histedit's instructions include things like "pick tip", it resolves
the "tip" revision early to a full nodeid (but plain hex nodeid
prefixes are not resolved to full nodeids). Then the nodeid (full or
not) is looked up using to a full nodeid later. This step is what
fails in pure mode. It has been failing since my c4131138eadb
(histedit: look up partial nodeid as partial nodeid, 2018-04-06). I
haven't verified, but I suspect histedit instructions like "pick <full
hex nodeid>" would have been failing before my commit too, though.
The fix is trivial: change a "< 40" to "<= 40".
Differential Revision: https://phab.mercurial-scm.org/D3428
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 24 Apr 2018 13:55:25 -0700] rev 37818
hgweb: reuse body file object when hgwebdir calls hgweb (issue5851)
An unintended side-effect of f0a851542a05 was that the request body
file object (which uses a util.cappedreader) was constructed twice
when hgwebdir called into hgweb. Since we attempt to read all remaining
data from this file object when Content-Length is defined and since there
were two instances of this object and the client supplied no additional
data to read, this resulted in deadlock.
The fix implemented in this commit is to reuse the request body file
object when it is passed from hgwebdir to hgweb.
A test demonstrating `hg clone` and `hg push` via hgwebdir has been
added. Without this patch, the test hangs when doing `hg clone`.
Surprisingly, this must mean that we have effectively no test coverage
of the wire protocol when run via hgwebdir.
Differential Revision: https://phab.mercurial-scm.org/D3427
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 25 Apr 2018 00:26:49 +0530] rev 37817
remotenames: mark the extension as EXPERIMENTAL
I still don't feel confident about locking the behavior of all the things in
the remotenames extension. Moreover the extension was introduced in this cycle
only. Let's mark this extension EXPERIMENTAL for now so that we can change
things especially the storage layer if required in next cycle.
I will like to use cbor at storage layer too.
Differential Revision: https://phab.mercurial-scm.org/D3426
Augie Fackler <augie@google.com> [Tue, 24 Apr 2018 22:47:14 -0400] rev 37816
tests: fix test-check-commit.t when all commits are public
I'm 99% sure this is a portable use of /bin/[, and it seems to fix the
issue I noticed on the buildbot on my machine.
Yuya Nishihara <yuya@tcha.org> [Tue, 24 Apr 2018 21:29:00 +0900] rev 37815
import: fix crash on --exact check of empty commit (issue5702)
Martin von Zweigbergk <martinvonz@google.com> [Tue, 24 Apr 2018 08:20:15 -0700] rev 37814
tests: mark test-check-interfaces.py as requiring a repo
This was failing our 4.6rc1 build like this:
mercurial.error.RepoError: repository /tmp/build-debs.zMTRhC/src-4.6rc1 not found
Differential Revision: https://phab.mercurial-scm.org/D3425
Boris Feld <boris.feld@octobus.net> [Mon, 23 Apr 2018 19:23:18 +0100] rev 37813
sshpeer: reflect actual command activity one handshake
The output from devel-peer-request is expected to give data about request and
roundtrip done to the server. Changeset a9cffd14aa04 changed some of that by
grouping hello and between commands call. However, the old sequence of command
was "emulated" in sshpeer.
Update the sshpeer to reflect this grouping of commands and update the tests
that use it.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 23 Apr 2018 23:24:53 -0400] rev 37812
tests: drop a useless glob in test-infinite-bundlestore.t
With the previous breakage tamed, the lack of test output difference was causing
the test runner to report "no result code from test" because of this glob.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 23 Apr 2018 23:22:52 -0400] rev 37811
infinitepush: ensure fileindex bookmarks use '/' separators (issue5840)
After loading up with status messages, I noticed that the subsequent matcher was
rejecting 'scratch\mybranch' on Windows. No bookmarks were reported back, and
the tests subsequently failed. I did a search for 'match', and nothing else
looks like it needs to be fixed up, but someone who understands this code should
also take a look.
I also tried setting `infinitepush.branchpattern=re:scratch\\.*` in
library-infinitepush.sh without this change, but that didn't work. Still,
should we ban '\' in these bookmarks to avoid confusion? I thought I saw code
that sandwiches a pattern between 're:^' and '.*', so perhaps regex characters
will need special care?
I also noticed comments in externalbundlestore.{read,write} that it won't work
on Windows because of opening an open file. But I don't see a test failure, so
this may lack test coverage.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 22 Apr 2018 11:54:10 -0700] rev 37810
interfaceutil: module to stub out zope.interface
The startup time of `hg` increased during the 4.6 development cycle. A
cause of that was importing more modules and doing more work at module
import time.
The import of zope.interface and the declaring of various interfaces
is partially responsible for the startup time regression.
Our current usage of zope.interface doesn't do much at run time: we are
merely declaring interfaces and stating that certain types implement
various interfaces. Core Mercurial is not (yet) using of any of
zope.interface features that actually require that interface plumbing be
defined. The only place we actually need the interface metadata is in
test-check-interfaces.py.
This commit establishes a new interfaceutil module. It exposes the subset
of the zope.interface API that we currently use. By default, the APIs
no-op. But if an environment variable is set, we export the real
zope.interface APIs.
Existing importers of zope.interface have been converted to use the new
module. test-check-interfaces.py has been updated to define the
environment variable so the real zope.interface is used.
The net effect of this change is we stop importing 9 zope.interface.*
modules and we no longer perform interface bookkeeping when registering
interfaces.
On my i7-6700K on Linux, a shell loop that runs `hg log -r .` 300 times
on a repo with 1 commit shows a significant CPU time improvement
(average of 4 runs):
4.5: 14.814s
before: 19.028s
after: 16.945s
And with `run-tests.py -j10` (single run):
4.5: ~3100s (~51.7m)
before: ~4450s (~74.2m)
after: ~3980s (~66.3m)
So this claws back about half of the regressions in 4.6.
Differential Revision: https://phab.mercurial-scm.org/D3419
Yuya Nishihara <yuya@tcha.org> [Mon, 23 Apr 2018 21:13:19 +0900] rev 37809
test-fix: normalize precision of mtime copied by 'cp -p'
Appears that MSYS cp only copies mtime in seconds.
Augie Fackler <augie@google.com> [Fri, 20 Apr 2018 14:43:45 -0400] rev 37808
merge stable heads
Augie Fackler <raf@durin42.com> [Fri, 20 Apr 2018 14:37:48 -0400] rev 37807
Added signature for changeset 1ec874717d8a
Augie Fackler <raf@durin42.com> [Fri, 20 Apr 2018 14:37:47 -0400] rev 37806
Added tag 4.6rc1 for changeset 1ec874717d8a
Kim Alvefur <zash@zash.se> [Fri, 20 Apr 2018 15:39:32 +0200] rev 37805
internals: correct capitalization of 'compression' stream level parameter
Yuya Nishihara <yuya@tcha.org> [Fri, 20 Apr 2018 20:54:32 +0900] rev 37804
test-check-code: prevent from adding Python modules shadowed by ancient C
Yuya Nishihara <yuya@tcha.org> [Fri, 20 Apr 2018 20:48:10 +0900] rev 37803
diffhelper: rename module to avoid conflicts with ancient C module (issue5846)
Historically we had had C extensions in mercurial/, which shadows the pure
Python modules of the same name forever unless we do clean build/install.
I'm sloppy to think about new name, so just dropped the "s".
Yuya Nishihara <yuya@tcha.org> [Thu, 19 Apr 2018 23:33:17 +0900] rev 37802
diffhelpers: backport 9e40bc4c1bde from C implementation
9e40bc4c1bde just says "harden testhunk." I don't think this would be
the case, but it makes some sense to avoid negative index.
Yuya Nishihara <yuya@tcha.org> [Fri, 20 Apr 2018 20:38:29 +0900] rev 37801
version: make parser more robust for rc variants and ill-formed strings
Tom Prince <mozilla@hocat.ca> [Wed, 18 Apr 2018 19:02:44 -0600] rev 37800
phabricator: specify some metadata compatibly with arc
Differential Revision: https://phab.mercurial-scm.org/D3414
Yuya Nishihara <yuya@tcha.org> [Thu, 19 Apr 2018 19:46:24 +0900] rev 37799
diff: restore original color scheme for worddiff
I'm not young. "red dim" is nearly invisible, and "red" vs "red dim" is
too subtle to see difference. And, "underline" is necessary to highlight
whitespace changes.
Yuya Nishihara <yuya@tcha.org> [Thu, 19 Apr 2018 22:12:10 +0900] rev 37798
test-fix: fix use of 'f --newer' to check that foo.whole is not updated
Here we want to assert not 'mtime(foo.whole.orig) < mtime(foo.whole)'.
The condition has to be inverted since 'f --newer' says "newer" if the
mtimes match.
Alternatively, we could insert 'sleep 2' before 'cp' to ensure that
foo.whole.orig is newer than foo.whole, but a fewer sleeps should be
better.
Yuya Nishihara <yuya@tcha.org> [Thu, 19 Apr 2018 19:55:51 +0900] rev 37797
context: translate FilteredIndex/LookupError at repo[changeid] (API)
This partially backs out ecd3f6909184. It seems layering violation for
repo[changeid] to raise storage-level exceptions transparently. Otherwise,
we would have to rewrite callers to catch all of them.
try:
repo[rev_or_node]
except (error.RepoLookupError, error.FilteredIndexError,
error.FilteredLookupError):
pass
This would also fix filectx._changectx(), which catches FilteredRepoLookupError
to fall back to the unfiltered path.
Yuya Nishihara <yuya@tcha.org> [Thu, 19 Apr 2018 20:22:33 +0900] rev 37796
inifinitepush: fix filebundlestore to close file
Boris Feld <boris.feld@octobus.net> [Wed, 18 Apr 2018 15:07:06 +0200] rev 37795
notify: only notify for non-filtered revision
This should fix issue5821 for hook targeting individual revision.
Boris Feld <boris.feld@octobus.net> [Wed, 18 Apr 2018 15:05:12 +0200] rev 37794
notify: access the initial revision on an unfiltered repository (issue5821)
This should avoid crash when the first revision pushed end up being hidden.
Boris Feld <boris.feld@octobus.net> [Wed, 18 Apr 2018 15:04:12 +0200] rev 37793
notify: use changelog API to iterate over revision number (issue5821)
This will avoid iterating over filtered revision.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 19 Apr 2018 00:55:36 -0400] rev 37792
infinitepush: open files in binary mode
This fixes the scary looking abort in test-infinitepush-ci.t when unbundling:
--- tests/test-infinitepush-ci.t
+++ tests/test-infinitepush-ci.t.err
@@ -84,15 +84,12 @@
$ hg unbundle .hg/scratchbranches/filebundlestore/a4/c2/a4c202c147a9c4bb91bbadb56321fc5f3950f7f2
adding changesets
- adding manifests
- adding file changes
- added 1 changesets with 1 changes to 1 files
- new changesets 6cb0989601f1
- (run 'hg update' to get a working copy)
-
- $ hg glog
- o 1:6cb0989601f1 added a
- | public
+ transaction abort!
+ rollback completed
+ abort: stream ended unexpectedly (got 68 bytes, expected 218759168)
+ [255]
+
+ $ hg glog
@ 0:67145f466344 initialcommit
public
This was found by grepping for '"r', "'r", '"w' and "'w" after manually creating
a bundle from the same revision, diffing against the corrupt one, and seeing CRs
sprinkled around. Sadly, the missing bookmarks are still a problem in the two
remaining test failures.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 30 Mar 2018 21:12:09 -0400] rev 37791
tests: stabilize test-fix.t for Windows
Augie Fackler <raf@durin42.com> [Wed, 18 Apr 2018 15:39:22 -0400] rev 37790
Added signature for changeset ed5448edcbfa
Augie Fackler <raf@durin42.com> [Wed, 18 Apr 2018 15:39:21 -0400] rev 37789
Added tag 4.6rc0 for changeset ed5448edcbfa
Augie Fackler <augie@google.com> [Wed, 18 Apr 2018 15:32:08 -0400] rev 37788
merge with default to begin 4.6 freeze
# no-check-commit because of many vendored packages
Augie Fackler <augie@google.com> [Sun, 04 Mar 2018 15:29:41 -0500] rev 37787
rebase: introduce support for automatically rebasing orphan changes
_destautorebase(SRC) is based on the _destrestack(SRC) revset from
fbamend. The supporting _possibledestination function is extracted
from evolve, with minor cleanups.
We've considered some alternatives here:
* This change, but with --auto as the flag name. We're hedging our bets
on this a little in this change so that if this ends up being the wrong
direction we haven't burned the valauble --auto name on rebase.
* --destination auto: I've got reservations about the discoverability of this,
and we don't currently have a good story for a revset alias of sorts that
changes behavior depending on the context in which it's used.
* A "rebase presets" feature, where we could use the currently-an-error
positional argument space for the rebase command to define presets, so that
users could define a 'linearize' preset that specifies
--revision='orphan()-obsolete()' and --dest=_destautoorphanrebase(SRC).
Personally, I find the third option somewhat appealing, but am
hesitant to "spend" the functionality space of positional arguments to
the rebase command. We should revisit the way we expose this
functionality sometime in the 4.7 cycle once we've had a chance to vet
the implementation of the functionality.
Differential Revision: https://phab.mercurial-scm.org/D2668
Augie Fackler <augie@google.com> [Wed, 18 Apr 2018 14:32:36 -0400] rev 37786
py3: whitelist six new passing tests
Differential Revision: https://phab.mercurial-scm.org/D3406
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 16 Apr 2018 22:21:54 -0700] rev 37785
wireproto: rename wireproto to wireprotov1server (API)
We have wireprotov2server, wireprotov1peer, and wireprotov2peer.
wireproto only contains server functionality. So it makes sense to
rename it to wireprotov1server so the naming aligns with everything
else.
Differential Revision: https://phab.mercurial-scm.org/D3400
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 16 Apr 2018 22:10:02 -0700] rev 37784
wireproto: move version 2 commands dict to wireprotov2server
This was the final piece of version 2 referenced in wireproto. The
break between server implementations is now much cleaner.
Differential Revision: https://phab.mercurial-scm.org/D3399
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 16 Apr 2018 22:08:13 -0700] rev 37783
wireproto: move supportedcompengines out of wireproto
This function is used by both version 1 and version 2. It belongs in
a common module.
"wireprototypes" may not be the best module name. I may rename it...
Differential Revision: https://phab.mercurial-scm.org/D3398
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 16 Apr 2018 22:00:52 -0700] rev 37782
wireproto: reimplement dispatch() for version 2 server
The code is minimal. I'm trying to create a cleaner break between
version 1 and version 2 server code.
Differential Revision: https://phab.mercurial-scm.org/D3397
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 16 Apr 2018 21:52:33 -0700] rev 37781
wireproto: move command registration types to wireprototypes
These are shared across wire protocol implementations. wireprototypes
is our module for common code.
Differential Revision: https://phab.mercurial-scm.org/D3396
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 16 Apr 2018 21:49:59 -0700] rev 37780
wireproto: make version 2 @wireprotocommand an independent function
Previously, the code for this decorator was shared between version 1
and version 2 commands. Very few parts of the function were identical.
So I don't think sharing is justified.
wireprotov2server now has its own @wireprotocommand decorator function.
Because the decorator is no longer shared, code for configuring the
transport policy has been removed. i.e. commands must have separate
implementations for each wire protocol version.
Differential Revision: https://phab.mercurial-scm.org/D3395
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 16 Apr 2018 21:38:52 -0700] rev 37779
wireproto: don't pass transportpolicy argument
The default is version 1 only. So we don't need to pass this argument
when declaring commands.
Differential Revision: https://phab.mercurial-scm.org/D3394
Sushil khanchi <sushilkhanchi97@gmail.com> [Wed, 18 Apr 2018 19:25:35 +0530] rev 37778
forget: rename --confirm to --interactive
Differential Revision: https://phab.mercurial-scm.org/D3405
Joerg Sonnenberger <joerg@bec.de> [Tue, 17 Apr 2018 13:46:18 +0200] rev 37777
notify: add maxdiffstat option to truncate long file lists
Large scale changes like a new GCC version can easily result in 1MB+
emails due to diffstat alone. The new maxdiffstat option truncates the
list similar to what maxdiff already provides for the diffs.
Differential Revision: https://phab.mercurial-scm.org/D3402
Yuya Nishihara <yuya@tcha.org> [Fri, 13 Apr 2018 23:45:07 +0900] rev 37776
log: fix crash on empty revision with --copies switch
If a revset is empty, .max() raises ValueError.
I don't see any reason to recompute the revs, so I made it reuse the one
returned by logcmdutil.getrevs(). If no revs specified by command line,
the endrev will be smartset.spanset(repo) + 1, which is basically the same
as len(repo), the default of getrenamedfn(). If --follow specified,
revs.max() points to the working parent, which seems more correct.
Yuya Nishihara <yuya@tcha.org> [Tue, 17 Apr 2018 21:59:58 +0900] rev 37775
revsetlang: do not pass in non-bytes to parse()
Since parse() isn't a simple function, we shouldn't expect it would raise
TypeError or ValueError for invalid inputs. Before, TypeError was raised
at 'if pos != len(spec)', which was quite late to report an error.
This patch also makes tokenize() detect invalid object before converting
it to a py3-safe bytes.
Spotted while adding the 'revset(...)' hack to _parsewith().
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Apr 2018 00:30:39 +0900] rev 37774
fix: use templater to substitute values in command string
bytes.format() isn't supported on Python 3. Luckily, our template syntax
is similar so we can reuse it. We need a hack to disable \-escapes as '\'
is a directory separator on Windows.
Yuya Nishihara <yuya@tcha.org> [Fri, 13 Apr 2018 23:07:12 +0900] rev 37773
logcmdutil: pass formatter to jsonchangeset as argument
And rename the class. Now we can reuse it for 'log -Tcbor' (and '-Tpickle'
if we want.)
Yuya Nishihara <yuya@tcha.org> [Fri, 13 Apr 2018 22:47:14 +0900] rev 37772
logcmdutil: rewrite jsonchangeset printer to be backed by jsonformatter
This is a bit slower than the original implementation, but I don't think
that would actually matter. It's still faster than full templating.
$ hg log -Tjson -r0:5000 --time > /dev/null
(orig) time: real 1.550 secs (user 1.500+0.000 sys 0.040+0.000)
(new) time: real 1.810 secs (user 1.740+0.000 sys 0.070+0.000)
cf.
$ hg log -Tdefault -r0:5000 --time > /dev/null
time: real 4.980 secs (user 4.850+0.000 sys 0.130+0.000)
$ hg log -r0:5000 --time > /dev/null
time: real 2.340 secs (user 2.220+0.000 sys 0.100+0.000)
$ hg log -r0:5000 -q --time > /dev/null
time: real 0.750 secs (user 0.670+0.000 sys 0.070+0.000)
The test output changes because keys are sorted alphabetically.
Yuya Nishihara <yuya@tcha.org> [Fri, 13 Apr 2018 22:20:13 +0900] rev 37771
logcmdutil: remove unused attribute 'cache' from jsonchangeset printer
Perhaps it's a copy-pasta of changeset_templater at bd15932846a4.
Yuya Nishihara <yuya@tcha.org> [Fri, 13 Apr 2018 22:58:49 +0900] rev 37770
formatter: convert timestamp to int
Spotted while porting jsonchangeset printer to formatter. A timestamp may be
float type, but its meaningful part is just int.
Joerg Sonnenberger <joerg@bec.de> [Tue, 17 Apr 2018 03:07:01 +0200] rev 37769
bundle: introduce per-engine compression level
If experimental.bundlecomplevel.$engine is set, prefer it over the
generic experimental.bundlecomplevel. Given that compression levels have
widely different meanings across engines, this allows much saner
configuration.
Differential Revision: https://phab.mercurial-scm.org/D3393
Joerg Sonnenberger <joerg@bec.de> [Tue, 17 Apr 2018 02:41:25 +0200] rev 37768
bundlespec: drop externalnames flag
Always provide the human readable version of compression and version.
Add the translated wire format name in the new wirecompression and
wireversion fields.
Differential Revision: https://phab.mercurial-scm.org/D3392
Martin von Zweigbergk <martinvonz@google.com> [Mon, 16 Apr 2018 23:29:09 -0700] rev 37767
revlog: make shortest() take a full binary nodeid (API)
Follow-up to 7b2955624777 (scmutil: make shortesthexnodeidprefix()
take a full binary nodeid, 2018-04-14).
Differential Revision: https://phab.mercurial-scm.org/D3403
Matt Harbison <matt_harbison@yahoo.com> [Fri, 06 Apr 2018 11:13:47 -0400] rev 37766
lfs: add the 'Authorization' property to the Batch API response, if present
The client copies all of these properties under 'header' to the HTTP Headers of
the subsequent GET or PUT request that it performs. That allows the Basic HTTP
authentication used to authorize the Batch API request to also authorize the
upload/download action.
There's likely further work to do here. There's an 'authenticated' boolean key
in the Batch API response that can be set, and there is an 'LFS-Authenticate'
header that is used instead of 'WWW-Authenticate'[1]. (We likely need to
support both, since some hosting solutions are likely to only respond with the
latter.) In any event, this works with SCM Manager, so there is real world
benefit.
I'm limiting the headers returned to 'Basic', because that's all the lfs spec
calls out. In practice, I've seen gitbucket emit custom header content[2].
[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md#response-errors
[2] https://github.com/gitbucket/gitbucket/blob/35655f33c7713f08515ed640ece0948acd6d6168/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala#L119
Matt Harbison <matt_harbison@yahoo.com> [Sat, 14 Apr 2018 21:16:35 -0400] rev 37765
lfs: enable the final download count status message
At this point, I think all of the core commands are prefetching, except grep and
verify. Verify will need some special handling, in case the revlogs are
corrupt.
Grep has an issue that still needs to be debugged, but we probably need to give
the behavior some thought too- it would be a shame to have to download
everything in order to search. I think the benefit of having this info for all
commands outweighs extra printing in a command that is arguably not well
behaved in this context anyway.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 14 Apr 2018 20:11:27 -0400] rev 37764
diff: invoke the file prefetch hook
By invoking it this deep within the command, we pick up both subrepo and hgweb
support, as well as --patch support for commands that implement logopts.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 14 Apr 2018 19:43:45 -0400] rev 37763
export: invoke the file prefetch hook
cmdutil.exportfile() is only called by shelve, mq and patchbomb. Those are
unlikely to mix with lfs, but it may as well be invoked there for completeness.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 14 Apr 2018 18:50:45 -0400] rev 37762
scmutil: teach the file prefetch hook to handle multiple commits
The remainder of the commands that need prefetch deal with multiple revisions.
I initially coded this as a separate hook, but then it needed a list of files
to handle `diff` and `grep`, so it didn't seem worth keeping them separate.
Not every matcher will emit bad file messages (some are built from a list of
files that are known to exist). But it seems better to filter this in one place
than to push this on either each caller or each hook implementation.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 16 Apr 2018 23:39:30 -0400] rev 37761
tests: arrange for a server in wireproto-command-capabilities.t to be killed
The stray servers were piling up after the test harness exited. On Windows,
this means the *.pyd files can't be rebuilt, which is why the build warning
count dropped to 1 recently.
Boris Feld <boris.feld@octobus.net> [Tue, 10 Apr 2018 16:06:52 +0200] rev 37760
revset: skip legacy lookup for revspec wrapped in 'revset(...)'
Currently, multiple labels can take forms that can be confused with revset
(eg: "rev(0)" is a valid tag). Since we look up for tags before evaluating
revset, this means a tag can shadow a valid revset at any time.
We now enforce the strict revset parsing when wrapped with 'revset(...)'. For
now, This only work on a whole revspec (but can be used within the revset
without effect). This might change in the future if we improve the
implementation.
The feature is undocumented for now, keeping it in the experimental namespace.
In case a better approach to achieve the same goal is found.
The syntax looks like a revset but is not implemented as such for now. Since the
goal is to avoid some preprocessing that happens before revset parsing, we
cannot simply implement it as a revset predicate.
There was other approaches discussed over the mailing-list but they were less
convincing.
Having a configuration flag to disable legacy lookup have been considered but
discarded. There are too many common uses of ambiguous identifier (eg: '+',
'-' or '..') to have the legacy lookup mechanism turned off.
In addition, the approach can control the parsing of each revset, making
it more flexible. For example, a revset used as the value of an existing
configuration option (eg: pushrev) could enforce its resolution as a revset (by
using the prefix) while user inputs would still use the legacy lookup.
In addition of offering a way to unambiguously input a revset, this prefix
allow skipping the name lookup providing a significant speedup in some case.
Boris Feld <boris.feld@octobus.net> [Thu, 01 Mar 2018 11:37:16 -0500] rev 37759
revset: add more test to show current behaviors with label looking like revset
There are very few constraints on what character can be put into tags and
other labels. We add more tests showing some of extreme cases that user can
currently use.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 14 Apr 2018 20:04:59 -0700] rev 37758
commands: use command executor interface
Differential Revision: https://phab.mercurial-scm.org/D3391
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 14 Apr 2018 18:36:00 -0700] rev 37757
exchange: use command executor interface for calling listkeys
So the requests are compatible with version 2 peers.
Differential Revision: https://phab.mercurial-scm.org/D3390
Sushil khanchi <sushilkhanchi97@gmail.com> [Thu, 22 Mar 2018 16:11:42 +0530] rev 37756
forget: add --confirm option
Also added confirmopts in cmdutil.py
Differential Revision: https://phab.mercurial-scm.org/D2934
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 23:12:07 -0400] rev 37755
largefiles: opts appears to already be bytes in this instance
test-largefiles.t now passes.
Differential Revision: https://phab.mercurial-scm.org/D3364
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 23:58:13 -0400] rev 37754
tests: port inline extensions in test-hook.t to py3
This test *almost* passes now, but some import errors print very
differently in ways that seem at least somewhat important.
Differential Revision: https://phab.mercurial-scm.org/D3363
Augie Fackler <augie@google.com> [Sat, 14 Apr 2018 00:10:17 -0400] rev 37753
tests: manually print list in test-hook.t
Changes the output a bit, but not in an important way.
Differential Revision: https://phab.mercurial-scm.org/D3362
Augie Fackler <augie@google.com> [Sat, 14 Apr 2018 00:08:59 -0400] rev 37752
hook: also use pprint on lists for stable output on py2/3
Differential Revision: https://phab.mercurial-scm.org/D3361
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 23:56:11 -0400] rev 37751
hook: use stringutil.pprint instead of reinventing it
Differential Revision: https://phab.mercurial-scm.org/D3360
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 23:57:57 -0400] rev 37750
stringutil: make b prefixes on string output optional
I need this to preserve some behavior in hook.py.
Differential Revision: https://phab.mercurial-scm.org/D3359
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 23:55:37 -0400] rev 37749
stringutil: teach pprint how to format None
Differential Revision: https://phab.mercurial-scm.org/D3358
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 23:50:17 -0400] rev 37748
tests: update inline extensions in test-bundle2-exchange.t to py3
The test doesn't pass for what superficially look like good
reasons. We'll need to come back to it later.
# skip-blame because it's b prefixes and a couple of b'N' instead of str(N)
Differential Revision: https://phab.mercurial-scm.org/D3357
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 23:11:46 -0400] rev 37747
wsgicgi: un-do some prior porting work that is now wrong
The Python 3 WSGI behavior is that the environ dict should be full of
unicodes. We previously tried Too Hard here, so we unwind that bit of porting.
Also add some bytesurl() encodes on status and headers.
test-clone-cgi.t now passes.
Differential Revision: https://phab.mercurial-scm.org/D3356
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 22:36:54 -0400] rev 37746
hgweb_mod: inform hgweb class about paths actually being bytes
Differential Revision: https://phab.mercurial-scm.org/D3355
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 22:33:38 -0400] rev 37745
hgweb: inform hgweb.hgweb() entrypoint that paths should be bytes
Differential Revision: https://phab.mercurial-scm.org/D3354
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 22:27:16 -0400] rev 37744
tests: port inline cgi script in test-largefiles.t to python 3
Differential Revision: https://phab.mercurial-scm.org/D3353
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 22:26:57 -0400] rev 37743
lfcommands: use %d on known-int in format string
Differential Revision: https://phab.mercurial-scm.org/D3352
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 22:12:30 -0400] rev 37742
py3: fix test-shelve.t on Python 3
We get a better error message out of the abort here, which is fine.
Differential Revision: https://phab.mercurial-scm.org/D3351
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:53:07 -0400] rev 37741
tests: fix up a couple of minor bytes inconsistencies in run-tests.py
Only in the automatic bisection code, so fortunately nothing major is
amiss. Fixes test-run-tests.t under Python 3.
Differential Revision: https://phab.mercurial-scm.org/D3350
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:51:10 -0400] rev 37740
tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
I'm fairly certain it hasn't been until now. Mercifully, there doesn't
appear to be any ninja breakage.
Differential Revision: https://phab.mercurial-scm.org/D3349
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:30:55 -0400] rev 37739
py3: another three passing
Differential Revision: https://phab.mercurial-scm.org/D3348
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:11:28 -0400] rev 37738
httppeer: work around API differences on urllib Request objects
Since this is only a problem in httppeer, I'd rather keep this a
local-to-the-module kludge rather than pile more on pycompat. We'll
still find it easily to clean up later because it checks
pycompat.ispy3.
Differential Revision: https://phab.mercurial-scm.org/D3347
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:07:18 -0400] rev 37737
httppeer: no matter what Python 3 might think, http headers are bytes
Differential Revision: https://phab.mercurial-scm.org/D3346
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:06:50 -0400] rev 37736
httppeer: fix debug prints to work on Python 3
Differential Revision: https://phab.mercurial-scm.org/D3345
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:04:25 -0400] rev 37735
url: some bytes/str cleanup where we interface with stdlib funcs
Differential Revision: https://phab.mercurial-scm.org/D3344
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:01:17 -0400] rev 37734
hgweb: these strings should be sysstrs, not bytes
Differential Revision: https://phab.mercurial-scm.org/D3343
Augie Fackler <augie@google.com> [Fri, 13 Apr 2018 21:22:05 -0400] rev 37733
tests: port inline extensions in test-http.t to Python 3
Differential Revision: https://phab.mercurial-scm.org/D3342
Jun Wu <quark@fb.com> [Mon, 09 Apr 2018 15:58:30 -0700] rev 37732
patch: implement a new worddiff algorithm
The previous worddiff algorithm has many problems. The major problem is it
does a "similarity check" that selects a subset of matched lines to do
inline diffs. It is a bad idea because:
- The "similarity check" is non-obvious to users. For example, a simple
change from "long long x" to "int64_t x" will fail the similarity check
and won't be diff-ed as expected.
- Selecting "lines" to diff won't work as people expect if there are line
wrapping changes.
- It has a sad time complexity if lines do not match, could be O(N^2)-ish.
There are other problems in implementation details.
- Lines can match across distant hunks (if the next hunk does not have
"-" lines).
- "difflib" is slow.
The solution would be removing the "similarity check", and just diff all
words in a same hunk. So no content will be missed and everything will be
diff-ed as expected. This is similar to what code review tool like
Phabricator does.
This diff implements the word diff algorithm as described above. It also
avoids difflib to be faster.
Note about colors: To be consistent, "changed inserted" parts and "purely
insertion blocks" should have a same color, since they do not exist in the
previous version. Instead of highlighting differences, this patch chooses to
dim common parts. This is also more consistent with Phabricator or GitHub
webpage. That said, the labels are defined in a way that people can still
highlight changed parts and leave purely inserted/deleted hunks use the
"non-highlighted" color.
As one example, running:
hg log -pr df50b87d8f736aff8dc281f816bddcd6f306930c mercurial/commands.py \
--config experimental.worddiff=1 --color=debug --config diff.unified=0
The previous algorithm outputs:
[diff.file_a|--- a/mercurial/commands.py Fri Mar 09 15:53:41 2018 +0100]
[diff.file_b|+++ b/mercurial/commands.py Sat Mar 10 12:33:19 2018 +0530]
[diff.hunk|@@ -2039,1 +2039,4 @@]
[diff.deleted|-][diff.deleted.highlight|@command('^forget',][diff.deleted| ][diff.deleted.highlight|walkopts,][diff.deleted| _('[OPTION]... FILE...'), inferrepo=True)]
[diff.inserted|+@command(]
[diff.inserted|+ '^forget',]
[diff.inserted|+ walkopts + dryrunopts,]
[diff.inserted|+ ][diff.inserted.highlight| ][diff.inserted| _('[OPTION]... FILE...'), inferrepo=True)]
[diff.hunk|@@ -2074,1 +2077,3 @@]
[diff.deleted|- rejected = cmdutil.forget(ui, repo, m, prefix="",][diff.deleted.highlight| explicitonly=False)[0]]
[diff.inserted|+ dryrun = opts.get(r'dry_run')]
[diff.inserted|+ rejected = cmdutil.forget(ui, repo, m, prefix="",]
[diff.inserted|+ explicitonly=False, dryrun=dryrun)[0]]
The new algorithm outputs:
[diff.file_a|--- a/mercurial/commands.py Fri Mar 09 15:53:41 2018 +0100]
[diff.file_b|+++ b/mercurial/commands.py Sat Mar 10 12:33:19 2018 +0530]
[diff.hunk|@@ -2039,1 +2039,4 @@]
[diff.deleted|-][diff.deleted.unchanged|@command(][diff.deleted.unchanged|'^forget',][diff.deleted.unchanged| ][diff.deleted.changed|walkopts][diff.deleted.unchanged|,][diff.deleted.changed| ][diff.deleted.unchanged|_('[OPTION]... FILE...'), inferrepo=True)]
[diff.inserted|+][diff.inserted.unchanged|@command(]
[diff.inserted|+][diff.inserted.changed| ][diff.inserted.unchanged|'^forget',]
[diff.inserted|+][diff.inserted.changed| walkopts][diff.inserted.unchanged| ][diff.inserted.changed|+ dryrunopts][diff.inserted.unchanged|,]
[diff.inserted|+][diff.inserted.changed| ][diff.inserted.unchanged|_('[OPTION]... FILE...'), inferrepo=True)]
[diff.hunk|@@ -2074,1 +2077,3 @@]
[diff.deleted|-][diff.deleted.unchanged| rejected = cmdutil.forget(ui, repo, m, prefix="",][diff.deleted.changed| ][diff.deleted.unchanged|explicitonly=False][diff.deleted.unchanged|)[0]]
[diff.inserted|+][diff.inserted.changed| dryrun = opts.get(r'dry_run')]
[diff.inserted|+][diff.inserted.unchanged| rejected = cmdutil.forget(ui, repo, m, prefix="",]
[diff.inserted|+][diff.inserted.changed| ][diff.inserted.unchanged|explicitonly=False][diff.inserted.changed|, dryrun=dryrun][diff.inserted.unchanged|)[0]]
Practically, when diffing a 8k line change, the time spent on worddiff
reduces from 4 seconds to 0.14 seconds.
Differential Revision: https://phab.mercurial-scm.org/D3212
Jun Wu <quark@fb.com> [Mon, 19 Mar 2018 04:28:30 -0700] rev 37731
patch: buffer lines for a same hunk
Instead of yielding tokens directly, buffer them if they belong to a same
hunk. This makes it easier for the upcoming new worddiff algorithm to only
focus on the diff hunk, instead of having to worry about other contents.
This breaks how the existing experimental worddiff algorithm works, so the
algorithm was removed, and related tests are disabled for now. The next patch
will add a new worddiff algorithm.
Differential Revision: https://phab.mercurial-scm.org/D3211
Jun Wu <quark@fb.com> [Mon, 19 Mar 2018 04:28:29 -0700] rev 37730
patch: move yielding "\n" to the end of loop
The original logic makes it harder to reason about - it yields the "\n"
character belonging to the last line in the next loop iteration.
The new code is in theory a little bit slower. But is more readable. It
makes the following changes easier to read.
Differential Revision: https://phab.mercurial-scm.org/D3210