Matt DeVore <matvore@google.com> [Wed, 17 Oct 2018 15:48:01 -0700] rev 40504
blackbox: add configitem for format of log timestamps
Sometimes blackbox logs are used to report performance problems, but the
timestamps are only at second granularity, so often the timings have to
stated separately by the reporter. This is inconvenient and error-prone,
so I would like to include %f in the date format. This patch makes that
possible.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 23 Oct 2018 14:04:17 -0700] rev 40503
narrow: replace filtering in list comprehension by set operations
I didn't think of this while reviewing the patch.
Differential Revision: https://phab.mercurial-scm.org/D5188
Boris Feld <boris.feld@octobus.net> [Tue, 23 Oct 2018 19:20:22 +0200] rev 40502
storage: update sqlitestore to use the new `deltamode` parameter
While updating the filelog class, I forget to update the sqlitestore, tests
are now passing with this patch.
Differential Revision: https://phab.mercurial-scm.org/D5185
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 23 Oct 2018 16:24:04 +0300] rev 40501
narrow: rework logic to check whether we need to widen and narrow
This patch reworks logic which calculates whether we need to extend or narrow
our working copy or not.
We filter the addincludes, removeincludes, addexcludes and removeexcludes passed
from user to the actual added and removed includes and excludes. What that means
is a user can pass an already included path as addincludes, a path which is not
included as removeincludes etc. In such situations the old logic use to think we
need to do some work, whereas we don't need to do that work.
In old logic, even if we don't have anything new to include but it believes we
need to call widen, this adds some good amount of work on large repository. A
widen calls involves computing incomming csets, calling the narrow_widen() which
in non-ellipses cases goes through all the set of csets which are available
which can take ~2-3 mins on large repos. Those 2-3 minutes are spend on doing
nothing which a client can prevent by checking is there really anything which
needs to be included.
The tests changes shows that we don't go to the server anymore in such cases
which is nice.
Differential Revision: https://phab.mercurial-scm.org/D5183
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 23 Oct 2018 14:26:17 +0300] rev 40500
tests: show that adding an already included path still calls narrow_widen()
This patch adds tests demonstrating that we still go to the server in
non-ellipses widening when we have that path already on the client and there is
nothing new to download.
The next patch will try to make client side logic smart and not go to the server
if we don't need to download anything.
Differential Revision: https://phab.mercurial-scm.org/D5182
Mads Kiilerich <mads@kiilerich.com> [Sun, 14 Oct 2018 17:08:18 +0200] rev 40499
graft: introduce --base option for using custom base revision while merging
The graft command usually performs an internal merge of the current parent
revision with the graft revision, using p1 of the grafted revision as base for
the merge.
As a trivial extension of this, we introduce the --base option to allow for
using another base revision.
This can be used as a building block for grafting and collapsing multiple
changesets at once, or for grafting the resulting change from a merge as a
single simple change. (This is kind of similar to backout --parent ... only
different: this graft base must be an ancestor, but is usually *not* a parent.)
This is probably an advanced use case, and we do thus not show it in the
non-verbose help.
Boris Feld <boris.feld@octobus.net> [Thu, 18 Oct 2018 12:31:06 +0200] rev 40498
changegroup: add a option to create bundle with full snapshot only
This is easy to implement now and can be useful for benchmarking.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:21:02 +0200] rev 40497
changegroup: allow to force delta to be against p1
This new developer option is useful to general more "generic" bundle. Without
this option, a bundle generated from the repository use deltas similar to the
one stored in the specific repository it was generated from. This makes
performance testing a bit tricky.
Using deltas similar to the final result means all delta stored in the bundle
can be applied to the target repository without any further processing (except
for the rare case of a full snapshot). The application of such bundles
(almost) never exercises the (slower) path of searching for a new valid delta.
This result in unrealistic and too favorable timing and profile.
Instead, we introduce an option to make sure all revisions are stored as a
delta against p1. It might not be the best generation option, but it
guarantees that the content will be "generic", not favoring a specific target.
Boris Feld <boris.feld@octobus.net> [Tue, 09 Oct 2018 23:26:35 +0200] rev 40496
storage: also use `deltamode argument` for ifiledata
Now that lower level uses such argument, we can propagate the change to higher
layers.
Boris Feld <boris.feld@octobus.net> [Tue, 09 Oct 2018 22:02:01 +0200] rev 40495
changegroup: refactor emitrevision to use a `deltamode` argument
This new argument gathers the semantic of `sendfulltext` and `deltaprevious`
in a single value. We are about to introduce a new type of constraints.
Avoiding yet another argument sounds like a plus.
Yuya Nishihara <yuya@tcha.org> [Tue, 23 Oct 2018 21:11:13 +0900] rev 40494
branchmap: do not specify changelog as an argument
Since (unfiltered)repo.changelog lookup gets as fast as __dict__ lookup,
there's no point to pass in changelog instance.
$ hg perfbranchmap --clear-revbranch -R mozilla-central
! base
(orig) wall 20.593091 comb 20.600000 user 20.520000 sys 0.080000 (best of 3)
(this) wall 20.129126 comb 20.130000 user 20.020000 sys 0.110000 (best of 3)
This backs out most of the changes in 76d4272bd57b and 47c03042cd1d.
Yuya Nishihara <yuya@tcha.org> [Sat, 20 Oct 2018 17:56:00 +0900] rev 40493
filecache: unimplement __set__() and __delete__() (API)
Implementing __set__() implies that the descriptor can't be overridden by
obj.__dict__, which means any property access involves slow function call.
"Data descriptors with __set__() and __get__() defined always override
a redefinition in an instance dictionary. In contrast, non-data descriptors
can be overridden by instances."
https://docs.python.org/2.7/reference/datamodel.html#invoking-descriptors
This patch basically backs out 236bb604dc39, "scmutil: update cached copy
when filecached attribute is assigned (issue3263)." The problem described
in issue3263 (which is #3264 in Bugzilla) should no longer happen since
repo._bookmarkcurrent has been moved to repo._bookmarks.active. We still
have a risk of introducing similar bugs, but I think that's the cost we
have to pay.
$ hg perfrevset 'branch(tip)' -R mercurial
(orig) wall 0.139511 comb 0.140000 user 0.140000 sys 0.000000 (best of 66)
(prev) wall 0.114195 comb 0.110000 user 0.110000 sys 0.000000 (best of 81)
(this) wall 0.099038 comb 0.110000 user 0.100000 sys 0.010000 (best of 93)
Yuya Nishihara <yuya@tcha.org> [Sat, 20 Oct 2018 19:13:05 +0900] rev 40492
filecache: use try-except for faster __dict__ lookup
Python function call is slow, and the cost could be significant here.
$ hg perfrevset 'branch(tip)' -R mercurial
(orig) wall 0.139511 comb 0.140000 user 0.140000 sys 0.000000 (best of 66)
(this) wall 0.114195 comb 0.110000 user 0.110000 sys 0.000000 (best of 81)
rdamazio@google.com [Thu, 18 Oct 2018 19:57:30 -0700] rev 40491
help: displaying extension commands by default
Differential Revision: https://phab.mercurial-scm.org/D5156
rdamazio@google.com [Thu, 18 Oct 2018 19:57:05 -0700] rev 40490
help: displaying documented aliases by default
This makes aliases be displayed in "hg help" when they have a :doc config
entry, and also allows them to be assigned to a category with :category.
Differential Revision: https://phab.mercurial-scm.org/D5087
rdamazio@google.com [Sat, 13 Oct 2018 05:43:39 -0700] rev 40489
help: allow hiding of help topics
Differential Revision: https://phab.mercurial-scm.org/D5077
rdamazio@google.com [Sat, 13 Oct 2018 05:02:55 -0700] rev 40488
help: allow commands to be hidden
This is useful in enterprise environments where some workflows are
discouraged.
Differential Revision: https://phab.mercurial-scm.org/D5076
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Oct 2018 00:12:20 +0300] rev 40487
py3: add one more passing test to whitelist
Caught by python 3 builder.
Differential Revision: https://phab.mercurial-scm.org/D5175
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Oct 2018 00:05:50 +0300] rev 40486
py3: make sure we pass sysstr in sqlite3.connect()
Differential Revision: https://phab.mercurial-scm.org/D5174
Martin von Zweigbergk <martinvonz@google.com> [Tue, 05 Sep 2017 15:24:25 -0700] rev 40485
archive: use manifest.matches() to simplify and speed up matching
manifest.matches() can avoid walking paths the user did not want to
archive.
Differential Revision: https://phab.mercurial-scm.org/D5178
Martin von Zweigbergk <martinvonz@google.com> [Tue, 05 Sep 2017 15:24:22 -0700] rev 40484
archive: create alwaysmatcher when no matcher provided
Differential Revision: https://phab.mercurial-scm.org/D5177
Martin von Zweigbergk <martinvonz@google.com> [Tue, 05 Sep 2017 15:21:21 -0700] rev 40483
archive: change "matcnfn" argument to a real matcher
All callers seem to be passing a real matcher, not just a function. We
were also passing it into match.subdirmatcher(), which assumes it is a
matcher.
Differential Revision: https://phab.mercurial-scm.org/D5176
rdamazio@google.com [Wed, 09 Jan 2019 20:00:35 -0800] rev 40482
histedit: crashing with a more useful error message on empty defaultrev
Before this, `hg --config histedit.defaultrev= histedit` would crash with
File "destutil.py", line 385, in desthistedit
if revs:
UnboundLocalError: local variable 'revs' referenced before assignment
Differential Revision: https://phab.mercurial-scm.org/D5543
Augie Fackler <raf@durin42.com> [Mon, 07 Jan 2019 19:00:54 -0500] rev 40481
Added signature for changeset 197f092b2cd9
Augie Fackler <raf@durin42.com> [Mon, 07 Jan 2019 19:00:51 -0500] rev 40480
Added tag 4.8.2 for changeset 197f092b2cd9
Augie Fackler <raf@durin42.com> [Sun, 06 Jan 2019 14:58:54 -0500] rev 40479
server: always close http socket if responding with an error (issue6033)
It's possible for hgweb to respond _very_ early with an error if we're
catching certain types of errors. When we do, we need to tell the client
the socket is toast when there's a POST involved because otherwise there
can be lingering POST data on the socket that will confuse any future
requests on the socket. This manifested as a flaky failure on Linux in an
lfs extension test and a reliable failure on FreeBSD. With this patch
applied, test-lfs-serve-access.t now passes for me on FreeBSD.
Differential Revision: https://phab.mercurial-scm.org/D5498
Yuya Nishihara <yuya@tcha.org> [Sat, 05 Jan 2019 15:44:55 +0900] rev 40478
match: fix assertion for fileset with no context (issue6046)
A falsy changectx should be allowed.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 04 Jan 2019 21:01:10 -0500] rev 40477
templatekw: fix documentation typos
Yuya Nishihara <yuya@tcha.org> [Wed, 02 Jan 2019 09:41:04 +0900] rev 40476
update: do not pass in user revspec as default destination (issue6044)
When the revsingle() was introduced at 61c0df2b089a, it couldn't handle
revspec=0 (not '0') properly. That's probably why the default was set to
rev.
This is technically BC since "hg update ''" was identical to "hg update '.'"
whereas "hg update -r ''" is "hg update", but I believe that's a bug given
no test fails with this change.
Boris Feld <boris.feld@octobus.net> [Sun, 30 Dec 2018 16:11:06 +0100] rev 40475
revlog: cache delta base value under -1
Such base are invalid so we better report them early.
Boris Feld <boris.feld@octobus.net> [Thu, 27 Dec 2018 23:34:37 +0100] rev 40474
revlog: catch revlog corruption in index_baserev
A revision cannot use a base above itself, it can only happens one corrupted
repository.
Ignoring such corrupted could lead to infinite loop.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 21 Dec 2018 17:36:12 -0500] rev 40473
phabricator: properly encode boolean types in the request body
I tripped over this playing with `hg debugcallconduit` to query for valid
reviewers. If the JSON on stdin is written as 'True' or 'False', python
complains it isn't valid JSON. If it's written as 'true' or 'false', it made it
to the server, but got kicked back with this:
abort: Conduit Error (ERR-CONDUIT-CORE): Error while reading "isBot":
Expected boolean (true or false), got something else.
The test isn't really relevant here (the code can be reverted, and it will
pass), but this gives us coverage for the debug command.
Yuya Nishihara <yuya@tcha.org> [Tue, 11 Dec 2018 22:34:07 +0900] rev 40472
worker: do not swallow exception occurred in main process
Before, SystemExit(255) would be most likely raised since the worker processes
were terminated by the main process and the status would be set to 255 in
response. We should instead re-raise the exception occurred first. It's pretty
hard to debug problems like the issue 6035 with no traceback.
Yuya Nishihara <yuya@tcha.org> [Mon, 17 Dec 2018 20:46:09 +0900] rev 40471
run-tests: fix permission to clean up unreadable directories
I found many hgtests.* directories left in $TMPDIR, which couldn't be deleted
because test-ssh-repoerror.t created some directories with a-rx mode.
Yuya Nishihara <yuya@tcha.org> [Sun, 16 Dec 2018 17:08:46 +0900] rev 40470
context: reimplement memfilectx.cmp()
If I added a sanity check to basefilectx, test-context.py exploded. This
patch copies the naive implementation from overlayworkingfilectx.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 15 Dec 2018 14:55:06 -0500] rev 40469
windows: ensure pure posixfile fd doesn't escape by entering context manager
There are tests in test-revlog-mmapindex.t and test-rebase-mq-skip.t that are
fixed by this, but we usually don't use --pure on Windows. For whatever reason,
the remaining --pure failures are various errors like $ENOTDIR$ and "Access is
denied" have a trailing '.'.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 15 Dec 2018 13:54:37 -0500] rev 40468
vfs: ensure closewrapbase fh doesn't escape by entering context manager
I'm not sure if there's a problem in practice here, as there's no test failure
either way. The __exit__() and close() methods raise an exception, so maybe
__exit__() and close() are being called directly on the underlying handle when
delayclosedfile is used on a context manager? I doubt that was intended.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 15 Dec 2018 13:41:34 -0500] rev 40467
windows: ensure mixedfilemodewrapper fd doesn't escape by entering context mgr
Otherwise it seems that the special read and write handling would be bypassed.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 15 Dec 2018 01:26:18 -0500] rev 40466
py3: ensure the proxied Windows fd doesn't escape by entering context manager
The purpose of the proxy class is to provide the `name` attribute which contains
the file path. But in tests that used a context manager, it still blew up
complaining that 'int' doesn't have a 'startswith' function.
Julien Cristau <jcristau@mozilla.com> [Wed, 12 Dec 2018 06:41:19 +0100] rev 40465
test: fix test-http-bad-server with current python 2.7
https://github.com/python/cpython/pull/2825 changed the exception
message for empty http status line.
Differential Revision: https://phab.mercurial-scm.org/D5412
Matt Harbison <matt_harbison@yahoo.com> [Sun, 09 Dec 2018 23:48:50 -0500] rev 40464
hgweb: register web.comparisoncontext to the config table
This was caught in some server side logging added to debug py3 issues.
Augie Fackler <raf@durin42.com> [Tue, 04 Dec 2018 17:04:19 -0500] rev 40463
Added signature for changeset 1c8c54cf9725
Augie Fackler <raf@durin42.com> [Tue, 04 Dec 2018 17:04:17 -0500] rev 40462
Added tag 4.8.1 for changeset 1c8c54cf9725
Martin von Zweigbergk <martinvonz@google.com> [Tue, 20 Nov 2018 14:43:27 -0800] rev 40461
rebase: fix path auditing to audit path relative to repo root (issue5818)
Before this patch, when rebasing a file called "foo/bar", we would
check e.g. if "/foo" (i.e. rooted at the file system root) was a
symlink.
Differential Revision: https://phab.mercurial-scm.org/D5361
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 08:56:43 -0800] rev 40460
tests: show bad path auditing in in-memory rebase
Thanks to Yuya for providing this test case in
https://bz.mercurial-scm.org/show_bug.cgi?id=5818.
Differential Revision: https://phab.mercurial-scm.org/D5368
Martin von Zweigbergk <martinvonz@google.com> [Tue, 04 Dec 2018 08:55:48 -0800] rev 40459
tests: add a missing "cd .." to test-rebase-inmemory.t
Differential Revision: https://phab.mercurial-scm.org/D5367
Yuya Nishihara <yuya@tcha.org> [Sun, 28 Oct 2018 21:29:04 +0900] rev 40458
rust: fix possible out-of-bounds read through index_get_parents()
index_get_parents() is an internal function, which doesn't check if the
specified rev is valid. If rustlazyancestors() were instantiated with an
invalid stoprev, it would access to invalid memory region.
This is NOT a security fix as there's no Python code triggering the bug,
but included in this series to not give a notion about the memory issue
fixed by the previous patch.
Yuya Nishihara <yuya@tcha.org> [Thu, 01 Nov 2018 20:32:59 +0900] rev 40457
revlog: fix out-of-bounds access by negative parents read from revlog (SEC)
82d6a35cf432 wasn't enough. Several callers don't check negative revisions
but for -1 (nullrev), which would directly lead to out-of-bounds read, and
buffer overflow could follow. RCE might be doable with carefully crafted
revlog structure, though I don't think this would be useful attack surface.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 11:14:44 -0800] rev 40456
rebase: fix dir/file conflict detection when using in-mem merge
Differential Revision: https://phab.mercurial-scm.org/D5360
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 11:11:34 -0800] rev 40455
tests: show that in-mem rebase does not find path dir/file conflicts
Differential Revision: https://phab.mercurial-scm.org/D5359
Matt Harbison <matt_harbison@yahoo.com> [Mon, 03 Dec 2018 20:59:48 -0500] rev 40454
extdiff: register the configuration generated commands with a help category
Otherwise, 'extdiff' shows up under file management and the rest of the commands
are at the bottom under 'Uncategorized'.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 09:36:40 -0800] rev 40453
rebase: abort in-mem rebase if there's a dirty merge state
In-memory merge uses the on-disk merge state, so we should not allow
it run in-memory merge when the merge state is not clean. We should
probably not use the on-disk merge state when running in-memory merge,
but chaning that is not suitable for the stable branch.
Differential Revision: https://phab.mercurial-scm.org/D5357
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Nov 2018 16:21:37 -0800] rev 40452
rebase: preserve working copy when redoing in-mem rebase on disk
When in-memory rebase runs into conflicts, we retry it on disk. But
before we do that, we abort the in-memory rebase. That is done because
even though it's mostly in memory, there are still a few state files
written (e.g. the merge state). We should make it not write those
files so we don't need to abort, but for the stable branch, let's
explicitly clear the state we need to clear instead of running the
usual abort code.
Differential Revision: https://phab.mercurial-scm.org/D5356
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Nov 2018 15:08:43 -0800] rev 40451
tests: show that in-mem rebase falling back loses state
Both working copy changes and the merge state is lost when in-memory
rebase falls back to on-disk mode.
Differential Revision: https://phab.mercurial-scm.org/D5355
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Dec 2018 21:45:15 +0900] rev 40450
commandserver: get around ETIMEDOUT raised by selectors2
selector.select() should exits with an empty event list on timed out, but
selectors2 raises OSError if timeout expires while recovering from EINTR.
Spotted while debugging new chg feature.
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Dec 2018 21:31:19 +0900] rev 40449
selectors2: backport minimal fix of timeout handling from 2.0.1
The original code would raise TypeError since OSError() doesn't support
keyword arguments.
We can't simply import the selectors 2.0.1, which still spawns "uname -p"
through platform.system(). We could switch to the unreleased version, but
I decided to not right now to minimize the change.
Augie Fackler <augie@google.com> [Wed, 14 Nov 2018 10:12:43 -0500] rev 40448
tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
When I upgraded the FreeBSD buildbot to 11.2 it seems we picked up
clang6, but the default clang on FreeBSD doesn't include libfuzzer. I
can't find a way to sniff for libfuzzer without running a compile, so
here we are.
Differential Revision: https://phab.mercurial-scm.org/D5270
Augie Fackler <augie@google.com> [Wed, 14 Nov 2018 10:11:37 -0500] rev 40447
tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
This isn't as robust as it probably should be, but for now it'll get
the job done on the buildbots.
Differential Revision: https://phab.mercurial-scm.org/D5269
Augie Fackler <augie@google.com> [Thu, 29 Nov 2018 16:25:37 -0500] rev 40446
tests: stabilize test-inherit-mode.t on FreeBSD and macOS (issue6026)
Symbolic links are funny permissions-wise, but on the linked issue
Yuya has convinced me that we can ignore this permissions issue on
macOS (FreeBSD allows setting permissions bits but ignores them) and
we'll be in fine shape.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 28 Nov 2018 12:52:23 -0800] rev 40445
wireprotov2peer: wait for initial object before resolving future
As part of rolling out wireprotov2 with redirect support, I
encountered an edge case with regards to future resolution.
Essentially, the initial response frame from the server did not
fully decode the initial CBOR object. The frame wasn't marked as
EOS. In the previous code, we resolved the future for the request
to response.objects(), which mapped to the commandresponse instance
which would eventually produce a redirect. Upon receiving
subsequent data, the initial CBOR object containing the redirect
would be decoded and we'd process the redirect. However, the
future would already have been resolved with the initial
commandresponse.objects() and the client iterating over the
objects wouldn't receive any objects from the redirect because
the redirect was populating a different commandresponse instance!
This commit changes the logic so we don't resolve futures until
the initial CBOR response object is fully decoded or until EOS
occurs. In cases where there is an empty or partial frame
associated with a redirect, the future will now resolve with the
commandresponse containing the proper series of decoded objects.