Yuya Nishihara <yuya@tcha.org> [Sun, 28 Oct 2018 21:29:04 +0900] rev 40458
rust: fix possible out-of-bounds read through index_get_parents()
index_get_parents() is an internal function, which doesn't check if the
specified rev is valid. If rustlazyancestors() were instantiated with an
invalid stoprev, it would access to invalid memory region.
This is NOT a security fix as there's no Python code triggering the bug,
but included in this series to not give a notion about the memory issue
fixed by the previous patch.
Yuya Nishihara <yuya@tcha.org> [Thu, 01 Nov 2018 20:32:59 +0900] rev 40457
revlog: fix out-of-bounds access by negative parents read from revlog (SEC)
82d6a35cf432 wasn't enough. Several callers don't check negative revisions
but for -1 (nullrev), which would directly lead to out-of-bounds read, and
buffer overflow could follow. RCE might be doable with carefully crafted
revlog structure, though I don't think this would be useful attack surface.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 11:14:44 -0800] rev 40456
rebase: fix dir/file conflict detection when using in-mem merge
Differential Revision: https://phab.mercurial-scm.org/D5360
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 11:11:34 -0800] rev 40455
tests: show that in-mem rebase does not find path dir/file conflicts
Differential Revision: https://phab.mercurial-scm.org/D5359
Matt Harbison <matt_harbison@yahoo.com> [Mon, 03 Dec 2018 20:59:48 -0500] rev 40454
extdiff: register the configuration generated commands with a help category
Otherwise, 'extdiff' shows up under file management and the rest of the commands
are at the bottom under 'Uncategorized'.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 03 Dec 2018 09:36:40 -0800] rev 40453
rebase: abort in-mem rebase if there's a dirty merge state
In-memory merge uses the on-disk merge state, so we should not allow
it run in-memory merge when the merge state is not clean. We should
probably not use the on-disk merge state when running in-memory merge,
but chaning that is not suitable for the stable branch.
Differential Revision: https://phab.mercurial-scm.org/D5357
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Nov 2018 16:21:37 -0800] rev 40452
rebase: preserve working copy when redoing in-mem rebase on disk
When in-memory rebase runs into conflicts, we retry it on disk. But
before we do that, we abort the in-memory rebase. That is done because
even though it's mostly in memory, there are still a few state files
written (e.g. the merge state). We should make it not write those
files so we don't need to abort, but for the stable branch, let's
explicitly clear the state we need to clear instead of running the
usual abort code.
Differential Revision: https://phab.mercurial-scm.org/D5356
Martin von Zweigbergk <martinvonz@google.com> [Fri, 30 Nov 2018 15:08:43 -0800] rev 40451
tests: show that in-mem rebase falling back loses state
Both working copy changes and the merge state is lost when in-memory
rebase falls back to on-disk mode.
Differential Revision: https://phab.mercurial-scm.org/D5355
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Dec 2018 21:45:15 +0900] rev 40450
commandserver: get around ETIMEDOUT raised by selectors2
selector.select() should exits with an empty event list on timed out, but
selectors2 raises OSError if timeout expires while recovering from EINTR.
Spotted while debugging new chg feature.
Yuya Nishihara <yuya@tcha.org> [Mon, 03 Dec 2018 21:31:19 +0900] rev 40449
selectors2: backport minimal fix of timeout handling from 2.0.1
The original code would raise TypeError since OSError() doesn't support
keyword arguments.
We can't simply import the selectors 2.0.1, which still spawns "uname -p"
through platform.system(). We could switch to the unreleased version, but
I decided to not right now to minimize the change.
Augie Fackler <augie@google.com> [Wed, 14 Nov 2018 10:12:43 -0500] rev 40448
tests: sniff for libfuzzer actually being available in test-fuzz-targets.t
When I upgraded the FreeBSD buildbot to 11.2 it seems we picked up
clang6, but the default clang on FreeBSD doesn't include libfuzzer. I
can't find a way to sniff for libfuzzer without running a compile, so
here we are.
Differential Revision: https://phab.mercurial-scm.org/D5270
Augie Fackler <augie@google.com> [Wed, 14 Nov 2018 10:11:37 -0500] rev 40447
tests: sniff for /usr/local/bin/gmake and use it in test-fuzz-targets.t
This isn't as robust as it probably should be, but for now it'll get
the job done on the buildbots.
Differential Revision: https://phab.mercurial-scm.org/D5269
Augie Fackler <augie@google.com> [Thu, 29 Nov 2018 16:25:37 -0500] rev 40446
tests: stabilize test-inherit-mode.t on FreeBSD and macOS (issue6026)
Symbolic links are funny permissions-wise, but on the linked issue
Yuya has convinced me that we can ignore this permissions issue on
macOS (FreeBSD allows setting permissions bits but ignores them) and
we'll be in fine shape.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 28 Nov 2018 12:52:23 -0800] rev 40445
wireprotov2peer: wait for initial object before resolving future
As part of rolling out wireprotov2 with redirect support, I
encountered an edge case with regards to future resolution.
Essentially, the initial response frame from the server did not
fully decode the initial CBOR object. The frame wasn't marked as
EOS. In the previous code, we resolved the future for the request
to response.objects(), which mapped to the commandresponse instance
which would eventually produce a redirect. Upon receiving
subsequent data, the initial CBOR object containing the redirect
would be decoded and we'd process the redirect. However, the
future would already have been resolved with the initial
commandresponse.objects() and the client iterating over the
objects wouldn't receive any objects from the redirect because
the redirect was populating a different commandresponse instance!
This commit changes the logic so we don't resolve futures until
the initial CBOR response object is fully decoded or until EOS
occurs. In cases where there is an empty or partial frame
associated with a redirect, the future will now resolve with the
commandresponse containing the proper series of decoded objects.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 28 Nov 2018 10:37:43 -0800] rev 40444
wireprotov2peer: always return a bool from _processredirect()
Without this, we may stop servicing the redirect response if the
future has already been resolved. And the future will often be
resolved very early, since many consumers iterate the decoded
CBOR object stream and expect data to lazily arrive.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 20 Nov 2018 18:47:19 -0500] rev 40443
tests: stabilize the recent checkexec changes on Windows
This goes with bd0874977a5e.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 03:09:23 +0100] rev 40442
checkexec: create destination directory if necessary
Since 460733327640, a "share" use the cache of the source repository. A side
effect is that no `.hg/cache` directory exists in the "share" anymore. As a
result, the checkexec logic can't use it to create its temporary file and have
to use the working copy for that.
This is suboptimal, it pollutes the working copy and prevents them to keep the
file around in cache. We do not want to use the cache directory for the share
target, it might be on a different file system.
So instead, we (try to) create the directory if it is missing. This is a
simple change that fixes the current behavior regression on stable.
On default, we should probably ensure the proper directories are created when
initializing the repository. We should also introduce a 'wcache' directory to
hold cache file related to the working copy. This would clarify the cache
situation regarding shares.
The tests catch a couple of other affected cases.
Yuya Nishihara <yuya@tcha.org> [Thu, 15 Nov 2018 22:59:38 +0900] rev 40441
graft: do not try to skip rev derived from ancestor more than once (issue6024)
We check 'x in revs' in other cases, so let's do the same.
The test case credits to Tom Prince.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 16 Nov 2018 18:37:26 -0500] rev 40440
subrepo: print the status line before creating the peer for better diagnostics
I ran into a problem where I tried updating to a different branch, and the
process appeared to hang. It turned out that the subrepo revision wasn't
available locally, and I must have originally cloned it from an `hg serve -S` on
a machine that currently wasn't serving anything. It took 2+ minutes to
timeout, and didn't mention what it was connecting to even then.
There are a couple of other issues in this scenario too.
- The repo is dirty after the failed checkout because the top level repo is
updated first. We should probably make 2 passes- top down to pull
everything needed, and then do an update once everything is in place.
- Something must be reading .hgsubstate from wdir because if the same merge
command is run after the timeout, a prompt is issued that the local and
remote subrepo diverged, instead of hanging. But it lists the local version
and remote version as having the same hash.
Augie Fackler <augie@google.com> [Wed, 14 Nov 2018 11:52:13 -0500] rev 40439
tests: allow for 100% of profiled time in sleep in test-profile.t
I'm getting an annoying failure in this test on our builder, and I
*think* what's happening is that the profiler is taking _just_ long
enough to start that we're spending 100% of the profiled time in the
sleep function, which was causing the leading space to not be printed
since the 100 was in the first column of output.
Differential Revision: https://phab.mercurial-scm.org/D5272
Anton Shestakov <av6@dwimlabs.net> [Wed, 14 Nov 2018 15:06:21 +0800] rev 40438
copystore: provide unit to ui.makeprogress()
Anton Shestakov <av6@dwimlabs.net> [Wed, 14 Nov 2018 15:07:02 +0800] rev 40437
verify: provide unit to ui.makeprogress()
Augie Fackler <augie@google.com> [Tue, 13 Nov 2018 19:47:48 -0500] rev 40436
tests: fix wireproto redirection test on systems without tls1.2
Our automated package builder has some ancient configuration that
lacks modern TLS, which is how we noticed this.
Tested: the test now passes on both macOS High Sierra (has tls1.2) and
Ubuntu Trusty (which does not).
Matt Harbison <matt_harbison@yahoo.com> [Sat, 10 Nov 2018 22:25:12 -0500] rev 40435
phabricator: ensure the command summaries are available in extension help
Previously, `hg help phabricator` listed the 3 supported commands at the bottom
of the extension help, but said "no help text available".
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 09 Nov 2018 23:49:39 +0000] rev 40434
hgweb: cast bytearray to bytes
PEP-3333 seems to indicate that bytes is the only allowed type that can
be used to express the output of a WSGI application. And some WSGI
environments seem to enforce this (mod_wsgi does).
This commit universally casts bytearray instances to bytes to appease
the WSGI specification.
I found this because wireprotov2 is emitting bytearray instances. I'd
like to keep things that way because the way it builds a data
structure, bytearray is more efficient. I'd rather keep the low-level
code efficient (and using bytearray) and cast at the edges than impose
a performance penalty on code that may run outside WSGI contexts.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 08 Nov 2018 20:04:07 -0500] rev 40433
help: unjumble the list of default config values for `internals.config`
Kyle Lippincott <spectral@google.com> [Mon, 05 Nov 2018 15:01:45 -0800] rev 40432
tweakdefaults: remove commands.resolve.mark-check=abort, it is too broken
See issue6020 for the current case. I don't want to continue attempting to fix
this on the stable branch, so I'm removing from tweakdefaults and will send
fixes meant for the default branch and 4.9.
Differential Revision: https://phab.mercurial-scm.org/D5225
Kyle Lippincott <spectral@google.com> [Fri, 02 Nov 2018 11:57:45 -0700] rev 40431
resolve: when resolve.mark-check=abort, downgrade to warning if pats specified
Previously, with --config resolve.mark-check=abort, running `hg resolve -m foo`
would abort and emit a message saying to use --all. This command does not work,
though: `hg resolve -m foo --all`, and it's really weird for --all to be the
"--force" flag.
My original goal with the option was to make it so that `hg resolve -m` (no
filename arguments) was safer, which is why --all is used; in my mind, `hg
resolve -m foo` should always mark it as resolved, and `--all` is how you
specify "all the files", so that's why I chose `hg resolve -m --all` as the way
out of `hg resolve -m` aborting. This commit makes all of this work the way it
was meant to in my head :)
Differential Revision: https://phab.mercurial-scm.org/D5218
Augie Fackler <raf@durin42.com> [Fri, 02 Nov 2018 14:18:29 -0400] rev 40430
Added signature for changeset a91a2837150b
Augie Fackler <raf@durin42.com> [Fri, 02 Nov 2018 14:18:26 -0400] rev 40429
Added tag 4.8 for changeset a91a2837150b
Yuya Nishihara <yuya@tcha.org> [Sun, 28 Oct 2018 21:16:36 +0900] rev 40428
rust: fix signature of rustlazyancestors_init() function
Obviously, sizeof(int) != mem::size_of::<usize>() on amd64, though the
argument would be passed in 64-bit register anyway.
Yuya Nishihara <yuya@tcha.org> [Fri, 02 Nov 2018 21:25:35 +0900] rev 40427
tests: require SQLite 3.8.3+ as sqlitestore relies on "WITH" clause
The test fails on gcc112 because the SQLite is too old.
https://sqlite.org/changes.html#version_3_8_3
Anton Shestakov <av6@dwimlabs.net> [Fri, 19 Oct 2018 22:09:53 +0800] rev 40426
relnotes: various tweaks for release notes
Stop filtering out commits that are expected to be covered by releasenotes
extension: now we want two lists, one for WhatsNew and one for ReleaseX.Y.
Use `only(stoprev, startrev)` to make `relnotes -h` output be actually true
about what revisions are included.
More filter rules, mostly obvious.
More classifying rules to have less things in "unsorted".
Looks like nargs=1 was just making args.startrev and args.stoprev be lists for
no reason.
BC and API sections are renamed to what we're using on the WhatsNew page, and
also just skipped if empty.
Boris Feld <boris.feld@octobus.net> [Thu, 01 Nov 2018 12:52:16 +0100] rev 40425
delta: skip "empty delta" optimisation for non-general case (issue6006)
Non-general delta repository cannot delta against anything than prev. So even if
the delta to prev is empty we should use it.
This is similar to the change made in bafa1c4bb7a8.
Differential Revision: https://phab.mercurial-scm.org/D5201
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 16:32:16 -0700] rev 40424
narrow: fix copies._fullcopytracing() narrowspec filtering in graft case
I broke this too in 707c3804e607 (narrow: move copies overrides to
core, 2018-09-28). Hopefully I'm done fixing things broken by that
commit now.
Differential Revision: https://phab.mercurial-scm.org/D5213
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 16:28:11 -0700] rev 40423
tests: demonstrate broken copies._fullcopytracing()
Turns out copies._fullcopytracing() was also broken.
Differential Revision: https://phab.mercurial-scm.org/D5212
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 13:20:12 -0700] rev 40422
narrow: make copies.pathcopies() filter with narrowspec again
I broke this in 707c3804e607 (narrow: move copies overrides to core,
2018-09-28).
Differential Revision: https://phab.mercurial-scm.org/D5203
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 11:24:45 -0700] rev 40421
tests: demonstrate broken copies.pathcopies()
Differential Revision: https://phab.mercurial-scm.org/D5202
"Paul Morelle <paul.morelle@octobus.net" [Wed, 31 Oct 2018 20:32:42 +0100] rev 40420
setup: explain to distutils how we write rc versions
When we use a rc version number (e.g. 4.8rc0), bdist_msi is using
distutils.StrictVersion to parse it into a tuple of numbers.
By default, StrictVersion.version_re only recognizes [ab] for alpha/beta,
where mercurial may use '-rc' or 'rc'.
This change makes StrictVersion parse correctly our version numbers, so that
bdist_msi doesn't fail on rc versions.
Boris Feld <boris.feld@octobus.net> [Wed, 31 Oct 2018 12:08:37 -0700] rev 40419
changegroup: restore default node ordering (issue6001)
Changeset db5501d9 changed the default node ordering from "storage" to
"linearize".
While the new API is more explicit and cleaner, the "linearize" order is
problematic on certain repositories like netbeans where it makes bundling
slower the more nodes we bundle.
Pushing and pulling 100 changesets was ~20% slower and pushing and pulling
1000 changesets was ~600% slower.
A very quick analysis of profile traces showed that the pull operation was
taking more time creating the delta.
Putting back the old default order seems to be the safe option. With more time
during the next cycle, we can understand better the impact of sorting with the
DAG order by default, the source of the regression and how to mitigate it.
/!\ We are still waiting for the full performance impact but with this patch,
bundling and pulling locally (not on the performance workstation) 1000
changesets on the netbeans repository is as fast as before the regression.
Differential Revision: https://phab.mercurial-scm.org/D5196
Boris Feld <boris.feld@octobus.net> [Mon, 29 Oct 2018 17:26:25 +0100] rev 40418
changegroup: introduce an explicit linear sorting
We still need to linearize the revisions in some cases, introduce an explicit
`linear` sorting before changing back the default order.
Differential Revision: https://phab.mercurial-scm.org/D5195
Yuya Nishihara <yuya@tcha.org> [Wed, 31 Oct 2018 21:16:54 +0900] rev 40417
fix: disable use of thread-based worker
getfixes() accesses to repo, changectx, filectx, etc., so I believe there
are code paths triggering data race. Mercurial API isn't thread safe in
general.
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 31 Oct 2018 15:27:06 +0300] rev 40416
configitems: rename the config to prevent adding an alias in future
Right now the config option looks like:
[experimental.server]
stream-narrow-clones=
which does not match how config options are generally defined in core. So let's
rename this to:
[experimental]
server.stream-narrow-clones=
before the new release so that we don't have to add an alias in future for this.
Differential Revision: https://phab.mercurial-scm.org/D5198
Boris Feld <boris.feld@octobus.net> [Wed, 31 Oct 2018 11:02:08 +0100] rev 40415
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Starting with 5aef5afa8654, a valid delta parent might be "refined". This
allows repository using sparse-revlog to produce better delta chain by using
better intermediate snapshot base.
However, this refining step was performed in all cases, including for
repository not using sparse-revlog. This could produce a strange chain in the
general delta case and corrupted repository in the non-general delta case.
We now skip this step unless sparse-revlog is in use.
In issue 6006, Yuya Nishihara provided a test case using an external
repository, so we did not include it. Finding "laboratory" condition to
reproduce this case and implementing an efficient test reproducing it is a bit
tricky. We do not foresee to have the time to provide one by the release date.
Differential Revision: https://phab.mercurial-scm.org/D5197
Augie Fackler <augie@google.com> [Mon, 29 Oct 2018 16:23:42 -0400] rev 40414
http: work around custom http client classes that refuse extra attrs
I have no idea what is going on with our custom http client code at Google,
but it chokes on these extra attributes we're tucking on http clients. Since
it feels more than a little wrong to just stuff extra data on a client, let's
degrade gracefully when the client class refuses the attributes.
Anton Shestakov <av6@dwimlabs.net> [Thu, 25 Oct 2018 21:33:43 +0800] rev 40413
crecord: make nextsametype() check that parent item exists (issue6009)
Items that represent files in curses interface don't have parents.
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Wed, 24 Oct 2018 10:05:13 -0400] rev 40412
help: describe what ui.tweakdefaults changes, concretely
Currently, one has to look at the code.
A couple things are suboptimal:
- probably not translatable
- lines don't get wrapped (a couple are a bit too long)
but it seems to better this way than without help at all.
Differential Revision: https://phab.mercurial-scm.org/D5187
Matt Harbison <matt_harbison@yahoo.com> [Thu, 25 Oct 2018 00:22:42 -0400] rev 40411
logexchange: convert paths to unix when detecting the active path
This fixes the problem in the tests[1] where Windows was showing the whole path
as the remotename for local repositories.
Somebody with a better understanding of this extension should probably take a
deeper look. There may be other cases that need to be converted- specifically
the `elif not instance` and the missing `else` cases in activepath(). I also
noticed when adding debug prints that the absolute path is stored in the file,
probably not normalized. (It's wrapped up in $TESTTMP.)
[1] https://buildbot.mercurial-scm.org/builders/Win7%20x86_64%20hg%20tests/builds/1042/steps/run-tests.py%20%28python%202.7.13%29/logs/stdio
Matt Harbison <matt_harbison@yahoo.com> [Wed, 24 Oct 2018 22:40:48 -0400] rev 40410
help: update the default value specified for `profiling.time-track`
I tried conditionalizing this in a `.. container::` block, but that seemed to
add an extra blank line between the main text and the parenthetical.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 24 Oct 2018 22:24:10 -0400] rev 40409
profiling: revert the default mode back to 'cpu' on Windows
On Windows, os.times() only returns user and system times. Real elapsed time is
0. That results in no actual times reported, an end wall time of 0.000000, and
seemingly randomly sorted stack frames. This at least provides test stability
in test-profile.t.
I kind of think that `default=pycompat.iswindows and 'cpu' or 'real'` would be a
better way to set the default in configitems, but I didn't see any other
examples of this, and thought maybe there's a reason for that. That might allow
plugging the value into the help text automatically- the documented default
wasn't updated in db0dba2d157d.
Boris Feld <boris.feld@octobus.net> [Wed, 17 Oct 2018 14:47:01 +0200] rev 40408
phase: add an archived phase
This phase allows for hidden changesets in the "user space". It differs from
the "internal" phase which is intended for internal by-product only. There
have been discussions at the 4.8 sprint to use such phase to speedup cleanup
after history rewriting operation.
Shipping it in the same release as the 'internal-phase' groups the associated
`requires` entry. The important bit is to have support for this phase in the
earliest version of mercurial possible. Adding the UI to manipulate this new
phase later seems fine.
The current plan for archived usage and user interface are as follow. On a
repository with internal-phase on and evolution off:
* history rewriting command set rewritten changeset in the archived phase.
(This mean updating the cleanupnodes method).
* keep `hg unbundle .hg/strip-backup/X.hg` as a way to restore changeset for
now
(backup bundle need to contains phase data)
* [maybe] add a `hg strip --soft` advance flag
(a light way to expose the feature without getting in the way of a better
UI)
Mercurial 4.8 freeze is too close to get the above in by then.
We don't introduce a new repository `requirement` as we reuse the one
introduced with the 'archived' phase during the 4.8 cycle.
Yuya Nishihara <yuya@tcha.org> [Tue, 23 Oct 2018 20:46:21 +0900] rev 40407
exewrapper: apply clang-format to silence test-check-clang-format.t
Augie Fackler <raf@durin42.com> [Mon, 22 Oct 2018 14:48:14 -0400] rev 40406
Added signature for changeset 956ec6f1320d
Augie Fackler <raf@durin42.com> [Mon, 22 Oct 2018 14:48:11 -0400] rev 40405
Added tag 4.8rc0 for changeset 956ec6f1320d
Augie Fackler <augie@google.com> [Mon, 22 Oct 2018 14:46:06 -0400] rev 40404
merge to stable for 4.8 release freeze
Martin von Zweigbergk <martinvonz@google.com> [Mon, 22 Oct 2018 11:34:35 -0700] rev 40403
shortest: never emit 0-length prefix even if unique
It turned out that the pure version of our code for finding the
shortest unique nodeid prefix would return a 0-length string if that
was unique (because there was at most one revision in the
disambiguation set). That's kind of correct, but it can't be used as
input, so we shouldn't return it. Let's just adjust the given
minlength up to at least 1. This fixes test-template-functions.t,
which was failing in pure mode.
Differential Revision: https://phab.mercurial-scm.org/D5181
Boris Feld <boris.feld@octobus.net> [Mon, 22 Oct 2018 15:51:01 +0200] rev 40402
logtoprocess: sends the canonical command name to the subprocess
One of the use-case of logtoprocess is to monitor command duration. With the
current code, we only get whatever command name the user typed (either
abbreviated or aliased).
This makes analytics on the collected data more difficult. Stores the
canonical command name in the request object. Pass the stored canonical name
in the `req.ui.log("commandfinish", ...)` call as keyword argument to not
break potential string formatting.
Pass the value as the environment variable named `LTP_COMMAND` to the called
script.
Differential Revision: https://phab.mercurial-scm.org/D4820
Boris Feld <boris.feld@octobus.net> [Mon, 22 Oct 2018 15:47:30 +0200] rev 40401
logtoprocess: fix message formatting
The logtoprocess used to try formatting the message using keyword options
instead of always using the rest of the arguments. Update it to match blackbox
behavior.
Differential Revision: https://phab.mercurial-scm.org/D5180
Boris Feld <boris.feld@octobus.net> [Sat, 18 Aug 2018 01:44:38 +0200] rev 40400
profiling: move default mode to "real" time
Mercurial operations involve a lot of disks or network access. These impact
command runtime significantly and it seems important to report them in our
default profiling output.
Having the right default means that we don't forget them when asking people to
produces profiling traces or when doing profiling ourselves.
Moving to "real time" by default will remove the need to think about
activating it on most occasions. The "CPU" time-based profiling is still
accessible when necessary.
Anton Shestakov <av6@dwimlabs.net> [Fri, 05 Oct 2018 23:40:12 +0800] rev 40399
streamclone: abort when client needs to handle obsmarkers, but doesn't
When client doesn't have any of obsolescence markers exchange capabilities,
then it's safe to say it can't handle obsmarkers. However, if it understands
even one format version, then stream clones are fine -- client can use
"obsmarkers" bundle2 part.
Anton Shestakov <av6@dwimlabs.net> [Fri, 05 Oct 2018 23:27:17 +0800] rev 40398
streamclone: include obsstore file into stream bundle if client can read it
Matt Harbison <matt_harbison@yahoo.com> [Fri, 19 Oct 2018 18:34:42 -0400] rev 40397
setup: build exewrapper with Unicode support on py3
I didn't see a compiler switch documented anywhere, but I diffed the command
line for full VC++ project when toggling between MBCS and Unicode. This is all
they do.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 19 Oct 2018 18:32:13 -0400] rev 40396
exewrapper: convert to _tcsxxx functions for Unicode compatability
This fixes more than 50 tests on py3 on Windows when enabled, mostly hooks and
such that invoked `hg` directly. 187 left to go.
I skipped doing the abort printing with Unicode because of apparent issues with
MinGW [1]. It may be moot though, as MinGW isn't listed as a supported compiler
after 3.4 [2].
[1] https://stackoverflow.com/questions/17700797/printf-wprintf-s-s-ls-char-and-wchar-errors-not-announced-by-a-compil
[2] https://wiki.python.org/moin/WindowsCompilers
Matt Harbison <matt_harbison@yahoo.com> [Fri, 19 Oct 2018 18:23:14 -0400] rev 40395
exewrapper: drop an unused variable
Yuya Nishihara <yuya@tcha.org> [Thu, 18 Oct 2018 21:14:22 +0900] rev 40394
commands: restore compatibility for "^cmd" registration (issue6005)
This is done at loading time, where ui is available.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Oct 2018 12:30:49 +0200] rev 40393
exchangev2: support fetching shallow files history
This commit teaches the exchangev2 client code to handle fetching shallow
files data.
Only shallow fetching of files data is supported: shallow fetching of
changeset and manifest data is explicitly not yet supported.
Previously, we would fetch file revisions for changesets that were received
by the current pull operation. In the new model, we calculate the set of
"relevant" changesets given the pull depth and only fetch files data for
those changesets.
We also teach the "filesdata" command invocation to vary parameters as needed.
The implementation here is far from complete or optimal. Subsequent pulls will
end up re-fetching a lot of files data. But the application of this data should
mostly be a no-op on the client, so it isn't a big deal.
Depending on the order file revisions are fetched in, revisions could get
inserted with the wrong revision number relationships. I think the best way
to deal with this is to remove revision numbers from storage and to either
dynamically derive them (by reconstructing a DAG from nodes/parents) or remove
revision numbers from the file storage interface completely.
A missing API that we'll likely want to write pretty soon is "ensure files
for revision(s) are present." We can kind of cajole exchangev2.pull() to do
this. But it isn't very efficient. For example, in simple cases like
widening the store to obtain data for a single revision, it is probably
more efficient to walk the manifest and find exactly which file revisions
are missing and to make explicit requests for just their data. In more
advanced cases, asking the server for all files data may be more efficient,
even though it requires sending data the client already has. There is tons
of room for future experimentation here. And TBH I'm not sure what the
final state will be.
Anyway, this commit gets us pretty close to being able to have shallow
and narrow checkouts with exchangev2/sqlite storage. Close enough that a
minimal extension should be able to provide fill in the gaps until the code
in core stabilizes and there is a user-facing way to trigger the
narrow/shallow bits from `hg clone` without also implying using of the
narrow extension...
Differential Revision: https://phab.mercurial-scm.org/D5169
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 17 Oct 2018 17:32:15 +0200] rev 40392
sqlitestore: support for storing revisions without their parents
This commit kinda/sorta implements the equivalent of ellipsis nodes for
the SQLite storage backend.
Without implementing full blown ellipsis nodes (and the necessary support for
them in the wire protocol), we instead teach the store to rewrite the p1 and
p2 nodes to nullid when the incoming parent isn't in the local store. This allows
servers to remain dumb and send the real parent and have the clients deal
with the missing parent problem.
This obviously isn't ideal because a benefit of ellipsis nodes is we can
insert a fake parent to ellide missing changesets. But neither solution is
ideal because it drops the original parent from storage. We could probably
teach the SQLite store to retain the original parent and handle missing
parents at read time. However, parent revisions are stored as integers and
it isn't trivial to store an "empty" revision in the store yet, which would
be necessary to represent the "missing" parent.
The store is somewhat intelligent in trying to remove the missing parents
metadata when the revision is re-added. But, revision numbers will be all
messed up in that case, so I'm not sure it is worth it. At some point we'll
likely want to remove the concept of revision numbers from the database and
have the store invent them at index generation time. Or even better, we can
do away with revision numbers from the file storage interface completely.
We'll get there eventually...
Differential Revision: https://phab.mercurial-scm.org/D5168
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Oct 2018 15:38:25 +0200] rev 40391
wireprotov2: support exposing linknode of file revisions
When supporting shallow file storage, clients may fetch file revisions
by changeset. But they may not readily know which changeset introduced a
specific file revision. The "linknode" is used to record which changeset
introduces which file revision.
This commit teaches the "filedata" and "filesdata" wire protocol commands
to expose the linknode for file revisions. The implementation is likely
wrong when hidden changesets are in play, since the linknode may refer to
a hidden changeset. We can deal with this problem later.
Differential Revision: https://phab.mercurial-scm.org/D5167
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Oct 2018 14:59:03 +0200] rev 40390
localrepo: support marking repos as having shallow file storage
Various operations against repositories need to know if repository
storage is full or partial. For example, a checkout (including possibly
a widening of a sparse checkout), needs to know if it can assume all file
revisions are available or whether to look for missing revisions first.
This commit lays the plumbing for doing that.
We define a repo creation option that indicates that shallow file storage
is desired.
The SQLite store uses this creation option to add an extra repo requirement
indicating file storage is shallow.
A new repository feature has been added to indicate that file storage is
shallow. The SQLite store adds this feature when the shallow file store
requirement is present.
Code can now look at repo.features to determine if repo file storage may
be shallow and take additional actions if so.
While we're here, we also teach the SQLite store to handle the narrow repo
requirement, which gets added when making narrow clones.
Differential Revision: https://phab.mercurial-scm.org/D5166
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 26 Sep 2018 14:41:15 -0700] rev 40389
repository: teach addgroup() to receive data with missing parents
The way the narrow extension works today, the server rewrites
outgoing changegroup data to lie about parents when the parents
data is missing. It adds the ellipsis flag to the revision so
it can be recorded as such in the revlog.
In the new wire protocol, such rewriting does not occur on
the server (at least not yet anyway). Instead, it is up to the
client to recognize when it has received a revision without its
parents. This means rewriting will be performed on the client.
Furthermore, the mechanism for storing a shallow revision may
differ from store to store. For example, the revlog store uses
the ellipsis flag to denote a revision's parents have been
rewritten. But a non-revlog store may wish to store things
differently. And, some stores may not even support receiving
shallow revision data!
Therefore, it makes sense for the store itself to be making
decisions about what to do when they receive revision data
without their parents.
This commit teaches the addgroup() bulk insert method to accept
a boolean argument that indicates whether the incoming data may
lack parent revisions. This flag can be set when receiving
"shallow" data from a remote.
The revlog implementation of this method has been taught to rewrite
the missing parent(s) to nullid and to add the ellipsis flag to
the revision when a missing parent is encountered. But it only
does this if ellipsis flags are enabled on the repo and the
incoming data is marked as possibly shallow. An error occurs
otherwise.
Differential Revision: https://phab.mercurial-scm.org/D5165
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Oct 2018 13:44:25 +0200] rev 40388
commands: support passing depth to hg.clone()
This will allow extensions to add --depth or other arguments to control
depth fetching.
Differential Revision: https://phab.mercurial-scm.org/D5164
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 03 Oct 2018 14:57:29 -0700] rev 40387
filelog: add a hasnode() method (API)
Missing in the file storage interface is the ability to query whether
a specified value is a known node.
This commit defines that interface member and implements it on the
revlog and sqlite file stores.
Storage unit tests have been added.
The revlog implementation is a bit more complicated because index lookups
don't consistently raise the same exception. For SQLite, we can simply look
for a key in a dict.
Differential Revision: https://phab.mercurial-scm.org/D5163
Matt Harbison <matt_harbison@yahoo.com> [Sun, 21 Oct 2018 22:26:00 -0400] rev 40386
lfs: consult the narrow matcher when extracting pointers from ctx (issue5794)
I added a testcase for lfs to all narrow tests, and the following failed:
test-narrow-acl.t
test-narrow-exchange.t
test-narrow-patterns.t
test-narrow-strip.t
test-narrow-trackedcmd.t
test-narrow-widen.t
test-narrow.t
The first two still have errors in the pretxnchangegroup on clone and (receiving
a) push, which I'm still looking into (4d63f3bc1e1a fixed something in this area
already). These two modified tests seem to cover the things that failed in the
remaining narrow tests, i.e. `hg tracked` and `hg strip`, so I didn't bother
enabling the testcases elsewhere. Maybe we should, but it's 68 tests total.
Yuya Nishihara <yuya@tcha.org> [Sat, 20 Oct 2018 20:25:56 +0900] rev 40385
statprof: fix overflow while skipping boilerplate parts
I got IndexError randomly because of stack[i] where i = len(stack).
Yuya Nishihara <yuya@tcha.org> [Sat, 20 Oct 2018 20:15:48 +0900] rev 40384
statprof: fix indent level of fp.write() (issue6004)
It was changed at 9d3034348c4f by mistake.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 19 Oct 2018 22:31:47 -0400] rev 40383
py3: stringify setupversion on Windows
This was stringified a few lines above for non Windows platforms, but `version`
remains bytes. The old code effectively undid the conversion, and triggered a
warning in setuptools when building.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 19 Oct 2018 23:47:38 -0400] rev 40382
tests: add coverage for some untested areas of hgweb
The fact that these mimetype guesses weren't blowing up anywhere on py3 prior to
9310037f0636 was the giveaway. The annotate function is a bit unusual in that
it renders the page with a 500 in the middle, so I left the HTML output. For
the other functions, checking the access log is enough.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 19 Oct 2018 23:30:56 +0300] rev 40381
statprof: update the name as the i increases (issue6003)
2864f8d3fcd6 while working on py3 fix, take out the name building out of the
loop so we were not building the new stack-name for each i, rather we were using
the first one again and again.
The test changes shows the profile is now working.
Differential Revision: https://phab.mercurial-scm.org/D5172
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 19 Oct 2018 23:18:29 +0300] rev 40380
test: show more profile lines in test-profile.t
This shows that we don't output anything after the first line and demonstrate
issue6003.
Differential Revision: https://phab.mercurial-scm.org/D5171
Augie Fackler <augie@google.com> [Fri, 19 Oct 2018 11:45:51 -0400] rev 40379
keepalive: use getattr to avoid AttributeErrors when vcr is in use
Fixes test-phabricator.t.
Differential Revision: https://phab.mercurial-scm.org/D5160
Augie Fackler <augie@google.com> [Fri, 19 Oct 2018 11:45:25 -0400] rev 40378
phabricator: do more of the VCR work in demandimport.deactivated()
If I don't do this, VCR gets confused looking for pycurl and other
libraries. I have no idea how this ever worked.
Differential Revision: https://phab.mercurial-scm.org/D5159
Augie Fackler <augie@google.com> [Fri, 19 Oct 2018 11:28:29 -0400] rev 40377
tests: sleep longer in test-logtoprocess.t
We should probably write some sort of helper that can wait N seconds
for all specified values to appear in a file or something, but for now
this will fix the FreeBSD buildbot.
Differential Revision: https://phab.mercurial-scm.org/D5157
Augie Fackler <augie@google.com> [Fri, 19 Oct 2018 11:31:18 -0400] rev 40376
tests: fix pyflakes warning in test-duplicateoptions.py
Differential Revision: https://phab.mercurial-scm.org/D5158
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Oct 2018 16:34:45 +0200] rev 40375
branchmap: avoid changelog and attribute lookups in replacecache()
This should make things faster. I'm not sure which operations would benefit
from it though. Maybe branchmap application on clone?
Differential Revision: https://phab.mercurial-scm.org/D5162
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Oct 2018 16:16:17 +0200] rev 40374
branchmap: pass changelog into branchmap functions
As part of building the branchmap, we loop over revs and call branchmap()
or _branchmap(). Previously, these functions were accessing repo.changelog.
We know from past experience that repo.changelog in loops is bad for
performance.
This commit teaches the branchmap code to pass a changelog instance into
branchmap() and _branchmap() so we don't need to pay this penalty.
On my MBP, this appears to show a speedup on a clone of the
mozilla-unified repo:
$ hg perfbranchmap --clear-revbranch
! base
! wall 21.078160 comb 21.070000 user 20.920000 sys 0.150000 (best of 3)
! wall 20.574682 comb 20.560000 user 20.400000 sys 0.160000 (best of 3)
$ hg perfbranchmap
! base
! wall 4.880413 comb 4.870000 user 4.860000 sys 0.010000 (best of 3)
! wall 4.573968 comb 4.560000 user 4.550000 sys 0.010000 (best of 3)
Differential Revision: https://phab.mercurial-scm.org/D5161
Augie Fackler <augie@google.com> [Thu, 18 Oct 2018 16:36:10 -0400] rev 40373
fuzz: move many initialization steps into LLVMFuzzerInitialize
Doing this means that things we intentionally leak (eg type objects)
no longer confuse AddressSanitizer, so now we can run the fuzzer MUCH
longer.
Differential Revision: https://phab.mercurial-scm.org/D5154
Martin von Zweigbergk <martinvonz@google.com> [Thu, 17 Nov 2016 15:51:33 -0800] rev 40372
bundle2: fix broken compression engine assertion
bundletype() is a function, so it needs to be called, and it is
documented to return a 2-tuple. This code is untested, so that's why
we haven't noticed the bad assertion.
Differential Revision: https://phab.mercurial-scm.org/D5155
Matt Harbison <matt_harbison@yahoo.com> [Thu, 18 Oct 2018 17:54:07 -0400] rev 40371
tests: glob over a difference between Windows 7 and Window 10
The error value is 11001 on Windows 10. I have no idea why it changed, but it
seems unimportant.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 18 Oct 2018 18:11:16 -0400] rev 40370
py3: fix module imports in test-highlight.t
The hash changes are because the *.py file is committed to the repo.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 17 Oct 2018 23:33:43 -0400] rev 40369
py3: fix module imports in tests, as flagged by test-check-module-imports.t
I have no idea why these aren't flagged with python2. I excluded
test-highlight.t for now to make this easier to review- the changed code is
committed to a repo, which has cascading changes on the rest of the test.
There's a mix of bytes and str in the imports dict of contrib/import-checker.py
that crashed it half way through listing out these errors. I couldn't figure
out how to fix that properly, so I was lazy and applied this on py3, to find the
rest of the errors:
diff --git a/contrib/import-checker.py b/contrib/import-checker.py
--- a/contrib/import-checker.py
+++ b/contrib/import-checker.py
@@ -626,7 +626,12 @@ def find_cycles(imports):
top.foo -> top.qux -> top.foo
"""
cycles = set()
- for mod in sorted(imports.keys()):
+ def sort(v):
+ if isinstance(v, bytes):
+ return v.decode('ascii')
+ return v
+
+ for mod in sorted(imports.keys(), key=sort):
try:
checkmod(mod, imports)
except CircularImport as e:
Matt Harbison <matt_harbison@yahoo.com> [Thu, 18 Oct 2018 21:55:47 -0400] rev 40368
lfs: don't add extension to hgrc after conversion (BC)
This is in the spirit of bcf72d7b1524.
Yuya Nishihara <yuya@tcha.org> [Thu, 18 Oct 2018 21:00:07 +0900] rev 40367
addremove: add "ui." prefix to message color keys
I don't like fully-colorized status/warning messages, and I want to disable
them at all. If we'd supported a syntax like 'color.ui.*=none', I could
easily turn addremove.added/removed off as well as ui.error. This patch is
just for that.
Since addremove colors aren't released yet, which were added at ddc1da134772,
there are no compatibility concerns.
Martin von Zweigbergk <martinvonz@google.com> [Thu, 09 Feb 2017 09:17:40 -0800] rev 40366
update: clarify update() call sites by specifying argument names
merge.update() takes a lot of parameters and I get confused all the
time which is which.
Differential Revision: https://phab.mercurial-scm.org/D5153
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Oct 2018 10:11:08 -0700] rev 40365
debugcommands: avoid stack trace from debugindexstats in pure mode
This has been broken since I added it in d71e0ba34d9b (debugcommands:
add a debugindexstats command, 2018-08-08). This patch also fixes the
test.
Differential Revision: https://phab.mercurial-scm.org/D5152
Augie Fackler <augie@google.com> [Thu, 18 Oct 2018 11:24:20 -0400] rev 40364
tests: fix up pure case of test-sqlitestore.t
This is clearly what the line should read based on the "force to zlib"
section below, so I'm guessing it just got overlooked during development.
Differential Revision: https://phab.mercurial-scm.org/D5151
Augie Fackler <augie@google.com> [Thu, 18 Oct 2018 11:14:04 -0400] rev 40363
tests: don't emit false failures when sqlite3 is missing
I'm honestly surprised we have buildbot coverage for this, but we do!
Differential Revision: https://phab.mercurial-scm.org/D5150