Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 03:50:47 -0700] rev 26628
destupdate: move the check related to the "clean" logic in the function
We want this function to exactly predict the behavior for update. Moreover, we
would like to remove all high level behavior logic out of the merge module so
this is a step forward.
Now that the 'destupdate' function both compute and validate the destination, we
can directly use it at the command level, ensuring that the 'hg update' command
never call 'merge.update' without a defined destination. This is a first (but
significant) step toward having 'merge.update' always feed with a properly
validated destination and free of high level logic.
Mads Kiilerich <madski@unity3d.com> [Mon, 12 Oct 2015 19:22:34 +0200] rev 26627
largefiles: better handling of merge of largefiles that not are available
Before, when merging revisions with missing largefiles, the missing largefiles
would be fetched as a part of the merge. If that failed (for example because
the main repository temporarily was unavailable), the largefile would be left
missing. However, the next commit would abort and (seemed to) fail when
markcommitted tried to mark the standin file as normal and thus had to hash the
largefile that didn't exist. (Actually, the commit would succeed but the
largefile update that follows right after the commit transaction would abort -
quite confusing.)
To fix that, make sure that synclfdirstate only marks files as normal if they
actually exist.
Pierre-Yves David <pierre-yves.david@fb.com> [Sun, 11 Oct 2015 22:13:03 -0700] rev 26626
patchbomb: check that targets exist at the publicurl
Advertising that the patch are available to be pulled requires that to be true.
So we check revision availability on the remote before sending any email.
Mads Kiilerich <madski@unity3d.com> [Mon, 12 Oct 2015 20:13:12 +0200] rev 26625
windows: read all global config files, not just the first (
issue4491) (BC)
On windows, hgrc.d/*.rc would not be read if mercurial.ini was found. That was
far from obvious from the documentation and different from the behavior on
posix systems.
As a consequence of this, TortoiseHg cacert configuration placed in hgrc.d
would not be read if an old global mercurial.ini still existed.
"hg config -g" could also crash when no global configuration files could be
found.
Instead, make windows behave like posix and read all global configuration
files.
The documentation was in a way right that individual config settings in the
global Mercurial.ini would override settings from for example .hgrc.d\*.rc, but
only because the .d files not would be read at all if a Mercurial.ini was
found. The ordering in the documentation is thus changed to match the code.
Ryan McElroy <rmcelroy@fb.com> [Fri, 09 Oct 2015 14:48:59 -0700] rev 26624
strip: factor out revset calculation for strip -B
This will allow reusing it in evolve and overriding it in other extensions.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 09 Oct 2015 11:22:01 -0700] rev 26623
clonebundles: support for seeding clones from pre-generated bundles
Cloning can be an expensive operation for servers because the server
generates a bundle from existing repository data at request time. For
a large repository like mozilla-central, this consumes 4+ minutes
of CPU time on the server. It also results in significant network
utilization. Multiplied by hundreds or even thousands of clients and
the ensuing load can result in difficulties scaling the Mercurial server.
Despite generation of bundles being deterministic until the next
changeset is added, the generation of bundles to service a clone request
is not cached. Each clone thus performs redundant work. This is
wasteful.
This patch introduces the "clonebundles" extension and related
client-side functionality to help alleviate this deficiency. The
client-side feature is behind an experimental flag and is not enabled by
default.
It works as follows:
1) Server operator generates a bundle and makes it available on a
server (likely HTTP).
2) Server operator defines the URL of a bundle file in a
.hg/clonebundles.manifest file.
3) Client `hg clone`ing sees the server is advertising bundle URLs.
4) Client fetches and applies the advertised bundle.
5) Client performs equivalent of `hg pull` to fetch changes made since
the bundle was created.
Essentially, the server performs the expensive work of generating a
bundle once and all subsequent clones fetch a static file from
somewhere. Scaling static file serving is a much more manageable
problem than scaling a Python application like Mercurial. Assuming your
repository grows less than 1% per day, the end result is 99+% of CPU
and network load from clones is eliminated, allowing Mercurial servers
to scale more easily. Serving static files also means data can be
transferred to clients as fast as they can consume it, rather than as
fast as servers can generate it. This makes clones faster.
Mozilla has implemented similar functionality of this patch on
hg.mozilla.org using a custom extension. We are hosting bundle files in
Amazon S3 and CloudFront (a CDN) and have successfully offloaded
>1 TB/day in data transfer from hg.mozilla.org, freeing up significant
bandwidth and CPU resources. The positive impact has been stellar and
I believe it has proved its value to be included in Mercurial core. I
feel it is important for the client-side support to be enabled in core
by default because it means that clients will get faster, more reliable
clones and will enable server operators to reduce load without
requiring any client-side configuration changes (assuming clients are
up to date, of course).
The scope of this feature is narrowly and specifically tailored to
cloning, despite "serve pulls from pre-generated bundles" being a valid
and useful feature. I would eventually like for Mercurial servers to
support transferring *all* repository data via statically hosted files.
You could imagine a server that siphons all pushed data to bundle files
and instructs clients to apply a stream of bundles to reconstruct all
repository data. This feature, while useful and powerful, is
significantly more work to implement because it requires the server
component have awareness of discovery and a mapping of which changesets
are in which files. Full, clone bundles, by contrast, are much simpler.
The wire protocol command is named "clonebundles" instead of something
more generic like "staticbundles" to leave the door open for a new, more
powerful and more generic server-side component with minimal backwards
compatibility implications. The name "bundleclone" is used by Mozilla's
extension and would cause problems since there are subtle differences
in Mozilla's extension.
Mozilla's experience with this idea has taught us that some form of
"content negotiation" is required. Not all clients will support all
bundle formats or even URLs (advanced TLS requirements, etc). To ensure
the highest uptake possible, a server needs to advertise multiple
versions of bundles and clients need to be able to choose the most
appropriate from that list one. The "attributes" in each
server-advertised entry facilitate this filtering and sorting. Their
use will become apparent in subsequent patches.
Initial inspiration and credit for the idea of cloning from static files
belongs to Augie Fackler and his "lookaside clone" extension proof of
concept.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 29 Sep 2015 16:17:32 -0700] rev 26622
sslutil: expose attribute indicating whether SNI is supported
This will be used so clone bundles can advertise whether URLs require
SNI. This will be explained more in a subsequent patch.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 23:58:07 -0700] rev 26621
resolve: perform all premerges before performing any file merges (BC)
Just like the BC to merge before it, this allows for a maximally consistent
state before providing any prompts to the user.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 23:56:44 -0700] rev 26620
test-resolve.t: add some tests for .orig file contents
An upcoming patch will touch some code around this area, and I couldn't find
any tests related to this.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 23:54:40 -0700] rev 26619
test-resolve.t: add some output to show order of operations
This basically shows the behavior of resolve with multiple files. An upcoming
behavior change will cause this output to also change.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 21:56:39 -0700] rev 26618
merge.mergestate: perform all premerges before any merges (BC)
We perform all that we can non-interactively before prompting the user for input
via their merge tool. This allows for a maximally consistent state when the user
is first prompted.
The test output changes indicate the actual behavior change happening.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:12:12 -0700] rev 26617
merge: introduce a preresolve function
The section of code that writes out the version of the file cached in the merge
state should only be run at preresolve time. This is so that if the premerge
keeps around conflict markers, those don't get overwritten before the main
merge.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 18:37:54 -0700] rev 26616
merge.mergestate._resolve: also return completed status
We'll need this for a new 'preresolve' function we're adding.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 18:29:50 -0700] rev 26615
merge.mergestate: add a wrapper around resolve
The resolve function will be broken up into separate pre-resolve and resolve
steps.
Siddharth Agarwal <sid0@fb.com> [Fri, 09 Oct 2015 13:54:52 -0700] rev 26614
simplemerge: move conflict warning message to filemerge
The current output for a failed merge with conflict markers looks something like:
merging foo
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
merging bar
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
We're going to change the way merges are done to perform all premerges before
all merges, so that the output above would look like:
merging foo
merging bar
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
The 'warning: conflicts during merge' line has no context, so is pretty
confusing.
This patch will change the future output to:
merging foo
merging bar
warning: conflicts while merging foo! (edit, then use 'hg resolve --mark')
warning: conflicts while merging bar! (edit, then use 'hg resolve --mark')
The hint on how to resolve the conflicts makes this a bit unwieldy, but solving
that is tricky because we already hint that people run 'hg resolve' to retry
unresolved merges. The 'hg resolve --mark' mostly applies to conflict marker
based resolution.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 15:04:00 -0700] rev 26613
filemerge: clean up some dead code
We now exit early if we do a premerge, so extra checks are no longer necessary.
Augie Fackler <augie@google.com> [Mon, 12 Oct 2015 14:15:04 -0400] rev 26612
run-tests: add b-prefix on two strings to fix python3 support
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:47:14 -0700] rev 26611
filemerge: break overall filemerge into separate premerge and merge steps
This means that in ms.resolve we must call merge after calling premerge. This
doesn't yet mean that all premerges happen before any merges -- however, this
does get us closer to our goal.
The output differences are because we recompute the merge tool. The only
user-visible difference caused by this patch is that if the tool is missing
we'll print the warning twice. Not a huge deal, though.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:04:40 -0700] rev 26610
filemerge: only copy to backup during premerge step
The premerge might leave the original file in an unclean state. Therefore it's
important to only copy the file in the beginning.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:02:53 -0700] rev 26609
filemerge: only print out "merging f" output at premerge step
We're soon going to call this function twice, once for premerge and once for
merge. This makes sure the "merging" output only gets printed during the
premerge step.
Siddharth Agarwal <sid0@fb.com> [Thu, 08 Oct 2015 00:19:20 -0700] rev 26608
filemerge: deindent the parts of filemerge outside the try block
It is no longer necessary to indent these parts.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:47:04 -0700] rev 26607
filemerge: introduce a premerge flag and function
This flag will let us get to our overall goal of performing all premerges
before any merges.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 12:56:21 -0700] rev 26606
filemerge: also return whether the merge is complete
In future patches, we'll pause merges after the premerge step. After the
premerge step we'll return complete = False.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 12:31:08 -0700] rev 26605
filemerge: add a wrapper around the filemerge function
We'll introduce a separate premerge function that calls the same code.
Mads Kiilerich <madski@unity3d.com> [Fri, 09 Oct 2015 01:19:37 +0200] rev 26604
context: don't hex encode all unknown 20 char revision specs (
issue4890)
d3908c911d5e introduced nice hexified display of missing nodes. It did however
also make missing 20 character revision specifications be shown as hex - very
confusing.
Users are often wrong and somehow specify revisions that don't exist. Nodes
will however rarely be missing ... and they will only look like a user provided
revision specification and be all ascii in 1 of 4*10**9.
With this change, missing revisions will only be hexified if they really look
like binary nodes. This change will thus improve the error reporting UI in the
common case and only very rarely make it confusing in the opposite direction of
how it was before.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 12 Oct 2015 00:45:24 -0700] rev 26603
discovery: put trivial branch first
Having the simple and tiny branch of the conditional first help readability. The
"else" that appears after a screen of code is harder to relate to a conditional.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 09 Oct 2015 15:31:50 -0700] rev 26602
shelve: rename 'publicancestors' to something accurate (
issue4737)
That function is actually not returning public ancestors at all. This is
pointed by the second line of the docstring...
The bundling behavior was made correct in
a5141977198d but with confusion
remaining regarding what each function was doing.
This close
issue4737, because this highlight that shelve is actually -not-
bundling too much data (this was actually properly tested).
Nathan Goldbaum <ngoldbau@ucsc.edu> [Fri, 09 Oct 2015 12:30:46 -0500] rev 26601
makefile: add wheel build target