Jun Wu <quark@fb.com> [Sat, 22 Apr 2017 16:50:08 -0700] rev 32166
worker: rewrite error handling so os._exit covers all cases
Previously the worker error handling is like:
pid = os.fork() --+
if pid == 0: |
.... | problematic
.... --+
try: --+
.... | worker error handling
--+
If a signal arrives when Python is executing the "problematic" lines, an
external error handling (dispatch.py) will take over the control flow and
it's no longer guaranteed "os._exit" is called (see 86cd09bc13ba for why it
is necessary).
This patch rewrites the error handling so it covers all possible code paths
for a worker even during fork.
Note: "os.getpid() == parentpid" is used to test if the process is parent or
not intentionally, instead of checking "pid", because "pid = os.fork()" may
be not atomic - it's possible that that a signal hits the worker before the
assignment completes [1]. The newly added test replaces "os.fork" to
exercise that extreme case.
[1]: CPython compiles "pid = os.fork()" to 2 byte codes: "CALL_FUNCTION" and
"STORE_FAST", so it's probably not atomic:
def f():
pid = os.fork()
dis.dis(f)
2 0 LOAD_GLOBAL 0 (os)
3 LOAD_ATTR 1 (fork)
6 CALL_FUNCTION 0
9 STORE_FAST 0 (pid)
12 LOAD_CONST 0 (None)
15 RETURN_VALUE
Jun Wu <quark@fb.com> [Sat, 22 Apr 2017 15:00:17 -0700] rev 32165
dispatch: take over SignalInterrupt handling from scmutil
dispatch handles KeyboardInterrupt already. This makes the code more
consistent, and makes worker not print "killed!" if it receives SIGTERM in
most cases (in rare cases there is still "killed!" printed, which will be
fixed by the next patch).
Kevin Bullock <kbullock+mercurial@ringworld.org> [Tue, 02 May 2017 17:29:01 -0500] rev 32164
merge stable into default
Augie Fackler <raf@durin42.com> [Wed, 05 Jul 2017 11:24:22 -0400] rev 32163
Added signature for changeset 26c49ed51a69
Augie Fackler <raf@durin42.com> [Wed, 05 Jul 2017 11:24:17 -0400] rev 32162
Added tag 4.2.2 for changeset 26c49ed51a69
Matt Harbison <matt_harbison@yahoo.com> [Tue, 04 Jul 2017 22:58:21 -0400] rev 32161
help: cleanup grammar in the hooks section
Augie Fackler <augie@google.com> [Tue, 04 Jul 2017 10:18:44 -0400] rev 32160
merge with i18n
Augie Fackler <augie@google.com> [Tue, 04 Jul 2017 10:17:14 -0400] rev 32159
revlog: backed out changeset e9d325cfe071
We'll move this to the default branch.
Wagner Bruna <wbruna@softwareexpress.com.br> [Thu, 29 Jun 2017 12:07:54 -0300] rev 32158
i18n-pt_BR: synchronized with efebc9f52ecb
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 23 Jun 2017 13:49:34 +0200] rev 32157
revlog: add an experimental option to mitigated delta issues (issue5480)
The general delta heuristic to select a delta do not scale with the number of
branch. The delta base is frequently too far away to be able to reuse a chain
according to the "distance" criteria. This leads to insertion of larger delta (or
even full text) that themselves push the bases for the next delta further away
leading to more large deltas and full texts. This full text and frequent
recomputation throw Mercurial performance in disarray.
For example of a slightly large repository
280 000 files (2 150 000 versions)
430 000 changesets (10 000 topological heads)
Number below compares repository with and without the distance criteria:
manifest size:
with: 21.4 GB
without: 0.3 GB
store size:
with: 28.7 GB
without 7.4 GB
bundle last 15 00 revisions:
with: 800 seconds
971 MB
without: 50 seconds
73 MB
unbundle time (of the last 15K revisions):
with: 1150 seconds (~19 minutes)
without: 35 seconds
Similar issues has been observed in other repositories.
Adding a new option or "feature" on stable is uncommon. However, given that this
issues is making Mercurial practically unusable, I'm exceptionally targeting
this patch for stable.
What is actually needed is a full rework of the delta building and reading
logic. However, that will be a longer process and churn not suitable for stable.
In the meantime, we introduces a quick and dirty mitigation of this in the
'experimental' config space. The new option introduces a way to set the maximum
amount of memory usable to store a diff in memory. This extend the ability for
Mercurial to create chains without removing all safe guard regarding memory
access. The option should be phased out when core has a more proper solution
available.
Setting the limit to '0' remove all limits, setting it to '-1' use the default
limit (textsize x 4).
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 27 Jun 2017 18:13:10 +0200] rev 32156
rebase: also test abort from pretxnclose error
Different hooks will have different properties so we cover more hooks to catch
further regressions.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 27 Jun 2017 18:10:55 +0200] rev 32155
rebase: reinforce testing around precommit hook interrupting a rebase
Different hooks will have different properties so we cover more hooks to catch
further regression.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 27 Jun 2017 17:45:58 +0200] rev 32154
rebase: provides test case for (issue5610)
The 4.2 release introduces a regression regarding the behavior of rebase with
some hook failures. We add the tests from the bug report from Henrik Stuart to
our test base to prevent further regression on this.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 27 Jun 2017 17:40:24 +0200] rev 32153
rebase: backed out changeset cf8ad0e6c0e4 (issue5610)
Having a single transaction for rebase means the whole transaction gets rolled back
on error. To work around this a small hack has been added to detect merge
conflict and commit the work done so far before exiting. This hack works because
there is nothing transaction related going on during the merge phase.
However, if a hook blocks the rebase to create a changeset, it is too late to commit the
work done in the transaction before the problematic changeset was created. This
leads to the whole rebase so far being rolled back. Losing merge resolution and
other work in the process. (note: rebase state will be fully lost too).
Since issue5610 is a pretty serious regression and the next stable release is a
couple day away, we are taking the backout route until we can figure out
something better to do.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 27 Jun 2017 17:39:55 +0200] rev 32152
rebase: backed out changeset 2519994d25ca
In the process of fixing issue5610 in 4.2.2, we are trying to backout
cf8ad0e6c0e4. This changeset is making changes that depend on cf8ad0e6c0e4,
so we need to back it out first.
Since issue5610 is pretty serious regression and the next stable release is a
couple of days away, we are taking the backout route until we can figure out
something better to do.
Anton Shestakov <av6@dwimlabs.net> [Fri, 23 Jun 2017 13:33:41 +0800] rev 32151
make: add Debian 9 (Stretch) docker target
Anton Shestakov <av6@dwimlabs.net> [Fri, 23 Jun 2017 13:08:46 +0800] rev 32150
make: templatize Debian build target a la e63dfbbdbd07
Anton Shestakov <av6@dwimlabs.net> [Fri, 23 Jun 2017 12:04:12 +0800] rev 32149
make: add Ubuntu Zesty docker targets (.deb and ppa)
Zesty Zapus was released on 2017-04-13 and will be supported until 2018-01.
Anton Shestakov <av6@dwimlabs.net> [Fri, 23 Jun 2017 10:05:01 +0800] rev 32148
docker: install less as a build-time dependency in deb-based distros
It's needed since 387a76cac28e, otherwise dpkg-checkbuilddeps errors out.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 23 Jun 2017 13:22:04 +0200] rev 32147
eol: fix 'error' parameter name in the commitctx wrapper
Since its introduction in 9dfee83c93c8, the parameter has always been name
"error". Yet the eol extension have been using 'haserror' as the argument name,
breaking extensions with subclass passing 'error' as a keyword argument.
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 23 Jun 2017 13:24:45 +0200] rev 32146
eol: import 'error' as 'errormod'
We need the 'error' name available to fix another bug, so we rename the imported
module.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 13 Jun 2017 22:24:41 -0400] rev 32145
largefiles: avoid a crash when archiving a subrepo with largefiles disabled
This path is also used for extdiff, which is how I crossed paths with it.
Without this, an AttributeError occurs looking for 'lfstatus' on
localrepository. See also d414c28db84d.
The other archive method is for the archival.py override, so it doesn't need to
be special cased like this. (It looks like it is only called for the top level
repo.) Likewise, the transplant override is also for commands.py. The other
overrides set lfstatus before examining it.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 12 Jun 2017 11:24:21 -0700] rev 32144
help: correct description of "glob:foo/*" matching
Unlike what the description says, it does not match recursively. Also
add an example of "glob:foo/**" (which does match recursively).
Kevin Bullock <kbullock@ringworld.org> [Sun, 04 Jun 2017 08:16:37 -0500] rev 32143
Added signature for changeset c850f0ed54c1