Siddharth Agarwal <sid0@fb.com> [Fri, 09 Oct 2015 13:54:52 -0700] rev 26614
simplemerge: move conflict warning message to filemerge
The current output for a failed merge with conflict markers looks something like:
merging foo
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
merging bar
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
We're going to change the way merges are done to perform all premerges before
all merges, so that the output above would look like:
merging foo
merging bar
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
The 'warning: conflicts during merge' line has no context, so is pretty
confusing.
This patch will change the future output to:
merging foo
merging bar
warning: conflicts while merging foo! (edit, then use 'hg resolve --mark')
warning: conflicts while merging bar! (edit, then use 'hg resolve --mark')
The hint on how to resolve the conflicts makes this a bit unwieldy, but solving
that is tricky because we already hint that people run 'hg resolve' to retry
unresolved merges. The 'hg resolve --mark' mostly applies to conflict marker
based resolution.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 15:04:00 -0700] rev 26613
filemerge: clean up some dead code
We now exit early if we do a premerge, so extra checks are no longer necessary.
Augie Fackler <augie@google.com> [Mon, 12 Oct 2015 14:15:04 -0400] rev 26612
run-tests: add b-prefix on two strings to fix python3 support
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:47:14 -0700] rev 26611
filemerge: break overall filemerge into separate premerge and merge steps
This means that in ms.resolve we must call merge after calling premerge. This
doesn't yet mean that all premerges happen before any merges -- however, this
does get us closer to our goal.
The output differences are because we recompute the merge tool. The only
user-visible difference caused by this patch is that if the tool is missing
we'll print the warning twice. Not a huge deal, though.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:04:40 -0700] rev 26610
filemerge: only copy to backup during premerge step
The premerge might leave the original file in an unclean state. Therefore it's
important to only copy the file in the beginning.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:02:53 -0700] rev 26609
filemerge: only print out "merging f" output at premerge step
We're soon going to call this function twice, once for premerge and once for
merge. This makes sure the "merging" output only gets printed during the
premerge step.
Siddharth Agarwal <sid0@fb.com> [Thu, 08 Oct 2015 00:19:20 -0700] rev 26608
filemerge: deindent the parts of filemerge outside the try block
It is no longer necessary to indent these parts.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 20:47:04 -0700] rev 26607
filemerge: introduce a premerge flag and function
This flag will let us get to our overall goal of performing all premerges
before any merges.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 12:56:21 -0700] rev 26606
filemerge: also return whether the merge is complete
In future patches, we'll pause merges after the premerge step. After the
premerge step we'll return complete = False.
Siddharth Agarwal <sid0@fb.com> [Sun, 11 Oct 2015 12:31:08 -0700] rev 26605
filemerge: add a wrapper around the filemerge function
We'll introduce a separate premerge function that calls the same code.
Mads Kiilerich <madski@unity3d.com> [Fri, 09 Oct 2015 01:19:37 +0200] rev 26604
context: don't hex encode all unknown 20 char revision specs (issue4890)
d3908c911d5e introduced nice hexified display of missing nodes. It did however
also make missing 20 character revision specifications be shown as hex - very
confusing.
Users are often wrong and somehow specify revisions that don't exist. Nodes
will however rarely be missing ... and they will only look like a user provided
revision specification and be all ascii in 1 of 4*10**9.
With this change, missing revisions will only be hexified if they really look
like binary nodes. This change will thus improve the error reporting UI in the
common case and only very rarely make it confusing in the opposite direction of
how it was before.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 12 Oct 2015 00:45:24 -0700] rev 26603
discovery: put trivial branch first
Having the simple and tiny branch of the conditional first help readability. The
"else" that appears after a screen of code is harder to relate to a conditional.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 09 Oct 2015 15:31:50 -0700] rev 26602
shelve: rename 'publicancestors' to something accurate (issue4737)
That function is actually not returning public ancestors at all. This is
pointed by the second line of the docstring...
The bundling behavior was made correct in a5141977198d but with confusion
remaining regarding what each function was doing.
This close issue4737, because this highlight that shelve is actually -not-
bundling too much data (this was actually properly tested).
Nathan Goldbaum <ngoldbau@ucsc.edu> [Fri, 09 Oct 2015 12:30:46 -0500] rev 26601
makefile: add wheel build target
Nathan Goldbaum <ngoldbau@ucsc.edu> [Fri, 09 Oct 2015 12:25:51 -0500] rev 26600
setup: import setup from setuptools if FORCE_SETUPTOOLS is set
This should allow easier experimentation with using setuptools in mercurial's
build automation, without breaking anything that currently depends on distutils
behavior
Gijs Kruitbosch <gijskruitbosch@gmail.com> [Mon, 12 Oct 2015 14:46:51 +0100] rev 26599
hgweb: remove obsolete -webkit-border-radius property
Anton Shestakov <av6@dwimlabs.net> [Mon, 12 Oct 2015 15:20:04 +0800] rev 26598
monoblue: add a link to the latest file revision
For reference, this was added to paper/coal in bb00a159e594 and to gitweb in
b3b57ecbda50.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 09 Oct 2015 15:44:00 -0700] rev 26597
discovery: reference relevant bug in the faulty code
We extend the comment about this code flaw with more code flaw.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 09 Oct 2015 15:37:05 -0700] rev 26596
discovery: fix a typo in a comment
The idea here is that the code is imperfect, not that it is impossible to get
something behaving properly.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 09 Oct 2015 14:59:37 -0700] rev 26595
getsubset: get the unpacker version from the bundler
The current setup requires to pass both a packer and, optionally, the version
of the unpacker. This is confusing and error prone as the two value cannot
mismatch. Instead, we simply grab the version from the packer. This fixes a bug
where requesting a cg2 from 'hg bundle' were reported as changegroup 1.
I should have caught that in the initial changeset but I missed it somehow.
Emanuele Giaquinta <emanuele.giaquinta@gmail.com> [Mon, 12 Oct 2015 15:42:32 +0300] rev 26594
test-convert-cvs: add sleep so cvs notices changes
This change makes the test pass on gcc112.fsffrance.org.
Emanuele Giaquinta <emanuele.giaquinta@gmail.com> [Wed, 07 Oct 2015 11:33:52 +0300] rev 26593
cvsps: fix computation of parent revisions when log caching is on
cvsps computes the parent revisions of log entries by walking the cvs log
sorted by (rcs, revision) and by iteratively maintaining a 'versions'
dictionary which maps a (rcs, branch) pair onto the last revision seen for that
pair. When log caching is on and a log cache exists, cvsps fails to set the
parent revisions of new log entries because it does not iterate over the log
cache in the parents computation. A complication is that a file rcs can change
(move to/from the attic), with respect to its value in the log cache, if the
file is removed/added back. This patch adds an iteration over the log cache to
update the rcs of cached log entries, if changed, and to properly populate the
'versions' dictionary.
Matt Mackall <mpm@selenic.com> [Tue, 06 Oct 2015 16:26:20 -0500] rev 26592
dirstate: batch calls to statfiles (issue4878)
This makes it more interruptible.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Oct 2015 18:30:47 +0900] rev 26591
parsers: fix infinite loop or out-of-bound read in fm1readmarkers (issue4888)
The issue4888 was caused by 0-length obsolete marker. If msize is zero,
fm1readmarkers() never ends.
This patch adds several bound checks to fm1readmarker(). Therefore, 0-length
and invalid-size marker should be rejected.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Oct 2015 18:41:41 +0900] rev 26590
parsers: read sizes of metadata pair of obsolete marker at once
This will make it easy to implement bound checking. Currently fm1readmarker()
has no protection for corrupted obsstore and can cause infinite loop or
out-of-bound reads.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 21:51:24 -0700] rev 26589
filemerge: clean up temp files in a finally block
This isn't really a big deal because the temp files are created in $TMPDIR, but
it makes some upcoming work simpler.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 08 Oct 2015 12:53:09 -0700] rev 26588
check-code: detect and ban 'util.Abort'
We have seen the light, please use the new way.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 08 Oct 2015 12:55:45 -0700] rev 26587
error: get Abort from 'error' instead of 'util'
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 22:49:24 -0700] rev 26586
eol: rename 'error' to 'haserror'
The variable 'error' conflict with the module name that we would like to import
and use in a coming changeset.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 22:29:57 -0700] rev 26585
discovery: rename 'error' to 'errormsg'
The variable 'error' conflict with the module name that we would like to import
and use in a coming changeset.
Christian Delahousse <cdelahousse@fb.com> [Mon, 05 Oct 2015 16:44:45 -0700] rev 26584
histedit: delete histedit statefile on any exception during abort
When an user aborts a histedit, many things could go wrong. At a minimum, after
a histedit abort failure, their repository should be out of that state. We've
found situations where the user could not exit the histedit state without
manually deleting the histedit state file. This patch ensures that if any
exception happens during an abort, the histedit statefile will be deleted so
that users are out of the histedit state and can at least manually get the repo
back to a workable condition.
Christian Delahousse <cdelahousse@fb.com> [Tue, 06 Oct 2015 15:09:28 -0700] rev 26583
histedit: check presence of statefile before deleting it
When the histeditstate class instance has it's clear() method called, there is
nothing to check to see if the state file exists before deleting it. It may not
exist, which would create an exception. This patch allows clear to be called at
any time.
This will be needed for the following patch.
Christian Delahousse <cdelahousse@fb.com> [Mon, 05 Oct 2015 16:34:17 -0700] rev 26582
histedit: add inprogress method to state class
If a histedit is progress, the 'histedit-state' file should exist. The patch
implements a convenience function to do check if a histedit is in progress.
This method will be use in next patch in the series.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Fri, 09 Oct 2015 03:53:47 +0900] rev 26581
commands: use dirstateguard instead of begin/end-parentchange for backout
Before this patch, "hg backout" uses 'begin'/'end'-'parentchange()'
of 'dirstate' class to avoid writing incomplete dirstate changes out
at failure.
But this framework doesn't work as expected, if 'dirstate.write()' is
invoked between them. In fact, in-memory dirstate changes may be
written out at 'repo.status()' implied by 'merge.update()', even
before this patch.
To restore dirstate as expected at failure of "hg backout", this patch
uses 'dirstateguard' instead of 'begin'/'end'-'parentchange()'.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Fri, 09 Oct 2015 03:53:47 +0900] rev 26580
commands: make "hg import" use dirstateguard only for --no-commit
Previous patch made dirstate changes in a transaction scope "all or
nothing". Therefore, 'dirstateguard' is meaningless, if its scope is
as same as one of the related transaction.
Before this patch, "hg import" uses 'dirstateguard' always, but
transaction is also started if '--no-commit' isn't specified.
To avoid redundancy, this patch makes "hg import" use dirstateguard
only if transaction isn't started (= '--no-commit' is specified).
In this patch, 'if dsguard' can be examined safely, because 'dsguard'
is initialized (with None) before outermost 'try'.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Fri, 09 Oct 2015 03:53:46 +0900] rev 26579
cmdutil: stop tryimportone from using dirstateguard (BC)
There is no user of 'cmdutil.tryimportone()' other than
'commands.import_()', which can restore dirstate at failure of
applying patches by transaction or dirstateguard.
Therefore, it is reasonable to stop 'tryimportone()' from using
redundant 'dirstateguard', even though it changes behavior of
'tryimportone()'.
After this patch, 3rd party extensions should use 'dirstateguard' or
so explicitly, if they want to restore dirstate at failure of
importing a patch.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Fri, 09 Oct 2015 03:53:46 +0900] rev 26578
dirstate: remove meaningless dirstateguard
Previous patch made dirstate changes in a transaction scope "all or
nothing". Therefore, 'dirstateguard' is meaningless, if its scope is
as same as one of the related transaction.
This patch removes such meaningless 'dirstateguard' usage.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Fri, 09 Oct 2015 03:53:46 +0900] rev 26577
localrepo: execute appropriate actions for dirstate at releasing transaction
Before this patch, in-memory dirstate changes are still kept over a
transaction scope boundary regardless of the result of it.
For "all or nothing" policy of the transaction, in-memory dirstate
changes should be:
- written out at successful closing a transaction, because
subsequent 'dirstate.invalidate()' can lose them
- discarded at failure of a transaction, because outer
'wlock.release()' or so may write them out
To discard all changes in a transaction completely, this patch also
restores '.hg/dirstate' by '.hg/journal.dirstate' at failure, because
'transaction' itself does nothing for files related to '.hg/journal.*'
in such case (therefore, renaming in this patch is safe enough).
This is a part of preparations for "transactional dirstate". See also
the wiki page below for detail about it.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
This patch also removes redundant 'dirstate.invalidate()' just before
aborting a transaction for shelve/unshelve.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Fri, 09 Oct 2015 03:53:46 +0900] rev 26576
transaction: add releasefn to notify the end of a transaction scope
'releasefn' is used by subsequent patch, to do appropriate action
according to the result of it at the end of a transaction scope.
To ensure that 'releasefn' is invoked only once, this patch invokes it
after assignment 'self.journal = None', because such assignment
prevents from invoked 'transaction._abort()' again via '__del__()'.
def __del__(self):
if self.journal:
self._abort()
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 23:35:30 -0700] rev 26575
filemerge: move post-merge checks into a separate function
This makes the overall filemerge function easier to follow, and makes upcoming
work simpler.
Siddharth Agarwal <sid0@fb.com> [Thu, 08 Oct 2015 14:18:43 -0700] rev 26574
filemerge._xmerge: drop no longer necessary 'if r:' check
Cleanup from an earlier patch to make premerge be directly called from the main
filemerge function.
Siddharth Agarwal <sid0@fb.com> [Thu, 08 Oct 2015 14:17:31 -0700] rev 26573
filemerge._idump: drop no longer necessary 'if r:' check
Cleanup from an earlier patch to make premerge be directly called from the main
filemerge function.
Siddharth Agarwal <sid0@fb.com> [Thu, 08 Oct 2015 14:16:19 -0700] rev 26572
filemerge._merge: drop no longer necessary 'if r:' check
Cleanup from an earlier patch to make premerge be directly called from the main
filemerge function.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 02:33:45 -0700] rev 26571
revset: delete _updatedefaultdest as it has no users
The revset is not used anywhere anymore. We delete the function until we use
(and therefore test it again).
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 28 Sep 2015 22:11:23 -0700] rev 26570
merge: get the default update destination from the function
There is no value in using the revset instead of the extracted function.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 01:46:47 -0700] rev 26569
update: move default destination computation to a function
We ultimately want this to be accessible through a revset, but there is too
much complexity here for that to work. Especially we'll have to return more
than just the destination to control the behavior (eg: bookmarks to activate,
etc).
To prevent cycle, a new module is created, it will receive other
destination/behavior function in the future.
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 08 Oct 2015 10:57:03 -0700] rev 26568
worker: restore old countcpus code (issue4869)
This is a backout of d29859cfcfc2. The stdlib implementation of
multiprocessing.cpu_count() attempts to invoke a process on BSD and
Darwin platforms (at least on 2.7). Under certain conditions (such as
cwd being removed) this could raise. Our old code was silently catching
the exception.
The old code was more robust, so restore it.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 21:22:16 -0700] rev 26567
filemerge: call premerge directly from main merge function
The merge code currently does (in pseudocode):
for f in tomerge:
premerge f
merge f
We'd like to change this to look more like:
for f in tomerge:
premerge f
for f in tomerge:
merge f
This makes sure as many files are resolved as possible before prompting for the
others. This restructuring is also necessary for custom merge drivers.
This function separates out the premerge step from the merge step. In future
patches we'll actually turn these into separate steps in the merge driver.
The 'if r:' occurrences will be cleaned up in subsequent patches.
Durham Goode <durham@fb.com> [Mon, 05 Oct 2015 16:19:54 -0700] rev 26566
bundle2: allow lazily acquiring the lock
In the external pushrebase extension, it is valuable to be able to do some work
without taking the lock (like running expensive hooks). This enables
significantly higher commit throughput.
This patch adds an option to lazily acquire the lock. It means that all bundle2
part handlers that require writing to the repo must first call
op.gettransction(), when in this mode.
Durham Goode <durham@fb.com> [Tue, 06 Oct 2015 14:42:29 -0700] rev 26565
bundle2: add op.gettransaction() to handlers that need the lock
A future patch will allow the bundle2 lock be taken lazily. We need to
introduce transaction-gets to each handler that needs the lock.
The tests caught these issues when I added lazy locking.
Matt Mackall <mpm@selenic.com> [Thu, 08 Oct 2015 17:44:22 -0500] rev 26564
merge with stable
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 23:13:57 -0700] rev 26563
patchbomb: add a 'bundletype' config under 'patchbomb'
patchbomb relies on the 'hg bundle' command to generate an attached bundle using
--bundle. However, while 'hg bundle' has a --type option, patchbomb did not.
This is becoming very relevant since we are about to issue bundle2 for
general-delta repository.
This was tracked as issue4863
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 07 Oct 2015 13:05:25 -0700] rev 26562
import: allow processing of extra part header after import
As we have a way for extension to add more header, we need a way for them to
actually process them. We add a basic hook point to do extra work after the
import have been committed.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 09:51:24 -0700] rev 26561
import: allow processing of extra part header during import
As we have a way for extension to add more header, we need a way for them to
actually process them. We add a basic hook points to alter the changeset
(especially extra) before we commit. There would be more to do for a full
featured hooking, but this currently fit my needs.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:23:21 -0700] rev 26560
extract: parse 'nodeid' using the generic mechanism
Parsing 'nodeid' is very simple and a good first test.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:22:23 -0700] rev 26559
extract: parse 'branch' using the generic mechanism
Parsing 'branch' is very simple and a good first test.
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 07 Oct 2015 01:13:36 -0700] rev 26558
extract: parse 'date' using the generic mechanism
Parsing 'date' is very simple and a good first test.
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 07 Oct 2015 01:20:49 -0700] rev 26557
extract: add some facility for extensible header parsing
We need a way for extension to extend the header we can parse. We start with a
very simple mechanism that will be used in the next changeset.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:16:24 -0700] rev 26556
extract: remove the 'user' variable
It is not heavily used enough to justify being something other than a
dictionary entry.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:11:09 -0700] rev 26555
extract: use a single return
The differences between both returns are now very thin, we factor out
that part.
Pierre-Yves David <pierre-yves.david@fb.com> [Wed, 07 Oct 2015 00:50:53 -0700] rev 26554
extract: move 'nodeid' assignment where it is parsed
The is one setter and no consumer, we can move it there directly.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:08:32 -0700] rev 26553
extract: move 'date' assignment where it is parsed
There is one setter and no consumer, we can move it there directly.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:07:33 -0700] rev 26552
extract: assign user to 'data' earlier
This is used in both case, we can assign it once.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:06:36 -0700] rev 26551
extract: move 'branch' assignment where it is parsed
There is one setter and no consumer, we can move it there directly.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:05:56 -0700] rev 26550
extract: directly assign parent to data dictionary
The temporary variables are not adding anything.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:04:46 -0700] rev 26549
extract: assign message only once
This return is shared between the two cases. We can move it earlier.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:04:06 -0700] rev 26548
extract: simplify parents assignement
We move to the more standard style of:
value = default
if special:
value = other
This is more consistent and compact.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 02:01:53 -0700] rev 26547
patch: move 'extract' return to a dictionnary
The final goal here is to be able to parse, return and process arbitrary data
from patch. This mirror the recently added ability to add arbitrary data to
patch headers.
The first step is to return something more flexible, so we return a dict without
changing any other logic.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 06 Oct 2015 01:49:04 -0700] rev 26546
patchbomb: add experimental config of a "pullurl" and export it
This config allows to specify a public location where your changeset can be
found. It then include a dedicated patch header show a command to be used to
retrieve the change. See the test for example.
This is flagged as experimental because this feature is not safe until we have
more logic to test that:
- changeset actually exists on destination
- changeset is draft on destination.
As all this is experimental, bike shedding can happily happens before we remove
the experimental flag.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 23:17:01 -0700] rev 26545
export: introduce a generic way to add patch header on export
Extensions currently have no easy way to add data to exported
patch. This is now fixed using a generic mechanism in the same fashion
used by bundle2. Tests are coming in the next changeset with its first
user.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 00:23:20 -0700] rev 26544
incoming: request a bundle2 when possible (BC)
Incoming was using bundle1 in all cases, as bundle1 is restricted to
changegroup1 and does not support general delta, this can lead to significant
CPU overhead if the server is using general delta storage. We now properly
request and store a bundle2 to disk.
If the server include any output or error in the bundle, they will be stored on
disk and replayed when the bundle is read. As 'hg incoming' is going to read the
bundle right away, we call that 'good' enough and go back to the bigger plan of
having general delta on by default.
This was tracked as 4864
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 00:18:11 -0700] rev 26543
bundlerepo: indent some code to prepare next patch
We are about to add a new condition. Code is indented in a separated patch for
readability.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 08 Oct 2015 01:40:21 -0700] rev 26542
bundle2: add a way to just forward the bundle2 stream to another user
There is use case for directly forward and bundle2 stream from the peer to a
file (eg: 'hg incoming --bundle'), However ssh peers have no way to know the
'getbundle' is over except for actually interpreting the bundle. So we need to
have the unbundle do the interpreting and forward job.
The function is marked as private to highlight that this is terrible and that we
are sorry.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 01:10:49 -0700] rev 26541
bundle2: split parameter retrieval and processing
We want to introduce a simple way to forward the content of a bundle2 stream.
For this purpose, we will need to both yield the parameters block and process it
(to apply any behavior change it might indicate).
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 00:14:47 -0700] rev 26540
changegroup: extract the file management part in its own function
The current writebundle function do two things:
- taking a changegroup-packer instance and storing it into a valid bundle with
proper header.
- creating a temporary or requested file to store that bundle
We would like to make it easier to forward bundle stream directly from a remote
peer to a file, so we split the two logic to be able to skip the one about
building a valid bundle (the remote is already sending one).
Pierre-Yves David <pierre-yves.david@fb.com> [Sun, 04 Oct 2015 21:48:19 -0700] rev 26539
unbundle: properly read head modification result from bundle2
We were reading the wrong key...
Yuya Nishihara <yuya@tcha.org> [Wed, 07 Oct 2015 23:04:31 +0900] rev 26538
revset: strip off "literal:" prefix from bookmark not found error
This is what branch() and tag() do.
Yuya Nishihara <yuya@tcha.org> [Wed, 07 Oct 2015 23:00:29 +0900] rev 26537
revset: do not fall through to revspec for literal: branch (issue4838)
If "literal:" is specified, it must not be a revset expression. It should
error out with a better message.
Gijs Kruitbosch <gijskruitbosch@gmail.com> [Wed, 07 Oct 2015 21:08:14 +0100] rev 26536
hgweb: ensure both foreground and background colors are specified (issue4872)
When users configure the default foreground or background color to
non-default (black on white) values, several hgweb styles lack
contrast for headers and table row items. This patch fixes that by
ensuring that where either foreground or background colors are
specified, both are specified.
Yuya Nishihara <yuya@tcha.org> [Thu, 08 Oct 2015 23:24:38 +0900] rev 26535
templater: do not pre-evaluate generator keyword at runsymbol (issue4868)
It was introduced by e06e9fd2d99f, but the important code was removed by
a3c2d9211294. So there was no positive effect other than exhausting memory.
The problem spotted by e06e9fd2d99f is that you can't use a generator keyword
more than once. For example, in hgweb template, "{child} {child}" doesn't work
because the first "{child}" consumes the generator. But as a3c2d9211294 says,
the fix was wrong because it could overwrite a callable keyword that returns
a generator. Also the fix didn't work for a generator of generator such as
"{diff}" keyword. So, the proper fix for that problem would be to not put
a generator in a keyword table. Instead, it should be a factory of a generator.
Note that this should fix the memory issue in hgweb, but my firefox killed by
OOM in place. Be careful to not use a modern web browser to test the issue4868.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 05 Oct 2015 01:47:33 -0700] rev 26534
merge: drop special parent assignment in the obsolete case
We can safely drop this because the very same assignment is enforcement later in
the function. Dropping it will make it simpler to extract the default
destination logic in its own function.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 20:31:43 -0700] rev 26533
bundle: use bundle2 if repository uses general delta
As bundle1 does not support generaldelta, this would mean recomputing delta at
bundle time. This is similar to what we do for strip and shelve and was tracked
as issue4865.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 20:21:16 -0700] rev 26532
parsebundletype: add a comment for future generation
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 19:16:00 -0700] rev 26531
bundle: extend the format of --type to support version and compression
We had some basic undocumented support for uncompressed bundle2 support. We now
have an official extensible syntax to specify both format type and compression
(eg: bzip2-v2).
In practice, this changeset introduce the 'v1' and 'v2' identifier to make it
possible to combine format and compression. The default format is still 'v1'.
We'll care about picking 'v1' or 'v2' in regard with general delta in the next
changesets.
Gijs Kruitbosch <gijskruitbosch@gmail.com> [Wed, 07 Oct 2015 20:19:20 +0100] rev 26530
hgweb: fix border-radius for standards-based browsers
While here, remove an old gecko-specific property, as gecko has
supported the unprefixed version for many years.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 21:28:43 -0700] rev 26529
filemerge: move precheck to before files are written out
This is much simpler, and also avoids unnecessary disk IO.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 22:58:52 -0700] rev 26528
filemerge: move 'merging' output to before file creation
This has no visible impact but makes some future work simpler.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 21:21:56 -0700] rev 26527
filemerge.filemerge: make a tuple containing merge paths on disk
We're going to need this same tuple elsewhere.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 16:14:57 -0700] rev 26526
filemerge: switch trymerge boolean to mergetype enum
trymerge = False becomes mergetype = nomerge, and trymerge = True becomes
mergetype = fullmerge or mergeonly, depending on whether a premerge happens.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 15:13:41 -0700] rev 26525
filemerge: add some merge types
We're going to turn the 'trymerge' boolean into a 'mergetype' enum with these
three possible values.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Thu, 08 Oct 2015 01:41:30 +0900] rev 26524
shelve: restore unshelved dirstate explicitly after aborting transaction
Before this patch, "hg unshelve" uses aborting a current transaction
to discard temporary changes while unshelving.
This assumes that dirstate changes in a transaction scope are kept
even after aborting it. But this assumption will be broken by
"transactional dirstate". See the wiki page below for detail about it.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
This patch explicitly saves shelved dirstate just before aborting
current transaction, and restore dirstate with it after aborting by
utility function '_aborttransaction()' added by previous patch.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Thu, 08 Oct 2015 01:41:30 +0900] rev 26523
shelve: restore shelved dirstate explicitly after aborting transaction
Before this patch, "hg shelve" uses aborting a current transaction to
discard temporary changes while shelving.
This assumes that dirstate changes in a transaction scope are kept
even after aborting it. But this assumption will be broken by
"transactional dirstate". See the wiki page below for detail about it.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
This patch explicitly saves shelved dirstate just before aborting
current transaction, and restore dirstate with it after aborting by
utility function '_aborttransaction()' added by previous patch.
This patch replaces 'if tr: tr.abort()' by 'lockmod.release(tr)',
because the former is already done in '_aborttransaction()' (and the
latter has no effect), if current transaction is aborted in it
successfully. Otherwise, the latter is enough to trigger aborting.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Thu, 08 Oct 2015 01:41:30 +0900] rev 26522
shelve: add utility to abort current transaction but keep dirstate
"hg shelve" and "hg unshelve" use aborting a current transaction to
discard temporary changes while (un)shelving.
This assumes that dirstate changes in a transaction scope are kept
even after aborting it. But this assumption will be broken by
"transactional dirstate". See the wiki page below for detail about it.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
This patch adds utility function "_aborttransaction()" to abort
current transaction but keep dirstate changes for (un)shelving.
'dirstate.invalidate()' just after aborting a transaction should be
removed soon by subsequent patch, which writes or discards in-memory
dirstate changes at releasing transaction according to the result of
it.
BTW, there are some other ways below, which (seem to, at first glance)
resolve this issue. But this patch chose straightforward way for ease
of review and future refactorring.
- commit transaction at first, and then rollback it
It causes unintentional "dirty read" of running transaction to
other processes at committing it.
- use dirstateguard to save and restore shelved dirstate
After DirstateTransactionPlan, making 'dirstate.write()' write
in-memory changes into actual file requires
'transaction.writepending()' while transaction running.
It causes meaningless writing other in-memory changes out, even
though they are never referred.
In addition to it, it isn't desirable that scope of dirstateguard
and transaction intersects each other.
- get list of files changed from the parent, keep it in memory, and
emulate that changes after aborting transaction
This additional memory consumption may block aborting transaction
in large repository (on small resource environment).
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Thu, 08 Oct 2015 01:41:30 +0900] rev 26521
dirstate: split write to write changes into files other than .hg/dirstate
'_writedirstate()' is used mainly for "transactional dirstate". See
the wiki page below for detail about it.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Thu, 08 Oct 2015 01:41:30 +0900] rev 26520
bookmarks: use recordchange instead of writing if transaction is active
Before this patch, 'bmstore.write()' always write in-memory bookmark
changes into '.hg/bookmarks' regardless of transaction activity.
If 'bmstore.write()' is invoked inside a transaction and it writes
changes into '.hg/bookmarks', then:
- original bookmarks aren't restored at failure of that transaction
This breaks "all or nothing" policy of the transaction.
BTW, "hg rollback" can restore bookmarks successfully even before
this patch, because original bookmarks are saved into
'.hg/journal.bookmarks' at the beginning of the transaction, and
it (actually renamed as '.hg/undo.bookmarks') is used by "hg
rollback".
- uncommitted bookmark changes are visible to other processes
This is a kind of "dirty read"
For example, 'rebase.rebase()' implies 'bmstore.write()', and it may
be executed inside the transaction of "hg unshelve". Then, intentional
aborting at the end of "hg unshelve" transaction doesn't restore
original bookmarks (this is obviously a bug).
This patch uses 'bmstore.recordchange()' instead of actual writing by
'bmstore._writerepo()', if any transaction is active
This patch also removes meaningless restoring bmstore explicitly at
the end of "hg shelve".
This patch doesn't choose fixing each 'bmstore.write()' callers as
like below, because writing similar code here and there is very
redundant.
before:
bmstore.write()
after:
tr = repo.currenttransaction()
if tr:
bmstore.recordchange(tr)
else:
bmstore.write()
Even though 'bmstore.write()' itself may have to be discarded by
putting bookmark operations into transaction scope, this patch chose
fixing it to implement "transactional dirstate" at first.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 00:27:23 -0700] rev 26519
filemerge: run symlink check for :merge3
Just like :merge, :merge3 doesn't support merging symlinks.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 00:24:44 -0700] rev 26518
filemerge: print correct name of tool for symlink checks
Earlier we'd print ':merge' even if the tool was something else like ':union'.
That's clearly a bug.
Siddharth Agarwal <sid0@fb.com> [Wed, 07 Oct 2015 00:01:16 -0700] rev 26517
filemerge: normalize 'internal:foo' names to ':foo'
In upcoming patches we're going to present these names in the UI -- it would be
good not to present deprecated names.
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 22:57:21 -0700] rev 26516
filemerge: use symlinkcheck for :merge and :union
This exposes a couple of bugs, both of which will be fixed in upcoming patches.
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 22:56:33 -0700] rev 26515
filemerge: add a precheck for symlinks
This will be used by internal merge tools.
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 22:55:21 -0700] rev 26514
filemerge: call precheck if available
In upcoming patches we'll define a precheck function for some merge tools.
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 22:54:14 -0700] rev 26513
filemerge: add a before-merge callback to internal merge tools
We're going to separate the pre-merge and merge steps for merge tools. The
merge step will be specific to the tool, but the pre-merge step will be common
to all merge tools that need it.
However, some merge tools run checks *before* the pre-merge step. This callback
will allow that to continue to work.
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 17:39:13 -0700] rev 26512
filemerge: indent filemerge.filemerge
This will make upcoming patches much easier to review.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 20:15:00 -0700] rev 26511
test-bundle-type: replace unbundle with debugbundle
We now have a convenient command to look at bundle contents, let's use
it.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 18:01:24 -0700] rev 26510
bundle: extract the parsing of the bundle type in a function
We are going to introduce significant extensions of the bundle parsing code to
support creation of bundle2 through the bundle command. As an early step, we
extract the logic in its own function.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 19:14:47 -0700] rev 26509
changegroup: add version argument to getchangegroup
For some obscure reasons (probably upsetting a Greek goddess),
getchangegroup did not had a 'version' argument to control the changegroup
version. We fixes this to allow cg02 to be used with 'hg bundle' in the future.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 19:14:33 -0700] rev 26508
changegroup: add version argument to getlocalchangegroup
For some obscure reasons (probably upsetting a Greek goddess),
getlocalchangegroup did not have a 'version' argument to control the
changegroup version. We fix this to allow cg02 to be used with 'hg
bundle' in the future.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 15:08:00 -0700] rev 26507
shelve: bundle using bundle2 if repository is general delta (issue4862)
This will prevent expensive delta computation on bundling and is similar to what we do
for strip backup.
Pierre-Yves David <pierre-yves.david@fb.com> [Thu, 01 Oct 2015 15:09:32 -0700] rev 26506
shelve: move changegroup generation inside writebundle
We will generate different changegroup if general delta is enabled so we gather
this in the lower level function. There wasn't any good reason to have it in
the main code anyway.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 29 Sep 2015 17:14:30 -0700] rev 26505
shelve: compress on-disk bundle
Shelve is using 'HG10UN' bundle for no obvious reason. Changing this to 'HG10BZ'
as we do for other bundle stored on disk.
Matt Mackall <mpm@selenic.com> [Wed, 07 Oct 2015 13:44:48 -0500] rev 26504
merge with stable
Bruce Cran <bruce.cran@gmail.com> [Mon, 05 Oct 2015 10:43:16 -0600] rev 26503
notify: fix fromauthor setting for 'incoming' hook type (issue4194)
Set the author field in notification emails for the 'incoming' hook type
in addition to 'changegroup' and 'outgoing' types.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Oct 2015 12:37:26 -0400] rev 26502
templater: protect word() from crashing on out of range negative value
The function isn't documented to work with negative values at all, but it does,
which can be useful. However, the range check didn't account for this.
Siddharth Agarwal <sid0@fb.com> [Fri, 02 Oct 2015 13:00:47 -0700] rev 26501
commands: add a new debug command to print merge state
We're going to be extending the merge state very soon, and this will give us a
way to test all that.
Siddharth Agarwal <sid0@fb.com> [Wed, 30 Sep 2015 21:22:31 -0700] rev 26500
merge.mergestate: factor out code to validate v1/v2 records
We're going to need this in another place in upcoming patches.
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 13:19:05 -0700] rev 26499
localrepo: prevent wlock from being inherited when a transaction is running
Review feedback from Pierre-Yves David. A separate line of work is working to
ensure that dirstate writes are written to a separate 'pending' file while a
transaction is active. Lock inheritance currently conflicts with that, so dodge
the issue by simply preventing inheritance while a transaction is running.
Custom merge drivers aren't going to run inside a transaction, so this doesn't
affect that.
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 13:13:31 -0700] rev 26498
lock: add a way to prevent locks from being inherited
We want to prevent locks from being inherited sometimes (e.g. when there's a
currently running transaction, which will break a lot of assumptions we're
making in here.)
Siddharth Agarwal <sid0@fb.com> [Tue, 06 Oct 2015 15:55:50 -0700] rev 26497
test-fncache: use args/kwargs for lock wrapper
This is annoying to keep up to date, and also just plain unnecessary.
timeless@mozdev.org [Tue, 06 Oct 2015 15:07:00 -0400] rev 26496
rebase: enable histedit for useful help with it
timeless@mozdev.org [Tue, 06 Oct 2015 15:01:25 -0400] rev 26495
rebase: suggest help -e histedit
Users unfamiliar with an extension should be reading the
documentation for the feature, not the command.
timeless@mozdev.org [Tue, 06 Oct 2015 15:05:40 -0400] rev 26494
rebase: factor out histedit help command
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Oct 2015 22:35:36 +0900] rev 26493
util: use tuple accessor to get accurate st_mtime value (issue4836)
Because st.st_mtime is computed as 'sec + 1e-9 * nsec' and double is too narrow
to represent nanoseconds, int(st.st_mtime) can be 'sec + 1'. Therefore, that
value could be different from the one got by osutils.listdir().
This patch fixes the problem by accessing to raw st_mtime by tuple index.
It catches TypeError to fall back to st.st_mtime because our osutil.stat does
not support tuple index. In dirstate.normal(), 'st' is always a Python stat,
but in dirstate.status(), it can be either a Python stat or an osutil.stat.
Thanks to vgatien-baron@janestreet.com for finding the root cause of this
subtle problem.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Oct 2015 22:25:29 +0900] rev 26492
util: extract stub function to get mtime with second accuracy
This function is trivial but will need a long comment why it can't use
st.st_mtime. See the next patch for details.
Siddharth Agarwal <sid0@fb.com> [Mon, 05 Oct 2015 14:37:59 -0700] rev 26491
scmutil: add a way for a repo's wlock to be inherited by a subprocess
This is part of a series that will allow locks to be inherited by subprocesses
in limited circumstances.
In the future, we'll call this for custom merge drivers.
Siddharth Agarwal <sid0@fb.com> [Mon, 05 Oct 2015 14:34:52 -0700] rev 26490
scmutil: add a way for a subprocess to be run with an inheritable lock
This is part of a series that will allow locks to be inherited by subprocesses
in limited circumstances.
In an upcoming patch, we'll add an API for the wlock to be inherited.
Siddharth Agarwal <sid0@fb.com> [Mon, 05 Oct 2015 14:27:37 -0700] rev 26489
localrepo: add a way to get the current wlock if it's held
This will be useful to pass around a reference to the lock to some functions
we're going to add to scmutil. We don't want those functions to live in
localrepo to avoid bloat.
Siddharth Agarwal <sid0@fb.com> [Mon, 05 Oct 2015 14:26:53 -0700] rev 26488
localrepo: add a way to get the current lock if it's held
We'll use this in upcoming patches to return a reference to the wlock if it's
held.
timeless@mozdev.org [Thu, 24 Sep 2015 13:58:18 -0400] rev 26487
test-help: verify that extension keywords appear in help templates
fixed by a2291c9c85a1
Matt Harbison <matt_harbison@yahoo.com> [Mon, 24 Aug 2015 23:30:17 -0400] rev 26486
templatekw: add {changes}, {distance} and {tag} to the {latesttag} keyword
Matt Harbison <matt_harbison@yahoo.com> [Sun, 23 Aug 2015 23:44:58 -0400] rev 26485
templater: introduce {latesttag()} function to match a pattern (issue4184)
This allows the latest class of tag to be found, such as a release candidate or
final build, instead of just the absolute latest.
It doesn't appear that the existing keyword can be given an optional argument.
There is a keyword, function and filter for 'date', so it doesn't seem harmful
to introduce a new function with the same name as an existing keyword. Most
functions are pretty Mercurial agnostic, but there is {revset()} as precedent.
Even though templatekw.getlatesttags() returns a single tuple, one entry of
which is a list, it is simplest to present this as a list of tags instead of a
single item, with each tag having a distance and change count attribute. It is
also closer to how {latesttag} returns a list of tags, and how this function
works when not given a '%' operator.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 24 Aug 2015 23:07:00 -0400] rev 26484
templatekw: introduce showlatesttags() to handle {latesttag} keywords
The keywords {changes}, {distance} and {tag} will be available on a future
template method that will allow pattern matching against tag names. For
consistency, these should be available on the existing {latesttag} keyword as
well.
I debated whether or not to add {tag} instead of just continuing with the
existing {latesttag}. But it seems clearer not to have the same name for two
distinct things (a list in the LHS of %, and an individual tag value on the
right).
The value of latesttags[0] is the date of commit for the cset to which the tag
is applied (i.e. not the date the tag was applied), and therefore isn't made
visible because it doesn't seem interesting. It appears that this is merely an
internal implementation detail for sorting csets in a stable manner when there
are different branches.
Matt Harbison <matt_harbison@yahoo.com> [Mon, 05 Oct 2015 21:11:50 -0400] rev 26483
templatekw: factor out the changessincetag calculation to a private method
This will be reused in the next patch.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 23 Aug 2015 23:22:55 -0400] rev 26482
templatekw: allow getlatesttags() to match a specific tag pattern
This will allow the latest class of tag to be found, such as a release candidate
or final build, instead of just the absolute latest. It will be exposed in a
future patch.
It's unfortunate that the original 'latesttags' cache can't be used to determine
the proper values, but it isn't fully populated for the entire repo. For
example, the {latesttagdistance} keyword on the Mecurial repo builds the cache
up back to the revision for 1.4. If the pattern was 're:^0\.\d$', that wouldn't
be in the cache. Maybe this can be optimized some other way, but for now, this
is the simpliest implementation.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 22 Aug 2015 22:52:18 -0400] rev 26481
util: extract stringmatcher() from revset
This is used to match against tags, bookmarks, etc in revsets. It will be used
in a future patch to do the same tag matching in templater.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Oct 2015 17:36:32 -0700] rev 26480
util.chunkbuffer: avoid extra mutations when reading partial chunks
Previously, a read(N) where N was less than the length of the first
available chunk would mutate the deque instance twice and allocate a new
str from the slice of the existing chunk. Profiling drawed my attention
to these as a potential hot spot during changegroup reading.
This patch makes the code more complicated in order to avoid the
aforementioned 3 operations.
On a pre-generated mozilla-central gzip bundle, this series has the
following impact on `hg unbundle` performance on my MacBook Pro:
before: 358.21 real 317.69 user 38.49 sys
after: 301.57 real 262.69 user 37.11 sys
delta: -56.64 real -55.00 user -1.38 sys
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Oct 2015 16:34:47 -0700] rev 26479
util.chunkbuffer: refactor chunk handling logic
This will make the next patch easier to read. It provides no benefit on
its own.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 05 Oct 2015 16:28:12 -0700] rev 26478
util.chunkbuffer: special case reading everything
The new code results in simpler logic within the while loop. It is also
faster since we avoid performing operations on the queue and buf
collections. However, there shouldn't be any super hot loops for this
since the whole point of chunkbuffer is to avoid reading large amounts
of data at once. This does, however, make it easier to optimize
chunkbuffer in a subsequent patch.
Mathias De Maré <mathias.demare@gmail.com> [Mon, 05 Oct 2015 07:13:35 +0200] rev 26477
revert: add reference to backout
Mathias De Maré <mathias.demare@gmail.com> [Mon, 05 Oct 2015 07:11:48 +0200] rev 26476
backout: add reference to revert
Siddharth Agarwal <sid0@fb.com> [Fri, 25 Sep 2015 12:39:23 -0700] rev 26475
localrepo: allow wlock to be inherited
This is part of a series that will allow locks to be inherited by subprocesses
in limited circumstances.
When allowed, the parent process will pass down requisite information to the
child process by way of this environment variable.
Siddharth Agarwal <sid0@fb.com> [Sun, 04 Oct 2015 20:04:44 -0700] rev 26474
lock.release: don't call postrelease functions for inherited locks
Review feedback from Pierre-Yves David. The postrelease functions typically
assume the lock is not held at all.
Siddharth Agarwal <sid0@fb.com> [Sun, 04 Oct 2015 20:02:50 -0700] rev 26473
lock: turn prepinherit/reacquire into a single context manager
Review feedback from Pierre-Yves David. This makes the overall code cleaner and
less error-prone, and makes a previously explicitly checked error state
impossible.
Siddharth Agarwal <sid0@fb.com> [Sun, 04 Oct 2015 19:28:43 -0700] rev 26472
localrepo: add a note about parentenvvar
Review feedback from Pierre-Yves David.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 12:11:44 -0700] rev 26471
exchange: add "streaming all changes" to bundle2 pulling
This is the beginning of client-side support for performing a stream
clone using bundle2. The main bundle2 pull function checks whether to
perform a streaming clone and outputs a message if so.
While we have a duplicate message, it seems easier to have all the
bundle2 console writing in one location and in an easy-to-read
conditional block.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 12:07:01 -0700] rev 26470
streamclone: move "streaming all changes" message location
Previously, the message was printed after we requested and started
processing the remote stream. This seems like something that we should
do before calling out to the remote. Moving it also makes it easier to
deal with the bundle2 implementation.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 19:06:06 -0700] rev 26469
streamclone: move payload header generation into own function
The stream clone data over the wire protocol contains a header line
indicating total file count and data size. In bundle2, this metadata can
be captured by a part parameter and doesn't need to be in the body.
In preparation for bundle2, have generatev1() return the raw metadata
and move the header generation to its own function.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 18:44:46 -0700] rev 26468
streamclone: move payload header line consumption
bundle2 parts have parameters. These are a logical place for "header"
data such as the file count and payload size of stream clone data. In
preparation for supporting stream clones with bundle2, move the
consumption of the header line from the payload into
maybeperformlegacystreamclone().
Note: the header line is still being emitted by generatev1(). This will
be addressed in a subsequent patch.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 18:35:19 -0700] rev 26467
streamclone: teach canperformstreamclone to be bundle2 aware
We add an argument to canperformstreamclone() to return False if a
bundle2 stream clone is available. This will enable the legacy stream
clone step to no-op when a bundle2 stream clone is supported.
The commented code will be made active when bundle2 supports streaming
clone.
This patch does foreshadow the introduction of the "stream" bundle2
capability and its "v1" sub-capability. The bundle2 capability mirrors
the existing "stream" capability and is needed so clients know whether a
server explicitly supports streaming clones over bundle2 (servers up to
this point support bundle2 without streaming clone support).
The sub-capability will denote which data formats and variations are
supported. Currently, the value "v1" denotes the existing streaming
clone data format, which I intend to reuse inside a bundle2 part. My
intent is to eventually introduce alternate data formats that can be
produced and consumed more efficiently. Having a sub-capability means
we don't need to introduce a new top-level bundle2 capability when new
formats are introduced. This doesn't really have any implications
beyond making the capabilities namespace more organized.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 11:50:42 -0700] rev 26466
streamclone: refactor canperformstreamclone to accept a pullop
This isn't strictly necessary. But a lot of pull functionality accepts a
pulloperation so extra state can be added easily. It also enables
extensions to perform more powerful things.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 12:03:30 -0700] rev 26465
exchange: expose bundle2 availability on pulloperation
Like the previous patch, the value is cached and will prevent a function
level import in streamclone.py.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 18:31:53 -0700] rev 26464
exchange: expose bundle2 capabilities on pulloperation
This adds a cache and makes accessing the capabilities slightly simpler,
as you don't need to directly go through the bundle2 module. This will
also help prevent a function-level import in streamclone.py.
This patch arguably isn't necessary. But I think it makes things
slightly nicer.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Sun, 04 Oct 2015 21:33:29 +0900] rev 26463
keyword: make restrict mode False while updating files for rollback
This is a preparation for using 'repo.rollback()' instead of aborting
a current running transaction for "shelve" and "unshelve".
Before this patch, updating files as a part of 'repo.rollback()'
overridden by keyword extension always follows 'restrict' mode of the
command currently executed.
"merge", "unshelve" and so on should be 'restrict'-ed, because keyword
expansion may cause unexpected conflicts at merging while these
commands.
But, if 'repo.rollback()' is invoked while executing 'restrict'-ed
commands, modified files in the working directory are marked as
"CLEAN" unexpectedly by code path below:
# 'lookup' below is True at updating modified files for rollback
kwcmd = self.restrict and lookup # kwexpand/kwshrink
:
if kwcmd:
self.repo.dirstate.normal(f)
On the other hand, "rollback" command isn't 'restrict'-ed, because
rollbacking itself doesn't imply merging.
Therefore, disabling 'restrict' mode while updating files as a part of
'repo.rollback()' regardless of current 'restrict' mode should be
reasonable.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 11:34:28 -0700] rev 26462
streamclone: rename and document maybeperformstreamclone()
Upcoming patches will introduce bundle2 based streaming clones. Add
"legacy" to the function name and add a docstring clarifying the intent of
the function.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 11:27:10 -0700] rev 26461
streamclone: move applyremotedata() into maybeperformstreamclone()
Future work around stream cloning will be implemented in a bundle2
world. This code will only be used in the legacy code path and
doesn't need to be abstracted or extensible.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 03 Oct 2015 09:53:56 -0700] rev 26460
branchmap: move branch cache code out of streamclone.py
This is low-level branch map and cache manipulation code. It deserves to
live next to similar code in branchmap.py. Moving it also paves the road
for multiple consumers, such as a bundle2 part handler that receives
branch mappings from a remote.
This is largely a mechanical move, with only variable names and
indentation being changed.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 23:08:15 -0700] rev 26459
streamclone: move streamin() into maybeperformstreamclone()
streamin() only had a single consumer. And it always only ever will
because it is strongly coupled with the current,
soon-to-be-superseded-by-bundle2 functionality.
The return value has been dropped because nobody was using it.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 11:20:52 -0700] rev 26458
streamclone: refactor maybeperformstreamclone to take a pullop
Just like all the other pull steps. Consistency is good.
This seems a little excessive right now since maybeperformstreamclone is
such a short function. This will be addressed in a subsequent patch.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 11:17:43 -0700] rev 26457
demandimport: replace more references to _demandmod instances
_demandmod instances may be referenced by multiple importing modules.
Before this patch, the _demandmod instance only maintained a reference
to its first consumer when using the "from X import Y" syntax. This is
because we only created a single _demandmod instance (attached to the
parent X module). If multiple modules A and B performed
"from X import Y", we'd produce a single _demandmod instance
"demandmod" with the following references:
X.Y = <demandmod>
A.Y = <demandmod>
B.Y = <demandmod>
The locals from the first consumer (A) would be stored in <demandmod1>.
When <demandmod1> was loaded, we'd look at the locals for the first
consumer and replace the symbol, if necessary. This resulted in state:
X.Y = <module>
A.Y = <module>
B.Y = <demandmod>
B's reference to Y wasn't updated and was still using the proxy object
because we just didn't record that B had a reference to <demandmod> that
needed updating!
With this patch, we add support for tracking which modules in addition
to the initial importer have a reference to the _demandmod instance and
we replace those references at module load time.
In the case of posix.py, this fixes an issue where the "encoding" module
was being proxied, resulting in hundreds of thousands of
__getattribute__ lookups on the _demandmod instance during dirstate
operations on mozilla-central, speeding up execution by many
milliseconds. There are likely several other operation that benefit from
this change as well.
The new mechanism isn't perfect: references in locals (not globals) may
likely linger. So, if there is an import inside a function and a symbol
from that module is used in a hot loop, we could have unwanted overhead
from proxying through _demandmod. Non-global imports are discouraged
anyway. So hopefully this isn't a big deal in practice. We could
potentially deploy a code checker that bans use of attribute lookups of
function-level-imported modules inside loops.
This deficiency in theory could be avoided by storing the set of globals
and locals dicts to update in the _demandmod instance. However, I tried
this and it didn't work. One reason is that some globals are _demandmod
instances. We could work around this, but it's a bit more work. There
also might be other module import foo at play. The solution as
implemented is better than what we had and IMO is good enough for the
time being.
It's worth noting that this sub-optimal behavior was made worse by the
introduction of absolute_import and its recommended "from . import X"
syntax for importing modules from the "mercurial" package. If we ever
wrote performance tests, measuring the amount of module imports and
__getattribute__ proxy calls through _demandmod instances would be
something I'd have it check.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 04 Oct 2015 10:36:54 -0700] rev 26456
demandimport: refactor processfromitem
This will match the next patch smaller and easier to read.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 03 Oct 2015 15:30:17 -0700] rev 26455
demandimport: consolidate code for processing items in fromlist
This code was mostly duplicated. An upcoming patch will add more
complexity, making the duplication harder to justify. Consolidate the
code.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Oct 2015 15:16:33 +0900] rev 26454
pager: recreate stdout to make it line-buffered
We want to see partial command results as soon as possible. But the buffering
mode of stdout (= pager's stdin) was set to fully-buffered because it isn't
associated with a tty. So, this patch recreates new stdout object to force its
buffering mode.
Because two file objects are associated with the same stdout fd and their
destructors will call close(), one of them must be closed carefully. Python
expects that the stdout fd never be closed even after sys.stdout.close() [1],
but newstdout has no such hack. So this patch calls newstdout.close()
immediately before duplicating the original stdout fd to sys.stdout.
operation sys.stdout newstdout fd
--------------------- ---------- --------- --------
newstdout.close() open closed closed
os.dup2(stdoutfd, ..) open closed open
del sys.stdout closed closed open [1]
[1]: https://hg.python.org/cpython/file/v2.7.10/Python/sysmodule.c#l1391
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Oct 2015 18:48:21 +0900] rev 26453
pager: rename variables of backup fds
The next patch will add backup file objects, so the original variable names
would be confusing.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Oct 2015 15:02:52 +0900] rev 26452
pager: inline _pagersubprocess() into _runpager()
We don't need _pagersubprocess() because the fork version was removed
at 59d794154e8d.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Oct 2015 23:10:40 +0900] rev 26451
ui: send traceback of devel warning to appropriate output stream
If ui.ferr is a command-server channel, traceback should be written to it.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Oct 2015 14:57:24 +0900] rev 26450
util.system: compare fileno to see if it needs stdout redirection
Future patches will reopen stdout to be line-buffered, so sys.stdout may
be different object than sys.__stdout__.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 23:04:52 -0700] rev 26449
exchange: move stream clone logic into pull code path
Stream clones are a special case of clones. Clones are a special case of
pull. Most of the logic for deciding what to do at pull time is in
exchange.py. It makes sense for the stream clone determination to live
there as well.
This patch moves the calling of the stream clone code into pull(). The
checks in streamclone.canperformstreamclone() ensure that we don't
perform a stream clone unless it is possible.
A future patch will convert maybeperformstreamclone() to accept a
pullop to make it consistent with everything else in pull(). It will
also grow some functionality (in case you doubted the necessity of a 4
line function).
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 22:16:34 -0700] rev 26448
exchange: teach pull about requested stream clones
An upcoming patch will move the invocation of stream cloning logic to
the normal pull code path (from localrepository.clone). In preparation
for this, we teach pull() and pulloperation about whether a streaming
clone is requested.
The return logic in localrepository.clone() has been reformatted
slightly because of line length issues.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 21:53:25 -0700] rev 26447
streamclone: add explicit check for empty local repo
Stream clone doesn't work with non-empty local repositories. In upcoming
patches, we'll move stream cloning to the regular pull code path. Add an
explicit check on the repository being empty to prevent streaming clones
to non-empty repos.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 22:22:11 -0700] rev 26446
streamclone: refactor code for deciding to stream clone
Having this in a standalone function will eventually enable bundle2 to
share code with the bundle1 code path.
While I was here, I also added some comments to add clarity.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 21:39:04 -0700] rev 26445
streamclone: move streaming clone logic from localrepo
This is the last remnants of streaming clone code in localrepo.py.
This is a mostly mechanical transplant of code to a new file. Only a
rewrite of "self" to "repo" was performed. The code will be
significantly refactored in upcoming patches. So don't scrutinize it too
closely.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 16:24:56 -0700] rev 26444
streamclone: move _allowstream() from wireproto
While we're moving things into streamclone.py...
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 16:05:52 -0700] rev 26443
streamclone: move code out of exchange.py
We bulk move functions from exchange.py related to streaming clones.
Function names were renamed slightly to drop a component redundant with
the module name. Docstrings and comments referencing old names and
locations were updated accordingly.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 15:58:24 -0700] rev 26442
streamclone: move stream_in() from localrepo
Another basic content move. The underscore from the function name was
removed to comply with naming standards.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 15:51:32 -0700] rev 26441
streamclone: move applystreamclone() from localrepo.py
Upcoming patches will modernize the streaming clone code. Streaming
clone data and code kind of lives in its own world. exchange.py is
arguably the most appropriate existing location for it. However, over
a dozen patches from now it became apparent that there was a lot of code
related to streaming clones and that having it contained within its own
module would make it easier to comprehend. So, we establish
streamclone.py.
It's worth noting that streamclone.py existed a long time ago, last seen
in the 1.6 release. It was removed in 04f76a954842.
The function was renamed as part of the move because its old name was
redundant with the new module name. The only other content change was
"self" was renamed to "repo" and minor grammar in the docstring was
updated.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 02 Oct 2015 15:36:00 -0700] rev 26440
exchange: add docstring to pull()
This seems like the kind of important function that should be documented
better.
Siddharth Agarwal <sid0@fb.com> [Fri, 25 Sep 2015 12:28:12 -0700] rev 26439
localrepo: allow creating inherited locks
This is part of a series that will allow locks to be inherited by subprocesses
in limited circumstances.
Siddharth Agarwal <sid0@fb.com> [Fri, 25 Sep 2015 12:56:05 -0700] rev 26438
error: make lock inheritance contract violations a subclass of RuntimeError
This is more appropriate, per Pierre-Yves David.
Yuya Nishihara <yuya@tcha.org> [Sat, 26 Sep 2015 12:19:39 +0900] rev 26437
templatekw: hide help of "branches" by DEPRECATED marker
See 43e872a35f8a for details.
Yuya Nishihara <yuya@tcha.org> [Sat, 26 Sep 2015 12:39:13 +0900] rev 26436
templatekw: remove dockeywords hack
Now all template keywords are defined as real functions.
Yuya Nishihara <yuya@tcha.org> [Sat, 26 Sep 2015 12:38:02 +0900] rev 26435
templatekw: port implementation of showparents() from changeset_templater
It isn't cool, but we can peek at ui flag via repo.ui. So, it is possible
to implement showparents() in templatekw, and therefore we can eliminate the
dockeywords hack.
Yuya Nishihara <yuya@tcha.org> [Sat, 26 Sep 2015 12:32:03 +0900] rev 26434
templatekw: reorder stub of showparents() function
It will have an implementation.
Yuya Nishihara <yuya@tcha.org> [Sat, 26 Sep 2015 12:29:09 +0900] rev 26433
changeset_printer: move _meaningful_parentrevs() to scmutil
It will be used by templatekw.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 01 Oct 2015 22:14:11 -0400] rev 26432
test-run-tests: add globs for Windows
There are two other unrelated failures in this test. For some reason child1 has
a space after it, thus 2 spaces before the glob, otherwise the glob is ignored
and removed.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 01 Oct 2015 21:49:50 -0400] rev 26431
test-commandserver: generate files with a fixed EOL for test stability
The hashes were different on Windows.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 01 Oct 2015 21:24:10 -0400] rev 26430
test-debugextensions: sanitize JSON output for Windows
Windows is printing a double backslash, which isn't handled by (glob):
--- c:/Users/Matt/Projects/hg/tests/test-debugextensions.t
+++ c:/Users/Matt/Projects/hg/tests/test-debugextensions.t.err
@@ -47,7 +47,7 @@
{
"buglink": "",
"name": "color",
- "source": "*/hgext/color.pyc", (glob)
+ "source": "c:\\Users\\Matt\\Projects\\hg\\hgext\\color.pyc",
"testedwith": "internal"
},
{
Matt Harbison <matt_harbison@yahoo.com> [Thu, 01 Oct 2015 21:10:38 -0400] rev 26429
test-strip: add glob for Windows
Ryan McElroy <rmcelroy@fb.com> [Thu, 01 Oct 2015 10:48:14 -0700] rev 26428
bundle2: generate check:heads in a independent function
Matt Mackall <mpm@selenic.com> [Thu, 01 Oct 2015 12:17:46 -0500] rev 26427
merge with stable
timeless@mozdev.org [Wed, 30 Sep 2015 12:36:30 -0400] rev 26426
cmdutil: handle multiline translations of HG: messages safely
Before this, if a localizer/localization included a multiline
message, and didn't prefix the intermediate lines with 'HG: ',
then the line would be a candidate for inclusion in the commit
message -- which isn't ideal.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 29 Sep 2015 14:42:03 -0700] rev 26425
strip: compress bundle2 backup using BZ
Storing uncompressed bundle on disk would be a regression. Strip backup using
bundle2 are now compressed when requested.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 29 Sep 2015 14:41:40 -0700] rev 26424
writebundle: add a compression argument for the bundle2 case
Bundle2 compression is more complex than the bundle1 one. Therefore it
is handled by the bundler itself. Moreover, on-disk bundle2 will
probably have a large number of flavors so simply adding a new "format"
for it does not seems the way to go.
This will be used in the next changeset to compress bundle2 strip backup.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 29 Sep 2015 13:16:51 -0700] rev 26423
strip: use bundle2 + cg2 by default when repository use general delta
The bundle10 format (plain changegroup-01) does not support general delta and
result into expensive delta re-computation when stripping. If the repository is
general delta, we store backups as bundle20 containing a changegroup-02 payload.
We remove the experimental feature related to strip backup bundle format because
this achieve the same goal in a leaner way. Removing the experimental option is
fine, that is why it experimental in the first place.
Compression of these bundles are coming in later changesets.