Mads Kiilerich <madski@unity3d.com> [Sun, 16 Oct 2016 02:29:45 +0200] rev 30190
largefiles: more safe handling of interruptions while updating modifications
Largefiles are fragile with the design where dirstate and lfdirstate must be
kept in sync.
To be less fragile, mark all clean largefiles as unsure ("normallookup") before
updating standins. After standins have been updated and we know exactly which
largefile standins actually was changed, mark the unchanged largefiles back to
clean ("normal").
This will make the failure mode more safe. If interrupted, the next command
will continue to perform extra hashing of all largefiles. That will do that all
largefiles that are out of sync with their standin will be marked dirty and
they will show up in status and can be cleaned with update --clean.
Mads Kiilerich <madski@unity3d.com> [Sun, 16 Oct 2016 02:26:38 +0200] rev 30189
largefiles: test coverage of fatal interruption of update
Test using existing changesets in a clean working directory, revealing problems
with files that don't show up as modified or do show up as removed when they
just not have been written yet.
Gábor Stefanik <gabor.stefanik@nng.com> [Wed, 12 Oct 2016 21:33:45 +0200] rev 30188
checkcopies: add a sanity check against false-positive copies
When grafting a copy backwards through a rename, a copy is wrongly detected,
which causes the graft to be applied inappropriately, in a destructive way.
Make sure that the old file name really exists in the common ancestor,
and bail out if it doesn't.
This fixes the aggravated case of bug 5343, although the basic issue
(failure to duplicate the copy information) still occurs.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 16 Oct 2016 10:38:52 -0700] rev 30187
exchange: refactor APIs to obtain bundle data (API)
Currently, exchange.getbundle() returns either a cg1unpacker or a
util.chunkbuffer (in the case of bundle2). This is kinda OK, as
both expose a .read() to consumers. However, localpeer.getbundle()
has code inferring what the response type is based on arguments and
converts the util.chunkbuffer returned in the bundle2 case to a
bundle2.unbundle20 instance. This is a sign that the API for
exchange.getbundle() is not ideal because it doesn't consistently
return an "unbundler" instance.
In addition, unbundlers mask the fact that there is an underlying
generator of changegroup data. In both cg1 and bundle2, this generator
is being fed into a util.chunkbuffer so it can be re-exposed as a
file object.
util.chunkbuffer is a nice abstraction. However, it should only be
used "at the edges." This is because keeping data as a generator is
more efficient than converting it to a chunkbuffer, especially if we
convert that chunkbuffer back to a generator (as is the case in some
code paths currently).
This patch refactors exchange.getbundle() into
exchange.getbundlechunks(). The new API returns an iterator of chunks
instead of a file-like object.
Callers of exchange.getbundle() have been updated to use the new API.
There is a minor change of behavior in test-getbundle.t. This is
because `hg debuggetbundle` isn't defining bundlecaps. As a result,
a cg1 data stream and unpacker is being produced. This is getting fed
into a new bundle20 instance via bundle2.writebundle(), which uses
a backchannel mechanism between changegroup generation to add the
"nbchanges" part parameter. I never liked this backchannel mechanism
and I plan to remove it someday. `hg bundle` still produces the
"nbchanges" part parameter, so there should be no user-visible
change of behavior. I consider this "regression" a bug in
`hg debuggetbundle`. And that bug is captured by an existing
"TODO" in the code to use bundle2 capabilities.
Pierre-Yves David <pierre-yves.david@ens-lyon.org> [Thu, 13 Oct 2016 01:30:14 +0200] rev 30186
mergecopies: rename 'ca' to 'base'
This variable was named after the common ancestor. It is actually the merge
base that might differ from the common ancestor in the graft case. We rename the
variable before a larger refactoring to clarify the situation. Similar rename
was also applied to 'checkcopies' in a prior changeset.
Pierre-Yves David <pierre-yves.david@ens-lyon.org> [Thu, 13 Oct 2016 01:26:33 +0200] rev 30185
copies: move variable document from checkcopies to mergecopies
It appears that 'mergecopies' is the function consuming these data so we move
the documentation there.
Pierre-Yves David <pierre-yves.david@ens-lyon.org> [Tue, 11 Oct 2016 02:21:42 +0200] rev 30184
checkcopies: pass data as a dictionary of dictionaries
more are coming
Pierre-Yves David <pierre-yves.david@ens-lyon.org> [Tue, 11 Oct 2016 02:15:23 +0200] rev 30183
checkcopies: move 'movewithdir' initialisation right before its usage
The 'movewithdir' had a lot of related logic all around the 'mergecopies'.
However it is actually never containing anything until the very last loop in
that function. We move the (simplified) variable definition there for clarity
Mads Kiilerich <madski@unity3d.com> [Fri, 14 Oct 2016 01:53:15 +0200] rev 30182
cmdutil: satisfy expections in dirstateguard.__del__, even if __init__ fails
Python "delstructors" are terrible - this one because it assumed that __init__
had completed before it was called. That would not necessarily be the case if
the repository was read only or broken and saving the dirstate thus failed in
unexpected ways. That could give confusing warnings about missing '_active'
after failures.
To fix that, make sure all member variables are "declared" before doing
anything that possibly could fail. [Famous last words.]
Mads Kiilerich <madski@unity3d.com> [Fri, 14 Oct 2016 01:53:15 +0200] rev 30181
util: increase filechunkiter size to 128k
util.filechunkiter has been using a chunk size of 64k for more than 10 years,
also in years where Moore's law still was a law. It is probably ok to bump it
now and perhaps get a slight win in some cases.
Also, largefiles have been using 128k for a long time. Specifying that size
multiple times (or forgetting to do it) seems a bit stupid. Decreasing it to
64k also seems unfortunate.
Thus, we will set the default chunksize to 128k and use the default everywhere.
Mads Kiilerich <madski@unity3d.com> [Wed, 12 Oct 2016 12:22:18 +0200] rev 30180
largefiles: always use filechunkiter when iterating files
Before, we would sometimes use the default iterator over large files. That
iterator is line based and would add extra buffering and use odd chunk sizes
which could give some overhead.
copyandhash can't just apply a filechunkiter as it sometimes is passed a
genuine generator when downloading remotely.
Yuya Nishihara <yuya@tcha.org> [Fri, 14 Oct 2016 23:33:00 +0900] rev 30179
revset: for x^2, do not take null as a valid p2 revision
Since we don't count null p2 revision as a parent, x^2 should never return
null even if null is explicitly populated.
Yuya Nishihara <yuya@tcha.org> [Mon, 10 Oct 2016 22:30:09 +0200] rev 30178
revset: make follow() reject more than one start revisions
Taking only the last revision is inconsistent because ancestors(set) follows
all revisions given, and theoretically follow(startrev=set) == ancestors(set).
I'm planning to add a support for multiple start revisions, but that won't fit
to the 4.0 time frame. So reject multiple revisions now to avoid future BC.
len(revs) might be slow if revs were large, but we don't care since a valid
revs should have only one element.
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 15 Oct 2016 17:10:53 -0700] rev 30177
bundle2: only emit compressed chunks if they have data
This is similar to 58467204cac0. Not all calls into the compressor
return compressed data, as the compressor may buffer compressed
output internally. It is cheaper to check for empty chunks than to
send empty chunks through the generator.
When generating a gzip-v2 bundle of the mozilla-unified repo, this
change results in 50,093 empty chunks not being sent through the
generator (out of 1,902,996 total input chunks).