Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 Aug 2018 13:27:33 -0400] rev 39188
test-absorb: stabilize for no-execbit platforms
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 Aug 2018 12:35:47 -0400] rev 39187
test-graft: add a missing output line for Windows
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 Aug 2018 01:07:43 -0400] rev 39186
share: allow more than one level of directories to be created
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 Aug 2018 00:51:26 -0400] rev 39185
clone: allow local cloning to create more than one level of directories
I figure cloning a remote repository is more common, thus it's more likely that
some people might be relying on the less restrictive behavior. Additionally,
`hg init` will also create more than one level of missing directories.
I also have a use case for reading the subrepos from .hgsub, and sharing them
into the normal nested location on the server. SCM Manager doesn't host
subrepos in the normal nested location, which is nice for deduplicating the
repository data, but confuses `hg verify`. Some of the subrepos are in the root
of the repositories, while others are one or two directories deep. So not
having to build up the parent path first is desirable.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 17 Aug 2018 23:28:37 -0400] rev 39184
tests: demonstrate an inconsistency when cloning to a missing directory tree
I noticed that `hg share` is unable to create more than one missing directory on
the path, and thought it was inconsistent with clone. It turns out that the
path for copying/linking the remote store has the same limitation, but cloning
from a remote repo doesn't.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 Aug 2018 23:39:26 -0400] rev 39183
tests: stabilize test-merge-tools.t on Windows
More fun with `hg import --bypass` to work around Windows limitations. The
diffs were generated on Linux, and had a tab to terminate the `+++b/...` lines.
But check-code complained about trailing whitespace, and it seems to run without
them.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 21:26:34 +0000] rev 39182
dagutil: remove module
The previous commit removed the last consumer of this module.
.. api:: dagutil module has been removed
Some functionality has been moved to the dagop module. Other
functionality can be accomplished via revsets.
Differential Revision: https://phab.mercurial-scm.org/D4330
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 21:21:50 +0000] rev 39181
dagop: port revlogdag.linearize() to standalone function
The code should functionally be identical.
We also port the one consumer in changegroup to use the new
standalone function.
After this commit, dagutil is no longer used!
Differential Revision: https://phab.mercurial-scm.org/D4329
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 19:48:52 +0000] rev 39180
dagutil: use revlog.parentrevs() for resolving parent revisions
And remove parents() since it is no longer used.
revlog.parentrevs() is almost the same as parents(). The main
difference is that parentrevs() can return nullrev. dagop.headrevs()
already handles nullrev. We add an inline check for nullrev in the
other call site to account for the difference.
.. api:: parents() removed from dagutil classes
Use parentrevs() on the storage object instead.
Differential Revision: https://phab.mercurial-scm.org/D4328
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 19:45:13 +0000] rev 39179
dagop: extract headsetofconnecteds() from dagutil
The functionality for resolving the set of DAG heads from a
subset simply requires a function to resolve parent revisions.
Let's establish a function in the dagop module to do this, which
seems to be where generic DAG functionality goes these days.
Differential Revision: https://phab.mercurial-scm.org/D4327
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 19:35:24 +0000] rev 39178
setdiscovery: precompute children revisions to avoid quadratic lookup
Moving away from dagutil a few commits ago introduced quadratic
behavior when resolving children revisions during discovery.
This commit introduces a precompute step of the children revisions
to avoid the bad behavior.
I believe the new code should have near identical performance to
what dagutil was doing before. Behavior is still slightly different
because we take into account filtered revisions. But this change was
made when we moved off dagutil.
I added a comment about multiple invocations of this function
redundantly calculating the children revisions. I believe this
potentially undesirable behavior was present when we used dagutil,
as the call to inverse() previously in this function created a new
object and required computing children on every invocation. I thought
we should document the potential for a performance issue rather than
let it go undocumented.
Differential Revision: https://phab.mercurial-scm.org/D4326
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 19:24:36 +0000] rev 39177
dagutil: remove unused classes
We only directly use revlogdag in changegroup code. We don't need
all this abstraction. So remove various classes and levels
of inheritance.
Differential Revision: https://phab.mercurial-scm.org/D4325
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 18:23:47 +0000] rev 39176
setdiscovery: use revset for resolving DAG heads in a subset
This was the final use of dagutil in setdiscovery!
For reasons I didn't investigate, feeding a set with nullrev
into the heads() revset resulted in a bunch of tests failing.
Filtering out nullrev from the input set fixes things.
Differential Revision: https://phab.mercurial-scm.org/D4324
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 19:12:25 +0000] rev 39175
dagutil: remove ability to invert instances
The previous commit removed the last consumer of this feature.
.. api:: remove inverse() methods from classes in dagutil
Differential Revision: https://phab.mercurial-scm.org/D4323
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 18:22:10 +0000] rev 39174
setdiscovery: don't use dagutil for parent resolution
_updatesample()'s one remaining use of revlogdag is for resolving
the parents of a revision.
In 2 cases, we actually resolve parents. In 1, we operate on the
inverted DAG and resolve children.
This commit teaches _updatesample() to receive an argument defining
the function to resolve "parent" revisions. Call sites pass in
changelog.parentrevs() or a wrapper around changelog.children()
accordingly.
The use of children() is semantically correct. But it is quadratic,
since revlog.children() does a range scan over all revisions starting
at its input and effectively calls parentrevs() to build up the list
of children. So calling it repeatedly in a loop is a recipe for
bad performance. I will be implementing something better in a
subsequent commit. I wanted to get the porting off of dagutil done
in a way that was simple and correct.
Like other patches in this series, this change is potentially impacted
but revlogdag's ignorance of filtered revisions. The new code is
filtering aware, since changelog's revs() (used by children() will
skip filtered revisions and therefore hidden children won't appear.
This is potentially backwards incompatible. But no tests fail and
I think this code should respect visibility.
Differential Revision: https://phab.mercurial-scm.org/D4322
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 18:05:36 +0000] rev 39173
setdiscovery: use revsets for computing a subset's heads and roots
revlogdag.headsetofconnecteds() obtains the set of DAG heads in a
given set of revs.
revlogdag.inverse() inverts the DAG order and makes
headsetofconnecteds() obtain the DAG roots in a given subset.
Both of these can be expressed with a revset.
Like other patches in this series, revlogdag uses revlog.index
and thus doesn't take filtering into account. Revsets do. So there
is a chance for regressions with this change. But no tests fail.
And I think this code should take filtering into account since
hidden changesets shouldn't factor into discovery (unless operating
on the hidden repository).
Differential Revision: https://phab.mercurial-scm.org/D4321
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:59:16 +0000] rev 39172
dagutil: remove heads() and localsubset from revlogdag.__init__
The previous commit removed the last consumer of this API.
I'm not going to mark as API incompatible because I doubt anybody
used this functionality (outside of possibly passing an argument
to revlogdag.__init__). I intend to remove revlogdag later in
this series and its API annotation will cover this one.
Differential Revision: https://phab.mercurial-scm.org/D4320
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:54:10 +0000] rev 39171
setdiscovery: pass head revisions into sample functions
This eliminates the last remaining consumer of heads() and
related functionality in dagutil.
Differential Revision: https://phab.mercurial-scm.org/D4319
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:48:15 +0000] rev 39170
setdiscovery: pass heads into _updatesample()
In preparation for eliminating the use of dagutil. Since
_takefullsample() operates on the inverted DAG, it is easier
to have the caller pass in the relevant set instead of teaching
_updatesample() about when to invert the DAG.
We keep the logic identical for now: future commits will remove
dagutil.
Differential Revision: https://phab.mercurial-scm.org/D4318
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:26:07 +0000] rev 39169
setdiscovery: use a revset for finding DAG heads in a subset
The march towards moving away from dagutil continues.
Like other patches moving us away from dagutil, there is the
potential for regressions to occur because revlogdag's
headsetofconnecteds() uses revlog.index, which doesn't take
filtering into account. The revset layer does. But no tests
fail, so we appear to be in the clear.
Differential Revision: https://phab.mercurial-scm.org/D4317
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:21:11 +0000] rev 39168
setdiscovery: reflect use of revs instead of nodes
This code all operates on revision numbers. Update variable names
and comments accordingly.
Differential Revision: https://phab.mercurial-scm.org/D4316
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:15:09 +0000] rev 39167
dagutil: remove descendantset() and ancestorset()
descendantset() is unused after the previous commit. And
ancestorset() was only used by descendantset(), so it can be removed
as well.
.. api:: descendantset() and ancestorset() removed from dagutil
Use a revset instead when operating on the changelog. Or use
various functionality in the ancestor or dagop modules.
Differential Revision: https://phab.mercurial-scm.org/D4315
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:13:26 +0000] rev 39166
setdiscovery: use a revset instead of dagutil.descendantset()
This is the only use of descendantset() in the repo.
Strictly speaking, the revset behaves slightly differently than
dagutil. The reason is that dagutil is using revlog.index for
DAG traversal and this data structure isn't aware of visibility /
filtering. So it can operate on revisions it shouldn't operate on.
But our test coverage of this code is pretty comprehensive and
this change causes no tests to fail. So I think we are good.
Also, the revset parser failed to parse `%ld:: - %ld::`, hence
the use of descendants(). I'm not sure if that is a feature or
a bug.
Differential Revision: https://phab.mercurial-scm.org/D4314
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 20:23:10 +0000] rev 39165
setdiscovery: don't use dagutil to compute heads
This is a relatively trivial operation to perform. We don't
need to use dagutil.
This brings us one step closer to eliminating dagutil.
We still need to populate the heads on revlogdag because later
functionality relies on it.
Differential Revision: https://phab.mercurial-scm.org/D4310
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 20:11:45 +0000] rev 39164
dagutil: remove nodeset()
AFAICT this was unused since the code was introduced by
cb98fed52495
in 2011!
Differential Revision: https://phab.mercurial-scm.org/D4309
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:55:55 +0000] rev 39163
debugcommands: use a revset instead of dagutil
All this code was doing was finding the nodes that are heads
from the ancestors of an input set. This can easily be expressed
with a revset without having to go through dagutil.
This was the last use of ancestorset() outside of dagutil itself.
Differential Revision: https://phab.mercurial-scm.org/D4308
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:51:01 +0000] rev 39162
dagutil: remove internalize() and internalizeall()
The previous commit removed the last consumer.
.. api:: removed internalize() and internalizeall() from dagutil
Use .rev(node) on storage objects to convert nodes to revisions.
Differential Revision: https://phab.mercurial-scm.org/D4307
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:47:30 +0000] rev 39161
setdiscovery: don't use dagutil for node -> rev conversion
The node -> rev conversion is possible using standard storage APIs
and doesn't need to involve the dagutil module.
Differential Revision: https://phab.mercurial-scm.org/D4306
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:40:46 +0000] rev 39160
dagutil: remove externalize() and externalizeall()
They are unused after the previous commit.
.. api:: externalize() and externalizeall() removed from dagutil
Use .node() on a storage primitive to perform revision to node
conversions.
Differential Revision: https://phab.mercurial-scm.org/D4305
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:39:47 +0000] rev 39159
setdiscovery: don't use dagutil for rev -> node conversions
We don't need to use dagutil to perform a simple rev -> node
conversion.
I haven't measured, but the new code is likely faster, as we
avoid extra function calls and avoid some attribute lookups.
Differential Revision: https://phab.mercurial-scm.org/D4304
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:23:24 +0000] rev 39158
exchange: don't use dagutil
We were only using it for simple node -> rev and parent revision
lookups. These are exposed via the storage interface and we don't
need to go through dagutil.
Differential Revision: https://phab.mercurial-scm.org/D4303
Paul Morelle <paul.morelle@octobus.net> [Fri, 20 Jul 2018 13:20:01 +0200] rev 39157
revlog: only consider the span of the delta section
Since the number of snapshots is limited we can exclude them from the logic
checking size and number of reads. Limiting the span computation to the delta
section will allow for further optimization.
Boris Feld <boris.feld@octobus.net> [Mon, 23 Jul 2018 16:21:58 +0200] rev 39156
revlog: ensure intermediate snapshot have decreasing size
If the intermediate snapshot is bigger than the previous one, there is likely
a better snapshot to be made at a different level.
Paul Morelle <paul.morelle@octobus.net> [Wed, 07 Mar 2018 12:28:04 +0100] rev 39155
revlog: bound number of snapshots in a chain
To limit the number of snapshot chained, we enforce them to be smaller and
smaller. This guarantee the number of snapshot in a chain will be bounded to a
small number.
Boris Feld <boris.feld@octobus.net> [Fri, 20 Jul 2018 14:32:56 +0200] rev 39154
revlog: compute snapshot depth on delta info
We need the information to be available when choosing delta.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 12:30:30 +0200] rev 39153
debugrevlog: display snapshot details per depth
This help in understanding the final structure of build manifest. All data
about snapshot (full and intermediate) are gathered into a sub-list for
clarity.
Since we do not produce such snapshots yet, the only thing changing in test
output is the way the information is presented.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 12:09:14 +0200] rev 39152
revlog: add a method to retrieve snapshot depth
Some snapshot property (eg: maximum size) will depend on their depth.
Boris Feld <boris.feld@octobus.net> [Fri, 27 Jul 2018 10:52:43 +0200] rev 39151
debugrevlog: include information about intermediate snapshots
As we are about to create intermediate snapshots, we need to have a way to
debug them. We start by adding very simple debug output and more detailed
output will comes in next changesets.
Paul Morelle <paul.morelle@octobus.net> [Fri, 20 Jul 2018 13:34:48 +0200] rev 39150
revlog: also detect intermediate snapshots
Also detect intermediate-snapshot done against another previous snapshot.
Doing an intermediate snapshot instead of a full one can reduce the number of
full snapshots we need. They are especially useful for content with a lot of
churn on the same line (eg: the manifest) where having a delta over multiple
revisions can end up being significantly smaller than the sum of these
revision deltas.
A revlog built using intermediate snapshots can be a bit smaller and reuse
snapshot much more efficiently. This last property is useful combined with
constraints on chain length. Using intermediate snapshot can produce
repository with delta chain ten times shorter without impact on the storage
size. Shorter chain lengths are faster to restore, greatly improving read
performance.
This changesets (and the following ones) focus on getting the core principle
of intermediate snapshots into Mercurial core. Later changeset will introduce
the strategy to create them.
Paul Morelle <paul.morelle@octobus.net> [Fri, 20 Jul 2018 13:32:17 +0200] rev 39149
revlog: add a method to tells whether rev is stored as a snapshot
For now we only have one type of snapshot: full snapshot versus nullrev.
However we are looking into adding intermediate snapshot where a large diff
against another snapshot is performed instead of storing a full new text.
The conditional is a bit strange and is done in order to help readability of a
some later changesets.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 15:20:44 +0200] rev 39148
debugrevlog: fix for non-manifest object
The `filelog` object is no longer an actual revlog. Instead, the actual revlog
is stored in the `_revlog` attribute.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 16:11:35 -0700] rev 39147
merge with stable
Boris Feld <boris.feld@octobus.net> [Fri, 17 Aug 2018 20:35:52 +0200] rev 39146
remotephase: avoid full changelog iteration (
issue5964)
Changeset
88efb7d6bcb6 introduced a performance regression by triggering a
full ancestors walk.
This changeset reworks this logic so that we no longer walk down the full
changelog. The motivation for
88efb7d6bcb6,
issue5939, is still fixed.
mercurial compared to a draft repository
----------------------------------------
8eeed92475d5: 0.012637 seconds
88efb7d6bcb6: 0.202699 seconds (x16)
46da52f4b820: 0.215551 seconds (+6%)
this code: 0.008397 seconds (-33% from base)
The payload size reduction we see in `test-bookmarks-pushpull.t` comes from a
more aggressive filter of nullid and is harmless.
Kyle Lippincott <spectral@google.com> [Fri, 17 Aug 2018 15:32:38 -0700] rev 39145
nodes: expand/comment the magic nodes so they are more easily searchable
We just encountered `
000000000000modified`, and it was quite annoying to search
for these, even though I knew they existed. For those that don't know that they
exist, this is essentially impossible to search for :)
(Technically we encountered it in its hex form,
3030303030303030303030306d6f646966696564, so I'm adding comments with those
forms in case that's helpful to people in the future).
Differential Revision: https://phab.mercurial-scm.org/D4331
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 13:07:33 +0900] rev 39144
revlog: obtain the first node at the lowest layer while building pure nodemap
Just for clarity. This doesn't matter in practice since changelog.nodemap
is accessed *before* filtered revisions get ready.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 12:54:50 +0900] rev 39143
revlog: fix pure nodemap to not access missing index entry
This bug was revealed by
a3dacabd476b and
a1f934573c0b.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 12:48:44 +0900] rev 39142
changelog: remove copy of revlog.nodemap()
It's been there since 2012, "clfilter: introduce `filteredrevs` attribute
on changelog." I don't think we can apply changelog filtering to nodemap at
this level, so this patch removes the nodemap stub completely.
Boris Feld <boris.feld@octobus.net> [Fri, 17 Aug 2018 16:00:32 -0700] rev 39141
remotephase: fast path newheads computation in simple case (
issue5964)
Changeset
88efb7d6bcb6 fixed the logic of `phases.newheads` but greatly
regressed its performance (up to many order of magnitude). The first step to
fix the regression is to exit early when there is no work to do. If there are
no heads to filter or not roots to filter them, we don't have to do any work.
This fixes the regression when talking to an all public changeset. The
performance is even better than before.
pypy, compared to an all public repo
------------------------------------
8eeed92475d5: 0.005758 seconds
88efb7d6bcb6: 0.602517 seconds (x104)
this code: 0.001508 seconds (-74% from base)
mercurial compared to an all public repo
----------------------------------------
8eeed92475d5: 0.000577 seconds
88efb7d6bcb6: 0.185316 seconds (x321)
this code: 0.000150 seconds (-74% from base)
The performance of newheads, when actual computations are required, is fixed
in the next changeset.
Boris Feld <boris.feld@octobus.net> [Fri, 17 Aug 2018 17:51:06 +0200] rev 39140
perf: add a perfphasesremote command
This command measure the time we spend analysing remote phase during push and
pull and display some information relevant to this computation.
The `test-contrib-perf.t` expected output has to be updated but I do need
these module for this perf command.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 14:43:40 +0200] rev 39139
sparse-revlog: fix delta validity computation
When considering the validity of a delta with sparse-revlog, we check the size
of the largest read. To do so, we use some regular logic with the extra delta
information. Some of this logic was not handling this extra delta properly,
confusing it with "nullrev". This confusion with nullrev lead to wrong results
for this computation but preventing a crash.
Changeset
781b2720d2ac on default revealed this error, crashing. This
changeset fixes the logic on stable so that the computation is correct (and
the crash is averted).
The fix is made on stable as this will impact 4.7 clients interacting with
sparse-revlog repositories (eg: created by later version).
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:51:05 +0900] rev 39138
branchmap: explicitly convert file into iterator
Follows up
2a4bfbb52111. This is required for httprangereader, which is not
an iterable itself.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:25:39 +0900] rev 39137
branchmap: close cache file properly
Follows up
2a4bfbb52111.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:24:29 +0900] rev 39136
branchmap: strip '\n' read from cache file as before
Follows up
2a4bfbb52111.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:21:25 +0900] rev 39135
rebase: do not pass in user option to rollback in-memory merge conflict
Nothing passed before
e9e742bd0501.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:19:17 +0900] rev 39134
rebase: add test for in-memory merge conflicts
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 18:53:51 +0000] rev 39133
rebase: call _dorebase() properly
This fixes a regression from
e9e742bd0501 where we failed to pass
all necessary arguments to _dorebase().
Differential Revision: https://phab.mercurial-scm.org/D4302
Pulkit Goyal <pulkit@yandex-team.ru> [Thu, 16 Aug 2018 16:59:40 +0300] rev 39132
context: make sure file is not deleted while checking path conflicts
If a file is deleted and a directory of same name is created in the same commit,
IMM thinks of that as a file conflict, however the file is deleted and hence the
directory can be created.
The test change demonstrate the fix.
Differential Revision: https://phab.mercurial-scm.org/D4300
Pulkit Goyal <pulkit@yandex-team.ru> [Thu, 16 Aug 2018 16:53:48 +0300] rev 39131
tests: demonstrate that IMM needs to be smarter with path conflicts
When we try to rebase a commit which deletes an existing file and make a
directory of the same name, rebase with IMM aborts. It should work fine just
like the without IMM case.
Differential Revision: https://phab.mercurial-scm.org/D4299
Pulkit Goyal <pulkit@yandex-team.ru> [Thu, 16 Aug 2018 16:36:32 +0300] rev 39130
tests: don't create new repo inside existing repo in test-rebase-inmemory.t
Differential Revision: https://phab.mercurial-scm.org/D4298
Augie Fackler <augie@google.com> [Wed, 25 Jul 2018 13:40:42 -0400] rev 39129
tests: remove test-py3-commands.t
This was a smoke test for early in the Python 3 porting effort, before
anything actually worked. Now that we've got over half the testsuite
passing, this test has outlived its utility.
Differential Revision: https://phab.mercurial-scm.org/D4288
Augie Fackler <augie@google.com> [Wed, 25 Jul 2018 13:41:21 -0400] rev 39128
tests: update test-check-py3-compat.t output in the py3exe branch
This hasn't been maintained in a while, it looks like.
Differential Revision: https://phab.mercurial-scm.org/D4289
Kyle Lippincott <spectral@google.com> [Wed, 15 Aug 2018 17:40:21 -0700] rev 39127
overlayworkingctx: fix exception in metadata-only inmemory merges (
issue5960)
If there was a metadata-only mutation, such as +x or -x on a file, we would
create a cache entry with None for data, and this would cause problems later on
when some code tried to run fctx.data() or similar, and was expecting a string.
My original fix for this involved passing data=self._wrappedctx[path].data() in
setflags(), but this version seems slightly better - this way, if we ever call
write() and then call setflags(), we don't destroy the data that we wrote that's
in the cache. I haven't verified that other fields aren't destroyed, such as
date or flags :)
Differential Revision: https://phab.mercurial-scm.org/D4287
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 22:20:28 +0900] rev 39126
filemerge: show actual capabilities of internal merge tools
This information is useful to know which internal merge tools can be
applied safely on binary files and/or symlinks.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 15 Aug 2018 22:24:50 +0900] rev 39125
filemerge: add config knob to check capabilities of internal merge tools
For historical reason, Mercurial assumes capabilities of internal
merge tools as below while examining rules to decide merge tool,
regardless of actual capabilities of them.
=============== ====== ========
specified via binary symlinks
=============== ====== ========
--tool o o
HGMERGE o o
merge-patterns o (*) x (*)
ui.merge x (*) x (*)
=============== ====== ========
This causes:
- unintentional internal merge tool is chosen for binary files via
merge-patterns section of configuration file
- explicit configuration of internal merge tool for symlinks is
ignored unintentionally
But on the other hand, simple "check capability strictly" might break
backward compatibility (e.g. existing merge automations), because it
changes the result of merge tool selection.
Therefore, this patch adds config knob "merge.strict-capability-check"
to control whether capabilities of internal merge tools should be
checked strictly or not.
If this configuration is true, capabilities of internal merge tools
are checked strictly in (*) cases above.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 15 Aug 2018 22:24:38 +0900] rev 39124
filemerge: show warning if chosen tool has no binary files capability
While matching patterns in "merge-patterns" configuration, Mercurial
silently assumes that all merge tools have binary files
capability. This implementation comes from
5af5f0f9d724 (or Mercurial
1.0).
At failure of merging binary files with incorrect internal merge tool,
there is no hint about this silent ignorance of binary files
capability.
This patch shows warning message, if chosen internal merge tool has no
binary files capability. This will help users to investigate why a
binary file isn't merged as expected.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 20:15:51 +0900] rev 39123
filemerge: add the function to examine a capability of a internal tool
For "symlink" and "binary" capabilities, _toolbool() can not examine
these of internal merge tools strictly, because it examines only
configurations in "merge-tools" section.
Users can configure them explicitly as below for example, but this is
not ordinary usage and not convenient:
[merge-tools]
:other.symlink = true
:other.binary = true
This patch adds hascapability() internal function, which can examine
actual capabilities of a internal merge tool strictly.
At this patch, hascapability() is still used with "strict=False".
Subsequent patches use it with "strict=True".
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 20:08:27 +0900] rev 39122
filemerge: set actual capabilities of internal merge tools
This information is used to detect actual capabilities of internal
merge tools by subsequent patches.
For convenience, this patch assumes that merge tools typed as
"nomerge" have both binary files and symlinks capabilities.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 20:05:36 +0900] rev 39121
help: describe more detail about capabilities while deciding merge tool
"hg help merge-tools" describes as below:
(internal merge tools) will by default not handle symlinks or
binary files.
But in some cases, Mercurial assumes that internal merge tools have
one or both of these capabilities.
"hg help merge-tools" also describes as below, for matching patterns in
merge-patterns configuration section. But this is not sufficient.
Here, binary capabilities of the merge tool are not considered.
This patch describes more detail about capabilities while deciding
merge tool.
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:50:53 -0400] rev 39120
tests: un-glob patchbomb test lines that were only globbing a hostname
Differential Revision: https://phab.mercurial-scm.org/D4296
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:42:04 -0400] rev 39119
tests: force a stable hostname in patchbomb tests
No visible output changes in this commit because everything is globbed
away, but on Python 3 the stable hostname will mean that the headers
don't trigger RFC2822 multi-line mode because they'll always be
consistently short.
Differential Revision: https://phab.mercurial-scm.org/D4295
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:40:20 -0400] rev 39118
patchbomb: allow using HGHOSTNAME to force a hostname
I'll update run-tests.py to set this globally to stabilize some
tests. The variable name is intentionally generic because I suspect we
should generalize this to other tests.
Differential Revision: https://phab.mercurial-scm.org/D4294
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:39:32 -0400] rev 39117
patchbomb: extract function for generating message-id
Differential Revision: https://phab.mercurial-scm.org/D4293
Sushil khanchi <sushilkhanchi97@gmail.com> [Wed, 15 Aug 2018 11:27:57 +0530] rev 39116
rebase: cover restorestatus() by lock to prevent it from being updated
To prevent it from being updated by another process
`restorestatus()` is moved under lock.
Differential Revision: https://phab.mercurial-scm.org/D4282
Martijn Pieters <mj@octobus.net> [Mon, 13 Aug 2018 21:22:14 +0100] rev 39115
branchmap: load branchmap as an iterable
This avoids reading all the file into memory if the cache turns out to be
invalid.
Differential Revision: https://phab.mercurial-scm.org/D4281
Martijn Pieters <mj@octobus.net> [Mon, 13 Aug 2018 20:31:01 +0100] rev 39114
perf: time loading branchmap caches
Differential Revision: https://phab.mercurial-scm.org/D4280
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 00:13:41 +0000] rev 39113
tests: add conditional output when simplestore extensions is loaded
This drops the number of failures with this extension to 3.
Differential Revision: https://phab.mercurial-scm.org/D4286
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 00:11:35 +0000] rev 39112
tests: conditionalize extension tests for extra extensions
If extra extensions are loaded (e.g. via --extra-config-opt),
the tests conditionalized in this commit fail in ways that
are dependent on the extensions that are loaded. So let's
skip them when that scenario is present.
This drops the number of failures for the simplestorerepo.py
extension to 4.
Differential Revision: https://phab.mercurial-scm.org/D4285
Yuya Nishihara <yuya@tcha.org> [Sat, 07 Jul 2018 22:40:39 +0900] rev 39111
commit: try hard to reuse p1 manifest if nothing changed
This is all for commit reproducibility on "hg convert".
With this change, p1 manifest is reused if ctx.files() *to be committed* is
empty, and if new manifest entry is identical to p1. This is important
property for "hg convert" since memctx.files() built from a convert source
may be either a) more narrowed thanks to a committed ctx.files() which
provides more accurate status, or b) containing redundant files because of
sloppy filtering on e.g. octopus merge.
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 18:44:42 +0900] rev 39110
merge: add tests for commit with no content change
It isn't easy to say when to reuse the p1 manifest. Basically, that's only
when wctx.files() is empty, but we need to know that wctx.files() is not
the same as repo['.'].files() after the commit.
This patch adds several examples of commits with empty ctx/wctx.files().
I don't think this is exhaustive, but it contains at least one failure
mode in which a converted repo result in a different hash.
I also note that the manifest revlog does NOT follow the DAG shape of the
changelog since p1 manifest is reused if wctx.files() is empty even at merge.
I don't know whether it is intentional or not, but it's the behavior since
2011,
301725c3df9a "localrepo: reuse parent manifest in commitctx if no files
have changed."
Yuya Nishihara <yuya@tcha.org> [Sat, 07 Jul 2018 22:32:49 +0900] rev 39109
commit: add debug message regarding manifest reuse
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 12:16:08 +0900] rev 39108
mail: convert encoding.encoding to sysstr
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 12:14:28 +0900] rev 39107
mail: call s.decode('ascii') explicitly to see if s is an ascii bytes
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 12:11:36 +0900] rev 39106
mail: pass in addr to _addressencode() in bytes
_addressencode() doesn't process a unicode addr well. For example,
acc.encode('ascii') would raise UnicodeEncodeError if acc was a unicode.
This patch temporarily restores the pre-
952bf3c948f0 _addressencode().
Several tests would be broken on Python 3.
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 12:01:34 +0900] rev 39105
mail: remove redundant bytesurl() from addrlistencode()
_addressencode() returns a bytes.
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 11:43:24 +0900] rev 39104
patchbomb: really use BytesGenerator API
It wraps around Generator type.
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 14:03:22 +0900] rev 39103
byteify-strings: prevent "__name__ == '__main__'" from being transformed
This was okay for import-time code transformer, but shouldn't be applied to
source code.
Augie Fackler <augie@google.com> [Wed, 15 Aug 2018 17:06:12 -0400] rev 39102
merge with stable
Yuya Nishihara <yuya@tcha.org> [Wed, 15 Aug 2018 10:57:47 +0900] rev 39101
rebase: turn off inmemory flag on --stop
Guessing from the inline comment, "in-memory rebase is not compatible with
resuming rebases", in-memory rebasing should be disabled.
Yuya Nishihara <yuya@tcha.org> [Wed, 15 Aug 2018 10:47:07 +0900] rev 39100
rebase: use action variable to select things to do
Yuya Nishihara <yuya@tcha.org> [Wed, 15 Aug 2018 10:41:26 +0900] rev 39099
rebase: consolidate check for --dry-run/--confirm on abort/stop/continue
Yuya Nishihara <yuya@tcha.org> [Wed, 15 Aug 2018 10:36:06 +0900] rev 39098
rebase: consolidate check for --abort/--stop/--continue conflicts
Yuya Nishihara <yuya@tcha.org> [Wed, 15 Aug 2018 10:32:10 +0900] rev 39097
rebase: do not set internal dryrun flag by --confirm
This prepares for cleaning up option checking.
Sushil khanchi <sushilkhanchi97@gmail.com> [Fri, 27 Jul 2018 14:30:28 +0530] rev 39096
rebase: add test to show --stop moves bookmarks
Added tests to show that --stop option moves bookmarks
of rebased csets to their succs.
Differential Revision: https://phab.mercurial-scm.org/D3982
Sushil khanchi <sushilkhanchi97@gmail.com> [Sun, 22 Jul 2018 22:08:14 +0530] rev 39095
rebase: include --stop option in documentation
Differential Revision: https://phab.mercurial-scm.org/D3970
Sushil khanchi <sushilkhanchi97@gmail.com> [Sun, 22 Jul 2018 22:04:49 +0530] rev 39094
rebase: raise error for options conflicting with --stop
Make sure --stop feature raise error for conflicting options
--continue, --abort, --confirm, --dry-run. Tests are added to
reflect the same.
Differential Revision: https://phab.mercurial-scm.org/D3969
Sushil khanchi <sushilkhanchi97@gmail.com> [Wed, 18 Jul 2018 00:25:52 +0530] rev 39093
rebase: add --stop option to stop rebase at any point (
issue5206)
Before this patch, during a rebase if you get a point where you can't
figure out what to do next, then either you had to complete your series
or abandon all the work you have done during this rebase.
Now, with this feature you can stop at any point by keeping the rebased
csets and mark original csets as obsolete. And if you don't have evolution
extension enabled then you can use --keep option as an alternative which
will keep original csets too, instead of marking them obsolete.
Differential Revision: https://phab.mercurial-scm.org/D3959
Boris Feld <boris.feld@octobus.net> [Fri, 01 Jun 2018 11:36:06 +0200] rev 39092
statprof: cleanup string construction
Use string substitutions instead of string additions.
Boris Feld <boris.feld@octobus.net> [Wed, 25 Jul 2018 17:11:59 +0200] rev 39091
addremove: add associated color for the new labels
We use classic color: green is added, red is removed.
Boris Feld <boris.feld@octobus.net> [Tue, 14 Aug 2018 11:39:48 +0200] rev 39090
remove: add a label for messages about removed files
Keeping consistency between addremove and remove seems a good idea.
Boris Feld <boris.feld@octobus.net> [Tue, 14 Aug 2018 11:15:05 +0200] rev 39089
add: add a label for messages about added files
Keeping consistency between addremove and add seems a good idea.
Boris Feld <boris.feld@octobus.net> [Wed, 25 Jul 2018 17:11:52 +0200] rev 39088
addremove: add labels for messages about added and removed files
This is the first step to add color for add and remove output.
Boris Feld <boris.feld@octobus.net> [Fri, 27 Jul 2018 12:08:10 +0200] rev 39087
revlog: filter out "invalid" delta base candidates
There are bases that we know won't produce a good delta chain. For example, if
the base is already bigger than twice the size of the text we store, we know
the resulting delta chain will never be valid.
We might make the check a bit more powerful and generic in the future, but
this looks a good start.
In particular, empty file (size 0) will never find a good base, so we should
stop spending time trying to find one.
Boris Feld <boris.feld@octobus.net> [Tue, 14 Aug 2018 13:47:07 -0700] rev 39086
revlog: use a symbolic constant for the deltas/text ration
We are about to use it somewhere else, we should as well make it easier to
access and change.
Boris Feld <boris.feld@octobus.net> [Fri, 27 Jul 2018 14:37:31 +0200] rev 39085
revlog: do not search for delta for empty content
We won't find any useful base to store an empty text. So we should not even try
to.
Boris Feld <boris.feld@octobus.net> [Tue, 14 Aug 2018 13:44:13 -0700] rev 39084
revlog: split and document good delta conditional
The logic is still identical, but having each conditional on its own helps to
document them and will help to edit them in the future.
Boris Feld <boris.feld@octobus.net> [Fri, 27 Jul 2018 19:09:41 +0200] rev 39083
revlog: skip over empty revision when looking for delta base
There is no point in keeping an empty revision in the middle of a chain since
they won't contribute any data. So we filter them out when looking for a valid
base delta.
Boris Feld <boris.feld@octobus.net> [Sat, 28 Jul 2018 14:14:59 +0200] rev 39082
debugrevlog: display details about empty revision
We now display details about the course of emptiness (empty text or empty
delta).
Boris Feld <boris.feld@octobus.net> [Fri, 27 Jul 2018 19:19:52 +0200] rev 39081
debugrevlog: track empty revlog entries
A revlog entry can be "empty" for two reasons:
* the stored text is empty
* we found another entry with the exact same content
Tracking them separately make revlog content clearer.
Boris Feld <boris.feld@octobus.net> [Fri, 27 Jul 2018 10:59:59 +0200] rev 39080
debugrevlog: document some of the variable used
This help to understand the code.
Boris Feld <boris.feld@octobus.net> [Tue, 14 Aug 2018 13:36:41 -0700] rev 39079
revlog: drop compatibility for `revlog.descendant` (API)
As advertised, the method is dropped in 4.8.
Boris Feld <boris.feld@octobus.net> [Tue, 14 Aug 2018 13:36:11 -0700] rev 39078
context: drop compatibility for `context.descendant` (API)
As advertised, the method is dropped in 4.8.
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Aug 2018 09:20:44 +0900] rev 39077
cmdutil: remove _updatecleanmsg() which is no longer used anywhere
Yuya Nishihara <yuya@tcha.org> [Fri, 10 Aug 2018 09:18:37 +0900] rev 39076
cmdutil: fix weird indent in _mergemsg()
Matt Harbison <matt_harbison@yahoo.com> [Fri, 10 Aug 2018 00:28:07 -0400] rev 39075
cext: fix a warning about differing const qualifiers on Windows
The prototype takes a 'void *', but self->offsets is a 'const char **'.
mercurial/cext/revlog.c(2011) : warning C4090: 'function' : different
'const' qualifiers
Matt Harbison <matt_harbison@yahoo.com> [Fri, 10 Aug 2018 00:14:47 -0400] rev 39074
cext: fix most truncation warnings in revlog on Windows
There's one more, and I'm not sure why it isn't being tripped on other
platforms:
mercurial/cext/revlog.c(430) : warning C4244: '=' : conversion from
'Py_ssize_t' to 'char', possible loss of data
Matt Harbison <matt_harbison@yahoo.com> [Thu, 09 Aug 2018 23:52:45 -0400] rev 39073
cext: fix Windows warning about implicit conversion of 32-bit shift to 64 bit
mercurial/cext/revlog.c(1541) : warning C4334: '<<' : result of 32-bit shift
implicitly converted to 64 bits (was 64-bit shift int ended?)
Martin von Zweigbergk <martinvonz@google.com> [Thu, 09 Aug 2018 00:09:03 -0700] rev 39072
index: make capacity argument to nt_init be measured in revisions
The nodetree's internal capacity field is measures in nodetree nodes,
which is not something the caller should have to know about.
Differential Revision: https://phab.mercurial-scm.org/D4166
Martin von Zweigbergk <martinvonz@google.com> [Wed, 08 Aug 2018 23:41:50 -0700] rev 39071
index: avoid duplicating capacity-growth expression
We were duplicating the "*2" instead of reusing it. It's overflow-safe
to reuse as long as the growth factor (i.e. currently 2) is not larger
than sizeof(nodetreenode) (currently 64 or 128).
Differential Revision: https://phab.mercurial-scm.org/D4165
Martin von Zweigbergk <martinvonz@google.com> [Wed, 08 Aug 2018 23:36:15 -0700] rev 39070
index: move check for too large capacity into nt_init()
It's clearer to have the check just before the allocation happens.
Differential Revision: https://phab.mercurial-scm.org/D4164
Martin von Zweigbergk <martinvonz@google.com> [Wed, 08 Aug 2018 22:26:57 -0700] rev 39069
shortest: don't include nullid in disambigution revset
As noted by Yuya in the review of D4118, the fact that the nodetree
always contains the nullid is not what we want for the disambiguation
case. It's not what the pure version of the code does.
Note that the updated test also passes before this patch, but it
wouldn't have passed in native-code mode once we start using the
nodetree for disambiguating nodeid prefixes.
Differential Revision: https://phab.mercurial-scm.org/D4163
Martin von Zweigbergk <martinvonz@google.com> [Wed, 08 Aug 2018 21:52:39 -0700] rev 39068
index: don't include nullid in the internal "length" field
This is a follow-up to
781b2720d2ac (index: don't include nullid in
len(), 2018-07-20). I intentionally didn't update the "index stats",
so it will also not include the space for nullid after this patch.
Differential Revision: https://phab.mercurial-scm.org/D4162
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 13:20:42 -0400] rev 39067
tests: add test coverage for revlogindex[-1] which was previously missing
Differential Revision: https://phab.mercurial-scm.org/D4184
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 01 Aug 2018 16:28:10 +0300] rev 39066
context: safegaurd from 'lx' being passed as file flag in manifest
Follows up for
d558e53cd6b619d5f671a7580513bfc66a398eac.
Differential Revision: https://phab.mercurial-scm.org/D4259
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 03:33:38 -0400] rev 39065
debugcommands: urlerror only has a read() method in Python 2
Differential Revision: https://phab.mercurial-scm.org/D4258
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 03:31:23 -0400] rev 39064
debugcommands: get_method should always return a sysstr
Differential Revision: https://phab.mercurial-scm.org/D4257
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 03:16:02 -0400] rev 39063
stringutil: if we get a memoryview in escapestr, coerce it to bytes
Otherwise we get an exception. Sadly, this manifesting deep inside the
wireproto code, inside a future. For some reason the exception was
/causing a hang/ rather than actually propagating out, which seems
like it might merit some investigation in the future.
Differential Revision: https://phab.mercurial-scm.org/D4256
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 03:14:52 -0400] rev 39062
debugcommands: fix a missing b prefix
This wasn't causing any immediate problems because of the source
transformer, but I noticed the inconsistency and it bugged me.
# skip-blame just a b prefix
Differential Revision: https://phab.mercurial-scm.org/D4255
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 02:31:44 -0400] rev 39061
wireprotohelpers: add missing b prefixes
# skip-blame just b prefixes
Differential Revision: https://phab.mercurial-scm.org/D4254
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 02:30:14 -0400] rev 39060
util: fix signature of setsockopt in socket observer
The format string expected the result, callers all passed the result,
but we didn't actually accept the result to this function. I swear
I've looked at this code a dozen times trying to find the problem, and
it was this simple all along.
Differential Revision: https://phab.mercurial-scm.org/D4253
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 02:03:26 -0400] rev 39059
tests: port test-glog.t to Python 3 by using extracted printrevset
Differential Revision: https://phab.mercurial-scm.org/D4252
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 02:02:40 -0400] rev 39058
tests: extract printrevset extension from test-glog-beautifygraph.t
Differential Revision: https://phab.mercurial-scm.org/D4251
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 02:18:41 -0400] rev 39057
tests: add missing b'' prefix in test-check-code.t
It now passes on Python 3.
# skip-blame b prefix
Differential Revision: https://phab.mercurial-scm.org/D4250
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 02:17:50 -0400] rev 39056
contrib: have check-code look at files in latin1 instead of ascii
This way all files open. So far none of our patterns look like they'll care.
Differential Revision: https://phab.mercurial-scm.org/D4249
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 00:37:24 -0400] rev 39055
tests: port inline extension in test for beautifygraph to Python 3
test-glog-beautifygraph.t now passes on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4247
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 00:08:06 -0400] rev 39054
beautifygraph: use slicing instead of subscripting on bytestr
Differential Revision: https://phab.mercurial-scm.org/D4246
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 00:07:45 -0400] rev 39053
beautifygraph: use sysstr for checking encoding._wide
# skip-blame just an r prefix
Differential Revision: https://phab.mercurial-scm.org/D4245
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 01:42:01 -0400] rev 39052
stringutil: have buildrepr delegate to pprint for unknown types
This stabilizes some revset reprs across Python 2 and 3.
Differential Revision: https://phab.mercurial-scm.org/D4244
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 01:41:31 -0400] rev 39051
stringutil: teach pprint about sets
This is the old (Python 2) way of printing sets. I actually prefer the
Python 3 version of the repr, but this will result in less test churn
in the short term.
Differential Revision: https://phab.mercurial-scm.org/D4243
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 00:12:55 -0400] rev 39050
parser: replace bespoke _brepr with stringutil.pprint
Differential Revision: https://phab.mercurial-scm.org/D4242
Augie Fackler <augie@google.com> [Fri, 10 Aug 2018 02:06:46 -0400] rev 39049
mq: use stringutil.pprint instead of pycompat.byterepr
Resolves a few output changes on Python 3 in test-mq.t, and gets us
two new passing tests.
Differential Revision: https://phab.mercurial-scm.org/D4248
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 13:27:05 -0400] rev 39048
tests: remove expected output of test-parseindex2.py
Everything uses unittest idioms now, so this can go away.
Differential Revision: https://phab.mercurial-scm.org/D4185
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 13:13:16 -0400] rev 39047
parsers: adjust pure-python version to mimic
a3dacabd476b
This was caught by the tests, to my surprise. I'll also follow up with
a test of the index[-1] behavior so we can be sure that remains
consistent, as I think that currently has no coverage.
Differential Revision: https://phab.mercurial-scm.org/D4183
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 13:34:33 -0400] rev 39046
tests: restore Python 3 compat in test-parseindex2.py
I think this was previously getting lucky in Python 3, and would have
stacktraced if something failed. Now that failure messages are eagerly
computed, this turned into a problem.
Differential Revision: https://phab.mercurial-scm.org/D4182
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 13:26:28 -0400] rev 39045
tests: port remaining bits of test-parseindex2 to unittest asserts
Differential Revision: https://phab.mercurial-scm.org/D4181
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 23:13:56 -0400] rev 39044
tests: allow more detailed error message from re.compile
test-grep.t now passes on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4240
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 23:13:35 -0400] rev 39043
grep: include format string in columns list
This lets us correctly use %d as needed.
Differential Revision: https://phab.mercurial-scm.org/D4239
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 23:13:01 -0400] rev 39042
grep: coerce username to bytestr, not str
Differential Revision: https://phab.mercurial-scm.org/D4238
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 23:12:44 -0400] rev 39041
grep: difflib sequencematcher opcodes are native strs
Differential Revision: https://phab.mercurial-scm.org/D4237
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 21:59:32 -0400] rev 39040
tests: allow for a bonus newline in base64'd email payload
Python 3 adds this newline, which is harmless.
test-patchbomb.t now passes on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4234
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 21:58:43 -0400] rev 39039
mail: be more cautious about bytes vs str for py3 compat
It's suboptimal that we get a bytes on 2 and a unicode on 3, but it's
easy to work with and I'm disinclined to change anything unless we
start using some sort of type inferencer.
Differential Revision: https://phab.mercurial-scm.org/D4233
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 21:57:46 -0400] rev 39038
patchbomb: use native strings when determining attachment disposition
Differential Revision: https://phab.mercurial-scm.org/D4232
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 21:04:15 -0400] rev 39037
mail: always fall back to iso-8859-1 if us-ascii won't work (BC)
It looks like this was a well-intentioned backwards compat hack for
previewing the output of `hg email` in a stable way. Unfortunately I
think this hack's time has come, because Python 3 does a much better
job of ensuring it actually emits *valid* email messages. In
particular, Python 2 would blindly trust us that the bytes we handed
it were valid for the encoding we claimed, but Python 3 has some more
sniff-tests that we end up failing.
As a result, if we're going to print an email to the terminal, try
us-ascii first, but if that fails go straight to iso-8859-1 which
should be reasonably readable for ascii-compatible patch bodies. This
*will* be a breaking change for ascii-incompatible textual patch
content, but I don't think that's avoidable if we want to continue
using the email library from the stdlib.
.. bc::
Emails from the patchbomb extension will always be printed as though
they are iso-8859-1 if they're not valid us-ascii. Previously,
previewed emails were always claimed to be us-ascii and might
contain invalid byte sequences.
Differential Revision: https://phab.mercurial-scm.org/D4231
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 20:57:27 -0400] rev 39036
tests: put some Python 3 polish on inline Python invocations
A couple of these became inline python sessions because they got more
involved, and one of them is super-annoying and writes directly to
sys.stdout.buffer because I just couldn't make it work any other way.
Differential Revision: https://phab.mercurial-scm.org/D4230
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 19:39:33 -0400] rev 39035
patchbomb: don't unintentionally duplicate headers
This dict-like of headers is only mostly dict-like: if you set a key
that already exists, you it appends another value, rather than
replacing the one that was already present.
This fixes test-patchbomb-bookmark.t on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4229
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 19:37:57 -0400] rev 39034
patchbomb: use sysstrs when describing content-type
Differential Revision: https://phab.mercurial-scm.org/D4228
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 18:28:43 -0400] rev 39033
patchbomb: use native strs for email header keys and values
Differential Revision: https://phab.mercurial-scm.org/D4223
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 18:28:19 -0400] rev 39032
patchbomb: use email.encoders instead of email.Encoders
No idea when this became a thing, but it exists for me in both 2.7 and 3.6.
Differential Revision: https://phab.mercurial-scm.org/D4222
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 18:27:36 -0400] rev 39031
mail: properly handle email addresses typically being unicodes
Visible progress in test-patchbomb.t on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D4221
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 20:08:30 -0700] rev 39030
tests: fix Python3 issues in Python one-liners in test-patchbomb.t
# skip-blame just byte prefixes
Differential Revision: https://phab.mercurial-scm.org/D4220
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 17:46:07 -0400] rev 39029
dummysmtpd: accept additional kwargs from stdlib smtpd
This was causing extremely mysterious failures because smtpd is
implemented using asynchat, which in turn doesn't appear to do
anything *remotely* helpful if an exception occurs.
# no-check-commit because I'm editing a foo_bar function signature
test-patchbomb-tls.t now passes in Python 3, but only because it
doesn't check message bodies like test-patchbomb.t. test-patchbomb.t
is *full* of doubled headers in the output (eg [0]) which seems like
an odd failure mode.
0:
@@ -141,12 +154,17 @@
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
+ MIME-Version: 1.0
+ Content-Type: text/plain; charset="us-ascii"
+ Content-Transfer-Encoding: 7bit
Subject: [PATCH] bookmark
X-Mercurial-Node:
8dab2639fd35f1e337ad866c372a5c44f1064e3c
X-Mercurial-Series-Index: 1
X-Mercurial-Series-Total: 1
Differential Revision: https://phab.mercurial-scm.org/D4219
Augie Fackler <augie@google.com> [Thu, 09 Aug 2018 17:44:36 -0400] rev 39028
mail: fix debug print, which appears to have been broken for a long time
I noticed this when trying to debug very mysterious dummysmtpd
problems. It turns out you can set self.debuglevel to a number greater
than 0 and have smtplib print helpful debug output, but this output on
our side of things was broken.
Differential Revision: https://phab.mercurial-scm.org/D4218
Augie Fackler <augie@google.com> [Mon, 16 Jul 2018 18:17:44 -0400] rev 39027
patchbomb: python 3 really wants those email addresses in unicode
At this point, test-patchbomb-tls.t almost passes on Python 3, but I'm
not really sure what the issue is: we just get a `abort: Connection
unexpectedly closed` that doesn't make sense to me.
Differential Revision: https://phab.mercurial-scm.org/D3957
Augie Fackler <augie@google.com> [Mon, 16 Jul 2018 18:17:19 -0400] rev 39026
mail: stop using the smtplib.SSLFakeFile and use socket.socket.makefile
They're equivalent, and the latter is what Python 3.3 says to use in
the release notes. Turns out it works on Python 2 as well.
Differential Revision: https://phab.mercurial-scm.org/D3956
Augie Fackler <augie@google.com> [Mon, 16 Jul 2018 18:16:26 -0400] rev 39025
mail: modernize check for Python-with-TLS
We used to be going indirectly through the socket module, but now we
just check for the ssl module.
Differential Revision: https://phab.mercurial-scm.org/D3955
Augie Fackler <augie@google.com> [Mon, 16 Jul 2018 17:49:17 -0400] rev 39024
mail: cope with Py3 unicode antics on email addresses
Differential Revision: https://phab.mercurial-scm.org/D3954
Augie Fackler <augie@google.com> [Mon, 16 Jul 2018 17:48:03 -0400] rev 39023
mail: fix _encode to be more correct on Python 3
This code appears to be on the wrong side of the law in Python 2, at
least some of the time. In Python 3, it's definitely wrong in places,
but fortunately that's easy to fix.
Differential Revision: https://phab.mercurial-scm.org/D3953
Augie Fackler <augie@google.com> [Mon, 16 Jul 2018 14:14:27 -0400] rev 39022
patchbomb: work around email module really wanting to write unicode data
Differential Revision: https://phab.mercurial-scm.org/D3951
Matt Harbison <matt_harbison@yahoo.com> [Tue, 14 Aug 2018 14:00:35 -0400] rev 39021
convert: don't drop missing or corrupt tag entries
Cleaning up the tags file could be a useful feature in some cases, so maybe
there should be a switch for this. However, the default hg -> hg convert tries
to maintain identical hashes (thus convert.hg.saverev is off by default, but is
on by default for other source types). It looks like _rewritesubstate() has a
`continue` in it, and therefore a similar problem.
I ran into this conversion divergence when a coworker "merged" two repositories
by copy/pasting all of the files from the source repo and massaging the code,
and forgetting to revert the .hg* files. That silently emptied the .hgtags file
after the conversion. (This isn't the manifest node bug Yuya has been helping
with- this occurred well after the bzr -> hg conversion and wasn't a merge
commit, which made it extra puzzling. That bug is still an issue.)
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 09 Aug 2018 12:25:06 -0700] rev 39020
changegroup: invert conditional and dedent
I don't like else-less blocks that are indented for no reason.
Differential Revision: https://phab.mercurial-scm.org/D4217
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 09 Aug 2018 09:40:29 -0700] rev 39019
changegroup: capture base node instead of rev in delta request
I want the API in storage to be defined in terms of nodes, not
revision numbers.
Differential Revision: https://phab.mercurial-scm.org/D4216
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 09 Aug 2018 09:28:26 -0700] rev 39018
changegroup: introduce requests to define delta generation
Currently, we iterate through each revision we will be producing
a delta for then call into 1 of 2 functions for generating that
delta. Deltas are emitted as we iterate.
A problem with this model is that revision generation is tightly
coupled to the changegroup code. And the storage layer needs to
expose APIs like deltaparent() so changegroup delta generation
can produce a delta with that knowledge.
Another problem is that in this model, deltas can only be produced
sequentially after the previous delta was produced and emitted.
Some storage backends might be capable of producing deltas in
parallel (e.g. if the changegroup deltas are cached somewhere).
This commit aims to solve these problems by turning delta generation
into a 2 phase implementation where the first phase determines
info about all the deltas that need to be generated and the 2nd
phase resolves those deltas.
We introduce a "revisiondeltarequest" object that holds data about
a to-be-generated delta. We perform a full pass over all revisions
whose delta is to be generated and generate a "revisiondeltarequest"
for each. Then we iterate over the "revisiondeltarequest" instances
and derive a "revisiondelta" for each.
This patch was quite large. In order to avoid even more churn, aspects
of the implementation are less than ideal. e.g. we're recording
revision numbers instead of nodes in a few places and we don't
yet have a formal API for resolving an iterable of revisiondeltarequest
instances. Things will be improved in subsequent commits.
Unfortunately, this commit reduces performance substantially. For
`hg perfchangegroupchangelog` on my hg repo:
! wall 1.512607 comb 1.510000 user 1.490000 sys 0.020000 (best of 7)
! wall 2.150863 comb 2.150000 user 2.150000 sys 0.000000 (best of 5)
And for `hg bundle -t none-v2 -a` for the mozilla-unified repo:
178.32user 4.22system 3:02.59elapsed
190.97user 4.17system 3:15.19elapsed
Some of this was attributed to changelog slowdown. `hg
perfchangegroupchangelog` on mozilla-unified:
! wall 21.688715 comb 21.690000 user 21.570000 sys 0.120000 (best of 3)
! wall 25.683659 comb 25.680000 user 25.540000 sys 0.140000 (best of 3)
Profiling seems to reveal that the changelog slowdown is due to reading
changelog revisions multiple times. First in the linknode callback
to resolve the set of files changed. Second in the delta generation.
Before, we likely had hit the last revision cache in the revlog when
doing delta generation since we performed that immediately after
performing the linknode callback.
I'm not exactly sure where the other ~8s are being spent. It might be
from overhead of constructing a few million revisiondeltarequest
objects. I'm OK with the regression for now because it is in service
of a larger cause (storage abstraction). I'll try to profile later
and claw back the performance.
Differential Revision: https://phab.mercurial-scm.org/D4215
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 20:17:48 -0700] rev 39017
changegroup: refactor delta parent code
We had recently abstracted the delta parent functions to
facilitate extracting code from cgpacker. Now that we're in
a better place, it is time to revisit the design.
Changegroup version 1 requires that the previous node be used as the
delta parent. Later versions allow any available node to be used
as the base.
In the case where an arbitrary parent can be used, the choice of
a delta parent is best left in the hands of the storage backend.
So it makes sense for the delta parent selection to be hidden
away in the storage layer. This means deferring the choice of the
delta parent selection function to as close to delta generation
time as possible.
This commit moves the delta selection logic to essentially just
before delta generation. However, because changegroup version 1
limits what we can do, we have retained the ability to force a
delta against the previous revision.
As part of this, I realized that the ellipsis parent function was
unused! That's because ellipsis mode always sends full revisions
and not deltas.
Differential Revision: https://phab.mercurial-scm.org/D4214
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 16:01:26 -0700] rev 39016
changegroup: differentiate between fulltext and diff based deltas
Previously, revisiondelta encoded a delta and an optional prefix
containing a delta header. The underlying code could populate
the delta with either a real delta or a fulltext revision.
Following the theme of wanting to defer serialization of revision
data to the changegroup format as long as possible, it seems
prudent for the revision delta instance to capture what type of
data is being represented. This could possibly allow us to
encode revision data differently in the future. But for the
short term, it makes the behavior of a revisiondelta more
explicit.
Differential Revision: https://phab.mercurial-scm.org/D4213
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 15:28:22 -0700] rev 39015
changegroup: minor cleanups to deltagroup()
Differential Revision: https://phab.mercurial-scm.org/D4212
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 14:44:48 -0700] rev 39014
changegroup: emit revisiondelta instances from deltagroup()
By abstracting the concept of a delta group away from its
serialization (the changegroup format), we make it easier
to establish alternate serialization formats. We also make
it possible to move aspects of delta group generation into
the storage layer. This will allow storage to make decisions
about e.g. delta parent choices without the changegroup code
needing storage APIs to determine delta parents. We're still
a bit of a way from there. Future commits will work towards
that world.
Differential Revision: https://phab.mercurial-scm.org/D4211
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 14:33:33 -0700] rev 39013
changegroup: move file chunk emission to generate()
Same deal as manifests. We want to get to a point where we can
emit data structures from deltagroup() and derive the raw
changegroup data as late as possible.
Differential Revision: https://phab.mercurial-scm.org/D4210
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 15:14:59 -0700] rev 39012
changegroup: move manifest chunk emission to generate()
We want to get to a point where we can emit data structures from
deltagroup() and derive the raw changegroup data as late as possible.
Differential Revision: https://phab.mercurial-scm.org/D4209
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 15:09:12 -0700] rev 39011
changegroup: move size tracking and end of manifests to generate()
Preparing for all the generate* functions to emit data structures
instead of raw chunks.
Differential Revision: https://phab.mercurial-scm.org/D4208
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 14:15:33 -0700] rev 39010
changegroup: emit delta group close chunk outside of deltagroup()
I want to make deltagroup() emit data structures rather than
serialized deltas. Upcoming commits will demonstrate why.
Differential Revision: https://phab.mercurial-scm.org/D4207
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 14:19:02 -0700] rev 39009
changegroup: extract cgpacker.group() to standalone function
It doesn't need to be part of the packer class.
Differential Revision: https://phab.mercurial-scm.org/D4206
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 14:02:31 -0700] rev 39008
changegroup: pass all state into group()
This will allow us to split it into a standalone function.
Differential Revision: https://phab.mercurial-scm.org/D4205
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 08 Aug 2018 13:50:54 -0700] rev 39007
changegroup: inline _prune() into call sites
The functionality is pretty simple. As a bonus, _prune() had special
code for the manifest case. We can now exclude this check from the
file call site.
Differential Revision: https://phab.mercurial-scm.org/D4199
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 15:31:03 -0700] rev 39006
changegroup: inline _packmanifests() into generatemanifests()
It is relatively small. Every other generate*() calls group()
directly. So the new code is consistent.
Differential Revision: https://phab.mercurial-scm.org/D4198
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 15:13:25 -0700] rev 39005
changegroup: invert conditional and dedent
I don't like seeing code that visually resembles the pyramid of
doom.
Differential Revision: https://phab.mercurial-scm.org/D4197
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 15:10:38 -0700] rev 39004
changegroup: make _revisiondeltanarrow() a standalone function
It doesn't require any state on the packer. Everything impacting
behavior is passed in as a function. So split it out, just like
what was done for _revisiondeltanormal().
Differential Revision: https://phab.mercurial-scm.org/D4196
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 15:08:29 -0700] rev 39003
changegroup: pass state into _revisiondeltanarrow
After this, the method no longer accesses self and can be split
into a standalone function.
Differential Revision: https://phab.mercurial-scm.org/D4195
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 14:53:42 -0700] rev 39002
changegroup: inline _close()
Now that it doesn't clear self._clrevtolocalrev on every invocation
and is a simple one-liner that calls another function, we can
do away with this method and inline its content into all call
sites.
Differential Revision: https://phab.mercurial-scm.org/D4194
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 14:52:00 -0700] rev 39001
changegroup: pass clrevtolocalrev to each group
clrevtolocalrev is a per-changegroup group mapping revisions to
aid with shallow clone.
Back when this functionality was implemented in an extension, this
dict was added to the packer instance so monkeypatched functions
could reference it there. Now that this code is part of core, we
can pass the dict to each consumer properly so it doesn't have to
live on the cgpacker instance. This commit does that.
Differential Revision: https://phab.mercurial-scm.org/D4193
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 12:44:56 -0700] rev 39000
changegroup: combine _generatefiles() into generatefiles()
These were split out in
a06aab274aef as part of moving the
narrow code into core. They don't need to be separate
functions.
Differential Revision: https://phab.mercurial-scm.org/D4192
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 15:45:56 -0700] rev 38999
changegroup: define linknodes callbacks in generatefiles()
This is how it is done everywhere else.
But the logic here is a bit more complex because shallow clone
needs to reference the original linknode implementation. But
at least now all function implementations are defined in the
same place.
Differential Revision: https://phab.mercurial-scm.org/D4191
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 10:55:32 -0700] rev 38998
changegroup: track changelog to manifest revision map explicitly
Previously, self._nextclrevtolocalrev was only populated as part
of the changelog lookup callback. But cgpacker._close() was looking
at self._nextclrevtolocalrev on every invocation.
Since self._nextclrevtolocalrev is for communicating the mapping
of changelog revisions to manifest revisions, this commit refactors
the code to make that explicit.
The changelog state now stores this mapping. And after the changelog
group is emitted, we update self._clrevtolocalrev with that dict.
self._nextclrevtolocalrev is unused and has been deleted.
Differential Revision: https://phab.mercurial-scm.org/D4190
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 07 Aug 2018 10:49:41 -0700] rev 38997
changegroup: remove _clnodetorev
cgpacker._clnodetorev is a glorified cache/index of changelog
nodes to revision numbers.
I'm not sure why it exists. Maybe performance? But its presence
is making refactoring of this code more complicated than it needs
to be.
This commit removes the cache and replaces it with direct lookups
against the changelog.
If this cache was for performance reasons, we should be able to
restore it easily enough... after the changegroup refactor is
complete.
Differential Revision: https://phab.mercurial-scm.org/D4189