Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:54:10 +0000] rev 39204
setdiscovery: pass head revisions into sample functions
This eliminates the last remaining consumer of heads() and
related functionality in dagutil.
Differential Revision: https://phab.mercurial-scm.org/D4319
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:48:15 +0000] rev 39203
setdiscovery: pass heads into _updatesample()
In preparation for eliminating the use of dagutil. Since
_takefullsample() operates on the inverted DAG, it is easier
to have the caller pass in the relevant set instead of teaching
_updatesample() about when to invert the DAG.
We keep the logic identical for now: future commits will remove
dagutil.
Differential Revision: https://phab.mercurial-scm.org/D4318
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:26:07 +0000] rev 39202
setdiscovery: use a revset for finding DAG heads in a subset
The march towards moving away from dagutil continues.
Like other patches moving us away from dagutil, there is the
potential for regressions to occur because revlogdag's
headsetofconnecteds() uses revlog.index, which doesn't take
filtering into account. The revset layer does. But no tests
fail, so we appear to be in the clear.
Differential Revision: https://phab.mercurial-scm.org/D4317
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:21:11 +0000] rev 39201
setdiscovery: reflect use of revs instead of nodes
This code all operates on revision numbers. Update variable names
and comments accordingly.
Differential Revision: https://phab.mercurial-scm.org/D4316
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:15:09 +0000] rev 39200
dagutil: remove descendantset() and ancestorset()
descendantset() is unused after the previous commit. And
ancestorset() was only used by descendantset(), so it can be removed
as well.
.. api:: descendantset() and ancestorset() removed from dagutil
Use a revset instead when operating on the changelog. Or use
various functionality in the ancestor or dagop modules.
Differential Revision: https://phab.mercurial-scm.org/D4315
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 17:13:26 +0000] rev 39199
setdiscovery: use a revset instead of dagutil.descendantset()
This is the only use of descendantset() in the repo.
Strictly speaking, the revset behaves slightly differently than
dagutil. The reason is that dagutil is using revlog.index for
DAG traversal and this data structure isn't aware of visibility /
filtering. So it can operate on revisions it shouldn't operate on.
But our test coverage of this code is pretty comprehensive and
this change causes no tests to fail. So I think we are good.
Also, the revset parser failed to parse `%ld:: - %ld::`, hence
the use of descendants(). I'm not sure if that is a feature or
a bug.
Differential Revision: https://phab.mercurial-scm.org/D4314
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 20:23:10 +0000] rev 39198
setdiscovery: don't use dagutil to compute heads
This is a relatively trivial operation to perform. We don't
need to use dagutil.
This brings us one step closer to eliminating dagutil.
We still need to populate the heads on revlogdag because later
functionality relies on it.
Differential Revision: https://phab.mercurial-scm.org/D4310
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 20:11:45 +0000] rev 39197
dagutil: remove nodeset()
AFAICT this was unused since the code was introduced by cb98fed52495
in 2011!
Differential Revision: https://phab.mercurial-scm.org/D4309
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:55:55 +0000] rev 39196
debugcommands: use a revset instead of dagutil
All this code was doing was finding the nodes that are heads
from the ancestors of an input set. This can easily be expressed
with a revset without having to go through dagutil.
This was the last use of ancestorset() outside of dagutil itself.
Differential Revision: https://phab.mercurial-scm.org/D4308
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:51:01 +0000] rev 39195
dagutil: remove internalize() and internalizeall()
The previous commit removed the last consumer.
.. api:: removed internalize() and internalizeall() from dagutil
Use .rev(node) on storage objects to convert nodes to revisions.
Differential Revision: https://phab.mercurial-scm.org/D4307
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:47:30 +0000] rev 39194
setdiscovery: don't use dagutil for node -> rev conversion
The node -> rev conversion is possible using standard storage APIs
and doesn't need to involve the dagutil module.
Differential Revision: https://phab.mercurial-scm.org/D4306
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:40:46 +0000] rev 39193
dagutil: remove externalize() and externalizeall()
They are unused after the previous commit.
.. api:: externalize() and externalizeall() removed from dagutil
Use .node() on a storage primitive to perform revision to node
conversions.
Differential Revision: https://phab.mercurial-scm.org/D4305
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:39:47 +0000] rev 39192
setdiscovery: don't use dagutil for rev -> node conversions
We don't need to use dagutil to perform a simple rev -> node
conversion.
I haven't measured, but the new code is likely faster, as we
avoid extra function calls and avoid some attribute lookups.
Differential Revision: https://phab.mercurial-scm.org/D4304
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 19:23:24 +0000] rev 39191
exchange: don't use dagutil
We were only using it for simple node -> rev and parent revision
lookups. These are exposed via the storage interface and we don't
need to go through dagutil.
Differential Revision: https://phab.mercurial-scm.org/D4303
Paul Morelle <paul.morelle@octobus.net> [Fri, 20 Jul 2018 13:20:01 +0200] rev 39190
revlog: only consider the span of the delta section
Since the number of snapshots is limited we can exclude them from the logic
checking size and number of reads. Limiting the span computation to the delta
section will allow for further optimization.
Boris Feld <boris.feld@octobus.net> [Mon, 23 Jul 2018 16:21:58 +0200] rev 39189
revlog: ensure intermediate snapshot have decreasing size
If the intermediate snapshot is bigger than the previous one, there is likely
a better snapshot to be made at a different level.
Paul Morelle <paul.morelle@octobus.net> [Wed, 07 Mar 2018 12:28:04 +0100] rev 39188
revlog: bound number of snapshots in a chain
To limit the number of snapshot chained, we enforce them to be smaller and
smaller. This guarantee the number of snapshot in a chain will be bounded to a
small number.
Boris Feld <boris.feld@octobus.net> [Fri, 20 Jul 2018 14:32:56 +0200] rev 39187
revlog: compute snapshot depth on delta info
We need the information to be available when choosing delta.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 12:30:30 +0200] rev 39186
debugrevlog: display snapshot details per depth
This help in understanding the final structure of build manifest. All data
about snapshot (full and intermediate) are gathered into a sub-list for
clarity.
Since we do not produce such snapshots yet, the only thing changing in test
output is the way the information is presented.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 12:09:14 +0200] rev 39185
revlog: add a method to retrieve snapshot depth
Some snapshot property (eg: maximum size) will depend on their depth.
Boris Feld <boris.feld@octobus.net> [Fri, 27 Jul 2018 10:52:43 +0200] rev 39184
debugrevlog: include information about intermediate snapshots
As we are about to create intermediate snapshots, we need to have a way to
debug them. We start by adding very simple debug output and more detailed
output will comes in next changesets.
Paul Morelle <paul.morelle@octobus.net> [Fri, 20 Jul 2018 13:34:48 +0200] rev 39183
revlog: also detect intermediate snapshots
Also detect intermediate-snapshot done against another previous snapshot.
Doing an intermediate snapshot instead of a full one can reduce the number of
full snapshots we need. They are especially useful for content with a lot of
churn on the same line (eg: the manifest) where having a delta over multiple
revisions can end up being significantly smaller than the sum of these
revision deltas.
A revlog built using intermediate snapshots can be a bit smaller and reuse
snapshot much more efficiently. This last property is useful combined with
constraints on chain length. Using intermediate snapshot can produce
repository with delta chain ten times shorter without impact on the storage
size. Shorter chain lengths are faster to restore, greatly improving read
performance.
This changesets (and the following ones) focus on getting the core principle
of intermediate snapshots into Mercurial core. Later changeset will introduce
the strategy to create them.
Paul Morelle <paul.morelle@octobus.net> [Fri, 20 Jul 2018 13:32:17 +0200] rev 39182
revlog: add a method to tells whether rev is stored as a snapshot
For now we only have one type of snapshot: full snapshot versus nullrev.
However we are looking into adding intermediate snapshot where a large diff
against another snapshot is performed instead of storing a full new text.
The conditional is a bit strange and is done in order to help readability of a
some later changesets.
Boris Feld <boris.feld@octobus.net> [Wed, 15 Aug 2018 15:20:44 +0200] rev 39181
debugrevlog: fix for non-manifest object
The `filelog` object is no longer an actual revlog. Instead, the actual revlog
is stored in the `_revlog` attribute.
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 17 Aug 2018 16:11:35 -0700] rev 39180
merge with stable
Kyle Lippincott <spectral@google.com> [Fri, 17 Aug 2018 15:32:38 -0700] rev 39179
nodes: expand/comment the magic nodes so they are more easily searchable
We just encountered `000000000000modified`, and it was quite annoying to search
for these, even though I knew they existed. For those that don't know that they
exist, this is essentially impossible to search for :)
(Technically we encountered it in its hex form,
3030303030303030303030306d6f646966696564, so I'm adding comments with those
forms in case that's helpful to people in the future).
Differential Revision: https://phab.mercurial-scm.org/D4331
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 13:07:33 +0900] rev 39178
revlog: obtain the first node at the lowest layer while building pure nodemap
Just for clarity. This doesn't matter in practice since changelog.nodemap
is accessed *before* filtered revisions get ready.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 12:54:50 +0900] rev 39177
revlog: fix pure nodemap to not access missing index entry
This bug was revealed by a3dacabd476b and a1f934573c0b.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 12:48:44 +0900] rev 39176
changelog: remove copy of revlog.nodemap()
It's been there since 2012, "clfilter: introduce `filteredrevs` attribute
on changelog." I don't think we can apply changelog filtering to nodemap at
this level, so this patch removes the nodemap stub completely.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:51:05 +0900] rev 39175
branchmap: explicitly convert file into iterator
Follows up 2a4bfbb52111. This is required for httprangereader, which is not
an iterable itself.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:25:39 +0900] rev 39174
branchmap: close cache file properly
Follows up 2a4bfbb52111.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:24:29 +0900] rev 39173
branchmap: strip '\n' read from cache file as before
Follows up 2a4bfbb52111.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:21:25 +0900] rev 39172
rebase: do not pass in user option to rollback in-memory merge conflict
Nothing passed before e9e742bd0501.
Yuya Nishihara <yuya@tcha.org> [Fri, 17 Aug 2018 10:19:17 +0900] rev 39171
rebase: add test for in-memory merge conflicts
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 18:53:51 +0000] rev 39170
rebase: call _dorebase() properly
This fixes a regression from e9e742bd0501 where we failed to pass
all necessary arguments to _dorebase().
Differential Revision: https://phab.mercurial-scm.org/D4302
Pulkit Goyal <pulkit@yandex-team.ru> [Thu, 16 Aug 2018 16:59:40 +0300] rev 39169
context: make sure file is not deleted while checking path conflicts
If a file is deleted and a directory of same name is created in the same commit,
IMM thinks of that as a file conflict, however the file is deleted and hence the
directory can be created.
The test change demonstrate the fix.
Differential Revision: https://phab.mercurial-scm.org/D4300
Pulkit Goyal <pulkit@yandex-team.ru> [Thu, 16 Aug 2018 16:53:48 +0300] rev 39168
tests: demonstrate that IMM needs to be smarter with path conflicts
When we try to rebase a commit which deletes an existing file and make a
directory of the same name, rebase with IMM aborts. It should work fine just
like the without IMM case.
Differential Revision: https://phab.mercurial-scm.org/D4299
Pulkit Goyal <pulkit@yandex-team.ru> [Thu, 16 Aug 2018 16:36:32 +0300] rev 39167
tests: don't create new repo inside existing repo in test-rebase-inmemory.t
Differential Revision: https://phab.mercurial-scm.org/D4298
Augie Fackler <augie@google.com> [Wed, 25 Jul 2018 13:40:42 -0400] rev 39166
tests: remove test-py3-commands.t
This was a smoke test for early in the Python 3 porting effort, before
anything actually worked. Now that we've got over half the testsuite
passing, this test has outlived its utility.
Differential Revision: https://phab.mercurial-scm.org/D4288
Augie Fackler <augie@google.com> [Wed, 25 Jul 2018 13:41:21 -0400] rev 39165
tests: update test-check-py3-compat.t output in the py3exe branch
This hasn't been maintained in a while, it looks like.
Differential Revision: https://phab.mercurial-scm.org/D4289
Kyle Lippincott <spectral@google.com> [Wed, 15 Aug 2018 17:40:21 -0700] rev 39164
overlayworkingctx: fix exception in metadata-only inmemory merges (issue5960)
If there was a metadata-only mutation, such as +x or -x on a file, we would
create a cache entry with None for data, and this would cause problems later on
when some code tried to run fctx.data() or similar, and was expecting a string.
My original fix for this involved passing data=self._wrappedctx[path].data() in
setflags(), but this version seems slightly better - this way, if we ever call
write() and then call setflags(), we don't destroy the data that we wrote that's
in the cache. I haven't verified that other fields aren't destroyed, such as
date or flags :)
Differential Revision: https://phab.mercurial-scm.org/D4287
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 22:20:28 +0900] rev 39163
filemerge: show actual capabilities of internal merge tools
This information is useful to know which internal merge tools can be
applied safely on binary files and/or symlinks.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 15 Aug 2018 22:24:50 +0900] rev 39162
filemerge: add config knob to check capabilities of internal merge tools
For historical reason, Mercurial assumes capabilities of internal
merge tools as below while examining rules to decide merge tool,
regardless of actual capabilities of them.
=============== ====== ========
specified via binary symlinks
=============== ====== ========
--tool o o
HGMERGE o o
merge-patterns o (*) x (*)
ui.merge x (*) x (*)
=============== ====== ========
This causes:
- unintentional internal merge tool is chosen for binary files via
merge-patterns section of configuration file
- explicit configuration of internal merge tool for symlinks is
ignored unintentionally
But on the other hand, simple "check capability strictly" might break
backward compatibility (e.g. existing merge automations), because it
changes the result of merge tool selection.
Therefore, this patch adds config knob "merge.strict-capability-check"
to control whether capabilities of internal merge tools should be
checked strictly or not.
If this configuration is true, capabilities of internal merge tools
are checked strictly in (*) cases above.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 15 Aug 2018 22:24:38 +0900] rev 39161
filemerge: show warning if chosen tool has no binary files capability
While matching patterns in "merge-patterns" configuration, Mercurial
silently assumes that all merge tools have binary files
capability. This implementation comes from 5af5f0f9d724 (or Mercurial
1.0).
At failure of merging binary files with incorrect internal merge tool,
there is no hint about this silent ignorance of binary files
capability.
This patch shows warning message, if chosen internal merge tool has no
binary files capability. This will help users to investigate why a
binary file isn't merged as expected.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 20:15:51 +0900] rev 39160
filemerge: add the function to examine a capability of a internal tool
For "symlink" and "binary" capabilities, _toolbool() can not examine
these of internal merge tools strictly, because it examines only
configurations in "merge-tools" section.
Users can configure them explicitly as below for example, but this is
not ordinary usage and not convenient:
[merge-tools]
:other.symlink = true
:other.binary = true
This patch adds hascapability() internal function, which can examine
actual capabilities of a internal merge tool strictly.
At this patch, hascapability() is still used with "strict=False".
Subsequent patches use it with "strict=True".
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 20:08:27 +0900] rev 39159
filemerge: set actual capabilities of internal merge tools
This information is used to detect actual capabilities of internal
merge tools by subsequent patches.
For convenience, this patch assumes that merge tools typed as
"nomerge" have both binary files and symlinks capabilities.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Tue, 14 Aug 2018 20:05:36 +0900] rev 39158
help: describe more detail about capabilities while deciding merge tool
"hg help merge-tools" describes as below:
(internal merge tools) will by default not handle symlinks or
binary files.
But in some cases, Mercurial assumes that internal merge tools have
one or both of these capabilities.
"hg help merge-tools" also describes as below, for matching patterns in
merge-patterns configuration section. But this is not sufficient.
Here, binary capabilities of the merge tool are not considered.
This patch describes more detail about capabilities while deciding
merge tool.
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:50:53 -0400] rev 39157
tests: un-glob patchbomb test lines that were only globbing a hostname
Differential Revision: https://phab.mercurial-scm.org/D4296
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:42:04 -0400] rev 39156
tests: force a stable hostname in patchbomb tests
No visible output changes in this commit because everything is globbed
away, but on Python 3 the stable hostname will mean that the headers
don't trigger RFC2822 multi-line mode because they'll always be
consistently short.
Differential Revision: https://phab.mercurial-scm.org/D4295
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:40:20 -0400] rev 39155
patchbomb: allow using HGHOSTNAME to force a hostname
I'll update run-tests.py to set this globally to stabilize some
tests. The variable name is intentionally generic because I suspect we
should generalize this to other tests.
Differential Revision: https://phab.mercurial-scm.org/D4294
Augie Fackler <raf@durin42.com> [Thu, 16 Aug 2018 00:39:32 -0400] rev 39154
patchbomb: extract function for generating message-id
Differential Revision: https://phab.mercurial-scm.org/D4293
Sushil khanchi <sushilkhanchi97@gmail.com> [Wed, 15 Aug 2018 11:27:57 +0530] rev 39153
rebase: cover restorestatus() by lock to prevent it from being updated
To prevent it from being updated by another process
`restorestatus()` is moved under lock.
Differential Revision: https://phab.mercurial-scm.org/D4282
Martijn Pieters <mj@octobus.net> [Mon, 13 Aug 2018 21:22:14 +0100] rev 39152
branchmap: load branchmap as an iterable
This avoids reading all the file into memory if the cache turns out to be
invalid.
Differential Revision: https://phab.mercurial-scm.org/D4281
Martijn Pieters <mj@octobus.net> [Mon, 13 Aug 2018 20:31:01 +0100] rev 39151
perf: time loading branchmap caches
Differential Revision: https://phab.mercurial-scm.org/D4280
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 00:13:41 +0000] rev 39150
tests: add conditional output when simplestore extensions is loaded
This drops the number of failures with this extension to 3.
Differential Revision: https://phab.mercurial-scm.org/D4286
Gregory Szorc <gregory.szorc@gmail.com> [Thu, 16 Aug 2018 00:11:35 +0000] rev 39149
tests: conditionalize extension tests for extra extensions
If extra extensions are loaded (e.g. via --extra-config-opt),
the tests conditionalized in this commit fail in ways that
are dependent on the extensions that are loaded. So let's
skip them when that scenario is present.
This drops the number of failures for the simplestorerepo.py
extension to 4.
Differential Revision: https://phab.mercurial-scm.org/D4285
Yuya Nishihara <yuya@tcha.org> [Sat, 07 Jul 2018 22:40:39 +0900] rev 39148
commit: try hard to reuse p1 manifest if nothing changed
This is all for commit reproducibility on "hg convert".
With this change, p1 manifest is reused if ctx.files() *to be committed* is
empty, and if new manifest entry is identical to p1. This is important
property for "hg convert" since memctx.files() built from a convert source
may be either a) more narrowed thanks to a committed ctx.files() which
provides more accurate status, or b) containing redundant files because of
sloppy filtering on e.g. octopus merge.
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 18:44:42 +0900] rev 39147
merge: add tests for commit with no content change
It isn't easy to say when to reuse the p1 manifest. Basically, that's only
when wctx.files() is empty, but we need to know that wctx.files() is not
the same as repo['.'].files() after the commit.
This patch adds several examples of commits with empty ctx/wctx.files().
I don't think this is exhaustive, but it contains at least one failure
mode in which a converted repo result in a different hash.
I also note that the manifest revlog does NOT follow the DAG shape of the
changelog since p1 manifest is reused if wctx.files() is empty even at merge.
I don't know whether it is intentional or not, but it's the behavior since
2011, 301725c3df9a "localrepo: reuse parent manifest in commitctx if no files
have changed."
Yuya Nishihara <yuya@tcha.org> [Sat, 07 Jul 2018 22:32:49 +0900] rev 39146
commit: add debug message regarding manifest reuse
Yuya Nishihara <yuya@tcha.org> [Sun, 12 Aug 2018 12:16:08 +0900] rev 39145
mail: convert encoding.encoding to sysstr