stream-clone: avoid opening a revlog in case we do not need it
Opening an revlog has a cost, especially if it is inline as we have to scan the
file and construct an index.
To prevent the associated slowdown, we just do a minimal scan to check that an
inline file is still inline, and simply stream the file without creating a
revlog when we can.
This provides a big boost compared to the previous changeset, even if the full
generation is still penalized by the initial gathering of information.
All benchmarks are run on linux with Python 3.10.7.
# benchmark.name = hg.exchange.stream.generate
# benchmark.variants.version = v2
### Compared to the previous changesets
We get a large win all across the board!
# mercurial-2018-08-01-zstd-sparse-revlog
before: 0.250694 seconds
after: 0.105986 seconds (-57.72%)
# pypy-2018-08-01-zstd-sparse-revlog
before: 3.885657 seconds
after: 1.709748 seconds (-56.00%)
# netbeans-2018-08-01-zstd-sparse-revlog
before: 16.679371 seconds
after: 7.687469 seconds (-53.91%)
# mozilla-central-2018-08-01-zstd-sparse-revlog
before: 38.575482 seconds
after: 17.520316 seconds (-54.58%)
# mozilla-try-2019-02-18-zstd-sparse-revlog
before: 81.160994 seconds
after: 37.073753 seconds (-54.32%)
### Compared to 6.4.3
We are still significantly slower than 6.4.3, the extra time is usually twice
slower than the extra time we observe on the locked section, which is a quite
interesting information.
Except for mercurial-central that is much faster. That discrepancy is not really
explained yet.
# mercurial-2018-08-01-zstd-sparse-revlog
6.4.3: 0.072560 seconds
after: 0.105986 seconds (+46.07%) (- 0.03 seconds)
# pypy-2018-08-01-zstd-sparse-revlog
6.4.3: 1.211193 seconds
after: 1.709748 seconds (+41.16%) (-0.45 seconds)
# netbeans-2018-08-01-zstd-sparse-revlog
6.4.3: 4.932843 seconds
after: 7.687469 seconds (+55.84%) (-2.75 seconds)
# mozilla-central-2018-08-01-zstd-sparse-revlog
6.4.3: 34.012226 seconds
after: 17.520316 seconds (-48.49%) (-16.49 seconds)
# mozilla-try-2019-02-18-zstd-sparse-revlog
6.4.3: 23.850555 seconds
after: 37.073753 seconds (+55.44%) (+13.22 seconds)
#require serve
setting up repo
$ hg init test
$ cd test
$ echo a > a
$ hg ci -Ama
adding a
$ hg rm a
$ hg ci -mdel
set up hgweb
$ hg serve -n test -p $HGPORT -d --pid-file=hg.pid -A access.log -E errors.log
$ cat hg.pid >> $DAEMON_PIDS
revision
$ get-with-headers.py localhost:$HGPORT 'rev/tip'
200 Script output follows
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US">
<head>
<link rel="icon" href="/static/hgicon.png" type="image/png" />
<meta name="robots" content="index, nofollow" />
<link rel="stylesheet" href="/static/style-paper.css" type="text/css" />
<script type="text/javascript" src="/static/mercurial.js"></script>
<title>test: c78f6c5cbea9</title>
</head>
<body>
<div class="container">
<div class="menu">
<div class="logo">
<a href="https://mercurial-scm.org/">
<img src="/static/hglogo.png" alt="mercurial" /></a>
</div>
<ul>
<li><a href="/shortlog/tip">log</a></li>
<li><a href="/graph/tip">graph</a></li>
<li><a href="/tags">tags</a></li>
<li><a href="/bookmarks">bookmarks</a></li>
<li><a href="/branches">branches</a></li>
</ul>
<ul>
<li class="active">changeset</li>
<li><a href="/raw-rev/tip">raw</a></li>
<li><a href="/file/tip">browse</a></li>
</ul>
<ul>
</ul>
<ul>
<li><a href="/help">help</a></li>
</ul>
</div>
<div class="main">
<h2 class="breadcrumb"><a href="/">Mercurial</a> </h2>
<h3>
changeset 1:<a href="/rev/c78f6c5cbea9">c78f6c5cbea9</a>
<span class="phase">draft</span> <span class="branchhead">default</span> <span class="tag">tip</span>
</h3>
<form class="search" action="/log">
<p><input name="rev" id="search1" type="text" size="30" value="" /></p>
<div id="hint">Find changesets by keywords (author, files, the commit message), revision
number or hash, or <a href="/help/revsets">revset expression</a>.</div>
</form>
<div class="description">del</div>
<table id="changesetEntry">
<tr>
<th class="author">author</th>
<td class="author">test</td>
</tr>
<tr>
<th class="date">date</th>
<td class="date age">Thu, 01 Jan 1970 00:00:00 +0000</td>
</tr>
<tr>
<th class="author">parents</th>
<td class="author"><a href="/rev/cb9a9f314b8b">cb9a9f314b8b</a> </td>
</tr>
<tr>
<th class="author">children</th>
<td class="author"></td>
</tr>
<tr>
<th class="files">files</th>
<td class="files">a </td>
</tr>
<tr>
<th class="diffstat">diffstat</th>
<td class="diffstat">
1 files changed, 0 insertions(+), 1 deletions(-)
<a id="diffstatexpand" class="diffstattoggle" href="#">[<tt>+</tt>]</a>
<div id="diffstatdetails" style="display:none;">
<a class="diffstattoggle" href="#">[<tt>-</tt>]</a>
<table class="diffstat-table stripes2"> <tr>
<td class="diffstat-file"><a href="#l1.1">a</a></td>
<td class="diffstat-total" align="right">1</td>
<td class="diffstat-graph">
<span class="diffstat-add" style="width:0.0%;"> </span>
<span class="diffstat-remove" style="width:100.0%;"> </span>
</td>
</tr>
</table>
</div>
</td>
</tr>
</table>
<div class="overflow">
<div class="sourcefirst linewraptoggle">line wrap: <a class="linewraplink" href="#">on</a></div>
<div class="sourcefirst"> line diff</div>
<div class="stripes2 diffblocks">
<div class="bottomline inc-lineno"><pre class="sourcelines wrap">
<span id="l1.1" class="minusline">--- a/a Thu Jan 01 00:00:00 1970 +0000</span><a href="#l1.1"></a>
<span id="l1.2" class="plusline">+++ /dev/null Thu Jan 01 00:00:00 1970 +0000</span><a href="#l1.2"></a>
<span id="l1.3" class="atline">@@ -1,1 +0,0 @@</span><a href="#l1.3"></a>
<span id="l1.4" class="minusline">-a</span><a href="#l1.4"></a></pre></div>
</div>
</div>
</div>
</div>
</body>
</html>
diff removed file
$ get-with-headers.py localhost:$HGPORT 'diff/tip/a'
200 Script output follows
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US">
<head>
<link rel="icon" href="/static/hgicon.png" type="image/png" />
<meta name="robots" content="index, nofollow" />
<link rel="stylesheet" href="/static/style-paper.css" type="text/css" />
<script type="text/javascript" src="/static/mercurial.js"></script>
<title>test: a diff</title>
</head>
<body>
<div class="container">
<div class="menu">
<div class="logo">
<a href="https://mercurial-scm.org/">
<img src="/static/hglogo.png" alt="mercurial" /></a>
</div>
<ul>
<li><a href="/shortlog/tip">log</a></li>
<li><a href="/graph/tip">graph</a></li>
<li><a href="/tags">tags</a></li>
<li><a href="/bookmarks">bookmarks</a></li>
<li><a href="/branches">branches</a></li>
</ul>
<ul>
<li><a href="/rev/tip">changeset</a></li>
<li><a href="/file/tip">browse</a></li>
</ul>
<ul>
<li><a href="/file/tip/a">file</a></li>
<li><a href="/file/tip/a">latest</a></li>
<li class="active">diff</li>
<li><a href="/comparison/tip/a">comparison</a></li>
<li><a href="/annotate/tip/a">annotate</a></li>
<li><a href="/log/tip/a">file log</a></li>
<li><a href="/raw-file/tip/a">raw</a></li>
</ul>
<ul>
<li><a href="/help">help</a></li>
</ul>
</div>
<div class="main">
<h2 class="breadcrumb"><a href="/">Mercurial</a> </h2>
<h3>
diff a @ 1:<a href="/rev/c78f6c5cbea9">c78f6c5cbea9</a>
<span class="phase">draft</span> <span class="branchhead">default</span> <span class="tag">tip</span>
</h3>
<form class="search" action="/log">
<p><input name="rev" id="search1" type="text" size="30" value="" /></p>
<div id="hint">Find changesets by keywords (author, files, the commit message), revision
number or hash, or <a href="/help/revsets">revset expression</a>.</div>
</form>
<div class="description">del</div>
<table id="changesetEntry">
<tr>
<th>author</th>
<td>test</td>
</tr>
<tr>
<th>date</th>
<td class="date age">Thu, 01 Jan 1970 00:00:00 +0000</td>
</tr>
<tr>
<th>parents</th>
<td><a href="/file/cb9a9f314b8b/a">cb9a9f314b8b</a> </td>
</tr>
<tr>
<th>children</th>
<td></td>
</tr>
</table>
<div class="overflow">
<div class="sourcefirst linewraptoggle">line wrap: <a class="linewraplink" href="#">on</a></div>
<div class="sourcefirst"> line diff</div>
<div class="stripes2 diffblocks">
<div class="bottomline inc-lineno"><pre class="sourcelines wrap">
<span id="l1.1" class="minusline">--- a/a Thu Jan 01 00:00:00 1970 +0000</span><a href="#l1.1"></a>
<span id="l1.2" class="plusline">+++ /dev/null Thu Jan 01 00:00:00 1970 +0000</span><a href="#l1.2"></a>
<span id="l1.3" class="atline">@@ -1,1 +0,0 @@</span><a href="#l1.3"></a>
<span id="l1.4" class="minusline">-a</span><a href="#l1.4"></a></pre></div>
</div>
</div>
</div>
</div>
</body>
</html>
$ cd ..