Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 15 Oct 2019 18:42:03 +0200] rev 43256
copies: use an unfiltered repository for the changeset centric algorithm
Since the algorithm work form heads to ancestors, we don't need to check
filtering for anything but the two entries.
Using an unfiltered version is noticeably more efficient.
Some timing on the pypy repo:
revision: large amount; added files: large amount; rename small amount;
c3b14617fbd7 9ba6ab77fd29
before: ! wall 2.717861 comb 2.720000 user 2.700000 sys 0.020000 (median of 10)
after: ! wall 2.582204 comb 2.580000 user 2.560000 sys 0.020000 (median of 10)
revision: large amount; added files: small amount; rename small amount;
c3b14617fbd7 f650a9b140d2
before: ! wall 4.003146 comb 4.010000 user 3.970000 sys 0.040000 (median of 10)
after: ! wall 3.814613 comb 3.810000 user 3.760000 sys 0.050000 (median of 10)
revision: large amount; added files: large amount; rename large amount;
08ea3258278e d9fa043f30c0
before: ! wall 0.704204 comb 0.700000 user 0.700000 sys 0.000000 (median of 13)
after: ! wall 0.657387 comb 0.650000 user 0.640000 sys 0.010000 (best of 14)
revision: small amount; added files: large amount; rename large amount;
df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013493 comb 0.020000 user 0.020000 sys 0.000000 (median of 219)
after: ! wall 0.013523 comb 0.020000 user 0.020000 sys 0.000000 (median of 218)
revision: small amount; added files: large amount; rename small amount;
4aa4e1f8e19a 169138063d63
before: ! wall 0.003017 comb 0.000000 user 0.000000 sys 0.000000 (median of 985)
after: ! wall 0.002876 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount;
4bc173b045a6 964879152e2e
before: ! wall 0.000073 comb 0.000000 user 0.000000 sys 0.000000 (median of 12672)
after: ! wall 0.000082 comb 0.000000 user 0.000000 sys 0.000000 (median of 11456)
revision: medium amount; added files: large amount; rename medium amount;
c95f1ced15f2 2c68e87c3efe
before: ! wall 0.478061 comb 0.470000 user 0.470000 sys 0.000000 (median of 19)
after: ! wall 0.452420 comb 0.450000 user 0.450000 sys 0.000000 (median of 21)
revision: medium amount; added files: medium amount; rename small amount;
d343da0c55a8 d7746d32bf9d
before: ! wall 0.116015 comb 0.110000 user 0.110000 sys 0.000000 (median of 84)
after: ! wall 0.109153 comb 0.100000 user 0.100000 sys 0.000000 (median of 90)
Differential Revision: https://phab.mercurial-scm.org/D7123
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 17:42:01 -0400] rev 43255
copies: extract data extraction into a `revinfo` function
The function is build once at the beginning of the algorithm and used fetch
appropriate information for each revision.
This abstracts some implementation details from the main algorithm and will help
us to access the data more efficiently in future changesets.
Differential Revision: https://phab.mercurial-scm.org/D7070
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Oct 2019 17:49:30 +0200] rev 43254
copies: avoid instancing more changectx to access parent revisions
We just need to know the revision numbers of the parents, creating full context
is needlessly expensive.
This provide a small, but noticeable performance boost.
revision: large amount; added files: large amount; rename small amount;
c3b14617fbd7 9ba6ab77fd29
before: ! wall 2.885636 comb 2.900000 user 2.870000 sys 0.030000 (median of 10)
after: ! wall 2.702270 comb 2.710000 user 2.690000 sys 0.020000 (median of 10)
revision: large amount; added files: small amount; rename small amount;
c3b14617fbd7 f650a9b140d2
before: ! wall 4.298271 comb 4.290000 user 4.240000 sys 0.050000 (median of 10)
after: ! wall 3.976610 comb 3.970000 user 3.920000 sys 0.050000 (median of 10)
revision: large amount; added files: large amount; rename large amount;
08ea3258278e d9fa043f30c0
before: ! wall 0.773397 comb 0.770000 user 0.770000 sys 0.000000 (median of 11)
after: ! wall 0.701634 comb 0.700000 user 0.700000 sys 0.000000 (median of 13)
revision: small amount; added files: large amount; rename large amount;
df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013585 comb 0.010000 user 0.010000 sys 0.000000 (median of 217)
after: ! wall 0.013550 comb 0.010000 user 0.010000 sys 0.000000 (median of 218)
revision: small amount; added files: large amount; rename small amount;
4aa4e1f8e19a 169138063d63
before: ! wall 0.003202 comb 0.000000 user 0.000000 sys 0.000000 (median of 929)
after: ! wall 0.002993 comb 0.010000 user 0.010000 sys 0.000000 (median of 992)
revision: small amount; added files: small amount; rename small amount;
4bc173b045a6 964879152e2e
before: ! wall 0.000077 comb 0.000000 user 0.000000 sys 0.000000 (median of 12060)
after: ! wall 0.000072 comb 0.000000 user 0.000000 sys 0.000000 (median of 12804)
revision: medium amount; added files: large amount; rename medium amount;
c95f1ced15f2 2c68e87c3efe
before: ! wall 0.510614 comb 0.500000 user 0.500000 sys 0.000000 (median of 18)
after: ! wall 0.473681 comb 0.470000 user 0.470000 sys 0.000000 (median of 20)
revision: medium amount; added files: medium amount; rename small amount;
d343da0c55a8 d7746d32bf9d
before: ! wall 0.126552 comb 0.130000 user 0.130000 sys 0.000000 (median of 77)
after: ! wall 0.115240 comb 0.110000 user 0.110000 sys 0.000000 (median of 85)
Differential Revision: https://phab.mercurial-scm.org/D7122
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 16 Oct 2019 09:52:05 +0200] rev 43253
copies: get copies information directly from _copies
Performance measurement does not show any significant performance movement. This
is not surprising since p1copies() code is `self._copies[0]`.
Differential Revision: https://phab.mercurial-scm.org/D7121
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 13:43:27 -0400] rev 43252
copies: simplify the handling of merges
Instead of stacking copies for both parent on the head, we move copies outside
of the heap into a dedicated dictionary. The two side of merge can we merged
sooner, making the algorithm simpler.
This simplicity reflect in the heap structure and speed up the execution for
copies involving a large amount of merges.
Here are timing for perfpathcopies of multiple revision pairs.
- filelog: timing using filelog (with the introrev condition dropped)
- base: this series base
- before: the parent of this changeset
- after: this changeset
revision: large amount; added files: large amount; rename small amount;
c3b14617fbd7 9ba6ab77fd29
filelog: ! wall 3.679613 comb 3.680000 user 3.580000 sys 0.100000 (median of 3)
base: ! wall 8.884369 comb 8.880000 user 8.850000 sys 0.030000 (median of 3)
before: ! wall 8.443747 comb 8.420000 user 8.410000 sys 0.010000 (median of 3)
after: ! wall 4.697917 comb 4.690000 user 4.660000 sys 0.030000 (median of 3)
revision: large amount; added files: small amount; rename small amount;
c3b14617fbd7 f650a9b140d2
filelog: ! wall 0.003357 comb 0.010000 user 0.010000 sys 0.000000 (median of 781)
base: ! wall 12.398524 comb 12.400000 user 12.330000 sys 0.070000 (median of 3)
before: ! wall 10.852593 comb 10.850000 user 10.800000 sys 0.050000 (median of 3)
after: ! wall 6.750832 comb 6.750000 user 6.640000 sys 0.110000 (median of 3)
revision: large amount; added files: large amount; rename large amount;
08ea3258278e d9fa043f30c0
filelog: ! wall 2.754687 comb 2.760000 user 2.650000 sys 0.110000 (median of 4)
base: ! wall 1.423166 comb 1.420000 user 1.400000 sys 0.020000 (median of 8)
before: ! wall 1.068041 comb 1.060000 user 1.050000 sys 0.010000 (median of 10)
after: ! wall 1.045916 comb 1.050000 user 1.040000 sys 0.010000 (median of 10)
revision: small amount; added files: large amount; rename large amount;
df6f7a526b60 a83dc6a2d56f
filelog: ! wall 1.552293 comb 1.550000 user 1.510000 sys 0.040000 (median of 6
base: ! wall 0.022662 comb 0.020000 user 0.020000 sys 0.000000 (median of 128)
before: ! wall 0.021111 comb 0.020000 user 0.020000 sys 0.000000 (median of 139)
after: ! wall 0.021577 comb 0.020000 user 0.020000 sys 0.000000 (median of 138)
revision: small amount; added files: large amount; rename small amount;
4aa4e1f8e19a 169138063d63
filelog: ! wall 1.500983 comb 1.500000 user 1.420000 sys 0.080000 (median of 7)
base: ! wall 0.006956 comb 0.010000 user 0.010000 sys 0.000000 (median of 392)
before: ! wall 0.004356 comb 0.010000 user 0.010000 sys 0.000000 (median of 675)
after: ! wall 0.004329 comb 0.000000 user 0.000000 sys 0.000000 (median of 682)
revision: small amount; added files: small amount; rename small amount;
4bc173b045a6 964879152e2e
filelog: ! wall 0.011745 comb 0.020000 user 0.020000 sys 0.000000 (median of 250)
base: ! wall 0.000156 comb 0.000000 user 0.000000 sys 0.000000 (median of 17180)
before: ! wall 0.000100 comb 0.000000 user 0.000000 sys 0.000000 (median of 26912)
after: ! wall 0.000105 comb 0.000000 user 0.000000 sys 0.000000 (median of 25689)
revision: medium amount; added files: large amount; rename medium amount;
c95f1ced15f2 2c68e87c3efe
filelog: ! wall 3.228230 comb 3.230000 user 3.110000 sys 0.120000 (median of 4)
base: ! wall 0.997640 comb 1.000000 user 0.980000 sys 0.020000 (median of 10)
before: ! wall 0.778291 comb 0.780000 user 0.780000 sys 0.000000 (median of 13)
after: ! wall 0.706594 comb 0.710000 user 0.710000 sys 0.000000 (median of 15)
revision: medium amount; added files: medium amount; rename small amount;
d343da0c55a8 d7746d32bf9d
filelog: ! wall 1.052501 comb 1.060000 user 1.040000 sys 0.020000 (median of 10
base: ! wall 0.214519 comb 0.220000 user 0.220000 sys 0.000000 (median of 45)
before: ! wall 0.160804 comb 0.160000 user 0.160000 sys 0.000000 (median of 62)
after: ! wall 0.163736 comb 0.160000 user 0.160000 sys 0.000000 (median of 60)
Differential Revision: https://phab.mercurial-scm.org/D7069
Raphaël Gomès <rgomes@octobus.net> [Mon, 07 Oct 2019 23:17:44 +0200] rev 43251
rust-utils: introduce a debug util to print the python stack trace
Differential Revision: https://phab.mercurial-scm.org/D7057
Raphaël Gomès <rgomes@octobus.net> [Wed, 16 Oct 2019 17:16:23 +0300] rev 43250
rust-cross-platform: remove `unimplemented!` to get compile-time errors
We should get a compile-time error that there is missing functionality for the
targeted platform instead of the program breaking at runtime.
Differential Revision: https://phab.mercurial-scm.org/D7107
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 00:18:34 -0700] rev 43249
widening: pass in matchers instead of patterns
This matches how it's done for the non-ellipsis case. The oldmatch is
not used yet, but it should be used eventually when widening no longer
resends manifests and files the client already has.
Differential Revision: https://phab.mercurial-scm.org/D7099
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:41:50 -0700] rev 43248
widening: trust user to give full "known" set
The new narrow_widen wire protocol command is supposed to return data
for exactly the revisions that the client requested (in order for it
to be strip-free). So we should not add ancestors the client didn't
ask for.
Differential Revision: https://phab.mercurial-scm.org/D7098
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 00:06:27 -0700] rev 43247
widening: remove "depth" argument since it's always None
Differential Revision: https://phab.mercurial-scm.org/D7097
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 00:03:10 -0700] rev 43246
widening: remove unused "heads" argument
Differential Revision: https://phab.mercurial-scm.org/D7096
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:36:55 -0700] rev 43245
widening: remove pointless code for second changegroup
The caller passes in common==heads, so we will never find any nodes to
visit in this code and there will therefore never be a second
changegroup emitted.
Differential Revision: https://phab.mercurial-scm.org/D7095
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:34:16 -0700] rev 43244
widening: remove always-true condition in widening code
The function is now specifically about widening, so we don't need to
check if we're widening.
Differential Revision: https://phab.mercurial-scm.org/D7094
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 21:51:07 -0700] rev 43243
narrow: drop server support for widening using the getbundle command (BC)
The client still supports widening using the getbundle, which we
(Google) still depend on a for a little while more (we've started the
migration to the new narrow_widen command, but we're not done yet).
Differential Revision: https://phab.mercurial-scm.org/D7093
Martin von Zweigbergk <martinvonz@google.com> [Thu, 10 Oct 2019 22:18:35 -0700] rev 43242
widening: duplicate generateellipsesbundle2() for widening
The widening and the non-widening code are quite different. It will be
clearer to have them as sepearate functions. To start with, I've just
copied it exactly, so it's clearer over the next few patches how
they're different.
The new function should gradually become more similar to
bundle2.widen_bundle(), and should perhaps eventually be merged with
that function. However, I've left it in narrowbundle2.py for now since
it still depends on constants like _KILLNODESIGNAL there.
Differential Revision: https://phab.mercurial-scm.org/D7092
Martin von Zweigbergk <martinvonz@google.com> [Fri, 11 Oct 2019 22:02:44 -0700] rev 43241
tests: show graph log before and after widening in more cases
I'm about to make some changes to which revisions get sent during
widening. Some more tests will make it clearer what changes. I've also
switched to graph log so we easily notice if the graph shape changes.
Differential Revision: https://phab.mercurial-scm.org/D7091
Augie Fackler <augie@google.com> [Wed, 19 Jun 2019 18:22:10 -0400] rev 43240
tracing: add a couple of trace points on obsolete and repoview
I was seeing weird stalls in some fast commands, and these trace
events help explain them.
Differential Revision: https://phab.mercurial-scm.org/D6550
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 11:05:32 -0400] rev 43239
dirstate: add some traces on listdir calls
I'm starting to spend some time trying to make `hg status` do less work, and
knowing what's happening in here and what's expensive is an important start.
Differential Revision: https://phab.mercurial-scm.org/D7109
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 11:06:04 -0400] rev 43238
cleanup: hgdemandimport.tracing accepts strings, not bytes
This does mean that the resulting traces will have some b'' goo in them on
Python 3, but I think that's a worthwile price to pay since it doesn't break
any tests.
Differential Revision: https://phab.mercurial-scm.org/D7108
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 09:48:27 -0400] rev 43237
tests: fix formatting test after move of black config
Differential Revision: https://phab.mercurial-scm.org/D7102
Raphaël Gomès <rgomes@octobus.net> [Wed, 16 Oct 2019 15:27:17 +0200] rev 43236
rust-dependencies: pin the dependencies of hg-core to prevent breakage
A new version of the `rand` crate broke two discovery tests. We should
upgrade our dependencies periodically manually.
Differential Revision: https://phab.mercurial-scm.org/D7115
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 15 Oct 2019 19:13:47 -0700] rev 43235
automation: install Python 3.8.0
Python 3.8 has been released. Let's install it instead of the
RC / development version.
Differential Revision: https://phab.mercurial-scm.org/D7114
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 14 Oct 2019 21:21:58 -0700] rev 43234
automation: schedule an EC2Launch run on next boot
Without this, launching EC2 instances constructed from the AMI
won't go through the normal EC2 instance launch machinery. This
missing machinery does important things like set up network
routes to use the instance metadata service and process any
UserData.
Since EC2Launch now runs on subsequent boots and UserData is
processed, we needed to make setting of UserData conditional
on bootstrapping mode.
Differential Revision: https://phab.mercurial-scm.org/D7113
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 09:54:05 -0400] rev 43233
dirs: give formatting oversight to clang-format
Differential Revision: https://phab.mercurial-scm.org/D7104
Augie Fackler <augie@google.com> [Tue, 15 Oct 2019 09:52:33 -0400] rev 43232
dirs: tag a struct as not being formattable
Differential Revision: https://phab.mercurial-scm.org/D7103
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 14:38:34 -0400] rev 43231
sidedatacopies: only store an entry if it has values
This will make for a shorter storage and help use to write faster code in simple
case. This change already provided a speed boost for copy tracing.
For example here is combined time of running copies tracing on a 6989 pairs of
revision on the pypy repos:
before: 771s
after: 631s - 18%
This also has a very positive impact on changelog size. For example here are the
number for the `00changelog.d` file of pypy.
before:
30449712 Bytes
after:
24973718 Bytes - 18%
This give an overall quite acceptable overhead for storing copies into the
changelog:
filelog-only:
23370586
sidedata:
24973718 Bytes + 7%
Differential Revision: https://phab.mercurial-scm.org/D7068
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 02 Oct 2019 14:16:30 -0400] rev 43230
sidedatacopies: only read from copies when in this mode
If we know we expect data from sidedata, we read them from sidedata and nothing
else.
This avoid looking into extra for revision without sidedata entries. Such
revision will be introduced in the next changesets.
Differential Revision: https://phab.mercurial-scm.org/D7067
Yuya Nishihara <yuya@tcha.org> [Sun, 13 Oct 2019 20:06:09 +0900] rev 43229
dirs: remove mutable string optimization at all
As far as I can see, the optimization trick has been dead since
42e89b87ca79
"dirs: speed up by storing number of direct children per dir". After
42e89b87ca79, the key variable is cleared to NULL at each iteration.
Denis Laxalde <denis.laxalde@logilab.fr> [Tue, 15 Oct 2019 12:14:44 +0200] rev 43228
py3: prevent comparison with None in curses histedit (
issue6196)
Martin von Zweigbergk <martinvonz@google.com> [Sat, 12 Oct 2019 11:30:25 -0700] rev 43227
fix: match patterns relative to root
I was surprised fixer patterns (used to determine which fixers to run)
are applies to the parent directory, not the repo root
directory. Danny Hooper (the author of the extension) seemed to agree
that it's better to apply them to the repo root, so that's what this
patch does.
Differential Revision: https://phab.mercurial-scm.org/D7101