Matt Harbison <matt_harbison@yahoo.com> [Fri, 23 Nov 2018 22:18:56 -0500] rev 40711
remotefilelog: drop some compatibility cruft for finding the hg exeutable
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 21 Nov 2018 17:17:26 +0300] rev 40710
branchmap: build the revbranchcache._namesreverse() only when required
On big repositories with a lot of named branches and that also increasing over
time, building of this dict can be expensive and shows up in profile.
For our internal repository, this saves ~0.05 seconds.
Thanks to Yuya for suggesting using util.propertycache() and
util.clearcachedproperty().
Differential Revision: https://phab.mercurial-scm.org/D5291
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 11:11:38 +0100] rev 40709
sparse-revlog: put the native implementation of slicechunktodensity to use
When possible, the C implementation of delta chain slicing will be used.
providing a large boost in performance for this operation.
To take a practical example of restoring manifest revision '
59547c40bc4c' for
a reference NetBeans repository (using sparse-revlog). The media time of the
step `slice-sparse-chain` of `perfrevlogrevision` improve from 0.660 ms to
0.098 ms;
The full series move delta chain slicing from 1.120 ms to 0.098 ms;
Implementing _slicechunktosize into C would yield further improvements.
However, the performance seems good enough for now.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 11:10:52 +0100] rev 40708
cext: increment the `parsers` module version
We introduced a new `slicechunktodensity` method to the index.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 11:09:58 +0100] rev 40707
sparse-revlog: introduce native (C) implementation of slicechunktodensity
This is a C implementation of `_slicechunktodensity` in the
`mercurial/revlogutils/deltas.py` file.
The algorithm involves a lot of integer manipulation and low-level access to
index data. Having a C implementation of it raises a large performance
improvement. See later changeset in this series for details.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Nov 2018 18:45:23 +0100] rev 40706
sparse-revlog: add a `trim_endidx` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first.
This function implement a subpart of the python function `_trimchunk` in
`mercurial/revlogutils/deltas.py`. Handling of actual Python objects is left
to the caller function.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Nov 2018 18:43:16 +0100] rev 40705
sparse-revlog: add a `index_segment_span` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first. This new function provides
an efficient way to retrieve some of the information needed by
`slicechunktodensity`.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Nov 2018 18:42:58 +0100] rev 40704
sparse-revlog: add a `index_get_length` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first. This new function provides
an efficient way to retrieve some of the information needed by
`slicechunktodensity`.
Boris Feld <boris.feld@octobus.net> [Tue, 20 Nov 2018 14:49:18 +0000] rev 40703
sparse-revlog: add a `index_get_start` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first. This new function provides
an efficient way to retrieve some of the information needed by
`slicechunktodensity`.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:32:32 +0100] rev 40702
perf: add --clear-revlog flag to branchmapload
Having the changelog index already loaded when loading the branchmap can have a
large impact on performance.
Example runs (large private repository):
hg perfbranchmapload -f base
! wall 0.116722 comb 0.120000 user 0.110000 sys 0.010000 (best of 59)
hg perfbranchmapload -f base --clear-revlogs
! wall 0.258246 comb 0.230000 user 0.220000 sys 0.010000 (best of 31)
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:32:28 +0100] rev 40701
perf: introduce a function to fully "unload" a changelog
The function remove various attributes and caches related to changelog.
This is getting a common requirement.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 01:32:57 +0100] rev 40700
perf: use an explicit function in perfbranchmapload
This make things clearer.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 01:20:18 +0100] rev 40699
perf: update function name to match `perfbranchmapload` command
Having function with the same name as the command is simpler.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 11:46:47 +0000] rev 40698
perf: fix a minor typo in perfbranchmapload
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:37 +0200] rev 40697
context: floor adjustlinkrev graph walk during copy tracing
The `_adjustlinkrev` method gains an optional "stoprev" argument. The linkrev
adjustment will give up once this floor is reached. The relevant functions
using `_adjustlinkrev` are updated to pass an appropriate value in the copy
tracing code.
In some private repository, about 10% of the status call triggered the
pathological case addressed by this change. The speedup varies from one call
to another, the best-observed win is moving from 170s to 11s.
The effect of this change can be seen in the public pypy repository, running the
following command:
hg perftracecopies --source
83c9ff0c0206 --destination
59c79103d5b0
before: 3.401753 seconds
after: 2.634897 seconds (-23%)
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 14:30:58 +0000] rev 40696
context: small refactoring of `isintroducedafter`
This make the next change clearer.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:36 +0200] rev 40695
context: split `introrev` logic in a sub function
We want to add a mechanism to stop iteration early associated to intro rev early
in some case. However, it does not make sense to expose it in the public
`filectx` API. So we split the code into an internal method instead.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:35 +0200] rev 40694
context: introduce an `isintroducedafter` method and use it in copies
Right now, copy tracing make effort to not traverse the graph too much to save
performance. It uses a "limit" acting as a floor revision past which data are
no longer relevant to the current copy tracing.
However, to enforce this limit, it does a call to `filectx.rev()` and that
call can trigger a graph traversal on its own. That extra graph traversal is
unaware of the current limit and can become very expensive. That cost is
increased by the nature of work done in adjust link rev, we are not only
walking down the graph, we are also checking the affected file for each
revision we walk through. Something significantly more expensive than the walk
itself.
To work around this we need to make the `filectx` operation aware of the
current limit. The first step is to introduce a dedicated method:
`isintroducedafter`. We'll then rework that method logic to stop traversal as
soon as possible.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:34 +0200] rev 40693
context: take advantage of `_descendantrev` in introrev if available
Before this changeset, `_descendantrev` was ignored and `introrev` could
return a "wrong" result. I was previously fine because there seems to be no
existing code using both `introrev` and `_descendantrev` at the same time.
However, we would like to change that.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:49:30 +0200] rev 40692
context: spell out the logic around linkrev adjustement starting point
We make the intent of the `_changeid` and `_changectx` checks explicit. The
same logic was previously performed by the `self.rev()` call. The new code is
a bit redundant, but much clearer.
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 14:14:56 +0000] rev 40691
perf: add a new `perfhelper-tracecopies` command
The command is not measuring performance itself, it digs interesting statistic
to help pick good arguments for the `perfcopytrace` command.
Martin von Zweigbergk <martinvonz@google.com> [Fri, 21 Sep 2018 09:19:42 -0700] rev 40690
narrow: extract helper for parsing narrowspec file
This will be used for parsing a file that's stored in a different
location.
Differential Revision: https://phab.mercurial-scm.org/D5277
Yuya Nishihara <yuya@tcha.org> [Thu, 22 Nov 2018 22:40:22 +0900] rev 40689
py3: cast attribute name to sysstr in clearcachedproperty()
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 21 Nov 2018 19:12:12 +0300] rev 40688
py3: whitelist three more passing test caught by buildbot
The number of failing test is back to two digits.
Differential Revision: https://phab.mercurial-scm.org/D5293
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Wed, 21 Nov 2018 13:08:23 -0500] rev 40687
tests: make test-check-module-imports more robust
It failed for me without this in this way:
tests/test-commandserver.t:19: relative import of stdlib module
tests/test-lfs-serve.t:108: relative import of stdlib module
tests/test-lfs-serve.t:255: relative import of stdlib module
tests/test-lfs-serve.t:362: relative import of stdlib module
tests/test-lfs-serve.t:406: relative import of stdlib module
tests/test-lock.py:9: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-lrucachedict.py:5: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-match.py:5: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-remotefilelog-datapack.py:15: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-remotefilelog-histpack.py:14: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-simplekeyvaluefile.py:4: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-sshserver.py:6: imports not lexically sorted: silenttestrunner < unittest, True, True
This is because every module is considered a stdlib module, because
the stdlib_prefixes is /usr, and my repo is in /usr/local/home, which
means that sys.path contains a couple of
/usr/local/home/../hg/.. entries that count as "in the stdlib".
Fix this by preventing any path in sys.path that's inside the mercurial
source from being considered "in the stdlib".
Differential Revision: https://phab.mercurial-scm.org/D5294
Anton Shestakov <av6@dwimlabs.net> [Sat, 27 Oct 2018 21:13:23 +0800] rev 40686
push: add --publish flag to change phase of pushed changesets
This flag is copied from topic extension, where it proved to be useful. It
makes all pushed changesets public on both ends if the push succeeds.
Doesn't currently work for any subrepos that are implicitly pushed together
with their owner repo.
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 17:19:54 +0000] rev 40685
match: reformat `syntaxes` dictionary for better maintainability
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 18:54:44 +0000] rev 40684
match: provide and use a quick way to escape a single byte
The previous function has a lot of overhead (including being a function). In
the `_globre` case, we always escape a single byte. So we provide a dictionary
dedicated to this use case. We directly use the dictionary to avoid a function
call, these are expensive in Python.
Again, this raise a very significant performance gain:
Before: ! wall 0.059793 comb 0.060000 user 0.060000 sys 0.000000 (median of 100)
After: ! wall 0.020390 comb 0.020000 user 0.020000 sys 0.000000 (median of 146)
Total improvement for the full series:
Before: ! wall 0.153153 comb 0.150000 user 0.150000 sys 0.000000 (median of 66)
After: ! wall 0.020390 comb 0.020000 user 0.020000 sys 0.000000 (median of 146)