Yuya Nishihara <yuya@tcha.org> [Sat, 24 Nov 2018 18:38:44 +0900] rev 40754
annotate: check and update command options first
It helps extracting helper class.
Yuya Nishihara <yuya@tcha.org> [Sat, 24 Nov 2018 18:36:44 +0900] rev 40753
annotate: format local tables in less-dense form
I'll move these to a class, where one-extra indent will be needed.
Pulkit Goyal <pulkit@yandex-team.ru> [Thu, 22 Nov 2018 18:14:21 +0300] rev 40752
perf: rename version flag of perfchangegroupchangelog to cgversion
The version flag conflicts with the global version flag and does not work.
Differential Revision: https://phab.mercurial-scm.org/D5297
Differential Revision: https://phab.mercurial-scm.org/D5298
Matt Harbison <matt_harbison@yahoo.com> [Fri, 23 Nov 2018 22:27:57 -0500] rev 40751
procutil: don't allow the main 'hg' script to be treated as the Windows exe
Previously, there were a handful of errors like this:
$ hg prefetch --repack
(running background incremental repack)
+ abort: %1 is not a valid Win32 application
+ [255]
CreateProcess() doesn't append .exe when `lpApplicationName` contains a path,
and a python script isn't directly executable.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 23 Nov 2018 22:18:56 -0500] rev 40750
remotefilelog: drop some compatibility cruft for finding the hg exeutable
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 21 Nov 2018 17:17:26 +0300] rev 40749
branchmap: build the revbranchcache._namesreverse() only when required
On big repositories with a lot of named branches and that also increasing over
time, building of this dict can be expensive and shows up in profile.
For our internal repository, this saves ~0.05 seconds.
Thanks to Yuya for suggesting using util.propertycache() and
util.clearcachedproperty().
Differential Revision: https://phab.mercurial-scm.org/D5291
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 11:11:38 +0100] rev 40748
sparse-revlog: put the native implementation of slicechunktodensity to use
When possible, the C implementation of delta chain slicing will be used.
providing a large boost in performance for this operation.
To take a practical example of restoring manifest revision '59547c40bc4c' for
a reference NetBeans repository (using sparse-revlog). The media time of the
step `slice-sparse-chain` of `perfrevlogrevision` improve from 0.660 ms to
0.098 ms;
The full series move delta chain slicing from 1.120 ms to 0.098 ms;
Implementing _slicechunktosize into C would yield further improvements.
However, the performance seems good enough for now.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 11:10:52 +0100] rev 40747
cext: increment the `parsers` module version
We introduced a new `slicechunktodensity` method to the index.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 11:09:58 +0100] rev 40746
sparse-revlog: introduce native (C) implementation of slicechunktodensity
This is a C implementation of `_slicechunktodensity` in the
`mercurial/revlogutils/deltas.py` file.
The algorithm involves a lot of integer manipulation and low-level access to
index data. Having a C implementation of it raises a large performance
improvement. See later changeset in this series for details.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Nov 2018 18:45:23 +0100] rev 40745
sparse-revlog: add a `trim_endidx` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first.
This function implement a subpart of the python function `_trimchunk` in
`mercurial/revlogutils/deltas.py`. Handling of actual Python objects is left
to the caller function.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Nov 2018 18:43:16 +0100] rev 40744
sparse-revlog: add a `index_segment_span` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first. This new function provides
an efficient way to retrieve some of the information needed by
`slicechunktodensity`.
Boris Feld <boris.feld@octobus.net> [Fri, 09 Nov 2018 18:42:58 +0100] rev 40743
sparse-revlog: add a `index_get_length` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first. This new function provides
an efficient way to retrieve some of the information needed by
`slicechunktodensity`.
Boris Feld <boris.feld@octobus.net> [Tue, 20 Nov 2018 14:49:18 +0000] rev 40742
sparse-revlog: add a `index_get_start` function in C
We are about to implement a native version of `slicechunktodensity`. For
clarity, we introduce the helper functions first. This new function provides
an efficient way to retrieve some of the information needed by
`slicechunktodensity`.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:32:32 +0100] rev 40741
perf: add --clear-revlog flag to branchmapload
Having the changelog index already loaded when loading the branchmap can have a
large impact on performance.
Example runs (large private repository):
hg perfbranchmapload -f base
! wall 0.116722 comb 0.120000 user 0.110000 sys 0.010000 (best of 59)
hg perfbranchmapload -f base --clear-revlogs
! wall 0.258246 comb 0.230000 user 0.220000 sys 0.010000 (best of 31)
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:32:28 +0100] rev 40740
perf: introduce a function to fully "unload" a changelog
The function remove various attributes and caches related to changelog.
This is getting a common requirement.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 01:32:57 +0100] rev 40739
perf: use an explicit function in perfbranchmapload
This make things clearer.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 01:20:18 +0100] rev 40738
perf: update function name to match `perfbranchmapload` command
Having function with the same name as the command is simpler.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 11:46:47 +0000] rev 40737
perf: fix a minor typo in perfbranchmapload
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:37 +0200] rev 40736
context: floor adjustlinkrev graph walk during copy tracing
The `_adjustlinkrev` method gains an optional "stoprev" argument. The linkrev
adjustment will give up once this floor is reached. The relevant functions
using `_adjustlinkrev` are updated to pass an appropriate value in the copy
tracing code.
In some private repository, about 10% of the status call triggered the
pathological case addressed by this change. The speedup varies from one call
to another, the best-observed win is moving from 170s to 11s.
The effect of this change can be seen in the public pypy repository, running the
following command:
hg perftracecopies --source 83c9ff0c0206 --destination 59c79103d5b0
before: 3.401753 seconds
after: 2.634897 seconds (-23%)
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 14:30:58 +0000] rev 40735
context: small refactoring of `isintroducedafter`
This make the next change clearer.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:36 +0200] rev 40734
context: split `introrev` logic in a sub function
We want to add a mechanism to stop iteration early associated to intro rev early
in some case. However, it does not make sense to expose it in the public
`filectx` API. So we split the code into an internal method instead.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:35 +0200] rev 40733
context: introduce an `isintroducedafter` method and use it in copies
Right now, copy tracing make effort to not traverse the graph too much to save
performance. It uses a "limit" acting as a floor revision past which data are
no longer relevant to the current copy tracing.
However, to enforce this limit, it does a call to `filectx.rev()` and that
call can trigger a graph traversal on its own. That extra graph traversal is
unaware of the current limit and can become very expensive. That cost is
increased by the nature of work done in adjust link rev, we are not only
walking down the graph, we are also checking the affected file for each
revision we walk through. Something significantly more expensive than the walk
itself.
To work around this we need to make the `filectx` operation aware of the
current limit. The first step is to introduce a dedicated method:
`isintroducedafter`. We'll then rework that method logic to stop traversal as
soon as possible.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:50:34 +0200] rev 40732
context: take advantage of `_descendantrev` in introrev if available
Before this changeset, `_descendantrev` was ignored and `introrev` could
return a "wrong" result. I was previously fine because there seems to be no
existing code using both `introrev` and `_descendantrev` at the same time.
However, we would like to change that.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:49:30 +0200] rev 40731
context: spell out the logic around linkrev adjustement starting point
We make the intent of the `_changeid` and `_changectx` checks explicit. The
same logic was previously performed by the `self.rev()` call. The new code is
a bit redundant, but much clearer.
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 14:14:56 +0000] rev 40730
perf: add a new `perfhelper-tracecopies` command
The command is not measuring performance itself, it digs interesting statistic
to help pick good arguments for the `perfcopytrace` command.
Martin von Zweigbergk <martinvonz@google.com> [Fri, 21 Sep 2018 09:19:42 -0700] rev 40729
narrow: extract helper for parsing narrowspec file
This will be used for parsing a file that's stored in a different
location.
Differential Revision: https://phab.mercurial-scm.org/D5277
Yuya Nishihara <yuya@tcha.org> [Thu, 22 Nov 2018 22:40:22 +0900] rev 40728
py3: cast attribute name to sysstr in clearcachedproperty()
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 21 Nov 2018 19:12:12 +0300] rev 40727
py3: whitelist three more passing test caught by buildbot
The number of failing test is back to two digits.
Differential Revision: https://phab.mercurial-scm.org/D5293
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Wed, 21 Nov 2018 13:08:23 -0500] rev 40726
tests: make test-check-module-imports more robust
It failed for me without this in this way:
tests/test-commandserver.t:19: relative import of stdlib module
tests/test-lfs-serve.t:108: relative import of stdlib module
tests/test-lfs-serve.t:255: relative import of stdlib module
tests/test-lfs-serve.t:362: relative import of stdlib module
tests/test-lfs-serve.t:406: relative import of stdlib module
tests/test-lock.py:9: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-lrucachedict.py:5: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-match.py:5: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-remotefilelog-datapack.py:15: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-remotefilelog-histpack.py:14: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-simplekeyvaluefile.py:4: imports not lexically sorted: silenttestrunner < unittest, True, True
tests/test-sshserver.py:6: imports not lexically sorted: silenttestrunner < unittest, True, True
This is because every module is considered a stdlib module, because
the stdlib_prefixes is /usr, and my repo is in /usr/local/home, which
means that sys.path contains a couple of
/usr/local/home/../hg/.. entries that count as "in the stdlib".
Fix this by preventing any path in sys.path that's inside the mercurial
source from being considered "in the stdlib".
Differential Revision: https://phab.mercurial-scm.org/D5294
Anton Shestakov <av6@dwimlabs.net> [Sat, 27 Oct 2018 21:13:23 +0800] rev 40725
push: add --publish flag to change phase of pushed changesets
This flag is copied from topic extension, where it proved to be useful. It
makes all pushed changesets public on both ends if the push succeeds.
Doesn't currently work for any subrepos that are implicitly pushed together
with their owner repo.
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 17:19:54 +0000] rev 40724
match: reformat `syntaxes` dictionary for better maintainability
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 18:54:44 +0000] rev 40723
match: provide and use a quick way to escape a single byte
The previous function has a lot of overhead (including being a function). In
the `_globre` case, we always escape a single byte. So we provide a dictionary
dedicated to this use case. We directly use the dictionary to avoid a function
call, these are expensive in Python.
Again, this raise a very significant performance gain:
Before: ! wall 0.059793 comb 0.060000 user 0.060000 sys 0.000000 (median of 100)
After: ! wall 0.020390 comb 0.020000 user 0.020000 sys 0.000000 (median of 146)
Total improvement for the full series:
Before: ! wall 0.153153 comb 0.150000 user 0.150000 sys 0.000000 (median of 66)
After: ! wall 0.020390 comb 0.020000 user 0.020000 sys 0.000000 (median of 146)
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 16:50:21 +0000] rev 40722
perf: make `clearfilecache` helper work with any object
If the object is not a localrepo, it won't have an `unfiltered` method (and
won't need one).
Boris Feld <boris.feld@octobus.net> [Tue, 20 Nov 2018 10:10:25 +0000] rev 40721
perf: move some of the perftags benchmark to the setup function
Creating fresh objects and clearing the cache should not be part of the
timing.
before: ! wall 0.020851 comb 0.020000 user 0.020000 sys 0.000000 (median of 138)
after: ! wall 0.018740 comb 0.020000 user 0.020000 sys 0.000000 (median of 141)
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 23:14:46 +0000] rev 40720
perf: use the new setup function in "perfbookmarks"
This command was picked arbitrarily to display the usefulness of the new
feature. In my Mercurial repository (with very few bookmarks), moving cache
cleanup in the dedicated setup function has a visible and stable effect on
the benchmark number.
before: ! wall 0.000061 comb 0.000000 user 0.000000 sys 0.000000 (median of 40837)
after: ! wall 0.000058 comb 0.000000 user 0.000000 sys 0.000000 (median of 40500)
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 22:55:07 +0000] rev 40719
perf: add a `setup` argument to run code outside of the timed section
With this new argument, it is possible to perform special setup and cleanup
outside of code actually timed. This is useful to provide more accurate
benchmark.
Boris Feld <boris.feld@octobus.net> [Mon, 19 Nov 2018 23:02:29 +0000] rev 40718
perf: explicitly pass title as a keyword argument in `perfdiffwd`
This will help to update the timer function arguments in a later changeset.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 17:59:23 +0900] rev 40717
ui: unify argument name of ui.log()
It's called an "event" in both blackbox and logtoprocess.
Yuya Nishihara <yuya@tcha.org> [Sun, 11 Nov 2018 17:45:18 +0900] rev 40716
logtoprocess: extract logger class from ui wrapper
It provides the same interface as the blackboxlogger. The ui wrapper will
be removed shortly.
Yuya Nishihara <yuya@tcha.org> [Wed, 21 Nov 2018 21:40:16 +0900] rev 40715
merge with stable
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 19 Nov 2018 21:12:13 +0300] rev 40714
py3: use node.hex(m.digest()) instead of m.hexdigest()
hashlib.sha1.hexdigest() returns str on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5287
Boris Feld <boris.feld@octobus.net> [Sun, 18 Nov 2018 02:40:47 +0100] rev 40713
tests: add Balto configuration file
I have been developing a Mercurial test runner plugin for being able to run
Mercurial tests with Balto
(https://bitbucket.org/lothiraldan/balto/src/default/).
Balto requires a configuration file so let's include it, I have added the
required information in the configuration file as comments.
I hope Balto would be an helpful tool for other people than me.
Differential Revision: https://phab.mercurial-scm.org/D5283
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Mon, 19 Nov 2018 13:40:03 -0500] rev 40712
tests: make test-alias.t pass with re2
Locally, these "non-ASCII character in alias" errors don't show up,
though I get them when the alias is defined at the command line rather
than in an hgrc.
The brokenness comes from the fact that hgrcs are parsed with regexes,
and re/re2 differ in this way:
$ python -c 'import re; print(re.compile("(.*)").match("aaa\xc0bbbb").groups())'
('aaa\xc0bbbb',)
$ python -c 'import re2; print(re2.compile("(.*)").match("aaa\xc0bbbb").groups())'
('aaa',)
Apparently re2 stops when it encounters invalid utf8 (which I suppose makes sense
given that '.' matches what appears to be a codepoint rather than a byte). This is
presumably a bug in hg, but not very important, so just change the test to stick
to valid utf8.
Differential Revision: https://phab.mercurial-scm.org/D5288
Martin von Zweigbergk <martinvonz@google.com> [Mon, 19 Nov 2018 23:08:09 -0800] rev 40711
context: remove seemingly impossible code branch
I'm not a Python expert, but I can't think of a way that the following
branch can ever be hit:
def _changeid(self):
if r'_changeid' in self.__dict__:
return self._changeid
It seems to me that if that condition is true, then this function
would not have been called. The only exception I can think of is if a
reference to the function had been stored beforehand, something like this:
c = fctx.__dict__['_changeid']
fctx._changeid
c()
But that seems like very unlikely code to exist.
The condition was added in 921b64e1f7b9 (filecontext: use 'is not
None' to check for filelog existence, 2013-05-01) as a "bonus" change
(in addition to what the patch was actually about)
Differential Revision: https://phab.mercurial-scm.org/D5289
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 19 Nov 2018 21:11:08 +0300] rev 40710
py3: use pycompat.byteskwargs() to covert keys of kwargs to bytes
Differential Revision: https://phab.mercurial-scm.org/D5286
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 19 Nov 2018 20:30:07 +0300] rev 40709
py3: use '%d' for integers instead of '%s'
Differential Revision: https://phab.mercurial-scm.org/D5285
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 19 Nov 2018 19:57:11 +0300] rev 40708
py3: add 9 new passing tests caught by buildbot
Differential Revision: https://phab.mercurial-scm.org/D5284
Anton Shestakov <av6@dwimlabs.net> [Fri, 09 Nov 2018 13:57:13 +0800] rev 40707
branch: allow changing branch of merge commits with --rev
Tests show that changing branch of merge commits works fine with evolution and
without, so let's allow it. Other safeguards should prevent users from shooting
themselves in the foot.