Yuya Nishihara <yuya@tcha.org> [Sun, 18 Nov 2018 18:21:39 +0900] rev 40827
blackbox: pass in options to _openlogfile() as arguments
This prepares for extracting utility function from the blackbox module.
Yuya Nishihara <yuya@tcha.org> [Sat, 17 Nov 2018 22:10:27 +0900] rev 40826
blackbox: just try writing to repo.vfs and update lastlogger on success
This is simpler and more robust. Before, an empty ".hg" directory would be
created if it's removed after checking vfs.isdir('.').
Yuya Nishihara <yuya@tcha.org> [Tue, 20 Nov 2018 22:31:12 +0900] rev 40825
vfs: add option to not create parent directories implicitly
In blackbox, we don't want to create a ".hg" directory by mistake. This
provides a race-safe option to achieve that.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:55:33 +0100] rev 40824
repo: add a `wcachevfs` to access the `.hg/wcache/` directory
This wvfs will allow us to migrate various cache to the new `wcache` directory.
Helping with cache issues with "share".
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:46:31 +0100] rev 40823
cache: create `wcache` directory at init time
The cache directory will be needed very quickly, so it seems simpler to create
it early to make sure it has the same owner and permission than the other
directory in the repository.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 02:38:55 +0100] rev 40822
cache: create `cache` directory at init time
The cache directory will be needed very quickly, so it seems simpler to create
it early to make sure it has the same owner and permission than the other
directory in the repository.
Boris Feld <boris.feld@octobus.net> [Thu, 15 Nov 2018 17:08:23 +0100] rev 40821
check-exec: write file in 'wcache' instead of 'cache'
Some cache are relevant or affected by the working copy used. So the `.hg/cache`
directory is not the best place for them because multiple shared repository can
end up fighting over them.
To address this issue, we introduce a new 'wcache' directory to host this kind
of cache.
The first user are the `checkisexec` type file. These files describe property of
the working copy and fit the use-case well.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:09:44 +0100] rev 40820
mmapindex: set default to 1MB
mmapping index is more efficient if we only need a small part of it.
The 1MB value has been picked arbitrarily, a lower value might be better.
On a large repository with a 60MB index, we see the following performance
gain:
hg perfindex
before: ! wall 0.032023 comb 0.040000 user 0.000000 sys 0.040000 (best of 100)
after: ! wall 0.000196 comb 0.000000 user 0.000000 sys 0.000000 (best of 1060)
The speed boost benefit all cases, including the one where the full index
needs to be parsed.
hg perfindex --rev 0
before: ! wall 0.040673 comb 0.030000 user 0.000000 sys 0.030000 (best of 100)
after ! wall 0.010713 comb 0.020000 user 0.010000 sys 0.010000 (best of 212)
This gain reflect in higher level operation:
hg perfbookmarks --clear-revlogs
before: ! wall 0.161339 comb 0.160000 user 0.130000 sys 0.030000 (best of 56)
after: ! wall 0.123228 comb 0.120000 user 0.120000 sys 0.000000 (best of 68)
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:07:33 +0100] rev 40819
mmapindex: move the 'mmapindexthreshold' option out of experimental
The option is useful and should be advertised more. We move it out of
experimental as a first step. The `storage` section is selected as this is
related to how the storage is accessed. A new 'performance' section might be
more appropriate.
We move from 'mmapindexthreshold` to `mmap-threshold` as non-index item are
also suitable for mmap (eg: the rev-branch-cache).
If relevant, we can introduce sub-option `mmap-threshold.revlog-index` later.
Boris Feld <boris.feld@octobus.net> [Sat, 01 Dec 2018 15:57:27 +0100] rev 40818
perf: add a --rev attribute to perfindex
This allow for benchmarking the time necessary to look for other version than
the tip.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 06:03:38 +0100] rev 40817
perf: update perfindex to be more realistic
The previous code was creating a revlog manually, we now use the actual
`localrepo` method to create it.
We have to jump though extra hops to work around the impact of filecache.
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:09:46 -0800] rev 40816
match: drop unnecessary wrapping of regex in group
It seems the regexes have been wrapped in an unnamed group since
b6c42714d900 (Add locate command., 2005-07-05). In that commit, the
grouping was needed because there was a "head" ('^') added before the
group and a "tail" (os.sep) added after it. It seems the head was
moved inside the group in 1c0c413cccdd (Get add and locate to use new
repo and dirstate walk code., 2005-07-18) and the tail was moved
inside the group in 89985a1b3427 (Clean up walk and changes code to
use normalised names properly., 2005-07-31), So it seems to me that
we've carried around the unnecessary group for 13 years. This patch
removes it.
Differential Revision: https://phab.mercurial-scm.org/D5352
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:45:20 -0800] rev 40815
match: use _BASE_SIZE instead of magic value 4
Differential Revision: https://phab.mercurial-scm.org/D5351
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:44:49 -0800] rev 40814
match: make "groupsize" include the trailing "|"
I think this is a little easier to follow and it will simplify later
patches too.
Differential Revision: https://phab.mercurial-scm.org/D5350
Martin von Zweigbergk <martinvonz@google.com> [Sun, 02 Dec 2018 13:09:43 -0800] rev 40813
match: fix an unaligned (but harmless) indent
Differential Revision: https://phab.mercurial-scm.org/D5349
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:41:10 +0100] rev 40812
match: raise an Abort error instead of OverflowError
This case of OverflowError (one single pattern being too large) has never been
properly caught in the past.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 21:02:02 +0100] rev 40811
match: avoid translating glob to matcher multiple times for large sets
For hgignore with many globs, the resulting regexp might not fit under the 20K
length limit. So the patterns need to be broken up in smaller pieces.
Before this change, the logic was re-starting the full process from scratch
for each smaller pieces, including the translation of globs into regexp.
Effectively doing the work over and over.
If the 20K limit is reached, we are likely in a case where there is many such
glob, so exporting them is especially expensive and we should be careful not
to do that work more than once.
To work around this, we now translate glob to regexp once and for all. Then,
we assemble the resulting individual regexp into valid blocks.
This raises a very significant performance win for large `.hgignore file`:
Before: ! wall 0.153153 comb 0.150000 user 0.150000 sys 0.000000 (median of 66)
After: ! wall 0.059793 comb 0.060000 user 0.060000 sys 0.000000 (median of 100)
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:25:49 +0100] rev 40810
match: extract function that group regexps
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:16:05 +0100] rev 40809
match: test for overflow error in pattern
If a single pattern is too large to handle, we raise an exception. This case is
now doctested.
Boris Feld <boris.feld@octobus.net> [Thu, 22 Nov 2018 17:20:32 +0100] rev 40808
match: extract a literal constant into a symbolic one
Matt Harbison <matt_harbison@yahoo.com> [Sat, 01 Dec 2018 21:42:48 -0500] rev 40807
tests: apply binary mode to output in seq.py
I noticed this when playing with running tests using WSL, and iterating over the
output yielded '0\r', '1\r',... Most of the other *.py tools do this, and `seq`
on MSYS lacks '\r' in the output, so this is more consistent.
Boris Feld <boris.feld@octobus.net> [Fri, 23 Nov 2018 01:09:37 +0100] rev 40806
perf: add a `--clear-caches` to `perfbranchmapupdate`
This flag will help to measure the time we spend loading various cache that
support the branchmap update.
Example for an 500 000 revisions repository:
hg perfbranchmapupdate --base 'not tip' --target 'tip'
! wall 0.000860 comb 0.000000 user 0.000000 sys 0.000000 (best of 336)
hg perfbranchmapupdate --base 'not tip' --target 'tip' --clear-caches
! wall 0.029494 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 21:11:47 +0000] rev 40805
perf: start from an existing branchmap if possible
If the --base set if a superset of one of the cached branchmap, we should use as
a starting point. This greatly help the overall runtime of
`hg perfbranchmapupdate`
For example, for a repository with about 500 000 revisions, using this trick
make the command runtime move from about 200 second to about 10 seconds. A 20x
gain.
Boris Feld <boris.feld@octobus.net> [Wed, 21 Nov 2018 20:35:22 +0000] rev 40804
perf: rely on repoview for perfbranchmapupdate
Using 'repoview' matching the base and target subset make the benchmark more
realistic. It also unlocks optimization to make the command initialization
faster.