mmapindex: set default to 1MB mmapping index is more efficient if we only need a small part of it. The 1MB value has been picked arbitrarily, a lower value might be better. On a large repository with a 60MB index, we see the following performance gain: hg perfindex before: ! wall 0.032023 comb 0.040000 user 0.000000 sys 0.040000 (best of 100) after: ! wall 0.000196 comb 0.000000 user 0.000000 sys 0.000000 (best of 1060) The speed boost benefit all cases, including the one where the full index needs to be parsed. hg perfindex --rev 0 before: ! wall 0.040673 comb 0.030000 user 0.000000 sys 0.030000 (best of 100) after ! wall 0.010713 comb 0.020000 user 0.010000 sys 0.010000 (best of 212) This gain reflect in higher level operation: hg perfbookmarks --clear-revlogs before: ! wall 0.161339 comb 0.160000 user 0.130000 sys 0.030000 (best of 56) after: ! wall 0.123228 comb 0.120000 user 0.120000 sys 0.000000 (best of 68)

mmapindex: move the 'mmapindexthreshold' option out of experimental The option is useful and should be advertised more. We move it out of experimental as a first step. The `storage` section is selected as this is related to how the storage is accessed. A new 'performance' section might be more appropriate. We move from 'mmapindexthreshold` to `mmap-threshold` as non-index item are also suitable for mmap (eg: the rev-branch-cache). If relevant, we can introduce sub-option `mmap-threshold.revlog-index` later.

perf: add a --rev attribute to perfindex This allow for benchmarking the time necessary to look for other version than the tip.

perf: update perfindex to be more realistic The previous code was creating a revlog manually, we now use the actual `localrepo` method to create it. We have to jump though extra hops to work around the impact of filecache.

match: drop unnecessary wrapping of regex in group It seems the regexes have been wrapped in an unnamed group since b6c42714d900 (Add locate command., 2005-07-05). In that commit, the grouping was needed because there was a "head" ('^') added before the group and a "tail" (os.sep) added after it. It seems the head was moved inside the group in 1c0c413cccdd (Get add and locate to use new repo and dirstate walk code., 2005-07-18) and the tail was moved inside the group in 89985a1b3427 (Clean up walk and changes code to use normalised names properly., 2005-07-31), So it seems to me that we've carried around the unnecessary group for 13 years. This patch removes it. Differential Revision: https://phab.mercurial-scm.org/D5352

match: use _BASE_SIZE instead of magic value 4 Differential Revision: https://phab.mercurial-scm.org/D5351

match: make "groupsize" include the trailing "|" I think this is a little easier to follow and it will simplify later patches too. Differential Revision: https://phab.mercurial-scm.org/D5350

match: fix an unaligned (but harmless) indent Differential Revision: https://phab.mercurial-scm.org/D5349

match: raise an Abort error instead of OverflowError This case of OverflowError (one single pattern being too large) has never been properly caught in the past.

match: avoid translating glob to matcher multiple times for large sets For hgignore with many globs, the resulting regexp might not fit under the 20K length limit. So the patterns need to be broken up in smaller pieces. Before this change, the logic was re-starting the full process from scratch for each smaller pieces, including the translation of globs into regexp. Effectively doing the work over and over. If the 20K limit is reached, we are likely in a case where there is many such glob, so exporting them is especially expensive and we should be careful not to do that work more than once. To work around this, we now translate glob to regexp once and for all. Then, we assemble the resulting individual regexp into valid blocks. This raises a very significant performance win for large `.hgignore file`: Before: ! wall 0.153153 comb 0.150000 user 0.150000 sys 0.000000 (median of 66) After: ! wall 0.059793 comb 0.060000 user 0.060000 sys 0.000000 (median of 100)