view relnotes/6.3 @ 50338:81c7d04f4722 stable

match: match explicit file using a set The matcher as all the logic to do quick comparison against explicit patterns, however the pattern matcher was shadowing the code using that set and used the compiled regex pattern in all cases, which is quite slow. We restore the usage of the set based matching to boost performance. Building the regexp is still consuming a large amount of time (actually, the majority of the time), which is still silly. Maybe using re2 would help that, but this is a quest for another adventure. Another path to improve this is to have a pattern type dedicated to match the exact path to a file only (not a directory). This pattern could use the set matching only and be skipped in the regex all together. Benchmarks ========== In the following benchmark we are comparing the `hg cat` and `hg files` run time when matching against all files in the repository. They are run: - without the rust extensions - with the standard python engine (so without re2) Performance improvement in this series -------------------------------------- ###### hg files ############################################################### ### mercurial-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 0.230092 seconds prev-changeset: 0.230069 seconds this-changeset: 0.211425 seconds (-8.36%) ### mercurial-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 0.234235 seconds prev-changeset: 0.231165 seconds (-1.38%) this-changeset: 0.212300 seconds (-9.43%) ### pypy-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 0.613567 seconds prev-changeset: 0.616799 seconds this-changeset: 0.510852 seconds (-16.82%) ### pypy-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 0.801880 seconds prev-changeset: 0.616393 seconds (-23.22%) this-changeset: 0.511903 seconds (-36.23%) ### netbeans-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 21.541828 seconds prev-changeset: 21.586773 seconds this-changeset: 13.648347 seconds (-36.76%) ### netbeans-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 172.759857 seconds prev-changeset: 21.908197 seconds (-87.32%) this-changeset: 13.945110 seconds (-91.93%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 62.474221 seconds prev-changeset: 61.279490 seconds (-1.22%) this-changeset: 29.529469 seconds (-52.40%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 1364.180218 seconds prev-changeset: 62.473549 seconds (-95.40%) this-changeset: 30.625249 seconds (-97.75%) ###### hg cat ################################################################# ### mercurial-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 0.764407 seconds prev-changeset: 0.763883 seconds this-changeset: 0.737326 seconds (-3.68%) ### mercurial-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 0.768924 seconds prev-changeset: 0.765848 seconds this-changeset: 0.174d0b seconds (-4.44%) ### pypy-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 2.065220 seconds prev-changeset: 2.070498 seconds this-changeset: 1.939482 seconds (-6.08%) ### pypy-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 2.276388 seconds prev-changeset: 2.069197 seconds (-9.15%) this-changeset: 1.931746 seconds (-15.19%) ### netbeans-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 40.967983 seconds prev-changeset: 41.392423 seconds this-changeset: 32.181681 seconds (-22.20%) ### netbeans-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 216.388709 seconds prev-changeset: 41.648689 seconds (-80.88%) this-changeset: 32.580817 seconds (-85.04%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### sorted base-changeset: 105.228510 seconds prev-changeset: 103.315670 seconds (-1.23%) this-changeset: 69.416118 seconds (-33.64%) ### mozilla-central-2018-08-01-zstd-sparse-revlog ### shuffled base-changeset: 1448.722784 seconds prev-changeset: 104.369358 seconds (-92.80%) this-changeset: 70.554789 seconds (-95.13%) Different way to list the same data with this revision ------------------------------------------------------ ###### hg files ############################################################### ### mercurial-2018-08-01-zstd-sparse-revlog root: 0.119182 seconds glob: 0.120697 seconds (+1.27%) sorted: 0.211425 seconds (+77.40%) shuffled: 0.212300 seconds (+78.13%) ### pypy-2018-08-01-zstd-sparse-revlog root: 0.121986 seconds glob: 0.124822 seconds (+2.32%) sorted: 0.510852 seconds (+318.78%) shuffled: 0.511903 seconds (+319.64%) ### netbeans-2018-08-01-zstd-sparse-revlog root: 0.173984 seconds glob: 0.227203 seconds (+30.59%) sorted: 13.648347 seconds (+7744.59%) shuffled: 13.945110 seconds (+7915.16%) ### mozilla-central-2018-08-01-zstd-sparse-revlog root: 0.366463 seconds glob: 0.491030 seconds (+33.99%) sorted: 29.529469 seconds (+7957.96%) shuffled: 30.625249 seconds (+8256.97%) ###### hg cat ################################################################# ### mercurial-2018-08-01-zstd-sparse-revlog glob: 0.647471 seconds root: 0.643120 seconds shuffled: 0.174d0b seconds (+13.92%) sorted: 0.737326 seconds (+13.88%) ### mozilla-central-2018-08-01-zstd-sparse-revlog glob: 40.596983 seconds root: 40.129136 seconds shuffled: 70.554789 seconds (+73.79%) sorted: 69.416118 seconds (+70.99%) ### netbeans-2018-08-01-zstd-sparse-revlog glob: 18.777924 seconds root: 18.613905 seconds shuffled: 32.580817 seconds (+73.51%) sorted: 32.181681 seconds (+71.38%) ### pypy-2018-08-01-zstd-sparse-revlog glob: 1.555319 seconds root: 1.536534 seconds shuffled: 1.931746 seconds (+24.20%) sorted: 1.939482 seconds (+24.70%)
author Pierre-Yves David <pierre-yves.david@octobus.net>
date Sat, 01 Apr 2023 05:58:59 +0200
parents 8830004967ad
children
line wrap: on
line source

= Mercurial 6.3.3 =

 * tests: filter out PEP 657 error locations in tracebacks (issue6780)
 * tests: optional PEP 657 error location in test-extension.t (issue6781)
 * tests: optional PEP 657 error location in test-lfs-serve-access.t (issue6782)
 * histedit: byteify the help for the multifold action
 * sparse: fix a py2 based usage of `map()`
 * convert: stop passing str to the dateutil API in darcs
 * convert: turn the last str regex into bytes in cvs.py (issue6789)
 * convert: change socket mode from b'r+' to 'rwb' in cvs.py (issue6789)
 * convert: replace repr() by pycompat.byterepr() in cvsps.py (issue6789)
 * tests: os module is frozen in Python 3.11 (issue6786)
 * hgweb: unbyteify the 100-continue check
 * resourceutil: start using importlib.resources.files() when possible
 * revset: the `random` sort should not depend on sys.maxsize (issue6770)
 * tests: make sure pygments can detect python script without extension
 * convert: brz 3.3.0 moved NoSuchFile exception to breezy.transport
 * tests: pygments 2.14+ highlight whitespace in python code
 * hghave: make different has_pyoxidizer functions have different names
 * hghave: refactor checks for pygments versions using checkvers()
 * rust-narrow: fix loop that never loops
 * scmutil: make checknewlabel() allow "_" in otherwise numeric names (issue6737)
 * bundlerepo: enforce the requirements declared by the underlying repository
 * setup: make the version computation process more resistant
 * fix: add more information to the debug output
 * tag: disallow tagging the working directory
 * dirstate: handle missing backup file on restoration
 * dirstate-v2: complain early on docket name collision
 * rust: upgrade minimum `rayon` dependency
 * setup: support building from an ongoing merge
 * rust: add debug log about skipping dirstate update
 * rust-dirstate: trace append/no append to help debugging
 * dirstate-v2: don't mmap the data file when on NFS
 * run-tests: make it possible to nest conditionals
 * dirstate: add some debug output when writing the dirstate
 * testing: introduce util function to synchronize concurrent commands on files
 * dirstate: add a way to test races happening during status
 * dirstate: use more than a bool to control append behavior
 * dirstate-v2: add devel config option to control write behavior
 * rhg: fix race when a fixup file is deleted on disk
 * rhg: fix race when an ambiguous file is deleted on disk
 * dirstate: deal with read-race for pure python code
 * dirstate: deal with read-race for python code using rust object
 * dirstate: deal with read-race for pure rust code path (rhg)
 * dirstate: set identity whenever we read the dirstate's v2 docket
 * rust-dirstate-v2: don't write dirstate if data file has changed
 * rhg: remember the inode of .hg/dirstate

= Mercurial 6.3.2 =

 * [ecfc84b956a8] tests: expect the message from 1baf0fffd82f in test-hghave.t (issue6762)
 * [5c095119bff4] tests: add the missing space to test-hghave.t (issue6762)
 * [2c346c1c75ec] tests: use an all too familiar executable in test-run-tests.t (issue6661)
 * [13c0e3b4fd35] tests: use `test -f` instead of `ls` to see if a file is present (issue6662)
 * [8ced4ca30ea1] bisect: correct message about aborting an in-progress bisect (issue6527)
 * filemerge: fix crash when using filesets in [partial-merge-tools]
 * help: fix a py3 error interpolating Set into b'%s'
 * match: make the FLAG_RE pattern a raw string
 * python-compat: adapt to Python 3.11 BC breakage with `random.sample`
 * rust-status: fix thread count ceiling
 * hg: show the correct message when cloning an LFS repo with extension disabled
 * extensions: process disabled external paths when `hgext` package is in-memory
 * emitrevision: consider ancestors revision to emit as available base
 * make: add a target for building pyoxidizer tests on macOS
 * run-tests: support --pyoxidized on macOS
 * packaging: add dependencies to the PyOxidizer build on macOS
 * Miscellaneous test fixes

= Mercurial 6.3.1 =

 * memory-usage: fix `hg log --follow --rev R F` space complexity (dcb2581e33be)
 * Improve portability and robustness of test harness
 * hg-core: relax dependencies pinning
 * matcher: fix issues regex flag contained in pattern (issue6759)
 * matcher: do not prepend '.*' to pattern using ^ after flags
 * packaging: refresh dependency hashes (issue6750)
 * cffi: fix a bytes vs str issue on macOS when listing directories
 * commit: properly consider file include and exclude options when closing branch
 * dirstate-v2: do not put the dirstate data file in a transaction

= Mercurial 6.3 =

== New Features ==

 * testlib: add `--raw-sha1` option to `f`
 * rhg: add `config.rhg` helptext
 * config: add alias from `hg help rhg` to `hg help rust`
 * rhg: add a config option to fall back immediately
 * bundle: introduce a --exact option
 * perf-bundle: add a new command to benchmark bundle creation time
 * perf-bundle: accept --rev arguments
 * perf-bundle: accept --type argument
 * perf-unbundle: add a perf command to time the unbundle operation
 * perf: introduce a benchmark for delta-find
 * contrib: add support for rhel9
 * phase-shelve: Implement a 'shelve.store' experimental config
 * debug-delta-find: introduce a quiet mode
 * sort-revset: introduce a `random` variant
 * phase: introduce a dedicated requirement for the `archived` phase
 * rebase: add boolean config item rebase.store-source
 * rhg: make [rhg status -v] work when it needs no extra output
 * rhg: support "!" syntax for disabling extensions
 * rhg: add debugrhgsparse command to help figure out bugs in rhg
 * rhg: add sparse support
 * rhg-status: add support for narrow clones
 * templates: add filter to reverse list
 * contrib: add pull_logger extension
 * revset: handle wdir() in `roots()`
 * revset: handle wdir() in `sort(..., -topo)`
 * rhg: support tweakdefaults
 * rhg: parallellize computation of [unsure_is_modified]

== Default Format Change ==

These changes affect newly created repositories (or new clones) done with
Mercurial 6.3.

== New Experimental Features ==

== Bug Fixes ==

 * shelve: demonstrate that the state is different across platforms (issue6735)
 * shelve: in test for trailing whitespace, strip commit (issue6735)
 * shelve: remove strip and rely on prior state (issue6735)
 * tests: fix http-bad-server expected errors for python 3.10 (issue6643)
 * status: let `--no-copies` override `ui.statuscopies`
 * releasenotes: use re.MULTILINE mode when checking admonitions
 * rhg: fallback to slow path on invalid patterns in hgignore
 * Fix a bunch of leftover str/bytes issues from Python 3 migration
 * keepalive: ensure `close_all()` actually closes all cached connections
 * lfs: fix blob corruption when tranferring with workers on posix
 * lfs: avoid closing connections when the worker doesn't fork
 * dirstate-v2: update constant that wasn't kept in sync
 * dirstate-v2: fix edge case where entries aren't sorted
 * upgrade: no longer keep all revlogs in memory at any point
 * rust-status: save new dircache even if just invalidated
 * dirstate-v2: hash the source of the ignore patterns as well
 * rhg: fallback when encountering ellipsis revisions
 * shelve: handle empty parents and nodestoremove in shelvedstate (issue6748)
 * profile: prevent a crash when line number is unknown
 * tags-fnode-cache: do not repeatedly open the filelog in a loop
 * tags-fnode-cache: skip building a changectx in getfnode
 * rust: create wrapper struct to reduce `regex` contention issues

== Backwards Compatibility Changes ==

 * chg worker processes will now correctly load per-repository configuration
   when given a both a relative `--repository` path and an alternate working
   directory via `--cwd`. A side-effect of this change is that these workers
   will now return an error if hg cannot find the current working directory,
   even when a different directory is specified via `--cwd`.
 * phase: rename the requirement for internal-phase from `internal-phase` to `use-internal-phase` (see 74fb1842f8b962cf03d7cd5b841dbcf2ae065587)

== Internal API Changes ==

== Miscellaneous ==

 * sslutil: use proper attribute to select python 3.7+
 * typing: suppress a few pyi-errors with more recent pytype
 * ci: bump pytype to 2022.03.29
 * bundlespec: add documentation about existing option
 * subrepo: avoid opening console window for non-native subrepos on Windows
 * setup: unconditionally enable the `long-paths-support` option on Windows
 * setup: use the full executable manifest from `python.exe`
 * tests: work around libmagic bug in svn subrepo tests
 * packagelib: use python3 by default
 * Improve `hg bisect` performance
 * perf: properly process formatter option in perf::unbundle
 * compare-disco: miscellaneous display improvements
 * fsmonitor: better compatibility with newer Pythons
 * revlog: finer computation of "issnapshot"
 * rhg: don't fallback if `strip` or `rebase` are activated
 * perf: make perf::bundle compatible before 61ba04693d65
 * perf: make perf::bundle compatible down to 5.2
 * perf-unbundle: improve compatibility
 * run-tests: display the time it took to install Mercurial
 * mergetools: don't let meld open all changed files on startup
 * dirstate-v2: skip evaluation of hgignore regex on cached directories