view tests/test-obsolete-tag-cache.t @ 26243:836291420d53

revlog: optionally cache the full text when adding revisions

revlog instances can cache the full text of a single revision. Typically
the most recently read revision is cached.

When adding a delta group via addgroup() and _addrevision(), the
full text isn't always computed: sometimes only the passed in delta is
sufficient for adding a new revision to the revlog.

When writing the changelog from a delta group, the just-added full
text revision is always read immediately after it is written because
the changegroup code needs to extract the set of files from the entry.
In other words, revision() is *always* being called and caching the full
text of the just-added revision is guaranteed to result in a cache hit,
making the cache worthwhile.

This patch adds support to _addrevision() for always building and
caching the full text. This option is currently only active when
processing changelog entries from a changegroup.

While the total number of revision() calls is the same, the location
matters: buildtext() calls into revision() on the base revision when
building the full text of the just-added revision. Since the previous
revision's _addrevision() built the full text and the the previous
revision is likely the base revision, this means that the base
revision's full text is likely cached and can be used to compute the
current full text from just a delta. No extra I/O required.

The end result is the changelog isn't opened and read after adding every
revision from a changegroup.

On my 2013 MacBook Pro running OS X 10.10.5 from an SSD and Python 2.7,
this patch impacted the time taken to apply ~262,000 changesets from a
mozilla-central gzip bundle:

  before: ~43s
  after:  ~32s

~25% reduction in changelog processing times. Not bad.

  $ cat >> $HGRCPATH << EOF
  > [extensions]
  > blackbox=
  > rebase=
  > mock=$TESTDIR/mockblackbox.py
  > 
  > [experimental]
  > evolution = createmarkers
  > EOF

Create a repo with some tags

  $ hg init repo
  $ cd repo
  $ echo initial > foo
  $ hg -q commit -A -m initial
  $ hg tag -m 'test tag' test1
  $ echo first > first
  $ hg -q commit -A -m first
  $ hg tag -m 'test2 tag' test2
  $ hg -q up -r 0
  $ echo newhead > newhead
  $ hg commit -A -m newhead
  adding newhead
  created new head
  $ hg tag -m 'test head 2 tag' head2

  $ hg log -G -T '{rev}:{node|short} {tags} {desc}\n'
  @  5:2942a772f72a tip test head 2 tag
  |
  o  4:042eb6bfcc49 head2 newhead
  |
  | o  3:c3cb30f2d2cd  test2 tag
  | |
  | o  2:d75775ffbc6b test2 first
  | |
  | o  1:5f97d42da03f  test tag
  |/
  o  0:55482a6fb4b1 test1 initial
  

Trigger tags cache population by doing something that accesses tags info

  $ hg tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49
  test2                              2:d75775ffbc6b
  test1                              0:55482a6fb4b1

  $ cat .hg/cache/tags2-visible
  5 2942a772f72a444bef4bef13874d515f50fa27b6
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2
  55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1
  d75775ffbc6bca1794d300f5571272879bd280da test2

Hiding a non-tip changeset should change filtered hash and cause tags recompute

  $ hg debugobsolete -d '0 0' c3cb30f2d2cd0aae008cc91a07876e3c5131fd22 -u dummyuser

  $ hg tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49
  test1                              0:55482a6fb4b1

  $ cat .hg/cache/tags2-visible
  5 2942a772f72a444bef4bef13874d515f50fa27b6 f34fbc9a9769ba9eff5aff3d008a6b49f85c08b1
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2
  55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1

  $ hg blackbox -l 4
  1970/01/01 00:00:00 bob (*)> tags (glob)
  1970/01/01 00:00:00 bob (*)> 2/2 cache hits/lookups in * seconds (glob)
  1970/01/01 00:00:00 bob (*)> writing .hg/cache/tags2-visible with 2 tags (glob)
  1970/01/01 00:00:00 bob (*)> tags exited 0 after * seconds (glob)

Hiding another changeset should cause the filtered hash to change

  $ hg debugobsolete -d '0 0' d75775ffbc6bca1794d300f5571272879bd280da -u dummyuser
  $ hg debugobsolete -d '0 0' 5f97d42da03fd56f3b228b03dfe48af5c0adf75b -u dummyuser

  $ hg tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49

  $ cat .hg/cache/tags2-visible
  5 2942a772f72a444bef4bef13874d515f50fa27b6 2fce1eec33263d08a4d04293960fc73a555230e4
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2

  $ hg blackbox -l 4
  1970/01/01 00:00:00 bob (*)> tags (glob)
  1970/01/01 00:00:00 bob (*)> 1/1 cache hits/lookups in * seconds (glob)
  1970/01/01 00:00:00 bob (*)> writing .hg/cache/tags2-visible with 1 tags (glob)
  1970/01/01 00:00:00 bob (*)> tags exited 0 after * seconds (glob)

Resolving tags on an unfiltered repo writes a separate tags cache

  $ hg --hidden tags
  tip                                5:2942a772f72a
  head2                              4:042eb6bfcc49
  test2                              2:d75775ffbc6b
  test1                              0:55482a6fb4b1

  $ cat .hg/cache/tags2
  5 2942a772f72a444bef4bef13874d515f50fa27b6
  042eb6bfcc4909bad84a1cbf6eb1ddf0ab587d41 head2
  55482a6fb4b1881fa8f746fd52cf6f096bb21c89 test1
  d75775ffbc6bca1794d300f5571272879bd280da test2

  $ hg blackbox -l 4
  1970/01/01 00:00:00 bob (*)> --hidden tags (glob)
  1970/01/01 00:00:00 bob (*)> 2/2 cache hits/lookups in * seconds (glob)
  1970/01/01 00:00:00 bob (*)> writing .hg/cache/tags2 with 3 tags (glob)
  1970/01/01 00:00:00 bob (*)> --hidden tags exited 0 after * seconds (glob)