annotate mercurial/pushkey.py @ 23497:5817f71c2336

obsstore: disable garbage collection during initialization (issue4456) Python garbage collection is triggered by container creation. So code that creates a lot of tuples tends to trigger GC a lot. We disable the gc during obsolescence marker parsing and associated initialization. This provides an interesting speedup (25%). Load marker function on my 58758 markers repo: before: 0.468247 seconds after: 0.344362 seconds The benefit is a bit less visible overall. With python2.6 on my system I see: after: 0.60 before: 0.53 The difference is probably explained by the delaying of a costly GC. (but there is still a win). Marking involved tuples, lists and dicts as ignorable by the garbage collector should give us more benefit. But this is another adventure. Thanks goes to Siddharth Agarwal for the lead.
author Pierre-Yves David <pierre-yves.david@fb.com>
date Wed, 26 Nov 2014 16:58:31 -0800
parents b1d694d3975e
children 7b200566e474
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11367
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
1 # pushkey.py - dispatching for pushing and pulling keys
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
2 #
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
3 # Copyright 2010 Matt Mackall <mpm@selenic.com>
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
4 #
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
6 # GNU General Public License version 2 or any later version.
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
7
21650
a2c7ae21e8f4 pushkey: introduce an ``encodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 17298
diff changeset
8 import bookmarks, phases, obsolete, encoding
13353
689bf32b3bbd bookmarks: move pushkey functions into core
Matt Mackall <mpm@selenic.com>
parents: 11367
diff changeset
9
11367
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
10 def _nslist(repo):
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
11 n = {}
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
12 for k in _namespaces:
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
13 n[k] = ""
22953
b1d694d3975e obsolete: add exchange option
Durham Goode <durham@fb.com>
parents: 21661
diff changeset
14 if not obsolete.isenabled(repo, obsolete.exchangeopt):
17298
59c14bf5a48c pushkey: do not exchange obsole markers if feature is disabled
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 17075
diff changeset
15 n.pop('obsolete')
11367
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
16 return n
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
17
13353
689bf32b3bbd bookmarks: move pushkey functions into core
Matt Mackall <mpm@selenic.com>
parents: 11367
diff changeset
18 _namespaces = {"namespaces": (lambda *x: False, _nslist),
15648
79cc89de5be1 phases: add basic pushkey support
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 13353
diff changeset
19 "bookmarks": (bookmarks.pushbookmark, bookmarks.listbookmarks),
79cc89de5be1 phases: add basic pushkey support
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 13353
diff changeset
20 "phases": (phases.pushphase, phases.listphases),
17075
28ed1c4511ce obsolete: exchange obsolete marker over pushkey
Pierre-Yves.David@ens-lyon.org
parents: 15648
diff changeset
21 "obsolete": (obsolete.pushmarker, obsolete.listmarkers),
15648
79cc89de5be1 phases: add basic pushkey support
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 13353
diff changeset
22 }
11367
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
23
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
24 def register(namespace, pushkey, listkeys):
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
25 _namespaces[namespace] = (pushkey, listkeys)
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
26
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
27 def _get(namespace):
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
28 return _namespaces.get(namespace, (lambda *x: False, lambda *x: {}))
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
29
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
30 def push(repo, namespace, key, old, new):
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
31 '''should succeed iff value was old'''
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
32 pk = _get(namespace)[0]
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
33 return pk(repo, key, old, new)
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
34
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
35 def list(repo, namespace):
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
36 '''return a dict'''
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
37 lk = _get(namespace)[1]
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
38 return lk(repo)
ca4fc993087c pushkey: add pushkey core
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
39
21661
2f52a16f2bee pushkey: add an ``encode`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21659
diff changeset
40 encode = encoding.fromlocal
2f52a16f2bee pushkey: add an ``encode`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21659
diff changeset
41
21659
a319842539f5 pushkey: add a ``decode`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21652
diff changeset
42 decode = encoding.tolocal
a319842539f5 pushkey: add a ``decode`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21652
diff changeset
43
21650
a2c7ae21e8f4 pushkey: introduce an ``encodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 17298
diff changeset
44 def encodekeys(keys):
a2c7ae21e8f4 pushkey: introduce an ``encodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 17298
diff changeset
45 """encode the content of a pushkey namespace for exchange over the wire"""
21661
2f52a16f2bee pushkey: add an ``encode`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21659
diff changeset
46 return '\n'.join(['%s\t%s' % (encode(k), encode(v)) for k, v in keys])
21652
ed6e61eaebc0 pushkey: introduce an ``decodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21650
diff changeset
47
ed6e61eaebc0 pushkey: introduce an ``decodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21650
diff changeset
48 def decodekeys(data):
ed6e61eaebc0 pushkey: introduce an ``decodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21650
diff changeset
49 """decode the content of a pushkey namespace from exchange over the wire"""
ed6e61eaebc0 pushkey: introduce an ``decodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21650
diff changeset
50 result = {}
ed6e61eaebc0 pushkey: introduce an ``decodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21650
diff changeset
51 for l in data.splitlines():
ed6e61eaebc0 pushkey: introduce an ``decodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21650
diff changeset
52 k, v = l.split('\t')
21659
a319842539f5 pushkey: add a ``decode`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21652
diff changeset
53 result[decode(k)] = decode(v)
21652
ed6e61eaebc0 pushkey: introduce an ``decodekeys`` function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21650
diff changeset
54 return result