annotate contrib/memory.py @ 26945:8a256cee72c8 stable

tags: create new sortdict for performance reasons sortdict internally maintains a list of keys in insertion order. When a key is replaced via __setitem__, we .remove() from this list. This involves a linear scan and array adjustment. This is an expensive operation. The tags reading code was calling into sortdict.__setitem__ for each tag in a read .hgtags revision. For repositories with thousands of tags or thousands of .hgtags revisions, the overhead from list.remove() noticeable. This patch creates a new sortdict() so __setitem__ calls don't incur a list.remove. This doesn't appear to have any performance impact on my Firefox repository. But that's only because tags reading doesn't show up in profiles to begin with. I'm still waiting to hear from a user with over 10,000 tags and hundreds of heads on the impact of this patch.
author Gregory Szorc <gregory.szorc@gmail.com>
date Thu, 12 Nov 2015 13:16:04 -0800
parents 08a0f04b56bd
children 3e0d27d298b7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
1 # memory.py - track memory usage
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
2 #
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
3 # Copyright 2009 Matt Mackall <mpm@selenic.com> and others
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
4 #
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
10264
d6512b3e9ac0 Merge with stable
Matt Mackall <mpm@selenic.com>
parents: 10017
diff changeset
6 # GNU General Public License version 2 or any later version.
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
7
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
8 '''helper extension to measure memory usage
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
9
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
10 Reads current and peak memory usage from ``/proc/self/status`` and
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
11 prints it to ``stderr`` on exit.
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
12 '''
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
13
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
14 import atexit
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
15
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
16 def memusage(ui):
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
17 """Report memory usage of the current process."""
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
18 status = None
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
19 result = {'peak': 0, 'rss': 0}
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
20 try:
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
21 # This will only work on systems with a /proc file system
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
22 # (like Linux).
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
23 status = open('/proc/self/status', 'r')
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
24 for line in status:
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
25 parts = line.split()
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
26 key = parts[0][2:-1].lower()
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
27 if key in result:
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
28 result[key] = int(parts[1])
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
29 finally:
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
30 if status is not None:
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
31 status.close()
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10264
diff changeset
32 ui.write_err(", ".join(["%s: %.1f MiB" % (key, value / 1024.0)
10017
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
33 for key, value in result.iteritems()]) + "\n")
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
34
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
35 def extsetup(ui):
253d0da256b2 contrib: helper extension to track memory usage
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
36 atexit.register(memusage, ui)