annotate mercurial/utils/memorytop.py @ 46296:eec8899407f4

shelve: also create class representing whole directory of shelves It's a little annoying to have to create and pass in a vfs into `listshelves()`. This patch attempts to start addressing that by creating a class that represents a directory containing shelves (the directory can be either `.hg/shelved/` or `.hg/shelve-backup/`). Differential Revision: https://phab.mercurial-scm.org/D9743
author Martin von Zweigbergk <martinvonz@google.com>
date Tue, 12 Jan 2021 09:02:47 -0800
parents 5b6c0af021da
children 1c5810ce737e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
45797
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
1 # memorytop requires Python 3.4
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
2 #
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
3 # Usage: set PYTHONTRACEMALLOC=n in the environment of the hg invocation,
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
4 # where n>= is the number of frames to show in the backtrace. Put calls to
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
5 # memorytop in strategic places to show the current memory use by allocation
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
6 # site.
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
7
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
8 import gc
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
9 import tracemalloc
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
10
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
11
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
12 def memorytop(limit=10):
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
13 gc.collect()
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
14 snapshot = tracemalloc.take_snapshot()
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
15
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
16 snapshot = snapshot.filter_traces(
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
17 (
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
18 tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
19 tracemalloc.Filter(False, "<frozen importlib._bootstrap_external>"),
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
20 tracemalloc.Filter(False, "<unknown>"),
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
21 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
22 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
23 stats = snapshot.statistics('traceback')
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
24
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
25 total = sum(stat.size for stat in stats)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
26 print("\nTotal allocated size: %.1f KiB\n" % (total / 1024))
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
27 print("Lines with the biggest net allocations")
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
28 for index, stat in enumerate(stats[:limit], 1):
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
29 print(
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
30 "#%d: %d objects using %.1f KiB"
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
31 % (index, stat.count, stat.size / 1024)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
32 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
33 for line in stat.traceback.format(most_recent_first=True):
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
34 print(' ', line)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
35
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
36 other = stats[limit:]
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
37 if other:
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
38 size = sum(stat.size for stat in other)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
39 count = sum(stat.count for stat in other)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
40 print(
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
41 "%s other: %d objects using %.1f KiB"
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
42 % (len(other), count, size / 1024)
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
43 )
5b6c0af021da utils: helper function to print top memory allocation site
Joerg Sonnenberger <joerg@bec.de>
parents:
diff changeset
44 print()