Mercurial > hg
changeset 23226:5dcaed20b27c stable
changegroup: sparsely populate fnodes
Previously, fnodes had a key and empty dict value for every element in
changedfiles. This is somewhat wasteful. Empty dicts in CPython consume
a lot more memory than you would expect - 280 bytes.
On mozilla-central, which has ~190,000 files/fnodes keys, the previous
loop populating fnodes allocated 91,924 KB of memory, most of that for
the empty dicts.
With this patch in place, our peak RSS during mozilla-central clone
drops:
before: 364,356 KB
after: 326,008 KB
delta: -38,348 KB
When combined with the previous patch, total peak RSS decrease is now
190,116 KB.
| author | Gregory Szorc <gregory.szorc@gmail.com> |
|---|---|
| date | Thu, 06 Nov 2014 22:48:20 -0800 |
| parents | bdf7b1ea1dae |
| children | 32dadb2637f4 |
| files | mercurial/changegroup.py |
| diffstat | 1 files changed, 1 insertions(+), 4 deletions(-) [+] |
line wrap: on
line diff
--- a/mercurial/changegroup.py Thu Nov 06 22:33:48 2014 -0800 +++ b/mercurial/changegroup.py Thu Nov 06 22:48:20 2014 -0800 @@ -335,9 +335,6 @@ yield chunk progress(msgbundling, None) - for f in changedfiles: - fnodes[f] = {} - # Callback for the manifest, used to collect linkrevs for filelog # revisions. # Returns the linkrev node (collected in lookupcl). @@ -349,7 +346,7 @@ if f in changedfiles: # record the first changeset introducing this filelog # version - fnodes[f].setdefault(n, clnode) + fnodes.setdefault(f, {}).setdefault(n, clnode) return clnode mfnodes = self.prune(mf, mfs, commonrevs, source)