Matt Mackall <mpm@selenic.com> [Thu, 11 Oct 2007 00:46:54 -0500] rev 5451
revlog: break up compression of large deltas
Python's zlib apparently makes an internal copy of strings passed to
compress(). To avoid this, compress strings 1M at a time, then join
them at the end if the result would be smaller than the original.
For initial commits of large but compressible files, this cuts peak
memory usage nearly in half.
Matt Mackall <mpm@selenic.com> [Thu, 11 Oct 2007 00:46:53 -0500] rev 5450
revlog: fix caching of buffer objects
Matt Mackall <mpm@selenic.com> [Thu, 11 Oct 2007 00:46:52 -0500] rev 5449
chunkiter: handle large reads more efficiently
- for large reads, don't attempt to read more than necessary
- if we've gathered the exact number of bytes needed, avoid a string copy
Matt Mackall <mpm@selenic.com> [Thu, 11 Oct 2007 00:46:51 -0500] rev 5448
revlog: avoid large yields in group()
Split large yields so that the downstream consumer (chunkbuffer) will avoid
some pain when reading ahead.
Matt Mackall <mpm@selenic.com> [Thu, 11 Oct 2007 00:46:49 -0500] rev 5447
chunkiter: simplify iter logic
Matt Mackall <mpm@selenic.com> [Thu, 11 Oct 2007 00:46:48 -0500] rev 5446
chunkbuffer: removed unused method and arg