Mercurial > hg
comparison mercurial/bundle2.py @ 30177:9626022feaa4
bundle2: only emit compressed chunks if they have data
This is similar to 58467204cac0. Not all calls into the compressor
return compressed data, as the compressor may buffer compressed
output internally. It is cheaper to check for empty chunks than to
send empty chunks through the generator.
When generating a gzip-v2 bundle of the mozilla-unified repo, this
change results in 50,093 empty chunks not being sent through the
generator (out of 1,902,996 total input chunks).
author | Gregory Szorc <gregory.szorc@gmail.com> |
---|---|
date | Sat, 15 Oct 2016 17:10:53 -0700 |
parents | 0f6d6fdd3c2a |
children | f81002f736d7 |
comparison
equal
deleted
inserted
replaced
30176:9f41b66cffc0 | 30177:9626022feaa4 |
---|---|
571 yield _pack(_fstreamparamsize, len(param)) | 571 yield _pack(_fstreamparamsize, len(param)) |
572 if param: | 572 if param: |
573 yield param | 573 yield param |
574 # starting compression | 574 # starting compression |
575 for chunk in self._getcorechunk(): | 575 for chunk in self._getcorechunk(): |
576 yield self._compressor.compress(chunk) | 576 data = self._compressor.compress(chunk) |
577 if data: | |
578 yield data | |
577 yield self._compressor.flush() | 579 yield self._compressor.flush() |
578 | 580 |
579 def _paramchunk(self): | 581 def _paramchunk(self): |
580 """return a encoded version of all stream parameters""" | 582 """return a encoded version of all stream parameters""" |
581 blocks = [] | 583 blocks = [] |
1322 z = util.compressors[comp]() | 1324 z = util.compressors[comp]() |
1323 subchunkiter = cg.getchunks() | 1325 subchunkiter = cg.getchunks() |
1324 def chunkiter(): | 1326 def chunkiter(): |
1325 yield header | 1327 yield header |
1326 for chunk in subchunkiter: | 1328 for chunk in subchunkiter: |
1327 yield z.compress(chunk) | 1329 data = z.compress(chunk) |
1330 if data: | |
1331 yield data | |
1328 yield z.flush() | 1332 yield z.flush() |
1329 chunkiter = chunkiter() | 1333 chunkiter = chunkiter() |
1330 | 1334 |
1331 # parse the changegroup data, otherwise we will block | 1335 # parse the changegroup data, otherwise we will block |
1332 # in case of sshrepo because we don't know the end of the stream | 1336 # in case of sshrepo because we don't know the end of the stream |