Mercurial > hg
comparison mercurial/util.py @ 11670:1b3b843e1100
chunkbuffer: split big strings directly in chunkbuffer
| author | Benoit Boissinot <benoit.boissinot@ens-lyon.org> |
|---|---|
| date | Sun, 25 Jul 2010 13:10:57 +0200 |
| parents | f070d284994c |
| children | 05deba16c5d5 |
comparison
equal
deleted
inserted
replaced
| 11669:c47cb3193c53 | 11670:1b3b843e1100 |
|---|---|
| 912 iterator over chunks of arbitrary size.""" | 912 iterator over chunks of arbitrary size.""" |
| 913 | 913 |
| 914 def __init__(self, in_iter): | 914 def __init__(self, in_iter): |
| 915 """in_iter is the iterator that's iterating over the input chunks. | 915 """in_iter is the iterator that's iterating over the input chunks. |
| 916 targetsize is how big a buffer to try to maintain.""" | 916 targetsize is how big a buffer to try to maintain.""" |
| 917 self.iter = iter(in_iter) | 917 def splitbig(chunks): |
| 918 for chunk in chunks: | |
| 919 if len(chunk) > 2**20: | |
| 920 pos = 0 | |
| 921 while pos < len(chunk): | |
| 922 end = pos + 2 ** 18 | |
| 923 yield chunk[pos:end] | |
| 924 pos = end | |
| 925 else: | |
| 926 yield chunk | |
| 927 self.iter = splitbig(in_iter) | |
| 918 self.buf = '' | 928 self.buf = '' |
| 919 | 929 |
| 920 def read(self, l): | 930 def read(self, l): |
| 921 """Read L bytes of data from the iterator of chunks of data. | 931 """Read L bytes of data from the iterator of chunks of data. |
| 922 Returns less than L bytes if the iterator runs dry.""" | 932 Returns less than L bytes if the iterator runs dry.""" |