rust: speed up zstd decompression by re-using the decompression context
Admittedly, zstd is already pretty fast, but this change makes it
a bit faster yet: it saves ~5% of time it takes to read our large repo.
The actual motivating use case is treemanifest: in treemanifest
we end up reading *lots* of small directories, and many of them
need decompression, and there the saving for [rhg files] is >10%.
(which also seems unreasonable, we should probably keep things uncompressed more)
import argparse
import os
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
def gather():
for p in opts.path:
if not os.path.exists(p):
return
if os.path.isdir(p):
yield p + os.path.sep
for dirpath, dirs, files in os.walk(p):
for d in dirs:
yield os.path.join(dirpath, d) + os.path.sep
for f in files:
yield os.path.join(dirpath, f)
else:
yield p
print('\n'.join(sorted(gather(), key=lambda x: x.replace(os.path.sep, '/'))))