Mercurial > hg

comparison contrib/python-zstandard/NEWS.rst @ 40121:73fef626dae3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
zstandard: vendor python-zstandard 0.10.1 This was just released. The upstream source distribution from PyPI was extracted. Unwanted files were removed. The clang-format ignore list was updated to reflect the new source of files. setup.py was updated to pass a new argument to python-zstandard's function for returning an Extension instance. Upstream had to change to use relative paths because Python 3.7's packaging doesn't seem to like absolute paths when defining sources, includes, etc. The default relative path calculation is relative to setup_zstd.py which is different from the directory of Mercurial's setup.py. The project contains a vendored copy of zstandard 1.3.6. The old version was 1.3.4. The API should be backwards compatible and nothing in core should need adjusted. However, there is a new "chunker" API that we may find useful in places where we want to emit compressed chunks of a fixed size. There are a pair of bug fixes in 0.10.0 with regards to compressobj() and decompressobj() when block flushing is used. I actually found these bugs when introducing these APIs in Mercurial! But existing Mercurial code is not affected because we don't perform block flushing. # no-check-commit because 3rd party code has different style guidelines Differential Revision: https://phab.mercurial-scm.org/D4911
author Gregory Szorc <gregory.szorc@gmail.com>
date Mon, 08 Oct 2018 16:27:40 -0700
parents b1fb341d8a61
children 675775c33ab6
comparison
equal deleted inserted replaced
40120:89742f1fa6cb 40121:73fef626dae3
28 * Audit for ability to control read/write sizes on all APIs. 28 * Audit for ability to control read/write sizes on all APIs.
29 * Detect memory leaks via bench.py. 29 * Detect memory leaks via bench.py.
30 * Remove low-level compression parameters from ``ZstdCompressor.__init__`` and 30 * Remove low-level compression parameters from ``ZstdCompressor.__init__`` and
31 require use of ``CompressionParameters``. 31 require use of ``CompressionParameters``.
32 * Expose ``ZSTD_getFrameProgression()`` from more compressor types. 32 * Expose ``ZSTD_getFrameProgression()`` from more compressor types.
33 * Support modifying compression parameters mid operation when supported by
34 zstd API.
35 * Expose ``ZSTD_CLEVEL_DEFAULT`` constant.
36 * Support ``ZSTD_p_forceAttachDict`` compression parameter.
37 * Use ``ZSTD_CCtx_getParameter()``/``ZSTD_CCtxParam_getParameter()`` for retrieving
38 compression parameters.
39 * Consider exposing ``ZSTDMT_toFlushNow()``.
40 * Expose ``ZDICT_trainFromBuffer_fastCover()``,
41 ``ZDICT_optimizeTrainFromBuffer_fastCover``.
42 * Expose and enforce ``ZSTD_minCLevel()`` for minimum compression level.
43 * Consider a ``chunker()`` API for decompression.
44 * Consider stats for ``chunker()`` API, including finding the last consumed
45 offset of input data.
33 46
34 Other Actions Not Blocking Release 47 Other Actions Not Blocking Release
35 --------------------------------------- 48 ---------------------------------------
36 49
37 * Support for block compression APIs. 50 * Support for block compression APIs.
38 * API for ensuring max memory ceiling isn't exceeded. 51 * API for ensuring max memory ceiling isn't exceeded.
39 * Move off nose for testing. 52 * Move off nose for testing.
53
54 0.10.1 (released 2018-10-08)
55 ============================
56
57 Backwards Compatibility Notes
58 -----------------------------
59
60 * ``ZstdCompressor.stream_reader().closed`` is now a property instead of a
61 method (#58).
62 * ``ZstdDecompressor.stream_reader().closed`` is now a property instead of a
63 method (#58).
64
65 Changes
66 -------
67
68 * Stop attempting to package Python 3.6 for Miniconda. The latest version of
69 Miniconda is using Python 3.7. The Python 3.6 Miniconda packages were a lie
70 since this were built against Python 3.7.
71 * ``ZstdCompressor.stream_reader()``'s and ``ZstdDecompressor.stream_reader()``'s
72 ``closed`` attribute is now a read-only property instead of a method. This now
73 properly matches the ``IOBase`` API and allows instances to be used in more
74 places that accept ``IOBase`` instances.
75
76 0.10.0 (released 2018-10-08)
77 ============================
78
79 Backwards Compatibility Notes
80 -----------------------------
81
82 * ``ZstdDecompressor.stream_reader().read()`` now consistently requires an
83 argument in both the C and CFFI backends. Before, the CFFI implementation
84 would assume a default value of ``-1``, which was later rejected.
85 * The ``compress_literals`` argument and attribute has been removed from
86 ``zstd.ZstdCompressionParameters`` because it was removed by the zstd 1.3.5
87 API.
88 * ``ZSTD_CCtx_setParametersUsingCCtxParams()`` is no longer called on every
89 operation performed against ``ZstdCompressor`` instances. The reason for this
90 change is that the zstd 1.3.5 API no longer allows this without calling
91 ``ZSTD_CCtx_resetParameters()`` first. But if we called
92 ``ZSTD_CCtx_resetParameters()`` on every operation, we'd have to redo
93 potentially expensive setup when using dictionaries. We now call
94 ``ZSTD_CCtx_reset()`` on every operation and don't attempt to change
95 compression parameters.
96 * Objects returned by ``ZstdCompressor.stream_reader()`` no longer need to be
97 used as a context manager. The context manager interface still exists and its
98 behavior is unchanged.
99 * Objects returned by ``ZstdDecompressor.stream_reader()`` no longer need to be
100 used as a context manager. The context manager interface still exists and its
101 behavior is unchanged.
102
103 Bug Fixes
104 ---------
105
106 * ``ZstdDecompressor.decompressobj().decompress()`` should now return all data
107 from internal buffers in more scenarios. Before, it was possible for data to
108 remain in internal buffers. This data would be emitted on a subsequent call
109 to ``decompress()``. The overall output stream would still be valid. But if
110 callers were expecting input data to exactly map to output data (say the
111 producer had used ``flush(COMPRESSOBJ_FLUSH_BLOCK)`` and was attempting to
112 map input chunks to output chunks), then the previous behavior would be
113 wrong. The new behavior is such that output from
114 ``flush(COMPRESSOBJ_FLUSH_BLOCK)`` fed into ``decompressobj().decompress()``
115 should produce all available compressed input.
116 * ``ZstdDecompressor.stream_reader().read()`` should no longer segfault after
117 a previous context manager resulted in error (#56).
118 * ``ZstdCompressor.compressobj().flush(COMPRESSOBJ_FLUSH_BLOCK)`` now returns
119 all data necessary to flush a block. Before, it was possible for the
120 ``flush()`` to not emit all data necessary to fully represent a block. This
121 would mean decompressors wouldn't be able to decompress all data that had been
122 fed into the compressor and ``flush()``ed. (#55).
123
124 New Features
125 ------------
126
127 * New module constants ``BLOCKSIZELOG_MAX``, ``BLOCKSIZE_MAX``,
128 ``TARGETLENGTH_MAX`` that expose constants from libzstd.
129 * New ``ZstdCompressor.chunker()`` API for manually feeding data into a
130 compressor and emitting chunks of a fixed size. Like ``compressobj()``, the
131 API doesn't impose restrictions on the input or output types for the
132 data streams. Unlike ``compressobj()``, it ensures output chunks are of a
133 fixed size. This makes this API useful when the compressed output is being
134 fed into an I/O layer, where uniform write sizes are useful.
135 * ``ZstdCompressor.stream_reader()`` no longer needs to be used as a context
136 manager (#34).
137 * ``ZstdDecompressor.stream_reader()`` no longer needs to be used as a context
138 manager (#34).
139 * Bundled zstandard library upgraded from 1.3.4 to 1.3.6.
140
141 Changes
142 -------
143
144 * Added ``zstd_cffi.py`` and ``NEWS.rst`` to ``MANIFEST.in``.
145 * ``zstandard.__version__`` is now defined (#50).
146 * Upgrade pip, setuptools, wheel, and cibuildwheel packages to latest versions.
147 * Upgrade various packages used in CI to latest versions. Notably tox (in
148 order to support Python 3.7).
149 * Use relative paths in setup.py to appease Python 3.7 (#51).
150 * Added CI for Python 3.7.
151
152 0.9.1 (released 2018-06-04)
153 ===========================
154
155 * Debian packaging support.
156 * Fix typo in setup.py (#44).
157 * Support building with mingw compiler (#46).
40 158
41 0.9.0 (released 2018-04-08) 159 0.9.0 (released 2018-04-08)
42 =========================== 160 ===========================
43 161
44 Backwards Compatibility Notes 162 Backwards Compatibility Notes
88 * Fixed memory leak of ``ZSTD_DDict`` instances in CFFI's ``ZstdDecompressor``. 206 * Fixed memory leak of ``ZSTD_DDict`` instances in CFFI's ``ZstdDecompressor``.
89 207
90 New Features 208 New Features
91 ------------ 209 ------------
92 210
93 * Bundlded zstandard library upgraded from 1.1.3 to 1.3.4. This delivers various 211 * Bundled zstandard library upgraded from 1.1.3 to 1.3.4. This delivers various
94 bug fixes and performance improvements. It also gives us access to newer 212 bug fixes and performance improvements. It also gives us access to newer
95 features. 213 features.
96 * Support for negative compression levels. 214 * Support for negative compression levels.
97 * Support for *long distance matching* (facilitates compression ratios that approach 215 * Support for *long distance matching* (facilitates compression ratios that approach
98 LZMA). 216 LZMA).