cpython/Misc/NEWS.d/next/Library/2022-09-30-09-22-37.gh-issue-95534.ndEfPj.rst at eae7dad40255bad42e4abce53ff8143dcbc66af5 - ilusha/cpython - Forgejo: Beyond coding. We forge.

ilusha/cpython

mirror of https://github.com/python/cpython synced 2026-09-27 00:43:20 +03:00

Ruben Vorderman eae7dad402

gh-95534: Improve gzip reading speed by 10% (#97664 )

Change summary:
+ There is now a `gzip.READ_BUFFER_SIZE` constant that is 128KB. Other programs that read in 128KB chunks: pigz and cat. So this seems best practice among good programs. Also it is faster than 8 kb chunks.
+ a zlib._ZlibDecompressor was added. This is the _bz2.BZ2Decompressor ported to zlib. Since the zlib.Decompress object is better for in-memory decompression, the _ZlibDecompressor is hidden. It only makes sense in file decompression, and that is already implemented now in the gzip library. No need to bother the users with this.
+ The ZlibDecompressor uses the older Cpython arrange_output_buffer functions, as those are faster and more appropriate for the use case. 
+ GzipFile.read has been optimized. There is no longer a `unconsumed_tail` member to write back to padded file. This is instead handled by the ZlibDecompressor itself, which has an internal buffer. `_add_read_data` has been inlined, as it was just two calls.

EDIT: While I am adding improvements anyway, I figured I could add another one-liner optimization now to the python -m gzip application. That read chunks in io.DEFAULT_BUFFER_SIZE previously, but has been updated now to use READ_BUFFER_SIZE chunks.

2022-10-16 19:10:58 -07:00

1 line

45 B

ReStructuredText

Raw Blame History

:meth:`gzip.GzipFile.read` reads 10% faster.