Merged
Conversation
Much faster and comparable quality to level 9
alexeagle
approved these changes
Aug 7, 2023
aiuto
approved these changes
Aug 9, 2023
Collaborator
aiuto
left a comment
There was a problem hiding this comment.
I know this will come back to haunt me when I try to import it into Google and someone's bloat detector goes off. But I'm feeling lucky.
Expect that in a few weeks I'll come back with a new version that creates a constant for the default at the top of the file, so that it is easy to change for the entire org while vendoring in the code.
flode
added a commit
to flode/rules_pkg
that referenced
this pull request
Aug 17, 2023
Zip offers different compression algorithms and levels. For packaging, sometimes faster compression speed is more important than size. With pkg_tar, there is already the option to use a custom compressor. For pkg_zip these use case are now possible with the exposed zip compression_level and compression_type arguments in this PR. There's a new test case that verifies that the compression settings work by comparing the compressed file sizes. The default compression level is the same as recently change for pkg_tar in bazelbuild#720
flode
added a commit
to flode/rules_pkg
that referenced
this pull request
Aug 17, 2023
Zip offers different compression algorithms and levels. For packaging, sometimes faster compression speed is more important than size. With pkg_tar, there is already the option to use a custom compressor. For pkg_zip these use case are now possible with the exposed zip compression_level and compression_type arguments in this PR. There's a new test case that verifies that the compression settings work by comparing the compressed file sizes. The default compression level is the same as recently change for pkg_tar in bazelbuild#720
flode
added a commit
to flode/rules_pkg
that referenced
this pull request
Aug 17, 2023
Zip offers different compression algorithms and levels. For packaging, sometimes faster compression speed is more important than size. With pkg_tar, there is already the option to use a custom compressor. For pkg_zip these use case are now possible with the exposed zip compression_level and compression_type arguments in this PR. There's a new test case that verifies that the compression settings work by comparing the compressed file sizes. The default compression level is the same as recently change for pkg_tar in bazelbuild#720
flode
added a commit
to flode/rules_pkg
that referenced
this pull request
Aug 17, 2023
Zip offers different compression algorithms and levels. For packaging, sometimes faster compression speed is more important than size. With pkg_tar, there is already the option to use a custom compressor. For pkg_zip these use case are now possible with the exposed zip compression_level and compression_type arguments in this PR. There's a new test case that verifies that the compression settings work by comparing the compressed file sizes. The default compression level is the same as recently change for pkg_tar in bazelbuild#720
alexeagle
added a commit
that referenced
this pull request
Aug 17, 2023
Zip offers different compression algorithms and levels. For packaging, sometimes faster compression speed is more important than size. With `pkg_tar`, there is already the option to use a custom compressor, there's no similar option for `pkg_zip` yet. This PR exposes the zip `compression_level` and `compression_type` arguments for `pkg_zip`, thus enabling the use case. The list of compressions is the same as Python's `ZipFile`: https://docs.python.org/3/library/zipfile.html#zipfile.ZIP_STORED There's a new test case that verifies that the compression settings work by comparing the compressed file sizes. The default compression level is the same as in this recent change for `pkg_tar` in #720
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Much faster and comparable quality to level 9
In the example below, level 9 (max) compression is 2% smaller and 50% slower compared to level 6. This is why most gzip implementations default to level 6.
BUILD.bazel
WORKSPACE.bazel
Level 6: 183MB, 36s
Level 9: 179MB, 54s
(FYI, as a note for anyone caring about performance, using an external gzip binary cut the time by a further 6s.)