[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566 by SteNicholas · Pull Request #3555 · apache/celeborn

SteNicholas · 2025-12-03T11:44:59Z

What changes were proposed in this pull request?

Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566.
Lz4Decompressor follows the suggestion to move from fastDecompressor to safeDecompressor to mitigate the performance.

Backport:

Why are the changes needed?

CVE‐2025‐12183: Various lz4-java compression and decompression implementations do not guard against out-of-bounds memory access. Untrusted input may lead to denial of service and information disclosure. Vulnerable Maven coordinates: org.lz4:lz4-java up to and including 1.8.0.
CVE-2025-66566: Insufficient clearing of the output buffer in Java-based decompressor implementations in lz4-java 1.10.0 and earlier allows remote attackers to read previous buffer contents via crafted compressed input. In applications where the output buffer is reused without being cleared, this may lead to disclosure of sensitive data. JNI-based implementations are not affected.

Therefore, lz4-java version should upgrade to 1.10.4.

Does this PR resolve a correctness bug?

No.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

CI.

yawkat · 2025-12-03T14:01:11Z

I recommend you stick with fastestInstance. It is secure as long as you are on 1.8.1+. It will be much slower than in previous versions, but that can be mitigated by moving to fastestInstance.safeDecompressor like you did in this PR, which is much faster.

SteNicholas · 2025-12-04T01:50:32Z

@yawkat, thanks for review. I have sticked with fastestInstance for the workaround With the 1.8.1 patch applied, these workarounds are not necessary. It is still recommended to move from fastDecompressor to safeDecompressor to mitigate the performance impact of the fix, however. PTAL.

client/src/main/java/org/apache/celeborn/client/compress/Lz4Decompressor.java

yawkat · 2025-12-11T06:42:11Z

Also fyi there was another cve (CVE-2025-66566) that needs a newer version

Marcono1234 · 2025-12-12T20:08:13Z

CVE-2025-66566 affects versions less than or equal to 1.10.0. You should upgrade to 1.10.1.

SteNicholas · 2025-12-16T06:41:11Z

Ping @pan3793, @yawkat, @Marcono1234.

client-tez/tez-shaded/pom.xml

pan3793 · 2025-12-16T08:10:37Z

lz4 is famous for its ultra-fast speed, the upgrade is not free, my test shows it has perf impact - apache/spark#53453

I understand that security takes precedence over performance, so I'm fine with this change.

for the suggestion of 'moving to fastestInstance.safeDecompressor', I think we can NOT do that blindly - Celeborn Spark/Flink clients use the lz4-java libs provided by the engine libs, since we support a wide range of Spark/Flink versions, it's possible that the engine still ships old lz4-java jar, we may need to dynamiclly check and bind the fastDecompressor or safeDecompressor based on runtime version of lz4-java

yawkat · 2025-12-16T08:13:30Z

@pan3793 safeDecompressor should work just fine on old versions, and even on those old versions, it should be slightly faster than fastDecompressor. In fact, using safeDecompressor gets rid of most (but not all) of the security impact of the CVEs on old versions.

client/benchmarks/LZ4TPCDSDataBenchmark-jdk17-results.txt

pan3793 · 2026-03-02T13:02:41Z

@SteNicholas Code change looks fine, but let's wait for a while to collect feedback from other reviewers, about the performance drop.

SteNicholas · 2026-03-02T13:12:58Z

@yawkat. could you please take a look at performance drop of safeDecompressor which refers to the benchmark report of LZ4TPCDSDataBenchmark-jdk17-results.txt?

yawkat · 2026-03-02T17:37:07Z

@SteNicholas I don't have time for deep benchmarking, but I just threw an AI agent at it, and it figured out that some build flag changes that distros do can improve performance by 10%. Could you test with the 1.10.4 I just released?

pan3793 · 2026-03-03T01:59:51Z

@yawkat thanks! it indeed solves the performance regression. our benchmark shows 1.10.4 is much faster than 1.10.3, and even faster than 1.8.0! and I got a similar result in Spark apache/spark#54585

RexXiong · 2026-03-03T02:02:16Z

@yawkat thanks! it indeed solves the performance regression. our benchmark shows 1.10.4 is much faster than 1.10.3, and even faster than 1.8.0! and I got a similar result in Spark apache/spark#54585

Sounds Great, I think we can keep LZ4 as default.

… CVE‐2025‐12183 and CVE-2025-66566

… CVE‐2025‐12183 and CVE-2025-66566 - Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566. - `Lz4Decompressor` follows the [suggestion](apache/spark#53290 (comment)) to move from `fastDecompressor` to `safeDecompressor` to mitigate the performance. Backport: - apache/spark#53327 - apache/spark#53347 - apache/spark#53971 - apache/spark#53454 - apache/spark#54585 - [CVE‐2025‐12183](https://sites.google.com/sonatype.com/vulnerabilities/cve-2025-12183): Various lz4-java compression and decompression implementations do not guard against out-of-bounds memory access. Untrusted input may lead to denial of service and information disclosure. Vulnerable Maven coordinates: org.lz4:lz4-java up to and including 1.8.0. - [CVE-2025-66566](GHSA-cmp6-m4wj-q63q): Insufficient clearing of the output buffer in Java-based decompressor implementations in lz4-java 1.10.0 and earlier allows remote attackers to read previous buffer contents via crafted compressed input. In applications where the output buffer is reused without being cleared, this may lead to disclosure of sensitive data. JNI-based implementations are not affected. Therefore, lz4-java version should upgrade to 1.10.4. No. No. CI. Closes #3555 from SteNicholas/CELEBORN-2218. Lead-authored-by: SteNicholas <[email protected]> Co-authored-by: Cheng Pan <[email protected]> Signed-off-by: SteNicholas <[email protected]> (cherry picked from commit dca3749) Signed-off-by: SteNicholas <[email protected]>

SteNicholas · 2026-03-03T03:30:36Z

Thanks for all. Merged to main(v0.7.0) and branch-0.6(v0.6.3).

yawkat · 2026-03-03T06:31:00Z

Given that zstd is from the same authors as lz4 but newer, it may still be a good idea to move to zstd as the default long-term.

Iceberg switched to `at.yawk.lz4:lz4-java` group for security reasons, but it unintentionally introduced performance regression. https://github.com/yawkat/lz4-java/releases/tag/v1.10.4 > These changes attempt to fix the native performance regression in 1.9+. They should have no functional or security impact. See the benchmark reports in Celeborn and Spark projects - CELEBORN-2218 / apache/celeborn#3555 - SPARK-55803 / apache/spark#54585

Trino switched to `at.yawk.lz4:lz4-java` group for security reasons, but it unintentionally introduced performance regression. https://github.com/yawkat/lz4-java/releases/tag/v1.10.4 > These changes attempt to fix the native performance regression in 1.9+. They should have no functional or security impact. See the benchmark reports in Celeborn and Spark projects - CELEBORN-2218 / apache/celeborn#3555 - SPARK-55803 / apache/spark#54585

ClickHouse Java Client switched to `at.yawk.lz4:lz4-java` for security reasons, but it unintentionally introduced performance regression. https://github.com/yawkat/lz4-java/releases/tag/v1.10.4 > These changes attempt to fix the native performance regression in 1.9+. They should have no functional or security impact. See the benchmark reports in Apache Celeborn and Apache Spark projects - CELEBORN-2218 / apache/celeborn#3555 - SPARK-55803 / apache/spark#54585

Iceberg switched to `at.yawk.lz4:lz4-java` group for security reasons, but it unintentionally introduced performance regression. https://github.com/yawkat/lz4-java/releases/tag/v1.10.4 > These changes attempt to fix the native performance regression in 1.9+. They should have no functional or security impact. See the benchmark reports in Celeborn and Spark projects - CELEBORN-2218 / apache/celeborn#3555 - SPARK-55803 / apache/spark#54585

github-actions bot added module:client kind:build labels Dec 3, 2025

SteNicholas force-pushed the CELEBORN-2218 branch from 8599c17 to 9d04928 Compare December 3, 2025 12:09

SteNicholas force-pushed the CELEBORN-2218 branch from 9d04928 to 241c31c Compare December 4, 2025 01:48

SteNicholas force-pushed the CELEBORN-2218 branch 2 times, most recently from c113f4f to 202ba06 Compare December 4, 2025 13:49

Marcono1234 reviewed Dec 10, 2025

View reviewed changes

client/src/main/java/org/apache/celeborn/client/compress/Lz4Decompressor.java Outdated Show resolved Hide resolved

SteNicholas force-pushed the CELEBORN-2218 branch 2 times, most recently from 95d3ed2 to 7ec6523 Compare December 11, 2025 05:56

SteNicholas requested a review from Marcono1234 December 11, 2025 06:01

SteNicholas force-pushed the CELEBORN-2218 branch 2 times, most recently from da9e0bc to 8043b4a Compare December 11, 2025 06:24

github-actions bot removed the module:client label Dec 11, 2025

SteNicholas force-pushed the CELEBORN-2218 branch from 8043b4a to 30ed24a Compare December 12, 2025 02:53

github-actions bot added module:client module:mr module:tez labels Dec 12, 2025

SteNicholas changed the title ~~[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.8.1 to resolve CVE‐2025‐12183~~ [CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.0 to resolve CVE‐2025‐12183 and CVE-2025-66566 Dec 12, 2025

SteNicholas force-pushed the CELEBORN-2218 branch from 30ed24a to 678a311 Compare December 12, 2025 02:57

SteNicholas force-pushed the CELEBORN-2218 branch from 678a311 to fc23474 Compare December 15, 2025 02:46

SteNicholas changed the title ~~[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.0 to resolve CVE‐2025‐12183 and CVE-2025-66566~~ [CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.1 to resolve CVE‐2025‐12183 and CVE-2025-66566 Dec 15, 2025

SteNicholas force-pushed the CELEBORN-2218 branch from fc23474 to 4b94a83 Compare December 16, 2025 03:31

pan3793 reviewed Dec 16, 2025

View reviewed changes

client-tez/tez-shaded/pom.xml Outdated Show resolved Hide resolved

pan3793 reviewed Mar 2, 2026

View reviewed changes

client/benchmarks/LZ4TPCDSDataBenchmark-jdk17-results.txt Outdated Show resolved Hide resolved

SteNicholas assigned RexXiong Mar 2, 2026

pan3793 added 2 commits March 3, 2026 09:51

Bump lz4-java 1.10.4

3cc43ce

update benchmark results

60b1f29

RexXiong approved these changes Mar 3, 2026

View reviewed changes

pan3793 mentioned this pull request Mar 3, 2026

HADOOP-19829. Bump lz4-java 1.10.4 apache/hadoop#8294

Merged

4 tasks

[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve…

1ff0da1

… CVE‐2025‐12183 and CVE-2025-66566

SteNicholas changed the title ~~[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.3 to resolve CVE‐2025‐12183 and CVE-2025-66566~~ [CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566 Mar 3, 2026

pan3793 force-pushed the CELEBORN-2218 branch from 60da719 to 1ff0da1 Compare March 3, 2026 02:32

pan3793 approved these changes Mar 3, 2026

View reviewed changes

SteNicholas closed this in dca3749 Mar 3, 2026

pan3793 mentioned this pull request Mar 5, 2026

Build: Bump lz4-java 1.10.4 apache/iceberg#15518

Merged

pan3793 mentioned this pull request Mar 5, 2026

Bump lz4-java 1.10.4 trinodb/trino#28542

Merged

This was referenced Mar 5, 2026

Update lz4-java from 1.10.2 to 1.10.4 ClickHouse/clickhouse-java#2778

Merged

HBASE-29969 Bump lz4-java 1.10.4 apache/hbase#7865

Merged

Conversation

SteNicholas commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR resolve a correctness bug?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

yawkat commented Dec 3, 2025

Uh oh!

SteNicholas commented Dec 4, 2025

Uh oh!

Uh oh!

yawkat commented Dec 11, 2025

Uh oh!

Marcono1234 commented Dec 12, 2025

Uh oh!

SteNicholas commented Dec 16, 2025

Uh oh!

Uh oh!

pan3793 commented Dec 16, 2025

Uh oh!

yawkat commented Dec 16, 2025

Uh oh!

Uh oh!

pan3793 commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SteNicholas commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yawkat commented Mar 2, 2026

Uh oh!

pan3793 commented Mar 3, 2026

Uh oh!

RexXiong commented Mar 3, 2026

Uh oh!

SteNicholas commented Mar 3, 2026

Uh oh!

yawkat commented Mar 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

SteNicholas commented Dec 3, 2025 •

edited

Loading

pan3793 commented Mar 2, 2026 •

edited

Loading

SteNicholas commented Mar 2, 2026 •

edited

Loading