The EROFS read-only file-system has temporarily disabled LZ4 rolling decompression support after Amazon Web Services (AWS) discovered data corruption issues. The issue stems from potential uncontrolled backward memory copies within the upstream LZ4 implementation, which breaks the assumptions of the rolling decompression optimization.
The LZ4 rolling decompression feature was introduced to reduce the memory footprint of temporary pages. It works by leveraging the LZ77-based nature of LZ4, which only references the most recent 64 KiB of decompressed data, theoretically requiring a bounded rolling window of temporary pages. For example, 601,960-byte data compressed into a 256k LZ4 extent would normally require 146 extra pages per request if rolling decompression were disabled.
However, developers found that the kernel’s LZ4 codebase, which EROFS does not control, contains a flaw. Specifically, the `memmove()` function can copy long literals backward on x86-based systems even when source and destination ranges don’t overlap, violating the rolling window assumption. According to a patch explaining the change, this “breaks the rolling assumption and makes the optimization broken.”
To ensure data correctness, particularly given EROFS’ widespread use, the developers have disabled LZ4 rolling decompression. The change is slated for the Linux 7.3-rc3 kernel release on Sunday. The merge note stated: “The most impactful fix here is to disable LZ4 rolling decompression for now.” The developers noted they need more time to plan a solution, as the upstream LZ4 implementation is outside of their control.
Read the original coverage
💬 Comments
📜 Comment Policy