mirrors/ghostty
1
0
Fork 0
mirror of https://github.com/ghostty-org/ghostty.git synced 2026-07-10 11:19:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

terminal: optimize LZ4 decoding and add differential tests

Browse Source 
The block decoder previously copied literals through variable-length
memcpy calls and expanded every match with word loops that carried an
overcopy fallback in each branch. Real page blocks decode as millions
of tiny sequences, so per-sequence overhead dominated restore time.

Decode short literal runs and in-token matches with blind fixed-width
copies whose margin checks subsume the exact bounds checks they
replace. Expand small repeating periods into pattern-word stores,
copy distant long matches with one exact memcpy, and propagate the
rare non-power-of-two short offsets bytewise. Page corpora restore
13% to 19% faster and text around twice as fast, while compressor
output stays byte-for-byte unchanged.

Replace the fuzz test with a differential property suite which
round-trips generated inputs, validates blocks with an independent
format walker, rejects wrong-size outputs, and decodes corrupted and
truncated blocks. A light version runs as a normal unit test; the
exhaustive version runs when GHOSTTY_LZ4_SLOW is set. An AGENTS.md
records the benchmarking, testing, and verification workflow for
this directory.
This commit is contained in:
Mitchell Hashimoto
2026-07-09 11:48:46 -07:00
parent 25e6245691
commit 9a4bd2120a
3 changed files with 782 additions and 226 deletions
Download Patch File Download Diff File Expand all files Collapse all files

90
src/terminal/compress/AGENTS.md Normal file
View File

@@ -0,0 +1,90 @@
# Terminal Compression
Guidance for the codecs and the compressed page representation
(`Page.zig`) in this directory. These compress terminal page backing
memory (`terminal.Page`).
## Priorities
When making tradeoffs, in order:
1. **Compression ratio on page-shaped data.** Encoded bytes are retained
scrollback memory, and raw `terminal.Page` backing memory is the only
thing we actually compress. Ratio on text files or synthetic data is a
secondary signal.
2. **Decompression throughput.** Pages are compressed once when they go
cold but restored on demand (scrollback access, search, inspection), so
restore latency is felt directly.
3. **Compression throughput.** Runs on idle pages in the background; being
fast is nice, being slow is tolerable.
## Testing
- Targeted tests: `zig build test -Dtest-filter=<codec>`
- Prefer `zig build test-lib-vt -Dtest-filter=<codec>` when practical;
this code ships in libghostty-vt.
- Codecs must keep building for `wasm32-freestanding` (libghostty-vt):
no libc, no `src/simd` (Highway) dependencies. Verify with
`zig build -Demit-lib-vt -Dtarget=wasm32-freestanding -Doptimize=ReleaseSmall`.
- Every codec needs a differential property suite: round-trip identity,
an independent format walker, wrong-size output rejection, and
corruption/truncation decoding. Keep a light version in normal unit
tests and gate the exhaustive version behind an environment variable so
the default test suite stays fast.
## Verifying Correctness
- Decoders must be memory-safe for arbitrary input bytes. Every blind or
wide copy needs a stated margin argument bounding it by the output
buffer; keep those arguments in comments next to the code.
- Writing scratch bytes past a copy's logical end is safe only inside the
output buffer, because in-order decoding rewrites them before any match
can read them back. Do not weaken the exact-size output contract.
- When a change should not alter compressor output, prove it: compare
encoded sizes (or a sequence-count fingerprint) on the same corpus
before and after. Ratio drift is a functional change, not noise.
## Benchmarking
- Use `ghostty-bench +page-compression` (see `src/benchmark/AGENTS.md`
for the general workflow). Modes: `compress`, `decompress`, `store`,
and `report` for ratio.
- Build: `zig build -Demit-bench -Doptimize=ReleaseFast -Demit-macos-app=false`
- The most representative corpus is a raw dump of real page backing
memory, chunked at the page size (400 KiB on ReleaseFast targets).
Supplement with a text corpus and random bytes for worst cases, but
weigh page corpora highest per the priorities above. Keep corpora
outside the repository and reuse identical files across comparisons.
- `ghostty-bench +scrollback-compression` measures the PageList
transitions around the codec rather than the codec itself.
- For fast iteration, keep codecs dependent only on `std` so a standalone
harness can build them directly with `zig build-exe -O ReleaseFast` and
time the codec in-process (report min-of-N, verify round-trips).
- Measure one change at a time and re-measure the final state; run-to-run
noise is a few percent, so re-run before believing small deltas.
## Performance Notes
- Real page data decodes as millions of tiny operations (in LZ4: mostly
zero literals plus a 4-18 byte match). Per-item overhead dominates, so
branch-light fast paths with blind fixed-size copies win.
- Wide copies are the only SIMD that pays here. Vectorized compares and
other wide-stride tricks measured as net losses because matches are
short; prefer the simple word loop unless a measurement on page corpora
says otherwise.
- `@memcpy` beats stride loops only for long copies (roughly 64 bytes and
up); call overhead loses below that.
## LZ4 Specific
- The codec is `lz4.zig`, an allocation-free raw block (not frame)
implementation. Blocks do not carry their decoded size; callers supply
an exact-size output buffer.
- Tests: `zig build test -Dtest-filter=lz4`. The differential suite is
`lz4_differential.zig`; run the exhaustive version for any codec
change:
`GHOSTTY_LZ4_SLOW=1 zig build test -Dtest-filter="lz4 differential"`
- The compressor must keep the standard format restrictions (final five
bytes literal, matches start at least twelve bytes before the end) so
blocks stay consumable by optimized external decoders. The differential
walker checks this.

425
src/terminal/compress/lz4.zig
View File

@@ -37,6 +37,8 @@
//! Format reference:
//! https://github.com/lz4/lz4/blob/dev/doc/lz4_Block_format.md
const std = @import("std");
const assert = std.debug.assert;
const testing = std.testing;
/// Maximum input accepted by the reference LZ4 block API. Keeping the same
/// limit means `compressBound` fits in the integer sizes used by LZ4 callers
@@ -125,93 +127,101 @@ pub fn compress(
// positions to the end of that match.
var op: usize = 0;
var anchor: usize = 0;
var ip: usize = 0;
var search_attempts: usize = 0;
// LZ4's format leaves the final five input bytes as literals and starts
// the final match at least twelve bytes before the end. This is not
// required by our safe decoder, but makes blocks compatible with fast
// decoders that rely on the standard format restrictions.
const search_end = if (input.len >= match_find_limit)
input.len - match_find_limit
else
0;
while (input.len >= match_find_limit and ip <= search_end) {
// Hash the next four bytes and replace the table entry immediately.
// Hash collisions are expected, so equality is checked below before
// accepting the saved position as a match.
const sequence = readU32(input, ip);
const hash = hashSequence(sequence);
const candidates = table[hash];
rememberPosition(table, hash, ip);
const match_pos_ = candidatePosition(ip, @truncate(candidates)) orelse
candidatePosition(ip, @truncate(candidates >> 16));
if (match_pos_ == null) {
advanceSearch(&ip, anchor, &search_attempts);
continue;
}
var match_pos = match_pos_.?;
if (readU32(input, match_pos) != sequence) {
const older = candidatePosition(
ip,
@truncate(candidates >> 16),
);
if (older == null or
readU32(input, older.?) != sequence or
input[older.? + min_match] != input[ip + min_match])
{
advanceSearch(&ip, anchor, &search_attempts);
continue;
}
match_pos = older.?;
}
// Pull the match backwards into the current literal run. This is
// particularly helpful around aligned cell records. As with forward
// extension, compare words before locating the first differing byte.
const match_begin = matchBegin(input, ip, match_pos, anchor);
ip = match_begin.position;
match_pos = match_begin.candidate;
// We already compared the first four bytes. Continue up to the point
// where the required last five literals begin. `matchEnd` compares a
// machine word at a time before locating the first differing byte.
// decoders that rely on the standard format restrictions. Inputs too
// short for any match are emitted below as one literal-only sequence.
if (input.len >= match_find_limit) {
const search_end = input.len - match_find_limit;
const match_end_limit = input.len - last_literals;
const match_end = matchEnd(
input,
ip + min_match,
match_pos + min_match,
match_end_limit,
);
var ip: usize = 0;
var search_attempts: usize = 0;
try emitSequence(
output,
&op,
input[anchor..ip],
@intCast(ip - match_pos),
match_end - ip,
);
search: while (ip <= search_end) {
// Hash the next four bytes and replace the table entry
// immediately. Hash collisions are expected, so equality is
// checked below before accepting a saved position as a match.
const sequence = readU32(input, ip);
const hash = hashSequence(sequence);
const candidates = table[hash];
rememberPosition(table, hash, ip);
// The main loop jumps over the matched bytes rather than hashing every
// position within them. Seed one position near the end so an adjacent
// repeated record can still refer back into this match. The next loop
// iteration will then seed `match_end` normally.
if (match_end >= 2 and match_end - 2 + min_match <= input.len) {
const seed = match_end - 2;
rememberPosition(
table,
hashSequence(readU32(input, seed)),
seed,
var match_pos = candidatePosition(ip, @truncate(candidates)) orelse
candidatePosition(ip, @truncate(candidates >> 16)) orelse
{
advanceSearch(&ip, anchor, &search_attempts);
continue :search;
};
if (readU32(input, match_pos) != sequence) {
// The nearest candidate collided. Fall back to the older
// one, which must additionally match one byte beyond the
// minimum so collision-prone minimum-length matches from
// the stale half are not emitted.
match_pos = older: {
if (candidatePosition(
ip,
@truncate(candidates >> 16),
)) |older| {
if (readU32(input, older) == sequence and
input[older + min_match] == input[ip + min_match])
{
break :older older;
}
}
advanceSearch(&ip, anchor, &search_attempts);
continue :search;
};
}
// Pull the match backwards into the current literal run. This is
// particularly helpful around aligned cell records. As with
// forward extension, compare words before locating the first
// differing byte.
const match_begin = matchBegin(input, ip, match_pos, anchor);
ip = match_begin.position;
match_pos = match_begin.candidate;
// We already compared the first four bytes. Continue up to the
// point where the required last five literals begin. `matchEnd`
// compares a machine word at a time before locating the first
// differing byte.
const match_end = matchEnd(
input,
ip + min_match,
match_pos + min_match,
match_end_limit,
);
}
ip = match_end;
anchor = ip;
search_attempts = 0;
try emitSequence(
output,
&op,
input[anchor..ip],
@intCast(ip - match_pos),
match_end - ip,
);
// The main loop jumps over the matched bytes rather than hashing
// every position within them. Seed one position near the end so
// an adjacent repeated record can still refer back into this
// match. The next loop iteration will then seed `match_end`
// normally.
if (match_end >= 2 and match_end - 2 + min_match <= input.len) {
const seed = match_end - 2;
rememberPosition(
table,
hashSequence(readU32(input, seed)),
seed,
);
}
ip = match_end;
anchor = ip;
search_attempts = 0;
}
}
// Whatever remains after the last match is the terminal literal-only
@@ -229,6 +239,17 @@ pub fn compress(
pub fn decompress(input: []const u8, output: []u8) DecompressError!usize {
// `ip` and `op` always identify the next unread input byte and the next
// unwritten output byte respectively.
//
// The decoder is written around one observation: almost every sequence
// in real blocks has a short literal run and a short match. Both fast
// paths below copy a fixed number of bytes blindly and let the length
// arithmetic sort out how many of them were meaningful. Writing past a
// run's logical end is safe within the output buffer because decoding
// is strictly in order: every byte past `op` is either rewritten by a
// later copy before anything can read it, or lies beyond the block's
// final length and is never part of the result. The margin conditions
// on the fast paths also subsume the exact bounds checks they replace,
// which keeps decoding of malformed blocks memory-safe.
var ip: usize = 0;
var op: usize = 0;
@@ -245,13 +266,28 @@ pub fn decompress(input: []const u8, output: []u8) DecompressError!usize {
ip += 1;
// The high nibble and any extension bytes describe the literal run.
// Bounds are checked before slicing so malformed blocks never cause a
// partial read or write.
const literal_len = try decodeLength(input, &ip, token >> 4);
if (literal_len > input.len - ip) return error.TruncatedInput;
if (literal_len > output.len - op) return error.OutputTooSmall;
// The literals are copied as a side effect of computing the length.
const literal_len: usize = len: {
const nibble: usize = token >> 4;
if (nibble != 15 and
@min(input.len - ip, output.len - op) >= 16)
{
// A run below the extension threshold is at most 14 bytes,
// so with a 16-byte margin on both buffers one wide copy
// covers it.
copyIntAt(u128, output, op, input, ip);
break :len nibble;
}
copyLiterals(input, ip, output, op, literal_len);
// Extended or margin-poor runs take the checked path. Bounds are
// verified before copying so malformed blocks never cause a
// partial read or write.
const len = try decodeLength(input, &ip, nibble);
if (len > input.len - ip) return error.TruncatedInput;
if (len > output.len - op) return error.OutputTooSmall;
@memcpy(output[op..][0..len], input[ip..][0..len]);
break :len len;
};
ip += literal_len;
op += literal_len;
@@ -263,13 +299,29 @@ pub fn decompress(input: []const u8, output: []u8) DecompressError!usize {
}
if (input.len - ip < 2) return error.TruncatedInput;
const offset = std.mem.readInt(u16, input[ip..][0..2], .little);
const offset = readIntAt(u16, input, ip);
ip += 2;
if (offset == 0 or offset > op) return error.InvalidOffset;
// The token stores the match length minus the four-byte minimum. As
// with literals, a low nibble of 15 is extended by following bytes.
const encoded_match_len = try decodeLength(input, &ip, token & 0x0F);
const match_nibble: usize = token & 0x0F;
// A match whose length fits its nibble spans at most 18 bytes, so
// three blind copies always cover it. They are overlap-safe when the
// offset is at least a word: each load lies a full word behind the
// store which could observe it, so repeating patterns propagate
// correctly.
if (match_nibble != 15 and offset >= 8 and output.len - op >= 18) {
const match = op - offset;
copyIntAt(u64, output, op, output, match);
copyIntAt(u64, output, op + 8, output, match + 8);
copyIntAt(u16, output, op + 16, output, match + 16);
op += match_nibble + min_match;
continue;
}
const encoded_match_len = try decodeLength(input, &ip, match_nibble);
const match_len = std.math.add(
usize,
encoded_match_len,
@@ -278,8 +330,8 @@ pub fn decompress(input: []const u8, output: []u8) DecompressError!usize {
if (match_len > output.len - op) return error.OutputTooSmall;
// Match copies may overlap, so this cannot always be one memcpy.
// `copyMatch` uses word copies where the offset permits them and
// expands the common one-, two-, and four-byte repeating patterns.
// `copyMatch` expands the common small repeating periods into wide
// stores and uses word copies for larger offsets.
copyMatch(output, op, offset, match_len);
op += match_len;
}
@@ -298,8 +350,8 @@ fn emitSequence(
offset: u16,
match_len: usize,
) CompressError!void {
std.debug.assert(match_len >= min_match);
std.debug.assert(offset > 0);
assert(match_len >= min_match);
assert(offset > 0);
const encoded_match_len = match_len - min_match;
@@ -325,7 +377,7 @@ fn emitSequence(
// Match length extensions follow the offset because this is where the
// decoder expects them in an LZ4 sequence.
std.mem.writeInt(u16, output[op.*..][0..2], offset, .little);
writeIntAt(u16, output, op.*, offset);
op.* += 2;
if (encoded_match_len >= 15)
writeLength(output, op, encoded_match_len - 15);
@@ -378,7 +430,7 @@ fn writeLength(output: []u8, op: *usize, length_: usize) void {
fn decodeLength(
input: []const u8,
ip: *usize,
nibble: u8,
nibble: usize,
) DecompressError!usize {
var length: usize = nibble;
if (nibble != 15) return length;
@@ -549,129 +601,69 @@ fn matchEnd(
return position;
}
/// Copy one literal run. Most non-final sequences contain only a handful of
/// literals. When both buffers have eight accessible bytes, one fixed-width
/// copy is cheaper than a variable-size memcpy; bytes beyond the logical run
/// are overwritten immediately by the match. Final literals use the exact
/// copy because the encoded block ends directly after them.
fn copyLiterals(
input: []const u8,
ip: usize,
output: []u8,
op: usize,
literal_len: usize,
) void {
if (literal_len == 0) return;
if (literal_len <= @sizeOf(u64) and
input.len - ip >= @sizeOf(u64) and
output.len - op >= @sizeOf(u64))
{
copyIntAt(u64, output, op, input, ip);
return;
}
@memcpy(output[op..][0..literal_len], input[ip..][0..literal_len]);
}
/// Copy one decoded match from `offset` bytes behind `op`.
///
/// Offsets of at least eight can be copied one word at a time even when the
/// complete match overlaps: every individual load still precedes its store by
/// a full word. One-, two-, and four-byte periods are expanded into a repeated
/// word. Other small offsets retain the required byte-wise propagation.
fn copyMatch(output: []u8, op: usize, offset: usize, match_len: usize) void {
const match_pos = op - offset;
const destination = output[op..][0..match_len];
const can_overcopy = output.len - op - match_len >= 3;
/// The caller has validated that the match fits: `op + match_len` never
/// exceeds `output.len`. Wide copies may write a few scratch bytes past the
/// match's logical end; as described in `decompress`, that is safe anywhere
/// the write stays inside the output buffer. Every path therefore bounds its
/// wide stores by both the match end and the buffer end, and the bytewise
/// loop at the bottom finishes whatever remains.
fn copyMatch(output: []u8, op_: usize, offset: usize, match_len: usize) void {
var op = op_;
const end = op_ + match_len;
if (offset >= @sizeOf(u64)) {
var copied: usize = 0;
while (match_len - copied >= @sizeOf(u64)) {
copyIntAt(
u64,
output,
op + copied,
output,
match_pos + copied,
);
copied += @sizeOf(u64);
}
if (copied < match_len and can_overcopy) {
const remaining = match_len - copied;
if (remaining <= @sizeOf(u32)) {
copyIntAt(
u32,
output,
op + copied,
output,
match_pos + copied,
);
} else {
copyIntAt(
u64,
output,
op + copied,
output,
match_pos + copied,
);
}
} else {
while (copied < match_len) : (copied += 1)
destination[copied] = output[match_pos + copied];
}
// A source which ends behind the copy can never overlap it. Long
// distant matches are common in structured pages (repeated rows and
// whole blank regions), and one exact memcpy moves them in cache-line
// units. Shorter matches are not worth the call overhead.
if (offset >= match_len and match_len >= 64) {
@memcpy(
output[op..][0..match_len],
output[op - offset ..][0..match_len],
);
return;
}
if (offset == 1 or offset == 2 or offset == 4) {
const pattern: u64 = switch (offset) {
1 => @as(u64, output[match_pos]) * 0x0101_0101_0101_0101,
2 => @as(u64, readIntAt(u16, output, match_pos)) *
0x0001_0001_0001_0001,
4 => @as(u64, readIntAt(u32, output, match_pos)) *
0x0000_0001_0000_0001,
else => unreachable,
};
switch (offset) {
// A period which divides the word size expands into one repeated
// pattern word. Long runs (blank lines, repeated cells) then become
// independent stores, with no load waiting on a preceding store.
// Stores advance by whole words from `op`, which preserves the
// pattern's phase.
1, 2, 4, 8 => {
const pattern: u64 = switch (offset) {
1 => @as(u64, output[op - 1]) * 0x0101_0101_0101_0101,
2 => @as(u64, readIntAt(u16, output, op - 2)) *
0x0001_0001_0001_0001,
4 => @as(u64, readIntAt(u32, output, op - 4)) *
0x0000_0001_0000_0001,
8 => readIntAt(u64, output, op - 8),
else => unreachable,
};
var copied: usize = 0;
while (match_len - copied >= @sizeOf(u64)) {
writeIntAt(u64, output, op + copied, pattern);
copied += @sizeOf(u64);
}
const limit = @min(end, output.len -| 7);
while (op < limit) : (op += 8)
writeIntAt(u64, output, op, pattern);
},
if (copied < match_len and can_overcopy) {
const remaining = match_len - copied;
if (remaining <= @sizeOf(u32)) {
writeIntAt(u32, output, op + copied, @truncate(pattern));
} else {
writeIntAt(u64, output, op + copied, pattern);
}
} else {
while (copied < match_len) : (copied += 1) {
destination[copied] = @truncate(pattern >> @intCast(
(copied % @sizeOf(u64)) * 8,
));
}
}
return;
// Wide copies are overlap-safe for the remaining offsets of at
// least a copy unit: each load lies a full unit behind the store
// which could observe it. Offsets 3, 5, 6, and 7 fall through to
// the bytewise loop; they are rare in real data and word tricks
// for them cost more in complexity than they return.
else => if (offset >= 16) {
const limit = @min(end, output.len -| 15);
while (op < limit) : (op += 16)
copyIntAt(u128, output, op, output, op - offset);
} else if (offset >= 8) {
const limit = @min(end, output.len -| 7);
while (op < limit) : (op += 8)
copyIntAt(u64, output, op, output, op - offset);
},
}
if (offset >= @sizeOf(u32) and can_overcopy) {
var copied: usize = 0;
while (copied < match_len) : (copied += @sizeOf(u32)) {
copyIntAt(
u32,
output,
op + copied,
output,
match_pos + copied,
);
}
return;
}
for (0..match_len) |i| destination[i] = output[match_pos + i];
while (op < end) : (op += 1) output[op] = output[op - offset];
}
/// Map a four-byte input sequence to its scratch-table slot.
@@ -681,7 +673,6 @@ inline fn hashSequence(sequence: u32) usize {
/// Shared round-trip assertion used by the corpus-style tests below.
fn expectRoundTrip(input: []const u8) !void {
const testing = std.testing;
const bound = try compressBound(input.len);
const encoded = try testing.allocator.alloc(u8, bound);
defer testing.allocator.free(encoded);
@@ -698,15 +689,12 @@ fn expectRoundTrip(input: []const u8) !void {
}
test "compressBound" {
const testing = std.testing;
try testing.expectEqual(@as(usize, 16), try compressBound(0));
try testing.expectEqual(@as(usize, 272), try compressBound(255));
try testing.expectError(error.InputTooLarge, compressBound(max_input_size + 1));
}
test "literal-only compatibility vectors" {
const testing = std.testing;
var empty: [0]u8 = .{};
try testing.expectEqual(@as(usize, 0), try decompress(&.{0}, &empty));
@@ -727,7 +715,6 @@ test "literal-only compatibility vectors" {
}
test "overlapping match compatibility vector" {
const testing = std.testing;
// One literal 'a', followed by a four-byte match at distance one.
var output: [5]u8 = undefined;
try testing.expectEqual(@as(usize, 5), try decompress(
@@ -738,7 +725,6 @@ test "overlapping match compatibility vector" {
}
test "extended overlapping match compatibility vector" {
const testing = std.testing;
// One literal followed by a 274-byte match. The match extension is
// encoded as 255 + 0 after the low token nibble's initial 15 bytes.
var output: [275]u8 = undefined;
@@ -750,7 +736,6 @@ test "extended overlapping match compatibility vector" {
}
test "short offset compatibility vectors" {
const testing = std.testing;
// These blocks end immediately after their match. Besides covering the
// repeating-pattern paths, they verify that the decoder uses exact copies
@@ -775,7 +760,6 @@ test "short offset compatibility vectors" {
}
test "bounded wild copies are overwritten by final literals" {
const testing = std.testing;
// The first sequence's nine-byte match leaves the five final literals
// required by the LZ4 block format. Its logical one-byte tail is copied as
@@ -807,7 +791,6 @@ test "bounded wild copies are overwritten by final literals" {
}
test "maximum match offset compatibility vector" {
const testing = std.testing;
const literal_len = std.math.maxInt(u16);
const extension_len = (literal_len - 15) / 255 + 1;
const encoded = try testing.allocator.alloc(
@@ -834,7 +817,6 @@ test "maximum match offset compatibility vector" {
}
test "round trips boundary-sized inputs" {
const testing = std.testing;
const lengths = [_]usize{
0, 1, 3, 4, 5, 12, 15, 16, 19,
20, 254, 255, 256, 269, 270, 271, 510, 511,
@@ -850,7 +832,6 @@ test "round trips boundary-sized inputs" {
}
test "round trips compressible page-sized inputs" {
const testing = std.testing;
const page_len = 400 * 1024;
const zeros = try testing.allocator.alloc(u8, page_len);
@@ -869,7 +850,6 @@ test "round trips compressible page-sized inputs" {
}
test "round trips deterministic random inputs" {
const testing = std.testing;
var prng = std.Random.DefaultPrng.init(0x4C5A_3401);
const random = prng.random();
@@ -883,7 +863,6 @@ test "round trips deterministic random inputs" {
}
test "compress reports short output" {
const testing = std.testing;
const input = "a terminal page needs enough output space";
var table: HashTable = undefined;
var output: [4]u8 = undefined;
@@ -894,7 +873,6 @@ test "compress reports short output" {
}
test "decompress rejects malformed blocks" {
const testing = std.testing;
var output: [32]u8 = undefined;
try testing.expectError(error.TruncatedInput, decompress(&.{0xF0}, &output));
@@ -908,11 +886,6 @@ test "decompress rejects malformed blocks" {
try testing.expectError(error.OutputSizeMismatch, decompress(&.{0}, output[0..1]));
}
test "fuzz decompressor safety" {
return std.testing.fuzz({}, fuzzDecompress, .{});
}
fn fuzzDecompress(_: void, input: []const u8) !void {
var output: [4096]u8 = undefined;
_ = decompress(input, &output) catch {};
test {
_ = @import("lz4_differential.zig");
}

493
src/terminal/compress/lz4_differential.zig Normal file
View File

@@ -0,0 +1,493 @@
//! Differential and property tests for the LZ4 block codec.
//!
//! Every generated input must compress into a block which:
//!
//! 1. fits within `compressBound`,
//! 2. is structurally valid LZ4 with the stricter guarantees our
//! compressor documents (final five bytes are literals, matches start
//! at least twelve bytes before the end), verified by an independent
//! walker which shares no code with the codec,
//! 3. decompresses to exactly the original bytes, and
//! 4. is rejected when decompressed into a buffer of the wrong size.
//!
//! Valid blocks are additionally mutated (bit flips, splices, truncations)
//! and fed to the decompressor, which must fail cleanly or succeed, but
//! never read or write out of bounds. The unit-test build enables runtime
//! safety, so any out-of-bounds slice access fails the test.
//!
//! The light suite below runs as a normal unit test and finishes quickly.
//! The exhaustive suite multiplies the same properties across far more
//! sizes, periods, seeds, and mutations; it is slow and therefore skipped
//! unless the environment variable `GHOSTTY_LZ4_SLOW` is set:
//!
//! GHOSTTY_LZ4_SLOW=1 zig build test -Dtest-filter="lz4 differential"
const std = @import("std");
const testing = std.testing;
const Allocator = std.mem.Allocator;
const lz4 = @import("lz4.zig");
/// Number of trailing block bytes which must be literals, mirroring the
/// documented guarantee of the compressor. Kept as an independent constant
/// so a codec regression cannot silently weaken the check.
const last_literals = 5;
/// A match may not begin in the final twelve bytes. See `last_literals`.
const match_find_limit = 12;
/// Sizes around every encoding boundary: token nibble limits (14/15),
/// minimum match and find limits (4/12), length-extension steps (15 + 255k),
/// and power-of-two neighborhoods.
const boundary_sizes = [_]usize{
0, 1, 2, 3, 4, 5, 6, 7,
8, 9, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 31, 32, 33, 63,
64, 65, 254, 255, 256, 269, 270, 271,
272, 1023, 1024, 4095, 4096, 4097,
};
/// Input generators exercising distinct codec behaviors. Every generator is
/// deterministic for a given random state.
const Generator = enum {
/// Uniform random bytes; largely incompressible.
random_bytes,
/// Runs of one repeated byte with random lengths; period-one matches.
runs,
/// One repeating pattern; exercises a fixed match period end to end.
periodic,
/// Eight-byte records with random small payloads and zero padding, with
/// some records repeated; resembles terminal cell memory.
cells,
/// Dictionary words with separators; text-like literal/match mix.
words,
/// Mostly zeros with scattered random bytes; long matches with isolated
/// literals.
sparse,
/// Random segments of all other generators; exercises transitions.
mixed,
fn fill(gen: Generator, random: std.Random, buf: []u8) void {
switch (gen) {
.random_bytes => random.bytes(buf),
.runs => {
var i: usize = 0;
while (i < buf.len) {
const run = @min(
random.intRangeAtMost(usize, 1, 300),
buf.len - i,
);
@memset(buf[i..][0..run], random.int(u8));
i += run;
}
},
.periodic => fillPeriodic(
random,
buf,
random.intRangeAtMost(usize, 1, 40),
),
.cells => {
var i: usize = 0;
while (i + 8 <= buf.len) : (i += 8) {
const cell = buf[i..][0..8];
if (i >= 8 and random.boolean()) {
// Repeat one of the last 256 records.
const back = 8 * random.intRangeAtMost(
usize,
1,
@min(i / 8, 256),
);
cell.* = buf[i - back ..][0..8].*;
} else {
@memset(cell, 0);
cell[0] = ' ' + random.uintLessThan(u8, 95);
cell[1] = random.uintLessThan(u8, 4);
}
}
random.bytes(buf[i..]);
},
.words => {
const words = [_][]const u8{
"the", "terminal", "page", "compress",
"row", "cell", "style", "zig",
"lz4", "block", "offset", "match",
"a", "of", "and", "literal",
"0x00", "0xFF", " ", "\r\n",
"-----", "=", "pub fn", "const",
};
var i: usize = 0;
while (i < buf.len) {
const word = words[random.uintLessThan(usize, words.len)];
const n = @min(word.len, buf.len - i);
@memcpy(buf[i..][0..n], word[0..n]);
i += n;
if (i < buf.len) {
buf[i] = if (random.boolean()) ' ' else '\n';
i += 1;
}
}
},
.sparse => {
@memset(buf, 0);
if (buf.len == 0) return;
for (0..buf.len / 32 + 1) |_| {
const at = random.uintLessThan(usize, buf.len);
buf[at] = random.int(u8);
}
},
.mixed => {
var i: usize = 0;
while (i < buf.len) {
const segment = @min(
random.intRangeAtMost(usize, 1, 2048),
buf.len - i,
);
const sub = random.enumValue(Generator);
if (sub != .mixed) sub.fill(random, buf[i..][0..segment]);
i += segment;
}
},
}
}
};
/// Fill `buf` with one repeating pattern of the given period.
fn fillPeriodic(random: std.Random, buf: []u8, period: usize) void {
if (buf.len == 0) return;
const head = @min(period, buf.len);
random.bytes(buf[0..head]);
for (head..buf.len) |i| buf[i] = buf[i - period];
}
/// Reusable buffers sized for the largest input a suite generates.
const Workspace = struct {
input: []u8,
encoded: []u8,
decoded: []u8,
table: *lz4.HashTable,
fn init(alloc: Allocator, max_input: usize) !Workspace {
const input = try alloc.alloc(u8, max_input);
errdefer alloc.free(input);
const encoded = try alloc.alloc(u8, try lz4.compressBound(max_input));
errdefer alloc.free(encoded);
// One extra byte so wrong-size decompression can be tested above
// the exact length as well as below it.
const decoded = try alloc.alloc(u8, max_input + 1);
errdefer alloc.free(decoded);
const table = try alloc.create(lz4.HashTable);
return .{
.input = input,
.encoded = encoded,
.decoded = decoded,
.table = table,
};
}
fn deinit(ws: *Workspace, alloc: Allocator) void {
alloc.free(ws.input);
alloc.free(ws.encoded);
alloc.free(ws.decoded);
alloc.destroy(ws.table);
ws.* = undefined;
}
};
/// Compress one input and verify every property promised by the codec.
/// Returns the encoded length so callers can reuse the encoded block.
fn expectCodecProperties(ws: *Workspace, input: []const u8) !usize {
const encoded_len = try lz4.compress(input, ws.encoded, ws.table);
try testing.expect(encoded_len <= try lz4.compressBound(input.len));
const encoded = ws.encoded[0..encoded_len];
try expectValidBlock(encoded, input.len);
// Exact-size decompression must reproduce the input bit for bit. The
// output is poisoned first so unwritten bytes cannot pass as correct.
const output = ws.decoded[0..input.len];
@memset(output, 0xAA);
try testing.expectEqual(input.len, try lz4.decompress(encoded, output));
try testing.expectEqualSlices(u8, input, output);
// The exact-size contract must reject both smaller and larger buffers.
if (input.len > 0) {
try testing.expectError(
error.OutputTooSmall,
lz4.decompress(encoded, ws.decoded[0 .. input.len - 1]),
);
}
try testing.expectError(
error.OutputSizeMismatch,
lz4.decompress(encoded, ws.decoded[0 .. input.len + 1]),
);
return encoded_len;
}
/// Structurally validate one encoded block against the LZ4 block format and
/// the stricter guarantees documented by our compressor. This deliberately
/// reimplements the format rather than reusing codec internals.
fn expectValidBlock(encoded: []const u8, raw_len: usize) !void {
var ip: usize = 0;
var op: usize = 0;
var last_match_end: usize = 0;
while (true) {
// Every sequence, including the final one, starts with a token.
try testing.expect(ip < encoded.len);
const token = encoded[ip];
ip += 1;
var literal_len: usize = token >> 4;
if (literal_len == 15) literal_len += try readExtension(encoded, &ip);
try testing.expect(encoded.len - ip >= literal_len);
ip += literal_len;
op += literal_len;
// The final sequence contains only literals and ends the block.
if (ip == encoded.len) {
try testing.expectEqual(raw_len, op);
if (last_match_end > 0)
try testing.expect(raw_len - last_match_end >= last_literals);
return;
}
try testing.expect(encoded.len - ip >= 2);
const offset = std.mem.readInt(u16, encoded[ip..][0..2], .little);
ip += 2;
try testing.expect(offset >= 1);
try testing.expect(offset <= op);
// Our compressor starts matches at least `match_find_limit` bytes
// before the end and never lets one run into the final literals.
try testing.expect(op + match_find_limit <= raw_len);
var match_len: usize = (token & 0x0F) + 4;
if (token & 0x0F == 15) match_len += try readExtension(encoded, &ip);
op += match_len;
try testing.expect(op + last_literals <= raw_len);
last_match_end = op;
}
}
/// Read one length extension: bytes of 255 accumulate until a terminator
/// below 255, which is included in the sum.
fn readExtension(encoded: []const u8, ip: *usize) !usize {
var total: usize = 0;
while (true) {
try testing.expect(ip.* < encoded.len);
const value = encoded[ip.*];
ip.* += 1;
total += value;
if (value != 255) return total;
}
}
/// Decode deterministic corruptions of a valid block. Any result is
/// acceptable except memory unsafety, which the safety-checked test build
/// turns into a failure. Mutations may still form a valid block, so output
/// contents are intentionally not asserted.
fn expectMutationSafety(
ws: *Workspace,
random: std.Random,
encoded_len: usize,
raw_len: usize,
mutations: usize,
) !void {
const original = try testing.allocator.dupe(u8, ws.encoded[0..encoded_len]);
defer testing.allocator.free(original);
for (0..mutations) |_| {
const block = ws.encoded[0..encoded_len];
@memcpy(block, original);
switch (random.uintLessThan(u8, 4)) {
// Flip up to eight random bits.
0 => for (0..random.intRangeAtMost(usize, 1, 8)) |_| {
const at = random.uintLessThan(usize, block.len);
block[at] ^= @as(u8, 1) << random.int(u3);
},
// Overwrite one random byte. Token and length bytes are the
// most interesting targets and small blocks are mostly tokens.
1 => block[random.uintLessThan(usize, block.len)] =
random.int(u8),
// Splice random garbage over a random span.
2 => {
const at = random.uintLessThan(usize, block.len);
const span = @min(
random.intRangeAtMost(usize, 1, 16),
block.len - at,
);
random.bytes(block[at..][0..span]);
},
// Decode a random prefix of the intact block.
else => {},
}
const len = if (random.boolean())
random.uintAtMost(usize, block.len)
else
block.len;
_ = lz4.decompress(block[0..len], ws.decoded[0..raw_len]) catch {};
}
}
/// Workload knobs shared by the light and exhaustive suites.
const Budget = struct {
/// Upper bound and step of the contiguous small-size sweep applied to
/// every generator.
sweep_max: usize,
sweep_step: usize,
/// Number and maximum size of random-parameter inputs.
random_inputs: usize,
random_max: usize,
/// Explicit match periods checked with `fillPeriodic`.
periods: []const usize,
period_len: usize,
/// Number of corrupted decode attempts per mutation base block.
mutations: usize,
fn maxInput(budget: Budget) usize {
return @max(
budget.random_max,
@max(budget.period_len, boundary_sizes[boundary_sizes.len - 1]),
);
}
};
const light_budget: Budget = .{
.sweep_max = 96,
.sweep_step = 1,
.random_inputs = 24,
.random_max = 32 * 1024,
// One period per copy strategy: byte propagation (3), pattern words
// (1/2/4/8), word strides (9), wide strides (17), plus the 64 KiB
// window edge cases.
.periods = &.{ 1, 2, 3, 4, 8, 9, 17, 65534, 65535, 65536, 65537 },
.period_len = 160 * 1024,
.mutations = 64,
};
const exhaustive_budget: Budget = .{
.sweep_max = 2048,
.sweep_step = 1,
.random_inputs = 512,
.random_max = 512 * 1024,
.periods = &.{
1, 2, 3, 4, 5, 6, 7, 8,
9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 23, 24, 31, 32, 33,
48, 63, 64, 65, 127, 128, 255, 256,
257, 4095, 4096, 32768, 65533, 65534, 65535, 65536,
65537, 65538,
},
.period_len = 320 * 1024,
.mutations = 4096,
};
fn runSuite(budget: Budget, seed: u64) !void {
var prng = std.Random.DefaultPrng.init(seed);
const random = prng.random();
var ws: Workspace = try .init(testing.allocator, budget.maxInput());
defer ws.deinit(testing.allocator);
// Every generator across every boundary size and the small-size sweep.
// Small inputs hit the literal-only format edges: below the minimum
// match sizes, around nibble limits, and around extension steps.
inline for (@typeInfo(Generator).@"enum".fields) |field| {
const gen: Generator = @enumFromInt(field.value);
for (boundary_sizes) |size| {
const input = ws.input[0..size];
gen.fill(random, input);
_ = try expectCodecProperties(&ws, input);
}
var size: usize = 0;
while (size <= budget.sweep_max) : (size += budget.sweep_step) {
const input = ws.input[0..size];
gen.fill(random, input);
_ = try expectCodecProperties(&ws, input);
}
}
// Random generator and size pairs, biased toward interesting sizes by
// squaring so both tiny and large inputs appear.
for (0..budget.random_inputs) |_| {
const gen = random.enumValue(Generator);
const scale = random.float(f64);
const size: usize = @intFromFloat(
scale * scale * @as(f64, @floatFromInt(budget.random_max)),
);
const input = ws.input[0..size];
gen.fill(random, input);
const encoded_len = try expectCodecProperties(&ws, input);
// Reuse a handful of these encodings as corruption bases.
try expectMutationSafety(
&ws,
random,
encoded_len,
input.len,
budget.mutations / budget.random_inputs + 1,
);
}
// Directed match periods, including the 64 KiB window edge where the
// compressor's 16-bit position arithmetic wraps.
for (budget.periods) |period| {
const input = ws.input[0..budget.period_len];
fillPeriodic(random, input, period);
_ = try expectCodecProperties(&ws, input);
}
// Dedicated corruption run over a text-like block, plus an exhaustive
// truncation sweep: every prefix of a valid block must decode cleanly
// or fail cleanly.
{
const input = ws.input[0..@min(16 * 1024, budget.random_max)];
Generator.words.fill(random, input);
const encoded_len = try expectCodecProperties(&ws, input);
try expectMutationSafety(
&ws,
random,
encoded_len,
input.len,
budget.mutations,
);
for (0..encoded_len) |prefix| {
_ = lz4.decompress(
ws.encoded[0..prefix],
ws.decoded[0..input.len],
) catch {};
}
}
}
test "lz4 differential light" {
try runSuite(light_budget, 0x4C5A_3403);
}
test "lz4 differential exhaustive" {
// Slow. Enable explicitly, ideally together with a test filter:
// GHOSTTY_LZ4_SLOW=1 zig build test -Dtest-filter="lz4 differential"
if (!std.process.hasEnvVarConstant("GHOSTTY_LZ4_SLOW"))
return error.SkipZigTest;
// Several independent seeds; the suite is deterministic per seed.
for (0..4) |seed| try runSuite(exhaustive_budget, 0x4C5A_4000 + seed);
}