terminal/snapshot: stage PAGE payloads while decoding

PAGE payloads were decoded through a stack of stream adapters:
a CRC32C-hashing reader over a length-limited reader over the
BLAKE3-hashing snapshot reader. Every row paid several adapter
crossings and both hashes were fed row-sized chunks, which kept
BLAKE3 out of its efficient many-block path and made adapter
overhead about a quarter of decode time.

Decode now reads the remaining payload into a scratch buffer with
one bulk read, so each hash sees the payload as a single update, and
then parses the tables and grid from a flat in-memory reader. Row
headers are also read as one three-byte read instead of two calls.
Staging is capped at 8 MiB, far above any standard-capacity page
payload, so a hostile declared length cannot force a large
allocation; larger payloads fall back to the streaming path. CRC
validation and exact-exhaustion checks are unchanged, with the
staged reader checked for leftover bytes to preserve
PayloadNotExhausted semantics.

Benchmark deltas at this commit (terminal-snapshot, 1 MB corpora):

  ascii lines 1-70:  decode 12.2 -> 8.1 ms (encode unchanged)
  ascii full-wrap:   decode 11.1 -> 7.2 ms
  utf8:              decode  3.1 -> 2.1 ms

Relative to the previous wire format and codecs, the series is a
16.0x encode and 14.8x decode improvement on line-shaped scrollback
at 4.5x smaller wire size.
This commit is contained in:
Mitchell Hashimoto
2026-08-02 09:37:25 -07:00
parent 9f66563479
commit 3e5d128353
3 changed files with 65 additions and 14 deletions

View File

@@ -515,11 +515,24 @@ pub fn decode(
hyperlink_remap: *const HyperlinkRemap,
) DecodeError!void {
for (0..page.size.rows) |y| {
// Every bit pattern is a valid header: booleans decode directly
// and the raw semantic value gets a default below. Reserved bits
// do not change the known fields.
const row_header: Row = @bitCast(try reader.takeByte());
const count = try io.readInt(reader, u16);
// Read the row header and cell count.
const row_header: Row, const count: u16 = header: {
// The staged payload path has every header buffered.
var row_header_bytes: [3]u8 = undefined;
if (reader.bufferedLen() >= 3) {
row_header_bytes = reader.buffered()[0..3].*;
reader.toss(3);
} else {
try reader.readSliceAll(&row_header_bytes);
}
// Every bit pattern is a valid header: booleans decode directly
// and the raw semantic value gets a default below. Reserved
// bits do not change the known fields.
const row_header: Row = @bitCast(row_header_bytes[0]);
const count = std.mem.readInt(u16, row_header_bytes[1..3], .little);
break :header .{ row_header, count };
};
const row = page.getRow(y);
row.wrap = row_header.wrap;

View File

@@ -204,10 +204,17 @@ pub const Decoder = struct {
return self.header.pageCapacity() catch unreachable;
}
/// Largest remaining PAGE payload staged into one contiguous buffer.
/// This comfortably covers every payload a standard-capacity page can
/// produce while keeping the allocation an untrusted length can force
/// far below the record framing's four-byte length limit.
const max_staged_payload = 8 * 1024 * 1024;
/// Decode the remaining payload into caller-owned native page storage.
///
/// The destination must be freshly initialized with `capacity`. `alloc`
/// is used only for temporary ID remaps and integrity-check storage.
/// is used only for temporary staging, ID remaps, and integrity-check
/// storage.
pub fn decode(
self: *Decoder,
destination: *TerminalPage,
@@ -221,12 +228,42 @@ pub const Decoder = struct {
.cols = self.header.columns,
.rows = self.header.rows,
};
try decodePayloadBody(
self.record_reader.payloadReader(),
alloc,
destination,
self.header,
);
// `init` consumed exactly the fixed header from the declared
// payload, so this is the byte count of the tables and grid.
const remaining = self.record_reader.header.payload_len - Header.len;
if (remaining <= max_staged_payload) {
// Stage the payload with one bulk read. The whole payload
// passes through both hashes as one update and the payload
// decoders then parse a flat buffer, which keeps per-row work
// free of stream adapters. The CRC and exact-length checks in
// `finish` are unaffected.
const staged = try alloc.alloc(u8, remaining);
defer alloc.free(staged);
try self.record_reader.payloadReader().readSliceAll(staged);
var staged_reader: std.Io.Reader = .fixed(staged);
try decodePayloadBody(
&staged_reader,
alloc,
destination,
self.header,
);
if (staged_reader.bufferedLen() != 0) {
return error.PayloadNotExhausted;
}
} else {
// A payload this large is either hostile or a page far beyond
// native capacities. Decode it through the streaming payload
// reader so its declared length cannot force an allocation.
try decodePayloadBody(
self.record_reader.payloadReader(),
alloc,
destination,
self.header,
);
}
try self.record_reader.finish();
// The decoder normalizes every semantic value, so a complete decode

View File

@@ -302,8 +302,9 @@ pub const Reader = struct {
// such as peek and discard.
limited_buffer: [1]u8,
// PAGE decoding performs many small reads. 256 bytes batches several cells
// while CRC32C is calculated without making Reader large on the stack.
// Fixed-header decoding performs several small reads. 256 bytes batches
// them while CRC32C is calculated without making Reader large on the
// stack. Bulk payload reads bypass this buffer entirely.
hashing_buffer: [256]u8,
limited: std.Io.Reader.Limited,