mirror of
https://github.com/ghostty-org/ghostty.git
synced 2026-08-24 16:11:43 +00:00
terminal/snapshot: remove BLAKE3 digests (#13680)
Remove BLAKE3 prefix digests. Keep READY/FINISH as empty records since they're semantically important markers. Our existing format (CRC32 per-record, declared counts, strict tag ordering requirements, etc.) already detect: accidental corruption, truncation, data omission, and duplication. BLAKE3 only protects against valid records being swapped or removed entirely. It is heavy for just that, and callers can solve that anyways via their own transport (like, just use TCP). For more adversarial protection, callers can also add layers like TLS or their own alternate signing methods depending on their own threat models. Removing the hash improves encode times by ~1.4x, decode times by ~1.3x. Time-to-READY decoding is effectively unchanged because it was such a small package to begin with. **AI usage:** I had it clean up the comments and the tests, but I did the blake3 removal and marker changes, and wrote the commit message myself. All reviewed.
This commit is contained in:
@@ -104,7 +104,7 @@ int main(void) {
|
||||
NULL, &incremental_decoder, reader);
|
||||
assert(result == GHOSTTY_SUCCESS);
|
||||
|
||||
// READY authenticates and returns a renderable terminal before old history.
|
||||
// READY returns a validated, renderable terminal before old history.
|
||||
GhosttyTerminal incremental_terminal = NULL;
|
||||
result = ghostty_snapshot_decoder_ready(
|
||||
incremental_decoder, &incremental_terminal);
|
||||
@@ -146,7 +146,7 @@ int main(void) {
|
||||
page_count++;
|
||||
}
|
||||
|
||||
// NO_VALUE means FINISH authenticated successfully and is idempotent.
|
||||
// NO_VALUE means FINISH validated successfully and is idempotent.
|
||||
assert(result == GHOSTTY_NO_VALUE);
|
||||
assert(page_count > 0);
|
||||
assert(ghostty_snapshot_decoder_next(incremental_decoder) ==
|
||||
|
||||
@@ -23,12 +23,12 @@ extern "C" {
|
||||
*
|
||||
* Encode and restore the complete state of a terminal via a binary format.
|
||||
*
|
||||
* A snapshot is an ordered, authenticated record stream. Its READY checkpoint
|
||||
* contains enough state to render and resume the terminal, including any
|
||||
* A snapshot is an ordered, CRC-protected record stream. Its READY marker
|
||||
* follows enough state to render and resume the terminal, including any
|
||||
* unfinished VT parser input. Older scrollback pages follow READY and the
|
||||
* FINISH checkpoint authenticates the complete snapshot.
|
||||
* FINISH marker terminates the complete snapshot.
|
||||
*
|
||||
* End-of-file before an operation's required READY or FINISH checkpoint is
|
||||
* End-of-file before an operation's required READY or FINISH marker is
|
||||
* malformed, truncated snapshot data and returns GHOSTTY_INVALID_VALUE.
|
||||
* GHOSTTY_IO_ERROR is reserved for a reader callback that returns false.
|
||||
*
|
||||
@@ -91,22 +91,22 @@ extern "C" {
|
||||
* +------------- CONTINUATION ---------------+
|
||||
* | unfinished VT/UTF-8 input, or ground |
|
||||
* +------------------ READY -----------------+
|
||||
* | BLAKE3-256 of every preceding byte | ready() returns here
|
||||
* | empty renderable-state marker | ready() returns here
|
||||
* +----------------- HISTORY ----------------+ repeated per screen
|
||||
* | scrollback manifest |
|
||||
* +------------------ PAGE ------------------+ next() consumes one page
|
||||
* | older screen rows |
|
||||
* +------------------ FINISH ----------------+
|
||||
* | BLAKE3-256 of every preceding byte | next() returns NO_VALUE
|
||||
* | empty end-of-snapshot marker | next() returns NO_VALUE
|
||||
* +------------------------------------------+
|
||||
* | trailing transport bytes (not consumed) |
|
||||
* +------------------------------------------+
|
||||
* @endcode
|
||||
*
|
||||
* READY authenticates the renderable prefix through CONTINUATION. FINISH
|
||||
* authenticates READY and every history record as well as the earlier prefix.
|
||||
* Thus record CRC32C detects local corruption while the BLAKE3 checkpoints
|
||||
* also bind the ordering and completeness of the record stream.
|
||||
* READY separates the renderable prefix through CONTINUATION from history.
|
||||
* FINISH terminates the record sequence. Both are empty records protected by
|
||||
* CRC32C, like every other record. Declared record counts, tags, and strict
|
||||
* decoding enforce the stream's ordering and completeness.
|
||||
*
|
||||
* Snapshot format version 1 is a work in progress and does not yet carry a
|
||||
* binary-compatibility guarantee.
|
||||
@@ -206,7 +206,7 @@ typedef enum GHOSTTY_ENUM_TYPED {
|
||||
/**
|
||||
* Rows prepended by the most recently decoded history page.
|
||||
*
|
||||
* Zero means the page was consumed and authenticated but could not be
|
||||
* Zero means the page was consumed and validated but could not be
|
||||
* applied to the live terminal.
|
||||
*
|
||||
* Output type: size_t *
|
||||
@@ -238,7 +238,7 @@ typedef enum GHOSTTY_ENUM_TYPED {
|
||||
* otherwise this returns GHOSTTY_INVALID_VALUE.
|
||||
*
|
||||
* Encoding begins at the writer's current position. If an error occurs, the
|
||||
* writer may contain a partial snapshot without a valid FINISH checkpoint.
|
||||
* writer may contain a partial snapshot without a valid FINISH marker.
|
||||
* Calls to the writer are synchronous; this function does not flush or make
|
||||
* the caller's destination durable.
|
||||
*
|
||||
@@ -319,7 +319,7 @@ GHOSTTY_API GhosttyResult ghostty_snapshot_encode_alloc(
|
||||
* wait outside the decoder or block in their callback. The read callback must
|
||||
* not call APIs, including ghostty_snapshot_decoder_free(), on the decoder
|
||||
* that owns it. Returning false reports GHOSTTY_IO_ERROR; returning true with
|
||||
* zero bytes before a required checkpoint reports truncated snapshot data as
|
||||
* zero bytes before a required marker reports truncated snapshot data as
|
||||
* GHOSTTY_INVALID_VALUE.
|
||||
*
|
||||
* @param allocator Allocator for decoder and decoded terminal state, or NULL
|
||||
@@ -390,7 +390,7 @@ GHOSTTY_API GhosttyResult ghostty_snapshot_decoder_set(
|
||||
const void* value);
|
||||
|
||||
/**
|
||||
* Decode and authenticate the renderable snapshot prefix through READY.
|
||||
* Decode and validate the renderable snapshot prefix through READY.
|
||||
*
|
||||
* On success, terminal receives a caller-owned terminal with its persistent
|
||||
* VT stream already restored from the snapshot continuation. The terminal is
|
||||
@@ -425,14 +425,14 @@ GHOSTTY_API GhosttyResult ghostty_snapshot_decoder_ready(
|
||||
/**
|
||||
* Decode one history page into the terminal returned by READY.
|
||||
*
|
||||
* Each GHOSTTY_SUCCESS consumes and authenticates one PAGE record. Query the
|
||||
* Each GHOSTTY_SUCCESS consumes and validates one PAGE record. Query the
|
||||
* GHOSTTY_SNAPSHOT_DECODER_DATA_PROGRESS_* values before calling next again.
|
||||
* GHOSTTY_NO_VALUE means FINISH was validated; repeated calls after FINISH
|
||||
* also return GHOSTTY_NO_VALUE.
|
||||
*
|
||||
* The terminal may be rendered, resized, and fed live PTY input between calls.
|
||||
* If a history page can no longer be applied safely, it is still consumed and
|
||||
* authenticated and progress reports zero rows. The decoder applies history
|
||||
* validated and progress reports zero rows. The decoder applies history
|
||||
* to the caller-owned terminal produced by its READY operation.
|
||||
*
|
||||
* A decoding error invalidates the decoder's source position. The terminal
|
||||
@@ -449,7 +449,7 @@ GHOSTTY_API GhosttyResult ghostty_snapshot_decoder_next(
|
||||
GhosttySnapshotDecoder decoder);
|
||||
|
||||
/**
|
||||
* Decode and authenticate one complete snapshot.
|
||||
* Decode and validate one complete snapshot.
|
||||
*
|
||||
* This is the one-shot form of READY followed by all history pages through
|
||||
* FINISH. It may only be called before decoding starts. Bytes following FINISH
|
||||
|
||||
@@ -87,7 +87,6 @@
|
||||
inherit pkgs lib stdenv;
|
||||
};
|
||||
python = python3.withPackages (python-pkgs: [
|
||||
python-pkgs.blake3
|
||||
python-pkgs.kaitaistruct
|
||||
python-pkgs.ucs-detect
|
||||
]);
|
||||
|
||||
@@ -356,7 +356,7 @@ pub fn decoder_ready(
|
||||
return .success;
|
||||
}
|
||||
|
||||
/// Decode and apply exactly one history page, or authenticate FINISH.
|
||||
/// Decode and apply exactly one history page, or validate FINISH.
|
||||
pub fn decoder_next(
|
||||
decoder_: Decoder,
|
||||
) callconv(lib.calling_conv) Result {
|
||||
@@ -374,14 +374,14 @@ pub fn decoder_next(
|
||||
const native = terminal_c.zigTerminal(terminal).?;
|
||||
history.progress = null;
|
||||
|
||||
// Consume one authenticated record and preserve READY metadata on failure.
|
||||
// Consume one validated record and preserve READY metadata on failure.
|
||||
const progress = decoder.decoder.next(decoder.alloc, native) catch |err| {
|
||||
const metadata = history.metadata;
|
||||
decoder.state = .{ .failed = metadata };
|
||||
return decoderMapError(decoder, err);
|
||||
};
|
||||
|
||||
// Publish page progress, or transition permanently to authenticated FINISH.
|
||||
// Publish page progress, or transition permanently to validated FINISH.
|
||||
if (progress) |value| {
|
||||
history.progress = value;
|
||||
return .success;
|
||||
@@ -392,7 +392,7 @@ pub fn decoder_next(
|
||||
return .no_value;
|
||||
}
|
||||
|
||||
/// Decode and authenticate the complete snapshot transactionally.
|
||||
/// Decode and validate the complete snapshot transactionally.
|
||||
pub fn decoder_decode(
|
||||
decoder_: Decoder,
|
||||
out_: ?*terminal_c.Terminal,
|
||||
@@ -413,7 +413,7 @@ pub fn decoder_decode(
|
||||
};
|
||||
const native = terminal_c.zigTerminal(ready.terminal).?;
|
||||
|
||||
// Apply every history page and require an authenticated FINISH record.
|
||||
// Apply every history page and require a valid FINISH record.
|
||||
while (true) {
|
||||
const progress = decoder.decoder.next(decoder.alloc, native) catch |err| {
|
||||
terminal_c.free(ready.terminal);
|
||||
@@ -477,7 +477,7 @@ fn decoderMapError(decoder: *DecoderWrapper, err: anyerror) Result {
|
||||
}
|
||||
|
||||
// A successful zero-byte read is clean EOF by contract. Reaching it before
|
||||
// the required checkpoint therefore means truncated snapshot data, not an
|
||||
// the required marker therefore means truncated snapshot data, not an
|
||||
// external I/O failure, and deliberately maps to invalid_value below.
|
||||
return switch (err) {
|
||||
error.OutOfMemory => .out_of_memory,
|
||||
|
||||
@@ -1,43 +1,20 @@
|
||||
//! READY and FINISH snapshot checkpoint records.
|
||||
//! READY and FINISH snapshot marker records.
|
||||
//!
|
||||
//! Each checkpoint payload is one BLAKE3-256 digest of every snapshot byte
|
||||
//! before that checkpoint's record header. This binds record order and detects
|
||||
//! omitted or duplicated records in ways that independent record CRCs cannot.
|
||||
//!
|
||||
//! READY covers the envelope, TERMINAL, all SCREEN/PAGE sequences, and the
|
||||
//! required CONTINUATION record. FINISH covers that same prefix plus the
|
||||
//! complete READY record and all HISTORY/PAGE sequences. Neither digest includes
|
||||
//! its own record. FINISH terminates one snapshot; bytes after it belong to the
|
||||
//! containing transport.
|
||||
//!
|
||||
//! The digest does not replace record framing. Its 32-byte payload is still
|
||||
//! protected by the record's CRC32C.
|
||||
//! READY separates the renderable terminal state from the history sequences.
|
||||
//! FINISH terminates one snapshot; bytes after it belong to the containing
|
||||
//! transport. Both markers have empty payloads, independently framed and
|
||||
//! protected by the same CRC32C as every other record.
|
||||
//!
|
||||
//! ## Binary Format
|
||||
//!
|
||||
//! READY and FINISH use the same fixed payload:
|
||||
//!
|
||||
//! ```text
|
||||
//! 0 +----------------------------+
|
||||
//! | BLAKE3-256 prefix digest |
|
||||
//! | 32 bytes |
|
||||
//! 32 +----------------------------+
|
||||
//! ```
|
||||
//! READY and FINISH contain only their record headers. Their declared payload
|
||||
//! length must be zero.
|
||||
|
||||
const std = @import("std");
|
||||
const test_fixture = @import("fixture.zig");
|
||||
const record = @import("record.zig");
|
||||
|
||||
const Blake3 = std.crypto.hash.Blake3;
|
||||
|
||||
/// The prefix digest exchanged by checkpoint codecs and the snapshot driver.
|
||||
pub const Digest = record.PrefixDigest;
|
||||
|
||||
comptime {
|
||||
std.debug.assert(@sizeOf(Digest) == 32);
|
||||
}
|
||||
|
||||
/// Selects one of the two checkpoint positions in a snapshot.
|
||||
/// Selects one of the two marker positions in a snapshot.
|
||||
pub const Kind = enum {
|
||||
ready,
|
||||
finish,
|
||||
@@ -50,66 +27,40 @@ pub const Kind = enum {
|
||||
}
|
||||
};
|
||||
|
||||
pub const EncodeError = std.Io.Writer.Error ||
|
||||
record.Writer.FinishError;
|
||||
pub const EncodeError = record.Writer.FinishError;
|
||||
|
||||
/// Append one checkpoint covering every byte already emitted by `stream`.
|
||||
///
|
||||
/// Finalizing does not consume the running hasher. READY is therefore included
|
||||
/// as the stream continues toward FINISH.
|
||||
pub fn encode(
|
||||
kind: Kind,
|
||||
stream: *record.Writer,
|
||||
) EncodeError!void {
|
||||
const digest = stream.prefixDigest();
|
||||
|
||||
const payload = stream.begin(kind.tag());
|
||||
/// Append one empty marker record.
|
||||
pub fn encode(kind: Kind, stream: *record.Writer) EncodeError!void {
|
||||
_ = stream.begin(kind.tag());
|
||||
errdefer stream.cancel();
|
||||
try payload.writeAll(&digest);
|
||||
try stream.finish();
|
||||
}
|
||||
|
||||
pub const DecodeError = record.Reader.InitError ||
|
||||
record.Reader.FinishError ||
|
||||
error{
|
||||
/// The next record is valid but is not the expected checkpoint.
|
||||
/// The next record is valid but is not the expected marker.
|
||||
UnexpectedRecordTag,
|
||||
|
||||
/// The checkpoint does not describe the preceding snapshot bytes.
|
||||
InvalidDigest,
|
||||
};
|
||||
|
||||
/// Decode and validate one checkpoint against the running prefix digest.
|
||||
///
|
||||
/// READY is consumed through the hashing reader so FINISH covers it. FINISH is
|
||||
/// consumed from the underlying source so neither checkpoint includes itself.
|
||||
/// Decode one marker and require its payload to be empty.
|
||||
pub fn decode(
|
||||
kind: Kind,
|
||||
stream: *record.StreamReader,
|
||||
source: *std.Io.Reader,
|
||||
) DecodeError!void {
|
||||
const expected = stream.prefixDigest();
|
||||
const source = if (kind == .finish)
|
||||
stream.source()
|
||||
else
|
||||
stream.reader();
|
||||
|
||||
var record_reader: record.Reader = undefined;
|
||||
try record_reader.init(source);
|
||||
if (record_reader.header.tag != kind.tag()) {
|
||||
return error.UnexpectedRecordTag;
|
||||
}
|
||||
|
||||
var actual: Digest = undefined;
|
||||
try record_reader.payloadReader().readSliceAll(&actual);
|
||||
try record_reader.finish();
|
||||
if (!std.mem.eql(u8, &expected, &actual)) return error.InvalidDigest;
|
||||
}
|
||||
|
||||
const test_ready_fixture = test_fixture.parse(
|
||||
@embedFile("testdata/checkpoint-ready-v1.hex"),
|
||||
);
|
||||
|
||||
test "READY golden encoding and BLAKE3-256 registry" {
|
||||
test "READY golden encoding" {
|
||||
const prefix = "abc";
|
||||
|
||||
var snapshot: std.Io.Writer.Allocating = .init(std.testing.allocator);
|
||||
@@ -130,40 +81,13 @@ test "READY golden encoding and BLAKE3-256 registry" {
|
||||
snapshot.written(),
|
||||
);
|
||||
|
||||
// The checked-in payload independently locks the BLAKE3-256 registry.
|
||||
var actual: Digest = undefined;
|
||||
Blake3.hash(prefix, &actual, .{});
|
||||
const payload_offset = prefix.len + record.Header.len;
|
||||
try std.testing.expectEqualSlices(
|
||||
u8,
|
||||
test_ready_fixture[payload_offset..][0..actual.len],
|
||||
&actual,
|
||||
);
|
||||
|
||||
// Finalizing a streaming hasher neither consumes nor resets it.
|
||||
var hasher = Blake3.init(.{});
|
||||
hasher.update("a");
|
||||
var prefix_digest: Digest = undefined;
|
||||
Blake3.hash("a", &prefix_digest, .{});
|
||||
hasher.final(&actual);
|
||||
try std.testing.expectEqual(prefix_digest, actual);
|
||||
|
||||
hasher.update("bc");
|
||||
hasher.final(&actual);
|
||||
try std.testing.expectEqualSlices(
|
||||
u8,
|
||||
test_ready_fixture[payload_offset..][0..actual.len],
|
||||
&actual,
|
||||
);
|
||||
|
||||
// Decode the checked-in record rather than the generated candidate.
|
||||
var source: std.Io.Reader = .fixed(&test_ready_fixture);
|
||||
var source_stream: record.StreamReader = .init(&source);
|
||||
try source_stream.reader().discardAll(prefix.len);
|
||||
try decode(.ready, &source_stream);
|
||||
try source.discardAll(prefix.len);
|
||||
try decode(.ready, &source);
|
||||
}
|
||||
|
||||
test "READY and FINISH checkpoint coverage" {
|
||||
test "READY and FINISH are empty marker records" {
|
||||
const testing = std.testing;
|
||||
|
||||
var snapshot: std.Io.Writer.Allocating = .init(testing.allocator);
|
||||
@@ -175,34 +99,29 @@ test "READY and FINISH checkpoint coverage" {
|
||||
defer stream.deinit();
|
||||
try stream.writer().writeAll("prefix");
|
||||
|
||||
// READY covers only the bytes that precede its own record.
|
||||
const ready_offset = snapshot.written().len;
|
||||
const ready_digest = stream.prefixDigest();
|
||||
try encode(.ready, &stream);
|
||||
|
||||
// History follows READY and is included by FINISH.
|
||||
try stream.writer().writeAll("history");
|
||||
const finish_offset = snapshot.written().len;
|
||||
const finish_digest = stream.prefixDigest();
|
||||
try encode(.finish, &stream);
|
||||
|
||||
// The running stream reaches both checkpoints without rehashing its prefix.
|
||||
var source: std.Io.Reader = .fixed(snapshot.written());
|
||||
var source_stream: record.StreamReader = .init(&source);
|
||||
try source_stream.reader().discardAll(ready_offset);
|
||||
try testing.expectEqual(ready_digest, source_stream.prefixDigest());
|
||||
try decode(.ready, &source_stream);
|
||||
try source_stream.reader().discardAll("history".len);
|
||||
try testing.expectEqual(finish_digest, source_stream.prefixDigest());
|
||||
try decode(.finish, &source_stream);
|
||||
|
||||
try testing.expectEqual(
|
||||
snapshot.written().len - finish_offset,
|
||||
record.Header.len + @sizeOf(Digest),
|
||||
record.Header.len,
|
||||
finish_offset - ready_offset - "history".len,
|
||||
);
|
||||
try testing.expectEqual(
|
||||
record.Header.len,
|
||||
snapshot.written().len - finish_offset,
|
||||
);
|
||||
|
||||
var source: std.Io.Reader = .fixed(snapshot.written());
|
||||
try source.discardAll(ready_offset);
|
||||
try decode(.ready, &source);
|
||||
try source.discardAll("history".len);
|
||||
try decode(.finish, &source);
|
||||
}
|
||||
|
||||
test "checkpoint rejects wrong tags and digests" {
|
||||
test "checkpoint rejects wrong tags and nonempty payloads" {
|
||||
const testing = std.testing;
|
||||
|
||||
var snapshot: std.Io.Writer.Allocating = .init(testing.allocator);
|
||||
@@ -212,20 +131,14 @@ test "checkpoint rejects wrong tags and digests" {
|
||||
&snapshot.writer,
|
||||
);
|
||||
defer stream.deinit();
|
||||
try stream.writer().writeAll("prefix");
|
||||
const checkpoint_offset = snapshot.written().len;
|
||||
try encode(.ready, &stream);
|
||||
|
||||
var wrong_tag_source: std.Io.Reader = .fixed(
|
||||
snapshot.written()[checkpoint_offset..],
|
||||
);
|
||||
var wrong_tag: record.StreamReader = .init(&wrong_tag_source);
|
||||
var wrong_tag_source: std.Io.Reader = .fixed(snapshot.written());
|
||||
try testing.expectError(
|
||||
error.UnexpectedRecordTag,
|
||||
decode(.finish, &wrong_tag),
|
||||
decode(.finish, &wrong_tag_source),
|
||||
);
|
||||
|
||||
// Build a correctly framed checkpoint containing an unrelated digest.
|
||||
var invalid: std.Io.Writer.Allocating = .init(testing.allocator);
|
||||
defer invalid.deinit();
|
||||
var invalid_stream: record.Writer = .init(
|
||||
@@ -233,22 +146,15 @@ test "checkpoint rejects wrong tags and digests" {
|
||||
&invalid.writer,
|
||||
);
|
||||
defer invalid_stream.deinit();
|
||||
try invalid_stream.writer().writeAll("prefix");
|
||||
var invalid_digest = invalid_stream.prefixDigest();
|
||||
invalid_digest[0] ^= 1;
|
||||
const invalid_payload = invalid_stream.begin(.ready);
|
||||
errdefer invalid_stream.cancel();
|
||||
try invalid_payload.writeAll(&invalid_digest);
|
||||
try invalid_payload.writeByte(0);
|
||||
try invalid_stream.finish();
|
||||
|
||||
var invalid_digest_source: std.Io.Reader = .fixed(invalid.written());
|
||||
var invalid_digest_stream: record.StreamReader = .init(
|
||||
&invalid_digest_source,
|
||||
);
|
||||
try invalid_digest_stream.reader().discardAll("prefix".len);
|
||||
var invalid_source: std.Io.Reader = .fixed(invalid.written());
|
||||
try testing.expectError(
|
||||
error.InvalidDigest,
|
||||
decode(.ready, &invalid_digest_stream),
|
||||
error.PayloadNotExhausted,
|
||||
decode(.ready, &invalid_source),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -262,15 +168,10 @@ test "FINISH leaves continuation bytes unread" {
|
||||
&finished.writer,
|
||||
);
|
||||
defer finished_stream.deinit();
|
||||
try finished_stream.writer().writeAll("prefix");
|
||||
try encode(.finish, &finished_stream);
|
||||
try finished.writer.writeByte(0);
|
||||
|
||||
var trailing_source: std.Io.Reader = .fixed(finished.written());
|
||||
var trailing_stream: record.StreamReader = .init(
|
||||
&trailing_source,
|
||||
);
|
||||
try trailing_stream.reader().discardAll("prefix".len);
|
||||
try decode(.finish, &trailing_stream);
|
||||
try testing.expectEqual(@as(u8, 0), try trailing_source.takeByte());
|
||||
var source: std.Io.Reader = .fixed(finished.written());
|
||||
try decode(.finish, &source);
|
||||
try testing.expectEqual(@as(u8, 0), try source.takeByte());
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
//! terminal and reconstruct its unfinished VT Stream state.
|
||||
//!
|
||||
//! After READY, we send history pages (scrollback). Finally, the
|
||||
//! snapshot ends with a FINISH payload.
|
||||
//! snapshot ends with an empty FINISH marker.
|
||||
//!
|
||||
//! ## Snapshot Format
|
||||
//!
|
||||
@@ -75,12 +75,12 @@
|
||||
//! VT parser/stream up to the same state, or no bytes if it should be
|
||||
//! in the ground state.
|
||||
//!
|
||||
//! READY and FINISH contain BLAKE3-256 digests of all preceding snapshot bytes.
|
||||
//! READY therefore validates the renderable active state and continuation.
|
||||
//! FINISH covers READY and all history as well, validating the complete snapshot
|
||||
//! and its record ordering. Each SCREEN declares its complete logical history
|
||||
//! extent, allowing a client to size its scrollbar at READY even though older
|
||||
//! PAGE records arrive afterward.
|
||||
//! READY and FINISH are empty marker records. READY separates the renderable
|
||||
//! active state and continuation from history; FINISH terminates the complete
|
||||
//! snapshot. Every record, including both markers, is independently protected
|
||||
//! by CRC32C. Each SCREEN declares its complete logical history extent, allowing
|
||||
//! a client to size its scrollbar at READY even though older PAGE records arrive
|
||||
//! afterward.
|
||||
//!
|
||||
//! ## Encoding
|
||||
//!
|
||||
@@ -99,12 +99,12 @@
|
||||
//!
|
||||
//! Encoding begins at the writer's current position, so unrelated bytes may
|
||||
//! precede the snapshot. The encoder buffers only the current record payload
|
||||
//! to calculate its length and CRC32C; completed records stream immediately
|
||||
//! and BLAKE3 checkpoint coverage is updated incrementally. Buffering is an
|
||||
//! encoder implementation detail, not a requirement of the wire format.
|
||||
//! to calculate its length and CRC32C; completed records stream immediately.
|
||||
//! Buffering is an encoder implementation detail, not a requirement of the
|
||||
//! wire format.
|
||||
//!
|
||||
//! A failure may leave prior complete records, or a partial record if the
|
||||
//! destination itself fails. Such a prefix has no valid FINISH checkpoint and
|
||||
//! destination itself fails. Such a prefix has no FINISH marker and
|
||||
//! cannot be restored as a complete snapshot.
|
||||
//!
|
||||
//! Each record type usually exposes an `encode` function that encodes
|
||||
|
||||
@@ -234,11 +234,11 @@ pub const Decoder = struct {
|
||||
const remaining = self.record_reader.header.payload_len - Header.len;
|
||||
|
||||
if (remaining <= max_staged_payload) {
|
||||
// Stage the payload with one bulk read. The whole payload
|
||||
// passes through both hashes as one update and the payload
|
||||
// decoders then parse a flat buffer, which keeps per-row work
|
||||
// free of stream adapters. The CRC and exact-length checks in
|
||||
// `finish` are unaffected.
|
||||
// Stage the payload with one bulk read. The whole payload passes
|
||||
// through the checksum hasher as one update and the payload
|
||||
// decoders then parse a flat buffer, which keeps per-row work free
|
||||
// of stream adapters. The CRC and exact-length checks in `finish`
|
||||
// are unaffected.
|
||||
const staged = try alloc.alloc(u8, remaining);
|
||||
defer alloc.free(staged);
|
||||
try self.record_reader.payloadReader().readSliceAll(staged);
|
||||
@@ -289,10 +289,8 @@ pub const DiscardError = record.Reader.InitError ||
|
||||
/// Consume exactly one complete PAGE record, validating its framing and
|
||||
/// CRC32C while discarding the payload without structural validation.
|
||||
///
|
||||
/// The payload bytes still stream through `source`, so an enclosing running
|
||||
/// digest (such as the FINISH checkpoint) continues to cover them. This lets
|
||||
/// a decoder stay aligned with the record sequence, and keep authenticating
|
||||
/// it, while dropping page content it can no longer apply.
|
||||
/// This lets a decoder stay aligned with the record sequence while dropping
|
||||
/// page content it can no longer apply.
|
||||
pub fn discard(source: *std.Io.Reader) DiscardError!void {
|
||||
var record_reader: record.Reader = undefined;
|
||||
try record_reader.init(source);
|
||||
@@ -1658,7 +1656,7 @@ test "decode reuses duplicate hyperlinks" {
|
||||
try std.testing.expectEqual(@as(usize, 0), decoded.hyperlink_set.count());
|
||||
}
|
||||
|
||||
test "discard consumes exactly one PAGE record and keeps digest coverage" {
|
||||
test "discard consumes exactly one PAGE record" {
|
||||
const testing = std.testing;
|
||||
|
||||
// Discard validates framing only, so an arbitrary payload keeps this test
|
||||
@@ -1676,19 +1674,8 @@ test "discard consumes exactly one PAGE record and keeps digest coverage" {
|
||||
const record_len = encoded.written().len;
|
||||
try encoded.writer.writeAll("next");
|
||||
|
||||
// Discarded payload bytes must still update an enclosing running digest,
|
||||
// which is what lets a snapshot FINISH checkpoint authenticate records
|
||||
// whose content was dropped.
|
||||
var source: std.Io.Reader = .fixed(encoded.written());
|
||||
var hashed: record.StreamReader = .init(&source);
|
||||
try discard(hashed.reader());
|
||||
var expected_digest: record.PrefixDigest = undefined;
|
||||
std.crypto.hash.Blake3.hash(
|
||||
encoded.written()[0..record_len],
|
||||
&expected_digest,
|
||||
.{},
|
||||
);
|
||||
try testing.expectEqual(expected_digest, hashed.prefixDigest());
|
||||
try discard(&source);
|
||||
try testing.expectEqualStrings("next", try source.take(4));
|
||||
|
||||
// Only PAGE records may be discarded; the tag remains strict.
|
||||
|
||||
@@ -24,11 +24,6 @@ const Allocator = std.mem.Allocator;
|
||||
const test_fixture = @import("fixture.zig");
|
||||
const io = @import("io.zig");
|
||||
|
||||
const Blake3 = std.crypto.hash.Blake3;
|
||||
|
||||
/// The running digest shared by snapshot stream codecs and checkpoints.
|
||||
pub const PrefixDigest = [Blake3.digest_length]u8;
|
||||
|
||||
/// CRC32C as specified by the snapshot format. This is the parameter set
|
||||
/// Zig's standard library names after its iSCSI use, backed by dedicated
|
||||
/// CRC32C instructions where the target has them.
|
||||
@@ -49,10 +44,10 @@ pub const Tag = enum(u16) {
|
||||
/// One screen's complete history manifest and page sequence.
|
||||
history = 4,
|
||||
|
||||
/// Digest marking the validated terminal-state prefix.
|
||||
/// Marker separating the renderable terminal state from history.
|
||||
ready = 5,
|
||||
|
||||
/// Digest validating the complete snapshot blob.
|
||||
/// Marker terminating the complete snapshot blob.
|
||||
finish = 6,
|
||||
|
||||
/// Canonical unfinished standard TerminalStream input.
|
||||
@@ -155,11 +150,10 @@ pub const Checksum = struct {
|
||||
|
||||
/// Streams complete records while retaining only one payload at a time.
|
||||
///
|
||||
/// All emitted bytes pass through one unbuffered BLAKE3 writer. The scratch
|
||||
/// allocation is retained between records so a stream's peak memory is the
|
||||
/// largest record payload rather than the complete snapshot.
|
||||
/// The scratch allocation is retained between records so a stream's peak
|
||||
/// memory is the largest record payload rather than the complete snapshot.
|
||||
pub const Writer = struct {
|
||||
hashing: std.Io.Writer.Hashed(Blake3),
|
||||
destination: *std.Io.Writer,
|
||||
scratch: std.Io.Writer.Allocating,
|
||||
active_tag: ?Tag,
|
||||
|
||||
@@ -168,7 +162,7 @@ pub const Writer = struct {
|
||||
destination: *std.Io.Writer,
|
||||
) Writer {
|
||||
return .{
|
||||
.hashing = destination.hashed(Blake3.init(.{}), &.{}),
|
||||
.destination = destination,
|
||||
.scratch = .init(alloc),
|
||||
.active_tag = null,
|
||||
};
|
||||
@@ -181,10 +175,10 @@ pub const Writer = struct {
|
||||
self.* = undefined;
|
||||
}
|
||||
|
||||
/// Return the digest-updating writer for unframed snapshot bytes.
|
||||
/// Return the destination writer for unframed snapshot bytes.
|
||||
pub fn writer(self: *Writer) *std.Io.Writer {
|
||||
assert(self.active_tag == null);
|
||||
return &self.hashing.writer;
|
||||
return self.destination;
|
||||
}
|
||||
|
||||
/// Begin one record and return its reusable payload writer.
|
||||
@@ -229,8 +223,8 @@ pub const Writer = struct {
|
||||
var header_writer: std.Io.Writer = .fixed(&header_bytes);
|
||||
header.encode(&header_writer) catch unreachable;
|
||||
|
||||
try self.hashing.writer.writeAll(&header_bytes);
|
||||
try self.hashing.writer.writeAll(payload);
|
||||
try self.destination.writeAll(&header_bytes);
|
||||
try self.destination.writeAll(payload);
|
||||
}
|
||||
|
||||
/// Discard the active record without emitting any bytes.
|
||||
@@ -241,53 +235,6 @@ pub const Writer = struct {
|
||||
self.scratch.shrinkRetainingCapacity(0);
|
||||
self.active_tag = null;
|
||||
}
|
||||
|
||||
/// Finalize the prefix written so far without consuming the hasher.
|
||||
pub fn prefixDigest(self: *const Writer) PrefixDigest {
|
||||
// Checkpoints require an exact byte boundary. Writer owns this
|
||||
// adapter and always constructs it without a buffer.
|
||||
assert(self.active_tag == null);
|
||||
assert(self.scratch.writer.end == 0);
|
||||
assert(self.hashing.writer.buffer.len == 0);
|
||||
assert(self.hashing.writer.buffered().len == 0);
|
||||
|
||||
var result: PrefixDigest = undefined;
|
||||
self.hashing.hasher.final(&result);
|
||||
return result;
|
||||
}
|
||||
};
|
||||
|
||||
/// Hashes snapshot bytes as they are consumed without reading ahead.
|
||||
pub const StreamReader = struct {
|
||||
hashing: std.Io.Reader.Hashed(Blake3),
|
||||
|
||||
pub fn init(input: *std.Io.Reader) StreamReader {
|
||||
return .{
|
||||
.hashing = input.hashed(Blake3.init(.{}), &.{}),
|
||||
};
|
||||
}
|
||||
|
||||
/// Return the digest-updating reader used before and through READY.
|
||||
pub fn reader(self: *StreamReader) *std.Io.Reader {
|
||||
return &self.hashing.reader;
|
||||
}
|
||||
|
||||
/// Return the source used to consume FINISH without hashing it.
|
||||
pub fn source(self: *StreamReader) *std.Io.Reader {
|
||||
return self.hashing.in;
|
||||
}
|
||||
|
||||
/// Finalize the prefix consumed so far without consuming the hasher.
|
||||
pub fn prefixDigest(self: *const StreamReader) PrefixDigest {
|
||||
// A nonempty adapter buffer could contain bytes beyond a checkpoint.
|
||||
// Construction is private to this type and fixes its capacity at zero.
|
||||
assert(self.hashing.reader.buffer.len == 0);
|
||||
assert(self.hashing.reader.bufferedLen() == 0);
|
||||
|
||||
var result: PrefixDigest = undefined;
|
||||
self.hashing.hasher.final(&result);
|
||||
return result;
|
||||
}
|
||||
};
|
||||
|
||||
/// Reads one complete record.
|
||||
@@ -297,7 +244,7 @@ pub const StreamReader = struct {
|
||||
pub const Reader = struct {
|
||||
header: Header,
|
||||
|
||||
// The limited reader normally streams directly into the hashing reader's
|
||||
// The limited reader normally streams directly into the checksum reader's
|
||||
// buffer. One byte is enough for operations that require it to buffer,
|
||||
// such as peek and discard.
|
||||
limited_buffer: [1]u8,
|
||||
|
||||
@@ -242,7 +242,7 @@ pub const EncodeError = Allocator.Error || PayloadEncodeError || page.EncodeErro
|
||||
///
|
||||
/// The suffix begins with the page containing the active area's first row and
|
||||
/// ends with the newest page. Completed records may already be emitted if a
|
||||
/// later record fails; the missing READY checkpoint makes that prefix invalid.
|
||||
/// later record fails; the missing READY marker makes that prefix invalid.
|
||||
pub fn encode(
|
||||
screen: *const TerminalScreen,
|
||||
key: TerminalScreenKey,
|
||||
|
||||
@@ -9,15 +9,15 @@ doc: |
|
||||
|
||||
A complete snapshot contains an envelope, terminal-wide state, one or two
|
||||
renderable screen sequences, one raw standard-Stream CONTINUATION, a READY
|
||||
checkpoint, matching history sequences, and a FINISH checkpoint. SCREEN pages
|
||||
marker, matching history sequences, and a FINISH marker. SCREEN pages
|
||||
are oldest-to-newest. HISTORY pages are newest-to-oldest. FINISH terminates the
|
||||
snapshot; bytes that follow belong to the containing transport and are outside
|
||||
this schema. Each SCREEN declares its complete logical history extent before
|
||||
READY.
|
||||
|
||||
Record CRC32C values and checkpoint BLAKE3-256 digests are represented here
|
||||
but cannot be calculated by portable Kaitai Struct expressions. The adjacent
|
||||
verify-kaitai.py script validates those values after parsing.
|
||||
Record CRC32C values are represented here but cannot be calculated by
|
||||
portable Kaitai Struct expressions. The adjacent verify-kaitai.py script
|
||||
validates those values after parsing.
|
||||
seq:
|
||||
- id: envelope
|
||||
type: envelope
|
||||
@@ -217,6 +217,7 @@ types:
|
||||
size: header.payload_length
|
||||
|
||||
checkpoint_record:
|
||||
doc: READY and FINISH are empty marker records.
|
||||
params:
|
||||
- id: expected_tag
|
||||
type: u2
|
||||
@@ -224,8 +225,9 @@ types:
|
||||
- id: header
|
||||
type: record_header(expected_tag)
|
||||
- id: payload
|
||||
type: checkpoint_payload
|
||||
size: header.payload_length
|
||||
valid:
|
||||
expr: _.size == 0
|
||||
|
||||
screen_sequence:
|
||||
doc: SCREEN followed by its declared PAGE records, oldest-to-newest.
|
||||
@@ -247,15 +249,6 @@ types:
|
||||
repeat: expr
|
||||
repeat-expr: history.payload.page_count
|
||||
|
||||
checkpoint_payload:
|
||||
seq:
|
||||
- id: prefix_digest
|
||||
size: 32
|
||||
- id: trailing_data
|
||||
size-eos: true
|
||||
valid:
|
||||
expr: _.size == 0
|
||||
|
||||
terminal_payload:
|
||||
seq:
|
||||
- id: header
|
||||
|
||||
@@ -40,7 +40,7 @@ pub const EncodeOptions = struct {
|
||||
/// Encoding starts at the destination's current position. Only one record
|
||||
/// payload is buffered at a time; completed records stream immediately. On
|
||||
/// failure, the destination may contain a snapshot prefix without its required
|
||||
/// checkpoints, and an output failure may have written part of a record.
|
||||
/// FINISH marker, and an output failure may have written part of a record.
|
||||
pub fn encode(
|
||||
alloc: Allocator,
|
||||
destination: *std.Io.Writer,
|
||||
@@ -74,7 +74,7 @@ pub fn encode(
|
||||
// 4. Standard Stream continuation.
|
||||
try continuation.encode(options.continuation, &stream);
|
||||
|
||||
// 5. Ready checkpoint.
|
||||
// 5. Ready marker.
|
||||
try checkpoint.encode(.ready, &stream);
|
||||
|
||||
// 6. History
|
||||
@@ -168,8 +168,8 @@ pub const Decoded = struct {
|
||||
/// Incrementally restore one snapshot: renderable state first, then history.
|
||||
///
|
||||
/// The snapshot record order makes a terminal renderable at the READY
|
||||
/// checkpoint, after only a small prefix of the stream. `Decoder` exposes
|
||||
/// that boundary: `ready` returns a complete, authenticated `Decoded` while
|
||||
/// marker, after only a small prefix of the stream. `Decoder` exposes
|
||||
/// that boundary: `ready` returns a complete, integrity-checked `Decoded` while
|
||||
/// scrollback is still in flight, and each `next` call then applies one
|
||||
/// history PAGE record off the critical path. `snapshot.decode` is the
|
||||
/// one-shot form and is implemented over this type.
|
||||
@@ -189,13 +189,13 @@ pub const Decoded = struct {
|
||||
/// while (try decoder.next(alloc, &terminal)) |progress| {
|
||||
/// _ = progress; // Scrollback grew; e.g. refresh the scrollbar.
|
||||
/// }
|
||||
/// // FINISH validated: the complete snapshot is authenticated.
|
||||
/// // FINISH validated: the complete snapshot has been decoded.
|
||||
/// ```
|
||||
///
|
||||
/// The terminal is live between `next` calls: the caller may render it and
|
||||
/// even feed it PTY bytes that arrived after the snapshot cut. `next`
|
||||
/// re-validates its destination on every call and degrades to discarding,
|
||||
/// consuming and authenticating wire bytes without applying them, when live
|
||||
/// consuming and validating wire bytes without applying them, when live
|
||||
/// changes have made a page inapplicable. See `next` for the drop rules.
|
||||
///
|
||||
/// The decoder owns no allocations and needs no deinit. It retains the
|
||||
@@ -205,9 +205,7 @@ pub const Decoded = struct {
|
||||
/// position are invalid: abandon the decoder, and destroy an already
|
||||
/// transferred Terminal or keep it with partial history, but do not resume.
|
||||
pub const Decoder = struct {
|
||||
/// Hashes every consumed byte so READY and FINISH can be validated
|
||||
/// without buffering or rehashing the stream.
|
||||
stream: record.StreamReader,
|
||||
source: *std.Io.Reader,
|
||||
state: State,
|
||||
|
||||
const State = union(enum) {
|
||||
@@ -259,10 +257,8 @@ pub const Decoder = struct {
|
||||
|
||||
/// The decoder begins reading only when `ready` is called.
|
||||
pub fn init(source: *std.Io.Reader) Decoder {
|
||||
// StreamReader owns a zero-buffer hashing adapter, making checkpoint
|
||||
// boundaries part of its API rather than a caller-held invariant.
|
||||
return .{
|
||||
.stream = .init(source),
|
||||
.source = source,
|
||||
.state = .start,
|
||||
};
|
||||
}
|
||||
@@ -281,11 +277,11 @@ pub const Decoder = struct {
|
||||
DuplicateScreen,
|
||||
};
|
||||
|
||||
/// Decode through the READY checkpoint and return the renderable state.
|
||||
/// Decode through the READY marker and return the renderable state.
|
||||
///
|
||||
/// The result is transactional and complete for interactive use: the
|
||||
/// Terminal, its continuation, and the advisory history extents are all
|
||||
/// authenticated by READY. Scrollback history is not present yet and
|
||||
/// validated through READY. Scrollback history is not present yet and
|
||||
/// arrives through `next`. Asserts this is the first call.
|
||||
pub fn ready(
|
||||
self: *Decoder,
|
||||
@@ -295,7 +291,7 @@ pub const Decoder = struct {
|
||||
) ReadyError!Decoded {
|
||||
assert(self.state == .start);
|
||||
errdefer self.state = .failed;
|
||||
const reader = self.stream.reader();
|
||||
const reader = self.source;
|
||||
|
||||
// Read the envelope, which is currently just a verification step.
|
||||
try envelope.decode(reader);
|
||||
@@ -363,10 +359,8 @@ pub const Decoder = struct {
|
||||
.bytes => |bytes| alloc.free(bytes),
|
||||
};
|
||||
|
||||
// READY covers the exact envelope-through-CONTINUATION prefix.
|
||||
// Finalizing does not consume the hasher, so the same stream
|
||||
// continues toward FINISH.
|
||||
try checkpoint.decode(.ready, &self.stream);
|
||||
// READY marks the exact envelope-through-CONTINUATION boundary.
|
||||
try checkpoint.decode(.ready, self.source);
|
||||
|
||||
// Record where history may be applied. The declared keys route the
|
||||
// HISTORY sequences, and the generations detect declared screens
|
||||
@@ -399,7 +393,7 @@ pub const Decoder = struct {
|
||||
key: TerminalScreenKey,
|
||||
|
||||
/// Rows prepended above that screen's existing content, or zero
|
||||
/// when the page was consumed and authenticated but dropped.
|
||||
/// when the page was consumed and validated but dropped.
|
||||
rows: usize,
|
||||
|
||||
/// PAGE records still pending in the same HISTORY sequence.
|
||||
@@ -437,11 +431,11 @@ pub const Decoder = struct {
|
||||
///
|
||||
/// `t` must be the Terminal restored by this decoder's `ready` call,
|
||||
/// wherever the caller now stores it. Returns null once FINISH has
|
||||
/// validated, which authenticates the complete snapshot; the source is
|
||||
/// validated, which completes the snapshot; the source is
|
||||
/// left positioned exactly after it. Further calls return null.
|
||||
///
|
||||
/// Wire validation stays strict: framing, CRCs, record order, routing,
|
||||
/// and both checkpoint digests are always enforced. Application is
|
||||
/// and both marker records are always enforced. Application is
|
||||
/// forgiving, because the terminal is live and may have changed since
|
||||
/// READY. A page is consumed but dropped, reported as zero rows, when:
|
||||
///
|
||||
@@ -482,11 +476,10 @@ pub const Decoder = struct {
|
||||
state.current = null;
|
||||
}
|
||||
|
||||
// All sequences are consumed: FINISH authenticates READY plus
|
||||
// all history and ends the snapshot, leaving trailing transport
|
||||
// bytes unread.
|
||||
// All sequences are consumed: FINISH ends the snapshot, leaving
|
||||
// trailing transport bytes unread.
|
||||
if (state.pending == 0) {
|
||||
try checkpoint.decode(.finish, &self.stream);
|
||||
try checkpoint.decode(.finish, self.source);
|
||||
if (comptime build_options.slow_runtime_safety) {
|
||||
var it = state.generations.iterator();
|
||||
while (it.next()) |entry| {
|
||||
@@ -505,7 +498,7 @@ pub const Decoder = struct {
|
||||
// may not repeat.
|
||||
state.pending -= 1;
|
||||
var manifest: history.Decoder = undefined;
|
||||
try manifest.init(self.stream.reader());
|
||||
try manifest.init(self.source);
|
||||
const key = manifest.header.key;
|
||||
if (state.generations.get(key) == null) {
|
||||
return error.UnexpectedHistoryKey;
|
||||
@@ -549,10 +542,10 @@ pub const Decoder = struct {
|
||||
sequence.remaining -= 1;
|
||||
const rows: usize = rows: {
|
||||
const restored = destination orelse {
|
||||
// The record must still be consumed and hashed so FINISH
|
||||
// can authenticate the complete snapshot.
|
||||
// The record must still be consumed so decoding remains
|
||||
// aligned with the following records and FINISH marker.
|
||||
sequence.apply = false;
|
||||
try page.discard(self.stream.reader());
|
||||
try page.discard(self.source);
|
||||
break :rows 0;
|
||||
};
|
||||
// The one-shot history decoder rejects a Screen which already has
|
||||
@@ -562,7 +555,7 @@ pub const Decoder = struct {
|
||||
// the destination compatible, while per-page finalization enforces
|
||||
// its current scrollback limits.
|
||||
break :rows history.decodePage(
|
||||
self.stream.reader(),
|
||||
self.source,
|
||||
alloc,
|
||||
restored,
|
||||
) catch |err| switch (err) {
|
||||
@@ -595,7 +588,7 @@ pub const Decoder = struct {
|
||||
/// the returned result is either complete and ready or not (error return).
|
||||
/// Individual record codecs normalize optional semantic state, and history
|
||||
/// beyond the declared scrollback limits is dropped rather than rejected,
|
||||
/// while framing, checkpoints, declared sequence counts, and unique
|
||||
/// while framing, markers, declared sequence counts, and unique
|
||||
/// cross-record screen routing remain strict.
|
||||
///
|
||||
/// This is the one-shot form of `Decoder`, which can additionally hand the
|
||||
@@ -814,24 +807,16 @@ test "complete snapshot round trip with history and alternate screen" {
|
||||
// Independently hash that output so both its length and complete byte
|
||||
// sequence are checked without retaining a second snapshot copy.
|
||||
var discard: std.Io.Writer.Discarding = .init(&.{});
|
||||
var hashing = discard.writer.hashed(
|
||||
std.crypto.hash.Blake3.init(.{}),
|
||||
&.{},
|
||||
);
|
||||
var hashing = discard.writer.hashed(record.Crc32c.init(), &.{});
|
||||
try encode(testing.allocator, &hashing.writer, &t, test_encode_options);
|
||||
try testing.expectEqual(
|
||||
@as(u64, test_complete_fixture.len),
|
||||
discard.fullCount(),
|
||||
);
|
||||
var expected_digest: checkpoint.Digest = undefined;
|
||||
std.crypto.hash.Blake3.hash(
|
||||
&test_complete_fixture,
|
||||
&expected_digest,
|
||||
.{},
|
||||
try testing.expectEqual(
|
||||
record.Crc32c.hash(&test_complete_fixture),
|
||||
hashing.hasher.final(),
|
||||
);
|
||||
var actual_digest: checkpoint.Digest = undefined;
|
||||
hashing.hasher.final(&actual_digest);
|
||||
try testing.expectEqual(expected_digest, actual_digest);
|
||||
|
||||
// Restore the checked-in reference rather than the just-generated bytes.
|
||||
var encoded_source: std.Io.Reader = .fixed(&test_complete_fixture);
|
||||
@@ -868,7 +853,7 @@ test "complete snapshot round trip with history and alternate screen" {
|
||||
);
|
||||
|
||||
// Re-encoding is a compact semantic equality check over all TERMINAL,
|
||||
// SCREEN, PAGE, and HISTORY fields and both checkpoint boundaries.
|
||||
// SCREEN, PAGE, and HISTORY fields and both marker boundaries.
|
||||
var reencoded: std.Io.Writer.Allocating = .init(testing.allocator);
|
||||
defer reencoded.deinit();
|
||||
try encode(
|
||||
@@ -1294,8 +1279,8 @@ test "complete snapshot encoding streams from the current writer position" {
|
||||
});
|
||||
defer t.deinit(testing.allocator);
|
||||
|
||||
// Prefix hashing begins with this call's envelope, independent of bytes
|
||||
// that were already present in the destination.
|
||||
// Encoding begins with this call's envelope, independent of bytes that
|
||||
// were already present in the destination.
|
||||
var nonempty: std.Io.Writer.Allocating = .init(testing.allocator);
|
||||
defer nonempty.deinit();
|
||||
try nonempty.writer.writeAll("prefix");
|
||||
@@ -1340,7 +1325,7 @@ test "complete snapshot encoding streams from the current writer position" {
|
||||
);
|
||||
}
|
||||
|
||||
test "complete snapshot rejects ordering and invalid checkpoints" {
|
||||
test "complete snapshot rejects ordering and invalid markers" {
|
||||
const testing = std.testing;
|
||||
|
||||
var t = try Terminal.init(testing.io, testing.allocator, .{
|
||||
@@ -1425,8 +1410,8 @@ test "complete snapshot rejects ordering and invalid checkpoints" {
|
||||
),
|
||||
);
|
||||
|
||||
// Construct a correctly framed READY with an intentionally unrelated
|
||||
// digest so the full driver, rather than record CRC validation, rejects it.
|
||||
// Construct a correctly framed READY with a nonempty payload so the full
|
||||
// driver rejects a marker that does not match the required empty shape.
|
||||
var invalid_ready: std.Io.Writer.Allocating = .init(testing.allocator);
|
||||
defer invalid_ready.deinit();
|
||||
var invalid_ready_stream: record.Writer = .init(
|
||||
@@ -1440,16 +1425,13 @@ test "complete snapshot rejects ordering and invalid checkpoints" {
|
||||
try continuation.encode(.ground, &invalid_ready_stream);
|
||||
const ready_payload = invalid_ready_stream.begin(.ready);
|
||||
errdefer invalid_ready_stream.cancel();
|
||||
try ready_payload.splatByteAll(
|
||||
0,
|
||||
@sizeOf(checkpoint.Digest),
|
||||
);
|
||||
try ready_payload.writeByte(0);
|
||||
try invalid_ready_stream.finish();
|
||||
var invalid_ready_source: std.Io.Reader = .fixed(
|
||||
invalid_ready.written(),
|
||||
);
|
||||
try testing.expectError(
|
||||
error.InvalidDigest,
|
||||
error.PayloadNotExhausted,
|
||||
decode(
|
||||
testing.allocator,
|
||||
testing.io,
|
||||
@@ -1925,7 +1907,7 @@ test "incremental decode discards history for screens changed since READY" {
|
||||
}
|
||||
|
||||
// A screen removed since READY discards its history. FINISH still
|
||||
// validates, proving discarded bytes remain digest-covered.
|
||||
// validates, proving discarded records leave the stream aligned.
|
||||
{
|
||||
var source: std.Io.Reader = .fixed(crafted.written());
|
||||
var decoder: Decoder = .init(&source);
|
||||
@@ -2014,7 +1996,7 @@ test "decode drops history beyond the declared scrollback limits" {
|
||||
|
||||
// Declare a byte limit the complete history cannot satisfy. The limit
|
||||
// is validated at page granularity on decode, so the snapshot itself
|
||||
// remains well formed and fully authenticated.
|
||||
// remains well formed and fully validated.
|
||||
var t = try testHistoryTerminal(3);
|
||||
defer t.deinit(testing.allocator);
|
||||
t.screens.get(.primary).?.pages.limits.set(.bytes, 1);
|
||||
@@ -2105,8 +2087,8 @@ test "incremental decode failure leaves applied history usable" {
|
||||
);
|
||||
|
||||
// The transferred terminal was never owned by the decoder: it remains
|
||||
// valid with the contiguous history prefix that did apply, and only
|
||||
// the FINISH authentication of the whole snapshot is lost.
|
||||
// valid with the contiguous history prefix that did apply; only the
|
||||
// incomplete snapshot stream is lost.
|
||||
const primary = restored.screens.get(.primary).?;
|
||||
try testing.expectEqual(@as(usize, 3), primary.pages.totalPages());
|
||||
try testing.expectEqual(@as(u21, 'A'), testTopLeftCodepoint(primary));
|
||||
|
||||
@@ -6,12 +6,5 @@
|
||||
# Generated by its snapshot test; review before replacing.
|
||||
# On mismatch, the candidate is copied to the repository root.
|
||||
|
||||
# prefix covered by READY
|
||||
61 62 63
|
||||
|
||||
# READY record header: tag, payload length, CRC32C
|
||||
05 00 20 00 00 00 90 5b 2d 06
|
||||
|
||||
# BLAKE3-256 digest of the prefix
|
||||
64 37 b3 ac 38 46 51 33 ff b6 3b 75 27 3a 8d b5
|
||||
48 c5 58 46 5d 79 db 03 fd 35 9c 6c d5 bd 9d 85
|
||||
# offset 0x00000000: encoded bytes
|
||||
61 62 63 05 00 00 00 00 00 e4 20 ef 0a # 0x00000000
|
||||
|
||||
36
src/terminal/snapshot/testdata/complete-v1.hex
vendored
36
src/terminal/snapshot/testdata/complete-v1.hex
vendored
@@ -97,28 +97,24 @@ ee ee 80 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000037a
|
||||
# offset 0x000004aa: continuation record, payload 0 bytes
|
||||
07 00 00 00 00 00 27 80 63 d1 # 0x000004aa
|
||||
|
||||
# offset 0x000004b4: ready record, payload 32 bytes
|
||||
05 00 20 00 00 00 4d 17 72 ed dd 87 26 75 cf 8e # 0x000004b4
|
||||
e5 1f 37 e3 05 92 0a e2 f8 ef c1 16 54 be 49 e7 # 0x000004c4
|
||||
b2 6c df 40 d2 77 fe 05 82 ea # 0x000004d4
|
||||
# offset 0x000004b4: ready record, payload 0 bytes
|
||||
05 00 00 00 00 00 e4 20 ef 0a # 0x000004b4
|
||||
|
||||
# offset 0x000004de: history record, payload 6 bytes
|
||||
04 00 06 00 00 00 20 32 ed e1 00 00 02 00 00 00 # 0x000004de
|
||||
# offset 0x000004be: history record, payload 6 bytes
|
||||
04 00 06 00 00 00 20 32 ed e1 00 00 02 00 00 00 # 0x000004be
|
||||
|
||||
# offset 0x000004ee: page record, payload 31 bytes
|
||||
03 00 1f 00 00 00 4a ed 2f c2 02 00 02 00 00 00 # 0x000004ee
|
||||
00 00 10 00 c0 00 00 04 00 00 00 08 00 00 00 01 # 0x000004fe
|
||||
00 42 00 00 00 00 00 00 00 # 0x0000050e
|
||||
# offset 0x000004ce: page record, payload 31 bytes
|
||||
03 00 1f 00 00 00 4a ed 2f c2 02 00 02 00 00 00 # 0x000004ce
|
||||
00 00 10 00 c0 00 00 04 00 00 00 08 00 00 00 01 # 0x000004de
|
||||
00 42 00 00 00 00 00 00 00 # 0x000004ee
|
||||
|
||||
# offset 0x00000517: page record, payload 31 bytes
|
||||
03 00 1f 00 00 00 23 6a 6b 19 02 00 02 00 00 00 # 0x00000517
|
||||
00 00 10 00 c0 00 00 04 00 00 00 08 00 00 00 01 # 0x00000527
|
||||
00 41 00 00 00 00 00 00 00 # 0x00000537
|
||||
# offset 0x000004f7: page record, payload 31 bytes
|
||||
03 00 1f 00 00 00 23 6a 6b 19 02 00 02 00 00 00 # 0x000004f7
|
||||
00 00 10 00 c0 00 00 04 00 00 00 08 00 00 00 01 # 0x00000507
|
||||
00 41 00 00 00 00 00 00 00 # 0x00000517
|
||||
|
||||
# offset 0x00000540: history record, payload 6 bytes
|
||||
04 00 06 00 00 00 b8 7a ba b1 01 00 00 00 00 00 # 0x00000540
|
||||
# offset 0x00000520: history record, payload 6 bytes
|
||||
04 00 06 00 00 00 b8 7a ba b1 01 00 00 00 00 00 # 0x00000520
|
||||
|
||||
# offset 0x00000550: finish record, payload 32 bytes
|
||||
06 00 20 00 00 00 5a 2d a8 f4 25 b2 49 b4 3e 64 # 0x00000550
|
||||
5a 2e d6 7d 7f 3d 60 5b db 9b 4c 3a 29 00 97 ec # 0x00000560
|
||||
d6 c7 1d ea da d3 a9 fd 3d 68 # 0x00000570
|
||||
# offset 0x00000530: finish record, payload 0 bytes
|
||||
06 00 00 00 00 00 3e eb 53 3e # 0x00000530
|
||||
|
||||
@@ -18,8 +18,8 @@ https://ide.kaitai.io/.
|
||||
The generated Python parser exists only in a temporary directory. This keeps
|
||||
snapshot.ksy as the source of truth and ensures this check cannot accidentally
|
||||
pass against stale generated code. After structural parsing, the script checks
|
||||
record CRC32C values, checkpoint BLAKE3 digests, and cross-record invariants
|
||||
that portable Kaitai expressions cannot represent.
|
||||
record CRC32C values and cross-record invariants that portable Kaitai
|
||||
expressions cannot represent.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -37,7 +37,6 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
try:
|
||||
from blake3 import blake3
|
||||
from kaitaistruct import KaitaiStream
|
||||
except ImportError as error:
|
||||
raise SystemExit(
|
||||
@@ -189,7 +188,7 @@ def validate_record(record: Any, data: bytes, offset: int) -> int:
|
||||
|
||||
|
||||
def all_snapshot_records(snapshot: Any) -> list[Any]:
|
||||
"""Return complete-snapshot records in their authenticated wire order."""
|
||||
"""Return complete-snapshot records in wire order."""
|
||||
records = [snapshot.terminal]
|
||||
for sequence in snapshot.screens:
|
||||
records.append(sequence.screen)
|
||||
@@ -204,7 +203,7 @@ def all_snapshot_records(snapshot: Any) -> list[Any]:
|
||||
|
||||
|
||||
def validate_complete_snapshot(snapshot: Any, data: bytes) -> None:
|
||||
"""Validate ordering relationships, record CRCs, and checkpoints."""
|
||||
"""Validate ordering relationships, record CRCs, and markers."""
|
||||
screen_keys = [
|
||||
sequence.screen.payload.header.key for sequence in snapshot.screens
|
||||
]
|
||||
@@ -282,14 +281,6 @@ def validate_complete_snapshot(snapshot: Any, data: bytes) -> None:
|
||||
|
||||
offset = SNAPSHOT_ENVELOPE_SIZE
|
||||
for record in all_snapshot_records(snapshot):
|
||||
if record is snapshot.ready:
|
||||
expected = blake3(data[:offset]).digest()
|
||||
if record.payload.prefix_digest != expected:
|
||||
raise ValueError("READY BLAKE3-256 digest does not match")
|
||||
elif record is snapshot.finish:
|
||||
expected = blake3(data[:offset]).digest()
|
||||
if record.payload.prefix_digest != expected:
|
||||
raise ValueError("FINISH BLAKE3-256 digest does not match")
|
||||
offset = validate_record(record, data, offset)
|
||||
|
||||
if offset != len(data):
|
||||
@@ -399,15 +390,6 @@ def main() -> int:
|
||||
raise ValueError(
|
||||
f"{fixture_label}: record has trailing data"
|
||||
)
|
||||
if hasattr(parsed.payload, "prefix_digest"):
|
||||
expected = blake3(
|
||||
fixture.data[:fixture.offset]
|
||||
).digest()
|
||||
if parsed.payload.prefix_digest != expected:
|
||||
raise ValueError(
|
||||
f"{fixture_label}: checkpoint digest does not match"
|
||||
)
|
||||
|
||||
print(f"ok {fixture_label}")
|
||||
|
||||
print(f"validated {len(fixture_paths)} snapshot fixtures")
|
||||
|
||||
Reference in New Issue
Block a user