mirrors/ghostty
1
0
Fork 0
mirror of https://github.com/ghostty-org/ghostty.git synced 2026-08-01 05:09:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

terminal/snapshot: binary snapshot format (#13534)

Browse Source 
This adds the first version of a binary snapshot format for terminal
state.

Use cases: replay software (like asciinema), multiplexers (like zmx),
scrollback-saving on disk, etc.

The intention of the binary snapshot format is to be able to fully
encode and decode terminal state across mediums such as network and
disk. You can also encode partial terminal state (e.g. only one screen
or even one page of contents). Long term, the intention is to also
support streaming state while a live terminal is running, but this
initial PR focuses on the full snapshot first (with some design choices
to get to the streaming state in the future).

The format is documented in the Zig code, but I also did a
[Kaitai](https://kaitai.io/) descriptor and both the Zig and Kaitai spec
verify they can parse committed fixtures. This helps identify drift in
the format or encoder/decoders in any way since this must ultimately be
a fixed format.

> [!NOTE]
>
> **On reviewability:** this is a massive PR that I don't expect anyone
to reasonably review. I'm going through it line-by-line (again) but I
purposely extracted any changes that affect other parts of Ghostty out
to other already-merged PRs. This one is isolated purely to a package
that isn't called by any client software. **So the plan is if this rough
shape looks good I'll merge it and we'll iterate from there.**

> [!WARNING]
>
> **Experimental.** The format can and will change. And we may also
decide that binary snapshotting in this way isn't the right direction
altogether (although, I'm pretty confident it is). It'd be impossible to
get a single large perfect PR because it'd be even larger than this by
multiples. So instead, we'll iterate on main so long as this work is not
touching any production code, which it isn't!

## Example

Encode:

```zig
const terminal = @import("terminal/main.zig");

var file_buffer: [16 * 1024]u8 = undefined;
var file_writer = file.writer(io, &file_buffer);
try terminal.snapshot.encode(alloc, &file_writer.interface, &t);
try file_writer.interface.flush();
```

Decode a full terminal:

```zig
var t = try terminal.snapshot.decode(&reader, io, alloc);
defer t.deinit(alloc);
```

## Future

This PR purposely only supports a synchronous encode/decode. I wanted to
get the large groundwork in before iterating further. Some iterations in
the future:

* Kitty graphics
* Live terminal snapshotting
* PTY stream continuation records (so VT state machines can stay in
sync)
* Performance work (encoding and decoding, maybe size)
* Configurable limits to prevent DoS
* C API
* etc...

## Wire format

The "robustness principle" is a guiding principle: "be conservative in
what you do, be liberal in what you accept from others." Our encoders
have a lot of extra validation, our decoders massage invalid data into
reasonable defaults (e.g. invalid styles become unstyled text).

> [!IMPORTANT]
>
> **Version 1 has no compatibility promise.** We use version 1 in the
envelope header. We will absolutely break this format as needed as we
iterate and improve on it...

### Envelope

Every snapshot starts with a fixed ten-byte envelope:

| Offset | Size | Field |
| ---: | ---: | :--- |
| 0 | 8 | Magic: `GHOSTSNP` |
| 8 | 2 | Snapshot version: `1` |

### Record framing

After the envelope, records are concatenated back-to-back. Every record
has a fixed header:

| Offset | Size | Field |
| ---: | ---: | :--- |
| 0 | 2 | Record tag |
| 2 | 4 | Payload length |
| 6 | 4 | CRC32C |
| 10 | variable | Payload |

CRC32C covers the encoded tag, payload length, and payload. The
payload-length boundary prevents a malformed record decoder from
consuming bytes belonging to the next record.

The registered record tags are:

| Value | Tag | Purpose |
| ---: | :--- | :--- |
| 1 | `TERMINAL` | Terminal-wide state and declared screens |
| 2 | `SCREEN` | One screen's live state and active page manifest |
| 3 | `PAGE` | One self-contained set of rows and cells |
| 4 | `HISTORY` | One screen's historical page manifest |
| 5 | `READY` | Digest of the renderable prefix |
| 6 | `FINISH` | Digest of the complete snapshot |

To view the format of each record, read its corresponding
`terminal/snapshot/<type>.zig` file.

### Complete record sequence

```text
+----------------------------------------+
| Envelope                               |
+----------------------------------------+
| TERMINAL                               |
+----------------------------------------+
| SCREEN * terminal.screen_count         |
| PAGE   * each screen.page_count        |
+----------------------------------------+
| READY                                  |
+----------------------------------------+
| HISTORY * terminal.screen_count        |
| PAGE    * each history.page_count       |
+----------------------------------------+
| FINISH                                 |
+----------------------------------------+
| Optional containing-transport bytes    |
+----------------------------------------+
```

`SCREEN` and `HISTORY` groups are routed by their encoded screen key and
may arrive in either key order.

## Checkpoints and validation

Each record has an independent CRC32C, but per-record checksums cannot
detect a valid record being reordered, omitted, or duplicated. `READY`
and `FINISH` therefore contain BLAKE3-256 digests over exact snapshot
prefixes:

- `READY` covers the envelope, `TERMINAL`, and all live `SCREEN`/`PAGE`
sequences. It does not include itself.
- `FINISH` covers that same prefix, the complete `READY` record, and all
`HISTORY`/`PAGE` sequences. It does not include itself.

This gives the format two useful integrity boundaries:

```text
envelope ... active pages | READY | history pages | FINISH
<------ renderable ------->
<------------- complete snapshot --------------->
```

## Performance

### Size, Compression Recommended

We intentionally use a simple grid over something like RLE (run-length
encoding). So every row contains exactly `columns` cells and each is
16-bytes! This is large! A 80x24, 10,000 line scrollback terminal
uncompressed would be ~13MB. However, with zstd level 1 compression that
goes down to 260K.

### Speed

We haven't benchmarked encoding or decoding speed yet. This PR focused
on getting a format in place. This will be heavily optimized later. I
suspect its probably pretty darn slow, actually.

## Kaitai Struct

I added a `snapshot.ksy` Kaita Struct spec that independently describes
the complete format. This is used by us for format validation but it can
also be used to programmatically generate parsers. For example, our test
fixture in the Kaita Struct web IDE decodes to:

<img width="523" height="779" alt="image"
src="https://github.com/user-attachments/assets/cd199c73-b6d6-4b35-8957-0cfe3d1a18f2"
/>

**AI Usage:** This work was done in concert with various models and
agents. Writing full encoders/decoders is tedious so it took a lot of
that way. A lot of review was done by AI (trying to find holes, issues,
inconsistencies). The actual binary protocol design and iteration was
done by me. This PR message was written by me.
This commit is contained in:
Mitchell Hashimoto
2026-07-31 19:26:59 -07:00
committed by GitHub
parent 4d605bf0d8 6b09bb3fca
commit 154ddc2a2f
42 changed files with 12296 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
.github/workflows/test.yml vendored
View File

@@ -111,6 +111,7 @@ jobs:
- test-sentry-linux
- test-i18n
- test-fuzz-libghostty
- test-kaitai
- test-lib-vt
- test-lib-vt-pkgconfig
- test-macos
@@ -1319,6 +1320,31 @@ jobs:
- name: Test
run: nix develop -c zig build test-lib-vt
test-kaitai:
if: github.repository == 'ghostty-org/ghostty' && needs.skip.outputs.skip != 'true'
needs: skip
runs-on: namespace-profile-ghostty-xsm
steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Cache
uses: namespacelabs/nscloud-cache-action@c5f8dab7560444c4bf8dbc64f1b203431873c547 # v1.6.1
with:
path: /nix
# Install Nix so the verifier uses the pinned compiler and runtimes.
- uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
with:
nix_path: nixpkgs=channel:nixos-unstable
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
with:
name: ghostty
authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"
- name: Verify Snapshot Kaitai Schema
run: nix develop -c src/terminal/snapshot/verify-kaitai.py
test-gtk:
strategy:
fail-fast: false

16
nix/devShell.nix
View File

@@ -59,6 +59,7 @@
zlib,
alejandra,
jq,
kaitai-struct-compiler,
minisign,
pandoc,
pinact,
@@ -85,6 +86,11 @@
gi_typelib_path = import ./build-support/gi-typelib-path.nix {
inherit pkgs lib stdenv;
};
python = python3.withPackages (python-pkgs: [
python-pkgs.blake3
python-pkgs.kaitaistruct
python-pkgs.ucs-detect
]);
in
mkShell {
name = "ghostty";
@@ -116,9 +122,10 @@ in
# Testing
parallel
python3
python
vttest
hyperfine
kaitai-struct-compiler
# wasm
wabt
@@ -138,11 +145,6 @@ in
blueprint-compiler
libadwaita
gtk4
# Python packages
(python3.withPackages (python-pkgs: [
python-pkgs.ucs-detect
]))
]
++ lib.optionals stdenv.hostPlatform.isLinux [
# My nix shell environment installs the non-interactive version
@@ -242,6 +244,6 @@ in
# We need to remove "xcrun" from the PATH. It is injected by
# some dependency but we need to rely on system Xcode tools
export PATH=$(echo "$PATH" | awk -v RS=: -v ORS=: '$0 !~ /xcrun/ || $0 == "/usr/bin" {print}' | sed 's/:$//')
export PATH="/opt/homebrew/opt/llvm/bin:/opt/homebrew/bin:/usr/local/opt/llvm/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$PATH"
export PATH="${python}/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/bin:/usr/local/opt/llvm/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$PATH"
'');
}

8
src/terminal/Screen.zig
View File

@@ -8,6 +8,7 @@ const ansi = @import("ansi.zig");
const charsets = @import("charsets.zig");
const fastmem = @import("../fastmem.zig");
const kitty = @import("kitty.zig");
const lib = @import("lib.zig");
const sgr = @import("sgr.zig");
const tripwire = @import("../tripwire.zig");
const unicode = @import("../unicode/main.zig");
@@ -114,7 +115,12 @@ pub const SemanticPrompt = struct {
.click = .none,
};
pub const SemanticClick = union(enum) {
pub const SemanticClickKind = lib.Enum(
lib.target,
&.{ "none", "click_events", "cl" },
);
pub const SemanticClick = union(SemanticClickKind) {
none,
click_events: osc.semantic_prompt.ClickEvents,
cl: osc.semantic_prompt.Click,

6
src/terminal/ScreenSet.zig
View File

@@ -30,9 +30,9 @@ active: *Screen,
/// All screens that are initialized.
all: std.EnumMap(Key, *Screen),
/// Monotonic generation counter for each screen key. This changes whenever a
/// screen is removed so external handles can distinguish a newly initialized
/// screen from stale references into destroyed screen storage.
/// Monotonic generation counter for each screen key. This changes whenever
/// screen storage is removed or replaced so external handles can distinguish a
/// newly initialized screen from stale references into destroyed storage.
generations: std.EnumMap(Key, usize),
pub fn init(

13
src/terminal/ansi.zig
View File

@@ -99,8 +99,11 @@ pub const ModifyKeyFormat = lib.Enum(
/// The protection modes that can be set for the terminal. See DECSCA and
/// ESC V, W.
pub const ProtectedMode = enum {
off,
iso, // ESC V, W
dec, // CSI Ps " q
};
pub const ProtectedMode = lib.Enum(
lib.target,
&.{
"off",
"iso", // ESC V, W
"dec", // CSI Ps " q
},
);

32
src/terminal/cursor.zig
View File

@@ -1,15 +1,25 @@
const lib = @import("../lib/main.zig");
/// The visual style of the cursor. Whether or not it blinks
/// is determined by mode 12 (modes.zig). This mode is synchronized
/// with CSI q, the same as xterm.
pub const Style = enum {
bar, // DECSCUSR 5, 6
block, // DECSCUSR 1, 2
underline, // DECSCUSR 3, 4
///
/// Bar, block, and underline correspond to DECSCUSR 5/6, 1/2, and 3/4.
/// Hollow block is Ghostty-specific and is reported as DECSCUSR 1 or 2.
pub const Style = lib.Enum(.zig, &.{
// DECSCUSR 5, 6
"bar",
/// The cursor styles below aren't known by DESCUSR and are custom
/// implemented in Ghostty. They are reported as some standard style
/// if requested, though.
/// Hollow block cursor. This is a block cursor with the center empty.
/// Reported as DECSCUSR 1 or 2 (block).
block_hollow,
};
// DECSCUSR 1, 2
"block",
// DECSCUSR 3, 4
"underline",
// The cursor styles below aren't known by DESCUSR and are custom
// implemented in Ghostty. They are reported as some standard style
// if requested, though.
// Hollow block cursor. This is a block cursor with the center empty.
// Reported as DECSCUSR 1 or 2 (block).
"block_hollow",
});

4
src/terminal/hyperlink.zig
View File

@@ -194,12 +194,12 @@ pub const PageEntry = struct {
const alloc = &page.string_alloc;
switch (self.id) {
.implicit => {},
.explicit => |v| alloc.free(
.explicit => |v| if (v.len > 0) alloc.free(
page.memory,
v.slice(page.memory),
),
}
alloc.free(
if (self.uri.len > 0) alloc.free(
page.memory,
self.uri.slice(page.memory),
);

1
src/terminal/main.zig
View File

@@ -21,6 +21,7 @@ pub const modes = @import("modes.zig");
pub const page = @import("page.zig");
pub const parse_table = @import("parse_table.zig");
pub const search = @import("search.zig");
pub const snapshot = @import("snapshot/main.zig");
pub const sgr = @import("sgr.zig");
pub const size = @import("size.zig");
pub const size_report = @import("size_report.zig");

80
src/terminal/osc/parsers/semantic_prompt.zig
View File

@@ -1,6 +1,7 @@
//! https://gitlab.freedesktop.org/Per_Bothner/specifications/blob/master/proposals/semantic-prompts.md
const std = @import("std");
const lib = @import("../../lib.zig");
const Parser = @import("../../osc.zig").Parser;
const OSCCommand = @import("../../osc.zig").Command;
const string_encoding = @import("../../../os/string_encoding.zig");
@@ -68,7 +69,10 @@ pub const Command = struct {
// See https://github.com/ghostty-org/ghostty/issues/10865 and
// https://github.com/kovidgoyal/kitty/issues/9500
// for further details.
pub const ClickEvents = enum { absolute, relative };
pub const ClickEvents = lib.Enum(
lib.target,
&.{ "absolute", "relative" },
);
pub const Option = enum {
aid,
@@ -199,7 +203,7 @@ pub const Option = enum {
return switch (self) {
.aid => value,
.cl => .init(value),
.cl => parseClick(value),
.prompt_kind => if (value.len == 1) PromptKind.init(value[0]) else null,
.err => value,
.redraw => if (std.mem.eql(u8, value, "0"))
@@ -234,42 +238,50 @@ pub const Option = enum {
/// The `cl` option specifies what kind of cursor key sequences are handled
/// by the application for click-to-move-cursor functionality.
pub const Click = enum {
/// Value: "line". Allows motion within a single input line using standard
/// left/right arrow escape sequences. Only a single left/right sequence
/// should be emitted for double-width characters.
line,
///
/// `line` allows movement within one input line. `multiple` allows movement
/// across lines with left/right sequences. The two vertical modes additionally
/// allow up/down sequences, with `smart_vertical` permitting editor-aware
/// column clamping.
pub const Click = lib.Enum(
lib.target,
&.{
// Value: "line". Allows motion within a single input line using
// standard left/right arrow escape sequences. Only a single left/right
// sequence should be emitted for double-width characters.
"line",
/// Value: "m". Allows movement between different lines in the same group,
/// but only using left/right arrow escape sequences.
multiple,
// Value: "m". Allows movement between different lines in the same
// group, but only using left/right arrow escape sequences.
"multiple",
/// Value: "v". Like `multiple` but cursor up/down should be used. The
/// terminal should be conservative when moving between lines: move the
/// cursor left to the start of line, emit the needed up/down sequences,
/// then move the cursor right to the clicked destination.
conservative_vertical,
// Value: "v". Like `multiple` but cursor up/down should be used. The
// terminal should be conservative when moving between lines: move the
// cursor left to the start of line, emit the needed up/down sequences,
// then move the cursor right to the clicked destination.
"conservative_vertical",
/// Value: "w". Like `conservative_vertical` but specifies that there are
/// no spurious spaces at the end of the line, and the application editor
/// handles "smart vertical movement" (moving 2 lines up from position 20,
/// where the intermediate line is 15 chars wide and the destination is
/// 18 chars wide, ends at position 18).
smart_vertical,
// Value: "w". Like `conservative_vertical` but specifies that there
// are no spurious spaces at the end of the line, and the application
// editor handles "smart vertical movement" (moving 2 lines up from
// position 20, where the intermediate line is 15 chars wide and the
// destination is 18 chars wide, ends at position 18).
"smart_vertical",
},
);
pub fn init(value: []const u8) ?Click {
return if (value.len == 1) switch (value[0]) {
'm' => .multiple,
'v' => .conservative_vertical,
'w' => .smart_vertical,
else => null,
} else if (std.mem.eql(
u8,
value,
"line",
)) .line else null;
}
};
fn parseClick(value: []const u8) ?Click {
return if (value.len == 1) switch (value[0]) {
'm' => .multiple,
'v' => .conservative_vertical,
'w' => .smart_vertical,
else => null,
} else if (std.mem.eql(
u8,
value,
"line",
)) .line else null;
}
pub const PromptKind = enum {
initial,

7
src/terminal/snapshot/AGENTS.md Normal file
View File

@@ -0,0 +1,7 @@
# Terminal Binary Snapshot
- Apply the robustness principle: "be conservative in what you do, be liberal
in what you accept from others."
- Validate on encode
- Gracefully degrade on decode, for example invalid styles degrade to
ignoring the style.

275
src/terminal/snapshot/checkpoint.zig Normal file
View File

@@ -0,0 +1,275 @@
//! READY and FINISH snapshot checkpoint records.
//!
//! Each checkpoint payload is one BLAKE3-256 digest of every snapshot byte
//! before that checkpoint's record header. This binds record order and detects
//! omitted or duplicated records in ways that independent record CRCs cannot.
//!
//! READY covers the envelope, TERMINAL, and all SCREEN/PAGE sequences. FINISH
//! covers that same prefix plus the complete READY record and all HISTORY/PAGE
//! sequences. Neither digest includes its own record. FINISH terminates one
//! snapshot; bytes after it belong to the containing transport.
//!
//! The digest does not replace record framing. Its 32-byte payload is still
//! protected by the record's CRC32C.
//!
//! ## Binary Format
//!
//! READY and FINISH use the same fixed payload:
//!
//! ```text
//! 0 +----------------------------+
//! | BLAKE3-256 prefix digest |
//! | 32 bytes |
//! 32 +----------------------------+
//! ```
const std = @import("std");
const test_fixture = @import("fixture.zig");
const record = @import("record.zig");
const Blake3 = std.crypto.hash.Blake3;
/// The prefix digest exchanged by checkpoint codecs and the snapshot driver.
pub const Digest = record.PrefixDigest;
comptime {
std.debug.assert(@sizeOf(Digest) == 32);
}
/// Selects one of the two checkpoint positions in a snapshot.
pub const Kind = enum {
ready,
finish,
fn tag(self: Kind) record.Tag {
return switch (self) {
.ready => .ready,
.finish => .finish,
};
}
};
pub const EncodeError = std.Io.Writer.Error ||
record.Writer.FinishError;
/// Append one checkpoint covering every byte already emitted by `stream`.
///
/// Finalizing does not consume the running hasher. READY is therefore included
/// as the stream continues toward FINISH.
pub fn encode(
kind: Kind,
stream: *record.Writer,
) EncodeError!void {
const digest = stream.prefixDigest();
const payload = stream.begin(kind.tag());
errdefer stream.cancel();
try payload.writeAll(&digest);
try stream.finish();
}
pub const DecodeError = record.Reader.InitError ||
record.Reader.FinishError ||
error{
/// The next record is valid but is not the expected checkpoint.
UnexpectedRecordTag,
/// The checkpoint does not describe the preceding snapshot bytes.
InvalidDigest,
};
/// Decode and validate one checkpoint against the running prefix digest.
///
/// READY is consumed through the hashing reader so FINISH covers it. FINISH is
/// consumed from the underlying source so neither checkpoint includes itself.
pub fn decode(
kind: Kind,
stream: *record.StreamReader,
) DecodeError!void {
const expected = stream.prefixDigest();
const source = if (kind == .finish)
stream.source()
else
stream.reader();
var record_reader: record.Reader = undefined;
try record_reader.init(source);
if (record_reader.header.tag != kind.tag()) {
return error.UnexpectedRecordTag;
}
var actual: Digest = undefined;
try record_reader.payloadReader().readSliceAll(&actual);
try record_reader.finish();
if (!std.mem.eql(u8, &expected, &actual)) return error.InvalidDigest;
}
const test_ready_fixture = test_fixture.parse(
@embedFile("testdata/checkpoint-ready-v1.hex"),
);
test "READY golden encoding and BLAKE3-256 registry" {
const prefix = "abc";
var snapshot: std.Io.Writer.Allocating = .init(std.testing.allocator);
defer snapshot.deinit();
var stream: record.Writer = .init(
std.testing.allocator,
&snapshot.writer,
);
defer stream.deinit();
try stream.writer().writeAll(prefix);
try encode(.ready, &stream);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/checkpoint-ready-v1.hex",
"snapshot_fixture-checkpoint-ready-v1.hex",
&test_ready_fixture,
snapshot.written(),
);
// The checked-in payload independently locks the BLAKE3-256 registry.
var actual: Digest = undefined;
Blake3.hash(prefix, &actual, .{});
const payload_offset = prefix.len + record.Header.len;
try std.testing.expectEqualSlices(
u8,
test_ready_fixture[payload_offset..][0..actual.len],
&actual,
);
// Finalizing a streaming hasher neither consumes nor resets it.
var hasher = Blake3.init(.{});
hasher.update("a");
var prefix_digest: Digest = undefined;
Blake3.hash("a", &prefix_digest, .{});
hasher.final(&actual);
try std.testing.expectEqual(prefix_digest, actual);
hasher.update("bc");
hasher.final(&actual);
try std.testing.expectEqualSlices(
u8,
test_ready_fixture[payload_offset..][0..actual.len],
&actual,
);
// Decode the checked-in record rather than the generated candidate.
var source: std.Io.Reader = .fixed(&test_ready_fixture);
var source_stream: record.StreamReader = .init(&source);
try source_stream.reader().discardAll(prefix.len);
try decode(.ready, &source_stream);
}
test "READY and FINISH checkpoint coverage" {
const testing = std.testing;
var snapshot: std.Io.Writer.Allocating = .init(testing.allocator);
defer snapshot.deinit();
var stream: record.Writer = .init(
testing.allocator,
&snapshot.writer,
);
defer stream.deinit();
try stream.writer().writeAll("prefix");
// READY covers only the bytes that precede its own record.
const ready_offset = snapshot.written().len;
const ready_digest = stream.prefixDigest();
try encode(.ready, &stream);
// History follows READY and is included by FINISH.
try stream.writer().writeAll("history");
const finish_offset = snapshot.written().len;
const finish_digest = stream.prefixDigest();
try encode(.finish, &stream);
// The running stream reaches both checkpoints without rehashing its prefix.
var source: std.Io.Reader = .fixed(snapshot.written());
var source_stream: record.StreamReader = .init(&source);
try source_stream.reader().discardAll(ready_offset);
try testing.expectEqual(ready_digest, source_stream.prefixDigest());
try decode(.ready, &source_stream);
try source_stream.reader().discardAll("history".len);
try testing.expectEqual(finish_digest, source_stream.prefixDigest());
try decode(.finish, &source_stream);
try testing.expectEqual(
snapshot.written().len - finish_offset,
record.Header.len + @sizeOf(Digest),
);
}
test "checkpoint rejects wrong tags and digests" {
const testing = std.testing;
var snapshot: std.Io.Writer.Allocating = .init(testing.allocator);
defer snapshot.deinit();
var stream: record.Writer = .init(
testing.allocator,
&snapshot.writer,
);
defer stream.deinit();
try stream.writer().writeAll("prefix");
const checkpoint_offset = snapshot.written().len;
try encode(.ready, &stream);
var wrong_tag_source: std.Io.Reader = .fixed(
snapshot.written()[checkpoint_offset..],
);
var wrong_tag: record.StreamReader = .init(&wrong_tag_source);
try testing.expectError(
error.UnexpectedRecordTag,
decode(.finish, &wrong_tag),
);
// Build a correctly framed checkpoint containing an unrelated digest.
var invalid: std.Io.Writer.Allocating = .init(testing.allocator);
defer invalid.deinit();
var invalid_stream: record.Writer = .init(
testing.allocator,
&invalid.writer,
);
defer invalid_stream.deinit();
try invalid_stream.writer().writeAll("prefix");
var invalid_digest = invalid_stream.prefixDigest();
invalid_digest[0] ^= 1;
const invalid_payload = invalid_stream.begin(.ready);
errdefer invalid_stream.cancel();
try invalid_payload.writeAll(&invalid_digest);
try invalid_stream.finish();
var invalid_digest_source: std.Io.Reader = .fixed(invalid.written());
var invalid_digest_stream: record.StreamReader = .init(
&invalid_digest_source,
);
try invalid_digest_stream.reader().discardAll("prefix".len);
try testing.expectError(
error.InvalidDigest,
decode(.ready, &invalid_digest_stream),
);
}
test "FINISH leaves continuation bytes unread" {
const testing = std.testing;
var finished: std.Io.Writer.Allocating = .init(testing.allocator);
defer finished.deinit();
var finished_stream: record.Writer = .init(
testing.allocator,
&finished.writer,
);
defer finished_stream.deinit();
try finished_stream.writer().writeAll("prefix");
try encode(.finish, &finished_stream);
try finished.writer.writeByte(0);
var trailing_source: std.Io.Reader = .fixed(finished.written());
var trailing_stream: record.StreamReader = .init(
&trailing_source,
);
try trailing_stream.reader().discardAll("prefix".len);
try decode(.finish, &trailing_stream);
try testing.expectEqual(@as(u8, 0), try trailing_source.takeByte());
}

104
src/terminal/snapshot/envelope.zig Normal file
View File

@@ -0,0 +1,104 @@
//! Snapshot envelope.
//!
//! Every snapshot binary blob begins with exactly one envelope at byte zero.
//! It is followed by a set of records. Each record provides their own
//! tag, payload length, CRC, etc.
//!
//! The envelope identifies the bytes as a terminal snapshot and selects the
//! single version governing the entire blob. A decoder validates both fields
//! before reading any records.
//!
//! The envelope is exactly ten bytes. All integers are unsigned and
//! little-endian.
//!
//! | Offset | Size | Field |
//! | -----: | ---: | :------------------- |
//! | 0 | 8 | Magic (`GHOSTSNP`) |
//! | 8 | 2 | Version (`u16`) |
const std = @import("std");
const test_fixture = @import("fixture.zig");
const io = @import("io.zig");
/// Identifies a Ghostty terminal snapshot and rejects unrelated input before
/// any record decoding begins. All eight bytes are part of the wire value.
pub const magic = "GHOSTSNP";
/// The complete compatibility boundary for snapshot layout and behavior.
/// Version 1 readers require this value to match exactly.
pub const version: u16 = 1;
/// Number of bytes in the fixed envelope: magic followed by version.
pub const encoded_len = computeLen();
comptime {
// We expect this so if it changes we should think carefully.
std.debug.assert(encoded_len == 10);
}
pub const DecodeError = std.Io.Reader.Error || error{
InvalidMagic,
UnsupportedVersion,
};
/// Encode the envelope.
pub fn encode(writer: *std.Io.Writer) std.Io.Writer.Error!void {
try writer.writeAll(magic);
try io.writeInt(writer, u16, version);
}
/// Decode and validate the envelope.
pub fn decode(reader: *std.Io.Reader) DecodeError!void {
var actual_magic: [magic.len]u8 = undefined;
try reader.readSliceAll(&actual_magic);
if (!std.mem.eql(u8, magic, &actual_magic)) return error.InvalidMagic;
const actual_version = try io.readInt(reader, u16);
if (actual_version != version) return error.UnsupportedVersion;
}
fn computeLen() usize {
comptime {
var buf: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
encode(&writer) catch unreachable;
return writer.end;
}
}
const test_golden_fixture = test_fixture.parse(@embedFile("testdata/envelope-v1.hex"));
test "golden encoding" {
var buf: [encoded_len]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
try encode(&writer);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/envelope-v1.hex",
"snapshot_fixture-envelope-v1.hex",
&test_golden_fixture,
writer.buffered(),
);
var reader: std.Io.Reader = .fixed(&test_golden_fixture);
try decode(&reader);
}
test "reject invalid magic and version" {
var invalid_magic: std.Io.Reader = .fixed("GHOSTSNX\x01\x00");
try std.testing.expectError(error.InvalidMagic, decode(&invalid_magic));
var invalid_version: std.Io.Reader = .fixed("GHOSTSNP\x00\x00");
try std.testing.expectError(
error.UnsupportedVersion,
decode(&invalid_version),
);
}
test "reject every truncation" {
for (0..encoded_len) |len| {
var reader: std.Io.Reader = .fixed(test_golden_fixture[0..len]);
try std.testing.expectError(error.EndOfStream, decode(&reader));
}
}

479
src/terminal/snapshot/fixture.zig Normal file
View File

@@ -0,0 +1,479 @@
//! Test-only support for reviewable snapshot golden fixtures.
//!
//! Snapshot compatibility references are stored as annotated hexadecimal text
//! under `testdata/`. Each byte is written as exactly two hexadecimal digits.
//! Whitespace is insignificant, and `#` begins a comment through the end of the
//! line. This permits field, record, and absolute-offset annotations without
//! changing the bytes embedded by a test.
//!
//! `parse` validates and decodes an `@embedFile` at comptime. Tests should
//! encode their native value, compare the generated bytes with that embedded
//! reference using `expectEqual`, and then decode the reference itself. This
//! keeps encoding regression coverage separate from backwards-decoding
//! coverage.
//!
//! `expectEqual` formats a fresh candidate on every run. Matching candidates
//! remain in the test's temporary directory. A mismatch copies the candidate
//! to the repository root, reports its path and first differing byte, and
//! fails without modifying the checked-in reference. After reviewing an
//! intentional wire change, a maintainer manually replaces the reference.
//! Fixture filenames include the snapshot version so a future format adds new
//! references instead of overwriting compatibility data for an older version.
//!
//! A typical fixed-value test looks like this:
//!
//! ```zig
//! const fixture = @import("fixture.zig");
//!
//! const reference = fixture.parse(
//! @embedFile("testdata/example-v1.hex"),
//! );
//!
//! test "example golden encoding and decoding" {
//! var encoded: [Example.len]u8 = undefined;
//! var writer: std.Io.Writer = .fixed(&encoded);
//! try Example.encode(value, &writer);
//!
//! try fixture.expectEqual(
//! .bytes,
//! "src/terminal/snapshot/testdata/example-v1.hex",
//! "snapshot_fixture-example-v1.hex",
//! &reference,
//! writer.buffered(),
//! );
//!
//! var reader: std.Io.Reader = .fixed(&reference);
//! try std.testing.expectEqualDeep(
//! value,
//! try Example.decode(&reader),
//! );
//! }
//! ```
//!
//! ## Kaitai
//!
//! Every fixture has three machine-readable comment fields:
//!
//! ```text
//! # Kaitai type: page_payload
//! # Kaitai params:
//! # Kaitai offset: 0
//! ```
//!
//! These select a type from `snapshot.ksy`, its whitespace-separated
//! parameters, and the byte offset where parsing begins. The Kaitai verifier
//! discovers fixtures from the filesystem and uses this metadata instead of
//! maintaining a parallel filename table. Generated candidates copy the
//! metadata from their checked-in reference.
//!
const std = @import("std");
const envelope = @import("envelope.zig");
const record = @import("record.zig");
const log = std.log.scoped(.snapshot_fixture);
pub const Kind = enum {
bytes,
page,
snapshot,
};
/// Decode an annotated hexadecimal fixture at comptime.
///
/// Whitespace is ignored. `#` begins a comment that continues through the end
/// of the line. Every other token must be exactly two hexadecimal digits.
pub fn parse(comptime source: []const u8) [decodedLen(source)]u8 {
comptime {
// Large complete-snapshot fixtures exceed Zig's default comptime
// branch budget while scanning their comments and byte tokens.
@setEvalBranchQuota(100_000_000);
// The first pass in decodedLen gives this result its exact array type.
// Keeping it an array lets callers use the fixture in comptime slices.
var result: [decodedLen(source)]u8 = undefined;
var source_index: usize = 0;
var result_index: usize = 0;
while (source_index < source.len) {
// Comments and layout are purely for reviewers and contribute no
// bytes to the embedded reference.
skipIgnored(source, &source_index);
if (source_index == source.len) break;
// Every data token is one complete byte. Parse it directly rather
// than accepting variable-width integers or other hex syntax.
if (source_index + 1 >= source.len) {
@compileError("snapshot fixture ends with one hex digit");
}
result[result_index] =
hexNibble(source[source_index]) << 4 |
hexNibble(source[source_index + 1]);
result_index += 1;
source_index += 2;
// Requiring a separator catches accidental third digits and makes
// the accepted grammar match the formatter below.
if (source_index < source.len and
!std.ascii.isWhitespace(source[source_index]) and
source[source_index] != '#')
{
@compileError(
"snapshot fixture hex bytes must be separated by whitespace",
);
}
}
return result;
}
}
/// Compare generated bytes with a checked-in reference.
///
/// A formatted candidate is always created in a temporary directory. On a
/// mismatch it is copied into the repository root before the test fails.
pub fn expectEqual(
kind: Kind,
reference_path: []const u8,
candidate_name: []const u8,
expected: []const u8,
actual: []const u8,
) !void {
const testing = std.testing;
const alloc = testing.allocator;
var tmp = testing.tmpDir(.{});
defer tmp.cleanup();
// Kaitai parser selection lives in the reference itself. Read the textual
// source so generated candidates preserve that metadata even though the
// byte comparison below uses the comptime-decoded reference.
const reference_source = try std.Io.Dir.cwd().readFileAlloc(
testing.io,
reference_path,
alloc,
.unlimited,
);
defer alloc.free(reference_source);
// Always materialize a candidate, even when it will match. This keeps the
// generation path exercised on every test run while avoiding repository
// artifacts for the normal successful case.
{
var candidate_file = try tmp.dir.createFile(
testing.io,
candidate_name,
.{},
);
defer candidate_file.close(testing.io);
var write_buffer: [4096]u8 = undefined;
var candidate_writer = candidate_file.writer(
testing.io,
&write_buffer,
);
try format(
kind,
reference_source,
actual,
&candidate_writer.interface,
);
try candidate_writer.interface.flush();
}
// Matching candidates remain in the temporary directory and disappear
// with its cleanup.
if (std.mem.eql(u8, expected, actual)) return;
// The test temp directory is not necessarily below the working directory.
// Resolve both paths before copying the candidate to the stable,
// user-visible repository-root name.
const tmp_path = try tmp.dir.realPathFileAlloc(
testing.io,
candidate_name,
alloc,
);
defer alloc.free(tmp_path);
const cwd = try std.Io.Dir.cwd().realPathFileAlloc(
testing.io,
".",
alloc,
);
defer alloc.free(cwd);
const candidate_path = try std.fs.path.join(
alloc,
&.{ cwd, candidate_name },
);
defer alloc.free(candidate_path);
try std.Io.Dir.copyFileAbsolute(
tmp_path,
candidate_path,
testing.io,
.{},
);
// Report the first useful comparison point. When one byte sequence is a
// prefix of the other, this is the shared length where they diverge.
const difference = difference: {
const shared_len = @min(expected.len, actual.len);
for (
expected[0..shared_len],
actual[0..shared_len],
0..,
) |expected_byte, actual_byte, index| {
if (expected_byte != actual_byte) break :difference index;
}
break :difference shared_len;
};
log.err(
"snapshot fixture differs reference={s} candidate={s} " ++
"offset=0x{x} expected_len={} actual_len={}",
.{
reference_path,
candidate_path,
difference,
expected.len,
actual.len,
},
);
return error.TestExpectedEqual;
}
fn decodedLen(comptime source: []const u8) usize {
comptime {
// This is a complete validation pass, not just a count. Invalid syntax
// therefore fails before parse allocates and fills its result array.
@setEvalBranchQuota(100_000_000);
var index: usize = 0;
var result: usize = 0;
while (index < source.len) {
skipIgnored(source, &index);
if (index == source.len) break;
if (index + 1 >= source.len) {
@compileError("snapshot fixture ends with one hex digit");
}
_ = hexNibble(source[index]);
_ = hexNibble(source[index + 1]);
result += 1;
index += 2;
if (index < source.len and
!std.ascii.isWhitespace(source[index]) and
source[index] != '#')
{
@compileError(
"snapshot fixture hex bytes must be separated by whitespace",
);
}
}
return result;
}
}
fn skipIgnored(comptime source: []const u8, index: *usize) void {
while (index.* < source.len) {
// Layout whitespace is unrestricted so fixtures can group fields and
// keep deterministic sixteen-byte output lines.
if (std.ascii.isWhitespace(source[index.*])) {
index.* += 1;
continue;
}
// A comment consumes the rest of its line. The newline itself is
// consumed by the whitespace case on the next iteration.
if (source[index.*] != '#') return;
while (index.* < source.len and source[index.*] != '\n') {
index.* += 1;
}
}
}
fn hexNibble(comptime value: u8) u8 {
return switch (value) {
'0'...'9' => value - '0',
'a'...'f' => value - 'a' + 10,
'A'...'F' => value - 'A' + 10,
else => @compileError("snapshot fixture contains a non-hex digit"),
};
}
fn format(
kind: Kind,
reference_source: []const u8,
bytes: []const u8,
writer: *std.Io.Writer,
) (std.Io.Writer.Error || error{InvalidKaitaiMetadata})!void {
// Every generated candidate starts with enough maintenance context to be
// understandable when copied out of a failing test's temp directory.
try writer.writeAll("# Ghostty snapshot fixture\n");
try writeKaitaiMetadata(reference_source, writer);
try writer.print(
"# Wire version: {}\n" ++
"# Generated by its snapshot test; review before replacing.\n" ++
"# On mismatch, the candidate is copied to the repository root.\n",
.{envelope.version},
);
// Small values only need a hex dump. Larger compound formats get
// structure-aware section labels without changing their decoded bytes.
switch (kind) {
.bytes => try writeSection(
"encoded bytes",
bytes,
0,
bytes.len,
writer,
),
.page => try formatPage(bytes, writer),
.snapshot => try formatSnapshot(bytes, writer),
}
}
fn writeKaitaiMetadata(
reference_source: []const u8,
writer: *std.Io.Writer,
) (std.Io.Writer.Error || error{InvalidKaitaiMetadata})!void {
const prefixes = [_][]const u8{
"# Kaitai type:",
"# Kaitai params:",
"# Kaitai offset:",
};
var found = [_]bool{false} ** prefixes.len;
var lines = std.mem.splitScalar(u8, reference_source, '\n');
while (lines.next()) |untrimmed| {
const line = std.mem.trim(u8, untrimmed, " \r");
for (prefixes, &found) |prefix, *seen| {
if (!std.mem.startsWith(u8, line, prefix)) continue;
if (seen.*) return error.InvalidKaitaiMetadata;
seen.* = true;
try writer.print("{s}\n", .{line});
break;
}
}
for (found) |seen| {
if (!seen) return error.InvalidKaitaiMetadata;
}
}
fn formatPage(
bytes: []const u8,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
// PAGE version 1 has a twenty-byte fixed header. The remainder is
// self-describing through the counts and dimensions in that header.
const header_end = @min(bytes.len, 20);
try writeSection("PAGE header", bytes, 0, header_end, writer);
if (header_end < bytes.len) {
try writeSection(
"style and hyperlink tables, rows, and cells",
bytes,
header_end,
bytes.len,
writer,
);
}
}
fn formatSnapshot(
bytes: []const u8,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
// A complete snapshot begins with its unframed envelope. Clamp this first
// section so a partially generated candidate can still be inspected.
const envelope_end = @min(bytes.len, envelope.encoded_len);
try writeSection("snapshot envelope", bytes, 0, envelope_end, writer);
// Walk record framing only far enough to annotate boundaries. The emitted
// hex remains authoritative; this formatter never repairs candidate data.
var offset: usize = envelope_end;
while (offset < bytes.len) {
// A partial header cannot provide a safe payload boundary, so preserve
// the remaining bytes as one explicitly incomplete section.
if (bytes.len - offset < record.Header.len) {
try writeSection(
"trailing partial record",
bytes,
offset,
bytes.len,
writer,
);
return;
}
// Tag and payload length are the first six bytes of every record
// header. CRC validation belongs to the codec test, not this formatter.
const tag_raw = std.mem.readInt(
u16,
bytes[offset..][0..2],
.little,
);
const payload_len: u32 = std.mem.readInt(
u32,
bytes[offset + 2 ..][0..4],
.little,
);
// Clamp the section to the available candidate bytes. This makes a
// truncated payload reviewable instead of indexing beyond the slice.
const record_end: usize = @min(
bytes.len,
offset + record.Header.len + @as(usize, payload_len),
);
// Known tags make the annotated diff readable. Preserve unknown tags
// numerically so formatter output is still useful for encoder bugs.
if (std.enums.fromInt(record.Tag, tag_raw)) |tag| {
try writer.print(
"\n# offset 0x{x:0>8}: {s} record, payload {} bytes\n",
.{ offset, @tagName(tag), payload_len },
);
} else {
try writer.print(
"\n# offset 0x{x:0>8}: unknown record {}, payload {} bytes\n",
.{ offset, tag_raw, payload_len },
);
}
try writeHex(bytes, offset, record_end, writer);
offset = record_end;
}
}
fn writeSection(
name: []const u8,
bytes: []const u8,
start: usize,
end: usize,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
// Sections begin on a fresh line and carry their absolute wire offset.
try writer.print(
"\n# offset 0x{x:0>8}: {s}\n",
.{ start, name },
);
try writeHex(bytes, start, end, writer);
}
fn writeHex(
bytes: []const u8,
start: usize,
end: usize,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
const digits = "0123456789abcdef";
var offset = start;
while (offset < end) {
// Lowercase, sixteen-byte lines provide stable textual diffs. The
// trailing offset is a comment and is ignored by parse.
const line_end = @min(end, offset + 16);
for (bytes[offset..line_end], 0..) |byte, index| {
if (index != 0) try writer.writeByte(' ');
try writer.writeByte(digits[byte >> 4]);
try writer.writeByte(digits[byte & 0x0f]);
}
try writer.print(" # 0x{x:0>8}\n", .{offset});
offset = line_end;
}
}

804
src/terminal/snapshot/grid.zig Normal file
View File

@@ -0,0 +1,804 @@
//! Grid (rows and cells) encoding.
//!
//! A grid contains the rows and cells of a terminal page. Its dimensions are
//! supplied by the containing record rather than repeated here. The encoder
//! writes exactly `rows` row records, and every row contains exactly `columns`
//! cell records.
//!
//! All records are tightly packed with no padding between them. All integers
//! are unsigned and little-endian.
//!
//! ## Row
//!
//! Each row has the following format:
//!
//! | Offset | Size | Field |
//! | -----: | -------: | :-------------------------- |
//! | 0 | 1 | Row flags |
//! | 1 | variable | Exactly `columns` cells |
//!
//! Cells are encoded consecutively. Since cells may contain a variable number
//! of grapheme codepoints, the next row begins immediately after the final
//! cell and its grapheme codepoints.
//!
//! The row flag byte has the following format:
//!
//! | Bits | Field |
//! | ---: | :------------------------ |
//! | 0 | Wrap |
//! | 1 | Wrap continuation |
//! | 2-3 | Semantic prompt |
//! | 4-7 | Reserved, zero |
//!
//! Semantic prompt values are:
//!
//! | Value | Meaning |
//! | ----: | :------------------ |
//! | 0 | None |
//! | 1 | Prompt |
//! | 2 | Prompt continuation |
//!
//! Value 3 is not emitted in snapshot version 1. Decoders treat it as none.
//!
//! Native row cache flags are not encoded. In particular, the Kitty virtual
//! placeholder hint is derived while decoding cells containing U+10EEEE.
//!
//! ## Cell
//!
//! Each cell has the following format:
//!
//! | Offset | Size | Field |
//! | -----: | --------: | :---------------------------- |
//! | 0 | 1 | Content kind |
//! | 1 | 1 | Width kind |
//! | 2 | 1 | Protected and semantic flags |
//! | 3 | 1 | Reserved, zero |
//! | 4 | 2 | Style ID |
//! | 6 | 2 | Hyperlink ID |
//! | 8 | 4 | Codepoint or packed color |
//! | 12 | 4 | Grapheme suffix count |
//! | 16 | 4 * count | Grapheme suffix codepoints |
//!
//! Content kinds are:
//!
//! | Value | Meaning |
//! | ----: | :----------------- |
//! | 0 | Codepoint |
//! | 1 | Palette background |
//! | 2 | RGB background |
//!
//! For codepoint content, the value is a Unicode scalar encoded as a `u32`.
//! For a palette background, the low byte is the palette index and the other
//! three bytes are zero. For an RGB background, the low three bytes are red,
//! green, and blue, and the high byte is zero.
//!
//! Width kinds are:
//!
//! | Value | Meaning |
//! | ----: | :---------- |
//! | 0 | Narrow |
//! | 1 | Wide |
//! | 2 | Spacer tail |
//! | 3 | Spacer head |
//!
//! The cell flag byte has the following format:
//!
//! | Bits | Field |
//! | ---: | :---------------- |
//! | 0 | Protected |
//! | 1-2 | Semantic content |
//! | 3-7 | Reserved, zero |
//!
//! Semantic content values are:
//!
//! | Value | Meaning |
//! | ----: | :------ |
//! | 0 | Output |
//! | 1 | Input |
//! | 2 | Prompt |
//!
//! Value 3 is not emitted in snapshot version 1. Decoders treat it as output.
//!
//! Style and hyperlink ID zero mean no style and no hyperlink. Other IDs
//! refer to entries in the containing record's separate style and hyperlink
//! tables.
//!
//! Grapheme suffixes are valid only for codepoint content. Each suffix is a
//! Unicode scalar encoded as a `u32`. A nonzero suffix count requires a
//! nonzero base codepoint.
const std = @import("std");
const test_fixture = @import("fixture.zig");
const io = @import("io.zig");
const kitty = @import("../kitty.zig");
const terminal_hyperlink = @import("../hyperlink.zig");
const terminal_page = @import("../page.zig");
const terminal_style = @import("../style.zig");
const TerminalCell = terminal_page.Cell;
const TerminalHyperlinkId = terminal_hyperlink.Id;
const TerminalPage = terminal_page.Page;
const TerminalRow = terminal_page.Row;
const TerminalStyleId = terminal_style.Id;
/// Maps encoded style table IDs to IDs assigned by the destination page.
///
/// Build this by inserting each decoded style into the page, then recording
/// the encoded ID and the ID returned by the page's style set. Style ID zero is
/// implicit and does not need an entry.
pub const StyleRemap = std.AutoHashMap(TerminalStyleId, TerminalStyleId);
/// Maps encoded hyperlink table IDs to IDs assigned by the destination page.
///
/// Build this by inserting each decoded hyperlink into the page, then
/// recording the encoded ID and the ID returned by the page's hyperlink set.
/// Hyperlink ID zero is implicit and does not need an entry.
pub const HyperlinkRemap = std.AutoHashMap(TerminalHyperlinkId, TerminalHyperlinkId);
pub const EncodeError = std.Io.Writer.Error || error{
/// Wide and spacer cells do not form a valid row.
InvalidWideCell,
};
/// Encode every row and cell directly from a page.
///
/// If an error is returned, partial data may have been written. If you
/// want transactional writing, the caller is responsible for using something
/// like a seekable stream and rolling back.
pub fn encode(
page: *const TerminalPage,
writer: *std.Io.Writer,
) EncodeError!void {
defer page.assertIntegrity();
for (0..page.size.rows) |y| {
// Row header
const row = page.getRow(y);
const row_header: RowHeader = .{
.wrap = row.wrap,
.wrap_continuation = row.wrap_continuation,
.semantic_prompt = row.semantic_prompt,
};
try writer.writeByte(@bitCast(row_header));
// Cells
const cells = page.getCells(row);
for (cells, 0..) |*cell, x| {
// Validate the wide state of this cell, we don't want
// to encode corrupt data.
switch (cell.wide) {
.narrow => {},
.wide => if (x + 1 == cells.len or
cells[x + 1].wide != .spacer_tail)
{
return error.InvalidWideCell;
},
.spacer_tail => if (x == 0 or
cells[x - 1].wide != .wide)
{
return error.InvalidWideCell;
},
.spacer_head => if (x + 1 != cells.len or !row.wrap) {
return error.InvalidWideCell;
},
}
const graphemes: []const u21 = if (cell.hasGrapheme())
page.lookupGrapheme(cell) orelse unreachable
else
&.{};
// The page has two codepoint tags depending on whether suffixes
// exist, but the wire represents both with one content kind.
const kind: CellHeader.Kind = switch (cell.content_tag) {
.codepoint, .codepoint_grapheme => .codepoint,
.bg_color_palette => .bg_color_palette,
.bg_color_rgb => .bg_color_rgb,
};
const value: CellHeader.Value = switch (kind) {
.codepoint => .{
.codepoint = cell.content.codepoint.data,
},
.bg_color_palette => .{ .bg_color_palette = .{
.index = cell.content.color_palette.data,
} },
.bg_color_rgb => .{ .bg_color_rgb = .{
.r = cell.content.color_rgb.r,
.g = cell.content.color_rgb.g,
.b = cell.content.color_rgb.b,
} },
};
const style_id = cell.style_id;
const hyperlink_id: TerminalHyperlinkId = if (cell.hyperlink)
page.lookupHyperlink(cell) orelse unreachable
else
0;
const header: CellHeader = .{
.content_kind = kind,
.width = cell.wide,
.protected = cell.protected,
.semantic_content = cell.semantic_content,
.style_id = style_id,
.hyperlink_id = hyperlink_id,
.value = value,
.grapheme_count = @intCast(graphemes.len),
};
try header.encode(writer);
for (graphemes) |suffix| try io.writeInt(
writer,
u32,
suffix,
);
}
}
}
/// Decode every row and cell directly into an initialized, empty page.
///
/// The grid does not encode dimensions, so `page` must already have the exact
/// row and column count expected by the containing record. This function reads
/// exactly `page.size.rows` rows with `page.size.cols` cells each. Capacity
/// hints are advisory. Graphemes and cell hyperlink references that do not fit
/// are discarded without affecting the rest of the grid.
///
/// Style and hyperlink table entries must be inserted into `page` before
/// calling this function. As each table entry is inserted, the caller records
/// its encoded ID and page-assigned ID in `style_remap` or `hyperlink_remap`.
/// ID zero always means the default style or no hyperlink. A nonzero ID missing
/// from its remap is also treated as zero so unknown table references do not
/// prevent the rest of the grid from decoding.
///
/// Invalid semantic data is normalized into a degraded form while preserving
/// the declared byte boundaries. Unknown semantic values use their neutral
/// variants, invalid Unicode becomes U+FFFD, invalid optional data is ignored,
/// and malformed wide-cell relationships become narrow cells.
pub fn decode(
page: *TerminalPage,
reader: *std.Io.Reader,
style_remap: *const StyleRemap,
hyperlink_remap: *const HyperlinkRemap,
) std.Io.Reader.Error!void {
for (0..page.size.rows) |y| {
const row_raw = try reader.takeByte();
const row_header: RowHeader = @bitCast(row_raw);
const semantic_prompt_raw: u2 = @truncate(row_raw >> @bitOffsetOf(RowHeader, "semantic_prompt"));
// Reserved bits do not change the known fields. The semantic enum is
// the only non-exhaustive field, so give its unknown value a default.
const row = page.getRow(y);
row.wrap = row_header.wrap;
row.wrap_continuation = row_header.wrap_continuation;
row.semantic_prompt = std.enums.fromInt(
TerminalRow.SemanticPrompt,
semantic_prompt_raw,
) orelse .none;
const cells = page.getCells(row);
for (cells, 0..) |*cell, x| {
const decoded_header = try CellHeader.decode(reader);
cell.* = .init(0);
var accept_graphemes = false;
const grapheme_count: u32 = switch (decoded_header) {
// The fixed header was fully consumed, but its content kind
// cannot be interpreted. Keep the cell empty and consume only
// the suffix bytes needed to reach the next cell.
.invalid => |count| count,
.valid => |header| valid: {
// IDs belong to the encoded page. Translate them to IDs
// assigned by the destination page before storing them on
// cells.
const encoded_style_id = header.style_id;
const style_id = if (encoded_style_id == 0)
0
else
style_remap.get(encoded_style_id) orelse 0;
const encoded_hyperlink_id = header.hyperlink_id;
const hyperlink_id = if (encoded_hyperlink_id == 0)
0
else
hyperlink_remap.get(encoded_hyperlink_id) orelse 0;
switch (header.content_kind) {
.codepoint => {
var cp = std.math.cast(
u21,
header.value.codepoint,
) orelse 0xFFFD;
if (cp > 0x10FFFF or
(cp >= 0xD800 and cp <= 0xDFFF))
{
cp = 0xFFFD;
}
cell.content = .{ .codepoint = .{ .data = cp } };
accept_graphemes = cp != 0;
// Kitty image and placement state is not part of this
// snapshot version, but the placeholder is still a
// valid Unicode scalar. Preserve it and derive the
// native row hint so later row operations remain
// correct.
if (cp == kitty.graphics.unicode.placeholder) {
row.kitty_virtual_placeholder = true;
}
},
.bg_color_palette => {
const palette = header.value.bg_color_palette;
cell.content_tag = .bg_color_palette;
cell.content = .{
.color_palette = .{ .data = palette.index },
};
},
.bg_color_rgb => {
const rgb = header.value.bg_color_rgb;
cell.content_tag = .bg_color_rgb;
cell.content = .{ .color_rgb = .{
.r = rgb.r,
.g = rgb.g,
.b = rgb.b,
} };
},
}
cell.wide = header.width;
cell.protected = header.protected;
cell.semantic_content = header.semantic_content;
if (style_id != 0) {
// The table owns one reference and each decoded cell owns
// one additional reference.
page.styles.use(page.memory, style_id);
cell.style_id = style_id;
row.styled = true;
}
if (hyperlink_id != 0) {
// setHyperlink records the cell mapping but intentionally
// does not increment the set's reference count. If its map
// is full, undo our reference and leave this cell unlinked.
page.hyperlink_set.use(page.memory, hyperlink_id);
page.setHyperlink(row, cell, hyperlink_id) catch {
page.hyperlink_set.release(page.memory, hyperlink_id);
};
}
break :valid header.grapheme_count;
},
};
// Always consume every declared suffix. Invalid scalars, suffixes
// on non-text cells, and suffixes that exceed the native capacity
// are optional detail and can be dropped independently.
for (0..grapheme_count) |_| {
const suffix_raw = try io.readInt(reader, u32);
if (!accept_graphemes) continue;
const suffix = std.math.cast(u21, suffix_raw) orelse continue;
if (suffix > 0x10FFFF or
(suffix >= 0xD800 and suffix <= 0xDFFF))
{
continue;
}
page.appendGrapheme(row, cell, suffix) catch {
accept_graphemes = false;
};
}
// A following cell resolves whether the previous wide marker owns
// a tail. Normalize the current marker immediately when possible,
// including a wide marker at the row end.
if (x > 0 and
cells[x - 1].wide == .wide and
cell.wide != .spacer_tail)
{
cells[x - 1].wide = .narrow;
}
switch (cell.wide) {
.narrow => {},
// A non-final wide marker remains pending until the next cell.
.wide => if (x + 1 == cells.len) {
cell.wide = .narrow;
},
.spacer_tail => if (x == 0 or
cells[x - 1].wide != .wide)
{
cell.wide = .narrow;
},
.spacer_head => if (x + 1 != cells.len or !row.wrap) {
cell.wide = .narrow;
},
}
}
}
}
/// The header before every row.
const RowHeader = packed struct(u8) {
wrap: bool = false,
wrap_continuation: bool = false,
semantic_prompt: TerminalRow.SemanticPrompt = .none,
_padding: u4 = 0,
};
/// The fixed fields that precede a cell's grapheme suffix codepoints.
const CellHeader = struct {
/// Number of bytes written by `encode`, calculated using the encoder itself
/// so this remains synchronized with the field-by-field wire format.
pub const len = computeLen();
comptime {
// This size is part of the wire format. If it changes, the snapshot
// version and golden fixtures must also change.
std.debug.assert(len == 16);
}
/// Interpretation of `value`.
content_kind: Kind = .codepoint,
/// Display width and spacer role of the cell.
width: TerminalCell.Wide = .narrow,
/// Whether selective erase operations protect the cell.
protected: bool = false,
/// Semantic role assigned by shell integration.
semantic_content: TerminalCell.SemanticContent = .output,
/// ID in the encoded page's style table, or zero for the default style.
style_id: TerminalStyleId = 0,
/// ID in the encoded page's hyperlink table, or zero for no hyperlink.
hyperlink_id: TerminalHyperlinkId = 0,
/// Codepoint or packed background color selected by `content_kind`.
value: Value = .{ .codepoint = 0 },
/// Number of grapheme suffix codepoints immediately following the header.
grapheme_count: u32 = 0,
/// Encode the fixed cell header.
pub fn encode(
self: CellHeader,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
const flags: Flags = .{
.protected = self.protected,
.semantic_content = self.semantic_content,
};
// 0 1 2 3 4 6 8 12 16
// +-------+-------+-------+-------+--------+--------+--------+--------+
// | kind | width | flags | zero | style | link | value | count |
// | u8 | u8 | u8 | u8 | u16 LE | u16 LE | u32 LE | u32 LE |
// +-------+-------+-------+-------+--------+--------+--------+--------+
try writer.writeByte(@intFromEnum(self.content_kind));
try writer.writeByte(@intFromEnum(self.width));
try writer.writeByte(@bitCast(flags));
try writer.writeByte(0);
try io.writeInt(writer, TerminalStyleId, self.style_id);
try io.writeInt(writer, TerminalHyperlinkId, self.hyperlink_id);
try io.writeInt(writer, u32, @bitCast(self.value));
try io.writeInt(writer, u32, self.grapheme_count);
}
/// A valid header, or the suffix count from an uninterpretable header.
const DecodeResult = union(enum) {
valid: CellHeader,
invalid: u32,
};
/// Decode a fixed cell header, normalizing unknown semantic values.
pub fn decode(reader: *std.Io.Reader) std.Io.Reader.Error!DecodeResult {
// Decode various fields. We need to be very defensive here
// because it can come from an untrusted source and may be invalid.
const content_kind_optional = kind: {
const raw = try reader.takeByte();
break :kind std.enums.fromInt(Kind, raw);
};
const width = width: {
const raw = try reader.takeByte();
break :width std.enums.fromInt(TerminalCell.Wide, raw) orelse .narrow;
};
const flags: Flags = @bitCast(try reader.takeByte());
// We can't trust `flags` to have a valid semantic content so
// we need extract the bits and do a safe conversion.
const semantic_content = semantic: {
const flags_raw: u8 = @bitCast(flags);
const raw: u2 = @truncate(flags_raw >> @bitOffsetOf(Flags, "semantic_content"));
break :semantic std.enums.fromInt(
TerminalCell.SemanticContent,
raw,
) orelse .output;
};
// This byte is reserved so the IDs and content value remain naturally
// aligned within the fixed cell header. Ignore it so future versions
// can give it meaning without making old decoders reject the cell.
_ = try reader.takeByte();
const style_id = try io.readInt(reader, TerminalStyleId);
const hyperlink_id = try io.readInt(reader, TerminalHyperlinkId);
const value: Value = @bitCast(try io.readInt(reader, u32));
const grapheme_count = try io.readInt(reader, u32);
// If we don't have a valid content kind, we return invalid and
// note the suffix bytes so that the reader can discard
const content_kind = content_kind_optional orelse
return .{ .invalid = grapheme_count };
return .{ .valid = .{
.content_kind = content_kind,
.width = width,
.protected = flags.protected,
.semantic_content = semantic_content,
.style_id = style_id,
.hyperlink_id = hyperlink_id,
.value = value,
.grapheme_count = grapheme_count,
} };
}
/// Computes the fixed header size using the encoder itself.
fn computeLen() usize {
comptime {
var buf: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
CellHeader.encode(.{}, &writer) catch unreachable;
return writer.end;
}
}
/// Determines how `value` is interpreted.
pub const Kind = enum(u8) {
codepoint = 0,
bg_color_palette = 1,
bg_color_rgb = 2,
};
/// The alternate interpretations of the four-byte content field.
pub const Value = packed union(u32) {
codepoint: u32,
bg_color_palette: packed struct(u32) {
index: u8,
_padding: u24 = 0,
},
bg_color_rgb: packed struct(u32) {
r: u8,
g: u8,
b: u8,
_padding: u8 = 0,
},
};
const Flags = packed struct(u8) {
protected: bool = false,
semantic_content: TerminalCell.SemanticContent = .output,
_padding: u5 = 0,
};
};
const test_golden_fixture = test_fixture.parse(
@embedFile("testdata/grid-v1.hex"),
);
test "grid golden encoding and decoding" {
const capacity: terminal_page.Capacity = .{
.cols = 3,
.rows = 2,
.styles = 0,
.hyperlink_bytes = 0,
.grapheme_bytes = 64,
.string_bytes = 0,
};
var source = try TerminalPage.init(capacity);
defer source.deinit();
// The first row covers semantic metadata, a wide-cell pair, and a
// multi-codepoint grapheme.
const wide = source.getRowAndCell(0, 0);
wide.row.semantic_prompt = .prompt;
wide.cell.* = .init('A');
wide.cell.wide = .wide;
wide.cell.protected = true;
wide.cell.semantic_content = .prompt;
const tail = source.getRowAndCell(1, 0);
tail.cell.wide = .spacer_tail;
tail.cell.semantic_content = .input;
const grapheme = source.getRowAndCell(2, 0);
grapheme.cell.* = .init('x');
try source.setGraphemes(
grapheme.row,
grapheme.cell,
&.{ 0x0301, 0x0302 },
);
// The second row covers both background-color kinds and a wrapped spacer
// head with the remaining row semantic value.
const palette = source.getRowAndCell(0, 1);
palette.cell.content_tag = .bg_color_palette;
palette.cell.content = .{ .color_palette = .{ .data = 7 } };
const rgb = source.getRowAndCell(1, 1);
rgb.cell.content_tag = .bg_color_rgb;
rgb.cell.content = .{ .color_rgb = .{
.r = 0xaa,
.g = 0xbb,
.b = 0xcc,
} };
rgb.cell.protected = true;
const head = source.getRowAndCell(2, 1);
head.cell.wide = .spacer_head;
head.row.wrap = true;
head.row.wrap_continuation = true;
head.row.semantic_prompt = .prompt_continuation;
var encoded: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&encoded);
try encode(&source, &writer);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/grid-v1.hex",
"snapshot_fixture-grid-v1.hex",
&test_golden_fixture,
writer.buffered(),
);
var destination = try TerminalPage.init(capacity);
defer destination.deinit();
var style_remap = StyleRemap.init(std.testing.allocator);
defer style_remap.deinit();
var hyperlink_remap = HyperlinkRemap.init(std.testing.allocator);
defer hyperlink_remap.deinit();
// Decode the checked-in reference through a one-byte reader buffer.
var fixture_reader: std.Io.Reader = .fixed(&test_golden_fixture);
var read_buffer: [1]u8 = undefined;
var limited = fixture_reader.limited(.unlimited, &read_buffer);
try decode(
&destination,
&limited.interface,
&style_remap,
&hyperlink_remap,
);
try destination.verifyIntegrity(std.testing.allocator);
const decoded_wide = destination.getRowAndCell(0, 0);
try std.testing.expectEqual(@as(u21, 'A'), decoded_wide.cell.codepoint());
try std.testing.expectEqual(TerminalCell.Wide.wide, decoded_wide.cell.wide);
try std.testing.expect(decoded_wide.cell.protected);
try std.testing.expectEqual(
TerminalCell.SemanticContent.prompt,
decoded_wide.cell.semantic_content,
);
try std.testing.expectEqual(
TerminalRow.SemanticPrompt.prompt,
decoded_wide.row.semantic_prompt,
);
const decoded_tail = destination.getRowAndCell(1, 0);
try std.testing.expectEqual(
TerminalCell.Wide.spacer_tail,
decoded_tail.cell.wide,
);
try std.testing.expectEqual(
TerminalCell.SemanticContent.input,
decoded_tail.cell.semantic_content,
);
const decoded_grapheme = destination.getRowAndCell(2, 0);
try std.testing.expectEqualSlices(
u21,
&.{ 0x0301, 0x0302 },
destination.lookupGrapheme(decoded_grapheme.cell).?,
);
const decoded_palette = destination.getRowAndCell(0, 1);
try std.testing.expectEqual(
TerminalCell.ContentTag.bg_color_palette,
decoded_palette.cell.content_tag,
);
try std.testing.expectEqual(
@as(u8, 7),
decoded_palette.cell.content.color_palette.data,
);
const decoded_rgb = destination.getRowAndCell(1, 1);
try std.testing.expectEqual(
TerminalCell.ContentTag.bg_color_rgb,
decoded_rgb.cell.content_tag,
);
try std.testing.expectEqual(
TerminalCell.RGB{ .r = 0xaa, .g = 0xbb, .b = 0xcc },
decoded_rgb.cell.content.color_rgb,
);
try std.testing.expect(decoded_rgb.cell.protected);
const decoded_head = destination.getRowAndCell(2, 1);
try std.testing.expectEqual(
TerminalCell.Wide.spacer_head,
decoded_head.cell.wide,
);
try std.testing.expect(decoded_head.row.wrap);
try std.testing.expect(decoded_head.row.wrap_continuation);
try std.testing.expectEqual(
TerminalRow.SemanticPrompt.prompt_continuation,
decoded_head.row.semantic_prompt,
);
// A re-encode proves the decoded native page retains every wire field.
var reencoded: [128]u8 = undefined;
var rewriter: std.Io.Writer = .fixed(&reencoded);
try encode(&destination, &rewriter);
try std.testing.expectEqualStrings(
&test_golden_fixture,
rewriter.buffered(),
);
}
test "grid normalizes incomplete wide cells" {
const testing = std.testing;
// One column puts the wide cell at the row end. Two columns put an
// ordinary narrow cell after it. Neither case supplies a spacer tail.
for ([_]u16{ 1, 2 }) |columns| {
const capacity: terminal_page.Capacity = .{
.cols = columns,
.rows = 1,
};
// Craft the malformed row directly on the wire. Decoding keeps its
// content but makes the incomplete wide cell narrow.
var payload: [64]u8 = undefined;
var payload_writer: std.Io.Writer = .fixed(&payload);
try payload_writer.writeByte(@bitCast(RowHeader{}));
try (CellHeader{
.width = .wide,
.value = .{ .codepoint = 'A' },
}).encode(&payload_writer);
if (capacity.cols > 1) try (CellHeader{
.value = .{ .codepoint = 'B' },
}).encode(&payload_writer);
var destination = try TerminalPage.init(capacity);
defer destination.deinit();
var style_remap = StyleRemap.init(testing.allocator);
defer style_remap.deinit();
var hyperlink_remap = HyperlinkRemap.init(testing.allocator);
defer hyperlink_remap.deinit();
var payload_reader: std.Io.Reader = .fixed(
payload_writer.buffered(),
);
try decode(
&destination,
&payload_reader,
&style_remap,
&hyperlink_remap,
);
try destination.verifyIntegrity(testing.allocator);
try testing.expectEqual(
@as(u21, 'A'),
destination.getRowAndCell(0, 0).cell.codepoint(),
);
try testing.expectEqual(
TerminalCell.Wide.narrow,
destination.getRowAndCell(0, 0).cell.wide,
);
if (columns > 1) {
const second = destination.getRowAndCell(1, 0).cell;
try testing.expectEqual(@as(u21, 'B'), second.codepoint());
try testing.expectEqual(TerminalCell.Wide.narrow, second.wide);
}
}
}

698
src/terminal/snapshot/history.zig Normal file
View File

@@ -0,0 +1,698 @@
//! HISTORY record payload encoding.
//!
//! One HISTORY record describes the complete history for a previously encoded
//! screen. It is followed immediately by the number of complete PAGE records
//! declared by `page_count`. Those pages contain the history older than the
//! first page in the corresponding SCREEN sequence and are ordered from newest
//! to oldest.
//!
//! Every encoded SCREEN has one corresponding HISTORY record. A screen with no
//! older pages uses a zero page count and has no following PAGE records.
//!
//! The first SCREEN page may begin above the active area. Those incidental
//! history rows are already present at READY and are not repeated after
//! HISTORY. The corresponding SCREEN header declares the complete logical
//! history extent, including both that resident overlap and the following PAGE
//! records.
//!
//! All integers are unsigned and little-endian.
//!
//! ## Binary Format
//!
//! A HISTORY record is followed immediately by its declared PAGE records:
//!
//! ```text
//! +----------------------+
//! | HISTORY record |
//! +----------------------+
//! | PAGE record 0 | newest
//! +----------------------+
//! | ... |
//! +----------------------+
//! | PAGE record (n - 1) | oldest
//! +----------------------+
//!
//! n = page_count
//! ```
//!
//! Newest-to-oldest order lets a reader start showing most-recent
//! history as soon as possible which is more useful to a user.
//!
//! The PAGE sequence may be empty when all history is already included in the
//! first SCREEN page or when the screen has no history.
//!
//! ### Header
//!
//! The HISTORY payload consists only of this fixed header:
//!
//! ```text
//! 0 +--------------------------------+
//! | Screen key (u16) |
//! 2 +--------------------------------+
//! | Following page count (u32) |
//! 6 +--------------------------------+
//! ```
//!
//! A decoder uses `page_count`, rather than another record tag, to find the end
//! of the page sequence.
const std = @import("std");
const Allocator = std.mem.Allocator;
const test_fixture = @import("fixture.zig");
const io = @import("io.zig");
const page = @import("page.zig");
const record = @import("record.zig");
const screen = @import("screen.zig");
const TerminalPage = @import("../page.zig").Page;
const TerminalPageList = @import("../PageList.zig");
const TerminalScreen = @import("../Screen.zig");
const TerminalScreenKey = @import("../ScreenSet.zig").Key;
/// The complete fixed payload of one HISTORY record.
pub const Header = struct {
/// Number of encoded bytes in the fixed payload, calculated by its encoder.
pub const len = computeLen();
comptime {
// This size is part of the wire format. If it changes, the snapshot
// version must also change.
std.debug.assert(len == 6);
}
/// Identifies the previously encoded screen that owns this history.
key: TerminalScreenKey,
/// Number of complete PAGE records immediately following this record.
page_count: u32,
/// Encode the fixed HISTORY payload.
pub fn encode(
self: Header,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
try io.writeInt(writer, u16, @intCast(@intFromEnum(self.key)));
try io.writeInt(writer, u32, self.page_count);
}
pub const DecodeError = std.Io.Reader.Error || error{InvalidKey};
/// Decode and validate the fixed HISTORY payload.
pub fn decode(reader: *std.Io.Reader) Header.DecodeError!Header {
const raw = try io.readInt(reader, u16);
const Tag = @typeInfo(TerminalScreenKey).@"enum".tag_type;
const value = std.math.cast(Tag, raw) orelse
return error.InvalidKey;
const key = std.enums.fromInt(
TerminalScreenKey,
value,
) orelse return error.InvalidKey;
return .{
.key = key,
.page_count = try io.readInt(reader, u32),
};
}
fn computeLen() usize {
comptime {
var buf: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
const value: Header = .{
.key = .primary,
.page_count = 0,
};
value.encode(&writer) catch unreachable;
return writer.end;
}
}
};
/// Errors possible while encoding one HISTORY and its complete PAGE sequence.
pub const EncodeError = Allocator.Error ||
page.EncodeError ||
record.Writer.FinishError ||
error{
/// The complete historical prefix has more pages than the header fits.
PageCountOverflow,
};
/// Encode one screen's HISTORY and its complete historical pages.
///
/// Pages are encoded newest-to-oldest. Compressed source pages are inspected
/// without changing their storage state. Completed records may already be
/// emitted if a later page fails.
pub fn encode(
terminal_screen: *const TerminalScreen,
key: TerminalScreenKey,
destination: *record.Writer,
) EncodeError!void {
// SCREEN begins at the page containing the active area's first row. Its
// leading rows are already resident; every previous complete page belongs
// to this HISTORY sequence.
const first = terminal_screen.pages.getTopLeft(.active).node.prev;
const page_count: usize = count: {
var page_count: usize = 0;
var node = first;
while (node) |current| : (node = current.prev) {
page_count += 1;
}
break :count page_count;
};
const header: Header = .{
.key = key,
.page_count = std.math.cast(
u32,
page_count,
) orelse return error.PageCountOverflow,
};
// HISTORY declares exactly how many PAGE records follow.
{
const payload = destination.begin(.history);
errdefer destination.cancel();
try header.encode(payload);
try destination.finish();
}
// Walk backward so each page can be prepended by the decoder immediately.
// PreservedPage borrows resident pages and clones compressed pages without
// changing the source node's representation.
var node = first;
while (node) |current| : (node = current.prev) {
var preserved = try current.pagePreservingState(terminal_screen.alloc);
defer preserved.deinit();
try page.encode(preserved.page(), destination);
}
}
/// Errors possible while restoring one HISTORY and its PAGE sequence.
pub const DecodeError = Decoder.InitError ||
Decoder.RestoreError ||
error{
/// The HISTORY key does not match the caller-selected screen.
UnexpectedScreenKey,
};
/// A decoded HISTORY manifest ready to restore its following PAGE records.
///
/// Keeping manifest decoding separate lets the full snapshot wrapper route a
/// HISTORY sequence by its encoded key before any pages mutate a Screen.
pub const Decoder = struct {
source: *std.Io.Reader,
header: Header,
pub const InitError = Header.DecodeError ||
record.Reader.InitError ||
record.Reader.FinishError ||
error{
/// The next record is valid but is not a HISTORY.
UnexpectedRecordTag,
};
pub const RestoreError = Allocator.Error ||
page.DecodeError ||
TerminalPageList.PageAllocation.FinalizeError ||
error{
/// The Screen already contains complete pages before its active page.
ExistingHistory,
};
/// Decode and finish the self-contained HISTORY manifest.
pub fn init(self: *Decoder, source: *std.Io.Reader) InitError!void {
var record_reader: record.Reader = undefined;
try record_reader.init(source);
if (record_reader.header.tag != .history) {
return error.UnexpectedRecordTag;
}
const header = try Header.decode(record_reader.payloadReader());
try record_reader.finish();
self.* = .{ .source = source, .header = header };
}
/// Restore the declared PAGE records into the selected native Screen.
pub fn decode(
self: *Decoder,
alloc: Allocator,
terminal_screen: *TerminalScreen,
) RestoreError!void {
// A freshly restored SCREEN may carry overlap inside its first page,
// but cannot already contain a complete historical page before that
// boundary.
const active_top = terminal_screen.pages.getTopLeft(.active);
if (terminal_screen.pages.getTopLeft(.screen).node != active_top.node) {
return error.ExistingHistory;
}
// Native row totals remain derived from the actual PAGE dimensions.
for (0..self.header.page_count) |_| {
// PAGE exposes its exact capacity before decoding the payload,
// allowing the destination PageList to allocate the final backing
// memory once.
var decoder: page.Decoder = undefined;
try decoder.init(self.source);
var allocation = try terminal_screen.pages.allocatePage(
decoder.capacity(),
);
defer allocation.deinit();
try decoder.decode(allocation.page(), alloc);
const contains_prompt = hasSemanticPrompt(allocation.page());
try allocation.finalize(.prepend);
if (contains_prompt) terminal_screen.semantic_prompt.seen = true;
}
terminal_screen.pages.assertIntegrity();
terminal_screen.assertIntegrity();
}
};
/// Restore one HISTORY and its declared PAGE records into a native Screen.
///
/// Each PAGE is decoded directly into a detached PageList-pooled allocation and
/// prepended only after its record and native integrity are validated. If a
/// later PAGE fails, earlier successful pages remain as a contiguous recent
/// history prefix.
pub fn decode(
source: *std.Io.Reader,
alloc: Allocator,
expected_key: TerminalScreenKey,
terminal_screen: *TerminalScreen,
) DecodeError!void {
var decoder: Decoder = undefined;
try decoder.init(source);
if (decoder.header.key != expected_key) return error.UnexpectedScreenKey;
try decoder.decode(alloc, terminal_screen);
}
fn hasSemanticPrompt(terminal_page: *const TerminalPage) bool {
const rows = terminal_page.rows.ptr(terminal_page.memory)[0..terminal_page.size.rows];
for (rows) |row| {
if (row.semantic_prompt != .none) return true;
const cells = row.cells.ptr(
terminal_page.memory,
)[0..terminal_page.size.cols];
for (cells) |cell| {
if (cell.semantic_content == .prompt) return true;
}
}
return false;
}
const test_header_fixture = test_fixture.parse(@embedFile("testdata/history-header-v1.hex"));
test "HISTORY header golden encoding and decoding" {
const expected: Header = .{
.key = .alternate,
.page_count = 0x01020304,
};
var encoded: [Header.len]u8 = undefined;
var writer: std.Io.Writer = .fixed(&encoded);
try expected.encode(&writer);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/history-header-v1.hex",
"snapshot_fixture-history-header-v1.hex",
&test_header_fixture,
writer.buffered(),
);
try std.testing.expectEqual(Header.len, test_header_fixture.len);
var reader: std.Io.Reader = .fixed(&test_header_fixture);
try std.testing.expectEqualDeep(expected, try Header.decode(&reader));
for (0..Header.len) |fixture_len| {
var truncated: std.Io.Reader = .fixed(
test_header_fixture[0..fixture_len],
);
try std.testing.expectError(
error.EndOfStream,
Header.decode(&truncated),
);
}
var invalid_key = test_header_fixture;
invalid_key[0] = 2;
var invalid_key_reader: std.Io.Reader = .fixed(&invalid_key);
try std.testing.expectError(
error.InvalidKey,
Header.decode(&invalid_key_reader),
);
}
test "HISTORY encodes newest first and restores complete history" {
// Choose an active height which spans two native pages, then grow until
// exactly two additional complete pages precede the active boundary.
var probe = try TerminalScreen.init(
std.testing.io,
std.testing.allocator,
.{ .cols = 80, .rows = 1, .max_scrollback_bytes = 0 },
);
const page_rows = probe.pages.getTopLeft(.screen).node.capacity().rows;
probe.deinit();
const screen_rows = page_rows + 1;
var source_screen = try TerminalScreen.init(
std.testing.io,
std.testing.allocator,
.{
.cols = 80,
.rows = screen_rows,
.max_scrollback_bytes = null,
},
);
defer source_screen.deinit();
source_screen.cursorAbsolute(0, screen_rows - 1);
while (source_screen.pages.totalPages() < 4) {
try source_screen.testWriteString("\n");
}
const active_top = source_screen.pages.getTopLeft(.active);
const newest_history = active_top.node.prev.?;
const oldest_history = newest_history.prev.?;
try std.testing.expectEqual(
source_screen.pages.getTopLeft(.screen).node,
oldest_history,
);
try std.testing.expectEqual(null, oldest_history.prev);
// Mark the two complete historical pages so the wire sequence and final
// native order are visible. The oldest prompt also verifies derived state
// is updated only when that page is restored.
oldest_history.page().getRowAndCell(0, 0).cell.* = .init('A');
oldest_history.page().getRowAndCell(
0,
0,
).cell.semantic_content = .prompt;
newest_history.page().getRowAndCell(0, 0).cell.* = .init('B');
// Encode SCREEN before history compression, then compress eligible history
// and require HISTORY encoding to preserve each source storage state.
var destination: std.Io.Writer.Allocating = .init(
std.testing.allocator,
);
defer destination.deinit();
var stream: record.Writer = .init(
std.testing.allocator,
&destination.writer,
);
defer stream.deinit();
try screen.encode(&source_screen, .primary, &stream);
const history_offset = destination.written().len;
_ = source_screen.pages.compress(.full);
const oldest_storage = oldest_history.storage();
const newest_storage = newest_history.storage();
try encode(&source_screen, .primary, &stream);
try std.testing.expectEqual(oldest_storage, oldest_history.storage());
try std.testing.expectEqual(newest_storage, newest_history.storage());
// Inspect the manifest and PAGE records independently. Newest history is
// sent first even though native PageList order is oldest-to-newest.
var history_source: std.Io.Reader = .fixed(
destination.written()[history_offset..],
);
var history_record: record.Reader = undefined;
try history_record.init(&history_source);
try std.testing.expectEqual(record.Tag.history, history_record.header.tag);
const header = try Header.decode(history_record.payloadReader());
try history_record.finish();
try std.testing.expectEqual(TerminalScreenKey.primary, header.key);
try std.testing.expectEqual(@as(u32, 2), header.page_count);
var decoded_newest = try page.decode(
&history_source,
std.testing.allocator,
);
defer decoded_newest.deinit();
try std.testing.expectEqual(
@as(u21, 'B'),
decoded_newest.getRowAndCell(0, 0).cell.codepoint(),
);
var decoded_oldest = try page.decode(
&history_source,
std.testing.allocator,
);
defer decoded_oldest.deinit();
try std.testing.expectEqual(
@as(u21, 'A'),
decoded_oldest.getRowAndCell(0, 0).cell.codepoint(),
);
try std.testing.expectError(error.EndOfStream, history_source.takeByte());
// Restore SCREEN first, then prepend HISTORY directly into that PageList.
var restore_source: std.Io.Reader = .fixed(destination.written());
var decoded_screen = try screen.decode(
&restore_source,
std.testing.io,
std.testing.allocator,
.{
.cols = 80,
.rows = screen_rows,
.max_scrollback_bytes = null,
},
);
defer decoded_screen.deinit();
try std.testing.expectEqual(
TerminalScreenKey.primary,
decoded_screen.key,
);
try std.testing.expectEqual(
@as(u64, @intCast(source_screen.pages.total_rows - screen_rows)),
decoded_screen.history_rows,
);
const restored = &decoded_screen.screen;
try std.testing.expect(!restored.semantic_prompt.seen);
try decode(
&restore_source,
std.testing.allocator,
.primary,
restored,
);
try std.testing.expectEqual(
source_screen.pages.totalPages(),
restored.pages.totalPages(),
);
const restored_oldest = restored.pages.getTopLeft(.screen).node;
try std.testing.expectEqual(
@as(u21, 'A'),
restored_oldest.page().getRowAndCell(0, 0).cell.codepoint(),
);
try std.testing.expectEqual(
@as(u21, 'B'),
restored_oldest.next.?
.page().getRowAndCell(0, 0).cell.codepoint(),
);
try std.testing.expect(restored.semantic_prompt.seen);
try std.testing.expectError(error.EndOfStream, restore_source.takeByte());
// Reuse a writable copy of the sequence for failure-path fixtures.
const encoded = try std.testing.allocator.dupe(
u8,
destination.written(),
);
defer std.testing.allocator.free(encoded);
const first_page_offset = history_offset + record.Header.len + Header.len;
const first_payload_len = std.mem.readInt(
u32,
encoded[first_page_offset + 2 ..][0..4],
.little,
);
const second_page_offset =
first_page_offset + record.Header.len + first_payload_len;
// Truncate the first PAGE only after its header has exposed a capacity.
// Its detached allocation is discarded and the live SCREEN list remains
// completely unchanged.
var truncated_source: std.Io.Reader = .fixed(
encoded[0 .. second_page_offset - 1],
);
var decoded_truncated = try screen.decode(
&truncated_source,
std.testing.io,
std.testing.allocator,
.{
.cols = 80,
.rows = screen_rows,
.max_scrollback_bytes = null,
},
);
defer decoded_truncated.deinit();
const truncated = &decoded_truncated.screen;
const truncated_screen_first = truncated.pages.getTopLeft(.screen).node;
const truncated_screen_page_count = truncated.pages.totalPages();
try std.testing.expectError(
error.EndOfStream,
decode(
&truncated_source,
std.testing.allocator,
.primary,
truncated,
),
);
try std.testing.expectEqual(
truncated_screen_page_count,
truncated.pages.totalPages(),
);
try std.testing.expectEqual(
truncated_screen_first,
truncated.pages.getTopLeft(.screen).node,
);
truncated.pages.assertIntegrity();
truncated.assertIntegrity();
// Corrupt only the older PAGE tag. A failure in a later PAGE keeps only
// the successfully prepended newer pages, contiguous with SCREEN.
std.mem.writeInt(
u16,
encoded[second_page_offset..][0..2],
@intFromEnum(record.Tag.screen),
.little,
);
var partial_source: std.Io.Reader = .fixed(encoded);
var decoded_partial = try screen.decode(
&partial_source,
std.testing.io,
std.testing.allocator,
.{
.cols = 80,
.rows = screen_rows,
.max_scrollback_bytes = null,
},
);
defer decoded_partial.deinit();
const partial = &decoded_partial.screen;
const screen_page_count = partial.pages.totalPages();
try std.testing.expectError(
error.UnexpectedRecordTag,
decode(
&partial_source,
std.testing.allocator,
.primary,
partial,
),
);
try std.testing.expectEqual(
screen_page_count + 1,
partial.pages.totalPages(),
);
try std.testing.expectEqual(
@as(u21, 'B'),
partial.pages.getTopLeft(.screen).node
.page().getRowAndCell(0, 0).cell.codepoint(),
);
try std.testing.expect(!partial.semantic_prompt.seen);
partial.pages.assertIntegrity();
partial.assertIntegrity();
}
test "HISTORY encodes and restores an empty sequence" {
var terminal_screen = try TerminalScreen.init(
std.testing.io,
std.testing.allocator,
.{
.cols = 2,
.rows = 2,
.max_scrollback_bytes = null,
},
);
defer terminal_screen.deinit();
var destination: std.Io.Writer.Allocating = .init(
std.testing.allocator,
);
defer destination.deinit();
var stream: record.Writer = .init(
std.testing.allocator,
&destination.writer,
);
defer stream.deinit();
try encode(&terminal_screen, .primary, &stream);
var inspect_source: std.Io.Reader = .fixed(destination.written());
var history_record: record.Reader = undefined;
try history_record.init(&inspect_source);
const header = try Header.decode(history_record.payloadReader());
try history_record.finish();
try std.testing.expectEqual(@as(u32, 0), header.page_count);
try std.testing.expectError(error.EndOfStream, inspect_source.takeByte());
var decode_source: std.Io.Reader = .fixed(destination.written());
const initial_first = terminal_screen.pages.getTopLeft(.screen).node;
try decode(
&decode_source,
std.testing.allocator,
.primary,
&terminal_screen,
);
try std.testing.expectEqual(
initial_first,
terminal_screen.pages.getTopLeft(.screen).node,
);
try std.testing.expectError(error.EndOfStream, decode_source.takeByte());
}
test "HISTORY rejects invalid routing and incomplete sequences" {
var terminal_screen = try TerminalScreen.init(
std.testing.io,
std.testing.allocator,
.{
.cols = 2,
.rows = 2,
.max_scrollback_bytes = null,
},
);
defer terminal_screen.deinit();
// Routing and sequence length still determine which native screen is
// mutated and where the following record begins, so they remain strict.
const Case = struct {
header: Header,
expected: anyerror,
};
const cases = [_]Case{
.{
.header = .{
.key = .alternate,
.page_count = 0,
},
.expected = error.UnexpectedScreenKey,
},
.{
.header = .{
.key = .primary,
.page_count = 1,
},
.expected = error.EndOfStream,
},
};
for (cases) |case| {
var destination: std.Io.Writer.Allocating = .init(
std.testing.allocator,
);
defer destination.deinit();
var stream: record.Writer = .init(
std.testing.allocator,
&destination.writer,
);
defer stream.deinit();
const payload = stream.begin(.history);
errdefer stream.cancel();
try case.header.encode(payload);
try stream.finish();
var source: std.Io.Reader = .fixed(destination.written());
try std.testing.expectError(
case.expected,
decode(
&source,
std.testing.allocator,
.primary,
&terminal_screen,
),
);
terminal_screen.pages.assertIntegrity();
terminal_screen.assertIntegrity();
}
}

493
src/terminal/snapshot/hyperlink.zig Normal file
View File

@@ -0,0 +1,493 @@
//! Snapshot hyperlink entry encoding.
//!
//! Each entry contains a URI and either an implicit numeric ID or an explicit
//! byte-string ID. Implicit IDs are generated by the terminal when the source
//! hyperlink has no explicit ID. A record can use these entries to build a
//! hyperlink table and assign indexes according to that record's format.
//! Indexing and ordering are properties of the containing record rather than
//! this codec.
//!
//! URIs and explicit IDs are non-empty arbitrary byte strings. Their lengths
//! are retained on the wire; they are not NUL-terminated and need not contain
//! UTF-8. Native page storage requires every allocated hyperlink string to
//! contain at least one byte. Encoding and standalone decoding reject empty
//! strings; PAGE decoding consumes but ignores entries it cannot represent.
//!
//! All integers are unsigned and little-endian.
//!
//! ## Implicit ID
//!
//! | Offset | Size | Field |
//! | -----: | --------: | :----------------- |
//! | 0 | 1 | Kind, `1` |
//! | 1 | 4 | Implicit ID (`u32`) |
//! | 5 | 4 | URI length (`u32`) |
//! | 9 | `uri_len` | URI bytes |
//!
//! ## Explicit ID
//!
//! | Offset | Size | Field |
//! | -----------: | --------: | :------------------------- |
//! | 0 | 1 | Kind, `2` |
//! | 1 | 4 | Explicit ID length (`u32`) |
//! | 5 | `id_len` | Explicit ID bytes |
//! | `5 + id_len` | 4 | URI length (`u32`) |
//! | `9 + id_len` | `uri_len` | URI bytes |
//!
//! Both variants have nine bytes of fixed overhead. The remaining bytes are
//! the URI plus the explicit ID, when present.
//!
//! The standalone decoder returns an allocator-owned hyperlink. PAGE decoding
//! reads strings directly into the destination page's allocator instead.
const std = @import("std");
const Allocator = std.mem.Allocator;
const test_fixture = @import("fixture.zig");
const io = @import("io.zig");
const terminal_hyperlink = @import("../hyperlink.zig");
const terminal_page = @import("../page.zig");
const terminal_size = @import("../size.zig");
const Kind = enum(u8) {
implicit = 1,
explicit = 2,
};
/// Errors possible while encoding one hyperlink entry.
pub const EncodeError = std.Io.Writer.Error || error{
/// A native hyperlink URI must contain at least one byte.
InvalidUri,
/// A native explicit hyperlink ID must contain at least one byte.
InvalidExplicitId,
/// A string does not fit the version 1 length field.
StringTooLong,
};
/// Errors possible while decoding one allocator-owned hyperlink entry.
pub const DecodeError = std.Io.Reader.Error || Allocator.Error || error{
/// The hyperlink kind is not defined by snapshot version 1.
InvalidKind,
/// A native hyperlink URI must contain at least one byte.
InvalidUri,
/// A native explicit hyperlink ID must contain at least one byte.
InvalidExplicitId,
};
/// Errors possible while decoding directly into a native page.
pub const DecodePageError = std.Io.Reader.Error || error{
/// The hyperlink kind is not defined by snapshot version 1.
InvalidKind,
};
/// Encode one hyperlink entry.
pub fn encode(
value: terminal_hyperlink.Hyperlink,
writer: *std.Io.Writer,
) EncodeError!void {
// Validate the complete value before writing any part of this variable-
// length entry. This keeps callers able to roll back at entry boundaries.
if (value.uri.len == 0) return error.InvalidUri;
if (value.uri.len > std.math.maxInt(u32)) return error.StringTooLong;
if (value.id == .explicit) {
const id = value.id.explicit;
if (id.len == 0) return error.InvalidExplicitId;
if (id.len > std.math.maxInt(u32)) return error.StringTooLong;
}
switch (value.id) {
.implicit => |id| {
try writer.writeByte(@intFromEnum(Kind.implicit));
try io.writeInt(writer, u32, id);
try io.writeInt(writer, u32, @intCast(value.uri.len));
try writer.writeAll(value.uri);
},
.explicit => |id| {
try writer.writeByte(@intFromEnum(Kind.explicit));
try io.writeInt(writer, u32, @intCast(id.len));
try writer.writeAll(id);
try io.writeInt(writer, u32, @intCast(value.uri.len));
try writer.writeAll(value.uri);
},
}
}
/// Decode one allocator-owned hyperlink entry.
///
/// The caller owns the returned value and must call `Hyperlink.deinit`.
pub fn decode(
reader: *std.Io.Reader,
alloc: Allocator,
) DecodeError!terminal_hyperlink.Hyperlink {
const kind_raw = try reader.takeByte();
const kind = std.enums.fromInt(Kind, kind_raw) orelse
return error.InvalidKind;
return switch (kind) {
.implicit => implicit: {
const id = try io.readInt(reader, u32);
const uri_len: usize = @intCast(try io.readInt(reader, u32));
if (uri_len == 0) return error.InvalidUri;
const uri = try alloc.alloc(u8, uri_len);
errdefer alloc.free(uri);
try reader.readSliceAll(uri);
break :implicit .{
.id = .{ .implicit = id },
.uri = uri,
};
},
.explicit => explicit: {
const id_len: usize = @intCast(try io.readInt(reader, u32));
if (id_len == 0) return error.InvalidExplicitId;
const id = try alloc.alloc(u8, id_len);
errdefer alloc.free(id);
try reader.readSliceAll(id);
const uri_len: usize = @intCast(try io.readInt(reader, u32));
if (uri_len == 0) return error.InvalidUri;
const uri = try alloc.alloc(u8, uri_len);
errdefer alloc.free(uri);
try reader.readSliceAll(uri);
break :explicit .{
.id = .{ .explicit = id },
.uri = uri,
};
},
};
}
/// Decode one hyperlink directly into page-owned storage.
///
/// Explicit ID and URI bytes are read into the page string allocator and the
/// completed entry is inserted into the page hyperlink set. The returned ID is
/// the native ID assigned by the destination page. Zero is returned when the
/// encoded value cannot be represented within the destination page, including
/// empty strings and insufficient advertised capacity.
pub fn decodePage(
page: *terminal_page.Page,
reader: *std.Io.Reader,
) DecodePageError!terminal_hyperlink.Id {
const kind_raw = try reader.takeByte();
const kind = std.enums.fromInt(Kind, kind_raw) orelse
return error.InvalidKind;
const entry: terminal_hyperlink.PageEntry = switch (kind) {
.implicit => implicit: {
const id = try io.readInt(reader, u32);
const uri_len: usize = @intCast(try io.readInt(reader, u32));
if (uri_len == 0) return 0;
const uri = decodePageString(
page,
reader,
uri_len,
) catch |err| switch (err) {
error.StringsOutOfMemory => {
try reader.discardAll(uri_len);
return 0;
},
else => |read_err| return read_err,
};
break :implicit .{
.id = .{ .implicit = id },
.uri = uri,
};
},
.explicit => explicit: {
const id_len: usize = @intCast(try io.readInt(reader, u32));
if (id_len == 0) {
const uri_len: usize = @intCast(
try io.readInt(reader, u32),
);
try reader.discardAll(uri_len);
return 0;
}
const id = decodePageString(
page,
reader,
id_len,
) catch |err| switch (err) {
error.StringsOutOfMemory => {
try reader.discardAll(id_len);
const uri_len: usize = @intCast(
try io.readInt(reader, u32),
);
try reader.discardAll(uri_len);
return 0;
},
else => |read_err| return read_err,
};
errdefer page.string_alloc.free(
page.memory,
id.slice(page.memory),
);
const uri_len: usize = @intCast(try io.readInt(reader, u32));
if (uri_len == 0) {
page.string_alloc.free(
page.memory,
id.slice(page.memory),
);
return 0;
}
const uri = decodePageString(
page,
reader,
uri_len,
) catch |err| switch (err) {
error.StringsOutOfMemory => {
try reader.discardAll(uri_len);
page.string_alloc.free(
page.memory,
id.slice(page.memory),
);
return 0;
},
else => |read_err| return read_err,
};
break :explicit .{
.id = .{ .explicit = id },
.uri = uri,
};
},
};
return page.hyperlink_set.addContext(
page.memory,
entry,
.{ .page = page },
) catch {
// The entry was fully consumed, so capacity failure only loses this
// optional hyperlink. Free its strings and let cells restore unlinked.
entry.free(page);
return 0;
};
}
fn decodePageString(
page: *terminal_page.Page,
reader: *std.Io.Reader,
len: usize,
) (std.Io.Reader.Error || error{StringsOutOfMemory})!terminal_size.Offset(u8).Slice {
std.debug.assert(len > 0);
// Allocate space for the string and read directly into it.
const value = page.string_alloc.alloc(
u8,
page.memory,
len,
) catch return error.StringsOutOfMemory;
errdefer page.string_alloc.free(page.memory, value);
try reader.readSliceAll(value);
return .{
.len = value.len,
.offset = terminal_size.getOffset(
u8,
page.memory,
&value[0],
),
};
}
const test_implicit_fixture = test_fixture.parse(@embedFile("testdata/hyperlink-implicit-v1.hex"));
const test_explicit_fixture = test_fixture.parse(@embedFile("testdata/hyperlink-explicit-v1.hex"));
test "golden implicit encoding" {
const value: terminal_hyperlink.Hyperlink = .{
.id = .{ .implicit = 0x01020304 },
.uri = "uri",
};
var buf: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
try encode(value, &writer);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/hyperlink-implicit-v1.hex",
"snapshot_fixture-hyperlink-implicit-v1.hex",
&test_implicit_fixture,
writer.buffered(),
);
var reader: std.Io.Reader = .fixed(&test_implicit_fixture);
var decoded = try decode(&reader, std.testing.allocator);
defer decoded.deinit(std.testing.allocator);
try std.testing.expectEqualDeep(value, decoded);
}
test "golden explicit encoding" {
const value: terminal_hyperlink.Hyperlink = .{
.id = .{ .explicit = "id" },
.uri = "uri",
};
var buf: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
try encode(value, &writer);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/hyperlink-explicit-v1.hex",
"snapshot_fixture-hyperlink-explicit-v1.hex",
&test_explicit_fixture,
writer.buffered(),
);
var reader: std.Io.Reader = .fixed(&test_explicit_fixture);
var decoded = try decode(&reader, std.testing.allocator);
defer decoded.deinit(std.testing.allocator);
try std.testing.expectEqualDeep(value, decoded);
}
test "encode rejects invalid strings before writing" {
const cases = [_]struct {
value: terminal_hyperlink.Hyperlink,
expected: anyerror,
}{
.{
.value = .{ .id = .{ .implicit = 1 }, .uri = "" },
.expected = error.InvalidUri,
},
.{
.value = .{ .id = .{ .explicit = "" }, .uri = "uri" },
.expected = error.InvalidExplicitId,
},
.{
.value = .{ .id = .{ .explicit = "id" }, .uri = "" },
.expected = error.InvalidUri,
},
};
for (cases) |case| {
var encoded: [32]u8 = undefined;
var writer: std.Io.Writer = .fixed(&encoded);
try std.testing.expectError(
case.expected,
encode(case.value, &writer),
);
try std.testing.expectEqual(@as(usize, 0), writer.end);
}
}
test "decode rejects empty strings" {
const cases = [_]struct {
fixture: []const u8,
expected: anyerror,
}{
.{
.fixture = "\x01\x04\x03\x02\x01\x00\x00\x00\x00",
.expected = error.InvalidUri,
},
.{
.fixture = "\x02\x00\x00\x00\x00",
.expected = error.InvalidExplicitId,
},
.{
.fixture = "\x02\x02\x00\x00\x00id\x00\x00\x00\x00",
.expected = error.InvalidUri,
},
};
for (cases) |case| {
var reader: std.Io.Reader = .fixed(case.fixture);
try std.testing.expectError(
case.expected,
decode(&reader, std.testing.allocator),
);
}
}
test "decodePage ignores empty strings" {
const fixtures = [_][]const u8{
"\x01\x04\x03\x02\x01\x00\x00\x00\x00",
"\x02\x00\x00\x00\x00\x03\x00\x00\x00uri",
"\x02\x02\x00\x00\x00id\x00\x00\x00\x00",
};
for (fixtures) |fixture| {
var page = try terminal_page.Page.init(.{
.cols = 1,
.rows = 1,
.hyperlink_bytes = 512,
.string_bytes = 16,
});
defer page.deinit();
var reader: std.Io.Reader = .fixed(fixture);
try std.testing.expectEqual(
@as(terminal_hyperlink.Id, 0),
try decodePage(&page, &reader),
);
try std.testing.expectEqual(@as(usize, 0), page.hyperlink_set.count());
try std.testing.expectError(error.EndOfStream, reader.takeByte());
}
}
test "decode rejects invalid kinds" {
for ([_]u8{ 0, 3, std.math.maxInt(u8) }) |kind| {
var fixture: [1]u8 = .{kind};
var reader: std.Io.Reader = .fixed(&fixture);
try std.testing.expectError(
error.InvalidKind,
decode(&reader, std.testing.allocator),
);
}
}
test "decode allocation failure" {
{
var reader: std.Io.Reader = .fixed(&test_implicit_fixture);
var failing = std.testing.FailingAllocator.init(
std.testing.allocator,
.{ .fail_index = 0 },
);
try std.testing.expectError(
error.OutOfMemory,
decode(&reader, failing.allocator()),
);
}
for (0..2) |fail_index| {
var failing = std.testing.FailingAllocator.init(
std.testing.allocator,
.{ .fail_index = fail_index },
);
var reader: std.Io.Reader = .fixed(&test_explicit_fixture);
try std.testing.expectError(
error.OutOfMemory,
decode(&reader, failing.allocator()),
);
}
}
test "reject every truncation" {
const fixtures: [2][]const u8 = .{
&test_implicit_fixture,
&test_explicit_fixture,
};
for (fixtures) |fixture| {
for (0..fixture.len) |fixture_len| {
var reader: std.Io.Reader = .fixed(fixture[0..fixture_len]);
try std.testing.expectError(
error.EndOfStream,
decode(&reader, std.testing.allocator),
);
}
}
}

55
src/terminal/snapshot/io.zig Normal file
View File

@@ -0,0 +1,55 @@
//! Wire-format I/O helpers.
//!
//! These are sometimes very thin layers over standard `std.Io.Writer`
//! but its to help ensure consistency in our behavior and avoid some
//! pitfalls.
const std = @import("std");
const testing = std.testing;
/// writeInt ensures we write in little-endian since that is our
/// standard form.
pub fn writeInt(
writer: *std.Io.Writer,
comptime T: type,
value: T,
) std.Io.Writer.Error!void {
try writer.writeInt(T, value, .little);
}
/// Read a little-endian integer.
///
/// This avoids `takeInt` because at the time of writing this the underlying
/// implementation uses `takeArray` which asserts that the reader buffer
/// is as large as T. Our approach works with caller-selected buffers that
/// may be smaller.
pub fn readInt(
reader: *std.Io.Reader,
comptime T: type,
) std.Io.Reader.Error!T {
var buf: [@sizeOf(T)]u8 = undefined;
try reader.readSliceAll(&buf);
return std.mem.readInt(T, &buf, .little);
}
test "integer round trip with a one-byte reader buffer" {
var encoded: [6]u8 = undefined;
var writer: std.Io.Writer = .fixed(&encoded);
try writeInt(&writer, u16, 0x1234);
try writeInt(&writer, u32, 0x56789abc);
// Keep the reader buffer smaller than either integer to verify readInt
// does not inherit std.Io.Reader.takeInt's buffer-size requirement.
var source: std.Io.Reader = .fixed(&encoded);
var buf: [1]u8 = undefined;
var limited = source.limited(.unlimited, &buf);
try testing.expectEqual(
@as(u16, 0x1234),
try readInt(&limited.interface, u16),
);
try testing.expectEqual(
@as(u32, 0x56789abc),
try readInt(&limited.interface, u32),
);
}

145
src/terminal/snapshot/main.zig Normal file
View File

@@ -0,0 +1,145 @@
//! Terminal snapshot binary representation and codecs.
//!
//! This is NOT a full transport-ready format to implement generic replay
//! software such as multiplexers, recorders (e.g. asciinema), etc. The goal
//! of this package is to provide a documented, binary-compatible representation
//! for a terminal state.
//!
//! We call this a "snapshot." The snapshot is purposely laid out in a way
//! that prioritizes making a terminal functional as quickly as possible.
//! To do that, it sends the active terminal state followed by a READY record,
//! then complete history.
//!
//! READY denotes that enough of the terminal state is down that it can
//! be fully rendered at that point. This is also the point where live
//! terminals can also start accepting pty bytes, typically. But the current
//! snapshot format lacks some of the information necessary to synchronize
//! pty byte state with an authoritative server.
//!
//! After READY, we send history pages (scrollback).
//!
//! ## Snapshot Format
//!
//! This documents snapshot format 1. Version 1 is the work-in-progress
//! format that we intended to continue to break until we can promise
//! binary compatibility.
//!
//! A snapshot is one envelope followed by a sequence of records. The envelope
//! occurs once at byte zero. Every record is independently framed as a fixed
//! header followed by the number of payload bytes declared by that header.
//!
//! ```text
//! +------------------+
//! | Envelope |
//! +------------------+
//! | Record 1 header |
//! +------------------+
//! | Record 1 payload |
//! +------------------+
//! | Record 2 header |
//! +------------------+
//! | Record 2 payload |
//! +------------------+
//! | ... |
//! +------------------+
//! ```
//!
//! Record groups have a strict order:
//!
//! ```text
//! +----------------------------------------+
//! | TERMINAL |
//! +----------------------------------------+
//! | SCREEN * terminal.screen_count |
//! | PAGE * each screen.page_count |
//! +----------------------------------------+
//! | READY |
//! +----------------------------------------+
//! | HISTORY * terminal.screen_count |
//! | PAGE * each history.page_count |
//! +----------------------------------------+
//! | FINISH |
//! +----------------------------------------+
//! ```
//!
//! The SCREEN and HISTORY sequence groups may each appear in any key order.
//! Each key must occur exactly once in each group and must identify one of the
//! screens declared by TERMINAL. SCREEN contains the complete pages needed to
//! restore each active area. HISTORY contains the older complete pages for its
//! screen in newest-to-oldest order so they can be prepended as they arrive.
//! Every SCREEN has one corresponding HISTORY, even when its history page count
//! is zero. FINISH terminates the snapshot. Bytes after FINISH belong to the
//! containing transport and are not consumed by snapshot decoding.
//!
//! READY and FINISH contain BLAKE3-256 digests of all preceding snapshot bytes.
//! READY therefore validates the renderable active-state prefix. FINISH covers
//! READY and all history as well, validating the complete snapshot and its
//! record ordering. Each SCREEN declares its complete logical history extent,
//! allowing a client to size its scrollbar at READY even though older PAGE
//! records arrive afterward.
//!
//! ## Encoding
//!
//! Encode a complete snapshot into any writer:
//!
//! ```zig
//! var output: std.Io.Writer.Allocating = .init(alloc);
//! defer output.deinit();
//!
//! try snapshot.encode(alloc, &output.writer, &terminal);
//!
//! const bytes = output.written();
//! ```
//!
//! Encoding begins at the writer's current position, so unrelated bytes may
//! precede the snapshot. The encoder buffers only the current record payload
//! to calculate its length and CRC32C; completed records stream immediately
//! and BLAKE3 checkpoint coverage is updated incrementally. Buffering is an
//! encoder implementation detail, not a requirement of the wire format.
//!
//! A failure may leave prior complete records, or a partial record if the
//! destination itself fails. Such a prefix has no valid FINISH checkpoint and
//! cannot be restored as a complete snapshot.
//!
//! Each record type usually exposes an `encode` function that encodes
//! a complete record, such as `screen.encode`.
//!
//! ## Decoding
//!
//! `snapshot.decode` consumes exactly one snapshot through FINISH and leaves
//! any following bytes unread. This permits multiple snapshots or live protocol
//! data to share a stream without waiting for the peer to close it. Transports
//! that deliver live PTY data before history finishes must multiplex that data
//! outside this ordered snapshot record sequence.
//!
//! ```zig
//! var terminal = try snapshot.decode(alloc, io, &reader);
//! defer terminal.deinit(alloc);
//! ```
//!
//! Use `snapshot.decodeExact` for a bounded file or buffer that must contain
//! only one snapshot. It preserves the stricter end-of-file check, which may
//! block when used with a live stream.
pub const checkpoint = @import("checkpoint.zig");
pub const envelope = @import("envelope.zig");
pub const grid = @import("grid.zig");
pub const history = @import("history.zig");
pub const hyperlink = @import("hyperlink.zig");
pub const page = @import("page.zig");
pub const record = @import("record.zig");
pub const screen = @import("screen.zig");
pub const style = @import("style.zig");
pub const terminal = @import("terminal.zig");
const codec = @import("snapshot.zig");
pub const EncodeError = codec.EncodeError;
pub const DecodeError = codec.DecodeError;
pub const DecodeExactError = codec.DecodeExactError;
pub const encode = codec.encode;
pub const decode = codec.decode;
pub const decodeExact = codec.decodeExact;
test {
@import("std").testing.refAllDecls(@This());
}

1462
src/terminal/snapshot/page.zig Normal file
View File

File diff suppressed because it is too large Load Diff

599
src/terminal/snapshot/record.zig Normal file
View File

@@ -0,0 +1,599 @@
//! Snapshot record framing.
//!
//! Records begin immediately after the single snapshot envelope and continue
//! back-to-back until FINISH. Each record has one fixed-size header followed
//! by exactly `payload_len` bytes. The payload length limits the record reader
//! so malformed payloads cannot consume bytes from the following record.
//!
//! The CRC covers the first six header bytes (tag and payload length)
//! followed by the payload. The CRC field itself is not included for obvious
//! reasons. All integers are unsigned and little-endian.
//!
//! | Offset | Size | Field |
//! | -----: | ------------: | :--------------------- |
//! | 0 | 2 | Tag (`u16`) |
//! | 2 | 4 | Payload length (`u32`) |
//! | 6 | 4 | CRC32C (`u32`) |
//! | 10 | `payload_len` | Payload |
//!
//! Supported tags are in `Tag`.
const std = @import("std");
const assert = std.debug.assert;
const Allocator = std.mem.Allocator;
const test_fixture = @import("fixture.zig");
const io = @import("io.zig");
const Blake3 = std.crypto.hash.Blake3;
/// The running digest shared by snapshot stream codecs and checkpoints.
pub const PrefixDigest = [Blake3.digest_length]u8;
/// CRC32C as specified by the snapshot format. Zig names this standard
/// parameter set after its iSCSI use.
pub const Crc32c = std.hash.crc.Crc32Iscsi;
/// Identifies the layout and meaning of a record payload.
/// The current snapshot version rejects every value not listed here.
pub const Tag = enum(u16) {
/// Terminal-wide live state and configuration.
terminal = 1,
/// One live screen and its page sequence.
screen = 2,
/// One complete logical terminal page.
page = 3,
/// One screen's complete history manifest and page sequence.
history = 4,
/// Digest marking the validated terminal-state prefix.
ready = 5,
/// Digest validating the complete snapshot blob.
finish = 6,
};
/// The fixed framing that precedes every record payload.
pub const Header = struct {
/// Number of bytes written by `encode`, calculated using the encoder itself
/// so this remains synchronized with the field-by-field wire format.
pub const len = computeLen();
comptime {
// This size is part of the wire format. If it changes, the snapshot
// version and golden fixtures must also change.
assert(len == 10);
}
/// Determines how the payload is decoded.
tag: Tag,
/// Number of payload bytes immediately following this header.
payload_len: u32,
/// CRC32C over the encoded tag, payload length, and payload.
crc32c: u32,
/// Errors possible while decoding a record header.
pub const DecodeError = std.Io.Reader.Error || error{InvalidTag};
/// Encode the fixed record header.
pub fn encode(
self: Header,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
try encodeChecksumPrefix(self.tag, self.payload_len, writer);
try io.writeInt(writer, u32, self.crc32c);
}
/// Decode a fixed record header and reject unknown tags.
/// Payload length and CRC validation occur while decoding the payload.
pub fn decode(reader: *std.Io.Reader) DecodeError!Header {
const tag_raw = try io.readInt(reader, u16);
const tag = std.enums.fromInt(Tag, tag_raw) orelse {
return error.InvalidTag;
};
return .{
.tag = tag,
.payload_len = try io.readInt(reader, u32),
.crc32c = try io.readInt(reader, u32),
};
}
/// Computes the required header length at comptime.
fn computeLen() usize {
comptime {
var buf: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
const header: Header = .{
.tag = .terminal,
.payload_len = 0,
.crc32c = 0,
};
header.encode(&writer) catch unreachable;
return writer.end;
}
}
};
/// A writer that calculates a record checksum without retaining payload bytes.
/// The checksum is initialized with the encoded tag and payload length.
pub const Checksum = struct {
hashing: std.Io.Writer.Hashing(Crc32c),
/// Begin a checksum with the encoded tag and payload length already added.
pub fn init(tag: Tag, payload_len: u32) Checksum {
var result: Checksum = .{
.hashing = .initHasher(.init(), &.{}),
};
encodeChecksumPrefix(
tag,
payload_len,
&result.hashing.writer,
) catch unreachable;
return result;
}
/// Return the writer through which the complete payload must be encoded.
pub fn writer(self: *Checksum) *std.Io.Writer {
return &self.hashing.writer;
}
/// Finish and return the checksum stored in the record header.
pub fn final(self: *Checksum) u32 {
self.hashing.writer.flush() catch unreachable;
return self.hashing.hasher.final();
}
};
/// Streams complete records while retaining only one payload at a time.
///
/// All emitted bytes pass through one unbuffered BLAKE3 writer. The scratch
/// allocation is retained between records so a stream's peak memory is the
/// largest record payload rather than the complete snapshot.
pub const Writer = struct {
hashing: std.Io.Writer.Hashed(Blake3),
scratch: std.Io.Writer.Allocating,
active_tag: ?Tag,
pub fn init(
alloc: Allocator,
destination: *std.Io.Writer,
) Writer {
return .{
.hashing = destination.hashed(Blake3.init(.{}), &.{}),
.scratch = .init(alloc),
.active_tag = null,
};
}
pub fn deinit(self: *Writer) void {
assert(self.active_tag == null);
assert(self.scratch.writer.end == 0);
self.scratch.deinit();
self.* = undefined;
}
/// Return the digest-updating writer for unframed snapshot bytes.
pub fn writer(self: *Writer) *std.Io.Writer {
assert(self.active_tag == null);
return &self.hashing.writer;
}
/// Begin one record and return its reusable payload writer.
pub fn begin(self: *Writer, tag: Tag) *std.Io.Writer {
assert(self.active_tag == null);
assert(self.scratch.writer.end == 0);
self.active_tag = tag;
return &self.scratch.writer;
}
pub const FinishError = std.Io.Writer.Error || error{
/// The payload cannot be represented by the record's `u32` length.
PayloadTooLarge,
};
/// Finish and emit the active record's header followed by its payload.
///
/// Payload validation failures occur before this function and emit no part
/// of the record. A destination failure here may have emitted a prefix.
pub fn finish(self: *Writer) FinishError!void {
assert(self.active_tag != null);
const tag = self.active_tag.?;
defer self.cancel();
const payload = self.scratch.written();
const payload_len = std.math.cast(
u32,
payload.len,
) orelse return error.PayloadTooLarge;
// The CRC prefix contains the payload length, so checksum the retained
// payload only after its final length is known.
var checksum: Checksum = .init(tag, payload_len);
checksum.writer().writeAll(payload) catch unreachable;
const header: Header = .{
.tag = tag,
.payload_len = payload_len,
.crc32c = checksum.final(),
};
var header_bytes: [Header.len]u8 = undefined;
var header_writer: std.Io.Writer = .fixed(&header_bytes);
header.encode(&header_writer) catch unreachable;
try self.hashing.writer.writeAll(&header_bytes);
try self.hashing.writer.writeAll(payload);
}
/// Discard the active record without emitting any bytes.
///
/// This is idempotent so an outer error cleanup may call it after `finish`
/// has already cleared state following a destination failure.
pub fn cancel(self: *Writer) void {
self.scratch.shrinkRetainingCapacity(0);
self.active_tag = null;
}
/// Finalize the prefix written so far without consuming the hasher.
pub fn prefixDigest(self: *const Writer) PrefixDigest {
// Checkpoints require an exact byte boundary. Writer owns this
// adapter and always constructs it without a buffer.
assert(self.active_tag == null);
assert(self.scratch.writer.end == 0);
assert(self.hashing.writer.buffer.len == 0);
assert(self.hashing.writer.buffered().len == 0);
var result: PrefixDigest = undefined;
self.hashing.hasher.final(&result);
return result;
}
};
/// Hashes snapshot bytes as they are consumed without reading ahead.
pub const StreamReader = struct {
hashing: std.Io.Reader.Hashed(Blake3),
pub fn init(input: *std.Io.Reader) StreamReader {
return .{
.hashing = input.hashed(Blake3.init(.{}), &.{}),
};
}
/// Return the digest-updating reader used before and through READY.
pub fn reader(self: *StreamReader) *std.Io.Reader {
return &self.hashing.reader;
}
/// Return the source used to consume FINISH without hashing it.
pub fn source(self: *StreamReader) *std.Io.Reader {
return self.hashing.in;
}
/// Finalize the prefix consumed so far without consuming the hasher.
pub fn prefixDigest(self: *const StreamReader) PrefixDigest {
// A nonempty adapter buffer could contain bytes beyond a checkpoint.
// Construction is private to this type and fixes its capacity at zero.
assert(self.hashing.reader.buffer.len == 0);
assert(self.hashing.reader.bufferedLen() == 0);
var result: PrefixDigest = undefined;
self.hashing.hasher.final(&result);
return result;
}
};
/// Reads one complete record.
///
/// `init` decodes the header, `payloadReader` returns a reader limited to the
/// declared payload, and `finish` verifies exact exhaustion and the CRC32C.
pub const Reader = struct {
header: Header,
// The limited reader normally streams directly into the hashing reader's
// buffer. One byte is enough for operations that require it to buffer,
// such as peek and discard.
limited_buffer: [1]u8,
// PAGE decoding performs many small reads. 256 bytes batches several cells
// while CRC32C is calculated without making Reader large on the stack.
hashing_buffer: [256]u8,
limited: std.Io.Reader.Limited,
hashing: std.Io.Reader.Hashed(Crc32c),
pub const InitError = Header.DecodeError;
/// Errors detected after a payload decoder returns.
pub const FinishError = error{
/// The payload bytes do not match the CRC in the record header.
InvalidChecksum,
/// The decoder did not consume exactly `payload_len` bytes.
PayloadNotExhausted,
};
/// Decode a record header and initialize its payload reader.
///
/// `self` must remain at a stable address until `finish` returns. Decode
/// the payload only through the reader returned by `payloadReader`.
pub fn init(
self: *Reader,
source: *std.Io.Reader,
) InitError!void {
self.* = undefined;
self.header = try Header.decode(source);
self.limited = .init(
source,
.limited(self.header.payload_len),
&self.limited_buffer,
);
const checksum: Checksum = .init(
self.header.tag,
self.header.payload_len,
);
self.hashing = .init(
&self.limited.interface,
checksum.hashing.hasher,
&self.hashing_buffer,
);
}
/// Return the length-limited, checksum-updating payload reader.
pub fn payloadReader(self: *Reader) *std.Io.Reader {
return &self.hashing.reader;
}
/// Require exact payload exhaustion and validate its CRC32C.
pub fn finish(self: *Reader) FinishError!void {
if (self.hashing.reader.bufferedLen() != 0 or
self.limited.remaining != .nothing)
{
return error.PayloadNotExhausted;
}
if (self.hashing.hasher.final() != self.header.crc32c) {
return error.InvalidChecksum;
}
}
};
/// Encode the portion of a record header covered by CRC32C.
fn encodeChecksumPrefix(
tag: Tag,
payload_len: u32,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
try io.writeInt(writer, u16, @intFromEnum(tag));
try io.writeInt(writer, u32, payload_len);
}
const test_page_header_fixture = test_fixture.parse(
@embedFile("testdata/record-page-header-v1.hex"),
);
test "golden PAGE record header and checksum" {
const page_header =
"\x50\x00\x18\x00\x00\x00\x00\x00" ++
"\x00\x00\x00\x00\x00\x00\x00\x00" ++
"\x00\x00\x00\x00\x00\x00\x00\x00";
var checksum: Checksum = .init(.page, page_header.len);
try checksum.writer().writeAll(page_header);
try std.testing.expectEqual(@as(u32, 0x7178441b), checksum.final());
const header: Header = .{
.tag = .page,
.payload_len = page_header.len,
.crc32c = 0x7178441b,
};
var buf: [Header.len]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
try header.encode(&writer);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/record-page-header-v1.hex",
"snapshot_fixture-record-page-header-v1.hex",
&test_page_header_fixture,
writer.buffered(),
);
var reader: std.Io.Reader = .fixed(&test_page_header_fixture);
try std.testing.expectEqual(header, try Header.decode(&reader));
}
test "reject invalid tags" {
for ([_]u16{ 0, 7, std.math.maxInt(u16) }) |tag| {
var fixture = [_]u8{0} ** Header.len;
std.mem.writeInt(u16, fixture[0..2], tag, .little);
var reader: std.Io.Reader = .fixed(&fixture);
try std.testing.expectError(error.InvalidTag, Header.decode(&reader));
}
}
test "reject every header truncation" {
for (0..Header.len) |len| {
var reader: std.Io.Reader = .fixed(
test_page_header_fixture[0..len],
);
try std.testing.expectError(error.EndOfStream, Header.decode(&reader));
}
}
test "record reader verifies exhaustion and checksum" {
const payload = "snapshot payload";
var checksum: Checksum = .init(.page, payload.len);
try checksum.writer().writeAll(payload);
const header: Header = .{
.tag = .page,
.payload_len = payload.len,
.crc32c = checksum.final(),
};
var encoded: [Header.len + payload.len + 4]u8 = undefined;
var writer: std.Io.Writer = .fixed(&encoded);
try header.encode(&writer);
try writer.writeAll(payload ++ "next");
var source: std.Io.Reader = .fixed(writer.buffered());
var record_reader: Reader = undefined;
try record_reader.init(&source);
var decoded: [payload.len]u8 = undefined;
try record_reader.payloadReader().readSliceAll(&decoded);
try record_reader.finish();
try std.testing.expectEqualStrings(payload, &decoded);
try std.testing.expectEqualStrings("next", try source.take(4));
}
test "record reader rejects remaining bytes and invalid checksum" {
const payload = "payload";
const header: Header = .{
.tag = .page,
.payload_len = payload.len,
.crc32c = 0,
};
var encoded: [Header.len + payload.len]u8 = undefined;
var writer: std.Io.Writer = .fixed(&encoded);
try header.encode(&writer);
try writer.writeAll(payload);
{
var source: std.Io.Reader = .fixed(writer.buffered());
var record_reader: Reader = undefined;
try record_reader.init(&source);
_ = try record_reader.payloadReader().takeByte();
try std.testing.expectError(
error.PayloadNotExhausted,
record_reader.finish(),
);
}
{
var source: std.Io.Reader = .fixed(writer.buffered());
var record_reader: Reader = undefined;
try record_reader.init(&source);
try record_reader.payloadReader().discardAll(payload.len);
try std.testing.expectError(
error.InvalidChecksum,
record_reader.finish(),
);
}
}
test "payload limit does not consume the next record" {
const payload = "ab";
var checksum: Checksum = .init(.page, payload.len);
try checksum.writer().writeAll(payload);
const header: Header = .{
.tag = .page,
.payload_len = payload.len,
.crc32c = checksum.final(),
};
var encoded: [Header.len + payload.len + 4]u8 = undefined;
var writer: std.Io.Writer = .fixed(&encoded);
try header.encode(&writer);
try writer.writeAll(payload ++ "next");
var source: std.Io.Reader = .fixed(writer.buffered());
var record_reader: Reader = undefined;
try record_reader.init(&source);
var too_long: [3]u8 = undefined;
try std.testing.expectError(
error.EndOfStream,
record_reader.payloadReader().readSliceAll(&too_long),
);
try record_reader.finish();
try std.testing.expectEqualStrings("next", try source.take(4));
}
test "record writer buffers a payload and appends framing" {
var destination: std.Io.Writer.Allocating = .init(std.testing.allocator);
defer destination.deinit();
try destination.writer.writeAll("prefix");
var stream: Writer = .init(
std.testing.allocator,
&destination.writer,
);
defer stream.deinit();
const payload = stream.begin(.page);
errdefer stream.cancel();
try payload.writeAll("payload");
try stream.finish();
const encoded = destination.written();
try std.testing.expectEqualStrings("prefix", encoded[0..6]);
var source: std.Io.Reader = .fixed(encoded[6..]);
var record_reader: Reader = undefined;
try record_reader.init(&source);
try std.testing.expectEqual(Tag.page, record_reader.header.tag);
try std.testing.expectEqual(
@as(u32, 7),
record_reader.header.payload_len,
);
try std.testing.expectEqualStrings(
"payload",
try record_reader.payloadReader().take(7),
);
try record_reader.finish();
}
test "record writer cancel preserves preceding bytes" {
var destination: std.Io.Writer.Allocating = .init(std.testing.allocator);
defer destination.deinit();
try destination.writer.writeAll("prefix");
var stream: Writer = .init(
std.testing.allocator,
&destination.writer,
);
defer stream.deinit();
const partial = stream.begin(.page);
errdefer stream.cancel();
try partial.writeAll("partial");
const scratch_capacity = stream.scratch.writer.buffer.len;
stream.cancel();
try std.testing.expectEqualStrings("prefix", destination.written());
try std.testing.expectEqual(
scratch_capacity,
stream.scratch.writer.buffer.len,
);
// Cancel retains the allocation but leaves it ready for the next record.
const next = stream.begin(.page);
try next.writeAll("next");
try stream.finish();
var source: std.Io.Reader = .fixed(destination.written()[6..]);
var record_reader: Reader = undefined;
try record_reader.init(&source);
try std.testing.expectEqualStrings(
"next",
try record_reader.payloadReader().take(4),
);
try record_reader.finish();
}
test "record writer clears active state after destination failure" {
// The header fits but the payload does not, leaving a permitted partial
// record in the destination while the reusable writer becomes idle again.
var destination_bytes: [Header.len]u8 = undefined;
var destination: std.Io.Writer = .fixed(&destination_bytes);
var stream: Writer = .init(std.testing.allocator, &destination);
defer stream.deinit();
const payload = stream.begin(.page);
try payload.writeByte(0);
try std.testing.expectError(error.WriteFailed, stream.finish());
try std.testing.expectEqual(@as(?Tag, null), stream.active_tag);
try std.testing.expectEqual(@as(usize, 0), stream.scratch.writer.end);
}

2299
src/terminal/snapshot/screen.zig Normal file
View File

File diff suppressed because it is too large Load Diff

994
src/terminal/snapshot/snapshot.ksy Normal file
View File

@@ -0,0 +1,994 @@
meta:
id: ghostty_snapshot
title: Ghostty terminal snapshot
application: Ghostty
license: MIT
endian: le
doc: |
Ghostty terminal snapshot format version 1.
A complete snapshot contains an envelope, terminal-wide state, one or two
renderable screen sequences, a READY checkpoint, matching history sequences,
and a FINISH checkpoint. SCREEN pages are oldest-to-newest. HISTORY pages are
newest-to-oldest. FINISH terminates the snapshot; bytes that follow belong to
the containing transport and are outside this schema. Each SCREEN declares
its complete logical history extent before READY.
Record CRC32C values and checkpoint BLAKE3-256 digests are represented here
but cannot be calculated by portable Kaitai Struct expressions. The adjacent
verify-kaitai.py script validates those values after parsing.
seq:
- id: envelope
type: envelope
- id: terminal
type: terminal_record
- id: screens
type: screen_sequence
repeat: expr
repeat-expr: terminal.payload.header.screen_count
- id: ready
type: checkpoint_record(5)
- id: histories
type: history_sequence
repeat: expr
repeat-expr: terminal.payload.header.screen_count
- id: finish
type: checkpoint_record(6)
enums:
record_tag:
1: terminal
2: screen
3: page
4: history
5: ready
6: finish
screen_key:
0: primary
1: alternate
cursor_style:
0: bar
1: block
2: underline
3: block_hollow
status_display:
0: main
1: status_line
shell_redraw:
0: full
1: none
2: last
mouse_event:
0: none
1: x10
2: normal
3: button
4: any
mouse_format:
0: x10
1: utf8
2: sgr
3: urxvt
4: sgr_pixels
mouse_shape:
0: default
1: context_menu
2: help
3: pointer
4: progress
5: wait
6: cell
7: crosshair
8: text
9: vertical_text
10: alias
11: copy
12: move
13: no_drop
14: not_allowed
15: grab
16: grabbing
17: all_scroll
18: col_resize
19: row_resize
20: n_resize
21: e_resize
22: s_resize
23: w_resize
24: ne_resize
25: nw_resize
26: se_resize
27: sw_resize
28: ew_resize
29: ns_resize
30: nesw_resize
31: nwse_resize
32: zoom_in
33: zoom_out
protected_mode:
0: "off"
1: iso
2: dec
semantic_click_kind:
0: none
1: click_events
2: cl
color_kind:
0: none
1: palette
2: rgb
hyperlink_kind:
0: none
1: implicit
2: explicit
cell_content_kind:
0: codepoint
1: background_palette
2: background_rgb
cell_width:
0: narrow
1: wide
2: spacer_tail
3: spacer_head
underline:
0: none
1: single
2: double
3: curly
4: dotted
5: dashed
semantic_content:
0: output
1: input
2: prompt
semantic_prompt:
0: none
1: prompt
2: prompt_continuation
types:
envelope:
doc: Fixed ten-byte snapshot identification and version header.
seq:
- id: magic
contents: [0x47, 0x48, 0x4f, 0x53, 0x54, 0x53, 0x4e, 0x50]
- id: version
type: u2
valid: 1
record_header:
doc: |
Common record framing. crc32c covers the encoded tag, payload length,
and payload bytes, excluding the crc32c field itself.
params:
- id: expected_tag
type: u2
seq:
- id: tag
type: u2
valid: expected_tag
- id: payload_length
type: u4
- id: crc32c
type: u4
terminal_record:
seq:
- id: header
type: record_header(1)
- id: payload
type: terminal_payload
size: header.payload_length
screen_record:
seq:
- id: header
type: record_header(2)
- id: payload
type: screen_payload
size: header.payload_length
page_record:
seq:
- id: header
type: record_header(3)
- id: payload
type: page_payload
size: header.payload_length
history_record:
seq:
- id: header
type: record_header(4)
- id: payload
type: history_payload
size: header.payload_length
checkpoint_record:
params:
- id: expected_tag
type: u2
seq:
- id: header
type: record_header(expected_tag)
- id: payload
type: checkpoint_payload
size: header.payload_length
screen_sequence:
doc: SCREEN followed by its declared PAGE records, oldest-to-newest.
seq:
- id: screen
type: screen_record
- id: pages
type: page_record
repeat: expr
repeat-expr: screen.payload.header.page_count
history_sequence:
doc: HISTORY followed by its declared PAGE records, newest-to-oldest.
seq:
- id: history
type: history_record
- id: pages
type: page_record
repeat: expr
repeat-expr: history.payload.page_count
checkpoint_payload:
seq:
- id: prefix_digest
size: 32
- id: trailing_data
size-eos: true
valid:
expr: _.size == 0
terminal_payload:
seq:
- id: header
type: terminal_header
- id: tab_stops
type: tab_stops(header.columns)
- id: original_palette
type: rgb
repeat: expr
repeat-expr: 256
- id: palette_override_mask
size: 32
- id: palette_overrides
type: palette_override(palette_override_mask, _index)
repeat: expr
repeat-expr: 256
- id: len_pwd
type: u4
- id: pwd
size: len_pwd
- id: len_title
type: u4
- id: title
size: len_title
- id: trailing_data
size-eos: true
valid:
expr: _.size == 0
terminal_header:
seq:
- id: columns
type: u2
valid:
min: 1
- id: rows
type: u2
valid:
min: 1
- id: width_px
type: u4
- id: height_px
type: u4
- id: scrolling_region_top
type: u2
- id: scrolling_region_bottom
type: u2
valid:
expr: _ >= scrolling_region_top and _ < rows
- id: scrolling_region_left
type: u2
- id: scrolling_region_right
type: u2
valid:
expr: _ >= scrolling_region_left and _ < columns
- id: status_display
type: u1
enum: status_display
valid:
expr: _.to_i <= 1
- id: active_screen_key
type: u2
enum: screen_key
valid:
expr: _.to_i <= 1
- id: screen_count
type: u2
valid:
expr: (_ == 1 or _ == 2) and (active_screen_key.to_i == 0 or _ == 2)
- id: previous_codepoint
type: u4
valid:
expr: _ == 0xffffffff or (_ <= 0x10ffff and not (_ >= 0xd800 and _ <= 0xdfff))
- id: cursor_is_default
type: u1
valid:
max: 1
- id: cursor_default_style
type: u1
enum: cursor_style
valid:
expr: _.to_i <= 3
- id: cursor_default_blink
type: u1
valid:
max: 2
- id: shell_redraw
type: u1
enum: shell_redraw
valid:
expr: _.to_i <= 2
- id: modify_other_keys_2
type: u1
valid:
max: 1
- id: mouse_event
type: u1
enum: mouse_event
valid:
expr: _.to_i <= 4
- id: mouse_format
type: u1
enum: mouse_format
valid:
expr: _.to_i <= 4
- id: mouse_shift_capture
type: u1
valid:
max: 2
- id: mouse_shape
type: u1
enum: mouse_shape
valid:
expr: _.to_i <= 33
- id: password_input
type: u1
valid:
max: 1
- id: current_modes
type: mode_set
- id: saved_modes
type: mode_set
- id: default_modes
type: mode_set
- id: background
type: dynamic_rgb
- id: foreground
type: dynamic_rgb
- id: cursor_color
type: dynamic_rgb
- id: max_scrollback_bytes
type: u8
- id: max_scrollback_rows
type: u8
mode_set:
doc: |
The stable packed registry shared by current, saved, and default modes.
Each named instance exposes one bit from the little-endian integer.
Arithmetic division is used instead of bitwise operations because the
JavaScript target implements those operations with signed 32-bit values.
All values remain exact because the registry occupies only 42 bits,
within JavaScript's 53-bit safe integer range.
seq:
- id: raw
type: u8
valid:
max: 4398046511103
instances:
disable_keyboard:
value: (raw / 1) % 2 != 0
insert:
value: (raw / 2) % 2 != 0
send_receive_mode:
value: (raw / 4) % 2 != 0
linefeed:
value: (raw / 8) % 2 != 0
cursor_keys:
value: (raw / 16) % 2 != 0
column_132:
value: (raw / 32) % 2 != 0
slow_scroll:
value: (raw / 64) % 2 != 0
reverse_colors:
value: (raw / 128) % 2 != 0
origin:
value: (raw / 256) % 2 != 0
wraparound:
value: (raw / 512) % 2 != 0
autorepeat:
value: (raw / 1024) % 2 != 0
mouse_event_x10:
value: (raw / 2048) % 2 != 0
cursor_blinking:
value: (raw / 4096) % 2 != 0
cursor_visible:
value: (raw / 8192) % 2 != 0
enable_mode_3:
value: (raw / 16384) % 2 != 0
reverse_wrap:
value: (raw / 32768) % 2 != 0
alt_screen_legacy:
value: (raw / 65536) % 2 != 0
keypad_keys:
value: (raw / 131072) % 2 != 0
backarrow_key_mode:
value: (raw / 262144) % 2 != 0
enable_left_and_right_margin:
value: (raw / 524288) % 2 != 0
mouse_event_normal:
value: (raw / 1048576) % 2 != 0
mouse_event_button:
value: (raw / 2097152) % 2 != 0
mouse_event_any:
value: (raw / 4194304) % 2 != 0
focus_event:
value: (raw / 8388608) % 2 != 0
mouse_format_utf8:
value: (raw / 16777216) % 2 != 0
mouse_format_sgr:
value: (raw / 33554432) % 2 != 0
mouse_alternate_scroll:
value: (raw / 67108864) % 2 != 0
mouse_format_urxvt:
value: (raw / 134217728) % 2 != 0
mouse_format_sgr_pixels:
value: (raw / 268435456) % 2 != 0
ignore_keypad_with_numlock:
value: (raw / 536870912) % 2 != 0
alt_esc_prefix:
value: (raw / 1073741824) % 2 != 0
alt_sends_escape:
value: (raw / 2147483648) % 2 != 0
reverse_wrap_extended:
value: (raw / 4294967296) % 2 != 0
alt_screen:
value: (raw / 8589934592) % 2 != 0
save_cursor:
value: (raw / 17179869184) % 2 != 0
alt_screen_save_cursor_clear_enter:
value: (raw / 34359738368) % 2 != 0
bracketed_paste:
value: (raw / 68719476736) % 2 != 0
synchronized_output:
value: (raw / 137438953472) % 2 != 0
grapheme_cluster:
value: (raw / 274877906944) % 2 != 0
report_color_scheme:
value: (raw / 549755813888) % 2 != 0
report_visibility:
value: (raw / 1099511627776) % 2 != 0
in_band_size_reports:
value: (raw / 2199023255552) % 2 != 0
tab_stops:
params:
- id: columns
type: u2
seq:
- id: complete_bytes
size: columns / 8
- id: partial_byte
type: u1
if: columns % 8 != 0
valid:
expr: _ < (1 << (columns % 8))
rgb:
seq:
- id: red
type: u1
- id: green
type: u1
- id: blue
type: u1
dynamic_rgb:
seq:
- id: default_present
type: u1
valid:
max: 1
- id: default_red
type: u1
valid:
expr: default_present == 1 or _ == 0
- id: default_green
type: u1
valid:
expr: default_present == 1 or _ == 0
- id: default_blue
type: u1
valid:
expr: default_present == 1 or _ == 0
- id: override_present
type: u1
valid:
max: 1
- id: override_red
type: u1
valid:
expr: override_present == 1 or _ == 0
- id: override_green
type: u1
valid:
expr: override_present == 1 or _ == 0
- id: override_blue
type: u1
valid:
expr: override_present == 1 or _ == 0
palette_override:
params:
- id: mask
type: bytes
- id: index
type: u2
seq:
- id: color
type: rgb
if: (mask[index / 8] & (1 << (index % 8))) != 0
screen_payload:
seq:
- id: header
type: screen_header
- id: saved_cursor
type: saved_cursor
if: header.saved_cursor_present == 1
- id: cursor_hyperlink
type: hyperlink(true, false)
- id: trailing_data
size-eos: true
valid:
expr: _.size == 0
screen_header:
seq:
- id: key
type: u2
enum: screen_key
valid:
expr: _.to_i <= 1
- id: page_count
type: u2
valid:
min: 1
- id: history_rows
type: u8
- id: cursor_x
type: u2
- id: cursor_y
type: u2
- id: cursor_style
type: u1
enum: cursor_style
valid:
expr: _.to_i <= 3
- id: cursor_flags
type: cursor_flags
- id: cursor_pen
type: style
- id: hyperlink_implicit_id
type: u4
- id: charset
type: charset_state
- id: protected_mode
type: u1
enum: protected_mode
valid:
expr: _.to_i <= 2
- id: kitty_keyboard_index
type: u1
valid:
max: 7
- id: kitty_keyboard_flags
type: kitty_keyboard_flags
repeat: expr
repeat-expr: 8
- id: semantic_click_kind
type: u1
enum: semantic_click_kind
valid:
expr: _.to_i <= 2
- id: semantic_click_value
type: u1
valid:
expr: |
semantic_click_kind.to_i == 0 ? _ == 0 :
semantic_click_kind.to_i == 1 ? _ <= 1 :
_ <= 3
- id: saved_cursor_present
type: u1
valid:
max: 1
cursor_flags:
seq:
- id: raw
type: u1
valid:
expr: (_ & 0xe0) == 0 and ((_ >> 2) & 0x3) <= 2
instances:
pending_wrap:
value: (raw & (1 << 0)) != 0
protected:
value: (raw & (1 << 1)) != 0
semantic_content:
value: (raw >> 2) & 0x3
semantic_content_clear_eol:
value: (raw & (1 << 4)) != 0
charset_state:
doc: |
Four selected character sets, the GL and GR slots, and an optional
single-shift slot. single_shift is zero for none or one plus a slot.
seq:
- id: raw
type: u2
valid:
expr: (_ & 0x8000) == 0 and ((_ >> 12) & 0x7) <= 4
instances:
g0:
value: (raw >> 0) & 0x3
g1:
value: (raw >> 2) & 0x3
g2:
value: (raw >> 4) & 0x3
g3:
value: (raw >> 6) & 0x3
gl:
value: (raw >> 8) & 0x3
gr:
value: (raw >> 10) & 0x3
single_shift:
value: (raw >> 12) & 0x7
kitty_keyboard_flags:
seq:
- id: raw
type: u1
valid:
expr: (_ & 0xe0) == 0
instances:
disambiguate:
value: (raw & (1 << 0)) != 0
report_events:
value: (raw & (1 << 1)) != 0
report_alternates:
value: (raw & (1 << 2)) != 0
report_all:
value: (raw & (1 << 3)) != 0
report_associated:
value: (raw & (1 << 4)) != 0
saved_cursor:
seq:
- id: x
type: u2
- id: y
type: u2
- id: pen
type: style
- id: flags
type: saved_cursor_flags
- id: charset
type: charset_state
saved_cursor_flags:
seq:
- id: raw
type: u1
valid:
expr: (_ & 0xf8) == 0
instances:
protected:
value: (raw & (1 << 0)) != 0
pending_wrap:
value: (raw & (1 << 1)) != 0
origin:
value: (raw & (1 << 2)) != 0
history_payload:
seq:
- id: key
type: u2
enum: screen_key
valid:
expr: _.to_i <= 1
- id: page_count
type: u4
- id: trailing_data
size-eos: true
valid:
expr: _.size == 0
page_payload:
seq:
- id: header
type: page_header
- id: styles
type: style_table_entry
repeat: expr
repeat-expr: header.style_count
- id: hyperlinks
type: hyperlink_table_entry
repeat: expr
repeat-expr: header.hyperlink_count
- id: grid
type: grid(header.rows, header.columns)
- id: trailing_data
size-eos: true
valid:
expr: _.size == 0
page_header:
seq:
- id: columns
type: u2
valid:
min: 1
- id: rows
type: u2
valid:
min: 1
- id: style_count
type: u2
- id: hyperlink_count
type: u2
- id: style_capacity
type: u2
- id: hyperlink_capacity_bytes
type: u2
- id: grapheme_capacity_bytes
type: u4
- id: string_capacity_bytes
type: u4
style_table_entry:
seq:
- id: encoded_id
type: u2
valid:
min: 1
- id: value
type: style
hyperlink_table_entry:
seq:
- id: encoded_id
type: u2
valid:
min: 1
- id: value
type: hyperlink(false, true)
style:
seq:
- id: foreground
type: style_color
- id: background
type: style_color
- id: underline_color
type: style_color
- id: flags
type: style_flags
- id: reserved
type: u2
valid: 0
style_flags:
seq:
- id: raw
type: u2
valid:
expr: (_ & 0xf800) == 0 and ((_ >> 8) & 0x7) <= 5
instances:
bold:
value: (raw & (1 << 0)) != 0
italic:
value: (raw & (1 << 1)) != 0
faint:
value: (raw & (1 << 2)) != 0
blink:
value: (raw & (1 << 3)) != 0
inverse:
value: (raw & (1 << 4)) != 0
invisible:
value: (raw & (1 << 5)) != 0
strikethrough:
value: (raw & (1 << 6)) != 0
overline:
value: (raw & (1 << 7)) != 0
underline:
value: (raw >> 8) & 0x7
style_color:
seq:
- id: kind
type: u1
enum: color_kind
valid:
expr: _.to_i <= 2
- id: first
type: u1
valid:
expr: kind.to_i != 0 or _ == 0
- id: second
type: u1
valid:
expr: kind.to_i == 2 or _ == 0
- id: third
type: u1
valid:
expr: kind.to_i == 2 or _ == 0
hyperlink:
params:
- id: allow_none
type: bool
- id: allow_empty
type: bool
seq:
- id: kind
type: u1
enum: hyperlink_kind
valid:
expr: _.to_i <= 2 and (allow_none or _.to_i != 0)
- id: implicit
type: implicit_hyperlink(allow_empty)
if: kind.to_i == 1
- id: explicit
type: explicit_hyperlink(allow_empty)
if: kind.to_i == 2
implicit_hyperlink:
params:
- id: allow_empty
type: bool
seq:
- id: id
type: u4
- id: len_uri
type: u4
valid:
expr: allow_empty or _ >= 1
- id: uri
size: len_uri
explicit_hyperlink:
params:
- id: allow_empty
type: bool
seq:
- id: len_id
type: u4
valid:
expr: allow_empty or _ >= 1
- id: id
size: len_id
- id: len_uri
type: u4
valid:
expr: allow_empty or _ >= 1
- id: uri
size: len_uri
grid:
params:
- id: num_rows
type: u2
- id: columns
type: u2
seq:
- id: rows
type: grid_row(columns)
repeat: expr
repeat-expr: num_rows
grid_row:
params:
- id: num_cells
type: u2
seq:
- id: flags
type: grid_row_flags
- id: cells
type: grid_cell(_index, num_cells, flags.wrap)
repeat: expr
repeat-expr: num_cells
grid_row_flags:
seq:
- id: raw
type: u1
valid:
expr: (_ & 0xf0) == 0 and ((_ >> 2) & 0x3) <= 2
instances:
wrap:
value: (raw & 1) != 0
wrap_continuation:
value: (raw & 2) != 0
semantic_prompt:
value: (raw >> 2) & 0x3
grid_cell:
params:
- id: index
type: u2
- id: columns
type: u2
- id: row_wrap
type: bool
seq:
- id: content_kind
type: u1
valid:
max: 2
- id: width
type: u1
valid:
expr: |
_ <= 3 and
(_ != 2 or
(index > 0 and _parent.cells[index - 1].width == 1)) and
(_ != 3 or
(index + 1 == columns and row_wrap))
- id: flags
type: grid_cell_flags
- id: reserved
type: u1
valid: 0
- id: style_id
type: u2
- id: hyperlink_id
type: u2
- id: value
type: u4
valid:
expr: |
content_kind == 0 ?
(_ <= 0x10ffff and not (_ >= 0xd800 and _ <= 0xdfff) and _ != 0x10eeee) :
content_kind == 1 ?
_ <= 0xff :
_ <= 0xffffff
- id: num_graphemes
type: u4
valid:
expr: |
content_kind == 0 ?
(_ == 0 or value != 0) :
_ == 0
- id: graphemes
type: u4
repeat: expr
repeat-expr: num_graphemes
valid:
expr: _ <= 0x10ffff and not (_ >= 0xd800 and _ <= 0xdfff) and _ != 0x10eeee
grid_cell_flags:
seq:
- id: raw
type: u1
valid:
expr: (_ & 0xf8) == 0 and ((_ >> 1) & 0x3) <= 2
instances:
protected:
value: (raw & (1 << 0)) != 0
semantic_content:
value: (raw >> 1) & 0x3

773
src/terminal/snapshot/snapshot.zig Normal file
View File

@@ -0,0 +1,773 @@
//! Complete terminal snapshot encoding and restoration.
const std = @import("std");
const build_options = @import("terminal_options");
const Allocator = std.mem.Allocator;
const checkpoint = @import("checkpoint.zig");
const envelope = @import("envelope.zig");
const test_fixture = @import("fixture.zig");
const history = @import("history.zig");
const record = @import("record.zig");
const screen = @import("screen.zig");
const terminal = @import("terminal.zig");
const Terminal = @import("../Terminal.zig");
const terminal_kitty = @import("../kitty.zig");
const TerminalPageList = @import("../PageList.zig");
const TerminalScreen = @import("../Screen.zig");
const TerminalScreenKey = @import("../ScreenSet.zig").Key;
const test_complete_fixture = test_fixture.parse(
@embedFile("testdata/complete-v1.hex"),
);
/// Errors possible while encoding one complete terminal snapshot.
pub const EncodeError = terminal.EncodeError ||
screen.EncodeError ||
history.EncodeError ||
checkpoint.EncodeError;
/// Encode one complete terminal snapshot.
///
/// Encoding starts at the destination's current position. Only one record
/// payload is buffered at a time; completed records stream immediately. On
/// failure, the destination may contain a snapshot prefix without its required
/// checkpoints, and an output failure may have written part of a record.
pub fn encode(
alloc: Allocator,
destination: *std.Io.Writer,
t: *const Terminal,
) EncodeError!void {
var stream: record.Writer = .init(alloc, destination);
defer stream.deinit();
// 1. Envelope
try envelope.encode(stream.writer());
// 2. Terminal
try terminal.encode(t, &stream);
// 3. Primary and alt screen
try screen.encode(
t.screens.get(.primary).?,
.primary,
&stream,
);
if (t.screens.get(.alternate)) |alternate| try screen.encode(
alternate,
.alternate,
&stream,
);
// 4. Ready checkpoint. In the future we'll put our continuation
// state before this so pty bytes can also flow.
try checkpoint.encode(.ready, &stream);
// 5. History
try history.encode(
t.screens.get(.primary).?,
.primary,
&stream,
);
if (t.screens.get(.alternate)) |alternate| try history.encode(
alternate,
.alternate,
&stream,
);
// 6. Finish
try checkpoint.encode(.finish, &stream);
}
/// Errors possible while restoring one complete terminal snapshot.
pub const DecodeError = envelope.DecodeError ||
terminal.DecodeError ||
screen.DecodeError ||
history.DecodeError ||
checkpoint.DecodeError ||
error{
/// A SCREEN names a key not declared by TERMINAL.
UnexpectedScreenKey,
/// More than one SCREEN names the same key.
DuplicateScreen,
/// A HISTORY names a key not declared by TERMINAL.
UnexpectedHistoryKey,
/// More than one HISTORY names the same key.
DuplicateHistory,
};
/// Restore one complete snapshot into a native terminal.
///
/// This consumes one snapshot through FINISH and leaves any following bytes in
/// the reader for the containing transport. Restoration is transactional: the
/// returned terminal is either complete and ready or not (error return).
/// Individual record codecs normalize optional semantic state, while framing,
/// checkpoints, declared sequence counts, and unique cross-record screen
/// routing remain strict.
pub fn decode(
alloc: Allocator,
io_: std.Io,
source: *std.Io.Reader,
) DecodeError!Terminal {
// StreamReader owns a zero-buffer hashing adapter, making checkpoint
// boundaries part of its API rather than a caller-maintained invariant.
var stream: record.StreamReader = .init(source);
const reader = stream.reader();
// Read the envelope, which is currently just a verification step.
try envelope.decode(reader);
// TERMINAL establishes terminal-wide state and allocates empty screen
// slots with their final routing. SCREEN values replace those slots in
// place so ScreenSet pointers, including the active pointer, stay valid.
var result = try terminal.decode(reader, io_, alloc);
errdefer result.deinit(alloc);
// TERMINAL initializes exactly the number of screen slots it declared.
// Decode that many SCREEN sequences and route each one by its encoded key.
const screen_count = result.screens.all.count();
const options: TerminalScreen.Options = options: {
const primary = result.screens.get(.primary).?;
const explicit_bytes = primary.pages.limits.bytes.explicit;
const explicit_lines = primary.pages.limits.lines.explicit;
break :options .{
.cols = result.cols,
.rows = result.rows,
.max_scrollback_bytes = if (explicit_bytes == std.math.maxInt(usize))
null
else
explicit_bytes,
.max_scrollback_lines = if (explicit_lines == std.math.maxInt(usize))
null
else
explicit_lines,
};
};
for (0..screen_count) |_| {
var decoded = try screen.decode(
reader,
io_,
alloc,
options,
);
errdefer decoded.deinit();
const slot = result.screens.get(decoded.key) orelse
return error.UnexpectedScreenKey;
// The fresh ScreenSet starts every declared slot at generation zero.
// Replacing a slot advances its generation, so a nonzero value means
// an earlier SCREEN in this snapshot already supplied the same key.
if (result.screens.generation(decoded.key) != 0) return error.DuplicateScreen;
slot.deinit();
slot.* = decoded.screen;
decoded.screen = undefined;
// We put an artificial generation in just so we can detect duplicates.
result.screens.generations.put(
decoded.key,
result.screens.generation(decoded.key) +% 1,
);
}
// READY covers the exact envelope-through-SCREEN prefix. Finalizing does
// not consume the hasher, so the same stream continues toward FINISH.
try checkpoint.decode(.ready, &stream);
// HISTORY keys make this sequence order-independent just like SCREEN.
// Although a decoder may publish recent pages as they validate, any later
// full-snapshot failure deinitializes the whole result.
for (0..screen_count) |_| {
var decoder: history.Decoder = undefined;
try decoder.init(reader);
const key = decoder.header.key;
const restored = result.screens.get(key) orelse
return error.UnexpectedHistoryKey;
// SCREEN routing advanced every decoded slot from generation zero to
// one. A value other than one means this key already received HISTORY.
if (result.screens.generation(key) != 1) {
return error.DuplicateHistory;
}
try decoder.decode(alloc, restored);
result.screens.generations.put(key, 2);
}
// FINISH authenticates READY and all history. Decode it directly from the
// underlying reader so the digest does not include FINISH itself.
try checkpoint.decode(.finish, &stream);
const keys = [_]TerminalScreenKey{ .primary, .alternate };
if (comptime build_options.slow_runtime_safety) {
for (keys) |key| {
const restored = result.screens.get(key) orelse continue;
restored.pages.assertIntegrity();
restored.assertIntegrity();
}
}
// Generations are only scratch state while routing the two keyed sequence
// groups. The completed terminal has not escaped yet, so reset them to the
// same initial state as any newly constructed ScreenSet.
for (keys) |key| result.screens.generations.put(key, 0);
return result;
}
/// Errors possible while restoring a snapshot that must end at end-of-file.
pub const DecodeExactError = DecodeError || std.Io.Reader.Error || error{
/// FINISH was followed by additional bytes.
TrailingData,
};
/// Restore one snapshot and require FINISH to be followed by end-of-file.
///
/// This is intended for bounded snapshot files and buffers. On a live stream,
/// checking for end-of-file may block; use `decode` to stop at FINISH instead.
pub fn decodeExact(
alloc: Allocator,
io_: std.Io,
source: *std.Io.Reader,
) DecodeExactError!Terminal {
var result = try decode(alloc, io_, source);
errdefer result.deinit(alloc);
_ = source.peekByte() catch |err| switch (err) {
error.EndOfStream => return result,
else => return err,
};
return error.TrailingData;
}
test "complete snapshot round trip with history and alternate screen" {
const testing = std.testing;
var t = try Terminal.init(testing.io, testing.allocator, .{
.cols = 2,
.rows = 3,
.max_scrollback_bytes = null,
.max_scrollback_lines = null,
});
defer t.deinit(testing.allocator);
// Exercise terminal-wide state.
t.width_px = 800;
t.height_px = 600;
t.colors.palette.set(7, .{ .r = 1, .g = 2, .b = 3 });
t.modes.values.bracketed_paste = true;
try t.setPwd("file:///tmp/snapshot");
try t.setTitle("complete snapshot");
const primary = t.screens.get(.primary).?;
// Use small exact capacities so this compound golden remains practical to
// review while still containing two complete history pages and one active
// page. Replacing the Screen in place preserves ScreenSet routing.
var replacement: TerminalScreen = replacement: {
var builder = try TerminalPageList.Builder.init(
testing.allocator,
.{
.cols = t.cols,
.rows = t.rows,
.max_size = null,
.max_lines = null,
},
);
defer builder.deinit();
const oldest = try builder.allocatePage(.{ .cols = 2, .rows = 2 });
oldest.size.rows = 2;
oldest.getRowAndCell(0, 0).cell.* = .init('A');
const recent = try builder.allocatePage(.{ .cols = 2, .rows = 2 });
recent.size.rows = 2;
recent.getRowAndCell(0, 0).cell.* = .init('B');
const active = try builder.allocatePage(.{ .cols = 2, .rows = 3 });
active.size.rows = 3;
active.getRowAndCell(0, 0).cell.* = .init('C');
active.getRowAndCell(0, 1).cell.* = .init('D');
active.getRowAndCell(0, 2).cell.* = .init('E');
var pages = try builder.finish();
errdefer pages.deinit();
const cursor_pin = try pages.trackPin(
pages.pin(.{ .active = .{} }).?,
);
const cursor_rac = cursor_pin.rowAndCell();
break :replacement .{
.io = testing.io,
.alloc = testing.allocator,
.pages = pages,
.cursor = .{
.page_pin = cursor_pin,
.page_row = cursor_rac.row,
.page_cell = cursor_rac.cell,
},
};
};
primary.deinit();
primary.* = replacement;
replacement = undefined;
try testing.expect(primary.pages.scrollbar().total > t.rows);
// Compression is an internal source representation and must remain
// unchanged while the complete history is inspected for encoding.
_ = primary.pages.compress(.full);
const source_memory = primary.pages.memoryStats();
// The optional alternate screen participates in both phases and remains
// the active screen after restoration.
_ = try t.switchScreen(.alternate);
try t.printString("alternate");
var encoded: std.Io.Writer.Allocating = .init(testing.allocator);
defer encoded.deinit();
try encode(testing.allocator, &encoded.writer, &t);
try testing.expectEqualDeep(source_memory, primary.pages.memoryStats());
try test_fixture.expectEqual(
.snapshot,
"src/terminal/snapshot/testdata/complete-v1.hex",
"snapshot_fixture-complete-v1.hex",
&test_complete_fixture,
encoded.written(),
);
// A complete snapshot can stream through a non-allocating destination.
// Independently hash that output so both its length and complete byte
// sequence are checked without retaining a second snapshot copy.
var discard: std.Io.Writer.Discarding = .init(&.{});
var hashing = discard.writer.hashed(
std.crypto.hash.Blake3.init(.{}),
&.{},
);
try encode(testing.allocator, &hashing.writer, &t);
try testing.expectEqual(
@as(u64, test_complete_fixture.len),
discard.fullCount(),
);
var expected_digest: checkpoint.Digest = undefined;
std.crypto.hash.Blake3.hash(
&test_complete_fixture,
&expected_digest,
.{},
);
var actual_digest: checkpoint.Digest = undefined;
hashing.hasher.final(&actual_digest);
try testing.expectEqual(expected_digest, actual_digest);
// Restore the checked-in reference rather than the just-generated bytes.
var encoded_source: std.Io.Reader = .fixed(&test_complete_fixture);
var source_buffer: [1]u8 = undefined;
var limited = encoded_source.limited(.unlimited, &source_buffer);
var restored = try decode(
testing.allocator,
testing.io,
&limited.interface,
);
defer restored.deinit(testing.allocator);
try testing.expectEqual(TerminalScreenKey.alternate, restored.screens.active_key);
try testing.expectEqual(
restored.screens.get(.alternate).?,
restored.screens.active,
);
try testing.expectEqualStrings(
"file:///tmp/snapshot",
restored.getPwd().?,
);
try testing.expectEqualStrings(
"complete snapshot",
restored.getTitle().?,
);
try testing.expectEqual(
primary.pages.scrollbar().total,
restored.screens.get(.primary).?.pages.scrollbar().total,
);
// Re-encoding is a compact semantic equality check over all TERMINAL,
// SCREEN, PAGE, and HISTORY fields and both checkpoint boundaries.
var reencoded: std.Io.Writer.Allocating = .init(testing.allocator);
defer reencoded.deinit();
try encode(testing.allocator, &reencoded.writer, &restored);
try testing.expectEqualStrings(
&test_complete_fixture,
reencoded.written(),
);
// SCREEN and HISTORY keys make both sequence groups order independent.
var reversed: std.Io.Writer.Allocating = .init(testing.allocator);
defer reversed.deinit();
var reversed_stream: record.Writer = .init(
testing.allocator,
&reversed.writer,
);
defer reversed_stream.deinit();
try envelope.encode(reversed_stream.writer());
try terminal.encode(&t, &reversed_stream);
try screen.encode(
t.screens.get(.alternate).?,
.alternate,
&reversed_stream,
);
try screen.encode(primary, .primary, &reversed_stream);
try checkpoint.encode(.ready, &reversed_stream);
try history.encode(
t.screens.get(.alternate).?,
.alternate,
&reversed_stream,
);
try history.encode(primary, .primary, &reversed_stream);
try checkpoint.encode(.finish, &reversed_stream);
var reversed_source: std.Io.Reader = .fixed(reversed.written());
var reversed_restored = try decode(
testing.allocator,
testing.io,
&reversed_source,
);
defer reversed_restored.deinit(testing.allocator);
try testing.expectEqual(
TerminalScreenKey.alternate,
reversed_restored.screens.active_key,
);
try testing.expectEqual(
@as(usize, 0),
reversed_restored.screens.generation(.primary),
);
try testing.expectEqual(
@as(usize, 0),
reversed_restored.screens.generation(.alternate),
);
}
test "complete snapshot preserves Kitty virtual placeholders" {
if (comptime !build_options.kitty_graphics) return error.SkipZigTest;
const testing = std.testing;
var t = try Terminal.init(testing.io, testing.allocator, .{
.cols = 2,
.rows = 1,
});
defer t.deinit(testing.allocator);
// Register a real virtual placement, then write its grid representation:
// U+10EEEE followed by row and column diacritics. The image and placement
// registry is intentionally omitted, but the grid content must remain
// decodable.
try t.screens.active.kitty_images.addImage(
testing.io,
testing.allocator,
.{ .id = 1 },
);
try t.screens.active.kitty_images.addPlacement(
testing.io,
testing.allocator,
1,
0,
.{
.location = .{ .virtual = {} },
.columns = 1,
.rows = 1,
},
);
try t.setAttribute(.{ .@"256_fg" = 1 });
try t.printString("\u{10EEEE}\u{0305}\u{0305}");
const source_cell = t.screens.active.pages.getCell(.{
.screen = .{},
}).?;
try testing.expectEqual(
terminal_kitty.graphics.unicode.placeholder,
source_cell.cell.codepoint(),
);
try testing.expect(source_cell.row.kitty_virtual_placeholder);
var encoded: std.Io.Writer.Allocating = .init(testing.allocator);
defer encoded.deinit();
try encode(testing.allocator, &encoded.writer, &t);
var encoded_source: std.Io.Reader = .fixed(encoded.written());
var restored = try decode(
testing.allocator,
testing.io,
&encoded_source,
);
defer restored.deinit(testing.allocator);
const restored_cell = restored.screens.active.pages.getCell(.{
.screen = .{},
}).?;
try testing.expectEqual(
terminal_kitty.graphics.unicode.placeholder,
restored_cell.cell.codepoint(),
);
try testing.expect(restored_cell.cell.hasGrapheme());
try testing.expect(restored_cell.row.kitty_virtual_placeholder);
try testing.expectEqual(
@as(usize, 0),
restored.screens.active.kitty_images.images.count(),
);
try testing.expectEqual(
@as(usize, 0),
restored.screens.active.kitty_images.placements.count(),
);
}
test "complete snapshot encoding streams from the current writer position" {
const testing = std.testing;
var t = try Terminal.init(testing.io, testing.allocator, .{
.cols = 2,
.rows = 1,
});
defer t.deinit(testing.allocator);
// Prefix hashing begins with this call's envelope, independent of bytes
// that were already present in the destination.
var nonempty: std.Io.Writer.Allocating = .init(testing.allocator);
defer nonempty.deinit();
try nonempty.writer.writeAll("prefix");
const snapshot_offset = nonempty.written().len;
try encode(testing.allocator, &nonempty.writer, &t);
try testing.expectEqualStrings(
"prefix",
nonempty.written()[0..snapshot_offset],
);
var appended_source: std.Io.Reader = .fixed(
nonempty.written()[snapshot_offset..],
);
var appended = try decode(
testing.allocator,
testing.io,
&appended_source,
);
appended.deinit(testing.allocator);
// Payload validation happens in the record-local scratch allocation. The
// already-streamed envelope remains, but no partial TERMINAL is emitted.
t.colors.palette.current[7] = .{ .r = 1, .g = 2, .b = 3 };
var destination: std.Io.Writer.Allocating = .init(testing.allocator);
defer destination.deinit();
try destination.writer.writeAll("prefix");
try testing.expectError(
error.InvalidPalette,
encode(testing.allocator, &destination.writer, &t),
);
var expected_envelope: [envelope.encoded_len]u8 = undefined;
var envelope_writer: std.Io.Writer = .fixed(&expected_envelope);
try envelope.encode(&envelope_writer);
try testing.expectEqualStrings(
&expected_envelope,
destination.written()["prefix".len..],
);
}
test "complete snapshot rejects ordering and invalid checkpoints" {
const testing = std.testing;
var t = try Terminal.init(testing.io, testing.allocator, .{
.cols = 2,
.rows = 1,
});
defer t.deinit(testing.allocator);
const primary = t.screens.get(.primary).?;
// HISTORY is individually valid here, but the full decoder requires the
// primary SCREEN before READY.
var reordered: std.Io.Writer.Allocating = .init(testing.allocator);
defer reordered.deinit();
var reordered_stream: record.Writer = .init(
testing.allocator,
&reordered.writer,
);
defer reordered_stream.deinit();
try envelope.encode(reordered_stream.writer());
try terminal.encode(&t, &reordered_stream);
try history.encode(primary, .primary, &reordered_stream);
var reordered_source: std.Io.Reader = .fixed(reordered.written());
try testing.expectError(
error.UnexpectedRecordTag,
decode(testing.allocator, testing.io, &reordered_source),
);
// Construct a correctly framed READY with an intentionally unrelated
// digest so the full driver, rather than record CRC validation, rejects it.
var invalid_ready: std.Io.Writer.Allocating = .init(testing.allocator);
defer invalid_ready.deinit();
var invalid_ready_stream: record.Writer = .init(
testing.allocator,
&invalid_ready.writer,
);
defer invalid_ready_stream.deinit();
try envelope.encode(invalid_ready_stream.writer());
try terminal.encode(&t, &invalid_ready_stream);
try screen.encode(primary, .primary, &invalid_ready_stream);
const ready_payload = invalid_ready_stream.begin(.ready);
errdefer invalid_ready_stream.cancel();
try ready_payload.splatByteAll(
0,
@sizeOf(checkpoint.Digest),
);
try invalid_ready_stream.finish();
var invalid_ready_source: std.Io.Reader = .fixed(
invalid_ready.written(),
);
try testing.expectError(
error.InvalidDigest,
decode(testing.allocator, testing.io, &invalid_ready_source),
);
// A SCREEN key must name one of the slots declared by TERMINAL.
var undeclared: std.Io.Writer.Allocating = .init(testing.allocator);
defer undeclared.deinit();
var undeclared_stream: record.Writer = .init(
testing.allocator,
&undeclared.writer,
);
defer undeclared_stream.deinit();
try envelope.encode(undeclared_stream.writer());
try terminal.encode(&t, &undeclared_stream);
try screen.encode(primary, .alternate, &undeclared_stream);
var undeclared_source: std.Io.Reader = .fixed(undeclared.written());
try testing.expectError(
error.UnexpectedScreenKey,
decode(testing.allocator, testing.io, &undeclared_source),
);
// HISTORY sequences are also routed by key, which must name a declared
// screen even when the sequence contains no PAGE records.
var undeclared_history: std.Io.Writer.Allocating = .init(testing.allocator);
defer undeclared_history.deinit();
var undeclared_history_stream: record.Writer = .init(
testing.allocator,
&undeclared_history.writer,
);
defer undeclared_history_stream.deinit();
try envelope.encode(undeclared_history_stream.writer());
try terminal.encode(&t, &undeclared_history_stream);
try screen.encode(primary, .primary, &undeclared_history_stream);
try checkpoint.encode(.ready, &undeclared_history_stream);
try history.encode(primary, .alternate, &undeclared_history_stream);
var undeclared_history_source: std.Io.Reader = .fixed(
undeclared_history.written(),
);
try testing.expectError(
error.UnexpectedHistoryKey,
decode(testing.allocator, testing.io, &undeclared_history_source),
);
// The declared count cannot be satisfied by repeating the same key.
_ = try t.switchScreen(.alternate);
var duplicate: std.Io.Writer.Allocating = .init(testing.allocator);
defer duplicate.deinit();
var duplicate_stream: record.Writer = .init(
testing.allocator,
&duplicate.writer,
);
defer duplicate_stream.deinit();
try envelope.encode(duplicate_stream.writer());
try terminal.encode(&t, &duplicate_stream);
try screen.encode(primary, .primary, &duplicate_stream);
try screen.encode(primary, .primary, &duplicate_stream);
var duplicate_source: std.Io.Reader = .fixed(duplicate.written());
try testing.expectError(
error.DuplicateScreen,
decode(testing.allocator, testing.io, &duplicate_source),
);
// The declared count cannot be satisfied by repeating one HISTORY key.
var duplicate_history: std.Io.Writer.Allocating = .init(testing.allocator);
defer duplicate_history.deinit();
var duplicate_history_stream: record.Writer = .init(
testing.allocator,
&duplicate_history.writer,
);
defer duplicate_history_stream.deinit();
try envelope.encode(duplicate_history_stream.writer());
try terminal.encode(&t, &duplicate_history_stream);
try screen.encode(primary, .primary, &duplicate_history_stream);
try screen.encode(
t.screens.get(.alternate).?,
.alternate,
&duplicate_history_stream,
);
try checkpoint.encode(.ready, &duplicate_history_stream);
try history.encode(primary, .primary, &duplicate_history_stream);
try history.encode(primary, .primary, &duplicate_history_stream);
var duplicate_history_source: std.Io.Reader = .fixed(
duplicate_history.written(),
);
try testing.expectError(
error.DuplicateHistory,
decode(testing.allocator, testing.io, &duplicate_history_source),
);
}
test "complete snapshot leaves continuation bytes unread" {
const testing = std.testing;
var t = try Terminal.init(testing.io, testing.allocator, .{
.cols = 2,
.rows = 1,
});
defer t.deinit(testing.allocator);
var encoded: std.Io.Writer.Allocating = .init(testing.allocator);
defer encoded.deinit();
try encode(testing.allocator, &encoded.writer, &t);
const snapshot_len = encoded.written().len;
try encoded.writer.writeAll("pty");
var source: std.Io.Reader = .fixed(encoded.written());
var restored = try decode(testing.allocator, testing.io, &source);
defer restored.deinit(testing.allocator);
var continuation: [3]u8 = undefined;
try source.readSliceAll(&continuation);
try testing.expectEqualStrings("pty", &continuation);
var exact_source: std.Io.Reader = .fixed(encoded.written());
try testing.expectError(
error.TrailingData,
decodeExact(testing.allocator, testing.io, &exact_source),
);
var bounded_source: std.Io.Reader = .fixed(
encoded.written()[0..snapshot_len],
);
var bounded = try decodeExact(
testing.allocator,
testing.io,
&bounded_source,
);
defer bounded.deinit(testing.allocator);
}
test "complete snapshots decode sequentially from one reader" {
const testing = std.testing;
var t = try Terminal.init(testing.io, testing.allocator, .{
.cols = 2,
.rows = 1,
});
defer t.deinit(testing.allocator);
var encoded: std.Io.Writer.Allocating = .init(testing.allocator);
defer encoded.deinit();
try encode(testing.allocator, &encoded.writer, &t);
try encode(testing.allocator, &encoded.writer, &t);
var source: std.Io.Reader = .fixed(encoded.written());
var first = try decode(testing.allocator, testing.io, &source);
defer first.deinit(testing.allocator);
var second = try decode(testing.allocator, testing.io, &source);
defer second.deinit(testing.allocator);
try testing.expectError(error.EndOfStream, source.takeByte());
}

392
src/terminal/snapshot/style.zig Normal file
View File

@@ -0,0 +1,392 @@
//! Snapshot style entry encoding.
//!
//! Each entry describes one terminal style. A record can use these entries to
//! build a style table and assign indexes according to that record's format.
//! Indexing, ordering, etc. are properties of the containing record.
//!
//! Styles are encoded field by field from the terminal's native `Style` type.
//! The native packed representation is deliberately not part of the snapshot
//! format. This gives flexibility for changing one side or the other.
//!
//! All integers are unsigned and little-endian.
//!
//! | Offset | Size | Field |
//! | -----: | ---: | :------------------------ |
//! | 0 | 4 | Foreground color |
//! | 4 | 4 | Background color |
//! | 8 | 4 | Underline color |
//! | 12 | 2 | Style flags (`u16`) |
//! | 14 | 2 | Reserved, must be zero |
//!
//! The trailing reserved field is explicit wire padding that rounds each
//! style entry from 14 bytes to 16 bytes. This makes entry offsets and byte
//! counts straightforward.
//!
//! Each color begins with a one-byte kind followed by three data bytes:
//!
//! | Kind | Meaning | Data bytes |
//! | ---: | :------ | :--------------------------------- |
//! | 0 | None | All zero |
//! | 1 | Palette | Palette index, then two zero bytes |
//! | 2 | RGB | Red, green, blue |
//!
//! Style flag bits 0 through 7 are bold, italic, faint, blink, inverse,
//! invisible, strikethrough, and overline. Bits 8 through 10 contain the
//! underline kind. Bits 11 through 15 must be zero.
//!
//! | Underline | Meaning |
//! | --------: | :------ |
//! | 0 | None |
//! | 1 | Single |
//! | 2 | Double |
//! | 3 | Curly |
//! | 4 | Dotted |
//! | 5 | Dashed |
//!
//! Underline values 6 and 7 are invalid in snapshot version 1.
const std = @import("std");
const test_fixture = @import("fixture.zig");
const io = @import("io.zig");
const sgr = @import("../sgr.zig");
const terminal_style = @import("../style.zig");
/// Number of bytes written by `encode`, calculated using the encoder itself
/// so this remains synchronized with the field-by-field wire format.
pub const len = computeLen();
comptime {
// This size is part of the wire format. If it changes, the snapshot
// version and golden fixtures must also change.
std.debug.assert(len == 16);
}
const Flags = packed struct(u16) {
bold: bool = false,
italic: bool = false,
faint: bool = false,
blink: bool = false,
inverse: bool = false,
invisible: bool = false,
strikethrough: bool = false,
overline: bool = false,
underline: u3 = 0,
reserved: u5 = 0,
};
const ColorKind = enum(u8) {
none = 0,
palette = 1,
rgb = 2,
};
/// Errors possible while decoding one style entry.
pub const DecodeError = std.Io.Reader.Error || error{
/// A color kind is not defined by snapshot version 1.
InvalidColorKind,
/// Bytes unused by the selected color kind are not zero.
InvalidColor,
/// The encoded underline kind is not defined by snapshot version 1.
InvalidUnderline,
/// One or more reserved style flag bits are set.
InvalidFlags,
/// The trailing reserved field is not zero.
InvalidReserved,
};
/// Encode one terminal style as a fixed-size snapshot style entry.
pub fn encode(
value: terminal_style.Style,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
try encodeColor(value.fg_color, writer);
try encodeColor(value.bg_color, writer);
try encodeColor(value.underline_color, writer);
const flags: Flags = .{
.bold = value.flags.bold,
.italic = value.flags.italic,
.faint = value.flags.faint,
.blink = value.flags.blink,
.inverse = value.flags.inverse,
.invisible = value.flags.invisible,
.strikethrough = value.flags.strikethrough,
.overline = value.flags.overline,
.underline = @intFromEnum(value.flags.underline),
};
try io.writeInt(writer, u16, @bitCast(flags));
try io.writeInt(writer, u16, 0);
}
/// Decode and validate one fixed-size snapshot style entry.
pub fn decode(reader: *std.Io.Reader) DecodeError!terminal_style.Style {
const fg_color = try decodeColor(reader);
const bg_color = try decodeColor(reader);
const underline_color = try decodeColor(reader);
const flags: Flags = @bitCast(try io.readInt(reader, u16));
if (flags.reserved != 0) return error.InvalidFlags;
const underline = std.enums.fromInt(
sgr.Attribute.Underline,
flags.underline,
) orelse return error.InvalidUnderline;
const reserved = try io.readInt(reader, u16);
if (reserved != 0) return error.InvalidReserved;
return .{
.fg_color = fg_color,
.bg_color = bg_color,
.underline_color = underline_color,
.flags = .{
.bold = flags.bold,
.italic = flags.italic,
.faint = flags.faint,
.blink = flags.blink,
.inverse = flags.inverse,
.invisible = flags.invisible,
.strikethrough = flags.strikethrough,
.overline = flags.overline,
.underline = underline,
},
};
}
/// Decode strictly after consuming one complete fixed-size style entry.
///
/// Unlike `decode`, a semantic error leaves `reader` at the next entry. This
/// lets an enclosing codec catch the error and choose its own fallback without
/// losing the surrounding payload boundary.
pub fn decodeOrDiscard(
reader: *std.Io.Reader,
) DecodeError!terminal_style.Style {
var encoded: [len]u8 = undefined;
try reader.readSliceAll(&encoded);
var source: std.Io.Reader = .fixed(&encoded);
return decode(&source);
}
fn encodeColor(
value: terminal_style.Style.Color,
writer: *std.Io.Writer,
) std.Io.Writer.Error!void {
var encoded: [4]u8 = @splat(0);
switch (value) {
.none => encoded[0] = @intFromEnum(ColorKind.none),
.palette => |index| {
encoded[0] = @intFromEnum(ColorKind.palette);
encoded[1] = index;
},
.rgb => |rgb| {
encoded[0] = @intFromEnum(ColorKind.rgb);
encoded[1] = rgb.r;
encoded[2] = rgb.g;
encoded[3] = rgb.b;
},
}
try writer.writeAll(&encoded);
}
fn decodeColor(
reader: *std.Io.Reader,
) DecodeError!terminal_style.Style.Color {
// Colors are always 4 bytes
var encoded: [4]u8 = undefined;
try reader.readSliceAll(&encoded);
// Kind must be something we know about.
const kind = std.enums.fromInt(ColorKind, encoded[0]) orelse {
return error.InvalidColorKind;
};
return switch (kind) {
.none => if (std.mem.eql(u8, encoded[1..], &.{ 0, 0, 0 }))
.none
else
error.InvalidColor,
.palette => if (encoded[2] == 0 and encoded[3] == 0)
.{ .palette = encoded[1] }
else
error.InvalidColor,
.rgb => .{ .rgb = .{
.r = encoded[1],
.g = encoded[2],
.b = encoded[3],
} },
};
}
fn computeLen() usize {
comptime {
var buf: [128]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
encode(.{}, &writer) catch unreachable;
return writer.end;
}
}
const test_golden_fixture = test_fixture.parse(@embedFile("testdata/style-v1.hex"));
test "golden encoding and decoding" {
const value: terminal_style.Style = .{
.fg_color = .none,
.bg_color = .{ .palette = 0x7f },
.underline_color = .{ .rgb = .{
.r = 0x12,
.g = 0x34,
.b = 0x56,
} },
.flags = .{
.bold = true,
.italic = true,
.faint = true,
.blink = true,
.inverse = true,
.invisible = true,
.strikethrough = true,
.overline = true,
.underline = .curly,
},
};
var buf: [len]u8 = undefined;
var writer: std.Io.Writer = .fixed(&buf);
try encode(value, &writer);
try test_fixture.expectEqual(
.bytes,
"src/terminal/snapshot/testdata/style-v1.hex",
"snapshot_fixture-style-v1.hex",
&test_golden_fixture,
writer.buffered(),
);
var source: std.Io.Reader = .fixed(&test_golden_fixture);
var read_buf: [1]u8 = undefined;
var limited = source.limited(.unlimited, &read_buf);
try std.testing.expect(value.eql(try decode(&limited.interface)));
}
test "flag bit layout" {
const Case = struct {
value: Flags,
expected: u16,
};
const cases = [_]Case{
.{ .value = .{ .bold = true }, .expected = 1 << 0 },
.{ .value = .{ .italic = true }, .expected = 1 << 1 },
.{ .value = .{ .faint = true }, .expected = 1 << 2 },
.{ .value = .{ .blink = true }, .expected = 1 << 3 },
.{ .value = .{ .inverse = true }, .expected = 1 << 4 },
.{ .value = .{ .invisible = true }, .expected = 1 << 5 },
.{ .value = .{ .strikethrough = true }, .expected = 1 << 6 },
.{ .value = .{ .overline = true }, .expected = 1 << 7 },
.{
.value = .{
.underline = @intFromEnum(sgr.Attribute.Underline.single),
},
.expected = 1 << 8,
},
.{
.value = .{
.underline = @intFromEnum(sgr.Attribute.Underline.double),
},
.expected = 2 << 8,
},
.{
.value = .{
.underline = @intFromEnum(sgr.Attribute.Underline.curly),
},
.expected = 3 << 8,
},
.{
.value = .{
.underline = @intFromEnum(sgr.Attribute.Underline.dotted),
},
.expected = 4 << 8,
},
.{
.value = .{
.underline = @intFromEnum(sgr.Attribute.Underline.dashed),
},
.expected = 5 << 8,
},
.{ .value = .{ .reserved = 1 }, .expected = 1 << 11 },
};
for (cases) |case| {
try std.testing.expectEqual(
case.expected,
@as(u16, @bitCast(case.value)),
);
}
}
test "reject invalid colors" {
for ([_]usize{ 0, 4, 8 }) |offset| {
var invalid_kind: [len]u8 = @splat(0);
invalid_kind[offset] = 3;
var reader: std.Io.Reader = .fixed(&invalid_kind);
try std.testing.expectError(error.InvalidColorKind, decode(&reader));
}
var invalid_none: [len]u8 = @splat(0);
invalid_none[1] = 1;
var none_reader: std.Io.Reader = .fixed(&invalid_none);
try std.testing.expectError(error.InvalidColor, decode(&none_reader));
var invalid_palette: [len]u8 = @splat(0);
invalid_palette[0] = @intFromEnum(ColorKind.palette);
invalid_palette[2] = 1;
var palette_reader: std.Io.Reader = .fixed(&invalid_palette);
try std.testing.expectError(error.InvalidColor, decode(&palette_reader));
}
test "reject invalid flags and reserved field" {
for ([_]u16{ 6, 7 }) |underline| {
var invalid_underline: [len]u8 = @splat(0);
std.mem.writeInt(
u16,
invalid_underline[12..14],
underline << 8,
.little,
);
var reader: std.Io.Reader = .fixed(&invalid_underline);
try std.testing.expectError(error.InvalidUnderline, decode(&reader));
}
var invalid_flags: [len]u8 = @splat(0);
std.mem.writeInt(u16, invalid_flags[12..14], 1 << 11, .little);
var flags_reader: std.Io.Reader = .fixed(&invalid_flags);
try std.testing.expectError(error.InvalidFlags, decode(&flags_reader));
var invalid_reserved: [len]u8 = @splat(0);
std.mem.writeInt(u16, invalid_reserved[14..16], 1, .little);
var reserved_reader: std.Io.Reader = .fixed(&invalid_reserved);
try std.testing.expectError(
error.InvalidReserved,
decode(&reserved_reader),
);
}
test "decodeOrDiscard preserves the next entry boundary" {
var fixture: [len + 1]u8 = @splat(0);
fixture[0] = 3;
fixture[len] = 0xFF;
var reader: std.Io.Reader = .fixed(&fixture);
try std.testing.expectError(error.InvalidColorKind, decodeOrDiscard(&reader));
try std.testing.expectEqual(@as(u8, 0xFF), try reader.takeByte());
}
test "reject every truncation" {
const fixture = [_]u8{0} ** len;
for (0..len) |fixture_len| {
var reader: std.Io.Reader = .fixed(fixture[0..fixture_len]);
try std.testing.expectError(error.EndOfStream, decode(&reader));
}
}

1840
src/terminal/snapshot/terminal.zig Normal file
View File

File diff suppressed because it is too large Load Diff

17
src/terminal/snapshot/testdata/checkpoint-ready-v1.hex vendored Normal file
View File

@@ -0,0 +1,17 @@
# Ghostty snapshot fixture
# Kaitai type: checkpoint_record
# Kaitai params: 5
# Kaitai offset: 3
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# prefix covered by READY
61 62 63
# READY record header: tag, payload length, CRC32C
05 00 20 00 00 00 90 5b 2d 06
# BLAKE3-256 digest of the prefix
64 37 b3 ac 38 46 51 33 ff b6 3b 75 27 3a 8d b5
48 c5 58 46 5d 79 db 03 fd 35 9c 6c d5 bd 9d 85

139
src/terminal/snapshot/testdata/complete-v1.hex vendored Normal file
View File

@@ -0,0 +1,139 @@
# Ghostty snapshot fixture
# Kaitai type: ghostty_snapshot
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: snapshot envelope
47 48 4f 53 54 53 4e 50 01 00 # 0x00000000
# offset 0x0000000a: terminal record, payload 952 bytes
01 00 b8 03 00 00 e3 5f 8b cc 02 00 03 00 20 03 # 0x0000000a
00 00 58 02 00 00 00 00 02 00 00 00 01 00 00 01 # 0x0000001a
00 02 00 65 00 00 00 01 01 01 00 00 00 00 00 08 # 0x0000002a
00 04 22 00 64 10 00 00 00 04 22 00 64 00 00 00 # 0x0000003a
00 04 22 00 64 00 00 00 00 00 00 00 00 00 00 00 # 0x0000004a
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000005a
00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff # 0x0000006a
ff 00 1d 1f 21 cc 66 66 b5 bd 68 f0 c6 74 81 a2 # 0x0000007a
be b2 94 bb 8a be b7 c5 c8 c6 66 66 66 d5 4e 53 # 0x0000008a
b9 ca 4a e7 c5 47 7a a6 da c3 97 d8 70 c0 b1 ea # 0x0000009a
ea ea 00 00 00 00 00 5f 00 00 87 00 00 af 00 00 # 0x000000aa
d7 00 00 ff 00 5f 00 00 5f 5f 00 5f 87 00 5f af # 0x000000ba
00 5f d7 00 5f ff 00 87 00 00 87 5f 00 87 87 00 # 0x000000ca
87 af 00 87 d7 00 87 ff 00 af 00 00 af 5f 00 af # 0x000000da
87 00 af af 00 af d7 00 af ff 00 d7 00 00 d7 5f # 0x000000ea
00 d7 87 00 d7 af 00 d7 d7 00 d7 ff 00 ff 00 00 # 0x000000fa
ff 5f 00 ff 87 00 ff af 00 ff d7 00 ff ff 5f 00 # 0x0000010a
00 5f 00 5f 5f 00 87 5f 00 af 5f 00 d7 5f 00 ff # 0x0000011a
5f 5f 00 5f 5f 5f 5f 5f 87 5f 5f af 5f 5f d7 5f # 0x0000012a
5f ff 5f 87 00 5f 87 5f 5f 87 87 5f 87 af 5f 87 # 0x0000013a
d7 5f 87 ff 5f af 00 5f af 5f 5f af 87 5f af af # 0x0000014a
5f af d7 5f af ff 5f d7 00 5f d7 5f 5f d7 87 5f # 0x0000015a
d7 af 5f d7 d7 5f d7 ff 5f ff 00 5f ff 5f 5f ff # 0x0000016a
87 5f ff af 5f ff d7 5f ff ff 87 00 00 87 00 5f # 0x0000017a
87 00 87 87 00 af 87 00 d7 87 00 ff 87 5f 00 87 # 0x0000018a
5f 5f 87 5f 87 87 5f af 87 5f d7 87 5f ff 87 87 # 0x0000019a
00 87 87 5f 87 87 87 87 87 af 87 87 d7 87 87 ff # 0x000001aa
87 af 00 87 af 5f 87 af 87 87 af af 87 af d7 87 # 0x000001ba
af ff 87 d7 00 87 d7 5f 87 d7 87 87 d7 af 87 d7 # 0x000001ca
d7 87 d7 ff 87 ff 00 87 ff 5f 87 ff 87 87 ff af # 0x000001da
87 ff d7 87 ff ff af 00 00 af 00 5f af 00 87 af # 0x000001ea
00 af af 00 d7 af 00 ff af 5f 00 af 5f 5f af 5f # 0x000001fa
87 af 5f af af 5f d7 af 5f ff af 87 00 af 87 5f # 0x0000020a
af 87 87 af 87 af af 87 d7 af 87 ff af af 00 af # 0x0000021a
af 5f af af 87 af af af af af d7 af af ff af d7 # 0x0000022a
00 af d7 5f af d7 87 af d7 af af d7 d7 af d7 ff # 0x0000023a
af ff 00 af ff 5f af ff 87 af ff af af ff d7 af # 0x0000024a
ff ff d7 00 00 d7 00 5f d7 00 87 d7 00 af d7 00 # 0x0000025a
d7 d7 00 ff d7 5f 00 d7 5f 5f d7 5f 87 d7 5f af # 0x0000026a
d7 5f d7 d7 5f ff d7 87 00 d7 87 5f d7 87 87 d7 # 0x0000027a
87 af d7 87 d7 d7 87 ff d7 af 00 d7 af 5f d7 af # 0x0000028a
87 d7 af af d7 af d7 d7 af ff d7 d7 00 d7 d7 5f # 0x0000029a
d7 d7 87 d7 d7 af d7 d7 d7 d7 d7 ff d7 ff 00 d7 # 0x000002aa
ff 5f d7 ff 87 d7 ff af d7 ff d7 d7 ff ff ff 00 # 0x000002ba
00 ff 00 5f ff 00 87 ff 00 af ff 00 d7 ff 00 ff # 0x000002ca
ff 5f 00 ff 5f 5f ff 5f 87 ff 5f af ff 5f d7 ff # 0x000002da
5f ff ff 87 00 ff 87 5f ff 87 87 ff 87 af ff 87 # 0x000002ea
d7 ff 87 ff ff af 00 ff af 5f ff af 87 ff af af # 0x000002fa
ff af d7 ff af ff ff d7 00 ff d7 5f ff d7 87 ff # 0x0000030a
d7 af ff d7 d7 ff d7 ff ff ff 00 ff ff 5f ff ff # 0x0000031a
87 ff ff af ff ff d7 ff ff ff 08 08 08 12 12 12 # 0x0000032a
1c 1c 1c 26 26 26 30 30 30 3a 3a 3a 44 44 44 4e # 0x0000033a
4e 4e 58 58 58 62 62 62 6c 6c 6c 76 76 76 80 80 # 0x0000034a
80 8a 8a 8a 94 94 94 9e 9e 9e a8 a8 a8 b2 b2 b2 # 0x0000035a
bc bc bc c6 c6 c6 d0 d0 d0 da da da e4 e4 e4 ee # 0x0000036a
ee ee 80 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000037a
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000038a
00 00 01 02 03 14 00 00 00 66 69 6c 65 3a 2f 2f # 0x0000039a
2f 74 6d 70 2f 73 6e 61 70 73 68 6f 74 11 00 00 # 0x000003aa
00 63 6f 6d 70 6c 65 74 65 20 73 6e 61 70 73 68 # 0x000003ba
6f 74 # 0x000003ca
# offset 0x000003cc: screen record, payload 54 bytes
02 00 36 00 00 00 67 31 0e c6 00 00 01 00 04 00 # 0x000003cc
00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 # 0x000003dc
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000003ec
00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000003fc
# offset 0x0000040c: page record, payload 119 bytes
03 00 77 00 00 00 48 b7 91 cb 02 00 03 00 00 00 # 0x0000040c
00 00 10 00 c0 00 00 04 00 00 00 08 00 00 00 00 # 0x0000041c
00 00 00 00 00 00 00 43 00 00 00 00 00 00 00 00 # 0x0000042c
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000043c
00 00 00 00 00 00 00 00 44 00 00 00 00 00 00 00 # 0x0000044c
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000045c
00 00 00 00 00 00 00 00 00 45 00 00 00 00 00 00 # 0x0000046c
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000047c
00 # 0x0000048c
# offset 0x0000048d: screen record, payload 54 bytes
02 00 36 00 00 00 ce 1f 9f 08 01 00 01 00 00 00 # 0x0000048d
00 00 00 00 00 00 01 00 02 00 01 00 00 00 00 00 # 0x0000049d
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000004ad
00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000004bd
# offset 0x000004cd: page record, payload 119 bytes
03 00 77 00 00 00 84 a6 e9 08 02 00 03 00 00 00 # 0x000004cd
00 00 80 00 c0 00 00 02 00 00 00 08 00 00 03 00 # 0x000004dd
00 00 00 00 00 00 00 72 00 00 00 00 00 00 00 00 # 0x000004ed
00 00 00 00 00 00 00 6e 00 00 00 00 00 00 00 03 # 0x000004fd
00 00 00 00 00 00 00 00 61 00 00 00 00 00 00 00 # 0x0000050d
00 00 00 00 00 00 00 00 74 00 00 00 00 00 00 00 # 0x0000051d
03 00 00 00 00 00 00 00 00 65 00 00 00 00 00 00 # 0x0000052d
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x0000053d
00 # 0x0000054d
# offset 0x0000054e: ready record, payload 32 bytes
05 00 20 00 00 00 43 0d 7b 3a 84 06 56 ba 02 59 # 0x0000054e
04 24 20 59 96 92 14 ba 76 53 80 40 09 12 85 a7 # 0x0000055e
f2 b5 b8 f9 63 98 e8 dc 9a 39 # 0x0000056e
# offset 0x00000578: history record, payload 6 bytes
04 00 06 00 00 00 20 32 ed e1 00 00 02 00 00 00 # 0x00000578
# offset 0x00000588: page record, payload 86 bytes
03 00 56 00 00 00 52 3b 6e 8d 02 00 02 00 00 00 # 0x00000588
00 00 10 00 c0 00 00 04 00 00 00 08 00 00 00 00 # 0x00000598
00 00 00 00 00 00 00 42 00 00 00 00 00 00 00 00 # 0x000005a8
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000005b8
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000005c8
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000005d8
# offset 0x000005e8: page record, payload 86 bytes
03 00 56 00 00 00 fb bc 15 06 02 00 02 00 00 00 # 0x000005e8
00 00 10 00 c0 00 00 04 00 00 00 08 00 00 00 00 # 0x000005f8
00 00 00 00 00 00 00 41 00 00 00 00 00 00 00 00 # 0x00000608
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x00000618
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x00000628
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x00000638
# offset 0x00000648: history record, payload 6 bytes
04 00 06 00 00 00 b8 7a ba b1 01 00 00 00 00 00 # 0x00000648
# offset 0x00000658: finish record, payload 32 bytes
06 00 20 00 00 00 8d d6 46 7a ac 02 c8 09 9d 1a # 0x00000658
5a 00 ce 47 5e d7 b0 ae 54 10 bb 0d f1 35 7c ab # 0x00000668
92 fd ae d3 5e 1a 77 1e 40 7f # 0x00000678

10
src/terminal/snapshot/testdata/envelope-v1.hex vendored Normal file
View File

@@ -0,0 +1,10 @@
# Ghostty snapshot fixture
# Kaitai type: envelope
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
47 48 4f 53 54 53 4e 50 01 00 # 0x00000000

32
src/terminal/snapshot/testdata/grid-v1.hex vendored Normal file
View File

@@ -0,0 +1,32 @@
# Ghostty snapshot fixture
# Kaitai type: grid
# Kaitai params: 2 3
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# row 0: prompt
04
# row 0, cell 0: protected prompt, wide "A"
00 01 05 00 00 00 00 00 41 00 00 00 00 00 00 00
# row 0, cell 1: input spacer tail
00 02 02 00 00 00 00 00 00 00 00 00 00 00 00 00
# row 0, cell 2: "x" with two grapheme suffixes
00 00 00 00 00 00 00 00 78 00 00 00 02 00 00 00
01 03 00 00 02 03 00 00
# row 1: wrapped prompt continuation
0b
# row 1, cell 0: palette background 7
01 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00
# row 1, cell 1: protected RGB background
02 00 01 00 00 00 00 00 aa bb cc 00 00 00 00 00
# row 1, cell 2: spacer head
00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00

10
src/terminal/snapshot/testdata/history-header-v1.hex vendored Normal file
View File

@@ -0,0 +1,10 @@
# Ghostty snapshot fixture
# Kaitai type: history_payload
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
01 00 04 03 02 01 # 0x00000000

10
src/terminal/snapshot/testdata/hyperlink-explicit-v1.hex vendored Normal file
View File

@@ -0,0 +1,10 @@
# Ghostty snapshot fixture
# Kaitai type: hyperlink
# Kaitai params: false false
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
02 02 00 00 00 69 64 03 00 00 00 75 72 69 # 0x00000000

10
src/terminal/snapshot/testdata/hyperlink-implicit-v1.hex vendored Normal file
View File

@@ -0,0 +1,10 @@
# Ghostty snapshot fixture
# Kaitai type: hyperlink
# Kaitai params: false false
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
01 04 03 02 01 03 00 00 00 75 72 69 # 0x00000000

12
src/terminal/snapshot/testdata/page-empty-record-v1.hex vendored Normal file
View File

@@ -0,0 +1,12 @@
# Ghostty snapshot fixture
# Kaitai type: page_record
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
03 00 25 00 00 00 8c 05 d6 d3 01 00 01 00 00 00 # 0x00000000
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x00000010
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x00000020

11
src/terminal/snapshot/testdata/page-header-v1.hex vendored Normal file
View File

@@ -0,0 +1,11 @@
# Ghostty snapshot fixture
# Kaitai type: page_header
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
02 01 04 03 06 05 08 07 0a 09 0c 0b 10 0f 0e 0d # 0x00000000
14 13 12 11 # 0x00000010

24
src/terminal/snapshot/testdata/page-v1.hex vendored Normal file
View File

@@ -0,0 +1,24 @@
# Ghostty snapshot fixture
# Kaitai type: page_payload
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: PAGE header
03 00 02 00 02 00 02 00 08 00 00 02 80 00 00 00 # 0x00000000
00 01 00 00 # 0x00000010
# offset 0x00000014: style and hyperlink tables, rows, and cells
01 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 # 0x00000014
00 00 03 00 00 00 00 00 01 2a 00 00 00 00 00 00 # 0x00000024
00 00 00 00 01 00 02 01 00 00 00 61 05 00 00 00 # 0x00000034
61 6c 70 68 61 03 00 01 04 03 02 01 04 00 00 00 # 0x00000044
62 65 74 61 04 00 01 05 00 01 00 01 00 41 00 00 # 0x00000054
00 00 00 00 00 00 02 02 00 03 00 03 00 00 00 00 # 0x00000064
00 00 00 00 00 01 00 00 00 01 00 01 00 07 00 00 # 0x00000074
00 00 00 00 00 0b 00 00 00 00 00 00 00 00 78 00 # 0x00000084
00 00 02 00 00 00 01 03 00 00 02 03 00 00 02 00 # 0x00000094
01 00 00 00 00 00 aa bb cc 00 00 00 00 00 00 03 # 0x000000a4
00 00 00 00 00 00 00 00 00 00 00 00 00 00 # 0x000000b4

10
src/terminal/snapshot/testdata/record-page-header-v1.hex vendored Normal file
View File

@@ -0,0 +1,10 @@
# Ghostty snapshot fixture
# Kaitai type: record_header
# Kaitai params: 3
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
03 00 18 00 00 00 1b 44 78 71 # 0x00000000

13
src/terminal/snapshot/testdata/screen-header-v1.hex vendored Normal file
View File

@@ -0,0 +1,13 @@
# Ghostty snapshot fixture
# Kaitai type: screen_header
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
01 00 03 02 08 07 06 05 04 03 02 01 05 04 07 06 # 0x00000000
03 19 00 00 00 00 01 7f 00 00 02 12 34 56 ff 03 # 0x00000010
00 00 0d 0c 0b 0a e4 3d 02 07 01 02 04 08 10 1f # 0x00000020
00 11 02 03 01 # 0x00000030

11
src/terminal/snapshot/testdata/screen-saved-cursor-v1.hex vendored Normal file
View File

@@ -0,0 +1,11 @@
# Ghostty snapshot fixture
# Kaitai type: saved_cursor
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
02 01 04 03 00 00 00 00 00 00 00 00 00 00 00 00 # 0x00000000
00 00 00 00 07 e4 3d # 0x00000010

10
src/terminal/snapshot/testdata/style-v1.hex vendored Normal file
View File

@@ -0,0 +1,10 @@
# Ghostty snapshot fixture
# Kaitai type: style
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
00 00 00 00 01 7f 00 00 02 12 34 56 ff 03 00 00 # 0x00000000

16
src/terminal/snapshot/testdata/terminal-header-v1.hex vendored Normal file
View File

@@ -0,0 +1,16 @@
# Ghostty snapshot fixture
# Kaitai type: terminal_header
# Kaitai params:
# Kaitai offset: 0
# Wire version: 1
# Generated by its snapshot test; review before replacing.
# On mismatch, the candidate is copied to the repository root.
# offset 0x00000000: encoded bytes
02 01 04 03 08 07 06 05 0c 0b 0a 09 01 00 02 00 # 0x00000000
03 00 04 00 01 01 00 02 00 41 00 00 00 01 03 02 # 0x00000010
02 01 04 04 01 21 01 01 00 00 00 00 00 00 00 00 # 0x00000020
00 00 00 00 02 00 00 01 00 00 00 00 02 00 00 01 # 0x00000030
01 02 03 00 00 00 00 00 00 00 00 01 04 05 06 01 # 0x00000040
07 08 09 01 0a 0b 0c ff ff ff ff ff ff ff ff 08 # 0x00000050
07 06 05 04 03 02 01 # 0x00000060

417
src/terminal/snapshot/verify-kaitai.py Executable file
View File

@@ -0,0 +1,417 @@
#!/usr/bin/env python3
"""Compile the snapshot Kaitai schema and parse every checked-in fixture.
Run this from a Ghostty development shell:
src/terminal/snapshot/verify-kaitai.py
To turn an annotated fixture into a binary suitable for the Kaitai Web IDE:
src/terminal/snapshot/verify-kaitai.py \
--write-binary \
src/terminal/snapshot/testdata/complete-v1.hex \
/tmp/complete-v1.bin
Then load `snapshot.ksy` and the generated binary into
https://ide.kaitai.io/.
The generated Python parser exists only in a temporary directory. This keeps
snapshot.ksy as the source of truth and ensures this check cannot accidentally
pass against stale generated code. After structural parsing, the script checks
record CRC32C values, checkpoint BLAKE3 digests, and cross-record invariants
that portable Kaitai expressions cannot represent.
"""
from __future__ import annotations
import argparse
import importlib
import io
import shutil
import struct
import subprocess
import sys
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any
try:
from blake3 import blake3
from kaitaistruct import KaitaiStream
except ImportError as error:
raise SystemExit(
"missing Kaitai verifier dependencies; run this inside "
"`nix develop`"
) from error
SNAPSHOT_DIR = Path(__file__).resolve().parent
SCHEMA_PATH = SNAPSHOT_DIR / "snapshot.ksy"
TESTDATA_DIR = SNAPSHOT_DIR / "testdata"
SNAPSHOT_ENVELOPE_SIZE = 10
RECORD_HEADER_SIZE = struct.calcsize("<HII")
@dataclass(frozen=True)
class Fixture:
path: Path
type_name: str
params: tuple[object, ...]
offset: int
data: bytes
def load_fixture(path: Path) -> Fixture:
"""Load one self-describing annotated hexadecimal fixture."""
metadata: dict[str, str] = {}
chunks: list[str] = []
for line in path.read_text(encoding="utf-8").splitlines():
content, separator, comment = line.partition("#")
chunks.extend(content.split())
if not separator:
continue
comment = comment.strip()
if not comment.startswith("Kaitai "):
continue
key, colon, value = comment.removeprefix("Kaitai ").partition(":")
if not colon or key not in {"type", "params", "offset"}:
raise ValueError(f"{path}: invalid Kaitai metadata: {comment}")
if key in metadata:
raise ValueError(f"{path}: duplicate Kaitai {key} metadata")
metadata[key] = value.strip()
try:
data = bytes.fromhex(" ".join(chunks))
except ValueError as error:
raise ValueError(f"{path}: invalid annotated hexadecimal") from error
required = {"type", "params", "offset"}
if metadata.keys() != required:
missing = sorted(required - metadata.keys())
extra = sorted(metadata.keys() - required)
raise ValueError(
f"{path}: Kaitai metadata drift: missing={missing}, extra={extra}"
)
if not metadata["type"]:
raise ValueError(f"{path}: empty Kaitai type")
params: list[object] = []
for value in metadata["params"].split():
if value == "true":
params.append(True)
elif value == "false":
params.append(False)
else:
try:
params.append(int(value, 0))
except ValueError as error:
raise ValueError(
f"{path}: invalid Kaitai parameter {value!r}"
) from error
try:
offset = int(metadata["offset"], 0)
except ValueError as error:
raise ValueError(f"{path}: invalid Kaitai offset") from error
if offset < 0 or offset > len(data):
raise ValueError(f"{path}: Kaitai offset is outside the fixture")
return Fixture(
path=path,
type_name=metadata["type"],
params=tuple(params),
offset=offset,
data=data,
)
def parse_type(
parser_type: type[Any],
data: bytes,
*params: object,
) -> Any:
"""Construct one generated Kaitai type and require exact exhaustion."""
stream = KaitaiStream(io.BytesIO(data))
result = parser_type(*params, stream)
if not stream.is_eof():
raise ValueError(
f"{parser_type.__name__} left "
f"{stream.size() - stream.pos()} trailing bytes"
)
return result
def crc32c(data: bytes) -> int:
"""Return CRC32C using the reflected Castagnoli polynomial."""
result = 0xFFFFFFFF
for byte in data:
result ^= byte
for _ in range(8):
result = (
(result >> 1) ^ 0x82F63B78
if result & 1
else result >> 1
)
return result ^ 0xFFFFFFFF
def validate_record(record: Any, data: bytes, offset: int) -> int:
"""Validate one parsed record's source bytes and CRC32C."""
payload = record._raw_payload
header = record.header
if header.payload_length != len(payload):
raise ValueError(
f"record at 0x{offset:x}: declared {header.payload_length} "
f"payload bytes but parsed {len(payload)}"
)
encoded_header = data[offset : offset + RECORD_HEADER_SIZE]
expected_header = struct.pack(
"<HII",
header.tag,
header.payload_length,
header.crc32c,
)
if encoded_header != expected_header:
raise ValueError(f"record at 0x{offset:x}: parsed header drift")
covered = struct.pack("<HI", header.tag, header.payload_length) + payload
actual_crc = crc32c(covered)
if header.crc32c != actual_crc:
raise ValueError(
f"record at 0x{offset:x}: CRC32C 0x{header.crc32c:08x} "
f"does not match 0x{actual_crc:08x}"
)
return offset + RECORD_HEADER_SIZE + len(payload)
def all_snapshot_records(snapshot: Any) -> list[Any]:
"""Return complete-snapshot records in their authenticated wire order."""
records = [snapshot.terminal]
for sequence in snapshot.screens:
records.append(sequence.screen)
records.extend(sequence.pages)
records.append(snapshot.ready)
for sequence in snapshot.histories:
records.append(sequence.history)
records.extend(sequence.pages)
records.append(snapshot.finish)
return records
def validate_complete_snapshot(snapshot: Any, data: bytes) -> None:
"""Validate ordering relationships, record CRCs, and checkpoints."""
screen_keys = [
sequence.screen.payload.header.key for sequence in snapshot.screens
]
history_keys = [
sequence.history.payload.key for sequence in snapshot.histories
]
if len(set(screen_keys)) != len(screen_keys):
raise ValueError("complete snapshot contains a duplicate SCREEN key")
if len(set(history_keys)) != len(history_keys):
raise ValueError("complete snapshot contains a duplicate HISTORY key")
if set(screen_keys) != set(history_keys):
raise ValueError("SCREEN and HISTORY keys do not match")
terminal_header = snapshot.terminal.payload.header
expected_key_values = (
{0} if terminal_header.screen_count == 1 else {0, 1}
)
screen_key_values = {key.value for key in screen_keys}
if screen_key_values != expected_key_values:
raise ValueError(
f"TERMINAL declares screen keys {expected_key_values}, "
f"but SCREEN records contain {screen_key_values}"
)
screens_by_key = {
sequence.screen.payload.header.key: sequence
for sequence in snapshot.screens
}
for sequence in snapshot.screens:
header = sequence.screen.payload.header
if (
header.cursor_x >= terminal_header.columns
or header.cursor_y >= terminal_header.rows
or (
header.cursor_flags.pending_wrap
and header.cursor_x != terminal_header.columns - 1
)
):
raise ValueError(f"SCREEN key {header.key}: invalid cursor position")
for sequence in snapshot.histories:
payload = sequence.history.payload
screen = screens_by_key[payload.key]
screen_rows = sum(
page.payload.header.rows for page in screen.pages
)
if screen_rows < terminal_header.rows:
raise ValueError(
f"SCREEN key {payload.key}: PAGE rows do not cover "
"the active area"
)
overlap_rows = screen_rows - terminal_header.rows
history_rows = overlap_rows + sum(
page.payload.header.rows for page in sequence.pages
)
if history_rows != screen.screen.payload.header.history_rows:
raise ValueError(
f"SCREEN key {payload.key}: history_rows "
f"{screen.screen.payload.header.history_rows} "
f"does not match {history_rows}"
)
for record in all_snapshot_records(snapshot):
if not hasattr(record, "payload"):
continue
payload = record.payload
if not hasattr(payload, "styles"):
continue
style_ids = [entry.encoded_id for entry in payload.styles]
hyperlink_ids = [entry.encoded_id for entry in payload.hyperlinks]
if len(set(style_ids)) != len(style_ids):
raise ValueError("PAGE contains a duplicate style ID")
if len(set(hyperlink_ids)) != len(hyperlink_ids):
raise ValueError("PAGE contains a duplicate hyperlink ID")
offset = SNAPSHOT_ENVELOPE_SIZE
for record in all_snapshot_records(snapshot):
if record is snapshot.ready:
expected = blake3(data[:offset]).digest()
if record.payload.prefix_digest != expected:
raise ValueError("READY BLAKE3-256 digest does not match")
elif record is snapshot.finish:
expected = blake3(data[:offset]).digest()
if record.payload.prefix_digest != expected:
raise ValueError("FINISH BLAKE3-256 digest does not match")
offset = validate_record(record, data, offset)
if offset != len(data):
raise ValueError(
f"complete snapshot accounted for {offset} of {len(data)} bytes"
)
def compile_schema(output_dir: Path) -> type[Any]:
"""Compile snapshot.ksy and import its temporary Python parser."""
compiler = shutil.which("kaitai-struct-compiler") or shutil.which("ksc")
if compiler is None:
raise SystemExit(
"kaitai-struct-compiler is not available; run this inside "
"`nix develop`"
)
subprocess.run(
[
compiler,
"--target=python",
f"--outdir={output_dir}",
str(SCHEMA_PATH),
],
check=True,
)
sys.path.insert(0, str(output_dir))
try:
module = importlib.import_module("ghostty_snapshot")
finally:
sys.path.pop(0)
return module.GhosttySnapshot
def resolve_type(parser: type[Any], type_name: str) -> type[Any]:
"""Resolve one KSY snake-case type name on the generated parser."""
root_type_name = "".join(
f"_{character.lower()}" if character.isupper() else character
for character in parser.__name__
).lstrip("_")
if type_name == root_type_name:
return parser
class_name = "".join(
part[:1].upper() + part[1:] for part in type_name.split("_")
)
try:
return getattr(parser, class_name)
except AttributeError as error:
raise ValueError(f"snapshot.ksy has no type {type_name!r}") from error
def main() -> int:
cli = argparse.ArgumentParser(description=__doc__)
cli.add_argument(
"--write-binary",
nargs=2,
metavar=("FIXTURE", "OUTPUT"),
type=Path,
help="decode one annotated fixture into a raw binary",
)
args = cli.parse_args()
if args.write_binary:
fixture_path, output_path = args.write_binary
fixture = load_fixture(fixture_path)
output_path.write_bytes(fixture.data)
print(f"wrote {len(fixture.data)} bytes to {output_path}")
return 0
with tempfile.TemporaryDirectory(prefix="ghostty-kaitai-") as temp:
parser = compile_schema(Path(temp))
fixture_paths = sorted(TESTDATA_DIR.rglob("*.hex"))
if not fixture_paths:
raise ValueError(f"{TESTDATA_DIR}: no snapshot fixtures")
for path in fixture_paths:
fixture = load_fixture(path)
parser_type = resolve_type(parser, fixture.type_name)
fixture_label = path.relative_to(TESTDATA_DIR)
try:
parsed = parse_type(
parser_type,
fixture.data[fixture.offset:],
*fixture.params,
)
except Exception as error:
raise ValueError(
f"{fixture_label}: Kaitai parse failed"
) from error
if parser_type is parser:
if fixture.offset != 0:
raise ValueError(
f"{fixture_label}: root parser must begin at offset zero"
)
validate_complete_snapshot(parsed, fixture.data)
elif hasattr(parsed, "_raw_payload"):
record_end = validate_record(
parsed,
fixture.data,
fixture.offset,
)
if record_end != len(fixture.data):
raise ValueError(
f"{fixture_label}: record has trailing data"
)
if hasattr(parsed.payload, "prefix_digest"):
expected = blake3(
fixture.data[:fixture.offset]
).digest()
if parsed.payload.prefix_digest != expected:
raise ValueError(
f"{fixture_label}: checkpoint digest does not match"
)
print(f"ok {fixture_label}")
print(f"validated {len(fixture_paths)} snapshot fixtures")
return 0
if __name__ == "__main__":
raise SystemExit(main())

2
typos.toml
View File

@@ -28,6 +28,8 @@ extend-exclude = [
"valgrind.supp",
# Fuzz corpus (binary test inputs)
"test/fuzz-libghostty/corpus/*",
# Annotated binary snapshot fixtures
"src/terminal/snapshot/testdata/*.hex",
# Other
"*.pdf",
"*.data",