This adds the first version of a binary snapshot format for terminal
state.
Use cases: replay software (like asciinema), multiplexers (like zmx),
scrollback-saving on disk, etc.
The intention of the binary snapshot format is to be able to fully
encode and decode terminal state across mediums such as network and
disk. You can also encode partial terminal state (e.g. only one screen
or even one page of contents). Long term, the intention is to also
support streaming state while a live terminal is running, but this
initial PR focuses on the full snapshot first (with some design choices
to get to the streaming state in the future).
The format is documented in the Zig code, but I also did a
[Kaitai](https://kaitai.io/) descriptor and both the Zig and Kaitai spec
verify they can parse committed fixtures. This helps identify drift in
the format or encoder/decoders in any way since this must ultimately be
a fixed format.
> [!NOTE]
>
> **On reviewability:** this is a massive PR that I don't expect anyone
to reasonably review. I'm going through it line-by-line (again) but I
purposely extracted any changes that affect other parts of Ghostty out
to other already-merged PRs. This one is isolated purely to a package
that isn't called by any client software. **So the plan is if this rough
shape looks good I'll merge it and we'll iterate from there.**
> [!WARNING]
>
> **Experimental.** The format can and will change. And we may also
decide that binary snapshotting in this way isn't the right direction
altogether (although, I'm pretty confident it is). It'd be impossible to
get a single large perfect PR because it'd be even larger than this by
multiples. So instead, we'll iterate on main so long as this work is not
touching any production code, which it isn't!
## Example
Encode:
```zig
const terminal = @import("terminal/main.zig");
var file_buffer: [16 * 1024]u8 = undefined;
var file_writer = file.writer(io, &file_buffer);
try terminal.snapshot.encode(alloc, &file_writer.interface, &t);
try file_writer.interface.flush();
```
Decode a full terminal:
```zig
var t = try terminal.snapshot.decode(&reader, io, alloc);
defer t.deinit(alloc);
```
## Future
This PR purposely only supports a synchronous encode/decode. I wanted to
get the large groundwork in before iterating further. Some iterations in
the future:
* Kitty graphics
* Live terminal snapshotting
* PTY stream continuation records (so VT state machines can stay in
sync)
* Performance work (encoding and decoding, maybe size)
* Configurable limits to prevent DoS
* C API
* etc...
## Wire format
The "robustness principle" is a guiding principle: "be conservative in
what you do, be liberal in what you accept from others." Our encoders
have a lot of extra validation, our decoders massage invalid data into
reasonable defaults (e.g. invalid styles become unstyled text).
> [!IMPORTANT]
>
> **Version 1 has no compatibility promise.** We use version 1 in the
envelope header. We will absolutely break this format as needed as we
iterate and improve on it...
### Envelope
Every snapshot starts with a fixed ten-byte envelope:
| Offset | Size | Field |
| ---: | ---: | :--- |
| 0 | 8 | Magic: `GHOSTSNP` |
| 8 | 2 | Snapshot version: `1` |
### Record framing
After the envelope, records are concatenated back-to-back. Every record
has a fixed header:
| Offset | Size | Field |
| ---: | ---: | :--- |
| 0 | 2 | Record tag |
| 2 | 4 | Payload length |
| 6 | 4 | CRC32C |
| 10 | variable | Payload |
CRC32C covers the encoded tag, payload length, and payload. The
payload-length boundary prevents a malformed record decoder from
consuming bytes belonging to the next record.
The registered record tags are:
| Value | Tag | Purpose |
| ---: | :--- | :--- |
| 1 | `TERMINAL` | Terminal-wide state and declared screens |
| 2 | `SCREEN` | One screen's live state and active page manifest |
| 3 | `PAGE` | One self-contained set of rows and cells |
| 4 | `HISTORY` | One screen's historical page manifest |
| 5 | `READY` | Digest of the renderable prefix |
| 6 | `FINISH` | Digest of the complete snapshot |
To view the format of each record, read its corresponding
`terminal/snapshot/<type>.zig` file.
### Complete record sequence
```text
+----------------------------------------+
| Envelope |
+----------------------------------------+
| TERMINAL |
+----------------------------------------+
| SCREEN * terminal.screen_count |
| PAGE * each screen.page_count |
+----------------------------------------+
| READY |
+----------------------------------------+
| HISTORY * terminal.screen_count |
| PAGE * each history.page_count |
+----------------------------------------+
| FINISH |
+----------------------------------------+
| Optional containing-transport bytes |
+----------------------------------------+
```
`SCREEN` and `HISTORY` groups are routed by their encoded screen key and
may arrive in either key order.
## Checkpoints and validation
Each record has an independent CRC32C, but per-record checksums cannot
detect a valid record being reordered, omitted, or duplicated. `READY`
and `FINISH` therefore contain BLAKE3-256 digests over exact snapshot
prefixes:
- `READY` covers the envelope, `TERMINAL`, and all live `SCREEN`/`PAGE`
sequences. It does not include itself.
- `FINISH` covers that same prefix, the complete `READY` record, and all
`HISTORY`/`PAGE` sequences. It does not include itself.
This gives the format two useful integrity boundaries:
```text
envelope ... active pages | READY | history pages | FINISH
<------ renderable ------->
<------------- complete snapshot --------------->
```
## Performance
### Size, Compression Recommended
We intentionally use a simple grid over something like RLE (run-length
encoding). So every row contains exactly `columns` cells and each is
16-bytes! This is large! A 80x24, 10,000 line scrollback terminal
uncompressed would be ~13MB. However, with zstd level 1 compression that
goes down to 260K.
### Speed
We haven't benchmarked encoding or decoding speed yet. This PR focused
on getting a format in place. This will be heavily optimized later. I
suspect its probably pretty darn slow, actually.
## Kaitai Struct
I added a `snapshot.ksy` Kaita Struct spec that independently describes
the complete format. This is used by us for format validation but it can
also be used to programmatically generate parsers. For example, our test
fixture in the Kaita Struct web IDE decodes to:
<img width="523" height="779" alt="image"
src="https://github.com/user-attachments/assets/cd199c73-b6d6-4b35-8957-0cfe3d1a18f2"
/>
**AI Usage:** This work was done in concert with various models and
agents. Writing full encoders/decoders is tedious so it took a lot of
that way. A lot of review was done by AI (trying to find holes, issues,
inconsistencies). The actual binary protocol design and iteration was
done by me. This PR message was written by me.
Exclude annotated snapshot fixture hex files from typo checking. Their arbitrary binary byte sequences can otherwise be misidentified as misspelled words.
Expose complete encode and decode entry points directly from terminal.snapshot instead of requiring terminal.snapshot.snapshot. Reorder the decode APIs to accept the allocator and I/O context before the reader.
Publish each screen's logical history extent before READY so clients can size scrollbars while older pages are still arriving. Keep the value advisory and continue deriving native PageList totals from decoded pages.
Reduce HISTORY to its structural screen key and page count, and update the format documentation, Kaitai schema, verifier, and versioned fixtures.
Treat FINISH as the self-delimiting snapshot boundary instead of peeking for end-of-file. Normal decoding now leaves continuation bytes unread, allowing snapshots and live protocol data to share a stream without waiting for closure.
Add decodeExact for bounded files that still require strict exhaustion, and update the Kaitai schema, documentation, and tests for continuation and sequential snapshot decoding.
Stream complete snapshot records to any std.Io.Writer while retaining one reusable payload buffer for length and CRC calculation. Update BLAKE3 incrementally so checkpoints no longer require rehashing an allocating destination.
Wrap decode hashing in StreamReader to enforce exact checkpoint boundaries. Preserve v1 bytes while allowing snapshots to begin at the current writer position and retaining only valid prefixes on failures.
Route HISTORY sequences by their encoded screen key so both keyed sequence groups can arrive in either order. Keep undeclared and duplicate routing strict, separate HISTORY manifest parsing from page restoration, and clear decoder-only generation state before returning the terminal.
Normalize unknown terminal-wide semantic fields during restore while keeping dimensions and screen count structural. Preserve canonical encoding, ignore reserved mode and tab-stop bits, reset invalid color and scrolling state, and clamp finite scrollback policies to the native range.
Treat HISTORY row counts as canonical metadata rather than a reason to reject otherwise usable history. Restore topology from the declared PAGE sequence while keeping record framing, routing keys, and sequence boundaries strict.
Keep the standalone style and hyperlink codecs strict while allowing PAGE and SCREEN decoders to discard invalid optional data at boundaries they own. Normalize invalid styles to defaults, ignore unrepresentable hyperlinks, reuse duplicate entries, and validate hyperlink values before encoding.
Keep SCREEN encoding strict while allowing decoding to recover from unknown or noncanonical semantic state. Cursor positions now clamp to the restored active area, and invalid enum values, reserved bits, and optional state degrade to native defaults.
Keep snapshot grid encoding strict by rejecting malformed wide-cell relationships before they can produce invalid wire data.
Decode untrusted grids liberally while preserving record alignment. Unknown semantic values and content kinds degrade to safe defaults, optional graphemes and hyperlinks are dropped when invalid or over capacity, and malformed wide-cell markers normalize to narrow cells.
Treat Kitty virtual placeholder codepoints as ordinary valid grid content during snapshot restore and derive the native row lookup hint from decoded cells. Image and placement registries remain intentionally omitted.
Cover the behavior with a complete snapshot round trip containing a real virtual placement and its grapheme diacritics.
Run the snapshot Kaitai verifier in its own required xsm job. This gives schema, fixture, checksum, and cross-record validation a distinct CI result without coupling it to the libghostty-vt test suite.
Describe the complete version 1 snapshot format with a Kaitai schema and make every golden fixture self-describing for automatic discovery. Add a verifier that compiles the schema, parses all fixtures, and checks record checksums, checkpoint digests, and cross-record invariants.
Preserve Kaitai metadata when generating fixture candidates and provide the compiler and Python runtime dependencies through the development shell. Keep the mode registry portable to Kaitai JavaScript targets so the complete fixture also works in the web IDE.
Extracted out the raw `src/terminal` changes needed for the future
snapshot work, 4 separate changes. These are uncontroversial and
relatively simple, summarized below. Tests AI assisted but the rest
including commit messages, this PR message, etc. all organic.
* **Add iterator to ref counted set.** Iterate over live entries and
their IDs. Const, doesn't mutate the set.
* **lib.Enum produces stable enums for Zig.** Basically the same as C
except it uses the smallest fitting integer including the holes.
* **PageList: a couple helpers for manually creating pages.** There is
`PageList.Builder` for creating a new pagelist and
`PageList.allocatePage` for modifying an existing one. This allows
PageList construction from raw pages.
Rename Builder.addPage and PageAllocation.cancel to their consistent allocatePage and deinit forms. Track successful ownership transfers so both builder APIs can use unconditional deferred cleanup without releasing pages transferred to a PageList.
Fixes#13522
Fixes unreachable when reflow dupes a hyperlink into a destination page
whose string allocator is nearly full.
The capacity precondition in ReflowCursor.writeCell performed a single
test allocation of `uri.len + id.len` bytes before duping a hyperlink
into the destination page. But PageEntry.dupe allocates the URI and the
explicit ID as two separate allocations, and the string allocator rounds
every allocation up to its 32-byte chunk size independently, so the two
separate allocations can require one more chunk than the single combined
test allocation.
Write a new helper to make sure we get the right amount of space using
the same allocation pattern of dupe.
**AI note:** Verified upstream via Fable. I told it to ignore any
conclusions and do its own validation and fix suggestion. It did
validate it with a failing test which I studied. It implement a fix, I
rewrote it to be more idiomatic.
Fixes#13522
Fixes unreachable when reflow dupes a hyperlink into a destination page
whose string allocator is nearly full.
The capacity precondition in ReflowCursor.writeCell performed a single
test allocation of `uri.len + id.len` bytes before duping a hyperlink
into the destination page. But PageEntry.dupe allocates the URI and
the explicit ID as two separate allocations, and the string allocator
rounds every allocation up to its 32-byte chunk size independently, so
the two separate allocations can require one more chunk than the
single combined test allocation.
Write a new helper to make sure we get the right amount of space
using the same allocation pattern of dupe.
This change improves the user experience for Pi TUI users on macOS.
As a user of Ghostty 1.3.1, Pi 0.80.7, and macOS 26.5, I noticed that
Command-click was not working.
With Pi's help (GPT-5.6 Terra High), I narrowed the cause down to Pi's
redraw
behavior and `src/config/url.zig`'s regular expression. More details are
in
[Vouch Request
#13491](https://github.com/ghostty-org/ghostty/discussions/13491).
The `trailing_spaces_at_eol` behavior in `src/config/url.zig` was
introduced in
[PR #9921](https://github.com/ghostty-org/ghostty/pull/9921) while
improving
Command-click handling for relative and local paths. The concern about
matching
trailing whitespace was also noted in [a review
comment](https://github.com/ghostty-org/ghostty/pull/9921#issuecomment-3661107609).
However, supporting file paths with trailing spaces does not seem like a
good
trade-off because it blocks Command-click for file paths displayed by Pi
TUI.
This PR removes that behavior.
I tested this on my Mac with a patched Ghostty build, and Command-click
worked
correctly for file paths in Pi TUI.
AI disclosure: I used Pi with GPT-5.6 Terra High to investigate and
implement this change.
I reviewed the code and tested the result myself.
I've come up with a way to avoid manually allocating each entry which
honestly makes the code flow much more smoothly. Basically you collect
all the applicable keybinds first, then try to bind them with their
stable memory addresses.
I've come up with a way to avoid manually allocating each entry which
honestly makes the code flow much more smoothly. Basically you collect
all the applicable keybinds first, then try to bind them with their
stable memory addresses.
reword: The doc said "macOS doesn't need any dependencies" and then
immediately listed things you needed to install for macOS 😁. This is
just rewording the doc to be more consistent.
