154ddc2a2f
This adds the first version of a binary snapshot format for terminal state. Use cases: replay software (like asciinema), multiplexers (like zmx), scrollback-saving on disk, etc. The intention of the binary snapshot format is to be able to fully encode and decode terminal state across mediums such as network and disk. You can also encode partial terminal state (e.g. only one screen or even one page of contents). Long term, the intention is to also support streaming state while a live terminal is running, but this initial PR focuses on the full snapshot first (with some design choices to get to the streaming state in the future). The format is documented in the Zig code, but I also did a [Kaitai](https://kaitai.io/) descriptor and both the Zig and Kaitai spec verify they can parse committed fixtures. This helps identify drift in the format or encoder/decoders in any way since this must ultimately be a fixed format. > [!NOTE] > > **On reviewability:** this is a massive PR that I don't expect anyone to reasonably review. I'm going through it line-by-line (again) but I purposely extracted any changes that affect other parts of Ghostty out to other already-merged PRs. This one is isolated purely to a package that isn't called by any client software. **So the plan is if this rough shape looks good I'll merge it and we'll iterate from there.** > [!WARNING] > > **Experimental.** The format can and will change. And we may also decide that binary snapshotting in this way isn't the right direction altogether (although, I'm pretty confident it is). It'd be impossible to get a single large perfect PR because it'd be even larger than this by multiples. So instead, we'll iterate on main so long as this work is not touching any production code, which it isn't! ## Example Encode: ```zig const terminal = @import("terminal/main.zig"); var file_buffer: [16 * 1024]u8 = undefined; var file_writer = file.writer(io, &file_buffer); try terminal.snapshot.encode(alloc, &file_writer.interface, &t); try file_writer.interface.flush(); ``` Decode a full terminal: ```zig var t = try terminal.snapshot.decode(&reader, io, alloc); defer t.deinit(alloc); ``` ## Future This PR purposely only supports a synchronous encode/decode. I wanted to get the large groundwork in before iterating further. Some iterations in the future: * Kitty graphics * Live terminal snapshotting * PTY stream continuation records (so VT state machines can stay in sync) * Performance work (encoding and decoding, maybe size) * Configurable limits to prevent DoS * C API * etc... ## Wire format The "robustness principle" is a guiding principle: "be conservative in what you do, be liberal in what you accept from others." Our encoders have a lot of extra validation, our decoders massage invalid data into reasonable defaults (e.g. invalid styles become unstyled text). > [!IMPORTANT] > > **Version 1 has no compatibility promise.** We use version 1 in the envelope header. We will absolutely break this format as needed as we iterate and improve on it... ### Envelope Every snapshot starts with a fixed ten-byte envelope: | Offset | Size | Field | | ---: | ---: | :--- | | 0 | 8 | Magic: `GHOSTSNP` | | 8 | 2 | Snapshot version: `1` | ### Record framing After the envelope, records are concatenated back-to-back. Every record has a fixed header: | Offset | Size | Field | | ---: | ---: | :--- | | 0 | 2 | Record tag | | 2 | 4 | Payload length | | 6 | 4 | CRC32C | | 10 | variable | Payload | CRC32C covers the encoded tag, payload length, and payload. The payload-length boundary prevents a malformed record decoder from consuming bytes belonging to the next record. The registered record tags are: | Value | Tag | Purpose | | ---: | :--- | :--- | | 1 | `TERMINAL` | Terminal-wide state and declared screens | | 2 | `SCREEN` | One screen's live state and active page manifest | | 3 | `PAGE` | One self-contained set of rows and cells | | 4 | `HISTORY` | One screen's historical page manifest | | 5 | `READY` | Digest of the renderable prefix | | 6 | `FINISH` | Digest of the complete snapshot | To view the format of each record, read its corresponding `terminal/snapshot/<type>.zig` file. ### Complete record sequence ```text +----------------------------------------+ | Envelope | +----------------------------------------+ | TERMINAL | +----------------------------------------+ | SCREEN * terminal.screen_count | | PAGE * each screen.page_count | +----------------------------------------+ | READY | +----------------------------------------+ | HISTORY * terminal.screen_count | | PAGE * each history.page_count | +----------------------------------------+ | FINISH | +----------------------------------------+ | Optional containing-transport bytes | +----------------------------------------+ ``` `SCREEN` and `HISTORY` groups are routed by their encoded screen key and may arrive in either key order. ## Checkpoints and validation Each record has an independent CRC32C, but per-record checksums cannot detect a valid record being reordered, omitted, or duplicated. `READY` and `FINISH` therefore contain BLAKE3-256 digests over exact snapshot prefixes: - `READY` covers the envelope, `TERMINAL`, and all live `SCREEN`/`PAGE` sequences. It does not include itself. - `FINISH` covers that same prefix, the complete `READY` record, and all `HISTORY`/`PAGE` sequences. It does not include itself. This gives the format two useful integrity boundaries: ```text envelope ... active pages | READY | history pages | FINISH <------ renderable -------> <------------- complete snapshot ---------------> ``` ## Performance ### Size, Compression Recommended We intentionally use a simple grid over something like RLE (run-length encoding). So every row contains exactly `columns` cells and each is 16-bytes! This is large! A 80x24, 10,000 line scrollback terminal uncompressed would be ~13MB. However, with zstd level 1 compression that goes down to 260K. ### Speed We haven't benchmarked encoding or decoding speed yet. This PR focused on getting a format in place. This will be heavily optimized later. I suspect its probably pretty darn slow, actually. ## Kaitai Struct I added a `snapshot.ksy` Kaita Struct spec that independently describes the complete format. This is used by us for format validation but it can also be used to programmatically generate parsers. For example, our test fixture in the Kaita Struct web IDE decodes to: <img width="523" height="779" alt="image" src="https://github.com/user-attachments/assets/cd199c73-b6d6-4b35-8957-0cfe3d1a18f2" /> **AI Usage:** This work was done in concert with various models and agents. Writing full encoders/decoders is tedious so it took a lot of that way. A lot of review was done by AI (trying to find holes, issues, inconsistencies). The actual binary protocol design and iteration was done by me. This PR message was written by me.
Ghostty
Fast, native, feature-rich terminal emulator pushing modern features.
A native GUI or embeddable library via libghostty.
About
·
Download
·
Documentation
·
Contributing
·
Developing
About
Ghostty is a terminal emulator that differentiates itself by being fast, feature-rich, and native. While there are many excellent terminal emulators available, they all force you to choose between speed, features, or native UIs. Ghostty provides all three.
libghostty is a cross-platform, zero-dependency C and Zig library
for building terminal emulators or utilizing terminal functionality
(such as style parsing). Anyone can use libghostty to build a terminal
emulator or embed a terminal into their own applications. See
Ghostling for a minimal complete project
example or the examples directory
for smaller examples of using libghostty in C and Zig.
For more details, see About Ghostty.
Download
See the download page on the Ghostty website.
Documentation
See the documentation on the Ghostty website.
Contributing and Developing
If you have any ideas, issues, etc. regarding Ghostty, or would like to contribute to Ghostty through pull requests, please check out our "Contributing to Ghostty" document. Those who would like to get involved with Ghostty's development as well should also read the "Developing Ghostty" document for more technical details.
Roadmap and Status
Ghostty is stable and in use by millions of people and machines daily.
The high-level ambitious plan for the project, in order:
| # | Step | Status |
|---|---|---|
| 1 | Standards-compliant terminal emulation | ✅ |
| 2 | Competitive performance | ✅ |
| 3 | Rich windowing features -- multi-window, tabbing, panes | ✅ |
| 4 | Native Platform Experiences | ✅ |
| 5 | Cross-platform libghostty for Embeddable Terminals |
✅ |
| 6 | Ghostty-only Terminal Control Sequences | ❌ |
Additional details for each step in the big roadmap below:
Standards-Compliant Terminal Emulation
Ghostty implements all of the regularly used control sequences and can run every mainstream terminal program without issue. For legacy sequences, we've done a comprehensive xterm audit comparing Ghostty's behavior to xterm and building a set of conformance test cases.
In addition to legacy sequences (what you'd call real "terminal" emulation), Ghostty also supports more modern sequences than almost any other terminal emulator. These features include things like the Kitty graphics protocol, Kitty image protocol, clipboard sequences, synchronized rendering, light/dark mode notifications, and many, many more.
We believe Ghostty is one of the most compliant and feature-rich terminal emulators available.
Terminal behavior is partially a de jure standard (i.e. ECMA-48) but mostly a de facto standard as defined by popular terminal emulators worldwide. Ghostty takes the approach that our behavior is defined by (1) standards, if available, (2) xterm, if the feature exists, (3) other popular terminals, in that order. This defines what the Ghostty project views as a "standard."
Competitive Performance
Ghostty is generally in the same performance category as the other highest performing terminal emulators.
"The same performance category" means that Ghostty is much faster than traditional or "slow" terminals and is within an unnoticeable margin of the well-known "fast" terminals. For example, Ghostty and Alacritty are usually within a few percentage points of each other on various benchmarks, but are both something like 100x faster than Terminal.app and iTerm. However, Ghostty is much more feature rich than Alacritty and has a much more native app experience.
This performance is achieved through high-level architectural decisions and low-level optimizations. At a high-level, Ghostty has a multi-threaded architecture with a dedicated read thread, write thread, and render thread per terminal. Our renderer uses OpenGL on Linux and Metal on macOS. Our read thread has a heavily optimized terminal parser that leverages CPU-specific SIMD instructions. Etc.
Rich Windowing Features
The Mac and Linux (build with GTK) apps support multi-window, tabbing, and splits with additional features such as tab renaming, coloring, etc. These features allow for a higher degree of organization and customization than single-window terminals.
Native Platform Experiences
Ghostty is a cross-platform terminal emulator but we don't aim for a least-common-denominator experience. There is a large, shared core written in Zig but we do a lot of platform-native things:
- The macOS app is a true SwiftUI-based application with all the things you would expect such as real windowing, menu bars, a settings GUI, etc.
- macOS uses a true Metal renderer with CoreText for font discovery.
- macOS supports AppleScript, Apple Shortcuts (AppIntents), etc.
- The Linux app is built with GTK.
- The Linux app integrates deeply with systemd if available for things like always-on, new windows in a single instance, cgroup isolation, etc.
Our goal with Ghostty is for users of whatever platform they run Ghostty on to think that Ghostty was built for their platform first and maybe even exclusively. We want Ghostty to feel like a native app on every platform, for the best definition of "native" on each platform.
Cross-platform libghostty for Embeddable Terminals
In addition to being a standalone terminal emulator, Ghostty is a
C-compatible library for embedding a fast, feature-rich terminal emulator
in any 3rd party project. This library is called libghostty.
Due to the scope of this project, we're breaking libghostty down into
separate libraries, starting with libghostty-vt. The goal of
this project is to focus on parsing terminal sequences and maintaining
terminal state. This is covered in more detail in this
blog post.
libghostty-vt is already available and usable today for Zig and C and
is compatible for macOS, Linux, Windows, and WebAssembly. The functionality
is extremely stable (since its been proven in Ghostty GUI for a long time),
but the API signatures are still in flux.
libghostty is already heavily in use. See examples
for small examples of using libghostty in C and Zig or the
Ghostling project for a
complete example. See awesome-libghostty
for a list of projects and resources related to libghostty.
We haven't tagged libghostty with a version yet and we're still working on a better docs experience, but our Doxygen website is a good resource for the C API.
Ghostty-only Terminal Control Sequences
We want and believe that terminal applications can and should be able to do so much more. We've worked hard to support a wide variety of modern sequences created by other terminal emulators towards this end, but we also want to fill the gaps by creating our own sequences.
We've been hesitant to do this up until now because we don't want to create more fragmentation in the terminal ecosystem by creating sequences that only work in Ghostty. But, we do want to balance that with the desire to push the terminal forward with stagnant standards and the slow pace of change in the terminal ecosystem.
We haven't done any of this yet.
Crash Reports
Ghostty has a built-in crash reporter that will generate and save crash
reports to disk. The crash reports are saved to the $XDG_STATE_HOME/ghostty/crash
directory. If $XDG_STATE_HOME is not set, the default is ~/.local/state.
Crash reports are not automatically sent anywhere off your machine.
Crash reports are only generated the next time Ghostty is started after a crash. If Ghostty crashes and you want to generate a crash report, you must restart Ghostty at least once. You should see a message in the log that a crash report was generated.
Note
Use the
ghostty +crash-reportCLI command to get a list of available crash reports. A future version of Ghostty will make the contents of the crash reports more easily viewable through the CLI and GUI.
Crash reports end in the .ghosttycrash extension. The crash reports are in
Sentry envelope format. You can
upload these to your own Sentry account to view their contents, but the format
is also publicly documented so any other available tools can also be used.
The ghostty +crash-report CLI command can be used to list any crash reports.
A future version of Ghostty will show you the contents of the crash report
directly in the terminal.
To send the crash report to the Ghostty project, you can use the following CLI command using the Sentry CLI:
SENTRY_DSN=https://e914ee84fd895c4fe324afa3e53dac76@o4507352570920960.ingest.us.sentry.io/4507850923638784 sentry-cli send-envelope --raw <path to ghostty crash>
Warning
The crash report can contain sensitive information. The report doesn't purposely contain sensitive information, but it does contain the full stack memory of each thread at the time of the crash. This information is used to rebuild the stack trace but can also contain sensitive data depending on when the crash occurred.