This adds opt-in continuation tracking to `terminal.Stream` that allows
any caller to call `writeContinuation` in order to get the minimum bytes
necessary from a grounded parser state to the identical state.
This enables reliable stream restart across serialization states, which
could be used for local restart, networked terminals, etc. For me, this
is used for multiplexers. :)
**LLM usage:** I wrote the continuation tracker myself, used a mix of
5.6+Fable to review it for me, applied their feedback directly. Only
place with predominantly AI code are tests, which I reviewed. Commit and
PR message written myself.
## Implementation
The implementation of this was really carefully done to avoid any
negative performance impact particularly when continuation tracking is
_off_.
The way this work is simple:
1. ESC is the only char that leaves the ground state and most ESC
sequences are short. So if we're in a non-ground state, we do a
backwards vectorized search to find the last `ESC` in the input slice.
If one doesn't exist, we assume we found it previously and store the
whole slice (rare, since ESC sequences are usually short like I said).
2. If we're in the ground state that means we only have a potential
incomplete UTF-8 codepoint, so we find the lead UTF-8 byte.
3. When writing, we normalize the suffix to drop things like BEL
commands that would've already been handled to avoid double-calling.
## Performance
No real impact.
Via `ghostty-bench +terminal-stream`.
Corpus | Main | PR w/ Tracking Off | PR w/ Tracking On
-- | -- | -- | --
Plain ASCII (256 MiB) | 175.4 ms | 175.8 ms | 175.5 ms
UTF-8 (32 MiB) | 268.4 ms | 270.0 ms | 269.9 ms
5% invalid UTF-8 (32 MiB) | 316.4 ms | 317.8 ms | 320.2 ms
CSI-heavy (32 MiB) | 145.0 ms | 146.3 ms | 145.9 ms
OSC (32 MiB) | 1621.0 ms | 1627.5 ms | 1638.3 ms
Kitty APC (128 MiB) | 95.5 ms | 96.9 ms | 96.3 ms
Mixed traffic (32 MiB) | 172.4 ms | 172.0 ms | 172.3 ms
Giant APC (128 MiB) | 38.3 ms | 38.3 ms | 40.8 ms
This adds opt-in continuation tracking to `terminal.Stream` that allows
any caller to call `writeContinuation` in order to get the minimum bytes
necessary from a grounded parser state to the identical state.
This enables reliable stream restart across serialization states, which
could be used for local restart, networked terminals, etc. For me, this
is used for multiplexers. :)
## Implementation
The implementation of this was really carefully done to avoid any
negative performance impact particularly when continuation tracking is
_off_.
The way this work is simple:
1. ESC is the only char that leaves the ground state and most
ESC sequences are short. So if we're in a non-ground state, we
do a backwards vectorized search to find the last `ESC` in the
input slice. If one doesn't exist, we assume we found it previously
and store the whole slice (rare, since ESC sequences are usually
short like I said).
2. If we're in the ground state that means we only have a potential
incomplete UTF-8 codepoint, so we find the lead UTF-8 byte.
3. When writing, we normalize the suffix to drop things like BEL
commands that would've already been handled to avoid
double-calling.
## Performance
Via `ghostty-bench +terminal-stream`
Corpus main tracking off tracking on
plain ASCII (256 MiB) 175.4ms 175.8ms 175.5ms
UTF-8 (32 MiB) 268.4ms 270.0ms 269.9ms
5% invalid UTF-8 (32 MiB) 316.4ms 317.8ms 320.2ms
CSI-heavy (32 MiB) 145.0ms 146.3ms 145.9ms
OSC (32 MiB) 1621.0ms 1627.5ms 1638.3ms
Kitty APC (128 MiB) 95.5ms 96.9ms 96.3ms
mixed traffic (32 MiB) 172.4ms 172.0ms 172.3ms
giant APC (128 MiB) 38.3ms 38.3ms 40.8ms
Adds "Copy" and "Export to file" buttons to the Terminal IO inspector
so recorded VT events can be saved outside the app for sharing or
analysis.
Export is wired up through a new export_terminal_io apprt action,
handled with a native save dialog on both macOS and GTK.
Scrollback limits were fixed when a terminal screen was initialized.
Move byte and line state and enforcement into a shared Limits type so
PageList can update both constraints after initialization and resize.
Add runtime setters to PageList and Terminal. Lowering a limit prunes
eligible history immediately, while zero bytes switches the primary
screen to no-scrollback behavior and clears retained history. Keep
alternate screens unchanged.
This commit represents the majority of the work necessary to upgrade
Ghostty to use Zig 0.16.0.
Key parts:
* In addition to its previous responsibilities, the global state now
houses state for global I/O implementations and the process
environment. It is now also utilized in the main application along
with the C library. Where necessary, global state is isolated from key
parts of the implementation (e.g., in libghostty subsystems), and it's
expected that this list will grow.
* We currently manage our own C translation layer where necessary. In
these cases, cImport has been removed in favor of the new external
translate-c package. Due to fixes that have needed be made to properly
translate the dependencies that were swapped out, as mentioned, we
have had to backport fixes from the current translate-c package (and
the upstream Arocc dependency). We will host this ourselves until Zig
0.17.0 is released with these fixes.
* Where necessary (only a small number of cases), some stdlib code from
0.15.2 (and even from 0.17.0) has been taken, adopted, and vendored in
lib/compat.
Co-authored-by: Leah Amelia Chen <hi@pluie.me>
The PageList overview mixed structural state with a long list of exact
byte counters, making the compression result difficult to interpret.
Keep the overview focused on grid and scrollback structure, and add a
dedicated compression section before scrollbar details. Present page
states, uncompressed size, encoded ratio, resident estimate, and savings
using readable units and scoped help text.
Scrollback compression was available only through explicit PageList
calls, so normal renderer-backed surfaces never reclaimed cold history.
Debounce renderer wakeups with a one-shot timer and run bounded
compression steps only when the terminal lock is immediately available.
Timer state is isolated in the renderer and becomes idle once PageList
reports that the pass is complete.
Keep inspector reads representation-preserving by decoding compressed
pages into temporary copies. Update the scrollback configuration and
benchmark documentation to describe the production behavior and its
memory accounting.
Cold-history compression previously required scanning every eligible
page in one call, which makes it unsuitable for an idle-time scheduler.
The inspector also restored compressed pages while traversing collapsed
entries, hiding the representation and undoing reclamation.
Add caller-owned serial state that resumes compression without retaining
node pointers. Each invocation inspects at most eight candidates and
attempts one resident page. List mutations restart safely, while
unsupported or historical viewports stop work. Keep a stateless
whole-history operation for measurement.
Expose metadata-only storage and memory accounting for diagnostics,
update the inspector to restore only expanded pages, and add an
incremental live scrollback benchmark. This remains disconnected from
production scheduling.
PageList nodes previously exposed Page directly, so introducing a
compressed representation would require every consumer to understand its
state and ownership.
Add resident and compressed node states behind a page access boundary.
Content access transparently recommits and restores retained mappings,
while metadata traversal stays compressed and lifecycle paths can
discard encoded contents without decoding. Compression borrows pool
memory for standard scratch and uses temporary aligned storage for
oversized pages.
Migrate terminal, rendering, search, formatting, and C API consumers to
the new boundary. Hot grow, scroll, and print paths reuse resolved
pages, with an explicit resident-only accessor where live cursor
pointers prove that the page cannot be compressed.
The compression entry point remains private with no production callers,
so normal terminal behavior and scrollback accounting are unchanged.
ReleaseFast terminal-stream comparisons across bulk output, scrolling,
redraw, and erase workloads remain within 2% of the parent revision.
Selection gestures now treat releases with invalidated anchors as dragged,
so a press that crosses screen boundaries cannot also activate links or
prompt clicks on release. Cell drags that create a same-cell selection also
mark the gesture as dragged, which keeps click-only actions from firing
after a threshold-crossing drag.
Autoscroll now resolves the drag pin after moving the viewport instead of
reusing the pin from before the scroll. This keeps the selection aligned
with the row currently under the pointer. The inspector also validates the
tracked click pin before displaying it so stale pins from inactive screens
are ignored.
The terminal.Stream next/nextSlice functions can now no longer fail.
All prior failure modes were fully isolated in the handler `vt`
callbacks. As such, vt callbacks are now required to not return an error
and handle their own errors somehow.
Allowing streams to be fallible before was an incorrect design. It
caused problematic scenarios like in `nextSlice` early terminating
processing due to handler errors. This should not be possible.
There is no safe way to bubble up vt errors through the stream because
if nextSlice is called and multiple errors are returned, we can't
coalesce them. We could modify that to return a partial result but its
just more work for stream that is unnecessary. The handler can do all of
this.
This work was discovered due to cleanups to prepare for more C APIs.
Less errors make C APIs easier to implement! And, it helps clean up our
Zig, too.
