Restore the error handling that the removed std.posix fork and waitpid
wrappers previously provided. Raw fork failures now propagate, waitpid
retries interruptions before reading status, and edit-config constructs the
sentinel-terminated argv required by execve.
Let global initialization own cleanup through its existing errdefer so
temporary paths are freed once. Report initialization failures with the
static synchronous I/O provider because global I/O has already been torn
down by that point.
Zig 0.16.0 made the criteria for reporting "failed command" stricter (or
looser, depending on your perspective I guess...) - now, tests that
print anything to stderr cause the message to appear.
Note that in this instance tests still pass and you get a return code of
0, but nonetheless, it can be confusing.
Additionally, having spammy passing tests in general is not necessarily
a great experience, so this should help with that.
Note that this change was already done to the main tests. We can add a
build argument to control this if need be.
The early-stage main Zig wrapper recognizes if main only needs the
minimal state (args and lower-level environment) and skips a bunch of
unneeded initialization (allocator, arena, threaded I/O, and the
higher-level environment map). Particularly, the fact that it does not
set up an I/O instance means that we won't have any unneeded signal
handlers set up for the unused threaded I/O implementation, which is
similar in spirit to the fixes we applied for the C VT implementation,
with the notable difference that we do actually set a threaded I/O up in
global state - hence, again, we don't want the duplicate unused one.
The C API is assumed to be single-threaded per VT instance.
Additionally, using fully-threaded I/O instances registers signal
handlers, and would do a pair of registrations once per instance, which
could easily get out of hand (and is not really what we intend anyway).
init_single_threaded does not register signal handlers, so it does not
have this issue, and matches the execution model of the C VT API
(single-threaded/not thread-safe within a single VT instance).
This also fixes an initialization issue with the threaded I/O instance
in general (needs allocation as the memory location would have gone out
of scope before).
- benchmark: avoid buffers to avoid a memcpy
- build: keep frame pointers on macOS. There was some debug changes from
Zig 0.15 and this helps. Also, Apple actually requires/expects x29 to
always be a frame pointer.
- build/macos: force libSystem symbols instead of compiler-rt
- global: add InitOpts.tool so that ghostty-gen/bench can parse their
own actions in `+action`
- quirks: provide our own vectorized memset. see the comment for more
details why.
- synthetic: fix UB by accessing global.io before it was initialized
- terminal/hash_map: force inline for unique repr types. Zig 0.15
inlined and 0.16 doesn't, measured a huge slowdown in hyperlink
benchmarks.
- terminal: add explicit `@Vector` usage for storing a run of identical cells
as well as for scanning printable cells. This auto-vectorized in Zig
0.15 but not in Zig 0.16. This produces the same assembly.
- unicode: properties and LUT need power-of-two backing integer to avoid
bad LLVM codegen
This commit represents the majority of the work necessary to upgrade
Ghostty to use Zig 0.16.0.
Key parts:
* In addition to its previous responsibilities, the global state now
houses state for global I/O implementations and the process
environment. It is now also utilized in the main application along
with the C library. Where necessary, global state is isolated from key
parts of the implementation (e.g., in libghostty subsystems), and it's
expected that this list will grow.
* We currently manage our own C translation layer where necessary. In
these cases, cImport has been removed in favor of the new external
translate-c package. Due to fixes that have needed be made to properly
translate the dependencies that were swapped out, as mentioned, we
have had to backport fixes from the current translate-c package (and
the upstream Arocc dependency). We will host this ourselves until Zig
0.17.0 is released with these fixes.
* Where necessary (only a small number of cases), some stdlib code from
0.15.2 (and even from 0.17.0) has been taken, adopted, and vendored in
lib/compat.
Co-authored-by: Leah Amelia Chen <hi@pluie.me>
Re: #13160 (related but not that issue)
PageList eraseRow and eraseRowBounded have the same issue previously
fixed for cursorScrollAbove: when shifting rows up across a page boundary,
the top row of the next page is cloned into the last row of the
previous page, and that clone can fail if the destination page lacks
capacity for the row's managed memory.
Handle the errors the same way the other cross-page copies do:
increase the destination page capacity for the dimension that ran
out and retry the row copy.
This type of logic was repeated EVERYWHERE so I extracted this into a
helper in PageList and Screen. They're slightly different due to the extra
accounting that Screen has to do for the cursor.
Don't know of any scenario this actually happened in the real world but
it was trivially reproducible with tests.
Kitty 0.48 added support for usage hints in the image protocol,
specifically for marking images as "transient", meaning that they should
be prioritized for eviction if there is memory pressure.
https://sw.kovidgoyal.net/kitty/graphics-protocol/#image-usage-hints
Also changed the eviction algorithm to use an allocated array for
organizing the images to be evicted rather than using an ArrayList to
minimize the number of allocations made (no real memory savings though).
Re: #13160
When cursorScrollAbove rotates rows across a page boundary, the last row
of the previous page is cloned into the destination page. That can cause
capacity failures we didn't previously handle.
The error propagated out of the operation after rows had already been
rotated, leaving the page list half-mutated. Subsequent operations on
the corrupted state can cause crashes since the state was incoherent.
Handle these errors the same way insertLines and deleteLines already do
for their cross-page copies: increase the destination page capacity for
the dimension that ran out and retry the row copy.
Re: #13160
When cursorScrollAbove rotates rows across a page boundary, the last
row of the previous page is cloned into the destination page. That can
cause capacity failures we didn't previously handle.
The error propagated out of the operation after rows had already been rotated,
leaving the page list half-mutated. Subsequent operations on the corrupted state
can cause crashes since the state was incoherent.
Handle these errors the same way insertLines and deleteLines already
do for their cross-page copies: increase the destination page capacity
for the dimension that ran out and retry the row copy.
Kitty 0.48 added support for usage hints in the image protocol,
specifically for marking images as "transient", meaning that they
should be prioritized for eviction if there is memory pressure.
https://sw.kovidgoyal.net/kitty/graphics-protocol/#image-usage-hints
Also changed the eviction algorithm to use an allocated array for
organizing the images to be evicted rather than using an ArrayList to
minimize the number of allocations made (no real memory savings though).
`ext-background-effect-v1` is finally seeing major adoption throughout
KWin, Mutter, cosmic-comp, Niri, etc. and we should update our comments
on that.
`ext-background-effect-v1` is finally seeing major adoption throughout
KWin, Mutter, cosmic-comp, Niri, etc. and we should update our comments
on that.
Cursor defaults were duplicated across stream handlers and it was a
pretty significant amount of simple and yet non-trivial logic to
understand.
Store these on Terminal itself and have methods to route things like
DECSCUSR through for consistent behaviors.
The libghostty-vt stream is made to be infallible: in the case of any error
it just logs and moves on. That's because a terminal can't really... stop,
under normal operations. But, under special operations (fuzzing, replays,
etc.) it can and should stop!
Rather than make the operation fallible, its simply enough for me at least
to know that something went wrong. This is a simple change that adds a simple
flag that is flagged to true when such a scenario happens.
For normal Ghostty GUI operations, this isn't used at all. For libghostty
consumers they can choose to read it if they want, but don't have to.
This also adds a C API to read it.
Terminal resize could leave tab stops, pixel geometry, synchronized output,
or screen dimensions partially updated when a later allocation failed. This
is now safe. There is one exception, see the code comments.
This makes the Terminal.resize handle more of the common elements that
a core terminal emulator should: cell geoemtry handling (if exists),
updates synchronized output modes.
This adds a new TerminalStream.resize that also handles the side effects
for more easy integration into downstream libghostty-vt consumers, namely
mode 2048 in-band signaling handling.
Using `clearRetainingCapacity` as it was being used in
Fingerprint.update produces undefined behavior; both the old "copy" and
current version of the fingerprint would still be using the same
storage.
This commit changes the function so that the storage is more clearly
re-used, and shrunk at the end if need be.
Retain the existing fingerprint's storage instead of allocating a fresh
owned slice on every update. The previous approach built a replacement
fingerprint before it knew whether the viewport changed, discarding it
for unchanged viewports.
The fingerprint now rebuilds in place after reserving viewport-sized
capacity and reports whether its contents changed. Unchanged viewport
updates skip the allocation, while changed viewports will usually reuse
the existing buffer for the lifetime of the active search.
Page maps handled removal with tombstones, which a fixed-capacity map
can never outgrow. Keeping probe lengths bounded required an insertion
headroom counter, an allocation-free rehash, and five separate recovery
paths with subtle invariants: removal created a tombstone without
restoring headroom, so the counter could reach zero while the map was
half empty and every insertion path had to be prepared to rebuild.
Replace tombstones with backward-shift deletion (Knuth vol. 3, 6.4
algorithm R). Removal restores the table to the state it would be in
had the key never been inserted, so probe chains stay canonical at all
times and fragmentation cannot accumulate by construction. This deletes
the headroom counter, the in-place rehash, and every recovery path.
Insertion no longer invalidates pointers; removal may now move other
entries instead, and no caller holds entry pointers across removals.
Removal costs a cluster scan instead of a byte write, paying
incrementally what tombstones deferred to later probes and periodic
rebuilds. Free slots remain all-zero bytes so probe loops keep fusing
the state and fingerprint checks into single-byte compares. A
randomized oracle test against the stdlib map and a canonical-placement
invariant check cover the new removal path, including full tables where
no free slot terminates the shift.
ReleaseFast benchmarks against the prior map commits:
| workload | delta |
|---|---:|
| map churn, 50% load | 1.23x faster |
| map churn, 75% load | 1.15x faster |
| map churn, max load | 1.07x slower |
| map lookup | neutral |
| clear and redraw stream | 1.32x faster |
| full OSC 8 stream | 1.06x faster |
| linked-line movement | neutral |
Hyperlink and grapheme maps are keyed by cell offset. Moving their
data removes the source entry and reinserts it at a destination known
to be absent. Generic clobbering insertion still searches the probe
sequence to rule out a duplicate.
Use no-clobber insertion for these moves so the first available
tombstone can be reused. If a destination reaches a free slot after
tombstones exhaust insertion headroom, rebuild the map in place and
retry before decrementing the budget. This keeps the known-absent fast
path safe at every load state.
The ReleaseFast movement benchmark improves from 133.7 ms to 126.4 ms,
a 5.5% reduction. The linked-line control remains neutral at 395.4 ms
before and 395.7 ms after.
#13292
Page maps previously allowed a 100% load factor. Once live entries
and tombstones filled every slot, a missing-key lookup or insertion
could scan the entire map.
Reserve 20% of each offset hash map as insertion headroom and track
that budget separately from the live count. Port the allocation-free
in-place rehash from Zig HashMapUnmanaged so canonical insertion can
rebuild fragmented probes and restore tombstone-exhausted headroom
without growing the page. Assumed-capacity insertion now applies the
same guard, preventing the headroom counter from wrapping.
Pass the exact requested hyperlink count into map layout before
load-factor scaling, avoiding a redundant power-of-two rounding step.
Screen-level recovery now grows only when live hyperlinks actually fill
usable capacity.
ReleaseFast terminal benchmarks compare main with the combined map
changes:
| workload | before | after | speedup |
|---|---:|---:|---:|
| map churn | 1.253 s | 12.5 ms | 100x |
| full OSC 8 stream | 3.576 s | 75.6 ms | 47x |
| clear and redraw | 299.1 ms | 118.3 ms | 2.5x |
Co-authored-by: Tim Culverhouse <tim@timculverhouse.com>
This changes our `page_serial` semantics from monotonic min to a
generation marker (any change means you should reload). This fixes a
number of real bugs that search had.
In reality, this invariant was already broken and not true. So this PR
comes to reality with that and fixes all our usage. The most visible
failures were panics when selecting cached search results after
partially erasing a history page or splitting a page, but the same bug
class affected viewport search fingerprints and asynchronous render
highlights.
It was mostly search, for now, but this type of fix is important for
some upcoming work I'm playing around with regarding deferred reflow and
disk offload.
## Page Serial Semantics
`Node.serial` is now explicitly a page generation. It changes whenever a
node is allocated or reused, and whenever an in-place mutation changes
the meaning or valid range of its row coordinates.
`page_serial_min` is renamed to `page_serial_epoch` to reflect its
remaining purpose. Only a whole-list reset advances the epoch. A serial
below the epoch is definitely stale and can be rejected in O(1); a
serial at or above it is only potentially valid and is checked against
the live list using its captured pointer and generation.
## Performance
The generation bump is constant work per node whose layout is already
being changed. There is no additional page-list scan on normal row
append or scrollback pruning.
I did add a new field to the render state, so I ran render state
benchmarks we already have that shows a very slight slowdown but this is
acceptable especially given recent speedups outweigh this significantly:
| render measurement | before | after | change |
|--------------------|--------|-------|--------|
| full rebuild | 2.915 us/update | 2.926 us/update | +0.27% |
| lock-held rebuild | 2.472 us/update | 2.489 us/update | +0.53% (within
noise) |
| clean update | 19.03 ns/update | 19.30 ns/update | +1.43% (+0.27 ns) |
| one dirty row | 52.97 ns/update | 53.40 ns/update | +0.96% (within
noise) |
| flattened highlight miss | 86.77 ns/update | 86.87 ns/update | +0.19%
(within noise) |
| flattened highlight match | 88.38 ns/update | 90.26 ns/update | +2.00%
(+1.88 ns) |
## LLM Notes
This work was done in concert with Codex. I reviewed and reshaped the
serial model, implementation, tests, documentation, and performance
analysis throughout. This PR message is hand-written.
reloadActive previously pruned every cached history result whenever the
active area changed. Search reconciliation runs under the terminal lock,
so this scaled with both result and page counts on a frame-paced path.
Validate only the selected history result during reload. Keep full
pruning before selection navigation so stale candidates are removed
before their coordinates are tracked.
Screen scroll fast paths and Terminal line insertion/deletion move Row
values directly instead of using PageList erasure helpers. Their node
generations therefore stayed valid after cached row coordinates changed.
Expose the PageList layout invalidator to sibling terminal modules and
renew each affected existing page once before full-row rotations or
swaps. Keep partial-width cell moves on the content-only path and cover
same-page, cross-page, and fresh-tail behavior.
libghostty-vt was already parsing OSC 52 into the clipboard_contents
action but the stream handler dropped, so there was no way to observe
clipboard writes (in this case, a program using go-libghostty). This
adds a clipboard_set effect following the existing bell/title_changed
pattern and expose it through the C API as
`GHOSTTY_TERMINAL_OPT_CLIPBOARD_SET`.
The callback receives the OSC 52 kind byte and the base64 payload
exactly as received; decoding and kind interpretation are left to the
embedder, matching how decoding is typically deferred.
Note this intentionally does not deal with clipboard read requests given
the security implications.
AI disclosure: Fable (via Claude Code) did the majority of the work
here, I validated it on the client side and fully understand the pattern
we're fitting into.
#13182
Replace the OSC 52-specific kind and encoded payload callback with an
atomic clipboard write containing a normalized destination and decoded
MIME representations. This keeps protocol details out of embedders and
lets iTerm2 Copy use the same semantic path.
Represent clears with an empty content list, preserve binary payloads,
and return a generic result for protocols that acknowledge writes. Add
the C ABI descriptors, layout metadata, and effects example so future
multipart protocols can reuse the callback without another API break.
page_serial_min no longer represented the minimum live page serial.
Its name still suggested an ordering relationship and obscured the
remaining reset-only invalidation behavior.
Rename the field to page_serial_epoch and document its O(1) rejection
and ScreenSearch bulk-pruning utility. Explicitly verify that ordinary
bounded pruning does not begin a new whole-list epoch.
Single-row erasure and trailing blank-row trimming can remap or remove
stored row coordinates while retaining the same node. External pointer-
plus-generation references could therefore survive these less common
PageList mutation paths.
Renew each affected node inside the traversal that already changes it,
and signal compression activity only when row erasure begins in
history. This adds constant work per touched page without another list
scan or normal append cost.
Compaction and capacity growth publish fresh page generations, so an
active incremental traversal can detect them. They did not update the
separate activity token, however, and a completed compressor could
remain idle after either operation restored or replaced a cold page.
Mark successful replacements as compression activity without resetting
the exact continuation marker. The next scheduled step can retain valid
progress or restart through the existing generation checks.
ScreenSearch reloaded active state before pruning stale flattened
history. Reload could compare a shifted tracked selection against
cached page coordinates and panic before the later validation step ran.
Prune history immediately after dimension reconciliation in
reloadActive, before any cached coordinate is inspected or converted to
a pin. Selection now relies on that ordering and avoids a redundant
second prune.
RenderState applied flattened highlights after releasing the terminal
lock and matched them to copied rows by node address alone. A recycled
address could therefore make a stale asynchronous highlight decorate
unrelated content.
Capture the live node generation with each render row and require both
the pointer and copied generation to match. Validation remains lock-
free and never dereferences a potentially stale node.
ViewportSearch compared only cached node addresses when deciding
whether to reuse its owned search window. An in-place layout change or
pool address reuse could therefore make stale text and coordinates
appear current.
Snapshot each node generation with its pointer and compare only those
captured values. This detects a renewed generation at the same address
without ever dereferencing cached node pointers.
Reverse multi-page highlight construction reordered node and row-
bound columns but left captured page generations in their original
order. Every cross-page result therefore paired each node with another
page generation and could be rejected as stale despite remaining live.
Reverse the serial column with the other flattened chunk metadata and
cover the node-plus-generation pairing in the existing boundary match
test.
PageList.split moved the source suffix into a fresh target but left
the shortened source on its old generation. Cached matches in that
suffix could therefore pass validation against the live source pointer
and reach pin tracking with invalid coordinates.
Renew the source generation only after target cloning succeeds, and
mark compression activity so restored history is reconsidered. The
conservative floor keeps bounded pruning safe even when the renewed
source and fresh target precede older successors.
Partial history erasure shifted retained rows while preserving the
node generation. Cached flattened matches then passed pointer-plus-
generation validation and could be tracked with coordinates beyond the
shortened page.
Renew the generation before reinitializing or shifting a page layout,
and mark compression activity so an incremental pass restarts and
revisits a restored cold page. The conservative serial floor permits
the fresh generation to remain before older live successors without
rejecting them.
Page generations are not ordered with the page list because splits
and in-place replacements insert fresh generations before older live
pages. Advancing page_serial_min while pruning could therefore reject
live successors and fail PageList integrity checks.
Keep the floor as a whole-list invalidation epoch and use the existing
pointer-plus-generation membership check for ordinary removals. Add
bounded pruning coverage for split and replacement ordering, and verify
reset still rejects a stale generation when its node address is reused.
Kitty graphics payloads are dispatched in bulk, but finding each slice
boundary still examines every byte with a scalar loop. This leaves large
direct base64 image transmissions parser-bound.
Scan ordinary APC bytes using the vector width recommended for the compile
target. Keep the scalar scan both as the tail and as the full fallback when
the target has no recommended vector width. Test state-machine boundaries
against byte-at-a-time parsing.
A ReleaseFast APC parser benchmark over the same 64 MiB Kitty graphics
corpus, with 10 warmups and 30 measured runs, produced:
mean median
scalar 37.6 ms 32.6 ms
vectorized 22.3 ms 19.0 ms
Hyperfine reports the vectorized version as 1.69 times faster overall, with
the median runtime improving by approximately 42 percent.
Count-limited PageIterator traversal took the minimum of the page and requested lengths, then tested whether that minimum exceeded the request. The condition was impossible, so iteration never crossed a page. Reverse traversal also excluded the current row and subtracted one from row zero, causing a runtime safety panic.
Count the current row in both directions, consume the returned length, and move to an adjacent page only when the request has rows remaining. Returned chunks now preserve their half-open bounds at row zero and across page boundaries.
Search selection could run after the terminal removed a screen but before the next refresh reconciled the cached searchers. Reloading that stale ScreenSearch dereferenced freed PageList nodes, while normal cleanup also tried to untrack pins from the destroyed list.
Reconcile under the terminal lock before selecting, track ScreenSet generations so allocator address reuse cannot hide replacements, and release stale search buffers without touching pins already freed with the screen. Cleanup now takes the same lock when live pins must be untracked.
History pruning only compared cached serials with page_serial_min. Replacing a historical node through compaction left its old serial above that cutoff, so selecting the cached match attempted to track a destroyed node and hit the PageList validity assertion.
Validate every flattened chunk by finding the live node and matching its serial without dereferencing stale pointers. Remove only the invalid result and adjust any later selection index so unrelated older results remain available.
PageListSearch.feed changed only the node of its tracked progress pin. If the preceding history page had fewer rows or columns after a split, the retained bottom-right coordinates fell outside that page and the next PageList integrity check panicked.
Reset both coordinates to the new node's actual bottom-right cell whenever feed advances. The progress pin now remains valid across heterogeneous page sizes.
Partial history erasure can remove a page without marking tracked pins as garbage because they move coherently to the next page. ScreenSearch therefore retained a selection whose cached history result was subsequently removed by pruneHistory. Selecting again indexed an empty history result list and panicked.
Clear the tracked selection when its combined result index falls within the history suffix being pruned. Retained active and newer history selections keep their existing indices.
