Commit Graph

16543 Commits

Author SHA1 Message Date
Mitchell Hashimoto
a55850c981 terminal: use previous page width for cursor cells
cursorCellEndOfPrev moved its pin to the previous row but then set the
column from the desired screen width. If incomplete reflow left that
previous page narrower, resolving the cell used an out-of-bounds column
and panicked in runtime safety builds.

Set the column from the page reached by the cursor pin so the returned
cell is always the actual final cell of that row.
2026-07-09 20:58:14 -07:00
Mitchell Hashimoto
a9f5b7eba2 terminal: clamp selection rows to page width
containedRowCached built full-row and rectangular selections using
the desired screen width. During incomplete reflow, an intermediate
narrower page received endpoints beyond its cell range, and consumers
could panic when resolving the returned pins.

Use the owning page width for full rows and clamp rectangular bounds to
that width. Every contained-row selection now returns resolvable pins.
2026-07-09 20:56:44 -07:00
Mitchell Hashimoto
fa8cae88b2 terminal: use destination width for line selection
Semantic line selection moved to the previous row when the next row
started with different content, then assigned the previous pin an x
coordinate from the next page. During incomplete reflow, a wider next
page produced an out-of-bounds pin and a runtime safety panic.

Set the end column from the page that owns the destination pin. Line
selection now remains valid while crossing mixed-width page boundaries.
2026-07-09 20:54:09 -07:00
Mitchell Hashimoto
3d08161b50 terminal: handle empty tabstop ranges
Tabstops.reset subtracted one from the column count before iterating
default stops. Although init and resize accept zero columns, resetting
that state with a nonzero interval underflowed and panicked.

Return after clearing when the grid has fewer than two columns. Empty
and single-column tabstop sets now preserve the normal no-stop result.
2026-07-09 20:50:23 -07:00
Mitchell Hashimoto
8f1c2fe959 terminal: handle backwards selection timestamps
SelectionGesture passed caller-supplied repeat timestamps directly to
Instant.since. A C API client or non-monotonic timer could provide an
earlier timestamp after a later one, causing a runtime safety panic
while converting negative elapsed seconds to u64.

Compare instants before calculating elapsed time and treat backwards
timestamps as failed repeats. The next press becomes a new single-click
anchor, matching other invalid repeat inputs.
2026-07-09 20:48:58 -07:00
Mitchell Hashimoto
0cb004734e terminal: ignore empty cell clears
Screen.clearCells accepted a slice but its runtime safety validation
indexed the first and last elements unconditionally. Passing an empty
range therefore panicked before the otherwise valid no-op clear.

Return immediately for an empty slice so validation and managed-memory
bookkeeping only run when there are cells to clear.
2026-07-09 20:38:29 -07:00
Mitchell Hashimoto
d6e24d9856 terminal: make pin traversal width-aware
Pin movement assumed every page had the same column count. During an
incomplete reflow, crossing into a narrower page could produce an
out-of-bounds x coordinate, while wrapped movement could land on the
wrong row or stop early.

Use destination page widths while moving vertically or wrapping, and
reject points that exceed the resolved page. Add synthetic mixed-width
coverage for movement, wrapping, overflow, and point conversion.
2026-07-09 20:38:28 -07:00
Mitchell Hashimoto
30b42f42a3 terminal: reject unrepresentable pin coordinates
pointFromPin accumulated scrollback rows directly into the u32 Y
field. An unbounded PageList with more than 2^32 rows could overflow
while converting a valid pin and panic in runtime safety builds.

Use checked additions for every cross-page row contribution. If the
pin cannot fit in point.Coordinate, return null through the existing
out-of-range result instead of trapping.
2026-07-09 20:38:18 -07:00
Mitchell Hashimoto
e6e4a9fdc1 terminal: widen cell screen coordinates
Cell.screenPoint accumulated page row counts in CellCountInt even
though screen point Y coordinates are u32. Once scrollback crossed
65,535 rows, walking back through page metadata overflowed and trapped
in runtime safety builds.

Accumulate directly in u32 so page-local u16 row counts widen before
addition and the result uses the full range promised by point.Coordinate.
2026-07-09 20:23:06 -07:00
Mitchell Hashimoto
afbf5ba156 terminal: handle minimum prompt scroll delta
Prompt scrolling negated negative deltas to count the requested jumps.
minInt(isize) has no positive signed representation, so a caller could
trigger a runtime safety panic before the search for an earlier prompt
started.

Use @abs to produce the full unsigned magnitude. An extreme negative
request now follows the normal prompt traversal and clamps at the oldest
available prompt.
2026-07-09 20:20:46 -07:00
Mitchell Hashimoto
c753fe4a4f terminal: handle minimum row scroll delta
PageList.scroll negated negative row deltas to obtain their
magnitude. minInt(isize) has no positive signed representation, so
callers could trigger a runtime safety panic before the existing
traversal had a chance to clamp at the top.

Use @abs to calculate an unsigned magnitude that represents every isize
value. The same value now drives both cached-pin and general traversal
paths.
2026-07-09 20:19:37 -07:00
Mitchell Hashimoto
35e1a0160c build(deps): bump cachix/install-nix-action from 31.10.6 to 31.10.7 (#13271)
Bumps
[cachix/install-nix-action](https://github.com/cachix/install-nix-action)
from 31.10.6 to 31.10.7.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/cachix/install-nix-action/releases">cachix/install-nix-action's
releases</a>.</em></p>
<blockquote>
<h2>v31.10.7</h2>
<h2>What's Changed</h2>
<ul>
<li>nix: 2.34.7 -&gt; 2.34.8 by <a
href="https://github.com/github-actions"><code>@​github-actions</code></a>[bot]
in <a
href="https://redirect.github.com/cachix/install-nix-action/pull/278">cachix/install-nix-action#278</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/cachix/install-nix-action/compare/v31.10.6...v31.10.7">https://github.com/cachix/install-nix-action/compare/v31.10.6...v31.10.7</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="a49548c11d"><code>a49548c</code></a>
Merge pull request <a
href="https://redirect.github.com/cachix/install-nix-action/issues/278">#278</a>
from cachix/create-pull-request/patch</li>
<li><a
href="147e749b5f"><code>147e749</code></a>
nix: 2.34.7 -&gt; 2.34.8</li>
<li><a
href="23cf0fec1d"><code>23cf0fe</code></a>
Merge pull request <a
href="https://redirect.github.com/cachix/install-nix-action/issues/276">#276</a>
from cachix/dependabot/github_actions/actions/checkout-7</li>
<li><a
href="8bdfc70a3e"><code>8bdfc70</code></a>
chore(deps): bump actions/checkout from 6 to 7</li>
<li>See full diff in <a
href="8aa03977d8...a49548c11d">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cachix/install-nix-action&package-manager=github_actions&previous-version=31.10.6&new-version=31.10.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>
2026-07-09 20:04:02 -07:00
Mitchell Hashimoto
6236d3859c Misc runtime safety fixes (#13275)
Runtime safety violating scenarios found by GPT 5.6. Verified each one
manually. See each commit.

I'm going to keep searching so not going to merge this yet.
2026-07-09 20:03:10 -07:00
Mitchell Hashimoto
0aaedf4360 terminal: saturate origin cursor offsets
setCursorPos added origin-mode margins to requested row and column
values before clamping them to the scrolling region. A request near
maxInt(usize) overflowed during that addition and crashed instead of
landing on the region boundary.

Use saturating addition for the origin offsets. The existing clamp then
places oversized requests on the bottom-right margin without changing
normal cursor positioning.
2026-07-09 19:56:06 -07:00
Mitchell Hashimoto
0ff4e41b22 terminal: fix pin wrapping at row boundaries
Pin.leftWrap and rightWrap calculated the destination using the
remainder after consuming the current row. When that remainder was an
exact multiple of the column count, rightWrap subtracted one from zero
and leftWrap produced a column equal to the width. Dereferencing either
pin could panic. A maximum usize offset on a one-column page also
overflowed the row count.

Base the row and column calculations on the remainder minus one. This
maps exact multiples to the final cell of the correct row and keeps the
maximum offset calculation in range so traversal reports overflow
normally.
2026-07-09 19:56:06 -07:00
Mitchell Hashimoto
5bc6588e43 terminal/search: ignore empty search needles
Low-level search state accepted an empty needle even though the search
thread normally filters it out. SlidingWindow treated the empty string
as a zero-length match and underflowed while calculating its inclusive
end offset. Active and viewport overlap calculations could also
underflow while loading adjacent pages.

Treat an empty needle as an inactive search with no matches or history,
and saturate the viewport overlap length.
2026-07-09 19:55:59 -07:00
Mitchell Hashimoto
a23d90c89a terminal/search: reset cached results after resize (#13274)
Screen searches only reset cached dimensions while feeding more history.
Selecting or reloading a result immediately after a resize left
flattened highlights pointing at page nodes freed by reflow. The next
selection operation could dereference those stale pointers and crash.

Centralize dimension invalidation and run it before feed, reload, and
selection paths inspect cached state. Add regression coverage for
selecting a cached active match after a column resize.
2026-07-09 19:46:37 -07:00
Mitchell Hashimoto
7f073c4cf2 terminal: dispatch APC string bytes in bulk slices (#13270)
APC payloads such as Kitty graphics images can be megabytes of base64
data, but every byte was dispatched individually: through the VT state
machine table, an apc_put action, the stream handler, the APC protocol
handler, and finally a per-byte ArrayList append in the Kitty command
parser. Five layers of dispatch per byte made large image transfers far
slower than they needed to be.

Add a bulk fast path alongside the existing CSI fast paths in
consumeUntilGround: scan the longest run of apc_put bytes (stopping at
any byte the parse table doesn't treat as APC payload: CAN, SUB, ESC,
and most C1 bytes exit or abort the string state, and 0xA0-0xFF are
ignored by it) and dispatch the run as a single new apc_put_slice
action. The APC handler identifies the protocol from the first few bytes
as before, then passes the remainder of each slice to the protocol
parser in bulk; the Kitty parser appends payload data with a single
appendSlice. Ignored/unknown APC sequences now drop each slice in O(1)
instead of per-byte dispatch.

The fast path is guarded the same way as the CSI fast paths: handlers
with a vtRaw hook (the inspector) keep receiving per-byte apc_put
actions, and the scalar next() path is unchanged.

Also add benchmark support: a `ghostty-gen +kitty` synthetic generator
emitting well-formed Kitty graphics transmit commands with 4 KiB random
base64 payloads (not valid image data; the corpus exercises the parsing
paths, not image decoding), and a `ghostty-bench +apc-parser` benchmark
that measures the stream -> APC -> Kitty parse path without image
decode/storage.

Benchmarks on a 64 MiB corpus (hyperfine, ReleaseFast, x86_64 Linux,
baseline is identical source with only the fast path disabled):

  apc-parser:               1.061 s -> 43 ms  (~25x)
  terminal-stream (kitty):  1.163 s -> 72 ms  (~16x)
  terminal-stream (ascii):  no change

The ascii case was verified with retired instruction counts (perf stat,
pinned to one core) since wall time on the test machine has 4-7 ms of
noise: 988,030,458 vs 988,045,833 instructions (+0.0016%), a fixed
startup-size delta; the ground-state hot loop never reaches the new
branch.

AI Disclosure: This code was written with the assistance of Fable 5 via
Amp.
2026-07-09 19:38:39 -07:00
Mitchell Hashimoto
6275184473 terminal/search: reset cached results after resize
Screen searches only reset cached dimensions while feeding more
history. Selecting or reloading a result immediately after a resize
left flattened highlights pointing at page nodes freed by reflow. The
next selection operation could dereference those stale pointers and
crash.

Centralize dimension invalidation and run it before feed, reload, and
selection paths inspect cached state. Add regression coverage for
selecting a cached active match after a column resize.
2026-07-09 19:38:29 -07:00
dependabot[bot]
035ae8ddb6 build(deps): bump cachix/install-nix-action from 31.10.6 to 31.10.7
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 31.10.6 to 31.10.7.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Changelog](https://github.com/cachix/install-nix-action/blob/master/RELEASE.md)
- [Commits](8aa03977d8...a49548c11d)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-version: 31.10.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-10 00:12:41 +00:00
Tim Culverhouse
f6f79acce6 terminal: dispatch APC string bytes in bulk slices
APC payloads such as Kitty graphics images can be megabytes of base64
data, but every byte was dispatched individually: through the VT state
machine table, an apc_put action, the stream handler, the APC protocol
handler, and finally a per-byte ArrayList append in the Kitty command
parser. Five layers of dispatch per byte made large image transfers
far slower than they needed to be.

Add a bulk fast path alongside the existing CSI fast paths in
consumeUntilGround: scan the longest run of apc_put bytes (stopping
at any byte the parse table doesn't treat as APC payload: CAN, SUB,
ESC, and most C1 bytes exit or abort the string state, and 0xA0-0xFF
are ignored by it) and dispatch the run as a single new apc_put_slice
action. The APC handler identifies the protocol from the first few
bytes as before, then passes the remainder of each slice to the
protocol parser in bulk; the Kitty parser appends payload data with a
single appendSlice. Ignored/unknown APC sequences now drop each slice
in O(1) instead of per-byte dispatch.

The fast path is guarded the same way as the CSI fast paths: handlers
with a vtRaw hook (the inspector) keep receiving per-byte apc_put
actions, and the scalar next() path is unchanged.

Also add benchmark support: a `ghostty-gen +kitty` synthetic generator
emitting well-formed Kitty graphics transmit commands with 4 KiB
random base64 payloads (not valid image data; the corpus exercises
the parsing paths, not image decoding), and a `ghostty-bench
+apc-parser` benchmark that measures the stream -> APC -> Kitty parse
path without image decode/storage.

Benchmarks on a 64 MiB corpus (hyperfine, ReleaseFast, x86_64 Linux,
baseline is identical source with only the fast path disabled):

  apc-parser:               1.061 s -> 43 ms  (~25x)
  terminal-stream (kitty):  1.163 s -> 72 ms  (~16x)
  terminal-stream (ascii):  no change

The ascii case was verified with retired instruction counts (perf
stat, pinned to one core) since wall time on the test machine has
4-7 ms of noise: 988,030,458 vs 988,045,833 instructions (+0.0016%),
a fixed startup-size delta; the ground-state hot loop never reaches
the new branch.
2026-07-09 17:07:55 -05:00
Mitchell Hashimoto
7e02af8798 terminal: scrollback page compression (70 to 90% memory savings) (#13264)
This adds transparent compression for non-active/non-viewport scrollback
pages, reducing physical memory for compressed pages by anywhere from
70% to 90%. Compression is obviously highly dependent on the shape of
the data, but these are the numbers I got for normal scrollback.

Due to compression being available, I bumped the default scrollback
limit from 10MB to 50MB. On average, a full scrollback still uses less
memory than the prior limit due to the compression ratios.

## Demo

Here is a demo video showing me filling my scrollback and using the
inspector so you can see the live compression/decompression activity and
results:


https://github.com/user-attachments/assets/7b9d0383-42f7-47bf-8b3f-853e3f89549c

## Resident vs. Virtual Memory

This PR works by lowering _resident/physicalmemory, but doesn't touch
_virtual_ memory.

Practically what this means is that users need to make sure they're
looking at resident memory to see the change.

We use OS primitives like `MADV_DONTNEED` on Linux or
`MADV_FREE_REUSABLE` on Darwin to discard our physical memory, but
retain our virtual memory allocations. This is awesome because it means
our decompression is infallible: the OS has already given us the memory,
but it just remaps it at that point.

This is baked into the core implementation, so compression only works on
systems that support an OS primitive to retain virtual mappings while
discarding physical. Today, that is macOS and 64-bit Linux. Other
operating systems have support we just haven't coded it up yet.

## Implementation

A major refactor had to happen to PageList to represent nodes as either
resident or compressed. Pins typically accessed node content directly so
we had to add a bunch of helpers to read metadata without decompression
(but some access requires it).

Compression is relatively slow and its important we don't impact IO
performance. So we support incremental compression passes and they only
run when the terminal is idle (250ms timer on the render thread that
resets on any activity). Benchmarks show zero regression in IO
throughput on this change.

In order to maintain the no-libc invariant for libghostty-vt, we use a
hand-written (an AI assisted optimization) LZ4 compression
implementation. The performance and compression ratio is _okay_. Its a
good first step for this. I think in the future I want to look at other
implementations we can bring in based on build configuration.

## Performance

Measured with a saved 7.3 MB corpus made by repeating `gh --help` output
into a 120x80 terminal with a 50 MB scrollback limit on my machine:

| compression measurement | result |
|-------------------------|--------|
| pages compressed | 121 |
| raw page backing | 49.56 MB |
| encoded storage | 3.03 MB (6.11% of raw) |
| estimated physical memory savings | 46.53 MB (93.89%) |
| full compression | 30.3 ms total, 12.2 ms over the 18.1 ms no-op
baseline (~101 µs/page) |
| incremental drain | 29.7 ms total, 11.6 ms over baseline (~96 µs/page)
|
| compress and restore | 33.5 ms total, 3.2 ms over full compression
(~26 µs/page to restore) |

The workload above is especially repetitive, so its 6.11% encoded ratio
is better than the 10% to 30% expected for text-heavy terminal history
in general. Steady-state throughput is unchanged within noise
(`terminal-stream` benchmarks and manual `cat` timings).

## libghostty-vt

The same caller-driven compression controls are exposed to Zig and C. 

Note that compression _is not automatic_ for libghostty users. Callers
must determine their own definition of "idle" and when to compress and
call our incremental callback APIs to perform the compression.
Decompression is automatic and as-needed (and will trigger
recompression-required flags so callers are aware).

## LLM Notes

This work was done in concert with Codex. I reviewed and
rewrote/reshaped pretty much every change extensively, particularly
PageList/Terminal. This PR message is written by hand, commit messages
are LLM written but reviewed.
2026-07-09 13:19:21 -07:00
Mitchell Hashimoto
11b9a6ef17 renderer: hand off state mutex to avoid starving frames (#13265)
The mutex is unfair. From my understanding (after a brief convo with
@rockorager), it's a race between the parse thread and the renderer
thread.

The parse thread is unlocking it then locking it faster than the
renderer thread can get a lock/unlock. Under sustained pty output the
parse thread never sleeps between batches, so the renderer's frame
snapshot can starve for as long as the output lasts.

The fix here makes the parse thread voluntarily stay off the lock until
the renderer has had one turn by introducing two atomics:

- `demand`: a waiter count. The renderer increments it before locking
and decrements it after acquiring, so demand > 0 means "the renderer is
queued on the lock or about to be."
- `handoff_gen`: a generation counter. The renderer bumps it (with a
futex wake) after unlocking, meaning "a waiter completed one full turn."

At each batch boundary the parse thread checks demand (a single relaxed
load in the common case, so this **should** cost nothing when the
renderer isn't waiting). If a waiter exists, it futex-waits on
handoff_gen until the renderer has taken and released the lock, bounded
by a 1ms timeout, trying to ensure a lost wake can't stall IO.

This is inspired from `parking_lot` form rust land, which has eventual
fairness, but applied only at the one site that misbehaves.

## LLM Note

I used Fable 5 in Amp to write the code, but only after I identified the
problem myself from previous experience with mutex fairness (and the
convo above with rockorager).

The diagnosis — the parse thread barging the unfair mutex and starving
the renderer — came first.Fable then traced the exact loop in
`Exec.zig/generic.zig` and implemented the handoff protocol.

I've reviewed the changes.
2026-07-09 12:47:46 -07:00
Uzair Aftab
4f53b846bc renderer: move State declaration to top of file 2026-07-09 21:41:42 +02:00
Mitchell Hashimoto
9a4bd2120a terminal: optimize LZ4 decoding and add differential tests
The block decoder previously copied literals through variable-length
memcpy calls and expanded every match with word loops that carried an
overcopy fallback in each branch. Real page blocks decode as millions
of tiny sequences, so per-sequence overhead dominated restore time.

Decode short literal runs and in-token matches with blind fixed-width
copies whose margin checks subsume the exact bounds checks they
replace. Expand small repeating periods into pattern-word stores,
copy distant long matches with one exact memcpy, and propagate the
rare non-power-of-two short offsets bytewise. Page corpora restore
13% to 19% faster and text around twice as fast, while compressor
output stays byte-for-byte unchanged.

Replace the fuzz test with a differential property suite which
round-trips generated inputs, validates blocks with an independent
format walker, rejects wrong-size outputs, and decodes corrupted and
truncated blocks. A light version runs as a normal unit test; the
exhaustive version runs when GHOSTTY_LZ4_SLOW is set. An AGENTS.md
records the benchmarking, testing, and verification workflow for
this directory.
2026-07-09 12:38:59 -07:00
Uzair Aftab
d34b54e9b4 renderer: hand off state mutex to avoid starving frames
The renderer state mutex is unfair on all platforms (os_unfair_lock
on macOS, a futex based lock elsewhere). A thread that unlocks and
right away locks again wins over a sleeping waiter, because the
waiter first has to be woken up and scheduled. The termio parse
thread does exactly this under heavy pty output: it relocks the
mutex for every batch and never sleeps in between, so the renderer
can starve in updateFrame for as long as the output lasts.

Fix this by letting the parse thread stay off the lock until the
renderer had its turn. `renderer.State` gets two atomics: a waiter
count (`demand`) and a generation counter (`handoff_gen`). The renderer
takes the mutex through lockDemand/unlockDemand which update these,
and the parse thread calls yieldToDemand between batches. If a
waiter exists it sleeps on a futex until the renderer took and
released the lock, with a 1ms timeout so a lost wake can not stall
IO forever.

All the atomics are monotonic on purpose: they are only a hint for
scheduling, the mutex still protects the terminal state itself.
When the renderer is not waiting the cost for the parse loop is a
single relaxed atomic load per batch.
2026-07-09 20:29:09 +02:00
Mitchell Hashimoto
25e6245691 renderer: avoid starving scrollback compression
Inspector rendering can hold the terminal mutex while waking the
renderer. When the compression scheduler failed to acquire that mutex,
it treated every wake as possible terminal activity and restarted the
idle timer. Frequent inspector frames could therefore postpone
compression indefinitely until another interaction changed the timing.

Keep an existing compression deadline when a wake encounters lock
contention. The timer callback already rechecks both terminal activity
and lock availability, while the first contended wake still arms a
timer when none is active.
2026-07-09 10:47:31 -07:00
Mitchell Hashimoto
172f15da3b terminal: expose compression through libghostty-vt
Scrollback compression scheduling was only available to Zig callers that
used Terminal directly, leaving C embedders unable to drive the same idle
compression policy.

Define ABI-aware mode and result enums on Terminal and export activity
and compression operations through the C API. Keep scheduling
caller-owned, validate C inputs, and document the incremental contract
with a complete example.

Report unsupported reclamation consistently for full passes so callers
can disable compression on targets that cannot retain decommitted
mappings.
2026-07-09 10:11:07 -07:00
Mitchell Hashimoto
6d5dda40db inspector: clarify page compression memory
The PageList overview mixed structural state with a long list of exact
byte counters, making the compression result difficult to interpret.

Keep the overview focused on grid and scrollback structure, and add a
dedicated compression section before scrollbar details. Present page
states, uncompressed size, encoded ratio, resident estimate, and savings
using readable units and scoped help text.
2026-07-09 09:47:13 -07:00
Mitchell Hashimoto
0fb89f4ffe terminal: configure scrollback compression
Idle compression was always enabled on supported renderer-backed
surfaces and the default logical scrollback limit remained sized for
fully resident history.

Add a default-on scrollback-compression option and make renderer
scheduling honor it across config reloads. Existing compressed pages
remain encoded when the option is disabled, while reenabling it starts a
fresh idle pass.

Raise the default logical scrollback limit from 10 MB to 50 MB and
document typical physical-memory savings, content-dependent behavior,
and retained virtual address usage.
2026-07-09 09:47:13 -07:00
Mitchell Hashimoto
95685afd26 terminal: compress offscreen scrollback history
Compression previously stopped whenever the viewport left the active
area, leaving all scrollback resident while a user viewed history.

Traverse complete historical pages through a metadata-only iterator
which skips the contiguous visible range. Restart incremental traversal
after every viewport movement so pages become eligible once they leave
view, while visible pages remain resident for immediate redraw.

Add a PageList-only drain mode for tests and benchmarks, and update
scrollback documentation to describe the offscreen eligibility rule.
2026-07-09 09:47:13 -07:00
Mitchell Hashimoto
181254d36a terminal: debounce compression by page activity
Compression scheduling previously postponed its idle timer after every
renderer wake. The inspector redraw loop wakes the renderer
continuously, so opening it could prevent pending scrollback compression
from ever starting.

Track compression-relevant PageList activity with a wrapping 48-bit
token and restart the timer only when the composite Terminal token
changes. This removes the separate dirty bit while reserving 16 bits for
future Terminal-owned compression triggers.

Expose target availability through the terminal package and leave
renderer compression state undefined on unsupported targets so its timer
is never initialized.
2026-07-09 09:47:13 -07:00
Mitchell Hashimoto
b9bb50c832 terminal: optimize page compression codec
The raw LZ4 codec previously kept one match candidate and extended and
copied matches byte by byte. This limited compression quality and made
page restoration substantially more expensive than the reference
decoder.

Pack two candidates into the existing 16 KiB table, accelerate long
literal searches, and compare matching runs a word at a time. Decode
short literals and common repeated patterns with fixed-width copies
while retaining exact bounds checks for malformed blocks.

This keeps the public API, workspace size, block format, and
allocation-free dependency model unchanged.
2026-07-09 09:47:13 -07:00
Mitchell Hashimoto
461562ca4f terminal: enable idle scrollback compression
Scrollback compression was available only through explicit PageList
calls, so normal renderer-backed surfaces never reclaimed cold history.

Debounce renderer wakeups with a one-shot timer and run bounded
compression steps only when the terminal lock is immediately available.
Timer state is isolated in the renderer and becomes idle once PageList
reports that the pass is complete.

Keep inspector reads representation-preserving by decoding compressed
pages into temporary copies. Update the scrollback configuration and
benchmark documentation to describe the production behavior and its
memory accounting.
2026-07-09 09:47:13 -07:00
Mitchell Hashimoto
f8c217e557 terminal: track pending scrollback compression
PageList previously kept only traversal position, so callers had no
central signal for deciding when an incremental compression pass should
run. Scheduling policy had to infer work from output and UI activity.

Track compression dirtiness alongside the PageList continuation state.
Growth preserves valid progress while marking work pending, and resize
or viewport transitions restart from the oldest page. A no-work
verification pass clears the state.

Expose Terminal helpers which report whether compression is required and
run compression against primary scrollback even while the alternate
screen is active. Unsupported retained-memory targets report no work.
2026-07-09 09:47:12 -07:00
Mitchell Hashimoto
0d015b2fce terminal: preserve compressed pages during search
Search previously used the normal page access boundary while formatting
history and checking soft-wrap boundaries. Inspecting compressed history
therefore restored its retained mapping and undid the memory
reclamation.

Format through preserved page snapshots and copy row counts and wrap
state into the sliding window's owned metadata. Overlap decisions reuse
the same snapshot, so compressed pages are decoded at most once per
append and remain compressed after matching.

Add a cross-page regression which searches compressed history and
verifies both source nodes retain their compressed representation.
2026-07-09 09:47:12 -07:00
Mitchell Hashimoto
e7969ed436 terminal: make PageList compression self-contained
Incremental compression previously exposed its traversal state to
callers, requiring them to coordinate the cursor with PageList lifetime
and topology changes. Read-only consumers also had to restore compressed
nodes to inspect their contents.

Move the continuation state into PageList and expose a single mode-based
compression entry point. Incremental passes restart safely across
mutations and verify a no-work pass before becoming idle, while full
passes leave incremental state fresh.

Add preserved page reads which decode compressed nodes into caller-owned
storage without changing their representation, and migrate the
scrollback compression benchmark to the new API.
2026-07-09 09:47:12 -07:00
Mitchell Hashimoto
70e788f066 terminal: add incremental scrollback compression
Cold-history compression previously required scanning every eligible
page in one call, which makes it unsuitable for an idle-time scheduler.
The inspector also restored compressed pages while traversing collapsed
entries, hiding the representation and undoing reclamation.

Add caller-owned serial state that resumes compression without retaining
node pointers. Each invocation inspects at most eight candidates and
attempts one resident page. List mutations restart safely, while
unsupported or historical viewports stop work. Keep a stateless
whole-history operation for measurement.

Expose metadata-only storage and memory accounting for diagnostics,
update the inspector to restore only expanded pages, and add an
incremental live scrollback benchmark. This remains disconnected from
production scheduling.
2026-07-09 09:47:12 -07:00
Mitchell Hashimoto
9156ada169 terminal: add cold-history compression pass
PageList could compress individual nodes but had no policy-level
operation for selecting pages that are safe to reclaim. The compressed
state therefore remained reachable only from tests.

Add a stateless pass that considers only complete history pages while
the viewport follows the active area. It gates work on supported
retained-mapping reclamation, reports attempts and retained bytes, and
leaves restoration lazy when a resize pulls compressed history back into
the active area.

Add a live scrollback-compression benchmark for measuring complete
PageList compression and restoration against saved VT corpora. The pass
still has no production callers, and ReleaseFast terminal-stream
comparisons remain within the existing throughput guardrail.
2026-07-09 09:47:12 -07:00
Mitchell Hashimoto
421fe8dabe terminal: integrate compressed pages into PageList
PageList nodes previously exposed Page directly, so introducing a
compressed representation would require every consumer to understand its
state and ownership.

Add resident and compressed node states behind a page access boundary.
Content access transparently recommits and restores retained mappings,
while metadata traversal stays compressed and lifecycle paths can
discard encoded contents without decoding. Compression borrows pool
memory for standard scratch and uses temporary aligned storage for
oversized pages.

Migrate terminal, rendering, search, formatting, and C API consumers to
the new boundary. Hot grow, scroll, and print paths reuse resolved
pages, with an explicit resident-only accessor where live cursor
pointers prove that the page cannot be compressed.

The compression entry point remains private with no production callers,
so normal terminal behavior and scrollback accounting are unchanged.
ReleaseFast terminal-stream comparisons across bulk output, scrolling,
redraw, and erase workloads remain within 2% of the parent revision.
2026-07-09 09:47:11 -07:00
Mitchell Hashimoto
f6fd4cb087 terminal: generalize page memory reclamation
PageList's virtual-memory helpers were tied to pool items even though
the underlying decommit and recommit operations also apply to retained
page mappings.

Move the helpers into terminal/mem.zig and express their different
failure contracts as generic modes. Zero mode preserves the existing
pool invariant and fallbacks, while strict mode only succeeds when the
operating system accepts reclamation and avoids touching memory that
will be restored.

PageList now uses the shared zero mode for its page pool. The strict
path is tested in isolation and remains unused, so this does not enable
page compression yet.
2026-07-09 09:47:11 -07:00
Mitchell Hashimoto
ebc3ffd222 terminal: add compressed page representation
The standalone LZ4 codec had no representation for terminal page
ownership or metadata, so PageList integration would otherwise need to
reconstruct every Page field independently.

Add compress.Page, which embeds the complete terminal Page while
retaining its original virtual mapping and owns only an exact-sized
encoded block. Compression is kept only when the encoded state is
strictly smaller, and scratch output is capped at that profitability
boundary so a future PageList caller can borrow a standard pool item.

Extend the page-compression benchmark with a store mode that measures
the encoded copy, allocation, bounded retention, and eviction path.
Nothing uses compression from PageList yet; this remains isolated
groundwork.
2026-07-09 09:47:11 -07:00
Mitchell Hashimoto
c62c159841 terminal: add standalone LZ4 block codec
Scrollback compression needs a codec that can be used from libghostty-vt
without pulling in libc, and we need to measure it before integrating it
with terminal page ownership.

This adds an allocation-free raw LZ4 block codec in scalar Zig.  Callers
provide the input, output, and fixed-size scratch table. The decoder
uses an exact-size output contract so page metadata mismatches fail
cleanly. Compatibility vectors, boundary cases, random round trips, and
fuzz coverage exercise the block format.

Also adds a page-compression benchmark that operates on reusable raw
page corpora. Compression and decompression have separate modes with
setup outside the timed region, plus a ratio report and no-op baseline.
Nothing uses compression in the terminal yet; this is the isolated codec
and measurement groundwork.
2026-07-09 09:47:11 -07:00
Mitchell Hashimoto
7cb44fea33 renderer: avoid allocating when there are no active links (#13258)
Determine if any links are active before building the string and
byte-to-cell map. Those buffers scale with viewport size, and this
function runs during frame updates, so avoid allocating them when the
current mouse/modifier state can't highlight any regex links.

This adds an additional `self.links` iteration, but that list is usually
small, the "active" check is cheap, and it breaks on the first hit.

---

GPT 5.5 spotted this and wrote the test case. I came up with the
tradeoffs myself and wrote the runtime code.
2026-07-09 09:19:51 -07:00
Jon Parise
9c9cf3e821 renderer: avoid allocating when there are no active links
Determine if any links are active before building the string and
byte-to-cell map. Those buffers scale with viewport size, and this
function runs during frame updates, so avoid allocating them when the
current mouse/modifier state can't highlight any regex links.

This adds an additional `self.links` iteration, but that list is usually
small, the "active" check is cheap, and it breaks on the first hit.
2026-07-09 11:17:49 -04:00
Mitchell Hashimoto
b14d923836 Revert "termio: bound POSIX read-ahead on non-Darwin" (#13253)
This reverts commit bed47168ca.

This commit reduced the buffer size and count on non-Darwin. This
introduced a ~20% perf loss on linux for high throughput loads on a
questionable claim. Producers of bytes should not be pacing frames based
on how fast they can write to the terminal.
2026-07-08 13:43:28 -07:00
Tim Culverhouse
60121a0399 Revert "termio: bound POSIX read-ahead on non-Darwin"
This reverts commit bed47168ca.
2026-07-08 15:37:39 -05:00
Mitchell Hashimoto
f815f84594 macos: fix quick terminal restoring stale size after display reconnect (#13250)
"Why is my quick terminal not taking up the entire top of my docked Mac
screen after I reconnect?" Boy howdy are you in the right PR.

It turns out that the quick terminal caches its last-closed window frame
per display so it can restore the user's size when reopened. The cache
entry was considered valid whenever the current screen was the same size
*or larger* than when the frame was saved ("persist when screens grow").
This has led to a pattern that was simply maddening. To wit: that rule
breaks across display changes.

When an external display is disconnected and later reconnected at a
different resolution (common after traveling with a laptop, do not even
get me started on projectors) the same display can come back larger than
when the frame was cached. The stale frame is still treated as valid and
restored, so the quick terminal no longer fills the screen (it appears
at a partial width/height). Because the cache is persisted, restarting
Ghostty does not clear it, and the user is slowly driven mad. Welcome to
madness; we have snacks.

This PR addresses this by treating a cached frame as valid when the
screen geometry matches exactly (both backing scale factor and frame
size). On any mismatch we drop the entry and fall back to the configured
quick-terminal-size. Manual resizes are still remembered across toggles
within a stable display configuration.

Fixes the regression reported in #12348.

AI disclaimer: I used AI for this. Of course I used AI for this, my code
is terrible on a good day. Specifically, Claude Code, as well as a
custom harness that has the curious tendency to write commit messages
containing conspiracy theories about the code because I am history's
greatest monster.

Fight me!
2026-07-08 10:49:39 -07:00
Lukas
91f66da245 macos: route IME preedit commits through key events (#13222) 2026-07-08 07:06:44 +02:00
Mitchell Hashimoto
ad692f1e85 terminal: return free-listed page memory to the OS (#13245)
This uses `madvise` flags on both macOS and Linux to mark our unused
pool items as free memory and not show up in RSS (until they're actually
used). We can do this since we do page-aligned/page-multiple
allocations, heyo!

Background: The PageList page pool never returns memory to the OS:
destroyed pages are zeroed and free-listed until the surface exits. Any
operation that shrinks the page count (clearing scrollback, pruning
churn, resets, reflow) therefore retains its high-water RSS forever.
Clearing a full scrollback keeps all of it resident, which at the
default 10MB scrollback-limit is 10MB per terminal of memory that can
never be used again for anything else.

Lots of memes on the internet about this, and it turns out operating
systems give us an answer for this (both Linux and macOS at least), so
let's do it kids.

On Linux, madvise(MADV_DONTNEED) reclaims the pages immediately and
guarantees zero-fill on the next touch, which also lets us skip the
zeroing memset entirely, making destroy cheaper.

On macOS, we zero in place and mark the item MADV_FREE_REUSABLE, which
removes it from the process footprint immediately. Reuse is paired with
MADV_FREE_REUSE when the pool hands a buffer back out so that footprint
accounting stays correct. The zero invariant required by page reuse
holds either way: reusable page contents are either preserved (our
zeroes) or reclaimed and zero-filled by the kernel.

Other platforms and test builds keep the existing memset behavior.

## LLM Notes

Fable 5 found the retention behavior while re-analyzing scrollback
memory, wrote the change and tests, and verified the madvise semantics
empirically with memory probes on macOS and a real Linux kernel, plus
before/after throughput benchmarks on both. I reviewed the analysis, the
diff, rewrote the code to be more idiomatic Zig, and wrote this commit
message you're reading.
2026-07-07 20:59:56 -07:00