Pin.leftWrap and rightWrap calculated the destination using the
remainder after consuming the current row. When that remainder was an
exact multiple of the column count, rightWrap subtracted one from zero
and leftWrap produced a column equal to the width. Dereferencing either
pin could panic. A maximum usize offset on a one-column page also
overflowed the row count.
Base the row and column calculations on the remainder minus one. This
maps exact multiples to the final cell of the correct row and keeps the
maximum offset calculation in range so traversal reports overflow
normally.
Scrollback compression scheduling was only available to Zig callers that
used Terminal directly, leaving C embedders unable to drive the same idle
compression policy.
Define ABI-aware mode and result enums on Terminal and export activity
and compression operations through the C API. Keep scheduling
caller-owned, validate C inputs, and document the incremental contract
with a complete example.
Report unsupported reclamation consistently for full passes so callers
can disable compression on targets that cannot retain decommitted
mappings.
Compression previously stopped whenever the viewport left the active
area, leaving all scrollback resident while a user viewed history.
Traverse complete historical pages through a metadata-only iterator
which skips the contiguous visible range. Restart incremental traversal
after every viewport movement so pages become eligible once they leave
view, while visible pages remain resident for immediate redraw.
Add a PageList-only drain mode for tests and benchmarks, and update
scrollback documentation to describe the offscreen eligibility rule.
Compression scheduling previously postponed its idle timer after every
renderer wake. The inspector redraw loop wakes the renderer
continuously, so opening it could prevent pending scrollback compression
from ever starting.
Track compression-relevant PageList activity with a wrapping 48-bit
token and restart the timer only when the composite Terminal token
changes. This removes the separate dirty bit while reserving 16 bits for
future Terminal-owned compression triggers.
Expose target availability through the terminal package and leave
renderer compression state undefined on unsupported targets so its timer
is never initialized.
PageList previously kept only traversal position, so callers had no
central signal for deciding when an incremental compression pass should
run. Scheduling policy had to infer work from output and UI activity.
Track compression dirtiness alongside the PageList continuation state.
Growth preserves valid progress while marking work pending, and resize
or viewport transitions restart from the oldest page. A no-work
verification pass clears the state.
Expose Terminal helpers which report whether compression is required and
run compression against primary scrollback even while the alternate
screen is active. Unsupported retained-memory targets report no work.
Incremental compression previously exposed its traversal state to
callers, requiring them to coordinate the cursor with PageList lifetime
and topology changes. Read-only consumers also had to restore compressed
nodes to inspect their contents.
Move the continuation state into PageList and expose a single mode-based
compression entry point. Incremental passes restart safely across
mutations and verify a no-work pass before becoming idle, while full
passes leave incremental state fresh.
Add preserved page reads which decode compressed nodes into caller-owned
storage without changing their representation, and migrate the
scrollback compression benchmark to the new API.
Cold-history compression previously required scanning every eligible
page in one call, which makes it unsuitable for an idle-time scheduler.
The inspector also restored compressed pages while traversing collapsed
entries, hiding the representation and undoing reclamation.
Add caller-owned serial state that resumes compression without retaining
node pointers. Each invocation inspects at most eight candidates and
attempts one resident page. List mutations restart safely, while
unsupported or historical viewports stop work. Keep a stateless
whole-history operation for measurement.
Expose metadata-only storage and memory accounting for diagnostics,
update the inspector to restore only expanded pages, and add an
incremental live scrollback benchmark. This remains disconnected from
production scheduling.
PageList could compress individual nodes but had no policy-level
operation for selecting pages that are safe to reclaim. The compressed
state therefore remained reachable only from tests.
Add a stateless pass that considers only complete history pages while
the viewport follows the active area. It gates work on supported
retained-mapping reclamation, reports attempts and retained bytes, and
leaves restoration lazy when a resize pulls compressed history back into
the active area.
Add a live scrollback-compression benchmark for measuring complete
PageList compression and restoration against saved VT corpora. The pass
still has no production callers, and ReleaseFast terminal-stream
comparisons remain within the existing throughput guardrail.
PageList nodes previously exposed Page directly, so introducing a
compressed representation would require every consumer to understand its
state and ownership.
Add resident and compressed node states behind a page access boundary.
Content access transparently recommits and restores retained mappings,
while metadata traversal stays compressed and lifecycle paths can
discard encoded contents without decoding. Compression borrows pool
memory for standard scratch and uses temporary aligned storage for
oversized pages.
Migrate terminal, rendering, search, formatting, and C API consumers to
the new boundary. Hot grow, scroll, and print paths reuse resolved
pages, with an explicit resident-only accessor where live cursor
pointers prove that the page cannot be compressed.
The compression entry point remains private with no production callers,
so normal terminal behavior and scrollback accounting are unchanged.
ReleaseFast terminal-stream comparisons across bulk output, scrolling,
redraw, and erase workloads remain within 2% of the parent revision.
PageList's virtual-memory helpers were tied to pool items even though
the underlying decommit and recommit operations also apply to retained
page mappings.
Move the helpers into terminal/mem.zig and express their different
failure contracts as generic modes. Zero mode preserves the existing
pool invariant and fallbacks, while strict mode only succeeds when the
operating system accepts reclamation and avoids touching memory that
will be restored.
PageList now uses the shared zero mode for its page pool. The strict
path is tested in isolation and remains unused, so this does not enable
page compression yet.
The PageList page pool never returns memory to the OS: destroyed pages
are zeroed and free-listed until the surface exits. Any operation that
shrinks the page count (clearing scrollback, pruning churn, resets,
reflow) therefore retains its high-water RSS forever.
Clearing a full scrollback keeps all of it resident, which at the default 10MB
scrollback-limit is 10MB per terminal of memory that can never be
used again for anything else.
Lots of memes on the internet about this, and it turns out operating
systems give us an answer for this (both Linux and macOS at least),
so let's do it kids.
Pool items can't be individually freed since they live inside arena
chunks, but they are page-aligned and page-multiple sized, so we can
decommit them while they sit in the free list. Our page-aligned
allocation pays off, again!
On Linux, madvise(MADV_DONTNEED) reclaims the pages immediately and
guarantees zero-fill on the next touch, which also lets us skip the
zeroing memset entirely, making destroy cheaper.
On macOS, we zero in place and mark the item MADV_FREE_REUSABLE, which
removes it from the process footprint immediately. Reuse is paired
with MADV_FREE_REUSE when the pool hands a buffer back out so that
footprint accounting stays correct. The zero invariant required by
page reuse holds either way: reusable page contents are either
preserved (our zeroes) or reclaimed and zero-filled by the kernel.
Other platforms and test builds keep the existing memset behavior.
## LLM Notes
Fable 5 found the retention behavior while re-analyzing scrollback
memory, wrote the change and tests, and verified the madvise semantics
empirically with memory probes on macOS and a real Linux kernel, plus
before/after throughput benchmarks on both. I reviewed the analysis,
the diff, rewrote the code to be more idiomatic Zig, and wrote this
commit message you're reading.
PageList skips zeroing pooled page buffers in release builds, relying
on the OS page allocator handing out zeroed pages and destroyNodeExt
zeroing buffers before returning them to the pool. There is a hidden
exception: std.heap.MemoryPool writes its free list node into the
first pointer-size bytes of a free-listed buffer, so a reused buffer
is not fully zero. This is only safe because the page rows array is
laid out at offset 0, a page always has at least one row, and initBuf
fully rewrites every row, overwriting the stale free list pointer.
None of that was written down or checked anywhere, so a future layout
reorder (or a zero-row page) would corrupt pages in release builds
only, in a way that depends on pool reuse patterns. This adds a
comptime assert that a Row covers at least a pointer, a runtime assert
that pages always have at least one row, and comments tying the
invariant together at the layout, initBuf, and pool reuse sites.
Also fixes stale doc comments: deinit referenced a clonePool function
that no longer exists, and Screen tests referenced increaseCapacity by
its old adjustCapacity name.
PageList.increaseCapacity grows a capacity dimension by doubling it.
If the dimension is zero, doubling "succeeds" without growing: the
page is reallocated and recloned with an identical capacity, violating
the documented guarantee that we always increase by at least one unit.
Every unbounded retry site (startHyperlink, cursorSetHyperlink, the
reflow probes, insertLines/deleteLines) then loops forever reallocating
a page per iteration, and the single-retry sites (styles, graphemes)
fail their retry and silently drop data.
No production page has a zero dimension today, which is why this has
never fired: standard capacities are nonzero and doubling keeps them
nonzero. But exactRowCapacity legitimately returns zero for dimensions
with no content (a compacted plain text page has zero styles, grapheme,
string, and hyperlink capacity), so any compaction work makes this
reachable.
Growth from zero now jumps straight to the standard default for the
dimension rather than doubling. The default is what every standard
page starts with, so single-retry callers are guaranteed enough room
for their pending allocation, whereas doubling from a minimum unit
could still come up short (a single grapheme can need multiple chunks,
and a style set below capacity 3 cannot store anything).
PageList decided whether a page's backing memory belongs to the memory
pool or the heap by comparing its length against std_size. This was super
error-prone and the source of many bugs historically. We locked it down
but its bothered me and has gotten in the way of another feature I've
wanted to do: memory compaction.
First, this commit records the ownership explicitly on each node and uses it
everywhere ownership was previously inferred from size.
Second, createPage gains an exact_size option that forces an exact-size
heap allocation even when the layout would fit a pool item.
Third, compact() now uses it to shrink any page to its minimum size,
including standard pages. Nothing calls compact [YET!] but this is going
to be the key to compressing scrollback history.
This is groundwork for a ton of memory savings. Coming soon.
This adds a GHOSTTY_SCROLL_VIEWPORT_ROW tag with a `size_t row` member
in the value union. The row is an absolute offset from the top of the
scrollable area, clamped to the active area, in the same row space as
the scrollbar offset so thumb positions round-trip cleanly:
ghostty_terminal_scroll_viewport(term,
(GhosttyTerminalScrollViewport){
.tag = GHOSTTY_SCROLL_VIEWPORT_ROW,
.value = {.row = 42},
});
The tag is appended to the existing enum and the union fits within the
reserved padding, so this is ABI compatible.
This also corrects the docs on GHOSTTY_TERMINAL_DATA_SCROLLBAR: the
getter is amortized O(1) (total is maintained incrementally, the offset
is cached), not "expensive". Since there is intentionally no change
callback, the docs now bless polling per frame or per write batch and
diffing, which is what Ghostty's own renderer does.
Motivation: Embedders building native scrollbars can already read scroll state via
GHOSTTY_TERMINAL_DATA_SCROLLBAR, but the write side only exposed
top/bottom/delta scrolling. Mapping a scrollbar thumb drag to an
absolute position required reading the current offset and computing a
delta, which is two calls that must be sequenced atomically by the
caller.
The core already supports absolute positioning and the macOS
app uses it for scroller drags via the scroll_to_row keybinding; this
exposes the same operation through the libghostty C API.
In the column-shrink (.lt) branch of PageList.resize, resizeWithoutReflow
lowers self.rows before resizeCols runs. Because the active area is anchored
to the bottom, shrinking rows moves the active-area top down; a cursor near
the top of the old active area then ends up above the new active area (in
scrollback).
resizeCols counts wrap continuations from the cursor pin up to the active-area
top via a .left_up rowIterator. When the cursor pin is above the limit, the
range is reversed and the iterator's order assertion fires (SIGABRT in debug;
silently iterates empty in release).
Count zero wraps when the cursor pin is above the active area, mirroring the
post-reflow preserved-cursor block which already no-ops for a non-active
cursor. Add a regression test.
Adds a PageList regression test exercising the underflow path fixed in
7fa6fffbc, and a libghostty-vt C API test mirroring the original repro
through ghostty_terminal_resize.
PageList.resize takes the .lt branch when columns shrink, which calls
resizeWithoutReflow (mutating self.rows to the new smaller value) and
then resizeCols with the original opts.cursor.y. When both axes shrink
in one call and the cursor sits at or past the new bottom row, the
expression `self.rows - c.y - 1` underflows and panics in safety builds.
Use saturating subtraction; "remaining rows below cursor" is 0 once the
cursor sits at or past the new bottom.
This PR fixes an issue where reflowing could leave the active cursor
attached to a clipped trailing blank cell instead of following the
current write position.
Maybe related to #12298?
When Screen resize forwards the active cursor into PageList reflow, a
history-pinned viewport can be remapped into the active area before the
preserved-cursor grow step finishes. The old code kept treating that
viewport as a history pin during the intermediate grow calls, which left
too few rows beneath the pin and tripped the viewport integrity checks.
Fix this by normalizing the viewport back to active as soon as reflow
moves the pinned row into the active area. Add a Screen-level regression
test that exercises the full resize path with bounded scrollback and
wrapped rows, and document the setup so the unwrap and viewport
transition are clear.
Fixes#11957
erasePage now updates page_serial_min when the first page is erased,
and asserts that only front or back pages are erased since
page_serial_min cannot represent serial gaps from middle erasure.
To enforce this invariant at the API level, PageList.eraseRows is
now private. Two public wrappers replace it: eraseHistory always
starts from the beginning of history, and eraseActive takes a y
coordinate (with bounds assertion) and always starts from the top
of the active area. This makes middle-page erasure impossible by
construction.
resizeWithoutReflowGrowCols has a fast path that reuses existing page
capacity when growing columns: it simply bumps page.size.cols without
touching cell data. If any row has a spacer_head at the old last column
(from a wide char that did not fit), that cell is no longer at the end
of the now-wider row, causing a page integrity violation.
Fix by checking for spacer_head cells at the old last column before
taking the fast path. If any are found, fall through to the slow path
which handles spacer heads correctly via cloneRowFrom.
Found by AFL++ stream fuzzer. #11109
Fixes#10369
When `resizeWithoutReflowGrowCols` copies rows to a previous page with
spare capacity, tracked pins pointing to those rows were not being remapped.
This left pins pointing to the original page which was subsequently destroyed.
The fix adds pin remapping for rows copied to the previous page,
matching the existing remapping logic for rows copied to new pages.
I also added new integrity checks to verify that our tracked pins are
always valid at points where internal operations complete.
Thanks to @grishy for finding this!
This never caused any known issues, but it's a bug! `increaseCapacity`
should produce a node with identical contents, just more capacity. We
were forgetting to copy over the dirty flag.
I looked back at `adjustCapacity` and it also didn't preserve the dirty
flag so presumably downstream consumers have been handling this case
manually. But, I think semantically it makes sense for
`increaseCapacity` to preserve the dirty flag.
This bug was found by AI (while I was doing another task). I fixed it
and wrote the test by hand though.
Fixes#10352
The bug was that non-standard pages would mix the old
`growRequiredForActive` check and make our active area insufficient in
the PageList.
But, since scrollbars now require we have a cached `total_rows` that our
safety checks always verify, we can remove the old linked list traversal
and switch to some simple math in general across all page sizes.
