Environment variables from the "real" environment leaked into the test
after the Zig 0.16 update which would cause them to fail if you ran them
on a Gnome system.
Environment variables from the "real" environment leaked into the test
after the Zig 0.16 update which would cause them to fail if you ran them
on a Gnome system.
Replaces: https://github.com/ghostty-org/ghostty/pull/13427
Zig's Mach-O linker does not emit LC_ENCRYPTION_INFO_64 for physical iOS
dylibs. This allows libghostty-vt to build successfully but causes
frameworks containing it to fail App Store validation.
I think it'd be cleaner to always just build Apple targets on Apple
hosts with the native linker. We don't need to rely on Zig being correct
and this helps ensure compatibility for details like this.
Replaces: https://github.com/ghostty-org/ghostty/pull/13427
Zig's Mach-O linker does not emit LC_ENCRYPTION_INFO_64 for physical
iOS dylibs. This allows libghostty-vt to build successfully but causes
frameworks containing it to fail App Store validation.
I think it'd be cleaner to always just build Apple targets on Apple hosts
with the native linker. We don't need to rely on Zig being correct and
this helps ensure compatibility for details like this.
Xcode 27's math.h uses the __need_infinity_nan protocol provided by
matching Clang resource headers. Zig 0.16's bundled float.h predates
that protocol, causing the bundled libc++ compilation to fail.
Overlay the SDK math.h through the Apple SDK libc include path and
provide the missing infinity and NaN definitions. The compatibility
header can be removed once Zig's bundled Clang headers support the
protocol.
Xcode 27's math.h uses the __need_infinity_nan protocol provided by
matching Clang resource headers. Zig 0.16's bundled float.h predates
that protocol, causing the bundled libc++ compilation to fail.
Overlay the SDK math.h through the Apple SDK libc include path and
provide the missing infinity and NaN definitions. The compatibility
header can be removed once Zig's bundled Clang headers support the
protocol.
#13417
The bundled upstream libc++ headers in Zig 0.16 skip the
Apple-configured availability setting. This causes the headers to assume
every LLVM 21 ABI symbol is present in the target system libc++,
producing binaries that fail at launch on macOS versions without
`std::__hash_memory`.
Enable the Apple vendor availability table for compile steps configured
by the Apple SDK helper. libc++ now selects its inline compatibility
implementation when the target system dylib does not provide the symbol.
References in the mega comment
#13417
The bundled upstream libc++ headers in Zig 0.16 skip the Apple-configured
availability setting. This causes the headers to assume every LLVM 21
ABI symbol is present in the target system libc++, producing binaries
that fail at launch on macOS versions without `std::__hash_memory`.
Enable the Apple vendor availability table for compile steps configured
by the Apple SDK helper. libc++ now selects its inline compatibility
implementation when the target system dylib does not provide the symbol.
References in the mega comment
Closes#12228
Supersedes #12388
**UPDATED** - Also check comments for additional details!
This commit represents the majority of the work necessary to upgrade
Ghostty to use Zig 0.16.0.
At this point, all tests pass under Linux, but more work may be
necessary to get them to build and function on other platforms.
There are some parts of this update that deserve commentary, so that
follows below:
## Expanded use of global state (IO/environment related)
Global state, once generally only used by the C library, has now been
expanded to be used across the project at large. The static local
variable that holds the state has been moved private in its source
container with all attributes that need to be accessed globally gated
behind accessors, most of which guard on testing and send test copies
instead. Use of the global state in non-testing scenarios asserts that
the state has been initialized through `init` naturally through the
optional assertion process.
The rationale for this change is to have a location to store a
general-purpose I/O implementation and environment variables, both of
which are now provided through [Juicy
Main](https://ziglang.org/download/0.16.0/release-notes.html#Juicy-Main)
and hence can no longer be accessed or mutated through stdlib without
use of lower-level system calls and hacks (some of which are employed,
but sparingly).
As the code matures, dependence on global state should naturally slim
down.
We do not allow global state to be used in libghostty-vt. There are
comptime guards that prevent this should compilation of libghostty-vt
end up pulling `global.zig`. This means that as per the last paragraph,
work has already begun to de-couple the codebase from global state where
necessary. Additionally, in some places where environment needs to be
updated and where it can be done in an isolated fashion, environment
maps are used - system-level injection of environment through the use of
`setenv` or `unsetenv` now only happens during early initialization (and
hopefully we can remove these in the future too, especially since they
require re-synchronization of the higher-level environment primitives
after this is done).
## The `lib/compat` Tree
Some stdlib features that have been removed but still either seem they
would be valuable to us or outright complex to move away from
(particularly `SegmentedList`) have been extracted from 0.15.2, updated
as needed, and placed in `src/lib/compat`. The intention again is to
allow for piecemeal migration to more modern implementations or possibly
straight local versions.
This paradigm has also allowed us to add `std.Io.Condition.waitTimeout`,
which incidentally was missed in the 0.16.0 shuffle and has been
re-added for 0.17.0. We can remove this in favor of the upstream when we
eventually migrate to that, obviously.
Note that there was a lot more of this extracted code when this work was
started, but a lot of said code has been removed (namely environment or
process/fd-related functionality).
## translate-c Issues (functional on Linux, Darwin WIP)
There have been a number of C translation issues that we have been
working through through submitted patches and the great help from folks
on the Arocc and Zig side. This is ongoing, with the remaining work to
getting things fixed mainly focused on the MacOS side. Stay tuned for
further developments.
As mentioned at the top, follow comments for more details!
Restore the error handling that the removed std.posix fork and waitpid
wrappers previously provided. Raw fork failures now propagate, waitpid
retries interruptions before reading status, and edit-config constructs the
sentinel-terminated argv required by execve.
Let global initialization own cleanup through its existing errdefer so
temporary paths are freed once. Report initialization failures with the
static synchronous I/O provider because global I/O has already been torn
down by that point.
Zig 0.16.0 made the criteria for reporting "failed command" stricter (or
looser, depending on your perspective I guess...) - now, tests that
print anything to stderr cause the message to appear.
Note that in this instance tests still pass and you get a return code of
0, but nonetheless, it can be confusing.
Additionally, having spammy passing tests in general is not necessarily
a great experience, so this should help with that.
Note that this change was already done to the main tests. We can add a
build argument to control this if need be.
The early-stage main Zig wrapper recognizes if main only needs the
minimal state (args and lower-level environment) and skips a bunch of
unneeded initialization (allocator, arena, threaded I/O, and the
higher-level environment map). Particularly, the fact that it does not
set up an I/O instance means that we won't have any unneeded signal
handlers set up for the unused threaded I/O implementation, which is
similar in spirit to the fixes we applied for the C VT implementation,
with the notable difference that we do actually set a threaded I/O up in
global state - hence, again, we don't want the duplicate unused one.
The C API is assumed to be single-threaded per VT instance.
Additionally, using fully-threaded I/O instances registers signal
handlers, and would do a pair of registrations once per instance, which
could easily get out of hand (and is not really what we intend anyway).
init_single_threaded does not register signal handlers, so it does not
have this issue, and matches the execution model of the C VT API
(single-threaded/not thread-safe within a single VT instance).
This also fixes an initialization issue with the threaded I/O instance
in general (needs allocation as the memory location would have gone out
of scope before).
- benchmark: avoid buffers to avoid a memcpy
- build: keep frame pointers on macOS. There was some debug changes from
Zig 0.15 and this helps. Also, Apple actually requires/expects x29 to
always be a frame pointer.
- build/macos: force libSystem symbols instead of compiler-rt
- global: add InitOpts.tool so that ghostty-gen/bench can parse their
own actions in `+action`
- quirks: provide our own vectorized memset. see the comment for more
details why.
- synthetic: fix UB by accessing global.io before it was initialized
- terminal/hash_map: force inline for unique repr types. Zig 0.15
inlined and 0.16 doesn't, measured a huge slowdown in hyperlink
benchmarks.
- terminal: add explicit `@Vector` usage for storing a run of identical cells
as well as for scanning printable cells. This auto-vectorized in Zig
0.15 but not in Zig 0.16. This produces the same assembly.
- unicode: properties and LUT need power-of-two backing integer to avoid
bad LLVM codegen
This commit represents the majority of the work necessary to upgrade
Ghostty to use Zig 0.16.0.
Key parts:
* In addition to its previous responsibilities, the global state now
houses state for global I/O implementations and the process
environment. It is now also utilized in the main application along
with the C library. Where necessary, global state is isolated from key
parts of the implementation (e.g., in libghostty subsystems), and it's
expected that this list will grow.
* We currently manage our own C translation layer where necessary. In
these cases, cImport has been removed in favor of the new external
translate-c package. Due to fixes that have needed be made to properly
translate the dependencies that were swapped out, as mentioned, we
have had to backport fixes from the current translate-c package (and
the upstream Arocc dependency). We will host this ourselves until Zig
0.17.0 is released with these fixes.
* Where necessary (only a small number of cases), some stdlib code from
0.15.2 (and even from 0.17.0) has been taken, adopted, and vendored in
lib/compat.
Co-authored-by: Leah Amelia Chen <hi@pluie.me>
Re: #13160 (related but not that issue)
PageList eraseRow and eraseRowBounded have the same issue previously
fixed for cursorScrollAbove: when shifting rows up across a page
boundary, the top row of the next page is cloned into the last row of
the previous page, and that clone can fail if the destination page lacks
capacity for the row's managed memory.
Handle the errors the same way the other cross-page copies do: increase
the destination page capacity for the dimension that ran out and retry
the row copy.
This type of logic was repeated EVERYWHERE so I extracted this into a
helper in PageList and Screen. They're slightly different due to the
extra accounting that Screen has to do for the cursor.
Don't know of any scenario this actually happened in the real world but
it was trivially reproducible with tests.
Re: #13160 (related but not that issue)
PageList eraseRow and eraseRowBounded have the same issue previously
fixed for cursorScrollAbove: when shifting rows up across a page boundary,
the top row of the next page is cloned into the last row of the
previous page, and that clone can fail if the destination page lacks
capacity for the row's managed memory.
Handle the errors the same way the other cross-page copies do:
increase the destination page capacity for the dimension that ran
out and retry the row copy.
This type of logic was repeated EVERYWHERE so I extracted this into a
helper in PageList and Screen. They're slightly different due to the extra
accounting that Screen has to do for the cursor.
Don't know of any scenario this actually happened in the real world but
it was trivially reproducible with tests.
Fixes https://github.com/ghostty-org/ghostty/issues/13390
Technically it would be safe to remove `#available(macOS 27, *)` check,
but I haven't tested all the os versions, so I kept it there.
### AI Disclosure
No AI is used for this one.
Kitty 0.48 added support for usage hints in the image protocol,
specifically for marking images as "transient", meaning that they should
be prioritized for eviction if there is memory pressure.
https://sw.kovidgoyal.net/kitty/graphics-protocol/#image-usage-hints
Also changed the eviction algorithm to use an allocated array for
organizing the images to be evicted rather than using an ArrayList to
minimize the number of allocations made (no real memory savings though).
Re: #13160
When cursorScrollAbove rotates rows across a page boundary, the last row
of the previous page is cloned into the destination page. That can cause
capacity failures we didn't previously handle.
The error propagated out of the operation after rows had already been
rotated, leaving the page list half-mutated. Subsequent operations on
the corrupted state can cause crashes since the state was incoherent.
Handle these errors the same way insertLines and deleteLines already do
for their cross-page copies: increase the destination page capacity for
the dimension that ran out and retry the row copy.
Re: #13160
When cursorScrollAbove rotates rows across a page boundary, the last
row of the previous page is cloned into the destination page. That can
cause capacity failures we didn't previously handle.
The error propagated out of the operation after rows had already been rotated,
leaving the page list half-mutated. Subsequent operations on the corrupted state
can cause crashes since the state was incoherent.
Handle these errors the same way insertLines and deleteLines already
do for their cross-page copies: increase the destination page capacity
for the dimension that ran out and retry the row copy.
Kitty 0.48 added support for usage hints in the image protocol,
specifically for marking images as "transient", meaning that they
should be prioritized for eviction if there is memory pressure.
https://sw.kovidgoyal.net/kitty/graphics-protocol/#image-usage-hints
Also changed the eviction algorithm to use an allocated array for
organizing the images to be evicted rather than using an ArrayList to
minimize the number of allocations made (no real memory savings though).
`ext-background-effect-v1` is finally seeing major adoption throughout
KWin, Mutter, cosmic-comp, Niri, etc. and we should update our comments
on that.
`ext-background-effect-v1` is finally seeing major adoption throughout
KWin, Mutter, cosmic-comp, Niri, etc. and we should update our comments
on that.
Cursor defaults were duplicated across stream handlers and it was a
pretty significant amount of simple and yet non-trivial logic to
understand.
Store these on Terminal itself and have methods to route things like
DECSCUSR through for consistent behaviors.
No AI usage here.
Cursor defaults were duplicated across stream handlers and it was a
pretty significant amount of simple and yet non-trivial logic to
understand.
Store these on Terminal itself and have methods to route things like
DECSCUSR through for consistent behaviors.
The libghostty-vt stream is made to be infallible: in the case of any
error it just logs and moves on. That's because a terminal can't
really... stop, under normal operations. But, under special operations
(fuzzing, replays, etc.) it can and should stop!
Rather than make the operation fallible, its simply enough for me at
least to know that something went wrong. This is a simple change that
adds a simple flag that is flagged to true when such a scenario happens.
For normal Ghostty GUI operations, this isn't used at all. For
libghostty consumers they can choose to read it if they want, but don't
have to.
This also adds a C API to read it.
The libghostty-vt stream is made to be infallible: in the case of any error
it just logs and moves on. That's because a terminal can't really... stop,
under normal operations. But, under special operations (fuzzing, replays,
etc.) it can and should stop!
Rather than make the operation fallible, its simply enough for me at least
to know that something went wrong. This is a simple change that adds a simple
flag that is flagged to true when such a scenario happens.
For normal Ghostty GUI operations, this isn't used at all. For libghostty
consumers they can choose to read it if they want, but don't have to.
This also adds a C API to read it.
This makes error returns in `Screen.resize` and `Terminal.resize` safe:
they leave the terminal/screen in its prior state (with one exception,
noted below). Previously, both were documented as leaving the terminal
in a garbage stage on error.
Unit tests with tripwire added to verify errdefer behaviors.
This also includes a refactor: the common behaviors that resize needs
such as updating cell size in pixels, disabling synchronized output,
handling mode 2048, are now pulled into libghostty-vt. This will make it
easier for downstream libghostty users to make proper terminals.
**The one perfect undo exception:** If the primary screen can resize but
the alt screen cannot, then we try our best to do something reasonable,
in order:
1. If we're on the primary screen, we just deallocate the alt screen.
It'll be reallocated lazily (and may fail) in the future. Worst case
here is we lose screen data if the future TUI doesn't expect a clear on
enter/exit.
2. If we're on the alt screen, we deallocate to try recover memory, then
reinitialize eagerly at the new size hoping to at least have a blank alt
screen. Similar to 1, we lose screen data here, but its likely screen
data that mattered since we were actively on the alt screen.
3. If the eager reinit fails, we switch to the primary screen. This is
the biggest issue, cause the TUI program won't know this happened and
probably do some crazy stuff on the primary screen. But, its also a
super exceptional situation.
In every case though, the terminal is consistent and safe to use.
**LLM notes:** Only used as a judge and to assist with test writing, not
used at all for commit messages, PR, or non-test logic.
Terminal resize could leave tab stops, pixel geometry, synchronized output,
or screen dimensions partially updated when a later allocation failed. This
is now safe. There is one exception, see the code comments.
