Fixes a frame time regression reported with fortio's `fps -fire`
benchmark (fortio.org/terminal/fps): frame times nearly doubled at
typical grid sizes after #13209 (the pipelined pty reader), and were
more than 5x worse at small grids.
## The problem
The `fps -fire` program follows a request/response pattern: write a
frame, end it with a cursor position query (CSI 6n), and block until
the reply arrives before starting the next frame.
The gather stage treats any burst of 1 KiB or more as a saturated
stream. When its spin retries come up dry, it sleeps in a 1ms poll
expecting the writer's next refill so it can publish fewer, larger
batches. But a frame-synced writer will never refill here: it is
blocked waiting for a reply to a query that is sitting inside the
very batch the gather stage is holding back. The poll always sleeps
its full timeout, adding ~1.2ms to every round trip. Ouch!
## The fix
Sleeping for a refill gap is only free while the parse stage is busy,
since the wait hides behind parse time. Once the parser is idle,
every additional microsecond spent bridging is added straight to
output latency. So:
1. When the spin retries exhaust and the parse stage is idle,
deliver the batch immediately instead of polling.
2. When the parse stage is busy, use a `pipe2` pipe to allow the parser
to notify the gather thread it is idle. In the middle of the `poll`
loop and sleep, we can get interrupted immediately and deliver
the batch.
The pipe is only written while the gather stage is actively polling, so an
interactive terminal never pays the syscalls, and a saturated stream
never idles the parser, preserving full batching and throughput for
bulk output. Win, win, win!
## Benchmarks
| workload | pre-#13209 | before | after |
|---|---|---|---|
| fps -fire 80x24 | 0.262 ms / 3674 fps | 1.435 ms / 694 fps | 0.234 ms / 4106 fps |
| fps -fire 160x45 | 1.012 ms / 975 fps | 1.823 ms / 545 fps | 0.701 ms / 1405 fps |
| fps -fire 284x68 | 2.443 ms / 407 fps | 2.391 ms / 414 fps | 1.351 ms / 732 fps |
| cat 19.3 MB | 0.204 s (95 MB/s) | 0.086 s (224 MB/s) | 0.082 s (236 MB/s) |
## LLM Notes
Bisect script written by hand and found the offending commit. Fable 5
found the likely cause. I manually came up with the proposed solution and
wrote it out. Fable helped benchmark for me to verify my assumptions
conceptually and in the real world.
The pipelined POSIX pty reader uses multiple large gather buffers to avoid
stalling on Darwin, where pty reads are capped around 1KiB. On Linux this can
let bulk terminal producers run too far ahead of terminal parsing/rendering.
Frame-style terminal apps such as DOOM-fire can then report very high producer
FPS while Ghostty displays stale frames from the buffered stream.
Keep the Darwin-tuned 4 x 64KiB pipeline, but reduce non-Darwin read-ahead to
2 x 8KiB so the pty can apply backpressure before multiple frames are queued.
This replaces the single-threaded pty read loop on posix systems with
a two-stage pipeline: a new `io-gather` thread drains the pty into a
small ring of large buffers while the existing `io-reader` thread
parses the previous batch concurrently.
The motivating discovery was actually found by Fable, but the
resulting code was hand-written and hand-verified (in addition to
model-verified as an extra check): on macOS the kernel tty output queue
caps every read on the pty master at about 1 KiB regardless of the
read buffer size. Instrumenting a pty with a 64 KB buffer while
streaming a 6.49 MB file produced 6,337 reads where every read was exactly
1024 bytes.
This immediately made me realize two things about the old loop that we've
had since like 2023 which called processOutput once per read: all per-call
overhead (terminal lock, render wakeup, cursor timestamp) was paid per
kilobyte of bulk output, and the child could never run more than 1 KiB
ahead of us, so while we parsed the child (e.g. `cat`) sat blocked on a full
kernel queue instead of producing.
In 2023, I justified this architecture by saying "reads are generally small"
but I didn't understand then that reads are generally small because the
kernel makes them small even if there is a lot of data.
To preserve latency for the more typical
small-reads-that-are-actually-small, sub-1 KB payloads deliver on the first
EAGAIN with no added latency.
End-to-end throughput was measured by timing `cat file > /dev/ttysN`
against a fresh app instance (M4 Max, macOS 26, ReleaseFast, medians
of repeated interleaved A/B runs):
| stream | before | after | change |
|-------------------------|--------------|---------------|---------|
| ascii.txt (6.5 MB) | 91-92 MB/s | 114-123 MB/s | +25-30% |
| unicode.txt (8 MB) | 116-117 MB/s | 180-183 MB/s | +55% |
| DOOM-Fire-Zig | 530 fps | 770 fps | +45% |
The pipeline now delivers the parse stage's full measured capacity
(the parse thread is pegged while gather spends ~33% of a core, so
any IO throughput improvements are now fully parser-bound).
**Linux note:** This needs to be verified on Linux. I think broadly
architecture is better and should never be worse. But its possible
some of the magic constants need to be tuned differently. Would love
more testing there.
Per review feedback, cover the four Windows branches added in the
parent commit:
- bare `cmd.exe` resolves via `%COMSPEC%` (with documented fallback)
- bare non-cmd shell (`pwsh.exe`) is passed through unchanged
- shell value with arguments (`wsl ~`) is split on whitespace
- direct command is passed through without modification
Co-authored-by: Claude <noreply@anthropic.com>
On Windows the shell value was always executed as `cmd.exe /C <shell>`.
For even a simple `command = wsl ~` this spawned two processes (the
cmd wrapper and the user's actual shell) and had visible side effects:
an extra cmd.exe in the process tree, and cmd AutoRun state (DOSKEY
aliases, `cd` in init.cmd, etc.) running in the wrapper rather than
the user's shell, since AutoRun is per-process.
Run the shell value directly. If it contains whitespace, split on
whitespace into argv. Bare `cmd.exe` is resolved via %COMSPEC% which
is the documented path to the current command processor; other bare
values are left to PATH resolution in Command.startWindows.
The simple whitespace split does not honor Windows CLI quoting rules.
Users who need quoted arguments should use the direct command form.
Also skip the termios focus timer on Windows since Windows has no
termios; the focusGained callback was starting a timer whose callback
would then do nothing.
Co-authored-by: Claude <noreply@anthropic.com>
This adds a function to the core surface to get process information
about the process(es) running in the terminal. Currently supported is
the PID of the foreground process and the name of the slave PTY.
If there is an error retrieving the information, or the platform does
not support retieving that information `null` is returned.
This will be useful in exposing the foreground PID and slave PTY name to
AppleScript or other APIs.
This changes the way Ghostty assigns itself and subprocesses to
cgroups and how resource controls are applied.
* Ghostty itself no longer modifies it's own cgroup or moves itself
to a transient scope. To modify the main Ghostty process' resource
controls ensure that you're launching Ghostty with a systemd unit and
use the standard systemd methods for overriding and applying changes
to systemd units.
* If configured (on by default), the process used to run your command
will be moved to a transient systemd scope after it is forked from
Ghostty but before the user's command is executed. Resource controls
will be applied to the transient scope at this time. Changes to
the `linux-cgroup*` configuration entries will not alter existing
commands. If changes are made to the `linux-cgroup*` configuration
entries commands will need to be relaunched. Resource limits can also
be modified after launch outside of Ghostty using systemd tooling. The
transient scope name can be shown by running `systemctl --user whoami`
in a shell running inside Ghostty.
Fixes#2084.
Related to #6669
The `cursor` shell feature always used a blinking bar (beam), often to
the surprise of users who set `cursor-style-blink = false`.
This change extends our GHOSTTY_SHELL_FEATURES format to include either
`cursor:blink` (default) or `cursor:steady` based on cursor-style-blink
when the `cursor` feature is enabled, and all shell integrations have
been updated to use that additional information to choose the DECSCUSR
cursor value (5=blinking bar, 6=steady bar).
I also considered passing a DECSCUSR value in GHOSTTY_SHELL_FEATURES
(e.g. `cursor:5`). This mostly worked well, but zsh also needs the blink
state on its own for its block cursor. We also don't support any other
shell feature cursor configurability (e.g. the shape), so this was an
over generalization.
This does change the behavior for users who like the blinking bar in the
shell but have `cursor-blink-style = false` for other reasons. We could
provide additional `cursor` shell feature configurability, but I think
that's best left to a separate change.
Nushell <https://www.nushell.sh/> is a modern interactive shell that
provides many shell features out-of-the-box, like `title` support. Our
shell integration therefore focuses on Ghostty-specific features like
`sudo`.
We use Nushell's module system to provide a `ghostty` module containing
our shell integration features. This module is automatically loaded from
$XDG_DATA_DIRS/nushell/vendor/autoload/ when `nushell` shell integration
is enabled.
Exported module functions need to be explicitly "used" before they're
available to the interactive shell environment. We do that automatically
by adding `--execute "use ghostty *"` to the `nu` command line.
This imports all available functions, and individual shell features are
runtime-guarded by the script code (using $GHOSTTY_SHELL_FEATURES). We
can consider further refining this later.
When automatic shell integration is disabled, users can still manually
source and enable the shell integration module:
source $GHOSTTY_RESOURCES_DIR/shell-integration/nushell/vendor/autoload/ghostty.nu
use ghostty *
This initial work implements our TERMINFO-aware `sudo` wrapper (via the
`sudo` shell feature). Support for additional features, like `ssh-env`
and `ssh-terminfo`, will follow.
Our existing logic already ensured that setupFeatures() was always
called, but that was happening from two code paths: explicitly when
shell integration is .none and implicitly via setup().
We can simplify this by always calling setupFeatures() once, outside of
the (automatic) shell integration path.
There's one small behavioral change: we previously didn't set up shell
features in the automatic shell integration path if we didn't have a
resources directory (as a side effect). Resources are required for shell
integrations, but we don't need them to export GHOSTTY_SHELL_FEATURES,
which could potentially still be useful on its on.
This cleans up some of our termio exec code by unifying process launch
state into a single union type. This makes it easier to distinguish
between the current two mutually exclusive modes of launching a process:
fork/exec and flatpak dbus commands.
It also ensures everyplace we touch related to process launching is
forced to address every case (exhaustive switch handling). I did find
one resource cleanup bug based on this cleanup, which is also fixed
here. This just improves memory slightly so it's not a big deal.
If we add future ways to launch processes, we can add a new union case.
For example, I originally had a `posix_spawn` option while I was
experimenting with that before abandoning it (see #9274).
this test previously didn't fail when accessing freed members of config
because deiniting `command_arena` was a no-op; `command_arena` was derived
from `arena`, which allocated memory after `command_arena` was created/used
Fixes#7792
Our error handling for `exec` failing within the forked process never
actually worked! It triggered all sorts of issues. We didn't catch this
before because it used to be exceptionally hard to fail an exec because
we used to wrap ALL commands in a `/bin/sh -c`.
However, we now support direction execution, most notably when you do
`ghostty -e <command>` but also via the `direct:` prefix on configured
commands.
This fixes up our exec failure handling by printing a useful error
message and avoiding any errdefers in the child which was causing the
double-close.
Fixes#7500
Supersedes #7582
This commit moves the child exit handling logic from the IO thead to the
apprt thread. The IO thread now only sends a `child_exited` message to
the apprt thread with metadata about the exit conditions (exit code,
runtime).
From there, the apprt thread can handle the exit situation however is
necessary. This commit doesn't change the behavior but it does fix the
issue #7500. The behavior is: exit immediately, show abnormal exit
message, wait for user input, etc.
This also gets us closer to #7649.
We previously only showed this message if the user had
`wait-after-command` set to true, since if its false the surface would
close anyways.
With the latest undo feature on macOS, this is no longer the case; a
exited process can be undone and reopened. I considered disallowing
undoing an exited surface, but I think there is value in being able to
go back and recapture output in scrollback if you wanted to.
This introduces a syntax for `command` and `initial-command` that allows
the user to specify whether it should be run via `/bin/sh -c` or not.
The syntax is a prefix `direct:` or `shell:` prior to the command,
with no prefix implying a default behavior as documented.
Previously, we unconditionally ran commands via `/bin/sh -c`, primarily
to avoid having to do any shell expansion ourselves. We also leaned on
it as a crutch for PATH-expansion but this is an easy problem compared
to shell expansion.
For the principle of least surprise, this worked well for configurations
specified via the config file, and is still the default. However, these
configurations are also set via the `-e` special flag to the CLI, and it
is very much not the principle of least surprise to have the command run via
`/bin/sh -c` in that scenario since a shell has already expanded all the
arguments and given them to us in a nice separated format. But we had no
way to toggle this behavior.
This commit introduces the ability to do this, and changes the defaults
so that `-e` doesn't shell expand. Further, we also do PATH lookups
ourselves for the non-shell expanded case because thats easy (using
execvpe style extensions but implemented as part of the Zig stdlib). We don't
do path expansion (e.g. `~/`) because thats a shell expansion.
So to be clear, there are no two polar opposite behavioes here with
clear semantics:
1. Direct commands are passed to `execvpe` directly, space separated.
This will not handle quoted strings, environment variables, path
expansion (e.g. `~/`), command expansion (e.g. `$()`), etc.
2. Shell commands are passed to `/bin/sh -c` and will be shell expanded
as per the shell's rules. This will handle everything that `sh`
supports.
In doing this work, I also stumbled upon a variety of smaller
improvements that could be made:
- A number of allocations have been removed from the startup path that
only existed to add a null terminator to various strings. We now
have null terminators from the beginning since we are almost always
on a system that's going to need it anyways.
- For bash shell integration, we no longer wrap the new bash command
in a shell since we've formed a full parsed command line.
- The process of creating the command to execute by termio is now unit
tested, so we can test the various complex cases particularly on
macOS of wrapping commands in the login command.
- `xdg-terminal-exec` on Linux uses the `direct:` method by default
since it is also assumed to be executed via a shell environment.
This allows `termio.Exec` to track processes spawned via
`FlatpakHostCommand`, finally allowing Ghostty to function as a
Flatpak.
Alongside this is a few bug fixes:
* Don't add ghostty to PATH when running in flatpak mode since it's
unreachable.
* Correctly handle exit status returned by Flatpak. Previously this was
not processed and contains extra status bits.
* Use correct type for PID returned by Flatpak.
Related to #3224
Previously, Ghostty used a static API for async event handling: io_uring
on Linux, kqueue on macOS. This commit changes the backend to be dynamic
on Linux so that epoll will be used if io_uring isn't available, or if
the user explicitly chooses it.
This introduces a new config `async-backend` (default "auto") which can
be set by the user to change the async backend in use. This is a
best-effort setting: if the user requests io_uring but it isn't
available, Ghostty will fall back to something that is and that choice
is up to us.
Basic benchmarking both in libxev and Ghostty (vtebench) show no
noticeable performance differences introducing the dynamic API, nor
choosing epoll over io_uring.
Fixes#5257
Specify environment variables to pass to commands launched in a terminal
surface. The format is `env=KEY=VALUE`.
`env = foo=bar`
`env = bar=baz`
Setting `env` to an empty string will reset the entire map to default
(empty).
`env =`
Setting a key to an empty string will remove that particular key and
corresponding value from the map.
`env = foo=bar`
`env = foo=`
will result in `foo` not being passed to the launched commands.
Setting a key multiple times will overwrite previous entries.
`env = foo=bar`
`env = foo=baz`
will result in `foo=baz` being passed to the launched commands.
These environment variables _will not_ be passed to commands run by Ghostty
for other purposes, like `open` or `xdg-open` used to open URLs in your
browser.
Caused by #5650
I actually don't understand how this didn't happen before or why we
didn't notice it but it seems like the envmap was never freed. In the
latest debug builds prior to this build GPA reports the leak.
We should free the envmap when the subprocess is deinitialized. But also
we can free the env map as soon as we start the subprocess which saves
some small amount of memory at runtime.
Additionally, we should only be freeing the envmap on error if we
created it.
`WINDOWID` is the conventional environment variable for scripts that
want to know the X11 window ID of the terminal, so that it may call
tools like `xprop` or `xdotool`. We already know the window ID for
window protocol handling, so we might as well throw this in for
convenience.
Fixes#4884
When our command exits, it will close the pty slave fd. This will
trigger a HUP on our poll. Previously, we only checked for IN. When a fd
is closed, IN triggers forever which would leave to an infinite loop and
100% CPU.
Now, detect the HUP and exit the read thread.
Multiple fixes to prevent file descriptor leaks:
- libxev eventfd now uses CLOEXEC
- linux: cgroup clone now uses CLOEXEC for the cgroup fd
- termio pipe uses pipe2 with CLOEXEC
- pty master always sets CLOEXEC because the child doesn't need it
- termio exec now closes pty slave fd after fork
There still appear to be some fd leaks happening. They seem related to
GTK, they aren't things we're accessig directly. I still want to
investigate them but this at least cleans up the major sources of fd
leakage.
## Descriptions
The code was short-circuiting the shell integration setup when
`shell-integration = none`, which prevented the feature environment
variables from being set. These environment variables are needed even
for manual shell integration to work properly.
## Changes
- Extracted feature environment variables setup into a separate
`setup_features` function
- Modified the shell integration initialization to ensure features are
set up even when `shell-integration = none`
<img width="1126" alt="image"
src="https://github.com/user-attachments/assets/ceeb33f5-26ee-4a3b-a6d5-eed57848c96c"
/>
Fixes https://github.com/ghostty-org/ghostty/issues/5046
login(1)'s .hushlogin logic was "fixed" in macOS Sonoma 14.4, so this
comment (and our workaround) is only relevant for versions earlier than
that.
The relevant change to login/login.c is part of system_cmds-979.100.8.
> login.c: look for .hushlogin in home directory (112854361)
- 1bca46ecc5
- https://github.com/apple-oss-distributions/distribution-macOS/tree/macos-144
If the read thread has already exited, it will have closed the read end
of the quit pipe. Unless SIGPIPE is masked with signal(SIGPIPE, SIG_IGN),
or the macOS-specific fcntl(F_SETNOSIGPIPE), writing to the write end of
a broken pipe kills the writer with SIGPIPE instead of returning -EPIPE
as an error. This causes a crash if the read thread exits before
threadExit.
This was already a possible race condition if read() returns
error.NotOpenForReading or error.InputOutput, but it's now much easier to
trigger due to the recent "termio/exec: fix 100% CPU usage after
wait-after-command process exits" fix.
Fix this by closing the quit pipe instead of writing to it.
`std.fs.accessAbsolute` asserts if the user proposed path is absolute,
which we are seemingly passing as-is with no validating that it is.
When running with safety checks on, passing non-absolute path to
--working-directory will make ghostty crash.
I changed it to use `Dir.access`, which is just `accessAbsolute` without
the check.
This has the side effect of also allowing relative working directory.
We're packaging more and more application-specific data directories in
our application bundle. It's helpful to add that path to XDG_DATA_DIRS
so those applications (that support XDG_DATA_DIRS) can locate their data
directories without additional user-level configuration.
This also fixes a typo ("MATHPATH") in the nearby MANPATH-building code.
Fixes#2273
On macOS, killpg is expected to fail with EPERM because of the way we
launch a login process around it. Before this commit, this caused us to
never call waitpid and reap the child process, which caused the child
process to stick around as a zombie.
This commit allows killpg to fail with EPERM on macOS and fall through
to waitpid.
