This PR changes our primary/alt screen tracking from being by-value
fields that are copied during switch to heap-allocated pointers that are
stable. This is motivated now by #189 since our search thread needs a
stable screen pointer, but this will also help us in the future with our
future N-screens proposal I have.
Also, as a nice bonus, alt screen memory is now initialized on demand
when you first enter alt-screen, so this saves a few MB of memory for a
new terminal if you never use alt screen!
This is something I've wanted to do for a veryyyyyy long time, but the
annoyance of the task really held me back. I finally pushed through and
did this with the help of some AI agents for the rote tasks (renaming
stuff).
Chugging along towards #189
This adds significantly more internal work for searching. A long time
ago, I added #2885 which had a hint of what I was thinking of. This
simultaneously builds on this and changes direction.
The change of direction is that instead of making PageList fully
concurrency safe and having a search thread access it concurrently, I'm
now making an architectural shift where our search thread will grab the
big lock (blocking all IO/rendering), but with the bet that we can make
our critical areas small enough and time them well enough that the
performance hit while actively searching will be minimal. **Results yet
to be seen, but the path to implement this is much, much simpler.**
## Rearchitecting Search
To that end, this PR builds on #2885 by making `src/terminal/search` and
entire package (rather than a single file).
```mermaid
graph TB
subgraph Layer5 ["<b>Layer 5: Thread Orchestration</b>"]
Thread["<b>Thread</b><br/>━━━━━━━━━━━━━━━━━━━━━<br/>• MPSC queue management<br/>• libxev event loop<br/>• Message handling<br/>• Surface mailbox communication<br/>• Forward progress coordination"]
end
subgraph Layer4 ["<b>Layer 4: Screen Coordination</b>"]
ScreenSearch["<b>ScreenSearch</b><br/>━━━━━━━━━━━━━━━━━━━━━<br/>• State machine (tick + feed)<br/>• Result caching<br/>• Per-screen (alt/primary)<br/>• Composes Active + History search<br/>• Interrupt handling"]
end
subgraph Layer3 ["<b>Layer 3: Domain-Specific Search</b>"]
ActiveSearch["<b>ActiveSearch</b><br/>━━━━━━━━━━━━━━━━━━━━━<br/>• Active area only<br/>• Invalidate & re-search<br/>• Small, volatile data"]
PageListSearch["<b>PageListSearch</b><br/>━━━━━━━━━━━━━━━━━━━━━<br/>• History search (reverse order)<br/>• Separated tick/feed ops<br/>• Immutable PageList assumption<br/>• Garbage pin detection"]
end
subgraph Layer2 ["<b>Layer 1: Primitive Operations</b>"]
SlidingWindow["<b>SlidingWindow</b><br/>━━━━━━━━━━━━━━━━━━━━━<br/>• Manual linked list node management<br/>• Circular buffer maintenance<br/>• Zero-allocation search<br/>• Match yielding<br/>• Page boundary handling"]
end
Thread --> ScreenSearch
ScreenSearch --> ActiveSearch
ScreenSearch --> PageListSearch
ActiveSearch --> SlidingWindow
PageListSearch --> SlidingWindow
classDef layer5 fill:#0a0a0a,stroke:#ff0066,stroke-width:3px,color:#ffffff
classDef layer4 fill:#0f0f0f,stroke:#ff6600,stroke-width:3px,color:#ffffff
classDef layer3 fill:#141414,stroke:#ffaa00,stroke-width:3px,color:#ffffff
classDef layer2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff
class Thread layer5
class ScreenSearch layer4
class ActiveSearch,PageListSearch layer3
class SlidingWindow layer2
style Layer5 fill:#050505,stroke:#ff0066,stroke-width:2px,color:#ffffff
style Layer4 fill:#080808,stroke:#ff6600,stroke-width:2px,color:#ffffff
style Layer3 fill:#0c0c0c,stroke:#ffaa00,stroke-width:2px,color:#ffffff
style Layer2 fill:#101010,stroke:#00ff00,stroke-width:2px,color:#ffffff
```
Within the package, we have composable layers that let us test each
point:
- `SlidingWindow`: The lowest layer, the caller manually adds linked
list page nodes and it maintains a sliding window we search over,
yielding results without allocation (besides the circular buffers to
maintain the sliding window).
- `PageListSearch`: Searches a PageList structure in reverse order
(assumption: more recent matches are more valuable than older), but
separates out the `tick` (search, but no PageList access) and `feed`
(PageList access, prep data for search but don't search) operations.
This lets us `feed` in a critical area and `tick` outside. **This
assumes an immutable PageList, so this is for history.**
- `ActiveSearch`: Searches only the active area of a PageList. The
expectation is that the active area changes much more regularly, but it
is also very small (relative to scrollback). Throws away and re-searches
the active area as necessary.
- `ScreenSearch`: Composes the previous three components to coordinate
searching an active terminal screen. You'd have one of these per screen
(alt vs primary). This also caches results unlike the other components,
with the expectation that the caller will revisit the results as screens
change (so if you switch from neovim back to your shell and vice versa
with a search active, it won't start over).
- `Thread`: A dedicated search thread that will receive messages via
MPSC queues while managing the forward progress of a `ScreenSearch` and
sending matches back to the surface mailbox for apprt rendering. **The
thread component is not functional, just boilerplate, in this PR.**
ScreenSearch is a state machine that moves in an iterative `tick` +
`feed` fashion. This will let us "interrupt" the search with updates on
the search thread (read our mailbox via libxev loops for example) and
will let us minimize critical areas with locks (only `feed`).
Each component is significantly unit tested, especially around page
boundary cases. Given the complexity, there is no way this is perfect,
but the architecture is such that we can easily add regression tests as
we find issues.
## Other Changes, Notes
The only change to actually used code is that tracked pins in a
`PageList` can now be flagged as "garbage." A garbage tracked pin is one
that had to be moved in a non-sensical way because the previous location
it tracked has been deleted. This is used by the searcher to detect that
our history was pruned.
**If my assumption about the big lock is wrong** and this ends up being
godawful for performance, then it should still be okay because more
granular locking and reference counting such as that down by @dave-fl in
#8850 can be pushed into these components and reused. So this work is
still valuable on its own.
## Future
This PR is still just a bunch of internals, split out into its own PR so
I don't make one huge 10K diff PR. There are a number of future tasks:
- Flesh out `ScreenSearch` and hook it up to `Thread`
- Pull search thread management into `Surface` (or possibly the render
thread or shared render state since active area changes can be
synchronized with renderer frame rebuilds. Not sure yet.)
- Send updates back to the surface thread so that apprts can update UI.
- Apprt actions, input bindings, etc. to hook this all up (the easy
part, really).
The next step is to continue to flesh out the `ScreenSearch` as required
and hook it up to `Thread`.
**AI disclosure:** AI reviewed the code and assisted with some tests,
but didn't write any of the logic or design. This is beyond its ability
(or my ability to spec it out clearly enough for AI to succeed).
Fixes#9579
Protect against panics caused by integer overflows by using functions
that allow integer overflows to be caught instead of causing a panic.
Also protect against DOS from images that might not cause an overflow
but do consume an absurd amount of memory by limiting images to a
maximum size of 4GiB.
Fixes#9579
Protect against panics caused by integer overflows by using functions
that allow integer overflows to be caught instead of causing a panic.
Also protect against DOS from images that might not cause an
overflow but do consume an absurd amount of memory by limiting
images to a maximum size of 4GiB (which is the maximum size of
`image-storage-limit`).
Fixes#9532
Supersedes #9554
Turns out the explicit `UTF8_STRING` atom was not needed after all and
GTK adds it automatically when running under X11; just having the
explicit UTF-8 charset type is enough.
This corrects situations where it may not be necessary to include
(Wayland), in addition to removing a duplicate atom under X11.
Importantly, this also corrects issues under Wayland in some scenarios,
such as using Electron-based apps (e.g., VSCode/Codium under Ubuntu
24.04 LTS).
Turns out this was not needed after all and GTK adds it automatically
when running under X11; just having the explicit UTF-8 charset type is
enough.
This corrects situations where it may not be necessary to include
(Wayland), in addition to removing a duplicate atom under X11.
Importantly, this also corrects issues under Wayland in some scenarios,
such as using Electron-based apps (e.g., VSCode/Codium under Ubuntu
24.04 LTS).
Bumps
[cachix/install-nix-action](https://github.com/cachix/install-nix-action)
from 31.8.2 to 31.8.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/cachix/install-nix-action/releases">cachix/install-nix-action's
releases</a>.</em></p>
<blockquote>
<h2>v31.8.3</h2>
<h2>What's Changed</h2>
<ul>
<li>nix: 2.32.2 -> 2.32.3 by <a
href="https://github.com/github-actions"><code>@github-actions</code></a>[bot]
in <a
href="https://redirect.github.com/cachix/install-nix-action/pull/260">cachix/install-nix-action#260</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/cachix/install-nix-action/compare/v31.8.2...v31.8.3">https://github.com/cachix/install-nix-action/compare/v31.8.2...v31.8.3</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="7ec16f2c06"><code>7ec16f2</code></a>
Merge pull request <a
href="https://redirect.github.com/cachix/install-nix-action/issues/260">#260</a>
from cachix/create-pull-request/patch</li>
<li><a
href="5afc2ac89d"><code>5afc2ac</code></a>
nix: 2.32.2 -> 2.32.3</li>
<li>See full diff in <a
href="456688f15b...7ec16f2c06">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
There is a sparkle-related 'issue' with the previous implementation.
When you download/install in the `updateAvailable` state, if you don't
install it, then check the updates again. Sparkle loses its downloaded
stage in the delegate (it's normal when I use the sparkle source code).
This time, when you click install in the `updateAvailable` state, it
just uses the previous downloaded package and starts to install, without
calling `showReady(toInstallAndRelaunch:)`.
I think removing `readyToInstall` in our customised ui makes sense,
since the current package is pretty small and only takes a few seconds
to download for a normal network. And most of users intend to install
this update.
Also, there is no way back once you reach "Install and Restart" stage in
`SPUStandardUserDriver`, unless you force quit ofc.
~~This pr also contains some further cleanup for #9170, since
`InstallingView` is no longer visible for users.~~
#### `auto-download`
When `auto-download = check`, Sparkle will call
`UpdateDriver/showUpdateFound(with:state:reply:)`.
When `auto-download = download`, previously no update ui was presented
to the user, neither Sparkle's nor Ghostty’s.
> I tried tick&untick auto download&install in Sparkle's alert, which
doesn't seem to make any difference.
With this pr, `auto-download = check` will behave the same,
**`auto-download = download` will now prompt the user with a capsule.**
https://github.com/user-attachments/assets/f994dd29-d348-4fea-8777-df3720d6e7af
> [!NOTE]
> I used AI to proofread my comments
There is a sparkle-related 'issue' with the previous implementation. When you download/install in the `updateAvailable` state, if you don't install it, then check the updates again. Sparkle loses its downloaded stage in the delegate (it's normal when I use the sparkle source code). This time, when you click install in the `updateAvailable` state, it just uses the previous downloaded package and starts to install, without calling `showReady(toInstallAndRelaunch:)`.
I think removing `readyToInstall` in our customed ui, will reduce one step to install an update for most of the users out there, which makes sense, since the current package is pretty small, only takes a few seconds to download for a normal network, and they intended to install this update.
