mirror of
https://github.com/ghostty-org/ghostty.git
synced 2026-07-10 19:29:37 +00:00
terminal: VT throughput optimizations from real-world dataset (~1.2x to ~3.4x) (#13226)
This is a series of seven commits that optimizes VT processing throughput. Each commit is isolated, individually benchmarked, and carries a detailed commit message so please read each for details about each change. Whereas #13220 was driven by synthetic data, this series was driven by profiling a 2.6 GB recording of real terminal sessions from an asciinema data dump. Through this, I've been able to improve throughput processing the full dump from 276 to 342 MB/s on my system. > [!NOTE] > > **LLM usage:** This series of work was largely driven by Fable 5 and the summaries below started as LLM-written. I've proofread (and mostly modified) every line of work and rewritten everything to be shorter and more in line with how I'd describe a change. Nothing here was unreviewed. I also threw away 3 sets of changes I didn't agree with the maintenance of, but did speed up things a bit. ## The changes 1. **decode ASCII inline in the SIMD scan for ESC**. Real streams call `utf8DecodeUntilControlSeq` on short runs (an escape every ~18 bytes), so ~9% of total time was simdutf setup plus its scalar tail paid per tiny run. The ESC scan and the UTF-8 to UTF-32 "decode" (a widening for ASCII) are now one pass. **Result: +5.4% on the real corpus.** 2. **handle CSI entry bytes inline in consumeUntilGround**. The `[` after ESC and the single `csi_entry` byte each paid a `nextNonUtf8` call, two to three calls for every one of the tens of millions of CSI sequences in the recording. Both transitions are now handled in the consumeUntilGround loop, so a typical CSI parses with no per-byte calls at all. **Result: +2.7% real corpus, +3.1% CSI-heavy stream.** 3. **fill style-only cell runs in bulk in printSliceFill**. The largest single item in the profile (~25%). The print fast path's two scans (run eligibility, simple-cell check) are early-exit search loops LLVM won't vectorize, and real traffic constantly lands in the general path because styled text overwrites cells styled differently (TUI redraws), paying a per-cell release/use pair. The scans are now vectorized and uniformly-styled runs are consumed wholesale: one vector scan, one releaseMultiple/useMultiple pair, one branch-free fill. **Result: +11% real corpus, +21% TUI redraw.** 4. **release style refs per run instead of per cell in clearCells**. Erasing styled rows released each cell's style reference one at a time even though styled cells overwhelmingly share one style per run (status bars, highlighted regions, solid rows). Runs now release with a single releaseMultiple. **Result: +1.1% real corpus, 2.1x on full-screen styled erase.** 5. **log unsupported-input messages once per distinct value**. The recording triggers ~120k warnings, dominated by a few messages some program re-emitted every frame ("unimplemented mode: 34"), each paying formatting plus a blocking writev while adding nothing beyond the first occurrence. A logUnsupportedOnce helper suppresses repeats per (call site, value) using a 64-byte lock-free table per site. **Result: +3.2% on the real corpus, system time halved.** ## Benchmarks Measured with `ghostty-bench +terminal-stream` (full terminal handler, 120x80 terminal, M4 Max, macOS 26, ReleaseFast, hyperfine means of 6 runs, 64KiB read chunks). These are parser-stage numbers, not end-to-end app numbers. | stream | before | after | throughput | change | |-------------------------------|---------|---------|------------------|--------| | real 2.6 GB session recording | 9.441 s | 7.609 s | 276 → 342 MB/s | 1.24x | | ascii (no escapes) | 119 ms | 84 ms | 838 → 1186 MB/s | 1.41x | | TUI redraw (rotating styles) | 417 ms | 293 ms | 240 → 342 MB/s | 1.42x | | styled paint + ED 2 erase | 418 ms | 124 ms | 239 → 808 MB/s | 3.38x | | csi mix (random-color SGR/CUP)| 695 ms | 696 ms | (adversarial) | ~1.0x | Note the "csi mix" benchmark above was a generated adversarial input e.g. a worst-case input for the changes we made. It wasn't based in real-world data or expectations. But I asked for it to be done so we can verify we don't see regressions too much (and were able to verify we see basically none).
This commit is contained in:
Mitchell Hashimoto
@@ -112,11 +112,14 @@ fn step(ptr: *anyopaque) Benchmark.Error!void {
|
||||
// aren't currently IO bound.
|
||||
const f = self.data_f orelse return;
|
||||
|
||||
var read_buf: [4096]u8 align(std.atomic.cache_line) = undefined;
|
||||
var read_buf: [64 * 1024]u8 align(std.atomic.cache_line) = undefined;
|
||||
var f_reader = f.reader(&read_buf);
|
||||
const r = &f_reader.interface;
|
||||
|
||||
var buf: [4096]u8 = undefined;
|
||||
// This buffer size matches the read buffer size used by the
|
||||
// real IO thread (see termio Exec.zig buffer_capacity) so that
|
||||
// the benchmark exercises the stream with realistic chunk sizes.
|
||||
var buf: [64 * 1024]u8 = undefined;
|
||||
while (true) {
|
||||
const n = r.readSliceShort(&buf) catch {
|
||||
log.warn("error reading data file err={?}", .{f_reader.err});
|
||||
|
||||
171
src/simd/vt.cpp
171
src/simd/vt.cpp
@@ -198,6 +198,92 @@ size_t DecodeUTF8(const uint8_t* HWY_RESTRICT input,
|
||||
return static_cast<size_t>(out - output);
|
||||
}
|
||||
|
||||
// Widen the N uint8 lanes of v into N uint32 values stored at out.
|
||||
// This is the UTF-8 to UTF-32 "decode" for ASCII bytes.
|
||||
template <class D>
|
||||
static HWY_INLINE void WidenAsciiStore(D d,
|
||||
hn::Vec<D> v,
|
||||
char32_t* HWY_RESTRICT out) {
|
||||
uint32_t* HWY_RESTRICT out32 = reinterpret_cast<uint32_t*>(out);
|
||||
#if HWY_TARGET == HWY_SCALAR
|
||||
// The scalar fallback target has single-lane vectors, which cannot
|
||||
// be halved; widen the one lane directly.
|
||||
(void)d;
|
||||
out32[0] = hn::GetLane(v);
|
||||
#else
|
||||
const hn::Half<D> dh;
|
||||
const hn::Half<hn::Half<D>> dq;
|
||||
const hn::Rebind<uint32_t, decltype(dq)> d32;
|
||||
const size_t N4 = hn::Lanes(dq);
|
||||
const auto lo = hn::LowerHalf(dh, v);
|
||||
const auto hi = hn::UpperHalf(dh, v);
|
||||
hn::StoreU(hn::PromoteTo(d32, hn::LowerHalf(dq, lo)), d32, out32 + 0 * N4);
|
||||
hn::StoreU(hn::PromoteTo(d32, hn::UpperHalf(dq, lo)), d32, out32 + 1 * N4);
|
||||
hn::StoreU(hn::PromoteTo(d32, hn::LowerHalf(dq, hi)), d32, out32 + 2 * N4);
|
||||
hn::StoreU(hn::PromoteTo(d32, hn::UpperHalf(dq, hi)), d32, out32 + 3 * N4);
|
||||
#endif
|
||||
}
|
||||
|
||||
// The general (non-ASCII) portion of DecodeUTF8UntilControlSeqImpl.
|
||||
// Continues scanning for ESC starting at byte offset `base` and decodes
|
||||
// input[base..stop) via simdutf. The caller must have already decoded
|
||||
// input[0..base) as ASCII into output[0..base) (one codepoint per byte).
|
||||
template <class D>
|
||||
static HWY_NOINLINE size_t DecodeNonAsciiUntilControlSeq(
|
||||
D d,
|
||||
const T* HWY_RESTRICT input,
|
||||
size_t count,
|
||||
size_t base,
|
||||
char32_t* output,
|
||||
size_t* output_count) {
|
||||
const size_t N = hn::Lanes(d);
|
||||
const hn::Vec<D> esc_vec = Set(d, 0x1B);
|
||||
|
||||
// Compare N elements at a time.
|
||||
size_t i = base;
|
||||
for (; i + N <= count; i += N) {
|
||||
// Load the N elements from our input into a vector.
|
||||
const hn::Vec<D> input_vec = hn::LoadU(d, input + i);
|
||||
|
||||
// If we don't have any escapes we keep going. We want to accumulate
|
||||
// the largest possible valid UTF-8 sequence before decoding.
|
||||
const size_t esc_idx = IndexOfChunk(d, esc_vec, input_vec);
|
||||
if (esc_idx == kNotFound) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// We have an ESC char, decode up to this point. We start by assuming
|
||||
// a valid UTF-8 sequence and slow-path into error handling if we find
|
||||
// an invalid sequence.
|
||||
*output_count = base + DecodeUTF8(input + base, i + esc_idx - base,
|
||||
output + base);
|
||||
return i + esc_idx;
|
||||
}
|
||||
|
||||
// If we have leftover input then we scan it one byte at a time (slow!)
|
||||
// using pretty much the same logic as above.
|
||||
for (; i < count; ++i) {
|
||||
if (input[i] == 0x1B) {
|
||||
*output_count = base + DecodeUTF8(input + base, i - base, output + base);
|
||||
return i;
|
||||
}
|
||||
}
|
||||
|
||||
// If we reached this point, its possible for our input to have an
|
||||
// incomplete sequence because we're consuming the full input. We need
|
||||
// to trim any incomplete sequences from the end of the input.
|
||||
//
|
||||
// We use our own trim instead of simdutf::trim_partial_utf8 because
|
||||
// we only want to trim sequences that are valid-so-far (true partial
|
||||
// sequences that may be completed by future input). Invalid bytes
|
||||
// like C0, C1, F5-FF should NOT be trimmed — they should be passed
|
||||
// through to DecodeUTF8 which will replace them with U+FFFD per the
|
||||
// maximal subpart algorithm.
|
||||
const size_t trimmed_len = TrimValidPartialUTF8(input + base, count - base);
|
||||
*output_count = base + DecodeUTF8(input + base, trimmed_len, output + base);
|
||||
return base + trimmed_len;
|
||||
}
|
||||
|
||||
/// Decode the UTF-8 text in input into output until an escape
|
||||
/// character is found. This returns the number of bytes consumed
|
||||
/// from input and writes the number of decoded characters into
|
||||
@@ -217,59 +303,62 @@ size_t DecodeUTF8UntilControlSeqImpl(D d,
|
||||
|
||||
// Create a vector containing ESC since that denotes a control sequence.
|
||||
const hn::Vec<D> esc_vec = Set(d, 0x1B);
|
||||
// Any byte >= 0x80 is part of a multi-byte UTF-8 sequence.
|
||||
const hn::Vec<D> high_vec = Set(d, 0x80);
|
||||
|
||||
// Compare N elements at a time.
|
||||
// ASCII fast path: terminal input is overwhelmingly ASCII, for which
|
||||
// UTF-8 decoding is a simple widening of each byte to 32 bits. We
|
||||
// fuse the ESC scan with the decode, one chunk at a time, and only
|
||||
// fall back to the full UTF-8 decoder (simdutf) when we encounter a
|
||||
// non-ASCII byte. This avoids a second pass over the input and, for
|
||||
// the common short runs between escape sequences, avoids the fixed
|
||||
// overhead of the general-purpose decoder.
|
||||
size_t i = 0;
|
||||
for (; i + N <= count; i += N) {
|
||||
// Load the N elements from our input into a vector.
|
||||
const hn::Vec<D> input_vec = hn::LoadU(d, input + i);
|
||||
|
||||
// If we don't have any escapes we keep going. We want to accumulate
|
||||
// the largest possible valid UTF-8 sequence before decoding.
|
||||
// TODO(mitchellh): benchmark this vs decoding every time
|
||||
const size_t esc_idx = IndexOfChunk(d, esc_vec, input_vec);
|
||||
if (esc_idx == kNotFound) {
|
||||
continue;
|
||||
// Find the first byte that stops the ASCII fast path: an ESC or
|
||||
// any non-ASCII byte.
|
||||
const hn::Mask<D> stop_mask =
|
||||
hn::Or(hn::Eq(input_vec, esc_vec), hn::Ge(input_vec, high_vec));
|
||||
const intptr_t stop = hn::FindFirstTrue(d, stop_mask);
|
||||
|
||||
// Widen the whole chunk unconditionally: output is guaranteed to
|
||||
// be at least as large as input, and if we stop mid-chunk only
|
||||
// the prefix is reported (the rest is scratch that the caller
|
||||
// never reads).
|
||||
WidenAsciiStore(d, input_vec, output + i);
|
||||
if (stop < 0) continue;
|
||||
|
||||
const size_t stop_idx = i + static_cast<size_t>(stop);
|
||||
if (input[stop_idx] == 0x1B) {
|
||||
// ESC: everything before it was ASCII, one codepoint per byte.
|
||||
*output_count = stop_idx;
|
||||
return stop_idx;
|
||||
}
|
||||
|
||||
// We have an ESC char, decode up to this point. We start by assuming
|
||||
// a valid UTF-8 sequence and slow-path into error handling if we find
|
||||
// an invalid sequence.
|
||||
*output_count = DecodeUTF8(input, i + esc_idx, output);
|
||||
return i + esc_idx;
|
||||
// Non-ASCII: decode the rest (up to an ESC) with the full decoder.
|
||||
return DecodeNonAsciiUntilControlSeq(d, input, count, stop_idx, output,
|
||||
output_count);
|
||||
}
|
||||
|
||||
// If we have leftover input then we decode it one byte at a time (slow!)
|
||||
// using pretty much the same logic as above.
|
||||
if (i != count) {
|
||||
const hn::CappedTag<T, 1> d1;
|
||||
using D1 = decltype(d1);
|
||||
const hn::Vec<D1> esc1 = Set(d1, hn::GetLane(esc_vec));
|
||||
for (; i < count; ++i) {
|
||||
const hn::Vec<D1> input_vec = hn::LoadU(d1, input + i);
|
||||
const size_t esc_idx = IndexOfChunk(d1, esc1, input_vec);
|
||||
if (esc_idx == kNotFound) {
|
||||
continue;
|
||||
}
|
||||
|
||||
*output_count = DecodeUTF8(input, i + esc_idx, output);
|
||||
return i + esc_idx;
|
||||
// Leftover input (< N bytes): process one byte at a time.
|
||||
for (; i < count; ++i) {
|
||||
const T b = input[i];
|
||||
if (b == 0x1B) {
|
||||
*output_count = i;
|
||||
return i;
|
||||
}
|
||||
if (b >= 0x80) {
|
||||
return DecodeNonAsciiUntilControlSeq(d, input, count, i, output,
|
||||
output_count);
|
||||
}
|
||||
output[i] = b;
|
||||
}
|
||||
|
||||
// If we reached this point, its possible for our input to have an
|
||||
// incomplete sequence because we're consuming the full input. We need
|
||||
// to trim any incomplete sequences from the end of the input.
|
||||
//
|
||||
// We use our own trim instead of simdutf::trim_partial_utf8 because
|
||||
// we only want to trim sequences that are valid-so-far (true partial
|
||||
// sequences that may be completed by future input). Invalid bytes
|
||||
// like C0, C1, F5-FF should NOT be trimmed — they should be passed
|
||||
// through to DecodeUTF8 which will replace them with U+FFFD per the
|
||||
// maximal subpart algorithm.
|
||||
const size_t trimmed_len = TrimValidPartialUTF8(input, i);
|
||||
*output_count = DecodeUTF8(input, trimmed_len, output);
|
||||
return trimmed_len;
|
||||
// The entire input was ASCII (no ESC, no partial sequences possible).
|
||||
*output_count = count;
|
||||
return count;
|
||||
}
|
||||
|
||||
size_t DecodeUTF8UntilControlSeq(const uint8_t* HWY_RESTRICT input,
|
||||
|
||||
@@ -80,13 +80,18 @@ fn utf8DecodeUntilControlSeqScalar(
|
||||
var valid: usize = 1; // lead byte is valid
|
||||
for (0..seq.len - 1) |ci| {
|
||||
if (decode_offset + valid >= decode.len) {
|
||||
// Truncated at end of buffer: treat as incomplete
|
||||
// input that may be completed later. Stop decoding
|
||||
// without consuming these bytes.
|
||||
return .{
|
||||
// The sequence is cut off by the end of the decode
|
||||
// region. If the region ends at the true end of the
|
||||
// input then it may be completed by future input, so
|
||||
// stop without consuming these bytes. If the region
|
||||
// was bounded by an ESC then the sequence can never
|
||||
// be completed; the valid-so-far prefix is a maximal
|
||||
// subpart which maps to a single U+FFFD below.
|
||||
if (decode.len == input.len) return .{
|
||||
.consumed = decode_offset,
|
||||
.decoded = decode_count,
|
||||
};
|
||||
break;
|
||||
}
|
||||
const cb = decode[decode_offset + valid];
|
||||
if (cb < seq.ranges[ci][0] or cb > seq.ranges[ci][1]) {
|
||||
@@ -148,6 +153,58 @@ fn utf8SeqInfo(lead: u8) Utf8SeqInfo {
|
||||
};
|
||||
}
|
||||
|
||||
// Differential test: the SIMD implementation must agree with the
|
||||
// scalar implementation on any input. Exercises random mixtures of
|
||||
// ASCII, escapes, controls, valid and invalid UTF-8, at various
|
||||
// lengths (including chunk-boundary straddling cases).
|
||||
test "decode simd matches scalar" {
|
||||
if (comptime !options.simd) return error.SkipZigTest;
|
||||
|
||||
const testing = std.testing;
|
||||
var prng = std.Random.DefaultPrng.init(0xf00dface);
|
||||
const rand = prng.random();
|
||||
|
||||
var input: [257]u8 = undefined;
|
||||
var out_simd: [input.len]u32 = undefined;
|
||||
var out_scalar: [input.len]u32 = undefined;
|
||||
|
||||
for (0..10_000) |_| {
|
||||
const len = rand.intRangeAtMost(usize, 0, input.len);
|
||||
const style = rand.intRangeAtMost(u8, 0, 2);
|
||||
for (input[0..len]) |*b| {
|
||||
b.* = switch (style) {
|
||||
// Mostly ASCII with occasional specials.
|
||||
0 => switch (rand.intRangeAtMost(u8, 0, 20)) {
|
||||
0 => 0x1B,
|
||||
1 => rand.intRangeAtMost(u8, 0, 0x1F),
|
||||
2 => rand.int(u8),
|
||||
else => rand.intRangeAtMost(u8, 0x20, 0x7E),
|
||||
},
|
||||
// Heavy multi-byte/invalid UTF-8.
|
||||
1 => switch (rand.intRangeAtMost(u8, 0, 3)) {
|
||||
0 => rand.intRangeAtMost(u8, 0x80, 0xBF),
|
||||
1 => rand.intRangeAtMost(u8, 0xC0, 0xFF),
|
||||
2 => 0x1B,
|
||||
else => rand.intRangeAtMost(u8, 0x20, 0x7E),
|
||||
},
|
||||
// Fully random bytes.
|
||||
else => rand.int(u8),
|
||||
};
|
||||
}
|
||||
|
||||
const res_simd = utf8DecodeUntilControlSeq(input[0..len], &out_simd);
|
||||
const res_scalar = utf8DecodeUntilControlSeqScalar(input[0..len], &out_scalar);
|
||||
errdefer std.debug.print("input={x}\n", .{input[0..len]});
|
||||
try testing.expectEqual(res_scalar.consumed, res_simd.consumed);
|
||||
try testing.expectEqual(res_scalar.decoded, res_simd.decoded);
|
||||
try testing.expectEqualSlices(
|
||||
u32,
|
||||
out_scalar[0..res_scalar.decoded],
|
||||
out_simd[0..res_simd.decoded],
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
test "decode no escape" {
|
||||
const testing = std.testing;
|
||||
|
||||
@@ -399,6 +456,35 @@ test "decode valid multibyte surrounded by invalid" {
|
||||
}
|
||||
}
|
||||
|
||||
test "decode partial UTF-8 before escape" {
|
||||
const testing = std.testing;
|
||||
|
||||
// A valid-so-far but incomplete sequence cut off by an ESC can
|
||||
// never be completed, so it is consumed and replaced by a single
|
||||
// U+FFFD (maximal subpart) rather than left pending. Only
|
||||
// sequences cut off by the true end of input are left pending.
|
||||
var output: [64]u32 = undefined;
|
||||
|
||||
// 2-byte lead cut off by ESC.
|
||||
{
|
||||
const str = "hi\xc2\x1b[0m";
|
||||
const result = utf8DecodeUntilControlSeq(str, &output);
|
||||
try testing.expectEqual(@as(usize, 3), result.consumed);
|
||||
try testing.expectEqual(@as(usize, 3), result.decoded);
|
||||
try testing.expectEqual(@as(u32, 0xFFFD), output[2]);
|
||||
}
|
||||
|
||||
// 3-byte lead plus one valid continuation cut off by ESC:
|
||||
// the whole prefix is one maximal subpart, one U+FFFD.
|
||||
{
|
||||
const str = "\xe0\xa0\x1bX";
|
||||
const result = utf8DecodeUntilControlSeq(str, &output);
|
||||
try testing.expectEqual(@as(usize, 2), result.consumed);
|
||||
try testing.expectEqual(@as(usize, 1), result.decoded);
|
||||
try testing.expectEqual(@as(u32, 0xFFFD), output[0]);
|
||||
}
|
||||
}
|
||||
|
||||
test "decode invalid byte before escape" {
|
||||
const testing = std.testing;
|
||||
|
||||
|
||||
@@ -1432,9 +1432,21 @@ pub fn clearCells(
|
||||
}
|
||||
|
||||
if (row.styled) {
|
||||
for (cells) |*cell| {
|
||||
if (cell.hasStyling())
|
||||
page.styles.release(page.memory, cell.style_id);
|
||||
// Styled cells overwhelmingly come in runs sharing the same
|
||||
// style (e.g. a colored status bar or a highlighted region),
|
||||
// so group them and release each run with a single ref-count
|
||||
// update rather than per cell.
|
||||
var i: usize = 0;
|
||||
while (i < cells.len) {
|
||||
const id = cells[i].style_id;
|
||||
if (id == style.default_id) {
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
var j = i + 1;
|
||||
while (j < cells.len and cells[j].style_id == id) j += 1;
|
||||
page.styles.releaseMultiple(page.memory, id, @intCast(j - i));
|
||||
i = j;
|
||||
}
|
||||
|
||||
// If we have no left/right scroll region we can be sure
|
||||
|
||||
@@ -528,7 +528,32 @@ fn printSliceFill(
|
||||
// in the run is always written as a fresh, single-codepoint cell,
|
||||
// so the grapheme break check against it is exact.
|
||||
const run_len: usize = run: {
|
||||
for (1..cps.len) |idx| {
|
||||
var idx: usize = 1;
|
||||
|
||||
// Vectorized scan for the narrow class: codepoints in
|
||||
// [0x10, 0xFF] are always eligible with no further checks
|
||||
// and dominate real-world input, so scan for the first
|
||||
// codepoint outside that range several lanes at a time.
|
||||
// Anything else (including eligible unicode) proceeds via
|
||||
// the scalar loop below.
|
||||
if (comptime width == .narrow) {
|
||||
const lanes = 8;
|
||||
const V = @Vector(lanes, u32);
|
||||
const lo: V = @splat(0x10);
|
||||
const hi: V = @splat(0xFF);
|
||||
while (idx + lanes <= cps.len) {
|
||||
const v: V = cps[idx..][0..lanes].*;
|
||||
const in_range = (v >= lo) & (v <= hi);
|
||||
if (!@reduce(.And, in_range)) {
|
||||
const bits: std.meta.Int(.unsigned, lanes) = @bitCast(in_range);
|
||||
idx += @ctz(~bits);
|
||||
break;
|
||||
}
|
||||
idx += lanes;
|
||||
}
|
||||
}
|
||||
|
||||
while (idx < cps.len) : (idx += 1) {
|
||||
const cp = cps[idx];
|
||||
if (comptime width == .narrow) {
|
||||
if (cp >= 0x10 and cp <= 0xFF) continue;
|
||||
@@ -630,11 +655,31 @@ fn printSliceFill(
|
||||
var k: usize = 0; // cells written
|
||||
fill: while (k < cell_count) {
|
||||
// Find the run of simple cells so the store loop below is
|
||||
// branch-free (and vectorizable).
|
||||
// branch-free (and vectorizable). This is an early-exit
|
||||
// search loop that LLVM won't auto-vectorize, and reused
|
||||
// rows typically match the whole way through, so scan
|
||||
// several cells at a time manually.
|
||||
var simple = k;
|
||||
while (simple < cell_count) : (simple += 1) {
|
||||
const bits: u64 = @bitCast(cells[simple]);
|
||||
if ((bits & simple_mask) != check_expected) break;
|
||||
simple: {
|
||||
const lanes = 4;
|
||||
const V = @Vector(lanes, u64);
|
||||
const mask_v: V = @splat(simple_mask);
|
||||
const expect_v: V = @splat(check_expected);
|
||||
const cells64: [*]const u64 = @ptrCast(cells);
|
||||
while (simple + lanes <= cell_count) {
|
||||
const v: V = cells64[simple..][0..lanes].*;
|
||||
const ok = (v & mask_v) == expect_v;
|
||||
if (!@reduce(.And, ok)) {
|
||||
const bits: std.meta.Int(.unsigned, lanes) = @bitCast(ok);
|
||||
simple += @ctz(~bits);
|
||||
break :simple;
|
||||
}
|
||||
simple += lanes;
|
||||
}
|
||||
while (simple < cell_count) : (simple += 1) {
|
||||
const bits: u64 = @bitCast(cells[simple]);
|
||||
if ((bits & simple_mask) != check_expected) break;
|
||||
}
|
||||
}
|
||||
|
||||
if (comptime width == .wide) {
|
||||
@@ -665,6 +710,68 @@ fn printSliceFill(
|
||||
}
|
||||
if (k >= cell_count) break;
|
||||
|
||||
// Bulk path for runs of cells that differ from the
|
||||
// expected simple cell only by their style: this is the
|
||||
// common case when styled text overwrites previously
|
||||
// styled (or default-styled) rows, e.g. TUI redraws.
|
||||
// These runs are handled wholesale: one scan to find the
|
||||
// run of identical old styles, two ref-count updates,
|
||||
// and a branch-free fill.
|
||||
if (comptime width == .narrow) bulk: {
|
||||
const cells64: [*]const u64 = @ptrCast(cells);
|
||||
const first = cells64[k] & simple_mask;
|
||||
|
||||
// The old cell must be a plain narrow codepoint cell
|
||||
// with no hyperlink whose only difference is the
|
||||
// style id (see printSliceCheckExpected: every other
|
||||
// masked field must be zero).
|
||||
const style_shift = @bitOffsetOf(Cell, "style_id");
|
||||
const old_style: style.Id = @truncate(first >> style_shift);
|
||||
if (first != printSliceCheckExpected(old_style)) break :bulk;
|
||||
assert(old_style != style_id); // it failed the simple check
|
||||
|
||||
// Find the run of cells with identical masked bits.
|
||||
var m = k + 1;
|
||||
scan: {
|
||||
const lanes = 4;
|
||||
const V = @Vector(lanes, u64);
|
||||
const mask_v: V = @splat(simple_mask);
|
||||
const first_v: V = @splat(first);
|
||||
while (m + lanes <= cell_count) {
|
||||
const v: V = cells64[m..][0..lanes].*;
|
||||
const ok = (v & mask_v) == first_v;
|
||||
if (!@reduce(.And, ok)) {
|
||||
const bits: std.meta.Int(.unsigned, lanes) = @bitCast(ok);
|
||||
m += @ctz(~bits);
|
||||
break :scan;
|
||||
}
|
||||
m += lanes;
|
||||
}
|
||||
while (m < cell_count) : (m += 1) {
|
||||
if ((cells64[m] & simple_mask) != first) break;
|
||||
}
|
||||
}
|
||||
|
||||
// Fix up the style ref counts for the whole run at
|
||||
// once. Each of the old cells held a reference to
|
||||
// old_style so the release is safe by construction.
|
||||
const n = m - k;
|
||||
if (old_style != style.default_id) {
|
||||
page.styles.releaseMultiple(page.memory, old_style, @intCast(n));
|
||||
}
|
||||
if (style_id != style.default_id) {
|
||||
page.styles.useMultiple(page.memory, style_id, @intCast(n));
|
||||
}
|
||||
|
||||
for (k..m) |idx| {
|
||||
cells[idx] = @bitCast(
|
||||
template_bits | (@as(u64, cps[printed + idx]) << cp_shift),
|
||||
);
|
||||
}
|
||||
k = m;
|
||||
continue :fill;
|
||||
}
|
||||
|
||||
// General path for cells that failed the masked check:
|
||||
// style-only mismatches are handled inline; anything that
|
||||
// needs cleanup (wide chars and their spacers, grapheme
|
||||
|
||||
@@ -612,10 +612,27 @@ pub fn Stream(comptime H: type) type {
|
||||
while (self.parser.state != .ground) {
|
||||
if (offset >= input.len) return input.len;
|
||||
|
||||
// Bulk-consume CSI parameter bytes. This can't be used
|
||||
// for handlers with a vtRaw hook because it dispatches
|
||||
// the CSI directly (see nextNonUtf8).
|
||||
// Fast path for CSI entry: "ESC [" is by far the most
|
||||
// common escape sequence prefix, so handle the '[' and
|
||||
// the byte that follows it here rather than paying a
|
||||
// nextNonUtf8 call for each.
|
||||
if (self.parser.state == .escape and input[offset] == '[') {
|
||||
self.parser.state = .csi_entry;
|
||||
offset += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (comptime !@hasDecl(T, "vtRaw")) {
|
||||
if (self.parser.state == .csi_entry) {
|
||||
if (self.csiEntryByte(input[offset])) {
|
||||
offset += 1;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// Bulk-consume CSI parameter bytes. This can't be
|
||||
// used for handlers with a vtRaw hook because it
|
||||
// dispatches the CSI directly (see nextNonUtf8).
|
||||
if (self.parser.state == .csi_param) {
|
||||
offset += self.consumeCsiParams(input[offset..]);
|
||||
if (offset >= input.len) return input.len;
|
||||
@@ -632,6 +649,47 @@ pub fn Stream(comptime H: type) type {
|
||||
return offset;
|
||||
}
|
||||
|
||||
/// Fast path for a byte in the csi_entry state, the state right
|
||||
/// after "ESC [". Virtually every CSI sequence spends exactly
|
||||
/// one byte in this state, on either a digit, a private marker,
|
||||
/// or a final byte. Returns true if the byte was fully handled;
|
||||
/// false means the caller must process it through the general
|
||||
/// state machine.
|
||||
///
|
||||
/// Must not be used by handlers with a vtRaw hook because the
|
||||
/// final byte case dispatches the CSI directly.
|
||||
inline fn csiEntryByte(self: *Self, c: u8) bool {
|
||||
comptime assert(!@hasDecl(T, "vtRaw"));
|
||||
assert(self.parser.state == .csi_entry);
|
||||
switch (c) {
|
||||
// First parameter digit.
|
||||
'0'...'9' => {
|
||||
self.parser.state = .csi_param;
|
||||
// param_acc is zero (cleared on escape entry)
|
||||
// so accumulating is just the digit value.
|
||||
self.parser.param_acc = c - '0';
|
||||
self.parser.param_acc_idx = 1;
|
||||
},
|
||||
// An empty first parameter.
|
||||
';' => {
|
||||
self.parser.state = .csi_param;
|
||||
self.parser.params[0] = 0;
|
||||
self.parser.params_idx = 1;
|
||||
},
|
||||
// Private marker (e.g. '?' in "ESC [ ? 2004 h").
|
||||
0x3C...0x3F => {
|
||||
self.parser.state = .csi_param;
|
||||
self.parser.collect(c);
|
||||
},
|
||||
// A final byte: a parameterless CSI.
|
||||
0x40...0x7E => self.csiDispatchFinal(c),
|
||||
// Defer to the state machine for anything else
|
||||
// (C0 controls, intermediates, colon).
|
||||
else => return false,
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/// Bulk-consume CSI parameter bytes (digits and separators)
|
||||
/// and, if reached, the final byte (dispatching the CSI).
|
||||
/// Returns the number of bytes consumed. Stops at the first
|
||||
@@ -828,38 +886,9 @@ pub fn Stream(comptime H: type) type {
|
||||
}
|
||||
|
||||
// Fast path for CSI entry, the state right after "ESC [".
|
||||
// Virtually every CSI sequence spends exactly one byte in
|
||||
// this state, on either a digit, a private marker, or a
|
||||
// final byte.
|
||||
if (comptime !has_vt_raw) {
|
||||
if (self.parser.state == .csi_entry) csi_entry: {
|
||||
switch (c) {
|
||||
// First parameter digit.
|
||||
'0'...'9' => {
|
||||
self.parser.state = .csi_param;
|
||||
// param_acc is zero (cleared on escape entry)
|
||||
// so accumulating is just the digit value.
|
||||
self.parser.param_acc = c - '0';
|
||||
self.parser.param_acc_idx = 1;
|
||||
},
|
||||
// An empty first parameter.
|
||||
';' => {
|
||||
self.parser.state = .csi_param;
|
||||
self.parser.params[0] = 0;
|
||||
self.parser.params_idx = 1;
|
||||
},
|
||||
// Private marker (e.g. '?' in "ESC [ ? 2004 h").
|
||||
0x3C...0x3F => {
|
||||
self.parser.state = .csi_param;
|
||||
self.parser.collect(c);
|
||||
},
|
||||
// A final byte: a parameterless CSI.
|
||||
0x40...0x7E => self.csiDispatchFinal(c),
|
||||
// Defer to the state machine for anything else
|
||||
// (C0 controls, intermediates, colon).
|
||||
else => break :csi_entry,
|
||||
}
|
||||
return;
|
||||
if (self.parser.state == .csi_entry) {
|
||||
if (self.csiEntryByte(c)) return;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -982,7 +1011,7 @@ pub fn Stream(comptime H: type) type {
|
||||
.SO => self.handler.vt(.invoke_charset, .{ .bank = .GL, .charset = .G1, .locking = false }),
|
||||
.SI => self.handler.vt(.invoke_charset, .{ .bank = .GL, .charset = .G0, .locking = false }),
|
||||
|
||||
else => log.warn("invalid C0 character, ignoring: 0x{x}", .{c}),
|
||||
else => logUnsupportedOnce("invalid C0 character, ignoring: 0x{x}", .{c}, c),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1500,7 +1529,11 @@ pub fn Stream(comptime H: type) type {
|
||||
if (req) |r| {
|
||||
self.handler.vt(.device_attributes, r);
|
||||
} else {
|
||||
log.warn("invalid device attributes command: {f}", .{input});
|
||||
logUnsupportedOnce(
|
||||
"invalid device attributes command: {f}",
|
||||
.{input},
|
||||
if (input.params.len > 0) input.params[0] else 0,
|
||||
);
|
||||
return;
|
||||
}
|
||||
},
|
||||
@@ -1586,7 +1619,7 @@ pub fn Stream(comptime H: type) type {
|
||||
if (modes.modeFromInt(mode_int, ansi_mode)) |mode| {
|
||||
self.handler.vt(.set_mode, .{ .mode = mode });
|
||||
} else {
|
||||
log.warn("unimplemented mode: {}", .{mode_int});
|
||||
logUnsupportedOnce("unimplemented mode: {}", .{mode_int}, mode_int);
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -1607,7 +1640,7 @@ pub fn Stream(comptime H: type) type {
|
||||
if (modes.modeFromInt(mode_int, ansi_mode)) |mode| {
|
||||
self.handler.vt(.reset_mode, .{ .mode = mode });
|
||||
} else {
|
||||
log.warn("unimplemented mode: {}", .{mode_int});
|
||||
logUnsupportedOnce("unimplemented mode: {}", .{mode_int}, mode_int);
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -1676,9 +1709,10 @@ pub fn Stream(comptime H: type) type {
|
||||
self.handler.vt(.modify_key_format, format);
|
||||
},
|
||||
|
||||
else => log.warn(
|
||||
else => logUnsupportedOnce(
|
||||
"unknown CSI m with intermediate: {}",
|
||||
.{input.intermediates[0]},
|
||||
input.intermediates[0],
|
||||
),
|
||||
},
|
||||
|
||||
@@ -2010,13 +2044,15 @@ pub fn Stream(comptime H: type) type {
|
||||
23 => self.handler.vt(.title_pop, index),
|
||||
else => @compileError("unreachable"),
|
||||
}
|
||||
} else log.warn(
|
||||
} else logUnsupportedOnce(
|
||||
"ignoring CSI 22/23 t with extra parameters: {f}",
|
||||
.{input},
|
||||
input.params[0],
|
||||
),
|
||||
else => log.warn(
|
||||
else => logUnsupportedOnce(
|
||||
"ignoring CSI t with unimplemented parameter: {f}",
|
||||
.{input},
|
||||
input.params[0],
|
||||
),
|
||||
}
|
||||
} else log.err(
|
||||
@@ -2191,7 +2227,11 @@ pub fn Stream(comptime H: type) type {
|
||||
|
||||
.change_window_icon => |icon| {
|
||||
@branchHint(.likely);
|
||||
log.info("OSC 1 (change icon) received and ignored icon={s}", .{icon});
|
||||
logUnsupportedOnce(
|
||||
"OSC 1 (change icon) received and ignored icon={s}",
|
||||
.{icon},
|
||||
0,
|
||||
);
|
||||
},
|
||||
|
||||
.clipboard_contents => |clip| {
|
||||
@@ -2383,7 +2423,11 @@ pub fn Stream(comptime H: type) type {
|
||||
else => {}, // fall through
|
||||
}
|
||||
|
||||
log.warn("unimplemented ESC action: {f}", .{action});
|
||||
logUnsupportedOnce(
|
||||
"unimplemented ESC action: {f}",
|
||||
.{action},
|
||||
action.final,
|
||||
);
|
||||
},
|
||||
|
||||
// IND - Index
|
||||
@@ -2577,12 +2621,72 @@ pub fn Stream(comptime H: type) type {
|
||||
@branchHint(.likely);
|
||||
},
|
||||
|
||||
else => log.warn("unimplemented ESC action: {f}", .{action}),
|
||||
else => logUnsupportedOnce(
|
||||
"unimplemented ESC action: {f}",
|
||||
.{action},
|
||||
action.final,
|
||||
),
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/// Logs an unsupported-input message at most once per distinct key
|
||||
/// per process.
|
||||
///
|
||||
/// These messages are emitted in response to input that the terminal
|
||||
/// application controls, so a misbehaving (or merely chatty) program
|
||||
/// can trigger the same message millions of times, e.g. by toggling
|
||||
/// an unimplemented mode on every frame. Each log call has a real
|
||||
/// throughput cost (formatting plus a blocking write per message)
|
||||
/// while adding no diagnostic value beyond the first occurrence.
|
||||
///
|
||||
/// The keys seen so far are tracked in a small fixed table (64 bytes)
|
||||
/// instantiated per (format, argument type) tuple, i.e. roughly per
|
||||
/// call site. Real streams only ever produce a handful of distinct
|
||||
/// unsupported values per site, so if the table ever fills, messages
|
||||
/// for further new values are suppressed as well: by that point the
|
||||
/// log already shows this class of problem and unbounded distinct
|
||||
/// values would flood it anyway.
|
||||
fn logUnsupportedOnce(
|
||||
comptime format: []const u8,
|
||||
args: anytype,
|
||||
key: u16,
|
||||
) void {
|
||||
// u32 slots so every u16 key is representable alongside an empty
|
||||
// sentinel and so 32-bit targets (e.g. wasm32) have native
|
||||
// atomics.
|
||||
const empty = std.math.maxInt(u32);
|
||||
const Static = struct {
|
||||
var seen: [16]u32 = @splat(empty);
|
||||
};
|
||||
|
||||
// The atomics make concurrent streams safe: slots are only ever
|
||||
// claimed, never changed, so the scan can stop at the first empty
|
||||
// slot. The worst case race is a benign duplicate message.
|
||||
for (&Static.seen) |*slot| {
|
||||
const cur = @atomicLoad(u32, slot, .acquire);
|
||||
if (cur == key) return; // already logged
|
||||
if (cur != empty) continue; // other key, keep scanning
|
||||
|
||||
// Empty slot: claim it for this key and log below.
|
||||
const actual = @cmpxchgStrong(
|
||||
u32,
|
||||
slot,
|
||||
empty,
|
||||
key,
|
||||
.acq_rel,
|
||||
.acquire,
|
||||
) orelse break;
|
||||
|
||||
// Lost the race: suppress if it was to the same key, keep
|
||||
// scanning otherwise.
|
||||
if (actual == key) return;
|
||||
} else return; // table full: suppress new values too
|
||||
|
||||
log.warn(format, args);
|
||||
}
|
||||
|
||||
test Action {
|
||||
// Forces the C type to be reified when the target is C, ensuring
|
||||
// all our types are C ABI compatible.
|
||||
|
||||
Reference in New Issue
Block a user