The processing of terminfo can be separated into two steps:
1. The initialization of terminfo, which includes trying to find $TERM
in a terminfo database file. As a fallback, common terminfo
definitions are compiled in. After this, we apply a lot of ad-hoc
patching to cover over limitations of terminfo.
2. While processing updates from nvim, actually using terminfo strings
and formatting them with runtime values. for this part, terminfo
essentially is a hyper-enhanced version of snprintf(), including
a sm0l stack based virtual machine which can manipulate the runtime
parameters.
This PR completely replaces libuniblium for step 2, with code
vendored from NetBSD's libtermkey which has been adapted to use typesafe
input parameters and to write into an output buffer in place.
The most immedatiate effects is a performance enhancement of
update_attrs() which is a very hot function when profiling the
TUI-process part of screen updates. In a stupid microbenchmark
(essentially calling nvim__screenshot over and over in a loop) this
leads to a speedup of ca 1.5x for redrawing the screen on the TUI-side.
What this means in practise when using nvim as a text editor is probably
no noticible effect at all, and when reabusing nvim as idk a full screen
RGB ASCII art rendrer maybe an increase from 72 to 75 FPS LMAO.
As nice side-effect, reduce the usage of unibilium to initialization only..
which will make it easier to remove, replace or make unibilium optional,
adressing #31989. Specifically, the builtin fallback doesn't use
unibilium at all, so a unibilium-free build is in principle possible
if the builtin definitions are good enough.
As a caveat, this PR doesn't touch libtermkey at all, which still has a
conditional dependency on unibilium. This will be investigated in a
follow-up PR
Note: the check of $TERMCOLOR was moved from tui/tui.c to
_defaults.lua in d7651b27d5 as we want to
skip the logic in _defaults.lua if the env var was set, but there
is no harm in TUI getting the right value when the TUI is trying to
initialize its terminfo shenanigans. Also this check is needed when
a TUI connects to a `--headless` server later, which will observe
a different $TERMCOLOR value than the nvim core process itself.
Problem: w_wrow/col calculation in terminal_check_cursor is wrong when the
terminal is smaller than the window. Common when there's a larger window open
with the same terminal, or just after a resize (as refresh_size is deferred).
Solution: don't calculate it; validate_cursor will correct it later if it's
out-of-date.
Note that the toplines set for the terminal (also before this PR) assume
'nowrap' (which is set by default for terminal windows), and that no weird stuff
like filler lines are around. That means, for example, it's possible for the
cursor to be moved off-screen if there's wrapped lines. If this happens, it's
likely update_topline will move the cursor back on screen via validate_cursor or
similar, but maybe this should be handled more elegantly in the future?
Problem: autocommands can cause various problems in terminal mode, which can
lead to crashes, for example.
Solution: fix found issues. Move some checks to terminal_check and guard against
autocommands messing with things. Trigger TermEnter/Leave after terminal mode
has changed/restored most state. Wipeout the correct buffer if TermLeave
switches buffers and fix a UAF if it or WinScrolled/Resized frees the terminal
prematurely.
These changes also allow us to remove the buffer restrictions on TextChangedT;
they were inadequate in stopping some issues, and WinScrolled/Resized was
lacking them anyway.
Problem: when creating a new tabpage with a terminal window, terminal size is
not updated if there is no statusline.
Solution: do not rely on last_status to implicitly call terminal_check_size as a
side effect of making room for a statusline; call it directly.
Problem: topline of a focused terminal window may not tail to terminal output if
events scroll the window.
Solution: set the topline in terminal_check_cursor.
Problem: missing redraws when restoring saved cursorline/column, plus missing
statusline and mode redraws when not updating the screen in terminal mode.
Solution: schedule the redraws in a similar manner to other modes and remove
some now unnecessary redrawing logic. Redraw if cursorline-related options
change from entering terminal mode. This fixes test failures in later commits.
WTF: TextChangedT triggers based on must_redraw, which is... fun...? Try to
preserve its behaviour as much as we can for now.
Problem: in terminal mode, adjust_topline moves curwin's cursor to the last row
so set_topline tails the non-scrollback area. This may result in the observed
cursor position remaining tailed in events within the focused terminal, rather
than reflecting the actual cursor position.
Solution: use the focused terminal's actual cursor position immediately, rather
than relying on the next terminal_check_cursor call in terminal_check to set it.
Note: Maybe also possible for terminal mode cursor position to be stale
(reporting the normal mode position) when switching buffers in events to another
terminal until the next terminal_check call? (or until refresh_terminal is
called for it) Maybe not worth fixing that, though.
Problem:
Not easy for a user to tell ":restart" to "run this command(s) after restarting".
Solution:
All ":restart" args following the optional +cmd arg are treated as a big cmdline that is passed as a "-c" CLI arg when restarting nvim.
Problem:
":restart" always executes ":qall" to exit the server.
Solution:
Support ":restart +cmd" so the user can control the command
used to exit the server.
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
Problem:
Cannot use `nvim_open_term()` to pipe terminal scrollback > 100000
Solution:
Increase scrollback limit to 1000000
If there's no technical consequences of doing this, can be set even
higher in the future.
Many terminals now include support for OSC 52 in their Primary Device
Attributes (DA1) response. This is preferable to using XTGETTCAP because
DA1 is _much_ more broadly supported.
Problem: Currently terminal highlight attribute buffers are statically allocated
be the size of `TERM_ATTRS_MAX`. This unique case isn't respected in
some places in the ui_compositor. Due to this, when a terminal window
has lines longer them `TERM_ATTRS_MAX`, the compositor will go past the
end of the buffer causing a crash due to out of bounds access.
Solution: Add check to ensure we don't query terminal highlight attrs
past `TERM_ATTRS_MAX` in `win_line()`.
Fixes#30374
Add '^' and '$' around the pattern. This makes it less likely to make
mistakes of when writing tests with {MATCH:}, as most such tests have
text before and after {MATCH:}.
vterm does not send us the terminator in the string fragment. Our OSC 8
parser assumed that it was and therefore treated short strings as
invalid (as it assumed it was missing a terminator).
Uses the undocumented "error_exit" UI event for a different purpose:
When :detach is used on the server, send an "error_exit" with 0 `status`
to indicate that the server shouldn't wait for client exit.
Problem:
Test often fails on cirrus CI (freebsd):
buffer cursor position is correct in terminal with number column in a line with single-cell multibyte chars and no trailing spaces, before_each
test/functional/terminal/cursor_spec.lua:805: Row 5 did not match.
Expected:
|{7: 1 } |
|{7: 2 } |
|{7: 3 } |
|{7: 4 } |
|*{7: 5 }Entering Ex mode. Type "visual" to go to Normal mode. |
|{7: 6 }:^ |
|{3:-- TERMINAL --} |
Actual:
|{7: 1 } |
|{7: 2 } |
|{7: 3 } |
|{7: 4 } |
|*{7: 5 } |
|{7: 6 }:^ |
|{3:-- TERMINAL --} |
Solution:
Skip it. Ex mode isn't that important.
Problem: A 'winblend' window floating over uninitialized cells loses
its highlighting.
Solution: Return the front attribute for uninitialized background cells.
Problem:
Developing/troubleshooting plugins has friction because "restarting"
Nvim requires quitting and manually starting again. #32484
Solution:
- Implement a `:restart` command which emits `restart` UI event.
- Handle the `restart` UI event in the builtin TUI client: stop the
`nvim --embed` server, start a new one, and attach to it.
Problem:
Currently undefined behavior can occur when `string_fragment()` is
called with `OSC_COMMAND`. This is because when the state changes to
`OSC_COMMAND`, `string_initial` is set to true. Then in some cases,
directly after this `string_initial` will be set back to false before
the on_osc callback is called, this leads to `term_settermprop()` never
initializing the title.
Solution:
In all of the no-op cases in `string_fragment()` currently, we continue
to the end of the function where `vt->parser.string_initial` is set to
false. This change returns in the no-op cases instead since in these
cases the string has not yet been terminated and sent to the callback.
Note:
This change also adds a test with a byte sequence from the file
in #34028 that caused nvim to crash. This byte sequences is the shortest
sequence I could trim down from that file that still would trigger the
crash. There are also two other tests I added which validate that
setting the title with OSC-0 and OSC-2 still works.
Fixes: #34028
Problem: The TUI doesn't forward a key properly when it has unsupported
modifiers like NumLock.
Solution: Don't try to add modifiers when only unsupported modifiers are
present.
Related #33791
This fixes the problem that sending a raw C0 control code to trigger a
mapping for it does not work in Terminal mode.
Note: this isn't done for 00 or 7F, as that'll be backward-incompatible.
Problem:
Default 'statusline' is implemented in C and not representable as
a statusline expression. This makes it hard for user configs/plugins to
extend it.
Solution:
- Change the default 'statusline' slightly to a statusline expression.
- Remove the C implementation.
Problem:
Nvim tries to use OSC 52 even when no TUIs are attached.
Solution:
On each UIEnter/UILeave event, check that there is a TUI client connected to Nvim's stdout.
Always allow the following four events to be nested, as they may contain
important information, and are triggered on the event loop, which may be
processed by a blocking call inside another autocommand.
- ChanInfo
- ChanOpen
- TermRequest
- TermResponse
There are some other events that are triggered on the event loop, but
they are mostly triggered by user actions in a UI client, and therefore
not very likely to happen during another autocommand, so leave them
unchanged for now.
Problem:
When a function like vim.wait() is used, we continuously drain the main
event queue until it is empty, never stopping for user input. This means
the libuv timer never runs and the terminal never gets refreshed, so
emit_termrequest continously reschedules itself onto the same event
queue, causing an infinite loop.
Solution:
Use a separate "pending" event queue, where events that require a
terminal refresh are temporarily placed. Drain this queue after a
terminal refresh and events are copied back onto the main queue. This
prevents infinite loops since the main event queue will always be able
to properly drain.
