Problem: Setting title while TUI buffer is almost full may cause the
end of a flush to be treated as a part of an OSC 2 or OSC 0
sequence, leading to problems like invisible cursor.
Solution: Make the whole sequence to set title a unibi_ext string.
Now that #16425 is fixed, use Unicode again to reduce screen height.
Unfortunately composing chars still can't be used, as it turns out that
composing chars may be missing when sent separately from the base char
at the last char a screen line.
Problem:
Using a single RPC request to child server isn't enough to prevent race
between nvim_paste and nvim_input.
Solution:
Ensure both child client and child server have processed pending events
by sending an empty DCS response to the child client and waiting for
TermResponse autocommand on the child server.
Before this PR, the behavior of nvim_paste is:
- When vim.paste() returns false, return false to the client, but treat
following chunks normally (i.e. rely on the client cancelling the
paste as expected).
- When vim.paste() throws an error, still return true to the client, but
drain the following chunks in the stream without calling vim.paste().
There are two problems with such behavior:
- When vim.paste() errors, the client is still supposed to send the
remaining chunks of the stream, even though they do nothing.
- Having different code paths for two uncommon but similar situations
complicates maintenance.
This PR makes both the cancel case and the error case return false to
the client and drain the remaining chunks of the stream, which, apart
from sharing the same code path, is beneficial whether the client checks
the return value of nvim_paste or not:
- If the client checks the return value, it can avoid sending the
following chunks needlessly after an error.
- If the client doesn't check the return value, chunks following a
cancelled chunk won't be pasted on the server regardless, which leads
to less confusing behavior.
Problem: When input immediately follows end of bracketed paste, the
nvim_input may be processed before the nvim_paste.
Solution: Ensure some waiting after the end of a bracketed paste.
- Fixes 'autoindent' being applied during redo.
- Makes redoing a large paste significantly faster.
- Stores pasted text in the register being recorded.
Fix#28561
Problem:
The default builtin UI client does not declare its client info. This
reduces discoverability and makes it difficult for plugins to identify
the UI.
Solution:
- Call nvim_set_client_info after attaching, as recommended by `:help dev-ui`.
- Also set the "pid" field.
- Also change `ui_active()` to return a count. Not directly relevant to
this commit, but will be useful later.
This module is generally used by any tests that need the full Nvim TUI
instead of `screen.lua`. Thus it should live in `functional/` instead of
in `functional/terminal/`.
Problem: both `StatusLineTerm`/`StatusLineTermNC` are now explicitly
used, but `:color vim` does not set them to the values used in Vim.
This might be fine if `:color vim` is treated as "the state of default
color scheme prior the big update", but it seems to be better treated
as "Vim's default color scheme" (how it is documented in its header).
Solution: add `StatusLineTerm`/`StatusLineTermNC` definitions to
'runtime/colors/vim.lua'.
Use explicit foreground colors ('Whte'/'Black') instead of `guifg=bg`
used in source, as the latter caused some problems in the past (if
`Normal` is not defined, `nvim_set_hl()` can't recognize `'bg'` as the
foreground value).
Also realign the rest of the background conditional highlight groups.
We currently check $COLORTERM in the TUI process to determine if the
terminal supports 24 bit color (truecolor). If $COLORTERM is "truecolor"
or "24bit" then we automatically assume that the terminal supports
truecolor, but if $COLORTERM is set to any other value we still query
the terminal.
The `rgb` flag of the UI struct is a boolean which only indicates
whether the UI supports truecolor, but does not have a 3rd state that we
can use to represent "we don't know if the UI supports truecolor". We
currently use `rgb=false` to represent this "we don't know" state, and
we use XTGETTCAP and DECRQSS queries to determine at runtime if the
terminal supports truecolor. However, if $COLORTERM is set to a value
besides "truecolor" or "24bit" (e.g. "256" or "16) that is a clear
indication that the terminal _does not_ support truecolor, so it is
incorrect to treat `rgb=false` as "we don't know" in that case.
Instead, in the TUI process we only check for the terminfo capabilities.
This must be done in the TUI process because we do not have access to
this information in the core Neovim process when `_defaults.lua` runs.
If the TUI cannot determine truecolor support from terminfo alone, we
set `rgb=false` to indicate "we don't know if the terminal supports
truecolor yet, keep checking". When we get to `_defaults.lua`, we can
then check $COLORTERM and only query the terminal if it is unset.
This means that users can explicitly opt out of truecolor determination
by setting `COLORTERM=256` (or similar) in their environment.
Problem: Fix added in #28676 worked accidentally(used variables were
themselves uninitialized at this point during startup) and
does not always work.
Solution: Reset attributes when clearing regions during startup.
Problem: Invalidated regions that are flushed during startup are
cleared with unitialized "clear_attrs", which is perceived as
flickering.
Solution: Initialize "clear_attrs" with current terminal background color.
Specifically, functions that are run in the context of the test runner
are put in module `test/testutil.lua` while the functions that are run
in the context of the test session are put in
`test/functional/testnvim.lua`.
Closes https://github.com/neovim/neovim/issues/27004.
With "intermediate" flag, only using minimal timeout is too short and
may lead to failures.
Also remove the fallback timeout in screen:expect_unchanged(), as having
a different fallback timeout than screen:expect() is confusing.
Problem: `tui->space_buf` may be smaller than the width of the TUI or widest
grid, causing an overflow when calling `tui_grid_clear` if `print_spaces` is
called from `clear_region` (clears the TUI's screen since #23428).
Solution: resize `space_buf` to be wide enough to fit the TUI or widest grid.
Didn't bother shrinking the allocation if the max widths decrease.
Problems:
1. The test case for querying truecolor support did not check which
capabilities were queried
2. The test case for querying truecolor support checked `&termguicolors`
in the Nvim test runner, not the child Nvim in the the embedded
terminal
3. The test case for querying truecolor support did not actually respond
to the XTGETTCAP requests. `'termguicolors'` is still enabled even
without responding to this query because libvterm understands and
responds to the DECRQSS request, but it is still good to respond to
the query explicitly instead of depending on hidden libvterm behavior
4. No test case exists at all for OSC 52
Solution:
Fix all of the problems listed above.
Extmarks can contain URLs which can then be drawn in any supporting UI.
In the TUI, for example, URLs are "drawn" by emitting the OSC 8 control
sequence to the TTY. On terminals which support the OSC 8 sequence this
will create clickable hyperlinks.
URLs are treated as inline highlights in the decoration subsystem, so
are included in the `DecorSignHighlight` structure. However, unlike
other inline highlights they use allocated memory which must be freed,
so they set the `ext` flag in `DecorInline` so that their lifetimes are
managed along with other allocated memory like virtual text.
The decoration subsystem then adds the URLs as a new highlight
attribute. The highlight subsystem maintains a set of unique URLs to
avoid duplicating allocations for the same string. To attach a URL to an
existing highlight attribute we call `hl_add_url` which finds the URL in
the set (allocating and adding it if it does not exist) and sets the
`url` highlight attribute to the index of the URL in the set (using an
index helps keep the size of the `HlAttrs` struct small).
This has the potential to lead to an increase in highlight attributes
if a URL is used over a range that contains many different highlight
attributes, because now each existing attribute must be combined with
the URL. In practice, however, URLs typically span a range containing a
single highlight (e.g. link text in Markdown), so this is likely just a
pathological edge case.
When a new highlight attribute is defined with a URL it is copied to all
attached UIs with the `hl_attr_define` UI event. The TUI manages its own
set of URLs (just like the highlight subsystem) to minimize allocations.
The TUI keeps track of which URL is "active" for the cell it is
printing. If no URL is active and a cell containing a URL is printed,
the opening OSC 8 sequence is emitted and that URL becomes the actively
tracked URL. If the cursor is moved while in the middle of a URL span,
we emit the terminating OSC sequence to prevent the hyperlink from
spanning multiple lines.
This does not support nested hyperlinks, but that is a rare (and,
frankly, bizarre) use case. If a valid use case for nested hyperlinks
ever presents itself we can address that issue then.
The motivation for this update is Issue #15365, where background=light
is not properly set for Nvim running from an Nvim :terminal. This can be
encountered when e.g., opening a terminal to make git commits, which
opens EDITOR=nvim in the nested terminal.
Under the implementation of this commit, the OSC response always
indicates a black or white foreground/background. While this may not
reflect the actual foreground/background color, it permits 'background'
to be retained for a nested Nvim instance running in the terminal
emulator. The behaviour matches Vim.
This commit implements a new TermRequest autocommand event and has Neovim
emit this event when children of terminal buffers emit an OSC or DCS sequence
libvterm does not handle.
The TermRequest autocommand event has additional data in the
v:termrequest variable.
Co-authored-by: Gregory Anders <greg@gpanders.com>
Since execl() is a variadic function, it requries a NULL-terminal to
indicate the end of its argument list, c.f. exec(3)
> The first argument, by convention, should point to the filename
> associated with the file being executed. The list of arguments *must*
> be terminated by a null pointer
This fixes the failure seen on aarch64 and i386, due to garbage data
being considered part of the variadic arguments.
This is the command invoked repeatedly to make the changes:
:%s/^\(.*\)|\%(\*\(\d\+\)\)\?$\n\1|\%(\*\(\d\+\)\)\?$/\=submatch(1)..'|*'..(max([str2nr(submatch(2)),1])+max([str2nr(submatch(3)),1]))/g
Rather than writing the synchronized update begin and end sequences into
the TUI's internal buffer (where it is later flushed to the TTY), write
these sequences directly to the TTY before and after the TUI's internal
buffer is itself flushed to the TTY.
This guarantees that a synchronized update is always used when we are
actually sending data to the TTY. This means we do not need to keep
track of the TUI's "dirty" state (any sequences which affect the TUI
state will be written in the TUI's internal buffer, which is now
guaranteed to only ever be written when a synchronized update is
active).
Currently, the value of $COLORTERM in :terminal in tests depends on
outer environment because of 'notermguicolors'.
If $COLORTERM is not set in :terminal, an inner Nvim instance will try
to detect 'termguicolors' support, which may interfere with tests.
So set 'termguicolors' in outer Nvim instance unless $COLORTERM needs to
be overridden, and unset it in inner Nvim instance when running TUI.
Set 'notermguicolors' in tests which spawn a child Nvim process to force
existing tests to use 16 colors. Also refactor the child process
invocation to make things a little bit less messy.
Problem: Default color scheme is suboptimal.
Solution: Start using new color scheme. Introduce new `vim` color scheme
for opt-in backward compatibility.
------
Main design ideas
- Be "Neovim branded".
- Be minimal for 256 colors with a bit more shades for true colors.
- Be accessible through high enough contrast ratios.
- Be suitable for dark and light backgrounds via exchange of dark and
light palettes.
------
Palettes
- Have dark and light variants. Implemented through exporeted
`NvimDark*` and `NvimLight*` hex colors.
- Palettes have 4 shades of grey for UI elements and 6 colors (red,
yellow, green, cyan, blue, magenta).
- Actual values are computed procedurally in Oklch color space based on
a handful of hyperparameters.
- Each color has a 256 colors variant with perceptually closest color.
------
Highlight groups
Use:
- Grey shades for general UI according to their design.
- Bold text for keywords (`Statement` highlight group). This is an
important choice to increase accessibility for people with color
deficiencies, as it doesn't rely on actual color.
- Green for strings, `DiffAdd` (as background), `DiagnosticOk`, and some
minor text UI elements.
- Cyan as main syntax color, i.e. for function usage (`Function`
highlight group), `DiffText`, `DiagnosticInfo`, and some minor text UI
elements.
- Red to generally mean high user attention, i.e. errors; in particular
for `ErrorMsg`, `DiffDelete`, `DiagnosticError`.
- Yellow very sparingly only with true colors to mean mild user
attention, i.e. warnings. That is, `DiagnosticWarn` and `WarningMsg`.
- Blue very sparingly only with true colors as `DiagnosticHint` and some
additional important syntax group (like `Identifier`).
- Magenta very carefully (if at all).
------
Notes
- To make tests work without relatively larege updates, each one is
prepended with an equivalent of the call `:colorscheme vim`.
Plus some tests which spawn new Neovim instances also now use 'vim'
color scheme.
In some cases tests are updated to fit new default color scheme.
Some escape sequences (in particular, OSC 52 paste responses) can be
very large, even unbounded in length. These can easily overflow
termkey's internal buffer. In order to process these long sequences,
dynamically grow termkey's internal buffer.
