Problem:
POSIX-compatible Ex-mode requires special-cases all over the codebase to
match various quirks that don't actually matter to users.
- The main utility of *interactive* Ex-mode is its REPL behavior, and
that can be achieved with `cmdwin`, which also gains extra UX
benefits.
- The main utility of *non-interactive* `nvim -es` is for shell
scripting, where Ex-mode quirks are mostly unhelpful (e.g. the
"Entering Ex mode" message).
Solution:
- Reimplement *interactive* Ex-mode as a "persistent, insert-mode
cmdwin" in Lua.
- "nvim -e/-E" is simply an alias to "gQ".
- Reframe *non-interactive* Ex-mode (`nvim -es`) as "script mode".
- Drop POSIX Ex-mode quirks.
Improvements:
- "nvim -V1 -es" output ends with a final newline!
- "nvim -V1 -es" no longer shows the "Entering Ex mode" msg. (This was
pointless noise, unwanted for scripting purposes.)
- stdin is no longer typeahead. Scripts (":lua io.read()") can read
stdin as data.
- Empty line is a no-op: a stray blank line no longer moves the cursor
(deviates from POSIX ex "+1"), no longer exits 1 at EOF (E501).
Preserved behavior:
- cursor starts at "$"
- mode()=="cv" (for non-interactive)
- multiline commands (:append/:function/heredoc pull continuation lines)
- bare-range print
- :print=>stdout
- -V1=>stderr
- CRLF input
- continue-after-error and exit codes
Dropped (regressed) POSIX behavior (non-interactive):
- Event loop only ticks while/between commands, not while blocked
waiting for a stdin line.
- ":g/pat/visual...Q"
- input()/getchar()/":s/x/y/c" no longer consume stdin lines as
answers: Nvim stops at end-of-input, skipping the rest of the script,
exit 0. Use ":lua io.read()" instead.
- If users care about this they should use interactive Ex-mode (`gQ`).
- ":@r" stops at end of the register instead of continuing to read
cmdline input from stdin.
Problem:
The magic globals `it`, `describe`, etc., are more trouble than they are
worth.
- Hooking into `after_each` requires `getfenv()` hacks.
- They confuse luals/emmylua, because the top-level `.luarc.json` isn't
merged with `test/.luarc.json` (apparently a luals limitation?)
- They totally defeat discoverability because the user just has to
"know" about the various magic symbols.
So they harm DX, which means they serve no purpose at all.
Solution:
- Expose the test API from `testutil`, so tests can call `t.it()`,
`t.describe()`, etc., in the conventional way.
- Drop `getfenv()` hacks.
- Drop the `setfenv()` injection in `load_chunk`.
- Drop `test/_meta.lua`.
FAILED test/functional/terminal/cursor_spec.lua @ 419: :terminal cursor uses the correct attributes
test/functional/terminal/cursor_spec.lua:448: Expected objects to be the same.
Passed in:
(string) 'block'
Expected:
(string) 'vertical'
stack traceback:
test/functional/terminal/cursor_spec.lua:448: in function <test/functional/terminal/cursor_spec.lua:419>
Problem:
When MsgArea highlight is changed, the next message may flash
and disappear because msg_start() renders with stale highlight attributes.
msg_puts_len() uses HL_ATTR(HLF_MSG) to render message text, which
happens before update_screen() calls highlight_changed().
So the message is rendered with outdated attrs.
Solution:
Call highlight_changed() in msg_start().
Problem: in terminal mode, adjust_topline moves curwin's cursor to the last row
so set_topline tails the non-scrollback area. This may result in the observed
cursor position remaining tailed in events within the focused terminal, rather
than reflecting the actual cursor position.
Solution: use the focused terminal's actual cursor position immediately, rather
than relying on the next terminal_check_cursor call in terminal_check to set it.
Note: Maybe also possible for terminal mode cursor position to be stale
(reporting the normal mode position) when switching buffers in events to another
terminal until the next terminal_check call? (or until refresh_terminal is
called for it) Maybe not worth fixing that, though.
Problem:
Test often fails on cirrus CI (freebsd):
buffer cursor position is correct in terminal with number column in a line with single-cell multibyte chars and no trailing spaces, before_each
test/functional/terminal/cursor_spec.lua:805: Row 5 did not match.
Expected:
|{7: 1 } |
|{7: 2 } |
|{7: 3 } |
|{7: 4 } |
|*{7: 5 }Entering Ex mode. Type "visual" to go to Normal mode. |
|{7: 6 }:^ |
|{3:-- TERMINAL --} |
Actual:
|{7: 1 } |
|{7: 2 } |
|{7: 3 } |
|{7: 4 } |
|*{7: 5 } |
|{7: 6 }:^ |
|{3:-- TERMINAL --} |
Solution:
Skip it. Ex mode isn't that important.
Problem:
Default 'statusline' is implemented in C and not representable as
a statusline expression. This makes it hard for user configs/plugins to
extend it.
Solution:
- Change the default 'statusline' slightly to a statusline expression.
- Remove the C implementation.
Problem: terminal mode cursor refresh logic has too many edge cases where it
fails when events change curbuf.
Solution: change the logic. Introduce cursor_visible to TerminalState to more
reliably track if terminal mode has changed busy. Move visibility handling to
refresh_cursor and move its call in refresh_terminal to terminal_check to avoid
temporarily changed curbufs from influencing cursor state.
This has the effect of "debouncing" shape/visibility updates to once per
terminal state tick (with the final attributes taking effect, as expected). I
think this is OK, but as a result it may also be warranted to update when
redrawing during the same state tick (e.g: from events executing :redraw); this
can be added later, if wanted.
Also move previous tests to a more appropriate place.
Problem: after #32458, it may still be possible for `busy_start` UI events to be
emitted without matching `busy_stop`s in the terminal.
Solution: do `terminal_enter`'s cursor visibility check immediately after
setting/restoring State so it occurs before events. This ensures that if pending
escape sequences are processed while in `terminal_enter`, the cursor's initial
visibility is set before `is_focused` is checked by `term_settermprop`.
As a result, we can move the call to `showmode` back to where it was originally.
Problem: `showmode` in `terminal_enter` may cause `vpeekc` to process events,
which may handle pending escape sequences. If `CSI ? 25 l` is handled to hide
the cursor, it may remain hidden even after leaving terminal mode if both
`terminal_enter` and (indirectly) `showmode` call `ui_busy_start`, as there is
only one matching call to `ui_busy_stop` after leaving terminal mode.
Solution: let `terminal_enter` handle setting the initial visibility of the
cursor before calling `showmode`.
Closes#32456.
This simple solution assumes it isn't possible for e.g. `os_breakcheck` to be
called indirectly by something else before `terminal_enter` initially handles
cursor visibility and after it restores it, which I think is true.
Problem:
`termopen` has long been a superficial wrapper around `jobstart`, and
has no real purpose. Also, `vim.system` and `nvim_open_term` presumably
will replace all features of `jobstart` and `termopen`, so centralizing
the logic will help with that.
Solution:
- Introduce `eval/deprecated.c`, where all deprecated eval funcs will live.
- Introduce "term" flag of `jobstart`.
- Deprecate `termopen`.
When a terminal application running inside the terminal emulator sets
the cursor shape or blink status of the cursor, update the cursor in the
parent terminal to match.
This removes the "virtual cursor" that has been in use by the terminal
emulator since the beginning. The original rationale for using the
virtual cursor was to avoid having to support additional UI methods to
change the cursor color for other (non-TUI) UIs, instead relying on the
TermCursor and TermCursorNC highlight groups.
The TermCursor highlight group is now used in the default 'guicursor'
value, which has a new entry for Terminal mode. However, the
TermCursorNC highlight group is no longer supported: since terminal
windows now use the real cursor, when the window is not focused there is
no cursor displayed in the window at all, so there is nothing to
highlight. Users can still use the StatusLineTermNC highlight group to
differentiate non-focused terminal windows.
BREAKING CHANGE: The TermCursorNC highlight group is no longer supported.
Before calling "attach" a screen object is just a dummy container for
(row, col) values whose purpose is to be sent as part of the "attach"
function call anyway.
Just create the screen in an attached state directly. Keep the complete
(row, col, options) config together. It is still completely valid to
later detach and re-attach as needed, including to another session.
This module is generally used by any tests that need the full Nvim TUI
instead of `screen.lua`. Thus it should live in `functional/` instead of
in `functional/terminal/`.
Specifically, functions that are run in the context of the test runner
are put in module `test/testutil.lua` while the functions that are run
in the context of the test session are put in
`test/functional/testnvim.lua`.
Closes https://github.com/neovim/neovim/issues/27004.
Problem: Updating default color scheme produced some feedback.
Solution: Address the feedback.
Outline of the changes:
- Colors `Grey1` and `Grey2` are made a little bit more extreme (dark -
darker, light - lighter) to increase overall contrast.
- `gui` colors are treated as base with `cterm` colors falling back to
using 0-15 colors which come from terminal emulator.
- Update highlight group definition to not include attribute definition
if it is intended to staty uncolored.
- Tweak some specific highlight groups.
- Add a list of Neovim specific highlight groups which are now defined
differently in a breaking way.
- Minor tweaks in several other places related to default color scheme.
This is the command invoked repeatedly to make the changes:
:%s/^\(.*\)|\%(\*\(\d\+\)\)\?$\n\1|\%(\*\(\d\+\)\)\?$/\=submatch(1)..'|*'..(max([str2nr(submatch(2)),1])+max([str2nr(submatch(3)),1]))/g
Set 'notermguicolors' in tests which spawn a child Nvim process to force
existing tests to use 16 colors. Also refactor the child process
invocation to make things a little bit less messy.
Problem: Default color scheme is suboptimal.
Solution: Start using new color scheme. Introduce new `vim` color scheme
for opt-in backward compatibility.
------
Main design ideas
- Be "Neovim branded".
- Be minimal for 256 colors with a bit more shades for true colors.
- Be accessible through high enough contrast ratios.
- Be suitable for dark and light backgrounds via exchange of dark and
light palettes.
------
Palettes
- Have dark and light variants. Implemented through exporeted
`NvimDark*` and `NvimLight*` hex colors.
- Palettes have 4 shades of grey for UI elements and 6 colors (red,
yellow, green, cyan, blue, magenta).
- Actual values are computed procedurally in Oklch color space based on
a handful of hyperparameters.
- Each color has a 256 colors variant with perceptually closest color.
------
Highlight groups
Use:
- Grey shades for general UI according to their design.
- Bold text for keywords (`Statement` highlight group). This is an
important choice to increase accessibility for people with color
deficiencies, as it doesn't rely on actual color.
- Green for strings, `DiffAdd` (as background), `DiagnosticOk`, and some
minor text UI elements.
- Cyan as main syntax color, i.e. for function usage (`Function`
highlight group), `DiffText`, `DiagnosticInfo`, and some minor text UI
elements.
- Red to generally mean high user attention, i.e. errors; in particular
for `ErrorMsg`, `DiffDelete`, `DiagnosticError`.
- Yellow very sparingly only with true colors to mean mild user
attention, i.e. warnings. That is, `DiagnosticWarn` and `WarningMsg`.
- Blue very sparingly only with true colors as `DiagnosticHint` and some
additional important syntax group (like `Identifier`).
- Magenta very carefully (if at all).
------
Notes
- To make tests work without relatively larege updates, each one is
prepended with an equivalent of the call `:colorscheme vim`.
Plus some tests which spawn new Neovim instances also now use 'vim'
color scheme.
In some cases tests are updated to fit new default color scheme.
Avoid consecutive RPC requests involving :startinsert or :stopinsert,
because consecutive RPC requests may be processed together, before the
:startinsert or :stopinsert takes effect.
Also change some feed_command() to command() to make tests faster.
Extend the capabilities of is_os to detect more platforms such as
freebsd and openbsd. Also remove `iswin()` helper function as it can be
replaced by `is_os("win")`.
This is essentially a convenience wrapper around the `pending()`
function, similar to `skip_fragile()` but more general-purpose.
Also remove `pending_win32` function as it can be replaced by
`skip(iswin())`.
Remove the command('qall!') from mksession_spec.lua because it prevents
helpers.rmdir() from retrying.
Allow extra trailing spaces when matching terminal lines.
It looks like Neovim has a bug: if `startinsert` is called using `command()`
then `-- TERMINAL --` gets replaced with `-- --` (and also a cursor appears).
