Problem: in terminal mode, adjust_topline moves curwin's cursor to the last row
so set_topline tails the non-scrollback area. This may result in the observed
cursor position remaining tailed in events within the focused terminal, rather
than reflecting the actual cursor position.
Solution: use the focused terminal's actual cursor position immediately, rather
than relying on the next terminal_check_cursor call in terminal_check to set it.
Note: Maybe also possible for terminal mode cursor position to be stale
(reporting the normal mode position) when switching buffers in events to another
terminal until the next terminal_check call? (or until refresh_terminal is
called for it) Maybe not worth fixing that, though.
Problem:
Test often fails on cirrus CI (freebsd):
buffer cursor position is correct in terminal with number column in a line with single-cell multibyte chars and no trailing spaces, before_each
test/functional/terminal/cursor_spec.lua:805: Row 5 did not match.
Expected:
|{7: 1 } |
|{7: 2 } |
|{7: 3 } |
|{7: 4 } |
|*{7: 5 }Entering Ex mode. Type "visual" to go to Normal mode. |
|{7: 6 }:^ |
|{3:-- TERMINAL --} |
Actual:
|{7: 1 } |
|{7: 2 } |
|{7: 3 } |
|{7: 4 } |
|*{7: 5 } |
|{7: 6 }:^ |
|{3:-- TERMINAL --} |
Solution:
Skip it. Ex mode isn't that important.
Problem:
Default 'statusline' is implemented in C and not representable as
a statusline expression. This makes it hard for user configs/plugins to
extend it.
Solution:
- Change the default 'statusline' slightly to a statusline expression.
- Remove the C implementation.
Problem: terminal mode cursor refresh logic has too many edge cases where it
fails when events change curbuf.
Solution: change the logic. Introduce cursor_visible to TerminalState to more
reliably track if terminal mode has changed busy. Move visibility handling to
refresh_cursor and move its call in refresh_terminal to terminal_check to avoid
temporarily changed curbufs from influencing cursor state.
This has the effect of "debouncing" shape/visibility updates to once per
terminal state tick (with the final attributes taking effect, as expected). I
think this is OK, but as a result it may also be warranted to update when
redrawing during the same state tick (e.g: from events executing :redraw); this
can be added later, if wanted.
Also move previous tests to a more appropriate place.
Problem: after #32458, it may still be possible for `busy_start` UI events to be
emitted without matching `busy_stop`s in the terminal.
Solution: do `terminal_enter`'s cursor visibility check immediately after
setting/restoring State so it occurs before events. This ensures that if pending
escape sequences are processed while in `terminal_enter`, the cursor's initial
visibility is set before `is_focused` is checked by `term_settermprop`.
As a result, we can move the call to `showmode` back to where it was originally.
Problem: `showmode` in `terminal_enter` may cause `vpeekc` to process events,
which may handle pending escape sequences. If `CSI ? 25 l` is handled to hide
the cursor, it may remain hidden even after leaving terminal mode if both
`terminal_enter` and (indirectly) `showmode` call `ui_busy_start`, as there is
only one matching call to `ui_busy_stop` after leaving terminal mode.
Solution: let `terminal_enter` handle setting the initial visibility of the
cursor before calling `showmode`.
Closes#32456.
This simple solution assumes it isn't possible for e.g. `os_breakcheck` to be
called indirectly by something else before `terminal_enter` initially handles
cursor visibility and after it restores it, which I think is true.
Problem:
`termopen` has long been a superficial wrapper around `jobstart`, and
has no real purpose. Also, `vim.system` and `nvim_open_term` presumably
will replace all features of `jobstart` and `termopen`, so centralizing
the logic will help with that.
Solution:
- Introduce `eval/deprecated.c`, where all deprecated eval funcs will live.
- Introduce "term" flag of `jobstart`.
- Deprecate `termopen`.
When a terminal application running inside the terminal emulator sets
the cursor shape or blink status of the cursor, update the cursor in the
parent terminal to match.
This removes the "virtual cursor" that has been in use by the terminal
emulator since the beginning. The original rationale for using the
virtual cursor was to avoid having to support additional UI methods to
change the cursor color for other (non-TUI) UIs, instead relying on the
TermCursor and TermCursorNC highlight groups.
The TermCursor highlight group is now used in the default 'guicursor'
value, which has a new entry for Terminal mode. However, the
TermCursorNC highlight group is no longer supported: since terminal
windows now use the real cursor, when the window is not focused there is
no cursor displayed in the window at all, so there is nothing to
highlight. Users can still use the StatusLineTermNC highlight group to
differentiate non-focused terminal windows.
BREAKING CHANGE: The TermCursorNC highlight group is no longer supported.
Before calling "attach" a screen object is just a dummy container for
(row, col) values whose purpose is to be sent as part of the "attach"
function call anyway.
Just create the screen in an attached state directly. Keep the complete
(row, col, options) config together. It is still completely valid to
later detach and re-attach as needed, including to another session.
This module is generally used by any tests that need the full Nvim TUI
instead of `screen.lua`. Thus it should live in `functional/` instead of
in `functional/terminal/`.
Specifically, functions that are run in the context of the test runner
are put in module `test/testutil.lua` while the functions that are run
in the context of the test session are put in
`test/functional/testnvim.lua`.
Closes https://github.com/neovim/neovim/issues/27004.
Problem: Updating default color scheme produced some feedback.
Solution: Address the feedback.
Outline of the changes:
- Colors `Grey1` and `Grey2` are made a little bit more extreme (dark -
darker, light - lighter) to increase overall contrast.
- `gui` colors are treated as base with `cterm` colors falling back to
using 0-15 colors which come from terminal emulator.
- Update highlight group definition to not include attribute definition
if it is intended to staty uncolored.
- Tweak some specific highlight groups.
- Add a list of Neovim specific highlight groups which are now defined
differently in a breaking way.
- Minor tweaks in several other places related to default color scheme.
This is the command invoked repeatedly to make the changes:
:%s/^\(.*\)|\%(\*\(\d\+\)\)\?$\n\1|\%(\*\(\d\+\)\)\?$/\=submatch(1)..'|*'..(max([str2nr(submatch(2)),1])+max([str2nr(submatch(3)),1]))/g
Set 'notermguicolors' in tests which spawn a child Nvim process to force
existing tests to use 16 colors. Also refactor the child process
invocation to make things a little bit less messy.
Problem: Default color scheme is suboptimal.
Solution: Start using new color scheme. Introduce new `vim` color scheme
for opt-in backward compatibility.
------
Main design ideas
- Be "Neovim branded".
- Be minimal for 256 colors with a bit more shades for true colors.
- Be accessible through high enough contrast ratios.
- Be suitable for dark and light backgrounds via exchange of dark and
light palettes.
------
Palettes
- Have dark and light variants. Implemented through exporeted
`NvimDark*` and `NvimLight*` hex colors.
- Palettes have 4 shades of grey for UI elements and 6 colors (red,
yellow, green, cyan, blue, magenta).
- Actual values are computed procedurally in Oklch color space based on
a handful of hyperparameters.
- Each color has a 256 colors variant with perceptually closest color.
------
Highlight groups
Use:
- Grey shades for general UI according to their design.
- Bold text for keywords (`Statement` highlight group). This is an
important choice to increase accessibility for people with color
deficiencies, as it doesn't rely on actual color.
- Green for strings, `DiffAdd` (as background), `DiagnosticOk`, and some
minor text UI elements.
- Cyan as main syntax color, i.e. for function usage (`Function`
highlight group), `DiffText`, `DiagnosticInfo`, and some minor text UI
elements.
- Red to generally mean high user attention, i.e. errors; in particular
for `ErrorMsg`, `DiffDelete`, `DiagnosticError`.
- Yellow very sparingly only with true colors to mean mild user
attention, i.e. warnings. That is, `DiagnosticWarn` and `WarningMsg`.
- Blue very sparingly only with true colors as `DiagnosticHint` and some
additional important syntax group (like `Identifier`).
- Magenta very carefully (if at all).
------
Notes
- To make tests work without relatively larege updates, each one is
prepended with an equivalent of the call `:colorscheme vim`.
Plus some tests which spawn new Neovim instances also now use 'vim'
color scheme.
In some cases tests are updated to fit new default color scheme.
Avoid consecutive RPC requests involving :startinsert or :stopinsert,
because consecutive RPC requests may be processed together, before the
:startinsert or :stopinsert takes effect.
Also change some feed_command() to command() to make tests faster.
Extend the capabilities of is_os to detect more platforms such as
freebsd and openbsd. Also remove `iswin()` helper function as it can be
replaced by `is_os("win")`.
This is essentially a convenience wrapper around the `pending()`
function, similar to `skip_fragile()` but more general-purpose.
Also remove `pending_win32` function as it can be replaced by
`skip(iswin())`.
Remove the command('qall!') from mksession_spec.lua because it prevents
helpers.rmdir() from retrying.
Allow extra trailing spaces when matching terminal lines.
It looks like Neovim has a bug: if `startinsert` is called using `command()`
then `-- TERMINAL --` gets replaced with `-- --` (and also a cursor appears).
Hope this will make people using feed_command less likely: this hides bugs.
Already found at least two:
1. msgpackparse() will show internal error: hash_add() in case of duplicate
keys, though it will still work correctly. Currently silenced.
2. ttimeoutlen was spelled incorrectly, resulting in option not being set when
expected. Test was still functioning somehow though. Currently fixed.
Removed the call to validate_cursor() because mb_check_adjust_col() is
already called in adjust_topline().
Closes#6378
References #6203https://s3.amazonaws.com/archive.travis-ci.org/jobs/215498258/log.txt
[ ERROR ] ...ovim/neovim/test/functional/terminal/scrollback_spec.lua @ 386: 'scrollback' option set to 0 behaves as 1 (10621.17 ms)
==================== File /home/travis/build/neovim/neovim/build/log/ubsan.12836 ====================
= =================================================================
= ==12836==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62100002cd00 at pc 0x000000eafe90 bp 0x7ffc8661fe50 sp 0x7ffc8661fe48
= READ of size 1 at 0x62100002cd00 thread T0
= #0 0xeafe8f in utf_head_off /home/travis/build/neovim/neovim/src/nvim/mbyte.c:1457:7
= #1 0x6b890e in getvcol /home/travis/build/neovim/neovim/src/nvim/charset.c:1169:15
= #2 0x6bc777 in getvvcol /home/travis/build/neovim/neovim/src/nvim/charset.c:1336:5
= #3 0xfc067b in curs_columns /home/travis/build/neovim/neovim/src/nvim/move.c:730:5
= #4 0xfbc8db in validate_cursor /home/travis/build/neovim/neovim/src/nvim/move.c:510:5
= #5 0x14479ed in setcursor /home/travis/build/neovim/neovim/src/nvim/screen.c:6363:5
= #6 0x17fe054 in redraw /home/travis/build/neovim/neovim/src/nvim/terminal.c:1175:5
= #7 0x17f95e4 in terminal_enter /home/travis/build/neovim/neovim/src/nvim/terminal.c:392:3
= #8 0x70eb2b in edit /home/travis/build/neovim/neovim/src/nvim/edit.c:1300:7
= #9 0x11097d1 in normal_finish_command /home/travis/build/neovim/neovim/src/nvim/normal.c:947:13
= #10 0x1081191 in normal_execute /home/travis/build/neovim/neovim/src/nvim/normal.c:1138:3
= #11 0x170b813 in state_enter /home/travis/build/neovim/neovim/src/nvim/state.c:58:26
= #12 0x103631b in normal_enter /home/travis/build/neovim/neovim/src/nvim/normal.c:464:3
= #13 0xdfb7a8 in main /home/travis/build/neovim/neovim/src/nvim/main.c:552:3
= #14 0x2b8a3c85bf44 in __libc_start_main /build/eglibc-MjiXCM/eglibc-2.19/csu/libc-start.c:287
= #15 0x447b25 in _start (/home/travis/build/neovim/neovim/build/bin/nvim+0x447b25)
=
= 0x62100002cd00 is located 0 bytes to the right of 4096-byte region [0x62100002bd00,0x62100002cd00)
= allocated by thread T0 here:
= #0 0x4f1e98 in malloc (/home/travis/build/neovim/neovim/build/bin/nvim+0x4f1e98)
= #1 0xf28774 in try_malloc /home/travis/build/neovim/neovim/src/nvim/memory.c:84:15
= #2 0xf28934 in xmalloc /home/travis/build/neovim/neovim/src/nvim/memory.c:118:15
= #3 0xec7be8 in mf_alloc_bhdr /home/travis/build/neovim/neovim/src/nvim/memfile.c:646:17
= #4 0xec58d4 in mf_new /home/travis/build/neovim/neovim/src/nvim/memfile.c:297:12
= #5 0xeda8a8 in ml_new_data /home/travis/build/neovim/neovim/src/nvim/memline.c:2697:16
= #6 0xed7beb in ml_open /home/travis/build/neovim/neovim/src/nvim/memline.c:349:8
= #7 0x643fcd in open_buffer /home/travis/build/neovim/neovim/src/nvim/buffer.c:109:7
= #8 0xa7038c in do_ecmd /home/travis/build/neovim/neovim/src/nvim/ex_cmds.c:2483:24
= #9 0xb5bb49 in do_exedit /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:6839:9
= #10 0xb7b6d8 in ex_edit /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:6767:3
= #11 0xb2a598 in do_one_cmd /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:2208:5
= #12 0xb08f47 in do_cmdline /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:602:20
= #13 0x109997b in nv_colon /home/travis/build/neovim/neovim/src/nvim/normal.c:4492:18
= #14 0x1081188 in normal_execute /home/travis/build/neovim/neovim/src/nvim/normal.c:1135:3
= #15 0x170b813 in state_enter /home/travis/build/neovim/neovim/src/nvim/state.c:58:26
= #16 0x103631b in normal_enter /home/travis/build/neovim/neovim/src/nvim/normal.c:464:3
= #17 0xdfb7a8 in main /home/travis/build/neovim/neovim/src/nvim/main.c:552:3
= #18 0x2b8a3c85bf44 in __libc_start_main /build/eglibc-MjiXCM/eglibc-2.19/csu/libc-start.c:287
=
= SUMMARY: AddressSanitizer: heap-buffer-overflow /home/travis/build/neovim/neovim/src/nvim/mbyte.c:1457:7 in utf_head_off
= Shadow bytes around the buggy address:
= 0x0c427fffd950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= =>0x0c427fffd9a0:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= Shadow byte legend (one shadow byte represents 8 application bytes):
= Addressable: 00
= Partially addressable: 01 02 03 04 05 06 07
= Heap left redzone: fa
= Heap right redzone: fb
= Freed heap region: fd
= Stack left redzone: f1
= Stack mid redzone: f2
= Stack right redzone: f3
= Stack partial redzone: f4
= Stack after return: f5
= Stack use after scope: f8
= Global redzone: f9
= Global init order: f6
= Poisoned by user: f7
= Container overflow: fc
= Array cookie: ac
= Intra object redzone: bb
= ASan internal: fe
= Left alloca redzone: ca
= Right alloca redzone: cb
= ==12836==ABORTING
=====================================================================================================
./test/helpers.lua:82: assertion failed!
stack traceback:
./test/helpers.lua:82: in function 'check_logs'
./test/functional/helpers.lua:643: in function <./test/functional/helpers.lua:642>
Let the terminal dictate the normal-mode cursor position. This will be
disorienting sometimes, but it is closer to what users expect vs always
going to the last line.
It is otherwise impossible to determine which test failed sanitizer/valgrind
check. test/functional/helpers.lua module return was changed so that tests which
do not provide after_each function to get new check will automatically fail.
