Problem:
- Lua<=>API roundtrips
- Although we prefer Lua for most business-logic code, doing this
conversion in C makes sense in this case because:
1. setting options is a hot path
2. most of the options logic lives in C
3. the current arrangement is MORE verbose and requires MORE code
Solution:
Move conversion to a C util.
- nvim_set_option_value passes the raw Object (scalar, Array, or Dict)
to `object_as_optval_for()` which flattens it to the canonical `:set`
string and validates the type.
- drop `convert_value_to_vim`, eliminate its roundtrip.
Problem:
`nvim_set_option_value` cannot "update" options similar to `:set opt=`,
`:set opt+=`, etc. The Lua impls of "vim.opt" / "vim.o" have incomplete,
bespoke reimplementations of those operations.
ref #38420
Solution:
- Add `operation` param to `nvim_set_option_value`, which may be "set",
"append", "prepend", or "remove".
- Use this feature to implement `vim.opt` / `vim.o`.
ArrayOf metadata existed before the LuaCATS type documentation refactor and is
useful to typed clients consuming api_info().
Keep Tuple metadata normalized to Array, since tuple element types can be mixed,
but preserve ArrayOf(...) for exported API metadata.
Fixes#38734
AI-assisted: Codex
Problem:
cmdwin (the `:q` cmdline buffer) has various limitations which require
special-casing all over the codebase.
Besides complicating the code, it also breaks async plugins if they try
to create buffers/windows after some work is done, if the user happens
to open cmdwin at the wrong the moment:
Lua callback: …/guh.nvim/lua/guh/util.lua:531:
E11: Invalid in command-line window; <CR> executes, CTRL-C quits
stack traceback:
[C]: in function 'nvim_buf_delete'
…/guh.nvim/lua/guh/util.lua:531: in function <…/guh.nvim/lua/guh/util.lua:526>
Solution:
Just say no to "inception". Reimplement cmdwin as a normal buffer+window.
All of the cmdwin contortions (in both core, and innocent plugins) exist
literally only to support "inception": recursive
cmdwin-in-cmdline-things, like `<c-r>=`, `/`, search-during-substitute,
`:input()`, etc. So we just won't support that (though I have
a potential plan for that later, which I call "modal parking lot").
The benefit is that plugins, and core, no longer have to care about
cmdwin.
BONUS:
- mouse-drag on vertical separators works (it only worked for
horizontal/statusline before)
- inccommand-in-cmdwin now works correctly, for free (thus don't need
#40077).
POTENTIAL FOLLOWUPS
- Drop `CHECK_CMDWIN` ("E11: Invalid in command-line window"), allow chaos.
- Unify `BUFLOCK_OK` / `LOCK_OK` ?
DESIGN:
- Eliminate lots of C globals, `EX_CMDWIN`, etc.
- `text_locked()` no longer reports true for cmdwin.
- cmdwin = a normal window with 'winfixbuf', 'bufhidden=wipe',
'buftype=nofile'. Invariants come from those options rather than
special cases throughout the codebase.
- `nv_record` for q:/q//q? calls Lua
`nlua_call_vimfn("vim._core.cmdwin", …)`. No `K_CMDWIN`
/ cmdline-reader detour.
- `cedit_key` (`c_CTRL-F`) schedules a deferred event that calls
`vim._core.cmdwin.open(type, content, pos)` and returns `Ctrl_C` so
the in-flight cmdline cancels. Reader state is not serialized; instead
the captured `(type, line, col)` is replayed via
`nvim_feedkeys(type..line.."<CR>", "nt", …)` after user confirms.
- On confirm/cancel: `<CR>` / `<C-C>` calls into Lua which closes the
window and re-feeds the cmdline.
BREAKING CHANGES:
- Expression-register cmdline (`<C-R>=` from insert-mode) no longer
supports cmdwin. Same applies to `input()` / `inputlist()` (already
covered by `text_locked`).
- Usage of cmdwin in macros/mappings will probably break (assuming they
ever worked).
Problem: Unecessary/unexpected cleanup happens when restoring the
current window for redraw purposes (since 28ffb334).
Solution: Don't call autocmd_restbuf() to restore the current window,
just do so directly.
Problem: Attach-time terminal probes cannot distinguish responses from
different attached UIs.
Solution: Identify the UI by RPC channel id in `TermResponse` and make
`vim.tty.request()` filter responses by channel.
Problem: Terminal background and truecolor detection runs only at startup,
gated on a UI being attached. A headless server has no UI then, so
`'background'` and `'termguicolors'` are never detected and remote
UIs ignore the terminal's theme.
Solution: Also (re)detect on UIEnter. The most recently attached terminal
wins; an explicit user value is preserved.
Problem: a font set via nvim_set_hl is lost or corrupted with font-only groups,
attribute combining, and update=true, and is dropped on any attr-table rebuild.
nvim__inspect_cell also frees the font name while a returned dict still borrows it.
Solution: register font-only groups, carry font through hl_combine_attr, inherit
it on update, and persist sg_font so a rebuild can restore it. Keep interned font
names across a rebuild instead of clearing them. Add the missing font field to
get_hl_info.
Co-authored-by: Ryan Patterson <cgamesplay@cgamesplay.com>
Problem:
stdpath() may return a DOS 8.3 "shortened" filename, because Windows
truncates long usernames into `6ch~N` names in `TEMP/TMP` env vars.
We don't want to "leak" them into Nvim.
Solution:
For "run", pass `true` to `vim_FullName` to expand 8.3 filenames.
For "cache", call `os_realpath` to expand 8.3 filenames.
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem:
stdpath() may return a DOS 8.3 "shortened" filename, because Windows
truncates some long usernames into `6ch~N` names. Then features such
as `nvim_get_runtime_file` fail to find the file.
Analysis:
When expanding an 8.3 filename path like `C:/Users/ADMINI~1/AppData/*`,
we treat `~` as a special character and first check whether a directory
named `ADMINI~1` exists under `Users`. Since no such directory actually
exists, the expansion fails.
Solution:
Treat `~` as a literal character in `do_path_expand`. Since the `~/`
case is already handled in `gen_expand_wildcards`, any remaining `~` is
just a literal character and will later be escaped to `\~` by
`file_pat_to_reg_pat` if needed.
Problem: Extmark has support for horizontal scrolling and truncating, but not wrapping.
Solution: Extend virt_lines_overflow flags to support "wrap" and "auto" based on proposed changes in #18282.
Problem: `nvim_exec_autocmds({ buf = ... })` matches the target buffer, but callbacks and modelines run with the caller buffer current rather than the target buffer.
Solution: Execute the buffered path in prepared target-buffer context and restore the caller afterward.
Problem:
Visual selection could end up in the wrong place after
nvim_buf_set_text or nvim_buf_set_lines. In some delete cases,
Visual.lnum was already clamped before the line shift happened, so the
adjustment got skipped.
Solution:
Split fix_cursor_cols into reusable fix_pos_col logic and reuse it
for Visual updates. Also adjust Visual.lnum before changed_lines so
the shift uses the original position before final clamping.
Problem:
2d795face6 added support for tab-local options ('cmdheight')
to `nvim_get_option_value`, but not to:
nvim_get_option_info2()
nvim_set_option_value(…, { tab = … })
gettabwinvar()
Solution:
- Update `options.lua` to model tab-local options. Introduce `kOptScopeTab`.
- Handle tab scope in the options layer so it works for all options APIs.
Note:
- No change to `gettabvar()`. Not sure if needed/wanted.
fix https://github.com/neovim/neovim/issues/31140
Problem:
API clients cannot query the tab-local value of 'cmdheight'.
Solution:
Allow nvim_get_option_value() to accept { tab = <tab-ID> } for 'cmdheight'.
Problem:
The Lua test harness still ran through standalone -ll mode, so tests
depended on the low-level Lua path instead of the regular Nvim Lua
environment. That also meant os.exit() coverage had to carry an ASAN
workaround because Lua's raw process exit skipped Nvim teardown and let
LeakSanitizer interfere with the observed exit code.
Solution:
Run the harness and related fixtures with nvim -l. Patch os.exit() in
the main Lua state to exit through getout(), so scripts observe normal
Nvim shutdown while standalone -ll remains available for generator-style
scripts. As a consequence, the startup test can assert os.exit() without
disabling leak detection.
AI-assisted: Codex
Problem: Message redirection column for captured output is not reset
after :echon since (4260f73, e63346df).
Solution: Ensure msg_ext_append is set before the kind with :echon.
Problem: Internal progress messages use the "nvim" source (since
ff68fd6b), plugins shouldn't be allowed to set the progress
message source to "nvim". The message ID used for internal
progress messages is not identifiable as such.
Solution: Disallow setting opts->source to "nvim" with nvim_echo().
Refactor msg_progress() and callees to bypass nvim_echo().
Prepend message id for internal progress messages with "nvim.".
Problem:
Can't get a command's description from nvim_get_commands when
cmd is string.
Solution:
Returns "desc" field in nvim_get_commands.
`definition` is now empty when cmd is function type.
Problem:
The `nvim_get_chan_info()` terminal channel test used `shell-test INTERACT` to verify that `jobstop()` reports an unhandled `SIGHUP` as exit code `129`.
`INTERACT` reads from stdin with `fgets()`, so closing the PTY could race with `SIGHUP` delivery. If `fgets()` observed EOF first, `shell-test` exited normally with code `0`, causing intermittent failures on slower sanitizer builds.
Solution:
Add a `shell-test HOLD` mode that prints a readiness prompt and then waits without reading stdin. Use it for the `SIGHUP` assertion so PTY EOF cannot make the helper exit normally before the signal path is observed.
Problem:
`gx` relies on `exepath` to get the fullpath of `cmd.exe`,
and that path must use `\`; otherwise, luv's spawn will fail.
Solution:
Revert `slash_adjust` in `exepath`, so that it still respects 'shellslash'
Problem:
After 55ceb31, z= and tselect don't work if `vim.ui.select` is an async
provider (especially terminal buffers).
Solution:
Drop the `vim.wait()` approach, use an async approach.
fix#39506
Problem:
Invalid `nvim_create_user_command` calls can leak the
`preview` callback reference after Neovim has taken ownership of it.
1. build with {a,l}san
2. run:
```sh
<path/to/nvim> --headless -u NONE --clean +'lua
for i = 1, 100 do
pcall(vim.api.nvim_create_user_command,
"some very epic stuff" .. i,
{}, -- NOTE: this is INVALID (not a function or string)
{ preview = function() end })
end
vim.cmd("qa!")
' +qa
```
3. see:
```
100 lua references were leaked!
```
Solution:
Clear `preview_luaref` in `err:`.
Problem: `nvim_set_keymap` leaks the `callback` `LuaRef` when the
LHS is too long.
Solution: Make `set_maparg_lhs_rhs` transfer `rhs_lua` to
`MapArguments` up front so the caller always owns the ref.
Problem:
On Windows, path separators may become inconsistent for various reasons,
which makes normalization quite painful.
Solution:
Normalize paths to `/` at the entry boundaries and always use it
internally, converting back only in rare cases where `\` is really
needed (e.g. cmd.exe/bat scripts?).
This is the first commit in a series of incremental steps.
Note:
* some funcs won't respect shellslash. e.g. `expand/fnamemodify`
* some funcs still respect shellslash, but will be updated in a follow
PR. e.g. `ex_pwd/f_chdir/f_getcwd`
* uv's built-in funcs always return `\`. e.g. `uv.cwd/uv.exepath`
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem: fg_indexed/bg_indexed were dropped from nvim_get_hl output due
to a wrong short_keys guard. HL_FG_INDEXED also wasn't cleared in
hl_blend_attrs, and HLATTRS_DICT_SIZE was too small.
Solution: Remove the short_keys guard, clear HL_FG_INDEXED in
hl_blend_attrs, bump HLATTRS_DICT_SIZE to 24, and clarify docs that
these flags mean rgb is an approximation of the cterm palette index.
Problem:
- Unable to "pin" a window to prevent closing without specifically
being targeted.
- :fclose closes hidden windows (even before visible windows).
Solution:
- Add 'winpinned' window-local option. When set, window is skipped by
:fclose and :only. Pin the ui2 cmdline window (which should always be
visible), so that it is not closed by :only/fclose.
- Skip over hidden (and pinned) windows with :fclose.
Co-authored-by: glepnir <glephunter@gmail.com>
Problem: Marks are not adjusted unloading a buffer that doesn't exist
on disk. E.g. extmarks are still valid (and will be beyond the
end of the buffer if the buffer is reloaded), even though the
text is lost.
Solution: Adjust marks for a cleared buffer when unloading a buffer that
doesn't exist on disk.
* fix(api): allow silencing "Too many highlight groups" error
Problem: Using Lua's `vim.api.nvim_set_hl(0, 'New', {...})` can fail if
there are too many existing highlight groups. However, this error can
not be silenced with `pcall`.
Solution: Make it possible to silence in `nvim_set_hl` and
`nvim_get_hl_id_by_name`.
* fix(lsp): limit number of groups created by `document_color()`
Problem: A file can contain many string colors that would be highlighted
by an LSP server. If this number crosses 19999 (maximum number of
allowed highlight groups), there are general issues with creating
other highlight groups, which can break functionality outside of
`vim.lsp.document_color`.
Solution: Limit number of highlight groups that are created by
`vim.lsp.document_color` to 10000 (half of allowed maximum).
This is not a 100% solution (since there can exist more than 10000
other highlight groups), but explicitly checking number of groups is
slow and 10000 should (hopefully) be enough for most use cases.
Problem: hlgroup2dict passes &ns_id to ns_get_hl twice. The first call
(link=true) sets *ns_hl = 0 when link_global is set, so the second call
and the sg_cleared guard both see ns_id == 0 and bail out. The group is
silently dropped from the result.
Solution: use a temporary copy of ns_id for each ns_get_hl call so the
original value is preserved.
Problem: Breaking a link with update=true loses colors inherited from
the linked group.
Solution: Copy color indices from the linked group so inherited colors
remain visible in :hi output.
Problem:
`:help dev-name-common` states that "buf" should be used instead of
"buffer" but there are cases where buffer is mentioned in the lua API.
Solution:
- Rename occurrences of "buffer" to "buf" for consistency with the
documentation.
- Support (but deprecate) "buffer" for backwards compatibility.
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem: nvim_clear_autocmds() does not type check "event" correctly, and also
treats an empty array "event" like nil.
Solution: fix type checking. Treat empty array "event" as a no-op, like
nvim_exec_autocmds(). Add some extra tests.
Likewise the nil handling change may be considered breaking if anyone
(unintentionally) relied on that. It was also true that integer, function, etc.
"event"s would also be treated like nil!
Note that an empty string "event" is still an error, as that's must be an exact
match on an event name.
Problem: nvim_exec_autocmds() documentation incorrectly describes the default
for "pattern" as *, when it's actually the current file name (like :doautocmd).
Solution: correct it. Add a test.
Problem: in autocmd APIs, a non-nil "pattern" containing only empty
'sub'-patterns is silently treated as nil, causing the fallback value to be
unexpectedly used instead.
Solution: for nvim_create_autocmd(), raise a validation error (as no autocmds
would be created). For nvim_{exec,clear}_autocmds(), make it a no-op (as
matching no autocmds is not an error).
Problem:
On exit, rpc_free() is called when processing main_loop.events after
libuv calls close callbacks of the channel's stream. However, when there
are no child processes, these libuv callbacks are called in loop_close()
instead of proc_teardown(), and main_loop.events isn't processed after
loop_close(). As a result, calling remote_ui_disconnect() in rpc_free()
causes UILeave to depend on the presence of child processes.
Solution:
Always call remote_ui_disconnect() in rpc_close_event(), and remove the
call in rpc_free().
Problem: setting one underline style to false clears bits belonging
to another style. `{underdouble = true, underdashed = false}` results
in undercurl because CHECK_FLAG_WITH_KEY does `m &= ~flag` which
doesn't work for multi-bit encoded values sharing HL_UNDERLINE_MASK.
Solution: use a local variable to derive the correct clear mask from
the flag. Clear the whole HL_UNDERLINE_MASK field instead of individual
bits, and only clear on false when the current style actually matches.
