Problem:
POSIX-compatible Ex-mode requires special-cases all over the codebase to
match various quirks that don't actually matter to users.
- The main utility of *interactive* Ex-mode is its REPL behavior, and
that can be achieved with `cmdwin`, which also gains extra UX
benefits.
- The main utility of *non-interactive* `nvim -es` is for shell
scripting, where Ex-mode quirks are mostly unhelpful (e.g. the
"Entering Ex mode" message).
Solution:
- Reimplement *interactive* Ex-mode as a "persistent, insert-mode
cmdwin" in Lua.
- "nvim -e/-E" is simply an alias to "gQ".
- Reframe *non-interactive* Ex-mode (`nvim -es`) as "script mode".
- Drop POSIX Ex-mode quirks.
Improvements:
- "nvim -V1 -es" output ends with a final newline!
- "nvim -V1 -es" no longer shows the "Entering Ex mode" msg. (This was
pointless noise, unwanted for scripting purposes.)
- stdin is no longer typeahead. Scripts (":lua io.read()") can read
stdin as data.
- Empty line is a no-op: a stray blank line no longer moves the cursor
(deviates from POSIX ex "+1"), no longer exits 1 at EOF (E501).
Preserved behavior:
- cursor starts at "$"
- mode()=="cv" (for non-interactive)
- multiline commands (:append/:function/heredoc pull continuation lines)
- bare-range print
- :print=>stdout
- -V1=>stderr
- CRLF input
- continue-after-error and exit codes
Dropped (regressed) POSIX behavior (non-interactive):
- Event loop only ticks while/between commands, not while blocked
waiting for a stdin line.
- ":g/pat/visual...Q"
- input()/getchar()/":s/x/y/c" no longer consume stdin lines as
answers: Nvim stops at end-of-input, skipping the rest of the script,
exit 0. Use ":lua io.read()" instead.
- If users care about this they should use interactive Ex-mode (`gQ`).
- ":@r" stops at end of the register instead of continuing to read
cmdline input from stdin.
Problem:
The magic globals `it`, `describe`, etc., are more trouble than they are
worth.
- Hooking into `after_each` requires `getfenv()` hacks.
- They confuse luals/emmylua, because the top-level `.luarc.json` isn't
merged with `test/.luarc.json` (apparently a luals limitation?)
- They totally defeat discoverability because the user just has to
"know" about the various magic symbols.
So they harm DX, which means they serve no purpose at all.
Solution:
- Expose the test API from `testutil`, so tests can call `t.it()`,
`t.describe()`, etc., in the conventional way.
- Drop `getfenv()` hacks.
- Drop the `setfenv()` injection in `load_chunk`.
- Drop `test/_meta.lua`.
Problem:
Continue separators normalization, and try to keep it at the
nvim <-> external boundary, e.g., `fn.xxx`, `api.xxx`, `:xxx`
Solution:
some key changes
- normalize `$HOME-windows`
- normalize entry points of changing directory
- `nvim_set_current_dir()`, `chdir()`
- normalize the named pipe
- `--listen` arg, `--server` arg, `:restart`
- `serverstart()`, `sockconnect()`, `serverstop()`
- make `expand()` respect 'shellslash' again
- clean up `did_set_shellslash`
- replace `forward_slash` with `TO_SLASH`
- remove obsolete `TMP_PATHSEPSTR`
- remove `slash_adjust` in `do_autocmd_dirchanged`?
- make `fnamemodify()` always return `/` (except when substituting
separators via `:s`, `:gs` ?)
Note:
- these funcs still apply `slash_adjust` on return, as before:
`:pwd`, `chdir()`, `exepath()`, `getcwd()`
- these funcs alwarys return `/`, unlike before the normalization PRs
- `getcompletion()`, `finddir()`, `findfile()`
Also
- clean up code, comments, formatting and tests
- move `TO_SLASH` from `f_chdir` to `changedir_func`
- move `TO_SLASH` from `f_bufadd` to `buflist_new`
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem:
Neovim can permanently stop accepting keyboard input after a large paste, or after
any sufficiently large input burst. The screen still redraws and honours window
resize, but typed keys have no effect and the session must be killed. A paste that
triggers the freeze is applied only partially.
Root cause:
`src/nvim/os/input.c` holds `input_buffer[INPUT_BUFFER_SIZE]` (16386 bytes),
compacted left rather than used as a ring:
- `input_get()` drains the buffer by advancing `input_read_pos`, but never rewinds
the cursors when it empties (`input_read_pos == input_write_pos`).
- The only rewind lives in `input_enqueue_raw()`, which `input_enqueue()` reaches
only inside `while (input_space() >= 19 && ptr < end)` (19 is the maximum
expansion of one `<x>` key form).
So once input fills the buffer to within 19 bytes of the top and is then fully
drained, the cursors are pinned near the top with `input_space() < 19`. The gate
never reopens, `input_enqueue()` never rewinds, and all further input is silently
dropped: `input_available()` stays 0 and the editor blocks forever in
`state_enter()` → `input_get()`. Redraw and resize run on independent paths, which
is why the UI looks alive while the keyboard is ignored.
Solution:
Rewind the read/write cursors when the buffer is empty, at the start of
`input_enqueue()`. The reset moves no data in the empty case and guarantees the
space gate can reopen; the non-empty case self-heals as the editor drains.
Test case:
A reproducer (no terminal required; drives `nvim --embed` over msgpack-RPC via
`nvim_input`) floods the input buffer, lets the editor drain it, then probes with
`:qa!`. It shows a sharp threshold at `INPUT_BUFFER_SIZE − 18`:
| `--fill` | master | with this fix |
|----------|-----------|---------------|
| ≤ 16367 | quits | quits |
| ≥ 16368 | **hangs** | quits |
- On current `master`, the freeze reproduces at `fill=16368`; the patched build
quits for every fill up to 100000.
- Interactive: a ~40 kB bracketed paste into Insert mode or a `:terminal` no longer
freezes.
Problem:
- Lua<=>API roundtrips
- Although we prefer Lua for most business-logic code, doing this
conversion in C makes sense in this case because:
1. setting options is a hot path
2. most of the options logic lives in C
3. the current arrangement is MORE verbose and requires MORE code
Solution:
Move conversion to a C util.
- nvim_set_option_value passes the raw Object (scalar, Array, or Dict)
to `object_as_optval_for()` which flattens it to the canonical `:set`
string and validates the type.
- drop `convert_value_to_vim`, eliminate its roundtrip.
Problem:
`nvim_set_option_value` cannot "update" options similar to `:set opt=`,
`:set opt+=`, etc. The Lua impls of "vim.opt" / "vim.o" have incomplete,
bespoke reimplementations of those operations.
ref #38420
Solution:
- Add `operation` param to `nvim_set_option_value`, which may be "set",
"append", "prepend", or "remove".
- Use this feature to implement `vim.opt` / `vim.o`.
Problem:
stdpath() may return a DOS 8.3 "shortened" filename, because Windows
truncates long usernames into `6ch~N` names in `TEMP/TMP` env vars.
We don't want to "leak" them into Nvim.
Solution:
For "run", pass `true` to `vim_FullName` to expand 8.3 filenames.
For "cache", call `os_realpath` to expand 8.3 filenames.
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem:
stdpath() may return a DOS 8.3 "shortened" filename, because Windows
truncates some long usernames into `6ch~N` names. Then features such
as `nvim_get_runtime_file` fail to find the file.
Analysis:
When expanding an 8.3 filename path like `C:/Users/ADMINI~1/AppData/*`,
we treat `~` as a special character and first check whether a directory
named `ADMINI~1` exists under `Users`. Since no such directory actually
exists, the expansion fails.
Solution:
Treat `~` as a literal character in `do_path_expand`. Since the `~/`
case is already handled in `gen_expand_wildcards`, any remaining `~` is
just a literal character and will later be escaped to `\~` by
`file_pat_to_reg_pat` if needed.
Problem:
2d795face6 added support for tab-local options ('cmdheight')
to `nvim_get_option_value`, but not to:
nvim_get_option_info2()
nvim_set_option_value(…, { tab = … })
gettabwinvar()
Solution:
- Update `options.lua` to model tab-local options. Introduce `kOptScopeTab`.
- Handle tab scope in the options layer so it works for all options APIs.
Note:
- No change to `gettabvar()`. Not sure if needed/wanted.
fix https://github.com/neovim/neovim/issues/31140
Problem:
API clients cannot query the tab-local value of 'cmdheight'.
Solution:
Allow nvim_get_option_value() to accept { tab = <tab-ID> } for 'cmdheight'.
Problem: Message redirection column for captured output is not reset
after :echon since (4260f73, e63346df).
Solution: Ensure msg_ext_append is set before the kind with :echon.
Problem: Internal progress messages use the "nvim" source (since
ff68fd6b), plugins shouldn't be allowed to set the progress
message source to "nvim". The message ID used for internal
progress messages is not identifiable as such.
Solution: Disallow setting opts->source to "nvim" with nvim_echo().
Refactor msg_progress() and callees to bypass nvim_echo().
Prepend message id for internal progress messages with "nvim.".
Problem:
The `nvim_get_chan_info()` terminal channel test used `shell-test INTERACT` to verify that `jobstop()` reports an unhandled `SIGHUP` as exit code `129`.
`INTERACT` reads from stdin with `fgets()`, so closing the PTY could race with `SIGHUP` delivery. If `fgets()` observed EOF first, `shell-test` exited normally with code `0`, causing intermittent failures on slower sanitizer builds.
Solution:
Add a `shell-test HOLD` mode that prints a readiness prompt and then waits without reading stdin. Use it for the `SIGHUP` assertion so PTY EOF cannot make the helper exit normally before the signal path is observed.
Problem:
`gx` relies on `exepath` to get the fullpath of `cmd.exe`,
and that path must use `\`; otherwise, luv's spawn will fail.
Solution:
Revert `slash_adjust` in `exepath`, so that it still respects 'shellslash'
Problem:
On Windows, path separators may become inconsistent for various reasons,
which makes normalization quite painful.
Solution:
Normalize paths to `/` at the entry boundaries and always use it
internally, converting back only in rare cases where `\` is really
needed (e.g. cmd.exe/bat scripts?).
This is the first commit in a series of incremental steps.
Note:
* some funcs won't respect shellslash. e.g. `expand/fnamemodify`
* some funcs still respect shellslash, but will be updated in a follow
PR. e.g. `ex_pwd/f_chdir/f_getcwd`
* uv's built-in funcs always return `\`. e.g. `uv.cwd/uv.exepath`
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem:
`nvim_echo(…, {id=…})` accepts user-defined id as a string or integer.
Generated ids are always higher than last highest msg-id used. Thus
plugins may accidentally advance the integer id "address space", which,
at minimum, could lead to confusion when troubleshooting, or in the
worst case, could overflow or "exhaust" the id address space.
There's no use-case for it, and it could be the mildly confusing, so we
should just disallow it.
Solution:
Disallow *integer* user-defined message-id.
Only allow *string* user-defined message-id.
Problem:
nvim_get_option_value with "filetype" set silently returns incorrect
defaults if autocommands are blocked, like when they're already running.
Solution:
Allow its FileType autocommands to nest: `do_filetype_autocmd(force=true)`.
Also error if executing them fails, rather than silently return wrong defaults.
Endless nesting from misbehaving scripts should be prevented by the recursion
limit in apply_autocmds_group, which is 10.
Problem: nvim_parse_cmd rejects valid commands like `:1` (range-only)
or `aboveleft` (modifier-only).
Solution: allow empty command when range or modifiers exist, and handle
execution using existing range command logic.
Problem:
When stopping a PTY process on Windows, the exit code indicates that the
process is stopped by SIGTERM even when closing all streams is enough to
terminate the process. This is inconsistent with other platforms.
Solution:
Set exit_signal to SIGHUP instead of SIGTERM when using SIGHUP.
Problem:
Exit code in :terminal channel test depends on whether the shell or Nvim
TUI in the terminal has registered its SIGHUP handler when jobstop() is
called.
Solution:
Don't use a shell as shells on different systems may handle SIGHUP
differently. Add a screen:expect() to wait for the TUI to start.
Problem: Exception error message only prints the first chunk of a
multi-chunk nvim_echo() message.
Solution: Concatenate consecutive message chunks in the exception
message list.
Problem:
When a terminal process exits, "[Process Exited]" text is added
to the buffer contents.
Solution:
- Return `exitcode` field from `nvim_get_chan_info`.
- Show it in the default 'statusline'.
- Show exitcode as virtual text in the terminal buffer.
Problem: Leading message newlines (not emitted with ext_messages since
4260f73) were responsible for resetting the redirection message
column (while the newline itself is later pruned...).
Solution: Ensure the redirection column is reset at the start of a message.
(Instead of re-adjusting all the newline callsites which can
themselves hopefully be pruned if ext_messages is enabled by
default.)
Define a CMake target for every subdirectory of test/functional that
contains functional tests, and a functionaltest_parallel target that
depends on all those targets, allowing multiple test runners to run in
parallel.
On CI, use at most 2 parallel test runners, as using more may increase
system load and make tests unstable.
Problem: Message ID is not incremented without ext_messages.
Solution: Increment where callstack reaches without ext_messages.
There is no clear place marking the end of internal
messages without ext_messages so IDs are only incremented
with `nvim_echo` calls. Message IDs are currently not exposed
anywhere but the msg_show event for this to matter.
Problem: nvim_parse_expression null pointer dereference with unmatched ]
followed by a node.
Solution: if ast_stack was empty, set new_top_node_p to top of the stack after
pushing the list literal node; similar to what's done for curlies.
This bug was originally found by a Matrix user, but I couldn't remember how to
trigger it... Ran into the other crash while finding a repro. :P
Problem: nvim_parse_expression null pointer dereference when parsing an
identifier followed by { with "highlight" parameter set to false.
Solution: only set opening_hl_idx if pstate->colors is not NULL.
Not added to parser_tests.lua as that uses highlight = true.
Problem: nvim_get_option_value with "filetype" set can crash if autocommands
open the dummy buffer in more windows, or if &bufhidden == "wipe".
Solution: Attempt to close all dummy buffer windows before wiping. Promote the
dummy buffer to a normal buffer if that fails.
Problem: When the "filetype" key is set for nvim_get_option_value, autocommands
can crash Nvim by prematurely wiping the dummy buffer, or cause options intended
for it to instead be set for unrelated buffers if switched during OptionSet.
Solution: Don't crash. Also quash side-effects from setting the buffer options.
Problem:
Using nvim_set_current_win() to switch windows while in Visual mode
causes E315 ml_get error when target buffer has fewer lines. This
doesn't happen with `:wincmd w` since it properly resets Visual mode
when switching buffers.
Solution:
Reset Visual mode when switching to another buffer, like `:wincmd w`.
Problem: nvim_parse_cmd('exe "ls"|edit foo', {}) fails to separate
nextcmd, returning args as { '"ls"|edit', 'foo' } instead of { '"ls"' }
with nextcmd='edit foo'.
Solution: Skip expressions before checking for '|' separator.
Problem: nvim_parse_cmd() incorrectly splits mapping commands like
into three arguments instead of preserving whitespace in the RHS.
Solution: Add special handling for mapping commands to parse them as exactly
two arguments - the LHS and the RHS with all whitespace preserved.
Problem: Since 48e2a736, prompt messages are handled by an actual
active cmdline, resulting in `State` no longer being equal
to `MODE_CONFIRM` which is used in some places. E.g. to
specify the current `mode()` or to re-emit a confirm message.
Solution: Replace `MODE_CONFIRM` with a new `MODE_CMDLINE` sub-mode when
`ccline.one_key/mouse_used` is set. Use it to avoid clearing
mouse_used prompt messages, and to re-emit one_key messages
(when ext_messages is inactive, for which this is unnecessary).
