Other (squashed) commits:
fix(tui): emit ui_send output atomically with the frame
Problem:
tui_ui_send() writes directly to the TTY, bypassing the output buffer.
Sequences sent via nvim_ui_send() (e.g. kitty multiple-cursors, or
visual-dot-repeat) always arrive in a separate TTY write from the frame
they were computed for. This manifests as "tearing", or e.g. in the case
of multicursor the terminal renders text with stale cursor overlays.
Solution:
- tui_ui_send(): while a frame is being assembled (pending invalid
regions or buffered output), buffer instead of writing directly.
- Out-of-frame sends (tty queries, clear-on-disable) still write
immediately.
- mcursor.lua: emit the terminal-cursor update at the end of the redraw
cycle (`on_end`, when screen positions are final) instead of
vim.schedule().
Problem: options related to ui2 (like `fillchars` with `msgsep`) do not
take effect if set during startup after enabling ui2.
Solution: explicitly check just after startup if relevant options were
changed during startup.
Problem:
Folding range markers are overwritten while ranges are evaluated. A range ending on a row can hide another range starting there, and multiple nested ranges ending together emit only the innermost ending level.
Solution:
Track starts and the number of ends per row before emitting markers. Prefer starts on shared boundary rows and use the outermost level when nested ranges end together.
AI-assisted
Problem:
Confirming cmdwin with a UTF-8 character containing 0x80 does not complete the
command.
Solution:
Escape K_SPECIAL bytes while feeding the cmdwin input after confirmation.
Problem: C keywords are matched using the characters from 'iskeyword',
so changing that option highlights part of an identifier as a
keyword and leaves keywords containing an underscore
unhighlighted.
Solution: Set the keyword characters with ":syn iskeyword".
fixes: vim/vim#21173closes: vim/vim#211765c9c5a43c1
Co-authored-by: Hirohito Higashi <h.east.727@gmail.com>
Co-Authored-By: Maxim Kim <habamax@gmail.com>
Problem: YAML syntax highlighting is slow; the yamlInteger and yamlFloat
rules alone account for over half of the parsing time.
Solution: The number, null and timestamp scalar patterns begin with a
lookbehind, which stops the regexp engine from using a
first-character search, so the automatic engine selects the
much slower NFA engine. Force the backtracking engine with
\%#=1 on these patterns for a large speedup with identical
matches (Jordan).
On a 40000-line YAML file the total :syntime drops by about 30%: the
yamlFloat rule goes from 0.61s to 0.17s and yamlInteger from 0.42s to 0.31s.
The engine override is applied only to the lookbehind-anchored number rules;
forcing it on the structural plain-scalar and mapping-key patterns regresses
them badly, so those are left on the automatic engine.
closes: vim/vim#21182560dfadac8
Co-authored-by: Julien Voisin <julien.voisin@dustri.org>
Problem:
The "." example mapping at `:h edit-repeat` doesn't work well with
`nvim_feedkeys(…, 'mt', false)`.
Solution:
Use `vim.b[ev.buf].maxseq` instead of `undotree()`.
Problem:
The cmdline_hide callback causes an unnecessary cursor move to the cmdline,
before the normal redraw updates the cursor back to the current window. This
appears as "flicker" when using plugins such as matchit (legacy ":" mappings
instead of "<cmd>" mappings).
Solution:
Skip the immediate redraw for cmdline_hide events. The normal redraw
still updates the cursor after the cmdline window is hidden.
Problem:
Replaying a deleted Lua mapping, may call an arbitrary function.
RHS of a Lua mapping embeds its LuaRef (`<K_LUA><ref><CR>`). The raw
keys may outlive the ref (redobuff ".", CmdAtom `keys`). If the mapping
is deleted, replaying it either (1) dereferences a freed registry slot,
or (2) calls whatever callback reused the slot (autocmd, timer, other
mapping).
Solution:
Assign a monotonic (never recycled) id to Lua mappings and encode the
mapping keys as `<K_LUA><id><CR>`.
Note: in the case of Vimscript, a deleted function raises E117, but if
the function is redefined with the same name, the mapping will find it.
Alternatives?:
- Globally ensure `LuaRef` ids are not recycled.
- Problem: could exhaust `int` in a long-lived Nvim session? Also,
difficult to impl bc the "recycling" is done by `luaL_ref` itself.
ref: 5ac2e47acc
Problem:
If a snippet does not have a placeholder, we use insert mode instead of
select mode. From here <Esc> leaves the session and highlight active.
Solution:
Cancel the session on <Esc>.
Problem:
When 'cmdheight' is 0, the MsgSeparator can obscure the statusline when
a message is displayed, making its contents disappear until
the next redraw.
Solution:
Skip the separator when 'cmdheight=0' available.
Problem:
A blank line before the Content-Length header makes header parsing
fail with "Content-Length not found in header". An LF in the 'name'
state falls through to the 'invalid' state, which only return to
'name' at the *next* LF. That LF terminates the next line, so the
line with Content-Length is swallowed.
The same issue happens when a junk line is a partial match of the
header name (e.g. "Cont\nContent-Length: ...").
Solution:
When seeing an LF in the 'name' state, reset the cursor and stay in
'name' rather than entering 'invalid'.
Problem:
The Vim "stuff" concept breaks the ability to reason about the call
stack and thus the boundaries of a `CmdAtom`: a stuffed translation ("x"
=> "dl") defers to the main loop. This "continuation" must be modeled in
`CmdAtom`, by checking global flags at undefined times, during undefined
circumstances.
Solution:
- After a stuffed "translation", eagerly execute the stuff buffer
(`exec_stuffed()`).
- Delete the CmdAtom "continuation" junk.
Note:
- op_colon runs its cmdline "nested", but that's fine because operators
already nest interactive sessions there (op_change runs edit()), and
the cmdline is frameless so the operator's frame can own/capture it.
Problem: Several tag and attribute names are highlighted with regular
expression alternations even though they are plain words. A
:syn-match with an alternation is retried at nearly every
position of every tag.
Solution: Move the plain-word tag names (b, i, u, em, strong, head,
body, title, h1-h6) and the "label", "href" and "title"
attribute names into syn-keyword lists, leaving only the
genuinely hyphenated names "accept-charset" and "http-equiv"
as matches.
Profiling a 6000-line HTML file with :syntime, the converted rules drop
from a combined 0.044s to 0.004s, lowering total per-rule syntax time
by about 15%. Highlighting is unchanged except that a bare valueless
"href" attribute is now highlighted as an attribute, consistent with a
bare "title", which already behaved that way.
closes: vim/vim#21155964a495bef
Co-authored-by: Julien Voisin <julien.voisin@dustri.org>
Problem:
Undo places the cursor wherever the cursor happened to sit at "save
time" (`uh_cursor` is sampled lazily on the first change).
Examples:
- `i` preserves, but `a` does not
- `diw`, `atest<Esc>`, `d^` abandon the original position
- `D`, `o` restore it (by accident).
Solution:
`composite` tracks the pending atom (and its `origin`) across frames.
A `stuffed` continuation frame inherits the `origin` + prepped redo.
Store `origin` info in the undo header, so undo can restore it.
- Not for a mid-command undo break (i_CTRL-G_u).
- Undoing a mapping restores where the mapping started (which
technically may be different than where the "edit" started).
Problem:
`nlua_pcall()` references `_G.debug.traceback`. If user code deletes it
or breaks it some other way, various Lua features are broken.
_G.debug = nil
vim.schedule(function() end)
vim.wait(100)
E5113: Lua chunk: attempt to index a nil value
stack traceback:
[C]: in function 'loop_poll'
[string "vim/_core/editor"]:176: in function 'wait'
crash.lua:3: in main chunk
PANIC: unprotected error in call to Lua API (attempt to index a nil value)
Solution:
Check `_G.debug.traceback` before using it as errfunc. If it's broken,
omit the traceback and say so in the error message.
Note: We could cache `_G.debug` in LUA_REGISTRYINDEX on startup, but
that would prevent plugins from providing custom functionality there
(and we happen to do so in `tui_spec.lua` for example).
Problem:
With these two lines (`\b` marking a backspace):
f\bfoo
xb\bb
line 1's bold run ends at byte 1 and line 2's begins at byte 1, so line 1
renders "fo" in bold rather than "f", and line 2's "b" is not bold at all.
Solution:
Only grow a highlight group that is on the current row.
Problem: A function used through 'omnifunc' or 'complete' is called the
same way whether 'autocomplete' started the completion or a
key asked for one, so it cannot answer differently.
Solution: Report which of the two it is in complete_info() as "auto". It
is returned only when asked for in {what}, so what
complete_info() says by itself does not change
(Hirohito Higashi).
closes: vim/vim#211431f56c351de
Co-authored-by: Hirohito Higashi <h.east.727@gmail.com>
Problem: filetype: bazelrc files are not recognized
Solution: Detect *.bazelrc and tools/bazel.rc files as bazelrc filetype,
include syntax und filetype plugins, update the menus
(Barrett Ruth).
closes: vim/vim#211461afe7ad1bb
Co-authored-by: Barrett Ruth <br@barrettruth.com>
Problem:
The runs ":helptags", which needs the tree-sitter-vimdoc parser.
This may cause problems for package maintainers / distros?
Solution:
Use running to generate the tags, like the zig build already does.
It also errors on duplicate tags, which is good.
The `:helptags` command still uses treesitter.
Problem:
The vimdoc parser does not treat "\r" as whitespace, so in a CRLF
helpfile the codeblock rule fails and "*" pairs inside examples become
tags. On Windows this generates duplicate "." and "/" tags:
D:/a/neovim/neovim/build/bin/nvim.exe -u NONE -i NONE -e --headless -c "helptags ++t doc" -c "exe 'cquit' !empty(v:errmsg)""
Error in command line:
E154: Duplicate tag "." in gui.txt and repeat.txt
E154: Duplicate tag "/" in pattern.txt and usr_08.txt
E154: Duplicate tag "/" in usr_08.txt and pattern.txt
Note: The old C parser was unaffected because it read helpfiles in text
mode (`os_fopen(…, "r")`).
Solution:
Strip "\r" before parsing.
Problem:
Parent commit regressed some behavior of the old C helptags-gen impl:
- helpfiles in sub-directories are named by basename, so :help fails
- "help-tags" always names "tags", never "tags-nl"
- E150 E151 E152 E153 are never reported
- existing tags file is not overwritten if no tags were found
- a duplicate tag aborts the run, skipping the remaining directories
- `*.TXT` and `*.FRX` (uppercase) are not recognized as helpfiles
- tree-sitter-vimdoc accepts tags that the C parser rejected: `*a|b*`
(breaks |links|) and unterminated `*tag`
- PUC Lua "<" compares with (localized) `strcoll()`, so the tags file is
not sorted by byte value (E432)
- requiring `vim.treesitter` at load time breaks :help itself, not just
:helptags, where the module is unavailable
- `vim.pack` runs :helptags for plugins that have no "doc/" directory,
so every install/update emits E150
Solution:
- Fail the build if generating helptags reports any `v:errmsg`.
- Restore old behavior: tags generated for runtime/doc are now identical
to those from the C implementation. Errors are non-fatal messages
instead of exceptions, so all directories are still processed.
- Load treesitter lazily, report a plain error if parser is missing.
Problem:
Tags are manually parsed in C which is not flexible and prone to errors.
Extending the help system to allow for other formats (e.g. Markdown)
would require a large rewrite in the C core, while with Treesitter it
only needs a query update.
Solution:
Use the power of treesitter to extract the tags from helpfiles.
- build: set `$VIMRUNTIME` when generating helptags, like
`cmake/Util.cmake` already does for other generators.
- fix(help): only accept tags delimited by whitespace. The old C parser
only accepted a `*tag*` preceded by start-of-line or whitespace and
followed by whitespace or end-of-line. The vimdoc parser also captures
tags followed by other text, e.g. `*$XDG_STATE_HOME*/.../logs` in
starting.txt, which caused an E154 duplicate tag error for docs that
were previously fine.
Problem:
In order for ui2 to graduate to the main "messages ui" its configuration
needs to graduate into actual options.
Solution:
Migrate some of its config to 'messagesopt':
- "maxheight" (note: currently this is an integer treated as
a "percentage"; if we want to support a row count we could allow
values with units, like `"42%"`)
- "pager"
- "timeout"
Also improves error messages:
messagesopt=hit-enter,history:500,bogus E474: Unknown item 'bogus'
messagesopt=hit-enter,history:500,progress:x E474: 'progress' must be one of: , c
messagesopt=hit-enter,history:abc E474: 'history' requires a number
Problem:
`pager_char` compares the `keytrans()` result exactly. E.g. `pager_char
= "<cr>"` does not work, it expects `<CR>` (uppercase).
Solution:
Normalize `pager_char` in enable().
Problem:
`Completor:accept()` applies the item against the range the server
answered with. Characters typed after the request but before accepting
lie inside neither that range nor the replacement, so they are left
behind by the insertion: typing `a` after `foob` and accepting `foobar`
within the 200 ms debounce yields `foobara`.
Solution:
Grow the item's range over the characters typed since the request, so
that `accept()` replaces them. Only while the candidate still matches,
decided by the longest common prefix `show()` already computes, so input
that contradicts it is left alone. Snippet items are unaffected, since
`accept()` ignores the range for them.
Problem:
`vim.on_key()` callbacks are invoked for keys that `getchar()` consumes,
which never reach the main loop, and there is no way to tell such a key
apart. With ui2 the `on_key` handler activated after interactive
cmdline, handles it as `pager_char` (thus enters the ui2 pager).
Solution:
Skip `vim.on_key()` callbacks during `getchar()`/`getcharstr()`.
Problem:
`Completor:accept()` hands the item's range straight to
`nvim_buf_set_text()`. That range is resolved when the response arrives,
so text deleted before accepting (backspacing within the 200 ms
debounce) leaves the range end past what is left of the line, and the
accept aborts with `Invalid 'end_col': out of range`.
18d6436 fixed the same staleness in `Completor:show()`, which drops an
item once its range *start* falls outside the buffer. That guard does
not cover the range end, and `accept()` was never given one, so this is
a hole left by that fix rather than a regression of it.
Solution:
Clamp the end of the range to the end of the line before writing. The
start needs no clamp: `show()` dropped the item one event loop tick
earlier if it had gone out of range.
Problem:
One item with an edit range before the word boundary changes the start
column for all items. For tsserver `str.`, completing `str.char` can
become `strcharAt` and break filtering.
Solution:
Prepend the missing text and filter on it.
Problem: complete_info() omits "abbr_hlgroup" and "kind_hlgroup".
Solution: Keep the highlight group ID with the match instead of the
resolved attribute and add both entries to the returned items
(glepnir).
closes: vim/vim#211051c32cede0a
1) The complete example plugin at :h write-plugin the end of the
documentation didn't reflect the snippets provided during the
tutorial. In most of the Add() function, interpolated strings are
used, but the final result used concatenated strings.
2) The example provided in readdir() to get a list of files ending in
".txt" didn't properly escape the dot in the regular expression:
readdir(dirname, {n -> n =~ '.txt$'})
This would match any file ending with "txt" that is at least 4
characters long, instead it should be:
readdir(dirname, {n -> n =~ '\.txt$'})
closes: vim/vim#2112363d08d4386
Co-authored-by: Josep Puigdemont <josep.puigdemont@gmail.com>
Problem:
Mapped keys can produce on_key callbacks with an empty `typed` value.
so mapped motions such as `j -> gj` are not recognized as typed input
and do not dismiss messages
Solution:
Track whether an empty `typed` callback follows typed input, allowing
mapped motions to dismiss messages like directly typed motions.
Problem:
A mapping that enters terminal-mode (`:FzfLua files` via `:startinsert`)
never reaches a composite-end: terminal-mode runs no "normal" CmdFrames.
The composite collects the entire terminal session (and more) into `lhs`.
E.g. if I have `<M-/>` mapped to open `:FzfLua files`, then interact
with fzf-lua UI, the emitted CmdAtom looks like:
<M-/>… => { type='mapping', lhs='<M-/><C-N><C-N><CR>', keys=nil }
Solution:
End the composite when terminal-mode is entered, which emits a more
meaningful and repeatable atom:
<M-/>… => { type='mapping', lhs='<M-/>', keys=nil }
Problem:
The "!" operator stuffs its cmdline continuation (`:.,.+1!`), so its
frame ends before capture (stuff pending) and the redo-prep disappears
with it.
!ipsort<CR> => { type='excmd', lhs=':.,.+1!sort<NL>', keys=':.,.+1!sort<NL>' }
Compare to builtin "." which works bc `do_bang()` completes the redo
(`!ip` + `sort<NL>`).
Solution:
Appoint the stuffed continuation frame as the "redo-prep" frame.
!ipsort<CR> => { type='operator', operator='!', lhs='!ipsort<NL>', keys='!ipsort<NL>' }
Notes:
- atom_cmd_end(): a frame ending with stuff pending re-points its
redo-prep to the next frame.
- atom_cmd_start(): a stuffed continuation frame (`KeyStuffed`) keeps
the redo-prep; flushed stuff discards it.
Problem:
CTRL-] via lsp tagfunc passes URI paths through :tag, which expands
\$VAR, so files like people_.\$personId.tsx fail with E429.
Solution:
fnameescape() the filename in tag items so \$ is treated as literal.
Problem:
man -w echoes any existent file back unchanged. :Man rejected all such
paths, so :Man /usr/share/man/man1/bash.1 failed even for real man pages.
Solution:
Accept an echoed path when it looks like a man page (.../man1/foo.1).
