The auto-refresh has a bit of a delay so it can happen that when a user
runs `codelens.run` it operates on an outdated state and either
does nothing, or fails.
This changes the logic for `.run` to always fetch the current lenses
before (optional) prompt and execution.
See discussion in https://github.com/neovim/neovim/pull/37689#discussion_r2764235931
This could potentially be optimized to first check if there's local
state with a version that matches the current buf-version, but in my
testing re-fetching them always was quickly enough that `run` still
feels instant and doing it this way simplifies the logic.
Side effect of the change is that `.run` also works if codelens aren't
enabled - for power users who know what the codelens would show that can
be useful.
From the LSP Spec:
> There are two uses cases where it can be beneficial to only compute
> semantic tokens for a visible range:
>
> - for faster rendering of the tokens in the user interface when a user
> opens a file. In this use case, servers should also implement the
> textDocument/semanticTokens/full request as well to allow for flicker
> free scrolling and semantic coloring of a minimap.
> - if computing semantic tokens for a full document is too expensive,
> servers can only provide a range call. In this case, the client might
> not render a minimap correctly or might even decide to not show any
> semantic tokens at all.
This commit unifies the usage of range and full/delta requests as
recommended by the LSP spec and aligns neovim with the way other LSP
clients use these request types for semantic tokens.
When a server supports range requests, neovim will simultaneously send a
range request and a full/delta request when first opening a file, and
will continue to issue range requests until a full response is
processed. At that point, range requests cease and full (or delta)
requests are used going forward. The range request should allow servers
to return a result faster for quicker highlighting of the file while it
works on the potentially more expensive full result. If a server decides
the full result is too expensive, it can just error out that request,
and neovim will continue to use range requests.
This commit also fixes and cleans up some other things:
- gen_lsp: registrationMethod or registrationOptions imply dynamic
registration support
- move autocmd creation/deletion to on_attach/on_detach
- debounce requests due to server refresh notifications
- fix off by one issue in tokens_to_ranges() iteration
Problem:
`Client.on_exit` runs `Client._on_detach` and the client removal logic
within two separate `vim.schedule` sequentially. However, since
`Client._on_detach` executes `LspNotify` inside `vim.schedule`, this
causes `LspNotify` to be executed after the client removal, which is
scheduled first. At that point, a valid `Client` can no longer be
retrieved within the autocmd callback.
Solution:
Put the client deletion inside the `vim.schedule` call.
Problem:
`on_list` is supposed to replace the default list-handler. With the current order of these `if` statements `on_list` won't be called if `loclist` is true.
Solution:
Change the order of the relevant blocks.
Previously, adjust_start_col returned nil when completion items had
different start position from lsp textEdit range
This caused the completion to fall back to \k*$ which ignores the
non-keyword characters
Changes:
- adjust_start_col: now returns the minimum start postion among all
items instead of nil
- _lsp_to_complete_items - normalizes the items by adding the gap between
current and minimum start
Fixes: https://github.com/neovim/neovim/issues/37441
Although powerful -- especially with chained modifiers --, the
readability (and therefore maintainability) of `fnamemodify()` and its
modifiers is often worse than a function name, giving less context and
having to rely on `:h filename-modifiers`. However, it is used plenty in
the Lua stdlib:
- 16x for the basename: `fnamemodify(path, ':t')`
- 7x for the parents: `fnamemodify(path, ':h')`
- 7x for the stem (filename w/o extension): `fnamemodify(path, ':r')`
- 6x for the absolute path: `fnamemodify(path, ':p')`
- 2x for the suffix: `fnamemodify(path, ':e')`
- 2x relative to the home directory: `fnamemodify(path, ':~')`
- 1x relative to the cwd: `fnamemodify(path, ':.')`
The `fs` module in the stdlib provides a cleaner interface for most of
these path operations: `vim.fs.basename` instead of `':t'`,
`vim.fs.dirname` instead of `':h'`, `vim.fs.abspath` instead of `':p'`.
This commit refactors the runtime to use these instead of fnamemodify.
Not all fnamemodify calls are removed; some have intrinsic differences
in behavior with the `vim.fs` replacement or do not yet have a
replacement in the Lua module, i.e. `:~`, `:.`, `:e` and `:r`.
* cache all tokens from various range requests for a given document
version
- all new token highlights are merged with previous highlights to
maintain order and the "marked" property
- this allows the tokens to stop flickering once they've loaded once
per document version
* abandon the processing coroutine if the request_id has changed instead
of relying only on the document version
- this will improve efficiency if a new range request is made while a
previous one was processing its result
* apply new highlights from processing coroutine directly to the current
result when the version hasn't changed
- this allows new highlights to be immediately drawable once they've
processed instead of waiting for the whole response to be processed
at once
* rpc layer was changed to provide the request ID back in success
callbacks, which is then provided as a request_id field on the handler
context to lsp handlers
Problem:
vim.lsp.tagfunc looks for the presence of 'c' (cursor) flag and issues
sync textDocument/definition requests to all clients, otherwise
workspace/symbol requests. But 'c' flag can also be set during the
insert mode completion, e.g. with an empty tag completion query, the tag
func receives pattern of '\<\k\k' with flags 'cir'.
Solution:
check for 'i' (insert mode completion) flag and don't issue any LSP
requests, return vim.NIL for immediate fallback to tags.
Problem:
`:lsp restart` detects when a client has exited by using the `LspDetach`
autocommand. This works correctly in common cases, but breaks when
restarting a client which is not attached to any buffer. It also breaks
if a client is detached in between `:lsp restart` and the actual
stopping of the client.
Solution:
Move restart logic into `vim/lsp/client.lua`, so it can hook in to
`_on_exit()`. The public `on_exit` callback cannot be used for this, as
`:lsp restart` needs to ensure the restart only happens once, even if
the command is run multiple times on the same client.
Work on #37166
- Dynamic Registration Tracking via Provider
- Supports_Method
- Multiple Registrations
- RegistrationOptions may dictate support for a method
Problem:
We want to encourage implementing core features in Lua instead of C, but
it's clumsy because:
- Core Lua code (built into `nvim` so it is available even if VIMRUNTIME
is missing/invalid) requires manually updating CMakeLists.txt, or
stuffing it into `_editor.lua`.
- Core Lua modules are not organized similar to C modules, `_editor.lua`
is getting too big.
Solution:
- Introduce `_core/` where core Lua code can live. All Lua modules added
there will automatically be included as bytecode in the `nvim` binary.
- Move these core modules into `_core/*`:
```
_defaults.lua
_editor.lua
_options.lua
_system.lua
shared.lua
```
TODO:
- Move `_extui/ => _core/ui2/`
Problem:
`on_accept` is a bit cumbersome to customize.
Solution:
* Before: users had to override the entire `on_accept` logic for their changes to be applied.
* Now: users can modify the item and return it to apply the modified changes, or return `nil` to fully customize how the changes are applied.
Problem: When fuzzy is enabled and the prefix is not empty,
items are not sorted by fuzzy score before calling fn.complete.
Solution: Use matchfuzzypos to get the scores and sort the items
by fuzzy score before calling fn.complete.
By simplifying the way range is supported, we can fix a couple issues as
well as making it less complex and more efficient:
* For non-range LSP servers, don't send requests on WinScrolled. The
semantic tokens module has been reworked to only send one active
request at a time, as it was before range support was added. If range
is not supported, then send_request() only fires if there's been a
change to the buffer's document version.
* Cache the server's support of range and delta requests when attaching
to a buffer to save the lookup on each request.
* Range requests always use the visible window, so just use that for the
`range` param when sending requests when range is supported by the
server. This reduces the API surface area of send_request().
* Debounce the WinScrolled autocmd requests in the same the way requests
are debounced when the buffer contents are changing. Should allow
scrolling via mouse wheel or holding down "j" or "k" work a bit
smoother.
The previous iteration of range support allowed multiple active requests
to be in progress simultaneously. However, a bug was preventing any but
the most recent request to actually apply to the client's highlighting
state so that complexity was unused. It was effectively only using one
active request at a time but was just using range requests on
WinScrolled events instead of a full (or delta) request when the
document version changed.
Refactor capability checks in Client:_supports_registration and
Client:supports_method to properly handle dynamicRegistration and unknown
methods. Now, dynamic capabilities are checked before assuming support for
unknown methods, ensuring more accurate LSP feature detection.
Problem:
Our LSP type system didnt have a concept of RegistrationMethods, this is where the method to dynamically register for a capability is sent to a different method endpoint then is used to call it. Eg `textDocument/semanticTokens` rather than the specific full/range/delta methods
Solution:
Extended generator to create `vim.lsp.protocol.Methods.Registration` with these registration methods. Also extend `_request_name_to_client_capability` to cover these methods. Adjust typing to suit
Problem:
If a `vim.lsp.config` explicitly sets `exit_timeout`, that indicates the
config wants that behavior for most usages of `:stop()`.
Solution:
Update `:stop()` to use `force=exit_timeout` if `force` was not
explicitly passed.
Also, don't start the timer at all when a previous shutdown failed, as
in this case a forced shutdown is used and no timer is needed.
This fixes most of the delays caused by #36750.
The delays caused by #36378 still seem to remain.
Allow the `request` parameter in `tokens_to_ranges` to be `nil` and
update version checking logic accordingly. This prevents errors when
the request is not present and improves robustness of semantic token
handling.
Problem:
The `flags` field calls its sub-fields "experimental".
But `exit_timeout` is now used for multiple purposes.
Solution:
Graduate `exit_timeout` to a top-level ClientConfig field.
Problem:
Nvim supports `textDocument/semanticTokens/full` and `…/full/delta`
already, but most servers don't support `…/full/delta` and Nvim will try
to request and process full semantic tokens response on every buffer
change. Even though the request is debounced, there is noticeable lag if
the token response is large (in a big file).
Solution:
Support `textDocument/semanticTokens/range`, which requests semantic
tokens for visible screen only.
Problem:
LSP incremental selection provides default visual-mode keymaps for `an`
and `in`. Operator-pending mode is not supported, so `dan` and `can` do
not apply the operation.
Solution:
Modify selection_range() to be synchronous.
Add operator-pending mappings.
Problem:
With the typescript LSes typescript-language-server and vtsls,
omnicompletion on partial tokens for certain types, such as array
methods, and functions that are attached as attributes to other
functions, either results in no entries populated in the completion menu
(typescript-language-server), or an unfiltered completion menu with all
array methods included, even if they don't share the same prefix as the
partial token being completed (vtsls).
Solution:
Enable insertReplaceSupport and uses the insert portion of the lsp
completion response in adjust_start_col if it's included in the
response.
Completion results are still filtered client side.
Problem: No way to customize completion order across multiple servers.
Solution: Add `cmp` function to `vim.lsp.completion.enable()` options
for custom sorting logic.
Problem:
Users often jump and navigate through LSP windows to yank text.
Concealed markdown can make navigation through hyperlinks and code
blocks more difficult.
Solution:
Change 'concealcursor' from 'n' to '' to preserve clean display
while improving navigation and selection of the LSP response.
Closes#36537
Problem:
Some servers write log to stdout and there's no way to avoid it.
See https://github.com/neovim/neovim/pull/35743#pullrequestreview-3379705828
Solution:
We can extract `content-length` field byte by byte and skip invalid
lines via a simple state machine (name/colon/value/invalid), with minimal
performance impact.
I chose byte parsing here instead of pattern. Although it's a bit more complex,
it provides more stable performance and allows for more accurate error info when
needed.
Here is a bench result and script:
parse header1 by pattern: 59.52377ms 45
parse header1 by byte: 7.531128ms 45
parse header2 by pattern: 26.06936ms 45
parse header2 by byte: 5.235724ms 45
parse header3 by pattern: 9.348495ms 45
parse header3 by byte: 3.452389ms 45
parse header4 by pattern: 9.73156ms 45
parse header4 by byte: 3.638386ms 45
Script:
```lua
local strbuffer = require('string.buffer')
--- @param header string
local function get_content_length(header)
for line in header:gmatch('(.-)\r?\n') do
if line == '' then
break
end
local key, value = line:match('^%s*(%S+)%s*:%s*(%d+)%s*$')
if key and key:lower() == 'content-length' then
return assert(tonumber(value))
end
end
error('Content-Length not found in header: ' .. header)
end
--- @param header string
local function get_content_length_by_byte(header)
local state = 'name'
local i, len = 1, #header
local j, name = 1, 'content-length'
local buf = strbuffer.new()
local digit = true
while i <= len do
local c = header:byte(i)
if state == 'name' then
if c >= 65 and c <= 90 then -- lower case
c = c + 32
end
if (c == 32 or c == 9) and j == 1 then
-- skip OWS for compatibility only
elseif c == name:byte(j) then
j = j + 1
elseif c == 58 and j == 15 then
state = 'colon'
else
state = 'invalid'
end
elseif state == 'colon' then
if c ~= 32 and c ~= 9 then -- skip OWS normally
state = 'value'
i = i - 1
end
elseif state == 'value' then
if c == 13 and header:byte(i + 1) == 10 then -- must end with \r\n
local value = buf:get()
return assert(digit and tonumber(value), 'value of Content-Length is not number: ' .. value)
else
buf:put(string.char(c))
end
if c < 48 and c ~= 32 and c ~= 9 or c > 57 then
digit = false
end
elseif state == 'invalid' then
if c == 10 then -- reset for next line
state, j = 'name', 1
end
end
i = i + 1
end
error('Content-Length not found in header: ' .. header)
end
--- @param fn fun(header: string): number
local function bench(label, header, fn, count)
local start = vim.uv.hrtime()
local value --- @type number
for _ = 1, count do
value = fn(header)
end
local elapsed = (vim.uv.hrtime() - start) / 1e6
print(label .. ':', elapsed .. 'ms', value)
end
-- header starting with log lines
local header1 =
'WARN: no common words file defined for Khmer - this language might not be correctly auto-detected\nWARN: no common words file defined for Japanese - this language might not be correctly auto-detected\nContent-Length: 45 \r\n\r\n'
-- header starting with content-type
local header2 = 'Content-Type: application/json-rpc; charset=utf-8\r\nContent-Length: 45 \r\n'
-- regular header
local header3 = ' Content-Length: 45\r\n'
-- regular header ending with content-type
local header4 = ' Content-Length: 45 \r\nContent-Type: application/json-rpc; charset=utf-8\r\n'
local count = 10000
collectgarbage('collect')
bench('parse header1 by pattern', header1, get_content_length, count)
collectgarbage('collect')
bench('parse header1 by byte', header1, get_content_length_by_byte, count)
collectgarbage('collect')
bench('parse header2 by pattern', header2, get_content_length, count)
collectgarbage('collect')
bench('parse header2 by byte', header2, get_content_length_by_byte, count)
collectgarbage('collect')
bench('parse header3 by pattern', header3, get_content_length, count)
collectgarbage('collect')
bench('parse header3 by byte', header3, get_content_length_by_byte, count)
collectgarbage('collect')
bench('parse header4 by pattern', header4, get_content_length, count)
collectgarbage('collect')
bench('parse header4 by byte', header4, get_content_length_by_byte, count)
```
Also, I removed an outdated test
accd392f4d/test/functional/plugin/lsp_spec.lua (L1950)
and tweaked the boilerplate in two other tests for reusability while keeping the final assertions the same.
accd392f4d/test/functional/plugin/lsp_spec.lua (L5704)accd392f4d/test/functional/plugin/lsp_spec.lua (L5721)
Problem: reuse_win will always jump to the first window containing the
target buffer rather even if the buffer is displayed in the current
window/tab
Solution: check to see if the buffer is already displayed in the
current window or any window of the current buffer
