From the LSP Spec:
> There are two uses cases where it can be beneficial to only compute
> semantic tokens for a visible range:
>
> - for faster rendering of the tokens in the user interface when a user
> opens a file. In this use case, servers should also implement the
> textDocument/semanticTokens/full request as well to allow for flicker
> free scrolling and semantic coloring of a minimap.
> - if computing semantic tokens for a full document is too expensive,
> servers can only provide a range call. In this case, the client might
> not render a minimap correctly or might even decide to not show any
> semantic tokens at all.
This commit unifies the usage of range and full/delta requests as
recommended by the LSP spec and aligns neovim with the way other LSP
clients use these request types for semantic tokens.
When a server supports range requests, neovim will simultaneously send a
range request and a full/delta request when first opening a file, and
will continue to issue range requests until a full response is
processed. At that point, range requests cease and full (or delta)
requests are used going forward. The range request should allow servers
to return a result faster for quicker highlighting of the file while it
works on the potentially more expensive full result. If a server decides
the full result is too expensive, it can just error out that request,
and neovim will continue to use range requests.
This commit also fixes and cleans up some other things:
- gen_lsp: registrationMethod or registrationOptions imply dynamic
registration support
- move autocmd creation/deletion to on_attach/on_detach
- debounce requests due to server refresh notifications
- fix off by one issue in tokens_to_ranges() iteration
* cache all tokens from various range requests for a given document
version
- all new token highlights are merged with previous highlights to
maintain order and the "marked" property
- this allows the tokens to stop flickering once they've loaded once
per document version
* abandon the processing coroutine if the request_id has changed instead
of relying only on the document version
- this will improve efficiency if a new range request is made while a
previous one was processing its result
* apply new highlights from processing coroutine directly to the current
result when the version hasn't changed
- this allows new highlights to be immediately drawable once they've
processed instead of waiting for the whole response to be processed
at once
* rpc layer was changed to provide the request ID back in success
callbacks, which is then provided as a request_id field on the handler
context to lsp handlers
By simplifying the way range is supported, we can fix a couple issues as
well as making it less complex and more efficient:
* For non-range LSP servers, don't send requests on WinScrolled. The
semantic tokens module has been reworked to only send one active
request at a time, as it was before range support was added. If range
is not supported, then send_request() only fires if there's been a
change to the buffer's document version.
* Cache the server's support of range and delta requests when attaching
to a buffer to save the lookup on each request.
* Range requests always use the visible window, so just use that for the
`range` param when sending requests when range is supported by the
server. This reduces the API surface area of send_request().
* Debounce the WinScrolled autocmd requests in the same the way requests
are debounced when the buffer contents are changing. Should allow
scrolling via mouse wheel or holding down "j" or "k" work a bit
smoother.
The previous iteration of range support allowed multiple active requests
to be in progress simultaneously. However, a bug was preventing any but
the most recent request to actually apply to the client's highlighting
state so that complexity was unused. It was effectively only using one
active request at a time but was just using range requests on
WinScrolled events instead of a full (or delta) request when the
document version changed.
Problem:
Our LSP type system didnt have a concept of RegistrationMethods, this is where the method to dynamically register for a capability is sent to a different method endpoint then is used to call it. Eg `textDocument/semanticTokens` rather than the specific full/range/delta methods
Solution:
Extended generator to create `vim.lsp.protocol.Methods.Registration` with these registration methods. Also extend `_request_name_to_client_capability` to cover these methods. Adjust typing to suit
Allow the `request` parameter in `tokens_to_ranges` to be `nil` and
update version checking logic accordingly. This prevents errors when
the request is not present and improves robustness of semantic token
handling.
Problem:
Nvim supports `textDocument/semanticTokens/full` and `…/full/delta`
already, but most servers don't support `…/full/delta` and Nvim will try
to request and process full semantic tokens response on every buffer
change. Even though the request is debounced, there is noticeable lag if
the token response is large (in a big file).
Solution:
Support `textDocument/semanticTokens/range`, which requests semantic
tokens for visible screen only.
Problem:
Closes#31453
Solution:
Introduce `vim.lsp.Capability`, which may serve as the base class for
all LSP features that require caching data. it
- was created if there is at least one client that supports the specific method;
- was destroyed if all clients that support the method were detached.
- Apply the refactor for `folding_range.lua` and `semantic_tokens.lua`.
- Show active features in :checkhealth.
Future:
I found that these features that are expected to be refactored by
`vim.lsp.Capability` have one characteristic in common: they all send
LSP requests once the document is modified. The following code is
different, but they are all for this purpose.
- semantic tokens:
fb8dba413f/runtime/lua/vim/lsp/semantic_tokens.lua (L192-L198)
- inlay hints, folding ranges, document color
fb8dba413f/runtime/lua/vim/lsp/inlay_hint.lua (L250-L266)
I think I can sum up this characteristic as the need to keep certain
data synchronized with the latest version computed by the server.
I believe we can handle this at the `vim.lsp.Capability` level, and
I think it will be very useful.
Therefore, my next step is to implement LSP request sending and data
synchronization on `vim.lsp.Capability`, rather than limiting it to the
current create/destroy data approach.
Problem: There is no check for buffer validity before processing
semantic tokens response. This can lead to `Invalid buffer id` error
if processing request takes a long time and the buffer is wiped out.
For example, this can happen after by accident navigating to a buffer
from different project which leads to first loading project's
workspace and *then* processing semantic tokens. During that time
a buffer can be wiped out, as navigation to it was by accident.
Solution: Add extra check for buffer validity before processing semantic
tokens response.
* deprecate old signatures
* move to new str_byteindex/str_utfindex signature
* use single-underscore name (double-underscore is reserved for Lua itself)
Problem:
- `vim.highlight` module does not follow `:help dev-name-common`, which
documents the name for "highlight" as "hl".
- Shorter names are usually preferred.
Solution:
Rename `vim.highlight` to `vim.hl`.
This is not a breaking change until 2.0 (or maybe never).
Problem:
str_byteindex_enc could return an error if the index was longer than the
lline length. This was handled in each of the calls to it individually
Solution:
* Fix the call at the source level so that if the index is higher than
the line length, line length is returned as per LSP specification
* Remove pcalls on str_byteindex_enc calls. No longer needed now that
str_byteindex_enc has a bounds check.
This reduces the number of nil checks around buf_versions usage
Test changes were lifted from 5c33815
Co-authored-by: Mathias Fussenegger <f.mathias@zignar.net>
* Revert "fix(lsp): account for changedtick version gap on modified reset (#29170)"
This reverts commit 2e6d295f79.
* Revert "refactor(lsp): replace util.buf_versions with changedtick (#28943)"
This reverts commit 5c33815448.
LspDetach is now triggered by the main on_detach callback that is added
when an LSP client is attached to a buffer. The semantic_tokens module
already includes a LspDetach handler that does the right thing. When the
LspDetach trigger was added to the main LSP on_detach, it created a race
condition in semantic tokens when a buffer was deleted that would
trigger both its own on_detach and the LspDetach handlers. If the former
came last, an error was thrown trying to delete a non-existent augroup
(destroy() was being called twice).
`lsp.util.buf_versions` was already derived from changedtick (`on_lines`
from `buf_attach` synced the version)
As far as I can tell there is no need to keep track of the state in a
separate table.
Experimental and subject to future changes.
Add a way to redraw certain elements that are not redrawn while Nvim is waiting
for input, or currently have no API to do so. This API covers all that can be
done with the :redraw* commands, in addition to the following new features:
- Immediately move the cursor to a (non-current) window.
- Target a specific window or buffer to mark for redraw.
- Mark a buffer range for redraw (replaces nvim__buf_redraw_range()).
- Redraw the 'statuscolumn'.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem: luals returns stricter diagnostics with bundled luarc.json
Solution: Improve some function and type annotations:
* use recognized uv.* types
* disable diagnostic for global `vim` in shared.lua
* docs: don't start comment lines with taglink (otherwise LuaLS will interpret it as a type)
* add type alias for lpeg pattern
* fix return annotation for `vim.secure.trust`
* rename local Range object in vim.version (shadows `Range` in vim.treesitter)
* fix some "missing fields" warnings
* add missing required fields for test functions in eval.lua
* rename lsp meta files for consistency
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
