feat(lsp)!: change semantic token highlighting
Change the default highlights used, and add more highlights per token.
Add an LspTokenUpdate event and a highlight_token function.
:Inspect now shows any highlights applied by token highlighting rules,
default or user-defined.
BREAKING CHANGE: change the default highlight groups used by semantic
token highlighting.
redraw! redraws the entire screen instead of just the windows with
the buffer which were actually changed.
I considered trying to calculating the range for the delta
but it looks tricky. Could a follow-up.
Fixes#21543
This should provide a better user experience when appending or prepending text to a word that has a semantic token extmark. More often than not, the appended/prepended text to the word will end up becoming part of the token anyway, so just use that extmark as the user types.
Apply semantic token modifiers as separate extmarks with corresponding
highlight groups (e.g., `@readonly`). This is a low-effort PR to enable
the most common use cases (applying, e.g., italics or backgrounds on top
of type highlights; language-specific fallbacks like `@global.lua` are
also available). This can be replaced by more complicated selector-style
themes later on.
Instead of testing for every possible modifier type, only test bits up
to the highest set in the token array. Saves many bit ops and
comparisons when there are no modifiers or when the highest set bit is a
lower bit than the highest possible in the legend on average.
Can be further simplified when non-luaJIT gets the full bit module (see #21222)
The spec indicates that the response may be `null`, but it doesn't
really say what a `null` response means. Since neovim raises an error if
the response is `null`, I figured that ignoring it would be the safest
bet.
Co-authored-by: Mathias Fussenegger <f.mathias@zignar.net>
1. The algorithm for applying edits was slightly incorrect. It needs to
preserve the original token list as the edits are applied instead of
mutating it as it iterates. From the spec:
Semantic token edits behave conceptually like text edits on
documents: if an edit description consists of n edits all n edits are
based on the same state Sm of the number array. They will move the
number array from state Sm to Sm+1.
2. Schedule the semantic token engine start() call in the
client._on_attach() function so that users who schedule_wrap() their
config.on_attach() functions (like nvim-lspconfig does) can still
disable semantic tokens by deleting the semanticTokensProvider from
their server capabilities.
* credit to @smolck and @theHamsta for their contributions in laying the
groundwork for this feature and for their work on some of the helper
utility functions and tests
