Apply semantic token modifiers as separate extmarks with corresponding
highlight groups (e.g., `@readonly`). This is a low-effort PR to enable
the most common use cases (applying, e.g., italics or backgrounds on top
of type highlights; language-specific fallbacks like `@global.lua` are
also available). This can be replaced by more complicated selector-style
themes later on.
Instead of testing for every possible modifier type, only test bits up
to the highest set in the token array. Saves many bit ops and
comparisons when there are no modifiers or when the highest set bit is a
lower bit than the highest possible in the legend on average.
Can be further simplified when non-luaJIT gets the full bit module (see #21222)
The spec indicates that the response may be `null`, but it doesn't
really say what a `null` response means. Since neovim raises an error if
the response is `null`, I figured that ignoring it would be the safest
bet.
Co-authored-by: Mathias Fussenegger <f.mathias@zignar.net>
1. The algorithm for applying edits was slightly incorrect. It needs to
preserve the original token list as the edits are applied instead of
mutating it as it iterates. From the spec:
Semantic token edits behave conceptually like text edits on
documents: if an edit description consists of n edits all n edits are
based on the same state Sm of the number array. They will move the
number array from state Sm to Sm+1.
2. Schedule the semantic token engine start() call in the
client._on_attach() function so that users who schedule_wrap() their
config.on_attach() functions (like nvim-lspconfig does) can still
disable semantic tokens by deleting the semanticTokensProvider from
their server capabilities.
* credit to @smolck and @theHamsta for their contributions in laying the
groundwork for this feature and for their work on some of the helper
utility functions and tests
