**Problem:** Top-level anonymous nodes are not being checked by the
query linter
**Solution:** Check them by adding them to the top-level query
This commit also moves a table construction out of the match iterator so
it is run less frequently.
**Problem:** `vim.treesitter.get_parser` will throw an error if no parser
can be found.
- This means the caller is responsible for wrapping it in a `pcall`,
which is easy to forget
- It also makes it slightly harder to potentially memoize `get_parser`
in the future
- It's a bit unintuitive since many other `get_*` style functions
conventionally return `nil` if no object is found (e.g. `get_node`,
`get_lang`, `query.get`, etc.)
**Solution:** Return `nil` if no parser can be found or created
- This requires a function signature change, and some new assertions in
places where the parser will always (or should always) be found.
- This commit starts by making this change internally, since it is
breaking. Eventually it will be rolled out to the public API.
For context, see https://github.com/neovim/neovim/pull/24738. Before
that PR, Nvim did not correctly handle captures with quantifiers. That
PR made the correct behavior opt-in to minimize breaking changes, with
the intention that the correct behavior would eventually become the
default. Users can still opt-in to the old (incorrect) behavior for now,
but this option will eventually be removed completely.
BREAKING CHANGE: Any plugin which uses `Query:iter_matches()` must
update their call sites to expect an array of nodes in the `match`
table, rather than a single node.
Problem: Installing treesitter parser is hard (harder than
climbing to heaven).
Solution: Add optional support for wasm parsers with `wasmtime`.
Notes:
* Needs to be enabled by setting `ENABLE_WASMTIME` for tree-sitter and
Neovim. Build with
`make CMAKE_EXTRA_FLAGS=-DENABLE_WASMTIME=ON
DEPS_CMAKE_FLAGS=-DENABLE_WASMTIME=ON`
* Adds optional Rust (obviously) and C11 dependencies.
* Wasmtime comes with a lot of features that can negatively affect
Neovim performance due to library and symbol table size. Make sure to
build with minimal features and full LTO.
* To reduce re-compilation times, install `sccache` and build with
`RUSTC_WRAPPER=<path/to/sccache> make ...`
**Problem:** With anonymous nodes toggled in the inspect tree, only
named nodes will be highlighted when moving the cursor in the source
code buffer.
**Solution:** Retrieve the anonymous node at the cursor (when toggled on
in the inspect tree) and highlight them when appropriate, for better
clarity/specificity.
This is identical to `named_node_for_range` except that it includes
anonymous nodes. This maintains consistency in the API because we
already have `descendant_for_range` and `named_descendant_for_range`.
**Problem:** A query file for something like `html_tags` will not be
given html node completion
**Solution:** Check for parser aliases before offering completions
Co-authored-by: Lewis Russell <me@lewisr.dev>
Problem:
Exiting the insert mode with ctrl-c does not trigger InsertLeave
autocmd. This may lead to nil error in treesitter foldexpr.
Solution:
Check nil. Folds still can be stale after exiting the insert mode with
ctrl-c, but it will be eventually updated correctly.
An alternative solution would be to ensure that exiting the insert mode
always triggers do_foldupdate. This can be done either by "fixing"
ctrl-c or with on_key callback that checks ctrl-c (nvim-cmp does this).
Problem:
Error when calling vim.treesitter.start() and vim.treesitter.stop() in
init.lua.
Solution:
Ensure syntaxset augroup exists after loading synload.vim.
Instead of looping over all captured nodes, just take the end range from
the last node in the list. This uses the fact that nodes returned by
iter_matches are ordered by their range (earlier to later).
Problem: Treesitter highlighter clears the already populated highlight
state when performing spell checking while drawing a
smoothscrolled topline.
Solution: Save and restore the highlight state in the highlighter's
_on_spell_nav callback.
Problem:
`o`-ing on a folded line opens the fold, because the new line gets the
fold level from the above line (level '='), which extends the fold to
the new line. `O` has a similar problem when run on the line below a
fold.
Solution:
Use -1 for the added line to get the lower level from the above/below
line.
Problem:
1. When interacting with multiple :InspectTree and the source buffer
windows there is a high chance of errors due to the window ids not
being updated and validated.
2. Not all InspectTree windows were closed when the source buffer was
closed.
Solution:
1. Update InspectTree window id on `CursorMoved` event and validate
source buffer window id before trying to navigate to it.
2. Close all InspectTree windows
Problem: `has-ancestor?` is O(n²) for the depth of the tree since it iterates over each of the node's ancestors (bottom-up), and each ancestor takes O(n) time.
This happens because tree-sitter's nodes don't store their parent nodes, and the tree is searched (top-down) each time a new parent is requested.
Solution: Make use of new `ts_node_child_containing_descendant()` in tree-sitter v0.22.6 (which is now the minimum required version) to rewrite the `has-ancestor?` predicate in C to become O(n).
For a sample file, decreases the time taken by `has-ancestor?` from 360ms to 6ms.
Problem: The column width 10 for parser name (lang) is too short.
For example, `markdown_inline` has 15 characters, which results in a
slight misalignment with other lines.
e.g. it looked like:
```
- OK Parser: markdown ABI: 14, path: .../parser/markdown.so
- OK Parser: markdown_inline ABI: 14, path: .../parser/markdown_inline.so
- OK Parser: php ABI: 14, path: .../parser/php.so
```
Solution: Use column width 20. As of now, the longest name among those
available in nvim-treesitter has length 18 (`haskell_persistent`).
e.g.:
```
- OK Parser: markdown ABI: 14, path: .../parser/markdown.so
- OK Parser: markdown_inline ABI: 14, path: .../parser/markdown_inline.so
- OK Parser: php ABI: 14, path: .../parser/php.so
```
Some parsers for, e.g., LaTeX or PHP have anonymous nodes like `"\"` or `"\text"` that behave wonkily (especially the first example) in the `InspectTree` window, so this PR escapes them by adding another backslash in front of them
Experimental and subject to future changes.
Add a way to redraw certain elements that are not redrawn while Nvim is waiting
for input, or currently have no API to do so. This API covers all that can be
done with the :redraw* commands, in addition to the following new features:
- Immediately move the cursor to a (non-current) window.
- Target a specific window or buffer to mark for redraw.
- Mark a buffer range for redraw (replaces nvim__buf_redraw_range()).
- Redraw the 'statuscolumn'.
* fix(treesitter): enforce lowercase language names
Problem: On case-insensitive file systems (e.g., macOS), `has_parser`
will return `true` for uppercase aliases, which will then try to inject
the uppercase language unsuccessfully.
Solution: Enforce and assume parser names to be lowercase when
resolving language names.
Problem:
While the fold level computation is incremental, the evaluation of the
foldexpr is done on the full buffer. Despite that the foldexpr reads
from the cache, it can take tens of milliseconds for moderately big (10K
lines) buffers.
Solution:
Track the range of lines on which the foldexpr should be evaluated.
Problem: Injecting languages for file redirects (e.g., in bash) is not
possible.
Solution: Add `@injection.filename` capture that is piped through
`vim.filetype.match({ filename = node_text })`; the resulting filetype
(if not `nil`) is then resolved as a language (either directly or
through the list maintained via `vim.treesitter.language.register()`).
Note: `@injection.filename` is a non-standard capture introduced by
Helix; having two editors implement it makes it likely to be upstreamed.
Problem:
`TSNode:_rawquery()` is complicated, has known issues and the Lua and
C code is awkwardly coupled (see logic with `active`).
Solution:
- Add `TSQueryCursor` and `TSQueryMatch` bindings.
- Replace `TSNode:_rawquery()` with `TSQueryCursor:next_capture()` and `TSQueryCursor:next_match()`
- Do more stuff in Lua
- API for `Query:iter_captures()` and `Query:iter_matches()` remains the same.
- `treesitter.c` no longer contains any logic related to predicates.
- Add `match_limit` option to `iter_matches()`. Default is still 256.
Using -1 as the initial value can cause the pattern offset to become
negative, which in turn results in a negative subpriority, which fails
validation in nvim_buf_set_extmark.
Tree-sitter queries can add URLs to a capture using the `#set!`
directive, e.g.
(inline_link
(link_text) @text.reference
(link_destination) @text.uri
(#set! @text.reference "url" @text.uri))
The pattern above is included by default in the `markdown_inline`
highlight query so that users with supporting terminals will see
hyperlinks. For now, this creates a hyperlink for *all* Markdown URLs of
the pattern [link text](link url), even if `link url` does not contain
a valid protocol (e.g. if `link url` is a path to a file). We may wish to
change this in the future to only linkify when the URL has a valid
protocol scheme, but for now we delegate handling this to the terminal
emulator.
In order to support directives which reference other nodes, the
highlighter must be updated to use `iter_matches` rather than
`iter_captures`. The former provides the `match` table which maps
capture IDs to nodes. However, this has its own challenges:
- `iter_matches` does not guarantee the order in which patterns are
iterated matches the order in the query file. So we must enforce
ordering manually using "subpriorities" (#27131). The pattern index of
each match dictates the extmark's subpriority.
- When injections are used, the highlighter contains multiple trees. The
pattern indices of each tree must be offset relative to the maximum
pattern index from all previous trees to ensure that extmarks appear
in the correct order.
- The `iter_captures` implementation currently has a bug where the
"match" table is only returned for the first capture within a pattern
(see #27274). This bug means that `#set!` directives in a query
apply only to the first capture within a pattern. Unfortunately, many
queries in the wild have come to depend on this behavior.
`iter_matches` does not share this flaw, so switching to
`iter_matches` exposed bugs in existing highlight queries. These
queries have been updated in this repo, but may still need to be
updated by users. The `#set!` directive applies to the _entire_ query
pattern when used without a capture argument. To make `#set!`
apply only to a single capture, the capture must be given as an
argument.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
