fix(treesitter): more distinctive highlight for EditQuery captures
Problem: EditQuery shows captures in the source buffer using the Title
highlight group, which could be too similar to Normal.
Solution: Use a virtual text diagnostic highlight group: they are
displayed in a similar manner to the query captures so we can assume
that the color scheme should have appropriate styling applied to make
them visible.
Problem:
`:EditQuery` command accepts a language argument, but it doesn't
highlight properly for injected languages.
Solution:
- Fully parse with the root language and then filter the query on the
child trees that are of the language requested.
- Also support completion (`EditQuery <tab>`).
Problem:
- Running `:h :EditQuery` throws error `E149: Sorry, no help for
:EditQuery`
- vim_diff.txt miss an entry for `:EditQuery`
Solution:
- Make tag `[:EditQuery]()` right-aligned, similar to command `:Open`
- Update vim_diff.txt
This commit changes the `offset!` directive so that instead of setting a
`metadata.range` value for the entire pattern, it will set a
`metadata.offset` value. This offset will be applied to the range only
in `vim.treesitter.get_range()`, rather than at directive application
time. This allows the offset to be applied to any and all nodes captured
by the given pattern, and removes the requirement that `#offset!` be
applied to only a single node.
The downside of this change is that plugins which read from
`metadata.range` may be thrown off course, but such plugins should
prefer `vim.treesitter.get_range()` when retrieving ranges anyway.
Note that `#trim!` still sets `metadata.range`, and
`vim.treesitter.get_range()` still reads from `metadata.range`, if it
exists.
This matches the `iter_captures` functionality to the `iter_matches`
functionality, allowing it to specify a match limit and start depth for
the query iterator.
Problem: Cannot disable individual captures and patterns in treesitter queries.
Solution:
* Expose the corresponding tree-sitter API functions for `TSQuery` object.
* Add documentation for `TSQuery`.
* Return the pattern ID from `get_captures_at_pos()` (and hence `:Inspect!`).
**Problem:** Some weird languages have different comment syntax
depending on the location in the code, and we do not have a way to
determine the correct `commentstring` for these special cases.
**Solution:** Allow queries to specify `commentstring` values in
metadata, allowing users/`nvim-treesitter` to provide a better
commenting experience without hugely increasing the scope of the code in
core.
TSHighlighter now places marks for conceal_lines metadata. A new
internal decor provider callback _on_conceal_line was added that
instructs the highlighter to place conceal_lines marks whenever the
editor needs to know whether a line is concealed. The bundled markdown
queries use conceal_lines metadata to conceal code block fence lines.
Problem:
`yx` uses "y" prefix, which shadows a builtin operator.
Solution:
Use `g=` (in the form of `g==` currently), drawing from precedent of
CTRL-= and 'tpope/vim-scriptease'.
**Problem:** Parsing can be slow for large files, and it is a blocking
operation which can be disruptive and annoying.
**Solution:** Provide a function for asynchronous parsing, which accepts
a callback to be run after parsing completes.
Co-authored-by: Lewis Russell <lewis6991@gmail.com>
Co-authored-by: Luuk van Baal <luukvbaal@gmail.com>
Co-authored-by: VanaIgr <vanaigranov@gmail.com>
**Problem:** Query parsing uses a weak cache which is invalidated
frequently
**Solution:** Make the cache strong, and invalidate it manually when
necessary (that is, when `rtp` is changed or `query.set()` is called)
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Problem:
char-wise folding for `#trim!` ranges are improperly calculated for nodes that
end at column 0, due to the way `get_node_text` works.
Solution:
Add the blank line that `get_node_text` removes for for nodes ending at column
0. Also properly set column positions when performing linewise trims.
This commit also implements more generic trimming, acting on all
whitespace (charwise) rather than just empty lines.
It will unblock
https://github.com/nvim-treesitter/nvim-treesitter/pull/3442 and allow
for properly concealing markdown bullet markers regardless of indent
width, e.g.
Problem: No clear way to check whether parsers are available for a given
language.
Solution: Make `language.add()` return `true` if a parser was
successfully added and `nil` otherwise. Use explicit `assert` instead of
relying on thrown errors.
For context, see https://github.com/neovim/neovim/pull/24738. Before
that PR, Nvim did not correctly handle captures with quantifiers. That
PR made the correct behavior opt-in to minimize breaking changes, with
the intention that the correct behavior would eventually become the
default. Users can still opt-in to the old (incorrect) behavior for now,
but this option will eventually be removed completely.
BREAKING CHANGE: Any plugin which uses `Query:iter_matches()` must
update their call sites to expect an array of nodes in the `match`
table, rather than a single node.
Problem: `has-ancestor?` is O(n²) for the depth of the tree since it iterates over each of the node's ancestors (bottom-up), and each ancestor takes O(n) time.
This happens because tree-sitter's nodes don't store their parent nodes, and the tree is searched (top-down) each time a new parent is requested.
Solution: Make use of new `ts_node_child_containing_descendant()` in tree-sitter v0.22.6 (which is now the minimum required version) to rewrite the `has-ancestor?` predicate in C to become O(n).
For a sample file, decreases the time taken by `has-ancestor?` from 360ms to 6ms.
Problem:
`TSNode:_rawquery()` is complicated, has known issues and the Lua and
C code is awkwardly coupled (see logic with `active`).
Solution:
- Add `TSQueryCursor` and `TSQueryMatch` bindings.
- Replace `TSNode:_rawquery()` with `TSQueryCursor:next_capture()` and `TSQueryCursor:next_match()`
- Do more stuff in Lua
- API for `Query:iter_captures()` and `Query:iter_matches()` remains the same.
- `treesitter.c` no longer contains any logic related to predicates.
- Add `match_limit` option to `iter_matches()`. Default is still 256.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
Query patterns can contain quantifiers (e.g. (foo)+ @bar), so a single
capture can map to multiple nodes. The iter_matches API can not handle
this situation because the match table incorrectly maps capture indices
to a single node instead of to an array of nodes.
The match table should be updated to map capture indices to an array of
nodes. However, this is a massively breaking change, so must be done
with a proper deprecation period.
`iter_matches`, `add_predicate` and `add_directive` must opt-in to the
correct behavior for backward compatibility. This is done with a new
"all" option. This option will become the default and removed after the
0.10 release.
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Co-authored-by: MDeiml <matthias@deiml.net>
Co-authored-by: Gregory Anders <greg@gpanders.com>
Document that the `start` and `stop` parameters in
`Query:iter_captures()` and `Query:iter_matches()` are optional.
The tree-sitter lib has been bumped up to 0.20.9, so we also no longer
need "Requires treesitter >= 0.20.9".
- `TSQuery`: userdata object for parsed query.
- `vim.treesitter.Query`: renamed from `Query`.
- Add a new field `lang`.
- `TSQueryInfo`:
- Move to `vim/treesitter/_meta.lua`, because C code owns it.
- Correct typing for `patterns`, should be a map from `integer`
(pattern_id) to `(integer|string)[][]` (list of predicates or
directives).
- `vim.treesitter.QueryInfo` is added.
- This currently has the same structure as `TSQueryInfo` (exported
from C code).
- Document the fields (see `TSQuery:inspect`).
- Add typing for `vim._ts_parse_query()`.
