**Problems:**
- `vim.treesitter.language.inspect()` returns duplicate
symbol names, sometimes up to 6 of one kind in the case of `markdown`
- The list-like `symbols` table can have holes and is thus not even a
valid msgpack table anyway, mentioned in a test
**Solution:** Return symbols as a map, rather than a list, where field
names are the names of the symbol. The boolean value associated with the
field encodes whether or not the symbol is named.
Note that anonymous nodes are surrounded with double quotes (`"`) to
prevent potential collisions with named counterparts that have the same
identifier.
Problem: Language names are only registered for filetype<->language
lookups when parsers are actually loaded; this means users cannot rely
on `vim.treesitter.language.get_lang()` or `get_filetypes()` to return
the correct value when language and filetype coincide and always need to
add explicit fallbacks.
Solution: Always return the language name as valid filetype in
`get_filetypes()`, and default to the filetype in `get_lang()`. Document
this behavior.
**Problem:** Top-level anonymous nodes are not being checked by the
query linter
**Solution:** Check them by adding them to the top-level query
This commit also moves a table construction out of the match iterator so
it is run less frequently.
**Problem:** `vim.treesitter.get_parser` will throw an error if no parser
can be found.
- This means the caller is responsible for wrapping it in a `pcall`,
which is easy to forget
- It also makes it slightly harder to potentially memoize `get_parser`
in the future
- It's a bit unintuitive since many other `get_*` style functions
conventionally return `nil` if no object is found (e.g. `get_node`,
`get_lang`, `query.get`, etc.)
**Solution:** Return `nil` if no parser can be found or created
- This requires a function signature change, and some new assertions in
places where the parser will always (or should always) be found.
- This commit starts by making this change internally, since it is
breaking. Eventually it will be rolled out to the public API.
For context, see https://github.com/neovim/neovim/pull/24738. Before
that PR, Nvim did not correctly handle captures with quantifiers. That
PR made the correct behavior opt-in to minimize breaking changes, with
the intention that the correct behavior would eventually become the
default. Users can still opt-in to the old (incorrect) behavior for now,
but this option will eventually be removed completely.
BREAKING CHANGE: Any plugin which uses `Query:iter_matches()` must
update their call sites to expect an array of nodes in the `match`
table, rather than a single node.
**Problem:** A query file for something like `html_tags` will not be
given html node completion
**Solution:** Check for parser aliases before offering completions
Co-authored-by: Lewis Russell <me@lewisr.dev>
Problem:
`TSNode:_rawquery()` is complicated, has known issues and the Lua and
C code is awkwardly coupled (see logic with `active`).
Solution:
- Add `TSQueryCursor` and `TSQueryMatch` bindings.
- Replace `TSNode:_rawquery()` with `TSQueryCursor:next_capture()` and `TSQueryCursor:next_match()`
- Do more stuff in Lua
- API for `Query:iter_captures()` and `Query:iter_matches()` remains the same.
- `treesitter.c` no longer contains any logic related to predicates.
- Add `match_limit` option to `iter_matches()`. Default is still 256.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
Query patterns can contain quantifiers (e.g. (foo)+ @bar), so a single
capture can map to multiple nodes. The iter_matches API can not handle
this situation because the match table incorrectly maps capture indices
to a single node instead of to an array of nodes.
The match table should be updated to map capture indices to an array of
nodes. However, this is a massively breaking change, so must be done
with a proper deprecation period.
`iter_matches`, `add_predicate` and `add_directive` must opt-in to the
correct behavior for backward compatibility. This is done with a new
"all" option. This option will become the default and removed after the
0.10 release.
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Co-authored-by: MDeiml <matthias@deiml.net>
Co-authored-by: Gregory Anders <greg@gpanders.com>
Memoizes a function, using a custom function to hash the arguments.
Private for now until:
- There are other places in the codebase that could benefit from this
(e.g. LSP), but might require other changes to accommodate.
- Invalidation of the cache needs to be controllable. Using weak tables
is an acceptable invalidation policy, but it shouldn't be the only
one.
- I don't think the story around `hash_fn` is completely thought out. We
may be able to have a good default hash_fn by hashing each argument,
so basically a better 'concat'.
