Problem: The default match limit of 256 can be too low for realistic
use cases, but was necessary to guard against catastrophic
performance cliffs.
Solution: Performance cliffs were fixed in upstream tree-sitter 0.27+,
so remove the fallback limit to return unlimited matches by default.
Problem:
`get_node_text()` returned inconsistent results between buffer and
string sources when a node's range ends at `end_col == 0` (i.e. the node
ends with a newline). The buffer path dropped the trailing newline; the
string path included it correctly.
Solution:
Append `'\n'` in `buf_range_get_text()` when `end_col == 0` and
`start_row ~= end_row`. The `start_row ~= end_row` guard excludes
zero-width nodes at column 0, which should return `""`.
Remove the workaround in the `#trim!` directive that manually
compensated for the missing newline.
Strip whitespace in `resolve_lang()` so injection language nodes ending
at `end_col == 0` (e.g. `">lua\n"`) still resolve correctly.
After an edit, LanguageTree:_edit() updates the current trees. When the
LanguageTree manages explicit regions, _edit() also refreshes _regions
from tree:included_ranges(true), so those regions have the edited byte
offsets.
A later injection pass may call set_included_regions() with a different
number of child regions. That path discards the old trees and emits
changedtree callbacks for them. invalidate(true) does the same when a
buffer is reloaded. Before this change, both discard paths called
tree:included_ranges(true) for every old tree, even if _edit() had just
collected those exact ranges.
That duplicate range extraction is expensive with many injection trees.
Realistic shapes include generated C files with many macro bodies parsed
by the C preproc_arg injection, Markdown documents with many fenced blocks
of the same language, and template files with many embedded-language
islands. The stock highlighter registers recursive changedtree callbacks,
so this is on the normal highlighting edit path.
Track whether _regions currently came from tree:included_ranges(true)
with _regions_from_tree_ranges. _do_changedtree_callbacks() reuses
_regions only in that state; otherwise it falls back to calling
tree:included_ranges(true). Clear the marker when regions are replaced by
injection ranges, when a tree is reparsed, or when trees are discarded.
This avoids keeping a second copy of the ranges while preserving callback
precision: changedtree still receives tree:included_ranges(true) for the
old tree, not the broader managed region.
Benchmark on 100k C macro injections, one-line edit, recursive
changedtree callback:
- HEAD median: edited parse 84.2 ms, child region replacement 58.7 ms
- This change: edited parse 34.6 ms, child region replacement 8.5 ms
That is about 2.4x faster for the edit parse and 6.9x faster for child
region replacement in this workload.
Add a regression test that replacing injection regions still fires
changedtree and still reports the old tree's exact included ranges.
AI-assisted: Codex
Problem:
Cannot remove a `@conceal` highlight when defined in highlights.scm.
Solution:
Support a `@noconceal` highlight that works similarly to `@nospell` where it
overrides the conceal set on the range to remove it. Additionally, can
set the conceal metadata field to false for the same behavior.
Problem:
Can't expand treesitter-incremental-selection to the next and previous
sibling nodes.
Solution:
Pressing `]N` in visual mode will expand the selection to the next
sibling node, and `[N` will do the same with the previous node.
Problem:
`TSNode:id()` returns the underlying c pointer as a string, which may include
NUL bytes. In PUC Lua, `('%s'):format('\0a\0')` returns `''` and not `'\0a\0'`
(i.e. treats the string as a c-string (which terminates at the NUL byte)).
This resulted in two different nodes being able to have the same id.
Solution:
Use concatenation `..` instead of `string.format()`.
Problem:
`:help dev-name-common` states that "buf" should be used instead of
"buffer" but there are cases where buffer is mentioned in the lua API.
Solution:
- Rename occurrences of "buffer" to "buf" for consistency with the
documentation.
- Support (but deprecate) "buffer" for backwards compatibility.
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem:
The fold refresh path for foldminlines/foldnestmax creates a new
FoldInfo and starts an async parse. If FileType or BufUnload re-enters
before that callback returns, foldinfos[bufnr] can be cleared or
replaced. The callback then indexes a stale slot and raises an "attempt
to index a nil value" error.
Solution:
Capture the FoldInfo created for the refresh and carry that object
through the async callback. Before calling foldupdate(), verify that the
buffer still points at the same FoldInfo generation; otherwise ignore
the stale callback.
AI-assisted: Codex
Fixes#38461
refactor(lua): add integer coercion helpers
Add vim._tointeger() and vim._ensure_integer(), including optional base
support, and switch integer-only tonumber()/assert call sites in the Lua
runtime to use them.
This also cleans up related integer parsing in LSP, health, loader, URI,
tohtml, and Treesitter code.
supported by AI
Hyphenated language names are silently dropped when used as injections
(see #38132).
This combines the normalization of language aliases into `resolve_lang`,
and also adds the normalization of hyphens to underscores, which allows
for handling of injected language tags with hyphens in their names.
Fixes#38132.
Problem:
:InspectTree don't show luadoc injection lua file. Since luadoc share
the same "root" with comment in their common primary (lua) tree.
Current logic simply show the largest (comment injection) and ignore all
smaller one (luadoc injection).
Solution:
Handle different lang injections separately. Then sort them by
byte_length to ensure the draw tree consistent.
fix(treesitter): more distinctive highlight for EditQuery captures
Problem: EditQuery shows captures in the source buffer using the Title
highlight group, which could be too similar to Normal.
Solution: Use a virtual text diagnostic highlight group: they are
displayed in a similar manner to the query captures so we can assume
that the color scheme should have appropriate styling applied to make
them visible.
Although powerful -- especially with chained modifiers --, the
readability (and therefore maintainability) of `fnamemodify()` and its
modifiers is often worse than a function name, giving less context and
having to rely on `:h filename-modifiers`. However, it is used plenty in
the Lua stdlib:
- 16x for the basename: `fnamemodify(path, ':t')`
- 7x for the parents: `fnamemodify(path, ':h')`
- 7x for the stem (filename w/o extension): `fnamemodify(path, ':r')`
- 6x for the absolute path: `fnamemodify(path, ':p')`
- 2x for the suffix: `fnamemodify(path, ':e')`
- 2x relative to the home directory: `fnamemodify(path, ':~')`
- 1x relative to the cwd: `fnamemodify(path, ':.')`
The `fs` module in the stdlib provides a cleaner interface for most of
these path operations: `vim.fs.basename` instead of `':t'`,
`vim.fs.dirname` instead of `':h'`, `vim.fs.abspath` instead of `':p'`.
This commit refactors the runtime to use these instead of fnamemodify.
Not all fnamemodify calls are removed; some have intrinsic differences
in behavior with the `vim.fs` replacement or do not yet have a
replacement in the Lua module, i.e. `:~`, `:.`, `:e` and `:r`.
Problem: Spell navigation skips words on the first line because
_on_spell_nav passes an empty range (0,0) to the highlighter.
Solution: Use math.max(erow, srow + 1) to ensure a valid search window.
Signed-off-by: ashab-k <ashabkhan2000@gmail.com>
Problem:
When the `#offset!` directive is used with `:EditQuery`, the query does not take the offset into consideration when creating the extmark to preview the capture.
Solution:
Use the capture metadata to modify the node range before creating the extmark.
Problem:
VimEnter clears foldinfo, so register_cbs is called again after
VimEnter. The duplicate parser callbacks break incremental fold
computation.
Solution:
Check if the callbacks are already registered.
Problem: Outdated query files in `runtimepath` can trigger errors
which are hard to diagnose.
Solution: Add section to `:check treesitter` that lists all query
files in `runtimepath`, sorted by language and query type. Files
are listed in `runtimepath` order so that the first of multiple entry
is typically the one that is used.
Note: Unlike the `nvim-treesitter` health check, this does not try
to parse the queries so will not flag incompatible ones (which would
be much more expensive).
Problem:
If the last visible line in a window is not fully displayed, this line
may not get injection highlighting. This happens because line('w$')
actually means the last *completely displayed* line.
Solution:
Use line('w$') + 1 for the botline.
This reverts 4244a96774
"test: fix failing lsp/utils_spec #36609",
which changed the test based on the wrong behavior.
**Problem:** Whenever `LanguageTree:parse()` is called, injection trees
from previously parsed ranges are dropped.
**Solution:** Allow the function to accept a list of ranges, so it can
return injection trees for all the given ranges.
Co-authored-by: Jaehwang Jung <tomtomjhj@gmail.com>
The iterator is meant to be fully reset in this code path, but only the
`next_row` state was being reset. This would only cause highlight
artifacts for very brief periods of time, though.
This commit changes `languagetree.lua` so that it creates a scratch
buffer under the hood when dealing with string parsers. This will make
it much easier to just use extmarks whenever we need to track injection
trees in `languagetree.lua`. This also allows us to remove the
`treesitter.c` code for parsing a string directly.
Note that the string parser's scratch buffer has `set noeol nofixeol` so
that the parsed source exactly matches the passed in string.
This partially reverts 0b8a72b739,
that is unreverts 15e77a56b7
"priority" is an internal neovim concept which does not occur in shared
queries. Ideally a single priority space should eventually be enough
for our needs. But as we don't want to poke at the usages of
priorities right now in the wider ecosystem,
introduce the "subpriorities" so that treesitter code can distinguish
highlights of the same priorities with different tree nesting depth.
This mainly affects `injection.combined` as parent-tree nodes might appear
in the middle of child-tree nodes which otherwise is not possible.
Continuing the work of #31400
That PR allowed the provider to be invoked multiple times per line.
We want only to do that when there actually is more data later on the
line. Additionally, we want to skip over lines which contain no new
highlight items. The TS query cursor already tells us what the next
position with more data is, so there is no need to reinvoke the range
callback before that.
NB: this removes the double buffering introduced in #32619 which
is funtamentally incompatible with this (nvim core is supposed to keep
track of long ranges by itself, without requiring a callback reinvoke
blitz). Need to adjust the priorities some other way to fix the same issue.
Problem: many FileType autocommands assume curbuf is the same as the target
buffer; this can cause &syntax to be restored for the wrong buffer in some cases
when TSHighlighter:destroy is called.
Solution: run nvim_exec_autocmds in the context of the target buffer via
nvim_buf_call.
Don't print ABI version of duplicated parsers that are later in the
runtime path (see [#35326]).
Change the sorting from `name > path` to `name > rtpath_index`, this
ensures the first (loaded) parser is first in the list and any
subsequent parsers can be considered "not loaded".
This is fuzzy at best since `vim.treesitter.language.add` can take a
path to a parser and change the load order.
The correct solution is for `vim.treesitter.language.inspect` to return
the parser path so we can compare against it and/or for it to also be
able to take a path to a parser so we can inspect it without loading it
first.
