Remove the `set_timeout` functions for `TSParser` and instead add a timeout
parameter to the regular parse function. Remove these deprecated tree-sitter
API functions and replace them with the preferred `TSParseOptions` style.
Problem: Error messages that cause a vim.ui_attach() namespace to
detach are not visible in the message history. Decoration
provider and vim.ui_attach error messages are dissimilar.
Solution: Emit vim.ui_attach() errors as an actual message in addition
to logging it. Adjust error message format.
Problem: Message kind logic for emitting an error message is convoluted
and still results in emitting an unfinished message earlier than
wanted.
Solution: Ensure emsg_multiline() always sets the kind wanted by the caller
and doesn't isn't unset to logic for emitting the source message.
Caller is responsible for making sure multiple message chunks are
not emitted as multiple events by setting `msg_ext_skip_flush`...
Problem: Cannot disable individual captures and patterns in treesitter queries.
Solution:
* Expose the corresponding tree-sitter API functions for `TSQuery` object.
* Add documentation for `TSQuery`.
* Return the pattern ID from `get_captures_at_pos()` (and hence `:Inspect!`).
Problem: No way to check the version of a treesitter parser.
Solution: Add version metadata (ABI 15 parsers only) as well as parser state count and supertype information (ABI 15) in `vim.treesitter.language.inspect()`. Also graduate the `abi_version` field, as this is now the official upstream name.
---------
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Problem: Sourcing buffer lines is too complicated.
Solution: Simplify the code. Make it possible to source Vim9 script lines.
(Yegappan Lakshmanan, closesvim/vim#9974)
85b43c6cb7
This commit changes the behavior of sourcing buffer lines to always have
a script ID, although sourcing the same buffer always produces the same
script ID.
vim-patch:9.1.0372: Calling CLEAR_FIELD() on the same struct twice
Problem: Calling CLEAR_FIELD() on the same struct twice.
Solution: Remove the second CLEAR_FIELD(). Move the assignment of
cookie.sourceing_lnum (zeertzjq).
closes: vim/vim#14627f68517c167
Co-authored-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Problem: Wrong script context for option set by function defined by
nvim_exec2 in a Lua script.
Solution: Call nlua_set_sctx() after adding SOURCING_LNUM and always set
sc_lnum for a Lua script.
This is a bug discovered when testing #28486. Not sure if this actually
happens in practice, but it's easy to fix and required for #28486.
Problem: When setting an option, mapping etc. from Lua without -V1, the
script ID is set to SID_LUA even if there already is a script
ID assigned by :source.
Solution: Don't set script ID to SID_LUA if it is already a Lua script.
Also add _editor.lua to ignorelist to make script context more
useful when using vim.cmd().
Problem:
luv callback `vim.uv.new_timer():start(0, 0, function() error() end)`
causes SIGSEGV, since `xstrdup` gets NULL from `lua_tostring`.
Similar to: a5b1b83a26
Solution:
Check NULL before `xstrdup`.
Problem:
When running an initial parse, parse() returns an empty table rather
than an actual range. In `languagetree.lua`, we manually check if
a parse was incremental to determine the changed parse region.
Solution:
- Always return a range (in the C side) from parse().
- Simplify the language tree code a bit.
- Logger no longer shows empty ranges on the initial parse.
Problem: nlua_call_ref_ctx() does not pop the return value in fast
context that did not error.
Solution: Fall through to end; calling nlua_call_pop_retval().
Problem:
Calling `xstrdup` with a NULL pointer causes a SIGSEGV if `lua_tostring` returns
NULL in `nlua_luv_thread_common_cfpcall`.
Crash stack trace:
- `_platform_strlen` → `xstrdup` (memory.c:469)
- `nlua_luv_thread_common_cfpcall` (executor.c:281)
Solution:
Check if `lua_tostring` returns NULL and pass NULL to `event_create` to avoid the crash.
**Problem:** Parsing can be slow for large files, and it is a blocking
operation which can be disruptive and annoying.
**Solution:** Provide a function for asynchronous parsing, which accepts
a callback to be run after parsing completes.
Co-authored-by: Lewis Russell <lewis6991@gmail.com>
Co-authored-by: Luuk van Baal <luukvbaal@gmail.com>
Co-authored-by: VanaIgr <vanaigranov@gmail.com>
Problem: We want to deprecate `nvim_err_write(ln)()` but there is no
obvious replacement (from Lua). Meanwhile we already have
`nvim_echo()` with an `opts` argument.
Solution: Add `err` argument to `nvim_echo()` that directly maps to
`:echoerr`.
Regression from 2a7d0ed614, which removed
header that is only needed if wasmtime support is enabled. Prevent this
from happening again by wrapping the include in a `HAVE_WASMTIME` check.
Problem: Two separate try/end wrappers, that only marginally differ by
restoring a few variables. Wrappers that don't restore
previous state are dangerous to use in "api-fast" functions.
Solution: Remove wrappers that don't restore the previous state.
Always use TRY_WRAP.
Problem: Separate message emitted for each newline present in Lua
print() arguments.
Solution: Make msg_multiline() handle NUL bytes. Refactor print() to use
msg_multiline(). Refactor vim.print() to use print().
Problem: Lua callbacks for "msg_show" events with vim.ui_attach() are
executed when it is not safe.
Solution: Disallow non-fast API calls for "msg_show" event callbacks.
Automatically detach callback after excessive errors.
Make sure fast APIs do not modify Nvim state.
**Problem:** Currently node names with non-alphanumeric, non
underscore/hyphen characters (only possible with anonymous nodes) are
not given a proper error message. See tree-sitter issue 3892 for more
details.
**Solution:** Apply a different scanning logic to anonymous nodes to
correctly identify the entire node name (i.e., up until the final double
quote)
* deprecate old signatures
* move to new str_byteindex/str_utfindex signature
* use single-underscore name (double-underscore is reserved for Lua itself)
PROBLEM:
There are several limitations to vim.str_byteindex, vim.str_utfindex:
1. They throw given out-of-range indexes. An invalid (often user/lsp-provided)
index doesn't feel exceptional and should be handled by the caller.
`:help dev-error-patterns` suggests that `retval, errmsg` is the preferred
way to handle this kind of failure.
2. They cannot accept an encoding. So LSP needs wrapper functions. #25272
3. The current signatures are not extensible.
* Calling: The function currently uses a fairly opaque boolean value to
indicate to identify the encoding.
* Returns: The fact it can throw requires wrapping in pcall.
4. The current name doesn't follow suggestions in `:h dev-naming` and I think
`get` would be suitable.
SOLUTION:
- Because these are performance-sensitive, don't introduce `opts`.
- Introduce an "overload" that accepts `encoding:string` and
`strict_indexing:bool` params.
```lua
local col = vim.str_utfindex(line, encoding, [index, [no_out_of_range]])
```
Support the old versions by dispatching on the type of argument 2, and
deprecate that form.
```lua
vim.str_utfindex(line) -- (utf-32 length, utf-16 length), deprecated
vim.str_utfindex(line, index) -- (utf-32 index, utf-16 index), deprecated
vim.str_utfindex(line, 'utf-16') -- utf-16 length
vim.str_utfindex(line, 'utf-16', index) -- utf-16 index
vim.str_utfindex(line, 'utf-16', math.huge) -- error: index out of range
vim.str_utfindex(line, 'utf-16', math.huge, false) -- utf-16 length
```
**Problem:** Tree-sitter 0.24.0 introduced a new symbol type to denote
supertype nodes (`TSSymbolTypeSupertype`). Now, `language.inspect()`
(and the query `omnifunc`) return supertype symbols, but with double
quotes around them.
**Solution:** Mark a symbol as "named" based on it *not* being an
anonymous node, rather than checking that it is a regular node (which a
supertype also is not).
**Problems:**
- `vim.treesitter.language.inspect()` returns duplicate
symbol names, sometimes up to 6 of one kind in the case of `markdown`
- The list-like `symbols` table can have holes and is thus not even a
valid msgpack table anyway, mentioned in a test
**Solution:** Return symbols as a map, rather than a list, where field
names are the names of the symbol. The boolean value associated with the
field encodes whether or not the symbol is named.
Note that anonymous nodes are surrounded with double quotes (`"`) to
prevent potential collisions with named counterparts that have the same
identifier.
This commit also marks `child_containing_descendant()` as deprecated
(per upstream's documentation), and uses `child_with_descendant()` in
its place. Minimum required tree-sitter version will now be `0.24`.
Problem:
Tree cursors can only be efficient when they are re-used.
Short-lived cursors are very slow.
Solution:
Reimplement functions that use short-lived cursors.
Problem:
Linematch used to use strchr to navigate a string, however strchr does
not supoprt embedded NULs.
Solution:
Use `mmfile_t` instead of `char *` in linematch and introduce `strnchr()`.
Also remove heap allocations from `matching_char_iwhite()`
Fixes: #30505
CID 497370: Overflowed constant (INTEGER_OVERFLOW)
Expression `tsize - ret.has_type_key`, where tsize=0 and
ret.has_type_key=1, underflows the type that
receives it, an unsigned integer 64 bits wide.
CID 509910: Overflowed constant (INTEGER_OVERFLOW)
Expression stack.size++, which is equal to 0, where stack.size is
known to be equal to 18446744073709551615, overflows the type that
receives it, an unsigned integer 64 bits wide
In the api_info() output:
:new|put =map(filter(api_info().functions, '!has_key(v:val,''deprecated_since'')'), 'v:val')
...
{'return_type': 'ArrayOf(Integer, 2)', 'name': 'nvim_win_get_position', 'method': v:true, 'parameters': [['Window', 'window']], 'since': 1}
The `ArrayOf(Integer, 2)` return type didn't break clients when we added
it, which is evidence that clients don't use the `return_type` field,
thus renaming Dictionary => Dict in api_info() is not (in practice)
a breaking change.
Problem: Installing treesitter parser is hard (harder than
climbing to heaven).
Solution: Add optional support for wasm parsers with `wasmtime`.
Notes:
* Needs to be enabled by setting `ENABLE_WASMTIME` for tree-sitter and
Neovim. Build with
`make CMAKE_EXTRA_FLAGS=-DENABLE_WASMTIME=ON
DEPS_CMAKE_FLAGS=-DENABLE_WASMTIME=ON`
* Adds optional Rust (obviously) and C11 dependencies.
* Wasmtime comes with a lot of features that can negatively affect
Neovim performance due to library and symbol table size. Make sure to
build with minimal features and full LTO.
* To reduce re-compilation times, install `sccache` and build with
`RUSTC_WRAPPER=<path/to/sccache> make ...`
Problem:
Empty dictionaries are converted into typed tables of the form `{ [true]
= 6}` instead of an empty dictionary representation `{}`. This leads to
incorrect table representation, along with failure in JSON encoding of
such tables as currently tables with only string and number type keys
can be encoded.
Solution:
The typed table logic has been removed from `nlua_push_Dictionary`. The
typed table logic is required only for float value conversions which is
already handled in `nlua_push_Float`. So, it is(was) no longer required
here.
Fixesneovim/neovim#29218
This is a breaking change which will make refactor of typval and shada
code a lot easier. In particular, code that would use or check for
v:msgpack_types.binary in the wild would be broken. This appears to be
rarely used in existing plugins.
Also some cases where v:msgpack_type.string would be used to represent a
binary string of "string" type, we use a BLOB instead, which is
vimscripts native type for binary blobs, and already was used for BIN
formats when necessary.
msgpackdump(msgpackparse(data)) no longer preserves the distinction
of BIN and STR strings. This is very common behavior for
language-specific msgpack bindings. Nvim uses msgpack as a tool to
serialize its data. Nvim is not a tool to bit-perfectly manipulate
arbitrary msgpack data out in the wild.
The changed tests should indicate how behavior changes in various edge
cases.
Do the expansion right after setting the expand context, so that the
length of the completion prefix can be set, but don't do that directly
in set_one_cmd_context(), as that's also called by getcmdcompltype().
It's a function to perform operations in their own sealed context,
similar to pythons `with`. This helps ease operations where you need to
perform an operation in a specific context, and then restore the
context.
Marked as private for now as it's not ready for public use. The current
plan is to start using this internally so we can discover and fix any
problems. Once this is ready to be exposed it will be renamed to
`vim.with`.
Usage:
```lua
local ret = vim._with({context = val}, function()
return "hello"
end)
```
, where `context` is any combination of:
- `buf`
- `emsg_silent`
- `hide`
- `horizontal`
- `keepalt`
- `keepjumps`
- `keepmarks`
- `keeppatterns`
- `lockmarks`
- `noautocmd`
- `options`
- `sandbox`
- `silent`
- `unsilent`
- `win`
(except for `win` and `buf` which can't be used at the same time). This
list will most likely be expanded in the future.
Work on https://github.com/neovim/neovim/issues/19832.
Co-authored-by: Lewis Russell <lewis6991@gmail.com>
