Problem: LuaLS struggles with the generics used in Nvim's runtime,
requiring broad diagnostic suppressions. Indexing is also slow.
Solution: Use EmmyLua for type checks in the build and CI. It offers
more sophisticated type checking, substantially better support for
generics, and much better flow analysis.
Correct the affected annotations. Use `@internal`, supported directly by
EmmyLua, instead of `@nodoc` for shared internal declarations, and
support it in the help parser.
AI-assisted
Problem:
When disabling, the loader list is traversed forwards while entries are
removed. As a result, some entries are skipped and remain active. Thus,
disabling `vim.loader` restores `_G.loadfile`, but does not fully
restore the original package loader chain.
Solution:
Traverse `package.loaders` in reverse when removing cached loaders.
Capture the original `package.loaders` list in the test, and assert that
the list is restored properly.
`opts.plain=true` does not expand tildes in addition to environment
variables, unlike `opts.expand_env=false`.
`opts.expand_env=false` is soft-deprecated.
refactor(lua): add integer coercion helpers
Add vim._tointeger() and vim._ensure_integer(), including optional base
support, and switch integer-only tonumber()/assert call sites in the Lua
runtime to use them.
This also cleans up related integer parsing in LSP, health, loader, URI,
tohtml, and Treesitter code.
supported by AI
Although powerful -- especially with chained modifiers --, the
readability (and therefore maintainability) of `fnamemodify()` and its
modifiers is often worse than a function name, giving less context and
having to rely on `:h filename-modifiers`. However, it is used plenty in
the Lua stdlib:
- 16x for the basename: `fnamemodify(path, ':t')`
- 7x for the parents: `fnamemodify(path, ':h')`
- 7x for the stem (filename w/o extension): `fnamemodify(path, ':r')`
- 6x for the absolute path: `fnamemodify(path, ':p')`
- 2x for the suffix: `fnamemodify(path, ':e')`
- 2x relative to the home directory: `fnamemodify(path, ':~')`
- 1x relative to the cwd: `fnamemodify(path, ':.')`
The `fs` module in the stdlib provides a cleaner interface for most of
these path operations: `vim.fs.basename` instead of `':t'`,
`vim.fs.dirname` instead of `':h'`, `vim.fs.abspath` instead of `':p'`.
This commit refactors the runtime to use these instead of fnamemodify.
Not all fnamemodify calls are removed; some have intrinsic differences
in behavior with the `vim.fs` replacement or do not yet have a
replacement in the Lua module, i.e. `:~`, `:.`, `:e` and `:r`.
Problem:
When using the Nvim appimage, `~/.cache/nvim/luac` directory can grow to
250,000+ files.
Example of 2 identical files in `./luac/`:
%2ftmp%2f.mount_nvim.a65Rja0%2fusr%2fshare%2fnvim%2fruntime%2flua%2fvim%2ftreesitter.luac
%2ftmp%2f.mount_nvim.aNpxXgo%2fusr%2fshare%2fnvim%2fruntime%2flua%2fvim%2ftreesitter.luac
Analysis:
The `nvim.appimage` mounts nvim at a different temporary path each time
it is invoked. The naming scheme of these cache files is random, which
defats the purpose of the cache creates N new files on every launch of
nvim.
Steps to reproduce:
1. install `nvim.appimage`
2. `mv ~/.cache/nvim/luac ~/.cache/nvim/luac.backup`
3. `nvim`
4. Observe contents of `~/.cache/nvim/luac/`
5. Close nvim and run `nvim` again
6. Observe contents of `~/.cache/nvim/luac/` and see that new identical
files have been added with a different mount prefix
Solution:
When running from an appimage, trim the random part of the filepaths.
Problem:
`vim.loader.disable` does not conform to `:help dev-name-common` and
`:help dev-patterns`.
Solution:
- Add `enable` parameter to `vim.loader.enable`
- Remove `vim.loader.disable`
- Note the change in `:help news-breaking-dev` (HEAD changes).
- This is not a breaking change (except to "HEAD") because
`vim.loader` is marked "experimental".
previous: 26765e8461
An implication of this current approach is that `NVIM_API_LEVEL` should be
bumped when a new Lua function is added.
TODO(future): add a lint check which requires `@since` on all new functions.
ref #25416
Problem:
`vim.fs.dirname([[C:\User\XXX\AppData\Local]])` returns "." on
mingw/msys2.
Solution:
- Check for "mingw" when deciding `iswin`.
- Use `has("win32")` where possible, it works in "fast" contexts since
b02eeb6a72.
Problem:
vim.fs.normalize() normalizes too much vim.loader and is slow.
Solution:
Make it faster by doing less. This reduces the times spent in
vim.fs.normalize in vim.loader from ~13ms -> 1-2ms.
Numbers from a relative benchmark:
- Skipping `vim.validate()`: 285ms -> 230ms
- Skipping `path_resolve_dot()`: 285ms -> 60ms
- Skipping `double_slash`: 60ms -> 35ms
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
Problem: Loading `vim.fs` via the `vim.loader` Lua package loader will
result in a stack overflow due to a cyclic dependency. This may happen
when the `vim.fs` module isn't byte-compiled, i.e. when `--luamod-dev`
is used (#27413).
Solution: `vim.loader` depends on `vim.fs`. Therefore `vim.fs` should
be loaded in advance.
Problem: luals returns stricter diagnostics with bundled luarc.json
Solution: Improve some function and type annotations:
* use recognized uv.* types
* disable diagnostic for global `vim` in shared.lua
* docs: don't start comment lines with taglink (otherwise LuaLS will interpret it as a type)
* add type alias for lpeg pattern
* fix return annotation for `vim.secure.trust`
* rename local Range object in vim.version (shadows `Range` in vim.treesitter)
* fix some "missing fields" warnings
* add missing required fields for test functions in eval.lua
* rename lsp meta files for consistency
Problem: cache paths are derived by replacing each reserved/filesystem-
path-sensitive char with a `%` char in the original path. With this
method, two different files at two different paths (each containing `%`
chars) can erroneously resolve to the very same cache path in certain
edge-cases.
Solution: derive cache paths by url-encoding the original (path) instead
using `vim.uri_encode()` with `"rfc2396"`. Increment `Loader.VERSION` to
denote this change.
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Enforce consistent terminology (defined in
`gen_help_html.lua:spell_dict`) for common misspellings.
This does not spellcheck English in general (perhaps a future TODO,
though it may be noisy).
* feat(lua): vim.tbl_contains supports general tables and predicates
Problem: `vim.tbl_contains` only works for list-like tables (integer
keys without gaps) and primitive values (in particular, not for nested
tables).
Solution: Rename `vim.tbl_contains` to `vim.list_contains` and add new
`vim.tbl_contains` that works for general tables and optionally allows
`value` to be a predicate function that is checked for every key.
Whenever we run fs_stat() on a path, save this information in the loader
so it can be re-used.
- Loader.loadfile: Remove arguments `hash` as it is no longer needed.
- Loader.loader: Use _G.loadstring instead of Loader.load
This allows plugins to wrap loadstring to inspection and profiling
- factor out read file logic
