Problem:
bw_rest was used as an extra buffer to save incomplete byte sequences
between calls to buf_write_bytes. Besides being unnecessarily
complicated, this introduced a number of issues:
1) The bytes stored in bw_rest could still be there at the end of
writing the file, never having been written, thus losing some of the
file content on write.
2) bw_rest was not cleared out after the "checking_conversion" phase,
leaving them to affect the written file content during the writing
phase, corrupting the file.
3) bw_rest could contain extra bytes that need to be written to the
output buffer during a buf_write_convert call, potentially before any
bytes are consumed. But some conversions are in-place, without a
separate output buffer. Writing bytes from bw_rest to the "output"
buffer actually overwrote bytes from the input buffer before they were
read, corrupting the data to be written.
4) The extra bytes in bw_rest that need to be written to the conversion
output buffer were not originally accounted for in the size calculation
for the output buffer, causing a buffer overflow (previously fixed in
Vim patch 9.1.2028).
Solution:
Rather than maintaining a separate buffer, the unconverted bytes at the
end of the buffer can just be shifted to the beginning of the buffer,
and the buffer size updated. This requires a bit of refactoring, and
buf_write_convert and buf_write_convert_with_iconv need to report the
number of bytes they consumed so that buf_write_bytes can handle the
remaining bytes.
Following conversion, bw_buf can be checked for any remaining bytes.
Leftover bytes in this case result in a conversion error, which is
better than silently dropping them.
A short section of dead code was removed from buf_write_convert, for
converting a non-UTF-8 buffer to UTF-8. Neovim buffers are always UTF-8.
A few additional tests for iconv conversions have been added. Vim's
iconv tests are disabled in Neovim because they use unsupported values
for 'encoding'.
Problem:
Iter:peek() only works if the iterator is a |list-iterator| (internally, an `ArrayIter`).
However, it is possible to implement :peek() support for any iterator.
Solution:
- add `_peeked` buffer for lookahead without actually consuming values
- `peek()` now works for function, pairs(), and array iterators
- `skip(predicate)` stops at the first non matching element without consuming it
- keep existing optimized behavior for `ArrayIter` to maintain backward compatibility
- use `pack`/`unpack` to support iterators that return multiple values
fix(treesitter): more distinctive highlight for EditQuery captures
Problem: EditQuery shows captures in the source buffer using the Title
highlight group, which could be too similar to Normal.
Solution: Use a virtual text diagnostic highlight group: they are
displayed in a similar manner to the query captures so we can assume
that the color scheme should have appropriate styling applied to make
them visible.
Problem: Changing terminal height immediately after outputting lines
may lead to wrong scrollback.
Solution: Insert pending scrollback lines before the old window height.
TermCursor already has cterm=reverse. Additionally, now that terminal
buffers have a real cursor, the cterm=reverse in TermCursor no longer
shows up in the screen state.
Problem:
`vim.json.decode()` could not parse JSONC (JSON with Comments)
extension, which is commonly used in configuration files.
Solution:
Introduce an `skip_comments` option, which is disabled by default. When
enabled, allows JavaScript-style comments within JSON data.
Problem:
Presumably Neovim contributors use Neovim to write the contributed code.
Chances are that they have some local Neovim-source-specific config, for
example setting the $TEST_FILE and 'makeprg' for files in test/.
Solution:
Ignore it.
Problem: Crash on failed sockconnect() if a new connection is accepted
while polling for uv events.
Solution: Don't use channel_destroy_early().
Also test "tcp" mode failure properly.
Problem:
Escaping logic for {subject} in ex cmd `:help {subject}` is done in a
messy 200+ lines C function which is hard to maintain and improve.
Solution:
Rewrite in Lua. Use `string.gsub()` instead of looping over characters
to improve clarity and add many more tests to be able to confidently
improve current code later on.
Problem: cterm field in Dict(highlight) is declared as Union(Integer, String)
but it actually expects a Dict(highlight_cterm).
Solution: change cterm type to DictAs(highlight__cterm) and simplify the
handling in dict2hlattrs since type validation and empty array compat are
already handled by api_dict_to_keydict.
Problem: Tests for deprecated highlight API were mixed with current API tests.
Solution: Move them to deprecated_spec.lua and update highlight_spec.lua to use nvim_get_hl.
In particular, also mention the difference between the regex atom \k and
what Vim considers for a word character.
closes: vim/vim#186889e456e52df
Co-authored-by: Christian Brabandt <cb@256bit.org>
The set_by_lua_block directive of the Lua module takes an additional
variable as an argument which currently breaks the detection of inline
Lua blocks. For example:
set_by_lua_block $myvar {
return tonumber(ngx.var.myothervar)-1
}
closes: vim/vim#19362e92998ea4d
Co-authored-by: Josef Schönberger <josef.schoenberger@tum.de>
Problem: Wrong scrollback when passing a buffer with many lines to
nvim_open_term().
Solution: Delete all buffer lines before opening the terminal.
I tried to use buf_clear(), but it crashes inside deleted_lines_mark(),
so I'll just use deleted_lines_buf() for now. The behavior of marks can
be decided later.
Problem:
After eaacdc9, complete with emmylua_ls error with:
runtime/lua/vim/lsp/completion.lua:586: attempt to get length of field
'items' (a nil value)
Solution:
Result can be CompletionItem[] according the spec:
> If a `CompletionItem[]` is provided, it is interpreted to be complete,
> so it is the same as `{ isIncomplete: false, items }`
Problem:
Mappings with a control modifier follow the (tag) format of:
{key_before}_CTRL-{key}_{key_after}
Where `{key_before}` and `{key_after}` can be any other key combination.
However, for the Nvim default mappings `[_CTRL-Q`, `]_CTRL-Q`,
`[_CTRL-L`, `]_CTRL-L`, `[_CTRL-T` and `]_CTRL-T`, the underscore
between the bracket and CTRL was absent. This lead to problems finding
the relevant docs with `:help [_CTRL-Q` and made parsing the {subject}
argument for `:help` harder.
Solution:
Use the right tag format.
Problem: heap use-after-free if OptionSet autocommands from setting a terminal's
&buftype wipe the buffer.
Solution: set b_locked during OptionSet for &buftype.
Problem: Terminal doesn't handle ED 3 (clear scrollback) properly.
Solution: Add vterm callback for sb_clear().
Also fix another problem that scrollback lines may be duplicated when
pushing to scrollback immediately after reducing window height, as can
be seen in the changes to test/functional/terminal/window_spec.lua.
- Change syntax file maintainer.
- Add Guile and Python command highlighting.
- Update command list to version 12.
- Add foldable regions for the commands 'define', 'if' and 'while'
multiline commands.
- Support documented partial command names.
- Add matchit, browsefilter, and comment formatting support.
- Support embedded C in compiler {code|print} commands.
- Add largely complete settings highlighting and folding.
- Add syntax tests (incomplete).
Thanks to Claudio Fleiner for many years of maintenance.
closes: vim/vim#10649b422a33ac2
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Problem: TermResponse deferred due to blocked autocommands lacks "data" payload.
Also, it may not fire if a new v:termresponse reuses the same string address.
Solution: add it. Use the value of v:termresponse for "data.sequence". Replace
pointer comparisons with a flag.
The removal of "old_termresponse" comparisons is required to pass the test on
the CI, or locally for me when compiled in RelWithDebInfo.
Problem:
Empty response affects server start boundary computed before.
Solution:
Ignore empty responses. This is mostly micro-optimization that avoids
extending existing results with empty responses.
Problem:
Diagnostic lifecycle invariants (clearing on empty publish and buffer
deletion) were previously implicit and not directly covered by functional
tests, allowing regressions to go unnoticed.
Solution:
Add functional regression tests asserting that diagnostics are cleared
when an LSP server publishes an empty diagnostic set and when the
associated buffer is deleted. Assertions are scoped to the client
diagnostic namespace and use public diagnostic APIs only.
Problem: Code lenses currently display as virtual text on the same line
and after the relevant item. While the spec does not say how lenses
should be rendered, above the line is most typical. For longer lines,
lenses rendered as virtual text can run off the side of the screen.
Solution: Display lenses as virtual lines above the text.
Closes https://github.com/neovim/neovim/issues/33923
Co-authored-by: Yi Ming <ofseed@foxmail.com>
Problem: if buf_free_all autocommands open a terminal, it will remain open after
the buffer is freed.
Solution: close terminals again later, this time while blocking autocommands.
Did consider terminal_open checking stuff like b_locked_split instead, but
that's set during BufHidden, etc., which doesn't mean the buffer's being wiped.
Problem: close_buffer autocmds may switch buffers at the last moment when
closing a window, causing terminal_check_size to prefer the size of a closed
window, or TabClosed to set an old <abuf>.
Solution: use the actual last buffer, similar to what TabClosed did before.
NOTE: If buffer was unloaded/deleted (not wiped), then TabClosed's <abuf> may
not use it. (w_buffer = NULL) Maybe odd, but it's how it worked before anyhow.
Relies on close_buffer reliably setting w_buffer to NULL if freed, otherwise
buf_valid is better. Only concern I see is if the window wasn't in the window
list after closing the buffer (close_buffer won't set it to NULL then), but then
win_close{_othertab} should've returned earlier.
