Problem:
Detection of the pynvim module is currently done by finding the first
Python interpreter in the `PATH` and checking if it can import pynvim.
This has several problems:
- Activation of an unrelated Python virtual environment will break
automatic detection, unless pynvim is also installed in that
environment.
- Installing pynvim to the expected location is difficult. User
installation into the system-wide or user-wide Python site area is now
deprecated. On Ubuntu 24.04 with Python 3.12, for example, the
command `pip install --user pynvim` now fails with the error message
`error: externally-managed-environment`.
- Users may create a dedicated virtual environment in which to install
pynvim, but Nvim won't detect it; instead, they must either activate
it before launching Nvim (which interferes with the user of other
virtual environments) or else hard-code the variable
`g:python3_host_prog` in their `init.vim` to the path of the correct
Python interpreter. Neither option is desirable.
Solution:
Expose pynvim's Python interpreter on the `PATH` under the
name `pynvim-python`. Typical user-flow:
1. User installs either uv or pipx.
2. User installs pynvim via:
```
uv tool install --upgrade pynvim
# Or:
pipx install --upgrade pynvim
```
With corresponding changes in pynvim https://github.com/neovim/pynvim/issues/593
the above user-flow is all that's needed for Nvim to detect the
installed location of pynvim, even if an unrelated Python virtual
environments is activated. It uses standard Python tooling to automate
the necessary creation of a Python virtual environment for pyenv and the
publication of `pynvim-python` to a directory on `PATH`.
Problem:
Additional include directories in DEPS_INCLUDE_FLAGS variable are not
quoted. Paths with spaces break the resulting compile command.
Solution:
Enclose values in double quotes.
Note: normally we should avoid manual quoting, but in this case we can't
because of how `DEPS_INCLUDE_FLAGS` is used in `BuildLuv.cmake`
and `BuildLpeg.cmake`.
Problem: Buffer menu does not handle unicode names correctly
(after v9.1.1622)
Solution: Fix the BMHash() function (Yee Cheng Chin)
The Buffers menu uses a BMHash() function to generate a sortable number
to be used for the menu index. It used a naive (and incorrect) way of
encoding multiple ASCII values into a single integer, but assumes each
character to be only in the ASCII 32-96 range. This means if we use
non-ASCII file names (e.g. Unicode values like CJK or emojis) we get
integer underflow and overflow, causing the menu index to wrap around.
Vim's GUI implementations internally use a signed 32-bit integer for the
`gui_mch_add_menu_item()` function and so we need to make sure the menu
index is in the (0, 2^31-1) range.
To do this, if the file name starts with a non-ASCII value, we just use
the first character's value and set the high bit so it sorts after the
other ASCII ones. Otherwise, we just take the first 5 characters, and
use 5 bit for each character to encode a 30-bit number that can be
sorted.
This means Unicode file names won't be sorted beyond the first
character. This is likely going to be fine as there are lots of ways to
query buffers.
related: vim/vim#17403closes: vim/vim#179288f9de4991e
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
Problem: tests: fuzzy buffer name completion test doesn't match
successfully (after 9.1.1627).
Solution: Update pattern to account for the change in case sensitivity.
Also mark Test_search_stat_option() as flaky as it can still
sometimes fail (zeertzjq).
closes: vim/vim#17992891353671a
Some terminals support the CSI 8 t sequence to resize their own window
area. Neovim will emit this sequence when the TUI is resized
programatically, but it should not be emitted when the TUI is resized
because the host terminal was resized, as that results in an infinite
resize loop.
Don't print ABI version of duplicated parsers that are later in the
runtime path (see [#35326]).
Change the sorting from `name > path` to `name > rtpath_index`, this
ensures the first (loaded) parser is first in the list and any
subsequent parsers can be considered "not loaded".
This is fuzzy at best since `vim.treesitter.language.add` can take a
path to a parser and change the load order.
The correct solution is for `vim.treesitter.language.inspect` to return
the parser path so we can compare against it and/or for it to also be
able to take a path to a parser so we can inspect it without loading it
first.
These are not needed after #35129 but making uncrustify still play nice
with them was a bit tricky.
Unfortunately `uncrustify --update-config-with-doc` breaks strings
with backslashes. This issue has been reported upstream,
and in the meanwhile auto-update on every single run has been disabled.
These are special names by convention, and giving them distinct
highlighting is a nice visual clue (using Identifier by default).
This group is named "pythonClassVar" to match the name used by
python-syntax. Some third-party color schemes are aware of this
name and customized their colors accordingly.
closes: vim/vim#179681ee1d9b43d
Co-authored-by: Jon Parise <jon@indelible.org>
Problem: fuzzy.c has a few issues
Solution: Use Vims memory management, update style
(glepnir)
Problem:
- Missing cleanup of lmatchpos lists causing memory leaks
- Missing error handling for list operations
- Use of malloc() instead of Vim's alloc() functions
- Inconsistent C-style comments
- Missing null pointer checks for memory allocation
- Incorrect use of vim_free() for list objects
Solution:
- Add proper cleanup of lmatchpos in done section using list_free()
- Set lmatchpos to NULL after successful transfer to avoid confusion
- Add error handling for list_append_tv() failures
- Replace malloc() with alloc() and add null pointer checks
- Convert C-style comments to C++ style for consistency
- Fix vim_free() calls to use list_free() for list objects
closes: vim/vim#1798417a6d696bd
Co-authored-by: glepnir <glephunter@gmail.com>
Problem: fuzzy-matching can be improved
Solution: Implement a better fuzzy matching algorithm
(Girish Palya)
Replace fuzzy matching algorithm with improved fzy-based implementation
The
[current](https://www.forrestthewoods.com/blog/reverse_engineering_sublime_texts_fuzzy_match/)
fuzzy matching algorithm has several accuracy issues:
* It struggles with CamelCase
* It fails to prioritize matches at the beginning of strings, often
ranking middle matches higher.
After evaluating alternatives (see my comments
[here](https://github.com/vim/vim/issues/17531#issuecomment-3112046897)
and
[here](https://github.com/vim/vim/issues/17531#issuecomment-3121593900)),
I chose to adopt the [fzy](https://github.com/jhawthorn/fzy) algorithm,
which:
* Resolves the aforementioned issues.
* Performs better.
Implementation details
This version is based on the original fzy
[algorithm](https://github.com/jhawthorn/fzy/blob/master/src/match.c),
with one key enhancement: **multibyte character support**.
* The original implementation supports only ASCII.
* This patch replaces ascii lookup tables with function calls, making it
compatible with multibyte character sets.
* Core logic (`match_row()` and `match_positions()`) remains faithful to
the original, but now operates on codepoints rather than single-byte
characters.
Performance
Tested against a dataset of **90,000 Linux kernel filenames**. Results
(in milliseconds) show a **\~2x performance improvement** over the
current fuzzy matching algorithm.
```
Search String Current Algo FZY Algo
-------------------------------------------------
init 131.759 66.916
main 83.688 40.861
sig 98.348 39.699
index 109.222 30.738
ab 72.222 44.357
cd 83.036 54.739
a 58.94 62.242
b 43.612 43.442
c 64.39 67.442
k 40.585 36.371
z 34.708 22.781
w 38.033 30.109
cpa 82.596 38.116
arz 84.251 23.964
zzzz 35.823 22.75
dimag 110.686 29.646
xa 43.188 29.199
nha 73.953 31.001
nedax 94.775 29.568
dbue 79.846 25.902
fp 46.826 31.641
tr 90.951 55.883
kw 38.875 23.194
rp 101.575 55.775
kkkkkkkkkkkkkkkkkkkkkkkkkkkkk 48.519 30.921
```
```vim
vim9script
var haystack = readfile('/Users/gp/linux.files')
var needles = ['init', 'main', 'sig', 'index', 'ab', 'cd', 'a', 'b',
'c', 'k',
'z', 'w', 'cpa', 'arz', 'zzzz', 'dimag', 'xa', 'nha', 'nedax',
'dbue',
'fp', 'tr', 'kw', 'rp', 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkk']
for needle in needles
var start = reltime()
var tmp = matchfuzzy(haystack, needle)
echom $'{needle}' (start->reltime()->reltimefloat() * 1000)
endfor
```
Additional changes
* Removed the "camelcase" option from both matchfuzzy() and
matchfuzzypos(), as it's now obsolete with the improved algorithm.
related: neovim/neovim#34101fixesvim/vim#17531closes: vim/vim#179007e0df5eee9
Co-authored-by: Girish Palya <girishji@gmail.com>
Problem: C-indent does not handle compound literals
(@44100hertz, @Jorenar)
Solution: Detect and handle compound literal and structure
initialization (Anttoni Erkkilä)
match '=' or "return" optionally followed by &, (typecast), {
Fixes also initialization which begins with multiple opening braces.
fixes: vim/vim#2090fixes: vim/vim#12491closes: vim/vim#178655ba6e41d37
Co-authored-by: Anttoni Erkkilä <anttoni.erkkila@protonmail.com>
Problem: Autocompletion slow with include- and tag-completion
Solution: Refactor ins_compl_interrupted() to also check for timeout,
further refactor code to skip outputting message when
performing autocompletion (Girish Palya).
Running `vim *` in `vim/src` was slower than expected when
'autocomplete' was enabled. Include-file and tag-file completion
sources were not subject to the timeout check, causing unnecessary
delays.
So apply the timeout check to these sources as well, improving
autocompletion responsiveness, refactor find_pattern_in_path() to take
an additional "silent" argument, to suppress any messages.
closes: vim/vim#1796659e1d7f353
Co-authored-by: Girish Palya <girishji@gmail.com>
BUILD.md: Add git to prerequisites
The git command is literally the first one in the build instructions, therefore it's reasonable to treat it as one of the prerequisites. Void Linux already had git as one of the prerequisites; this commits adds git to all the other Unix systems.
Problem: parse_cmdline() sets eap->cmdlinep to address of local parameter,
causing invalid memory access when expand_filename() tries to modify it.
This leads to crashes when typing '%' in user commands with preview=true
and complete=file.
Solution: Change parse_cmdline() signature to accept char **cmdline,
allowing cmdlinep to point to caller's variable for safe reallocation.
A file containing only async functions (`async def func()`) wouldn't
previously match the pythonSync pattern.
Also, this pattern only matches at the beginning of the line, so it
won't ever match method definitions (which are indented within class
scopes). Update the comment accordingly.
closes: vim/vim#17963dba9eb46e6
Co-authored-by: Jon Parise <jon@indelible.org>
problem: most shada entries use weird `PossiblyFreedShadaEntry` type
solution: delet it
Shada entries can either be allocated by shada.c when reading,
or be constructed to represent the state of the current instance,
with direct references to live instance data to avoid extra allocations.
shada.c needs to carefully only free memory allocated by the first case,
and not free memory owned by other subsystems.
In some part of the code, this is inferred by the context but in others
we are mixing entries from different sources and need to indicate
the provenance by a `can_free_entry` flag. However constantly
frontloading this distinction in the name of the type and with
extra nesting levels, cause extra cognitive overhead when trying
to understand the code in any other aspects than the specific detail
of avoiding leaks/double frees.
As we always know if the memory is owned or not for any entry, we
can just put `can_free_entry` directly on the ShadaEntry struct.
That only one state is possible in a given context, is indicated
by this neat little syntactical construct called a constant field
initializer.
Tested using cross-compiling from linux:
zig build -Dcross=true -Dtarget=x86_64-windows nvim_bin
Note: not fully functional without a runtime, which still has to be
fuddled with manually
Macos and windows builds require a recent zig 0.15+dev version
As this zig master branch is currently too much in flux, we can't make
our CI depend on zig master.
Revisit CI after zig 0.15 release or at least feature freeze.
Problem:
During preview, the `input` still prompts the user to enter something
that won't be used later, which could be a bit confusing.
e.g., `:s/a/\=input("")`.
Solution:
Make the input() return early during 'inccommand' preview.
Problem: `vim.diagnostic.get_prev()` / `vim.diagnostic.get_next()` use
logical diagnostic positions and consider extmark validity.
`vim.diagnostic.get()` only uses original positions and doesn't care
about extmark validity. This results in inconsistency between these
APIs.
Solution: use original positions in `vim.diagnostic.get_prev()` and
`vim.diagnostic.get_next()` and don't consider extmark validity to match
previous behavior, which is consistent with `vim.diagnostic.get`.
Problem:
gen_help_html.lua script misinterprets parts of ASCII diagrams as help tags
(e.g., `|_________|` in `usr_28.txt`). This incorrectly triggered
special alignment-fixing logic that is meant for columnar text.
Signed-off-by: Shashwat Agrawal <shashwatagrawal473@gmail.com>
Problem: Temporary cmdline config is saved to be restored later.
Solution: Close the cmdline window so that it is recreated with the appropriate config.
Problem: Byte2line() does not work correctly with text properties. (Billie
Cleek)
Solution: Take the bytes of the text properties into account.
(closesvim/vim#5334)
9df53b62de
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Listener callback called at the wrong moment
Solution: Invoke listeners before calling ml_delete_int(). (closesvim/vim#4657)
acf7544cf6
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Listener callback called for the wrong buffer.
Solution: Invoke listeners before calling ml_append_int().
250e3112c6
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Highlight f-string replacement fields, including
- Comments
- Debugging flags
- Conversion fields
- Format specifications
- Delimiters
Syntax inside fields will be addressed in a separate commit.
related: vim/vim#10734
related: vim/vim#14033closes: vim/vim#17784a94a0555d9
Co-authored-by: Rob B <github@0x7e.net>
Class and function definitions previously shared a single highlight
group (pythonFunction). This change gives classes their own highlight
group (pythonClass) that's linked to Structure.
closes: vim/vim#1785648b7eb1ceb
Co-authored-by: Jon Parise <jon@indelible.org>
