According to `CaseFolding-15.1.0.txt`, full casefolding should be
preferred over simple casefolding as it's considered to be more correct.
Since utf8proc already provides full casefolding it makes sense to
switch to it. This will also remove a lot of unnecessary build code.
Temporary exceptions are made for two sets characters:
- `ß` will still be considered `ß` (instead of `ss`) as using a full
casefolding requires interfering with upstream spell files in some
form.
- `İ` will still be considered `İ` (instead of `i̇`) as using full
casefolding requires making a value judgement on the "correct"
behavior. There are two, equally valid case-insensetive comparison for
this character according to unicode. It is essentially up to the
implementor to decide which conversion is correct. For this reason it
might make sense to allow users to decide which conversion should be
done as an added option to `casemap` in a future PR.
Problem: Code related to findfile() is spread out.
Solution: Put findfile() related code into a new source file. (Yegappan
Lakshmanan, closesvim/vim#3934)
5fd0f5052f
Keep functions related to wildcard expansion in path.c, as in Vim they
are now spread out among multiple files, which isn't really ideal.
Problem: filecopy() may return wrong value when readlink() fails.
Solution: Set ret to -1 so that 0 is returned when readlink() fails.
(zeertzjq)
closes: vim/vim#15438da090f95df
In Vim a lot of filesystem functions have been moved to filepath.c.
However, some of these functions actually deal with file contents, and
Nvim's filesystem-related functions are spread out in a different way.
Therefore, it's better to use a different file for these functions.
Since #29315 we are also preprocessing header files in order to find
functions which need prototype declarations. gcc emits a spurious
"warning: #pragma once in main file" even when when you are just
using `gcc -E` which causes a bit of noise in compiler output.
As a workaround, surpress all warnings for this step, this should be pretty
much harmless as we will still get preprocessor warnings when doing
actual compilation `gcc -c` later.
reference: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89808
This also makes shada reading slightly faster due to avoiding
some copying and allocation.
Use keysets to drive decoding of msgpack maps for shada entries.
- common_function() has always been in evalfunc.c in Vim
- return_register() has always been in evalfunc.c in Vim
- get_user_input() was moved to ex_getln.c in Vim 8.1.1957
- tv_get_lnum_buf() was moved to typval.c in Vim 8.2.0847
When the cursor is moved we terminate any active OSC 8 sequences to
prevent the sequence from inadvertently spanning regions it is not meant
to span. However, if we do not also reset the TUI's active attr id
(print_attr_id) then the TUI does not "know" that it's current attribute
set has changed. When cursor_goto is called to wrap a line, the TUI does
not recompute the attributes so the OSC 8 sequence is not restarted
again.
When we terminate an OSC 8 sequence before moving the cursor, also reset
the active attr id so that the attributes are recomputed for URLs.
Problem: Wrong comment for "len" argument of call_simple_func().
Solution: Remove the "or -1 to use strlen()". Also change its type to
size_t to remove one cast. (zeertzjq)
closes: vim/vim#15410c1ed788c1b
Problem: Calling a function from an "expr" option has too much overhead.
Solution: Add call_simple_func() and use it for 'foldexpr'
87b4e5c5db
Cherry-pick a call_func() change from patch 8.2.1343.
Add expr-option-function docs to options.txt.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Vim9: using a script-local function requires using "s:" when
setting 'completefunc'.
Solution: Do not require "s:" in Vim9 script. (closesvim/vim#9796)
1fca5f3e86
vim-patch:8.2.4417: using NULL pointer
Problem: Using NULL pointer.
Solution: Set offset after checking for NULL pointer.
e89bfd212b
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: too many strlen() calls in syntax.c
Solution: refactor code to reduce the number or strlen() calls,
get rid of un-used SYN_NAMELEN macro
(John Marriott)
closes: vim/vim#15368b4ea77185c
Co-authored-by: John Marriott <basilisk@internode.on.net>
Problem: ex: trailing dot is optional for :g and :insert/:append
Solution: don't break out early, when the next command is empty.
(Mohamed Akram)
The terminating period is optional for the last command in a global
command list.
closes: vim/vim#154070214680a8e
Co-authored-by: Mohamed Akram <mohd.akram@outlook.com>
Problem: [security] use-after-free in tagstack_clear_entry
(Suyue Guo )
Solution: Instead of manually calling vim_free() on each of the tagstack
entries, let's use tagstack_clear_entry(), which will
also free the stack, but using the VIM_CLEAR macro,
which prevents a use-after-free by setting those pointers
to NULL
This addresses CVE-2024-41957
Github advisory:
https://github.com/vim/vim/security/advisories/GHSA-f9cr-gv85-hcr48a0bbe7b8a
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: Coverity warning in cstrncmp()
(after v9.1.0645)
Solution: Change the type of n2 to int.
(zeertzjq)
________________________________________________________________________________________________________
*** CID 1615684: Integer handling issues (INTEGER_OVERFLOW)
/src/regexp.c: 1757 in cstrncmp()
1751 n1 -= mb_ptr2len(s1);
1752 MB_PTR_ADV(p);
1753 n2++;
1754 }
1755 // count the number of bytes to advance the same number of chars for s2
1756 p = s2;
>>> CID 1615684: Integer handling issues (INTEGER_OVERFLOW)
>>> Expression "n2--", which is equal to 18446744073709551615, where "n2" is known to be equal to 0, underflows the type that receives it, an unsigned integer 64 bits wide.
1757 while (n2-- > 0 && *p != NUL)
1758 MB_PTR_ADV(p);
1759
1760 n2 = p - s2;
1761
1762 result = MB_STRNICMP2(s1, s2, *n, n2);
closes: vim/vim#15409e8feaa354e
Problem: Cannot use an autoload function from a package under start.
Solution: Also look in the "start" package directory. (Bjorn Linse,
closesvim/vim#7193)
223a950a85
Nvim already does this in do_in_cached_path(), and this change has no
effect in Nvim as Nvim removes DIP_START after do_in_cached_path().
Accidentally failed to mark as ported:
vim-patch:8.2.1731: Vim9: cannot use += to append to empty NULL list
Co-authored-by: bfredl <bjorn.linse@gmail.com>
Problem: eval5() and eval7 are too complex
Solution: Refactor eval5() and eval7() in eval.c
(Yegappan Lakshmanan)
closes: vim/vim#14900734286e4c6
Co-authored-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Problem:
`'scrollbind'` does not work properly if the window being scrolled
automatically contains any filler/virtual lines (except for diff filler
lines).
This is because when the scrollbind check is done, the logic only
considers changes to topline which are represented as line numbers.
Solution:
Write the logic for determine the scroll amount to take into account
filler/virtual lines.
Fixes#29751
Problem: Coverity warning after 9.1.0440
Solution: Fix Coverity warning, add a test and
reduce the calls to clear_tv()
(Yegappan Lakshmanan).
closes: vim/vim#14845dbac0da631
Co-authored-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Problem: Some functions are not tested
Solution: Add a few more tests, fix a few minor problems
(Yegappan Lakshmanan)
closes: vim/vim#14789fe424d13ef
Co-authored-by: Yegappan Lakshmanan <yegappan@yahoo.com>
Problem: Cannot use a method with a complex expression.
Solution: Evaluate the expression after "->" and use the result.
c665dabdf4
Cherry-pick a "verbose" check from patch 8.2.4123.
N/A patches for version.c:
vim-patch:8.2.4102: Vim9: import cannot be used after method
vim-patch:8.2.4110: Coverity warns for using NULL pointer
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Vim9: cannot use += to append to empty NULL list.
Solution: Copy the list instead of extending it. (closesvim/vim#6998)
81ed496048
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: regex: wrong match when searching multi-byte char
case-insensitive (diffsetter)
Solution: Apply proper case-folding for characters and search-string
This patch does the following 4 things:
1) When the regexp engine compares two utf-8 codepoints case
insensitive it may match an adjacent character, because it assumes
it can step over as many bytes as the pattern contains.
This however is not necessarily true because of case-folding, a
multi-byte UTF-8 character can be considered equal to some
single-byte value.
Let's consider the pattern 'ſ' and the string 's'. When comparing and
ignoring case, the single character 's' matches, and since it matches
Vim will try to step over the match (by the amount of bytes of the
pattern), assuming that since it matches, the length of both strings is
the same.
However in that case, it should only step over the single byte value
's' by 1 byte and try to start matching after it again. So for the
backtracking engine we need to ensure:
* we try to match the correct length for the pattern and the text
* in case of a match, we step over it correctly
There is one tricky thing for the backtracing engine. We also need to
calculate correctly the number of bytes to compare the 2 different
utf-8 strings s1 and s2. So we will count the number of characters in
s1 that the byte len specified. Then we count the number of bytes to
step over the same number of characters in string s2 and then we can
correctly compare the 2 utf-8 strings.
2) A similar thing can happen for the NFA engine, when skipping to the
next character to test for a match. We are skipping over the regstart
pointer, however we do not consider the case that because of
case-folding we may need to adjust the number of bytes to skip over.
So this needs to be adjusted in find_match_text() as well.
3) A related issue turned out, when prog->match_text is actually empty.
In that case we should try to find the next match and skip this
condition.
4) When comparing characters using collections, we must also apply case
folding to each character in the collection and not just to the
current character from the search string. This doesn't apply to the
NFA engine, because internally it converts collections to branches
[abc] -> a\|b\|c
fixes: vim/vim#14294closes: vim/vim#1475622e8e12d9f
N/A patches:
vim-patch:9.0.1771: regex: combining chars in collections not handled
vim-patch:9.0.1777: patch 9.0.1771 causes problems
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: Illegal memory access when pattern starts with illegal byte.
Solution: Do not match a character with an illegal byte.
f50940531d
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem:
Variables are often assigned multiple places in common patterns.
Solution:
Replace these common patterns with different patterns that reduce the
number of assignments.
Use `MAX` and `MIN`:
```c
if (x < y) {
x = y;
}
// -->
x = MAX(x, y);
```
```c
if (x > y) {
x = y;
}
// -->
x = MIN(x, y);
```
Use ternary:
```c
int a;
if (cond) {
a = b;
} els {
a = c;
}
// -->
int a = cond ? b : c;
```
Problem: Cannot use items() on a string.
Solution: Make items() work on a string. (closesvim/vim#11016)
3e518a8ec7
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: items() does not work on a list. (Sergey Vlasov)
Solution: Make items() work on a list. (closesvim/vim#11013)
976f859763
Skip CHECK_LIST_MATERIALIZE.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Check that mapping rhs starts with lhs doesn't work if lhs is
not simplified.
Solution: Keep track of the mapblock containing the alternative lhs and
also compare with it (zeertzjq).
fixes: vim/vim#15376closes: vim/vim#153849d997addc7
Cherry-pick removal of save_m_str from patch 8.2.4059.
Problem: E1510 may happen when formatting a message
(after 9.1.0181).
Solution: Only give E1510 when using typval. (zeertzjq)
closes: vim/vim#153910dff31576a
Problem: Rename of pum hl_group is incomplete in source.
Solution: Also rename the test function. Rename to user_hlattr in code
to avoid confusion with pum_extra. Add test with matched text
highlighting (zeertzjq).
closes: vim/vim#153484100852e09
