vim-patch:9.1.0089: qsort() comparison functions should be transitive

Problem:  qsort() comparison functions should be transitive
Solution: Do not subtract values, but rather use explicit comparisons

Improve qsort() comparison functions

There has been a recent report on qsort() causing out-of-bounds read &
write in glibc for non transitive comparison functions
https://www.qualys.com/2024/01/30/qsort.txt

Even so the bug is in glibc's implementation of the qsort() algorithm,
it's bad style to just use substraction for the comparison functions,
which may cause overflow issues and as hinted at in OpenBSD's manual
page for qsort(): "It is almost always an error to use subtraction to
compute the return value of the comparison function."

So check the qsort() comparison functions and change them to be safe.

closes: vim/vim#13980

e06e437665

Co-authored-by: Christian Brabandt <cb@256bit.org>
This commit is contained in:
zeertzjq
2024-02-10 21:26:54 +08:00
parent 71429c90ee
commit 5bbb733a1b
5 changed files with 29 additions and 13 deletions

View File

@@ -2792,8 +2792,10 @@ static int tv_nr_compare(const void *a1, const void *a2)
{
const listitem_T *const li1 = tv_list_first(*(const list_T **)a1);
const listitem_T *const li2 = tv_list_first(*(const list_T **)a2);
const varnumber_T n1 = TV_LIST_ITEM_TV(li1)->vval.v_number;
const varnumber_T n2 = TV_LIST_ITEM_TV(li2)->vval.v_number;
return (int)(TV_LIST_ITEM_TV(li1)->vval.v_number - TV_LIST_ITEM_TV(li2)->vval.v_number);
return n1 == n2 ? 0 : n1 > n2 ? 1 : -1;
}
/// "setcellwidths()" function