This involves two redesigns of the map.c implementations:
1. Change of macro style and code organization
The old khash.h and map.c implementation used huge #define blocks with a
lot of backslash line continuations.
This instead uses the "implementation file" .c.h pattern. Such a file is
meant to be included multiple times, with different macros set prior to
inclusion as parameters. we already use this pattern e.g. for
eval/typval_encode.c.h to implement different typval encoders reusing a
similar structure.
We can structure this code into two parts. one that only depends on key
type and is enough to implement sets, and one which depends on both key
and value to implement maps (as a wrapper around sets, with an added
value[] array)
2. Separate the main hash buckets from the key / value arrays
Change the hack buckets to only contain an index into separate key /
value arrays
This is a common pattern in modern, state of the art hashmap
implementations. Even though this leads to one more allocated array, it
is this often is a net reduction of memory consumption. Consider
key+value consuming at least 12 bytes per pair. On average, we will have
twice as many buckets per item.
Thus old implementation:
2*12 = 24 bytes per item
New implementation
1*12 + 2*4 = 20 bytes per item
And the difference gets bigger with larger items.
One might think we have pulled a fast one here, as wouldn't the average size of
the new key/value arrays be 1.5 slots per items due to amortized grows?
But remember, these arrays are fully dense, and thus the accessed memory,
measured in _cache lines_, the unit which actually matters, will be the
fully used memory but just rounded up to the nearest cache line
boundary.
This has some other interesting properties, such as an insert-only
set/map will be fully ordered by insert only. Preserving this ordering
in face of deletions is more tricky tho. As we currently don't use
ordered maps, the "delete" operation maintains compactness of the item
arrays in the simplest way by breaking the ordering. It would be
possible to implement an order-preserving delete although at some cost,
like allowing the items array to become non-dense until the next rehash.
Finally, in face of these two major changes, all code used in khash.h
has been integrated into map.c and friends. Given the heavy edits it
makes no sense to "layer" the code into a vendored and a wrapper part.
Rather, the layered cake follows the specialization depth: code shared
for all maps, code specialized to a key type (and its equivalence
relation), and finally code specialized to value+key type.
Adds new API helper macros `CSTR_AS_OBJ()`, `STATIC_CSTR_AS_OBJ()`, and `STATIC_CSTR_TO_OBJ()`, which cleans up a lot of the current code. These macros will also be used extensively in the upcoming option refactor PRs because then API Objects will be used to get/set options. This PR also modifies pre-existing code to use old API helper macros like `CSTR_TO_OBJ()` to make them cleaner.
This reduces the total number of khash_t instantiations from 22 to 8.
Make the khash internal functions take the size of values as a runtime
parameter. This is abstracted with typesafe Map containers which
are still specialized for both key, value type.
Introduce `Set(key)` type for when there is no value.
Refactor shada.c to use Map/Set instead of khash directly.
This requires `map_ref` operation to be more flexible.
Return pointers to both key and value, plus an indicator for new_item.
As a bonus, `map_key` is now redundant.
Instead of Map(cstr_t, FileMarks), use a pointer map as the FileMarks struct is
humongous.
Make `event_strings` actually work like an intern pool instead of wtf it
was doing before.
Problem: Only created files can be cleaned up with one call.
Solution: Add flags to mkdir() to delete with a deferred function.
Expand the writefile() name to a full path to handle changing
directory.
6f14da15ac
vim-patch:8.2.3742: dec mouse test fails without gnome terminfo entry
Problem: Dec mouse test fails without gnome terminfo entry.
Solution: Check if there is a gnome entry. Also fix 'acd' test on
MS-Windows. (Dominique Pellé, closesvim/vim#9282)
f589fd3e10
Cherry-pick test_autochdir.vim changes from patch 9.0.0313.
Cherry-pick test_autocmd.vim changes from patch 9.0.0323.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Vim9: value of 'magic' is still relevant.
Solution: Always behave like 'magic' is on in Vim9 script (closesvim/vim#7509)
f4e2099e39
EX_NONWHITE_OK is N/A: only applies to Vim9 script.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Allow Include What You Use to remove unnecessary includes and only
include what is necessary. This helps with reducing compilation times
and makes it easier to visualise which dependencies are actually
required.
Work on https://github.com/neovim/neovim/issues/549, but doesn't close
it since this only works fully for .c files and not headers.
Problem: Some tests fail on Cirrus CI and/or with FreeBSD.
Solution: Make 'backupskip' empty. Do not run tests as root. Check for
directory when using viminfo. (Ozaki Kiichi, closesvim/vim#6596)
b86abadf87
Enable and fix bugprone-misplaced-widening-cast warning.
Fix some modernize-macro-to-enum and readability-else-after-return
warnings, but don't enable them. While the warnings can be useful, they
are in general too noisy to enable.
Uncrustify struggles to format function-like macros which are defined in
deeply nested areas of the code. Un-nesting them unblocks useful
formatting rules from uncrustify.
vim-patch:8.2.0559: clearing a struct is verbose
Problem: Clearing a struct is verbose.
Solution: Define and use CLEAR_FIELD() and CLEAR_POINTER().
a80faa8930
Problem: Command line history code is spread out.
Solution: Put the code in a new file. (Yegappan Lakshmanan, closesvim/vim#4779)
Also graduate the +cmdline_hist feature.
d7663c22c6
** Refactor
Previously most functions used to "get" a mark returned a position,
changed the line number and sometimes changed even the current buffer.
Now functions return a {x}fmark_T making calling context aware whether
the mark is in another buffer without arcane casting. A new function is
provided for switching to the mark buffer and returning a flag style
Enum to convey what happen in the movement. If the cursor changed, line,
columns, if it changed buffer, etc.
The function to get named mark was split into multiple functions.
- mark_get() -> fmark_T
- mark_get_global() -> xfmark_T
- mark_get_local() -> fmark_T
- mark_get_motion() -> fmark_T
- mark_get_visual() -> fmark_T
Functions that manage the changelist and jumplist were also modified to
return mark types.
- get_jumplist -> fmark_T
- get_changelist -> fmark_T
The refactor is also seen mainly on normal.c, where all the mark
movement has been siphoned through one function nv_gomark, while the
other functions handle getting the mark and setting their movement
flags. To handle whether context marks should be left, etc.
** Mark View
While doing the refactor the concept of a mark view was also
implemented:
The view of a mark currently implemented as the number of lines between
the mark position on creation and the window topline. This allows for
moving not only back to the position of a mark but having the window
look similar to when the mark was defined. This is done by carrying and
extra element in the fmark_T struct, which can be extended later to also
restore horizontal shift.
*** User space features
1. There's a new option, jumpoptions+=view enables the mark view restoring
automatically when using the jumplist, changelist, alternate-file and
mark motions. <C-O> <C-I> g; g, <C-^> '[mark] `[mark]
** Limitations
- The view information is not saved in shada.
- Calls to get_mark should copy the value in the pointer since we are
using pos_to_mark() to wrap and provide a homogeneous interfaces. This
was also a limitation in the previous state of things.
The size of long varies depending on architecture, in contrast to the
MAXLNUM constant which sets the maximum allowable number of lines to
2^32-1. This discrepancy may lead to hard to detect bugs, for example
https://github.com/neovim/neovim/issues/18454. Setting linenr_T to a
fix maximum size of 2^32-1 will prevent this type of errors in the
future.
Also change the variables `amount` and `amount_after` to be linenr_T
since they're referring to "the line number difference" between two
texts.
