The removes the previous restriction that nvim_buf_set_extmark()
could not be used to highlight arbitrary multi-line regions
The problem can be summarized as follows: let's assume an extmark with a
hl_group is placed covering the region (5,0) to (50,0) Now, consider
what happens if nvim needs to redraw a window covering the lines 20-30.
It needs to be able to ask the marktree what extmarks cover this region,
even if they don't begin or end here.
Therefore the marktree needs to be augmented with the information covers
a point, not just what marks begin or end there. To do this, we augment
each node with a field "intersect" which is a set the ids of the
marks which overlap this node, but only if it is not part of the set of
any parent. This ensures the number of nodes that need to be explicitly
marked grows only logarithmically with the total number of explicitly
nodes (and thus the number of of overlapping marks).
Thus we can quickly iterate all marks which overlaps any query position
by looking up what leaf node contains that position. Then we only need
to consider all "start" marks within that leaf node, and the "intersect"
set of that node and all its parents.
Now, and the major source of complexity is that the tree restructuring
operations (to ensure that each node has T-1 <= size <= 2*T-1) also need
to update these sets. If a full inner node is split in two, one of the
new parents might start to completely overlap some ranges and its ids
will need to be moved from its children's sets to its own set.
Similarly, if two undersized nodes gets joined into one, it might no
longer completely overlap some ranges, and now the children which do
needs to have the have the ids in its set instead. And then there are
the pivots! Yes the pivot operations when a child gets moved from one
parent to another.
This involves two redesigns of the map.c implementations:
1. Change of macro style and code organization
The old khash.h and map.c implementation used huge #define blocks with a
lot of backslash line continuations.
This instead uses the "implementation file" .c.h pattern. Such a file is
meant to be included multiple times, with different macros set prior to
inclusion as parameters. we already use this pattern e.g. for
eval/typval_encode.c.h to implement different typval encoders reusing a
similar structure.
We can structure this code into two parts. one that only depends on key
type and is enough to implement sets, and one which depends on both key
and value to implement maps (as a wrapper around sets, with an added
value[] array)
2. Separate the main hash buckets from the key / value arrays
Change the hack buckets to only contain an index into separate key /
value arrays
This is a common pattern in modern, state of the art hashmap
implementations. Even though this leads to one more allocated array, it
is this often is a net reduction of memory consumption. Consider
key+value consuming at least 12 bytes per pair. On average, we will have
twice as many buckets per item.
Thus old implementation:
2*12 = 24 bytes per item
New implementation
1*12 + 2*4 = 20 bytes per item
And the difference gets bigger with larger items.
One might think we have pulled a fast one here, as wouldn't the average size of
the new key/value arrays be 1.5 slots per items due to amortized grows?
But remember, these arrays are fully dense, and thus the accessed memory,
measured in _cache lines_, the unit which actually matters, will be the
fully used memory but just rounded up to the nearest cache line
boundary.
This has some other interesting properties, such as an insert-only
set/map will be fully ordered by insert only. Preserving this ordering
in face of deletions is more tricky tho. As we currently don't use
ordered maps, the "delete" operation maintains compactness of the item
arrays in the simplest way by breaking the ordering. It would be
possible to implement an order-preserving delete although at some cost,
like allowing the items array to become non-dense until the next rehash.
Finally, in face of these two major changes, all code used in khash.h
has been integrated into map.c and friends. Given the heavy edits it
makes no sense to "layer" the code into a vendored and a wrapper part.
Rather, the layered cake follows the specialization depth: code shared
for all maps, code specialized to a key type (and its equivalence
relation), and finally code specialized to value+key type.
This reduces the total number of khash_t instantiations from 22 to 8.
Make the khash internal functions take the size of values as a runtime
parameter. This is abstracted with typesafe Map containers which
are still specialized for both key, value type.
Introduce `Set(key)` type for when there is no value.
Refactor shada.c to use Map/Set instead of khash directly.
This requires `map_ref` operation to be more flexible.
Return pointers to both key and value, plus an indicator for new_item.
As a bonus, `map_key` is now redundant.
Instead of Map(cstr_t, FileMarks), use a pointer map as the FileMarks struct is
humongous.
Make `event_strings` actually work like an intern pool instead of wtf it
was doing before.
fix(extmarks): disallow removing extmarks in on_lines callbacks
decor_redraw_start (which runs before decor_providers_invoke_lines) gets
references for the extmarks on a specific line. If these extmarks are
deleted in on_lines callbacks then this results in a heap-use-after-free
error.
Fixes#22801
Problem: Can not get all extmarks in a buffer. Properties are missing
from the details array.
Solution: Allow getting all extmarks in a buffer by supplying a -1
"ns_id". Add missing properties to the details array.
extranges and a bunch of other improvements are coming for 0.10
This gets in some minor surrounding API changes to avoid rebase
conflicts until then.
- decorations will be able to be specific to windows
- adjust deletion API to fit with extranges
Allow Include What You Use to remove unnecessary includes and only
include what is necessary. This helps with reducing compilation times
and makes it easier to visualise which dependencies are actually
required.
Work on https://github.com/neovim/neovim/issues/549, but doesn't close
it since this only works fully for .c files and not headers.
00cfc1d (from #20249) reduced the amount of unnecessary redraws. This
surfaced an issue where if and extmark with a specific ID is
repositioned to a different row, the decorations from the old row were
not redrawn and removed. This change fixes that by redrawing the
old row.
- Added 'spell' option to extmarks:
Extmarks with this set will have the region spellchecked.
- Added 'noplainbuffer' option to 'spelloptions':
This is used to tell Neovim not to spellcheck the buffer. The old
behaviour was to spell check the whole buffer unless :syntax was set.
- Added spelling support to the treesitter highlighter:
@spell captures in highlights.scm are used to define regions which
should be spell checked.
- Added support for navigating spell errors for extmarks:
Works for both ephemeral and static extmarks
- Added '_on_spell_nav' callback for decoration providers:
Since ephemeral callbacks are only drawn for the visible screen,
providers must implement this callback to instruct Neovim which
regions in the buffer need can be spell checked.
The callback takes a start position and an end position.
Note: this callback is subject to change hence the _ prefix.
- Added spell captures for built-in support languages
Co-authored-by: Lewis Russell <lewis6991@gmail.com>
Co-authored-by: Björn Linse <bjorn.linse@gmail.com>
The size of long varies depending on architecture, in contrast to the
MAXLNUM constant which sets the maximum allowable number of lines to
2^32-1. This discrepancy may lead to hard to detect bugs, for example
https://github.com/neovim/neovim/issues/18454. Setting linenr_T to a
fix maximum size of 2^32-1 will prevent this type of errors in the
future.
Also change the variables `amount` and `amount_after` to be linenr_T
since they're referring to "the line number difference" between two
texts.
Add space around arithmetic operators '+' and '-'.
Remove space between back-to-back parentheses, i.e. ')(' vs. ') ('.
Remove space between '((' or '))' of control statements.
Add space between ')' and '{' of control statements.
Remove space between function name and '(' on function declaration.
Collapse empty blocks between '{' and '}'.
Remove newline at the end of the file.
Remove newline between 'enum' and '{'.
Remove newline between '}' and ')' in a function invocation.
Remove newline between '}' and 'while' of 'do' statement.
Previously b_signcols was invalidated whenever a sign was added/removed
or when a buffer line was added/removed.
This change introduces a sentinel linenr_T into the buffer state which
is a line number used to determine the signcolumn. With this
information, we can invalidate the signcolumn less often. Now the
signcolumn is only invalidated when a sign or line at the sentinel line
number is removed.
Add the following options to extmarks:
- sign_text
- sign_hl_group
- number_hl_group
- line_hl_group
- cursorline_hl_group
Note: ranges are unsupported and decorations are only applied to
start_row
marktree.c was originally constructed as a "generic" datatype,
to make the prototyping of its internal logic as simple as possible
and also as the usecases for various kinds of extmarks/decorations was not yet decided.
As a consequence of this, various extra indirections and allocations was
needed to use marktree to implement extmarks (ns/id pairs) and
decorations of different kinds (some which is just a single highlight
id, other an allocated list of virtual text/lines)
This change removes a lot of indirection, by making Marktree specialized
for the usecase. In particular, the namespace id and mark id is stored
directly, instead of the 64-bit global id particular to the Marktree
struct. This removes the two maps needed to convert between global and
per-ns ids.
Also, "small" decorations are stored inline, i.e. those who
doesn't refer to external heap memory anyway. That is highlights (with
priority+flags) are stored inline, while virtual text, which anyway
occurs a lot of heap allocations, do not. (previously a hack was used
to elide heap allocations for highlights with standard prio+flags)
TODO(bfredl): the functionaltest-lua CI version of gcc is having
severe issues with uint16_t bitfields, so splitting up compound
assignments and redundant casts are needed. Clean this up once we switch
to a working compiler version.
This allows to more quickly skip though regions which has non-decorative
marks when redrawing. This might seem like a gratuitous
micro-optimization in isolation.
But!
Soon decorations are gonna crop into other hot inner-loop paths,
including the plines.c code for calculating the horizontal and
vertical space of text. Then we want to quickly skip over regions with
"only" overlaying decorations (which do not affect text size)
Note: the reason for removing them is not that there after this refactor
is no use of them, but rather that having them available is an
anti-pattern: they manange an _extra_ heap allocation which has
nothing to do with the functionality of the map itself (khash
manages the real buffers internally). In case there happens to
be a reason to allocate the map structure itself later, this
should be made explicit using xcalloc/xfree calls.
54ce101 changed the way undo entries are created when adding decorations.
This creates all sorts of problems.This change fixes the problem by
reverting to the previous behavior.
On empty buffers, when editing the first line, the line is buffered, causing offset to be < 0. While the buffer is not actually empty, the buffered line has not been flushed (and should not be) yet, so the call is valid but an edge case.
