If nvim exited with nonzero status this is for one of the two reasons
- `:cquit` was invoked. This is used by users and plugins to communicate
a result, like a nonzero status will fail a `git commit` operation
- There was an internal error or deadly signal. in this case an error
message was likely written to stderr or to the screen.
In the latter case, the error message was often hidden by the TUI
exiting altscreen mode, which erases all visible terminal text.
This change prevents this in the latter case, while still cleaning up
the terminal properly when `:cquit` was deliberatily invoked.
Other cleanup like exiting mouse mode and raw mode is still done.
Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.
As an example, even a pretty modest 50*80 screen would consume
50*80*2*32 = 256000, i e a quarter megabyte
With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.
This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
glyph cache
This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.
When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing
A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
The removes the previous restriction that nvim_buf_set_extmark()
could not be used to highlight arbitrary multi-line regions
The problem can be summarized as follows: let's assume an extmark with a
hl_group is placed covering the region (5,0) to (50,0) Now, consider
what happens if nvim needs to redraw a window covering the lines 20-30.
It needs to be able to ask the marktree what extmarks cover this region,
even if they don't begin or end here.
Therefore the marktree needs to be augmented with the information covers
a point, not just what marks begin or end there. To do this, we augment
each node with a field "intersect" which is a set the ids of the
marks which overlap this node, but only if it is not part of the set of
any parent. This ensures the number of nodes that need to be explicitly
marked grows only logarithmically with the total number of explicitly
nodes (and thus the number of of overlapping marks).
Thus we can quickly iterate all marks which overlaps any query position
by looking up what leaf node contains that position. Then we only need
to consider all "start" marks within that leaf node, and the "intersect"
set of that node and all its parents.
Now, and the major source of complexity is that the tree restructuring
operations (to ensure that each node has T-1 <= size <= 2*T-1) also need
to update these sets. If a full inner node is split in two, one of the
new parents might start to completely overlap some ranges and its ids
will need to be moved from its children's sets to its own set.
Similarly, if two undersized nodes gets joined into one, it might no
longer completely overlap some ranges, and now the children which do
needs to have the have the ids in its set instead. And then there are
the pivots! Yes the pivot operations when a child gets moved from one
parent to another.
Problem:
* The guessed botline might be smaller than the actual botline e.g. when
there are folds and the user is typing in insert mode. This may result
in incorrect treesitter highlights for injections.
* botline can be larger than the last line number of the buffer, which
results in errors when placing extmarks.
Solution:
* Take a more conservative approximation. I am not sure if it is
sufficient to guarantee correctness, but it seems to be good enough
for the case mentioned above.
* Clamp it to the last line number.
Co-authored-by: Lewis Russell <me@lewisr.dev>
This involves two redesigns of the map.c implementations:
1. Change of macro style and code organization
The old khash.h and map.c implementation used huge #define blocks with a
lot of backslash line continuations.
This instead uses the "implementation file" .c.h pattern. Such a file is
meant to be included multiple times, with different macros set prior to
inclusion as parameters. we already use this pattern e.g. for
eval/typval_encode.c.h to implement different typval encoders reusing a
similar structure.
We can structure this code into two parts. one that only depends on key
type and is enough to implement sets, and one which depends on both key
and value to implement maps (as a wrapper around sets, with an added
value[] array)
2. Separate the main hash buckets from the key / value arrays
Change the hack buckets to only contain an index into separate key /
value arrays
This is a common pattern in modern, state of the art hashmap
implementations. Even though this leads to one more allocated array, it
is this often is a net reduction of memory consumption. Consider
key+value consuming at least 12 bytes per pair. On average, we will have
twice as many buckets per item.
Thus old implementation:
2*12 = 24 bytes per item
New implementation
1*12 + 2*4 = 20 bytes per item
And the difference gets bigger with larger items.
One might think we have pulled a fast one here, as wouldn't the average size of
the new key/value arrays be 1.5 slots per items due to amortized grows?
But remember, these arrays are fully dense, and thus the accessed memory,
measured in _cache lines_, the unit which actually matters, will be the
fully used memory but just rounded up to the nearest cache line
boundary.
This has some other interesting properties, such as an insert-only
set/map will be fully ordered by insert only. Preserving this ordering
in face of deletions is more tricky tho. As we currently don't use
ordered maps, the "delete" operation maintains compactness of the item
arrays in the simplest way by breaking the ordering. It would be
possible to implement an order-preserving delete although at some cost,
like allowing the items array to become non-dense until the next rehash.
Finally, in face of these two major changes, all code used in khash.h
has been integrated into map.c and friends. Given the heavy edits it
makes no sense to "layer" the code into a vendored and a wrapper part.
Rather, the layered cake follows the specialization depth: code shared
for all maps, code specialized to a key type (and its equivalence
relation), and finally code specialized to value+key type.
Some more reasonable defaults for topline:
- if topline was replaced with another line, that now becomes topline
- if line was inserted just before topline, display it. This is more
similar to the previous API behavior.
The change in #248240081549 was not a regression, however it was an
incomplete change. Unfortunately some common plugins come to depend on
this exising self-inconsistent behavior. These plugins are going to need
to update for 0.10
nvim_buf_set_lines used to NOT adjust the topline correctly if a buffer
was displayed in just one window. However, if displayed in multiple
windows, it was correctly adjusted for any window not deemed the
current window for the buffer (which could be an arbitrary choice if the
buffer was not already current, as noted in the last rafactor)
This fixes so that all windows have their topline adjusted. The added
tests show this behavior, which should be the reasonable one.
Problem: Now way to show text at the bottom part of floating window
border (a.k.a. "footer").
Solution: Allows `footer` and `footer_pos` config fields similar to
`title` and `title_pos`.
Most of the messy things when changing a non-current buffer is
not about the buffer, it is about windows. In particular, it is about
`curwin`.
When editing a non-current buffer which is displayed in some other
window in the current tabpage, one such window will be "borrowed" as the
curwin. But this means if two or more non-current windows displayed the buffers,
one of them will be treated differenty. this is not desirable.
In particular, with nvim_buf_set_text, cursor _column_ position was only
corrected for one single window. Two new tests are added: the test
with just one non-current window passes, but the one with two didn't.
Two corresponding such tests were also added for nvim_buf_set_lines.
This already worked correctly on master, but make sure this is
well-tested for future refactors.
Also, nvim_create_buf no longer invokes autocmds just because you happened
to use `scratch=true`. No option value was changed, therefore OptionSet
must not be fired.
ml_get_buf() takes a third parameters to indicate whether the
caller wants to mutate the memline data in place. However
the vast majority of the call sites is using this function
just to specify a buffer but without any mutation. This makes
it harder to grep for the places which actually perform mutation.
Solution: Remove the bool param from ml_get_buf(). it now works
like ml_get() except for a non-current buffer. Add a new
ml_get_buf_mut() function for the mutating use-case, which can
be grepped along with the other ml_replace() etc functions which
can modify the memline.
Problem: mapset() not properly handling script ID
Solution: replace_termcodes() may accept a script ID
closes: vim/vim#12699closes: vim/vim#126977e0bae024d
This aligns its behaviour better with `nvim_win_close`.
Note that `:hide` is actually incapable of closing the cmdwin, unlike `:close`
and `:quit`, so this is a bit of a difference in behaviour.
Disallow closing the previous window from `nvim_win_close`, as this will cause
issues.
Again, no telling how safe this is. It also requires exposing old_curwin. :/
Also note that it's possible for the `&cmdheight` to change if, for example,
there are 2 tabpages and `nvim_win_close` is used to close the last window in
the other tabpage while `&stal` is 1. This is addressed in a later commit.
Problem: As discussed on Matrix, there was some interest in having
`nvim_open_win` again be able to open floats in the cmdwin (e.g: displaying a
hover doc related to what's in the cmdwin). After #23228, this was disallowed.
Solution: Allow `nvim_open_win` in the cmdwin as long as `!enter` and
`buffer != curbuf` (the former can cause all sorts of issues, and the latter
can crash Nvim after closing cmdwin). Also allow `nvim_win_set_buf` in a similar
fashion.
Note that we're not *entirely* sure if this is 100% safe (cmdwin is a
global-state-using-main-loop-calling beast), but this seems to work OK..?
Also:
- Check the buffer argument of `nvim_open_win` earlier, and abort if it's
invalid (it used to still open a window in this case).
- Untranslate `e_cmdwin` errors in the API (other errors in the API are not
translated: although not detailed in the API contract yet, errors are
supposed to be stable).
* perf(rtp): reduce rtp scans
Problem:
Scanning the filesystem is expensive and particularly affects
startuptime.
Solution:
Reduce the amount of redundant directory scans by relying less on glob
patterns and handle vim and lua sourcing lower down.
Problem:
Lua functions that return multiple results are declared by using
multiple `@return` docstring directives. But the generated docs don't
make it obvious what this represents.
Solution:
- Generate a "Return (multiple)" heading for multiple-value functions.
- Fix `@note` directives randomly placed after `@return`.
