problem: checks for wp->w_p_rl are all over the place, making simple
things like "advance column one cell" incredibly complicated.
solution: always fill linebuf_char[] using an incrementing counter,
and then mirror the buffer as a post-processing step
This was "easier" that I first feared, because the stupid but simple
workaround for things like keeping linenumbers still left-right,
e.g. "mirror them and them mirror them once more" is more or less
what vim did already. So let's just keep doing that.
long is 32 bits on windows, while it is 64 bits on other architectures.
This makes the type suboptimal for a codebase meant to be
cross-platform. Replace it with more appropriate integer types.
Problem: Peeking and flushing output slows down execution.
Solution: Do not update the mode message when global_busy is set. Do not
flush when only peeking for a character. (Ken Takata)
cb574f4154
Problem: linebreak applies for leading whitespace
Solution: only apply linebreak, once we have found non-breakat chars in
the line
closes: vim/vim#13228closes: vim/vim#13243dd75fcfbdf
Co-authored-by: Christian Brabandt <cb@256bit.org>
The 'arabicshape' feature of vim is a transformation of unicode text to
make arabic and some related scripts look better at display time. In
particular the content of a cell will be adjusted depending on the
(original) content of the cells just before and after it.
This is implemented by the arabic_shape() function in nvim. Before this
commit, shaping was invoked in four different contexts:
- when rendering buffer text in win_line()
- in line_putchar() for rendering virtual text
- as part of grid_line_puts, used by messages and statuslines and
similar
- as part of draw_cmdline() for drawing the cmdline
This replaces all these with a post-processing step in grid_put_linebuf(),
which has become the entry point for all text rendering after recent
refactors.
An aim of this is to make the handling of multibyte text yet simpler.
One of the main reasons multibyte chars needs to be "parsed" into
codepoint arrays of composing chars is so that these could be inspected
for the purpose of shaping. This can likely be vastly simplified in many
contexts where only the total length (in bytes) and width of composed
char is needed.
This finalizes the long running refactor from the old TUI-focused grid
implementation where text-drawing cursor was not separated from the
visible cursor.
Still, the pattern of setting cursor position together with updating a
line was convenient. Introduce grid_line_cursor_goto() to still allow
this but now being explicit about it.
Only having batched drawing functions makes code involving drawing
a bit longer. But it is better to be explicit, and this highlights
cases where multiple small redraws can be grouped together. This was the
case for most of the changed places (messages, lastline, and :intro)
- Move vimoption_T to option.h
- option_defs.h is for option-related types
- option_vars.h corresponds to Vim's option.h
- option_defs.h and option_vars.h don't include each other
The screen grid refactors will continue until morale improves.
Jokes aside, this is quite a central installment in the series.
Before this refactor, there were two fundamentally distinct codepaths
for getting some text on the screen:
- the win_line() -> grid_put_linebuf() -> ui_line() call chain used for
buffer text, with linebuf_char as a temporary scratch buffer
- the grid_line_start/grid_line_puts/grid_line_flush() -> ui_line()
path used for every thing else: statuslines, messages and the command line.
Here the grid->chars[] array itself doubles as a scratch buffer.
With this refactor, the later family of functions still exist, however
they now as well render to linebuf_char just like win_line() did, and
grid_put_linebuf() is called in the end to calculate delta changes.
This means we don't need any duplicate logic for delta calculations anymore.
Later down the line, it will be possible to share more logic operating
on this scratch buffer, like doing 'rightleft' reversal and arabic
shaping as a post-processing step.
Problem: multispace wrong when scrolling horizontally
Solution: Update position in "multispace" or "leadmultispace" also in
skipped chars. Reorder conditions to be more consistent.
closes: vim/vim#13145closes: vim/vim#13147abc808112e
This is not used as part of the logic to actually implement TUI line wrapping
In vim (especially gvim) it is used to emulate terminal-style text
selection. But in nvim we don't do that, and have no plans to reintroduce it.
Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.
As an example, even a pretty modest 50*80 screen would consume
50*80*2*32 = 256000, i e a quarter megabyte
With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.
This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
glyph cache
This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.
When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing
A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
The removes the previous restriction that nvim_buf_set_extmark()
could not be used to highlight arbitrary multi-line regions
The problem can be summarized as follows: let's assume an extmark with a
hl_group is placed covering the region (5,0) to (50,0) Now, consider
what happens if nvim needs to redraw a window covering the lines 20-30.
It needs to be able to ask the marktree what extmarks cover this region,
even if they don't begin or end here.
Therefore the marktree needs to be augmented with the information covers
a point, not just what marks begin or end there. To do this, we augment
each node with a field "intersect" which is a set the ids of the
marks which overlap this node, but only if it is not part of the set of
any parent. This ensures the number of nodes that need to be explicitly
marked grows only logarithmically with the total number of explicitly
nodes (and thus the number of of overlapping marks).
Thus we can quickly iterate all marks which overlaps any query position
by looking up what leaf node contains that position. Then we only need
to consider all "start" marks within that leaf node, and the "intersect"
set of that node and all its parents.
Now, and the major source of complexity is that the tree restructuring
operations (to ensure that each node has T-1 <= size <= 2*T-1) also need
to update these sets. If a full inner node is split in two, one of the
new parents might start to completely overlap some ranges and its ids
will need to be moved from its children's sets to its own set.
Similarly, if two undersized nodes gets joined into one, it might no
longer completely overlap some ranges, and now the children which do
needs to have the have the ids in its set instead. And then there are
the pivots! Yes the pivot operations when a child gets moved from one
parent to another.
Problem: 'linebreak' is incorrectly drawn after 'breakindent'.
Solution: Don't include 'breakindent' size when already after it.
closes: vim/vim#12937closes: vim/vim#129401d3e0e8f31
Problem: Cursor position still wrong with 'showbreak' and virtual text
after last character or 'listchars' "eol".
Solution: Remove unnecessary w_wcol adjustment in curs_columns(). Also
fix first char of virtual text not shown at the start of a screen
line.
closes: vim/vim#12478closes: vim/vim#12532closes: vim/vim#129046a3897232a
ml_get_buf() takes a third parameters to indicate whether the
caller wants to mutate the memline data in place. However
the vast majority of the call sites is using this function
just to specify a buffer but without any mutation. This makes
it harder to grep for the places which actually perform mutation.
Solution: Remove the bool param from ml_get_buf(). it now works
like ml_get() except for a non-current buffer. Add a new
ml_get_buf_mut() function for the mutating use-case, which can
be grepped along with the other ml_replace() etc functions which
can modify the memline.
Problem: Wrong display with wrapping virtual text or unprintable chars,
'showbreak' and 'smoothscroll'.
Solution: Don't skip cells taken by 'showbreak' in screen lines before
"w_skipcol". Combined "n_skip" and "skip_cells".
closes: vim/vim#12597b557f48982
Problem: Visual highlight not working with cursor at end of screen line
and 'showbreak'.
Solution: Only update "vcol_prev" when drawing buffer text.
closes: vim/vim#128658fc6a1dae0
Problem: Wrong cursor position when clicking after concealed text
with 'virtualedit'.
Solution: Store virtual columns in ScreenCols[] instead of text
columns, and always use coladvance() when clicking.
This also fixes incorrect curswant when clicking on a TAB, so now
Test_normal_click_on_ctrl_char() asserts the same results as the ones
before patch 9.0.0048.
closes: vim/vim#12808e500ae8e29
Remove the mouse_adjust_click() function.
There is a difference in behavior with the old mouse_adjust_click()
approach: when clicking on the character immediately after concealed
text that is completely hidden, cursor is put on the clicked character
rather than at the start of the concealed text. The new behavior is
better, but it causes unnecessary scrolling in a functional test (which
is an existing issue unrelated to these patches), so adjust the test.
Now fully merged:
vim-patch:9.0.0177: cursor position wrong with 'virtualedit' and mouse click
Problem: blockwise Visual highlight not working with virtual text
Solution: Reset the correct variable at the end of virtual selection and
Check for double-width char inside virtual text.
closes: vim/vim#126066e940d9a1d
Need to remove area_active and use wlv.fromcol and wlv.tocol directly.
Extreme testcase:
```lua
vim.fn.setline(1, 'foobar')
local ns = vim.api.nvim_create_namespace('')
for _ = 1, 100000 do
vim.api.nvim_buf_set_extmark(0, ns, 0, 3, {})
end
local start_time = vim.loop.hrtime()
vim.fn.virtcol('$')
local stop_time = vim.loop.hrtime()
print(stop_time - start_time)
```
Before #20130: 31696
On master branch: 26191344
After this PR: 37692
