decor->text.str pointer must go. This removes it for conceal char,
in preparation for a larger PR which will also handle the sign case.
By actually allowing composing chars for a conceal chars, this
becomes a feature and not just a refactor, as a bonus.
Problem: buffer text with composing chars are converted from UTF-8
to an array of up to seven UTF-32 values and then converted back
to UTF-8 strings.
Solution: Convert buffer text directly to UTF-8 based schar_T values.
The limit of the text size is now in schar_T bytes, which is currently
31+1 but easily could be raised as it no longer multiplies the size
of the entire screen grid when not used, the full size is only required
for temporary scratch buffers.
Also does some general cleanup to win_line text handling, which was
unnecessarily complicated due to multibyte rendering being an "opt-in"
feature long ago. Nowadays, a char is just a char, regardless if it consists
of one ASCII byte or multiple bytes.
We already have an extensive suite of static analysis tools we use,
which causes a fair bit of redundancy as we get duplicate warnings. PVS
is also prone to give false warnings which creates a lot of work to
identify and disable.
problem: checks for wp->w_p_rl are all over the place, making simple
things like "advance column one cell" incredibly complicated.
solution: always fill linebuf_char[] using an incrementing counter,
and then mirror the buffer as a post-processing step
This was "easier" that I first feared, because the stupid but simple
workaround for things like keeping linenumbers still left-right,
e.g. "mirror them and them mirror them once more" is more or less
what vim did already. So let's just keep doing that.
Previously, 'rightleftcmd' was implemented by having all code which
would affect msg_col or output screen cells be conditional on `cmdmsg_rl`.
This change removes all that and instead implements rightleft as a
mirroring post-processing step.
Problem: Peeking and flushing output slows down execution.
Solution: Do not update the mode message when global_busy is set. Do not
flush when only peeking for a character. (Ken Takata)
cb574f4154
The 'arabicshape' feature of vim is a transformation of unicode text to
make arabic and some related scripts look better at display time. In
particular the content of a cell will be adjusted depending on the
(original) content of the cells just before and after it.
This is implemented by the arabic_shape() function in nvim. Before this
commit, shaping was invoked in four different contexts:
- when rendering buffer text in win_line()
- in line_putchar() for rendering virtual text
- as part of grid_line_puts, used by messages and statuslines and
similar
- as part of draw_cmdline() for drawing the cmdline
This replaces all these with a post-processing step in grid_put_linebuf(),
which has become the entry point for all text rendering after recent
refactors.
An aim of this is to make the handling of multibyte text yet simpler.
One of the main reasons multibyte chars needs to be "parsed" into
codepoint arrays of composing chars is so that these could be inspected
for the purpose of shaping. This can likely be vastly simplified in many
contexts where only the total length (in bytes) and width of composed
char is needed.
This finalizes the long running refactor from the old TUI-focused grid
implementation where text-drawing cursor was not separated from the
visible cursor.
Still, the pattern of setting cursor position together with updating a
line was convenient. Introduce grid_line_cursor_goto() to still allow
this but now being explicit about it.
Only having batched drawing functions makes code involving drawing
a bit longer. But it is better to be explicit, and this highlights
cases where multiple small redraws can be grouped together. This was the
case for most of the changed places (messages, lastline, and :intro)
msg_puts_display was more complex than necessary in nvim, as in
nvim, it no longer talks directly with a terminal.
In particular we don't need to scroll the grid before emiting the last
char. The TUI already takes care of things like that, for terminals
where it matters.
- Move vimoption_T to option.h
- option_defs.h is for option-related types
- option_vars.h corresponds to Vim's option.h
- option_defs.h and option_vars.h don't include each other
The screen grid refactors will continue until morale improves.
Jokes aside, this is quite a central installment in the series.
Before this refactor, there were two fundamentally distinct codepaths
for getting some text on the screen:
- the win_line() -> grid_put_linebuf() -> ui_line() call chain used for
buffer text, with linebuf_char as a temporary scratch buffer
- the grid_line_start/grid_line_puts/grid_line_flush() -> ui_line()
path used for every thing else: statuslines, messages and the command line.
Here the grid->chars[] array itself doubles as a scratch buffer.
With this refactor, the later family of functions still exist, however
they now as well render to linebuf_char just like win_line() did, and
grid_put_linebuf() is called in the end to calculate delta changes.
This means we don't need any duplicate logic for delta calculations anymore.
Later down the line, it will be possible to share more logic operating
on this scratch buffer, like doing 'rightleft' reversal and arabic
shaping as a post-processing step.
This is a step in an ongoing refactor where the "grid_puts" and
"grid_put_linebuf" code paths will share more of the implementation (in
particular for delta calculation, doublewidth and 'arabicshape'
handling). But it also makes sense by its own as a cleanup, and is thus
committed separately.
Before this change many of the low level grid functions grid_puts,
grid_fill etc could both be used in a standalone fashion but also as
part of a batched line update which would be finally transmitted as a
single grid_line call (via ui_line() ). This was initially useful to
quickly refactor pre-existing vim code to use batched logic safely.
However, this pattern is not really helpful for maintainable and newly
written code, where the "grid" and "row" arguments are just needlessly
repeated. This simplifies these calls to just use grid and row as
specified in the initial grid_line_start(grid, row) call.
This also makes the intent clear whether any grid_puts() call is actually
part of a batch or not, which is better in the long run when more things
get refactored to use effective (properly batched) updates.
This is not used as part of the logic to actually implement TUI line wrapping
In vim (especially gvim) it is used to emulate terminal-style text
selection. But in nvim we don't do that, and have no plans to reintroduce it.
Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.
As an example, even a pretty modest 50*80 screen would consume
50*80*2*32 = 256000, i e a quarter megabyte
With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.
This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
glyph cache
This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.
When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing
A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
Problem: Wrong curswant when clicking and the second cell of a
double-width char.
Solution: Don't copy virtcol of the first char to the second one.
closes: vim/vim#128429994160bfe
Problem: Wrong cursor position when clicking after concealed text
with 'virtualedit'.
Solution: Store virtual columns in ScreenCols[] instead of text
columns, and always use coladvance() when clicking.
This also fixes incorrect curswant when clicking on a TAB, so now
Test_normal_click_on_ctrl_char() asserts the same results as the ones
before patch 9.0.0048.
closes: vim/vim#12808e500ae8e29
Remove the mouse_adjust_click() function.
There is a difference in behavior with the old mouse_adjust_click()
approach: when clicking on the character immediately after concealed
text that is completely hidden, cursor is put on the clicked character
rather than at the start of the concealed text. The new behavior is
better, but it causes unnecessary scrolling in a functional test (which
is an existing issue unrelated to these patches), so adjust the test.
Now fully merged:
vim-patch:9.0.0177: cursor position wrong with 'virtualedit' and mouse click
Problem: Visual area not shown when using 'showbreak' and start of line is
not visible. (Jaehwang Jung)
Solution: Adjust "fromcol" for the space taken by 'showbreak'.
(closesvim/vim#12514)
f578ca2c8f
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Stray character is visible if 'smoothscroll' marker is displayed
on top of a double-wide character.
Solution: When overwriting a double-width character with the 'smoothscroll'
marker clear the second half. (closesvim/vim#12469)
ecb87dd7d3
Problem: screenchar(), screenchars() and screenstring() do not work
properly when 'encoding' is set to a double-byte encoding.
Solution: Fix the way the bytes of the characters are obtained.
(issue vim/vim#12469)
47eec6716b
Problem: "precedes" from 'listchars' overwritten by <<< for 'smoothscroll'.
Solution: Keep the "precedes" character.
13cdde3952
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Line number not visisble with 'smoothscroll', 'nu' and 'rnu'.
Solution: Put the ">>>" after the line number instead of on top.
eb4de62931
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: "<<<" shows for 'smoothscroll' even when 'showbreak is set.
Solution: When 'showbreak' is set do not display "<<<".
0937b9fb24
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: No indication when the first line is broken for 'smoothscroll'.
Solution: Show "<<<" in the first line.
406b5d89e1
Co-authored-by: Bram Moolenaar <Bram@vim.org>
libnvim couldn't be easily used in C++ due to the use of reserved keywords.
Additionally, add explicit casts to *alloc function calls used in inline
functions, as C++ doesn't allow implicit casts from void pointers.
Problem: Code is indented more than necessary.
Solution: Use an early return where it makes sense. (Yegappan Lakshmanan,
closesvim/vim#11858)
6ec6666047
Co-authored-by: Yegappan Lakshmanan <yegappan@yahoo.com>
