Problem: Visual selection isn't drawn with 'breakindent' when the line
doesn't fit in the window (Jaehwang Jung)
Solution: Adjust wlv->fromcol also for 'breakindent' (zeertzjq)
closes: vim/vim#13767closes: vim/vim#1376823627722d3
Problem: Default number column has incorrect width after 'statuscolumn'
is unset due to an error, but was also truncated.
Solution: Reverse 'statuscolumn' error and truncate return branches.
Problem: Unable to predict which byte-offset to place virtual text to
make it repeat visually in the wrapped part of a line.
Solution: Add a flag to nvim_buf_set_extmark() that causes virtual
text to repeat in wrapped lines.
We no longer return to the main loop in win_line() to put column
characters on the screen. Simplify and sync statuscolumn drawing
logic with that of statusline.
We do not need an enum to keep track of what place in win_line() we
currently are at. We already have a variable which keeps track where
in the code we currently are (and thus what part of the line we are
currently rendering), it is called the _program counter_. When we need
non-linear or self-referential control-flow anyway for a laugh, we
have a mechanism for that, it is called _function calls_.
Do not "save" and "restore" the wlv->n_extra state every time the
columns are to be drawn. This sort of thing needs to go away. Instead of
setting the n_extra variables and then going to the outer while loop,
the text in the columns can be rendered by just simply putting the text
into the cells of the screen line, right away. Even in nvim this can be
tricky sometimes, luckily we can use function calls to abstract this
logic, which means that this handy data structure called the _call
stack_ is handling saving away state temporarily, and restoring it back
when we need it again.
Lastly, but not least, as we now have direct control how signs
are rendered, these can be stored as schar_T[2] and be directly
put on screen as such.
Problem: 'breakindent' is not drawn after diff filler lines.
Solution: Correct check for whether 'breakindent' should be drawn.
closes: vim/vim#13624588f20dece
Cherry-pick Test_diff_with_syntax() change from patch 9.0.1257.
decor->text.str pointer must go. This removes it for conceal char,
in preparation for a larger PR which will also handle the sign case.
By actually allowing composing chars for a conceal chars, this
becomes a feature and not just a refactor, as a bonus.
Remove the monolithic Decoration struct. Before this change, each extmark
could either represent just a hl_id + priority value as a inline
decoration, or it would take a pointer to this monolitic 112 byte struct
which has to be allocated.
This change separates the decorations into two pieces: DecorSignHighlight
for signs, highlights and simple set-flag decorations (like spell,
ui-watched), and DecorVirtText for virtual text and lines.
The main separation here is whether they are expected to allocate more
memory. Currently this is not really true as sign text has to be an
allocated string, but the plan is to get rid of this eventually (it can
just be an array of two schar_T:s). Further refactors are expected to
improve the representation of each decoration kind individually. The
goal of this particular PR is to get things started by cutting the
Gordian knot which was the monolithic struct Decoration.
Now, each extmark can either contain chained indicies/pointers to
these kinds of objects, or it can fit a subset of DecorSignHighlight
inline.
The point of this change is not only to make decorations smaller in
memory. In fact, the main motivation is to later allow them to grow
_larger_, but on a dynamic, on demand fashion. As a simple example, it
would be possible to augment highlights to take a list of multiple
`hl_group`:s, which then would trivially map to a chain of multiple
DecorSignHighlight entries.
One small feature improvement included with this refactor itself, is
that the restriction that extmarks cannot be removed inside a decoration
provider has been lifted. These are instead safely lifetime extended
on a "to free" list until the current iteration of screen drawing is done.
NB: flags is a mess. but DecorLevel is useless, this slightly less so
Problem: The legacy signlist data structures and associated functions are
redundant since the introduction of extmark signs.
Solution: Store signs defined through the legacy commands in a hashmap, placed
signs in the extmark tree. Replace signlist associated functions.
Usage of the legacy sign commands should yield no change in behavior with the
exception of:
- "orphaned signs" are now always removed when the line it is placed on is
deleted. This used to depend on the value of 'signcolumn'.
- It is no longer possible to place multiple signs with the same identifier
in a single group on multiple lines. This will now move the sign instead.
Moreover, both signs placed through the legacy sign commands and through
|nvim_buf_set_extmark()|:
- Will show up in both |sign-place| and |nvim_buf_get_extmarks()|.
- Are displayed by increasing sign identifier, left to right.
Extmark signs used to be ordered decreasingly as opposed to legacy signs.
Problem: buffer text with composing chars are converted from UTF-8
to an array of up to seven UTF-32 values and then converted back
to UTF-8 strings.
Solution: Convert buffer text directly to UTF-8 based schar_T values.
The limit of the text size is now in schar_T bytes, which is currently
31+1 but easily could be raised as it no longer multiplies the size
of the entire screen grid when not used, the full size is only required
for temporary scratch buffers.
Also does some general cleanup to win_line text handling, which was
unnecessarily complicated due to multibyte rendering being an "opt-in"
feature long ago. Nowadays, a char is just a char, regardless if it consists
of one ASCII byte or multiple bytes.
We already have an extensive suite of static analysis tools we use,
which causes a fair bit of redundancy as we get duplicate warnings. PVS
is also prone to give false warnings which creates a lot of work to
identify and disable.
problem: checks for wp->w_p_rl are all over the place, making simple
things like "advance column one cell" incredibly complicated.
solution: always fill linebuf_char[] using an incrementing counter,
and then mirror the buffer as a post-processing step
This was "easier" that I first feared, because the stupid but simple
workaround for things like keeping linenumbers still left-right,
e.g. "mirror them and them mirror them once more" is more or less
what vim did already. So let's just keep doing that.
long is 32 bits on windows, while it is 64 bits on other architectures.
This makes the type suboptimal for a codebase meant to be
cross-platform. Replace it with more appropriate integer types.
Problem: Peeking and flushing output slows down execution.
Solution: Do not update the mode message when global_busy is set. Do not
flush when only peeking for a character. (Ken Takata)
cb574f4154
Problem: linebreak applies for leading whitespace
Solution: only apply linebreak, once we have found non-breakat chars in
the line
closes: vim/vim#13228closes: vim/vim#13243dd75fcfbdf
Co-authored-by: Christian Brabandt <cb@256bit.org>
The 'arabicshape' feature of vim is a transformation of unicode text to
make arabic and some related scripts look better at display time. In
particular the content of a cell will be adjusted depending on the
(original) content of the cells just before and after it.
This is implemented by the arabic_shape() function in nvim. Before this
commit, shaping was invoked in four different contexts:
- when rendering buffer text in win_line()
- in line_putchar() for rendering virtual text
- as part of grid_line_puts, used by messages and statuslines and
similar
- as part of draw_cmdline() for drawing the cmdline
This replaces all these with a post-processing step in grid_put_linebuf(),
which has become the entry point for all text rendering after recent
refactors.
An aim of this is to make the handling of multibyte text yet simpler.
One of the main reasons multibyte chars needs to be "parsed" into
codepoint arrays of composing chars is so that these could be inspected
for the purpose of shaping. This can likely be vastly simplified in many
contexts where only the total length (in bytes) and width of composed
char is needed.
This finalizes the long running refactor from the old TUI-focused grid
implementation where text-drawing cursor was not separated from the
visible cursor.
Still, the pattern of setting cursor position together with updating a
line was convenient. Introduce grid_line_cursor_goto() to still allow
this but now being explicit about it.
Only having batched drawing functions makes code involving drawing
a bit longer. But it is better to be explicit, and this highlights
cases where multiple small redraws can be grouped together. This was the
case for most of the changed places (messages, lastline, and :intro)
- Move vimoption_T to option.h
- option_defs.h is for option-related types
- option_vars.h corresponds to Vim's option.h
- option_defs.h and option_vars.h don't include each other
The screen grid refactors will continue until morale improves.
Jokes aside, this is quite a central installment in the series.
Before this refactor, there were two fundamentally distinct codepaths
for getting some text on the screen:
- the win_line() -> grid_put_linebuf() -> ui_line() call chain used for
buffer text, with linebuf_char as a temporary scratch buffer
- the grid_line_start/grid_line_puts/grid_line_flush() -> ui_line()
path used for every thing else: statuslines, messages and the command line.
Here the grid->chars[] array itself doubles as a scratch buffer.
With this refactor, the later family of functions still exist, however
they now as well render to linebuf_char just like win_line() did, and
grid_put_linebuf() is called in the end to calculate delta changes.
This means we don't need any duplicate logic for delta calculations anymore.
Later down the line, it will be possible to share more logic operating
on this scratch buffer, like doing 'rightleft' reversal and arabic
shaping as a post-processing step.
Problem: multispace wrong when scrolling horizontally
Solution: Update position in "multispace" or "leadmultispace" also in
skipped chars. Reorder conditions to be more consistent.
closes: vim/vim#13145closes: vim/vim#13147abc808112e
This is not used as part of the logic to actually implement TUI line wrapping
In vim (especially gvim) it is used to emulate terminal-style text
selection. But in nvim we don't do that, and have no plans to reintroduce it.
Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.
As an example, even a pretty modest 50*80 screen would consume
50*80*2*32 = 256000, i e a quarter megabyte
With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.
This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
glyph cache
This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.
When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing
A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
