refactor(grid): change schar_T representation to be more compact

Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.

As an example, even a pretty modest 50*80 screen would consume

50*80*2*32 = 256000, i e a quarter megabyte

With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.

This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
  glyph cache

This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.

When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing

A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
This commit is contained in:
bfredl
2023-09-13 13:39:18 +02:00
parent 46402c16c0
commit 8da986ea87
25 changed files with 439 additions and 171 deletions

View File

@@ -9,9 +9,13 @@
#include "nvim/types.h"
#define MAX_MCO 6 // fixed value for 'maxcombine'
// Includes final NUL. at least 4*(MAX_MCO+1)+1
#define MAX_SCHAR_SIZE 32
// The characters and attributes drawn on grids.
typedef char schar_T[(MAX_MCO + 1) * 4 + 1];
// if data[0] is 0xFF, then data[1..4] is a 24-bit index (in machine endianess)
// otherwise it must be a UTF-8 string of length maximum 4 (no NUL when n=4)
typedef uint32_t schar_T;
typedef int sattr_T;
enum {
@@ -41,6 +45,9 @@ enum {
///
/// vcols[] contains the virtual columns in the line. -1 means not available
/// (below last line), MAXCOL means after the end of the line.
/// -2 or -3 means in fold column and a mouse click should:
/// -2: open a fold
/// -3: close a fold
///
/// line_offset[n] is the offset from chars[], attrs[] and vcols[] for the start
/// of line 'n'. These offsets are in general not linear, as full screen scrolling