This involves two redesigns of the map.c implementations:
1. Change of macro style and code organization
The old khash.h and map.c implementation used huge #define blocks with a
lot of backslash line continuations.
This instead uses the "implementation file" .c.h pattern. Such a file is
meant to be included multiple times, with different macros set prior to
inclusion as parameters. we already use this pattern e.g. for
eval/typval_encode.c.h to implement different typval encoders reusing a
similar structure.
We can structure this code into two parts. one that only depends on key
type and is enough to implement sets, and one which depends on both key
and value to implement maps (as a wrapper around sets, with an added
value[] array)
2. Separate the main hash buckets from the key / value arrays
Change the hack buckets to only contain an index into separate key /
value arrays
This is a common pattern in modern, state of the art hashmap
implementations. Even though this leads to one more allocated array, it
is this often is a net reduction of memory consumption. Consider
key+value consuming at least 12 bytes per pair. On average, we will have
twice as many buckets per item.
Thus old implementation:
2*12 = 24 bytes per item
New implementation
1*12 + 2*4 = 20 bytes per item
And the difference gets bigger with larger items.
One might think we have pulled a fast one here, as wouldn't the average size of
the new key/value arrays be 1.5 slots per items due to amortized grows?
But remember, these arrays are fully dense, and thus the accessed memory,
measured in _cache lines_, the unit which actually matters, will be the
fully used memory but just rounded up to the nearest cache line
boundary.
This has some other interesting properties, such as an insert-only
set/map will be fully ordered by insert only. Preserving this ordering
in face of deletions is more tricky tho. As we currently don't use
ordered maps, the "delete" operation maintains compactness of the item
arrays in the simplest way by breaking the ordering. It would be
possible to implement an order-preserving delete although at some cost,
like allowing the items array to become non-dense until the next rehash.
Finally, in face of these two major changes, all code used in khash.h
has been integrated into map.c and friends. Given the heavy edits it
makes no sense to "layer" the code into a vendored and a wrapper part.
Rather, the layered cake follows the specialization depth: code shared
for all maps, code specialized to a key type (and its equivalence
relation), and finally code specialized to value+key type.
Removes the `getoption_T` struct and also introduces the `OptVal` struct
to unify the methods of getting/setting different option value types.
This is the first of many PRs to reduce code duplication in the Vim
option code as well as to make options easier to maintain. It also
increases the flexibility and extensibility of options. Which opens the
door for things like Array and Dictionary options.
Adds new API helper macros `CSTR_AS_OBJ()`, `STATIC_CSTR_AS_OBJ()`, and `STATIC_CSTR_TO_OBJ()`, which cleans up a lot of the current code. These macros will also be used extensively in the upcoming option refactor PRs because then API Objects will be used to get/set options. This PR also modifies pre-existing code to use old API helper macros like `CSTR_TO_OBJ()` to make them cleaner.
This reduces the total number of khash_t instantiations from 22 to 8.
Make the khash internal functions take the size of values as a runtime
parameter. This is abstracted with typesafe Map containers which
are still specialized for both key, value type.
Introduce `Set(key)` type for when there is no value.
Refactor shada.c to use Map/Set instead of khash directly.
This requires `map_ref` operation to be more flexible.
Return pointers to both key and value, plus an indicator for new_item.
As a bonus, `map_key` is now redundant.
Instead of Map(cstr_t, FileMarks), use a pointer map as the FileMarks struct is
humongous.
Make `event_strings` actually work like an intern pool instead of wtf it
was doing before.
Previously, there were three low-level delay entry points
- os_delay(ms, ignoreinput=true): sleep for ms, only break on got_int
- os_delay(ms, ignoreinput=false): sleep for ms, break on any key input
os_microdelay(us, false): equivalent, but in μs (not directly called)
- os_microdelay(us, true): sleep for μs, never break.
The implementation of the latter two both used uv_cond_timedwait()
This could have been for two reasons:
1. allow another thread to "interrupt" the wait
2. uv_cond_timedwait() has higher resolution than uv_sleep()
However we (1) never used the first, even when TUI was a thread, and
(2) nowhere in the codebase are we using μs resolution, it is always a ms
multiplied with 1000.
In addition, os_delay(ms, false) would completely block the thread for
100ms intervals and in between check for input. This is not how event handling
is done alound here.
Therefore:
Replace the implementation of os_delay(ms, false) to use
LOOP_PROCESS_EVENTS_UNTIL which does a proper epoll wait with a timeout,
instead of the 100ms timer panic.
Replace os_microdelay(us, false) with a direct wrapper of uv_sleep.
Problem:
has('gui_running') is still common in the wild and our answer has
changed over time, causing frustration.
95a6ccbe9f
Solution:
Use stdin_tty/stdout_tty to decide if a UI is (not) a GUI.
Problem:
- API validation involves too much boilerplate.
- API validation errors are not consistently worded.
Solution:
Introduce some macros. Currently these are clumsy, but they at least
help with consistency and avoid some nesting.
- <expr> mapping has no business saving and restoring the
low-level UI cursor. The cursor will be put in a reasonable
position after input is processed, chill out.
- TUI handles output needed for suspend
- vgetc() family of function does flushing
Problem: Code is indented more than necessary.
Solution: Use an early return where it makes sense. (Yegappan Lakshmanan,
closesvim/vim#11813)
e857598896
Partial port as this depends on some previous eval and 'smoothscroll'
patches.
Co-authored-by: Yegappan Lakshmanan <yegappan@yahoo.com>
- The defined interface for the UI is only the RPC protocol. The original
UI interface as an array of function pointers fill no function.
- On the server, all the UI:s are all RPC channels.
- ui.c is only used on the server.
- The compositor is a preprocessing step for single-grid UI:s
- on the client, ui_client and tui talk directly to each other
- we still do module separation, as ui_client.c could form the basis
of a libnvim client module later.
Items for later PR:s
- vim.ui_attach is still an unhappy child, reconsider based on plugin experience.
- the flags in ui_events.in.h are still a mess. Can be simplified now.
- UX for remote attachment needs more work.
- startup for client can be simplified further (think of the millisecs we can save)
This was previously disabled due to build issues on windows.
Any reasonable platform can now be expected to have the necessary
interfaces to build and run the TUI subsystem.
Runtime quality issues of using the TUI (on any new platform) are not
relevant here. Just run Nvim in an external UI instead of the TUI as always.
Allow Include What You Use to remove unnecessary includes and only
include what is necessary. This helps with reducing compilation times
and makes it easier to visualise which dependencies are actually
required.
Work on https://github.com/neovim/neovim/issues/549, but doesn't close
it since this only works fully for .c files and not headers.
Problem: Assert_fails() setting emsg_silent changes normal execution.
Solution: Use a separate flag in_assert_fails.
28ee892ac4
Cherry-pick no_wait_return from patch 9.0.0846.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: The screen.c file is much too big.
Solution: Split it in three parts. (Yegappan Lakshmanan, closesvim/vim#4943)
7528d1f6b5
This is an approximation vim-patch 8.1.2057. Applying the patch directly
isn't feasible since our version of screen.c has diverged too much,
however we still introduce drawscreen.c and drawline.c:
- screen.c is now a much smaller file used for low level screen functions
- drawline.c contains everything needed for win_line()
- drawscreen.c contains everything needed for update_screen()
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
vim-patch:8.1.2082: some files have a weird name to fit in 8.3 characters
Problem: Some files have a weird name to fit in 8.3 characters.
Solution: Use a nicer names.
30e8e73506
Note for external UIs: Nvim can now emit multiple "redraw" event batches
before a final "flush" event is received. To retain existing behavior,
clients should make sure to update visible state at an explicit "flush"
event, not just the end of a "redraw" batch of event.
* Get rid of copy_object() blizzard in the auto-generated ui_event layer
* Special case "grid_line" by encoding screen state directly to
msgpack events with no intermediate API events.
* Get rid of the arcane notion of referring to the screen as the "shell"
* Array and Dictionary are kvec_t:s, so define them as such.
* Allow kvec_t:s, such as Arrays and Dictionaries, to be allocated with
a predetermined size within an arena.
* Eliminate redundant capacity checking when filling such kvec_t:s
with values.
Problem:
Piping NodeJS output into Neovim makes the editor unusable.
This happens because NodeJS changes the tty state on exit after
Nvim calls uv_tty_set_mode(). (May not always happen due to race
condition.)
This should have been fixed by 4ba5b4a864#13084. But some
commands and functions (:sleep, system(), …) call ui_flush()
internally, in particular the first tui_mode_change() is called before
the end of startup.
Steps to reproduce:
1. node -e "setTimeout(()=>{console.log('test')}, 1000)" | nvim -u NORC +"sleep 500m" -
2. The cursor key letters just overwrite the editor screen, and CTRL+C exits.
Solution:
Skip pending_mode_update during startup.
Note: Delaying ui_flush() entirely could be a more general solution
(emit a new UI event on VimEnter?). But "remote/coprocess TUI" #18375
could make all of this moot anyway.
Fixes#18470
drawback: tracing memory errors with ASAN is less accurate for arena
allocated memory.
Therefore, to start with it is being used for Object types around
serialization/deserialization exclusively. This is going to have
a large impact especially when TUI is refactored as a co-prosess
as all UI events will be serialized and deserialized by nvim itself.
