We already have an extensive suite of static analysis tools we use,
which causes a fair bit of redundancy as we get duplicate warnings. PVS
is also prone to give false warnings which creates a lot of work to
identify and disable.
refactor: use a more idiomatic loop to iterate over the cells
There are two cases in which the following assertion would fail:
```c
assert(g->icell < g->ncells);
```
1. If `g->ncells = 0`. Update this to be legal.
2. If an EOF is reached while parsing `wrap`. In this case, the unpacker
attempts to resume from `cells`, which is a bug. Create a new state
for parsing `wrap`.
Reference: https://neovim.io/doc/user/ui.html#ui-event-grid_line
Problem: The style guide states that all switch statements that are not conditional on an enum must have a `default` case, but does not give any explicit guideline for switch statements that are conditional on enums. As a result, a `default` case is added in many enum switch statements, even when the switch statement is exhaustive. This is not ideal because it removes the ability to have compiler errors to easily detect unchanged switch statements when a new possible value for an enum is added.
Solution: Add explicit guidelines for switch statements that are conditional on an enum, clarifying that a `default` case is not necessary if the switch statement is exhaustive. Also refactor pre-existing code with unnecessary `default` cases.
Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.
As an example, even a pretty modest 50*80 screen would consume
50*80*2*32 = 256000, i e a quarter megabyte
With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.
This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
glyph cache
This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.
When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing
A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
This involves two redesigns of the map.c implementations:
1. Change of macro style and code organization
The old khash.h and map.c implementation used huge #define blocks with a
lot of backslash line continuations.
This instead uses the "implementation file" .c.h pattern. Such a file is
meant to be included multiple times, with different macros set prior to
inclusion as parameters. we already use this pattern e.g. for
eval/typval_encode.c.h to implement different typval encoders reusing a
similar structure.
We can structure this code into two parts. one that only depends on key
type and is enough to implement sets, and one which depends on both key
and value to implement maps (as a wrapper around sets, with an added
value[] array)
2. Separate the main hash buckets from the key / value arrays
Change the hack buckets to only contain an index into separate key /
value arrays
This is a common pattern in modern, state of the art hashmap
implementations. Even though this leads to one more allocated array, it
is this often is a net reduction of memory consumption. Consider
key+value consuming at least 12 bytes per pair. On average, we will have
twice as many buckets per item.
Thus old implementation:
2*12 = 24 bytes per item
New implementation
1*12 + 2*4 = 20 bytes per item
And the difference gets bigger with larger items.
One might think we have pulled a fast one here, as wouldn't the average size of
the new key/value arrays be 1.5 slots per items due to amortized grows?
But remember, these arrays are fully dense, and thus the accessed memory,
measured in _cache lines_, the unit which actually matters, will be the
fully used memory but just rounded up to the nearest cache line
boundary.
This has some other interesting properties, such as an insert-only
set/map will be fully ordered by insert only. Preserving this ordering
in face of deletions is more tricky tho. As we currently don't use
ordered maps, the "delete" operation maintains compactness of the item
arrays in the simplest way by breaking the ordering. It would be
possible to implement an order-preserving delete although at some cost,
like allowing the items array to become non-dense until the next rehash.
Finally, in face of these two major changes, all code used in khash.h
has been integrated into map.c and friends. Given the heavy edits it
makes no sense to "layer" the code into a vendored and a wrapper part.
Rather, the layered cake follows the specialization depth: code shared
for all maps, code specialized to a key type (and its equivalence
relation), and finally code specialized to value+key type.
Added to support MessagePack-RPC fully compliant clients that do
not return responses in request order.
Although it is currently not an efficient implementation for full
compliance and full compliance cannot be guaranteed, the addition
of the new client type `msgpack-rpc` creates a situation where "if
the client type is `msgpack-rpc`, then backward compatibility is
ignored and full compliance with MessagePack- RPC compliance is
justified even if backward compatibility is ignored if the client
type is `msgpack-rpc`.
Adds new API helper macros `CSTR_AS_OBJ()`, `STATIC_CSTR_AS_OBJ()`, and `STATIC_CSTR_TO_OBJ()`, which cleans up a lot of the current code. These macros will also be used extensively in the upcoming option refactor PRs because then API Objects will be used to get/set options. This PR also modifies pre-existing code to use old API helper macros like `CSTR_TO_OBJ()` to make them cleaner.
This reduces the total number of khash_t instantiations from 22 to 8.
Make the khash internal functions take the size of values as a runtime
parameter. This is abstracted with typesafe Map containers which
are still specialized for both key, value type.
Introduce `Set(key)` type for when there is no value.
Refactor shada.c to use Map/Set instead of khash directly.
This requires `map_ref` operation to be more flexible.
Return pointers to both key and value, plus an indicator for new_item.
As a bonus, `map_key` is now redundant.
Instead of Map(cstr_t, FileMarks), use a pointer map as the FileMarks struct is
humongous.
Make `event_strings` actually work like an intern pool instead of wtf it
was doing before.
problem: can we have Serde?
solution: we have Serde at home
This by itself is just a change of notation, that could be quickly
merged to avoid messy merge conflicts, but upcoming changes are planned:
- keysets no longer need to be defined in one single file. `keysets.h` is
just the initial automatic conversion of the previous `keysets.lua`.
keysets just used in a single api/{scope}.h can be moved to that file, later on.
- Typed dicts will have more specific types than Object. this will
enable most of the existing manual typechecking boilerplate to be eliminated.
We will need some annotation for missing value, i e a boolean will
need to be represented as a TriState (none/false/true) in some cases.
- Eventually: optional parameters in form of a `Dict opts` final
parameter will get added in some form to metadata. this will require
a discussion/desicion about type forward compatibility.
This commit implements the ability to control all of the XDG paths
Neovim should use. This is done by setting an environment variable named
NVIM_APPNAME. For example, setting $NVIM_APPNAME makes Neovim look for
its configuration directory in $XDG_CONFIG_HOME/$NVIM_APPNAME instead of
$XDG_CONFIG_HOME/nvim.
If NVIM_APPNAME is not set or is an empty string, "nvim" will be used as
default.
The usecase for this feature is to enable an easy way to switch from
configuration to configuration. One might argue that the various $XDG
environment variables can already be used for this usecase. However,
setting $XDG environment variables also affects tools spawned by Neovim.
For example, while setting $XDG_CONFIG_HOME will enable Neovim to use a
different configuration directory, it will also prevent Git from finding
its "default" configuration.
Closes https://github.com/neovim/neovim/issues/21691
Problem:
Tests that _intentionally_ fail certain conditions cause noise in
$NVIM_LOG_FILE:
$NVIM_LOG_FILE: /home/runner/work/neovim/neovim/build/.nvimlog
(last 100 lines)
WRN 2023-01-16T18:26:27.673 T599.7799.0 unsubscribe:519: RPC: ch 1: tried to unsubscribe unknown event 'doesnotexist'
WRN 2023-01-16T18:29:00.557 ?.11151 server_start:163: Failed to start server: no such file or directory: /X/X/X/...
WRN 2023-01-16T18:33:07.269 127.0.0.1:12345 server_start:163: Failed to start server: address already in use: 127.0.0.1
...
-- Output to stderr:
module 'vim.shared' not found:
no field package.preload['vim.shared']
no file './vim/shared.lua'
no file '/home/runner/nvim-deps/usr/share/lua/5.1/vim/shared.lua'
no file '/home/runner/nvim-deps/usr/share/lua/5.1/vim/shared/init.lua'
no file '/home/runner/nvim-deps/usr/lib/lua/5.1/vim/shared.lua'
no file '/home/runner/nvim-deps/usr/lib/lua/5.1/vim/shared/init.lua'
no file './vim/shared.so'
...
E970: Failed to initialize builtin lua modules
Solution:
- Log to a private $NVIM_LOG_FILE in tests that intentionally fail and
cause ERR log messages.
- Assert that the expected messages are actually logged.
Fix remaining clint errors and remove error suppression completely.
Rename the lint targets to align with the established naming convention:
- lintc-clint lints with clint.py.
- lintc-uncrustify lints with uncrustify.
- lintc runs both targets.
lintc is also provided as a make target for convenience.
After this change we can remove these files:
https://github.com/neovim/doc/tree/gh-pages/reports/clinthttps://github.com/neovim/doc/blob/main/ci/clint-errors.sh
* fix: log and clear error in ui_comp_event
* fix: handling error in each map_foreach_value iteration
* fix: handling error decl in for_each loop
* fix: updating initerr to const, removing initerr free-ing
* fix: using ERROR_SET for error check
* fix: wrapping ERROR_INIT in parens to allow for including inside macro
* fix(PVS/V009): start file with special comment
* fix(PVS/V501): identical sub-expressions for comparison
* fix(PVS/V560): part of conditional expression is always true/false
* fix(PVS/V593): review expression of type A = B < C
* fix(PVS/V614): potentially uninitialized variable used
Allow Include What You Use to remove unnecessary includes and only
include what is necessary. This helps with reducing compilation times
and makes it easier to visualise which dependencies are actually
required.
Work on https://github.com/neovim/neovim/issues/549, but doesn't close
it since this only works fully for .c files and not headers.
Problem: Macros for MS-Windows are inconsistent, using "32", "3264 and
others.
Solution: Use MSWIN for all MS-Windows builds. Use FEAT_GUI_MSWIN for the
GUI build. (Hirohito Higashi, closesvim/vim#3932)
4f97475d32
In the rare case that exit_event is called from process_close_handles,
it stalls waiting for the process to exit (the routine is currently
underway to do just that). This causes `job_spec.lua` to sometimes
stall.
REJECTED IDEAS:
==============================================================
1. Currently `exit_event` is placed on `main_loop.fast_events`. Would the problem
be solved by using `main_loop.events` instead?
- A: Maybe, but it will cause other problems, such as queuing exit_event()
during "Press Enter..." prompt which may result in the event not being
processed, leading to another stall.
2. Can we avoid the timer?
- A: Using a timer is just the easiest way to queue a delayed event without
causing an infinite loop in the queue currently being processed.
3. Can we avoid the new `exit_need_delay` global...
1. by using `process_is_tearing_down` instead?
- A: Can't use `process_is_tearing_down` because its semantics are different.
2. by checking a similar condition as `process_teardown`? f50135a32e/src/nvim/event/process.c (L141-L142)
```
if (!process_is_tearing_down || (kl_empty(main_loop.children) && multiqueue_empty(main_loop.events))) {
uv_timer_start(&main_loop.exit_delay_timer, exit_delay_cb, 0, 0);
return;
}
```
- A: Tried but it did not work (other stalls occurred). Maybe
exit_event() is called from a source other than
process_close_handles() and is delayed, the delayed exit_event() will
be executed before main_loop.events is processed, resulting in an
infinite loop.
Problem:
- Since c57f6b28d7#8519, sockets are created in ~/.local/… but XDG
spec says: "XDG_RUNTIME_DIR: Must be on the local filesystem", which
implies that XDG_STATE_DIR is potentially non-local.
- Not easy to inspect Nvim-created temp files (for debugging etc).
Solution:
- Store sockets in stdpath('run') ($XDG_RUNTIME_DIR).
- Establish "/tmp/nvim.user/" as the tempdir root shared by all Nvims.
- Make ok() actually useful.
- Introduce assert_nolog().
closes#3517closes#17093
