According to `CaseFolding-15.1.0.txt`, full casefolding should be
preferred over simple casefolding as it's considered to be more correct.
Since utf8proc already provides full casefolding it makes sense to
switch to it. This will also remove a lot of unnecessary build code.
Temporary exceptions are made for two sets characters:
- `ß` will still be considered `ß` (instead of `ss`) as using a full
casefolding requires interfering with upstream spell files in some
form.
- `İ` will still be considered `İ` (instead of `i̇`) as using full
casefolding requires making a value judgement on the "correct"
behavior. There are two, equally valid case-insensetive comparison for
this character according to unicode. It is essentially up to the
implementor to decide which conversion is correct. For this reason it
might make sense to allow users to decide which conversion should be
done as an added option to `casemap` in a future PR.
In Vim a lot of filesystem functions have been moved to filepath.c.
However, some of these functions actually deal with file contents, and
Nvim's filesystem-related functions are spread out in a different way.
Therefore, it's better to use a different file for these functions.
This also makes shada reading slightly faster due to avoiding
some copying and allocation.
Use keysets to drive decoding of msgpack maps for shada entries.
Before this change, "static inline" functions in headers needed to have
their function attributes specified in a completely different way. The
prototype had to be duplicated, and REAL_FATTR_ had to be used instead
of the public FUNC_ATTR_ names.
TODO: need a check that a "header.h.inline.generated.h" file is not
forgotten when the first "static inline" function with attributes
is added to a header (they would just be silently missing).
Problem: Unsetting global variables earlier in #28578 to avoid
recursiveness, caused superfluous or even unlimited
showmode().
Solution: Partly revert #28578 so that the globals are unset at the end
of showmode(), and avoid recursiveness for ext UI by adding a
recursive function guard to each generated UI call that may
call a Lua callback.
Problem: Calling :redraw from vim.ui_attach() callback results in
recursive cmdline/message events.
Solution: Avoid recursiveness where possible and replace global "call_buf"
with separate, temporary buffers for each event so that when a Lua
callback for one event fires another event, that does not result
in invalid memory access.
There's no "rule" or bad practice or whatever that says we cannot
generate c files. it is is just that we have ~20 generated headers
and ~2 generated sources and there is nothing in these two generated
source files which sets them aparts. Lua bindings are not different from
rpc bindings, and pathdef is not different from versiondef.
So to simplify build logic and ease the future port to build.zig,
streamline the build to only have generated headers, no direct generated
.c files.
Also "nlua_add_api_functions" had its prototype duplicated twice which
defeated the point of having mandatory prototypes (one source of truth).
Problem: Currently, the `immutable` property of options can be applied for options that are hidden and options whose value simply can't be changed. Which is problematic when attempting to convert an option like `'maxcombine'` into an immutable option, because trying to `:set` an immutable option currently gives an error, which is only desired behavior for hidden options, not options that are actually immutable.
Solution: Separate the `immutable` property into two distinct `hidden` and `immutable` properties. Change all options with the `immutable` property to use the `hidden` property instead. Also add `p_mco` as an `immutable` option, as its value cannot be changed, and the underlying variable is not used anywhere.
Just some basic spring cleaning.
In the distant past, not all UI:s where remote UI:s. They still aren't,
but both of the "UI" and "UIData" structs are now only for remote UI:s.
Thus join them as "RemoteUI".
Before, we needed to always pack an entire msgpack_rpc Object to
a continous memory buffer before sending it out to a channel.
But this is generally wasteful. it is better to just flush
whatever is in the buffer and then continue packing to a new buffer.
This is also done for the UI event packer where there are some extra logic
to "finish" of an existing batch of nevents/ncalls. This doesn't really
stop us from flushing the buffer, just that we need to update the state
machine accordingly so the next call to prepare_call() always will
start with a new event (even though the buffer might contain overflow
data from a large event).
Then we can just load metadata in C as a single msgpack blob. Which also
can be used directly as binarly data, instead of first unpacking all the
functions and ui_events metadata to immediately pack it again, which was
a bit of a silly walk (and one extra usecase of `msgpack_rpc_from_object`
which will get yak shaved in the next PR)
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
As only a few API functions make use of explicit freeing of the return
value, make it opt-in instead. The arena is always present under the
hood, so `Arena *arena` arg now doesn't mean anything other than getting
access to this arena. Also it is in principle possible to return an
allocated value while still using the arena as scratch space for other
stuff (unlikely, but there no reason to not allow it).
Problem: Setting some options changes curswant unnecessarily.
Solution: Add a P_HLONLY flag that prevents changing curswant.
(zeertzjq)
closes: vim/vim#14044fcaed6a70f
Note: this contains two _temporary_ changes which can be reverted
once the Arena vs no-Arena distinction in API wrappers has been removed.
Both nlua_push_Object and object_to_vim_take_luaref() has been changed
to take the object argument as a pointer. This is not going to be
necessary once these are only used with arena (or not at all) allocated
Objects.
The object_to_vim() variant which leaves luaref untouched might need to
stay for a little longer.
and for return value of nlua_exec/nlua_call_ref, as this uses
the same family of functions.
NB: the handling of luaref:s is a bit of a mess.
add api_luarefs_free_XX functions as a stop-gap as refactoring
luarefs is a can of worms for another PR:s.
as a minor feature/bug-fix, nvim_buf_call and nvim_win_call now preserves
arbitrary return values.
Currently having two separate memory strategies for API return values is
a bit unnecessary, and mostly a consequence of converting the hot spot
cases which needed it first. But there is really no downside to using
arena everywhere (which implies also directly using strings which are
allocated earlier or even statically, without copy).
There only restriction is we need to know the size of arrays in advance,
but this info can often be passed on from some earlier stage if it is
missing.
This collects some "small" cases. The more complex stuff will get a PR
each.
Implement api_keydict_to_dict as the complement to api_dict_to_keydict
Fix a conversion error when nvim_get_win_config gets called from lua,
where Float values "x" and "y" didn't get converted to lua numbers.
Problem:
APIs get wrong boolean option default values on big-endian platforms.
Solution:
Use a union for def_val.
Cannot use OptVal or OptValData yet as it needs to have the same types
as option variables.
"export" only prevents IWYU from adding these headers if the headers
that export them are included, while "private" ensures that IWYU never
adds these headers.
It isn't really useful to put anonymous enums only used as arguments to
functions calls in _defs.h headers, as they will only be used by a file
that calls those functions, which requires including a non-defs header.
Also move os_msg() and os_errmsg() back to message.h, as on Windows they
are actual functions instead of macros.
Also remove gettext.h and globals.h from private/helpers.h.
Problem:
`findoption()` searches through the options[] table linearly for option
names, even though hashy can be used to generate a compile-time hash
table for it.
Solution:
Use hashy to generate a compile time hash table for finding options.
This also allows handling option aliases, so we don't need separate
options[] table entries for things like 'viminfo'.
Problem: We have `P_(BOOL|NUM|STRING)` macros to represent an option's type, which is redundant because `OptValType` can already do that. The current implementation of option type flags is also too limited to allow adding multitype options in the future.
Solution: Remove `P_(BOOL|NUM|STRING)` and replace it with a new `type_flags` attribute in `vimoption_T`. Also do some groundwork for adding multitype options in the future.
Side-effects: Attempting to set an invalid keycode option (e.g. `set t_foo=123`) no longer gives an error.
Add a macro to indicate the option count so that we can iterate through the options[] table more clearly. This also removes the need for having an option with NULL fullname at the end of `options[]`.
Problem: Many places in the code use `findoption()` to access an option using its name, even if the option index is available. This is very slow because it requires looping through the options array over and over.
Solution: Use option index instead of name wherever possible. Also introduce an `OptIndex` enum which contains the index for every option as enum constants, this eliminates the need to pass static option names as strings.
Problem:
Not all Lua code is checked by stylua. Automating code-style is an
important mechanism for reducing time spent on accidental
(non-essential) complexity.
Solution:
- Enable lintlua for `src/` directory.
followup to 517f0cc634
FUNC_ATTR_* should only be used in .c files with generated headers.
Defining FUNC_ATTR_* as empty in headers causes misuses of them to be
silently ignored. Instead don't define them by default, and only define
them as empty after a .c file has included its generated header.
Problem: We use the `p_force_on` and `p_force_off` variables to check if a variable is immutable and what its default value is. This is not only hacky and unintuitive, but also is limited to only boolean options.
Solution: Replace `p_force_on` and `p_force_off` with an `immutable` property for options, which indicates if an option is immutable. Immutable options cannot be changed from their default value.
Ref: #25672.
We already have an extensive suite of static analysis tools we use,
which causes a fair bit of redundancy as we get duplicate warnings. PVS
is also prone to give false warnings which creates a lot of work to
identify and disable.
long is 32 bits on windows, while it is 64 bits on other architectures.
This makes the type suboptimal for a codebase meant to be
cross-platform. Replace it with more appropriate integer types.
