From a very high level point of view, Vim/Nvim can be described as state
machines following these instructions in a loop:
- Read user input
- Peform some action. The action is determined by the current state and can
switch states.
- Possibly display some feedback to the user.
This is not immediately visible because these instructions spread across dozens
of nested loops and function calls, making it very hard to modify the state
machine(to support more event types, for example).
So far, the approach Nvim has taken to allow more events is this:
- At the very core function that blocks for input, poll for arbitrary events.
- If the event received from the OS is user input, just return it normally to
the callers.
- If the event is not direct result of user input(possibly a vimscript function
call coming from a msgpack-rpc socket or a job control callback), return a
special key code(`K_EVENT`) that is handled by callers where it is safer to
perform arbitrary actions.
One problem with this approach is that the `K_EVENT` signal is being sent across
multiple states that may be unaware of it. This was partially fixed with the
`input_enable_events`/`input_disable_events` functions, which were added as a
mechanism that the upper layers can use to tell the core input functions that it
is ready to accept `K_EVENT`.
Another problem is that the mapping engine is implemented in getchar.c
which is called from every state, but the mapping engine is not aware of
`K_EVENT` so events can break mappings.
While it is theoretically possible to modify getchar.c to make it aware of
`K_EVENT`, this commit fixes the problem with a different approach: Model Nvim
as a pushdown automaton(https://en.wikipedia.org/wiki/Pushdown_automaton). This
design has many advantages which include:
- Decoupling the event loop from the states reponsible for handling events.
- Better control of state transition with less dependency on global variable
hacks(eg: 'restart_edit' global variable).
- Easier removal of global variables and function splitting. That is because
many variables are for state-specific information, and probably ended up being
global to simplify communication between functions, which we fix by storing
state-specific information in specialized structures.
The final goal is to let Nvim have a single top-level event loop represented by
the following pseudo-code:
```
while not quitting
let event = read_event
current_state(event)
update_screen()
```
This closely mirrors the state machine description above and makes it easier to
understand, extend and debug the program.
Note that while the pseudo code suggests an explicit stack of states that
doesn't rely on return addresses(as suggested by the principles of
automata-based programming:
https://en.wikipedia.org/wiki/Automata-based_programming), for now we'll use the
call stack as a structure to manage state transitioning as it would be very
difficult to refactor Nvim to use an explicit stack of states, and the benefits
would be small.
While this change may seem like an endless amount of work, it is possible to
do it incrementally as was shown in the previous commits. The general procedure
is:
1- Find a blocking `vgetc()`(or derivatives) call. This call represents an
implicit state of the program.
2- Split the code before and after the `vgetc()` call into functions that match
the signature of `state_check_callback` and `state_execute_callback.
Only `state_execute_callback` is required.
3- Create a `VimState` "subclass" and a initializer function that sets the
function pointers and performs any other required initialization steps. If
the state has no local variables, just use `VimState` without subclassing.
4- Instead of calling the original function containing the `vgetc()`,
initialize a stack-allocated `VimState` subclass, then call `state_enter` to
begin processing events in the state.
5- The check/execute callbacks can return 1 to continue normally, 0 to break the
loop or -1 to skip to the next iteration. These callbacks contain code that
execute before and after the old `vgetc()` call.
The functions created in step 2 may contain other `vgetc()` calls. These
represent implicit sub-states of the current state, but it is fine to remove
them later in smaller steps since we didn't break compatibility with existing
code.
This makes it impossible for K_EVENT to interfere with mappings, but it also
disables processing of events while in the middle of a mapping (Though this will
be fixed later as this refactoring progresses).
`may_sync_undo` is now called when K_EVENT is received. This is necessary to
correctly update undo entry lists before executing some action.
`normal_prepare` is now called by `normal_check` before returning 1(to
continue).
Also remove `input_{enable,disable}_events` calls from `normal_cmd`, which only
exists now as a compatibility function to run normal commands with keys inserted
into the typeahead buffer(We don't want to process events in these cases
anyway).
Refactor input.c, normal.c and edit.c to use the K_EVENT special key to trigger
the CURSORHOLD event. In normal and edit mode, K_EVENT is treated as
K_CURSORHOLD, which enables better handling of arbitrary actions in those
states(eg: In normal mode the previous operator counts will be restored).
Also fix a test in vim_spec.lua. The test had a wrong assumption: cmdheight is
only used to determine when the press enter screen will be shown, not to limit
how many lines or control pagination.
The new function contains logic that must be executed after handling input in
normal mode and also before the first main loop iteration. Also rename
`main_loop` to `normal_enter` and move it to normal.c
What works:
1. ShaDa file dumping: header, registers, jump list, history, search patterns,
substitute strings, variables.
2. ShaDa file reading: registers, global marks, variables.
Most was not tested.
TODO:
1. Merging.
2. Reading history, local marks, jump and buffer lists.
3. Documentation update.
4. Converting some data from &encoding.
5. Safer variant of dumping viminfo (dump to temporary file then rename).
6. Removing old viminfo code (currently masked with `#if 0` in a ShaDa file for
reference).
- new feature: if the first character of 'keywordprg' is ":", the
command is invoked as a Vim ex-command prefixed with [count].
- change default 'keywordprg' to :Man
- Improve the implementation of deferred/immediate events.
- Use the new queue module to change how/when events are queued/processed by
giving a private queue to each emitter.
- Immediate events(which only exist to break uv_run recursion) are now
represented in the `loop->fast_events` queue.
- Events pushed to child queues are propagated to the event loop main queue and
processed as K_EVENT keys.
- Add event loop abstraction module under src/nvim/event. The
src/nvim/event/loop module replaces src/nvim/os/event
- Remove direct dependency on libuv signal/timer API and use the new abstraction
instead.
- Replace all references to uv_default_loop() by &loop.uv, a new global variable
that wraps libuv main event loop but allows the event loop functions to be
reused in other contexts.
Problem: Redrawing problem with 'relativenumber' and 'linebreak'.
Solution: Temporarily reset 'linebreak' and restore it in more places.
(Christian Brabandt)
https://github.com/vim/vim/commit/v7-4-576Closes#1946
After left clicking on a tab in the tabline, the "in_tab_line" variable wasn't
set to 'false' and every following mouse action assumed still being on the
tabline which messed up visual selection etc.
We already use wrappers for allocation, the new `xfree` function is the
equivalent for deallocation and provides a way to fully replace the malloc
implementation used by Neovim.
Problem: A search with end offset gets stuck at end of file. (Gary Johnson)
Solution: When a search doesn't move the cursor repeat it with a higher
count. (Christian Brabandt)
https://github.com/vim/vim/releases/tag/v7-4-636
Switching cursor off is only necessary in two occasions:
- When redrawing to avoid terminal flickering
- When the editor is busy
The first can now be handled by the TUI, so most calls to ui_cursor_off can be
removed from the core.
So, before this commit it was only necessary to switch the cursor off to notify
the user that nvim was running some long operation. Now the cursor_{on,off}
functions have been replaced by busy_{stop,start} which can be handled in a
UI-specific way(turning the cursor off or showing a busy indicator, for
example).
To make things even more simpler, nvim is always busy except when waiting for
user input or other asynchronous events: It automatically switches to a non-busy
state when the event loop is about to be entered for more than 100 milliseconds.
`ui_busy_start` can be called when its not desired to change the busy state in
the event loop (As its now done by functions that perform blocking shell
invocations).
- Removed term.c, term.h and term_defs.h
- Tests for T_* values were removed. screen.c was simplified as a
consequence(the best strategy for drawing is implemented in the UI layer)
- Redraw functions now call ui.c functions directly. Updates are flushed with
`ui_flush()`
- Removed all termcap options(they now return empty strings for compatibility)
- &term/&ttybuiltin options return a constant value(nvim)
- &t_Co is still available, but it mirrors t_colors directly
- Remove cursor tracking from screen.c and the `screen_start` function. Now the
UI is expected to maintain cursor state across any call, and reset it when
resized.
- Remove unused code
- Remove abstract_ui global, now it is always active
- Remove some terminal handling code
- Remove unused functions
- Remove HAVE_TGETENT/TERMINFO/TERMIOS/IOCTL #ifdefs
- Remove tgetent/terminfo from version.c
- Remove curses/terminfo dependencies
- Only start/stop termcap when starting/exiting the program
- msg_use_printf will return true if there are no attached UIs(
messages will be written to stdout)
- Remove `ex_winpos`(implement `:winpos` with `ex_ni`)
Problem : Negative array index read @ 909.
Diagnostic : False positive.
Rationale : Suggested error path assigns a negative value to idx at
line 836 (`idx = find_command(ca.cmdchar);`). That's
impossible, as `ca.cmdchar` is set to Ctrl_BSL just two
lines above, so we know that value will be in the table.
Resolution : Assert idx >= 0.
If you Google for this phrase found in the Vim documentation you'll find
almost exclusively hits from the Vim documentation. I think changing
"halfway a line" to "halfway through a line" makes more sense.
There seems to be an pervasive odd use of the word 'halfway' in the
original docs which I'm updating everywhere.
Problem : Out-of-bound read from a buffer.
Diagnostic : False positive.
Rationale : nv_max_linear should always be less than nv_cmd_idx size
(NV_CMDS_SIZE).
Resolution : Assert rationale.
