It appears that used msgpack library is not able to parse back message created
by msgpack_rpc_from_object() if nesting level is too high, so log_server_msg now
cares about msgpack_unpack_next() return value. Also error message from
server_notifications_spec.lua is not readable if something is wrong (though at
least now it does not crash when parsing deeply nested structures).
log_server_msg() in the test reports
[msgpack-rpc] nvim -> client(1) [error] "parse error"
Current name is inappropriate for the following reasons:
1. It is often masked by local `loop` variables.
2. It cannot be searched for. There are many `loop` variables where `loop` is
some local variable. There are many cases when “loop” word is used in
a comment.
3. It is in any case bad idea to use a generic name as a name of the global
variable. Best if global has module prefix: this is why it is in `main.h`:
`main_loop` both stands for “a main loop” and “a loop defined in `main.*`”.
Since I have no idea how to list every occurrence of this variable method used
to rename it is “remove it from globals.h, try to compile, fix errors”. Thus if
some occurrence was hidden under false `#if` branch it was not replaced.
The call to `event_init()` was too late. `command_line_scan()` in `main()` could
already need the loop initialized. Ref https://github.com/neovim/neovim/issues/3045#issuecomment-123405833.
A consequence of this change is that it was necessary to move the
`channel_from_stdio()` call to `command_line_scan()` when embedded_mode is set.
- Improve the implementation of deferred/immediate events.
- Use the new queue module to change how/when events are queued/processed by
giving a private queue to each emitter.
- Immediate events(which only exist to break uv_run recursion) are now
represented in the `loop->fast_events` queue.
- Events pushed to child queues are propagated to the event loop main queue and
processed as K_EVENT keys.
API functions exposed via msgpack-rpc now fall into two categories:
- async functions, which are executed as soon as the request is parsed
- sync functions, which are invoked in nvim main loop when processing the
`K_EVENT special key
Only a few functions which can be safely executed in any context are marked as
async.
parse_msgpack() closes a channel's stream on EOF error and the stream's
close callback close_cb() is queued for the next libuv loop iteration.
When parse_msgpack() returns, it has freed the channel and the queued
stream callback will access this freed memory.
To prevent this, increase the channel's reference count and let the
stream's close callback call decref().
Fixes#3128
- New libuv/pty process abstraction with simplified API and no globals.
- Remove nvim/os/job*. Jobs are now a concept that apply only to programs
spawned by vimscript job* functions.
- Refactor shell.c/channel.c to use the new module, which brings a number of
advantages:
- Simplified API, less code
- No slots in the user job table are used
- Not possible to acidentally receive data from vimscript
- Implement job table in eval.c, which is now a hash table with unilimited job
slots and unique job ids.
- Simplify RStream/WStream API and make it more consistent with libuv.
- Move into the event loop layer(event subdirectory)
- Remove uv_helpers module.
- Simplify job/process internal modules/API.
- Unify RStream and WStream into a single structure. This is necessary because
libuv streams can be readable and writable at the same time(and because the
uv_helpers.c hack to associate multiple streams with libuv handle was removed)
- Make struct definition public, allowing more flexible/simple memory
management by users of the module.
- Adapt channel/job modules to cope with the changes.
- Add event loop abstraction module under src/nvim/event. The
src/nvim/event/loop module replaces src/nvim/os/event
- Remove direct dependency on libuv signal/timer API and use the new abstraction
instead.
- Replace all references to uv_default_loop() by &loop.uv, a new global variable
that wraps libuv main event loop but allows the event loop functions to be
reused in other contexts.
Extract the RBuffer class from rstream.c and reimplement it as a ring buffer,
a more efficient version that doesn't need to relocate memory.
The old rbuffer_read/rbuffer_write interfaces are kept for simple
reading/writing, and the RBUFFER_UNTIL_{FULL,EMPTY} macros are introduced to
hide wrapping logic when more control is required(such as passing the buffer
pointer to a library function that writes directly to the pointer)
Also add a basic infrastructure for writing helper C files that are only
compiled in the unit test library, and use this to write unit tests for RBuffer
which contains some macros that can't be accessed directly by luajit.
Helped-by: oni-link <knil.ino@gmail.com>
Reviewed-by: oni-link <knil.ino@gmail.com>
Reviewed-by: Scott Prager <splinterofchaos@gmail.com>
Reviewed-by: Justin M. Keyes <justinkz@gmail.com>
Reviewed-by: Michael Reed <m.reed@mykolab.com>
Consider: `let vim = rpcstart('nvim', ['--embed'])`
Allows `rpcnotify(vim, ...)` to work like an asynchronous
`rpcrequest(nvim, ...)`.
Helped-by: Michael Reed <m.reed@mykolab.com>
Helped-by: Justin M. Keyes <>
We already use wrappers for allocation, the new `xfree` function is the
equivalent for deallocation and provides a way to fully replace the malloc
implementation used by Neovim.
Klib pools were used to improve allocation efficiency for some small objects,
but it is not a thread-safe approach. Thread safety in allocations will be
required for implementing #2371).
channel_write() uses a ref-counted buffer for writing. This buffer
should be released if it was used in "refcount" channel_write() calls.
But calling channel_write() on a closed channel would return early and
not decrease the refcount of the used buffer.
Notifications for a channel will be sent directly if there are no
pending requests (for this channel). Otherwise notifications are queued
for later sending.
But in two cases a notification could be sent with pending requests:
* Broadcasting a notification
* A channel that has just finished its last pending request
would call send_delayed_notifications() for all channels.
To prevent this, every channel can now only send its own delayed
notifications and broadcasting checks for pending requests.
- Remove JobActivity autocmd and v:job_data variable
- Simplify `jobstart` to receive:
- An argument vector
- An optional dictionary which may contain any of the current `jobstart`
options plus `on_stdout`, `on_stderr` and `on_exit` callbacks.
- Refactor and add more job tests
- Update documentation
Add missing parentheses and whatnot, move dangling comment, etc. Some
specific items worth mentioning:
Fixed some references to non-existent tags, found via `make html`
msgpack_rpc/channel.c:
ELOG already prefixes each line with "error @ ..."
- process spawning was decoupled from the rest of the job control logic. The
goal is reusing it for spawning processes connected to pseudo terminal file
descriptors.
- job_start now receives a JobOptions structure containing all the startup
options.
- Removed term.c, term.h and term_defs.h
- Tests for T_* values were removed. screen.c was simplified as a
consequence(the best strategy for drawing is implemented in the UI layer)
- Redraw functions now call ui.c functions directly. Updates are flushed with
`ui_flush()`
- Removed all termcap options(they now return empty strings for compatibility)
- &term/&ttybuiltin options return a constant value(nvim)
- &t_Co is still available, but it mirrors t_colors directly
- Remove cursor tracking from screen.c and the `screen_start` function. Now the
UI is expected to maintain cursor state across any call, and reset it when
resized.
- Remove unused code
- Remove abstract_ui global, now it is always active
- Remove some terminal handling code
- Remove unused functions
- Remove HAVE_TGETENT/TERMINFO/TERMIOS/IOCTL #ifdefs
- Remove tgetent/terminfo from version.c
- Remove curses/terminfo dependencies
- Only start/stop termcap when starting/exiting the program
- msg_use_printf will return true if there are no attached UIs(
messages will be written to stdout)
- Remove `ex_winpos`(implement `:winpos` with `ex_ni`)
Problem : Read from pointer after free @ {242, 391}.
Diagnostic : Real issues.
Rationale : Channel gets indeed freed on error case, producing
incorrect accesses to freed pointer later on.
Resolution : Implement reference counting mechanism to know when to free
channel.
ASAN detected this heap-use-after-free.
A job started by channel_from_job() could terminate and result in a call
to free_channel(), while channel_send_call() was still active/pending
and accessing Channel elements.
Original patch by @tarruda.
