Current name is inappropriate for the following reasons:
1. It is often masked by local `loop` variables.
2. It cannot be searched for. There are many `loop` variables where `loop` is
some local variable. There are many cases when “loop” word is used in
a comment.
3. It is in any case bad idea to use a generic name as a name of the global
variable. Best if global has module prefix: this is why it is in `main.h`:
`main_loop` both stands for “a main loop” and “a loop defined in `main.*`”.
Since I have no idea how to list every occurrence of this variable method used
to rename it is “remove it from globals.h, try to compile, fix errors”. Thus if
some occurrence was hidden under false `#if` branch it was not replaced.
The whole stream buffer is now put on screen at once instead of only
data up to the last newline. This has some advantages:
* RBuffer cannot wrap around, so we never forget to output second
half of the buffer.
* Stream data is not delayed anymore, because we don't have to wait for
a newline.
This works by remembering the last used screen column.
Problem: On some systems automatically adding the missing EOL causes
problems. Setting 'binary' has too many side effects.
Solution: Add the 'fixeol' option, default on. (Pavel Samarkin)
34d72d4b6c
The new event processing architecture changed `jobwait()` semantics: Only one
job is processed at time since process_wait only focuses on one queue.
This fixes the problem with a few changes:
- Allow the event queue polled by `process_wait` to be overriden by a new
argument.
- Allow the parent queue to be overriden with `queue_replace_parent`
- Create a temporary queue that serves as the parent for all jobs passed to
`jobwait()`
- Improve the implementation of deferred/immediate events.
- Use the new queue module to change how/when events are queued/processed by
giving a private queue to each emitter.
- Immediate events(which only exist to break uv_run recursion) are now
represented in the `loop->fast_events` queue.
- Events pushed to child queues are propagated to the event loop main queue and
processed as K_EVENT keys.
out_data_cb() can return without emptying the full RBuffer (no NL was
seen). Because the shell output stream is stopped until space in the
Rbuffer is freed up, no more shell output is written.
To prevent this, output the full RBuffer when write_output() did not
write anything.
write_output() can also process the same RBuffer content more than once,
if no NL was seen. To prevent NUL bytes from producing new lines (if
lines are not written to a buffer), translate NUL to SOH(1).
Fixes#2983
- New libuv/pty process abstraction with simplified API and no globals.
- Remove nvim/os/job*. Jobs are now a concept that apply only to programs
spawned by vimscript job* functions.
- Refactor shell.c/channel.c to use the new module, which brings a number of
advantages:
- Simplified API, less code
- No slots in the user job table are used
- Not possible to acidentally receive data from vimscript
- Implement job table in eval.c, which is now a hash table with unilimited job
slots and unique job ids.
- Simplify RStream/WStream API and make it more consistent with libuv.
- Move into the event loop layer(event subdirectory)
- Remove uv_helpers module.
- Simplify job/process internal modules/API.
- Unify RStream and WStream into a single structure. This is necessary because
libuv streams can be readable and writable at the same time(and because the
uv_helpers.c hack to associate multiple streams with libuv handle was removed)
- Make struct definition public, allowing more flexible/simple memory
management by users of the module.
- Adapt channel/job modules to cope with the changes.
- Add event loop abstraction module under src/nvim/event. The
src/nvim/event/loop module replaces src/nvim/os/event
- Remove direct dependency on libuv signal/timer API and use the new abstraction
instead.
- Replace all references to uv_default_loop() by &loop.uv, a new global variable
that wraps libuv main event loop but allows the event loop functions to be
reused in other contexts.
Extract the RBuffer class from rstream.c and reimplement it as a ring buffer,
a more efficient version that doesn't need to relocate memory.
The old rbuffer_read/rbuffer_write interfaces are kept for simple
reading/writing, and the RBUFFER_UNTIL_{FULL,EMPTY} macros are introduced to
hide wrapping logic when more control is required(such as passing the buffer
pointer to a library function that writes directly to the pointer)
Also add a basic infrastructure for writing helper C files that are only
compiled in the unit test library, and use this to write unit tests for RBuffer
which contains some macros that can't be accessed directly by luajit.
Helped-by: oni-link <knil.ino@gmail.com>
Reviewed-by: oni-link <knil.ino@gmail.com>
Reviewed-by: Scott Prager <splinterofchaos@gmail.com>
Reviewed-by: Justin M. Keyes <justinkz@gmail.com>
Reviewed-by: Michael Reed <m.reed@mykolab.com>
Problem : Argument with 'nonnull' attribute passed null @ 203.
Diagnostic : False positive.
Rationale : Problem is supposed to appear when argv[0] is NULL within
do_os_system. But argv is being generated by
shell_build_argv(), which implies argv[0] is the current
value for 'shell' option. Now, option has a non-null
default ($SHELL or "sh"), and, if set by the user, it can
be empty, but not NULL. So, argv[0] can never be NULL.
Resolution : Assert shell_build_argv() postcondition.
For any of these functions, if {cmd} is a string, execute
"&shell &shellcmdflag '{cmd}'", or simply {cmd} if it's a list.
In termopen(), if the 'name' option is not supplied, try to guess using
'{cmd}' (string) or {cmd}[0] (list). Simplify ex_terminal to use the
string form of termopen().
termopen: get name from argument
Convert list_to_argv to tv_to_argv.
Helped-by: Björn Linse <@bfredl>
Helped-by: oni-link <knil.ino@gmail.com>
Helped-by: Thiago de Arruda <@tarruda>
We already use wrappers for allocation, the new `xfree` function is the
equivalent for deallocation and provides a way to fully replace the malloc
implementation used by Neovim.
Even though assuming nvim is busy most times is simpler, it has a problem: A lot
of unnecessary busy_start/busy_stop notifications are sent to the UI. That's
because in the majority of scenarios almost no time is spent between
`event_poll` calls.
This restores the normal behavior which is to call busy_start only when nvim is
going to perform some task that can take a significant amount of time. Also
improve the usage of buffering in the TUI when changing the cursor state.
Switching cursor off is only necessary in two occasions:
- When redrawing to avoid terminal flickering
- When the editor is busy
The first can now be handled by the TUI, so most calls to ui_cursor_off can be
removed from the core.
So, before this commit it was only necessary to switch the cursor off to notify
the user that nvim was running some long operation. Now the cursor_{on,off}
functions have been replaced by busy_{stop,start} which can be handled in a
UI-specific way(turning the cursor off or showing a busy indicator, for
example).
To make things even more simpler, nvim is always busy except when waiting for
user input or other asynchronous events: It automatically switches to a non-busy
state when the event loop is about to be entered for more than 100 milliseconds.
`ui_busy_start` can be called when its not desired to change the busy state in
the event loop (As its now done by functions that perform blocking shell
invocations).
- process spawning was decoupled from the rest of the job control logic. The
goal is reusing it for spawning processes connected to pseudo terminal file
descriptors.
- job_start now receives a JobOptions structure containing all the startup
options.
- Removed term.c, term.h and term_defs.h
- Tests for T_* values were removed. screen.c was simplified as a
consequence(the best strategy for drawing is implemented in the UI layer)
- Redraw functions now call ui.c functions directly. Updates are flushed with
`ui_flush()`
- Removed all termcap options(they now return empty strings for compatibility)
- &term/&ttybuiltin options return a constant value(nvim)
- &t_Co is still available, but it mirrors t_colors directly
- Remove cursor tracking from screen.c and the `screen_start` function. Now the
UI is expected to maintain cursor state across any call, and reset it when
resized.
- Remove unused code
Shell command output was broken in @8a5a8db, which refactored nvim to no longer
switch to cooked mode(linefeeds are processed differently).
Fix the problem by refactoring write_output to accept to extra arguments that
control the flushing behavior and where data will be written to: buffer or
directly to the screen.
Nvim now relies much less on setting terminal mode to cooked mode, remove most
calls to settmode, except for those that happen on startup or when suspending.
Eventually even those will be handled by the UI layer.
Commit @45525853d352 removed usage of the `job_write_cb` for closing stdin due
to a memory error, but that doesn't work anymore because `job_close_in` closes
stdin immediately, possibly trimming input data before it is fully written.
Since most memory issues with jobs have been fixed, re-add the `job_write_cb`
call to ensure stdin is only closed when it should. Also add tests for scenarios
where using the callback makes a difference.
Passing NULL as the callback for stdout/stderr will result in job_start ignoring
stdout/stderr, respectively. A 'writable' boolean argument was also added, and
when false `job_start` will ignore stdin.
Also, refactor os_system to allow passing NULL as the `output` argument.
All files under the os, api and msgpack_rpc directories have -Wconversion
automatically applied. CONV_SOURCES is also checked for missing files(when
renaming, for example)
RBuffer instances represent the internal buffer used by RStreams.
This changes RStream constructor to receive RBuffer pointers and adds a set of
RBuffer methods that expose the lower level buffer manipulation to consumers of
the RStream API.
The os_system function uses a write callback to close the input stream when the
write completes, but this causes a memory error because the callback is invoked
right before the stream is freed by the caller.
This fixes the problem by removing the callback set by os_system. Instead, it
calls job_close_in immediately after writing(the stream will only close after
the write completes). The 'pending' parameter was also removed from the
'write_cb' as it should be hidden by the wstream module.
While the `wstream_set_write_cb` and `job_write_cb` are no longer used, they
will remain in the codebase for future use.
* With the changes in commit
"events: Refactor how event deferral is handled"
(2e4ea29d2c) the function argument
'defer' of 'job_start' and member variable 'defer' of 'struct job'
can be removed.
* Update/Fix the documentation for function 'job_start'.
With the goal to support pipe-only system() calls.
Notes on the second (vim) argument to f_system() (i.e.: redirected input)
and its implications:
- When calling system('cat -', ['some', 'list']), vanilla vim (before a
recent patch that added support for passing lists) just passes an empty
file to the process. This is the same as immediately closing the pipe,
which os_system does when no input is given. If we wouldn't close the
pipe, the process will linger forever (as is the case with `cat -`).
As of now, it's not allowed to pass a non-NULL pointer as the `output`
parameter. In other words, it's not possible to signal disinterst in the
process output. That may change in the future.
- The 'stripdecls.py' script replaces declarations in all headers by includes to
generated headers.
`ag '#\s*if(?!ndef NEOVIM_).*((?!#\s*endif).*\n)*#ifdef INCLUDE_GENERATED'`
was used for this.
- Add and integrate gendeclarations.lua into the build system to generate the
required includes.
- Add -Wno-unused-function
- Made a bunch of old-style definitions ANSI
This adds a requirement: all type and structure definitions must be present
before INCLUDE_GENERATED_DECLARATIONS-protected include.
Warning: mch_expandpath (path.h.generated.h) was moved manually. So far it is
the only exception.
Problem: Now that nvim/strings.h is correctly namespaced, an issue
that had been masked until now arises:
When compiling, we get a lot of errors because of everywhere
the functions in nvim/strings.h are used, there's no include
to import them.
But, how could this compile and work previously, then? It
turns out that:
- In every such case, we are also including vim.h, which in
turn includes os_unix_defs.h.
- os_unix_defs.h includes <string.h> and also <strings.h> in
some systems (e.g. OSX).
- Build had been modified previously to (even when importing
system headers), prefer equally-named local ones. That was
in fact done as a previous attempt to solve the same issue
we are trying to solve another way now.
So, we were including our "strings.h" as a side-effect of
including <strings.h> through "vim.h" --> "os_unix_defs.h".
Solution: Correctly include "nvim/strings.h" in every file needing it.
Move files from src/ to src/nvim/.
- src/nvim/ becomes the new root dir for nvim executable sources.
- src/libnvim/ is planned to become root dir of the neovim library.
