Coverity warning is a false positive: if rbuffer_read_ptr() returns
NULL then `cnt` is zero.
Revert 76ea97c809 (which caused
the TSan build to hang often--possibly because of the missing ui_flush()).
Instead, modify out_data_append_to_screen() to check for NULL.
ref #6862
rbuffer_read_ptr may return a null
if ptr == null && cnt == 0 && !out_data_decide_throttle(cnt)
then we would have called out_data_append_to_screen(ptr, cnt, eof)
which dereferences the null pointer.
- Establish ERROR log level as "critical". Such errors are rare and will
be valuable when users encounter unusual circumstances.
- Set -DMIN_LOG_LEVEL=3 for release-type builds
Tokenize p_sh if used as default in ex_terminal(). Previously p_sh was
used as the first arg in a list when calling termopen(), this would try
to call an untokenized version of shell, meaning if you had an argument
in 'shell':
set shell=/bin/bash\ --login
the command would fail.
Helped-by: oni-link <knil.ino@gmail.com>
Closes#3999
Instead of managing max_visits, check the time every N visits. This avoids edge
cases where max_visits is large but the chunk frequency slowed down, which would
causing the "..." pulse to show for a very long time. It's more important to
show output at reasonable intervals than to avoid calling os_hrtime().
This ameliorates use-cases like:
:!cat foo.txt
:make
where the user is interested in the last few lines of output.
Try these shell-based ex-commands before/after this commit:
:grep -r '' *
:make
:!yes
:!grep -r '' *
:!git grep ''
:!cat foo
:!echo foo
:!while true; do date; done
:!for i in `seq 1 20000`; do echo XXXXXXXXXX $i; done
In all cases the last few lines of the command should always be shown,
regardless of where throttling was triggered.
Periodically skip :! spam. This is a "cheat" that works for all UIs and greatly
improves responsiveness when :! spams MB or GB of output:
:!yes
:!while true; do date; done
:!git grep ''
:grep -r '' *
After ~10KB of data is seen from a single :! invocation, output will be skipped
for ~1s and three dots "..." will pulse in the bottom-left. Thereafter the
behavior alternates at every:
* 10KB received
* ~1s throttled
This also avoids out-of-memory which could happen with large :! outputs.
Note: This commit does not change the behavior of execute(':!foo').
execute(':!foo') returns the string ':!foo^M', it captures *only* Vim
messages, *not* shell command output. Vim behaves the same way.
Use system('foo') for capturing shell command output.
Closes#1234
Helped-by: oni-link <knil.ino@gmail.com>
Closes#3529Closes#5241
In Vim,
:echo system('cat - &', 'foo')
works because for both system() and :! Vim writes input to a temp file and uses
shell syntax to redirect the file to the backgrounded `cat` (get_cmd_output()
.. make_filter_cmd()).
In Nvim,
:echo system('cat - &', 'foo')
fails because we write the input directly via pipes (shell.c:do_os_system()),
but (per POSIX[1]) backgrounded process input stream is redirected from
/dev/null (unless overridden by shell redirection; supported only by some shells
[2]), so our writes are ignored, the process exits quickly, and if we are
writing data larger than the buffer size we'll see EPIPE.
This still works:
:%w !tee > foo1358.txt &
but this does not:
:%w !tee foo1358.txt &
though it *should* (why doesn't it?) because we still do the temp file dance
in do_bang() .. do_filter().
[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_03_02
[2] http://unix.stackexchange.com/a/71218
`lib/queue.h` implements a basic queue. `event/queue.c` implements
a specialized data structure on top of lib/queue.h; it is not a "normal"
queue.
Rename the specialized multi-level queue implemented in event/queue.c to
"multiqueue", to avoid confusion when reading the code.
Before this change one can eventually notice that "macros (uppercase
symbols) are for the normal queue, lowercase operations are for the
multi-level queue", but that is unnecessary friction for new developers
(or existing developers just visiting this part of the codebase).
move `call_shell` to misc1.c
Move some fns to state.c
Move some fns to option.c
Move some fns to memline.c
Move `vim_chdir*` fns to file_search.c
Move some fns to new module, bytes.c
Move some fns to fileio.c
Current name is inappropriate for the following reasons:
1. It is often masked by local `loop` variables.
2. It cannot be searched for. There are many `loop` variables where `loop` is
some local variable. There are many cases when “loop” word is used in
a comment.
3. It is in any case bad idea to use a generic name as a name of the global
variable. Best if global has module prefix: this is why it is in `main.h`:
`main_loop` both stands for “a main loop” and “a loop defined in `main.*`”.
Since I have no idea how to list every occurrence of this variable method used
to rename it is “remove it from globals.h, try to compile, fix errors”. Thus if
some occurrence was hidden under false `#if` branch it was not replaced.
The whole stream buffer is now put on screen at once instead of only
data up to the last newline. This has some advantages:
* RBuffer cannot wrap around, so we never forget to output second
half of the buffer.
* Stream data is not delayed anymore, because we don't have to wait for
a newline.
This works by remembering the last used screen column.
Problem: On some systems automatically adding the missing EOL causes
problems. Setting 'binary' has too many side effects.
Solution: Add the 'fixeol' option, default on. (Pavel Samarkin)
34d72d4b6c
The new event processing architecture changed `jobwait()` semantics: Only one
job is processed at time since process_wait only focuses on one queue.
This fixes the problem with a few changes:
- Allow the event queue polled by `process_wait` to be overriden by a new
argument.
- Allow the parent queue to be overriden with `queue_replace_parent`
- Create a temporary queue that serves as the parent for all jobs passed to
`jobwait()`
- Improve the implementation of deferred/immediate events.
- Use the new queue module to change how/when events are queued/processed by
giving a private queue to each emitter.
- Immediate events(which only exist to break uv_run recursion) are now
represented in the `loop->fast_events` queue.
- Events pushed to child queues are propagated to the event loop main queue and
processed as K_EVENT keys.
out_data_cb() can return without emptying the full RBuffer (no NL was
seen). Because the shell output stream is stopped until space in the
Rbuffer is freed up, no more shell output is written.
To prevent this, output the full RBuffer when write_output() did not
write anything.
write_output() can also process the same RBuffer content more than once,
if no NL was seen. To prevent NUL bytes from producing new lines (if
lines are not written to a buffer), translate NUL to SOH(1).
Fixes#2983
- New libuv/pty process abstraction with simplified API and no globals.
- Remove nvim/os/job*. Jobs are now a concept that apply only to programs
spawned by vimscript job* functions.
- Refactor shell.c/channel.c to use the new module, which brings a number of
advantages:
- Simplified API, less code
- No slots in the user job table are used
- Not possible to acidentally receive data from vimscript
- Implement job table in eval.c, which is now a hash table with unilimited job
slots and unique job ids.
- Simplify RStream/WStream API and make it more consistent with libuv.
- Move into the event loop layer(event subdirectory)
- Remove uv_helpers module.
- Simplify job/process internal modules/API.
- Unify RStream and WStream into a single structure. This is necessary because
libuv streams can be readable and writable at the same time(and because the
uv_helpers.c hack to associate multiple streams with libuv handle was removed)
- Make struct definition public, allowing more flexible/simple memory
management by users of the module.
- Adapt channel/job modules to cope with the changes.
- Add event loop abstraction module under src/nvim/event. The
src/nvim/event/loop module replaces src/nvim/os/event
- Remove direct dependency on libuv signal/timer API and use the new abstraction
instead.
- Replace all references to uv_default_loop() by &loop.uv, a new global variable
that wraps libuv main event loop but allows the event loop functions to be
reused in other contexts.
Extract the RBuffer class from rstream.c and reimplement it as a ring buffer,
a more efficient version that doesn't need to relocate memory.
The old rbuffer_read/rbuffer_write interfaces are kept for simple
reading/writing, and the RBUFFER_UNTIL_{FULL,EMPTY} macros are introduced to
hide wrapping logic when more control is required(such as passing the buffer
pointer to a library function that writes directly to the pointer)
Also add a basic infrastructure for writing helper C files that are only
compiled in the unit test library, and use this to write unit tests for RBuffer
which contains some macros that can't be accessed directly by luajit.
Helped-by: oni-link <knil.ino@gmail.com>
Reviewed-by: oni-link <knil.ino@gmail.com>
Reviewed-by: Scott Prager <splinterofchaos@gmail.com>
Reviewed-by: Justin M. Keyes <justinkz@gmail.com>
Reviewed-by: Michael Reed <m.reed@mykolab.com>
