ui_bridge:ui_bridge_stop() calls ui_detach_impl() last, so the check for
ui_active() in ui:ui_refresh() doesn't help: tui_main() already freed
the `ui` object.
There is a race between ui_bridge_stop (thread T0) and tui_main (thread T1).
UIBridgeData.stopped could be set while ui_bridge_stop() is in the
middle of loop_poll_events(), which may invoke tui_scheduler() on T0.
The pointers in tui_scheduler() may be invalid by then.
Solution(?): Use the `UI.data` field as a "stopped" flag and check it in
tui_scheduler().
ASAN use-after-free report observed in #7908:
= ==20066==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000000cd0 at pc 0x00000182abed bp 0x7ffe23b07070 sp 0x7ffe23b07068
= READ of size 8 at 0x611000000cd0 thread T0
= 0 0x182abec in tui_scheduler /home/travis/build/neovim/neovim/src/nvim/tui/tui.c:393:23
= 1 0x1876afd in ui_bridge_update_fg /home/travis/build/neovim/neovim/build/src/nvim/auto/ui_events_bridge.generated.h:205:3
= 2 0x186c130 in ui_resize /home/travis/build/neovim/neovim/src/nvim/ui.c:310:3
= 3 0x146b9c2 in screen_resize /home/travis/build/neovim/neovim/src/nvim/screen.c:7483:3
= 4 0x186a6f0 in ui_refresh /home/travis/build/neovim/neovim/src/nvim/ui.c:284:3
= 5 0x186bbe0 in ui_refresh_event /home/travis/build/neovim/neovim/src/nvim/ui.c:297:3
= 6 0xa2219a in multiqueue_process_events /home/travis/build/neovim/neovim/src/nvim/event/multiqueue.c:150:7
= 7 0xa1bd7f in loop_poll_events /home/travis/build/neovim/neovim/src/nvim/event/loop.c:63:3
= 8 0x1872709 in ui_bridge_stop /home/travis/build/neovim/neovim/src/nvim/ui_bridge.c:121:5
= 9 0x1864247 in ui_builtin_stop /home/travis/build/neovim/neovim/src/nvim/ui.c:143:3
= 10 0x1249ec8 in mch_exit /home/travis/build/neovim/neovim/src/nvim/os_unix.c:140:3
= 11 0xe56ba9 in getout /home/travis/build/neovim/neovim/src/nvim/main.c:671:3
= 12 0xfc4c8f in preserve_exit /home/travis/build/neovim/neovim/src/nvim/misc1.c:2653:3
= 13 0x1247c02 in deadly_signal /home/travis/build/neovim/neovim/src/nvim/os/signal.c:137:3
= 14 0x1247921 in on_signal /home/travis/build/neovim/neovim/src/nvim/os/signal.c:162:9
= 15 0xa35618 in signal_event /home/travis/build/neovim/neovim/src/nvim/event/signal.c:47:3
= 16 0xa2219a in multiqueue_process_events /home/travis/build/neovim/neovim/src/nvim/event/multiqueue.c:150:7
= 17 0xa1bd7f in loop_poll_events /home/travis/build/neovim/neovim/src/nvim/event/loop.c:63:3
= 18 0x1237bd6 in input_poll /home/travis/build/neovim/neovim/src/nvim/os/input.c:349:3
= 19 0x123334f in inbuf_poll /home/travis/build/neovim/neovim/src/nvim/os/input.c:372:24
= 20 0x123316d in os_inchar /home/travis/build/neovim/neovim/src/nvim/os/input.c:110:19
= 21 0x170d20e in state_enter /home/travis/build/neovim/neovim/src/nvim/state.c:55:13
= 22 0xbd7441 in command_line_enter /home/travis/build/neovim/neovim/src/nvim/ex_getln.c:384:3
= 23 0xbd0a60 in getcmdline /home/travis/build/neovim/neovim/src/nvim/ex_getln.c:1920:10
= 24 0xbdb365 in getexline /home/travis/build/neovim/neovim/src/nvim/ex_getln.c:2100:10
= 25 0xb00a6b in do_cmdline /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:528:47
= 26 0x10a7837 in nv_colon /home/travis/build/neovim/neovim/src/nvim/normal.c:4552:18
= 27 0x1091e15 in normal_execute /home/travis/build/neovim/neovim/src/nvim/normal.c:1136:3
= 28 0x170d439 in state_enter /home/travis/build/neovim/neovim/src/nvim/state.c:67:26
= 29 0x104ee14 in normal_enter /home/travis/build/neovim/neovim/src/nvim/normal.c:466:3
= 30 0xe4295c in main /home/travis/build/neovim/neovim/src/nvim/main.c:572:3
= 31 0x2b2ba340bf44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287
= 32 0x44d24b in _start (/home/travis/build/neovim/neovim/build/bin/nvim+0x44d24b)
=
= 0x611000000cd0 is located 16 bytes inside of 240-byte region [0x611000000cc0,0x611000000db0)
= freed by thread T1 here:
= 0 0x4ee0e2 in __interceptor_free /local/mnt/workspace/tmp/ubuntu_rel/llvm/utils/release/final/llvm.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:47:3
= 1 0xf4f6d4 in xfree /home/travis/build/neovim/neovim/src/nvim/memory.c:133:3
= 2 0x182a963 in tui_main /home/travis/build/neovim/neovim/src/nvim/tui/tui.c:383:3
= 3 0x18792b0 in ui_thread_run /home/travis/build/neovim/neovim/src/nvim/ui_bridge.c:106:3
= 4 0x2b2ba2697183 in start_thread /build/eglibc-ripdx6/eglibc-2.19/nptl/pthread_create.c:312
=
= previously allocated by thread T0 here:
= 0 0x4ee61a in calloc /local/mnt/workspace/tmp/ubuntu_rel/llvm/utils/release/final/llvm.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:76:3
= 1 0xf4f787 in xcalloc /home/travis/build/neovim/neovim/src/nvim/memory.c:147:15
= 2 0x182000a in tui_start /home/travis/build/neovim/neovim/src/nvim/tui/tui.c:127:12
= 3 0x1863f7c in ui_builtin_start /home/travis/build/neovim/neovim/src/nvim/ui.c:125:3
= 4 0xe41bb9 in main /home/travis/build/neovim/neovim/src/nvim/main.c:457:5
= 5 0x2b2ba340bf44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287
=
= Thread T1 created by T0 here:
= 0 0x4d774d in __interceptor_pthread_create /local/mnt/workspace/tmp/ubuntu_rel/llvm/utils/release/final/llvm.src/projects/compiler-rt/lib/asan/asan_interceptors.cc:317:3
= 1 0x1aae6b0 in uv_thread_create /home/travis/nvim-deps/build/src/libuv/src/unix/thread.c:75
= 2 0x18217fa in tui_start /home/travis/build/neovim/neovim/src/nvim/tui/tui.c:159:10
= 3 0x1863f7c in ui_builtin_start /home/travis/build/neovim/neovim/src/nvim/ui.c:125:3
= 4 0xe41bb9 in main /home/travis/build/neovim/neovim/src/nvim/main.c:457:5
= 5 0x2b2ba340bf44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287
---
Alternative attempt:
commit 6ad9c02491606a0c31e907f38c9931f324327aa5
Author: Justin M. Keyes <justinkz@gmail.com>
Date: Sat Jan 27 15:12:58 2018 +0100
tui: fix use-after-free: swap in empty scheduler
This should make life easier for UIs like VimR which implement their own
in-process bridged UI: they don't need to worry that their `scheduler`
might receive an invalid pointer.
To avoid that, ui_bridge_stopped() swaps in an empty scheduler. Note
that this requires the call to loop_poll_events() to be moved into the
critical section.
diff --git a/src/nvim/ui_bridge.c b/src/nvim/ui_bridge.c
index 779585416f80..491052d19d3b 100644
--- a/src/nvim/ui_bridge.c
+++ b/src/nvim/ui_bridge.c
@@ -93,10 +93,18 @@ UI *ui_bridge_attach(UI *ui, ui_main_fn ui_main, event_scheduler scheduler)
return &rv->bridge;
}
+static void ui_bridge_null_scheduler(Event event, void *d)
+{
+ WLOG("ignoring event (bridge stopped)");
+}
+
void ui_bridge_stopped(UIBridgeData *bridge)
{
uv_mutex_lock(&bridge->mutex);
bridge->stopped = true;
+ // Replace with an empty scheduler, so that the UI internal scheduler does
+ // not get invoked with an invalid pointer. #7922
+ bridge->scheduler = ui_bridge_null_scheduler;
uv_mutex_unlock(&bridge->mutex);
}
@@ -111,14 +119,11 @@ static void ui_bridge_stop(UI *b)
UIBridgeData *bridge = (UIBridgeData *)b;
bool stopped = bridge->stopped = false;
UI_BRIDGE_CALL(b, stop, 1, b);
- for (;;) {
+ while (!stopped) {
uv_mutex_lock(&bridge->mutex);
stopped = bridge->stopped;
- uv_mutex_unlock(&bridge->mutex);
- if (stopped) {
- break;
- }
loop_poll_events(&main_loop, 10); // Process one event (at most).
+ uv_mutex_unlock(&bridge->mutex);
}
uv_thread_join(&bridge->ui_thread);
uv_mutex_destroy(&bridge->mutex);
- Revert timer-based approach.
- Instead, call loop_poll_events() with a timeout in an "active" loop,
to infer that "TUI startup activity has mostly finished", but also to
enforce a mininum time (100 ms) before emitting "enable focus
reporting" termcode. (If TUI startup takes longer than that minimum
time, it's probably a slow environment anyways.)
- Tickle `main_loop` by sending a dummy event. Without this, the
initial "focus-gained" response from the terminal may not get
processed until the user hits a key.
ref #7720
ref #7664
ref #7649
ref #7664
ref 27f9b1c7b0
ref #7649
ref #766427f9b1c7b0 caused a regression: it uses loop_schedule_deferred() to
defer emitting the "enable focus reporting" termcode. tui_main() never
processes `tui_loop.events` (which loop_schedule_deferred() depends on),
so the event was never actually processed.
But fixing that (by processing `tui_loop.events`) would bring back the
problem 27f9b1c7b0 tried to fix: it still emits the event too soon.
Instead, do a little dance: schedule the event on `main_loop` and then
forward it to `tui_loop`.
NOTE: after this commit, in tmux 2.3 with `focus-events` enabled,
FocusGained is fired on startup and when resuming from suspend.
Using `script` to record the terminal session (and `vterm-dump` to
post-process the result):
BEFORE:
{DECSM 1049}{DECSM 1}{ESC =}
{CUP *}{ED *}{DECSM 2004}{DECSM 1004}{CSI 1,43 r}
{CUP 1,1}
{CUP *}{ED *}{SM 34}{DECSM 25}
{DECRM 25}{CSI 2 q}{CSI 2 q}
{CUP *}{ED *}{LF}
{SGR *}{LS1}{SGR 94}~
...
AFTER:
{CUP *}{ED *}{CSI 1,43 r}
{CUP 1,1}
{CUP *}{ED *}{SM 34}{DECSM 25}
{DECRM 25}{CSI 2 q}{CSI 2 q}
{CUP *}{ED *}{DECSM 2004}{DECSM 1004}{LF}
{SGR *}{LS1}{SGR 94}~
...
Doc for UV_PROCESS_DETACHED in uv.h mentions:
> child process will still keep the parent's event loop alive unless
> the parent process calls uv_unref() on the child's process handle.
ref #3944
Although in_port_t is a typedef for uint16_t, GCC in Ubuntu 12.04
complains about potential loss of data due to converting int to
uint16_t. Since we know this isn't possible, silence the warning to
avoid breaking QB until it gets upgraded to a newer Ubuntu.
When using serverstart("ip.ad.d.r:") to listen on a random port, we need
to abide by getaddrinfo()'s API and pass in a NULL service, rather than
an empty string.
When given an empty string, getaddrinfo() is free to search for a
service by the given name (since the string isn't a number) which will
fail. At least FreeBSD does perform this lookup.
This change implicitly adds IPv6 support.
If the address contains ":", we try to use a TCP socket instead of a Unix domain
socket. Everything in front of the last occurrence of ":" is the hostname and
everything after it the port.
If the hostname lookup fails, we fall back to using a Unix domain socket.
If the port is empty ("localhost:"), a random port will be assigned.
Examples:
NVIM_LISTEN_ADDRESS=localhost:12345 -> TCP (IPv4 or IPv6), port: 12345
NVIM_LISTEN_ADDRESS=localhost: -> TCP (IPv4 or IPv6), port: random (> 1024)
NVIM_LISTEN_ADDRESS=localhost:0 -> TCP (IPv4 or IPv6), port: random (> 1024)
NVIM_LISTEN_ADDRESS=localhost -> Unix domain socket "localhost" in current dir
Asynchronous API functions are served immediately, which means pending
input could change the state of Nvim shortly after an async API function
result is returned.
nvim_get_mode() is different:
- If RPCs are known to be blocked, it responds immediately (without
flushing the input/event queue)
- else it is handled just-in-time before waiting for input, after
pending input was processed. This makes the result more reliable
(but not perfect).
Internally this is handled as a special case, but _semantically_ nothing
has changed: API users never know when input flushes, so this internal
special-case doesn't violate that. As far as API users are concerned,
nvim_get_mode() is just another asynchronous API function.
In all cases nvim_get_mode() never blocks for more than the time it
takes to flush the input/event queue (~µs).
Note: This doesn't address #6166; nvim_get_mode() will provoke #6166 if
e.g. `d` is operator-pending.
Closes#6159
Disable CommandLineToArgvW-standard quoting for cmd.exe.
libuv assumes spawned processes follow the convention expected by
CommandLineToArgvW(). But cmd.exe is non-conformant, so for cmd.exe:
- With system([]), the caller has full control (and responsibility) to
quote arguments correctly.
- With system(''), shell* options are used.
libuv quoting is disabled if argv[0] is:
- cmd.exe
- cmd
- $COMSPEC resolving to a path with filename cmd.exe
Closes#6329
References #6387
Closes#1234
multiqueue:
- Implement multiqueue_size()
- Rename MultiQueueItem.parent to MultiQueueItem.parent_item, to avoid confusion
with MultiQueue.parent.
Closes#3529Closes#5241
In Vim,
:echo system('cat - &', 'foo')
works because for both system() and :! Vim writes input to a temp file and uses
shell syntax to redirect the file to the backgrounded `cat` (get_cmd_output()
.. make_filter_cmd()).
In Nvim,
:echo system('cat - &', 'foo')
fails because we write the input directly via pipes (shell.c:do_os_system()),
but (per POSIX[1]) backgrounded process input stream is redirected from
/dev/null (unless overridden by shell redirection; supported only by some shells
[2]), so our writes are ignored, the process exits quickly, and if we are
writing data larger than the buffer size we'll see EPIPE.
This still works:
:%w !tee > foo1358.txt &
but this does not:
:%w !tee foo1358.txt &
though it *should* (why doesn't it?) because we still do the temp file dance
in do_bang() .. do_filter().
[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_03_02
[2] http://unix.stackexchange.com/a/71218
`lib/queue.h` implements a basic queue. `event/queue.c` implements
a specialized data structure on top of lib/queue.h; it is not a "normal"
queue.
Rename the specialized multi-level queue implemented in event/queue.c to
"multiqueue", to avoid confusion when reading the code.
Before this change one can eventually notice that "macros (uppercase
symbols) are for the normal queue, lowercase operations are for the
multi-level queue", but that is unnecessary friction for new developers
(or existing developers just visiting this part of the codebase).
Because terminfo_stop() already ran, there is not much reason to wait
for the loop to teardown.
Helped-by: Björn Linse <bjorn.linse@gmail.com>
Helped-by: oni-link <knil.ino@gmail.com>
Closes#4778
References #3541
---
Bug report:
> After pressing `ZZ` I can find two threads freezing, occupying 100% CPU:
|-systemd-+
|-nvim,11567 /home/lz/code/1.rs +set title
| `-{nvim},11574
> 11567 has two threads:
(gdb) info threads
Id Target Id Frame
* 1 Thread 0x7f7622907780 (LWP 11567) "nvim" 0x00007f76222e66bd in pthread_join () from /usr/lib/libpthread.so.0
2 Thread 0x7f761f5ff700 (LWP 11574) "nvim" 0x00007ffcec9e9c59 in clock_gettime ()
(gdb) thread apply all bt
Thread 2 (Thread 0x7f761f5ff700 (LWP 11574)):
#0 0x00007ffcec9e9c59 in clock_gettime ()
#1 0x00007f76210b9356 in clock_gettime () from /usr/lib/libc.so.6
#2 0x00007f7622513e3c in ?? () from /usr/lib/libuv.so.1
#3 0x00007f7622505e94 in uv_run () from /usr/lib/libuv.so.1
#4 0x00000000004822ed in loop_close (loop=loop@entry=0x7f761f5fe870) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/event/loop.c:87
#5 0x00000000005a7ec0 in tui_main (bridge=0x7f761f6ac000, ui=0x7f761f69ace0) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/tui/tui.c:234
#6 0x00000000005a9b47 in ui_thread_run (data=<optimized out>) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/ui_bridge.c:87
#7 0x00007f7622510d07 in ?? () from /usr/lib/libuv.so.1
#8 0x00007f76222e5474 in start_thread () from /usr/lib/libpthread.so.0
#9 0x00007f76210ac69d in clone () from /usr/lib/libc.so.6
Thread 1 (Thread 0x7f7622907780 (LWP 11567)):
#0 0x00007f76222e66bd in pthread_join () from /usr/lib/libpthread.so.0
#1 0x00007f7622510dae in uv_thread_join () from /usr/lib/libuv.so.1
#2 0x00000000005aac9e in ui_bridge_stop (b=0x7f761f6ac000) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/ui_bridge.c:104
#3 0x00000000005a90e4 in ui_builtin_stop () at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/ui.c:91
#4 0x000000000052be09 in mch_exit (r=r@entry=1) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/os_unix.c:143
#5 0x00000000004db1cc in getout (exitval=exitval@entry=1) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/main.c:623
#6 0x00000000004fa43c in preserve_exit () at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/misc1.c:2652
#7 0x000000000052b77a in deadly_signal (signum=1) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/os/signal.c:120
#8 0x000000000052b7cf in on_signal (handle=<optimized out>, signum=<optimized out>, data=<optimized out>) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/os/signal.c:145
#9 0x0000000000484178 in signal_event (argv=<optimized out>) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/event/signal.c:44
#10 0x0000000000483b7f in queue_process_events (queue=0x7f7620417360) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/event/queue.c:142
#11 0x0000000000482208 in loop_poll_events (loop=0x84dec0 <loop>, ms=ms@entry=4000) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/event/loop.c:56
#12 0x000000000052a364 in input_poll (ms=ms@entry=4000) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/os/input.c:325
#13 0x000000000052a3e2 in inbuf_poll (ms=4000) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/os/input.c:347
#14 0x000000000052a839 in os_inchar (buf=buf@entry=0x0, maxlen=maxlen@entry=0, ms=ms@entry=-1, tb_change_cnt=tb_change_cnt@entry=0) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/os/input.c:107
#15 0x0000000000592eeb in state_enter (s=s@entry=0x7ffcec9d3560) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/state.c:46
#16 0x0000000000508533 in normal_enter (cmdwin=cmdwin@entry=false, noexmode=noexmode@entry=false) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/normal.c:464
#17 0x00000000004dc17e in main (argc=<optimized out>, argv=<optimized out>) at /tmp/yaourt-tmp-lz/aur-neovim-git/src/neovim-git/src/nvim/main.c:538
Inherited signal mask may block SIGCHLD, which causes libuv to hang at
epoll_wait.
Closes#5230
Helped-by: Nicolas Hillegeer <nicolas@hillegeer.com>
Helped-by: John Szakmeister <john@szakmeister.net>
Note: the #pragma gymnastics are a workaround for broken system headers on
macOS.
signal.h:
int sigaddset(sigset_t *, int);
#define sigaddset(set, signo) (*(set) |= __sigbits(signo), 0)
sys/_types/_sigset.h:
typedef __darwin_sigset_t sigset_t;
sys/_types.h:
typedef __uint32_t __darwin_sigset_t; /* [???] signal set */
sigset_t is defined as unsigned int, but the sigaddset() ORs it with an int,
mixing the types. So GCC generates a sign-conversion warning:
sig.c:9:13: warning: implicit conversion changes signedness: 'int' to 'unsigned int' [-Wsign-conversion]
(*(&s) |= __sigbits((sigset_t) 20), 0);
~~ ^~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
System headers are normally ignored when the compiler generates warnings:
https://gcc.gnu.org/onlinedocs/cpp/System-Headers.html
> GCC gives code found in system headers special treatment. All warnings,
> other than those generated by ‘#warning’ (see Diagnostics), are suppressed
> while GCC is processing a system header. Macros defined in a system header
> are immune to a few warnings wherever they are expanded. This immunity is
> granted on an ad-hoc basis, when we find that a warning generates lots of
> false positives because of code in macros defined in system headers.
Instead of the #pragma workaround, we could cast the sigset_t pointer:
# if defined(__APPLE__)
sigaddset((int *)&mask, SIGCHLD);
# else
sigaddset(&mask, SIGCHLD);
# endif
but that could break if the headers are later fixed.
