Commit Graph

265 Commits

Author SHA1 Message Date
Justin M. Keyes
507bda1c95 server: introduce --listen, deprecate $NVIM_LISTEN_ADDRESS 2018-04-11 02:41:05 +02:00
Justin M. Keyes
9f598e5765 serverstop(): return FALSE for invalid address 2018-04-11 01:58:41 +02:00
Björn Linse
90e5cc5484 channels: generalize jobclose() 2017-11-25 09:37:00 +01:00
Björn Linse
1ebc96fe10 channels: allow bytes sockets and stdio, and buffered bytes output 2017-11-24 14:54:15 +01:00
Björn Linse
5215e3205a channels: refactor 2017-11-24 14:50:00 +01:00
Björn Linse
3717e2157f Revert channel logging, rebased on new code below 2017-11-24 14:04:56 +01:00
Phlosioneer
df10714991 server.c: Fix bug in release mode (#7594)
When compiling with CMAKE_BUILD_TYPE=RelWithDebInfo, several
-Wmaybe-uninitialized warnings are printed. These were thought to
be false positives (#5061); there are no control paths that lead
to an uninitialized value. However, when gcc is run in -O2 mode,
it makes a mistake while generating the necessary logic.

Specifically, for the code:
...
  int = 0; // Index of the server whose address equals addr.
  for (; i < watchers.ga_len; i++) {
    watcher = ((SocketWatcher **)watchers.ga_data)[i];
    // <snip>
  }
  if (i >= watchers.ga_len) {
    ELOG("Not listening on %s", addr);
    return;
  }
...

Gcc generates:
...
<+98>:  cmp  %ebx, %ebp
<+100>: jg   0x530f13   <server_stop+55>
<+102>: cmp  %ebp, ebx
<+104>: jl   0x530f7e   <server_stop+162>
...

Normally, the if statement should catch the only control path
where watcher is not assigned: watchers.ga_len <= 0. When
compiled, the assembly lines 98 and 100 correspond to checking if
i < watchers.ga_len, and the lines 102 and 104 correspond to
checking if i >= watchers.ga_len. The assembly seems to compare
ebp (which is watchers.ga_len) with ebx (which is i), and jump
if greater; then do the same comparison and jump if less. This is
where gcc makes a mistake: it flips the order of the cmp
instruction. This means that the REAL behavior is first check if
i < watchers.ga_len and then check if i < watchers.ga_len. Which
means the code inside the if statement is NEVER executed; no
combination of i and watchers.ga_len will ever trigger ELOG().

So not only is this a use of an uninitialized value if
watchers.ga_len == 0 (or technically, if it's less than zero too),
it also clobbers any error detection if the for loop reaches the
last entry (which would normally cause i == watchers.ga_len too).

This commit fixes this issue by adding a bool to keep track of
whether a watcher was found during the loop. This makes gcc
generate the correct code, avoiding both bugs.
2017-11-20 01:55:28 +01:00
Björn Linse
2a3bcd1ff8 rpc: Don't delay notifications when request is pending (#6544)
With the old behavior, if a GUI makes a blocking request that requires user
interaction (like nvim_input()), it would not get any screen updates.

The client, not nvim, should decide how to handle notifications during a
pending request. If an rplugin wants to avoid async calls while a sync call is
busy, it likely wants to avoid processing async calls while another async call
also is handled as well.

This may break the expectation of some existing rplugins. For compatibility,
remote/define.vim reimplements the old behavior. Clients can opt-out by
specifying `sync=urgent`.

- Legacy hosts should be updated to use `sync=urgent`. They could add a flag
  indicating which async methods are always safe to call and which must wait
  until the main loop returns.
- New hosts can expose the full asyncness, they don't need to offer both
  behaviors.

ref #6532
ref #1398 d83868fe90
2017-10-29 03:06:53 +01:00
Justin M. Keyes
cdd9e868ef doc: channel, eventloop 2017-09-05 15:01:07 +02:00
Justin M. Keyes
0f442c328e channel.c:call_set_error(): fix memory leak 2017-08-21 01:04:28 +02:00
Justin M. Keyes
af993da435 rpc: close channel if stream was closed
f_jobstop()/f_rpcstop() .. process_stop() .. process_close_in(proc)
closes the write-stream of a RPC channel. But there might be
a pending RPC notification on the queue, which may get processed just
before the channel is closed.

To handle that case, check the Stream.closed in
channel.c:receive_msgpack().

Before this change, the above scenario could trigger
this assert(!stream->closed) in wstream_write():

    0x00007f96e1cd3428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
    0x00007f96e1cd502a in __GI_abort () at abort.c:89
    0x00007f96e1ccbbd7 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x768f9b "!stream->closed",
    file=file@entry=0x768f70 "../src/nvim/event/wstream.c", line=line@entry=77,
    function=function@entry=0x768fb0 <__PRETTY_FUNCTION__.13735> "wstream_write") at assert.c:92
    0x00007f96e1ccbc82 in __GI___assert_fail (assertion=0x768f9b "!stream->closed", file=0x768f70 "../src/nvim/event/wstream.c", line=77,
    function=0x768fb0 <__PRETTY_FUNCTION__.13735> "wstream_write") at assert.c:101
    0x00000000004d2c1f in wstream_write (stream=0x7f96e0a35078, buffer=0x7f96e09f9b40) at ../src/nvim/event/wstream.c:77
    0x00000000005857b2 in channel_write (channel=0x7f96e0ae5800, buffer=0x7f96e09f9b40) at ../src/nvim/msgpack_rpc/channel.c:551
    0x000000000058567d in on_request_event (argv=0x7ffed792efa0) at ../src/nvim/msgpack_rpc/channel.c:523
    0x00000000005854c8 in handle_request (channel=0x7f96e0ae5800, request=0x7ffed792f1b8) at ../src/nvim/msgpack_rpc/channel.c:503
    0x00000000005850cb in parse_msgpack (channel=0x7f96e0ae5800) at ../src/nvim/msgpack_rpc/channel.c:423
    0x0000000000584f90 in receive_msgpack (stream=0x7f96e0a35218, rbuf=0x7f96e0d1d4c0, c=22, data=0x7f96e0ae5800, eof=false)
    at ../src/nvim/msgpack_rpc/channel.c:389
    0x00000000004d0b20 in read_event (argv=0x7ffed792f4a8) at ../src/nvim/event/rstream.c:190
    0x00000000004ce462 in multiqueue_process_events (this=0x7f96e18172d0) at ../src/nvim/event/multiqueue.c:150
    0x000000000059b630 in nv_event (cap=0x7ffed792f620) at ../src/nvim/normal.c:7908
    0x000000000058be69 in normal_execute (state=0x7ffed792f580, key=-25341) at ../src/nvim/normal.c:1137
    0x0000000000652463 in state_enter (s=0x7ffed792f580) at ../src/nvim/state.c:61
    0x000000000058a1fe in normal_enter (cmdwin=false, noexmode=false) at ../src/nvim/normal.c:467
    0x00000000005500c2 in main (argc=2, argv=0x7ffed792f8d8) at ../src/nvim/main.c:554

Alternative approach suggested by bfredl is to use close_cb of the
process. My unsuccessful attempt is below. (It seems close_cb is queued
too late, which is the similar problem addressed by this commit):

    commit 75fc12c6ab15711bdb7b18c6d42ec9d157f5145e
    Author: Justin M. Keyes <justinkz@gmail.com>
    Date:   Fri Aug 18 01:30:41 2017 +0200

        rpc: use Stream's close_cb instead of explicit check in receive_msgpack()

    diff --git a/src/nvim/event/process.c b/src/nvim/event/process.c
    index 8371d3cd482e..e52da23cdc40 100644
    --- a/src/nvim/event/process.c
    +++ b/src/nvim/event/process.c
    @@ -416,6 +416,10 @@ static void on_process_exit(Process *proc)
     static void on_process_stream_close(Stream *stream, void *data)
     {
       Process *proc = data;
    +  ILOG("on_process_stream_close");
    +  if (proc->stream_close_cb != NULL) {
    +    proc->stream_close_cb(stream, proc->stream_close_data);
    +  }
       decref(proc);
     }

    diff --git a/src/nvim/event/process.h b/src/nvim/event/process.h
    index 5c00e8e7ecd5..34a8d54f6f8c 100644
    --- a/src/nvim/event/process.h
    +++ b/src/nvim/event/process.h
    @@ -26,6 +26,11 @@ struct process {
       Stream *in, *out, *err;
       process_exit_cb cb;
       internal_process_cb internal_exit_cb, internal_close_cb;
    +
    +  // Called when any of the process streams (in/out/err) closes.
    +  stream_close_cb stream_close_cb;
    +  void *stream_close_data;
    +
       bool closed, detach;
       MultiQueue *events;
     };
    @@ -50,6 +55,8 @@ static inline Process process_init(Loop *loop, ProcessType type, void *data)
         .closed = false,
         .internal_close_cb = NULL,
         .internal_exit_cb = NULL,
    +    .stream_close_cb = NULL,
    +    .stream_close_data = NULL,
         .detach = false
       };
     }
    diff --git a/src/nvim/event/stream.c b/src/nvim/event/stream.c
    index 7c865bfe1e8c..c8720d1e45d9 100644
    --- a/src/nvim/event/stream.c
    +++ b/src/nvim/event/stream.c
    @@ -95,7 +95,11 @@ void stream_close(Stream *stream, stream_close_cb on_stream_close, void *data)
     void stream_close_handle(Stream *stream)
       FUNC_ATTR_NONNULL_ALL
     {
    +  ILOG("stream=%d", stream);
    +  // LOG_CALLSTACK();
       if (stream->uvstream) {
    +    // problem:  this schedules on the queue, but channel.c:receive_msgpack may
    +    // be processed before close_cb is called by libuv.
         uv_close((uv_handle_t *)stream->uvstream, close_cb);
       } else {
         uv_close((uv_handle_t *)&stream->uv.idle, close_cb);
    @@ -105,6 +109,7 @@ void stream_close_handle(Stream *stream)
     static void close_cb(uv_handle_t *handle)
     {
       Stream *stream = handle->data;
    +  ILOG(">>>>>>>>>>>>>>>>>>>>>>> stream=%p stream->internal_close_cb=%p", stream, stream->internal_close_cb);
       if (stream->buffer) {
         rbuffer_free(stream->buffer);
       }
    diff --git a/src/nvim/msgpack_rpc/channel.c b/src/nvim/msgpack_rpc/channel.c
    index 782eabe04e4a..dc2b794e366a 100644
    --- a/src/nvim/msgpack_rpc/channel.c
    +++ b/src/nvim/msgpack_rpc/channel.c
    @@ -128,6 +128,8 @@ uint64_t channel_from_process(Process *proc, uint64_t id, char *source)
                                           source);
       incref(channel);  // process channels are only closed by the exit_cb
       channel->data.proc = proc;
    +  channel->data.proc->stream_close_cb = close_cb2;
    +  channel->data.proc->stream_close_data = channel;

       wstream_init(proc->in, 0);
       rstream_init(proc->out, 0);
    @@ -387,17 +389,6 @@ static void receive_msgpack(Stream *stream, RBuffer *rbuf, size_t c,
         goto end;
       }

    -  if ((chan_wstream(channel) != NULL && chan_wstream(channel)->closed)
    -      || (chan_rstream(channel) != NULL && chan_rstream(channel)->closed)) {
    -    char buf[256];
    -    snprintf(buf, sizeof(buf),
    -             "ch %" PRIu64 ": stream closed unexpectedly. "
    -             "closing channel",
    -             channel->id);
    -    call_set_error(channel, buf, WARN_LOG_LEVEL);
    -    goto end;
    -  }
    -
       size_t count = rbuffer_size(rbuf);
       DLOG("ch %" PRIu64 ": parsing %u bytes from msgpack Stream: %p",
            channel->id, count, stream);
    @@ -571,23 +562,6 @@ static Stream *chan_wstream(Channel *chan)
       abort();
     }

    -/// Returns the Stream that a Channel reads from.
    -static Stream *chan_rstream(Channel *chan)
    -{
    -  switch (chan->type) {
    -    case kChannelTypeSocket:
    -      return &chan->data.stream;
    -    case kChannelTypeProc:
    -      return chan->data.proc->out;
    -    case kChannelTypeStdio:
    -      return &chan->data.std.in;
    -    case kChannelTypeInternal:
    -      return NULL;
    -  }
    -  abort();
    -}
    -
    -
     static bool channel_write(Channel *channel, WBuffer *buffer)
     {
       bool success = false;
    @@ -799,6 +773,12 @@ static void close_cb(Stream *stream, void *data)
       decref(data);
     }

    +static void close_cb2(Stream *stream, void *data)
    +{
    +  ILOG("close_cb2");
    +  close_channel(data);
    +}
    +
     /// @param source description of source function, rplugin name, TCP addr, etc
     static Channel *register_channel(ChannelType type, uint64_t id,
                                      MultiQueue *events, char *source)
2017-08-21 01:04:28 +02:00
Justin M. Keyes
e006b1d98d log: some DEBUG-level stream logging 2017-08-21 01:04:28 +02:00
Justin M. Keyes
bc6a3fe784 log: caller provides the source details 2017-07-23 15:01:31 +02:00
Justin M. Keyes
502af39f62 log: channel registration 2017-07-22 21:26:03 +02:00
Justin M. Keyes
fe1af9c2bc log: Always enable; remove DISABLE_LOG
- Establish ERROR log level as "critical". Such errors are rare and will
  be valuable when users encounter unusual circumstances.
- Set -DMIN_LOG_LEVEL=3 for release-type builds
2017-06-07 00:26:21 +02:00
Justin M. Keyes
fcc9d99967 channel_write: fix compiler warning 2017-05-31 14:10:04 +02:00
Björn Linse
5a151555c8 sockets: don't deadlock when connecting to own pipe address 2017-05-29 19:02:49 +02:00
Björn Linse
6a75938758 channels: implement sockopen() to connect to socket
Helped-By: oni-link <knil.ino@gmail.com>
2017-05-29 19:02:49 +02:00
Marco Hinz
fd5e4e2e4c Server: don't fall back to Unix sockets 2017-05-22 23:38:09 +02:00
Marco Hinz
3efc82cbb2 Server: use uv_getaddrinfo() for $NVIM_LISTEN_ADDRESS
This change implicitly adds IPv6 support.

If the address contains ":", we try to use a TCP socket instead of a Unix domain
socket. Everything in front of the last occurrence of ":" is the hostname and
everything after it the port.

If the hostname lookup fails, we fall back to using a Unix domain socket.

If the port is empty ("localhost:"), a random port will be assigned.

Examples:

  NVIM_LISTEN_ADDRESS=localhost:12345 -> TCP (IPv4 or IPv6), port: 12345
  NVIM_LISTEN_ADDRESS=localhost:      -> TCP (IPv4 or IPv6), port: random (> 1024)
  NVIM_LISTEN_ADDRESS=localhost:0     -> TCP (IPv4 or IPv6), port: random (> 1024)
  NVIM_LISTEN_ADDRESS=localhost       -> Unix domain socket "localhost" in current dir
2017-05-22 01:05:39 +02:00
Justin M. Keyes
0e873a30f3 Merge #4411 from ZyX-I/luaviml'/lua 2017-05-09 00:39:17 +02:00
Carlo Abelli
aace622ca5 refactor/single-include (#6687) 2017-05-08 15:08:12 +02:00
ZyX
09f849b600 Merge branch 'master' into luaviml'/lua 2017-05-08 15:43:45 +03:00
Justin M. Keyes
1483800cdf coverity/161682: FP: Dereference after null check (#6630) 2017-05-01 17:04:56 +02:00
Justin M. Keyes
f17a818519 api/nvim_get_mode: Use child-queue instead of "priority". 2017-04-28 19:20:09 +02:00
Justin M. Keyes
acfd2a2a29 input.c: Process only safe events before blocking.
Introduce multiqueue_process_priority() to process only events at or
above a certain priority.
2017-04-28 19:19:51 +02:00
Justin M. Keyes
3ea1007753 api: nvim_get_mode()
Asynchronous API functions are served immediately, which means pending
input could change the state of Nvim shortly after an async API function
result is returned.

nvim_get_mode() is different:
  - If RPCs are known to be blocked, it responds immediately (without
    flushing the input/event queue)
  - else it is handled just-in-time before waiting for input, after
    pending input was processed. This makes the result more reliable
    (but not perfect).

Internally this is handled as a special case, but _semantically_ nothing
has changed: API users never know when input flushes, so this internal
special-case doesn't violate that. As far as API users are concerned,
nvim_get_mode() is just another asynchronous API function.

In all cases nvim_get_mode() never blocks for more than the time it
takes to flush the input/event queue (~µs).

Note: This doesn't address #6166; nvim_get_mode() will provoke #6166 if
e.g. `d` is operator-pending.

Closes #6159
2017-04-28 19:14:34 +02:00
Justin M. Keyes
086c354a0a api: Do not translate error messages.
Also re-word some error messages:
- "Key does not exist: %s"
- "Invalid channel: %<PRIu64>"
- "Request array size must be 4 (request) or 3 (notification)"
- "String cannot contain newlines"

References #6150
2017-04-24 00:11:27 +02:00
Justin M. Keyes
3fbc660d57 api_set_error(): rename 2017-04-23 23:44:57 +02:00
Justin M. Keyes
2ed91f222f api/internal: Remove set field from Error type. 2017-04-23 23:44:32 +02:00
Justin M. Keyes
2a49163103 api_clear_error() 2017-04-23 23:44:32 +02:00
Sander Bosma
5c9860a0a2 api: Do not truncate errors <1 MB. #6237
Closes #5984
2017-04-23 23:44:29 +02:00
ZyX
c2f3e361c5 *: Add comment to all C files 2017-04-19 19:11:50 +03:00
ZyX
043d8ff9f2 Merge branch 'master' into luaviml'/lua 2017-04-08 01:54:58 +03:00
James McCoy
f4a3a96b68 Add handling for MSGPACK_OBJECT_FLOAT{32,64}
msgpack-c previously only had MSGPACK_OBJECT_FLOAT, which was a 64-bit
value.  Now, 32-bit and 64-bit floats are supported as distinct types,
but we'll simply continue to treat everything as 64-bit types.
2017-03-30 21:35:36 -04:00
ZyX
28dafe3ff0 eval,*: Move get_tv_string to typval.c
Function was renamed and changed to return `const char *`.
2017-03-29 10:08:05 +03:00
ZyX
62fde31936 api: Also shift numbers in api_metadata output
Fixes problem introduced by “api: Allow kObjectTypeNil to be zero without 
breaking compatibility”: apparently there are clients which use metadata and 
there are which aren’t. For the first that commit would not be needed, for the 
second that commit misses this critical piece.
2017-03-27 00:12:23 +03:00
ZyX
ae4adcc707 gendeclarations: Make declarations generator work with macros funcs
Now it checks functions also after every semicolon and closing figure brace, 
possibly preceded by whitespaces (tabs and spaces). This should make messing 
with declarations in macros not needed.
2017-03-27 00:12:23 +03:00
ZyX
22d3ce9c29 msgpack_rpc: Fix #HANDLE_TYPE_CONVERSION_IMPL
Function declarations generator is able to handle properly only the *first* 
function definition that is in macros, and only if it is the first entity in the 
macros. So msgpack_rpc_from_* was already really a static function, additionally 
its attributes were useless. This commit switches to explicit declarations and 
makes generated functions static.
2017-03-27 00:12:23 +03:00
ZyX
ca4c8b7f8a api: Allow kObjectTypeNil to be zero without breaking compatibility 2017-03-27 00:12:23 +03:00
ZyX
ba2f615cd4 functests: Test for error conditions
During testing found the following bugs:

1. msgpack-gen.lua script is completely unprepared for Float values either in 
   return type or in arguments. Specifically:

   1. At the time of writing relevant code FLOAT_OBJ did not exist as well as 
      FLOATING_OBJ, but it would be used by msgpack-gen.lua should return type 
      be Float. I added FLOATING_OBJ macros later because did not know that 
      msgpack-gen.lua uses these _OBJ macros, otherwise it would be FLOAT_OBJ.
   2. msgpack-gen.lua should use .data.floating in place of .data.float. But it 
      did not expect that .data subattribute may have name different from 
      lowercased type name.

2. vim_replace_termcodes returned its argument as-is if it receives an empty 
   string (as well as _vim_id*() functions did). But if something in returned 
   argument lives in an allocated memory such action will cause double free: 
   once when freeing arguments, then when freeing return value. It did not cause 
   problems yet because msgpack bindings return empty string as {NULL, 0} and 
   nothing was actually allocated.
3. New code in msgpack-gen.lua popped arguments in reversed order, making lua 
   bindings’ signatures be different from API ones.
2017-03-27 00:11:28 +03:00
Justin M. Keyes
857113ca8c terminal: Disable some options in terminal-mode.
In terminal-mode these options are nonsense because cursor is placed at
end of buffer to "follow" output.

Closes #2259
2017-03-01 14:47:49 +01:00
Justin M. Keyes
f7908b6f49 channel.c: logging 2017-02-26 13:00:02 +01:00
James McCoy
b37d062459 msgpack_rpc: Allocate empty string in msgpack_rpc_to_object
STR_CASE previously used a NULL data pointer for the String object, but
this pushes the NULL checks to the rest of the code.  Instead,
allocating an empty string solves the same issue of there not being any
data but ensures that we're not passing NULL to functions that don't
expect it.

Closes #5627
2016-11-29 08:19:48 -05:00
James McCoy
ca292c9768 Avoid serializing NULL string through msgpack
Attempting to serialize a NULL string through msgpack results in
msgpack_sbuffer_write attempting to memcpy from a NULL pointer, which is
undefined behavior.
2016-11-23 20:30:00 -05:00
James McCoy
654e92186b lint 2016-11-02 10:06:27 -04:00
James McCoy
37e64d79cc object_to_vim: Fix buffer/window/tabpage conversion on BE systems
Since data.integer is a different (larger) integer type than
data.{buffer,window,tabpage}, we cannot abuse the union by using
data.integer to access the value for all 4 types.  Instead, remove the
{buffer,window,tabpage} members and always use the integer member.

In order to accomodate this, perform distinct validation and coercion
between the Integer type and Buffer/Window/Tabpage types in
object_to_vim, msgpack_rpc helpers, and gendispatch.lua.
2016-11-02 10:06:27 -04:00
Björn Linse
f6968dc0f7 api: call multiple methods atomically (useful in async contexts)
remove unused response_id parameter of handle_nvim_... helpers
2016-10-22 10:51:32 +02:00
Justin M. Keyes
6186df3562 event/multiqueue.c: Rename "queue" to "multiqueue".
`lib/queue.h` implements a basic queue. `event/queue.c` implements
a specialized data structure on top of lib/queue.h; it is not a "normal"
queue.

Rename the specialized multi-level queue implemented in event/queue.c to
"multiqueue", to avoid confusion when reading the code.

Before this change one can eventually notice that "macros (uppercase
symbols) are for the normal queue, lowercase operations are for the
multi-level queue", but that is unnecessary friction for new developers
(or existing developers just visiting this part of the codebase).
2016-10-02 00:24:49 +02:00
Björn Linse
acb7c826b3 api: fix leak when a api function is incorrectly called with a list.
This applies both to msgpack-rpc and eval.
2016-08-31 21:57:06 +02:00