This is a regression after PR #7704:
mac: Set $LANG based on the system locale
CFStringGetCStringPtr sometimes returns "lang_region" = NULL,
in this case CFStringGetCString is used instead,
which places output to "buf", but "buf" was not used
by the code.
Problem: MS-Windows: Problem with $HOME when is was set internally.
Solution: Only use the $HOME default internally. (Yasuhiro Matsumoto, closes
vim/vim#2013)
48340b62e8
Restore vim_getenv() behaviour for $HOME on Windows.
If a second and third child exit while we are already in the handler, we
will only see a single additional SIGCHLD. Therefore the handler must
not stop after processing a single child but should check all children.
Fixes#8740
It seems as though in an AppImage there's an extra child process that
dies at some early point, before we have set up a SIGCHLD handler. So
when we later get a SIGCHLD from a child that we do care about,
waitpid(-1, ...) tells us about the extra child - and we don't notice
that the interesting child has exited.
Or something like that!
See also:
* https://patchwork.kernel.org/patch/9949491/ in which perf hit
something similar
* discussion at the AppImage repository:
https://github.com/AppImage/AppImageKit/issues/812#issuecomment-404662110.
Fix is to be explicit about which process we are waitpid()'ing for, so
we never need be distracted by children that we don't know about.
Fixes#8104
ref be68f218ff
[OLDTEST] Running test_options
Failed: F /tests/oldtests|test_options :: Nvim exited with non-zero code
-en travis_fold:start:-tests-oldtests-test-options
=================================================================
==26191==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x621000067900 at pc 0x0000004768bd bp 0x7ffe6bd02de0 sp 0x7ffe6bd02580
READ of size 4096 at 0x621000067900 thread T0
0 0x4768bc in __interceptor_getpwnam /local/mnt/workspace/tmp/ubuntu_rel/llvm/utils/release/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1726:3
1 0x12847c5 in os_get_user_directory /home/travis/build/neovim/neovim/build/../src/nvim/os/users.c:82:8
2 0x125a3e3 in expand_env_esc /home/travis/build/neovim/neovim/build/../src/nvim/os/env.c:380:25
3 0x1257fdb in expand_env_save_opt /home/travis/build/neovim/neovim/build/../src/nvim/os/env.c:255:3
4 0x1291b77 in gen_expand_wildcards /home/travis/build/neovim/neovim/build/../src/nvim/path.c:1195:13
5 0x129e2e7 in expand_wildcards /home/travis/build/neovim/neovim/build/../src/nvim/path.c:2018:12
6 0x129e193 in expand_wildcards_eval /home/travis/build/neovim/neovim/build/../src/nvim/path.c:1986:11
7 0xc2ddc3 in ExpandFromContext /home/travis/build/neovim/neovim/build/../src/nvim/ex_getln.c:4685:11
8 0xc29412 in ExpandOne /home/travis/build/neovim/neovim/build/../src/nvim/ex_getln.c:3775:9
9 0x931364 in f_expand /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:8257:32
10 0x811954 in call_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6373:11
11 0x8256b8 in get_func_tv /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6119:11
12 0x8ad6a1 in eval7 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:4217:15
13 0x8a9c6b in eval6 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3914:7
14 0x8a797f in eval5 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3765:7
15 0x8a319f in eval4 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3502:7
16 0x8a263c in eval3 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3420:7
17 0x8a1adc in eval2 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3351:7
18 0x809b21 in eval1 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3278:7
19 0x824f24 in get_func_tv /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6092:9
20 0x81e674 in ex_call /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:2726:9
21 0xb4db73 in do_one_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:2238:5
22 0xb30119 in do_cmdline /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:602:20
23 0x845232 in call_user_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:21396:3
24 0x81127b in call_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6358:11
25 0x8256b8 in get_func_tv /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6119:11
26 0x81e674 in ex_call /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:2726:9
27 0xb4db73 in do_one_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:2238:5
28 0xb30119 in do_cmdline /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:602:20
29 0x8645f1 in ex_execute /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:19541:7
30 0xb4db73 in do_one_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:2238:5
31 0xb30119 in do_cmdline /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:602:20
32 0x845232 in call_user_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:21396:3
33 0x81127b in call_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6358:11
34 0x8256b8 in get_func_tv /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6119:11
35 0x81e674 in ex_call /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:2726:9
36 0xb4db73 in do_one_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:2238:5
37 0xb30119 in do_cmdline /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:602:20
38 0xb19f2d in do_source /home/travis/build/neovim/neovim/build/../src/nvim/ex_cmds2.c:2973:3
39 0xb16580 in cmd_source /home/travis/build/neovim/neovim/build/../src/nvim/ex_cmds2.c:2718:14
40 0xb16677 in ex_source /home/travis/build/neovim/neovim/build/../src/nvim/ex_cmds2.c:2699:3
41 0xb4db73 in do_one_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:2238:5
42 0xb30119 in do_cmdline /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:602:20
43 0xb362c5 in do_cmdline_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:278:10
44 0xe8e3a3 in exe_commands /home/travis/build/neovim/neovim/build/../src/nvim/main.c:1705:5
45 0xe7bbba in main /home/travis/build/neovim/neovim/build/../src/nvim/main.c:518:5
46 0x2b0e17bd1f44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287
47 0x44dcfb in _start (/home/travis/build/neovim/neovim/build/bin/nvim+0x44dcfb)
0x621000067900 is located 0 bytes to the right of 4096-byte region [0x621000066900,0x621000067900)
allocated by thread T0 here:
0 0x4eeed3 in malloc /local/mnt/workspace/tmp/ubuntu_rel/llvm/utils/release/final/llvm.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:67:3
1 0xf87981 in try_malloc /home/travis/build/neovim/neovim/build/../src/nvim/memory.c:87:15
2 0xf87ba9 in xmalloc /home/travis/build/neovim/neovim/build/../src/nvim/memory.c:121:15
3 0x1257f6f in expand_env_save_opt /home/travis/build/neovim/neovim/build/../src/nvim/os/env.c:254:15
4 0x1291b77 in gen_expand_wildcards /home/travis/build/neovim/neovim/build/../src/nvim/path.c:1195:13
5 0x129e2e7 in expand_wildcards /home/travis/build/neovim/neovim/build/../src/nvim/path.c:2018:12
6 0x129e193 in expand_wildcards_eval /home/travis/build/neovim/neovim/build/../src/nvim/path.c:1986:11
7 0xc2ddc3 in ExpandFromContext /home/travis/build/neovim/neovim/build/../src/nvim/ex_getln.c:4685:11
8 0xc29412 in ExpandOne /home/travis/build/neovim/neovim/build/../src/nvim/ex_getln.c:3775:9
9 0x931364 in f_expand /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:8257:32
10 0x811954 in call_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6373:11
11 0x8256b8 in get_func_tv /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6119:11
12 0x8ad6a1 in eval7 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:4217:15
13 0x8a9c6b in eval6 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3914:7
14 0x8a797f in eval5 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3765:7
15 0x8a319f in eval4 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3502:7
16 0x8a263c in eval3 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3420:7
17 0x8a1adc in eval2 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3351:7
18 0x809b21 in eval1 /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:3278:7
19 0x824f24 in get_func_tv /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6092:9
20 0x81e674 in ex_call /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:2726:9
21 0xb4db73 in do_one_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:2238:5
22 0xb30119 in do_cmdline /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:602:20
23 0x845232 in call_user_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:21396:3
24 0x81127b in call_func /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6358:11
25 0x8256b8 in get_func_tv /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:6119:11
26 0x81e674 in ex_call /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:2726:9
27 0xb4db73 in do_one_cmd /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:2238:5
28 0xb30119 in do_cmdline /home/travis/build/neovim/neovim/build/../src/nvim/ex_docmd.c:602:20
29 0x8645f1 in ex_execute /home/travis/build/neovim/neovim/build/../src/nvim/eval.c:19541:7
SUMMARY: AddressSanitizer: heap-buffer-overflow /local/mnt/workspace/tmp/ubuntu_rel/llvm/utils/release/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1726:3 in __interceptor_getpwnam
Shadow bytes around the buggy address:
0x0c4280004ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4280004ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4280004ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4280004f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c4280004f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c4280004f20:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4280004f30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4280004f40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4280004f50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4280004f60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4280004f70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==26191==ABORTING
Failed: E /tests/oldtests|logs :: Runtime errors detected.
Job exited with code 1
Screen (23 lines)
Problem: Using uninitialized memory when 'isfname' is empty.
Solution: Don't call getpwnam() without an argument. (Dominique Pelle,
closesvim/vim#1464)
187a4f2814
Fixes 2 failing tests in startup_spec.lua.
The Windows-only `--literal` option complicates support of "stdin-as-text
+ file-args" (#7679). Could work around it, but it's not worth
the trouble:
- users have a reasonable (and englightening) alternative: nvim +"n *"
- "always literal" is more consistent/predictable
- avoids platform-specific special-case
Unrelated changes:
- Replace fileno(stdxx) with STDXX_FILENO for consistency (not motivated
by any observed technical reason).
silent-mode (-es/-Es) has been broken for years. The workaround up to
now was to include --headless. But --headless is not equivalent because
it prints all messages, not the limited subset defined by silent-mode.
After this change we never release blocks from memory (in practice it
never happened because the memory limits are never reached). Let the OS
take care of that.
---
On today's systems the 'maxmem' and 'maxmemtot' values are huge (4+ GB)
so the limits are never reached in practice, but Vim wastes a lot of
time checking if the limit was reached.
If the limit is reached Vim starts saving pieces of the swap file that were in
memory to the disk. Said in a different way: Vim implements its own
memory-paging mechanism. This is unnecessary and inefficient since the
operating system already has virtual memory and will swap to the disk if
programs start using too much memory.
This change does...
1. Reduce the number of config options and need for documentation.
2. Make the code more efficient as we don't have to keep track of memory
usage nor check if the memory limits were reached to start swapping
to disk every time we need memory for buffers.
3. Simplify the code. Once memfile.c is simple enough it could be
replaced by actual operating system memory mapping (mmap,
MemoryViewOfFile...). This change does not prevent Vim to recover
changes from swap files since the swapping code is never triggered
with the huge limits set by default.
Vim has the 'swapsync' option which we removed in 62d137ce09.
Instead let 'fsync' control swapfile-fsync.
These cases ALWAYS force fsync (ignoring 'fsync' option):
- Idle (CursorHold).
- Exit caused by deadly signal.
- SIGPWR signal.
- Explicit :preserve command.
* Reading from stdin on Windows is fixed in the same way as it was in
#8267.
* The file_read function was returning without filling the
destination buffer when it was called with a non-blocking file
descriptor.
closes#8196
For historical reasons, uint64_t and friends are defined both as
typedefs and macros. Some platforms that do that define the macros as
identity (#define uint64_t uint64_t), others like NetBSD define to the
backing type (#define uint64_t __uint64_t). This is normally
transparent, except when multiple levels of macro expansions are used
inconsistently.
TODO: Raymond Chen explains[1] racy behavior of the
CreateToolhelp32Snapshot approach. Better approach:
> create a job object and put process P in it. Then call
> QueryInformationJobObject with JobObjectBasicProcessIdList to get the
> list of child processes.
[1] "Why is CreateToolhelp32Snapshot returning incorrect parent process IDs all of a sudden?"
https://blogs.msdn.microsoft.com/oldnewthing/20150403-00/?p=44313
XXX: comment at https://stackoverflow.com/q/1173342 :
> Windows recycles PIDs quite fast, you have to be extra careful not
> to kill unrelated processes. These APIs will report PPIDs for long
> dead processes whose PIDs may have been recycled. Check the parent
> start date to make sure it is related to the processes you spawned.
UV_PROCESS_DETACHED compels libuv:uv__process_child_init() to call
setsid() in the child just after fork(). That ensures the process and
its descendants are grouped in a separate session (and process group).
The following jobstart() call correctly groups `sh` and `sleep` in a new
session (and process-group), where `sh` is the "session leader" (and
process-group leader):
:call jobstart(['sh','-c','sleep 60'])
SESN PGRP PID PPID Command
30383 30383 30383 3620 │ ├─ -bash
30383 31432 31432 30383 │ │ └─ nvim -u NORC
30383 31432 31433 30383 │ │ ├─ nvim -u NORC
8105 8105 8105 31432 │ │ └─ sh -c sleep 60
8105 8105 8106 8105 │ │ └─ sleep 60
closes#6530
ref: https://stackoverflow.com/q/1046933
ref: https://unix.stackexchange.com/a/404065
Helped-by: Marco Hinz <mh.codebro+github@gmail.com>
Discussion
------------------------------------------------------------------------
On my linux box before this patch, the termclose_spec.lua:'kills job
trapping SIGTERM' test indirectly causes cmake/busted to wait for 60s.
That's because the test spawns a `sleep 60` descendant process which
hangs around even after nvim exits: nvim killed the parent PID, but not
PGID (process-group), so the grandchild "reparented" to init (PID 1).
Session contains processes (and process-groups) which are logically part
of the same "login session". Process-group is a set of
logically/informally-related processes within a session; for example,
shells assign a process group to each "job". Session IDs and PGIDs both
have type pid_t (like PIDs).
These OS-level mechanisms are, as usual, legacy accidents whose purpose
is upheld by convention and folklore. We can use session-level grouping
(setsid), or we could use process-group-level grouping (setpgid).
Vim uses setsid() if available, otherwise setpgid(0,0).
Windows
------------------------------------------------------------------------
UV_PROCESS_DETACHED on win32 sets CREATE_NEW_PROCESS_GROUP flag.
But uv_kill() does not kill the process-group:
https://github.com/nodejs/node/issues/3617
Ideas:
- Set UV_PROCESS_DETACHED (CREATE_NEW_PROCESS_GROUP), then call
GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, pid)
- Maybe won't work because MSDN says "Only processes that share the
same console as the calling process receive the signal."
https://docs.microsoft.com/en-us/windows/console/generateconsolectrlevent
But CREATE_NEW_PROCESS_GROUP creates a new console ...
ref https://stackoverflow.com/q/1453520
- Group processes within a "job". libuv does that *globally* for
non-detached processes: uv__init_global_job_handle.
- Iterate through CreateToolhelp32Snapshot().
- https://stackoverflow.com/q/1173342
- Vim does this, see terminate_all()
With MSVC, STDOUT_FILENO and STDERR_FILENO are defined as function calls instead of constants, meaning they can't be assigned to enum values. The enum was only used in one file, so it has been removed. A definition for STDIN_FILENO has been added that is consistent with the other two definitions.
