`InterlockedCompareExchange64 `(winnt.h) is used instead of gcc atomics
when compiling with MSVC on Windows, but the function signatures are
`InterlockedCompareExchange64(ptr int64, int64, int64)` and
`InterlockedCompareExchange32(ptr int32, int32, int32)` as opposed to
`(ptr T, ptr T, T)` for `__atomic_compare_exchange_n`.
Passing a pointer to the expected value (parameter two) instead of the
value itself causes the comparison to unconditionally fail, with stalls
in threaded code using atomic comparisons.
Fix the function signature for MSVC.
Signed-off-by: Ryan Walklin <ryan@testtoast.com>
(cherry picked from commit 2d0b62aa51)
The forward declarations cause `Exception` to be inferred - also,
`llrecv` is an internal implementation detail and the type of the
received item is controlled by generics, thus the ValueError raised
there seems out of place for the generic api.
(cherry picked from commit 91febf1f4c)
- Removed redundant `len` and `reserved` sets already performed by prior
`rawNewStringNoInit` calls.
- Reuse `appendChar`
- Removed never used `newOwnedString`
- Added internal `toOwnedCopy`
- Documents differences in impls of internal procs used for
`system.string.setLen`:
+ `strs_v2.setLengthStrV2`:
- does not set the terminating zero byte when new length is 0
- does not handle negative new length
+ `sysstr.setLengthStr`:
- sets the terminating zero byte when new length is 0
- bounds negative new length to 0
(cherry picked from commit b539adf829)
Continuation of #25180. This one refactors the sequence routines.
Preparation for extending with new routines.
Mostly removes repeating code to simplify debugging.
Removes:
- `incrSeqV2` superseded by `incrSeqV3`,
- `setLengthSeq` superseded by `setLengthSeqV2`
Note comment on line 338, acknowledging that implementation of
`setLenUninit` from #25022 does zero the new memory in this branch,
having been copied from `setLengthSeqV2`. This PR does not fix this.
(cherry picked from commit 01c084077e)
fixes#25007
```nim
proc setLengthSeqUninit(s: PGenericSeq, typ: PNimType, newLen: int, isTrivial: bool): PGenericSeq {.
compilerRtl.} =
```
In this added function, only the line `zeroMem(dataPointer(result,
elemAlign, elemSize, newLen), (result.len-%newLen) *% elemSize)` is
removed from `proc setLengthSeqV2` when enlarging a sequence.
JS and VM versions simply use `setLen`.
(cherry picked from commit 611b8bbf67)
fixes#19728
don't zero-filling memory for "trivial types" without destructor in
refc. I tested locally with internal apis.
(cherry picked from commit b421d0f8ee)
While `a.push.apply(a, b)` is better for performance than the previous
`a = a.concat(b)` due to the fact that it doesn't create a new array,
there is a pretty big problem with it: depending on the JS engine, if
the second array is too long, it can [cause a
crash](https://tanaikech.github.io/2020/04/20/limitation-of-array.prototype.push.apply-under-v8-for-google-apps-script/)
due to the function `push` taking too many arguments. This has
unfortunately been what the codegen produces since 1.4.0 (commit
707367e1ca).
So string addition is now moved to a compilerproc that just uses a `for`
loop. From what I can tell this is the most compatible and the fastest.
Only potential problem compared to `concat` etc is with aliasing, i.e.
adding an array to itself, but I'm guessing it's enough that the length
from before the iteration is used, since it can only grow. The test
checks for aliased nim strings but I don't know if there's an extra
protection for them.
(cherry picked from commit 839cbeb371)
This PR, courtesy of @NagyZoltanPeter
(https://github.com/waku-org/nwaku/pull/3522) adds the ability to track
memory allocations in a program suitable for use with
[heaptrack](https://github.com/KDE/heaptrack).
By passing `-d:heaptrack --debugger:native` to compilation, calls to
heaptrack will be injected when memory is being allocated and released -
unlike `-d:useMalloc` this strategy also works with `refc` and the
default memory pool.
See https://github.com/KDE/heaptrack for usage examples. The resulting
binary needs to be run with `heaptrack` and with the shared
`libheaptrack_preload.so` in the `LD_LIBRARY_PATH`.
(cherry picked from commit 861ebc0f19)
fixes#25240
> Deque items behavior is not the same on 2.0.16 and 2.2.0
The behavior seems to be caused by the temp introduced for the parameter
`deq.len`, which prevents it from being evaluated multiple times
(cherry picked from commit b7c02e9bad)
POSIX signal has an identical definition to ISO C signal:
https://pubs.opengroup.org/onlinepubs/9799919799/functions/signal.html
```c
void (*signal(int sig, void (*func)(int)))(int);
/* more readably restated by glibc as */
typedef void (*sighandler_t)(int);
sighandler_t signal(int signum, sighandler_t handler);
```
However, std/posix had omitted the function's return value; this fixes
that.
To prevent breaking every single line of code ever that touched this
binding (including mine...), I've also made it discardable.
Additionally, I have noticed that bsd_signal's type signature is wrong -
it should have been identical to signal. But bsd_signal was already
removed in POSIX 2008, and sigaction is the recommended, portable POSIX
signal interface. So I just deleted the bsd_signal binding.
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
(cherry picked from commit 483389d399)
TODO list, copied from the documentation:
- [x] compiler/platform.nim Add os/cpu properties.
- [x] lib/system.nim Add os/cpu to the documentation for system.hostOS
and system.hostCPU.
- [x] ~~compiler/options.nim Add special os/cpu property checks in
isDefined.~~ seems unnecessary; isn't dont for most CPUs
- [x] compiler/installer.ini Add os/cpu to Project.Platforms field.
- [x] lib/system/platforms.nim Add os/cpu.
- [x] ~~std/private/osseps.nim Add os specializations.~~
- [x] ~~lib/pure/distros.nim Add os, package handler.~~
- [x] ~~tools/niminst/makefile.nimf Add os/cpu compiler/linker flags.~~
already done in https://github.com/nim-lang/Nim/pull/20943
- [x] tools/niminst/buildsh.nimf Add os/cpu compiler/linker flags.
For csource:
- [x] have compiler/platform.nim updated
- [x] have compiler/installer.ini updated
- [x] have tools/niminst/buildsh.nimf updated
- [x] have tools/niminst/makefile.nimf updated
- [ ] be backported to the Nim version used by the csources
- [ ] the new csources must be pushed
- [ ] the new csources revision must be updated in
config/build_config.txt
Additionally:
- [x] check relation to https://github.com/nim-lang/Nim/pull/20943
Possible future work:
- Porting Nim to s390x-specific operating systems, notably z/OS
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
(cherry picked from commit f4497c6158)
...introduced by me in #24792. Sorry.
This fix doesn't avoid copying the `restrictedBody` twice in the
generated code but has the benefit of working.
Proper fix needs a detection that can set a const bool for a module
once. `when nimvm` is restricted in use and is difficult to dance
around. Some details in: #12517, #12518, #13038
I might have copied the buggy solution from some discussion and it might
have worked at some point, but it's small excuse.
(cherry picked from commit 6938fce40c)
fixes#24998
Basically it retraces back to the situation before
https://github.com/nim-lang/Nim/pull/18366 and
https://github.com/nim-lang/Nim/pull/18362, i.e.
```nim
import fuzz/a
import fuzz/a
```
```nim
import fuzz/a
from buzz/a
```
```nim
import fuzz/a except nil
from fuzz/a import addInt
```
All of these cases are now flagged as invalid and triggers a
redefinition error, i.e., each module name importing is treated as
consistent as the symbol definition
kinda annoying for importing/exporting with `when conditions` though
ref https://github.com/nim-lang/Nim/issues/18762https://github.com/nim-lang/Nim/issues/20907
```nim
from std/strutils import toLower
when not defined(js):
from std/strutils import toUpper
```
(cherry picked from commit 87ee9c84cb)
Inside a signal handler, you cannot allocate memory because the signal
handler, being implemented with a C
[`signal`](https://en.cppreference.com/w/c/program/signal) call, can be
called _during_ a memory allocation - when that happens, the CTRL-C
handler causes a segfault and/or other inconsistent state.
Similarly, the call can happen from a non-nim thread or inside a C
library function call etc, most of which do not support reentrancy and
therefore cannot be called _from_ a signal handler.
The stack trace facility used in the default handler is unfortunately
beyond fixing without more significant refactoring since it uses
garbage-collected types in its API and implementation.
As an alternative to https://github.com/nim-lang/Nim/pull/25110, this PR
removes the most problematic signal handler, namely the one for SIGINT
(ctrl-c) - SIGINT is special because it's meant to cause a regular
shutdown of the application and crashes during SIGINT handling are both
confusing and, if turned into SIGSEGV, have downstream effects like core
dumps and OS crash reports.
The signal handlers for the various crash scenarios remain as-is - they
may too cause their own crashes but we're already going down in a bad
way, so the harm is more limited - in particular, crashing during a
crash handler corrupts `core`/crash dumps. Users wanting to keep their
core files pristine should continue to use `-d:noSignalHandler` - this
is usually the better option for production applications since they
carry more detail than the Nim stack trace that gets printed.
Finally, the example of a ctrl-c handler performs the same mistake of
calling `echo` which is not well-defined - replace it with an example
that is mostly correct (except maybe for the lack of `volatile` for the
`stop` variable).
(cherry picked from commit 41ce86b577)
Raising exceptions halfway through a memory allocation is undefined
behavior since exceptions themselves require multiple allocations and
the allocator functions are not reentrant.
It is of course also expensive performance-wise to introduce lots of
exception-raising code everywhere since it breaks many optimisations and
bloats the code.
Finally, performing pointer arithmetic with signed integers is incorrect
for example on on a 32-bit systems that allows up to 3gb of address
space for applications (large address extensions) and unnecessary
elsewhere - broadly, stuff inside the memory allocator is generated by
the compiler or controlled by the standard library meaning that
applications should not be forced to pay this price.
If we wanted to check for overflow, the right way would be in the
initial allocation location where both the size and count of objects is
known.
The code is updated to use the same arithmetic operator style as for
refc with unchecked operations rather than disabling overflow checking
wholesale in the allocator module - there are reasons for both, but
going with the existing flow seems like an easier place to start.
(cherry picked from commit 8b9972c8b6)
Also added some documentation to the header.
See: https://forum.nim-lang.org/t/13311
> I did try using the flag, but couldn't get it to work. If I do
-d:nimIoSelector, the defined check passes, but the other code fails to
compile because there is no const named nimIoSelector. It seemed like a
bug to me, do you have a working number compiler invocation?
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
(cherry picked from commit d60e0211bc)
`DWORD` is defined as `int32`, so `DWORD(...)` would not work as
expected. When writing to files larger than 2GB, exception occurs:
```
unhandled exception: value out of range: 4294967295 notin -2147483648 .. 2147483647 [RangeDefect]
```
This PR is a quick fix for this.
P.S. Why `DWORD` is defined as `int32`?
(cherry picked from commit 4f09675d8a)
`strutils.formatSize` returns correct strings from large values close to
`int64.high`.
Round down `bytes` when it is converted to float.
(cherry picked from commit 065c4b443b)
- Added support for SOCKS5h (h for proxy-side DNS resolving) to
httpclient
- Deprecated `auth` arguments for `newProxy` constructors, for auth to
be embedded in the url.
Unfortunately `http://example.com` is not currently reachable from
github CI, so the tests fail there for a few days already, I'm not sure
what can be done here.
(cherry picked from commit 161b321796)
revert #24896
Partially reverting #24896 in #25024 broke CI. So better revert it
completely so the CI is green. I'll investigate the issue later.
(cherry picked from commit 08642ffe34)
Reworked closureiter transformation.
- Convolutedly nested finallies should cause no problems now.
- CurrentException state now follows nim runtime rules (pushes and pops
appropriately), and mimics normal code, which is somewhat buggy, see
#25031
- Previously state optimization (removing empty states or extra jumps)
missed some opportunities, I've reimplemented it to do everything
possible to optimize the states. At this point any extra states or jumps
should be considered a bug.
The resulting codegen (compiled binaries) is also slightly smaller.
**BUT:**
- I had to change C++ reraising logic, see expt.nim. Because with
closure iters `currentException` is not always in sync with C++'s notion
of current exception. From my tests and understanding of C++ runtime
there should not be any problems, but I'm only 99% sure :)
- I've reused `nfNoRewrite` flag in one specific case during the
transformation. This flag is also used in term-rewriting logic. Again,
99% sure, these 2 scenarios will never intersect.
(cherry picked from commit 36f8cefa85)
Docs are routinely compiled on a different OS so often don't reflect
reality of CT-conditionals.
I bet there's a few of other places like this in the stdlib.
(cherry picked from commit 6bdb069a66)
todo: We can also give a deprecation message for `ltPtr`/`lePtr`
matching for cstring in `magicsAfterOverloadResolution`
follow up https://github.com/nim-lang/Nim/pull/24942
(cherry picked from commit ade500b2cb)
explaining why the result may not be so surprising. Clean-up of stray
whitespace and insert of missing "in" along for the ride.
It's just not always faster or slower than `Table`. The difference
depends upon many factors such as (at least!): A) how much (if anything
- for `int` keys it is nothing) hash-comparison before `==` comparison
saves B) how much resizing happens (which may even vary from run to run
if end users are allowed to provide scale guess input), C) how much
comparison happens at all (i.e., table density), D) how much space/size
matters - like how close to a specific deployment "available" cache size
the table is.
If we want, we could add a sentence suggesting performance fans also try
`Table`, but the kind of low-level nature of the explanation strikes me
as already along those lines.
(cherry picked from commit 091fb5057b)
