ref #26144
The C backend must not use `sfNeverRaises` to remove exception checks
from
virtual method calls. The flag describes only the selected base method
body,
while a vtable override may raise a catchable exception.
This change makes `canRaiseDisp` conservative for `skMethod` symbols and
adds a
regression test covering an exception raised by a child method invoked
through a
base reference.
Fixes#11797.
Imported scalar and pointer aliases inherit their external C spelling,
but
receive a different Nim symbol. Signature hashing previously used that
symbol
identity, so aliases that emit exactly the same C type could produce
different
backend names for tuples, sequences, and other generic types.
For example, `cint` and `type CIntAlias = cint` both emit `int`, but
`seq[cint]` and `seq[CIntAlias]` could be emitted as incompatible C
structs.
The Nim type checker nevertheless permits assignments and calls between
them,
causing the generated C or C++ compilation to fail.
This changes the backend hash to use the external type spelling when
available.
A symbol-based fallback remains for imported types without a resolved
spelling.
The change deliberately does not collapse imported types into their
underlying
Nim builtin. Types such as `pid_t`, imported pointers with qualifiers,
and
platform typedefs may require distinct backend representations.
## NIF and incremental compilation
This does not change NIF serialization, NIF type keys, or the IC cache
format.
The bug is in backend type-name generation. An IC regression test is
included
to ensure that the corrected backend identity is preserved when
compilation
passes through the NIF pipeline.
## Tests
The regressions cover:
- tuple and sequence assignments between an imported type and its alias
- cross-module sequence parameters and mutation
- C and C++ backends
- NIF-backed incremental compilation
Existing C-type tests were also run under C/C++, refc, and ARC.
## Remaining scope
This does not solve the broader question of compatibility between
imported and
builtin types that have different backend identities, such as
`seq[cdouble]` and `seq[float]`. That remains tracked by #19374.
Grinding a small figdraw-based program under `nim ic` and diffing its
output against the classic backend surfaced eight bugs, four of which
silently produced a wrong binary rather than an error.
Frontend / build graph (`deps.nim`):
* Dead `when`-guarded imports were compiled anyway. `when someStrdefine
== "x": import y` is `cvUnknown` to the scanner, which conservatively
keeps the edge — right for an edge, but it also gave `y` its own `nim m`
rule, so a build died on a package the user never installed because they
never selected that backend. Track which edges are speculative and drop
a speculative subtree that cannot compile; if the guard was in fact
live, the discovery fixpoint puts the node back with the honest `cannot
open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so
nothing re-fires and `nim ic` relinked a stale binary while `nim c`
reported `cannot open file`. Report an unresolvable import from a
non-speculatively reached module during the graph scan.
* Macro-generated imports were discovered once and then forgotten.
Discovery only ran after a failure and the graph is re-derived
statically every run, so on a warm build the discovered module had no
rules at all and editing it changed nothing. Seed the graph from the
`.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from
file mtimes and never looks at a rule's command line, so `-d:foo=bar` /
`--mm:` / `--threads:` regenerated the build file with the new switches
and re-fired zero rules. Reify the configuration as a file and make it
an input of every rule.
* Command-line switches never reached the children: they replay the
project's config files, never the driver's argv, so `nim ic --opt:speed`
produced a byte-identical debug binary (likewise `--panics`,
`--experimental`, `--passC`). Forward the driver's switches, minus the
ones that must differ per child.
Artifacts and codegen:
* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw
the rule as satisfied on the next run: `nim ic` then reported success
for a program that does not compile, and generated code from
error-bearing AST (or hit an internal error in `ccgexprs`). Never
persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives
on the module symbol, which `moduleFromNifFile` rebuilds from scratch,
so `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a
module-level `block: let h = openHandle()` never ran `=destroy`. Persist
the flag as a `(modflags)` record. `injectdestructors` also has to
tolerate the `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym
node is written as a bare NIF `SymUse` token, which has nowhere to put
node flags, so the frontend's move analysis never reached the backend:
EVERY first assignment to a destructor-bearing local compiled as
`=sink`, i.e. `=destroy` on still-zeroed memory followed by a copy, and
no read was ever a move. Wrap a sym use in `(nflags ...)` when it
carries persistent node flags.
In the JS backend `toOpenArray` used `slice` (a copy), so writes through
a `var openArray` parameter silently vanished.
This emits `subarray` (a live shared-buffer view) for homogeneous
numeric arrays, otherwise such parameters are passed as a `{base, off,
len}` view that always aliases the caller's storage. Sliced seq/array
args become `{base, off, len}`, whole values `{base, off:0, len}`,
re-slices rebase.
Un-skips the JS guard in tests/openarray/topenarray.nim
Fixes#15952.
#23835 tried to do this, but it also switched to `long int` on the GNU
ABI, where it's actually just `int`.
Fixes#26111
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
fixes#26015
Fixes imported global variables with codegenDecl being emitted as
definitions instead of extern declarations.
A variable’s codegenDecl format should customize its definition in the
owning module. Other modules referencing the variable must emit a normal
declaration:
```c
extern NI variable;
```
After the variable-declaration builder refactor, genVarPrototype passed
Extern visibility to addVar. However, the sfCodegenDecl branch returned
before applying that visibility. This caused importing modules to emit
another tentative definition, resulting in duplicate-symbol linker
errors.
The fix restores the previous distinction between the custom definition
and cross-module prototypes. It also adds C and C++ regression coverage
for both direct access and access through an inline procedure.
follows up https://github.com/nim-lang/Nim/pull/24423
Specialize `rawAlloc` for alignment (cherry-picked from the other PR).
This cuts the frame of the normal unaligned path down enough to regain
the performance lost from loading the cold page in #26110
Also cleans up `MemRegion` a bit, the regressions are either gone or
were measurement errors.
fixes#26123
`cast[T](x)` is a transparent path expression for compiler analysis.
Previously, move/alias analysis could fail to see a later use through a
cast and incorrectly mark the source as moved, causing the issue’s
segmentation fault.
for views,
https://nim-lang.org/docs/manual_experimental.html#view-types-path-expressions:
A cast expression cast[T](e) is a path expression.
It also affects skipConvDfa, isAnalysableFieldAccess, and aliases. And I
might narrow it down for the two cases above mentioned if it causes
problems
Issue popped up when using `fromJson` into an array but the JSON passed
is an object
```nim
import std/[jsonutils, json]
let data = parseJson """
{"key": "value"}
"""
var foo: seq[int]
foo.fromJson(data)
echo foo #> @[0]
```
Basically the `setLen` would set the size to be equal to the number of
keys, but `getElems` just returns an empty array if the JSON isn't an
array which lead to it just creating zero'd items in the seq without
letting the user know.
Felt adding the checks was better than just skipping the `setLen` since
it lets the user know that there is a problem with the JSON
fixes#25992
```nim
type
Foo = object
case kind: bool
of true:
a: ref Bar # 8 bytes (pointer)
of false:
b: int # 4 bytes
```
specializeResetT for b emits accessor.b = 0 — writes 4 bytes
But the union is 8 bytes wide (sized by the largest branch)
The remaining 4 bytes where a used to live are untouched
Those stale bytes could contain a heap pointer the GC traces → crash
Add nimZeroMem after specializeResetN for case objects to clear the
entire union including unused branch bytes.
We defer copying the AST during generic instantiation until a cache miss
and fetch from cache based on bindings. On miss, we fall back to the old
logic and populate the cache.
This gains us a roughly 36% reduction in memory usage during bootstrap,
from `799.242MiB` down to `508.672MiB` on my machine.
For another point of reference, nimbus-eth2 goes from `10.5GB` memory
usage to `7.5GB`.
Independent companion to #26090 which together with this one yields a
bit under 20% faster compiles (or at least `--compileOnly` bootstraps)
on ORC.
fixes#26088
in https://github.com/nim-lang/Nim/pull/25772, `beginStores` requires
`newLen` to be passed for setting up the new length. So `streams.nim`
must make the string length exactly newLen.
…haracterization with--warning:ResultUsed:on
fixes#26062
> A return statement with no expression is shorthand for return result.
> ResultUsed: Warn about the usage of the built-in result variable.
> A procedure that does not have any return statement and does not use
the special result variable returns the value of its last expression.
`unapplyBlockContext` reset `inEnforcedGcSafe` and
`inEnforcedNoSideEffects` to false whenever a `{.cast(gcsafe).}` or
`{.cast(noSideEffect).}` block ended. When such a block is nested inside
another block of the same cast, the inner block's exit switched
enforcement back on for the rest of the enclosing block, so statements
lexically inside the outer cast were rejected.
The fix saves both flags in `PragmaBlockContext` when the block context
is created and restores the saved values on exit. That matches how every
other piece of block state there (`locked`, `exc`, `tags`, `forbids`) is
already handled; these two bools were the only ones reset to a constant
instead of restored.
No change for non-nested blocks: entering from the non-enforced state
saves false, so exit still clears the flag. A statement after the outer
block is still rejected as before.
Test covers nested `cast(gcsafe)` and nested `cast(noSideEffect)`.
`tests/effects` passes unchanged (49/49, same as stock).
Boostrap without linking is (conservatively) 17% faster on my machine.
Test setup runs the compiler compiling itself under ORC in release mode
from a clean cache and no C compilation (`--compileOnly`.)
Multiple samples are gathered before judging a potential optimization.
The test suite passes locally before push and bootstrap is tested with
strict views (mostly as a sanity check).
Tests are fair in the sense that they compile the same git worktree and
produce identical C output.
Refc will see less or no benefit.
Changes:
- Save tree traversals in `considerGenSyms` when no mappings exist
- Inline `maybeSkipDistinct` into `typeRel` for a safe cursor
- Only skip to static when there is a static type to reach in
`paramTypesMatchAux`
- Disable overflow checks for hashes
- Disable bounds checks for `nextIdentIter`
- Use cursor annotation when judged safe
- Use lent annotation for ast accessors
Spiritual companion to #26084
## Fix `globalSymbols` support on POSIX
### Summary
Fix `-d:globalSymbols` on POSIX platforms by defining `RTLD_GLOBAL`
alongside `RTLD_NOW` in `system/dyncalls.nim`.
On Linux and macOS, `RTLD_NOW` is defined locally in `dyncalls.nim`, but
`RTLD_GLOBAL` was not. As a result, enabling `-d:globalSymbols` could
fail because `RTLD_GLOBAL` was undeclared.
This change:
* defines `RTLD_GLOBAL` as `0x100` on Linux,
* defines `RTLD_GLOBAL` as `0x8` on macOS,
* imports `RTLD_GLOBAL` from `<dlfcn.h>` on other POSIX platforms.
These values are consistent with the existing POSIX constants already
used elsewhere in the Nim source tree.
### Motivation
`globalSymbols` is intended to load dynamic libraries with `RTLD_GLOBAL`
so that their exported symbols are available to subsequently loaded
shared libraries.
This is needed, for example, when a dynamically loaded library later
loads a plugin or provider that depends on symbols from the first
library.
Without this fix, `-d:globalSymbols` cannot be used reliably for that
purpose on POSIX systems.
### Testing
Tested on Linux with an AArch64 target.
A program using dynamically loaded OpenSSL libraries and a subsequently
loaded OpenSSL provider failed when the OpenSSL libraries were loaded
with the default local symbol visibility.
Using `RTLD_GLOBAL` made the same program work correctly.
After this change, building the original Nim program with:
```text
-d:globalSymbols
```
successfully loads the OpenSSL libraries with global symbol visibility,
and the provider-based TLS 1.2 and TLS 1.3 tests both pass.
The same behavior was also independently reproduced using direct
`dlopen()` / `dlsym()` calls:
```text
RTLD_LOCAL -> TLS 1.2 failed
RTLD_GLOBAL -> TLS 1.2 passed
```
Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>
`bundleChecksums` compiles the two NIF host tools through
`nimCompileFold`, which spawns a fresh `nim c` with only the `options`
string. Unlike the other `bundle*` procs it takes no `args`, so the
`--skipUserCfg --skipParentCfg` that `koch boot` was invoked with never
reached these two sub-builds.
For a Nim checked out inside another project (nimbus-eth1/eth2, nimbos
vendor Nim under `vendor/nimbus-build-system/vendor/Nim`) Nim then walks
the parent directories of `dist/nimony/src/nifler` and applies the host
project's `config.nims` to the tool build: an injected
`--import:libbacktrace`, plus `warningAsError[UnusedImport]`,
`hintAsError[DuplicateModuleImport]` and
`hintAsError[ConvFromXtoItselfNotNeeded]` all turn ordinary nimony code
into hard errors and break `koch boot`.
Host tools must not be configurable by whatever directory Nim happens to
sit under, so hardcode the skip flags (and `--noNimblePath`, matching
the other bundlers) rather than threading `args` through all seven call
sites.
`nimDecRefIsLast` always performed an atomic decrement. When the biased
count is already zero the destroying thread holds the only reference, so
there is nothing to adjudicate and the read-modify-write can be skipped.
Soundness: a counted reference can only be derived from the location
being destroyed -- which happens-before this destructor unless the
program races on that location -- or from another counted reference,
whose contribution is already in `rc` and therefore forces the slow
path. Observing zero proves no other thread holds a reference and that
none can appear. This relies on `--mm:atomicArc` having no collector;
ORC and YRC mutate `rc` from a participant that holds no counted
reference at all, so the fast path is deliberately not enabled for them.
The slow path keeps deciding on the value its own RMW returned. That is
what separates this from nim-lang/threading#45, where the "who frees"
role was decided from a separate load and the RMW result was discarded,
so the role could be dropped by every participant at once.
gcbench, -d:danger, median of 21 pinned runs:
--mm:arc (non-atomic RC) 0.1310
--mm:atomicArc 0.1742
--mm:atomicArc + this 0.1330
-23.7%, closing 95% of the gap to non-atomic reference counting. gcbench
builds its trees with `sink` parameters, so it performs almost no
incRefs and the whole atomicArc penalty is decRef traffic. The worst
case -- a decrement that always sees rc > 0, so the load never pays off
-- measures +1.1%.
`-d:nimNoAtomicArcFastPath` restores the previous code path.
`strbasics.add` uses `copymem` when available. CT conditional
scaffolding mirrors the same from system.
Follow-up to #15951
Compile-time test for `strbasics.add` undiscarded and passes, though
pending bug #15952 is still open.
## Summary
- nimpretty with non-default \--indent\ (e.g. 3 or 10) produced invalid
indentation in if/block/try expression regions because layouter kept the
original column when \keepIndents > 0\ and ignored \indWidth\.
- Rebase the column onto \indWidth\ using the relative offset from the
enclosing block baseline (\indentStack[^1]\).
## Root cause
\parser.nim\'s \
imprettyDontTouch\ template sets \keepIndents\ for if/block/try
expressions. layouter in the \keepIndents > 0\ branch used \ ok.indent\
(source column) directly as \indentLevel\ without scaling by \indWidth\,
so lines in these regions kept the original column and misaligned with
the rest of the file when \--indent\ differed from source indent width.
## Fix
\\\
im
em.indentLevel = em.indentStack.high * em.indWidth +
(tok.indent - em.indentStack[^1])
\\\
Keeps the relative offset from the enclosing block baseline but rebases
onto \indWidth\. At default \--indent:2\ the offset equals \indWidth\,
so output is unchanged (backwards compatible).
## Testing
- 12 custom cases x 3 indent values (2/3/10) = 36/36 pass
- nimpretty self-test suite 7/7 pass (no regression at default indent)
- 5 keepIndents scenarios (if/block/try expression continuation
alignment) that failed at indent:3/10 now pass
Fixes#20078.
fixes#25942fixes#25938
After a successful match to a concrete static T, normalizes an empty
static container literal to the formal payload type before binding it.
This prevents `static[set[empty]]({})` from leaking into the
instantiated proc body.
