`interfHidden` was 1.05s of a cold Atlas build: 1.70M hidden-symbol
stubs against 0.29M exported ones, created by every `nim m` for every
module it imports. The table is reached ONLY through
`modulegraphs.interfSelect` with `optImportHidden`, and that flag is set
in exactly one place — an `import x {.all.}`. Almost nothing reads it.
So it is built when something asks. Every read goes through
`interfSelect`, so guarding its four call sites is complete;
`modulegraphs` already imports `ast2nif`, so the call is direct.
The reason the first attempt at this failed, recorded because it is not
guessable: **a module has two FileIndexes.** `registerNifSuffix` keys
`filenameToIndexTbl` by the NIF SUFFIX and mints a `fikNifModule` entry,
while the graph indexes `g.ifaces` by the module's `fikSource` file, and
`DecodeContext.mods` is keyed by the former. Asking it with the latter
misses every time, silently — `import x {.all.}` then reported
"undeclared identifier" for a symbol that was right there. The lazy
builder therefore takes a SUFFIX. Two more conditions are load-bearing:
clear the pending flag only when the build SUCCEEDS, since an import
whose `.s.bif` the build has not produced yet must be retried rather than
written off for the rest of the process; and build into a LOCAL table
before assigning it back, since loading symbols can grow `g.ifaces` and
leave a `var` alias into it dangling.
Atlas, 204 modules:
frontend 9.98s -> 8.96s loading 4.57s -> 3.55s
InterfTables 1161ms -> 79ms hidden stubs 1.70M -> 0
cold serial 21.59s -> 19.79s cold parallel 11.11s -> 10.47s
Also fixes a scanner gap the test exposed. `import x {.all.}` serialises
as `(pragmax x (pragmas all))`, which `deps.nim.parseImportPath` did not
recognise, so it fell into the unknown-subtree skip and the import was
DROPPED from the static graph — the build only learned about it from the
`.s.deps` sidecar a round later, after a round that failed with
"requires precompiled NIF for import". Correct, but a wasted round and an
alarming error line for an ordinary import.
`tests/ic/timporthidden.nim` covers it: `{.all.}` sees the private
symbols, and the sibling case (a plain `import`) still rejects them under
both `--ic:on` and `--ic:off`. ic 42/42, `koch boot -d:release` equal
executables.
`nim c --ic:on` could not compile anything that reached `std/tempfiles`:
lib/std/tempfiles.nim(115, 38) Error: type mismatch
Expression: initRand()
Expected one of: [1] proc initRand(seed: int64)
That is `std/random`. It declares `proc initRand(): Rand` WITHOUT a `*`
and then, forty lines later, `since (1, 5, 1): export initRand`.
Two mechanisms make a symbol importable and the writer only knew one.
`sfExported` is the `*` on the declaration; `semExport` instead calls
`reexportSym`, which adds the symbol to the module's interface table and
sets no flag. `writeSymDef` decided the NIF `x` marker — "importable as a
bare identifier" — from `sfExported` alone, so a symbol exported the
second way shipped as private and its importer reported an undeclared
identifier. Nothing about this is IC-specific in the source; it is
ordinary stdlib code that classic compilation accepts.
So ask the interface too: `reexportedLocalSyms` collects the symbols a
module defines that reached its interface without the flag, and they get
the marker. Symbols that are genuinely private still do not — the
regression test's sibling case (`proc g()`, never exported) is still
rejected under both `--ic:on` and `--ic:off`.
This was found by pointing `--ic:on` at Atlas, which is now the first
sizeable third-party program it builds. `nim c --ic:off` and `--ic:on`
produce the same working binary from the same 209-module closure.
ic 41/41 (the new test is the 41st), `koch boot -d:release` equal
executables.
One conflict, in `canRaiseDisp`, where both sides added to the same guard:
* devel (#26145) short-circuits `skMethod` to "can raise", because a base
method's inferred effects describe only the base body, not every vtable
target;
* this branch resets `markCanRaiseBranch` to 5 on entry, so that a `-d:
icCanRaiseLog` differential attributes an answer to the branch that actually
decided it rather than to whatever the previous call left behind.
Both kept: the reset first, then devel's method branch, then the existing
flags branch. Marker 5 already means "decided here, neither predicate ran",
which is what the new branch does too, so it needs no new number — the comment
now says both short-circuits land there.
Verified on the merged tree: `tests/ic` 40/40, including devel's new
`timportcalias`; the cursor-driven and `PNode`-driven backends still generate
byte-identical C (67/67 under `--ic:on`); ccgbugs 146, concepts 48, method 22,
destructor 97, arc 140, gc 78, closure 23, iter 71, exception 47, all clean.
`tests/generics/tparser_generator.nim` fails, and does NOT come from this
merge: it fails the same way on pristine `origin/devel`, built and run in a
throwaway worktree to check. The compile succeeds; the spec expects no output
and the compiler now emits `typed`-deprecation and unused-import warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FMyRHByv7hhaQJ4Pa1bHbE
The metamorphic tests compile the program twice per step — once under `nim ic`,
once as the `nim c` reference — which is 100+ compilations for the category,
each fanning out a compiler process per module per stage. On a machine short of
RAM that swaps, and the symptoms read exactly like a deadlock: processes at 0%
CPU, no output, a different test "stuck" every run, and the same compilation
finishing in seconds standalone.
It is not a deadlock. A `nim ic` parent at 0% CPU is waiting on its children,
and the test name does keep changing if you watch long enough. Writing this down
because it has now been misdiagnosed as a testament/`nim ic` interaction more
than once, by me, before I measured `vm_stat` and `vm.swapusage`.
Also records the fan-out caps (`--parallelBuild:N`, `-d:icJobs:N`,
`-d:icNoParallel`), that serial mode is what makes per-process diagnostic output
readable rather than interleaved, that `testament r` cannot run a multi-step
metamorphic file, and that `*_temp.nim` are gitignored scratch rather than tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEF7FJvUkGKvG9LSGuEaNR
ref #26144
The C backend must not use `sfNeverRaises` to remove exception checks
from
virtual method calls. The flag describes only the selected base method
body,
while a vtable override may raise a catchable exception.
This change makes `canRaiseDisp` conservative for `skMethod` symbols and
adds a
regression test covering an exception raised by a child method invoked
through a
base reference.
Fixes#11797.
Imported scalar and pointer aliases inherit their external C spelling,
but
receive a different Nim symbol. Signature hashing previously used that
symbol
identity, so aliases that emit exactly the same C type could produce
different
backend names for tuples, sequences, and other generic types.
For example, `cint` and `type CIntAlias = cint` both emit `int`, but
`seq[cint]` and `seq[CIntAlias]` could be emitted as incompatible C
structs.
The Nim type checker nevertheless permits assignments and calls between
them,
causing the generated C or C++ compilation to fail.
This changes the backend hash to use the external type spelling when
available.
A symbol-based fallback remains for imported types without a resolved
spelling.
The change deliberately does not collapse imported types into their
underlying
Nim builtin. Types such as `pid_t`, imported pointers with qualifiers,
and
platform typedefs may require distinct backend representations.
## NIF and incremental compilation
This does not change NIF serialization, NIF type keys, or the IC cache
format.
The bug is in backend type-name generation. An IC regression test is
included
to ensure that the corrected backend identity is preserved when
compilation
passes through the NIF pipeline.
## Tests
The regressions cover:
- tuple and sequence assignments between an imported type and its alias
- cross-module sequence parameters and mutation
- C and C++ backends
- NIF-backed incremental compilation
Existing C-type tests were also run under C/C++, refc, and ARC.
## Remaining scope
This does not solve the broader question of compatibility between
imported and
builtin types that have different backend identities, such as
`seq[cdouble]` and `seq[float]`. That remains tracked by #19374.
Grinding a small figdraw-based program under `nim ic` and diffing its
output against the classic backend surfaced eight bugs, four of which
silently produced a wrong binary rather than an error.
Frontend / build graph (`deps.nim`):
* Dead `when`-guarded imports were compiled anyway. `when someStrdefine
== "x": import y` is `cvUnknown` to the scanner, which conservatively
keeps the edge — right for an edge, but it also gave `y` its own `nim m`
rule, so a build died on a package the user never installed because they
never selected that backend. Track which edges are speculative and drop
a speculative subtree that cannot compile; if the guard was in fact
live, the discovery fixpoint puts the node back with the honest `cannot
open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so
nothing re-fires and `nim ic` relinked a stale binary while `nim c`
reported `cannot open file`. Report an unresolvable import from a
non-speculatively reached module during the graph scan.
* Macro-generated imports were discovered once and then forgotten.
Discovery only ran after a failure and the graph is re-derived
statically every run, so on a warm build the discovered module had no
rules at all and editing it changed nothing. Seed the graph from the
`.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from
file mtimes and never looks at a rule's command line, so `-d:foo=bar` /
`--mm:` / `--threads:` regenerated the build file with the new switches
and re-fired zero rules. Reify the configuration as a file and make it
an input of every rule.
* Command-line switches never reached the children: they replay the
project's config files, never the driver's argv, so `nim ic --opt:speed`
produced a byte-identical debug binary (likewise `--panics`,
`--experimental`, `--passC`). Forward the driver's switches, minus the
ones that must differ per child.
Artifacts and codegen:
* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw
the rule as satisfied on the next run: `nim ic` then reported success
for a program that does not compile, and generated code from
error-bearing AST (or hit an internal error in `ccgexprs`). Never
persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives
on the module symbol, which `moduleFromNifFile` rebuilds from scratch,
so `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a
module-level `block: let h = openHandle()` never ran `=destroy`. Persist
the flag as a `(modflags)` record. `injectdestructors` also has to
tolerate the `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym
node is written as a bare NIF `SymUse` token, which has nowhere to put
node flags, so the frontend's move analysis never reached the backend:
EVERY first assignment to a destructor-bearing local compiled as
`=sink`, i.e. `=destroy` on still-zeroed memory followed by a copy, and
no read was ever a move. Wrap a sym use in `(nflags ...)` when it
carries persistent node flags.
In the JS backend `toOpenArray` used `slice` (a copy), so writes through
a `var openArray` parameter silently vanished.
This emits `subarray` (a live shared-buffer view) for homogeneous
numeric arrays, otherwise such parameters are passed as a `{base, off,
len}` view that always aliases the caller's storage. Sliced seq/array
args become `{base, off, len}`, whole values `{base, off:0, len}`,
re-slices rebase.
Un-skips the JS guard in tests/openarray/topenarray.nim
Fixes#15952.
A closure's environment type — and the `=destroy`/`=copy`/`=sink` the compiler
lifts for it — is minted by the BACKEND and exists in no module's semmed NIF.
Grinding a corpus of closure programs against `nim c` as the oracle turned up
three ways the per-module backend gets those wrong. `tests/closure` is now green
under `--ic:on`, and `tests/iter` goes from 13 failures to 9.
* **The env hook is emitted by nobody.** `emitsBodyInThisModule` walks up to the
outermost enclosing routine to pick the TU that emits a body. For the env
`=destroy` of a generic closure iterator defined in one module and instantiated
in another, that walk lands on the module of the ORIGINAL generic — a module
that never sees the instance — so the routine was emitted nowhere
(`undefined reference to eqdestroy__c485__…`). A backend-minted routine has no
owning module NIF at all: it is written into the `.t.bif` of every module that
references it, re-homed there with `@bk`. Every referencing TU emits it now and
the merge stage keeps one, like any other content-addressed definition.
* **The `:up` link is stored without an increment.** `env.:up = enclosingEnv` has
to go through `=copy` (with the cyclic incref) or the parent's refcount is one
too low, and at teardown the two envs' mutually recursive `=destroy`s each
believe they hold the last reference and recurse until the stack is gone — a
SIGSEGV after the program's own output has already been printed
(`tests/iter/tnestedclosures`, "Test 3"). Whether it becomes a `=copy` depends
on the up-field type's hooks existing at injection time. Whole-program cgen got
that for free: a LATER lifting pass creates them and it runs before any
routine's injection. The per-module backend injects a routine right after
lifting it (`lower`), long before the module's top level is transformed at all
(`cg`) — so both up-field assignment sites create the ops themselves.
* **Two backend hooks collide on one C name.** A backend-minted symbol is named
`_c<itemId.item>`, a PER-PROCESS counter — fine while "nifc lifts, emits and
compiles them in one run", which is what `mangleProcNameExt` assumed. But
`lower` mints the env hooks of nested routines and `cg` mints those of the
module's top level, and both land in the same translation unit: two unrelated
`=destroy`s became one C function. `mangleProcNameExt` already makes an
exception for hooks whose `disamb` is content-derived (`HookDisambBit`);
`toNifSymName` now mirrors it, so the content-derived value survives the round
trip instead of being overwritten by the loader.
`InstanceDisambBit`/`HookDisambBit` move to `astdef` — `ast2nif` names symbols by
them and cannot import `modulegraphs`.
Two new metamorphic tests, both of which fail on the previous compiler:
`tclosure_hooks` (generic closure iterator instantiated across modules) and
`tclosure_nested_iter` (closure iterator nested in a closure iterator).
`icFormatVersion` 37 -> 38 for the hook naming. `koch bootic` reaches its
byte-identical fixed point; `tests/closure`, `tests/destructor` (3 known
`--newruntime` failures), `tests/cpp` and the classic categories are unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`testament --ic` on `tests/destructor` went from 10 failures to 3; the three
left are all `--newruntime`, whose owned-ref RTTI destructor is still missing
(the object type's `=destroy` never reaches the `cg` that emits the type info).
* **global destructors are never run.** `graph.globalDestructors` is filled
while a module's top level goes through `injectDestructorCalls`, and
whole-program cgen empties the list into the main module's init proc — which
IS the program body, so the calls land at exit. Under `nim ic` every module's
`cg` is a separate process and main's only ever saw its own entries: a
module-level `var` with a `=destroy` in an imported module was simply never
destroyed. Each module now wraps its own list in a nullary exported proc and
announces the name in its `.c.nif` meta head (`CnifVersion` 4 -> 5); main's
`cg` reads the heads it already reads for init/datInit and calls them in
reverse dependency order. New test `tests/ic/tglobal_dtors.nim` pins the
order against the oracle.
* **`spawn` died with "system module needs: nimArgsPassingDone".** A module
loaded from a NIF is named by its mangled suffix, so `loadCompilerProc`'s
`module.name.s == "threadpool"` could never match. The backend loads the whole
program before codegen, so consult every loaded module's index instead; misses
are final and remembered (`getCompilerProc` doubles as a presence probe).
* **`new(x, finalizer)` died with "environment misses: x".** `semmagic`'s
finalizer-to-destructor wrapper copies the original's AST but only rewrites
the parameter, leaving `ast[namePos]` naming the ORIGINAL proc. ast2nif
re-derives a routine's serialized AST from `ast[namePos].sym.ast`, so the
wrapper serialized the original's body — whose parameter belongs to the
original — and lambda lifting saw it as a captured variable of another proc.
The copy now names itself, the invariant every other routine AST keeps.
* **a top-level `let (a, b) = f()` copied instead of moved.** A module's own
symbol is the owner of every top-level symbol and is written as a real `(sd)`,
so the loader minted a SECOND `skModule` PSym for it — and `sym.owner == owner`
is an identity test in `aliasanalysis.isAnalysableFieldAccess`, which made
every module-level location un-analysable. Hard error for a type with a
disabled `=copy`. Bind the NIF name to the one registered module symbol.
Backend only: doing it under `nim m` costs `times.toDateTimeByWeek` its
inferred `raises`.
* **a doubly linked list leaked its whole contents.** A field USE serializes as
a bare `SymUse` with nowhere to put symbol flags, so `trees.isCursor` said
"not a cursor" for every loaded field and `DoublyLinkedNode.prev` became a
COUNTED reference: every node held its predecessor alive and no refcount ever
hit zero. `{.cursor.}` now rides in the NIF name marker (`` `fc `` next to
`` `f ``), which def and use derive from the same `PSym`.
* **`--expandArc` came out shuffled.** `moduleSymbolStubs` iterated a `Table`,
i.e. hash order, so the `lower` stage transformed a module's routines in an
arbitrary order — not even stable between two compilers. Order by index
offset, which is source order.
Also: `testament`'s `generatedFile` did not include the matrix entry in the
nimcache key its caller uses, so every `ccodeCheck` test with a `matrix:`
reported `reCodeNotFound`.
`koch bootic` still reaches its fixed point; `tests/ic` and the classic
`tests/destructor` are unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every invariant `tests/ic` checked was IC-against-IC — clean == incremental, a
no-op edit changes nothing, a body edit moves no interface cookie. An IC that is
*consistently* wrong satisfies all of them, and that is exactly how two silent
miscompilations survived. `koch bootic` has the same blind spot: it proves the
compiler reproduces itself, not that it agrees with the reference backend.
Two mechanisms, at different scales.
**The oracle, in the metamorphic runner.** Every successful step is now also
compiled with `nim c` and run, and the two outputs must agree. Unlike the
hand-written `expect:` strings this needs no foresight from the test author: it
compares everything the program does, not only what someone thought to print,
which is what a silently-skipped destructor evades. `no-oracle` opts out.
The format also grew the expressiveness the recent bug hunt showed was missing —
every one of these described a state the suite could not reach:
* `#!DELETE <file>` removes a module. Deleting a still-imported file moves no
mtime, so nothing re-fires.
* `#!FLAGS <switches>` changes the compiler switches between steps. A config
change is not a file either.
* `fails: <substring>` asserts that BOTH compilers reject the program with that
text. Previously every step had to succeed, so the whole error path — and
recovery from it — was untested.
Six regression tests cover the eight bugs the last round fixed.
`testament r <file>` now dispatches metamorphic tests like `testament cat ic`.
**`testament --ic` runs the whole corpus through the incremental compiler**, so
IC inherits ~10k programs with expected output instead of 30 bespoke tests.
Two things had to change for that to mean anything:
* `nimcacheDir` now keys on the `matrix:` entry too. Two matrix variants of one
file are two different compilations; sharing a cache meant each run
invalidated what the previous left — harmless for a backend that caches only
object files, useless for an incremental one.
* About half the corpus overrides the command wholesale (`cmd: "nim c --gc:arc
$file"`), bypassing both `$target` and `$options`. Those are rewritten to `ic`
and given a private cache.
**Warm cache, hastur-style.** A generated warmup program pulling in `system` and
the most-imported stdlib modules is compiled once per distinct compile
configuration into `nimcache/ic_warmup_<hash>`, and each test's empty cache is
seeded from it with mtimes preserved (nifmake compares output-mtime >
input-mtime, so stamping the copies "now" re-fires the whole graph). Only
program-independent artifacts are copied: the frontend NIFs and cookies plus the
per-module `lower`/`cg` output. The `.c`/`.o` are left behind on purpose — the
merge decision is whole-program, so they are re-rendered for every program
anyway.
`tests/destructor` (97 runs): `nim c` 35s cold / 32s warm; `nim ic` ~3m30 cold /
**9.8s warm**.
Compiler changes this required or uncovered:
* `merge` read the live-module list from a manifest the driver writes instead of
globbing `*.c.nif` off the nimcache. Globbing absorbed artifacts belonging to
any other program sharing the directory — which is what made a prefilled cache
produce undefined symbols at link.
* The build-arg signature no longer includes `--icproject:`/`--icPreparsedConfig:`
(they name where a build lives, not what it produces, so two caches holding
identical artifacts got different signatures). The precompiled config still
counts, by content hash, minus its `(nimcache …)` line.
* `.s.deps` seeding is speculative and runs before the prune, so a sidecar entry
that has gone stale (an import that a `when` no longer takes) can be dropped
instead of lingering forever; a pruned module's scan artifacts are deleted so
an edit-accumulated cache still matches a clean one.
* A failed nifmake run no longer prints an `Error:` of its own. The children have
already reported; adding a build-system status as the LAST error hid the
compiler's real message from anything reading the final error — every
reject-style test under `nim ic` said "nifmake failed with exit code: 1".
* `--mm:hooks` fed the mm mode to an on/off switch and failed outright with
"'on' or 'off' expected, but 'hooks' found". Pre-existing and unrelated to IC;
only reachable through the explicit switch, since `--newruntime` sets
`selectedGC` directly.
Running `tests/destructor` under `--ic` currently leaves 10 failures. They are
genuine IC defects, not harness noise (all 97 pass under `nim c`) — the clearest
is `tglobaldestructor`: `graph.globalDestructors` is accumulated while injecting
destructors into a module's top level, but the main module's `cg` — which emits
the teardown — is a different process, so a module-level `var` with a `=destroy`
is never destroyed. Same shape as the init/datInit metas, and it wants the same
fix: record it in the `.c.nif` head.
Validation: `koch bootic` reaches its byte-identical fixed point; `tests/ic` is
36/36; arc, destructor, macros, template, iter, closure, ccg, codegen, types and
effects pass under `nim c`, with generics showing only its pre-existing failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fixes#26015
Fixes imported global variables with codegenDecl being emitted as
definitions instead of extern declarations.
A variable’s codegenDecl format should customize its definition in the
owning module. Other modules referencing the variable must emit a normal
declaration:
```c
extern NI variable;
```
After the variable-declaration builder refactor, genVarPrototype passed
Extern visibility to addVar. However, the sfCodegenDecl branch returned
before applying that visibility. This caused importing modules to emit
another tentative definition, resulting in duplicate-symbol linker
errors.
The fix restores the previous distinction between the custom definition
and cross-module prototypes. It also adds C and C++ regression coverage
for both direct access and access through an inline procedure.
follows up https://github.com/nim-lang/Nim/pull/24423
fixes#26123
`cast[T](x)` is a transparent path expression for compiler analysis.
Previously, move/alias analysis could fail to see a later use through a
cast and incorrectly mark the source as moved, causing the issue’s
segmentation fault.
for views,
https://nim-lang.org/docs/manual_experimental.html#view-types-path-expressions:
A cast expression cast[T](e) is a path expression.
It also affects skipConvDfa, isAnalysableFieldAccess, and aliases. And I
might narrow it down for the two cases above mentioned if it causes
problems
Issue popped up when using `fromJson` into an array but the JSON passed
is an object
```nim
import std/[jsonutils, json]
let data = parseJson """
{"key": "value"}
"""
var foo: seq[int]
foo.fromJson(data)
echo foo #> @[0]
```
Basically the `setLen` would set the size to be equal to the number of
keys, but `getElems` just returns an empty array if the JSON isn't an
array which lead to it just creating zero'd items in the seq without
letting the user know.
Felt adding the checks was better than just skipping the `setLen` since
it lets the user know that there is a problem with the JSON
fixes#25992
```nim
type
Foo = object
case kind: bool
of true:
a: ref Bar # 8 bytes (pointer)
of false:
b: int # 4 bytes
```
specializeResetT for b emits accessor.b = 0 — writes 4 bytes
But the union is 8 bytes wide (sized by the largest branch)
The remaining 4 bytes where a used to live are untouched
Those stale bytes could contain a heap pointer the GC traces → crash
Add nimZeroMem after specializeResetN for case objects to clear the
entire union including unused branch bytes.
fixes#26088
in https://github.com/nim-lang/Nim/pull/25772, `beginStores` requires
`newLen` to be passed for setting up the new length. So `streams.nim`
must make the string length exactly newLen.
…haracterization with--warning:ResultUsed:on
fixes#26062
> A return statement with no expression is shorthand for return result.
> ResultUsed: Warn about the usage of the built-in result variable.
> A procedure that does not have any return statement and does not use
the special result variable returns the value of its last expression.
`unapplyBlockContext` reset `inEnforcedGcSafe` and
`inEnforcedNoSideEffects` to false whenever a `{.cast(gcsafe).}` or
`{.cast(noSideEffect).}` block ended. When such a block is nested inside
another block of the same cast, the inner block's exit switched
enforcement back on for the rest of the enclosing block, so statements
lexically inside the outer cast were rejected.
The fix saves both flags in `PragmaBlockContext` when the block context
is created and restores the saved values on exit. That matches how every
other piece of block state there (`locked`, `exc`, `tags`, `forbids`) is
already handled; these two bools were the only ones reset to a constant
instead of restored.
No change for non-nested blocks: entering from the non-enforced state
saves false, so exit still clears the flag. A statement after the outer
block is still rejected as before.
Test covers nested `cast(gcsafe)` and nested `cast(noSideEffect)`.
`tests/effects` passes unchanged (49/49, same as stock).
`nimDecRefIsLast` always performed an atomic decrement. When the biased
count is already zero the destroying thread holds the only reference, so
there is nothing to adjudicate and the read-modify-write can be skipped.
Soundness: a counted reference can only be derived from the location
being destroyed -- which happens-before this destructor unless the
program races on that location -- or from another counted reference,
whose contribution is already in `rc` and therefore forces the slow
path. Observing zero proves no other thread holds a reference and that
none can appear. This relies on `--mm:atomicArc` having no collector;
ORC and YRC mutate `rc` from a participant that holds no counted
reference at all, so the fast path is deliberately not enabled for them.
The slow path keeps deciding on the value its own RMW returned. That is
what separates this from nim-lang/threading#45, where the "who frees"
role was decided from a separate load and the RMW result was discarded,
so the role could be dropped by every participant at once.
gcbench, -d:danger, median of 21 pinned runs:
--mm:arc (non-atomic RC) 0.1310
--mm:atomicArc 0.1742
--mm:atomicArc + this 0.1330
-23.7%, closing 95% of the gap to non-atomic reference counting. gcbench
builds its trees with `sink` parameters, so it performs almost no
incRefs and the whole atomicArc penalty is decRef traffic. The worst
case -- a decrement that always sees rc > 0, so the load never pays off
-- measures +1.1%.
`-d:nimNoAtomicArcFastPath` restores the previous code path.
`strbasics.add` uses `copymem` when available. CT conditional
scaffolding mirrors the same from system.
Follow-up to #15951
Compile-time test for `strbasics.add` undiscarded and passes, though
pending bug #15952 is still open.
fixes#25942fixes#25938
After a successful match to a concrete static T, normalizes an empty
static container literal to the formal payload type before binding it.
This prevents `static[set[empty]]({})` from leaking into the
instantiated proc body.
fixes#26045;
fixes#26046
The fix isolates compiler option state while semantically checking each
when nimvm branch.
compiler/semexprs.nim:2745 snapshots the option stack, compiler options,
diagnostics settings, and enabled features. It analyzes one branch and
restores that state in finally. Both the nimvm and else branches use
this function.
This prevents:
```nim
when nimvm:
{.push overflowChecks: off.}
```
from disabling overflow checks in following runtime code. It also means
a {.pop.} in the opposite branch correctly reports that it has no
corresponding {.push.}.
Fixes#26027.
Map the single-order compare-exchange failure ordering from `release` to
`relaxed` and from `acquire-release` to `acquire`. Apply the mapping to
the trivial and non-trivial strong and weak overloads, and correct the
explicit-order test cases.
Tested `tests/stdlib/concurrency/tatomics.nim` across C/C++, refc/orc,
and native/C++ atomics (8 combinations). Also verified the original GCC
16.1 assertion reproducer.
`distinctBase` results in typedesc, so `set[T.distinctBase]` received
`typedesc[range[...]]` as its element type, which `isOrdinalType`
rejects. Strip the wrapper in `semSet` before storing the element type
and checking ordinality.
Also add `tyFromExpr` to the deferred-check set so the error doesn't
fire prematurely inside generic bodies - same pattern already used by
`semArray`.
`toStrLit()` uses `repr()` internally, which forwards quotes and messes
with dashes in the output. Let's use `newStrLitNode()` directly instead.
GitHub: fixes https://github.com/nim-lang/Nim/issues/26039
fixes#26010
Cursors do not own their values and therefore cannot transfer ownership
through move.
Reject move(cursor) during semantic analysis and share the
cursor-location check
between semantic analysis and destructor injection.
Fixes#23615
## Root cause
The leak does not require async at all -- this minimal closure iterator
leaks the exception and its stacktrace seq under ARC/ORC:
```nim
iterator it(): int {.closure.} =
try:
yield 1 # try spanning a yield => closureiters transform
raise newException(ValueError, "x")
except ValueError: # typed except => generated `of` check
discard
yield 2
```
A bare `except:` does not leak; a *typed* `except` does:
1. `collectExceptState` in `compiler/closureiters.nim` generates the
except-branch type check as `of(getCurrentException(), T)`, using the
raw generic magic sym from `getSysMagic("of", mOf)`.
2. `injectdestructors` skips call arguments whose *formal* parameter
type is `isCompileTimeOnly`, and the raw generic `of` sym's formal
params are `tyGenericParam` so both arguments of the generated `of` call
are never processed.
3. `getCurrentException()` increfs `currException` via `=copy` into its
result. Since the arc pass never wraps that owned temp in a destroy
(`--expandArc` shows the condition left untouched, while a user-written
`if f() of ValueError` in the same iterator gets a `:tmpD` +
`=destroy`), the caught exception's refcount stays +1 forever.
Every `try: await x() except SomeError` in async code has this shape, so
each caught async exception leaked once.
## Fix
The state-machine wrapper already stores the active exception in the
`:curExc` env field before jumping to the except landing state, and
`currException == :curExc` on every path into that state. The generated
condition now references the env field via `ctx.newCurExcAccess()`
instead of calling `getCurrentException()` again -- no ownership
transfer, no temp to destroy, one fewer runtime call.
Note: the underlying `injectdestructors` behavior (skipping args of
calls whose formal params are raw `tyGenericParam`, e.g. from
`getSysMagic`) is a separate latent gap that could affect other
compiler-generated code; it is intentionally left untouched here.
## Valgrind, before and after
Exact code and command from the issue, on Linux (Valgrind 3.19):
```
nim c -d:danger --mm:orc --debugger:native --threads:off -d:useMalloc bug.nim
valgrind --leak-check=full --show-leak-kinds=all ./bug
```
Before (devel):
```
==14663== HEAP SUMMARY:
==14663== in use at exit: 136 bytes in 2 blocks
==14663== total heap usage: 22 allocs, 20 frees, 116,466 bytes allocated
==14663==
==14663== 56 bytes in 1 blocks are indirectly lost in loss record 1 of 2
==14663== at 0x488A1C4: realloc (vg_replace_malloc.c:1437)
==14663== by 0x10C663: prepareSeqAddUninit (seqs_v2.nim:212)
==14663== by 0x10CF6F: raiseExceptionEx (excpt.nim:538)
==14663== by 0x11661F: amain::amainX20X28AsyncX29_(Future<void>) (bug.nim:10)
==14663== ...
==14663==
==14663== 136 (80 direct, 56 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 2
==14663== at 0x48850C8: malloc (vg_replace_malloc.c:381)
==14663== by 0x10C8E3: nimNewObj (arc.nim:122)
==14663== by 0x115403: err::errX20X28AsyncX29_(Future<void>) (asyncmacro.nim:274)
==14663== by 0x116027: err::errNimAsyncContinue(Future<void>, ClosureIt<void>) (asyncmacro.nim:44)
==14663== by 0x1163D3: bug::err (bug.nim:3)
==14663== ...
==14663==
==14663== LEAK SUMMARY:
==14663== definitely lost: 80 bytes in 1 blocks
==14663== indirectly lost: 56 bytes in 1 blocks
==14663== possibly lost: 0 bytes in 0 blocks
==14663== still reachable: 0 bytes in 0 blocks
==14663== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
```
After (this PR):
```
==14675== HEAP SUMMARY:
==14675== in use at exit: 0 bytes in 0 blocks
==14675== total heap usage: 22 allocs, 22 frees, 116,466 bytes allocated
==14675==
==14675== All heap blocks were freed -- no leaks are possible
==14675==
==14675== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
```
## Testing
- New `tests/async/t23615.nim` (modeled on `t23212.nim`: `valgrind:
true` + alloc-stats assertion) covers both the pure closure-iterator
form and the async form from the issue, with the caught exception looped
50x so the leak blows well past the slack threshold. It passes with this
PR and fails against devel.
- Testament categories `async`, `arc`, `iter`, `exception` all pass with
the patched compiler (323 tests).
- Behavior is unchanged on a sanity program covering multi-branch
dispatch, `as e` binding, nested try, and re-raise across yields: output
is byte-identical to devel; the patched build just frees 2 more blocks
per caught exception.
## Summary
Adds `--genBif:on|off`, allowing regular compiler builds to generate
per-module semantic BIF artifacts in `nimcache`.
This reuses the semantic artifact format produced by incremental
compilation without enabling IC or changing the normal code-generation
and linking pipeline.
In comparison to `nim check --compress ...` this new flag `nim c
--genBif:on --compileOnly yourlib.nim` is considerably more useful for
tooling.
That produced full semantic proc declarations, Nim visibility,
signatures, overload disambiguators, and pragmas. For a proc that was
actually code-generated, it also recorded the exact backend name, for
example.
## Motivation
External tools such as language servers, debuggers, and binding
generators can benefit from resolved symbol and type information
produced during an ordinary build. Previously, these semantic BIF
artifacts were tied to the incremental compiler workflow.
## Details
With the option enabled:
```sh
nim c --genBif:on project.nim
```
the compiler writes semantic `.s.bif` files and their supporting
sidecars for each semantically checked module while continuing with the
requested backend normally.
The option:
- Works with non-IC builds.
- Does not enable incremental compilation.
- Does not change generated program behavior.
- Does not enable or introduce native ABI exports.
- Does not generate `.abi.nif` manifests.
- Is ignored for NimScript compilation.
The `genBif` name follows existing artifact-generation options such as
`genScript`, `genMapping`, and `genCDeps`.
## Testing
Added a focused C backend test that runs a regular build with
`--genBif:on` and verifies that semantic `.s.bif` artifacts are
generated.
A release-mode temporary compiler build and the focused Testament test
both pass.
…pile time
fixes#26000
vm: preserve lvalues for mutations of broadcast array elements
Load in-place mutation targets through their address so mutations don't
operate on detached copies of broadcast defaults. This also preserves
nested lvalues and evaluates indexed destinations only once.
Cover sequence, string, and set mutations through direct, nested, field,
enum-indexed, and range-indexed array elements.
