The rule deciding which integer identifies a backend-minted symbol —
content-derived `disamb` for a lifted hook (`setHookDisamb`), `itemId.item`
otherwise — was written out at three sites: `mangleProcNameExt` and
`ccgutils.makeUnique` for the C name, `ast2nif.toNifSymName` for the NIF name,
each carrying its own copy of the ten-line rationale.
They drifted, which is exactly cce17461d: `toNifSymName` lacked the hook
exception, so the loader overwrote a content-derived value and two unrelated
`=destroy` hooks collided on one C name (C accepted the mistyped call, C++
rejected it). `astdef.backendMintedDisamb` is now the single definition and all
three call it. `globalName` still reads `disamb` directly — correct for a loaded
symbol, and the round-trip invariant that makes it agree is now stated in the
shared function instead of left implicit.
Pure de-duplication, verified as such: all 219 generated `.c` files of a
219-module corpus and the linked binary are byte-identical to the previous
commit. Plus 16/16 metamorphic IC tests, the 17-file `koch ic` suite, 13/13
differential edit checks against `nim c`, and the debug and release `bootic`
fixed points.
Note for the record: this started as an attempt to replace the per-process
`_c<itemId.item>` counter with a content hash. Instrumenting both mangling
sites showed that branch is never taken — 0 of 166 backend-minted manglings on
the corpus, 0 of 257 on the compiler, all going through the content-derived
path — so rewriting the scheme would have shifted every backend-minted C name
and forced an `icFormatVersion` bump for no demonstrable benefit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Grinding a small figdraw-based program under `nim ic` and diffing its
output against the classic backend surfaced eight bugs, four of which
silently produced a wrong binary rather than an error.
Frontend / build graph (`deps.nim`):
* Dead `when`-guarded imports were compiled anyway. `when someStrdefine
== "x": import y` is `cvUnknown` to the scanner, which conservatively
keeps the edge — right for an edge, but it also gave `y` its own `nim m`
rule, so a build died on a package the user never installed because they
never selected that backend. Track which edges are speculative and drop
a speculative subtree that cannot compile; if the guard was in fact
live, the discovery fixpoint puts the node back with the honest `cannot
open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so
nothing re-fires and `nim ic` relinked a stale binary while `nim c`
reported `cannot open file`. Report an unresolvable import from a
non-speculatively reached module during the graph scan.
* Macro-generated imports were discovered once and then forgotten.
Discovery only ran after a failure and the graph is re-derived
statically every run, so on a warm build the discovered module had no
rules at all and editing it changed nothing. Seed the graph from the
`.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from
file mtimes and never looks at a rule's command line, so `-d:foo=bar` /
`--mm:` / `--threads:` regenerated the build file with the new switches
and re-fired zero rules. Reify the configuration as a file and make it
an input of every rule.
* Command-line switches never reached the children: they replay the
project's config files, never the driver's argv, so `nim ic --opt:speed`
produced a byte-identical debug binary (likewise `--panics`,
`--experimental`, `--passC`). Forward the driver's switches, minus the
ones that must differ per child.
Artifacts and codegen:
* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw
the rule as satisfied on the next run: `nim ic` then reported success
for a program that does not compile, and generated code from
error-bearing AST (or hit an internal error in `ccgexprs`). Never
persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives
on the module symbol, which `moduleFromNifFile` rebuilds from scratch,
so `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a
module-level `block: let h = openHandle()` never ran `=destroy`. Persist
the flag as a `(modflags)` record. `injectdestructors` also has to
tolerate the `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym
node is written as a bare NIF `SymUse` token, which has nowhere to put
node flags, so the frontend's move analysis never reached the backend:
EVERY first assignment to a destructor-bearing local compiled as
`=sink`, i.e. `=destroy` on still-zeroed memory followed by a copy, and
no read was ever a move. Wrap a sym use in `(nflags ...)` when it
carries persistent node flags.
In the JS backend `toOpenArray` used `slice` (a copy), so writes through
a `var openArray` parameter silently vanished.
This emits `subarray` (a live shared-buffer view) for homogeneous
numeric arrays, otherwise such parameters are passed as a `{base, off,
len}` view that always aliases the caller's storage. Sliced seq/array
args become `{base, off, len}`, whole values `{base, off:0, len}`,
re-slices rebase.
Un-skips the JS guard in tests/openarray/topenarray.nim
Fixes#15952.
A closure's environment type — and the `=destroy`/`=copy`/`=sink` the compiler
lifts for it — is minted by the BACKEND and exists in no module's semmed NIF.
Grinding a corpus of closure programs against `nim c` as the oracle turned up
three ways the per-module backend gets those wrong. `tests/closure` is now green
under `--ic:on`, and `tests/iter` goes from 13 failures to 9.
* **The env hook is emitted by nobody.** `emitsBodyInThisModule` walks up to the
outermost enclosing routine to pick the TU that emits a body. For the env
`=destroy` of a generic closure iterator defined in one module and instantiated
in another, that walk lands on the module of the ORIGINAL generic — a module
that never sees the instance — so the routine was emitted nowhere
(`undefined reference to eqdestroy__c485__…`). A backend-minted routine has no
owning module NIF at all: it is written into the `.t.bif` of every module that
references it, re-homed there with `@bk`. Every referencing TU emits it now and
the merge stage keeps one, like any other content-addressed definition.
* **The `:up` link is stored without an increment.** `env.:up = enclosingEnv` has
to go through `=copy` (with the cyclic incref) or the parent's refcount is one
too low, and at teardown the two envs' mutually recursive `=destroy`s each
believe they hold the last reference and recurse until the stack is gone — a
SIGSEGV after the program's own output has already been printed
(`tests/iter/tnestedclosures`, "Test 3"). Whether it becomes a `=copy` depends
on the up-field type's hooks existing at injection time. Whole-program cgen got
that for free: a LATER lifting pass creates them and it runs before any
routine's injection. The per-module backend injects a routine right after
lifting it (`lower`), long before the module's top level is transformed at all
(`cg`) — so both up-field assignment sites create the ops themselves.
* **Two backend hooks collide on one C name.** A backend-minted symbol is named
`_c<itemId.item>`, a PER-PROCESS counter — fine while "nifc lifts, emits and
compiles them in one run", which is what `mangleProcNameExt` assumed. But
`lower` mints the env hooks of nested routines and `cg` mints those of the
module's top level, and both land in the same translation unit: two unrelated
`=destroy`s became one C function. `mangleProcNameExt` already makes an
exception for hooks whose `disamb` is content-derived (`HookDisambBit`);
`toNifSymName` now mirrors it, so the content-derived value survives the round
trip instead of being overwritten by the loader.
`InstanceDisambBit`/`HookDisambBit` move to `astdef` — `ast2nif` names symbols by
them and cannot import `modulegraphs`.
Two new metamorphic tests, both of which fail on the previous compiler:
`tclosure_hooks` (generic closure iterator instantiated across modules) and
`tclosure_nested_iter` (closure iterator nested in a closure iterator).
`icFormatVersion` 37 -> 38 for the hook naming. `koch bootic` reaches its
byte-identical fixed point; `tests/closure`, `tests/destructor` (3 known
`--newruntime` failures), `tests/cpp` and the classic categories are unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`nim ic` was a command of its own, which made it the C backend only and cut it
off from everything the ordinary compile commands accept. `--ic:on` is a switch
on `nim c` / `nim cpp` / `nim objc` instead, so `-r`, `-d:release`,
`--exceptions:`, and a project-wide opt-in from `nim.cfg` / `config.nims` all
work. `nim ic` still resolves to the same driver; `koch bootic`, `koch ic` and
`testament --ic` now go through the switch.
The switch was already parsed into a `conf.ic` nobody read. It is now read in
`passCmd1` as well, because `nim.nim` has to decide whether this run is an IC
DRIVER before config loading (`ensureIcConfig` produces the precompiled config
the driver itself replays); when the switch comes from a config file instead,
`main.nim` produces it late.
**C++.** `tests/cpp` passes under `--ic:on`, matching its classic result. Four
fixes, three of them the shape of every "C++ needs the whole program" problem the
per-module backend has:
* The driver DECLARES each module's translation unit to nifmake without loading a
module, so it cannot ask `cgen.getCFile` — and it hardcoded `.nim.c`, so the
merge stage went looking for `.c.nif` next to the `.cpp.nif` the children had
written. `options.icCFileExt` mirrors the formula at backend granularity.
* C++ has no designated initializers, so the RTTI record is a bare variable that
`DatInit` fills field by field. A bare `TNimTypeV2 x;` is a tentative
definition — C's linker merges those, C++'s does not — so every TU that
demanded the type defined it ("multiple definition of NTIv2__…"). It now gets
the same extern-declaration + owned-`'d'`-definition split the C flavour has.
* `memberProcsPerType` / `initializersPerType` live only in the sem process, so
the backend emitted a struct WITHOUT its in-class member declarations and the
out-of-class definitions did not match ("no declaration matches
'void Doo::memberProc()'"). They are replayed from a new `(repcppmember …)`
log entry; `replayCppMember` re-derives the type from the routine's signature
exactly as `semCppMember` does, so no type key has to survive the round trip.
* Two follow-ons for members: `loc.snippet` is a CALL PATTERN (`#->salute(@)`),
and only `genMemberProcHeader` derives it — whole-program cgen got it for free
by generating the defining module first, but the per-module backend emits that
body in another process, leaving the caller with the mangled Nim name
(`loo->salute_u0__vireouyks1()`). And that pattern is not a linker name: every
`salute` member in every class mints the same one, so the merge stage handed
them all to one artifact and dropped the rest (undefined vtable at link).
Member definitions are keyed by their NIF name there instead.
`--run` is now dropped when re-invoking for the config artifact: the producer has
no output binary and `nim.nim`'s run step asserted on the empty `outFile`.
testament's `--ic` appends the switch rather than rewriting the compile verb, so
a test that overrides `cmd:` wholesale keeps its verb, and the C++ corpus is
covered too (it never was — the old rewrite only matched `nim c `).
`icFormatVersion` 36 -> 37 for the new log entry. `koch bootic` reaches its
byte-identical fixed point through the new entry point; `koch ic` passes;
`tests/ic`, `tests/destructor` (3 known `--newruntime` failures) and the classic
categories are unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`testament --ic` on `tests/destructor` went from 10 failures to 3; the three
left are all `--newruntime`, whose owned-ref RTTI destructor is still missing
(the object type's `=destroy` never reaches the `cg` that emits the type info).
* **global destructors are never run.** `graph.globalDestructors` is filled
while a module's top level goes through `injectDestructorCalls`, and
whole-program cgen empties the list into the main module's init proc — which
IS the program body, so the calls land at exit. Under `nim ic` every module's
`cg` is a separate process and main's only ever saw its own entries: a
module-level `var` with a `=destroy` in an imported module was simply never
destroyed. Each module now wraps its own list in a nullary exported proc and
announces the name in its `.c.nif` meta head (`CnifVersion` 4 -> 5); main's
`cg` reads the heads it already reads for init/datInit and calls them in
reverse dependency order. New test `tests/ic/tglobal_dtors.nim` pins the
order against the oracle.
* **`spawn` died with "system module needs: nimArgsPassingDone".** A module
loaded from a NIF is named by its mangled suffix, so `loadCompilerProc`'s
`module.name.s == "threadpool"` could never match. The backend loads the whole
program before codegen, so consult every loaded module's index instead; misses
are final and remembered (`getCompilerProc` doubles as a presence probe).
* **`new(x, finalizer)` died with "environment misses: x".** `semmagic`'s
finalizer-to-destructor wrapper copies the original's AST but only rewrites
the parameter, leaving `ast[namePos]` naming the ORIGINAL proc. ast2nif
re-derives a routine's serialized AST from `ast[namePos].sym.ast`, so the
wrapper serialized the original's body — whose parameter belongs to the
original — and lambda lifting saw it as a captured variable of another proc.
The copy now names itself, the invariant every other routine AST keeps.
* **a top-level `let (a, b) = f()` copied instead of moved.** A module's own
symbol is the owner of every top-level symbol and is written as a real `(sd)`,
so the loader minted a SECOND `skModule` PSym for it — and `sym.owner == owner`
is an identity test in `aliasanalysis.isAnalysableFieldAccess`, which made
every module-level location un-analysable. Hard error for a type with a
disabled `=copy`. Bind the NIF name to the one registered module symbol.
Backend only: doing it under `nim m` costs `times.toDateTimeByWeek` its
inferred `raises`.
* **a doubly linked list leaked its whole contents.** A field USE serializes as
a bare `SymUse` with nowhere to put symbol flags, so `trees.isCursor` said
"not a cursor" for every loaded field and `DoublyLinkedNode.prev` became a
COUNTED reference: every node held its predecessor alive and no refcount ever
hit zero. `{.cursor.}` now rides in the NIF name marker (`` `fc `` next to
`` `f ``), which def and use derive from the same `PSym`.
* **`--expandArc` came out shuffled.** `moduleSymbolStubs` iterated a `Table`,
i.e. hash order, so the `lower` stage transformed a module's routines in an
arbitrary order — not even stable between two compilers. Order by index
offset, which is source order.
Also: `testament`'s `generatedFile` did not include the matrix entry in the
nimcache key its caller uses, so every `ccodeCheck` test with a `matrix:`
reported `reCodeNotFound`.
`koch bootic` still reaches its fixed point; `tests/ic` and the classic
`tests/destructor` are unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every invariant `tests/ic` checked was IC-against-IC — clean == incremental, a
no-op edit changes nothing, a body edit moves no interface cookie. An IC that is
*consistently* wrong satisfies all of them, and that is exactly how two silent
miscompilations survived. `koch bootic` has the same blind spot: it proves the
compiler reproduces itself, not that it agrees with the reference backend.
Two mechanisms, at different scales.
**The oracle, in the metamorphic runner.** Every successful step is now also
compiled with `nim c` and run, and the two outputs must agree. Unlike the
hand-written `expect:` strings this needs no foresight from the test author: it
compares everything the program does, not only what someone thought to print,
which is what a silently-skipped destructor evades. `no-oracle` opts out.
The format also grew the expressiveness the recent bug hunt showed was missing —
every one of these described a state the suite could not reach:
* `#!DELETE <file>` removes a module. Deleting a still-imported file moves no
mtime, so nothing re-fires.
* `#!FLAGS <switches>` changes the compiler switches between steps. A config
change is not a file either.
* `fails: <substring>` asserts that BOTH compilers reject the program with that
text. Previously every step had to succeed, so the whole error path — and
recovery from it — was untested.
Six regression tests cover the eight bugs the last round fixed.
`testament r <file>` now dispatches metamorphic tests like `testament cat ic`.
**`testament --ic` runs the whole corpus through the incremental compiler**, so
IC inherits ~10k programs with expected output instead of 30 bespoke tests.
Two things had to change for that to mean anything:
* `nimcacheDir` now keys on the `matrix:` entry too. Two matrix variants of one
file are two different compilations; sharing a cache meant each run
invalidated what the previous left — harmless for a backend that caches only
object files, useless for an incremental one.
* About half the corpus overrides the command wholesale (`cmd: "nim c --gc:arc
$file"`), bypassing both `$target` and `$options`. Those are rewritten to `ic`
and given a private cache.
**Warm cache, hastur-style.** A generated warmup program pulling in `system` and
the most-imported stdlib modules is compiled once per distinct compile
configuration into `nimcache/ic_warmup_<hash>`, and each test's empty cache is
seeded from it with mtimes preserved (nifmake compares output-mtime >
input-mtime, so stamping the copies "now" re-fires the whole graph). Only
program-independent artifacts are copied: the frontend NIFs and cookies plus the
per-module `lower`/`cg` output. The `.c`/`.o` are left behind on purpose — the
merge decision is whole-program, so they are re-rendered for every program
anyway.
`tests/destructor` (97 runs): `nim c` 35s cold / 32s warm; `nim ic` ~3m30 cold /
**9.8s warm**.
Compiler changes this required or uncovered:
* `merge` read the live-module list from a manifest the driver writes instead of
globbing `*.c.nif` off the nimcache. Globbing absorbed artifacts belonging to
any other program sharing the directory — which is what made a prefilled cache
produce undefined symbols at link.
* The build-arg signature no longer includes `--icproject:`/`--icPreparsedConfig:`
(they name where a build lives, not what it produces, so two caches holding
identical artifacts got different signatures). The precompiled config still
counts, by content hash, minus its `(nimcache …)` line.
* `.s.deps` seeding is speculative and runs before the prune, so a sidecar entry
that has gone stale (an import that a `when` no longer takes) can be dropped
instead of lingering forever; a pruned module's scan artifacts are deleted so
an edit-accumulated cache still matches a clean one.
* A failed nifmake run no longer prints an `Error:` of its own. The children have
already reported; adding a build-system status as the LAST error hid the
compiler's real message from anything reading the final error — every
reject-style test under `nim ic` said "nifmake failed with exit code: 1".
* `--mm:hooks` fed the mm mode to an on/off switch and failed outright with
"'on' or 'off' expected, but 'hooks' found". Pre-existing and unrelated to IC;
only reachable through the explicit switch, since `--newruntime` sets
`selectedGC` directly.
Running `tests/destructor` under `--ic` currently leaves 10 failures. They are
genuine IC defects, not harness noise (all 97 pass under `nim c`) — the clearest
is `tglobaldestructor`: `graph.globalDestructors` is accumulated while injecting
destructors into a module's top level, but the main module's `cg` — which emits
the teardown — is a different process, so a module-level `var` with a `=destroy`
is never destroyed. Same shape as the init/datInit metas, and it wants the same
fix: record it in the `.c.nif` head.
Validation: `koch bootic` reaches its byte-identical fixed point; `tests/ic` is
36/36; arc, destructor, macros, template, iter, closure, ccg, codegen, types and
effects pass under `nim c`, with generics showing only its pre-existing failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Grinding a small figdraw-based program under `nim ic` and diffing its output
against the classic backend surfaced eight bugs, four of which silently produced
a wrong binary rather than an error.
Frontend / build graph (`deps.nim`):
* Dead `when`-guarded imports were compiled anyway. `when someStrdefine == "x":
import y` is `cvUnknown` to the scanner, which conservatively keeps the edge —
right for an edge, but it also gave `y` its own `nim m` rule, so a build died
on a package the user never installed because they never selected that backend.
Track which edges are speculative and drop a speculative subtree that cannot
compile; if the guard was in fact live, the discovery fixpoint puts the node
back with the honest `cannot open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so nothing
re-fires and `nim ic` relinked a stale binary while `nim c` reported `cannot
open file`. Report an unresolvable import from a non-speculatively reached
module during the graph scan.
* Macro-generated imports were discovered once and then forgotten. Discovery
only ran after a failure and the graph is re-derived statically every run, so
on a warm build the discovered module had no rules at all and editing it
changed nothing. Seed the graph from the `.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from file mtimes
and never looks at a rule's command line, so `-d:foo=bar` / `--mm:` / `--threads:`
regenerated the build file with the new switches and re-fired zero rules.
Reify the configuration as a file and make it an input of every rule.
* Command-line switches never reached the children: they replay the project's
config files, never the driver's argv, so `nim ic --opt:speed` produced a
byte-identical debug binary (likewise `--panics`, `--experimental`, `--passC`).
Forward the driver's switches, minus the ones that must differ per child.
Artifacts and codegen:
* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw the rule
as satisfied on the next run: `nim ic` then reported success for a program that
does not compile, and generated code from error-bearing AST (or hit an internal
error in `ccgexprs`). Never persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives on the
module symbol, which `moduleFromNifFile` rebuilds from scratch, so
`genTopLevelStmt` skipped `injectDestructorCalls` entirely: a module-level
`block: let h = openHandle()` never ran `=destroy`. Persist the flag as a
`(modflags)` record. `injectdestructors` also has to tolerate the
`nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym node is
written as a bare NIF `SymUse` token, which has nowhere to put node flags, so
the frontend's move analysis never reached the backend: EVERY first assignment
to a destructor-bearing local compiled as `=sink`, i.e. `=destroy` on
still-zeroed memory followed by a copy, and no read was ever a move. Wrap a sym
use in `(nflags ...)` when it carries persistent node flags.
`icFormatVersion` 34 -> 35 for the two new NIF records.
Validation: `koch bootic` reaches its byte-identical fixed point; two clean-cache
builds from the same compiler are identical; testament `arc`, `destructor`,
`macros`, `template`, `iter`, `closure`, `ccg`, `codegen`, `types` and `effects`
pass, and `generics`, `ic` and `stdlib` show exactly the pre-existing failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fixes#26015
Fixes imported global variables with codegenDecl being emitted as
definitions instead of extern declarations.
A variable’s codegenDecl format should customize its definition in the
owning module. Other modules referencing the variable must emit a normal
declaration:
```c
extern NI variable;
```
After the variable-declaration builder refactor, genVarPrototype passed
Extern visibility to addVar. However, the sfCodegenDecl branch returned
before applying that visibility. This caused importing modules to emit
another tentative definition, resulting in duplicate-symbol linker
errors.
The fix restores the previous distinction between the custom definition
and cross-module prototypes. It also adds C and C++ regression coverage
for both direct access and access through an inline procedure.
follows up https://github.com/nim-lang/Nim/pull/24423
fixes#26123
`cast[T](x)` is a transparent path expression for compiler analysis.
Previously, move/alias analysis could fail to see a later use through a
cast and incorrectly mark the source as moved, causing the issue’s
segmentation fault.
for views,
https://nim-lang.org/docs/manual_experimental.html#view-types-path-expressions:
A cast expression cast[T](e) is a path expression.
It also affects skipConvDfa, isAnalysableFieldAccess, and aliases. And I
might narrow it down for the two cases above mentioned if it causes
problems
fixes#25992
```nim
type
Foo = object
case kind: bool
of true:
a: ref Bar # 8 bytes (pointer)
of false:
b: int # 4 bytes
```
specializeResetT for b emits accessor.b = 0 — writes 4 bytes
But the union is 8 bytes wide (sized by the largest branch)
The remaining 4 bytes where a used to live are untouched
Those stale bytes could contain a heap pointer the GC traces → crash
Add nimZeroMem after specializeResetN for case objects to clear the
entire union including unused branch bytes.
We defer copying the AST during generic instantiation until a cache miss
and fetch from cache based on bindings. On miss, we fall back to the old
logic and populate the cache.
This gains us a roughly 36% reduction in memory usage during bootstrap,
from `799.242MiB` down to `508.672MiB` on my machine.
For another point of reference, nimbus-eth2 goes from `10.5GB` memory
usage to `7.5GB`.
Independent companion to #26090 which together with this one yields a
bit under 20% faster compiles (or at least `--compileOnly` bootstraps)
on ORC.
…haracterization with--warning:ResultUsed:on
fixes#26062
> A return statement with no expression is shorthand for return result.
> ResultUsed: Warn about the usage of the built-in result variable.
> A procedure that does not have any return statement and does not use
the special result variable returns the value of its last expression.
`unapplyBlockContext` reset `inEnforcedGcSafe` and
`inEnforcedNoSideEffects` to false whenever a `{.cast(gcsafe).}` or
`{.cast(noSideEffect).}` block ended. When such a block is nested inside
another block of the same cast, the inner block's exit switched
enforcement back on for the rest of the enclosing block, so statements
lexically inside the outer cast were rejected.
The fix saves both flags in `PragmaBlockContext` when the block context
is created and restores the saved values on exit. That matches how every
other piece of block state there (`locked`, `exc`, `tags`, `forbids`) is
already handled; these two bools were the only ones reset to a constant
instead of restored.
No change for non-nested blocks: entering from the non-enforced state
saves false, so exit still clears the flag. A statement after the outer
block is still rejected as before.
Test covers nested `cast(gcsafe)` and nested `cast(noSideEffect)`.
`tests/effects` passes unchanged (49/49, same as stock).
Boostrap without linking is (conservatively) 17% faster on my machine.
Test setup runs the compiler compiling itself under ORC in release mode
from a clean cache and no C compilation (`--compileOnly`.)
Multiple samples are gathered before judging a potential optimization.
The test suite passes locally before push and bootstrap is tested with
strict views (mostly as a sanity check).
Tests are fair in the sense that they compile the same git worktree and
produce identical C output.
Refc will see less or no benefit.
Changes:
- Save tree traversals in `considerGenSyms` when no mappings exist
- Inline `maybeSkipDistinct` into `typeRel` for a safe cursor
- Only skip to static when there is a static type to reach in
`paramTypesMatchAux`
- Disable overflow checks for hashes
- Disable bounds checks for `nextIdentIter`
- Use cursor annotation when judged safe
- Use lent annotation for ast accessors
Spiritual companion to #26084
## Summary
- nimpretty with non-default \--indent\ (e.g. 3 or 10) produced invalid
indentation in if/block/try expression regions because layouter kept the
original column when \keepIndents > 0\ and ignored \indWidth\.
- Rebase the column onto \indWidth\ using the relative offset from the
enclosing block baseline (\indentStack[^1]\).
## Root cause
\parser.nim\'s \
imprettyDontTouch\ template sets \keepIndents\ for if/block/try
expressions. layouter in the \keepIndents > 0\ branch used \ ok.indent\
(source column) directly as \indentLevel\ without scaling by \indWidth\,
so lines in these regions kept the original column and misaligned with
the rest of the file when \--indent\ differed from source indent width.
## Fix
\\\
im
em.indentLevel = em.indentStack.high * em.indWidth +
(tok.indent - em.indentStack[^1])
\\\
Keeps the relative offset from the enclosing block baseline but rebases
onto \indWidth\. At default \--indent:2\ the offset equals \indWidth\,
so output is unchanged (backwards compatible).
## Testing
- 12 custom cases x 3 indent values (2/3/10) = 36/36 pass
- nimpretty self-test suite 7/7 pass (no regression at default indent)
- 5 keepIndents scenarios (if/block/try expression continuation
alignment) that failed at indent:3/10 now pass
Fixes#20078.
fixes#25942fixes#25938
After a successful match to a concrete static T, normalizes an empty
static container literal to the formal payload type before binding it.
This prevents `static[set[empty]]({})` from leaking into the
instantiated proc body.
fixes#26045;
fixes#26046
The fix isolates compiler option state while semantically checking each
when nimvm branch.
compiler/semexprs.nim:2745 snapshots the option stack, compiler options,
diagnostics settings, and enabled features. It analyzes one branch and
restores that state in finally. Both the nimvm and else branches use
this function.
This prevents:
```nim
when nimvm:
{.push overflowChecks: off.}
```
from disabling overflow checks in following runtime code. It also means
a {.pop.} in the opposite branch correctly reports that it has no
corresponding {.push.}.
`distinctBase` results in typedesc, so `set[T.distinctBase]` received
`typedesc[range[...]]` as its element type, which `isOrdinalType`
rejects. Strip the wrapper in `semSet` before storing the element type
and checking ordinality.
Also add `tyFromExpr` to the deferred-check set so the error doesn't
fire prematurely inside generic bodies - same pattern already used by
`semArray`.
fixes#26010
Cursors do not own their values and therefore cannot transfer ownership
through move.
Reject move(cursor) during semantic analysis and share the
cursor-location check
between semantic analysis and destructor injection.
Fixes#23615
## Root cause
The leak does not require async at all -- this minimal closure iterator
leaks the exception and its stacktrace seq under ARC/ORC:
```nim
iterator it(): int {.closure.} =
try:
yield 1 # try spanning a yield => closureiters transform
raise newException(ValueError, "x")
except ValueError: # typed except => generated `of` check
discard
yield 2
```
A bare `except:` does not leak; a *typed* `except` does:
1. `collectExceptState` in `compiler/closureiters.nim` generates the
except-branch type check as `of(getCurrentException(), T)`, using the
raw generic magic sym from `getSysMagic("of", mOf)`.
2. `injectdestructors` skips call arguments whose *formal* parameter
type is `isCompileTimeOnly`, and the raw generic `of` sym's formal
params are `tyGenericParam` so both arguments of the generated `of` call
are never processed.
3. `getCurrentException()` increfs `currException` via `=copy` into its
result. Since the arc pass never wraps that owned temp in a destroy
(`--expandArc` shows the condition left untouched, while a user-written
`if f() of ValueError` in the same iterator gets a `:tmpD` +
`=destroy`), the caught exception's refcount stays +1 forever.
Every `try: await x() except SomeError` in async code has this shape, so
each caught async exception leaked once.
## Fix
The state-machine wrapper already stores the active exception in the
`:curExc` env field before jumping to the except landing state, and
`currException == :curExc` on every path into that state. The generated
condition now references the env field via `ctx.newCurExcAccess()`
instead of calling `getCurrentException()` again -- no ownership
transfer, no temp to destroy, one fewer runtime call.
Note: the underlying `injectdestructors` behavior (skipping args of
calls whose formal params are raw `tyGenericParam`, e.g. from
`getSysMagic`) is a separate latent gap that could affect other
compiler-generated code; it is intentionally left untouched here.
## Valgrind, before and after
Exact code and command from the issue, on Linux (Valgrind 3.19):
```
nim c -d:danger --mm:orc --debugger:native --threads:off -d:useMalloc bug.nim
valgrind --leak-check=full --show-leak-kinds=all ./bug
```
Before (devel):
```
==14663== HEAP SUMMARY:
==14663== in use at exit: 136 bytes in 2 blocks
==14663== total heap usage: 22 allocs, 20 frees, 116,466 bytes allocated
==14663==
==14663== 56 bytes in 1 blocks are indirectly lost in loss record 1 of 2
==14663== at 0x488A1C4: realloc (vg_replace_malloc.c:1437)
==14663== by 0x10C663: prepareSeqAddUninit (seqs_v2.nim:212)
==14663== by 0x10CF6F: raiseExceptionEx (excpt.nim:538)
==14663== by 0x11661F: amain::amainX20X28AsyncX29_(Future<void>) (bug.nim:10)
==14663== ...
==14663==
==14663== 136 (80 direct, 56 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 2
==14663== at 0x48850C8: malloc (vg_replace_malloc.c:381)
==14663== by 0x10C8E3: nimNewObj (arc.nim:122)
==14663== by 0x115403: err::errX20X28AsyncX29_(Future<void>) (asyncmacro.nim:274)
==14663== by 0x116027: err::errNimAsyncContinue(Future<void>, ClosureIt<void>) (asyncmacro.nim:44)
==14663== by 0x1163D3: bug::err (bug.nim:3)
==14663== ...
==14663==
==14663== LEAK SUMMARY:
==14663== definitely lost: 80 bytes in 1 blocks
==14663== indirectly lost: 56 bytes in 1 blocks
==14663== possibly lost: 0 bytes in 0 blocks
==14663== still reachable: 0 bytes in 0 blocks
==14663== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
```
After (this PR):
```
==14675== HEAP SUMMARY:
==14675== in use at exit: 0 bytes in 0 blocks
==14675== total heap usage: 22 allocs, 22 frees, 116,466 bytes allocated
==14675==
==14675== All heap blocks were freed -- no leaks are possible
==14675==
==14675== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
```
## Testing
- New `tests/async/t23615.nim` (modeled on `t23212.nim`: `valgrind:
true` + alloc-stats assertion) covers both the pure closure-iterator
form and the async form from the issue, with the caught exception looped
50x so the leak blows well past the slack threshold. It passes with this
PR and fails against devel.
- Testament categories `async`, `arc`, `iter`, `exception` all pass with
the patched compiler (323 tests).
- Behavior is unchanged on a sanity program covering multi-branch
dispatch, `as e` binding, nested try, and re-raise across yields: output
is byte-identical to devel; the patched build just frees 2 more blocks
per caught exception.
## Summary
Adds `--genBif:on|off`, allowing regular compiler builds to generate
per-module semantic BIF artifacts in `nimcache`.
This reuses the semantic artifact format produced by incremental
compilation without enabling IC or changing the normal code-generation
and linking pipeline.
In comparison to `nim check --compress ...` this new flag `nim c
--genBif:on --compileOnly yourlib.nim` is considerably more useful for
tooling.
That produced full semantic proc declarations, Nim visibility,
signatures, overload disambiguators, and pragmas. For a proc that was
actually code-generated, it also recorded the exact backend name, for
example.
## Motivation
External tools such as language servers, debuggers, and binding
generators can benefit from resolved symbol and type information
produced during an ordinary build. Previously, these semantic BIF
artifacts were tied to the incremental compiler workflow.
## Details
With the option enabled:
```sh
nim c --genBif:on project.nim
```
the compiler writes semantic `.s.bif` files and their supporting
sidecars for each semantically checked module while continuing with the
requested backend normally.
The option:
- Works with non-IC builds.
- Does not enable incremental compilation.
- Does not change generated program behavior.
- Does not enable or introduce native ABI exports.
- Does not generate `.abi.nif` manifests.
- Is ignored for NimScript compilation.
The `genBif` name follows existing artifact-generation options such as
`genScript`, `genMapping`, and `genCDeps`.
## Testing
Added a focused C backend test that runs a regular build with
`--genBif:on` and verifies that semantic `.s.bif` artifacts are
generated.
A release-mode temporary compiler build and the focused Testament test
both pass.
…pile time
fixes#26000
vm: preserve lvalues for mutations of broadcast array elements
Load in-place mutation targets through their address so mutations don't
operate on detached copies of broadcast defaults. This also preserves
nested lvalues and evaluates indexed destinations only once.
Cover sequence, string, and set mutations through direct, nested, field,
enum-indexed, and range-indexed array elements.
fix#25976
Initialize tagEffects for proc types that declare .forbids but omit
.tags,
so they behave like explicit tags: [] during indirect-call effect
tracking.
Add a regression for the nested callback assignment case.
fixes#25886fixes#25883
See #25883
Tuples only hash leaves so if 2 tuples have the same flattened
representation they collide in the C codegen.
Fix by hashing the length as well to disambiguate nesting levels.
---------
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
