Compare commits

..

14 Commits

Author SHA1 Message Date
araq
8d4ddd5516 IC: introduce BNode, the seam for running codegen off a .bif Cursor
Building `PNode` trees out of a module's `.bif` is the bulk of the `lower` and
`cg` stages: their cost tracks the size of the dependency CLOSURE a stage loads,
not the module it compiles (a 370-byte module costs 0.20s/0.16s in lower/cg, the
main module 3.40s/3.16s). Since the link stage stopped loading the graph those
two are ~85% of the serial backend critical path. The goal is to let the backend
read the `.bif` directly through a `Cursor`, constructing no `PNode` at all.

`BNode` is the seam: a `Cursor` with `-d:newIcBackend`, a `PNode` without, which
is what every build does today. Codegen migrates one area at a time and the
compiler keeps building throughout.

The seam is thin on purpose. `ast`/`astdef` already give `PNode` the whole
vocabulary — `kind`, `len`, `safeLen`, `sym`, `typ`, `info`, `firstSon`,
`secondSon`, `lastSon` and `sons`/`isons`/`sonsFrom` — so bnode deliberately does
NOT redefine them for `PNode`: an identical second overload makes every call site
ambiguous (tried it; `lastSon` breaks first). It adds only `son` and `hasSons`,
and supplies the full vocabulary on the `Cursor` side as `{.error.}` stubs, so
flipping the define names the exact missing accessor AT ITS CALL SITE instead of
collapsing into a cascade of type errors.

Two notes for whoever continues this:

* The migration front is SIGNATURES, not call sites. Converting `n[i]` to the
  iterator vocabulary changes nothing while the value is still `PNode`-typed —
  flipping the define then reports a plain type mismatch. Changing one proc's
  parameter from `PNode` to `BNode` is what moves it, and is a no-op with the
  define off. `containsResult` and `allPathsAsgnResult` are converted as the
  worked example.
* `BNode`'s cost model differs and the vocabulary is shaped around it: reading
  child `i` of a `Cursor` is O(size of children 0..<i), so prefer `sons`/
  `sonsFrom`/`isons` over indexing, and avoid `len` in a loop condition.

Note `type BNode = when defined(x): Cursor else: PNode` does not parse — `when`
is not an expression in type position; it needs a `when` block over two `type`
sections.

Inert: all 219 generated `.c` files and the linked binary are byte-identical to
the parent commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 05:44:39 +02:00
araq
f84f53bff4 cgen: replace indexed child loops with the sons/isons/sonsFrom iterators
`for i in k..<n.len: ... n[i] ...` is the dominant shape for walking a `PNode`'s
children in the code generator: 41 such loops across the cgen files, and 777
indexed node accesses in total. It reads worse than iterating, it bounds-checks
every subscript, and it is quadratic the moment the backend reads children off a
NIF `Cursor` rather than a materialised tree (a child is `firstSon` plus one
`skip` per preceding sibling, and `skip` steps over a whole subtree).

31 of the 41 are converted:

* 20 to `sons`/`sonsFrom` — the index only ever subscripted `n`.
* 9 to `isons`, which now takes a `start` index (defaulting to 0, so its eight
  existing call sites are unchanged). These genuinely need `i`: a parallel index
  into the routine's `PType` (`typ.n[i]`, `typ[i]`), a `needTmp[i-1]` lookup, an
  `i == field.position` test, `$i` in a generated struct name, or the index
  passed straight to `genOtherArg`.
* 2 to `sonsFrom` with a variable start (`firstParam`, `offset`).

`sonsFrom` is new, next to `sons`/`isons` in astdef.

The remaining 10 are deliberate. Eight are not `PNode` at all — `varargs[Snippet]`,
`seq[PSym]`, `string`, and `PType`, where `sons` is a `proc ...: var TTypeSeq`
rather than an iterator, so a blind rewrite would compile into something quite
different. Two iterate `0..<it.len-1`, excluding the last child, which no
iterator expresses cleanly.

Pure refactor, and verified as one: all 219 generated `.c` files of a 219-module
program and the linked binary are byte-identical to the parent commit. That is
the bar that matters here, because the index arithmetic (`i-1`, `i == position`,
`$i`) is the easy thing to get wrong. It also caught a real slip on the way:
`genFieldCheck` reassigns its loop variable, which a `for` binding cannot do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 05:44:14 +02:00
araq
e0c0724b62 IC: the link stage no longer loads the module graph
`--icBackendStage:link` called `loadBackendModules` — a whole-program
deserialization — for exactly two things: each module's `.c` path via
`getCFile`, and its recorded C compile/link directives via
`replayBackendActions`. That was 3.7s of the ~11s serial backend critical path
on a 219-module program, spent recovering a list of paths and a handful of
strings.

Both now come from artifacts the earlier stages already produce:

* `.c` paths from the driver's existing `LiveModulesFile` manifest (which
  `merge` already reads); the `.c` sits beside each listed `.c.nif`, so no new
  manifest was needed. The merge-decision fallback for conditionally-imported
  nodes that own a live symbol is kept.
* C directives from a new `<module>.c.cflags` sidecar. The module's own `cg`
  already replays them, so it writes them down too — one tab-separated line per
  `compile`/`link`/`passl`/`passc`/`localpassc`/`cppdefine` action. `localpassc`
  needs the module's source path, which only the writer can resolve, so it is
  baked into the line. Written unconditionally, empty included: it is a declared
  nifmake output of the `cg` rule and a missing output re-fires the rule for
  ever.

link: 3.70s -> 0.44s. A one-line code edit on the corpus goes 15.2s -> 11.8s;
cold 124s -> 120s; no-op unchanged at 0.27s.

Green: 16/16 metamorphic IC tests, the 17-file `koch ic` suite, 13/13
differential edit checks against `nim c`, and `bootic` iteration 1 (the full
fixed-point check was still running when this was committed).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 23:13:16 +02:00
araq
d81e764f98 IC: the cg stage must not write emit's .c
`registerModuleCode` writes the module's `.c` and registers it for compilation.
Under `--icBackendStage:cg` it must do neither: `cg`'s product is the `.c.nif`,
the `.c` is rendered by `emit` from that artifact plus the GLOBAL merge
decision, and the compile registration belongs to `link`.

Writing it in `cg` puts a second, differently-filtered `.c` — `cg` can only
filter by the liveness its own process sees — at the exact path `emit` declares
as its nifmake output. Two stages claim one output, and the `.c` ends up newer
than `emit`'s own `.c.nif` input, so nifmake considers `emit` up to date and
skips it: `cg`'s unfiltered text goes to the linker and every emit-everywhere
body is defined twice ("multiple definition of eqdup__u727466987__OOZ…").

Nothing surfaces this today because `merge` rewrites the decision file on every
run and every `emit` lists it as an input, so all of them re-fire and overwrite
the stray file. The fire-all is therefore load-bearing, not the "insurance" it
is documented as — and it is what makes a content-stable merge decision
(219 fewer processes and ~13s less backend CPU per edit on a 219-module
program) impossible: that experiment is exactly how this was found.

No behaviour change on its own, since the fire-all still runs. Green: `bootic`
fixed point, 16/16 metamorphic IC tests, the 17-file `koch ic` suite, and 13/13
differential edit checks against `nim c`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 22:38:14 +02:00
araq
3c53629164 IC: don't build interface tables for dep-of-a-dep module loads
`loadTransitiveHooks` loads a module only to register its hooks, macro-cache
replay and generic-instance offers. It is a dep-of-a-dep, not an import, so
none of its symbols are visible to the module being semchecked — yet it went
through the full `loadNifModule`, which builds the interface string tables by
calling `loadSymFromIndexEntry` on every index entry, and `processTopLevel`,
whose `export` branch calls `resolveSym` on every exported symbol. Both write
into an `interf`/`interfHidden` pair that is scratch, shared across all
iterations, and never read. Their only other effect is warming the name-keyed
`c.syms` cache, which `resolveSym` refills lazily from the same index on a miss.

New `LoadFlag.SkipInterfaceTables`, set only by `loadTransitiveHooks`, gates
both. Measured on a 219-module program: a one-line edit rebuild goes 5.58s ->
3.21s (-42%), and the compiler's own cold IC build 202s -> 164s (-19%), since
every `nim m` in a cold build pays this too. Profile of the `nim m` before:
direct imports 17 modules/1.40s, transitive hooks 209 modules/2.99s,
re-exports 39/0.23s.

This is NOT byte-inert, unlike the previous commit: skipping the eager stub
creation shifts the per-module load-order counter, so lazily-created stubs get
different serialized item ids and 44 of 238 `.s.bif` change. The artifacts stay
deterministic (238/238 reproducible across two independent cold builds) and
16/16 metamorphic tests pass, including `clean cache == incremental cache` and
`a no-op edit changes no artifact` — so an existing nimcache re-sems those
modules once and then converges. Also green: the 17-file `koch ic` suite, 13/13
differential edit checks against `nim c`, both `bootic` fixed points (debug and
release), and 482 tests across tests/arc, destructor, closure, iter, gc, async
with a failure set identical to the parent commit's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 20:03:42 +02:00
araq
1f504865ed IC: one definition of a backend-minted symbol's disambiguator
The rule deciding which integer identifies a backend-minted symbol —
content-derived `disamb` for a lifted hook (`setHookDisamb`), `itemId.item`
otherwise — was written out at three sites: `mangleProcNameExt` and
`ccgutils.makeUnique` for the C name, `ast2nif.toNifSymName` for the NIF name,
each carrying its own copy of the ten-line rationale.

They drifted, which is exactly cce17461d: `toNifSymName` lacked the hook
exception, so the loader overwrote a content-derived value and two unrelated
`=destroy` hooks collided on one C name (C accepted the mistyped call, C++
rejected it). `astdef.backendMintedDisamb` is now the single definition and all
three call it. `globalName` still reads `disamb` directly — correct for a loaded
symbol, and the round-trip invariant that makes it agree is now stated in the
shared function instead of left implicit.

Pure de-duplication, verified as such: all 219 generated `.c` files of a
219-module corpus and the linked binary are byte-identical to the previous
commit. Plus 16/16 metamorphic IC tests, the 17-file `koch ic` suite, 13/13
differential edit checks against `nim c`, and the debug and release `bootic`
fixed points.

Note for the record: this started as an attempt to replace the per-process
`_c<itemId.item>` counter with a content hash. Instrumenting both mangling
sites showed that branch is never taken — 0 of 166 backend-minted manglings on
the corpus, 0 of 257 on the compiler, all going through the content-derived
path — so rewriting the scheme would have shifted every backend-minted C name
and forced an `icFormatVersion` bump for no demonstrable benefit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 20:03:14 +02:00
araq
0c0cc1e496 remove owner-checking-tower 2026-08-28 15:12:56 +02:00
araq
1387093f99 don't redo nifler steps unnecessarily 2026-08-28 14:28:30 +02:00
Andreas Rumpf
546a518b22 Merge branch 'devel' into araq-ic-fixes 2026-08-28 11:10:25 +02:00
araq
cce17461de IC: three closure/environment fixes found by differential grinding
A closure's environment type — and the `=destroy`/`=copy`/`=sink` the compiler
lifts for it — is minted by the BACKEND and exists in no module's semmed NIF.
Grinding a corpus of closure programs against `nim c` as the oracle turned up
three ways the per-module backend gets those wrong. `tests/closure` is now green
under `--ic:on`, and `tests/iter` goes from 13 failures to 9.

* **The env hook is emitted by nobody.** `emitsBodyInThisModule` walks up to the
  outermost enclosing routine to pick the TU that emits a body. For the env
  `=destroy` of a generic closure iterator defined in one module and instantiated
  in another, that walk lands on the module of the ORIGINAL generic — a module
  that never sees the instance — so the routine was emitted nowhere
  (`undefined reference to eqdestroy__c485__…`). A backend-minted routine has no
  owning module NIF at all: it is written into the `.t.bif` of every module that
  references it, re-homed there with `@bk`. Every referencing TU emits it now and
  the merge stage keeps one, like any other content-addressed definition.

* **The `:up` link is stored without an increment.** `env.:up = enclosingEnv` has
  to go through `=copy` (with the cyclic incref) or the parent's refcount is one
  too low, and at teardown the two envs' mutually recursive `=destroy`s each
  believe they hold the last reference and recurse until the stack is gone — a
  SIGSEGV after the program's own output has already been printed
  (`tests/iter/tnestedclosures`, "Test 3"). Whether it becomes a `=copy` depends
  on the up-field type's hooks existing at injection time. Whole-program cgen got
  that for free: a LATER lifting pass creates them and it runs before any
  routine's injection. The per-module backend injects a routine right after
  lifting it (`lower`), long before the module's top level is transformed at all
  (`cg`) — so both up-field assignment sites create the ops themselves.

* **Two backend hooks collide on one C name.** A backend-minted symbol is named
  `_c<itemId.item>`, a PER-PROCESS counter — fine while "nifc lifts, emits and
  compiles them in one run", which is what `mangleProcNameExt` assumed. But
  `lower` mints the env hooks of nested routines and `cg` mints those of the
  module's top level, and both land in the same translation unit: two unrelated
  `=destroy`s became one C function. `mangleProcNameExt` already makes an
  exception for hooks whose `disamb` is content-derived (`HookDisambBit`);
  `toNifSymName` now mirrors it, so the content-derived value survives the round
  trip instead of being overwritten by the loader.

`InstanceDisambBit`/`HookDisambBit` move to `astdef` — `ast2nif` names symbols by
them and cannot import `modulegraphs`.

Two new metamorphic tests, both of which fail on the previous compiler:
`tclosure_hooks` (generic closure iterator instantiated across modules) and
`tclosure_nested_iter` (closure iterator nested in a closure iterator).

`icFormatVersion` 37 -> 38 for the hook naming. `koch bootic` reaches its
byte-identical fixed point; `tests/closure`, `tests/destructor` (3 known
`--newruntime` failures), `tests/cpp` and the classic categories are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 12:12:51 +02:00
araq
f3bdc6c5f2 IC: nim c --ic:on / nim cpp --ic:on replace the nim ic command
`nim ic` was a command of its own, which made it the C backend only and cut it
off from everything the ordinary compile commands accept. `--ic:on` is a switch
on `nim c` / `nim cpp` / `nim objc` instead, so `-r`, `-d:release`,
`--exceptions:`, and a project-wide opt-in from `nim.cfg` / `config.nims` all
work. `nim ic` still resolves to the same driver; `koch bootic`, `koch ic` and
`testament --ic` now go through the switch.

The switch was already parsed into a `conf.ic` nobody read. It is now read in
`passCmd1` as well, because `nim.nim` has to decide whether this run is an IC
DRIVER before config loading (`ensureIcConfig` produces the precompiled config
the driver itself replays); when the switch comes from a config file instead,
`main.nim` produces it late.

**C++.** `tests/cpp` passes under `--ic:on`, matching its classic result. Four
fixes, three of them the shape of every "C++ needs the whole program" problem the
per-module backend has:

* The driver DECLARES each module's translation unit to nifmake without loading a
  module, so it cannot ask `cgen.getCFile` — and it hardcoded `.nim.c`, so the
  merge stage went looking for `.c.nif` next to the `.cpp.nif` the children had
  written. `options.icCFileExt` mirrors the formula at backend granularity.

* C++ has no designated initializers, so the RTTI record is a bare variable that
  `DatInit` fills field by field. A bare `TNimTypeV2 x;` is a tentative
  definition — C's linker merges those, C++'s does not — so every TU that
  demanded the type defined it ("multiple definition of NTIv2__…"). It now gets
  the same extern-declaration + owned-`'d'`-definition split the C flavour has.

* `memberProcsPerType` / `initializersPerType` live only in the sem process, so
  the backend emitted a struct WITHOUT its in-class member declarations and the
  out-of-class definitions did not match ("no declaration matches
  'void Doo::memberProc()'"). They are replayed from a new `(repcppmember …)`
  log entry; `replayCppMember` re-derives the type from the routine's signature
  exactly as `semCppMember` does, so no type key has to survive the round trip.

* Two follow-ons for members: `loc.snippet` is a CALL PATTERN (`#->salute(@)`),
  and only `genMemberProcHeader` derives it — whole-program cgen got it for free
  by generating the defining module first, but the per-module backend emits that
  body in another process, leaving the caller with the mangled Nim name
  (`loo->salute_u0__vireouyks1()`). And that pattern is not a linker name: every
  `salute` member in every class mints the same one, so the merge stage handed
  them all to one artifact and dropped the rest (undefined vtable at link).
  Member definitions are keyed by their NIF name there instead.

`--run` is now dropped when re-invoking for the config artifact: the producer has
no output binary and `nim.nim`'s run step asserted on the empty `outFile`.

testament's `--ic` appends the switch rather than rewriting the compile verb, so
a test that overrides `cmd:` wholesale keeps its verb, and the C++ corpus is
covered too (it never was — the old rewrite only matched `nim c `).

`icFormatVersion` 36 -> 37 for the new log entry. `koch bootic` reaches its
byte-identical fixed point through the new entry point; `koch ic` passes;
`tests/ic`, `tests/destructor` (3 known `--newruntime` failures) and the classic
categories are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 10:32:31 +02:00
araq
7ddfc44c0f IC: six more correctness fixes found by the nim c oracle
`testament --ic` on `tests/destructor` went from 10 failures to 3; the three
left are all `--newruntime`, whose owned-ref RTTI destructor is still missing
(the object type's `=destroy` never reaches the `cg` that emits the type info).

* **global destructors are never run.** `graph.globalDestructors` is filled
  while a module's top level goes through `injectDestructorCalls`, and
  whole-program cgen empties the list into the main module's init proc — which
  IS the program body, so the calls land at exit. Under `nim ic` every module's
  `cg` is a separate process and main's only ever saw its own entries: a
  module-level `var` with a `=destroy` in an imported module was simply never
  destroyed. Each module now wraps its own list in a nullary exported proc and
  announces the name in its `.c.nif` meta head (`CnifVersion` 4 -> 5); main's
  `cg` reads the heads it already reads for init/datInit and calls them in
  reverse dependency order. New test `tests/ic/tglobal_dtors.nim` pins the
  order against the oracle.

* **`spawn` died with "system module needs: nimArgsPassingDone".** A module
  loaded from a NIF is named by its mangled suffix, so `loadCompilerProc`'s
  `module.name.s == "threadpool"` could never match. The backend loads the whole
  program before codegen, so consult every loaded module's index instead; misses
  are final and remembered (`getCompilerProc` doubles as a presence probe).

* **`new(x, finalizer)` died with "environment misses: x".** `semmagic`'s
  finalizer-to-destructor wrapper copies the original's AST but only rewrites
  the parameter, leaving `ast[namePos]` naming the ORIGINAL proc. ast2nif
  re-derives a routine's serialized AST from `ast[namePos].sym.ast`, so the
  wrapper serialized the original's body — whose parameter belongs to the
  original — and lambda lifting saw it as a captured variable of another proc.
  The copy now names itself, the invariant every other routine AST keeps.

* **a top-level `let (a, b) = f()` copied instead of moved.** A module's own
  symbol is the owner of every top-level symbol and is written as a real `(sd)`,
  so the loader minted a SECOND `skModule` PSym for it — and `sym.owner == owner`
  is an identity test in `aliasanalysis.isAnalysableFieldAccess`, which made
  every module-level location un-analysable. Hard error for a type with a
  disabled `=copy`. Bind the NIF name to the one registered module symbol.
  Backend only: doing it under `nim m` costs `times.toDateTimeByWeek` its
  inferred `raises`.

* **a doubly linked list leaked its whole contents.** A field USE serializes as
  a bare `SymUse` with nowhere to put symbol flags, so `trees.isCursor` said
  "not a cursor" for every loaded field and `DoublyLinkedNode.prev` became a
  COUNTED reference: every node held its predecessor alive and no refcount ever
  hit zero. `{.cursor.}` now rides in the NIF name marker (`` `fc `` next to
  `` `f ``), which def and use derive from the same `PSym`.

* **`--expandArc` came out shuffled.** `moduleSymbolStubs` iterated a `Table`,
  i.e. hash order, so the `lower` stage transformed a module's routines in an
  arbitrary order — not even stable between two compilers. Order by index
  offset, which is source order.

Also: `testament`'s `generatedFile` did not include the matrix entry in the
nimcache key its caller uses, so every `ccodeCheck` test with a `matrix:`
reported `reCodeNotFound`.

`koch bootic` still reaches its fixed point; `tests/ic` and the classic
`tests/destructor` are unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 08:43:42 +02:00
araq
467c911dc5 IC testing: compare against nim c, and run the real corpus under nim ic
Every invariant `tests/ic` checked was IC-against-IC — clean == incremental, a
no-op edit changes nothing, a body edit moves no interface cookie. An IC that is
*consistently* wrong satisfies all of them, and that is exactly how two silent
miscompilations survived. `koch bootic` has the same blind spot: it proves the
compiler reproduces itself, not that it agrees with the reference backend.

Two mechanisms, at different scales.

**The oracle, in the metamorphic runner.** Every successful step is now also
compiled with `nim c` and run, and the two outputs must agree. Unlike the
hand-written `expect:` strings this needs no foresight from the test author: it
compares everything the program does, not only what someone thought to print,
which is what a silently-skipped destructor evades. `no-oracle` opts out.

The format also grew the expressiveness the recent bug hunt showed was missing —
every one of these described a state the suite could not reach:

* `#!DELETE <file>` removes a module. Deleting a still-imported file moves no
  mtime, so nothing re-fires.
* `#!FLAGS <switches>` changes the compiler switches between steps. A config
  change is not a file either.
* `fails: <substring>` asserts that BOTH compilers reject the program with that
  text. Previously every step had to succeed, so the whole error path — and
  recovery from it — was untested.

Six regression tests cover the eight bugs the last round fixed.
`testament r <file>` now dispatches metamorphic tests like `testament cat ic`.

**`testament --ic` runs the whole corpus through the incremental compiler**, so
IC inherits ~10k programs with expected output instead of 30 bespoke tests.
Two things had to change for that to mean anything:

* `nimcacheDir` now keys on the `matrix:` entry too. Two matrix variants of one
  file are two different compilations; sharing a cache meant each run
  invalidated what the previous left — harmless for a backend that caches only
  object files, useless for an incremental one.
* About half the corpus overrides the command wholesale (`cmd: "nim c --gc:arc
  $file"`), bypassing both `$target` and `$options`. Those are rewritten to `ic`
  and given a private cache.

**Warm cache, hastur-style.** A generated warmup program pulling in `system` and
the most-imported stdlib modules is compiled once per distinct compile
configuration into `nimcache/ic_warmup_<hash>`, and each test's empty cache is
seeded from it with mtimes preserved (nifmake compares output-mtime >
input-mtime, so stamping the copies "now" re-fires the whole graph). Only
program-independent artifacts are copied: the frontend NIFs and cookies plus the
per-module `lower`/`cg` output. The `.c`/`.o` are left behind on purpose — the
merge decision is whole-program, so they are re-rendered for every program
anyway.

`tests/destructor` (97 runs): `nim c` 35s cold / 32s warm; `nim ic` ~3m30 cold /
**9.8s warm**.

Compiler changes this required or uncovered:

* `merge` read the live-module list from a manifest the driver writes instead of
  globbing `*.c.nif` off the nimcache. Globbing absorbed artifacts belonging to
  any other program sharing the directory — which is what made a prefilled cache
  produce undefined symbols at link.
* The build-arg signature no longer includes `--icproject:`/`--icPreparsedConfig:`
  (they name where a build lives, not what it produces, so two caches holding
  identical artifacts got different signatures). The precompiled config still
  counts, by content hash, minus its `(nimcache …)` line.
* `.s.deps` seeding is speculative and runs before the prune, so a sidecar entry
  that has gone stale (an import that a `when` no longer takes) can be dropped
  instead of lingering forever; a pruned module's scan artifacts are deleted so
  an edit-accumulated cache still matches a clean one.
* A failed nifmake run no longer prints an `Error:` of its own. The children have
  already reported; adding a build-system status as the LAST error hid the
  compiler's real message from anything reading the final error — every
  reject-style test under `nim ic` said "nifmake failed with exit code: 1".
* `--mm:hooks` fed the mm mode to an on/off switch and failed outright with
  "'on' or 'off' expected, but 'hooks' found". Pre-existing and unrelated to IC;
  only reachable through the explicit switch, since `--newruntime` sets
  `selectedGC` directly.

Running `tests/destructor` under `--ic` currently leaves 10 failures. They are
genuine IC defects, not harness noise (all 97 pass under `nim c`) — the clearest
is `tglobaldestructor`: `graph.globalDestructors` is accumulated while injecting
destructors into a module's top level, but the main module's `cg` — which emits
the teardown — is a different process, so a module-level `var` with a `=destroy`
is never destroyed. Same shape as the init/datInit metas, and it wants the same
fix: record it in the `.c.nif` head.

Validation: `koch bootic` reaches its byte-identical fixed point; `tests/ic` is
36/36; arc, destructor, macros, template, iter, closure, ccg, codegen, types and
effects pass under `nim c`, with generics showing only its pre-existing failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 21:00:13 +02:00
araq
c01e58c146 IC: eight correctness fixes found by differential testing against nim c
Grinding a small figdraw-based program under `nim ic` and diffing its output
against the classic backend surfaced eight bugs, four of which silently produced
a wrong binary rather than an error.

Frontend / build graph (`deps.nim`):

* Dead `when`-guarded imports were compiled anyway. `when someStrdefine == "x":
  import y` is `cvUnknown` to the scanner, which conservatively keeps the edge —
  right for an edge, but it also gave `y` its own `nim m` rule, so a build died
  on a package the user never installed because they never selected that backend.
  Track which edges are speculative and drop a speculative subtree that cannot
  compile; if the guard was in fact live, the discovery fixpoint puts the node
  back with the honest `cannot open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so nothing
  re-fires and `nim ic` relinked a stale binary while `nim c` reported `cannot
  open file`. Report an unresolvable import from a non-speculatively reached
  module during the graph scan.
* Macro-generated imports were discovered once and then forgotten. Discovery
  only ran after a failure and the graph is re-derived statically every run, so
  on a warm build the discovered module had no rules at all and editing it
  changed nothing. Seed the graph from the `.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from file mtimes
  and never looks at a rule's command line, so `-d:foo=bar` / `--mm:` / `--threads:`
  regenerated the build file with the new switches and re-fired zero rules.
  Reify the configuration as a file and make it an input of every rule.
* Command-line switches never reached the children: they replay the project's
  config files, never the driver's argv, so `nim ic --opt:speed` produced a
  byte-identical debug binary (likewise `--panics`, `--experimental`, `--passC`).
  Forward the driver's switches, minus the ones that must differ per child.

Artifacts and codegen:

* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw the rule
  as satisfied on the next run: `nim ic` then reported success for a program that
  does not compile, and generated code from error-bearing AST (or hit an internal
  error in `ccgexprs`). Never persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives on the
  module symbol, which `moduleFromNifFile` rebuilds from scratch, so
  `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a module-level
  `block: let h = openHandle()` never ran `=destroy`. Persist the flag as a
  `(modflags)` record. `injectdestructors` also has to tolerate the
  `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym node is
  written as a bare NIF `SymUse` token, which has nowhere to put node flags, so
  the frontend's move analysis never reached the backend: EVERY first assignment
  to a destructor-bearing local compiled as `=sink`, i.e. `=destroy` on
  still-zeroed memory followed by a copy, and no read was ever a move. Wrap a sym
  use in `(nflags ...)` when it carries persistent node flags.

`icFormatVersion` 34 -> 35 for the two new NIF records.

Validation: `koch bootic` reaches its byte-identical fixed point; two clean-cache
builds from the same compiler are identical; testament `arc`, `destructor`,
`macros`, `template`, `iter`, `closure`, `ccg`, `codegen`, `types` and `effects`
pass, and `generics`, `ic` and `stdlib` show exactly the pre-existing failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 19:24:08 +02:00
38 changed files with 947 additions and 783 deletions

View File

@@ -314,22 +314,15 @@ proc toNifSymName(w: var Writer; sym: PSym): string =
# during a VM transform): re-home to the current module with the `@bk`
# marker so each referencing module self-contains it. See transformBody.
#
# Use `itemId.item` (the writer's dedup identity, see `emittedBackendSyms`)
# as the numeric name component, NOT `disamb`: closure `:env` syms in one
# module are minted from TWO id spaces — the backend lower stage's
# `tb.idgen` and sem's `vmTransfIdgen` (transf.transformBody) — whose
# `disambTable`s each start `:env` at the same low count, so a macro-lowered
# `:env` (e.g. `implementSendProcBody`) and a backend-lowered one
# (`peerTrimmerHeartbeat`) collide on `:env.2.<mod>@bk`. Two distinct syms
# then share a NIF name; the loader's name-keyed index/`c.syms` return the
# first for both, so one proc's `:env` gets the OTHER proc's env type
# (mismatched-pointer C, "has no member colonup_" at link). `itemId.item` is
# unique per `@bk` sym (both are emitted as defs, see writeSym), mirroring
# how `@bk` TYPES already key off `itemId.item` (nifTypeName). The loader
# copies this back into `disamb` (sn.count), so `globalName` round-trips.
# The numeric name component comes from `astdef.backendMintedDisamb` — the
# ONE definition of which integer identifies a backend-minted symbol, shared
# with the two C-name manglers (`mangleProcNameExt`, `ccgutils.makeUnique`)
# so the NIF name and the C name cannot disagree. `@bk` TYPES key off
# `itemId.item` the same way (see `nifTypeName`). The loader copies this back
# into `disamb` (sn.count), so `globalName` round-trips.
result = sym.name.s
result.add '.'
result.addInt sym.itemId.item
result.addInt backendMintedDisamb(sym)
result.add '.'
result.add modname(w.currentModule, w.infos.config)
result.add BackendLocalMarker
@@ -2552,6 +2545,18 @@ proc cursorFromIndexEntry(c: var DecodeContext; module: FileIndex; entry: NifInd
type
LoadFlag* = enum
LoadFullAst, AlwaysLoadInterface
SkipInterfaceTables
## Do not eagerly build the module's interface string tables. Set by
## `modulegraphs.loadTransitiveHooks`, which loads a module only to
## register its hooks / macro-cache replay / generic-instance offers and
## throws the tables away — the module is a dep-of-a-dep, not an import, so
## none of its symbols are visible to the module being semchecked.
##
## The eager pass calls `loadSymFromIndexEntry` for EVERY index entry, and
## its only other effect is pre-populating the name-keyed `c.syms` cache —
## which `resolveSym` fills lazily on a miss anyway, straight from the same
## index. So for these loads it is pure work: on a 219-module program a
## one-line edit paid it 209 times over.
proc isGlobalIndexSym(s, dottedSuffix: string): bool =
## Mirror of `nifbuilder.addSymbolDefRetIsGlobal` / `bif.isGlobalSymbol`: a sym
@@ -3841,6 +3846,13 @@ proc processTopLevel(c: var DecodeContext; cur: var Cursor; flags: set[LoadFlag]
elif tagIs(cur, "reppureenum"): loadLogOp(c, result.logOps, cur, PureEnumEntry, attachedTrace, module)
elif tagIs(cur, "repcppmember"): loadLogOp(c, result.logOps, cur, CppMemberEntry, attachedTrace, module)
elif tagIs(cur, "export"):
if SkipInterfaceTables in flags:
# Same reason the interface tables are skipped: `interf` is a scratch
# table this caller throws away, so every `resolveSym` here (one per
# exported symbol, plus `addReexportedEnumFields`) only warms the
# name-keyed `c.syms` cache that `resolveSym` refills lazily on a miss.
skip cur
continue
cur.into:
while cur.hasMore and cur.kind == DotToken: skip cur # flags / type
while cur.hasMore:
@@ -3980,7 +3992,8 @@ proc loadNifModule*(c: var DecodeContext; suffix: ModuleSuffix; interf, interfHi
# Populate interface tables from the NIF index structure
# Symbols are created as stubs (Partial state) and will be loaded lazily via loadSym
# Use exports collected by processTopLevel
populateInterfaceTablesFromIndex(c, module, interf, interfHidden, string(suffix))
if SkipInterfaceTables notin flags:
populateInterfaceTablesFromIndex(c, module, interf, interfHidden, string(suffix))
proc loadNifModule*(c: var DecodeContext; f: FileIndex; interf, interfHidden: var TStrTable;
flags: set[LoadFlag] = {}): PrecompiledModule =

View File

@@ -960,10 +960,22 @@ iterator sons*(n: PNode): PNode =
## as it does not rely on random indexed access (see doc/ic_backend_nif_native.md).
for i in 0..<n.safeLen: yield n[i]
iterator isons*(n: PNode): tuple[i: int, n: PNode] =
## Like `sons` but also yields the child index. Replaces
## `for i in 0..<n.len: ... n[i] ...` when `i` itself is still needed.
for i in 0..<n.safeLen: yield (i, n[i])
iterator isons*(n: PNode; start = 0): tuple[i: int, n: PNode] =
## Like `sons` but also yields the child index, and optionally skips the first
## `start` children. Replaces `for i in start..<n.len: ... n[i] ...` when `i`
## itself is still needed — for a parameter position, a `needTmp[i-1]` lookup,
## a parallel index into the routine's `PType`, and so on. `start` is almost
## always 1, to step over a call's callee or a case statement's selector.
##
## Use `sonsFrom` instead when the index is only ever used to subscript `n`.
for i in start..<n.safeLen: yield (i, n[i])
iterator sonsFrom*(n: PNode; start: int): PNode =
## `sons` skipping the first `start` children. Replaces
## `for i in start..<n.len: ... n[i] ...`, which is by far the commonest
## indexed shape in the code generator — `start` is almost always 1, to step
## over a case/try statement's selector or a call's callee.
for i in start..<n.safeLen: yield n[i]
when defined(useNodeIds):
const nodeIdToDebug* = -1 # 2322968
@@ -1046,6 +1058,52 @@ proc newStrNode*(strVal: string; info: TLineInfo): PNode =
# handling for IC, they end up in IC indexes etc. Thus we "log" them in the module graph
# and to pass them around to the NIF writer. This is not very elegant but it works.
const
InstanceDisambBit* = 0x4000_0000'i32
## Set in the `disamb` of routine instances whose value is content-derived
## (see `modulegraphs.setInstanceDisamb`); keeps them disjoint from the
## small counter range ordinary symbols draw from, so the NIF name
## `name.disamb.module` stays collision-free within a module.
HookDisambBit* = 0x2000_0000'i32
## Set in the `disamb` of synthesized type-bound operators and `$enum`
## procs whose value is content-derived (see `modulegraphs.setHookDisamb`);
## disjoint from both the small counter range and `InstanceDisambBit`.
##
## Both live here rather than in `modulegraphs` because `ast2nif` — which
## cannot import that module — names symbols by them.
proc backendMintedDisamb*(s: PSym): int32 {.inline.} =
## The integer that identifies a BACKEND-MINTED symbol (`isBackendMinted`) in
## every name derived from it: its NIF name (`ast2nif.toNifSymName`) and its C
## name (`mangleutils.mangleProcNameExt`, `ccgutils.makeUnique`).
##
## Two cases, and the whole point of having ONE function is that all three
## sites take the same one:
##
## * A lifted HOOK's `disamb` is CONTENT-derived (`modulegraphs.setHookDisamb`),
## so it is identical in every process. Such a hook really does cross process
## boundaries — `lower` mints the env hooks of nested routines while `cg`
## mints those of the module's top level, and both land in the same
## translation unit — and its C name is also baked into emit-everywhere RTTI
## tables. `itemId.item` would differ per process, so two unrelated hooks
## collided on one `_c<item>` and the merge stage kept a single body for both
## (C accepted the mistyped call, C++ rejected it).
## * Otherwise `itemId.item` — the writer's dedup identity, unique per `@bk`
## sym. `disamb` cannot serve here: a module's `:env` syms are minted from TWO
## id spaces (the backend `lower` stage's idgen and sem's `vmTransfIdgen`)
## whose `disambTable`s each start `:env` at the same low count, so a
## macro-lowered and a backend-lowered `:env` collide on `:env.2.<mod>@bk`.
##
## The loader copies the name's numeric component back into `disamb`, so after a
## round trip `disamb` equals this value and `ast2nif.globalName` — which always
## reads `disamb` — agrees with the name the writer produced.
##
## This rule used to be written out at each of the three sites. They drifted:
## `toNifSymName` lacked the hook exception, so a content-derived value was
## overwritten by the loader and two backend hooks merged into one C function.
if (s.disamb and HookDisambBit) != 0'i32: s.disamb
else: s.itemId.item
type
LogEntryKind* = enum
HookEntry, ConverterEntry, MethodEntry, EnumToStrEntry, GenericInstEntry,

86
compiler/bnode.nim Normal file
View File

@@ -0,0 +1,86 @@
#
#
# The Nim Compiler
# (c) Copyright 2026 Andreas Rumpf
#
# See the file "copying.txt", included in this
# distribution, for details about the copyright.
#
## `BNode` — the backend's node type, and the seam for running codegen off a
## `.bif` `Cursor` instead of a deserialized `PNode` tree.
##
## Building those trees is the bulk of the `lower` and `cg` stages: their cost
## tracks the size of the dependency CLOSURE a stage loads, not the module it
## compiles (measured: a 370-byte module costs 0.20s/0.16s in lower/cg, the main
## module 3.40s/3.16s, and the two are ~85% of the serial backend critical path).
##
## With `-d:newIcBackend` `BNode` is a `Cursor`; without it a plain `PNode`,
## which is what every build does today. Codegen migrates to the vocabulary
## below one area at a time and the compiler keeps building throughout, because
## on the `PNode` side the vocabulary is what `ast`/`astdef` already provide —
## `kind`, `len`, `safeLen`, `sym`, `typ`, `info`, `firstSon`, `secondSon`,
## `lastSon` and the `sons`/`isons`/`sonsFrom` iterators all exist. This module
## deliberately does
## NOT redefine them for `PNode`: an identical second overload would make every
## call site ambiguous. It adds only what the AST lacks (`son`, `hasSons`), and
## supplies the whole vocabulary on the `Cursor` side.
##
## THE COST MODEL DIFFERS, and that is what the vocabulary is shaped around. A
## `Cursor` is a copyable position in a token buffer, so a child is reached by
## `firstSon` plus one `skip` per preceding sibling — and `skip` steps over a
## whole subtree. Reading child `i` is therefore O(size of children 0..<i):
##
## * `firstSon` / `secondSon` / `son(n, k)` with small constant `k` — cheap, and
## already how most structural access reads (344 of 777 indexed accesses in
## the cgen files use a constant or a `*Pos` index).
## * `for x in sons(n)` / `sonsFrom(n, k)` — one linear pass. ALWAYS migrate an
## indexed loop to these: `for i in 0..<n.len: n[i]` is O(n^2) once `BNode` is
## a `Cursor`, and ~196 such accesses remain.
## * `lastSon(n)` — O(len). Fine once, a trap inside a loop; 28 `n[^1]` uses.
## * `len(n)` — O(len) on a `Cursor`, which has to count. Do not put it in a loop
## condition; use `sons`/`sonsFrom`, or `hasSons` for an emptiness test.
import ast, lineinfos
when defined(newIcBackend):
import "../dist/nimony/src/lib" / nifcursors
# Imported only under the define: `cgen` is compiled during the koch
# bootstrap, where the nimony libs are unavailable (`ast2nif` is guarded the
# same way).
type BNode* = Cursor
# Migrated one accessor per step. Until then the `{.error.}` stubs make
# flipping the define report the exact missing piece AT ITS CALL SITE, rather
# than collapsing into a cascade of unrelated type errors.
proc kind*(n: BNode): TNodeKind {.error:
"BNode.kind: not implemented for Cursor yet — map the tag id to TNodeKind. " &
"`ic/enum2nif.parse(TNodeKind, string)` is the reverse of `toNifTag`, but a " &
"per-call string compare is too slow here: build a tag-id -> TNodeKind table once.".} = discard
proc len*(n: BNode): int {.error: "BNode.len: not implemented for Cursor yet (counts children; prefer sons/hasSons)".} = discard
proc safeLen*(n: BNode): int {.error: "BNode.safeLen: not implemented for Cursor yet".} = discard
proc son*(n: BNode; i: int): BNode {.error: "BNode.son: not implemented for Cursor yet (firstSon + i skips)".} = discard
proc firstSon*(n: BNode): BNode {.error: "BNode.firstSon: not implemented for Cursor yet".} = discard
proc secondSon*(n: BNode): BNode {.error: "BNode.secondSon: not implemented for Cursor yet".} = discard
proc lastSon*(n: BNode): BNode {.error: "BNode.lastSon: not implemented for Cursor yet (O(len))".} = discard
proc hasSons*(n: BNode): bool {.error: "BNode.hasSons: not implemented for Cursor yet".} = discard
proc sym*(n: BNode): PSym {.error: "BNode.sym: not implemented for Cursor yet".} = discard
proc typ*(n: BNode): PType {.error: "BNode.typ: not implemented for Cursor yet".} = discard
proc info*(n: BNode): TLineInfo {.error: "BNode.info: not implemented for Cursor yet".} = discard
iterator sons*(n: BNode): BNode {.error: "BNode.sons: not implemented for Cursor yet".} = discard
iterator sonsFrom*(n: BNode; start: int): BNode {.error: "BNode.sonsFrom: not implemented for Cursor yet".} = discard
else:
type BNode* = PNode
# Only the two the AST does not already have. Everything else in the
# vocabulary is `ast`/`astdef`'s own `PNode` API — see the module doc.
template son*(n: BNode; i: int): BNode =
## Named indexed access. Exists so a call site states "child i" in a form
## that survives `BNode` becoming a `Cursor`; keep `i` small and constant.
n[i]
template hasSons*(n: BNode): bool =
## Emptiness test that does not compute a length — `len` counts on a
## `Cursor`.
n.safeLen > 0

View File

@@ -11,11 +11,7 @@
proc canRaiseDisp(p: BProc; n: PNode): bool =
# we assume things like sysFatal cannot raise themselves
if n.kind == nkSym and n.sym.kind == skMethod:
# A base method may be overridden by a branch with a wider exception set.
# Its inferred effects describe only the base body, not every vtable target.
result = true
elif n.kind == nkSym and {sfNeverRaises, sfImportc, sfCompilerProc} * n.sym.flags != {}:
if n.kind == nkSym and {sfNeverRaises, sfImportc, sfCompilerProc} * n.sym.flags != {}:
result = false
elif optPanics in p.config.globalOptions or
(n.kind == nkSym and sfSystemModule in getModule(n.sym).flags and
@@ -53,8 +49,7 @@ proc preventNrvo(p: BProc; dest, le, ri: PNode): bool =
result = false
if le != nil:
for i in 1..<ri.len:
let r = ri[i]
for r in sonsFrom(ri, 1):
if isPartOf(le, r, {pfStructural}) != arNo: return true
# we use the weaker 'canRaise' here in order to prevent too many
# annoying warnings, see #14514
@@ -63,8 +58,7 @@ proc preventNrvo(p: BProc; dest, le, ri: PNode): bool =
message(p.config, le.info, warnObservableStores, $le)
# bug #19613 prevent dangerous aliasing too:
if dest != nil and dest != le:
for i in 1..<ri.len:
let r = ri[i]
for r in sonsFrom(ri, 1):
if isPartOf(dest, r, {pfStructural}) != arNo: return true
proc hasNoInit(call: PNode): bool {.inline.} =
@@ -478,19 +472,19 @@ proc genParams(p: BProc, ri: PNode, typ: PType; result: var Builder, argBuilder:
# Optimization: don't use a temp, if we would only take the address anyway
needTmp[i - 1] = false
for i in 1..<ri.len:
for i, it in isons(ri, 1):
if i < typ.n.len:
assert(typ.n[i].kind == nkSym)
let paramType = typ.n[i]
if not paramType.typ.isCompileTimeOnly:
var arg = newBuilder("")
genArg(p, ri[i], paramType.sym, ri, arg, needTmp[i-1])
genArg(p, it, paramType.sym, ri, arg, needTmp[i-1])
if arg.buf.len != 0:
result.addArgument(argBuilder):
result.add(extract(arg))
else:
var arg = newBuilder("")
genArgNoParam(p, ri[i], arg, needTmp[i-1])
genArgNoParam(p, it, arg, needTmp[i-1])
if arg.buf.len != 0:
result.addArgument(argBuilder):
result.add(extract(arg))
@@ -731,7 +725,7 @@ proc genPatternCall(p: BProc; ri: PNode; pat: string; typ: PType; result: var Bu
case pat[i]
of '@':
var callBuilder = default(CallBuilder) # not init call builder
for k in j..<ri.len:
for k, _ in isons(ri, j):
genOtherArg(p, ri, k, typ, result, callBuilder)
inc i
of '#':
@@ -815,7 +809,7 @@ proc genInfixCall(p: BProc, le, ri: PNode, d: var TLoc) =
pl.add(op.snippet)
var res = newBuilder("")
var call = initCallBuilder(res, extract(pl))
for i in 2..<ri.len:
for i, _ in isons(ri, 2):
genOtherArg(p, ri, i, typ, res, call)
fixupCall(p, le, ri, d, res, call)
@@ -846,7 +840,7 @@ proc genNamedParamCall(p: BProc, ri: PNode, d: var TLoc) =
if ri.len > 2:
pl.add(": ")
genArg(p, ri[2], typ.n[2].sym, ri, pl)
for i in start..<ri.len:
for i, it in isons(ri, start):
if i >= typ.n.len:
internalError(p.config, ri.info, "varargs for objective C method?")
assert(typ.n[i].kind == nkSym)
@@ -854,7 +848,7 @@ proc genNamedParamCall(p: BProc, ri: PNode, d: var TLoc) =
pl.add(" ")
pl.add(param.name.s)
pl.add(": ")
genArg(p, ri[i], param, ri, pl)
genArg(p, it, param, ri, pl)
if typ.returnType != nil:
if isInvalidReturnType(p.config, typ):
if ri.len > 1: pl.add(" ")

View File

@@ -1073,8 +1073,8 @@ proc genInExprAux(p: BProc, e: PNode, a, b, d: var TLoc)
proc genFieldCheck(p: BProc, e: PNode, obj: Rope, field: PSym, ty: PType) =
var test, u, v: TLoc
for i in 1..<e.len:
var it = e[i]
for child in sonsFrom(e, 1):
var it = child
assert(it.kind in nkCallKinds)
assert(it.firstSon.kind == nkSym)
let op = it.firstSon.sym
@@ -1932,15 +1932,15 @@ proc genObjConstr(p: BProc, e: PNode, d: var TLoc) =
r = rdLoc(d)
discard getTypeDesc(p.module, t)
let ty = getUniqueType(t)
for i in 1..<e.len:
if nfPreventCg in e[i].flags:
for it in sonsFrom(e, 1):
if nfPreventCg in it.flags:
# this is an object constructor node generated by the VM and
# this field is in an inactive case branch, don't generate assignment
continue
var check: PNode = nil
if e[i].len == 3 and optFieldCheck in p.options:
check = e[i][2]
genFieldObjConstr(p, ty, useTemp, isRef, e[i].firstSon, e[i][1], check, d, r, e.info)
if it.len == 3 and optFieldCheck in p.options:
check = it[2]
genFieldObjConstr(p, ty, useTemp, isRef, it.firstSon, it[1], check, d, r, e.info)
if useTemp:
if d.k == locNone:
@@ -2447,8 +2447,7 @@ proc genInOp(p: BProc, e: PNode, d: var TLoc) =
b = initLoc(locExpr, e, OnUnknown)
if e[1].len > 0:
var val: Snippet = ""
for i in 0..<e[1].len:
let it = e[1][i]
for it in sons(e[1]):
var currentExpr: Snippet
if it.kind == nkRange:
x = initLocExpr(p, it.firstSon)
@@ -3861,8 +3860,7 @@ proc containsOpaqueImportcFieldAux(t: PType; n: PNode): bool =
of nkRecCase:
if containsOpaqueImportcFieldAux(t, n.firstSon):
return true
for i in 1..<n.len:
let branch = n[i]
for branch in sonsFrom(n, 1):
if branch.kind == nkOfBranch or branch.kind == nkElse:
if containsOpaqueImportcFieldAux(t, branch.lastSon):
return true
@@ -4003,13 +4001,13 @@ proc getNullValueAux(p: BProc; t: PType; obj, constOrNil: PNode,
var branch = Zero
if constOrNil != nil:
## find kind value, default is zero if not specified
for i in 1..<constOrNil.len:
if constOrNil[i].kind == nkExprColonExpr:
if constOrNil[i].firstSon.sym.name.id == obj.firstSon.sym.name.id:
branch = getOrdValue(constOrNil[i][1])
for i, it in isons(constOrNil, 1):
if it.kind == nkExprColonExpr:
if it.firstSon.sym.name.id == obj.firstSon.sym.name.id:
branch = getOrdValue(it[1])
break
elif i == obj.firstSon.sym.position:
branch = getOrdValue(constOrNil[i])
branch = getOrdValue(it)
break
let selectedBranch = caseObjDefaultBranch(obj, branch)
@@ -4050,14 +4048,14 @@ proc getNullValueAux(p: BProc; t: PType; obj, constOrNil: PNode,
result.addField(init, name = sname):
block fieldInit:
if constOrNil != nil:
for i in 1..<constOrNil.len:
if constOrNil[i].kind == nkExprColonExpr:
assert constOrNil[i].firstSon.kind == nkSym, "illformed object constr; the field is not a sym"
if constOrNil[i].firstSon.sym.name.id == field.name.id:
genBracedInit(p, constOrNil[i][1], isConst, field.typ, result)
for i, it in isons(constOrNil, 1):
if it.kind == nkExprColonExpr:
assert it.firstSon.kind == nkSym, "illformed object constr; the field is not a sym"
if it.firstSon.sym.name.id == field.name.id:
genBracedInit(p, it[1], isConst, field.typ, result)
break fieldInit
elif i == field.position:
genBracedInit(p, constOrNil[i], isConst, field.typ, result)
genBracedInit(p, it, isConst, field.typ, result)
break fieldInit
# not found, produce default value:
getDefaultValue(p, field.typ, info, result)

View File

@@ -19,8 +19,8 @@ proc specializeResetN(p: BProc, accessor: Rope, n: PNode;
if n == nil: return
case n.kind
of nkRecList:
for i in 0..<n.len:
specializeResetN(p, accessor, n[i], typ)
for it in sons(n):
specializeResetN(p, accessor, it, typ)
of nkRecCase:
if (n[0].kind != nkSym): internalError(p.config, n.info, "specializeResetN")
let disc = n[0].sym
@@ -29,8 +29,7 @@ proc specializeResetN(p: BProc, accessor: Rope, n: PNode;
internalError(p.config, n.info, "specializeResetN()")
let discField = dotField(accessor, disc.loc.snippet)
p.s(cpsStmts).addSwitchStmt(discField):
for i in 1..<n.len:
let branch = n[i]
for branch in sonsFrom(n, 1):
assert branch.kind in {nkOfBranch, nkElse}
var caseBuilder: SwitchCaseBuilder
p.s(cpsStmts).addSwitchCase(caseBuilder):

View File

@@ -329,18 +329,18 @@ proc genCppParamsForCtor(p: BProc; call: PNode; didGenTemp: var bool): Snippet =
var argBuilder = default(CallBuilder) # not init, only building params
let typ = skipTypes(call.firstSon.typ, abstractInst)
assert(typ.kind == tyProc)
for i in 1..<call.len:
for i, child in isons(call, 1):
#if it's a type we can just generate here another initializer as we are in an initializer context
if call[i].kind == nkCall and call[i].firstSon.kind == nkSym and call[i].firstSon.sym.kind == skType:
if child.kind == nkCall and child.firstSon.kind == nkSym and child.firstSon.sym.kind == skType:
res.addArgument(argBuilder):
res.add genCppInitializer(p.module, p, call[i].firstSon.sym.typ, didGenTemp)
res.add genCppInitializer(p.module, p, child.firstSon.sym.typ, didGenTemp)
else:
#We need to test for temp in globals, see: #23657
let param =
if typ[i].kind in {tyVar} and call[i].kind == nkHiddenAddr:
call[i].firstSon
if typ[i].kind in {tyVar} and child.kind == nkHiddenAddr:
child.firstSon
else:
call[i]
child
if not param.typ.isCompileTimeOnly and (param.kind != nkBracketExpr or param.typ.kind in
{tyRef, tyPtr, tyUncheckedArray, tyArray, tyOpenArray,
tyVarargs, tySequence, tyString, tyCstring, tyTuple}):
@@ -574,10 +574,10 @@ proc genReturnStmt(p: BProc, t: PNode) =
p.s(cpsStmts).addGoto("BeforeRet_")
proc genGotoForCase(p: BProc; caseStmt: PNode) =
for i in 1..<caseStmt.len:
for child in sonsFrom(caseStmt, 1):
var scope: ScopeBuilder
startSimpleBlock(p, scope)
let it = caseStmt[i]
let it = child
for j in 0..<it.len-1:
if it[j].kind == nkRange:
localError(p.config, it.info, "range notation not available for computed goto")
@@ -646,10 +646,10 @@ proc genComputedGoto(p: BProc; n: PNode) =
# first goto:
p.s(cpsStmts).addComputedGoto(subscript(tmp, ra))
for i in 1..<caseStmt.len:
for child in sonsFrom(caseStmt, 1):
var scope: ScopeBuilder
startSimpleBlock(p, scope)
let it = caseStmt[i]
let it = child
for j in 0..<it.len-1:
if it[j].kind == nkRange:
localError(p.config, it.info, "range notation not available for computed goto")
@@ -992,18 +992,18 @@ proc genCaseStringBranch(p: BProc, b: PNode, e: TLoc, labl: TLabel,
proc genStringCase(p: BProc, t: PNode, stringKind: TTypeKind, d: var TLoc) =
# count how many constant strings there are in the case:
var strings = 0
for i in 1..<t.len:
if t[i].kind == nkOfBranch: inc(strings, t[i].len - 1)
for it in sonsFrom(t, 1):
if it.kind == nkOfBranch: inc(strings, it.len - 1)
if strings > stringCaseThreshold:
var bitMask = math.nextPowerOfTwo(strings) - 1
var branches: seq[Builder]
newSeq(branches, bitMask + 1)
var a: TLoc = initLocExpr(p, t.firstSon) # first pass: generate ifs+goto:
var labId = p.labels
for i in 1..<t.len:
for it in sonsFrom(t, 1):
inc(p.labels)
if t[i].kind == nkOfBranch:
genCaseStringBranch(p, t[i], a, "LA" & rope(p.labels) & "_",
if it.kind == nkOfBranch:
genCaseStringBranch(p, it, a, "LA" & rope(p.labels) & "_",
stringKind, branches)
else:
# else statement: nothing to do yet
@@ -1048,8 +1048,7 @@ proc branchHasTooBigRange(b: PNode): bool =
proc ifSwitchSplitPoint(p: BProc, n: PNode): int =
result = 0
for i in 1..<n.len:
var branch = n[i]
for i, branch in isons(n, 1):
var stmtBlock = lastSon(branch)
if stmtBlock.stmtsContainPragma(wLinearScanEnd):
result = i
@@ -1300,39 +1299,39 @@ proc genTryCpp(p: BProc, t: PNode, d: var TLoc) =
var catchAllPresent = false
incl p.flags, noSafePoints # mark as not needing 'popCurrentException'
if hasImportedCppExceptions:
for i in 1..<t.len:
if t[i].kind != nkExceptBranch: break
for it in sonsFrom(t, 1):
if it.kind != nkExceptBranch: break
# bug #4230: avoid false sharing between branches:
if d.k == locTemp and isEmptyType(t.typ): d.k = locNone
if t[i].len == 1:
if it.len == 1:
# general except section:
startBlockWith(p):
p.s(cpsStmts).add("catch (...) {\n")
genExceptBranchBody(t[i].firstSon)
genExceptBranchBody(it.firstSon)
endBlockWith(p):
p.s(cpsStmts).add("}\n")
catchAllPresent = true
else:
for j in 0..<t[i].len-1:
var typeNode = t[i][j]
if t[i][j].isInfixAs():
typeNode = t[i][j][1]
for j in 0..<it.len-1:
var typeNode = it[j]
if it[j].isInfixAs():
typeNode = it[j][1]
if isImportedException(typeNode.typ, p.config):
let exvar = t[i][j][2] # ex1 in `except ExceptType as ex1:`
let exvar = it[j][2] # ex1 in `except ExceptType as ex1:`
fillLocalName(p, exvar.sym)
backendEnsureMutable exvar.sym
fillLoc(exvar.sym.locImpl, locTemp, exvar, OnStack)
startBlockWith(p):
lineCg(p, cpsStmts, "catch ($1& $2) {$n", [getTypeDesc(p.module, typeNode.typ), rdLoc(exvar.sym.loc)])
genExceptBranchBody(t[i][^1]) # exception handler body will duplicated for every type
genExceptBranchBody(it[^1]) # exception handler body will duplicated for every type
endBlockWith(p):
p.s(cpsStmts).add("}\n")
elif isImportedException(typeNode.typ, p.config):
startBlockWith(p):
lineCg(p, cpsStmts, "catch ($1&) {$n", [getTypeDesc(p.module, t[i][j].typ)])
genExceptBranchBody(t[i][^1]) # exception handler body will duplicated for every type
lineCg(p, cpsStmts, "catch ($1&) {$n", [getTypeDesc(p.module, it[j].typ)])
genExceptBranchBody(it[^1]) # exception handler body will duplicated for every type
endBlockWith(p):
p.s(cpsStmts).add("}\n")
@@ -1360,8 +1359,8 @@ proc bodyCanRaise(p: BProc; n: PNode): bool =
result = canRaiseDisp(p, n.firstSon)
if not result:
# also check the arguments:
for i in 1 ..< n.len:
if bodyCanRaise(p, n[i]): return true
for it in sonsFrom(n, 1):
if bodyCanRaise(p, it): return true
of nkRaiseStmt:
result = true
of nkTypeSection, nkProcDef, nkConverterDef, nkMethodDef, nkIteratorDef,
@@ -1710,8 +1709,7 @@ proc genAsmOrEmitStmt(p: BProc, t: PNode, isAsmStmt=false; result: var Rope) =
if isAsmStmt: 1 # first son is pragmas
else: 0
for i in offset..<t.len:
let it = t[i]
for it in sonsFrom(t, offset):
case it.kind
of nkStrLit..nkTripleStrLit:
res.add(it.strVal)

View File

@@ -31,8 +31,8 @@ proc genTraverseProc(c: TTraversalClosure, accessor: Rope, n: PNode;
if n == nil: return
case n.kind
of nkRecList:
for i in 0..<n.len:
genTraverseProc(c, accessor, n[i], typ)
for it in sons(n):
genTraverseProc(c, accessor, it, typ)
of nkRecCase:
if (n[0].kind != nkSym): internalError(c.p.config, n.info, "genTraverseProc")
var p = c.p
@@ -42,8 +42,7 @@ proc genTraverseProc(c: TTraversalClosure, accessor: Rope, n: PNode;
internalError(c.p.config, n.info, "genTraverseProc()")
let discField = dotField(accessor, disc.loc.snippet)
p.s(cpsStmts).addSwitchStmt(discField):
for i in 1..<n.len:
let branch = n[i]
for branch in sonsFrom(n, 1):
assert branch.kind in {nkOfBranch, nkElse}
var caseBuilder: SwitchCaseBuilder
p.s(cpsStmts).addSwitchCase(caseBuilder):

View File

@@ -611,9 +611,9 @@ proc genMemberProcParams(m: BModule; prc: PSym, superCall, rettype, name, params
types.add getTypeDescWeak(m, this.typ, check, dkParam)
let firstParam = if isCtor: 1 else: 2
for i in firstParam..<t.n.len:
if t.n[i].kind != nkSym: internalError(m.config, t.n.info, "genMemberProcParams")
var param = t.n[i].sym
for it in sonsFrom(t.n, firstParam):
if it.kind != nkSym: internalError(m.config, t.n.info, "genMemberProcParams")
var param = it.sym
var descKind = dkParam
if optByRef in param.options:
if param.typ.kind == tyGenericInst:
@@ -623,7 +623,7 @@ proc genMemberProcParams(m: BModule; prc: PSym, superCall, rettype, name, params
var typ, name: string
backendEnsureMutable param
fillParamName(m, param)
fillLoc(param.locImpl, locParam, t.n[i],
fillLoc(param.locImpl, locParam, it,
param.paramStorageLoc)
if ccgIntroducedPtr(m.config, param, t.returnType) and descKind == dkParam:
typ = getTypeDescWeak(m, param.typ, check, descKind) & "*"
@@ -668,9 +668,9 @@ proc genProcParams(m: BModule; t: PType, rettype: var Rope, params: var Builder,
rettype = getTypeDescWeak(m, t.returnType, check, dkResult)
var paramBuilder: ProcParamBuilder
params.addProcParams(paramBuilder):
for i in 1..<t.n.len:
if t.n[i].kind != nkSym: internalError(m.config, t.n.info, "genProcParams")
var param = t.n[i].sym
for child in sonsFrom(t.n, 1):
if child.kind != nkSym: internalError(m.config, t.n.info, "genProcParams")
var param = child.sym
# The hidden closure environment param (`:envP`) is not a real C parameter:
# the environment is passed via the trailing `ClE_0` (added below) and
# `closureSetup` materialises `:envP` as a local cast of it. In a from-source
@@ -692,7 +692,7 @@ proc genProcParams(m: BModule; t: PType, rettype: var Rope, params: var Builder,
if isCompileTimeOnly(param.typ): continue
backendEnsureMutable param
fillParamName(m, param)
fillLoc(param.locImpl, locParam, t.n[i],
fillLoc(param.locImpl, locParam, child,
param.paramStorageLoc)
if isClosureEnv: continue # name/loc filled, but not part of the C signature
var typ: Rope
@@ -775,10 +775,10 @@ proc genRecordFieldsAux(m: BModule; n: PNode,
# prefix mangled name with "_U" to avoid clashes with other field names,
# since identifiers are not allowed to start with '_'
var unionBody = newBuilder("")
for i in 1..<n.len:
case n[i].kind
for i, it in isons(n, 1):
case it.kind
of nkOfBranch, nkElse:
let k = lastSon(n[i])
let k = lastSon(it)
if k.kind != nkSym:
let structName = "_" & mangleRecFieldName(m, n.firstSon.sym) & "_" & $i
var a = newBuilder("")
@@ -1552,8 +1552,7 @@ proc genObjectFields(m: BModule; typ, origType: PType, n: PNode, expr: Rope;
else:
m.s[cfsData].addArrayVar(kind = Local, name = tmp,
elementType = ptrType("TNimNode"), len = toInt(L)+1)
for i in 1..<n.len:
var b = n[i] # branch
for b in sonsFrom(n, 1):
var tmp2 = getNimNode(m)
genObjectFields(m, typ, origType, lastSon(b), tmp2, info)
case b.kind

View File

@@ -22,13 +22,13 @@ proc getPragmaStmt*(n: PNode, w: TSpecialWord): PNode =
case n.kind
of nkStmtList:
result = nil
for i in 0..<n.len:
result = getPragmaStmt(n[i], w)
for it in sons(n):
result = getPragmaStmt(it, w)
if result != nil: break
of nkPragma:
result = nil
for i in 0..<n.len:
if whichPragma(n[i]) == w: return n[i]
for it in sons(n):
if whichPragma(it) == w: return it
else:
result = nil
@@ -113,20 +113,12 @@ proc encodeName*(name: string): string =
proc makeUnique(m: BModule; s: PSym, name: string = ""): string =
result = if name == "": s.name.s else: name
# keep backend-minted ids out of the `_u` namespace; their item counter
# restarts at 0 and would collide with loaded symbols' ids
# restarts at 0 and would collide with loaded symbols' ids. Which integer
# identifies such a symbol is decided ONCE, in `astdef.backendMintedDisamb`,
# shared with `mangleProcNameExt` and `ast2nif.toNifSymName`.
if s.itemId.isBackendMinted:
result.add "_c"
if (s.disamb and HookDisambBit) != 0'i32:
# A backend-minted sym whose `disamb` is content-derived (setHookDisamb gave
# it HookDisambBit) — e.g. the `rttiDestroy` wrapper. Its `itemId.item` is a
# PER-PROCESS backend counter, so using it makes the C name diverge across
# the emit-everywhere processes: the type's RTTI table (emit-everywhere,
# merge-deduped) ends up referencing one process's `_c<item>` while the
# wrapper is defined with another's -> undefined at link (`rttiDestroy_c23`).
# The content-derived disamb is stable across processes, so use it.
result.add $s.disamb
else:
result.add $s.itemId.item
result.add $backendMintedDisamb(s)
else:
result.add "_u"
# Mirror `mangleProcNameExt`: use the per-(module,name) `disamb`, NOT

View File

@@ -16,7 +16,7 @@ import
rodutils, renderer, cgendata, aliases,
lowerings, lineinfos, pathutils, transf,
injectdestructors, astmsgs, modulepaths, pushpoppragmas,
mangleutils, cbuilderbase, modulegraphs
mangleutils, cbuilderbase, modulegraphs, bnode
from expanddefaults import caseObjDefaultBranch
from ast2nif import globalName, toNifFilename, icNifTypeName
@@ -113,57 +113,144 @@ proc icNifName(m: BModule; t: PType): string =
result = ""
proc signatureHasMetaType*(t: PType; depth: int = 0): bool =
## Whether a routine signature mentions a compile-time/meta element type
## (`typed`/`untyped` — e.g. `echo`'s `varargs[typed]` — typedesc, static,
## generic param). Such routines are expanded at their call sites and never
## emitted standalone, so the per-module owned-routine seeding must skip them
## (`getTypeDescAux(tyTyped)` otherwise). `tfHasMeta` alone misses the varargs
## element case, hence the explicit scan.
result = false
if t == nil or depth > 8: return false
if t.kind == tyGenericBody:
# The uninstantiated template carried as a `tyGenericInst`'s first child
# always mentions its `tyGenericParam` placeholders, but the instance
# itself is fully concrete (e.g. `var CountTable[SigHash]`). Descending
# here would wrongly flag every routine with a generic-instance parameter
# as meta and drop it from the owned-routine seeding -> undefined symbols
# at link (its only definer never emits it).
return false
if t.kind == tyStatic:
# A RESOLVED static value (the `256` in `MDigest[256]`, the `N` in
# `HashList[T, N]`, …) is carried as a `tyStatic` node inside the otherwise
# fully-concrete `tyGenericInst`, but it is NOT meta: the routine is a normal
# runtime routine the owner must emit. Only an UNRESOLVED `static T` parameter
# (no bound value, `t.n == nil`) is meta. Without this, every routine whose
# signature touches a `static`-parameterized generic instance (the bulk of
# the SSZ/`MDigest` API) is dropped from the owned-routine seeding and ends up
# an undefined reference at link (mirrors the tyGenericBody case above).
return t.n == nil
if t.kind in {tyTyped, tyUntyped, tyTypeDesc, tyGenericParam,
tyAnything, tyFromExpr, tyError}:
return true
for k in t.kids:
if signatureHasMetaType(k, depth + 1): return true
proc ownsRuntimeRoutine*(s: PSym; modPos: int): bool =
## A concrete, non-generic, runtime routine with a real body, OWNED by the
## module at `modPos`. Shared by the `cg` stage's owned-routine seeding (so a
## routine called only from other modules is still emitted by somebody) and
## the `lower` stage's owned-routine enumeration, so both stages see exactly
## the same set. The exclusions:
## - nested/closure procs (owner is a proc, not a module): emitted via their
## enclosing routine's lambda-lifting, never standalone;
## - generic instances (`sfFromGeneric`): emitted by demand, deduped by merge;
## - `importc`/`compileTime`/`error`/forward sentinels and meta signatures:
## not real codegen targets.
## - method DISPATCHERS (`sfDispatcher`): their bodies are (re)synthesized into
## the main TU by `emitMethodDispatchers`/`generateIfMethodDispatchers`, never
## per module. A dispatcher is a `copySym` clone of the method that shares the
## method's body sub-tree (incl. its closure iterator); transforming it here
## would lambda-lift that SHARED iterator a SECOND time under a different owner
## identity, baking a conflicting `up` field → "up references do not agree"
## (the divergence is impossible in non-IC, where the dispatcher body is empty
## at lift time). So a dispatcher is never an owned runtime routine.
## A `{.closure.}` iterator IS a standalone runtime routine (unlike an inline
## iterator, which is expanded at each call site) and must be emitted by its
## owner — else a cross-module `for` over it links to nothing.
##
## Generic INSTANCES (`sfFromGeneric`) are NEVER an owned runtime routine — not
## in `cg` and not in the `lower` stage. They are demanded by the backend's
## emit-everywhere path and deduped by `merge` (content C name); the frontend
## materialises them through the `(offer)` mechanism. The `lower` stage must
## not transform an instance: a not-fully-concrete instance (a closure factory
## over a `static` param, or a `$`/`=` op instance whose body resolves only at
## its further-specialised use sites) still carries unresolved overload choices
## and crashes `transformBody` (empty-`namePos` lambda, nil-typed const-fold).
s.itemId.module == modPos and
(s.kind in {skProc, skFunc, skConverter, skMethod} or
(s.kind == skIterator and s.typ != nil and s.typ.callConv == ccClosure)) and
s.skipGenericOwner != nil and s.skipGenericOwner.kind == skModule and
s.magic == mNone and
sfFromGeneric notin s.flags and
sfDispatcher notin s.flags and
{sfForward, sfImportc, sfCompileTime, sfError} * s.flags == {} and
s.typ != nil and not signatureHasMetaType(s.typ) and
s.ast != nil and s.ast.safeLen > bodyPos and
s.ast[genericParamsPos].kind == nkEmpty
# NOTE: an `nkEmpty` body is NOT a disqualifier. A concrete, owned, non-
# forward/-importc/-magic routine whose body folds to nothing is still a real
# definition the owner must emit (`void f(void){}`), exactly as whole-program
# cgen does — else a cross-module caller links to nothing. This bites e.g.
# Nimbus' `extras.incInternalErrors`, a plain `proc` whose sole statement is a
# metrics-counter `.inc()` that the `metrics` library expands to a no-op when
# the importing tool (ncli) builds with `-u:metrics`; the body is then a bare
# `nkEmpty`, but `state_transition_epoch` still calls it. Forward declarations
# (the other empty-body case) carry `sfForward` and are excluded above.
proc bodyIsSeededByItsOwner(prc: PSym): bool =
## Whether SOME module's `cg` is guaranteed to emit `prc`'s body on its own,
## without this TU asking for it. There are exactly two seeders in the
## per-module backend, and this enumerates them:
##
## * `nifbackend.generateCodeForModule` walks its module's index and
## `requestProcDef`s every `ownsRuntimeRoutine` — the SAME predicate the
## `lower` stage uses to decide what it transforms into that module's
## `.t.bif`. So asking it about `prc`'s OWN defining module answers
## "will that module's cg seed this?".
## * `nifbackend.emitMethodDispatchers` synthesizes every method dispatcher
## into the MAIN TU. A dispatcher is a `copySym` clone that no module's
## index enumerates, so the first rule cannot see it.
##
## Anything else — a generic instance, a synthesized hook, a nested routine
## (emitted as part of its enclosing routine's lambda-lifted body), an inline
## iterator (expanded at each call site) — is seeded by nobody. Those are
## emitted by EVERY demander and `merge` keeps one per content-addressed C
## name. That is the single default, and it is the safe direction: emitting a
## body twice costs a merge dedup, while emitting it nowhere is a link error.
##
## A BACKEND-MINTED routine (a hook or nested proc that lambda-lifting /
## `injectDestructorCalls` created during `lower`) exists in no module's semmed
## NIF: it is written into the `.t.bif` of every module that references it,
## re-homed there with `@bk`. Its `itemId.module` therefore names whichever
## `.t.bif` it was read from rather than a module that seeds it, so it must not
## be routed through the ownership question at all.
if isBackendMinted(prc.itemId): return false
result = sfDispatcher in prc.flags or
ownsRuntimeRoutine(prc, prc.itemId.module)
proc emitsBodyInThisModule(m: BModule, prc: PSym): bool =
## Per-module backend codegen is concerned with ONE module: it emits the
## bodies of the routines that module OWNS (its own top-level defs) and only
## *prototypes* a routine owned by another module — that routine's body is
## emitted by its own module's `cg` process, and the merge stage's DCE prunes
## whatever ends up globally dead. The funnel where the main module re-emitted
## its entire transitive closure (≈1.8 GB, a 56 MB `.c.nif`) is exactly this
## rule being absent.
## bodies whose owner is this module and only *prototypes* a body some other
## module's `cg` process is going to emit. The funnel where the main module
## re-emitted its entire transitive closure (~1.8 GB, a 56 MB `.c.nif`) is
## exactly this rule being absent.
##
## Generic instances and synthesized hooks (`=destroy`, `$`, …) have no single
## owning-module top-level — they are minted on demand — so each demander emits
## them and the merge stage deduplicates by their content-addressed C name.
##
## A NESTED routine is not emitted on its own: it is lambda-lifted and emitted
## as part of its ENCLOSING routine's body, into the same TU. So the decision
## must follow the OUTERMOST enclosing routine (the one directly under the
## module — `skipGenericOwner` stops at a generic *instance*, not its
## originating generic), never the nested symbol's own identity. Otherwise a
## nested proc whose enclosing is a generic instance (content-addressed,
## emitted by every demander) — e.g. nim-serialization's per-field `readField`
## inside the `makeFieldReadersTable[R,W]` instance, whose address fills the
## returned table — is gated out (its own `itemId.module` is the minting module
## and its disamb is a plain counter), so the enclosing's lift degrades it to a
## prototype and its body lands in no TU → undefined at link.
## The decision is a lookup against `bodyIsSeededByItsOwner`, i.e. against the
## very predicates that drive the seeding, rather than a re-derivation from
## symbol ancestry. Re-derivation is what made this function a five-clause
## tower and the source of a run of "emitted by nobody" / "two hooks on one C
## name" bugs: the walk answered a question about who WILL emit by inspecting
## who DECLARED, and the two drifted apart for every symbol the backend mints.
if not (m.config.cmd == cmdNifC and m.config.icBackendStage == "cg"):
return true
# The symbol may ITSELF be content-addressed (a synthesized hook or a generic
# instance carries `Hook/InstanceDisambBit` on its OWN `disamb`): then it has no
# single owning module and every demander emits it (merge dedups by C name),
# regardless of what it is nested under. This must be checked on `prc` directly,
# not on `top`: a `=destroy`/`=sink` lifted while compiling some enclosing proc
# (e.g. system's `isZeroMemory` destroying a `ptr array`) has that PROC as its
# `skipGenericOwner`, so `top` walks up to a plain routine whose own disamb has
# no bit — gating the hook to that routine's owner module, which mints it
# on demand and emits it nowhere → undefined at link.
if (prc.disamb and (InstanceDisambBit or HookDisambBit)) != 0'i32:
return true
var top = prc
while top.skipGenericOwner != nil and top.skipGenericOwner.kind != skModule:
top = top.skipGenericOwner
result = top.itemId.module == m.module.position or
(top.disamb and (InstanceDisambBit or HookDisambBit)) != 0'i32 or
# An INLINE iterator has no standalone body — it is expanded at each
# call site — so it is materialized in every module that iterates over
# it, never in its owner. A proc nested in one (e.g. std/uri's
# `parseData` inside `iterator decodeQuery`) is lambda-lifted into each
# of those consumer TUs and must be emitted there (its stable
# owner-suffixed name + `'u'` flag let the merge stage keep one); gating
# it to the iterator's owner module leaves it in no TU → undefined.
(top.kind == skIterator and top.typ != nil and
top.typ.callConv != ccClosure)
if not bodyIsSeededByItsOwner(prc):
# Seeded by nobody: every demander emits it, merge keeps one.
result = true
elif sfDispatcher in prc.flags:
result = sfMainModule in m.module.flags
else:
result = prc.itemId.module == m.module.position
proc initLoc(k: TLocKind, lode: PNode, s: TStorageLoc, flags: TLocFlags = {}): TLoc =
result = TLoc(k: k, storage: s, lode: lode,
@@ -1107,8 +1194,11 @@ proc symInDynamicLib(m: BModule, sym: PSym) =
var a: TLoc = initLocExpr(m.initProc, n.firstSon)
let callee = rdLoc(a)
var params: seq[Snippet] = @[]
for i in 1..<n.len-1:
a = initLocExpr(m.initProc, n[i])
var remaining = n.len - 2 # children 1 ..< len-1
for it in sonsFrom(n, 1):
if remaining <= 0: break
dec remaining
a = initLocExpr(m.initProc, it)
params.add(rdLoc(a))
params.add(makeCString($extname))
template load(builder: var Builder) =
@@ -1254,7 +1344,7 @@ const harmless = {nkConstSection, nkTypeSection, nkEmpty, nkCommentStmt, nkTempl
nkMacroDef, nkMixinStmt, nkBindStmt, nkFormalParams} +
declarativeDefs
proc containsResult(n: PNode): bool =
proc containsResult(n: BNode): bool =
result = false
case n.kind
of succ(nkEmpty)..pred(nkSym), succ(nkSym)..nkNilLit, harmless:
@@ -1290,7 +1380,10 @@ proc easyResultAsgn(n: PNode): PNode =
type
InitResultEnum = enum Unknown, InitSkippable, InitRequired
proc allPathsAsgnResult(p: BProc; n: PNode): InitResultEnum =
proc allPathsAsgnResult(p: BProc; n: BNode): InitResultEnum =
## Migrated to `BNode` (see bnode.nim). With `newIcBackend` off this is
## `PNode` and nothing changes; with it on, this body is where the Cursor
## vocabulary has to exist, and its `{.error.}` stubs name what is missing.
# Exceptions coming from calls don't have not be considered here:
#
# proc bar(): string = raise newException(...)
@@ -1357,8 +1450,7 @@ proc allPathsAsgnResult(p: BProc; n: PNode): InitResultEnum =
result = InitSkippable
var exhaustive = skipTypes(n.firstSon.typ,
abstractVarRange-{tyTypeDesc}).kind notin {tyFloat..tyFloat128, tyString, tyCstring}
for i in 1..<n.len:
let it = n[i]
for it in sonsFrom(n, 1):
allPathsInBranch(it.lastSon)
if it.kind == nkElse: exhaustive = true
if not exhaustive: result = Unknown
@@ -1390,11 +1482,11 @@ proc allPathsAsgnResult(p: BProc; n: PNode): InitResultEnum =
# is 'finally: result = x'
result = InitSkippable
allPathsInBranch(n.firstSon)
for i in 1..<n.len:
if n[i].kind == nkFinally:
result = allPathsAsgnResult(p, n[i].lastSon)
for it in sonsFrom(n, 1):
if it.kind == nkFinally:
result = allPathsAsgnResult(p, it.lastSon)
else:
allPathsInBranch(n[i].lastSon)
allPathsInBranch(it.lastSon)
of nkCallKinds:
if canRaiseDisp(p, n.firstSon) or
(n.firstSon.kind == nkSym and sfNoReturn in n.firstSon.sym.flags):
@@ -1548,8 +1640,8 @@ proc genProcLvl3*(m: BModule, prc: PSym) =
backendEnsureMutable res
res.locImpl.storage = OnUnknown
for i in 1..<prc.typ.n.len:
let param = prc.typ.n[i].sym
for paramNode in sonsFrom(prc.typ.n, 1):
let param = paramNode.sym
if param.typ.isCompileTimeOnly: continue
if prc.typ.callConv == ccClosure and param.name.s == ":envP":
# The hidden closure-env param is materialised by `closureSetup`, never a
@@ -2842,6 +2934,23 @@ proc genModuleCode(m: BModule; cf: var Cfile): string =
proc registerModuleCode(m: BModule; cf: var Cfile; code: string) =
## Second half of `writeModule`: writes the .c file if it changed and
## registers it for compilation.
##
## NOT under the per-module backend's `cg` stage. There the `.c` belongs to
## `emit`, which renders it from the `.c.nif` using the GLOBAL merge decision;
## `cg` can only filter by the liveness its own process can see, so writing
## here puts a second, differently-filtered `.c` at the very path `emit`
## declares as its nifmake output. Two stages then claim one output, and the
## `.c` ends up newer than `emit`'s own `.c.nif` input — so any build in which
## `emit` is not forced to run anyway keeps `cg`'s unfiltered text and hands it
## to the linker ("multiple definition of eqdup__…").
##
## Today nothing surfaces this: `merge` rewrites the decision file on every
## run and every `emit` lists it as an input, so all of them re-fire and
## overwrite the stray file. That makes the fire-all load-bearing rather than
## the "insurance" it is documented as, and it silently blocks making the
## decision content-stable. `cg`'s product is the `.c.nif`; the compile
## registration is likewise the `link` stage's job.
if m.config.cmd == cmdNifC and m.config.icBackendStage == "cg": return
if code != "" or m.config.symbolFiles != disabledSf:
when hasTinyCBackend:
if m.config.cmd == cmdTcc:

View File

@@ -11,8 +11,7 @@
## for details. Note this is a first implementation and only the "Concept matching"
## section has been implemented.
import ast, semdata, lookups, lineinfos, idents, msgs, renderer, types,
layeredtable, semtypinst
import ast, semdata, lookups, lineinfos, idents, msgs, renderer, types, layeredtable
import std/sets
@@ -72,8 +71,7 @@ proc semConceptDeclaration*(c: PContext; n: PNode): PNode =
type
MatchFlags* = enum
mfDontBind # Do not export bindings from the concept match
mfBindGenericParam # Export inferred invocation parameters despite mfDontBind
mfDontBind # Do not bind generic parameters
mfCheckGeneric # formal <- formal comparison as opposed to formal <- operand
ConceptTypePair = tuple[conceptId, typeId: ItemId]
@@ -575,17 +573,7 @@ proc conceptMatchNode(c: PContext; n: PNode; m: var MatchCon): bool =
# error was reported earlier.
result = false
proc resolvedBinding(c: PContext; t: PType; m: MatchCon): PType =
## An inferred concept parameter can refer to an implementation-local
## generic parameter, for example `Elem[Impl.T]`. Resolve it while the
## matcher's private bindings (`Impl.T -> int`) are still available.
if t.containsUnresolvedType:
prepareMetatypeForSigmatch(c, m.bindings, m.concpt.sym.info, t)
else:
t
proc fixBindings(c: PContext; bindings: var LayeredIdTable; concpt: PType;
invocation: PType; m: var MatchCon) =
proc fixBindings(bindings: var LayeredIdTable; concpt: PType; invocation: PType; m: var MatchCon) =
# invocation != nil means we have a non-atomic concept:
if invocation != nil and invocation.kind == tyGenericInvocation:
assert concpt.sym.typ.kind == tyGenericBody
@@ -597,9 +585,8 @@ proc fixBindings(c: PContext; bindings: var LayeredIdTable; concpt: PType;
continue
let found = m.bindings.lookup(thisSym)
if found != nil:
let resolved = resolvedBinding(c, found, m)
when logBindings: echo "Invocation bind: ", thisSym, " ", resolved
bindings.put(thisSym, resolved)
when logBindings: echo "Invocation bind: ", thisSym, " ", found
bindings.put(thisSym, found)
# bind even more generic parameters
let genBody = invocation.base
@@ -615,20 +602,6 @@ proc fixBindings(c: PContext; bindings: var LayeredIdTable; concpt: PType;
bindings.put(invocation[i], boundV)
bindings.put(concpt, m.potentialImplementation)
proc fixConstraintBindings(c: PContext; bindings: var LayeredIdTable;
invocation: PType; m: MatchCon) =
## Propagates only the dependent parameters of a concept constraint. The
## concept itself and its private matcher bindings must remain unbound so
## that independent constraints using the same concept don't get coupled.
if invocation != nil and invocation.kind == tyGenericInvocation:
let genBody = invocation.base
assert genBody.kind == tyGenericBody
for i in FirstGenericParamAt ..< invocation.kidsLen:
if lookup(bindings, invocation[i]) == nil:
let boundValue = m.bindings.lookup(genBody[i - 1])
if boundValue != nil:
bindings.put(invocation[i], resolvedBinding(c, boundValue, m))
proc processConcept(c: PContext; concpt, invocation: PType, bindings: var LayeredIdTable; m: var MatchCon): bool =
m.bindings = m.bindings.newTypeMapLayer()
if invocation != nil and invocation.kind == tyGenericInst:
@@ -638,11 +611,8 @@ proc processConcept(c: PContext; concpt, invocation: PType, bindings: var Layere
if invocation[i].kind != tyVoid:
bindParam(c, m, genericBody[i-1], invocation[i])
result = conceptMatchNode(c, concpt.conceptBody, m)
if result:
if mfDontBind notin m.flags:
fixBindings(c, bindings, concpt, invocation, m)
elif mfBindGenericParam in m.flags:
fixConstraintBindings(c, bindings, invocation, m)
if result and mfDontBind notin m.flags:
fixBindings(bindings, concpt, invocation, m)
proc conceptMatch*(c: PContext; concpt, arg: PType; bindings: var LayeredIdTable; invocation: PType, flags: set[MatchFlags] = {}): bool =
## Entry point from sigmatch. 'concpt' is the concept we try to match (here still a PType but

View File

@@ -18,6 +18,7 @@ import options, msgs, lineinfos, pathutils, condsyms,
import "../dist/nimony/src/lib" / [nifstreams, bitabs, nifreader, nifbuilder]
import icmodnames
import icnifcore
from ic/replayer import BackendActionsExt
type
FilePair = object
@@ -63,6 +64,13 @@ proc depsFile(c: DepContext; f: FilePair): string =
proc parsedFile(c: DepContext; f: FilePair): string =
getNimcacheDir(c.config).string / f.modname & ".p.nif"
proc parsedDepsFile(c: DepContext; f: FilePair): string =
## The deps sidecar `nifler parse --deps <src> <out>.p.nif` actually writes: it
## appends `.deps.nif` to the OUTPUT path, giving `<mod>.p.deps.nif`. Not to be
## confused with `depsFile` (`<mod>.deps.nif`), which the driver's own
## `nifler deps` pre-scan writes.
parsedFile(c, f).changeFileExt("") & ".deps.nif"
proc semmedFile(c: DepContext; f: FilePair): string =
getNimcacheDir(c.config).string / f.modname & ".s.bif"
@@ -803,18 +811,35 @@ proc pruneDeadSpeculative(c: var DepContext) =
for d in c.nodes[v].deps:
if not dead[d] and not alive[d]: stack.add d
# Drop the scan artifacts of a module that just left the graph, so an
# edit-accumulated cache does not differ from a clean one for no reason
# (`tests/ic/tdead_when_import` pins that). Re-running nifler if it ever comes
# back costs a single parse.
#
# But a FILE can belong to several nodes, and only the NODE is dead.
# `lib/system/inclrtl.nim` is `include`d by dozens of live stdlib modules and
# also sits in the file set of a dead-speculative one; a clean build therefore
# has its `.p.nif`, and deleting it here does not tidy the cache, it corrupts
# it. The consequences compound: the missing output re-fires that file's
# `nifler` rule, which rewrites the parsed file with a fresh mtime, which
# re-fires every `nim_m` rule listing it as an input — 16 full module re-sems
# (system, os, times, strutils, macros, unicode, ...) on every warm build, for
# ever, because the scanner is stateless and rediscovers the dead node each
# run. Measured on a 219-module program: an 11 s NO-OP build. So delete only
# what no live node claims.
var liveFiles = initHashSet[string]()
for i in 0 ..< n:
if alive[i]:
for f in c.nodes[i].files: liveFiles.incl f.nimFile
var cascaded = 0
for i in 0 ..< n:
if not alive[i]:
# Drop the scan artifacts of a module that just left the graph. `nifler`
# ran on it during `traverseDeps` (that is how we learned it cannot
# build), and leaving its `.p.nif`/`.deps.nif` behind makes an
# edit-accumulated cache differ from a clean one for no reason. Re-running
# nifler if it ever comes back costs a single parse.
for f in c.nodes[i].files:
if f.nimFile in liveFiles: continue
removeFile(c.parsedFile(f))
removeFile(c.depsFile(f))
removeFile(c.parsedFile(f).changeFileExt("") & ".deps.nif")
removeFile(c.parsedDepsFile(f))
if c.nodes[i].missingImport.len > 0:
rawMessage(c.config, hintSuccess,
"ic: skipping " & c.nodes[i].files[0].nimFile &
@@ -1102,8 +1127,13 @@ proc generateFrontendBuildFile(c: DepContext; forwardedArgs: seq[string]): strin
b.addTree "output"
b.addStrLit parsed
b.endTree()
# The deps sidecar this command really produces is `<mod>.p.deps.nif`,
# not `<mod>.deps.nif` (which only the driver's `nifler deps` pre-scan
# writes). Declaring the latter made the rule permanently stale — a
# missing output is nifmake's strongest rebuild trigger — for every
# module the pre-scan does not also cover.
b.addTree "output"
b.addStrLit c.depsFile(pair)
b.addStrLit c.parsedDepsFile(pair)
b.endTree()
b.endTree()
@@ -1332,6 +1362,8 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
if fileExists(cnifFiles[i]) or fileExists(cFiles[i]): prunedStale = true
removeFile(cnifFiles[i])
removeFile(cFiles[i])
removeFile(cFiles[i] & ".stamp")
removeFile(cFiles[i] & BackendActionsExt)
# The merge decision is a pure function of the set of `.c.nif`s present; if we
# just removed an over-approximated module's artifacts, a decision computed
# while they were present is stale — it can name a now-absent module as a
@@ -1429,6 +1461,10 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
if c.nodes[j].id != 0 and live[j]:
inputStr cnifFiles[j]
outputStr cnifFiles[i]
# The module's C compile/link directives (`{.passL.}` etc.), recorded so the
# `link` stage recovers them without loading the module graph. See
# `replayer.writeBackendActions`.
outputStr cFiles[i] & BackendActionsExt
b.endTree()
# merge: read the live modules' `.c.nif`, write the ownership/liveness
@@ -1473,6 +1509,10 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
inputStr cnifFiles[i]
inputStr mergeFile
outputStr cFiles[i]
# The freshness proof for this rule; see nifbackend.generateEmitStage. The
# `.c` alone cannot serve: it is written OnlyIfChanged, so a rule that ran
# and produced identical bytes looks exactly like a rule that never ran.
outputStr cFiles[i] & ".stamp"
b.endTree()
# link: compile + link every emitted `.c` in one process.
@@ -1486,7 +1526,9 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
# path splits back into outDir+outFile in the child).
b.addStrLit "--out:" & exeFile
for i in 0 ..< c.nodes.len:
if live[i]: inputStr cFiles[i]
if live[i]:
inputStr cFiles[i]
inputStr cFiles[i] & BackendActionsExt
inputStr argsFile
outputStr exeFile
b.endTree()

View File

@@ -14,11 +14,77 @@
import ".." / [ast, modulegraphs, trees, extccomp, btrees,
msgs, lineinfos, pathutils, options, cgmeth]
import std/tables
import std/[tables, os, strutils, syncio]
when defined(nimPreviewSlimSystem):
import std/assertions
const BackendActionsExt* = ".cflags"
## Sidecar written by a module's `cg` stage next to its `.c`, carrying the C
## compile/link directives that module's `{.passL.}`/`{.compile.}`/… pragmas
## recorded. See `writeBackendActions`.
proc writeBackendActions*(g: ModuleGraph; module: PSym; list: PNode;
outfile: string) =
## Serialize the backend-relevant replay actions of ONE module to `outfile`,
## one tab-separated action per line.
##
## The `link` stage used to recover these by loading the whole import closure
## as `PrecompiledModule`s and re-running `replayBackendActions` over each —
## a 3.7s whole-program graph load, per link, purely to recover a handful of
## strings and the modules' `.c` paths. The producing `cg` process already has
## them in hand, so it writes them down instead and `link` reads them back
## (`applyBackendActions`). Written unconditionally, even when empty: it is a
## declared nifmake output of the `cg` rule, and a missing output re-fires the
## rule for ever.
##
## `localpassc` needs the module's own source path, which only the writer can
## resolve, so it is baked in here as a third field.
var content = ""
if list != nil:
for n in list:
if n.kind == nkReplayAction and n.len >= 2 and
n[0].kind == nkStrLit and n[1].kind == nkStrLit:
case n[0].strVal
of "compile":
if n.len == 4 and n[2].kind == nkStrLit and n[3].kind == nkStrLit:
content.add "compile\t" & n[1].strVal & "\t" & n[2].strVal & "\t" &
n[3].strVal & "\n"
of "link", "passl", "passc", "cppdefine":
content.add n[0].strVal & "\t" & n[1].strVal & "\n"
of "localpassc":
content.add "localpassc\t" & n[1].strVal & "\t" &
toFullPathConsiderDirty(g.config, module.info.fileIndex).string & "\n"
else: discard
writeFile(outfile, content)
proc applyBackendActions*(g: ModuleGraph; infile: string) =
## Apply one module's recorded C directives (see `writeBackendActions`). The
## `link` stage's replacement for loading that module and replaying its AST.
if not fileExists(infile): return
for line in lines(infile):
if line.len == 0: continue
let f = line.split('\t')
case f[0]
of "compile":
if f.len == 4:
let cname = AbsoluteFile f[1]
var cf = Cfile(nimname: splitFile(cname).name, cname: cname,
obj: AbsoluteFile f[2],
flags: {CfileFlag.External}, customArgs: f[3])
extccomp.addExternalFileToCompile(g.config, cf)
of "link":
if f.len == 2: extccomp.addExternalFileToLink(g.config, AbsoluteFile f[1])
of "passl":
if f.len == 2: extccomp.addLinkOption(g.config, f[1])
of "passc":
if f.len == 2: extccomp.addCompileOption(g.config, f[1])
of "localpassc":
if f.len == 3: extccomp.addLocalCompileOption(g.config, f[1], AbsoluteFile f[2])
of "cppdefine":
if f.len == 2: options.cppDefine(g.config, f[1])
else: discard
proc replayStateChanges*(module: PSym; g: ModuleGraph; list: PNode) =
## `list` is an `nkStmtList` of `nkReplayAction` nodes (macro-cache puts/incs/
## adds/incls and a few pragmas) recorded for `module`. Under the NIF backend a

View File

@@ -423,20 +423,6 @@ proc genDefaultCall(t: PType; c: Con; info: TLineInfo): PNode =
result.add(newSymNode(createMagic(c.graph, c.idgen, "default", mDefault)))
result.typ = t
proc stabilizeBracketIndex(n: PNode; c: var Con; body: var PNode): PNode =
## Evaluate a side-effecting index once and return the stable access.
doAssert n.kind == nkBracketExpr and not isAtom(n[1])
let temp = newSym(skLet, getIdent(c.graph.cache, "bracketTmp"), c.idgen,
c.owner, n[1].info)
temp.typ = n[1].typ
let tempAsNode = newSymNode(temp)
body.add newTree(nkLetSection, n[1].info,
newTree(nkIdentDefs, tempAsNode,
newNodeI(nkEmpty, tempAsNode.info), n[1]))
result = copyNode(n)
result.add n[0]
result.add tempAsNode
proc destructiveMoveVar(n: PNode; c: var Con; s: var Scope): PNode =
# generate: (let tmp = v; reset(v); tmp)
if (not hasDestructor(c, n.typ)) and c.inEnsureMove == 0:
@@ -448,10 +434,6 @@ proc destructiveMoveVar(n: PNode; c: var Con; s: var Scope): PNode =
else:
result = newNodeIT(nkStmtListExpr, n.info, n.typ)
var n = n
if n.kind == nkBracketExpr and not isAtom(n[1]):
n = stabilizeBracketIndex(n, c, result)
var temp = newSym(skLet, getIdent(c.graph.cache, "blitTmp"), c.idgen, c.owner, n.info)
temp.typ = n.typ
var v = newNodeI(nkLetSection, n.info)
@@ -1173,11 +1155,24 @@ proc sameLocation*(a, b: PNode): bool =
else: false
proc genFieldAccessSideEffects(c: var Con; s: var Scope; dest, ri: PNode; flags: set[MoveOrCopyFlag] = {}): PNode =
result = newNodeI(nkStmtList, ri.info)
let newAccess = stabilizeBracketIndex(ri, c, result)
let snk = c.genSink(s, dest, newAccess, flags)
result.add snk
result.add c.genWasMoved(newAccess)
# with side effects
var temp = newSym(skLet, getIdent(c.graph.cache, "bracketTmp"), c.idgen, c.owner, ri[1].info)
temp.typ = ri[1].typ
var v = newNodeI(nkLetSection, ri[1].info)
let tempAsNode = newSymNode(temp)
var vpart = newNodeI(nkIdentDefs, tempAsNode.info, 3)
vpart[0] = tempAsNode
vpart[1] = newNodeI(nkEmpty, tempAsNode.info)
vpart[2] = ri[1]
v.add(vpart)
var newAccess = copyNode(ri)
newAccess.add ri[0]
newAccess.add tempAsNode
var snk = c.genSink(s, dest, newAccess, flags)
result = newTree(nkStmtList, v, snk, c.genWasMoved(newAccess))
proc ownsData(c: var Con; s: var Scope; orig: PNode; flags: set[MoveOrCopyFlag]): PNode =
var n = orig

View File

@@ -1787,11 +1787,9 @@ proc genVarOpenArrayArg(p: PProc, n: PNode, r: var TCompRes) =
r.res = "{base: $1, off: 0, len: ($1).length}" % [v.rdLoc]
r.kind = resExpr
proc genArg(p: PProc, n: PNode, param: PSym, r: var TCompRes;
emitted: ptr int = nil; skipVarOpenArray = false) =
proc genArg(p: PProc, n: PNode, param: PSym, r: var TCompRes; emitted: ptr int = nil) =
var a: TCompRes = default(TCompRes)
if (not skipVarOpenArray) and param.typ != nil and param.typ.kind == tyVar and
param.typ[0].kind == tyOpenArray:
if param.typ != nil and param.typ.kind == tyVar and param.typ[0].kind == tyOpenArray:
# `var openArray` params are passed as a `{base, off, len}` slice view.
genVarOpenArrayArg(p, n, a)
r.res.add(a.rdLoc)
@@ -1849,8 +1847,7 @@ proc genArgs(p: PProc, n: PNode, r: var TCompRes; start=1) =
r.kind = resExpr
proc genOtherArg(p: PProc; n: PNode; i: int; typ: PType;
generated: var int; r: var TCompRes;
skipVarOpenArray = false) =
generated: var int; r: var TCompRes) =
if i >= n.len:
globalError(p.config, n.info, "wrong importcpp pattern; expected parameter at position " & $i &
" but got only: " & $(n.len-1))
@@ -1863,12 +1860,11 @@ proc genOtherArg(p: PProc; n: PNode; i: int; typ: PType;
if paramType.isNil:
genArgNoParam(p, it, r)
else:
genArg(p, it, paramType.sym, r, skipVarOpenArray = skipVarOpenArray)
genArg(p, it, paramType.sym, r)
inc generated
proc genPatternCall(p: PProc; n: PNode; pat: string; typ: PType;
r: var TCompRes) =
let skipVarOpenArray = sfImportc in n[0].sym.flags
var i = 0
var j = 1
r.kind = resExpr
@@ -1878,11 +1874,11 @@ proc genPatternCall(p: PProc; n: PNode; pat: string; typ: PType;
var generated = 0
for k in j..<n.len:
if generated > 0: r.res.add(", ")
genOtherArg(p, n, k, typ, generated, r, skipVarOpenArray)
genOtherArg(p, n, k, typ, generated, r)
inc i
of '#':
var generated = 0
genOtherArg(p, n, j, typ, generated, r, skipVarOpenArray)
genOtherArg(p, n, j, typ, generated, r)
inc j
inc i
of '\31':

View File

@@ -675,6 +675,17 @@ proc rawClosureCreation(owner: PSym;
if up != nil and upField.typ.skipTypes({tyOwned, tyRef, tyPtr}) == up.typ.skipTypes({tyOwned, tyRef, tyPtr}):
result.add(newAsgnStmt(rawIndirectAccess(env, upField, env.info),
up, env.info))
# That assignment stores a real `ref`, so `injectDestructorCalls` has to
# find the up-field type's ops — otherwise it stays a raw pointer store,
# the enclosing env's refcount is one too low, and at teardown the two
# envs' mutually recursive `=destroy`s each believe they hold the last
# reference and recurse until the stack is gone. Whole-program cgen never
# noticed: some LATER lifting pass creates this very ref type's ops, and it
# runs before any routine's destructor injection. The per-module backend
# injects a routine right after lifting it (the `lower` stage), long before
# the module's top level is transformed at all (that is `cg`).
if up.typ != nil and up.typ.kind == tyRef and up.typ.elementType != nil:
createTypeBoundOpsLL(d.graph, up.typ, env.info, d.idgen, owner)
#elif oldenv != nil and oldenv.typ == upField.typ:
# result.add(newAsgnStmt(rawIndirectAccess(env, upField, env.info),
# oldenv, env.info))
@@ -732,6 +743,10 @@ proc closureCreationForIter(owner: PSym, iter: PNode;
if u != nil and u.typ.skipTypes({tyOwned, tyRef, tyPtr}) == expectedUpTyp:
result.add(newAsgnStmt(rawIndirectAccess(vnode, upField, iter.info),
u, iter.info))
# See the identical call in `rawClosureCreation`: the up-field's ops must
# exist by the time this assignment is destructor-injected.
if u.typ != nil and u.typ.kind == tyRef and u.typ.elementType != nil:
createTypeBoundOpsLL(d.graph, u.typ, iter.info, d.idgen, owner)
else:
localError(d.graph.config, iter.info, "internal error: cannot create up reference for iter")
result.add makeClosure(d.graph, d.idgen, iter.sym, vnode, iter.info)

View File

@@ -61,22 +61,12 @@ proc mangleProcNameExt*(graph: ModuleGraph, s: PSym): string =
# starts with an EMPTY per-name disamb table, so its `disamb` restarts at 0
# and collides with same-named sem-time symbols loaded from NIFs (two
# `=destroy` hooks both mangling to `_u2` → "conflicting types for ..." in
# the generated C). Most such symbols never cross a process boundary (nifc
# lifts, emits and compiles them in one run), so the per-module-unique
# item id is a safe and deterministic discriminator; the `_c` marker keeps
# the namespace disjoint from `_u<disamb>`.
# the generated C). The `_c` marker keeps the namespace disjoint from
# `_u<disamb>`; `backendMintedDisamb` (astdef) is the ONE definition of which
# integer identifies such a symbol, shared with `ccgutils.makeUnique` and
# `ast2nif.toNifSymName` so the C name and the NIF name cannot drift apart.
result = "_c"
if (s.disamb and HookDisambBit) != 0'i32:
# EXCEPTION: a backend-minted sym whose `disamb` is content-derived
# (setHookDisamb gave it HookDisambBit) — e.g. the `rttiDestroy` wrapper —
# DOES cross process boundaries: its C name is baked into the type's RTTI
# table, which is emit-everywhere and merge-deduped, so one process's
# `_c<item>` (a per-process backend counter) ends up referenced while the
# wrapper is defined with another's → undefined at link (`rttiDestroy_c23`).
# The content-derived disamb is stable across processes; use it.
result.addInt s.disamb
else:
result.addInt s.itemId.item
result.addInt backendMintedDisamb(s)
else:
result = "_u"
# Use `disamb` rather than `itemId.item`: under incremental compilation a

View File

@@ -574,12 +574,6 @@ proc logGenericInstance*(g: ModuleGraph; inst: PSym) =
let ownerModule = inst.itemId.module.int
g.opsLog.add LogEntry(kind: GenericInstEntry, module: ownerModule, sym: inst)
const
InstanceDisambBit* = 0x4000_0000'i32
## Set in the `disamb` of routine instances whose value is content-derived
## (see `setInstanceDisamb`); keeps them disjoint from the small counter
## range ordinary symbols draw from, so the NIF name `name.disamb.module`
## stays collision-free within a module.
proc setInstanceDisamb*(g: ModuleGraph; inst, generic: PSym;
concreteTypes: openArray[PType]) =
@@ -618,12 +612,6 @@ proc setInstanceDisamb*(g: ModuleGraph; inst, generic: PSym;
break
inst.disamb = h
const
HookDisambBit* = 0x2000_0000'i32
## Set in the `disamb` of synthesized type-bound operators and `$enum`
## procs whose value is content-derived (see `setHookDisamb`); disjoint
## from both the small counter range and the `InstanceDisambBit` range.
proc setHookDisamb*(g: ModuleGraph; hook: PSym; opName: string; typ: PType) =
## Under IC, replace a synthesized hook's counter-based `disamb` with a
## content-derived one: a hash of the operation name plus the `typeKey` of
@@ -1047,7 +1035,13 @@ when not defined(nimKochBootstrap):
var isKnownFile = false
let fileIdx = g.config.registerNifSuffix(string suffix, isKnownFile)
if not g.hookClosure.containsOrIncl(fileIdx.int):
let precomp = loadNifModule(ast.program, suffix, interf, interfHidden, {})
# `SkipInterfaceTables`: `interf`/`interfHidden` here are scratch tables
# shared by every iteration and never read — this module is a
# dep-of-a-dep, so none of its symbols are visible to the module being
# semchecked. Building them called `loadSymFromIndexEntry` for every
# index entry of every closure member.
let precomp = loadNifModule(ast.program, suffix, interf, interfHidden,
{SkipInterfaceTables})
registerLoadedHooks(g, precomp.logOps)
# Record this transitively-loaded module so the sem driver applies its
# VM-level load effects (macro-cache replay + `{.compileTime.}` global init)

View File

@@ -134,91 +134,6 @@ proc emitMethodDispatchers(g: ModuleGraph) =
if not containsOrIncl(mainMod.declaredThings, disp.id):
genProcLvl3(mainMod, disp)
proc signatureHasMetaType(t: PType; depth: int = 0): bool =
## Whether a routine signature mentions a compile-time/meta element type
## (`typed`/`untyped` — e.g. `echo`'s `varargs[typed]` — typedesc, static,
## generic param). Such routines are expanded at their call sites and never
## emitted standalone, so the per-module owned-routine seeding must skip them
## (`getTypeDescAux(tyTyped)` otherwise). `tfHasMeta` alone misses the varargs
## element case, hence the explicit scan.
result = false
if t == nil or depth > 8: return false
if t.kind == tyGenericBody:
# The uninstantiated template carried as a `tyGenericInst`'s first child
# always mentions its `tyGenericParam` placeholders, but the instance
# itself is fully concrete (e.g. `var CountTable[SigHash]`). Descending
# here would wrongly flag every routine with a generic-instance parameter
# as meta and drop it from the owned-routine seeding -> undefined symbols
# at link (its only definer never emits it).
return false
if t.kind == tyStatic:
# A RESOLVED static value (the `256` in `MDigest[256]`, the `N` in
# `HashList[T, N]`, …) is carried as a `tyStatic` node inside the otherwise
# fully-concrete `tyGenericInst`, but it is NOT meta: the routine is a normal
# runtime routine the owner must emit. Only an UNRESOLVED `static T` parameter
# (no bound value, `t.n == nil`) is meta. Without this, every routine whose
# signature touches a `static`-parameterized generic instance (the bulk of
# the SSZ/`MDigest` API) is dropped from the owned-routine seeding and ends up
# an undefined reference at link (mirrors the tyGenericBody case above).
return t.n == nil
if t.kind in {tyTyped, tyUntyped, tyTypeDesc, tyGenericParam,
tyAnything, tyFromExpr, tyError}:
return true
for k in t.kids:
if signatureHasMetaType(k, depth + 1): return true
proc ownsRuntimeRoutine(s: PSym; modPos: int): bool =
## A concrete, non-generic, runtime routine with a real body, OWNED by the
## module at `modPos`. Shared by the `cg` stage's owned-routine seeding (so a
## routine called only from other modules is still emitted by somebody) and
## the `lower` stage's owned-routine enumeration, so both stages see exactly
## the same set. The exclusions:
## - nested/closure procs (owner is a proc, not a module): emitted via their
## enclosing routine's lambda-lifting, never standalone;
## - generic instances (`sfFromGeneric`): emitted by demand, deduped by merge;
## - `importc`/`compileTime`/`error`/forward sentinels and meta signatures:
## not real codegen targets.
## - method DISPATCHERS (`sfDispatcher`): their bodies are (re)synthesized into
## the main TU by `emitMethodDispatchers`/`generateIfMethodDispatchers`, never
## per module. A dispatcher is a `copySym` clone of the method that shares the
## method's body sub-tree (incl. its closure iterator); transforming it here
## would lambda-lift that SHARED iterator a SECOND time under a different owner
## identity, baking a conflicting `up` field → "up references do not agree"
## (the divergence is impossible in non-IC, where the dispatcher body is empty
## at lift time). So a dispatcher is never an owned runtime routine.
## A `{.closure.}` iterator IS a standalone runtime routine (unlike an inline
## iterator, which is expanded at each call site) and must be emitted by its
## owner — else a cross-module `for` over it links to nothing.
##
## Generic INSTANCES (`sfFromGeneric`) are NEVER an owned runtime routine — not
## in `cg` and not in the `lower` stage. They are demanded by the backend's
## emit-everywhere path and deduped by `merge` (content C name); the frontend
## materialises them through the `(offer)` mechanism. The `lower` stage must
## not transform an instance: a not-fully-concrete instance (a closure factory
## over a `static` param, or a `$`/`=` op instance whose body resolves only at
## its further-specialised use sites) still carries unresolved overload choices
## and crashes `transformBody` (empty-`namePos` lambda, nil-typed const-fold).
s.itemId.module == modPos and
(s.kind in {skProc, skFunc, skConverter, skMethod} or
(s.kind == skIterator and s.typ != nil and s.typ.callConv == ccClosure)) and
s.skipGenericOwner != nil and s.skipGenericOwner.kind == skModule and
s.magic == mNone and
sfFromGeneric notin s.flags and
sfDispatcher notin s.flags and
{sfForward, sfImportc, sfCompileTime, sfError} * s.flags == {} and
s.typ != nil and not signatureHasMetaType(s.typ) and
s.ast != nil and s.ast.safeLen > bodyPos and
s.ast[genericParamsPos].kind == nkEmpty
# NOTE: an `nkEmpty` body is NOT a disqualifier. A concrete, owned, non-
# forward/-importc/-magic routine whose body folds to nothing is still a real
# definition the owner must emit (`void f(void){}`), exactly as whole-program
# cgen does — else a cross-module caller links to nothing. This bites e.g.
# Nimbus' `extras.incInternalErrors`, a plain `proc` whose sole statement is a
# metrics-counter `.inc()` that the `metrics` library expands to a no-op when
# the importing tool (ncli) builds with `-u:metrics`; the body is then a bare
# `nkEmpty`, but `state_transition_epoch` still calls it. Forward declarations
# (the other empty-body case) carry `sfForward` and are excluded above.
proc generateCodeForModule(g: ModuleGraph; precomp: PrecompiledModule) =
## Generate C code for a single module.
let moduleId = precomp.module.position
@@ -713,6 +628,11 @@ proc generateCgStage(g: ModuleGraph; mainFileIdx: FileIndex) =
let tb = bl.mods[target.module.position]
if tb != nil:
finishModule(g, tb)
# Record this module's C compile/link directives next to its `.c` so the
# `link` stage can recover them without loading the module graph. See
# `replayer.writeBackendActions`.
writeBackendActions(g, target.module, target.topLevel,
getCFile(tb).string & BackendActionsExt)
# Writes only the target's `.c.nif` (every other loaded module's TU is empty,
# so `cgenWriteModules` emits no artifact for it). cc/link are NOT run here.
@@ -814,6 +734,15 @@ proc generateEmitStage(g: ModuleGraph; mainFileIdx: FileIndex) =
# up-to-date check, not a shared prerequisite in nifmake's mtime ordering.
if not fileExists(cfile) or readFile(cfile) != code:
writeFile(cfile, code)
# ... but nifmake needs SOME output whose mtime proves "this rule ran since its
# inputs last moved". With the `.c` as the only output, the content-stable write
# above is indistinguishable from not having run: `merge` rewrites the decision
# file unconditionally, so every `emit` whose `.c` came out byte-identical stays
# older than a declared input and re-fires on every warm build from then on
# (measured: all 218 emit rules of a 219-module program, on a NO-OP build).
# The stamp is written unconditionally and is the rule's freshness proof; the
# `.c` keeps its content-stable mtime so `callCCompiler` still reuses the `.o`.
writeFile(cfile & ".stamp", $code.len & " " & $dropped & "\n")
if isDefined(g.config, "icDceCheck"):
stderr.writeLine "[icEmit] " & extractFilename(cfile) & " dropped " &
$dropped & " bodies (" & $code.len & " bytes)"
@@ -822,53 +751,62 @@ proc generateLinkStage(g: ModuleGraph; mainFileIdx: FileIndex) =
## Per-module backend link (`--icBackendStage:link`): the `emit` stages have
## written every module's `.c`; register them and run the C compiler + linker
## once via `extccomp.callCCompiler` (which parallelizes the per-file cc and
## skips up-to-date objects itself). No codegen runs — the graph is loaded only
## so `getCFile` yields each module's emitted `.c` path.
let (modules, precompSys, _) = loadBackendModules(g, mainFileIdx)
if modules.len == 0:
rawMessage(g.config, errGenerated,
"Cannot load NIF file for main module: " & toFullPath(g.config, mainFileIdx))
return
# The per-module `cg` processes each collect their module's C compile/link
# directives (`{.passL: "-lm".}` etc.) via `replayBackendActions`, but those
# live in the cg process and never reach this separate link process. Re-collect
# every loaded module's directives here so the final `callCCompiler` sees them
# (without this, math's `-lm` is lost → undefined `floor`/`pow`/… at link).
for m in modules:
replayBackendActions(g, m.module, m.topLevel)
if precompSys.module != nil:
replayBackendActions(g, precompSys.module, precompSys.topLevel)
let bl = BModuleList(g.backend)
## skips up-to-date objects itself). No codegen runs and NO MODULE GRAPH IS
## LOADED.
##
## It used to load the whole import closure (`loadBackendModules`) for two
## things only: each module's `.c` path via `getCFile`, and its recorded C
## directives via `replayBackendActions`. That was 3.7s of the ~11s serial
## backend critical path on a 219-module program — a whole-program
## deserialization to recover a list of paths and a handful of strings. Both
## are now read from artifacts the earlier stages already produce:
## * the driver's `LiveModulesFile` manifest lists every live module's
## `.c.nif`, and the `.c` sits beside it (`emit`'s output);
## * each module's `cg` wrote its directives to a `.cflags` sidecar.
let nimcache = getNimcacheDir(g.config).string
var cfiles: seq[string] = @[]
let manifest = nimcache / LiveModulesFile
if fileExists(manifest):
for line in lines(manifest):
let p = line.strip()
if p.len > 0 and p.endsWith(".nif"): cfiles.add p[0 ..< p.len - ".nif".len]
else:
# A cache written by an older compiler has no manifest; fall back to the
# `.c` files sitting next to the artifacts.
for artifact in walkFiles(nimcache / ("*" & icCFileExt(g.config) & ".nif")):
cfiles.add artifact[0 ..< artifact.len - ".nif".len]
sort cfiles
var addedCFiles = initHashSet[string]()
for m in bl.mods:
if m != nil:
let cfile = getCFile(m)
# Only modules that are their own cg/emit target produced a `.c`; the rest
# (extra members of system's closure that no build rule targets) had their
# code emit-everywhere'd into the targets, so they have no file to compile.
if not fileExists(cfile.string): continue
addedCFiles.incl extractFilename(cfile.string)
var cf = Cfile(nimname: m.module.name.s, cname: cfile,
obj: completeCfilePath(g.config, toObjFile(g.config, cfile)),
flags: {})
# `addExternalFileToCompile` (not `addFileToCompile`) gates each `.c` on its
# SHA1 footprint: an unchanged `.c` keeps its `.o` and is flagged Cached, so
# `callCCompiler` skips its compile but still links the existing object. This
# is what makes a localized edit recompile only the handful of `.c`s the
# `emit` stage actually rewrote, instead of every object every time — the
# final piece of per-module backend incrementality after the merge barrier.
addExternalFileToCompile(g.config, cf)
for cpath in cfiles:
# Only modules that are their own cg/emit target produced a `.c`; the rest
# had their code emit-everywhere'd into the targets, so there is nothing to
# compile for them.
if not fileExists(cpath): continue
addedCFiles.incl extractFilename(cpath)
# The directives this module recorded (`{.passL: "-lm".}` etc.); without
# them math's `-lm` is lost -> undefined `floor`/`pow`/… at link.
applyBackendActions(g, cpath & BackendActionsExt)
let cfile = AbsoluteFile cpath
var cf = Cfile(nimname: splitFile(cfile).name, cname: cfile,
obj: completeCfilePath(g.config, toObjFile(g.config, cfile)),
flags: {})
# `addExternalFileToCompile` (not `addFileToCompile`) gates each `.c` on its
# SHA1 footprint: an unchanged `.c` keeps its `.o` and is flagged Cached, so
# `callCCompiler` skips its compile but still links the existing object. This
# is what makes a localized edit recompile only the handful of `.c`s the
# `emit` stage actually rewrote, instead of every object every time.
addExternalFileToCompile(g.config, cf)
# deps.nim's static scanner can keep a CONDITIONALLY-imported module as a build
# node (e.g. `net`'s `when defineSsl: import openssl`, or a `when defined(os)`
# import) that the NIF-`deps` walk above never reaches because the condition is
# off. Such a node still emitted a `.c`, and it can OWN a live generic instance
# that a REACHABLE module reuses (openssl owns `toHex[uint8]`, reused by
# `strutils.escape`) — so its body must be at link or that reference is
# node (e.g. `net`'s `when defineSsl: import openssl`) that the manifest above
# may not cover. Such a node still emitted a `.c`, and it can OWN a live generic
# instance that a REACHABLE module reuses (openssl owns `toHex[uint8]`, reused
# by `strutils.escape`) — so its body must be at link or that reference is
# undefined. Link every emitted `.c` the merge decision says OWNS a LIVE symbol;
# a node that owns nothing live (a Windows-only winsock node on Linux) is
# correctly skipped.
block:
let nimcache = getNimcacheDir(g.config).string
let decision = readMergeDecision(nimcache / MergeDecisionFile)
if not decision.broken:
var liveOwners = initHashSet[string]()
@@ -880,6 +818,7 @@ proc generateLinkStage(g: ModuleGraph; mainFileIdx: FileIndex) =
if addedCFiles.containsOrIncl(cbase): continue
let cfile = AbsoluteFile(nimcache / cbase)
if not fileExists(cfile.string): continue
applyBackendActions(g, cfile.string & BackendActionsExt)
var cf = Cfile(nimname: cbase, cname: cfile,
obj: completeCfilePath(g.config, toObjFile(g.config, cfile)),
flags: {})

View File

@@ -29,7 +29,7 @@ const
nimEnableCovariance* = defined(nimEnableCovariance)
icFormatVersion* = "37"
icFormatVersion* = "38"
## Version of the IC cache format (the sem-NIF module layout written by
## ast2nif.nim plus the iface/impl/edges side files). Bump it whenever
## that layout changes: `commandIc` wipes a nimcache whose `ic.version`

View File

@@ -2160,54 +2160,47 @@ proc checkedForDestructor(t: PType): bool =
return true
result = false
proc normalizeTypeHook(t: PType; markAsgn = false): PType =
proc whereToBindTypeHook(c: PContext; t: PType): PType =
result = t
while true:
if markAsgn:
incl(result, tfHasAsgn)
if result.kind == tyCompositeTypeClass and result.base.kind == tyGenericBody:
result = result.base
elif result.kind in {tyGenericBody, tyGenericInst}:
result = result.skipModifier
elif result.kind == tyGenericInvocation:
result = result.genericHead
else:
break
proc whereToBindTypeHook(c: PContext; t: PType): PType =
result = normalizeTypeHook(t)
if result.kind in {tyGenericBody, tyGenericInst}: result = result.skipModifier
elif result.kind == tyGenericInvocation: result = result[0]
else: break
if result.kind in {tyObject, tyDistinct, tySequence, tyString}:
result = canonType(c, result)
proc bindHookToType(c: PContext; s: PSym; n: PNode; op: TTypeAttachedOp;
typeToBind: PType): bool =
var obj = typeToBind
if obj.kind notin {tyObject, tyDistinct, tySequence, tyString}:
return false
obj = canonType(c, obj)
let ao = getAttachedOp(c.graph, obj, op)
if ao == s:
discard "forward declared hook"
elif ao.isNil and not checkedForDestructor(obj):
setAttachedOp(c.graph, c.module.position, obj, op, s)
else:
prevDestructor(c, op, ao, obj, n.info)
if obj.owner.getModule != s.getModule:
localError(c.config, n.info, errGenerated,
"type bound operation `" & s.name.s & "` can be defined only in the same module with its type (" & obj.typeToString() & ")")
result = true
proc bindDupHook(c: PContext; s: PSym; n: PNode; op: TTypeAttachedOp) =
let t = s.typ
var noError = false
let cond = t.len == 2 and t.returnType != nil
if cond:
var obj = normalizeTypeHook(t.firstParamType, markAsgn = true)
let res = normalizeTypeHook(t.returnType)
var obj = t.firstParamType
while true:
incl(obj, tfHasAsgn)
if obj.kind in {tyGenericBody, tyGenericInst}: obj = obj.skipModifier
elif obj.kind == tyGenericInvocation: obj = obj.genericHead
else: break
if sameType(obj, res):
noError = bindHookToType(c, s, n, op, obj)
var res = t.returnType
while true:
if res.kind in {tyGenericBody, tyGenericInst}: res = res.skipModifier
elif res.kind == tyGenericInvocation: res = res.genericHead
else: break
if obj.kind in {tyObject, tyDistinct, tySequence, tyString} and sameType(obj, res):
obj = canonType(c, obj)
let ao = getAttachedOp(c.graph, obj, op)
if ao == s:
discard "forward declared destructor"
elif ao.isNil and not checkedForDestructor(obj):
setAttachedOp(c.graph, c.module.position, obj, op, s)
else:
prevDestructor(c, op, ao, obj, n.info)
noError = true
if obj.owner.getModule != s.getModule:
localError(c.config, n.info, errGenerated,
"type bound operation `" & s.name.s & "` can be defined only in the same module with its type (" & obj.typeToString() & ")")
if not noError and sfSystemModule notin s.owner.flags:
localError(c.config, n.info, errGenerated,
@@ -2237,8 +2230,25 @@ proc bindTypeHook(c: PContext; s: PSym; n: PNode; op: TTypeAttachedOp) =
t.len >= 2 and t.returnType == nil
if cond:
var obj = normalizeTypeHook(t.firstParamType.skipTypes({tyVar}), markAsgn = true)
noError = bindHookToType(c, s, n, op, obj)
var obj = t.firstParamType.skipTypes({tyVar})
while true:
incl(obj, tfHasAsgn)
if obj.kind in {tyGenericBody, tyGenericInst}: obj = obj.skipModifier
elif obj.kind == tyGenericInvocation: obj = obj.genericHead
else: break
if obj.kind in {tyObject, tyDistinct, tySequence, tyString}:
obj = canonType(c, obj)
let ao = getAttachedOp(c.graph, obj, op)
if ao == s:
discard "forward declared destructor"
elif ao.isNil and not checkedForDestructor(obj):
setAttachedOp(c.graph, c.module.position, obj, op, s)
else:
prevDestructor(c, op, ao, obj, n.info)
noError = true
if obj.owner.getModule != s.getModule:
localError(c.config, n.info, errGenerated,
"type bound operation `" & s.name.s & "` can be defined only in the same module with its type (" & obj.typeToString() & ")")
if not noError and sfSystemModule notin s.owner.flags:
case op
of attachedTrace:
@@ -2305,12 +2315,35 @@ proc semOverride(c: PContext, s: PSym, n: PNode) =
message(c.config, n.info, warnDeprecated, "Overriding `=` hook is deprecated; Override `=copy` hook instead")
let t = s.typ
if t.len == 3 and t.returnType == nil and t.firstParamType.kind == tyVar:
var obj = normalizeTypeHook(t.firstParamType.elementType, markAsgn = true)
let objB = normalizeTypeHook(t[2])
if sameType(obj, objB):
var obj = t.firstParamType.elementType
while true:
incl(obj, tfHasAsgn)
if obj.kind == tyGenericBody: obj = obj.skipModifier
elif obj.kind == tyGenericInvocation: obj = obj.genericHead
else: break
var objB = t[2]
while true:
if objB.kind == tyGenericBody: objB = objB.skipModifier
elif objB.kind in {tyGenericInvocation, tyGenericInst}:
objB = objB.genericHead
else: break
if obj.kind in {tyObject, tyDistinct, tySequence, tyString} and sameType(obj, objB):
# attach these ops to the canonical tySequence
obj = canonType(c, obj)
#echo "ATTACHING TO ", obj.id, " ", s.name.s, " ", cast[int](obj)
let k = if name == "=" or name == "=copy": attachedAsgn else: attachedSink
if bindHookToType(c, s, n, k, obj): return
let ao = getAttachedOp(c.graph, obj, k)
if ao == s:
discard "forward declared op"
elif ao.isNil and not checkedForDestructor(obj):
setAttachedOp(c.graph, c.module.position, obj, k, s)
else:
prevDestructor(c, k, ao, obj, n.info)
if obj.owner.getModule != s.getModule:
localError(c.config, n.info, errGenerated,
"type bound operation `" & name & "` can be defined only in the same module with its type (" & obj.typeToString() & ")")
return
if sfSystemModule notin s.owner.flags:
localError(c.config, n.info, errGenerated,
"signature for '" & s.name.s & "' must be proc[T: object](x: var T; y: T)")

View File

@@ -209,11 +209,7 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
# backend spelling instead of collapsing into the generic Nim builtin:
c &= char(t.kind)
if t.sym != nil and {sfImportc, sfExportc} * t.sym.flags != {}:
# Aliases inherit the external name, but have a different symbol.
if t.sym.loc.snippet != "":
c &= t.sym.loc.snippet
else:
c.hashSym(t.sym)
c.hashSym(t.sym)
of tyObject, tyEnum:
if t.typeInstImpl != nil:
# prevent against infinite recursions here, see bug #8883:

View File

@@ -1166,8 +1166,6 @@ proc enterConceptMatch(c: var TCandidate; f,a: PType, flags: TTypeRelFlags): TTy
return typeRel(c, prev, a, flags)
if trDontBind in flags:
conceptFlags.incl mfDontBind
if trBindGenericParam in flags:
conceptFlags.incl mfBindGenericParam
if trCheckGeneric in flags:
conceptFlags.incl mfCheckGeneric
let mres = concepts.conceptMatch(c.c, concpt, a, c.bindings, container, flags = conceptFlags)

View File

@@ -3304,21 +3304,13 @@ proc dirInclude(p: var RstParser): PRstNode =
## Only the content before the first occurrence of the specified
## text (but after any after text) will be included. If text is
## not found inclusion will happen until the end of the file.
##
## :literal: flag (empty)
##
## The entire included text is inserted into the document as a single
## literal block (useful for program listings).
##
## :code: language (if empty, `nim` is assumed by default)
##
## The argument and the included content are passed to the code directive
## (useful for program listings).
##
## :encoding: name of text encoding
##
## The text encoding of the external data file. Defaults to the document's
## encoding (if specified).
#literal : flag (empty)
# The entire included text is inserted into the document as a single
# literal block (useful for program listings).
#encoding : name of text encoding
# The text encoding of the external data file. Defaults to the document's
# encoding (if specified).
#
result = nil
var n = parseDirective(p, rnDirective, {hasArg, argIsFile, hasOptions}, nil)
var filename = strip(addNodes(n.sons[0]))
@@ -3351,19 +3343,6 @@ proc dirInclude(p: var RstParser): PRstNode =
if getFieldValue(n, "literal") != "":
result = newRstNode(rnLiteralBlock)
result.add newLeaf(inputString[startPosition..endPosition])
elif getFieldValue(n, "code") != "":
result = newRstNode(rnCodeBlock)
result.sons.setLen(3)
let lang = getFieldValue(n, "code").strip()
if lang notin ["", "\x01\x01"]:
var codeArg = newRstNode(rnDirArg)
codeArg.add(newLeaf(lang))
result.sons[0] = codeArg
result.sons[1] = newRstNode(rnFieldList)
defaultCodeLangNim(p, result)
var litBlock = newRstNode(rnLiteralBlock)
litBlock.add newLeaf(inputString[startPosition..endPosition])
result.sons[2] = litBlock
else:
var q: RstParser
initParser(q, p.s)

View File

@@ -1254,9 +1254,7 @@ proc del*[T](x: var seq[T], i: Natural) {.noSideEffect.} =
a.del(2)
assert a == @[10, 11, 14, 13]
let xl = x.len - 1
# Avoid moving the element onto itself when deleting the last item.
if i != xl:
movingCopy(x[i], x[xl])
movingCopy(x[i], x[xl])
setLen(x, xl)
proc insert*[T](x: var seq[T], item: sink T, i = 0.Natural) {.noSideEffect.} =

View File

@@ -35,20 +35,6 @@ proc bug20303() =
bug20303()
block: # bug #26143
var indexCalls = 0
proc nextIndex(): int =
result = indexCalls
inc indexCalls
proc consume(value: sink string) =
doAssert value == "A"
var values = @["A", "B"]
consume(values[nextIndex()])
doAssert indexCalls == 1
proc main() = # todo bug with templates
block: # bug #11267
var a: seq[char] = block: @[]

View File

@@ -1,5 +0,0 @@
proc resizeCints*(s: var seq[cint], n: int) =
s.setLen(n)
proc cintLen*(s: seq[cint]): int =
result = s.len

View File

@@ -1,15 +0,0 @@
discard """
action: run
targets: "c cpp"
"""
import mseq_importc_alias
type CIntAlias = cint
var fds: seq[CIntAlias]
doAssert cintLen(@[1.cint, 2.cint]) == 2
doAssert cintLen(fds) == 0
resizeCints(fds, 3)
fds[1] = CIntAlias(7)
doAssert cintLen(fds) == 3

View File

@@ -1,20 +0,0 @@
discard """
action: run
targets: "c cpp"
"""
type CIntAlias = cint
var x: (cint,) = (1.cint,)
var y: (CIntAlias,) = x
x = y
doAssert x[0] == 1.cint
var a: seq[cint]
var b: seq[CIntAlias]
a.add 1.cint
a.add 2.cint
b = a
a = b
doAssert a[0] == 1.cint
doAssert b[1] == CIntAlias(2)

View File

@@ -1,75 +0,0 @@
discard """
action: run
"""
type Indexable[T] = concept
proc `[]`(a: Self; index: int): T
proc len(a: Self): int
iterator items[T; I: Indexable[T]](indexable: I): T =
for index in 0 ..< indexable.len:
yield indexable[index]
type Dummy[T] = distinct seq[T]
proc `[]`[T](d: Dummy[T], i: int): T = seq[T](d)[i]
proc len[T](d: Dummy[T]): int = seq[T](d).len
var acc = 0
for x in Dummy(@[1, 2, 3]):
acc += x
doAssert acc == 6
# Inferred concept parameters are resolved through the implementation's own
# generic bindings before being exported to the surrounding routine.
type
Elem[T] = object
value: T
NestedDummy[T] = ref object
data: seq[T]
proc `[]`[T](d: NestedDummy[T], i: int): Elem[T] =
Elem[T](value: d.data[i])
proc len[T](d: NestedDummy[T]): int = d.data.len
iterator directItems[T](indexable: Indexable[T]): T =
for index in 0 ..< indexable.len:
yield indexable[index]
var nestedAcc = 0
for x in NestedDummy[int](data: @[4, 5, 6]):
nestedAcc += x.value
doAssert nestedAcc == 15
var directNestedAcc = 0
for x in directItems(NestedDummy[int](data: @[7, 8, 9])):
directNestedAcc += x.value
doAssert directNestedAcc == 24
# All dependent parameters inferred while checking a concept constraint must
# be propagated to the constrained routine.
type
KeyValue[K, V] = concept
proc key(x: Self): K
proc value(x: Self): V
Pair[K, V] = object
k: K
v: V
proc key[K, V](x: Pair[K, V]): K = x.k
proc value[K, V](x: Pair[K, V]): V = x.v
proc unpack[K, V; P: KeyValue[K, V]](x: P): (K, V) =
(x.key, x.value)
let pair = Pair[int, string](k: 7, v: "seven")
doAssert unpack(pair) == (7, "seven")
doAssert not compiles(unpack[string, int](pair))
proc unpackBoth[K1, V1, K2, V2;
P1: KeyValue[K1, V1]; P2: KeyValue[K2, V2]](
x: P1; y: P2): ((K1, V1), (K2, V2)) =
(unpack(x), unpack(y))
let otherPair = Pair[string, float](k: "eight", v: 8.0)
doAssert unpackBoth(pair, otherPair) == ((7, "seven"), ("eight", 8.0))

View File

@@ -166,99 +166,3 @@ type Vector*[T] = object
# proc `=destroy`*(x: var Vector[int]) = discard # this will remove error
proc `=destroy`*[T](x: var Vector[T]) = discard
var a: Vector[int] # Error: unresolved generic parameter
# issue #26132
block:
type UnparameterizedGeneric[T] = object
proc `=destroy`(x: var UnparameterizedGeneric) = discard
proc `=wasMoved`(x: var UnparameterizedGeneric) = discard
proc `=trace`(x: var UnparameterizedGeneric; env: pointer) = discard
var x: UnparameterizedGeneric[int]
discard x
# Exercise every type-bound hook with the generic parameter omitted.
block:
type
Generic[T] = object
value: T
var destroys, moves, traces, copies, sinks, dups: int
proc `=destroy`(x: var Generic) = inc destroys
proc `=wasMoved`(x: var Generic) =
inc moves
x.value = default(typeof(x.value))
proc `=trace`(x: var Generic; env: pointer) = inc traces
proc `=copy`(dest: var Generic; src: Generic) =
inc copies
dest.value = src.value
proc `=sink`(dest: var Generic; src: Generic) =
inc sinks
dest.value = src.value
proc `=dup`(src: Generic): Generic =
inc dups
Generic(value: src.value)
proc deepCopy(src: ref Generic): ref Generic = src
proc exercise[T]() =
var first = Generic[T](value: default(T))
var second = Generic[T](value: default(T))
second = first
doAssert second.value == first.value
second = Generic[T](value: default(T))
doAssert second.value == default(T)
`=trace`(first, nil)
`=wasMoved`(first)
let implicitDuplicate = first
discard implicitDuplicate
let duplicate = `=dup`(first)
discard duplicate
let original = new(Generic[T])
doAssert deepCopy(original) == original
exercise[string]()
exercise[int]()
exercise[seq[int]]()
doAssert copies > 0
doAssert sinks > 0
doAssert dups > 0
doAssert moves > 0
doAssert traces > 0
doAssert destroys > 0
block:
type GenericDistinct[T] = distinct Generic[T]
proc `=destroy`(x: var GenericDistinct) = discard
proc `=wasMoved`(x: var GenericDistinct) = discard
proc `=trace`(x: var GenericDistinct; env: pointer) = discard
proc `=copy`(dest: var GenericDistinct; src: GenericDistinct) = discard
proc `=sink`(dest: var GenericDistinct; src: GenericDistinct) = discard
proc `=dup`(src: GenericDistinct): GenericDistinct = src
proc deepCopy(src: ref GenericDistinct): ref GenericDistinct = src
var first = GenericDistinct[string](Generic[string](value: "first"))
var second = GenericDistinct[string](Generic[string](value: "second"))
second = first
second = GenericDistinct[string](Generic[string](value: "third"))
`=trace`(first, nil)
`=wasMoved`(first)
let moved = move(first)
let duplicate = `=dup`(moved)
discard duplicate
let original = new(GenericDistinct[string])
doAssert deepCopy(original) == original
block:
type GenericPair[A, B] = object
left: A
right: B
proc `=destroy`(x: var GenericPair) = discard
var pair = GenericPair[int, string](left: 42, right: "pair")
discard pair

101
tests/ic/tclosure_hooks.nim Normal file
View File

@@ -0,0 +1,101 @@
discard """
description: '''IC vs `nim c`: closure environments, their hooks and their owners'''
"""
#? metamorphic
# A closure's environment type — and the `=destroy`/`=copy` the compiler lifts
# for it — is minted by the BACKEND, during the `lower` stage, and exists in no
# module's semmed NIF. The per-module backend has to decide which translation
# unit emits such a routine, and the owner walk it uses lands on the module of
# the ORIGINAL generic: for a generic closure iterator defined in one module and
# instantiated in another, that is a module which never sees the instance, so the
# env's `=destroy` was emitted by nobody (`undefined reference to
# eqdestroy__c485__…`). Every referencing TU emits it now.
#
# The steps then move the captured state around, because the env's LAYOUT is what
# decides whether those hooks are trivial: a body-only edit that adds a capture
# changes the env type of a routine whose importers do not re-sem.
#!FILE clleaf.nim
type Ev* = proc (s: string): string {.closure.}
proc leafMaker*(tag: string): Ev =
var n = 0
proc outer(s: string): string =
proc inner(t: string): string =
inc n
tag & ":" & t & ":" & $n
inner(s)
result = outer
iterator leafIter*[T](xs: seq[T]): T {.closure.} =
for x in xs: yield x
#!FILE clmid.nim
import clleaf
proc midMaker*(tag: string): Ev =
let base = leafMaker(tag & "/mid")
var calls = 0
result = proc (s: string): string =
inc calls
base(s) & "#" & $calls
proc midIter*(): seq[string] =
# instantiates `leafIter[string]` HERE, not where it is defined
result = @[]
for x in leafIter(@["p", "q"]): result.add x
#!FILE main.nim
import clleaf, clmid
let t = midMaker("top")
echo t("Alpha")
echo t("Beta")
echo midIter()
# an instance only the main module has
var fs: seq[float] = @[]
for x in leafIter(@[1.5, 2.5]): fs.add x
echo fs
#!STEP
# body-only edit that GROWS the environment: a second captured local
#!FILE clleaf.nim
type Ev* = proc (s: string): string {.closure.}
proc leafMaker*(tag: string): Ev =
var n = 0
var seen: seq[string] = @[]
proc outer(s: string): string =
proc inner(t: string): string =
inc n
seen.add t
tag & ":" & t & ":" & $n & ":" & $seen.len
inner(s)
result = outer
iterator leafIter*[T](xs: seq[T]): T {.closure.} =
var i = 0
for x in xs:
inc i
yield x
#!STEP
# and shrink it again
#!FILE clleaf.nim
type Ev* = proc (s: string): string {.closure.}
proc leafMaker*(tag: string): Ev =
var n = 0
proc outer(s: string): string =
proc inner(t: string): string =
inc n
tag & ":" & t & ":" & $n
inner(s)
result = outer
iterator leafIter*[T](xs: seq[T]): T {.closure.} =
for x in xs: yield x
#!STEP

View File

@@ -0,0 +1,90 @@
discard """
description: '''IC vs `nim c`: a closure iterator nested in a closure iterator'''
"""
#? metamorphic
# `env.:up = enclosingEnv` links a nested routine's environment to its parent,
# and the two environments then reference each other. That assignment has to go
# through `=copy` (with the cyclic increment) or the parent's refcount is one too
# low, and at teardown both `=destroy`s believe they hold the last reference and
# recurse until the stack is gone — a SIGSEGV, after the program's own output has
# already been printed. (`tests/iter/tnestedclosures.nim`, "Test 3".)
#
# Whether it becomes a `=copy` depends on the up-field type's hooks existing when
# the routine is destructor-injected. Whole-program cgen got that for free: a
# LATER lifting pass creates them, and it runs before any routine's injection.
# The per-module backend injects a routine right after lifting it (the `lower`
# stage), long before the module's top level is transformed at all (that is
# `cg`) — so the hooks are created at the assignment site now.
#!FILE main.nim
iterator foo(): int {.closure.} =
let x = 34
proc bar() = echo "bar sees ", x
iterator bar2(): int {.closure.} =
bar()
yield x
for y in bar2():
yield y
for v in foo(): echo v
# a closure iterator nested in a closure iterator, inside a proc
proc factory() =
iterator outerIt(): int {.closure.} =
iterator innerIt(): int {.closure.} =
yield 0
yield 1
yield 2
for x in innerIt(): yield x
for x in outerIt(): echo x
factory()
# the iterator's env outlives the proc that made it
proc keep(): iterator (): string =
let held = "kept"
result = iterator (): string =
yield held
yield held & "!"
for s in keep()(): echo s
#!STEP
# growing the captured state changes both env layouts
#!FILE main.nim
iterator foo(): int {.closure.} =
let x = 34
var log: seq[string] = @[]
proc bar() =
log.add "bar"
echo "bar sees ", x, " ", log.len
iterator bar2(): int {.closure.} =
bar()
bar()
yield x
for y in bar2():
yield y
for v in foo(): echo v
proc factory() =
iterator outerIt(): int {.closure.} =
var emitted = 0
iterator innerIt(): int {.closure.} =
yield 0
yield 1
yield 2
for x in innerIt():
inc emitted
yield x * emitted
for x in outerIt(): echo x
factory()
proc keep(): iterator (): string =
let held = "kept"
let extra = "+"
result = iterator (): string =
yield held & extra
yield held & "!" & extra
for s in keep()(): echo s
#!STEP

View File

@@ -1,7 +0,0 @@
import ../ccgbugs/mseq_importc_alias
type CIntAlias = cint
var values: seq[CIntAlias]
resizeCints(values, 2)
doAssert cintLen(values) == 2

View File

@@ -49,13 +49,6 @@ proc bar(s: var seq[int], a: int) =
s.bar(5)
doAssert(s == @[123, 1])
# Imported JavaScript patterns must receive the underlying array, not the
# `{base, off, len}` view used for regular `var openArray` parameters.
proc jsSort[T](x: var openArray[T], cmp: proc(a, b: T): int) {.importcpp: "#.sort(#)", nodecl.}
var sorted = @[2, 1]
sorted.jsSort(proc(a, b: int): int = a - b)
doAssert(sorted == @[1, 2])
import tables
block: # Test get addr of byvar return value
var t = initTable[string, int]()

View File

@@ -1,20 +0,0 @@
discard """
output: '''caught'''
"""
type
Base = ref object of RootObj
Child = ref object of Base
method run(value: Base): string {.base.} =
result = "base"
method run(value: Child): string =
raise newException(ValueError, "child")
let value: Base = Child()
try:
discard value.run()
quit "virtual method did not raise"
except ValueError:
echo "caught"

View File

@@ -1,24 +0,0 @@
discard """
matrix: "--mm:orc --undef:nimPreviewNonVarDestructor"
output: "hello"
"""
# bug #26134
type MyObject = object
proc `=destroy`(v: var MyObject) =
echo "hello"
proc remove(v: var seq[MyObject]) =
v.del(0)
proc aaa(v: var seq[MyObject], i: sink MyObject) =
v.add(i)
proc main =
var v: seq[MyObject]
v.aaa(MyObject())
v.remove()
main()