IC: eight correctness fixes found by differential testing against nim c

Grinding a small figdraw-based program under `nim ic` and diffing its output
against the classic backend surfaced eight bugs, four of which silently produced
a wrong binary rather than an error.

Frontend / build graph (`deps.nim`):

* Dead `when`-guarded imports were compiled anyway. `when someStrdefine == "x":
  import y` is `cvUnknown` to the scanner, which conservatively keeps the edge —
  right for an edge, but it also gave `y` its own `nim m` rule, so a build died
  on a package the user never installed because they never selected that backend.
  Track which edges are speculative and drop a speculative subtree that cannot
  compile; if the guard was in fact live, the discovery fixpoint puts the node
  back with the honest `cannot open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so nothing
  re-fires and `nim ic` relinked a stale binary while `nim c` reported `cannot
  open file`. Report an unresolvable import from a non-speculatively reached
  module during the graph scan.
* Macro-generated imports were discovered once and then forgotten. Discovery
  only ran after a failure and the graph is re-derived statically every run, so
  on a warm build the discovered module had no rules at all and editing it
  changed nothing. Seed the graph from the `.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from file mtimes
  and never looks at a rule's command line, so `-d:foo=bar` / `--mm:` / `--threads:`
  regenerated the build file with the new switches and re-fired zero rules.
  Reify the configuration as a file and make it an input of every rule.
* Command-line switches never reached the children: they replay the project's
  config files, never the driver's argv, so `nim ic --opt:speed` produced a
  byte-identical debug binary (likewise `--panics`, `--experimental`, `--passC`).
  Forward the driver's switches, minus the ones that must differ per child.

Artifacts and codegen:

* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw the rule
  as satisfied on the next run: `nim ic` then reported success for a program that
  does not compile, and generated code from error-bearing AST (or hit an internal
  error in `ccgexprs`). Never persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives on the
  module symbol, which `moduleFromNifFile` rebuilds from scratch, so
  `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a module-level
  `block: let h = openHandle()` never ran `=destroy`. Persist the flag as a
  `(modflags)` record. `injectdestructors` also has to tolerate the
  `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym node is
  written as a bare NIF `SymUse` token, which has nowhere to put node flags, so
  the frontend's move analysis never reached the backend: EVERY first assignment
  to a destructor-bearing local compiled as `=sink`, i.e. `=destroy` on
  still-zeroed memory followed by a copy, and no read was ever a move. Wrap a sym
  use in `(nflags ...)` when it carries persistent node flags.

`icFormatVersion` 34 -> 35 for the two new NIF records.

Validation: `koch bootic` reaches its byte-identical fixed point; two clean-cache
builds from the same compiler are identical; testament `arc`, `destructor`,
`macros`, `template`, `iter`, `closure`, `ccg`, `codegen`, `types` and `effects`
pass, and `generics`, `ic` and `stdlib` show exactly the pre-existing failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
araq
2026-08-25 19:24:08 +02:00
parent 8ca7b75b8b
commit c01e58c146
6 changed files with 341 additions and 30 deletions

View File

@@ -1330,6 +1330,29 @@ var sigTag = registerTag("sig")
# by construction (replaces relying on the `@bk` module-marker bit, which the
# loader dropped on type USES). Mirrors NIF's `.unusedname` directive.
var unusedIdTag = registerTag("unusedid")
# `(modflags <int>)` — the MODULE symbol's backend-relevant flags. Only
# `sfInjectDestructors` (bit 0) so far: sempass2 sets it on the module sym when
# the module's TOP-LEVEL statements need the destructor pass, and `cgen.
# genTopLevelStmt` gates `injectDestructorCalls` on it. `moduleFromNifFile`
# builds the module PSym from scratch, so without this record the flag was lost
# and a NIF-loaded module's top-level locals were never destroyed (`block: let
# h = openHandle()` leaked, silently and only under `nim ic`).
const ModFlagInjectDestructors* = 1'i32
var modFlagsTag = registerTag("modflags")
# `(nflags <ident> <symuse>)` — an `nkSym` NODE's own flags. A sym node is
# normally emitted as a bare NIF `SymUse` token, which has nowhere to put them,
# so every node flag on a sym use was silently dropped. Two of those flags are
# the frontend's move/first-write analysis results (`nfFirstWrite`, `nfLastRead`,
# both listed in `PersistentNodeFlags`) that `injectdestructors` reads in the
# backend: without them EVERY first assignment to a destructor-bearing local
# compiled as `=sink` (i.e. `=destroy` on still-zeroed memory, then a copy)
# instead of a plain construction, and no read was ever recognised as a move.
# Only wrap when there is something to say, so the common sym use stays a bare
# token.
const symNodeFlagsTagName = "nflags"
var symNodeFlagsTag = registerTag(symNodeFlagsTagName)
const PersistedSymNodeFlags = PersistentNodeFlags - {nfLazyType, nfHasComment}
proc registerNifAstTags*() =
## (Re)registers ast2nif's NIF tags explicitly. The top-level `registerTag`
@@ -1345,6 +1368,8 @@ proc registerNifAstTags*() =
tdefTag = registerTag(typeDefTagName)
hiddenTypeTag = registerTag(hiddenTypeTagName)
bindingIdTag = registerTag(bindingIdTagName)
modFlagsTag = registerTag("modflags")
symNodeFlagsTag = registerTag(symNodeFlagsTagName)
replayTag = registerTag("replay")
repConverterTag = registerTag("repconverter")
repDestroyTag = registerTag("repdestroy")
@@ -1437,7 +1462,14 @@ proc writeNode(w: var Writer; dest: var IcBuilder; n: PNode; forAst = false) =
w.withNode dest, n:
dest.addIdent n.ident.s
of nkSym:
writeSymNode(w, dest, n, n.sym)
let persisted = n.flags * PersistedSymNodeFlags
if persisted == {}:
writeSymNode(w, dest, n, n.sym)
else:
dest.addParLe symNodeFlagsTag, trLineInfo(w, n.info)
writeFlags(dest, persisted)
writeSymNode(w, dest, n, n.sym)
dest.addParRi
of nkCharLit:
w.withNode dest, n:
dest.add charToken(n.intVal.char, NoLineInfo)
@@ -2085,7 +2117,8 @@ proc writeNifModule*(config: ConfigRef; thisModule: int32; n: PNode;
typeOffers: seq[tuple[generic: PSym; inst: PType]] = @[];
resolvedImportDeps: seq[FileIndex] = @[];
firstUnusedId: int32 = 0;
expansions: seq[(PSym, TLineInfo)] = @[]) =
expansions: seq[(PSym, TLineInfo)] = @[];
moduleFlags: int32 = 0) =
var w = Writer(infos: newLineInfoWriter(config), currentModule: thisModule)
w.deps = newIcBuilder(64)
var content = newIcBuilder(300)
@@ -2239,6 +2272,10 @@ proc writeNifModule*(config: ConfigRef; thisModule: int32; n: PNode;
dest.addParLe unusedIdTag, NoLineInfo
dest.addIntLit firstUnusedId.int64
dest.addParRi()
# The module symbol's backend-relevant flags (see `(modflags)`).
dest.addParLe modFlagsTag, NoLineInfo
dest.addIntLit moduleFlags.int64
dest.addParRi()
addAll(dest, w.deps)
# do not write the (stmts .. ) wrapper:
addStmtsBody(dest, content)
@@ -3259,6 +3296,13 @@ proc loadNode(c: var DecodeContext; n: var Cursor; thisModule: string;
loadSymFromCursor(c, sym, n, thisModule, localSyms)
result = newSymNode(sym, info)
result.flags.incl nfLazyType
elif tagIs(n, symNodeFlagsTagName):
# `(nflags <ident> <symuse>)`: node flags for the wrapped sym use.
n.into:
let flags = loadAtom(TNodeFlags, n)
result = loadNode(c, n, thisModule, localSyms)
if result != nil: result.flags = result.flags + flags
while n.hasMore: skip n
elif tagIs(n, typeDefTagName):
raiseAssert "`td` tag in invalid context"
elif tagIs(n, "none"):
@@ -3592,6 +3636,8 @@ type
## `typeInstCache` from them so a consumer reuses the baked instance
## (e.g. a `mixin`/`compiles()`-dependent array bound) instead of
## re-instantiating it with a different bound in its own scope.
moduleFlags*: int32 ## the module SYMBOL's backend-relevant flags; see
## `(modflags)` / `ModFlagInjectDestructors`.
includes*: seq[string] # resolved full paths of files this module `include`s;
# replayed into `inclToMod` by modulegraphs.nim so that
# nimsuggest can map a query in an include file back to
@@ -3737,6 +3783,12 @@ proc processTopLevel(c: var DecodeContext; cur: var Cursor; flags: set[LoadFlag]
# backend id seed — consumed eagerly by `moduleId`/`readUnusedId`; just
# skip past it here so the rest of the header still loads.
skip cur
elif tagIs(cur, "modflags"):
cur.into:
if cur.hasMore and cur.kind == IntLit:
result.moduleFlags = int32 intVal(cur)
skip cur
while cur.hasMore: skip cur
elif tagIs(cur, "repconverter"): loadLogOp(c, result.logOps, cur, ConverterEntry, attachedTrace, module)
elif tagIs(cur, "repdestroy"): loadLogOp(c, result.logOps, cur, HookEntry, attachedDestructor, module)
elif tagIs(cur, "repwasmoved"): loadLogOp(c, result.logOps, cur, HookEntry, attachedWasMoved, module)
@@ -4010,6 +4062,10 @@ proc writeLoweredModule*(c: var DecodeContext; config: ConfigRef;
dest.addParLe unusedIdTag, NoLineInfo
dest.addIntLit loweredSeed.int64
dest.addParRi()
# Carry the module flags forward: `cg` loads THIS `.t.bif`, not the `.s.bif`.
dest.addParLe modFlagsTag, NoLineInfo
dest.addIntLit precomp.moduleFlags.int64
dest.addParRi()
addAll(dest, w.deps)
addStmtsBody(dest, content)
dest.addParRi()

View File

@@ -26,6 +26,14 @@ type
Node = ref object
files: seq[FilePair] # main file + includes
deps: seq[int] # indices into DepContext.nodes
specDeps: seq[int] # the subset of `deps` reached ONLY through a `when`
# condition the scanner could not evaluate
missingImport: string # an `import` path this module's source names, under a
# `when` the scanner could not decide, that does not
# exist on disk (empty when all resolved)
missingHardImport: string ## ditto but NOT under any undecidable `when`: the
## real compile would reach this `import`, so it is
## a genuine "cannot open file" error
id: int
DepContext = object
@@ -41,6 +49,9 @@ type
scanningMain: bool # currently scanning the project main module's deps;
# makes `when isMainModule` conditions evaluate true
# only there (every other module is imported)
speculating: int # nesting depth of `when` guards the scanner could not
# decide; every import edge added while this is > 0 is
# recorded as speculative (see pruneDeadSpeculative)
proc toPair(c: DepContext; f: string): FilePair =
FilePair(nimFile: f, modname: moduleSuffix(f, cast[seq[string]](c.config.searchPaths)))
@@ -206,6 +217,18 @@ proc getsImplicitImports(c: DepContext; nimFile: string): bool =
## system.nim and never reaches them). Stdlib == under conf.libpath.
not isRelativeTo(nimFile, c.config.libpath.string)
proc addDepEdge(c: DepContext; current: Node; depId: int) =
## Record `current -> depId`. While the scanner is inside a `when` guard it
## could not evaluate (`c.speculating > 0`) the edge is *speculative*: it may
## not exist in the real compile at all. An edge seen at least once outside
## such a guard is hard and stays hard.
if depId notin current.deps: current.deps.add depId
if c.speculating > 0:
if depId notin current.specDeps: current.specDeps.add depId
else:
let i = current.specDeps.find(depId)
if i >= 0: current.specDeps.delete i
proc processImport(c: var DepContext; importPath: string; current: Node; origin: string) =
# `origin` = the file the `import` literally appears in. Crucial for imports
# inside `include`d files: e.g. `system.nim` includes `system/excpt.nim`, which
@@ -217,6 +240,14 @@ proc processImport(c: var DepContext; importPath: string; current: Node; origin:
# only after the post-sem `.s.deps` revealed the edge.
let resolved = resolveImport(c, origin, importPath)
if resolved.len == 0 or not fileExists(resolved):
# The module does not exist on disk. Silently ignoring this is right for the
# scanner (the `import` may sit in a dead `when` branch and the real compile
# never looks at it), but remember it: `pruneDeadSpeculative` uses it to tell
# a module that is merely unused apart from one that cannot compile at all.
if c.speculating > 0:
if current.missingImport.len == 0: current.missingImport = importPath
elif current.missingHardImport.len == 0:
current.missingHardImport = importPath
return
let pair = c.toPair(resolved)
@@ -225,7 +256,7 @@ proc processImport(c: var DepContext; importPath: string; current: Node; origin:
if existingIdx == -1:
# New module - create node and process it
let newNode = Node(files: @[pair], id: c.nodes.len)
current.deps.add newNode.id
addDepEdge(c, current, newNode.id)
# Every module depends on system.nim
if c.systemNodeId >= 0:
newNode.deps.add c.systemNodeId
@@ -243,8 +274,7 @@ proc processImport(c: var DepContext; importPath: string; current: Node; origin:
traverseDeps(c, pair, newNode)
else:
# Already processed - just add dependency
if existingIdx notin current.deps:
current.deps.add existingIdx
addDepEdge(c, current, existingIdx)
proc skipSubtree(s: var Stream; first: PackedToken) =
## Consume tokens until the ParLe at `first` is balanced. Caller has
@@ -533,14 +563,19 @@ proc readDepsFile(c: var DepContext; pair: FilePair; current: Node) =
# entirely. Otherwise advance past the marker and parse the path.
t = next(s)
var live = true
var speculative = false
if t.kind == ParLe and pool.tags[t.tagId] == "when":
# whenMarkerHolds consumes everything up to and including the
# closing `)` of the `(when ...)` subtree. Drop the import only when
# the condition is PROVABLY false; a `cvUnknown` condition (e.g. an
# `else:` branch guarded by `not <unevaluatable call>`, as in
# `when tryImport x: ... else: import x`) keeps the dependency so the
# static graph never misses a real import.
live = whenMarkerHolds(c, s) != cvFalse
# static graph never misses a real import — but marks every edge it
# creates speculative, so `pruneDeadSpeculative` can still drop a
# subtree that provably cannot compile in this configuration.
let cond = whenMarkerHolds(c, s)
live = cond != cvFalse
speculative = cond == cvUnknown
t = next(s)
if not live:
# Drain the rest of this import/include node.
@@ -558,6 +593,7 @@ proc readDepsFile(c: var DepContext; pair: FilePair; current: Node) =
# that expand to several imports. A plain `import a, b, c` lists several
# modules as siblings; a `fromimport` has a single path followed by the
# imported symbol list, which must not be treated as modules.
if speculative: inc c.speculating
if tag == "fromimport" or tag == "importexcept":
# `from m import syms` / `import m except syms`: the first child is the
# module path; the rest is the (in/ex)cluded symbol list, which must not
@@ -573,6 +609,7 @@ proc readDepsFile(c: var DepContext; pair: FilePair; current: Node) =
processInclude(c, importPath, current, pair.nimFile)
else:
processImport(c, importPath, current, pair.nimFile)
if speculative: dec c.speculating
# Drain any remaining tokens of this node (e.g. the symbol list of a
# `fromimport`), up to and including the node's closing ')'.
var depth = 1
@@ -689,6 +726,122 @@ proc traverseDeps(c: var DepContext; pair: FilePair; current: Node) =
return
readDepsFile(c, pair, current)
proc pruneDeadSpeculative(c: var DepContext) =
## Drop modules that are reachable only through a `when` guard the scanner
## cannot evaluate AND that cannot possibly compile because they import a
## module which does not exist on disk.
##
## The motivating shape is the ordinary `{.strdefine.}` backend switch:
##
## const figdrawTextBackend* {.strdefine.} = "pixie"
## when figdrawTextBackend == "harfbuzzy":
## import ./textrasters/glyphid_raster # imports `pkg/harfbuzzy`
##
## The value of that const needs sem, so `evalCondCmp` answers `cvUnknown` and
## the conservative rule keeps the import — the right call for an edge, but it
## also gives `glyphid_raster` its own `nim m` rule. The classic compiler never
## looks at that file; IC compiles it, cannot find `pkg/harfbuzzy`, and the
## whole build dies on a package the user never installed because they never
## selected that backend.
##
## Dropping is safe: if the guard *was* live, the importer's own `nim m` fails
## on the missing NIF, records the import in its `.s.deps` sidecar, and the
## discovery fixpoint re-adds the node — this time reporting the honest
## `cannot open file: pkg/harfbuzzy/raw` instead of a cascade of
## `undeclared identifier` noise.
let n = c.nodes.len
if n == 0: return
var roots = @[0]
if c.systemNodeId >= 0: roots.add c.systemNodeId
for i in c.implicitNodeIds: roots.add i
# Reachability through NON-speculative edges only: these modules are compiled
# for certain, so a missing import in them is a genuine user error to report.
var hard = newSeq[bool](n)
var stack = roots
while stack.len > 0:
let v = stack.pop()
if hard[v]: continue
hard[v] = true
for d in c.nodes[v].deps:
if d notin c.nodes[v].specDeps and not hard[d]: stack.add d
# A module the real compile DOES reach, naming an import that is not on disk,
# is a plain user error — and one nifmake cannot notice on its own: deleting
# `effects.nim` moves no mtime, so the importer's `nim m` never re-fires and
# `nim ic` happily relinked a stale binary while `nim c` said "cannot open
# file". Report it here, where the graph scan is the only thing that looks at
# import paths at all.
var reported = false
for i in 0 ..< n:
if hard[i] and c.nodes[i].missingHardImport.len > 0:
rawMessage(c.config, errGenerated,
c.nodes[i].files[0].nimFile & ": cannot open file: " &
c.nodes[i].missingHardImport)
reported = true
if reported: return
var dead = newSeq[bool](n)
var anyDead = false
for i in 0 ..< n:
if not hard[i] and c.nodes[i].missingImport.len > 0:
dead[i] = true
anyDead = true
if not anyDead: return
# Anything left reachable only through a dead node is dead too.
var alive = newSeq[bool](n)
stack = @[]
for r in roots:
if not dead[r]: stack.add r
while stack.len > 0:
let v = stack.pop()
if alive[v]: continue
alive[v] = true
for d in c.nodes[v].deps:
if not dead[d] and not alive[d]: stack.add d
var cascaded = 0
for i in 0 ..< n:
if not alive[i]:
if c.nodes[i].missingImport.len > 0:
rawMessage(c.config, hintSuccess,
"ic: skipping " & c.nodes[i].files[0].nimFile &
" (reached only under an undecidable `when`, and imports " &
c.nodes[i].missingImport & ", which is not installed)")
else:
inc cascaded
if cascaded > 0:
rawMessage(c.config, hintSuccess,
"ic: " & $cascaded & " further module(s) skipped, reachable only through those")
# Compact `c.nodes`; node ids ARE indices everywhere, so remap them all.
var remap = newSeq[int](n)
var newNodes: seq[Node] = @[]
for i in 0 ..< n:
if alive[i]:
remap[i] = newNodes.len
newNodes.add c.nodes[i]
else:
remap[i] = -1
proc remapped(remap: seq[int]; src: seq[int]): seq[int] =
result = @[]
for x in src:
if remap[x] >= 0 and remap[x] notin result: result.add remap[x]
for node in newNodes:
node.id = remap[node.id]
node.deps = remapped(remap, node.deps)
node.specDeps = remapped(remap, node.specDeps)
c.nodes = newNodes
var pm = initTable[string, int]()
for name, idx in c.processedModules:
if idx >= 0 and idx < n and remap[idx] >= 0: pm[name] = remap[idx]
c.processedModules = pm
if c.systemNodeId >= 0: c.systemNodeId = remap[c.systemNodeId]
c.implicitNodeIds = remapped(remap, c.implicitNodeIds)
proc computeSCCs(c: DepContext): seq[seq[int]] =
## Tarjan's strongly-connected-components over the module dependency graph
## (`node.deps`). Each returned component is a list of node indices; a module
@@ -798,6 +951,46 @@ proc computeForwardedArgs(c: DepContext): seq[string] =
# replayed (`conf.icPreparsedConfig`); `commandIc` has already guaranteed it
# exists, else it bailed.
result.add "--icPreparsedConfig:" & c.config.icPreparsedConfig
# Everything else the user typed on the `nim ic` command line. The children
# replay the project's CONFIG FILES (ic_config.cfg.nif), never the driver's
# argv, so a switch that exists only there — `--opt:speed`, `--panics:on`,
# `--experimental:…`, `--passC:…` — silently did not reach them: `nim ic
# --opt:speed` produced a byte-identical debug binary. Forward the switches
# verbatim, minus the ones that MUST differ per child (the output/cache paths,
# the command itself, and IC's own per-rule switches, which each rule sets).
const notForwarded = [
"nimcache", "out", "o", "outdir", "usenimcache", "run", "r",
"incremental", "ic", "symbolfiles", "genbif",
"icproject", "icpreparsedconfig", "icconfigout", "icgroup",
"icbackendstage", "icbackendmodule", "ismainmodule",
"help", "h", "fullhelp", "version", "v", "advanced"]
for a in commandLineParams():
if a.len < 2 or a[0] != '-': continue
var i = 1
if i < a.len and a[i] == '-': inc i
var name = ""
while i < a.len and a[i] notin {':', '='}:
name.add a[i]
inc i
if normalize(name) notin notForwarded and a notin result:
result.add a
proc configSignatureFile(c: DepContext; forwardedArgs: seq[string]): string =
## nifmake decides staleness from file mtimes alone — it never looks at a
## rule's command line. So changing `-d:someDefine`, `--mm:` or `--threads:`
## between two `nim ic` runs re-generated the build file with the new switches
## but re-fired nothing: the user got a silently stale binary built with the
## OLD configuration. Reify the configuration as a FILE and make every rule
## that consumes it an input, so a config change moves an mtime like any edit.
## Written `OnlyIfChanged` so a genuine no-op run stays a no-op.
result = getNimcacheDir(c.config).string / "ic_build_args.txt"
var content = ""
for p in c.config.searchPaths:
content.add "--path:" & p.string & "\n"
for a in forwardedArgs:
content.add a & "\n"
if not fileExists(result) or readFile(result) != content:
writeFile(result, content)
proc generateFrontendBuildFile(c: DepContext; forwardedArgs: seq[string]): string =
## Frontend build file: the nifler (parse) and `nim m` (sem) rules only. The
@@ -878,6 +1071,7 @@ proc generateFrontendBuildFile(c: DepContext; forwardedArgs: seq[string]): strin
# a NIF for each. Only dependencies *outside* the component become build-graph
# inputs — intra-component edges are produced by this very rule and listing
# them would reintroduce the cycle nifmake just rejected.
let argsFile = configSignatureFile(c, forwardedArgs)
let sccs = computeSCCs(c)
var sccOf = newSeq[int](c.nodes.len)
for sccId, comp in sccs:
@@ -906,6 +1100,10 @@ proc generateFrontendBuildFile(c: DepContext; forwardedArgs: seq[string]): strin
# Input 0 (the project file passed to `nim m`): the representative's .nim.
b.withTree "input":
b.addStrLit repPair.nimFile
# The configuration this child is invoked with (see configSignatureFile).
b.addTree "input"
b.addStrLit argsFile
b.endTree()
# All parsed files of every member (nifler outputs this group consumes).
for m in members:
for f in c.nodes[m].files:
@@ -1096,6 +1294,8 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
if prunedStale:
removeFile(mergeFile)
let argsFile = configSignatureFile(c, forwardedArgs)
var b = nifbuilder.open(result)
defer: b.close()
@@ -1153,6 +1353,7 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
b.addStrLit "--icBackendStage:lower"
b.addStrLit "--icBackendModule:" & node.files[0].modname
inputStr c.semmedFile(node.files[0])
inputStr argsFile
outputStr tFiles[i]
b.endTree()
@@ -1174,6 +1375,7 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
b.addStrLit "--icBackendStage:cg"
b.addStrLit "--icBackendModule:" & node.files[0].modname
inputStr tFiles[i]
inputStr argsFile
if node.id == 0:
for j in 0 ..< c.nodes.len:
if c.nodes[j].id != 0 and live[j]:
@@ -1221,11 +1423,49 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
b.addStrLit "--out:" & exeFile
for i in 0 ..< c.nodes.len:
if live[i]: inputStr cFiles[i]
inputStr argsFile
outputStr exeFile
b.endTree()
b.endTree() # stmts
proc deriveFromSemDeps(c: var DepContext): bool =
## Fold every already-compiled module's `.s.deps` sidecar (its REAL post-sem
## imports, macro-generated ones included) back into the graph. Returns true
## if anything new was added.
##
## Run BEFORE the first nifmake pass as well as after a failure. The static
## scanner cannot see `parseStmt("import dyn")`, so on the run that first hits
## it the frontend fails, this recovers the node, and the retry succeeds. But
## the graph is rebuilt from scratch on every `nim ic`, so on the NEXT run the
## frontend succeeds on round one — with `dyn` absent from the graph again,
## hence with no nifler/`nim m` rule of its own and no edge into its importer.
## Editing `dyn.nim` then changed nothing at all: the build silently reused the
## `.s.bif` from the run that discovered it. Seeding from the sidecars makes
## the discovery stick across runs.
result = false
let n0 = c.nodes.len # snapshot: new nodes are traversed as they're added
for ni in 0 ..< n0:
for p in readSemDeps(c, c.nodes[ni].files[0]):
let pair = c.toPair(p)
var idx = c.processedModules.getOrDefault(pair.modname, -1)
if idx == -1:
if not fileExists(pair.nimFile): continue
let newNode = Node(files: @[pair], id: c.nodes.len)
if c.systemNodeId >= 0:
newNode.deps.add c.systemNodeId
if getsImplicitImports(c, pair.nimFile):
for impId in c.implicitNodeIds:
if impId != newNode.id: newNode.deps.add impId
c.processedModules[pair.modname] = newNode.id
c.nodes.add newNode
idx = newNode.id
traverseDeps(c, pair, newNode)
result = true
if idx != ni and idx notin c.nodes[ni].deps:
c.nodes[ni].deps.add idx
result = true
proc commandIc*(conf: ConfigRef; frontendOnly = false) =
## Main entry point for `nim ic`. With `frontendOnly` (used by `nim track` for
## IDE queries) it runs only Phase 1 — the incremental nifler + `nim m`
@@ -1323,6 +1563,16 @@ proc commandIc*(conf: ConfigRef; frontendOnly = false) =
# Process dependencies
traverseDeps(c, rootPair, rootNode)
# Modules that only a `when` the scanner cannot decide pulls in, and that
# import something not installed, are dead in this configuration; scheduling
# them would fail the build over code the classic compiler never reads.
pruneDeadSpeculative(c)
# Re-apply what earlier runs discovered post-sem (macro-generated imports),
# so those modules keep their rules on a warm build instead of vanishing from
# the graph until the next failure. No-op on a cold cache.
discard deriveFromSemDeps(c)
# Discovery via `.s.deps`: imports GENERATED by macros (chronicles builds
# `import chronicles/textlines` via parseStmt from the chronicles_sinks
# define) are invisible to the static scanner. Each `nim m` records the
@@ -1393,26 +1643,7 @@ proc commandIc*(conf: ConfigRef; frontendOnly = false) =
var discovered = false
inc rounds
if rounds <= 20:
let n0 = c.nodes.len # snapshot: new nodes are traversed as they're added
for ni in 0 ..< n0:
for p in readSemDeps(c, c.nodes[ni].files[0]):
let pair = c.toPair(p)
var idx = c.processedModules.getOrDefault(pair.modname, -1)
if idx == -1:
let newNode = Node(files: @[pair], id: c.nodes.len)
if c.systemNodeId >= 0:
newNode.deps.add c.systemNodeId
if getsImplicitImports(c, pair.nimFile):
for impId in c.implicitNodeIds:
if impId != newNode.id: newNode.deps.add impId
c.processedModules[pair.modname] = newNode.id
c.nodes.add newNode
idx = newNode.id
traverseDeps(c, pair, newNode)
discovered = true
if idx != ni and idx notin c.nodes[ni].deps:
c.nodes[ni].deps.add idx
discovered = true
discovered = deriveFromSemDeps(c)
if not discovered:
rawMessage(conf, errGenerated, "nifmake failed with exit code: " & $exitCode)
break

View File

@@ -1119,6 +1119,11 @@ proc p(n: PNode; c: var Con; s: var Scope; mode: ProcessMode; tmpFlags = {sfSing
result[i] = n[i]
of nkGotoState, nkState, nkAsmStmt:
result = n
of nkReplayAction:
# A `.rod`/NIF replay record. It only ever appears in a NIF-loaded
# module's TOP-LEVEL statements (the loader prepends the `(replay ...)`
# entries there); cgen discards it, so pass it through untouched.
result = n
else:
result = nil
internalError(c.graph.config, n.info, "cannot inject destructors to node kind: " & $n.kind)

View File

@@ -1070,6 +1070,10 @@ when not defined(nimKochBootstrap):
g.ifaces[fileIdx.int].interf,
g.ifaces[fileIdx.int].interfHidden, flags)
result.module = m
# Restore the module symbol's persisted flags (see ast2nif `(modflags)`);
# `cgen.genTopLevelStmt` gates the destructor pass on `sfInjectDestructors`.
if (result.moduleFlags and ModFlagInjectDestructors) != 0:
m.incl sfInjectDestructors
for (mname, msuffix) in result.reexportedModules:
let ms = materializeReexportedModule(g, mname, msuffix)
if ms != nil:

View File

@@ -29,7 +29,7 @@ const
nimEnableCovariance* = defined(nimEnableCovariance)
icFormatVersion* = "34"
icFormatVersion* = "35"
## Version of the IC cache format (the sem-NIF module layout written by
## ast2nif.nim plus the iface/impl/edges side files). Bump it whenever
## that layout changes: `commandIc` wipes a nimcache whose `ic.version`

View File

@@ -248,7 +248,15 @@ proc processPipelineModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator
# current strongly-connected import group (`--icGroup`) are the exception:
# they are compiled from source here, so each must write its own NIF.
let shouldWriteNif =
if graph.config.ideActive:
if graph.config.errorCounter > 0:
# Never persist an artifact built from erroneous AST. `nim m` does exit
# non-zero, but its outputs would still land on disk NEWER than their
# inputs, so nifmake sees the rule as satisfied on the next run: the
# build then "succeeds" from a poisoned NIF — a silently wrong binary,
# or an internal error once codegen meets an `nkError` body. Leaving the
# outputs missing keeps the rule dirty so it re-fires and re-reports.
false
elif graph.config.ideActive:
# nimsuggest (cmdM): persist NIF for cleanly-compiled, SAVED modules so
# later queries load them instead of recompiling. Never persist the
# actively edited buffer (it may hold unsaved/incomplete code) nor a
@@ -320,10 +328,17 @@ proc processPipelineModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator
let firstUnusedId = max(idgen.symId, idgen.typeId)
var expansions: seq[(PSym, TLineInfo)] = @[]
discard graph.nifExpansions.take(module.position.int32, expansions)
# The module symbol's own backend-relevant flags. `sfInjectDestructors` is
# set by sempass2 when the module's TOP-LEVEL statements need the
# destructor pass; `moduleFromNifFile` builds a fresh module PSym, so
# without persisting it `cgen.genTopLevelStmt` skipped
# `injectDestructorCalls` and top-level locals were never destroyed.
let moduleFlags =
if sfInjectDestructors in module.flags: ModFlagInjectDestructors else: 0'i32
writeNifModule(graph.config, module.position.int32, topLevelStmts, graph.opsLog,
replayActions, implDeps, reexportedModuleSyms(graph, module),
genericOffers, typeOffers, resolvedImportDeps, firstUnusedId,
expansions)
expansions, moduleFlags)
# The module's REAL direct imports (incl. macro-generated) for `nim ic`'s
# graph re-derivation; see ast2nif.writeSemDeps / semdata.addImportFileDep.
var semDepPaths: seq[string] = @[]