mirror of
https://github.com/nim-lang/Nim.git
synced 2026-09-01 19:33:42 +00:00
IC: eight correctness fixes found by differential testing against nim c
Grinding a small figdraw-based program under `nim ic` and diffing its output against the classic backend surfaced eight bugs, four of which silently produced a wrong binary rather than an error. Frontend / build graph (`deps.nim`): * Dead `when`-guarded imports were compiled anyway. `when someStrdefine == "x": import y` is `cvUnknown` to the scanner, which conservatively keeps the edge — right for an edge, but it also gave `y` its own `nim m` rule, so a build died on a package the user never installed because they never selected that backend. Track which edges are speculative and drop a speculative subtree that cannot compile; if the guard was in fact live, the discovery fixpoint puts the node back with the honest `cannot open file`. * Deleting a still-imported module went unnoticed: no mtime moves, so nothing re-fires and `nim ic` relinked a stale binary while `nim c` reported `cannot open file`. Report an unresolvable import from a non-speculatively reached module during the graph scan. * Macro-generated imports were discovered once and then forgotten. Discovery only ran after a failure and the graph is re-derived statically every run, so on a warm build the discovered module had no rules at all and editing it changed nothing. Seed the graph from the `.s.deps` sidecars up front. * Config changes invalidated nothing. nifmake decides staleness from file mtimes and never looks at a rule's command line, so `-d:foo=bar` / `--mm:` / `--threads:` regenerated the build file with the new switches and re-fired zero rules. Reify the configuration as a file and make it an input of every rule. * Command-line switches never reached the children: they replay the project's config files, never the driver's argv, so `nim ic --opt:speed` produced a byte-identical debug binary (likewise `--panics`, `--experimental`, `--passC`). Forward the driver's switches, minus the ones that must differ per child. Artifacts and codegen: * A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw the rule as satisfied on the next run: `nim ic` then reported success for a program that does not compile, and generated code from error-bearing AST (or hit an internal error in `ccgexprs`). Never persist an artifact when `errorCounter > 0`. * Top-level destructors were never injected. `sfInjectDestructors` lives on the module symbol, which `moduleFromNifFile` rebuilds from scratch, so `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a module-level `block: let h = openHandle()` never ran `=destroy`. Persist the flag as a `(modflags)` record. `injectdestructors` also has to tolerate the `nkReplayAction` entries the loader prepends to `topLevel`. * `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym node is written as a bare NIF `SymUse` token, which has nowhere to put node flags, so the frontend's move analysis never reached the backend: EVERY first assignment to a destructor-bearing local compiled as `=sink`, i.e. `=destroy` on still-zeroed memory followed by a copy, and no read was ever a move. Wrap a sym use in `(nflags ...)` when it carries persistent node flags. `icFormatVersion` 34 -> 35 for the two new NIF records. Validation: `koch bootic` reaches its byte-identical fixed point; two clean-cache builds from the same compiler are identical; testament `arc`, `destructor`, `macros`, `template`, `iter`, `closure`, `ccg`, `codegen`, `types` and `effects` pass, and `generics`, `ic` and `stdlib` show exactly the pre-existing failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1330,6 +1330,29 @@ var sigTag = registerTag("sig")
|
||||
# by construction (replaces relying on the `@bk` module-marker bit, which the
|
||||
# loader dropped on type USES). Mirrors NIF's `.unusedname` directive.
|
||||
var unusedIdTag = registerTag("unusedid")
|
||||
# `(modflags <int>)` — the MODULE symbol's backend-relevant flags. Only
|
||||
# `sfInjectDestructors` (bit 0) so far: sempass2 sets it on the module sym when
|
||||
# the module's TOP-LEVEL statements need the destructor pass, and `cgen.
|
||||
# genTopLevelStmt` gates `injectDestructorCalls` on it. `moduleFromNifFile`
|
||||
# builds the module PSym from scratch, so without this record the flag was lost
|
||||
# and a NIF-loaded module's top-level locals were never destroyed (`block: let
|
||||
# h = openHandle()` leaked, silently and only under `nim ic`).
|
||||
const ModFlagInjectDestructors* = 1'i32
|
||||
var modFlagsTag = registerTag("modflags")
|
||||
|
||||
# `(nflags <ident> <symuse>)` — an `nkSym` NODE's own flags. A sym node is
|
||||
# normally emitted as a bare NIF `SymUse` token, which has nowhere to put them,
|
||||
# so every node flag on a sym use was silently dropped. Two of those flags are
|
||||
# the frontend's move/first-write analysis results (`nfFirstWrite`, `nfLastRead`,
|
||||
# both listed in `PersistentNodeFlags`) that `injectdestructors` reads in the
|
||||
# backend: without them EVERY first assignment to a destructor-bearing local
|
||||
# compiled as `=sink` (i.e. `=destroy` on still-zeroed memory, then a copy)
|
||||
# instead of a plain construction, and no read was ever recognised as a move.
|
||||
# Only wrap when there is something to say, so the common sym use stays a bare
|
||||
# token.
|
||||
const symNodeFlagsTagName = "nflags"
|
||||
var symNodeFlagsTag = registerTag(symNodeFlagsTagName)
|
||||
const PersistedSymNodeFlags = PersistentNodeFlags - {nfLazyType, nfHasComment}
|
||||
|
||||
proc registerNifAstTags*() =
|
||||
## (Re)registers ast2nif's NIF tags explicitly. The top-level `registerTag`
|
||||
@@ -1345,6 +1368,8 @@ proc registerNifAstTags*() =
|
||||
tdefTag = registerTag(typeDefTagName)
|
||||
hiddenTypeTag = registerTag(hiddenTypeTagName)
|
||||
bindingIdTag = registerTag(bindingIdTagName)
|
||||
modFlagsTag = registerTag("modflags")
|
||||
symNodeFlagsTag = registerTag(symNodeFlagsTagName)
|
||||
replayTag = registerTag("replay")
|
||||
repConverterTag = registerTag("repconverter")
|
||||
repDestroyTag = registerTag("repdestroy")
|
||||
@@ -1437,7 +1462,14 @@ proc writeNode(w: var Writer; dest: var IcBuilder; n: PNode; forAst = false) =
|
||||
w.withNode dest, n:
|
||||
dest.addIdent n.ident.s
|
||||
of nkSym:
|
||||
writeSymNode(w, dest, n, n.sym)
|
||||
let persisted = n.flags * PersistedSymNodeFlags
|
||||
if persisted == {}:
|
||||
writeSymNode(w, dest, n, n.sym)
|
||||
else:
|
||||
dest.addParLe symNodeFlagsTag, trLineInfo(w, n.info)
|
||||
writeFlags(dest, persisted)
|
||||
writeSymNode(w, dest, n, n.sym)
|
||||
dest.addParRi
|
||||
of nkCharLit:
|
||||
w.withNode dest, n:
|
||||
dest.add charToken(n.intVal.char, NoLineInfo)
|
||||
@@ -2085,7 +2117,8 @@ proc writeNifModule*(config: ConfigRef; thisModule: int32; n: PNode;
|
||||
typeOffers: seq[tuple[generic: PSym; inst: PType]] = @[];
|
||||
resolvedImportDeps: seq[FileIndex] = @[];
|
||||
firstUnusedId: int32 = 0;
|
||||
expansions: seq[(PSym, TLineInfo)] = @[]) =
|
||||
expansions: seq[(PSym, TLineInfo)] = @[];
|
||||
moduleFlags: int32 = 0) =
|
||||
var w = Writer(infos: newLineInfoWriter(config), currentModule: thisModule)
|
||||
w.deps = newIcBuilder(64)
|
||||
var content = newIcBuilder(300)
|
||||
@@ -2239,6 +2272,10 @@ proc writeNifModule*(config: ConfigRef; thisModule: int32; n: PNode;
|
||||
dest.addParLe unusedIdTag, NoLineInfo
|
||||
dest.addIntLit firstUnusedId.int64
|
||||
dest.addParRi()
|
||||
# The module symbol's backend-relevant flags (see `(modflags)`).
|
||||
dest.addParLe modFlagsTag, NoLineInfo
|
||||
dest.addIntLit moduleFlags.int64
|
||||
dest.addParRi()
|
||||
addAll(dest, w.deps)
|
||||
# do not write the (stmts .. ) wrapper:
|
||||
addStmtsBody(dest, content)
|
||||
@@ -3259,6 +3296,13 @@ proc loadNode(c: var DecodeContext; n: var Cursor; thisModule: string;
|
||||
loadSymFromCursor(c, sym, n, thisModule, localSyms)
|
||||
result = newSymNode(sym, info)
|
||||
result.flags.incl nfLazyType
|
||||
elif tagIs(n, symNodeFlagsTagName):
|
||||
# `(nflags <ident> <symuse>)`: node flags for the wrapped sym use.
|
||||
n.into:
|
||||
let flags = loadAtom(TNodeFlags, n)
|
||||
result = loadNode(c, n, thisModule, localSyms)
|
||||
if result != nil: result.flags = result.flags + flags
|
||||
while n.hasMore: skip n
|
||||
elif tagIs(n, typeDefTagName):
|
||||
raiseAssert "`td` tag in invalid context"
|
||||
elif tagIs(n, "none"):
|
||||
@@ -3592,6 +3636,8 @@ type
|
||||
## `typeInstCache` from them so a consumer reuses the baked instance
|
||||
## (e.g. a `mixin`/`compiles()`-dependent array bound) instead of
|
||||
## re-instantiating it with a different bound in its own scope.
|
||||
moduleFlags*: int32 ## the module SYMBOL's backend-relevant flags; see
|
||||
## `(modflags)` / `ModFlagInjectDestructors`.
|
||||
includes*: seq[string] # resolved full paths of files this module `include`s;
|
||||
# replayed into `inclToMod` by modulegraphs.nim so that
|
||||
# nimsuggest can map a query in an include file back to
|
||||
@@ -3737,6 +3783,12 @@ proc processTopLevel(c: var DecodeContext; cur: var Cursor; flags: set[LoadFlag]
|
||||
# backend id seed — consumed eagerly by `moduleId`/`readUnusedId`; just
|
||||
# skip past it here so the rest of the header still loads.
|
||||
skip cur
|
||||
elif tagIs(cur, "modflags"):
|
||||
cur.into:
|
||||
if cur.hasMore and cur.kind == IntLit:
|
||||
result.moduleFlags = int32 intVal(cur)
|
||||
skip cur
|
||||
while cur.hasMore: skip cur
|
||||
elif tagIs(cur, "repconverter"): loadLogOp(c, result.logOps, cur, ConverterEntry, attachedTrace, module)
|
||||
elif tagIs(cur, "repdestroy"): loadLogOp(c, result.logOps, cur, HookEntry, attachedDestructor, module)
|
||||
elif tagIs(cur, "repwasmoved"): loadLogOp(c, result.logOps, cur, HookEntry, attachedWasMoved, module)
|
||||
@@ -4010,6 +4062,10 @@ proc writeLoweredModule*(c: var DecodeContext; config: ConfigRef;
|
||||
dest.addParLe unusedIdTag, NoLineInfo
|
||||
dest.addIntLit loweredSeed.int64
|
||||
dest.addParRi()
|
||||
# Carry the module flags forward: `cg` loads THIS `.t.bif`, not the `.s.bif`.
|
||||
dest.addParLe modFlagsTag, NoLineInfo
|
||||
dest.addIntLit precomp.moduleFlags.int64
|
||||
dest.addParRi()
|
||||
addAll(dest, w.deps)
|
||||
addStmtsBody(dest, content)
|
||||
dest.addParRi()
|
||||
|
||||
@@ -26,6 +26,14 @@ type
|
||||
Node = ref object
|
||||
files: seq[FilePair] # main file + includes
|
||||
deps: seq[int] # indices into DepContext.nodes
|
||||
specDeps: seq[int] # the subset of `deps` reached ONLY through a `when`
|
||||
# condition the scanner could not evaluate
|
||||
missingImport: string # an `import` path this module's source names, under a
|
||||
# `when` the scanner could not decide, that does not
|
||||
# exist on disk (empty when all resolved)
|
||||
missingHardImport: string ## ditto but NOT under any undecidable `when`: the
|
||||
## real compile would reach this `import`, so it is
|
||||
## a genuine "cannot open file" error
|
||||
id: int
|
||||
|
||||
DepContext = object
|
||||
@@ -41,6 +49,9 @@ type
|
||||
scanningMain: bool # currently scanning the project main module's deps;
|
||||
# makes `when isMainModule` conditions evaluate true
|
||||
# only there (every other module is imported)
|
||||
speculating: int # nesting depth of `when` guards the scanner could not
|
||||
# decide; every import edge added while this is > 0 is
|
||||
# recorded as speculative (see pruneDeadSpeculative)
|
||||
|
||||
proc toPair(c: DepContext; f: string): FilePair =
|
||||
FilePair(nimFile: f, modname: moduleSuffix(f, cast[seq[string]](c.config.searchPaths)))
|
||||
@@ -206,6 +217,18 @@ proc getsImplicitImports(c: DepContext; nimFile: string): bool =
|
||||
## system.nim and never reaches them). Stdlib == under conf.libpath.
|
||||
not isRelativeTo(nimFile, c.config.libpath.string)
|
||||
|
||||
proc addDepEdge(c: DepContext; current: Node; depId: int) =
|
||||
## Record `current -> depId`. While the scanner is inside a `when` guard it
|
||||
## could not evaluate (`c.speculating > 0`) the edge is *speculative*: it may
|
||||
## not exist in the real compile at all. An edge seen at least once outside
|
||||
## such a guard is hard and stays hard.
|
||||
if depId notin current.deps: current.deps.add depId
|
||||
if c.speculating > 0:
|
||||
if depId notin current.specDeps: current.specDeps.add depId
|
||||
else:
|
||||
let i = current.specDeps.find(depId)
|
||||
if i >= 0: current.specDeps.delete i
|
||||
|
||||
proc processImport(c: var DepContext; importPath: string; current: Node; origin: string) =
|
||||
# `origin` = the file the `import` literally appears in. Crucial for imports
|
||||
# inside `include`d files: e.g. `system.nim` includes `system/excpt.nim`, which
|
||||
@@ -217,6 +240,14 @@ proc processImport(c: var DepContext; importPath: string; current: Node; origin:
|
||||
# only after the post-sem `.s.deps` revealed the edge.
|
||||
let resolved = resolveImport(c, origin, importPath)
|
||||
if resolved.len == 0 or not fileExists(resolved):
|
||||
# The module does not exist on disk. Silently ignoring this is right for the
|
||||
# scanner (the `import` may sit in a dead `when` branch and the real compile
|
||||
# never looks at it), but remember it: `pruneDeadSpeculative` uses it to tell
|
||||
# a module that is merely unused apart from one that cannot compile at all.
|
||||
if c.speculating > 0:
|
||||
if current.missingImport.len == 0: current.missingImport = importPath
|
||||
elif current.missingHardImport.len == 0:
|
||||
current.missingHardImport = importPath
|
||||
return
|
||||
|
||||
let pair = c.toPair(resolved)
|
||||
@@ -225,7 +256,7 @@ proc processImport(c: var DepContext; importPath: string; current: Node; origin:
|
||||
if existingIdx == -1:
|
||||
# New module - create node and process it
|
||||
let newNode = Node(files: @[pair], id: c.nodes.len)
|
||||
current.deps.add newNode.id
|
||||
addDepEdge(c, current, newNode.id)
|
||||
# Every module depends on system.nim
|
||||
if c.systemNodeId >= 0:
|
||||
newNode.deps.add c.systemNodeId
|
||||
@@ -243,8 +274,7 @@ proc processImport(c: var DepContext; importPath: string; current: Node; origin:
|
||||
traverseDeps(c, pair, newNode)
|
||||
else:
|
||||
# Already processed - just add dependency
|
||||
if existingIdx notin current.deps:
|
||||
current.deps.add existingIdx
|
||||
addDepEdge(c, current, existingIdx)
|
||||
|
||||
proc skipSubtree(s: var Stream; first: PackedToken) =
|
||||
## Consume tokens until the ParLe at `first` is balanced. Caller has
|
||||
@@ -533,14 +563,19 @@ proc readDepsFile(c: var DepContext; pair: FilePair; current: Node) =
|
||||
# entirely. Otherwise advance past the marker and parse the path.
|
||||
t = next(s)
|
||||
var live = true
|
||||
var speculative = false
|
||||
if t.kind == ParLe and pool.tags[t.tagId] == "when":
|
||||
# whenMarkerHolds consumes everything up to and including the
|
||||
# closing `)` of the `(when ...)` subtree. Drop the import only when
|
||||
# the condition is PROVABLY false; a `cvUnknown` condition (e.g. an
|
||||
# `else:` branch guarded by `not <unevaluatable call>`, as in
|
||||
# `when tryImport x: ... else: import x`) keeps the dependency so the
|
||||
# static graph never misses a real import.
|
||||
live = whenMarkerHolds(c, s) != cvFalse
|
||||
# static graph never misses a real import — but marks every edge it
|
||||
# creates speculative, so `pruneDeadSpeculative` can still drop a
|
||||
# subtree that provably cannot compile in this configuration.
|
||||
let cond = whenMarkerHolds(c, s)
|
||||
live = cond != cvFalse
|
||||
speculative = cond == cvUnknown
|
||||
t = next(s)
|
||||
if not live:
|
||||
# Drain the rest of this import/include node.
|
||||
@@ -558,6 +593,7 @@ proc readDepsFile(c: var DepContext; pair: FilePair; current: Node) =
|
||||
# that expand to several imports. A plain `import a, b, c` lists several
|
||||
# modules as siblings; a `fromimport` has a single path followed by the
|
||||
# imported symbol list, which must not be treated as modules.
|
||||
if speculative: inc c.speculating
|
||||
if tag == "fromimport" or tag == "importexcept":
|
||||
# `from m import syms` / `import m except syms`: the first child is the
|
||||
# module path; the rest is the (in/ex)cluded symbol list, which must not
|
||||
@@ -573,6 +609,7 @@ proc readDepsFile(c: var DepContext; pair: FilePair; current: Node) =
|
||||
processInclude(c, importPath, current, pair.nimFile)
|
||||
else:
|
||||
processImport(c, importPath, current, pair.nimFile)
|
||||
if speculative: dec c.speculating
|
||||
# Drain any remaining tokens of this node (e.g. the symbol list of a
|
||||
# `fromimport`), up to and including the node's closing ')'.
|
||||
var depth = 1
|
||||
@@ -689,6 +726,122 @@ proc traverseDeps(c: var DepContext; pair: FilePair; current: Node) =
|
||||
return
|
||||
readDepsFile(c, pair, current)
|
||||
|
||||
proc pruneDeadSpeculative(c: var DepContext) =
|
||||
## Drop modules that are reachable only through a `when` guard the scanner
|
||||
## cannot evaluate AND that cannot possibly compile because they import a
|
||||
## module which does not exist on disk.
|
||||
##
|
||||
## The motivating shape is the ordinary `{.strdefine.}` backend switch:
|
||||
##
|
||||
## const figdrawTextBackend* {.strdefine.} = "pixie"
|
||||
## when figdrawTextBackend == "harfbuzzy":
|
||||
## import ./textrasters/glyphid_raster # imports `pkg/harfbuzzy`
|
||||
##
|
||||
## The value of that const needs sem, so `evalCondCmp` answers `cvUnknown` and
|
||||
## the conservative rule keeps the import — the right call for an edge, but it
|
||||
## also gives `glyphid_raster` its own `nim m` rule. The classic compiler never
|
||||
## looks at that file; IC compiles it, cannot find `pkg/harfbuzzy`, and the
|
||||
## whole build dies on a package the user never installed because they never
|
||||
## selected that backend.
|
||||
##
|
||||
## Dropping is safe: if the guard *was* live, the importer's own `nim m` fails
|
||||
## on the missing NIF, records the import in its `.s.deps` sidecar, and the
|
||||
## discovery fixpoint re-adds the node — this time reporting the honest
|
||||
## `cannot open file: pkg/harfbuzzy/raw` instead of a cascade of
|
||||
## `undeclared identifier` noise.
|
||||
let n = c.nodes.len
|
||||
if n == 0: return
|
||||
|
||||
var roots = @[0]
|
||||
if c.systemNodeId >= 0: roots.add c.systemNodeId
|
||||
for i in c.implicitNodeIds: roots.add i
|
||||
|
||||
# Reachability through NON-speculative edges only: these modules are compiled
|
||||
# for certain, so a missing import in them is a genuine user error to report.
|
||||
var hard = newSeq[bool](n)
|
||||
var stack = roots
|
||||
while stack.len > 0:
|
||||
let v = stack.pop()
|
||||
if hard[v]: continue
|
||||
hard[v] = true
|
||||
for d in c.nodes[v].deps:
|
||||
if d notin c.nodes[v].specDeps and not hard[d]: stack.add d
|
||||
|
||||
# A module the real compile DOES reach, naming an import that is not on disk,
|
||||
# is a plain user error — and one nifmake cannot notice on its own: deleting
|
||||
# `effects.nim` moves no mtime, so the importer's `nim m` never re-fires and
|
||||
# `nim ic` happily relinked a stale binary while `nim c` said "cannot open
|
||||
# file". Report it here, where the graph scan is the only thing that looks at
|
||||
# import paths at all.
|
||||
var reported = false
|
||||
for i in 0 ..< n:
|
||||
if hard[i] and c.nodes[i].missingHardImport.len > 0:
|
||||
rawMessage(c.config, errGenerated,
|
||||
c.nodes[i].files[0].nimFile & ": cannot open file: " &
|
||||
c.nodes[i].missingHardImport)
|
||||
reported = true
|
||||
if reported: return
|
||||
|
||||
var dead = newSeq[bool](n)
|
||||
var anyDead = false
|
||||
for i in 0 ..< n:
|
||||
if not hard[i] and c.nodes[i].missingImport.len > 0:
|
||||
dead[i] = true
|
||||
anyDead = true
|
||||
if not anyDead: return
|
||||
|
||||
# Anything left reachable only through a dead node is dead too.
|
||||
var alive = newSeq[bool](n)
|
||||
stack = @[]
|
||||
for r in roots:
|
||||
if not dead[r]: stack.add r
|
||||
while stack.len > 0:
|
||||
let v = stack.pop()
|
||||
if alive[v]: continue
|
||||
alive[v] = true
|
||||
for d in c.nodes[v].deps:
|
||||
if not dead[d] and not alive[d]: stack.add d
|
||||
|
||||
var cascaded = 0
|
||||
for i in 0 ..< n:
|
||||
if not alive[i]:
|
||||
if c.nodes[i].missingImport.len > 0:
|
||||
rawMessage(c.config, hintSuccess,
|
||||
"ic: skipping " & c.nodes[i].files[0].nimFile &
|
||||
" (reached only under an undecidable `when`, and imports " &
|
||||
c.nodes[i].missingImport & ", which is not installed)")
|
||||
else:
|
||||
inc cascaded
|
||||
if cascaded > 0:
|
||||
rawMessage(c.config, hintSuccess,
|
||||
"ic: " & $cascaded & " further module(s) skipped, reachable only through those")
|
||||
|
||||
# Compact `c.nodes`; node ids ARE indices everywhere, so remap them all.
|
||||
var remap = newSeq[int](n)
|
||||
var newNodes: seq[Node] = @[]
|
||||
for i in 0 ..< n:
|
||||
if alive[i]:
|
||||
remap[i] = newNodes.len
|
||||
newNodes.add c.nodes[i]
|
||||
else:
|
||||
remap[i] = -1
|
||||
proc remapped(remap: seq[int]; src: seq[int]): seq[int] =
|
||||
result = @[]
|
||||
for x in src:
|
||||
if remap[x] >= 0 and remap[x] notin result: result.add remap[x]
|
||||
for node in newNodes:
|
||||
node.id = remap[node.id]
|
||||
node.deps = remapped(remap, node.deps)
|
||||
node.specDeps = remapped(remap, node.specDeps)
|
||||
c.nodes = newNodes
|
||||
|
||||
var pm = initTable[string, int]()
|
||||
for name, idx in c.processedModules:
|
||||
if idx >= 0 and idx < n and remap[idx] >= 0: pm[name] = remap[idx]
|
||||
c.processedModules = pm
|
||||
if c.systemNodeId >= 0: c.systemNodeId = remap[c.systemNodeId]
|
||||
c.implicitNodeIds = remapped(remap, c.implicitNodeIds)
|
||||
|
||||
proc computeSCCs(c: DepContext): seq[seq[int]] =
|
||||
## Tarjan's strongly-connected-components over the module dependency graph
|
||||
## (`node.deps`). Each returned component is a list of node indices; a module
|
||||
@@ -798,6 +951,46 @@ proc computeForwardedArgs(c: DepContext): seq[string] =
|
||||
# replayed (`conf.icPreparsedConfig`); `commandIc` has already guaranteed it
|
||||
# exists, else it bailed.
|
||||
result.add "--icPreparsedConfig:" & c.config.icPreparsedConfig
|
||||
# Everything else the user typed on the `nim ic` command line. The children
|
||||
# replay the project's CONFIG FILES (ic_config.cfg.nif), never the driver's
|
||||
# argv, so a switch that exists only there — `--opt:speed`, `--panics:on`,
|
||||
# `--experimental:…`, `--passC:…` — silently did not reach them: `nim ic
|
||||
# --opt:speed` produced a byte-identical debug binary. Forward the switches
|
||||
# verbatim, minus the ones that MUST differ per child (the output/cache paths,
|
||||
# the command itself, and IC's own per-rule switches, which each rule sets).
|
||||
const notForwarded = [
|
||||
"nimcache", "out", "o", "outdir", "usenimcache", "run", "r",
|
||||
"incremental", "ic", "symbolfiles", "genbif",
|
||||
"icproject", "icpreparsedconfig", "icconfigout", "icgroup",
|
||||
"icbackendstage", "icbackendmodule", "ismainmodule",
|
||||
"help", "h", "fullhelp", "version", "v", "advanced"]
|
||||
for a in commandLineParams():
|
||||
if a.len < 2 or a[0] != '-': continue
|
||||
var i = 1
|
||||
if i < a.len and a[i] == '-': inc i
|
||||
var name = ""
|
||||
while i < a.len and a[i] notin {':', '='}:
|
||||
name.add a[i]
|
||||
inc i
|
||||
if normalize(name) notin notForwarded and a notin result:
|
||||
result.add a
|
||||
|
||||
proc configSignatureFile(c: DepContext; forwardedArgs: seq[string]): string =
|
||||
## nifmake decides staleness from file mtimes alone — it never looks at a
|
||||
## rule's command line. So changing `-d:someDefine`, `--mm:` or `--threads:`
|
||||
## between two `nim ic` runs re-generated the build file with the new switches
|
||||
## but re-fired nothing: the user got a silently stale binary built with the
|
||||
## OLD configuration. Reify the configuration as a FILE and make every rule
|
||||
## that consumes it an input, so a config change moves an mtime like any edit.
|
||||
## Written `OnlyIfChanged` so a genuine no-op run stays a no-op.
|
||||
result = getNimcacheDir(c.config).string / "ic_build_args.txt"
|
||||
var content = ""
|
||||
for p in c.config.searchPaths:
|
||||
content.add "--path:" & p.string & "\n"
|
||||
for a in forwardedArgs:
|
||||
content.add a & "\n"
|
||||
if not fileExists(result) or readFile(result) != content:
|
||||
writeFile(result, content)
|
||||
|
||||
proc generateFrontendBuildFile(c: DepContext; forwardedArgs: seq[string]): string =
|
||||
## Frontend build file: the nifler (parse) and `nim m` (sem) rules only. The
|
||||
@@ -878,6 +1071,7 @@ proc generateFrontendBuildFile(c: DepContext; forwardedArgs: seq[string]): strin
|
||||
# a NIF for each. Only dependencies *outside* the component become build-graph
|
||||
# inputs — intra-component edges are produced by this very rule and listing
|
||||
# them would reintroduce the cycle nifmake just rejected.
|
||||
let argsFile = configSignatureFile(c, forwardedArgs)
|
||||
let sccs = computeSCCs(c)
|
||||
var sccOf = newSeq[int](c.nodes.len)
|
||||
for sccId, comp in sccs:
|
||||
@@ -906,6 +1100,10 @@ proc generateFrontendBuildFile(c: DepContext; forwardedArgs: seq[string]): strin
|
||||
# Input 0 (the project file passed to `nim m`): the representative's .nim.
|
||||
b.withTree "input":
|
||||
b.addStrLit repPair.nimFile
|
||||
# The configuration this child is invoked with (see configSignatureFile).
|
||||
b.addTree "input"
|
||||
b.addStrLit argsFile
|
||||
b.endTree()
|
||||
# All parsed files of every member (nifler outputs this group consumes).
|
||||
for m in members:
|
||||
for f in c.nodes[m].files:
|
||||
@@ -1096,6 +1294,8 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
|
||||
if prunedStale:
|
||||
removeFile(mergeFile)
|
||||
|
||||
let argsFile = configSignatureFile(c, forwardedArgs)
|
||||
|
||||
var b = nifbuilder.open(result)
|
||||
defer: b.close()
|
||||
|
||||
@@ -1153,6 +1353,7 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
|
||||
b.addStrLit "--icBackendStage:lower"
|
||||
b.addStrLit "--icBackendModule:" & node.files[0].modname
|
||||
inputStr c.semmedFile(node.files[0])
|
||||
inputStr argsFile
|
||||
outputStr tFiles[i]
|
||||
b.endTree()
|
||||
|
||||
@@ -1174,6 +1375,7 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
|
||||
b.addStrLit "--icBackendStage:cg"
|
||||
b.addStrLit "--icBackendModule:" & node.files[0].modname
|
||||
inputStr tFiles[i]
|
||||
inputStr argsFile
|
||||
if node.id == 0:
|
||||
for j in 0 ..< c.nodes.len:
|
||||
if c.nodes[j].id != 0 and live[j]:
|
||||
@@ -1221,11 +1423,49 @@ proc generateBackendBuildFile(c: DepContext; forwardedArgs: seq[string]): string
|
||||
b.addStrLit "--out:" & exeFile
|
||||
for i in 0 ..< c.nodes.len:
|
||||
if live[i]: inputStr cFiles[i]
|
||||
inputStr argsFile
|
||||
outputStr exeFile
|
||||
b.endTree()
|
||||
|
||||
b.endTree() # stmts
|
||||
|
||||
proc deriveFromSemDeps(c: var DepContext): bool =
|
||||
## Fold every already-compiled module's `.s.deps` sidecar (its REAL post-sem
|
||||
## imports, macro-generated ones included) back into the graph. Returns true
|
||||
## if anything new was added.
|
||||
##
|
||||
## Run BEFORE the first nifmake pass as well as after a failure. The static
|
||||
## scanner cannot see `parseStmt("import dyn")`, so on the run that first hits
|
||||
## it the frontend fails, this recovers the node, and the retry succeeds. But
|
||||
## the graph is rebuilt from scratch on every `nim ic`, so on the NEXT run the
|
||||
## frontend succeeds on round one — with `dyn` absent from the graph again,
|
||||
## hence with no nifler/`nim m` rule of its own and no edge into its importer.
|
||||
## Editing `dyn.nim` then changed nothing at all: the build silently reused the
|
||||
## `.s.bif` from the run that discovered it. Seeding from the sidecars makes
|
||||
## the discovery stick across runs.
|
||||
result = false
|
||||
let n0 = c.nodes.len # snapshot: new nodes are traversed as they're added
|
||||
for ni in 0 ..< n0:
|
||||
for p in readSemDeps(c, c.nodes[ni].files[0]):
|
||||
let pair = c.toPair(p)
|
||||
var idx = c.processedModules.getOrDefault(pair.modname, -1)
|
||||
if idx == -1:
|
||||
if not fileExists(pair.nimFile): continue
|
||||
let newNode = Node(files: @[pair], id: c.nodes.len)
|
||||
if c.systemNodeId >= 0:
|
||||
newNode.deps.add c.systemNodeId
|
||||
if getsImplicitImports(c, pair.nimFile):
|
||||
for impId in c.implicitNodeIds:
|
||||
if impId != newNode.id: newNode.deps.add impId
|
||||
c.processedModules[pair.modname] = newNode.id
|
||||
c.nodes.add newNode
|
||||
idx = newNode.id
|
||||
traverseDeps(c, pair, newNode)
|
||||
result = true
|
||||
if idx != ni and idx notin c.nodes[ni].deps:
|
||||
c.nodes[ni].deps.add idx
|
||||
result = true
|
||||
|
||||
proc commandIc*(conf: ConfigRef; frontendOnly = false) =
|
||||
## Main entry point for `nim ic`. With `frontendOnly` (used by `nim track` for
|
||||
## IDE queries) it runs only Phase 1 — the incremental nifler + `nim m`
|
||||
@@ -1323,6 +1563,16 @@ proc commandIc*(conf: ConfigRef; frontendOnly = false) =
|
||||
# Process dependencies
|
||||
traverseDeps(c, rootPair, rootNode)
|
||||
|
||||
# Modules that only a `when` the scanner cannot decide pulls in, and that
|
||||
# import something not installed, are dead in this configuration; scheduling
|
||||
# them would fail the build over code the classic compiler never reads.
|
||||
pruneDeadSpeculative(c)
|
||||
|
||||
# Re-apply what earlier runs discovered post-sem (macro-generated imports),
|
||||
# so those modules keep their rules on a warm build instead of vanishing from
|
||||
# the graph until the next failure. No-op on a cold cache.
|
||||
discard deriveFromSemDeps(c)
|
||||
|
||||
# Discovery via `.s.deps`: imports GENERATED by macros (chronicles builds
|
||||
# `import chronicles/textlines` via parseStmt from the chronicles_sinks
|
||||
# define) are invisible to the static scanner. Each `nim m` records the
|
||||
@@ -1393,26 +1643,7 @@ proc commandIc*(conf: ConfigRef; frontendOnly = false) =
|
||||
var discovered = false
|
||||
inc rounds
|
||||
if rounds <= 20:
|
||||
let n0 = c.nodes.len # snapshot: new nodes are traversed as they're added
|
||||
for ni in 0 ..< n0:
|
||||
for p in readSemDeps(c, c.nodes[ni].files[0]):
|
||||
let pair = c.toPair(p)
|
||||
var idx = c.processedModules.getOrDefault(pair.modname, -1)
|
||||
if idx == -1:
|
||||
let newNode = Node(files: @[pair], id: c.nodes.len)
|
||||
if c.systemNodeId >= 0:
|
||||
newNode.deps.add c.systemNodeId
|
||||
if getsImplicitImports(c, pair.nimFile):
|
||||
for impId in c.implicitNodeIds:
|
||||
if impId != newNode.id: newNode.deps.add impId
|
||||
c.processedModules[pair.modname] = newNode.id
|
||||
c.nodes.add newNode
|
||||
idx = newNode.id
|
||||
traverseDeps(c, pair, newNode)
|
||||
discovered = true
|
||||
if idx != ni and idx notin c.nodes[ni].deps:
|
||||
c.nodes[ni].deps.add idx
|
||||
discovered = true
|
||||
discovered = deriveFromSemDeps(c)
|
||||
if not discovered:
|
||||
rawMessage(conf, errGenerated, "nifmake failed with exit code: " & $exitCode)
|
||||
break
|
||||
|
||||
@@ -1119,6 +1119,11 @@ proc p(n: PNode; c: var Con; s: var Scope; mode: ProcessMode; tmpFlags = {sfSing
|
||||
result[i] = n[i]
|
||||
of nkGotoState, nkState, nkAsmStmt:
|
||||
result = n
|
||||
of nkReplayAction:
|
||||
# A `.rod`/NIF replay record. It only ever appears in a NIF-loaded
|
||||
# module's TOP-LEVEL statements (the loader prepends the `(replay ...)`
|
||||
# entries there); cgen discards it, so pass it through untouched.
|
||||
result = n
|
||||
else:
|
||||
result = nil
|
||||
internalError(c.graph.config, n.info, "cannot inject destructors to node kind: " & $n.kind)
|
||||
|
||||
@@ -1070,6 +1070,10 @@ when not defined(nimKochBootstrap):
|
||||
g.ifaces[fileIdx.int].interf,
|
||||
g.ifaces[fileIdx.int].interfHidden, flags)
|
||||
result.module = m
|
||||
# Restore the module symbol's persisted flags (see ast2nif `(modflags)`);
|
||||
# `cgen.genTopLevelStmt` gates the destructor pass on `sfInjectDestructors`.
|
||||
if (result.moduleFlags and ModFlagInjectDestructors) != 0:
|
||||
m.incl sfInjectDestructors
|
||||
for (mname, msuffix) in result.reexportedModules:
|
||||
let ms = materializeReexportedModule(g, mname, msuffix)
|
||||
if ms != nil:
|
||||
|
||||
@@ -29,7 +29,7 @@ const
|
||||
|
||||
nimEnableCovariance* = defined(nimEnableCovariance)
|
||||
|
||||
icFormatVersion* = "34"
|
||||
icFormatVersion* = "35"
|
||||
## Version of the IC cache format (the sem-NIF module layout written by
|
||||
## ast2nif.nim plus the iface/impl/edges side files). Bump it whenever
|
||||
## that layout changes: `commandIc` wipes a nimcache whose `ic.version`
|
||||
|
||||
@@ -248,7 +248,15 @@ proc processPipelineModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator
|
||||
# current strongly-connected import group (`--icGroup`) are the exception:
|
||||
# they are compiled from source here, so each must write its own NIF.
|
||||
let shouldWriteNif =
|
||||
if graph.config.ideActive:
|
||||
if graph.config.errorCounter > 0:
|
||||
# Never persist an artifact built from erroneous AST. `nim m` does exit
|
||||
# non-zero, but its outputs would still land on disk NEWER than their
|
||||
# inputs, so nifmake sees the rule as satisfied on the next run: the
|
||||
# build then "succeeds" from a poisoned NIF — a silently wrong binary,
|
||||
# or an internal error once codegen meets an `nkError` body. Leaving the
|
||||
# outputs missing keeps the rule dirty so it re-fires and re-reports.
|
||||
false
|
||||
elif graph.config.ideActive:
|
||||
# nimsuggest (cmdM): persist NIF for cleanly-compiled, SAVED modules so
|
||||
# later queries load them instead of recompiling. Never persist the
|
||||
# actively edited buffer (it may hold unsaved/incomplete code) nor a
|
||||
@@ -320,10 +328,17 @@ proc processPipelineModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator
|
||||
let firstUnusedId = max(idgen.symId, idgen.typeId)
|
||||
var expansions: seq[(PSym, TLineInfo)] = @[]
|
||||
discard graph.nifExpansions.take(module.position.int32, expansions)
|
||||
# The module symbol's own backend-relevant flags. `sfInjectDestructors` is
|
||||
# set by sempass2 when the module's TOP-LEVEL statements need the
|
||||
# destructor pass; `moduleFromNifFile` builds a fresh module PSym, so
|
||||
# without persisting it `cgen.genTopLevelStmt` skipped
|
||||
# `injectDestructorCalls` and top-level locals were never destroyed.
|
||||
let moduleFlags =
|
||||
if sfInjectDestructors in module.flags: ModFlagInjectDestructors else: 0'i32
|
||||
writeNifModule(graph.config, module.position.int32, topLevelStmts, graph.opsLog,
|
||||
replayActions, implDeps, reexportedModuleSyms(graph, module),
|
||||
genericOffers, typeOffers, resolvedImportDeps, firstUnusedId,
|
||||
expansions)
|
||||
expansions, moduleFlags)
|
||||
# The module's REAL direct imports (incl. macro-generated) for `nim ic`'s
|
||||
# graph re-derivation; see ast2nif.writeSemDeps / semdata.addImportFileDep.
|
||||
var semDepPaths: seq[string] = @[]
|
||||
|
||||
Reference in New Issue
Block a user