IC: instrument nim m, and find 1.05s of interface stubs nobody reads

The frontend was the one phase with no timers inside it — 180 processes
and 9.9s of Atlas's cold build, entirely opaque. `tStage` around
`commandCheck` and `tWriteNif` around `writeNifModule` close that, and
with the loading slots ast2nif already had, a `nim m` process is now
fully accounted:

    startup (exec+runtime+config)    0.17 s    2%
    loading imported `.s.bif`        4.57 s   46%
    writing this module's `.s.bif`   1.84 s   18%
    sem + parse                      3.41 s   34%

Two thirds of the frontend is artifact I/O, not compilation. That is the
per-module model's structural cost: 180 processes each rebuild their
imports' interfaces from nothing.

One part of it is not structural at all. `interfHidden` accounts for
1.05s of the loading: 1.70M hidden-symbol stubs against 0.29M exported
ones, built by every `nim m` for every module it imports. The table is
reached ONLY through `modulegraphs.interfSelect` with `optImportHidden`,
and that flag is set in exactly one place — an `import x {.all.}`. The
backend already skips building it for this reason; the frontend cannot
skip it unconditionally, but it does not have to build it eagerly either.

Measured with a probe rather than estimated: skipping it takes
`InterfTables` 1161ms -> 80ms, the frontend 9.98s -> 8.93s, and the whole
Atlas build 22.19s -> 20.47s wall.

Not taking it in this commit. The correct form is lazy population on
first `interfSelect(true)` — deciding up front cannot work, because a
macro-generated `{.all.}` import is invisible syntactically and guessing
wrong loses symbols silently. `loaderCtx` is the hook (the module index
survives in the DecodeContext), but this is symbol visibility, and it
deserves its own pass rather than the tail of a long one.

`bnode.nim` carries the numbers and the design note. All behind
`-d:icBNodeProf`. ic 41/41, `koch boot -d:release` equal executables.
This commit is contained in:
araq
2026-08-31 17:03:05 +02:00
parent d680599038
commit 3f014bbd37
4 changed files with 35 additions and 7 deletions

View File

@@ -129,6 +129,26 @@
## processes are 9.9s, and the 28 backend ones are 6.8s of which 6.7s is
## inside the stage bodies — `.t.bif` writing 1.68s and cg's demand-driven
## generation 1.91s are the two largest single items.
##
## The frontend splits (same run, `Stage`/`WriteNif` + the loading slots):
##
## startup (exec+runtime+config) 0.17s 2%
## loading imported `.s.bif` 4.57s 46%
## writing this module's `.s.bif` 1.84s 18%
## sem + parse 3.41s 34%
##
## So two thirds of the frontend is artifact I/O, not compilation. Within the
## loading, `interfHidden` is 1.05s of it: 1.70M hidden-symbol stubs against
## 0.29M exported ones, built by every `nim m` for every module it imports.
## That table is reached ONLY through `modulegraphs.interfSelect` when
## `optImportHidden` is set, which happens in exactly one place — an
## `import x {.all.}`. Skipping it outright (measured with a probe, not a
## guess) takes `InterfTables` 1161ms -> 80ms, the frontend 9.98s -> 8.93s and
## the whole Atlas build 22.19s -> 20.47s. Doing it CORRECTLY means populating
## the table lazily on first `interfSelect(true)` rather than deciding
## up front — a macro-generated `{.all.}` import cannot be seen syntactically,
## and guessing wrong loses symbols silently. `loaderCtx` is the hook: the
## module index is still in the DecodeContext after loading.
## * `-d:icBridgeOnly` builds the buffer but generates off the tree, which
## separates the ENCODER's cost from the READER's. Encoding is free — it does
## not show in wall time at all.

View File

@@ -44,7 +44,10 @@ when defined(icBNodeProf):
tStage,
tLowerOwned, tLowerHooks, tLowerWrite,
tCgGen, tCgInit, tCgFinish, tCgWrite,
tMergeStage, tEmitRender, tLinkStage
tMergeStage, tEmitRender, tLinkStage,
# `nim m` (the frontend): the sem pass as a whole, and writing the module's
# `.s.bif`. `Stage - WriteNif - <the loading slots>` is then sem proper.
tWriteNif
let procStart = getMonoTime()
## Set when this module initialises, i.e. essentially at process start, so

View File

@@ -29,6 +29,7 @@ when defined(nimPreviewSlimSystem):
import ../dist/checksums/src/checksums/sha1
import pipelines
import icprof
from icconfig import produceIcConfig, ensureIcConfig
when not defined(nimKochBootstrap):
@@ -445,7 +446,9 @@ proc mainCommand*(graph: ModuleGraph) =
# per-module compilation model cannot provide (yet); methods dispatch
# through the classic if-chain dispatchers instead
excl conf.features, Feature.vtables
commandCheck(graph)
# `tStage` for a `nim m` process, so `Process - Stage` is its real startup
# (exec, runtime init, config replay) rather than its whole runtime.
timed tStage: commandCheck(graph)
of cmdNifC:
setUseIc(true)
excl conf.features, Feature.vtables

View File

@@ -10,6 +10,7 @@ when not defined(nimKochBootstrap):
import "../dist/nimony/src/lib" / bitabs
import pipelineutils
import icprof
import ../dist/checksums/src/checksums/sha1
@@ -336,11 +337,12 @@ proc processPipelineModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator
# `injectDestructorCalls` and top-level locals were never destroyed.
let moduleFlags =
if sfInjectDestructors in module.flags: ModFlagInjectDestructors else: 0'i32
writeNifModule(graph.config, module.position.int32, topLevelStmts, graph.opsLog,
replayActions, implDeps, reexportedModuleSyms(graph, module),
genericOffers, typeOffers, resolvedImportDeps, firstUnusedId,
expansions, moduleFlags,
reexportedLocalSyms(graph, module))
timed tWriteNif:
writeNifModule(graph.config, module.position.int32, topLevelStmts, graph.opsLog,
replayActions, implDeps, reexportedModuleSyms(graph, module),
genericOffers, typeOffers, resolvedImportDeps, firstUnusedId,
expansions, moduleFlags,
reexportedLocalSyms(graph, module))
# The module's REAL direct imports (incl. macro-generated) for `nim ic`'s
# graph re-derivation; see ast2nif.writeSemDeps / semdata.addImportFileDep.
var semDepPaths: seq[string] = @[]