--ic:on turns an ordinary compile into an incremental one. It decomposes compilation into per-module steps whose results are cached as NIF files, and uses the external nifmake build tool to re-run only the steps whose inputs changed.
nim c --ic:on myproject.nim nim cpp --ic:on myproject.nim
It is a switch on the normal compile commands, not a command of its own, so everything else keeps working unchanged: cpp and objc backends, -r, -d:release, --exceptions:, and a project-wide opt-in from nim.cfg / config.nims. The older spelling nim ic still works and drives the same code, but it is the C backend only and cannot run the binary it built.
This document describes how IC works today, including the edge cases that shaped the current design. The per-module backend rewrite that earlier editions of this document listed as a Plan has landed: the whole-program, reuse/redirect/def-retention backend is gone and codegen is now a set of nifmake-driven per-module rules (see The backend).
The pipeline has two halves driven by one process (the driver, commandIc in compiler/deps.nim) that constructs a dependency graph, writes a build file, and hands it to nifmake:
nifmake orders the steps by their input/output files: every nim m runs before the nim nifc step that consumes its NIF, and a step re-fires only when one of its inputs is newer than its outputs. The driver invokes nifmake run --parallel by default, so independent steps at the same DAG depth fan out across cores; pass -d:icNoParallel to serialize (readable child output when debugging a build).
--genBif:on makes a regular compiler invocation write each semantically checked module as <suffix>.s.bif under the build's nimcache directory. This reuses the semantic artifact format used by IC without enabling incremental compilation or changing how the program is generated and linked. Tools such as language servers, debuggers, and binding generators can request these artifacts when they need resolved symbols and types from an ordinary build.
Per module <suffix> (a content hash of the path; see NIF symbols below), under the nimcache directory:
| File | Producer | Purpose |
|---|---|---|
| <s>.p.nif | nifler | parsed AST (syntactic) |
| <s>.deps.nif | nifler | static import list (syntactic imports) |
| <s>.s.deps.nif | nim m | real post-sem imports (incl. macro-generated); see Discovery |
| <s>.nif | nim m | semmed module (symbols resolved, typed) |
| <s>.iface.nif | nim m | iface cookie: hash of the importer-visible surface |
| <s>.impl.nif | nim m | impl cookie: hash of the entire content (bodies included) |
| <s>.edges.nif | nim m | NeedsImpl edges: modules whose bodies this sem consumed |
| <s>.c.nif | nim nifc | the C text as a NIF, with def/ref markers for DCE & dedup |
| ic_config.cfg.nif | driver | precompiled config replayed by every child (icconfig.nim) |
| ic.version | driver | format stamp; a mismatch wipes the cache (icFormatVersion) |
(See ../nifspec/doc/nif-spec.md.) A global symbol is <ident>.<disamb>.<moduleSuffix>. For a generic instantiation the <disamb> is not a counter but a content hash — setInstanceDisamb (modulegraphs.nim) MD5s the generic's identity plus the typeKey of every concrete type argument, masks it to 30 bits and tags it with InstanceDisambBit. So the only part of the name that varies between two modules making the same instantiation (seq[Foo]) is the <moduleSuffix>. Two consequences drive the backend:
A dependent must re-sem only when a dependency's relevant surface changed. Two hashes per module (ast2nif.nim):
NeedsImpl edges (.edges.nif): if a module consumed another module's body during sem — a macro expansion, a generic instantiation, a getImpl, or a compile-time call run in the VM — it records a strong edge. The dependent is then gated on that dependency's impl cookie instead of its iface cookie, so e.g. const x = dep.foo() re-sems when foo's body changes. Recording sites: semExprs.semTemplateExpr (templates), seminst.generateInstance (generics), vmgen.genProc (VM/macros/CT procs), vm.opcGetImpl (getImpl). Inline iterators and inline procs are not tracked — they are inlined at codegen, where the backend's NIF-mtime invalidation re-codegens their users.
The static scanner only sees syntactic imports. A macro can synthesize one (chronicles does parseStmt("import chronicles/textlines") driven by the chronicles_sinks define). Such an import is invisible until sem runs the macro. Each nim m records the imports it actually resolved (via the semdata.addImportFileDep hook → graph.importDeps → ast2nif.writeSemDeps) into <s>.s.deps.nif; a child that fails on a not-yet-built import flushes it before erroring. The driver re-derives the graph from those sidecars — adding the missing node + the importer→import edge — and reruns to a fixpoint. (This replaced an earlier icmissing.txt side channel.)
Codegen is no longer one whole-program process. nim nifc (cmdNifC, compiler/nifbackend.nim) is invoked once per stage via --icBackendStage:<stage>; commandIc emits these as ordinary nifmake rules so "which TUs rebuild" is just "which rules nifmake re-fires from input mtimes" — exactly as the frontend already works. There are four stages:
Because each stage is a nifmake rule keyed on file mtimes, a body-only edit to one module re-fires that module's cg+emit (and the merge/link), not the whole program — and an unchanged module's cg does not run at all.
The whole-program backend's hand-rolled mini-nifmake — computeModuleReuse, enforceDefRetention, redirectToLiveModule, the cached-defs/claim bookkeeping and the standalone dce.nim — is gone. Reuse is now just per-rule nifmake mtime checks, and the single-writer decision is the merge stage. The old cross-mm / `--force` `var not init` hazard dissolved with it: every codegen rule's config (including --mm) is a declared nifmake input, so a stale-config TU is simply rebuilt rather than mixed in. koch bootic is green under both orc and --mm:refc.
IC self-builds the compiler (koch bootic's byte-identical fixed-point check) under both orc and --mm:refc, and passes the external-package CI set.
Cold full bootstrap on a 32-core box (-d:release, no edits — IC's worst case, since incremental reuse is not exercised):
| wall | notes |
This is down from ~7.5× in the whole-program-backend era. IC does modestly more aggregate work (more processes, NIF re-parsing of imports per process), but on a many-core box that overhead is absorbed by the parallel nim m/nifc fan-out, and the C compile+link floor is shared with the classic backend. On few-core machines the cold gap is correspondingly wider — IC trades single-build latency for incremental latency.
The cold number is the least favourable comparison: it pays IC's full per-process overhead while using none of its incremental machinery. Warm rebuilds — the actual point of IC — recompile only the modules whose inputs changed (a body-only edit re-fires one module's cg+emit, not the program), so an edit-driven rebuild is a small fraction of either full build.
The strategic direction (decided 2026-06-13) is to make this NIF backend (cmdNifC) the default code generator. The per-module pipeline above is the realization of that direction; remaining work is promotion + deletion of the classic path, not new machinery.
The per-module backend (above) mirrors Nimony's src/nimony/deps.nim: the backend stopped re-implementing nifmake; each stage is a build rule, so reuse is just mtime checks and the merge stage is the only cross-module coordination.
Settled vs. open:
Validation bar (held on every change): koch bootic must reach its byte-identical fixed point, and binary size must not regress (DCE parity), across the external-package CI set.
A warm-edit profiling pass (2026-07-02, self-compiling the compiler into a dedicated --nimcache, editing one private proc body — internalErrorImpl — in the hub module compiler/msgs.nim) surfaced where a hub-module warm rebuild actually spends its time. The result refines the "a body-only edit re-fires one module" claim above: that holds for the backend, but the frontend can still cascade.
Measured: no-op 0.05s; hub body edit ~15s, split ~13s frontend / ~1.6s backend. Editing a body in a leaf (few importers) is fast; editing a body in a widely-imported module is not, and the cost is almost entirely frontend re-sem.
Frontend over-invalidation (the dominant hub-edit cost). Editing any body in a module — even a private routine that is only ever called — flips that module's whole-module impl cookie (writeImplCookie hashes the entire serialized module). Every module carrying a NeedsImpl edge on it then re-sems, even though the symbol it actually consumed is unchanged (e.g. a dependent that expanded the internalError template needs the template body, which is untouched; it does not need internalErrorImpl's body). In the msgs edit this re-fires 57 nim m processes. A .s.bif mtime diff hides this — .s.bif is content-stable, so a re-semmed-but-identical module keeps its timestamp; count actual nim m PIDs to see the fan-out.
The precise fix is per-symbol NeedsImpl gating: record which symbols' bodies a dependent consumed (the recording site modulegraphs.recordIcImplDep already receives the PSym; it currently coarsens to module(s.itemId)) and gate the dependent on only those. The obstacle is that nifmake gates on file mtimes, so per-symbol granularity needs either many cookie files or a bucketing scheme, and "which bodies are compile-time-consumable" is entangled with getImpl and the CT call graph (a macro that runs a private helper at CT does consume its body). A conservative narrowing — keep template/generic/macro/sfCompileTime bodies (plus getImpl targets) in the impl cookie but drop ordinary runtime routine bodies — captures the common "edit a private implementation proc" case, at the cost of proving the exclusion is complete.
Core modules:
Manual workflow:
Two mechanisms, at very different scales.
`tests/ic` — metamorphic tests. A t*.nim whose body contains #? metamorphic drives a sequence of cross-module edits through the IC driver in one fixed build directory (see testament/categories.nim, runMetamorphicIcTest). Directives:
| directive | effect |
|---|---|
| #!FILE <name> | (re)write a module in the virtual file system |
| #!DELETE <name> | remove a module, from the vfs and from disk |
| #!FLAGS <switches> | change the compiler switches from here on |
| #!STEP <attrs> | materialise the files, build, run, check |
Step attributes: expect: <stdout>, fails: <substring> (BOTH compilers must reject it, with that text), noop, body-edit, iface-edit, modules: <n>, clean, no-oracle.
Every successful step is also compiled with `nim c` and run, and the two outputs must agree. That oracle is the only check in the suite that is not IC-against-IC: clean == incremental, noop changes nothing and the cookie invariants are all satisfied by an IC that is consistently wrong, which is how two silent miscompilations survived (a NIF-loaded module's sfInjectDestructors was lost, so top-level destructors were never injected; nfFirstWrite/nfLastRead had nowhere to live on a serialized sym node, so every first assignment to a destructor-bearing local became =sink over zeroed memory). koch bootic has the same blind spot — it proves the compiler reproduces itself.
`testament --ic` — the whole corpus. Appends --ic:on to every C and C++ test compile, so IC inherits the existing ~10k programs and their expected output instead of the handful written for it by hand. Because it is a switch and not a command, a test that overrides the command wholesale (cmd: "nim cpp -r $file") simply gains the switch — no verb rewriting, and the C++ corpus comes along for free. Each also gets a private nimcache; without one they would share a cache and thrash it.
To keep that affordable, testament borrows nimony's hastur model (warmupSharedCache + prefillFromWarmup): a generated warmup program pulling in system and the most-imported stdlib modules is compiled once per distinct compile configuration into nimcache/ic_warmup_<hash>, and each test's empty cache is seeded from it with mtimes preserved (nifmake compares output-mtime > input-mtime, so stamping the copies "now" would re-fire the whole graph). Only program-independent artifacts are copied — the frontend NIFs and cookies plus the per-module lower/cg outputs. The .c/.o are deliberately left behind: the merge decision (which module owns each emit-everywhere definition) is whole-program, so those are re-rendered for every program anyway.
Measured on tests/destructor (97 test runs, 32-core box):
| cold | warm |
The warm number is the developer loop and it is 3.2x faster than the classic backend; the cold number is paid once per configuration and then cached on disk. The disk cost is real and worth knowing: ~3.4 GB of nimcache for that one category.
One property of an incremental compiler is worth spelling out because it looks like a test bug: a cached stage emits no diagnostics. --expandArc output, a hint, a warning — all of it is produced by the process that actually runs, so a build that reuses every artifact prints nothing. Tests that check nimout (and anything you are debugging by eye) therefore need a cold cache; running the same test twice in a row makes the second run's nimout empty.
nim cpp --ic:on works, and tests/cpp passes under it. Three things had to change for that, and they are worth knowing because they are the shape of every "C++ needs the whole program" problem the per-module backend has: