Every invariant `tests/ic` checked was IC-against-IC — clean == incremental, a
no-op edit changes nothing, a body edit moves no interface cookie. An IC that is
*consistently* wrong satisfies all of them, and that is exactly how two silent
miscompilations survived. `koch bootic` has the same blind spot: it proves the
compiler reproduces itself, not that it agrees with the reference backend.
Two mechanisms, at different scales.
**The oracle, in the metamorphic runner.** Every successful step is now also
compiled with `nim c` and run, and the two outputs must agree. Unlike the
hand-written `expect:` strings this needs no foresight from the test author: it
compares everything the program does, not only what someone thought to print,
which is what a silently-skipped destructor evades. `no-oracle` opts out.
The format also grew the expressiveness the recent bug hunt showed was missing —
every one of these described a state the suite could not reach:
* `#!DELETE <file>` removes a module. Deleting a still-imported file moves no
mtime, so nothing re-fires.
* `#!FLAGS <switches>` changes the compiler switches between steps. A config
change is not a file either.
* `fails: <substring>` asserts that BOTH compilers reject the program with that
text. Previously every step had to succeed, so the whole error path — and
recovery from it — was untested.
Six regression tests cover the eight bugs the last round fixed.
`testament r <file>` now dispatches metamorphic tests like `testament cat ic`.
**`testament --ic` runs the whole corpus through the incremental compiler**, so
IC inherits ~10k programs with expected output instead of 30 bespoke tests.
Two things had to change for that to mean anything:
* `nimcacheDir` now keys on the `matrix:` entry too. Two matrix variants of one
file are two different compilations; sharing a cache meant each run
invalidated what the previous left — harmless for a backend that caches only
object files, useless for an incremental one.
* About half the corpus overrides the command wholesale (`cmd: "nim c --gc:arc
$file"`), bypassing both `$target` and `$options`. Those are rewritten to `ic`
and given a private cache.
**Warm cache, hastur-style.** A generated warmup program pulling in `system` and
the most-imported stdlib modules is compiled once per distinct compile
configuration into `nimcache/ic_warmup_<hash>`, and each test's empty cache is
seeded from it with mtimes preserved (nifmake compares output-mtime >
input-mtime, so stamping the copies "now" re-fires the whole graph). Only
program-independent artifacts are copied: the frontend NIFs and cookies plus the
per-module `lower`/`cg` output. The `.c`/`.o` are left behind on purpose — the
merge decision is whole-program, so they are re-rendered for every program
anyway.
`tests/destructor` (97 runs): `nim c` 35s cold / 32s warm; `nim ic` ~3m30 cold /
**9.8s warm**.
Compiler changes this required or uncovered:
* `merge` read the live-module list from a manifest the driver writes instead of
globbing `*.c.nif` off the nimcache. Globbing absorbed artifacts belonging to
any other program sharing the directory — which is what made a prefilled cache
produce undefined symbols at link.
* The build-arg signature no longer includes `--icproject:`/`--icPreparsedConfig:`
(they name where a build lives, not what it produces, so two caches holding
identical artifacts got different signatures). The precompiled config still
counts, by content hash, minus its `(nimcache …)` line.
* `.s.deps` seeding is speculative and runs before the prune, so a sidecar entry
that has gone stale (an import that a `when` no longer takes) can be dropped
instead of lingering forever; a pruned module's scan artifacts are deleted so
an edit-accumulated cache still matches a clean one.
* A failed nifmake run no longer prints an `Error:` of its own. The children have
already reported; adding a build-system status as the LAST error hid the
compiler's real message from anything reading the final error — every
reject-style test under `nim ic` said "nifmake failed with exit code: 1".
* `--mm:hooks` fed the mm mode to an on/off switch and failed outright with
"'on' or 'off' expected, but 'hooks' found". Pre-existing and unrelated to IC;
only reachable through the explicit switch, since `--newruntime` sets
`selectedGC` directly.
Running `tests/destructor` under `--ic` currently leaves 10 failures. They are
genuine IC defects, not harness noise (all 97 pass under `nim c`) — the clearest
is `tglobaldestructor`: `graph.globalDestructors` is accumulated while injecting
destructors into a module's top level, but the main module's `cg` — which emits
the teardown — is a different process, so a module-level `var` with a `=destroy`
is never destroyed. Same shape as the init/datInit metas, and it wants the same
fix: record it in the `.c.nif` head.
Validation: `koch bootic` reaches its byte-identical fixed point; `tests/ic` is
36/36; arc, destructor, macros, template, iter, closure, ccg, codegen, types and
effects pass under `nim c`, with generics showing only its pre-existing failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fixes#25650
This pull request refactors and improves the dependency resolution logic
in the Nim compiler, The most important changes are grouped below:
### Dependency Resolution Refactor
* Replaced the `resolveFile` procedure with two more specialized
procedures: `resolveImport` (which uses the compiler's module lookup
rules for imports) and `resolveInclude` (which resolves includes
relative to the including file or search paths). Updated all usages
accordingly, improving clarity and correctness of dependency handling.
[[1]](diffhunk://#diff-1203947eecb9ef641ce7ee029677f875eb983de050b82c65ca286517feed00e6L82-R94)
[[2]](diffhunk://#diff-1203947eecb9ef641ce7ee029677f875eb983de050b82c65ca286517feed00e6L106-R103)
[[3]](diffhunk://#diff-1203947eecb9ef641ce7ee029677f875eb983de050b82c65ca286517feed00e6L121-R118)
* Removed the unused `strutils` import from `compiler/deps.nim` for
cleaner dependencies.
### Testing Improvements
* Added `import std/strbasics` to `tests/ic/tmiscs.nim` to ensure
required symbols are available for tests.
I tried to improve `resolveFile`, which is harder because either we need
to add `lib/std` to search path and all of other nested directory to
`--path` in `config/nim.cfg`. So I choose toi reuse `findModule` for
imports
fixes#25637
This pull request refactors the way the `sfInjectDestructors` flag is
set on symbols during lambda lifting in the Nim compiler. The main
change is the introduction of a helper procedure to encapsulate the
logic for marking symbols that require destructor injection, improving
code clarity and maintainability.
Refactoring and code quality improvements:
* Introduced the `markInjectDestructors` procedure to encapsulate the
logic for marking a symbol with the `sfInjectDestructors` flag, ensuring
that `backendEnsureMutable` is always called before modifying the
symbol's flags.
* Replaced direct flag manipulation (`owner.incl sfInjectDestructors`
and `prc.incl sfInjectDestructors`) with calls to the new
`markInjectDestructors` procedure in multiple locations, including
`makeClosure`, `createTypeBoundOpsLL`, and `rawClosureCreation`.
[[1]](diffhunk://#diff-19193904ba011a2bcc1e1a9768a7eb57cac57a274cad73d388149776ec2901e6L231-R235)
[[2]](diffhunk://#diff-19193904ba011a2bcc1e1a9768a7eb57cac57a274cad73d388149776ec2901e6L243-R247)
[[3]](diffhunk://#diff-19193904ba011a2bcc1e1a9768a7eb57cac57a274cad73d388149776ec2901e6L639-R643)
```nim
template compute(body: untyped): int =
block:
body
let x = compute:
var sum = 0
for i in 1..10: sum += i
sum
echo x
```
supersedes https://github.com/nim-lang/Nim/pull/25653
which in
02893e2f4c
```nim
of nkSym:
genSingleVar(p, it.sym, newSymNode(it.sym), it.sym.astdef)
```
A new branch for `nkSym` is added, though more changes might be needed
if `nkSym` is handled specifically
This pull request includes a few targeted changes across the codebase,
primarily focusing on improving symbol locality detection in the
compiler, adding a utility function for integer division and modulus,
and simplifying a test case.
- **Compiler Improvements**
* Improved the `isLocalSym` function in `compiler/ast2nif.nim` to more
accurately determine if a symbol is local by checking that the symbol's
owner is not a module.
- **Utility Function Addition**
* Added a new `divmod` procedure in `tests/ic/tmiscs.nim` that returns
both the quotient and remainder of integer division, along with a usage
example.
- **Test Simplification**
* Simplified the `showMeters` test in `tests/ic/tconverter.nim` by
removing a floating-point assertion, leaving only an output statement.
------------------------------------------------------------------------------------------------------------------
```nim
proc divmod(a, b: int): (int, int) =
(a div b, a mod b)
let (q, r) = divmod(17, 5)
echo q
echo r
```
gives `Error: unhandled exception: local symbol 'tmpTuple.0' not found
in localSyms. [AssertionDefect]`
`makeVarTupleSection` uses a temp of which the globalness and localness
is not specified. Turning it a global variable for top level scope broke
some Nim programs. So I think it's better to check the owner of the
symbol
```nim
if useTemp:
# use same symkind for compatibility with original section
let temp = newSym(symkind, getIdent(c.cache, "tmpTuple"), c.idgen, getCurrOwner(c), n.info)
```
fixes#25620
This pull request includes a fix to the type key generation logic in the
compiler and updates to a test file to cover additional language
features. The most important changes are summarized below:
### Compiler logic fix
* In `compiler/typekeys.nim`, the `typeKey` procedure was updated to
iterate over all elements in `t.sonsImpl` starting from index 0 instead
of 1, ensuring that all type sons are considered during type key
generation.
### Test suite improvements
* The test file `tests/ic/tenum.nim` was renamed to
`tests/ic/tmiscs.nim`, and its output expectations were updated to
reflect the new test cases.
* Added new test cases to `tests/ic/tmiscs.nim` to cover sink and move
semantics, including the definition of a `BigObj` type and a `consume`
procedure that demonstrates moving and consuming large objects.
```nim
# Sink and move semantics
type
BigObj = object
data: seq[int]
proc consume(x: sink BigObj) =
echo x.data.len
var b = BigObj(data: @[1, 2, 3, 4, 5])
consume(move b)
```
gives
```
error: passing 'tySequence__qwqHTkRvwhrRyENtudHQ7g' (aka 'struct tySequence__qwqHTkRvwhrRyENtudHQ7g') to parameter of incompatible type 'tySequence__cTyVHeHOWk5jStsToosJ8Q' (aka 'struct tySequence__cTyVHeHOWk5jStsToosJ8Q')
84 | eqdestroy___sysma2dyk_u75((*dest_p0).data);
```
follows up https://github.com/nim-lang/Nim/pull/25614
Attempts to move the generic instantiation to the module that uses it.
This should decrease re-compilation times as the source module where the
generic lives doesnt need to be recompiled
---------
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* default threads on
* make rst gcsafe
* ignore threads option for nimscript
* threads off
* use createShared for threads
* test without threads
* avr threds off
* avr threads off
* async threads off
* threads off
* fix ci
* restore option
* make CI pleased
* fix ic tests
* Update config.nims
* add changelog
* Update changelog.md
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
* IC exposes typedesc implementation shenanigans; so I change system.default's definition to what it should have been to begin with
* Update lib/system.nim
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
* IC: renamed to_packed_ast module to ic module
* IC: don't store the --forceBuild flag, makes it easier to test
* IC: enable hello world test
* Codegen: refactorings for IC; changed the name mangling algorithm
* fixed the HCR regressions
* life is too short for HCR
* tconvexhull is now allowed to use deepCopy
* IC exposed a stdlib bug, required a refactoring
* codegen: code cleanups
* IC: even if a module is outdated, its dependencies might come from disk
* IC: progress
* IC: better name mangling, module IDs are not stable
* IC: another refactoring helping with --ic:on --gc:arc
* disable arraymancer on Windows for the time being
* disable arraymancer altogether
* IC: make basic test work with 'nim cpp'
* IC: progress on --ic:on --gc:arc
* wip; name mangling for type info
* minor improvements
* IC: added the required logic for compilerProcs
* LazySym ftw
* we need this testing logic
* reimplement the old way we use for module package creation
* fixes a regression; don't pick module names if you can avoid it
* IC: C codegen is aware of IC
* manual: minor change to make VSCode's RST plugin render it properly
* IC: minor refactoring
* testament: code refactorings
* rodutils: removed dead code
* IC: always build the compiler with the IC feature
* IC: C codegen improvements
* IC: implement the undocumented -d:nimMustCache option for testing purposes
* IC: added first basic tests
* IC: extensive testing of the deserialization feature
* testament: refactoring; better IC tests
* IC: removes 'nimMustCache' flag; readonly does the same
* testament: minor refactoring
* update Nimble version
* testament: removed dead code and imports; IC: added simple test
* IC: progress
