Grinding a small figdraw-based program under `nim ic` and diffing its
output against the classic backend surfaced eight bugs, four of which
silently produced a wrong binary rather than an error.
Frontend / build graph (`deps.nim`):
* Dead `when`-guarded imports were compiled anyway. `when someStrdefine
== "x": import y` is `cvUnknown` to the scanner, which conservatively
keeps the edge — right for an edge, but it also gave `y` its own `nim m`
rule, so a build died on a package the user never installed because they
never selected that backend. Track which edges are speculative and drop
a speculative subtree that cannot compile; if the guard was in fact
live, the discovery fixpoint puts the node back with the honest `cannot
open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so
nothing re-fires and `nim ic` relinked a stale binary while `nim c`
reported `cannot open file`. Report an unresolvable import from a
non-speculatively reached module during the graph scan.
* Macro-generated imports were discovered once and then forgotten.
Discovery only ran after a failure and the graph is re-derived
statically every run, so on a warm build the discovered module had no
rules at all and editing it changed nothing. Seed the graph from the
`.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from
file mtimes and never looks at a rule's command line, so `-d:foo=bar` /
`--mm:` / `--threads:` regenerated the build file with the new switches
and re-fired zero rules. Reify the configuration as a file and make it
an input of every rule.
* Command-line switches never reached the children: they replay the
project's config files, never the driver's argv, so `nim ic --opt:speed`
produced a byte-identical debug binary (likewise `--panics`,
`--experimental`, `--passC`). Forward the driver's switches, minus the
ones that must differ per child.
Artifacts and codegen:
* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw
the rule as satisfied on the next run: `nim ic` then reported success
for a program that does not compile, and generated code from
error-bearing AST (or hit an internal error in `ccgexprs`). Never
persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives
on the module symbol, which `moduleFromNifFile` rebuilds from scratch,
so `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a
module-level `block: let h = openHandle()` never ran `=destroy`. Persist
the flag as a `(modflags)` record. `injectdestructors` also has to
tolerate the `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym
node is written as a bare NIF `SymUse` token, which has nowhere to put
node flags, so the frontend's move analysis never reached the backend:
EVERY first assignment to a destructor-bearing local compiled as
`=sink`, i.e. `=destroy` on still-zeroed memory followed by a copy, and
no read was ever a move. Wrap a sym use in `(nflags ...)` when it
carries persistent node flags.
follows up #24279
`discard finishTest` was wrong if the test still had a `retries` option:
it would just ignore the result of the test. This is an unlikely mistake
but we safeguard against it by splitting `finishTest` into two, one that
completely ignores the retries option and `finishTestRetryable` which
has to be checked for a retry. This also makes the code look slightly
better.
Testament now retries a test by a specified amount if it fails in any
way other than an invalid spec. This is to deal with the flaky GC tests
on Windows CI that fail in many different ways, from the linker randomly
erroring, segfaults, etc.
Unfortunately I couldn't do this cleanly in testament's current code.
The proc `addResult`, which is the "final" proc called in a test run's
lifetime, is now wrapped in a proc `finishTest` that returns a bool
`true` if the test failed and has to be retried. This result is
propagated up from `cmpMsgs` and `compilerOutputTests` until it reaches
`testSpecHelper`, which handles these results by recursing if the test
has to be retried. Since calling `testSpecHelper` means "run this test
with one given configuration", this means every single matrix
option/target etc. receive an equal amount of retries each.
The result of `finishTest` is ignored in cases where it's known that it
won't be retried due to passing, being skipped, having an invalid spec
etc. It's also ignored in `testNimblePackages` because it's not
necessary for those specific tests yet and similar retry behavior is
already implemented for part of it.
This was a last resort for the flaky GC tests but they've been a problem
for years at this point, they give us more work to do and turn off
contributors. Ideally GC tests failing should mark as "needs review" in
the CI rather than "failed" but I don't know if Github supports
something like this.
This code will crash `check`/`nimsuggest` since the `ra` register is
uninitialised
```nim
import macros
static:
discard parseExpr("'")
```
Now it assigns an empty node so that it has something
Testament changes were so I could properly write a test. It would pass
even with a segfault since it could find the error
* clean up some test categories
* mention exact slice issue
* magics into system
* move trangechecks into overflow
* move tmemory to system
* try fix CI
* try fix CI
* final CI fix
* test azure
* use exit 1
* try again
* use useSysAssert
* disable i386
* use refc for tlsEmulation on i386
* use refc
* disable i386
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* [Testament] Extend and document message testing aids
* Enable inline msgs when not reject action.
Eliminates the pain of changing the line and column numbers in `nimout`
or `output` while making changes to the test.
* Enable using inline msgs and nimout together.
Allows ease of inline msgs for the test as well as testing msgs from
other modules.
* Add path separator and test filename variable interpolation in
msgs.
Eases handling path separators in the msgs.
* Add some documentation.
* Fixed lots of broken tests
* Fixed more broken tests
* Support multiple inline messages per a line
* Fix a broken test
* Revert variable substitution in `output`
* Remove uneeded params
* Update doc/testament.md
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Update testament/specs.nim
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Update testament/specs.nim
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Fix indentation
Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* fixes#17943
* IC: test 'nim check --ic:on' for the full Nim compiler
* Update testament/testament.nim
Co-authored-by: Clyybber <darkmine956@gmail.com>
Co-authored-by: Clyybber <darkmine956@gmail.com>
* refs #17946; refactor testament test summary, show test duration for failures; increase timeout tchannels
* revert workarounds from https://github.com/nim-lang/Nim/pull/16698 and add allowPrefixMatch optional param to greedyOrderedSubsetLines
* add test
* workaround for yet another testament bug
* gitutils: add diffStrings, diffFiles, and use it in testament to compare expected vs gotten
* refactor with createTempDir
* cleanup
* refacotr
* PRTEMP fake test spec changes to show effect of diffStrings
* add runnableExamples for experimental/diff + cross-reference with gitutils
* Revert "PRTEMP fake test spec changes to show effect of diffStrings"
This reverts commit 57dc8d642d.
* Implement https://forum.nim-lang.org/t/7848#50018 with just the same
`SuccessX` hint category, build mode on a separate, final line, and
no change to how the mode is spelled for -d:release/-d:danger.
* Change to add a new BuildMode hint category and keep testament in sync
as per comment.
* Add "--hint:buildmode:off" to `defaultHintsOff`.
* Remove as requested.
* As requested for tests clean up.
* Address code review.
* Address code review.
* Mirror db45642311
* use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups
* workaround for nimcrypto not having `--path:.`; remove workaround "two are special snowflakes"
* add comment explaining why nimcrypto fails and what packages need to be testable
