follows up #24279
`discard finishTest` was wrong if the test still had a `retries` option:
it would just ignore the result of the test. This is an unlikely mistake
but we safeguard against it by splitting `finishTest` into two, one that
completely ignores the retries option and `finishTestRetryable` which
has to be checked for a retry. This also makes the code look slightly
better.
Testament now retries a test by a specified amount if it fails in any
way other than an invalid spec. This is to deal with the flaky GC tests
on Windows CI that fail in many different ways, from the linker randomly
erroring, segfaults, etc.
Unfortunately I couldn't do this cleanly in testament's current code.
The proc `addResult`, which is the "final" proc called in a test run's
lifetime, is now wrapped in a proc `finishTest` that returns a bool
`true` if the test failed and has to be retried. This result is
propagated up from `cmpMsgs` and `compilerOutputTests` until it reaches
`testSpecHelper`, which handles these results by recursing if the test
has to be retried. Since calling `testSpecHelper` means "run this test
with one given configuration", this means every single matrix
option/target etc. receive an equal amount of retries each.
The result of `finishTest` is ignored in cases where it's known that it
won't be retried due to passing, being skipped, having an invalid spec
etc. It's also ignored in `testNimblePackages` because it's not
necessary for those specific tests yet and similar retry behavior is
already implemented for part of it.
This was a last resort for the flaky GC tests but they've been a problem
for years at this point, they give us more work to do and turn off
contributors. Ideally GC tests failing should mark as "needs review" in
the CI rather than "failed" but I don't know if Github supports
something like this.
Followup to #24154, packages aren't ready for macos 14 (M1/ARM CPU) yet
and it seems to be preview on azure, so upgrade to macos 13 for now.
Macos 12 gives a warning:
```
You are using macOS 12.
We (and Apple) do not provide support for this old version.
It is expected behaviour that some formulae will fail to build in this old version.
It is expected behaviour that Homebrew will be buggy and slow.
Do not create any issues about this on Homebrew's GitHub repositories.
Do not create any issues even if you think this message is unrelated.
Any opened issues will be immediately closed without response.
Do not ask for help from Homebrew or its maintainers on social media.
You may ask for help in Homebrew's discussions but are unlikely to receive a response.
Try to figure out the problem yourself and submit a fix as a pull request.
We will review it but may or may not accept it.
```
* clean up some test categories
* mention exact slice issue
* magics into system
* move trangechecks into overflow
* move tmemory to system
* try fix CI
* try fix CI
* final CI fix
* fix/workaround for nimrtl and nimhcr on arc/orc
fixes#21803
* try fix clang, debug linux failure
* just make duplicated procs not rtl
* actual fix for duplicated procs
* remove deprecated pragma syntax from 0.20.0
closes#4651, closes#16653 with a cheap fix for now due to
how early `tfFinal` is set
* remove type pragma between name and generics
* undo removal, try removing bind expression (0.8.14)
* fix test, unremove bind expr
* remove again
* Update changelog.md
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
* dependencies @ HEAD & weave test dependencies
* try fix package ci
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
* default threads on
* make rst gcsafe
* ignore threads option for nimscript
* threads off
* use createShared for threads
* test without threads
* avr threds off
* avr threads off
* async threads off
* threads off
* fix ci
* restore option
* make CI pleased
* fix ic tests
* Update config.nims
* add changelog
* Update changelog.md
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
* silence error output from template_various.nim
* any => auto in tests
* avoid showing failed for parseSpec since this is expected behavior in 2 cases: tincludefile.nim, tnav1.nim
* enforce InheritFromException
* fixup
* {.used: symbol}
* add tests
* fix tests with --import
* --import works without giving spurious unused warnings
* new warning warnDuplicateModuleImport for `import foo; import foo`
* fix test, add resolveModuleAlias, use proper line info for module aliases
* fix spurious warnings
* fix deprecation msg for deprecated modules even with `import foo as bar`
* disable a test for i386 pending sorting XDeclaredButNotUsed errors
* UnusedImport now works with re-exported symbols
* fix typo [skip ci]
* ic support
* add genPNode to allow writing PNode-based compiler code similarly to `genAst`
* fix DuplicateModuleImport warning
* adjust test
* fixup
* fixup
* fixup
* fix after rebase
* fix for IC
* keep the proc inline, move the const out
* [skip ci] fix changelog
* experiment: remove calls to resolveModuleAlias
* followup
* fixup
* fix tests/modules/tselfimport.nim
* workaround tests/deprecated/tmodule1.nim
* fix properly
* simplify
* fixes#17943
* IC: test 'nim check --ic:on' for the full Nim compiler
* Update testament/testament.nim
Co-authored-by: Clyybber <darkmine956@gmail.com>
Co-authored-by: Clyybber <darkmine956@gmail.com>
* gitutils: add diffStrings, diffFiles, and use it in testament to compare expected vs gotten
* refactor with createTempDir
* cleanup
* refacotr
* PRTEMP fake test spec changes to show effect of diffStrings
* add runnableExamples for experimental/diff + cross-reference with gitutils
* Revert "PRTEMP fake test spec changes to show effect of diffStrings"
This reverts commit 57dc8d642d.
* ic fixed navigator crash when track wrong/missed
Also fixed an issue with getNimcacheDir not observing the outDir.
* closer, but not sure how to test[skip ci][ci skip]
* IC navigator: added support for include files
* update
* make posix happy via expandFilename
* update
Co-authored-by: Saem Ghani <saemghani+github@gmail.com>
* use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups
* workaround for nimcrypto not having `--path:.`; remove workaround "two are special snowflakes"
* add comment explaining why nimcrypto fails and what packages need to be testable
* minor improvements
* IC: added the required logic for compilerProcs
* LazySym ftw
* we need this testing logic
* reimplement the old way we use for module package creation
* fixes a regression; don't pick module names if you can avoid it
