Commit Graph

65 Commits

Author SHA1 Message Date
ringabout
0b363442e5 fixes broken doc links (#23255)
https://nim-lang.github.io/Nim/testament.html#writing-unit-tests 

https://nim-lang.github.io/Nim/testament.html#writing-unit-tests-output-message-variable-interpolation
2024-01-25 14:10:32 +08:00
Andrey Makarov
a660c17d30 Markdown code blocks migration part 8 (#22478) 2023-08-15 06:27:36 +02:00
metagn
3f51b6f73d set module symbol type to None instead of nil for discard check (#21657)
* set module symbol type to None instead of nil

fixes #19225

* alright
2023-04-13 11:56:16 +02:00
ringabout
b981f3eeb7 ship a modern nimble with lock files support (#21061)
* change `include genode/env` to an import 

ref 0b262e9496 (diff-8718bd20d8f61d6638d3d64b19efc31bcd40a6d5be8215b2a1f0b75ed93e8d56)

* fixes comments

* ship a modern nimble with lock files support

* not sure whether the latest nimble has a regression

now I'm trying  0.14.0

* change `pkgs` to `pkgs2` (#21073)

* overwrite problematic packages

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review
2022-12-12 07:25:39 +01:00
quantimnot
6289b002b6 [Testament] Extend and document message testing aids (#19996)
* [Testament] Extend and document message testing aids

* Enable inline msgs when not reject action.
Eliminates the pain of changing the line and column numbers in `nimout`
or `output` while making changes to the test.
* Enable using inline msgs and nimout together.
Allows ease of inline msgs for the test as well as testing msgs from
other modules.
* Add path separator and test filename variable interpolation in
msgs.
Eases handling path separators in the msgs.
* Add some documentation.

* Fixed lots of broken tests

* Fixed more broken tests

* Support multiple inline messages per a line

* Fix a broken test

* Revert variable substitution in `output`

* Remove uneeded params

* Update doc/testament.md

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Update testament/specs.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Update testament/specs.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Fix indentation

Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-09-01 17:52:13 +02:00
quantimnot
eb5358dcdb Testament now checks OS, CPU, endianess, bitsizes 8-64 in discard statements (#19137)
* Testament checks host/cpu in compiler/platform for discard spec statement

* Remove duplicated value checks, add comments, add all cpu bit sizes and endianesses

Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
2021-11-19 08:19:01 +01:00
Constantine Molchanov
b2edc3468c Use testament to check Norm test pass (#19018)
* Use testament to check Norm test pass

This is what I actually use to test Norm, so it's better to use it.

This should not currently pass. This is expected because this is exactly the problem I want to highlight with this PR. My tests do indeed not pass at the moment.

* Remove clearNimblePath from testament command.

Co-authored-by: flywind <xzsflywind@gmail.com>
2021-11-03 09:16:24 +01:00
Timothee Cour
a36efb59b5 fix #16256: nimout: <empty> should give error (vacuously true); improve a few tests (#18089)
* fix #16256: nimout: <empty> should give error (vacuously true); improve some tests

* renamed:    tests/stdlib/t9710.nim -> tests/misc/t9710.nim

* improve tests

* fix non-DRY tests

* improve $nim_prs_D/tests/stdlib/t9091.nim

* renamed:    tests/stdlib/t9091.nim -> tests/misc/t9091.nim

* fixup

* address comment: doAssert => result.parseErrors
2021-05-31 13:39:19 +02:00
Timothee Cour
f68f28d157 make testament isSuccess more robust and allow tests with --hints:off to succeed (#17968)
* fix testament isSuccess

* show givenSpec in addResult

* simplify tstatictypes.nim
2021-05-11 06:54:52 +02:00
Timothee Cour
fb677bf5c9 fix #17952: fix both false positives and false negatives for reInvalidSpec (#17956)
* fix #17952: fix both false positives and false negatives for reInvalidSpec

* handle megatest properly

* fix for tests/stdlib/tbase64.nim
2021-05-08 00:22:21 +02:00
Timothee Cour
a236002e54 testament: add nimoutFull: bool spec (#17867)
* testament: add `nimoutFull: bool` spec
* PRTEMP
* works
* cleanup
* add test for #12741
* PRTEMP failing test
* remove unrelated changes
* changelog
2021-04-27 11:11:28 +02:00
Timothee Cour
b667e288e2 move travis, appveyor, ci.yml.disabled to unmaintained/ (#17828)
* move travis, appveyor, ci.yml.disabled to unmaintained

* update some mentions of travis, appevyor

* fix test
2021-04-24 07:54:31 +02:00
Andreas Rumpf
0b116310bf unit separator (#17730)
* use the ASCII Unit Separator so that error messages can be handled precisely by the tooling
* updated testament
2021-04-20 16:30:17 +02:00
Timothee Cour
8049d0befc testament: remove unused tfile,tline,tcolumn (#17515) 2021-03-26 07:07:33 +01:00
Timothee Cour
c6f9c38892 use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups (#17301)
* use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups
* workaround for nimcrypto not having `--path:.`; remove workaround "two are special snowflakes"
* add comment explaining why nimcrypto fails and what packages need to be testable
2021-03-12 15:45:51 +01:00
Timothee Cour
5d95137ce0 remove tests/deps/ (#17132)
* remove tests/deps/

* fix tests

* fix tests/manyloc/keineschweine/lib/zlib_helpers.nim

* fixup
2021-02-23 13:31:53 +01:00
Timothee Cour
55a33e1664 testament: support disabled:osx as alias to disabled:macosx (#17124) 2021-02-22 11:01:15 +01:00
Timothee Cour
b9994925f5 remove all uses of condsyms symbols defined prior to bootstrap nim 0.20.0 (#16918)
* nimNoArrayToCstringConversion deadcode
* nimbabel deadcode
* nimHasalignOf deadcode
* nimvarargstyped deadcode
* nimhygiene deadcode
* nimNewTypedesc deadcode
* nimlocks deadcode
* nimHasCppDefine deadcode
* nimHasRunnableExamples deadcode
* nimHasNilChecks deadcode
* nimSymKind deadcode
* minor macros refactoring
* nimVmEqIdent deadcode
* nimNoNil deadcode
* nimNoZeroTerminator deadcode
* nimHasSymOwnerInMacro deadcode
* nimVmExportFixed deadcode
* nimNewRuntime deadcode
* nimAshr deadcode
* nimUncheckedArrayTyp deadcode
* nimHasTypeof deadcode
* nimErrorProcCanHaveBody deadcode
* nimHasHotCodeReloading deadcode
* nimHasSignatureHashInMacro deadcode
* nimHasDefault deadcode
* nimMacrosSizealignof deadcode
2021-02-17 09:32:36 +01:00
Juan Carlos
f1d165adf2 Testament small fixes (#16788)
* ReSync with Devel

* ReSync

* Fix Deprecations, Convertion to itself, code style, etc

* Fix Deprecations, Convertion to itself, code style, etc
2021-01-22 07:50:28 +01:00
flywind
da28df6113 remove deprecated specs (#16684) 2021-01-11 18:02:53 +01:00
Timothee Cour
00144ee4e9 fix #16576: honor matrix in testament by making such tests non joinable (#16577)
* refs #16576: honor matrix in testament by making such tests non joinable

* add tests + misc fixes

* fix test for i386 with -d:danger
2021-01-04 19:45:46 +01:00
Timothee Cour
7e1ae35195 testament: error instead of silently ignore invalid targets; remove pointless alias target vs targets; document matrix; DRY (#16343)
* testament: error instead of silently ignore invalid targets
* s/target/targets/
* fix test; refs #16344
* address comments
* Update testament/specs.nim

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2020-12-14 10:58:29 +01:00
Timothee Cour
31a8cf16bb testament spec: remove errmsg alias of errormsg (#16188) 2020-11-29 17:32:34 -08:00
Timothee Cour
6114df3c24 testament: error instead of silently overwrite a spec (#16166) 2020-11-29 10:42:50 -08:00
Timothee Cour
1d786c0a54 fix #14242 testament r tests/js/foo now works; testament now honors --targets (#16163)
* fix #14242 `testament r tests/js/foo` now works; testament now honors --targets

* fix shouldfail
2020-11-28 19:33:36 +01:00
Timothee Cour
b809562c7c make megatest consistent with unjoined tests wrt newlines, honor newlines in output spec (#16151)
* fix megatest newlines
* still allow missing trailing newline for now but in a more strict way than before
2020-11-28 09:09:31 +01:00
n5m
436e1fa5b1 fix #15631 (#15632)
* trigger valgrind failure on memory leak

* remove non-malloc tests

* remove ORC test

is redundant because we already have an ARC test

* only run valgrind tests on 64-bit Linux

* disable freebsd and openbsd

* Remove tleak_refc

As to not test implementation details (or bug)

* Fix test failures by removing redundant test

Since this tests/shoulfail/tvalgrind.nim was specified here to fail this test
itself fails since it will be skipped on non-linux CI

* Remove test, reason detailed in the previous commit

* Remove redundant disables

* Revert removing disables

* Add and use valgrind: leaks

* Fix

Co-authored-by: Clyybber <darkmine956@gmail.com>
Co-authored-by: n5m
2020-10-19 22:25:43 +02:00
Andreas Rumpf
9a110047cb testament improvement: allow inline error messages inside test cases (#15294)
* testament support for inline error messages
* adapt teffects1.nim test to show the potential
2020-09-10 16:30:20 +02:00
Timothee Cour
89a15e417d CI openbsd: 3x batching via NIM_TESTAMENT_BATCH ; overall CI finishes in 21m instead of 34m (#14851)
* CI openbsd: 2x batching via NIM_TESTAMENT_BATCH

* auto-generate .builds/openbsd_x.yml to avoid code duplication

* 3x batching
2020-06-30 14:21:30 +02:00
Timothee Cour
90808877c5 testament: generic N-fold batching: windows CI 37mn=>16m (#14823)
* testament: run CI faster thanks to batching
* move ta_in, tstdin into existing tosproc
* move ta_out,tafalse,texitcode,tstderr into existing tosproc
* joinable osproc
* move tstdout into existing tosproc
* spec: batchable; fix tests
* fixup
2020-06-27 16:51:17 +02:00
Timothee Cour
dfe51d10a1 addQuitProc now works with closures, and c, js(node/browser) backend; fix some bugs in testament (#14342)
* make addQuitProc great again

* fix bugs in testament

* fix test

* change 2016 => 2020

* addQuitProc => addExitProc + locks

* move to std/exitprocs
2020-06-16 11:43:48 +02:00
hlaaftana
76ffa4fa25 fixes #14112, tests for #12892, #12671, #11697 (#14125) 2020-04-26 10:12:16 +02:00
Euan
123f9fb77a Fix #14091 and #14093 - test failures on NetBSD (#14096) 2020-04-24 09:18:44 +02:00
Euan
7828199827 #12103 - CI for OpenBSD (#12105)
* Working on OpenBSD CI
* Condense steps into 2 steps to make output easier to follow.
* Move up one directory after csources build.
* Remove FreeBSD build manifest and add OpenBSD test ignores for coroutines and hot code reloading.
* If runCI fails, run the test results script.
* Add email trigger for build failure
* Remove .git from repository URL
* Disable SFML test on OpenBSD
* Disable tgetaddrinfo on OpenBSD as only UDP and TCP protocols are supported.
* Remove getFilePermissions as it causes CI test failures with NimScript.
* Set clang as cc in nim.cfg and use gmake to build csources.
* Add getCurrentDir to nimscript.
* Remove duplicate getCurrentDir and check for not weirdTarget.
* Add CI badge for OpenBSD.
* Disable tests which allocate lots of memory for OpenBSD.
* Use `CORO_BACKEND_SETJMP` on OpenBSD rather than ucontext.
* Simplify building of koch
* Disable t8657 on OpenBSD. See issue #13760.
* Fix #12142 - tarray_of_channels fails on OpenBSD
* Disable thhtpclient_ssl and tosprocterminate on OpenBSD. These tests can be enabled at a later date after fixing them.
* Install libffi.
* Set path to libc for openbsd.
* Disable tevalffi for now.
* Remove tevalffi.nim.
* Use ncpuonline sysctl rather than ncpu.
* Disable tacceptcloserace and tasynchttpserver on OpenBSD.
* Enable tacceptcloserace and tasynchttpserver.
* Fix #13775 as suggested by @alaviss - use /bin/cat on OpenBSD rather than /bin/sh.
* Enable test on OpenBSD.
* Disable tflowvar on OpenBSD.
2020-04-21 15:05:21 +02:00
Miran
cdedb86348 testament: introduce 'matrix' for testing multiple options (#13343) 2020-02-07 16:45:49 +01:00
Jasper Jenkins
d245d4cf09 fix tsizeof3 for aarch64 (#13169) 2020-01-16 14:21:12 +01:00
Andreas Rumpf
83a736a34a ARC: cycle detector (#12823)
* first implementation of the =trace and =dispose hooks for the cycle collector
* a cycle collector for ARC: progress
* manual: the .acyclic pragma is a thing once again
* gcbench: adaptations for --gc:arc
* enable valgrind tests for the strutils tests
* testament: better valgrind support
* ARC refactoring: growable jumpstacks
* ARC cycle detector: non-recursive algorithm
* moved and renamed core/ files back to system/
* refactoring: --gc:arc vs --gc:orc since 'orc' is even more experimental and we want to ship --gc:arc soonish
2019-12-17 17:37:50 +01:00
Euan
c5c6bae2a4 #12103 - CI for FreeBSD (#12179)
* Ref #12103 - adds FreeBSD CI
* Fix getApplFreebsd - length of the string includes the null terminator byte, so minus 1 for result length
* Show last commit in setup task.
* Remove .git from repository URL
* Don't include noisy details showing last commit.
* Add FreeBSD build status badge
* Fix #12182 - disable tconsole on FreeBSD
* Disable tgetaddrinfo on FreebSD as getaddrinfo doesn't support the ICMP protocol.
* Install boehm-gc-threaded
* Use libgc-threaded.so on FreeBSD rather than libgc.so.
* Simplify build failure handling. Update alt text for CI badge.
* Disable test on FreeBSD
* Simplify build config

- use GNU make to build csources
- set PATH variable using the environment key
- remove modification of config to set CC as this is already set

* Install git which seems to be missing from current freebsd images
* Revert change to how path is set
* Add a comment explaining why the length is truncated
* Fix tconsole.
2019-11-29 21:00:54 +01:00
Andreas Rumpf
223e65e529 attempt to add valgrind support to the CIs and testament (#12646)
* attempt to add valgrind support to the CIs and testament

* valgrind (currently) works only on 64-bit Linux
2019-11-18 09:10:48 +01:00
Leorize
acebcd7899 testament: add azure integration 2019-10-03 18:36:07 +02:00
Araq
c9f3a8b269 added a testcase for #12195; testament now supports a 'timeout' spec field 2019-09-17 15:15:12 +02:00
Andreas Rumpf
c8cffaf420 Incremental compilation (IC): Improvements (#11881)
* IC: C codegen is aware of IC
* manual: minor change to make VSCode's RST plugin render it properly
* IC: minor refactoring
* testament: code refactorings
* rodutils: removed dead code
* IC: always build the compiler with the IC feature
* IC: C codegen improvements
* IC: implement the undocumented -d:nimMustCache option for testing purposes
* IC: added first basic tests
* IC: extensive testing of the deserialization feature
* testament: refactoring; better IC tests
* IC: removes 'nimMustCache' flag; readonly does the same
* testament: minor refactoring
* update Nimble version
* testament: removed dead code and imports; IC: added simple test
* IC: progress
2019-08-08 08:41:05 +02:00
Andreas Rumpf
7e9cda7dba fixes #10299 (#11324) 2019-05-25 07:33:33 +02:00
Jacek Sieka
3d9269195f testament: apply skip to hard-coded categories (#11168)
* testament: apply skip to hard-coded categories

* allow disabling megatest
2019-05-08 12:35:13 +02:00
Arne Döring
28394153ab 32 bit fixes (#10608) 2019-02-13 23:30:14 +01:00
Andreas Rumpf
04cdf99448 koch and testament improvement; make testing command easier to get right 2019-02-13 23:30:14 +01:00
Araq
c7c495f08a testament: use bin/nim.exe as the default 2019-02-07 20:35:23 +01:00
Araq
6e1d2df74e testament: test nimble packages is beginning to work 2019-02-04 15:59:18 +01:00
Timothee Cour
5f2b66751a fix #10049 (#10083) 2018-12-23 23:00:37 +01:00
Andreas Rumpf
ac785b0623 testament: joinable is now an explicit concept of a test spec 2018-12-11 21:23:24 +01:00