Commit Graph

233 Commits

Author SHA1 Message Date
quantimnot
6289b002b6 [Testament] Extend and document message testing aids (#19996)
* [Testament] Extend and document message testing aids

* Enable inline msgs when not reject action.
Eliminates the pain of changing the line and column numbers in `nimout`
or `output` while making changes to the test.
* Enable using inline msgs and nimout together.
Allows ease of inline msgs for the test as well as testing msgs from
other modules.
* Add path separator and test filename variable interpolation in
msgs.
Eases handling path separators in the msgs.
* Add some documentation.

* Fixed lots of broken tests

* Fixed more broken tests

* Support multiple inline messages per a line

* Fix a broken test

* Revert variable substitution in `output`

* Remove uneeded params

* Update doc/testament.md

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Update testament/specs.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Update testament/specs.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Fix indentation

Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-09-01 17:52:13 +02:00
Timothee Cour
4e0f38fbb1 testament :show duration also for failed tests; improve tshould_not_work; mitigate #17946 tchannels timeouts (#17947)
* refs #17946; refactor testament test summary, show test duration for failures; increase timeout tchannels

* revert workarounds from https://github.com/nim-lang/Nim/pull/16698 and add allowPrefixMatch optional param to greedyOrderedSubsetLines

* add test

* workaround for yet another testament bug
2021-05-08 17:13:47 +02:00
Timothee Cour
a236002e54 testament: add nimoutFull: bool spec (#17867)
* testament: add `nimoutFull: bool` spec
* PRTEMP
* works
* cleanup
* add test for #12741
* PRTEMP failing test
* remove unrelated changes
* changelog
2021-04-27 11:11:28 +02:00
Timothee Cour
fe77c3c72f megatest: allow duplicate file names, show compilation progress (#17713)
* megatest: allow duplicate file names

* megatest: show compilation progress
2021-04-13 23:16:19 +02:00
Timothee Cour
2b5841cd2b fix testament regression: installed testament works again with testament r path (#16767)
* fix testament regression: installed testament works again with testament r path

* fixup
2021-01-20 14:04:08 +01:00
Timothee Cour
00144ee4e9 fix #16576: honor matrix in testament by making such tests non joinable (#16577)
* refs #16576: honor matrix in testament by making such tests non joinable

* add tests + misc fixes

* fix test for i386 with -d:danger
2021-01-04 19:45:46 +01:00
Timothee Cour
555cfd1d59 fix #16330 tshould_not_work (#16342) 2020-12-14 11:37:27 +01:00
Timothee Cour
1d786c0a54 fix #14242 testament r tests/js/foo now works; testament now honors --targets (#16163)
* fix #14242 `testament r tests/js/foo` now works; testament now honors --targets

* fix shouldfail
2020-11-28 19:33:36 +01:00
n5m
066f3ebc2a add tests for Testament "reject" action (#15709)
* add test for reject action

* highlight that a compilation failure is expected

* add the converse test for action=reject

* add trailing newline

* fix wording
2020-10-27 08:48:14 +01:00
n5m
8da0e01769 expect valgrind test failure on memory leak (#15669)
* expect valgrind test failure on memory leak

* alloc 1 instead of 0
2020-10-22 11:25:11 +02:00
Timothee Cour
a9eee6db65 misc testament cleanups (#14764)
* misc testament cleanups extracted cleanups from #14530

* make sure all CI failures can be searched with `FAIL: ` including megatest failures
2020-06-23 10:53:03 +02:00
Araq
80b508d337 test suite: rename tests containing 'fail' for easier search in logs 2019-11-26 16:04:28 +01:00
Araq
c9f3a8b269 added a testcase for #12195; testament now supports a 'timeout' spec field 2019-09-17 15:15:12 +02:00
Andreas Rumpf
82d5e773e3 make testament a tool we ship with Nim; fixes #12084 (#12088)
* make testament a tool we ship with Nim; fixes #12084
* moved config to fit
* adapt testament tests to use the testament binary
2019-08-30 15:43:07 +02:00
Timothee Cour
f9d58b9305 lots of testament bug fixes and improvements: (#10044) 2018-12-20 11:49:32 +01:00
Arne Döring
fb1f5c62b5 tshouldfail is stricter 2018-12-11 21:23:21 +01:00
Arne Döring
ca394ebd95 allow to disable backend logging 2018-12-11 21:23:20 +01:00
Arne Döring
cc0364e72f Tester tests (#9787)
* remove --lib:lib
* added some tests for testatment
* tests are addad WIP: add colors switch to tester
* meta tester is integrated
2018-11-27 10:00:32 +01:00
Jacek Sieka
97738a4f28 Testament pre parallel (#9137)
* testament: move to root dir (it's not a test)

* osproc: fix process index passed to afterRunEvent for parallel runs

it was passing the index of the process, not index of all commands

* testament: complete file move
2018-10-12 17:27:47 +02:00
xzfc
8fc7cecfa2 compiler: show name of instantiating context in error traces (#6763) (#9207) 2018-10-11 09:34:56 +02:00
Sergey Avseyev
756f50d0a3 Fix ioTests category after rename
Broken in e39f2a9283
2018-09-29 23:26:06 +03:00
Araq
3d768738f8 testament: quote categories too because of 'dir with space' test 2018-09-26 11:31:17 +02:00
Araq
d2208091fa disable Nimble dependencies for appveyor 2018-09-14 09:45:03 +02:00
LemonBoy
8e435d7b0c HTML generation in testament works again (#8702)
Fallout from the nil-str patch. Nothing to see here, move along.
2018-08-21 15:15:14 +02:00
Araq
2870ca5d71 Merge branch 'araq-misc' into devel 2018-08-16 12:30:00 +02:00
Timothee Cour
e05d0ea0bb tester.nim add more diagnostics (#8631) 2018-08-16 00:54:39 -04:00
Andreas Rumpf
e4c759fbff Merge branch 'devel' into araq-misc 2018-08-16 01:22:21 +02:00
alaviss
7ef268274f Haiku support for Nim (#8542)
* posix_other: Haiku now has spawn.h

This is added per https://dev.haiku-os.org/ticket/13446

* posix_other: Add Haiku specific Dirent members

* cpuinfo: Add an implementation for Haiku

* distros: Add basic Haiku support

* encodings: update Haiku support

* fenv, math: Haiku now provides libm

* times: Add Haiku struct members

* ansi_c, osalloc: Add Haiku constants

* threads: Add Haiku support

* testament: Haiku uses LIBRARY_PATH

* nim.cfg: Update Haiku support

libnetwork should only be linked if network functions are used

* threads: Haiku does not support -pthread switch

* tworkingdir: Haiku's env is in /bin

* posix_other: add SIGKILLTHR for Haiku

* sockets: link with libnetwork on Haiku

* coro: correct ucontext.h location

http://pubs.opengroup.org/onlinepubs/009696699/basedefs/ucontext.h.html

* coro: ucontext backend is not available on Haiku

Haiku doesn't provide the <ucontext.h> header, as it was removed from POSIX

* coro: fix setjmp backend

The compiler does not allow statements after a noreturn function

* nativesockets: Haiku doesn't support AI_V4MAPPED

* system: hostOS can contains "haiku"

* os: add support for Haiku's packagefs

packagefs is read-only, but there are writable holes to the underlying
file system as well

* os: update constant for Haiku
2018-08-14 09:35:07 +02:00
Araq
a0cde8cee6 Merge branch 'devel' into araq-misc 2018-08-13 20:16:18 +02:00
Araq
99384c0234 make NimInAction tests green 2018-08-13 20:12:00 +02:00
Andreas Rumpf
81645deb87 testament: change how JS tests are disable for travis on OSX 2018-08-13 12:51:19 +02:00
Naveen Arunachalam
e839c01f5b Clarify usage of "Test" argument in Koch (#8613)
Minor change to help people (e.g. like myself) who are starting out :)
Currently, the readme mentions the use of `./koch tests cat async`, but
the help text for koch doesn't mention it. Helptexts are modified
accordingly.
2018-08-12 22:10:06 +02:00
LemonBoy
089529c16a Preliminary check for test category folder existence (#8579)
Warn the user if nothing is executed.
2018-08-10 23:03:42 -04:00
Andreas Rumpf
6c319d4f74 travis/testament: disable JS tests on OSX 2018-08-06 19:16:23 +02:00
Andreas Rumpf
7f8eccb46f make 'flags' test green on OSX 2018-08-06 15:27:17 +02:00
genotrance
c115090f6e Testament should run tests with no action (#8232)
* Testament should run tests with no action

* Fix strutils tests for JS
2018-07-08 20:38:46 -04:00
Andreas Rumpf
ae82406c8b make tests green again 2018-07-05 19:14:18 +02:00
Andreas Rumpf
3ca11def6d Merge pull request #7771 from nim-lang/nim-in-action-fixes
Higher Nim in Action coverage plus regression fix.
2018-06-19 17:52:05 +02:00
Dominik Picheta
03b073d541 Workaround VM bug in strutils 2018-06-17 23:26:18 +01:00
Andreas Rumpf
dbcdc4331a testament: minor code formating change 2018-06-17 14:16:01 +02:00
Dominik Picheta
9e86237855 Adds smaller code samples from Chapters 1-3 to the tester. 2018-06-17 00:12:51 +01:00
Oscar Nihlgård
eb8f15d97c Support multi byte characters in module names (#7916) 2018-05-31 10:38:43 +03:00
Andreas Rumpf
700a2e125b make the tester compile without warnings 2018-05-06 09:00:55 +02:00
genotrance
e931f3b5a9 Fixes #802, #803 and #3775 - genscript issues (#7677)
* Fixes #802, #803 and #3775 - genscript issues

* Test case for genscript

* Test script

* Verify on Linux

* Update categories.nim

* Fix merge

* Improve test framework

* Windows fixes
2018-04-25 10:52:32 +02:00
Xiao-Yong
f543388959 unittest: default no color if stdout is not a tty (#7424)
We accept a new environment variable, NIMTEST_COLOR,
which override the effect of NIMTEST_NO_COLOR.
The environment variable, NIMTEST_COLOR, can be 'never'
or 'always', which set the color output to false or true,
respectively.
2018-04-12 17:57:34 +02:00
Andreas Rumpf
3bf1ce008f fixes #7236 2018-02-26 10:51:56 +01:00
Araq
5fe0c386ea disable symbol file tests for the old symbol file implementation 2018-02-19 13:19:43 +01:00
Sergey Avseyev
73af427f23 Fix longgc category tests (#7095)
testC command already has "gcc" as command, second "gcc" as option
generates "file not found error", and terminates the compiler with
error code
2018-01-22 15:08:27 +01:00
Dominik Picheta
b754bfabb6 Merge pull request #7023 from yglukhov/unify-async
Unify async macro and futures for js and native targets
2018-01-17 15:06:32 +00:00
Andreas Rumpf
b38f6d49b7 travis: attempt to make tests green again 2018-01-16 11:06:22 +01:00