Commit Graph

79 Commits

Author SHA1 Message Date
ringabout
04f3df4c87 fixes testament matrix doesn't work with other backends which left many JS tests untested (#23592)
Targets are not changes, which means the C binary is actually tested for
JS backend
2024-05-14 11:33:08 +02:00
Jake Leahy
db9d8003b0 Don't crash for invalid toplevel parseStmt/Expr calls (#23089)
This code will crash `check`/`nimsuggest` since the `ra` register is
uninitialised

```nim
import macros

static:
  discard parseExpr("'")
```
Now it assigns an empty node so that it has something

Testament changes were so I could properly write a test. It would pass
even with a segfault since it could find the error
2023-12-19 17:27:24 +01:00
metagn
b97d603cd0 some test cleanups & category reorganization (#22010)
* clean up some test categories

* mention exact slice issue

* magics into system

* move trangechecks into overflow

* move tmemory to system

* try fix CI

* try fix CI

* final CI fix
2023-06-06 06:54:07 +02:00
ringabout
afc30ca879 fixes #19863; move sha1, md5 to nimble packages for 2.0 (#21702)
* move sha1, md5 to nimble packages

* boot the compiler

* fixes tests

* build the documentation

* fixes docs

* lol, I forgot koch.nim

* add `nimHasChecksums` define

* clone checksums but maybe copying is better

* bump nimble hash

* use ChecksumsStableCommit

* fixes tests

* deprecate them

* fixes paths

* fixes koch
2023-05-02 10:49:17 +02:00
Anna
e4e947232b testament: make valgrind optional (#21238)
Add a simple switch to enable/disable valgrind support. On by default
2023-01-11 12:23:43 -05:00
ringabout
cdbf5b4699 fixes a severe bug of testament (#20832)
* test azure

* use exit 1

* try again

* use useSysAssert

* disable i386

* use refc for tlsEmulation on i386

* use refc

* disable i386

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-11-17 09:38:07 +08:00
Lee Matos
53643c9bc4 Update testament usage to clarify testament all (#20757) 2022-11-07 18:23:44 -05:00
quantimnot
6289b002b6 [Testament] Extend and document message testing aids (#19996)
* [Testament] Extend and document message testing aids

* Enable inline msgs when not reject action.
Eliminates the pain of changing the line and column numbers in `nimout`
or `output` while making changes to the test.
* Enable using inline msgs and nimout together.
Allows ease of inline msgs for the test as well as testing msgs from
other modules.
* Add path separator and test filename variable interpolation in
msgs.
Eases handling path separators in the msgs.
* Add some documentation.

* Fixed lots of broken tests

* Fixed more broken tests

* Support multiple inline messages per a line

* Fix a broken test

* Revert variable substitution in `output`

* Remove uneeded params

* Update doc/testament.md

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Update testament/specs.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Update testament/specs.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Fix indentation

Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-09-01 17:52:13 +02:00
metagn
f4af9e955b remove some deprecated pre-1.0 stdlib modules (#20202)
* remove pre-1.0 stdlib deprecations

notable exceptions:
* ze, toU8 etc in system/arithmetics
* potentially callsite

* undo macros, ospaths, securehash, oswalkdir

* add sets back

* add back future, document deprecated versions

* add to changelog [skip ci]
2022-08-23 20:18:40 +02:00
Jacek Sieka
094d86f997 testament: use full test name in skips [backport] (#19937)
testament: use full test name in skips
2022-07-16 16:56:54 -04:00
quantimnot
a8426fc789 Fix default testament target in docs and cli help (#19796)
Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
2022-05-23 06:12:34 +02:00
Jacek Sieka
63cca93ea9 testament: include extra options in test name (#19801)
there's currently no (simple) way to disambiguate which option failed
2022-05-19 17:11:53 +02:00
flywind
99f8793502 remove std/sharedstrings (#19228)
* remove std/sharedstrings

it has been broken since 0.18.0

* rephrase the changelog entry
2021-12-09 22:37:59 +08:00
Timothee Cour
64165f03fc testament --megatest:off now makes tests non-joinable so that they all run (#18787) 2021-09-03 07:15:08 +02:00
Timothee Cour
e1e8af535e merge BuildMode into SuccessX, remove code duplication w drnim, add useful info to successx, add gc to compilesettings (#18252)
* merge BuildMode into SuccessX, add more info

* refactor duplicated with drnim

* fixup

* address comment
2021-06-14 08:51:40 +02:00
Timothee Cour
1e0165186b testament: remove deadcode related to realtimeGC, testC, callCCompiler (#18087) 2021-05-26 11:49:01 -07:00
Timothee Cour
0b2bbcaa23 fix #18077 testament now parses cmd properly (#18086) 2021-05-26 09:46:53 +02:00
Timothee Cour
b59dc3b255 remove some custom logic in testament around flags, testExec (#18090)
* remove some custom logic in testament around flags, testExec

* remove testExec, custom logic around flags from testament

* fixup
2021-05-26 09:41:50 +02:00
Andreas Rumpf
558644725d fixes #17943 (#18045)
* fixes #17943

* IC: test 'nim check --ic:on' for the full Nim compiler

* Update testament/testament.nim

Co-authored-by: Clyybber <darkmine956@gmail.com>

Co-authored-by: Clyybber <darkmine956@gmail.com>
2021-05-19 16:00:14 +02:00
Timothee Cour
f68f28d157 make testament isSuccess more robust and allow tests with --hints:off to succeed (#17968)
* fix testament isSuccess

* show givenSpec in addResult

* simplify tstatictypes.nim
2021-05-11 06:54:52 +02:00
Timothee Cour
69710e4548 fix #17960: honor matrix also for action: compile, action: reject (#17980)
* fix #17960: honor `matrix` for all action
2021-05-09 09:50:08 +02:00
Timothee Cour
4e0f38fbb1 testament :show duration also for failed tests; improve tshould_not_work; mitigate #17946 tchannels timeouts (#17947)
* refs #17946; refactor testament test summary, show test duration for failures; increase timeout tchannels

* revert workarounds from https://github.com/nim-lang/Nim/pull/16698 and add allowPrefixMatch optional param to greedyOrderedSubsetLines

* add test

* workaround for yet another testament bug
2021-05-08 17:13:47 +02:00
Timothee Cour
20248a68fd gitutils: add diffStrings, diffFiles, and use it in testament to compare expected vs gotten (#17892)
* gitutils: add diffStrings, diffFiles, and use it in testament to compare expected vs gotten
* refactor with createTempDir
* cleanup
* refacotr
* PRTEMP fake test spec changes to show effect of diffStrings
* add runnableExamples for experimental/diff + cross-reference with gitutils
* Revert "PRTEMP fake test spec changes to show effect of diffStrings"

This reverts commit 57dc8d642d.
2021-04-30 11:00:33 +02:00
c-blake
601c050fcb Implement https://forum.nim-lang.org/t/7848#50018 (#17874)
* Implement https://forum.nim-lang.org/t/7848#50018 with just the same
`SuccessX` hint category, build mode on a separate, final line, and
no change to how the mode is spelled for -d:release/-d:danger.

* Change to add a new BuildMode hint category and keep testament in sync
as per comment.

* Add "--hint:buildmode:off" to `defaultHintsOff`.

* Remove as requested.

* As requested for tests clean up.

* Address code review.

* Address code review.

* Mirror db45642311
2021-04-29 08:55:43 +02:00
Timothee Cour
a236002e54 testament: add nimoutFull: bool spec (#17867)
* testament: add `nimoutFull: bool` spec
* PRTEMP
* works
* cleanup
* add test for #12741
* PRTEMP failing test
* remove unrelated changes
* changelog
2021-04-27 11:11:28 +02:00
Timothee Cour
e4a3feeb92 add --processing:dots|filenames|off to customize hintProcessing (#17817) 2021-04-23 08:07:22 +02:00
Andreas Rumpf
0b116310bf unit separator (#17730)
* use the ASCII Unit Separator so that error messages can be handled precisely by the tooling
* updated testament
2021-04-20 16:30:17 +02:00
Timothee Cour
68e7ed9c57 important_packages: allowed_failures batch (#17757)
* important_packages: reserve batch 0 for allowed failures

* custom batch name: allowed_failures
2021-04-20 09:02:35 +02:00
Andreas Rumpf
606288974f ic navigator tests (#17735)
* IC navigator: first basic test
2021-04-16 12:20:43 +02:00
Timothee Cour
25b4a0ab0a testament --verbose: prints commands; gitignore ic artifacts (#17716) 2021-04-14 09:44:07 +02:00
Timothee Cour
6a355a4db0 nim: unbreak CI; testament: add allowedFailure logic for tests that may fail but should still run (#17513) 2021-03-28 00:29:06 -07:00
Timothee Cour
8049d0befc testament: remove unused tfile,tline,tcolumn (#17515) 2021-03-26 07:07:33 +01:00
jiro
5c49480d8c remove 'stats' from help document of testament tool (#17364) 2021-03-15 12:08:47 +01:00
Timothee Cour
c6f9c38892 use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups (#17301)
* use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups
* workaround for nimcrypto not having `--path:.`; remove workaround "two are special snowflakes"
* add comment explaining why nimcrypto fails and what packages need to be testable
2021-03-12 15:45:51 +01:00
Timothee Cour
a4e6b242d5 asyncjs: add then, catch for promise pipelining (#16871)
* asyncjs: add then
* improve tests, changelog, API
* fix cryptic windows error: The parameter is incorrect
* address comments
2021-02-24 21:03:21 +01:00
Timothee Cour
31bb67a309 add -d:nimStrictMode in CI to keep code from regressing; fixes ConvFromXtoItselfNotNeeded, UnusedImport notes (#16764) 2021-02-17 19:30:09 +01:00
Timothee Cour
6f1289b80c remove .github/workflows/ci_ssl.yml; instead run via trunner_thirdparty (#16221) 2021-02-02 18:32:33 -08:00
Timothee Cour
11a54ab2b2 testament: fix #16829, fix partially #16830 (#16831)
* fix #16829
* address comments
2021-01-27 17:05:48 +01:00
Juan Carlos
f1d165adf2 Testament small fixes (#16788)
* ReSync with Devel

* ReSync

* Fix Deprecations, Convertion to itself, code style, etc

* Fix Deprecations, Convertion to itself, code style, etc
2021-01-22 07:50:28 +01:00
Timothee Cour
2b5841cd2b fix testament regression: installed testament works again with testament r path (#16767)
* fix testament regression: installed testament works again with testament r path

* fixup
2021-01-20 14:04:08 +01:00
Juan Carlos
78a99587a4 Deprecate TaintedString (#15423)
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
Co-authored-by: Dominik Picheta <dominikpicheta@googlemail.com>
2021-01-15 18:56:38 -08:00
flywind
510e383d92 add error messages (#16679) 2021-01-10 22:26:52 -08:00
Timothee Cour
00144ee4e9 fix #16576: honor matrix in testament by making such tests non joinable (#16577)
* refs #16576: honor matrix in testament by making such tests non joinable

* add tests + misc fixes

* fix test for i386 with -d:danger
2021-01-04 19:45:46 +01:00
Timothee Cour
555cfd1d59 fix #16330 tshould_not_work (#16342) 2020-12-14 11:37:27 +01:00
Timothee Cour
82bb4db4b7 unittest: use defines instead of env vars (#16165)
* unittest: use defines instead of env vars

* use defines in testament

* fixup

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2020-12-10 00:57:52 +01:00
Timothee Cour
e0b4f05053 nimout now consistently uses nimoutCheck (#16189) 2020-12-01 21:12:40 +01:00
Timothee Cour
6114df3c24 testament: error instead of silently overwrite a spec (#16166) 2020-11-29 10:42:50 -08:00
Timothee Cour
1d786c0a54 fix #14242 testament r tests/js/foo now works; testament now honors --targets (#16163)
* fix #14242 `testament r tests/js/foo` now works; testament now honors --targets

* fix shouldfail
2020-11-28 19:33:36 +01:00
Timothee Cour
b809562c7c make megatest consistent with unjoined tests wrt newlines, honor newlines in output spec (#16151)
* fix megatest newlines
* still allow missing trailing newline for now but in a more strict way than before
2020-11-28 09:09:31 +01:00
Timothee Cour
96930b9cae targets: use cpp instead of c++ everywhere (was by far the most common) (#15961) 2020-11-14 10:08:15 +01:00