Commit Graph

66 Commits

Author SHA1 Message Date
Timothee Cour
64165f03fc testament --megatest:off now makes tests non-joinable so that they all run (#18787) 2021-09-03 07:15:08 +02:00
Timothee Cour
e1e8af535e merge BuildMode into SuccessX, remove code duplication w drnim, add useful info to successx, add gc to compilesettings (#18252)
* merge BuildMode into SuccessX, add more info

* refactor duplicated with drnim

* fixup

* address comment
2021-06-14 08:51:40 +02:00
Timothee Cour
1e0165186b testament: remove deadcode related to realtimeGC, testC, callCCompiler (#18087) 2021-05-26 11:49:01 -07:00
Timothee Cour
0b2bbcaa23 fix #18077 testament now parses cmd properly (#18086) 2021-05-26 09:46:53 +02:00
Timothee Cour
b59dc3b255 remove some custom logic in testament around flags, testExec (#18090)
* remove some custom logic in testament around flags, testExec

* remove testExec, custom logic around flags from testament

* fixup
2021-05-26 09:41:50 +02:00
Andreas Rumpf
558644725d fixes #17943 (#18045)
* fixes #17943

* IC: test 'nim check --ic:on' for the full Nim compiler

* Update testament/testament.nim

Co-authored-by: Clyybber <darkmine956@gmail.com>

Co-authored-by: Clyybber <darkmine956@gmail.com>
2021-05-19 16:00:14 +02:00
Timothee Cour
f68f28d157 make testament isSuccess more robust and allow tests with --hints:off to succeed (#17968)
* fix testament isSuccess

* show givenSpec in addResult

* simplify tstatictypes.nim
2021-05-11 06:54:52 +02:00
Timothee Cour
69710e4548 fix #17960: honor matrix also for action: compile, action: reject (#17980)
* fix #17960: honor `matrix` for all action
2021-05-09 09:50:08 +02:00
Timothee Cour
4e0f38fbb1 testament :show duration also for failed tests; improve tshould_not_work; mitigate #17946 tchannels timeouts (#17947)
* refs #17946; refactor testament test summary, show test duration for failures; increase timeout tchannels

* revert workarounds from https://github.com/nim-lang/Nim/pull/16698 and add allowPrefixMatch optional param to greedyOrderedSubsetLines

* add test

* workaround for yet another testament bug
2021-05-08 17:13:47 +02:00
Timothee Cour
20248a68fd gitutils: add diffStrings, diffFiles, and use it in testament to compare expected vs gotten (#17892)
* gitutils: add diffStrings, diffFiles, and use it in testament to compare expected vs gotten
* refactor with createTempDir
* cleanup
* refacotr
* PRTEMP fake test spec changes to show effect of diffStrings
* add runnableExamples for experimental/diff + cross-reference with gitutils
* Revert "PRTEMP fake test spec changes to show effect of diffStrings"

This reverts commit 57dc8d642d.
2021-04-30 11:00:33 +02:00
c-blake
601c050fcb Implement https://forum.nim-lang.org/t/7848#50018 (#17874)
* Implement https://forum.nim-lang.org/t/7848#50018 with just the same
`SuccessX` hint category, build mode on a separate, final line, and
no change to how the mode is spelled for -d:release/-d:danger.

* Change to add a new BuildMode hint category and keep testament in sync
as per comment.

* Add "--hint:buildmode:off" to `defaultHintsOff`.

* Remove as requested.

* As requested for tests clean up.

* Address code review.

* Address code review.

* Mirror db45642311
2021-04-29 08:55:43 +02:00
Timothee Cour
a236002e54 testament: add nimoutFull: bool spec (#17867)
* testament: add `nimoutFull: bool` spec
* PRTEMP
* works
* cleanup
* add test for #12741
* PRTEMP failing test
* remove unrelated changes
* changelog
2021-04-27 11:11:28 +02:00
Timothee Cour
e4a3feeb92 add --processing:dots|filenames|off to customize hintProcessing (#17817) 2021-04-23 08:07:22 +02:00
Andreas Rumpf
0b116310bf unit separator (#17730)
* use the ASCII Unit Separator so that error messages can be handled precisely by the tooling
* updated testament
2021-04-20 16:30:17 +02:00
Timothee Cour
68e7ed9c57 important_packages: allowed_failures batch (#17757)
* important_packages: reserve batch 0 for allowed failures

* custom batch name: allowed_failures
2021-04-20 09:02:35 +02:00
Andreas Rumpf
606288974f ic navigator tests (#17735)
* IC navigator: first basic test
2021-04-16 12:20:43 +02:00
Timothee Cour
25b4a0ab0a testament --verbose: prints commands; gitignore ic artifacts (#17716) 2021-04-14 09:44:07 +02:00
Timothee Cour
6a355a4db0 nim: unbreak CI; testament: add allowedFailure logic for tests that may fail but should still run (#17513) 2021-03-28 00:29:06 -07:00
Timothee Cour
8049d0befc testament: remove unused tfile,tline,tcolumn (#17515) 2021-03-26 07:07:33 +01:00
jiro
5c49480d8c remove 'stats' from help document of testament tool (#17364) 2021-03-15 12:08:47 +01:00
Timothee Cour
c6f9c38892 use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups (#17301)
* use NIM_TESTAMENT_BATCH for important_packages, improve formatting, code cleanups
* workaround for nimcrypto not having `--path:.`; remove workaround "two are special snowflakes"
* add comment explaining why nimcrypto fails and what packages need to be testable
2021-03-12 15:45:51 +01:00
Timothee Cour
a4e6b242d5 asyncjs: add then, catch for promise pipelining (#16871)
* asyncjs: add then
* improve tests, changelog, API
* fix cryptic windows error: The parameter is incorrect
* address comments
2021-02-24 21:03:21 +01:00
Timothee Cour
31bb67a309 add -d:nimStrictMode in CI to keep code from regressing; fixes ConvFromXtoItselfNotNeeded, UnusedImport notes (#16764) 2021-02-17 19:30:09 +01:00
Timothee Cour
6f1289b80c remove .github/workflows/ci_ssl.yml; instead run via trunner_thirdparty (#16221) 2021-02-02 18:32:33 -08:00
Timothee Cour
11a54ab2b2 testament: fix #16829, fix partially #16830 (#16831)
* fix #16829
* address comments
2021-01-27 17:05:48 +01:00
Juan Carlos
f1d165adf2 Testament small fixes (#16788)
* ReSync with Devel

* ReSync

* Fix Deprecations, Convertion to itself, code style, etc

* Fix Deprecations, Convertion to itself, code style, etc
2021-01-22 07:50:28 +01:00
Timothee Cour
2b5841cd2b fix testament regression: installed testament works again with testament r path (#16767)
* fix testament regression: installed testament works again with testament r path

* fixup
2021-01-20 14:04:08 +01:00
Juan Carlos
78a99587a4 Deprecate TaintedString (#15423)
Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
Co-authored-by: Dominik Picheta <dominikpicheta@googlemail.com>
2021-01-15 18:56:38 -08:00
flywind
510e383d92 add error messages (#16679) 2021-01-10 22:26:52 -08:00
Timothee Cour
00144ee4e9 fix #16576: honor matrix in testament by making such tests non joinable (#16577)
* refs #16576: honor matrix in testament by making such tests non joinable

* add tests + misc fixes

* fix test for i386 with -d:danger
2021-01-04 19:45:46 +01:00
Timothee Cour
555cfd1d59 fix #16330 tshould_not_work (#16342) 2020-12-14 11:37:27 +01:00
Timothee Cour
82bb4db4b7 unittest: use defines instead of env vars (#16165)
* unittest: use defines instead of env vars

* use defines in testament

* fixup

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2020-12-10 00:57:52 +01:00
Timothee Cour
e0b4f05053 nimout now consistently uses nimoutCheck (#16189) 2020-12-01 21:12:40 +01:00
Timothee Cour
6114df3c24 testament: error instead of silently overwrite a spec (#16166) 2020-11-29 10:42:50 -08:00
Timothee Cour
1d786c0a54 fix #14242 testament r tests/js/foo now works; testament now honors --targets (#16163)
* fix #14242 `testament r tests/js/foo` now works; testament now honors --targets

* fix shouldfail
2020-11-28 19:33:36 +01:00
Timothee Cour
b809562c7c make megatest consistent with unjoined tests wrt newlines, honor newlines in output spec (#16151)
* fix megatest newlines
* still allow missing trailing newline for now but in a more strict way than before
2020-11-28 09:09:31 +01:00
Timothee Cour
96930b9cae targets: use cpp instead of c++ everywhere (was by far the most common) (#15961) 2020-11-14 10:08:15 +01:00
n5m
436e1fa5b1 fix #15631 (#15632)
* trigger valgrind failure on memory leak

* remove non-malloc tests

* remove ORC test

is redundant because we already have an ARC test

* only run valgrind tests on 64-bit Linux

* disable freebsd and openbsd

* Remove tleak_refc

As to not test implementation details (or bug)

* Fix test failures by removing redundant test

Since this tests/shoulfail/tvalgrind.nim was specified here to fail this test
itself fails since it will be skipped on non-linux CI

* Remove test, reason detailed in the previous commit

* Remove redundant disables

* Revert removing disables

* Add and use valgrind: leaks

* Fix

Co-authored-by: Clyybber <darkmine956@gmail.com>
Co-authored-by: n5m
2020-10-19 22:25:43 +02:00
Andreas Rumpf
9a110047cb testament improvement: allow inline error messages inside test cases (#15294)
* testament support for inline error messages
* adapt teffects1.nim test to show the potential
2020-09-10 16:30:20 +02:00
Bung
c2f80de1c7 fix #14822 copy test into var in matrix process, so can reset startTime before actully run (#15000)
* TTest pass by ref , so can reset startTime before actully run

* change TTest to ref type

* clone test in matrix process
2020-07-17 11:01:47 +02:00
narimiran
46aacf917f Fix style inconsistencies due to the previous commit 2020-07-06 11:24:43 +02:00
narimiran
ebaf603c84 revert 0944b0f4 2020-07-06 10:55:51 +02:00
Timothee Cour
dc5a40f3f3 {.deprecated: [existsFile: fileExists].} (#14735)
* {.deprecated: [existsFile: fileExists].}

* s/existsFile/fileExists/ except under deps

* workaround pending #14819

* fix test
2020-07-02 16:19:13 +02:00
Timothee Cour
3732556468 fix ./koch tests following #14823 (#14845) 2020-06-28 14:34:34 -07:00
Timothee Cour
90808877c5 testament: generic N-fold batching: windows CI 37mn=>16m (#14823)
* testament: run CI faster thanks to batching
* move ta_in, tstdin into existing tosproc
* move ta_out,tafalse,texitcode,tstderr into existing tosproc
* joinable osproc
* move tstdout into existing tosproc
* spec: batchable; fix tests
* fixup
2020-06-27 16:51:17 +02:00
Timothee Cour
a9eee6db65 misc testament cleanups (#14764)
* misc testament cleanups extracted cleanups from #14530

* make sure all CI failures can be searched with `FAIL: ` including megatest failures
2020-06-23 10:53:03 +02:00
Timothee Cour
dfe51d10a1 addQuitProc now works with closures, and c, js(node/browser) backend; fix some bugs in testament (#14342)
* make addQuitProc great again

* fix bugs in testament

* fix test

* change 2016 => 2020

* addQuitProc => addExitProc + locks

* move to std/exitprocs
2020-06-16 11:43:48 +02:00
Timothee Cour
bf604c6829 normalizeExe (#14668) 2020-06-15 10:57:34 +02:00
alaviss
4b43d42b98 testament: don't rely on Nim source structure [backport:1.2] (#14077)
It's also just faster to get the current file name instead of scanning
the PATH.
2020-04-23 09:26:06 +02:00
Miran
8088633250 faster CIs (#13803)
* ttables: smaller table, 5x speedup

* thavlak: less iterations, less loops; 30% speedup

* tasyncclosestall: shorter timeout; 35% speedup

* gcleak4: less iterations, 2x speedup

* ttimes: remove deprecated stuff

* tdangerisrelease: remove cpp backend, 3x speedup

* tfrexp1: smaller range, 2x speedup

* trtree: fix warnings, less iterations, 6x speedup

* tasyncawait_cyclebreaker: smaller swarm size; 2x speedup

* trealloc: smaller number of iterations; 10x speedup

* towned_binary_tree: less iterations, 4x speedup

* tclosure: remove unused code, less iterations; 2x speedup

* twaitany: less durations; 1.4x speedup

* tasync_misc: less iterations, 2x speedup

* t8535: smaller sleep, 1.5x speedup

* tmanyjoin: smaller sleep, 2x speedup

* t12221: shorter sleeps, removed two slower tests; 1.6x speedup

* tfuturestream: smaller sleep; 1.5x speedup

* growobjcrash: less iterations; 2x speedup

* ttryrecv: smaller sleep; 1.5x speedup

* treusetvar: less threads; 2x speedup

* delete tthreadanalysis2, basically a duplicate of tthreadanalysis

* t7758: less iterations, 1.5x speedup

* tasyncawait: smaller swarm, less messages; 1.5x speedup

* tjsandnativeasync: smaller sleep, 1.5x speedup

* tpendingcheck: smaller sleep, 1.5x speedup

* remove rodfiles test category

* move tseq from its own category to 'collections' category

* remove unneeded tests and helpers from 'assert' category

* stdlib: merge tbitops2 into tbitops

* remove 'trepr2' from 'stdlib' cat

* merge 'tstreams' into one file

* remove 'tinefficient_const_table' from 'ccbugs' cat

* merge 'tcollections_to_string' into 'tcollections'

* tblocking_channel: smaller sleep, small speedup

* tconvexhull: less iterartions; 1.2x speedup

* merge 'tdeepcopy2' into 'tdeepcopy'

* merge 'tdisjoint_slice2' into 'tdisjoint_slice1'

* tmissing_deepcopy: smaller sequence

* tsendtwice: smaller arrays; 5x speedup

* remove 'tindexerrorformatbounds'

* disable multimethod tests

* remove 'gc:none' and 'refc' without 'd:useRealtimeGC' from gc tests

* koch.nim: bootstrap just with '-d:release', no need for 'csource'

* add github workflow for documentation

* testament: no need for 8 sub-second decimals
2020-03-30 13:18:12 +02:00