Commit Graph

476 Commits

Author SHA1 Message Date
Ivan Bobev
ccd77b42af Add some enhancements to jsonutils.nim (#15133)
* Add some enhancements to `jsonutils.nim`

* Use `jsonutils.nim` hookable API to add a possibility to deserialize
  JSON arrays directly to `HashSet` and `OrderedSet` types and
  respectively to serialize those types to JSON arrays.

* Also add a possibility to deserialize JSON `null` objects to Nim
  option objects and respectively to serialize Nim option object to JSON
  object if some or to JSON `null` object if none.

* Move serialization/deserialization functionality for `Table` and
  `OrderedTable` types from `jsonutils.nim` to `tables.nim` via the
  hookable API.

* Add object `jsonutils.Joptions` and parameter from its type to
  `jsonutils.fromJson` procedure to control whether to allow
  deserializing JSON objects to Nim objects when the JSON has some
  extra or missing keys.

* Add unit tests for the added functionalities to `tjsonutils.nim`.

* improve fromJsonFields

* Add changelog entry for the jsonutils enhancements

* Add TODO in `jsonutils.nim`

* Added an entry to "Future directions" section in `jsonutils.nim` as
  suggestion for future support of serialization and de-serialization of
  nested variant objects.

* Added currently disabled test case in `tjsonutils.nim` for testing
  serialization and de-serialization of nested variant objects.

* Move JSON hooks to `jsonutils.nim`

Move `fromJsonHook` and `toJsonHook` procedures for different types to
`jsonutils.nim` module to avoid a dependency of collections modules to
the `json.nim` module.

The hooks are removed from the following modules:

  * `tables.nim`

  * `sets.nim`

  * `options.nim`

  * `strtabs.nim`

* Add some tests about `StringTableRef`

Add tests for `StringTableRef`'s `fromJsonHook` and `toJsonHook` to
`tjsonutils.nim`.

* Disable a warning in `jsonutils.nim`

Mark `fun` template in `jsonutils` module with `{.used.}` pragma in
order to disable `[XDeclaredButNotUsed]` hint. The template is actually
used by the `initCaseObject` macro in the same module.

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
2020-09-09 14:18:59 +02:00
flywind
8c82144ba5 add getprotobyname (#15273)
* add getprotobyname

* tests, docs, changelog

* add since
2020-09-09 12:47:22 +02:00
flywind
e08b802d79 more Protocol supports in windows (#15274) [backport:1.2] 2020-09-07 12:10:20 +02:00
Miran
4fb17bc03b fix #15257, toHex couldn't handle large uint64 (#15261) [backport:1.2] 2020-09-04 09:23:27 +02:00
Miran
15ff89cec1 [backport] fix #15064, strscans.scanf edge case for '$+' (#15223) 2020-08-25 09:57:15 +02:00
Elijah Shaw-Rutschman
8a004e2fc0 Add test coverage for atomics (#15193)
* Add test coverage for atomics

Signed-off-by: Elijah Shaw-Rutschman <elijahr@gmail.com>

* Fix compareExchange bugs for non-trivial objects

Bugs fixed:

1. compareExchange would not set the desired value in the success case.
2. compareExchange would not set var expected to the found value in the failure case.
3. withLock would spin forever running the unit tests. try..body..finally prevents this. Not sure why this makes a difference, since an exception wasn’t being raised, but clearing the guard in a finally block seems correct anyways.

Signed-off-by: Elijah Shaw-Rutschman <elijahr@gmail.com>
2020-08-18 18:02:10 +02:00
haxscramper
e5dbdc1d37 [FIX] strtabs interpolation in nimscript (#15172)
* [FIX] strtabs interpolation in nimscript

* [TEST] Use `static` in strtabs test
2020-08-09 14:12:53 +02:00
alaviss
c619cedd7c net: allow close() to ignore SSL failures due to disconnections (#15120)
* net: allow close() to ignore SSL failures due to disconnections

Comes with this PR is also a SIGPIPE handling contraption.

* net: don't do selectSigpipe() on macOS

macOS sockets have SO_NOSIGPIPE set, so an EPIPE doesn't necessary mean
that a SIGPIPE happened.

* net: fix alreadyBlocked logic

* net: WSAESHUTDOWN is also a disconnection error
2020-08-01 21:27:55 +02:00
alaviss
20315637aa asyncnet, net: don't attempt SSL_shutdown if a fatal error occurred (#15066)
* asyncnet, net: don't attempt SSL_shutdown if a fatal error occurred

Per TLS standard and SSL_shutdown(3ssl). This should prevent errors
coming from a close() after a bad event (ie. the other end of the pipe
is closed before shutdown can be negotiated).

Ref #9867

* tssl: try sending until an error occur

* tssl: cleanup

* tssl: actually run the test

I forgot to make the test run :P

* tssl: run the test on ARC, maybe then it'll be happy

* tssl: turns off ARC, switch tlsEmulation on for freebsd

* tssl: document why tlsEmulation is employed

* net: move SafeDisconn handling logic to socketError
2020-07-30 00:35:02 +02:00
flywind
c645dba038 fixes #14139 (#15107)
* fix #14139
* Update lib/pure/collections/heapqueue.nim

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Co-authored-by: Clyybber <darkmine956@gmail.com>
2020-07-29 16:18:27 +02:00
Tomohiro
c983466c15 Fix #14906 (#14949)
* Fix #14906 by wrapping outputStream with PipeOutStream

* Fix compile error when ./build_all.sh

* Use PipeOutStream on posix

* Fix compile error when build_all.sh

* Use ptr UncheckedArray

* Replace copyRefObj

* Remove tmp buffer from posPeekData

* Add more tests for outputStream

* Add comments about PipeOutStream.buffer

* Fix bug in posReadLine

* Move implementation of newPipeOutStream to streamwrapper module
2020-07-18 10:41:33 +02:00
Timothee Cour
e07d661d16 fix #14475; unittest.require now works with nim c; require and check now works with -d:nodejs (#14676)
* fix #14475; make unittest work with -d:nodejs

* fixup

* fixup

* disable inim, delaunay which failed after unittest.require got fixed

* re-enable tests that have been fixed
2020-07-14 13:14:32 +02:00
flywind
9ec9a31f46 Fix #12759 (#14967)
* add testcase for #5926

* fix #12759
2020-07-12 03:31:03 -04:00
flywind
3db6d9ea0c add docs and more tests for debug format strings (#14861)
* add debug format string

* remove try except

* add changelog

* add docs and more tests

* Update lib/pure/strformat.nim

Co-authored-by: Juan Carlos <juancarlospaco@gmail.com>

* minor

Co-authored-by: Juan Carlos <juancarlospaco@gmail.com>
2020-07-08 22:21:30 +02:00
Araq
49cea86d8b weaken tosproc test for my Windows machine which doesn't have 'ls' 2020-07-07 15:17:58 +02:00
Timothee Cour
dc5a40f3f3 {.deprecated: [existsFile: fileExists].} (#14735)
* {.deprecated: [existsFile: fileExists].}

* s/existsFile/fileExists/ except under deps

* workaround pending #14819

* fix test
2020-07-02 16:19:13 +02:00
flywind
8be54b8fa7 add debug fmt string like python's (#14808)
* add debug format string

* remove try except

* add changelog
2020-06-30 15:21:37 +02:00
Timothee Cour
90808877c5 testament: generic N-fold batching: windows CI 37mn=>16m (#14823)
* testament: run CI faster thanks to batching
* move ta_in, tstdin into existing tosproc
* move ta_out,tafalse,texitcode,tstderr into existing tosproc
* joinable osproc
* move tstdout into existing tosproc
* spec: batchable; fix tests
* fixup
2020-06-27 16:51:17 +02:00
Timothee Cour
37081e2945 fromJson: support object variants (#14694) 2020-06-24 18:03:01 +02:00
Timothee Cour
7df27b5035 [cleanups] doassert => doAssert; mark deadcode (#14711) 2020-06-17 20:25:25 -07:00
Miran
e7f280bd26 Remove deprecated stuff from stdlib (#14699)
* update to the latest Jester

* remove deprecated procs from some stdlib modules

* 'criterion' is not maintained anymore and relies on obsolete stuff
2020-06-17 15:25:02 +02:00
Timothee Cour
dfe51d10a1 addQuitProc now works with closures, and c, js(node/browser) backend; fix some bugs in testament (#14342)
* make addQuitProc great again

* fix bugs in testament

* fix test

* change 2016 => 2020

* addQuitProc => addExitProc + locks

* move to std/exitprocs
2020-06-16 11:43:48 +02:00
Timothee Cour
49033eb531 make tests/stdlib tests joinable (#14626)
* make tests/stdlib tests joinable

* fixup
2020-06-15 13:27:33 +02:00
Timothee Cour
d51beb7b20 make fromJson/toJson work with array[range, typ], + 1 bugfix (#14669)
* make toJson more robust

* properly handle array
2020-06-15 13:22:43 +02:00
Timothee Cour
bf604c6829 normalizeExe (#14668) 2020-06-15 10:57:34 +02:00
Timothee Cour
c83a22b76e Disable tfdleak_multiple on platforms other than Windows (#14624) 2020-06-11 12:54:32 -07:00
Timothee Cour
8bbdb8f43f fix #14545 windows CI docs (#14590)
* fix #14545 windows paths
* add lib/std/private/strutils2.nim
* remove dependency strutils2 => strutils
* address comment: lib/std/private/strutils2.nim => compiler/strutils2.nim
2020-06-10 14:10:57 +02:00
Timothee Cour
c7a1a7b8bf toJson, jsonTo, json (de)serialization for custom types; remove dependency on strtabs thanks to a hooking mechanism (#14563)
* json custom serialization; application for strtabs
* serialize using nesting
* make toJson more feature complete
* add since
* Revert "Improve JSON serialisation of strtabs (#14549)"

This reverts commit 7cb4ef26ad.

* better approach via mixin
* toJson, jsonTo
* fix test
* address comments
* move to jsonutils
* doc
* cleanups
* also test for js
* also test for vm
2020-06-08 10:35:23 +02:00
Leorize
6cb94b5da6 asyncnet, net: clear openssl error queue before performing I/O
Per SSL_get_error(3):

  The current thread's error queue must be empty before the TLS/SSL I/O
  operation is attempted, or SSL_get_error() will not work reliably.

There has been records of not clearing the error queue causing weird SSL
errors when there shouldn't be any, see:

https://github.com/openssl/openssl/issues/11889
2020-06-06 21:11:53 +02:00
Leorize
44d0b4bdfd thttpclient_ssl: be less specific
The error we're looking for is "certificate verify failed". The routine
that reports this will be different between openssl versions, so it
makes no sense to track the routine name as well.
2020-06-06 21:11:53 +02:00
Leorize
82092b3bb7 asyncnet, net: call SSL_shutdown only when connection established
This commit prevents "SSL_shutdown while in init" errors from happening.

See https://github.com/openssl/openssl/issues/710#issuecomment-253897666
2020-06-06 21:11:53 +02:00
Timothee Cour
d573581eb7 remove isMainModule from json,os,sequtils (#14572)
* move json.isMainModule => tjson

* move isMainModule => tos,tsequtils
2020-06-06 11:50:46 +02:00
alaviss
c1ca06b452 tfdleak: fix flakyness on Windows (#14550)
* tfdleak_multiple: introduce stress tester for tfdleak

Imported from #14548 and tweaked for consumption by testament.

This test seems to be really good at bringing out the flakyness of
tfdleadk.

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>

* tfdleak: increase accuracy of the test on Windows

This commit implements a new testing strategy for Windows:
1. We duplicate the handle that will be tested and enable inheritance.
   This duplicate will serve as a reference handle.
2. In addition to checking whether the handle is valid, we also verify
   whether the handle is the same as the reference. This gives us
   complete certainty on whether the handle in question is inherited
   from the parent.
   A side effect is that this uses Windows 10+ APIs. But since
   this is just for the test, we don't have to be picky about it.

Ideally we would want to do something like this for other POSIX-based
system, but most of them lack a facility to do this, and as of writing
there isn't any false positive for them, so we won't need the additional
checks.

MemFile.fHandle will also no longer be tested, as this handle defaults
to being invalid.

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
2020-06-04 13:25:38 +02:00
Timothee Cour
17d08ff71c close #14284 document semantics for start for re,nre; improve examples (#14483) 2020-05-28 20:43:36 +02:00
alaviss
4ae341353d asyncdispatch, asyncnet: add inheritance control (#14362)
* asyncdispatch, asyncnet: add inheritance control

* asyncnet, asyncdispatch: cleanup
2020-05-20 09:42:55 +02:00
Timothee Cour
e909486e5c trunner was not actually being tested in non-CTFFI mode; minor testament cleanups (#14377)
* use check
* trunner now works with cpp
* cleanup: move compiler/unittest_light => stdtest/unittest_light
* fix tests/readme.md
* remove deadcode references to rodfiles
* fix for windows
2020-05-19 09:41:31 +02:00
Timothee Cour
041ee92bba osproc.execCmdEx now takes an optional input for stdin, env, workingDir (#14211)
* `osproc.execCmdEx` now takes an optional `input` for stdin

* execCmdEx now also takes an optional ``workingDir` and `env`
2020-05-13 13:45:36 +02:00
Timothee Cour
d11cb9d495 fix a critical bug in windows.osproc leading to resource leaks and blocking IO [backport] (#14296) 2020-05-11 11:14:21 +02:00
awr1
b8e6ea7547 Added bitslice operations for bitops (#14016)
* added bit operations based on bit slices, clarified documentation, made non-mutating versions of mask ops
* Added since annotations, some runnable examples
* Added mask()/masked() functions, changed internal workings of mask ops to use new bit* funcs
* Changelog updated for new bitops improvements
* Reorganization, added runnable examples
* Documentation adjustments
* Add incltrl for since annotation
* Fix masked() impl
* Fix mask() return type
* Don't call toUnsigned on already unsigned types
* Remove improper `var T` for flipMasked()
* Fix return types for flipMasked()
* Slight syntactic cleanup for *masked ops
* Added tests for bitslice operations, new mask() operation, non-mutating mask ops
* Fix setmasked() var T issue
* More comprehensive tests
* Fix runnable example for bitsliced()
* Fix runnable example for mask()
2020-05-06 12:21:49 +02:00
Oscar Nihlgård
48e7775ad1 Make the fields of times.DateTime private (#14197)
* Make the fields of `times.DateTime` private

* PR fixes
2020-05-06 12:20:34 +02:00
hlaaftana
fbc97e712a move since from inclrtl to std/private/since (#14188)
* move since from inclrtl to std/private/since
* move since import in system below for HCR
2020-05-02 23:51:59 +02:00
Andreas Rumpf
cc60caedb3 fixes #14054 [backport:1.2] (#14061)
* fixes #14054

* make tests green again

* more tests are green

* maybe now
2020-04-30 06:48:57 +02:00
cooldome
3b5a504692 parseEnum_regression (#14150)
Co-authored-by: cooldome <ariabushenko@bk.ru>
2020-04-28 19:56:50 +02:00
Jacek Sieka
7d6cbf290a Error -> Defect for defects (#13908)
* Error -> Defect for defects

The distinction between Error and Defect is subjective,
context-dependent and somewhat arbitrary, so when looking at an
exception, it's hard to guess what it is - this happens often when
looking at a `raises` list _without_ opening the corresponding
definition and digging through layers of inheritance.

With the help of a little consistency in naming, it's at least possible
to start disentangling the two error types and the standard lib can set
a good example here.
2020-04-28 19:56:01 +02:00
hlaaftana
cd9af6b804 StringStream & more stdlib modules support for JS/NimScript (#14095)
* StringStream & more stdlib modules support for JS/NimScript

* change back pegs test in line with #14134
2020-04-28 19:44:52 +02:00
Timothee Cour
9384f7ad35 since now takes an optional patch, eg: since: (1, 3, 1) (#14124)
add tests for tinclrtl
2020-04-26 14:55:25 +02:00
hlaaftana
76ffa4fa25 fixes #14112, tests for #12892, #12671, #11697 (#14125) 2020-04-26 10:12:16 +02:00
Timothee Cour
d5b7e9902c fix nim CI; fix local testament (#14102) 2020-04-24 10:24:30 +02:00
Euan
123f9fb77a Fix #14091 and #14093 - test failures on NetBSD (#14096) 2020-04-24 09:18:44 +02:00
Vindaar
d42c5a575d base parseEnum on a case statement, fixes #14030 (#14046)
* base `parseEnum` on a case statement, fixes #14030

* apply simplifactions / clean up, remove `norm` node, use strVal

* export `normalize` in json.nim

* cmp using nimIdentNormalize, error at CT if ambiguous enum found

`nimIdentNormalize` provided by @cooldome.

We track all names of the branches we have created so far and error if
a duplicate is found.

Dummy change to make github react...

* fix docstring of `nimIdentNormalize`

* make `typ` arg `typedesc`, add lineinfo, call norm. only once
2020-04-22 09:41:56 +01:00