Commit Graph

9487 Commits

Author SHA1 Message Date
tersec
e8a70ff179 don't access void* out of alignment in refc GC to avoid UB (#21560) 2023-03-22 22:05:20 +01:00
Federico Ceratto
9df8ca0d81 Add URI parsing warning (#21547)
Related to CVE-2021-41259
https://github.com/nim-lang/security/security/advisories/GHSA-3gg2-rw3q-qwgc
https://github.com/nim-lang/Nim/pull/19128#issuecomment-1181944367
2023-03-20 18:51:58 +01:00
Eric N. Vander Weele
da7833c68b fixes #21538; expand len template parameter once in newSeqWith (#21543)
`len` could contain side effects and may result in different values when
substituted twice in the template expansion. Instead, capture the result
of substituting `len` once.

closes: #21538
2023-03-20 18:51:31 +01:00
Mark Leyva
285ea3c48e Fix: #21541. Add support for xnVerbatimText (#21542)
to text and text= procs. Remove unnecessary LF for xnVerbatimText
in $ proc.
2023-03-20 18:50:58 +01:00
Regis Caillaud
ba22826e9b Add check for nimMaxJeap on occupied memory + allocation size (#21521)
* fix nimMAxHeap checks

* move check to alloc pages

* remove debug trace

* Fix bad indentation

How the hell did that pass through CI ?
2023-03-20 18:49:18 +01:00
Hiroki Noda
ae06c6623d NuttX: use posix_spawn for osproc (#21539)
NuttX has standard posix_spawn interface, and can be used with it.

* https://nuttx.apache.org/docs/12.0.0/reference/user/01_task_control.html#c.posix_spawn
2023-03-20 18:43:10 +01:00
Hiroki Noda
0c1d595fae NuttX: use accept4 (#21544)
NuttX supports accept4 since 48c9d10336
2023-03-20 18:41:25 +01:00
Amjad Ben Hedhili
c52e44d845 Add cursor annotations to lists iterator variables (#21507)
Add `cursor` annotations to iterator variables

* See https://nim-lang.github.io/Nim/destructors.html#the-cursor-pragma
2023-03-13 08:43:45 +01:00
Jake Leahy
25eef64fe1 Remove Defect from raises list in std/times (#21473)
* Remove Defect from raises list

Since defects aren't tracked anymore this causes a hint to pop up mentioning it

* Still track Defect when getting ran with an older Nim version

The raises followed a pattern so moving them into a pragma didn't seem to cause any extra problems
2023-03-06 15:31:53 +01:00
Sultan Al Isaiee
04a494f8cf Add warning to specify timeout value in milliseconds, Fix #21449 (#21471)
a warning message been added to the documentation to remind users that the timeout parameter is expressed in milliseconds, not seconds. to help prevent confusion and unexpected behaviours.
2023-03-04 15:53:57 +01:00
Jake Leahy
d4d28f2ffe Allow futureLogging in release builds (#21448)
* Add test case

* Fixes 21447: Keeps stackTrace around when using futureLogging

* Remove extra whitespace
2023-03-02 14:27:10 -05:00
tersec
612abda4f4 remove decades-deprecated Win32 API *A function support (#21315) 2023-03-02 05:51:54 +01:00
Century Systems
dd629c8f45 asyncdispatch: for NuttX, add destructor to clear global dispatcher. (#21432)
* asyncdispatch: for NuttX, add destructor to clear global dispatcher using atexit().

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>

* std: exitprocs: remove "when defined(nuttx)" block.

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>

---------

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>
2023-02-28 14:38:50 +01:00
Antonis Geralis
83e3320725 Use a cast to suppress KeyError raises (#21451) 2023-02-28 14:17:21 +01:00
ringabout
6fea221d65 Overrides =copy for PackedSets (#21417) 2023-02-27 00:57:36 +01:00
Dmitry Arkhipenko
b2edfe7a02 Fix: nintendoswitch compilation (#21368)
* Fix: make nintendoswitch someGcc, remove symlink support for nintendoswitch, add getAppFilename for nintendoswitch

* Fix: use getApplHeuristic on nintendoswitch
2023-02-25 11:47:19 +01:00
Francis Thérien
7837e57e93 Disable threads for os:any (#21425) 2023-02-24 09:07:35 +01:00
Andreas Rumpf
8a19ac2070 fixes #21393 and misc style changes (#21419)
* fixes #21393 and misc style changes

* progress

---------

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
2023-02-24 09:02:15 +01:00
c-blake
fdd7520257 Fix the TODO portion of recently added posix_fallocate on OS X. (#21387) 2023-02-22 13:54:04 -05:00
Century Systems
ab1d4a5d58 ioselectors_epoll: for NuttX, limit initial numFD to configured value. (#21421)
ioselectors: ioselectors_epoll: for NuttX, limit initial numFD to configured value.

In the NuttX build config, there is a setting called "FS_NEPOLL_DESCRIPTORS".

--------
config FS_NEPOLL_DESCRIPTORS
	int "Maximum number of default epoll descriptors for epoll_create1(2)"
	default 8
	---help---
		The maximum number of default epoll descriptors for epoll_create1(2)
--------

For NuttX, change the number of fd arrays allocated by newSelector() to that value.

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>
2023-02-22 13:53:04 -05:00
Ecorous
de65b380ed Add getDataDir proc (#21408)
* Add getDataDir()

* Update lib/std/private/osappdirs.nim

---------

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2023-02-21 12:14:41 +01:00
Jake Leahy
6d423f1856 Make Time work with std/strformat (#21409)
* Add test case

* Remove formatValue template for `Time`

It didn't handle empty specifier correctly which caused it to be blank with strformat
2023-02-21 12:04:27 +01:00
Century Systems
c73c76fdc6 NuttX added supports getrlimit(RLIMIT_NOFILE), so remove NuttX specific codes. (#21385)
async: NuttX added supports getrlimit(RLIMIT_NOFILE), so remove NuttX-specific codes.

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>
2023-02-21 10:38:25 +08:00
Jake Leahy
13711b101d std/xmltree Add the type of the node when the assertion fails (#21383)
Print the type of the node when the assertion fails

This way the user actually knows what the type was instead of just knowing it failed
2023-02-17 11:24:25 +01:00
Qinsi (James) ZHU
03dd853815 add .replace() with callback to jsre (#21371) 2023-02-16 18:12:48 +01:00
Century Systems
3b9e9fd7b2 Add support for NuttX RTOS. (#21372)
* Add support for NuttX RTOS.

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>

* lib: pure: asyncdispatch: assign to result.

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>

* lib: std: typedthreads: add support for parameters to adjust Thread Stack Size.

Like FreeRTOS/Zephyr, add support for following configurations.

  -d:nimThreadStackSize=xxxxx
  -d:nimThreadStackGuard=yyyy

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>

---------

Signed-off-by: Takeyoshi Kikuchi <kikuchi@centurysys.co.jp>
2023-02-16 13:47:52 +01:00
c-blake
c91ef1a09f Fix closeHandle bug, add setFileSize, make resize work on Windows (#21375)
* Add general purpose `setFileSize` (unexported for now).  Use to simplify
`memfiles.open` as well as make robust (via hard allocation, not merely
`ftruncate` address space allocation) on systems with `posix_fallocate`.

As part of this, fix a bad `closeHandle` return check bug on Windows and
add `MemFile.resize` for Windows now that setFileSize makes that easier.

* Adapt existing test to exercise newly portable `MemFile.resize`.

* Since Apple has never provided `posix_fallocate`, provide a fallback.
This is presently written in terms of `ftruncate`, but it can be
improved to use `F_PREALLOCATE` instead, as mentioned in a comment.
2023-02-15 17:41:28 +01:00
ringabout
56a4d246a4 Refines raises list in osproc (#21323)
* Remove Exception from raises in closeImpl

* Update osproc.nim

* refine errors

* add ValueError

* cast raises

* refactor raises lists

* Update lib/pure/osproc.nim

* Update lib/pure/osproc.nim

---------

Co-authored-by: Antonis Geralis <43617260+planetis-m@users.noreply.github.com>
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2023-02-14 13:11:14 +01:00
Jake Leahy
cac49694c0 std/asyncjs allow transforming proc types (#21356)
* Add test case

* Implement JS async transform for nnkProcTy
2023-02-14 08:14:19 +01:00
c-blake
1d06c2b6cf This adds parseutils.parseSize, an inverse to strutils.formatSize (#21349)
* This adds `parseutils.parseSize`, an inverse to `strutils.formatSize`
which has existed since 2017.

It is useful for parsing the compiler's own output logs (like SuccessX)
or many other scenarios where "human readable" units have been chosen.
The doc comment and tests explain accepted syntax in detail.

Big units lead to small numbers, often with a fractional part, but we
parse into an `int64` since that is what `formatSize` stringifies and
this is an inverse over partial function slots.  Although metric
prefixes z & y for zettabyte & yottabyte are accepted, these will
saturate the result at `int64.high` unless the qualified number is a
small fraction.  This should not be much of a problem until such sizes
are common (at which point another overload with the parse result
either `float64` or `int128` could be added).

Tests avoids `test()` because of a weakly related static: test() failure
as mentioned in https://github.com/nim-lang/Nim/pull/21325. This is a
more elemental VM failure.  As such, it needs its own failure exhibition
issue that is a smaller test case.  (I am working on that, but unless
there is a burning need to `parseSize` at compile-time before run-time
it need not hold up this PR.)

* This worked with `int` but fails with `int64`.  Try for green tests.

* Lift 2-result matching into a `checkParseSize` template and format as a
table of input & 2 expected outputs which seems nicer and to address
https://github.com/nim-lang/Nim/pull/21349#pullrequestreview-1294407679

* Fix (probably) the i386 trouble by using `int64` consistently.

* Improve documentation by mentioning saturation.

* Improve documentation with `runnableExamples` and a little more detail in
the main doc comment based on excellent code review by @juancarlospaco:
https://github.com/nim-lang/Nim/pull/21349#pullrequestreview-1294564155

* Address some more @juancarlospaco code review concerns.

* Remove a stray space.

* Mention milli-bytes in docs to maybe help clarify why wild conventions
are so prone to going case-insensitive-metric.

* Add some parens.
2023-02-14 08:00:30 +01:00
ringabout
cbb6001d1e fixes backticks in the documentation (#21340) 2023-02-08 18:27:44 +08:00
ringabout
17115cbc73 fixes SSL version check logic [backport] (#21324)
* fixed version check logic [backport]

* add ciphersuites

* debug nimble

* fixes returns omission

* finally

* remove debug message

* add ciphersuites

---------

Co-authored-by: Araq <rumpf_a@web.de>
2023-02-02 16:44:14 +01:00
Jake Leahy
900fe8f501 Add contains to std/macrocache (#21304)
* Add test cases

* Implement contains for CacheSeq

* Implement contains for CacheTable

* Fix implementation of hasKey

* Remove contains for CacheSeq

Fix runnable examples

I was accidently using --doccmd:skip so I didn't spot the failure locally

* Implement hasKey as a VM callback instead of magic

* Implement suggestions from PR

Co-Authored-By: ringabout <ringabout@users.noreply.github.com>

* Update lib/core/macrocache.nim

---------

Co-authored-by: ringabout <ringabout@users.noreply.github.com>
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
2023-02-01 10:00:10 +01:00
ringabout
1431f90d8c Revert "Fix #13093 C++ Atomics: operator= is implicitly deleted because the default definition would be ill-formed " (#21307)
Revert "Fix #13093 C++ Atomics: operator= is implicitly deleted because the default definition would be ill-formed  (#21169)"

This reverts commit a7bae919ad.
2023-01-27 21:41:59 +01:00
Bung
a7bae919ad Fix #13093 C++ Atomics: operator= is implicitly deleted because the default definition would be ill-formed (#21169)
* add test

* fix #17982 Invalid C++ code generation when returning discardable var T

* fix #13093

* cpp atomic good example

* clearify the condition
2023-01-27 09:50:44 +01:00
ringabout
fc068ee06d fixes #18134; registers formatBiggestFloat in VM (#21299)
fixes #18134; registers formatBiggestFloat in vmops

strformat supports float format in VM
2023-01-26 23:03:59 +01:00
ringabout
c4d3d650ba fixes #21273; fixes an io.readLine off by one bug [backport 1.0] (#21276)
fixes #21273; io.readLine off by one
2023-01-25 14:56:19 -05:00
ringabout
5e7f7109e1 fixes #21290; deindent if the last same level is a text node (#21293)
* fixes #21290; deindent if the last same level is a text node

* add one more test
2023-01-24 13:35:19 +01:00
ringabout
b82b5d44af fixes #21278; deques.shrink off by one bug (#21284)
fixes #21278; deques.shrink off ny one bug
2023-01-20 11:43:45 +01:00
Andreas Rumpf
00ef27f4d1 minor parseopt.nim improvements (#21256)
* minor parseopt.nim improvements

* attempt to make CI happy

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2023-01-18 21:37:26 +01:00
Phil Krylov
f04f3e8bf7 std/parsesql: Add OFFSET keyword support. (#21255)
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2023-01-18 13:11:15 -05:00
PhilippMDoerner
fd207827c5 Add docs informing about gotcha of clients (#21262)
* Add docs informing about gotcha of clients

It is a hidden problem that AsyncHttpClient-instances *can not* deal with multiple requests at once.
Similar to normal HttpClients they can only deal with sending one request at a time.
This is not told anywhere in the documentation, but critical information that should be available to the user.

* Update lib/pure/httpclient.nim

fixes typo

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
2023-01-18 13:10:05 -05:00
ringabout
c4035d7f7c fixes unittest warnings (#21271) 2023-01-18 09:03:03 +01:00
Jake Leahy
1e52423774 Fix getting custom pragma from generic object (#20481)
* Merge devel

Add another test case

* Fix test

Use getCustomPragmaVal instead of hasCustomPragma

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2023-01-11 20:44:33 -05:00
Russell Brinson
8be46f3d16 parseopt.nim documentation clarity - default values & cmdEnd in getopt (#21047)
parseopt.nim documentation clarity

Added example for default values to cmd line parameters.
Additionally, added lines in getopt documentation about case switching still requiring the `cmdEnd` kind. Hopefully this clears up any vagueness for those following along in the example but omitting the `cmdEnd` in the case because the documentation said it wasn't needed.
2023-01-11 12:20:39 -05:00
ringabout
a55604a3d8 fixes #21232; highlite.getNextToken() returns gtOperator for '/' in C (#21234) 2023-01-08 15:40:27 +01:00
Andrey Makarov
2620da9bf9 docgen: implement cross-document links (#20990)
* docgen: implement cross-document links

Fully implements https://github.com/nim-lang/RFCs/issues/125
Follow-up of: https://github.com/nim-lang/Nim/pull/18642 (for internal links)
and https://github.com/nim-lang/Nim/issues/20127.

Overview
--------

Explicit import-like directive is required, called `.. importdoc::`.
(the syntax is % RST, Markdown will use it for a while).

Then one can reference any symbols/headings/anchors, as if they
were in the local file (but they will be prefixed with a module name
or markup document in link text).
It's possible to reference anything from anywhere (any direction
in `.nim`/`.md`/`.rst` files).

See `doc/docgen.md` for full description.

Working is based on `.idx` files, hence one needs to generate
all `.idx` beforehand. A dedicated option `--index:only` is introduced
(and a separate stage for `--index:only` is added to `kochdocs.nim`).

Performance note
----------------

Full run for `./koch docs` now takes 185% of the time before this PR.
(After: 315 s, before: 170 s on my PC).
All the time seems to be spent on `--index:only` run, which takes
almost as much (85%) of normal doc run -- it seems that most time
is spent on file parsing, turning off HTML generation phase has not
helped much.
(One could avoid it by specifying list of files that can be referenced
and pre-processing only them. But it can become error-prone and I assume
that these linke will be **everywhere** in the repository anyway,
especially considering https://github.com/nim-lang/RFCs/issues/478.
So every `.nim`/`.md` file is processed for `.idx` first).

But that's all without significant part of repository converted to
cross-module auto links. To estimate impact I checked the time for
`doc`ing a few files (after all indexes have been generated), and
everywhere difference was **negligible**.
E.g. for `lib/std/private/osfiles.nim` that `importdoc`s large
`os.idx` and hence should have been a case with relatively large
performance impact, but:

* After: 0.59 s.
* Before: 0.59 s.

So Nim compiler works so slow that doc part basically does not matter :-)

Testing
-------

1) added `extlinks` test to `nimdoc/`
2) checked that `theindex.html` is still correct
2) fixed broken auto-links for modules that were derived from `os.nim`
   by adding appropriate ``importdoc``

Implementation note
-------------------

Parsing and formating of `.idx` entries is moved into a dedicated
`rstidx.nim` module from `rstgen.nim`.

`.idx` file format changed:

* fields are not escaped in most cases because we need original
  strings for referencing, not HTML ones
  (the exception is linkTitle for titles and headings).
  Escaping happens later -- on the stage of `rstgen` buildIndex, etc.
* all lines have fixed number of columns 6
* added discriminator tag as a first column,
  it always allows distinguish Nim/markup entries, titles/headings, etc.
  `rstgen` does not rely any more (in most cases) on ad-hoc logic
  to determine what type each entry is.
* there is now always a title entry added at the first line.
* add a line number as 6th column
* linkTitle (4th) column has a different format: before it was like
  `module: funcName()`, now it's `proc funcName()`.
  (This format is also propagated to `theindex.html` and search results,
  I kept it that way since I like it more though it's discussible.)
  This column is what used for Nim symbols resolution.
* also changed details on column format for headings and titles:
  "keyword" is original, "linkTitle" is HTML one

* fix paths on Windows + more clear code

* Update compiler/docgen.nim

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>

* Handle .md and .nim paths uniformly in findRefFile

* handle titles better + more comments

* don't allow markup overwrite index title for .nim files

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2023-01-04 15:19:01 -05:00
Andreas Rumpf
cf1b16ef8b minor code cleanups (#21215) 2023-01-02 10:39:17 +01:00
rockcavera
e5639408d5 avoiding unnecessary allocation for dollar IpAddress (#21199)
* avoiding allocating an unnecessary byte for IPv4

The inet.h file uses 16 as the string in C needs the last null byte
1b929c02af/include/linux/inet.h (L49)

However, strings in Nim do not need this. So one byte is being allocated unnecessary and will never be used.

* avoid unnecessary allocation in IPv6 dollar

It is currently allocating 48 bytes. However, the Nim implementation for IPv6 will print a maximum of 39 characters. Nim does not implement IPv6 "0000:0000:0000:0000:0000:ffff:255.255.255.255" (45 characters) nor "0000:0000:0000:0000:0000:ffff:255.255.255.255%3" (47 characters). 

The indication in inet.h for 48 is due to the maximum use of 47 characters of a C string that needs a null byte at the end. So 48.
1b929c02af/include/linux/inet.h (L50)
2022-12-30 09:20:32 +01:00
ringabout
ebd1c678be fixes #21195; std/assertions continue to use sysFatal when nimPreviewSlimSystem is not defined (#21196)
* fixes #21195; `std/assertions` continue to use `sysFatal`

* try includes

* make `std/assertions` self-contained

* fixes tests

* fixes tests
2022-12-29 14:15:08 -05:00