Commit Graph

272 Commits

Author SHA1 Message Date
Andrey Makarov
8ed2431db0 Implement Unix file regularity check (#20448) (#20628)
* Implement Unix file regularity check

* update std/dirs also
2022-10-25 07:42:47 +02:00
Jake Leahy
eed1b6df20 Add better error messages to std/jsonutils (#20629)
* Add better error messages

* Add fmt on tuple msg
2022-10-24 12:57:48 -04:00
Jacek Sieka
e2f412145f fix fwrite prototype (#20644)
* fix fwrite prototype

* Update lib/std/syncio.nim

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
2022-10-24 17:25:34 +02:00
Jake Leahy
d261135c5c Fix tuple size check in std/jsonutils (#20637)
* Add test for tuple being invalid size

* Test tuple size before accessing fields

* Fix formatting for import

* Fix not being able to build from csources_v1

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2022-10-24 17:17:14 +08:00
Jake Leahy
69cb671d8d Pass json options to hooks (#20638)
* Added tests

* Fix expected string

* Allow hooks to take an optional parameter that is the current options

* Add options to hooks for other generic types

* Fix doc links
2022-10-24 08:14:17 +02:00
ringabout
c75cfb363f improve error message when using toTask with a call with a return value (#20622) 2022-10-23 08:27:34 +02:00
ringabout
9361938ccd add std/appdirs (#20618) 2022-10-22 13:42:46 +02:00
ringabout
3c12b72168 add typesafe std/paths, std/files, std/dirs, std/symlinks (#20582)
* split std/os; add typesafe std/paths
* add more files, dirs, paths
* add documentation
* add testcase
* remove tryRemoveFile
* clean up
* Delete test.nim
* apply changes
* add `add` and fixes
2022-10-21 21:53:44 +02:00
ringabout
ff2cb113ad move procs in os to appdirs (#20605)
* move procs in os to `appdirs`

* Update lib/std/private/osappdirs.nim
2022-10-20 23:14:50 +02:00
ringabout
f6a002c8a5 [std/os] split and re-export (#20593)
* [std/os] split and export
* move to private modules
* fixes docs and tests

Co-authored-by: xflywind <43030857+xflywind@users.noreply.github.com>
2022-10-20 07:58:29 +02:00
ringabout
c0824b9b80 [std/os clean up] import and export osseps (#20580)
import and export osseps
2022-10-17 20:29:00 +02:00
Andreas Rumpf
e83f27e6a0 out parameters: enforce that 'out' is only used as a parameter (#20510)
* out parameters: enforce that 'out' is only used as a parameter

* make tables.nim use 'out' parameters

* better backwards compat
2022-10-07 22:26:53 +02:00
Matt Haggard
594e93a66b macOS use SecRandomCopyBytes instead of getentropy (#20466)
* On macOS use SecRandomCopyBytes instead of getentropy (which is only available on macOS 10.12+)

* Change passL to passl
2022-10-05 13:59:10 -04:00
metagn
0b1650576c move widestrs out of system (#20462)
* move widestrs out of system

* fix osproc
2022-10-01 22:35:09 +02:00
ringabout
3e43ea3384 make koch and tools work with nimPreviewSlimSystem (#20459) 2022-09-30 08:09:40 +08:00
ringabout
65c2518d5c fix #19500; remove find optimization [backport: 1.6] (#19714)
* remove find optimization

close #19500

* save find to std

* add simple tests

* Apply suggestions from code review

Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>

Co-authored-by: sandytypical <43030857+xflywind@users.noreply.github.com>
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
2022-09-28 16:05:41 -04:00
metagn
919a889ba8 moderate system cleanup & refactor (#20355)
* system refactor, move out 600 lines

* compilation, slice, backwardsindex, misc_num moved out of system
* some procs/types moved into arithmetics, basic_types
* system no longer depends on syncio
* some procs moved around to fit with their surroundings

* make exceptions an import, old ops to misc_num

* move instantiationInfo back

* move back nim version, fix windows echo

* include compilation

* better docs for imported modules, fix unsigned ops

also remove ze, ze64, toU8, toU16, toU32 with nimPreviewSlimSystem

* fix terminal

* workaround IC test & weird csize bug, changelog

* move NimMajor etc back to compilation, rebase for CI

* try ic fix

* form single `indices`, slim out TaintedString, try fix IC

* fix CI, update changelog, addQuitProc

* fix CI

* try fix CI

* actually fix CI finally hopefully

* Update lib/system/compilation.nim

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>

* update kochdocs

* hopefully fix csize uses for slimsystem

* fix tquit

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
2022-09-28 15:28:45 -04:00
ringabout
3d2f0e2c7c make more standard libraries work with nimPreviewSlimSystem (#20343)
* make more standard libraries work with `nimPreviewSlimSystem`

* typo

* part two

* Delete specutils.nim

* fixes more tests

* more fixes

* fixes tests

* fixes three more tests

* add formatfloat import

* fix

* last
2022-09-27 20:06:23 +02:00
Andrey Makarov
088487f652 Implement Markdown definition lists (+ migration) (#20333)
Implements definition lists Markdown extension adopted in a few
implementations including:
* [Pandoc](
  https://pandoc.org/MANUAL.html#definition-lists)
* [kramdown](
  https://kramdown.gettalong.org/quickref.html#definition-lists)
* [PHP extra Markdown](
  https://michelf.ca/projects/php-markdown/extra/#def-list)

Also affected files have been migrated.
RST definition lists are turned off for Markdown: this solves the
problem of broken formatting mentioned in
https://github.com/nim-lang/Nim/pull/20292.
2022-09-11 13:52:43 -04:00
Andrey Makarov
cde6b2aab8 Implement Pandoc Markdown concise link extension (#20304)
* Implement Pandoc Markdown concise link extension

This implements https://github.com/nim-lang/Nim/issues/20127.
Besides reference to headings we also support doing references
to Nim symbols inside Nim modules.

Markdown:
```
Some heading
------------

Ref. [Some heading].
```

Nim:
```
proc someFunction*() ...

... ## Ref. [someFunction]
```

This is substitution for RST syntax like `` `target`_ ``.
All 3 syntax variants of extension from Pandoc Markdown are supported:
`[target]`, `[target][]`, `[description][target]`.

This PR also fixes clashes in existing files, particularly
conflicts with RST footnote feature, which does not work with
this PR (but there is a plan to adopt a popular [Markdown footnote
extension](https://pandoc.org/MANUAL.html#footnotes) to make footnotes work).

Also the PR fixes a bug that Markdown links did not work when `[...]`
section had a line break.

The implementation is straightforward since link resolution did not
change w.r.t. RST implementation, it's almost only about new syntax
addition. The only essential difference is a possibility to add a custom
link description: form `[description][target]` which does not have an
RST equivalent.

* fix nim 1.0 gotcha
2022-09-04 14:52:21 -04:00
havardjohn
23e0160af2 Add improved Windows UNC path support in std/os (#20281)
* Add improved Windows UNC path support in std/os

Original issue: `std/os.createDir` tries to create every component of
the given path as a directory. The problem is that `createDir`
interprets every backslash/slash as a path separator. For a UNC path
this is incorrect. E.g. one UNC form is `\\Server\Volume\Path`. It's an
error to create the `\\Server` directory, as well as creating
`\\Server\Volume`.

Add `ntpath.nim` module with `splitDrive` proc. This implements UNC path
parsing as implemented in the Python `ntpath.py` module. The following
UNC forms are supported:

* `\\Server\Volume\Path`
* `\\?\Volume\Path`
* `\\?\UNC\Server\Volume\Path`

Improves support for UNC paths in various procs in `std/os`:
---

* pathnorm.addNormalizePath
  * Issue: This had incomplete support for UNC paths
    * The UNC prefix (first 2 characters of a UNC path) was assumed to
      be exactly `\\`, but it can be `//` and `\/`, etc. as well
    * Also, the UNC prefix must be normalized to the `dirSep` argument
      of `addNormalizePath`
  * Resolution: Changed to account for different UNC prefixes, and
    normalizing the prefixes according to `dirSep`
    * Affected procs that get tests: `relativePath`, `joinPath`
  * Issue: The server/volume part of UNC paths can be stripped when
    normalizing `..` path components
    * This error should be negligable, so ignoring this
* splitPath
  * Now make sure the UNC drive is not split; return the UNC drive as
    `head` if the UNC drive is the only component of the path
  * Consequently fixes `extractFilename`, `lastPathPart`
* parentDir / `/../`
  * Strip away drive before working on the path, prepending the drive
    after all work is done - prevents stripping UNC components
  * Return empty string if drive component is the only component; this
    is the behavior for POSIX paths as well
  * Alternative implementation: Just call something like
    `pathnorm.normalizePath(path & "/..")` for the whole proc - maybe
    too big of a change
* tailDir
  * If drive is present in path, just split that from path and return
    path
* parentDirs iterator
  * Uses `parentDir` for going backwards
  * When going forwards, first `splitDrive`, yield the drive field, and
    then iterate over path field as normal
* splitFile
  * Make sure path parsing stops at end of drive component
* createDir
  * Fixed by skipping drive part before creating directories
  * Alternative implementation: use `parentDirs` iterator instead of
    iterating over characters
    * Consequence is that it will try to create the root directory
* isRootDir
  * Changed to treat UNC drive alone as root (e.g. "//?/c:" is root)
  * This change prevents the empty string being yielded by the
    `parentDirs` iterator with `fromRoot = false`
* Internal `sameRoot`
  * The "root" refers to the drive, so `splitDrive` can be used here

This adds UNC path support to all procs that could use it in std/os. I
don't think any more work has to be done to support UNC paths. For the
future, I believe the path handling code can be refactored due to
duplicate code. There are multiple ways of manipulating paths, such as
manually searching string for path separator and also having a path
normalizer (pathnorm.nim). If all path manipulation used `pathnorm.nim`,
and path component splitting used `parentDirs` iterator, then a lot of
code could be removed.

Tests
---

Added test file for `pathnorm.nim` and `ntpath.nim`.
`pathnorm.normalizePath` has no tests, so I'm adding a few unit tests.
`ntpath.nim` contains tests copied from Python's test suite.

Added integration tests to `tos.nim` that tests UNC paths.

Removed incorrect `relativePath` runnableExamples from being tested on Windows:
---

`relativePath("/Users///me/bar//z.nim", "//Users/", '/') == "me/bar/z.nim"`

This is incorrect on Windows because the `/` and `//` are not the same
root. `/` (or `\`) is expanded to the drive in the current working
directory (e.g. `C:\`). `//` (or `\\`), however, are the first two
characters of a UNC path. The following holds true for normal Windows
installations:

* `dirExists("/Users") != dirExists("//Users")`
* `dirExists("\\Users") != dirExists("\\\\Users")`

Fixes #19103

Questions:
---

* Should the `splitDrive` proc be in `os.nim` instead with copyright
  notice above the proc?
* Is it fine to put most of the new tests into the `runnableExamples`
  section of the procs in std/os?

* [skipci] Apply suggestions from code review

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* [skip ci] Update lib/pure/os.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Move runnableExamples tests in os.nim to tos.nim

* tests/topt_no_cursor: Change from using splitFile to splitDrive

`splitFile` can no longer be used in the test, because it generates
different ARC code on Windows and Linux. This replaces `splitFile` with
`splitDrive`, because it generates same ARC code on Windows and Linux,
and returns a tuple. I assume the test wants a proc that returns a
tuple.

* Drop copyright attribute to Python

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-09-03 20:47:09 -04:00
ringabout
5ff06fd3c2 Revert "fix #19600 No error checking on fclose (#19836)" (#20297)
This reverts commit 04e4a5ec0e.
2022-09-02 19:04:58 +02:00
Bung
04e4a5ec0e fix #19600 No error checking on fclose (#19836)
* fix #19600 No error checking on fclose

* add IOError to open
2022-08-29 18:09:14 +02:00
ringabout
b6bfe38ff5 move formatfloat out of system (#20195)
* move formatfloat out of system

* fixes doc

* Update changelog.md

* careless

* fixes

* deprecate system/formatfloat

* better handling
2022-08-24 13:38:30 +02:00
metagn
b8dc58d884 test removing dollar for objects out of system (#20242)
* test removing dollar for objects out of system

* test & fixes

* fix bootstrap

* use nimPreviewSlimSystem, test stdlib category

* fix test
2022-08-24 09:44:16 +02:00
metagn
d1d141b135 new .redefine pragma for templates, warn on redefinition without it (#20211)
* test CI for template redefinitions

* adapt asyncmacro

* fix quote

* fix again

* try something else

* revert

* fix ioselectors_select, disable packages CI

* adapt more tests & simplify

* more

* more

* more

* rename to redefine, warn on implicit redefinition

* basic documentation [skip ci]

* Update compiler/lineinfos.nim

Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>

Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
2022-08-23 21:41:30 +02:00
havardjohn
f4bbf3bf0b Add use of Windows Wide CRT API for env. vars (#20084)
* Add use of Windows Wide CRT API for env. vars

Replaces use of CRT API `getenv` and `putenv` with respectively
`_wgetenv` and `_wputenv`. Motivation is to reliably convert environment
variables to UTF-8, and the wide API is best there, because it's
reliably UTF-16.

Changed the hack in `lib/std/private/win_setenv.nim` by switching the
order of the Unicode and MBCS environment update; Unicode first, MBCS
second. Because `_wgetenv`/`_wputenv` is now used, the Unicode
environment will be initialized, so it should always be updated.

Stop updating MBCS environment with the name of `getEnv`. It's not
necessarily true that MBCS encoding and the `string` encoding is the
same. Instead convert UTF-16 to current Windows code page with
`wcstombs`, and use that string to update MBCS.

Fixes regression in `6b3c77e` that caused `std/envvars.getEnv` or
`std/os.getEnv` on Windows to return non-UTF-8 encoded strings.

Add tests that test environment variables with Unicode characters in
their name or value.

* Fix test issues

Fixes

* `nim cpp` didn't compile the tests
* Nimscript import of `tosenv.nim` from `test_nimscript.nims` failed
  with "cannot importc"

* Fix missing error check on `wcstombs`

* Fix ANSI testing errors

* Separate ANSI-related testing to their own tests, and only executing
  them if running process has a specific code page
  * Setting locale with `setlocale` was not reliable and didn't work on
    certain machines
* Add handling of a "no character representation" error in second
  `wcstombs` call

* tests/newruntime_misc: Increment allocCount

Increments overall allocations in `tnewruntime_misc` test. This is
because `getEnv` now does an additional allocation: allocation of the
UTF-16 string used as parameter to `c_wgetenv`.

* Revert "tests/newruntime_misc: Increment allocCount"

This reverts commit 4d4fe8bd3e.

* tests/newruntime_misc: Increment allocCount on Windows

Increments overall allocations in `tnewruntime_misc` test for Windows.
This is because `getEnv` on Windows now does an additional allocation:
allocation of the UTF-16 string used as parameter to `c_wgetenv`.

* Refactor, adding suggestions from code review

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Document, adding suggestions

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-08-20 04:30:11 -04:00
ringabout
e8ae2dc90b bootstrap the compiler with nimPreviewSlimSystem (#20176)
* bootstrap the compiler with nimPreviewSlimSystem

* threads
2022-08-09 16:32:29 +08:00
Lancer11211
efd5c571bf .forbids pragma: defining forbidden tags (#20050)
* .forbids pragma: defining illegal effects for proc types

This patch intends to define the opposite of the .tags pragma: a way to define effects which are not allowed in a proc.

* updated documentation and changelogs for the forbids pragma

* renamed notTagEffects to forbiddenEffects

* corrected issues of forbids pragma

the forbids pragma didn't handle simple restrictions properly and it also had issues with subtyping

* removed incorrect character from changelog

* added test to cover the interaction of methods and the forbids pragma

* covering the interaction of the tags and forbids pragmas

* updated manual about the forbids pragma

* removed useless statement

* corrected the subtyping of proc types using the forbids pragma

* updated manual for the forbids pragma

* updated documentations for forbids pragma

* updated nim docs

* updated docs with rsttester.nim

* regenerated documentation

* updated rst docs

* Update changelog.md

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>

* updated changelog

* corrected typo

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
2022-07-26 07:40:49 +02:00
Jacek Sieka
ad0aee5354 sysrand: fix syscall signature [backport] (#19982)
sysrand: fix syscall signature

`syscall` is a `C` varags function
2022-07-07 21:26:58 +08:00
Jake Leahy
8fa2c0b532 Pass headers and body correctly to FetchOptions (#19884) [backport]
* Pass headers to FetchOptions

Don't pass body if method is HttpGet or HttpHead

* Syntax fixes

* Restart CI
2022-06-13 08:03:40 +02:00
Antonis Geralis
836b95c896 Deprecate std/sums (#18439)
* Deprecate sums

* Update changelog.md

* Update lib/std/sums.nim

* log

* format

* remove

* Update changelog.md

Co-authored-by: sandytypical <43030857+xflywind@users.noreply.github.com>
2022-06-09 08:27:31 +02:00
flywind
98cebad7de enable style:usages for stdlib tests [backport: 1.6] (#19715)
* enable style:usages for stdlib tests

* freeAddrInfo

* more tests

* importc

* bufSize

* fix more

* => parseSql and renderSql
2022-04-13 11:53:02 +02:00
Timothee Cour
e78ef57c93 typetraits: add toSigned, toUnsigned (#18445)
* typetraits: add toSigned, toUnsigned

* improve and add tests

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Co-authored-by: flywind <xzsflywind@gmail.com>
2022-04-07 17:38:01 -04:00
flywind
c8aeea9d62 improve the error messages for std/tasks [backport: 1.6] (#19695) 2022-04-07 20:56:34 +02:00
Jacek Sieka
cb6ce80cb8 std/tasks: fix spelling (#19691) [backport]
why aren't these not being caught by style check options?
--styleCheck:usages finds it.

Co-authored-by: flywind <xzsflywind@gmail.com>
2022-04-06 20:28:37 +02:00
flywind
7f6e800caf move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
Juan Carlos
f0bfc0bd3f Remove define for jsfetch (#19530)
* Remove define nimExperimentalAsyncjsThen for std/asyncjs.then and std/jsfetch

* Remove define nimExperimentalAsyncjsThen for std/asyncjs.then and std/jsfetch

* Remove define nimExperimentalAsyncjsThen for std/asyncjs.then and std/jsfetch

* Remove define nimExperimentalAsyncjsThen for std/asyncjs.then and std/jsfetch
2022-02-25 20:34:16 +01:00
Sizhe Zhao
b2c5d7b4ff Documentation: Fix word usage (#19529) 2022-02-14 22:15:59 +00:00
flywind
33bfe5b12c fix parseEnum cannot parse enum with const fields (#19466)
fix #19463
2022-02-04 13:47:03 +01:00
flywind
891329cd4b move io out of system (#19442)
* move io out of system

* fix tests

* fix tests

* next step

* rename to syncio

* rename

* fix nimscript

* comma

* fix

* fix parts of errors

* good for now

* fix test
2022-02-02 17:10:11 +01:00
tandy1000
22ae0bef63 Update jsfetch with latest API and fix missing bindings (#19473)
* Update with latest API and fix missing bindings

remove deprecated `Body`
remove implicit `cstring` convs
add `Headers` to `FetchOptions`
add `Request` init proc which takes `FetchOptions`

* Update lib/std/jsfetch.nim

Co-authored-by: Juan Carlos <juancarlospaco@gmail.com>

* Update lib/std/jsfetch.nim

Co-authored-by: Juan Carlos <juancarlospaco@gmail.com>

* remove experimental flag

Co-authored-by: Juan Carlos <juancarlospaco@gmail.com>
2022-02-01 12:12:55 +01:00
metagn
2bd1aa186e New/better macro pragmas, mark some as experimental (#19406)
* New/better macro pragmas, make some experimental

fix #15920, close #18212, close #14781, close #6696,
close https://github.com/nim-lang/RFCs/issues/220

Variable macro pragmas have been changed to
only take a unary section node.
They can now also be applied in sections with multiple variables,
as well as `const` sections. They also accept arguments.

Templates now support macro pragmas, mirroring other routine types.

Type and variable macro pragmas have been made experimental.
Symbols without parentheses instatiating nullary macros or templates
has also been documented in the experimental manual.

A check for a redefinition error based on the left hand side of variable
definitions when using variable macro pragmas was disabled.
This nerfs `byaddr` specifically, however this has been documented as
a consequence of the experimental features `byaddr` uses.

Given how simple these changes are I'm worried if I'm missing something.

* accomodate compiler boot

* allow weird pragmas

* add test for #10994

* remove some control flow, try remove some logic
2022-01-20 20:57:50 +01:00
flywind
4a38092ac1 Added std/envvars for env vars handling (#19421) 2022-01-20 13:55:19 +01:00
flywind
bc3bee5469 Added std/oserrors for OS error reporting (#19390)
* Added 'std/oserrors' for OS error reporting

* add a simple test

* New code should not support -d:useWinAnsi anymore

thanks to @Araq
2022-01-17 13:06:31 +01:00
flywind
15e3813d96 add mm to compilesettings; deprecate gc (#19394) 2022-01-16 21:10:35 +01:00
rockcavera
c7d5b8c83d Fix #19038 - making the Nim compiler work again on Windows XP (#19331)
* Update osenv.nim

* Update win_setenv.nim

* Update lib/pure/includes/osenv.nim

* Update lib/pure/includes/osenv.nim

* fixing cstring

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2022-01-07 08:26:55 +01:00
flywind
1869826668 add std/private/win_getsysteminfo; refactor the usage of GetSystemInfo (#19310)
* add std/private/win_getsysteminfo

* import at the top level

* wrappers follow nep1 too

* follow review comment
2022-01-04 11:49:54 +01:00
rockcavera
77ad8b81e4 fix 19292 (#19293) 2021-12-29 07:26:54 +01:00
Jaremy Creechley
6976d18519 Implement zephyr urandom and monotime (#19142)
* implement urandom for Zephyr

* add monotime on zephyr

Co-authored-by: Jaremy Creechley <jaremy.creechley@panthalassa.com>
2021-11-14 12:49:30 +01:00