* FTP client is now able to connect to server over TLS by set `useTls = true` in newAsyncFtpClient proc
* Update asyncftpclient.nim
* fix CI
* shouldn't use {.error.}
* Update lib/pure/asyncftpclient.nim
Co-authored-by: flywind <xzsflywind@gmail.com>
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Implement Markdown indented code blocks
Additional indentation of 4 spaces makes a block an "indented code block"
(monospaced text without syntax highlighting).
Also `::` RST syntax for code blocks is disabled.
So instead of
```rst
see::
Some code
```
the code block should be written as
```markdown
see:
Some code
```
* Migrate RST literal blocks :: to Markdown's ones
The strutils `delete` func with signature
func delete*(s: var string, first, last: int)
was deprecated in adba5eb45e, in favor of one with signature
func delete*(s: var string, slice: Slice[int])
However, a few procedures still used the deprecated form. This commit
updates them, resolving these deprecation warnings:
rstgen.nim(766, 12) Warning: use `delete(s, first..last)`; delete is deprecated [Deprecated]
strutils.nim(1651, 19) Warning: use `delete(s, first..last)`; delete is deprecated [Deprecated]
strutils.nim(1679, 7) Warning: use `delete(s, first..last)`; delete is deprecated [Deprecated]
strutils.nim(2472, 7) Warning: use `delete(s, first..last)`; delete is deprecated [Deprecated]
Before this commit:
- `trimZeros` called `s.delete(i+1, i)` for an input that lacks a
trailing zero (like "1.23").
- `removePrefix*(s: var string, prefix: string)` called
`s.delete(0, -1)` when the prefix was the empty string.
which did not modify `s`, nor raise an error. But the newer slice
`delete` raises an `IndexDefect` when the start of the slice is greater
than the end, so we avoid calling the new `delete` for such a case.
Recall that exceptions inheriting from `system.Defect` are not tracked
with the `.raises: []` exception tracking mechanism [1], so this commit
does not break existing code like:
proc foo {.raises: [].} =
var s = "abc1.20"
s.removePrefix("abc")
s.trimZeros()
doAssert s == "1.2"
The `strutils.delete` deprecation was motivated by a problem with
`system.delete` [2][3]:
`system.delete` had surprising behavior when the index passed to it
was out of bounds (it would delete the last entry then). Compile
with `-d:nimStrictDelete` so that an index error is produced
instead.
Be aware however that your code might depend on this quirky behavior
so a review process is required on your part before you can use
`-d:nimStrictDelete`. To make this review easier, use the
`-d:nimAuditDelete` switch, which pretends that `system.delete` is
deprecated so that it is easier to see where it was used in your
code.
`-d:nimStrictDelete` will become the default in upcoming versions.
A similar deprecation happened with `sequtils.delete` [4], but that
deprecated form is already not used in this repo.
[1] https://github.com/nim-lang/Nim/blob/2dec69fe5aa6/doc/manual.md#exception-tracking
[2] https://github.com/nim-lang/Nim/blob/2dec69fe5aa6/changelogs/changelog_1_6_0.md#system
[3] https://github.com/nim-lang/Nim/commit/92cb76571432
[4] https://github.com/nim-lang/Nim/commit/1d6863a7899f
* system refactor, move out 600 lines
* compilation, slice, backwardsindex, misc_num moved out of system
* some procs/types moved into arithmetics, basic_types
* system no longer depends on syncio
* some procs moved around to fit with their surroundings
* make exceptions an import, old ops to misc_num
* move instantiationInfo back
* move back nim version, fix windows echo
* include compilation
* better docs for imported modules, fix unsigned ops
also remove ze, ze64, toU8, toU16, toU32 with nimPreviewSlimSystem
* fix terminal
* workaround IC test & weird csize bug, changelog
* move NimMajor etc back to compilation, rebase for CI
* try ic fix
* form single `indices`, slim out TaintedString, try fix IC
* fix CI, update changelog, addQuitProc
* fix CI
* try fix CI
* actually fix CI finally hopefully
* Update lib/system/compilation.nim
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
* update kochdocs
* hopefully fix csize uses for slimsystem
* fix tquit
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
* make more standard libraries work with `nimPreviewSlimSystem`
* typo
* part two
* Delete specutils.nim
* fixes more tests
* more fixes
* fixes tests
* fixes three more tests
* add formatfloat import
* fix
* last
* Optimize `base64.encodeMime`
* 5x faster for common scenarios, 13x faster if `lineLen` <= encoded
string's length or `newLine` is empty.
* Changed `lineLen`'s type to `Positive` to disallow `0`.
* Fix
* Revert "fixes #20285; prevent oid time overflow at year 2038"
This reverts commit dfcdb6ec2a.
* increase time to 64 bits and clean up
* add testcase
* inline consts
* add a changelog
* fixes #20285; prevent oid time overflow at year 2038
* conservative partial revert of #19814
* fix
* revert tssl
* revert azure CI change
* keep azure, revert version range
* fully revert CI, add changelog
* useOpenssl3 as separate define, .3 is a version
* Fix cannot create Windows directory in root
Fixes#20306, a regression bug with `createDir` caused by
`23e0160af283bb0bb573a86145e6c1c792780d49`.
The issue is that, if the path consists only of a drive and a single
directory (e.g. "Y:\nimcache2" in the original issue), then no
directories will be created. This works fine if there are multiple
directories (e.g. "Y:\nimcache2\test").
In the case of "Y:\nimcache2", `omitNext` in `createDir` is `false` on
the last condition in `createDir`. This means that the "nimcache2"
directory will not be created, and no exception will be raised.
Fixed by refactoring to use `parentDirs` iterator instead of iterating
over the string characters. Motivation is reduced code complexity.
Will not test the specific "C:\test" `createDir` case, since there is no
standard Windows drive with write permissions in the root. Creating a
custom drive-mapping to Windows Temp is a non-option. That could mess
up some users running the test.
Added `parentDirs` tests since they are lacking on for POSIX paths.
* Fix `createDir("")` causing error
The change to `createDir` caused `createDir("")` to raise an error,
where it previously didn't. Fixed so `createDir("")` does not fail, and
added test case.
* Implement Pandoc Markdown concise link extension
This implements https://github.com/nim-lang/Nim/issues/20127.
Besides reference to headings we also support doing references
to Nim symbols inside Nim modules.
Markdown:
```
Some heading
------------
Ref. [Some heading].
```
Nim:
```
proc someFunction*() ...
... ## Ref. [someFunction]
```
This is substitution for RST syntax like `` `target`_ ``.
All 3 syntax variants of extension from Pandoc Markdown are supported:
`[target]`, `[target][]`, `[description][target]`.
This PR also fixes clashes in existing files, particularly
conflicts with RST footnote feature, which does not work with
this PR (but there is a plan to adopt a popular [Markdown footnote
extension](https://pandoc.org/MANUAL.html#footnotes) to make footnotes work).
Also the PR fixes a bug that Markdown links did not work when `[...]`
section had a line break.
The implementation is straightforward since link resolution did not
change w.r.t. RST implementation, it's almost only about new syntax
addition. The only essential difference is a possibility to add a custom
link description: form `[description][target]` which does not have an
RST equivalent.
* fix nim 1.0 gotcha
* Add improved Windows UNC path support in std/os
Original issue: `std/os.createDir` tries to create every component of
the given path as a directory. The problem is that `createDir`
interprets every backslash/slash as a path separator. For a UNC path
this is incorrect. E.g. one UNC form is `\\Server\Volume\Path`. It's an
error to create the `\\Server` directory, as well as creating
`\\Server\Volume`.
Add `ntpath.nim` module with `splitDrive` proc. This implements UNC path
parsing as implemented in the Python `ntpath.py` module. The following
UNC forms are supported:
* `\\Server\Volume\Path`
* `\\?\Volume\Path`
* `\\?\UNC\Server\Volume\Path`
Improves support for UNC paths in various procs in `std/os`:
---
* pathnorm.addNormalizePath
* Issue: This had incomplete support for UNC paths
* The UNC prefix (first 2 characters of a UNC path) was assumed to
be exactly `\\`, but it can be `//` and `\/`, etc. as well
* Also, the UNC prefix must be normalized to the `dirSep` argument
of `addNormalizePath`
* Resolution: Changed to account for different UNC prefixes, and
normalizing the prefixes according to `dirSep`
* Affected procs that get tests: `relativePath`, `joinPath`
* Issue: The server/volume part of UNC paths can be stripped when
normalizing `..` path components
* This error should be negligable, so ignoring this
* splitPath
* Now make sure the UNC drive is not split; return the UNC drive as
`head` if the UNC drive is the only component of the path
* Consequently fixes `extractFilename`, `lastPathPart`
* parentDir / `/../`
* Strip away drive before working on the path, prepending the drive
after all work is done - prevents stripping UNC components
* Return empty string if drive component is the only component; this
is the behavior for POSIX paths as well
* Alternative implementation: Just call something like
`pathnorm.normalizePath(path & "/..")` for the whole proc - maybe
too big of a change
* tailDir
* If drive is present in path, just split that from path and return
path
* parentDirs iterator
* Uses `parentDir` for going backwards
* When going forwards, first `splitDrive`, yield the drive field, and
then iterate over path field as normal
* splitFile
* Make sure path parsing stops at end of drive component
* createDir
* Fixed by skipping drive part before creating directories
* Alternative implementation: use `parentDirs` iterator instead of
iterating over characters
* Consequence is that it will try to create the root directory
* isRootDir
* Changed to treat UNC drive alone as root (e.g. "//?/c:" is root)
* This change prevents the empty string being yielded by the
`parentDirs` iterator with `fromRoot = false`
* Internal `sameRoot`
* The "root" refers to the drive, so `splitDrive` can be used here
This adds UNC path support to all procs that could use it in std/os. I
don't think any more work has to be done to support UNC paths. For the
future, I believe the path handling code can be refactored due to
duplicate code. There are multiple ways of manipulating paths, such as
manually searching string for path separator and also having a path
normalizer (pathnorm.nim). If all path manipulation used `pathnorm.nim`,
and path component splitting used `parentDirs` iterator, then a lot of
code could be removed.
Tests
---
Added test file for `pathnorm.nim` and `ntpath.nim`.
`pathnorm.normalizePath` has no tests, so I'm adding a few unit tests.
`ntpath.nim` contains tests copied from Python's test suite.
Added integration tests to `tos.nim` that tests UNC paths.
Removed incorrect `relativePath` runnableExamples from being tested on Windows:
---
`relativePath("/Users///me/bar//z.nim", "//Users/", '/') == "me/bar/z.nim"`
This is incorrect on Windows because the `/` and `//` are not the same
root. `/` (or `\`) is expanded to the drive in the current working
directory (e.g. `C:\`). `//` (or `\\`), however, are the first two
characters of a UNC path. The following holds true for normal Windows
installations:
* `dirExists("/Users") != dirExists("//Users")`
* `dirExists("\\Users") != dirExists("\\\\Users")`
Fixes#19103
Questions:
---
* Should the `splitDrive` proc be in `os.nim` instead with copyright
notice above the proc?
* Is it fine to put most of the new tests into the `runnableExamples`
section of the procs in std/os?
* [skipci] Apply suggestions from code review
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* [skip ci] Update lib/pure/os.nim
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Move runnableExamples tests in os.nim to tos.nim
* tests/topt_no_cursor: Change from using splitFile to splitDrive
`splitFile` can no longer be used in the test, because it generates
different ARC code on Windows and Linux. This replaces `splitFile` with
`splitDrive`, because it generates same ARC code on Windows and Linux,
and returns a tuple. I assume the test wants a proc that returns a
tuple.
* Drop copyright attribute to Python
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* remove deprecated pragma syntax from 0.20.0
closes#4651, closes#16653 with a cheap fix for now due to
how early `tfFinal` is set
* remove type pragma between name and generics
* undo removal, try removing bind expression (0.8.14)
* fix test, unremove bind expr
* remove again
* Update changelog.md
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
* dependencies @ HEAD & weave test dependencies
* try fix package ci
Co-authored-by: konsumlamm <44230978+konsumlamm@users.noreply.github.com>
* micro implementation of rfc 149
refs https://github.com/nim-lang/RFCs/issues/149
* number/array/seq literals, more statements
* try fix number literal alias issue
* renew expectedType with if/case/try branch types
* fix (nerf) index type handling and float typed int
* use typeAllowed
* tweaks + const test (tested locally) [skip ci]
* fill out more of the checklist
* more literals, change @ order, type conversions
Not copying the full call tree before the typedesc call check
in `semIndirectOp` is also a small performance improvement.
* disable self-conversion warning
* revert type conversions (maybe separate op later)
* deal with CI for now (seems unrelated), try enums
* workaround CI different way
* proper fix
* again
* see sizes
* lol
* overload selection, simplify int literal -> float
* range, new @ solution, try use fitNode for nil
* use new magic
* try fix ranges, new magic, deal with #20193
* add documentation, support templates
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* Minor refactor
* Add OpenSSL 3 support
Remove symbols noOpenSSLHacksq and openssl10
* Drop loading of older openssl versions
* Add library path
* Use only versioned libssl soname os OSX
* Update .github/workflows/ci_packages.yml
Co-authored-by: Hein Thant <official.heinthanth@gmail.com>
* On Mac OS X CI, link OpenSSL in /usr/local/lib/
* Install OpenSSL on Mac OS X on azure pipeline
* Remove DYLD_LIBRARY_PATH
Co-authored-by: Hein Thant <official.heinthanth@gmail.com>
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Co-authored-by: Hein Thant <official.heinthanth@gmail.com>
* fixes#19973; switch to poll on posix
* it is fd
* exclude lwip
* fixes lwip
* rename select to timeoutRead
* refactor into timeoutRead/timeoutWrite
* refactor common parts
Co-authored-by: xflywind <43030857+xflywind@users.noreply.github.com>
* Add use of Windows Wide CRT API for env. vars
Replaces use of CRT API `getenv` and `putenv` with respectively
`_wgetenv` and `_wputenv`. Motivation is to reliably convert environment
variables to UTF-8, and the wide API is best there, because it's
reliably UTF-16.
Changed the hack in `lib/std/private/win_setenv.nim` by switching the
order of the Unicode and MBCS environment update; Unicode first, MBCS
second. Because `_wgetenv`/`_wputenv` is now used, the Unicode
environment will be initialized, so it should always be updated.
Stop updating MBCS environment with the name of `getEnv`. It's not
necessarily true that MBCS encoding and the `string` encoding is the
same. Instead convert UTF-16 to current Windows code page with
`wcstombs`, and use that string to update MBCS.
Fixes regression in `6b3c77e` that caused `std/envvars.getEnv` or
`std/os.getEnv` on Windows to return non-UTF-8 encoded strings.
Add tests that test environment variables with Unicode characters in
their name or value.
* Fix test issues
Fixes
* `nim cpp` didn't compile the tests
* Nimscript import of `tosenv.nim` from `test_nimscript.nims` failed
with "cannot importc"
* Fix missing error check on `wcstombs`
* Fix ANSI testing errors
* Separate ANSI-related testing to their own tests, and only executing
them if running process has a specific code page
* Setting locale with `setlocale` was not reliable and didn't work on
certain machines
* Add handling of a "no character representation" error in second
`wcstombs` call
* tests/newruntime_misc: Increment allocCount
Increments overall allocations in `tnewruntime_misc` test. This is
because `getEnv` now does an additional allocation: allocation of the
UTF-16 string used as parameter to `c_wgetenv`.
* Revert "tests/newruntime_misc: Increment allocCount"
This reverts commit 4d4fe8bd3e.
* tests/newruntime_misc: Increment allocCount on Windows
Increments overall allocations in `tnewruntime_misc` test for Windows.
This is because `getEnv` on Windows now does an additional allocation:
allocation of the UTF-16 string used as parameter to `c_wgetenv`.
* Refactor, adding suggestions from code review
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Document, adding suggestions
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
The Nim manual says that an implicit conversion to cstring will
eventually not be allowed [1]:
A Nim `string` is implicitly convertible to `cstring` for convenience.
[...]
Even though the conversion is implicit, it is not *safe*: The garbage collector
does not consider a `cstring` to be a root and may collect the underlying
memory. For this reason, the implicit conversion will be removed in future
releases of the Nim compiler. Certain idioms like conversion of a `const` string
to `cstring` are safe and will remain to be allowed.
And from Nim 1.6.0, such a conversion triggers a warning [2]:
A dangerous implicit conversion to `cstring` now triggers a `[CStringConv]` warning.
This warning will become an error in future versions! Use an explicit conversion
like `cstring(x)` in order to silence the warning.
However, some files in this repo produced such a warning. For example,
before this commit, compiling `parsejson.nim` would produce:
/foo/Nim/lib/pure/parsejson.nim(221, 37) Warning: implicit conversion to 'cstring' from a non-const location: my.buf; this will become a compile time error in the future [CStringConv]
/foo/Nim/lib/pure/parsejson.nim(231, 39) Warning: implicit conversion to 'cstring' from a non-const location: my.buf; this will become a compile time error in the future [CStringConv]
This commit resolves the most visible `CStringConv` warnings, making the
cstring conversions explicit.
[1] https://github.com/nim-lang/Nim/blob/d2318d9ccfe6/doc/manual.md#cstring-type
[2] https://github.com/nim-lang/Nim/blob/d2318d9ccfe6/changelogs/changelog_1_6_0.md#type-system
When checking conditions, pass `check` untyped argument directly to if. This results in better error messages when the condition is malformed.
Previously `check 1` would fail at compile time with `Error: type mismatch: got 'int literal(-2)' for '-2' but expected 'bool'`
Now it fails with `Error: type mismatch: got 'int literal(1)' for '1' but expected 'bool'`.
Similarly `check "foo"` would fail with
```
Error: type mismatch: got <string>
but expected one of:
proc `not`(a: typedesc): typedesc
first type mismatch at position: 1
required type for a: typedesc
but expression '"somestring"' is of type: string
...
```
Now it fails with `Error: type mismatch: got 'string' for '"somestring"' but expected 'bool'`
Based on what I understand from [Wikipedia](https://en.wikipedia.org/wiki/ISO_week_date#Weeks_per_year), 2001 does not have 53 weeks, but 2004, 2009, 2015, 2020 do. The years 2000 and 2001 seem to be copy pasted from the `getDaysInYear` example above. The result of `getWeeksInIsoYear` also seem to match up with Wikipedia.
That means these runnableExamples were never tested. Why is this the case? I only discovered this in #20091.
Without this, the httpclient examples are essentially setting you up for failure. I was bitten by this when my app became unable to open any more sockets.
I'm not entirely sure how long this will relevant, as I hope destructors will be added to an upcoming version of the stdlib. But figured it was worth submitting anyways!
