* implements: [C++] constructor pragma improvement (fix#21921)
t
* fix test so it doesnt use echo in globals
* Update compiler/ccgtypes.nim
* Update lib/std/private/dragonbox.nim
---------
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* Fix: make nintendoswitch someGcc, remove symlink support for nintendoswitch, add getAppFilename for nintendoswitch
* Fix: use getApplHeuristic on nintendoswitch
* docgen: implement cross-document links
Fully implements https://github.com/nim-lang/RFCs/issues/125
Follow-up of: https://github.com/nim-lang/Nim/pull/18642 (for internal links)
and https://github.com/nim-lang/Nim/issues/20127.
Overview
--------
Explicit import-like directive is required, called `.. importdoc::`.
(the syntax is % RST, Markdown will use it for a while).
Then one can reference any symbols/headings/anchors, as if they
were in the local file (but they will be prefixed with a module name
or markup document in link text).
It's possible to reference anything from anywhere (any direction
in `.nim`/`.md`/`.rst` files).
See `doc/docgen.md` for full description.
Working is based on `.idx` files, hence one needs to generate
all `.idx` beforehand. A dedicated option `--index:only` is introduced
(and a separate stage for `--index:only` is added to `kochdocs.nim`).
Performance note
----------------
Full run for `./koch docs` now takes 185% of the time before this PR.
(After: 315 s, before: 170 s on my PC).
All the time seems to be spent on `--index:only` run, which takes
almost as much (85%) of normal doc run -- it seems that most time
is spent on file parsing, turning off HTML generation phase has not
helped much.
(One could avoid it by specifying list of files that can be referenced
and pre-processing only them. But it can become error-prone and I assume
that these linke will be **everywhere** in the repository anyway,
especially considering https://github.com/nim-lang/RFCs/issues/478.
So every `.nim`/`.md` file is processed for `.idx` first).
But that's all without significant part of repository converted to
cross-module auto links. To estimate impact I checked the time for
`doc`ing a few files (after all indexes have been generated), and
everywhere difference was **negligible**.
E.g. for `lib/std/private/osfiles.nim` that `importdoc`s large
`os.idx` and hence should have been a case with relatively large
performance impact, but:
* After: 0.59 s.
* Before: 0.59 s.
So Nim compiler works so slow that doc part basically does not matter :-)
Testing
-------
1) added `extlinks` test to `nimdoc/`
2) checked that `theindex.html` is still correct
2) fixed broken auto-links for modules that were derived from `os.nim`
by adding appropriate ``importdoc``
Implementation note
-------------------
Parsing and formating of `.idx` entries is moved into a dedicated
`rstidx.nim` module from `rstgen.nim`.
`.idx` file format changed:
* fields are not escaped in most cases because we need original
strings for referencing, not HTML ones
(the exception is linkTitle for titles and headings).
Escaping happens later -- on the stage of `rstgen` buildIndex, etc.
* all lines have fixed number of columns 6
* added discriminator tag as a first column,
it always allows distinguish Nim/markup entries, titles/headings, etc.
`rstgen` does not rely any more (in most cases) on ad-hoc logic
to determine what type each entry is.
* there is now always a title entry added at the first line.
* add a line number as 6th column
* linkTitle (4th) column has a different format: before it was like
`module: funcName()`, now it's `proc funcName()`.
(This format is also propagated to `theindex.html` and search results,
I kept it that way since I like it more though it's discussible.)
This column is what used for Nim symbols resolution.
* also changed details on column format for headings and titles:
"keyword" is original, "linkTitle" is HTML one
* fix paths on Windows + more clear code
* Update compiler/docgen.nim
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* Handle .md and .nim paths uniformly in findRefFile
* handle titles better + more comments
* don't allow markup overwrite index title for .nim files
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* fix =#13790 ptr char (+friends) should not implicitly convert to cstring
* Apply suggestions from code review
* first round; compiles on windows
* nimPreviewSlimSystem
* conversion is unsafe, cast needed
* fixes more tests
* fixes asyncnet
* another try another error
* last one
* true
* one more
* why bugs didn't show at once
* add `nimPreviewCstringConversion` switch
* typo
* fixes ptr to cstring warnings[backport]
* add fixes
Co-authored-by: xflywind <43030857+xflywind@users.noreply.github.com>
* Implement Pandoc Markdown concise link extension
This implements https://github.com/nim-lang/Nim/issues/20127.
Besides reference to headings we also support doing references
to Nim symbols inside Nim modules.
Markdown:
```
Some heading
------------
Ref. [Some heading].
```
Nim:
```
proc someFunction*() ...
... ## Ref. [someFunction]
```
This is substitution for RST syntax like `` `target`_ ``.
All 3 syntax variants of extension from Pandoc Markdown are supported:
`[target]`, `[target][]`, `[description][target]`.
This PR also fixes clashes in existing files, particularly
conflicts with RST footnote feature, which does not work with
this PR (but there is a plan to adopt a popular [Markdown footnote
extension](https://pandoc.org/MANUAL.html#footnotes) to make footnotes work).
Also the PR fixes a bug that Markdown links did not work when `[...]`
section had a line break.
The implementation is straightforward since link resolution did not
change w.r.t. RST implementation, it's almost only about new syntax
addition. The only essential difference is a possibility to add a custom
link description: form `[description][target]` which does not have an
RST equivalent.
* fix nim 1.0 gotcha
* Add improved Windows UNC path support in std/os
Original issue: `std/os.createDir` tries to create every component of
the given path as a directory. The problem is that `createDir`
interprets every backslash/slash as a path separator. For a UNC path
this is incorrect. E.g. one UNC form is `\\Server\Volume\Path`. It's an
error to create the `\\Server` directory, as well as creating
`\\Server\Volume`.
Add `ntpath.nim` module with `splitDrive` proc. This implements UNC path
parsing as implemented in the Python `ntpath.py` module. The following
UNC forms are supported:
* `\\Server\Volume\Path`
* `\\?\Volume\Path`
* `\\?\UNC\Server\Volume\Path`
Improves support for UNC paths in various procs in `std/os`:
---
* pathnorm.addNormalizePath
* Issue: This had incomplete support for UNC paths
* The UNC prefix (first 2 characters of a UNC path) was assumed to
be exactly `\\`, but it can be `//` and `\/`, etc. as well
* Also, the UNC prefix must be normalized to the `dirSep` argument
of `addNormalizePath`
* Resolution: Changed to account for different UNC prefixes, and
normalizing the prefixes according to `dirSep`
* Affected procs that get tests: `relativePath`, `joinPath`
* Issue: The server/volume part of UNC paths can be stripped when
normalizing `..` path components
* This error should be negligable, so ignoring this
* splitPath
* Now make sure the UNC drive is not split; return the UNC drive as
`head` if the UNC drive is the only component of the path
* Consequently fixes `extractFilename`, `lastPathPart`
* parentDir / `/../`
* Strip away drive before working on the path, prepending the drive
after all work is done - prevents stripping UNC components
* Return empty string if drive component is the only component; this
is the behavior for POSIX paths as well
* Alternative implementation: Just call something like
`pathnorm.normalizePath(path & "/..")` for the whole proc - maybe
too big of a change
* tailDir
* If drive is present in path, just split that from path and return
path
* parentDirs iterator
* Uses `parentDir` for going backwards
* When going forwards, first `splitDrive`, yield the drive field, and
then iterate over path field as normal
* splitFile
* Make sure path parsing stops at end of drive component
* createDir
* Fixed by skipping drive part before creating directories
* Alternative implementation: use `parentDirs` iterator instead of
iterating over characters
* Consequence is that it will try to create the root directory
* isRootDir
* Changed to treat UNC drive alone as root (e.g. "//?/c:" is root)
* This change prevents the empty string being yielded by the
`parentDirs` iterator with `fromRoot = false`
* Internal `sameRoot`
* The "root" refers to the drive, so `splitDrive` can be used here
This adds UNC path support to all procs that could use it in std/os. I
don't think any more work has to be done to support UNC paths. For the
future, I believe the path handling code can be refactored due to
duplicate code. There are multiple ways of manipulating paths, such as
manually searching string for path separator and also having a path
normalizer (pathnorm.nim). If all path manipulation used `pathnorm.nim`,
and path component splitting used `parentDirs` iterator, then a lot of
code could be removed.
Tests
---
Added test file for `pathnorm.nim` and `ntpath.nim`.
`pathnorm.normalizePath` has no tests, so I'm adding a few unit tests.
`ntpath.nim` contains tests copied from Python's test suite.
Added integration tests to `tos.nim` that tests UNC paths.
Removed incorrect `relativePath` runnableExamples from being tested on Windows:
---
`relativePath("/Users///me/bar//z.nim", "//Users/", '/') == "me/bar/z.nim"`
This is incorrect on Windows because the `/` and `//` are not the same
root. `/` (or `\`) is expanded to the drive in the current working
directory (e.g. `C:\`). `//` (or `\\`), however, are the first two
characters of a UNC path. The following holds true for normal Windows
installations:
* `dirExists("/Users") != dirExists("//Users")`
* `dirExists("\\Users") != dirExists("\\\\Users")`
Fixes#19103
Questions:
---
* Should the `splitDrive` proc be in `os.nim` instead with copyright
notice above the proc?
* Is it fine to put most of the new tests into the `runnableExamples`
section of the procs in std/os?
* [skipci] Apply suggestions from code review
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* [skip ci] Update lib/pure/os.nim
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Move runnableExamples tests in os.nim to tos.nim
* tests/topt_no_cursor: Change from using splitFile to splitDrive
`splitFile` can no longer be used in the test, because it generates
different ARC code on Windows and Linux. This replaces `splitFile` with
`splitDrive`, because it generates same ARC code on Windows and Linux,
and returns a tuple. I assume the test wants a proc that returns a
tuple.
* Drop copyright attribute to Python
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Add use of Windows Wide CRT API for env. vars
Replaces use of CRT API `getenv` and `putenv` with respectively
`_wgetenv` and `_wputenv`. Motivation is to reliably convert environment
variables to UTF-8, and the wide API is best there, because it's
reliably UTF-16.
Changed the hack in `lib/std/private/win_setenv.nim` by switching the
order of the Unicode and MBCS environment update; Unicode first, MBCS
second. Because `_wgetenv`/`_wputenv` is now used, the Unicode
environment will be initialized, so it should always be updated.
Stop updating MBCS environment with the name of `getEnv`. It's not
necessarily true that MBCS encoding and the `string` encoding is the
same. Instead convert UTF-16 to current Windows code page with
`wcstombs`, and use that string to update MBCS.
Fixes regression in `6b3c77e` that caused `std/envvars.getEnv` or
`std/os.getEnv` on Windows to return non-UTF-8 encoded strings.
Add tests that test environment variables with Unicode characters in
their name or value.
* Fix test issues
Fixes
* `nim cpp` didn't compile the tests
* Nimscript import of `tosenv.nim` from `test_nimscript.nims` failed
with "cannot importc"
* Fix missing error check on `wcstombs`
* Fix ANSI testing errors
* Separate ANSI-related testing to their own tests, and only executing
them if running process has a specific code page
* Setting locale with `setlocale` was not reliable and didn't work on
certain machines
* Add handling of a "no character representation" error in second
`wcstombs` call
* tests/newruntime_misc: Increment allocCount
Increments overall allocations in `tnewruntime_misc` test. This is
because `getEnv` now does an additional allocation: allocation of the
UTF-16 string used as parameter to `c_wgetenv`.
* Revert "tests/newruntime_misc: Increment allocCount"
This reverts commit 4d4fe8bd3e.
* tests/newruntime_misc: Increment allocCount on Windows
Increments overall allocations in `tnewruntime_misc` test for Windows.
This is because `getEnv` on Windows now does an additional allocation:
allocation of the UTF-16 string used as parameter to `c_wgetenv`.
* Refactor, adding suggestions from code review
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Document, adding suggestions
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* fixes#17369
* megatest is green for --cpu:arm64
* docgen output includes more tags/raises
* implemented 'effectsOf'
* algorithm.nim: uses new effectsOf annotation
* closes#18376
* closes#17475
* closes#13905
* allow effectsOf: [a, b]
* added a test case
* parameters that are not ours cannot be declared as .effectsOf
* documentation
* manual: added the 'sort' example
* bootstrap with the new better options
