refs https://github.com/nim-lang/Nim/pull/23873#discussion_r1687995060,
fixes#23386, fixes#23385, supersedes #23572
Turns the `nfOpenSym` node flag implemented in #23091 and extended in
#23102 and #23873, into a node kind `nkOpenSym` that forms a unary node
containing either `nkSym` or `nkOpenSymChoice`. Since this affects
macros working on generic proc AST, the node kind is now only generated
when the experimental switch `genericsOpenSym` is enabled, and a new
node flag `nfDisabledOpenSym` is set to the `nkSym` or `nkOpenSymChoice`
when the switch is not enabled so that we can give a warning.
Now that the experimental switch has more reasonable semantics, we
define `nimHasGenericsOpenSym2`.
Continued from #23096 which was reverted due to breaking a package and
failing docgen tests. Docgen should now work, but ~~a PR is still
pending for the package: https://github.com/SciNim/Unchained/pull/45~~
has been merged
Fixes the assertion defect that happens when using `jsondoc --stdout`
(There is no outfile since its just stdout)
```
Error: unhandled exception: options.nim(732, 3) `not conf.outFile.isEmpty` [AssertionDefect]
```
Also makes the output easier to parse by ending each module output with
a new line.
* fix server caching issue causing Theme failures
* Fix tester to ignore version cache param
* fix case of people using -d:nimTestsNimdocFixup
* rsttester needed the same fix
Follow-up to #21576 (for solving #19607).
1) errors in Markdown mode for `.nim` doc comments are reported with
red color but allow to generate `.html` with the comment represented by
literate block (monospaced text). We suppose that it's what people want
for (supposedly) small doc comments. And this behavior is also a bit
more Markdown-ish in the sense that Markdown generally does not have
the concept of parsing error.
- However, for standalone `.md` it's **not** applied because for large
files the consequences are way bigger.
(In {.doctype: rst.} mode the behavior is the same as before -- report
the error and stop.)
In future, when our parser can handle Markdown without errors according to
the spec, this code will most probably be not needed.
This fixes a regression introduced in #20990 . When a group referencing
is used and one of the overloaded symbols is in `include`d file, then
`nim doc` crashes. The fix is in distinguishing (the index of) module
and file where the symbol is defined, and using only module as the
key in hash table for group referencing.
* suppresses non-exported fields of types and adds command-line option to re-enable this if desired
* corrected the doctest that produced a CI error
* an embarrassingly bad error in reasoning
* modified a nimdoc test to reflect updated behavior
* needed another change to bring utils.html doctest in sync with update
* add info
* fix nimdoc
* lint
* render postfix
* fixes a problem
* fixes nimdoc
* fix nimdoc
---------
Co-authored-by: johnperry-math <john.perry@usm.edu>
Co-authored-by: johnperry-math <devotus@yahoo.com>
* docgen: implement cross-document links
Fully implements https://github.com/nim-lang/RFCs/issues/125
Follow-up of: https://github.com/nim-lang/Nim/pull/18642 (for internal links)
and https://github.com/nim-lang/Nim/issues/20127.
Overview
--------
Explicit import-like directive is required, called `.. importdoc::`.
(the syntax is % RST, Markdown will use it for a while).
Then one can reference any symbols/headings/anchors, as if they
were in the local file (but they will be prefixed with a module name
or markup document in link text).
It's possible to reference anything from anywhere (any direction
in `.nim`/`.md`/`.rst` files).
See `doc/docgen.md` for full description.
Working is based on `.idx` files, hence one needs to generate
all `.idx` beforehand. A dedicated option `--index:only` is introduced
(and a separate stage for `--index:only` is added to `kochdocs.nim`).
Performance note
----------------
Full run for `./koch docs` now takes 185% of the time before this PR.
(After: 315 s, before: 170 s on my PC).
All the time seems to be spent on `--index:only` run, which takes
almost as much (85%) of normal doc run -- it seems that most time
is spent on file parsing, turning off HTML generation phase has not
helped much.
(One could avoid it by specifying list of files that can be referenced
and pre-processing only them. But it can become error-prone and I assume
that these linke will be **everywhere** in the repository anyway,
especially considering https://github.com/nim-lang/RFCs/issues/478.
So every `.nim`/`.md` file is processed for `.idx` first).
But that's all without significant part of repository converted to
cross-module auto links. To estimate impact I checked the time for
`doc`ing a few files (after all indexes have been generated), and
everywhere difference was **negligible**.
E.g. for `lib/std/private/osfiles.nim` that `importdoc`s large
`os.idx` and hence should have been a case with relatively large
performance impact, but:
* After: 0.59 s.
* Before: 0.59 s.
So Nim compiler works so slow that doc part basically does not matter :-)
Testing
-------
1) added `extlinks` test to `nimdoc/`
2) checked that `theindex.html` is still correct
2) fixed broken auto-links for modules that were derived from `os.nim`
by adding appropriate ``importdoc``
Implementation note
-------------------
Parsing and formating of `.idx` entries is moved into a dedicated
`rstidx.nim` module from `rstgen.nim`.
`.idx` file format changed:
* fields are not escaped in most cases because we need original
strings for referencing, not HTML ones
(the exception is linkTitle for titles and headings).
Escaping happens later -- on the stage of `rstgen` buildIndex, etc.
* all lines have fixed number of columns 6
* added discriminator tag as a first column,
it always allows distinguish Nim/markup entries, titles/headings, etc.
`rstgen` does not rely any more (in most cases) on ad-hoc logic
to determine what type each entry is.
* there is now always a title entry added at the first line.
* add a line number as 6th column
* linkTitle (4th) column has a different format: before it was like
`module: funcName()`, now it's `proc funcName()`.
(This format is also propagated to `theindex.html` and search results,
I kept it that way since I like it more though it's discussible.)
This column is what used for Nim symbols resolution.
* also changed details on column format for headings and titles:
"keyword" is original, "linkTitle" is HTML one
* fix paths on Windows + more clear code
* Update compiler/docgen.nim
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* Handle .md and .nim paths uniformly in findRefFile
* handle titles better + more comments
* don't allow markup overwrite index title for .nim files
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* Trying to fix by changing renderer
* add renderExpandUsing flag
This flag makes the renderer expand parameters that use using statement to have their full type
* Update docs
* Make comment better explain why checking for nkSym
* Fix nil access when macro/template has parameter with no type
* Fix nil access when node is not semmed yet
* Make rstgen work with gcsafe
Co-authored-by: Danil Yarantsev <tiberiumk12@gmail.com>
* add tests and fixes
* if nimHasWarningAsError
Co-authored-by: Danil Yarantsev <tiberiumk12@gmail.com>
Fix links to subheader when TOC is present
It was observed (in https://github.com/nim-lang/Nim/pull/20112)
that links to 2nd- (and subsequent) -level headings
fail if TOC is present, e.g.:
```nim
.. contents::
Type relations
==============
Convertible relation
--------------------
Ref. `Convertible relation`_
```
The problem here is that links are resolved in `rst.nim` but later
`rstgen.nim` fixes ("fixes") anchors to make them unique so that
TOC always works (if e.g. there was another sub-section like
"Convertible relation").
The solution implemented in this PR is to move that fix-up of anchors
into `rst.nim`, so that link resolution could know final anchors.
The bug seems to be added in https://github.com/nim-lang/Nim/pull/2332
in 2015, that is it is present in Nim 1.0.
* Add `doctype: RST|Markdown|RstMarkdown` pragma
Implements https://github.com/nim-lang/RFCs/issues/68 ,
see also discussion in https://github.com/nim-lang/Nim/issues/17987
The permitted values:
* `markdown`, which is default. It still contains nearly all of
the RST supported but it is assumed that in time we will give up
most or all RST features in this mode
* `rst`, without any extensions
* `RstMarkdown` — compatibility with Nim 1.x. It's basically RST
with those Markdown features enabled that don't conflict with RST.
* Apply suggestions from code review
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* Additional fix in spirit of review
* Fix test after #20188
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
* .forbids pragma: defining illegal effects for proc types
This patch intends to define the opposite of the .tags pragma: a way to define effects which are not allowed in a proc.
* updated documentation and changelogs for the forbids pragma
* renamed notTagEffects to forbiddenEffects
* corrected issues of forbids pragma
the forbids pragma didn't handle simple restrictions properly and it also had issues with subtyping
* removed incorrect character from changelog
* added test to cover the interaction of methods and the forbids pragma
* covering the interaction of the tags and forbids pragmas
* updated manual about the forbids pragma
* removed useless statement
* corrected the subtyping of proc types using the forbids pragma
* updated manual for the forbids pragma
* updated documentations for forbids pragma
* updated nim docs
* updated docs with rsttester.nim
* regenerated documentation
* updated rst docs
* Update changelog.md
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
* updated changelog
* corrected typo
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
- add additional parameters parsing (other implementations will just
ignore them). E.g. if in RST we have:
.. code:: nim
:test: "nim c $1"
...
then in Markdown that will be:
```nim test="nim c $1"
...
```
- implement Markdown interpretation of additional indentation which is
less than 4 spaces (>=4 spaces is a code block but it's not
implemented yet). RST interpretes it as quoted block, for Markdown it's
just normal paragraphs.
- add separate `md2html` and `md2tex` commands. This is to separate
Markdown behavior in cases when it diverges w.r.t. RST significantly —
most conspicously like in the case of additional indentation above, and
also currently the contradicting inline rule of Markdown is also turned
on only in `md2html` and `md2tex`. **Rationale:** mixing Markdown and
RST arbitrarily is a way to nowhere, we need to provide a way to fix the
particular behavior. Note that still all commands have **both** Markdown
and RST features **enabled**. In this PR `*.nim` files can be processed
only in Markdown mode, while `md2html` is for `*.md` files and
`rst2html` for `*.rst` files.
- rename `*.rst` files to `.*md` as our current default behavior is
already Markdown-ish
- convert code blocks in `docgen.rst` to Markdown style as an example.
Other code blocks will be converted in the follow-up PRs
- fix indentation inside Markdown code blocks — additional indentation
is preserved there
- allow more than 3 backticks open/close blocks (tildas \~ are still not
allowed to avoid conflict with RST adornment headings) see also
https://github.com/nim-lang/RFCs/issues/355
- better error messages
- (other) fix a bug that admonitions cannot be used in sandbox mode; fix
annoying warning on line 2711
* Refactor and doc package handling, module name mangling
* Consolidate, de-duplicate and extend package handling
* Alter how duplicate module names of a package are handled
* Alter how module names are mangled
* Fix crash when another package is named 'stdlib' (test case added)
* Doc what defines a package in the manual
Modules with duplicate names within a package used to be given 'fake'
packages to resolve conflicts. That prevented the ability to discern if
a module belonged to the current project package or a foreign package.
They now have the proper package owner and the names are mangled in a
consistent manner to prevent codegen clashes.
All module names are now mangled the same. Stdlib was treated special
before, but now it is same as any other package. This fixes a crash
when a foreign package is named 'stdlib'.
Module mangling is altered for both file paths and symbols used by the
backends.
Removed an unused module name to package mapping that may have been
intended for IC. The mapping was removed because it wasn't being used
and was complicating the issue of package modules with duplicate names
not having the proper package owner assigned.
* Fix some tests
* Refactor `packagehandling`
* Remove `packagehandling.withPackageName` and its uses
* Move module path mangling from `packagehandling` to `modulepaths`
* Move `options.toRodFile` to `ic` to break import cycle
* Changed import style to match preferred style
Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
