Extract markdown/rst doc into separate file (#20404)

* Extract Markdown & Rst doc into separate file

This documentation should be extracted into separate file
as it's user's documentation, which can be used as a separate
utility for compiling `.md/.rst` files.

* Restructure: move markup info into markdown_rst.md

+Markdown link migration
This commit is contained in:
Andrey Makarov
2022-09-22 13:05:53 +03:00
committed by GitHub
parent 2afce84616
commit ae3dd759c4
3 changed files with 271 additions and 247 deletions

View File

@@ -20,16 +20,6 @@ from input ``.nim`` files and projects, as well as HTML and LaTeX from input RST
dependencies (`import`), any top-level documentation comments (`##`), and
exported symbols (`*`), including procedures, types, and variables.
=================== ====================== ============ ==============
command runs on... input format output format
=================== ====================== ============ ==============
`nim doc`:cmd: documentation comments ``.nim`` ``.html`` HTML
`nim doc2tex`:cmd: ″ ″ ``.tex`` LaTeX
`nim jsondoc`:cmd: ″ ″ ``.json`` JSON
`nim rst2html`:cmd: standalone rst files ``.rst`` ``.html`` HTML
`nim rst2tex`:cmd: ″ ″ ``.tex`` LaTeX
=================== ====================== ============ ==============
Quick start
-----------
@@ -490,18 +480,8 @@ highlighting with the ``.. code-block:: nim`` prefix. ``code-block`` also
supports highlighting of a few other languages supported by the
[packages/docutils/highlite module](highlite.html).
Usage:
```cmd
nim rst2html docgen.rst
```
Output::
You're reading it!
The `rst2tex`:option: command is invoked identically to `rst2html`:option:,
but outputs a ``.tex`` file instead of ``.html``.
See [Markdown and RST markup languages](markdown_rst.html) for
usage of those commands.
HTML anchor generation
======================
@@ -628,10 +608,9 @@ Additional resources
* [Nim Compiler User Guide](nimc.html#compiler-usage-commandminusline-switches)
* Documentation for [rst module](rst.html) -- Nim RST/Markdown parser.
* [RST Quick Reference](
http://docutils.sourceforge.net/docs/user/rst/quickref.html)
* already mentioned documentation for
[Markdown and RST markup languages](markdown_rst.html), which also
contains the list of implemented features of these markup languages.
The output for HTML and LaTeX comes from the ``config/nimdoc.cfg`` and
``config/nimdoc.tex.cfg`` configuration files. You can add and modify these

258
doc/markdown_rst.md Normal file
View File

@@ -0,0 +1,258 @@
==========================================
Nim-flavored Markdown and reStructuredText
==========================================
:Author: Andrey Makarov
:Version: |nimversion|
.. default-role:: code
.. include:: rstcommon.rst
.. contents::
Both `Markdown`:idx: (md) and `reStructuredText`:idx: (RST) are markup
languages whose goal is to typeset texts with complex structure,
formatting and references using simple plaintext representation.
Command line usage
==================
Usage (to convert Markdown into HTML):
```cmd
nim md2html markdown_rst.md
```
Output::
You're reading it!
The `md2tex`:option: command is invoked identically to `md2html`:option:,
but outputs a ``.tex`` file instead of ``.html``.
These tools embedded into Nim compiler; the compiler can output
the result to HTML \[#html] or Latex \[#latex].
\[#html] commands `nim doc`:cmd: for ``*.nim`` files and
`nim rst2html`:cmd: for ``*.rst`` files
\[#latex] commands `nim doc2tex`:cmd: for ``*.nim`` and
`nim rst2tex`:cmd: for ``*.rst``.
Full list of supported commands:
=================== ====================== ============ ==============
command runs on... input format output format
=================== ====================== ============ ==============
`nim md2html`:cmd: standalone md files ``.md`` ``.html`` HTML
`nim md2tex`:cmd: same same ``.tex`` LaTeX
`nim rst2html`:cmd: standalone rst files ``.rst`` ``.html`` HTML
`nim rst2tex`:cmd: same same ``.tex`` LaTeX
`nim doc`:cmd: documentation comments ``.nim`` ``.html`` HTML
`nim doc2tex`:cmd: same same ``.tex`` LaTeX
`nim jsondoc`:cmd: same same ``.json`` JSON
=================== ====================== ============ ==============
Basic markup
============
If you are new to Markdown/RST please consider reading the following:
1) [Markdown Basic Syntax]
2) a long specification of Markdown: [CommonMark Spec]
3) a short [quick introduction] to RST
4) an [RST reference]: a comprehensive cheatsheet for RST
5) a more formal 50-page [RST specification].
Features
--------
A large subset is implemented with some [limitations] and
[additional Nim-specific features].
Supported standard RST features:
* body elements
+ sections
+ transitions
+ paragraphs
+ bullet lists using \+, \*, \-
+ enumerated lists using arabic numerals or alphabet
characters: 1. ... 2. ... *or* a. ... b. ... *or* A. ... B. ...
+ footnotes (including manually numbered, auto-numbered, auto-numbered
with label, and auto-symbol footnotes) and citations
+ definition lists
+ field lists
+ option lists
+ indented literal blocks
+ quoted literal blocks
+ line blocks
+ simple tables
+ directives (see official documentation in [RST directives list]):
- ``image``, ``figure`` for including images and videos
- ``code``
- ``contents`` (table of contents), ``container``, ``raw``
- ``include``
- admonitions: "attention", "caution", "danger", "error", "hint",
"important", "note", "tip", "warning", "admonition"
- substitution definitions: `replace` and `image`
+ comments
* inline markup
+ *emphasis*, **strong emphasis**,
``inline literals``, hyperlink references (including embedded URI),
substitution references, standalone hyperlinks,
internal links (inline and outline)
+ \`interpreted text\` with roles ``:literal:``, ``:strong:``,
``emphasis``, ``:sub:``/``:subscript:``, ``:sup:``/``:superscript:``
(see [RST roles list] for description).
+ inline internal targets
Additional Nim-specific features
--------------------------------
* directives: ``code-block`` \[cmp:Sphinx], ``title``,
``index`` \[cmp:Sphinx]
* predefined roles
- ``:nim:`` (default), ``:c:`` (C programming language),
``:python:``, ``:yaml:``, ``:java:``, ``:cpp:`` (C++), ``:csharp`` (C#).
That is every language that [highlite](highlite.html) supports.
They turn on appropriate syntax highlighting in inline code.
.. Note:: default role for Nim files is ``:nim:``,
for ``*.rst`` it's currently ``:literal:``.
- generic command line highlighting roles:
- ``:cmd:`` for commands and common shells syntax
- ``:console:`` the same for interactive sessions
(commands should be prepended by ``$``)
- ``:program:`` for executable names \[cmp:Sphinx]
(one can just use ``:cmd:`` on single word)
- ``:option:`` for command line options \[cmp:Sphinx]
- ``:tok:``, a role for highlighting of programming language tokens
* ***triple emphasis*** (bold and italic) using \*\*\*
* ``:idx:`` role for \`interpreted text\` to include the link to this
text into an index (example: [Nim index]).
* double slash `//` in option lists serves as a prefix for any option that
starts from a word (without any leading symbols like `-`, `--`, `/`)::
//compile compile the project
//doc generate documentation
Here the dummy `//` will disappear, while options `compile`:option:
and `doc`:option: will be left in the final document.
\[cmp:Sphinx] similar but different from the directives of
Python [Sphinx directives] and [Sphinx roles] extensions
Extra features
--------------
Optional additional features, by default turned on:
* emoji / smiley symbols
* Markdown tables
* Markdown code blocks. For them the same additional arguments as for RST
code blocks can be provided (e.g. `test` or `number-lines`) but with
a one-line syntax like this::
```nim test number-lines=10
echo "ok"
```
* Markdown links
* Markdown headlines
* Markdown block quotes
* using ``1`` as auto-enumerator in enumerated lists like RST ``#``
(auto-enumerator ``1`` can not be used with ``#`` in the same list)
.. Note:: By default Nim has ``roSupportMarkdown`` and
``roSupportRawDirective`` turned **on**.
.. warning:: Using Nim-specific features can cause other RST implementations
to fail on your document.
Idiosyncrasies
--------------
Currently we do **not** aim at 100% Markdown or RST compatibility in inline
markup recognition rules because that would provide very little user value.
This parser has 2 modes for inline markup:
1) Markdown-like mode which is enabled by `roPreferMarkdown` option
(turned **on** by default).
.. Note:: RST features like directives are still turned **on**
2) Compatibility mode which is RST rules.
.. Note:: in both modes the parser interpretes text between single
backticks (code) identically:
backslash does not escape; the only exception: ``\`` folowed by `
does escape so that we can always input a single backtick ` in
inline code. However that makes impossible to input code with
``\`` at the end in *single* backticks, one must use *double*
backticks::
`\` -- WRONG
``\`` -- GOOD
So single backticks can always be input: `\`` will turn to ` code
.. Attention::
We don't support some obviously poor design choices of Markdown (or RST).
- no support for the rule of 2 spaces causing a line break in Markdown
(use RST "line blocks" syntax for making line breaks)
- interpretation of Markdown block quotes is also slightly different,
e.g. case
::
>>> foo
> bar
>>baz
is a single 3rd-level quote `foo bar baz` in original Markdown, while
in Nim we naturally see it as 3rd-level quote `foo` + 1st level `bar` +
2nd level `baz`:
>>> foo
> bar
>>baz
Limitations
-----------
* no Unicode support in character width calculations
* body elements
- no roman numerals in enumerated lists
- no doctest blocks
- no grid tables
- some directives are missing (check official [RST directives list]):
``parsed-literal``, ``sidebar``, ``topic``, ``math``, ``rubric``,
``epigraph``, ``highlights``, ``pull-quote``, ``compound``,
``table``, ``csv-table``, ``list-table``, ``section-numbering``,
``header``, ``footer``, ``meta``, ``class``
- no ``role`` directives and no custom interpreted text roles
- some standard roles are not supported (check [RST roles list])
- no generic admonition support
* inline markup
- no simple-inline-markup
- no embedded aliases
Additional resources
--------------------
* See [Nim DocGen Tools Guide](docgen.html) for the details about
`nim doc`:cmd: command and idiosyncrasies of documentation markup in
``.nim`` files and Nim programming language projects.
* See also documentation for [rst module](rst.html) -- Nim RST/Markdown parser.
.. _Markdown Basic Syntax: https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax
.. _CommonMark Spec: https://spec.commonmark.org/0.30
.. _quick introduction: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
.. _RST reference: https://docutils.sourceforge.io/docs/user/rst/quickref.html
.. _RST specification: https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html
.. _RST directives list: https://docutils.sourceforge.io/docs/ref/rst/directives.html
.. _RST roles list: https://docutils.sourceforge.io/docs/ref/rst/roles.html
.. _Nim index: https://nim-lang.org/docs/theindex.html
.. _Sphinx directives: https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html
.. _Sphinx roles: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html

View File

@@ -7,231 +7,18 @@
# distribution, for details about the copyright.
#
## ==================================
## packages/docutils/rst
## ==================================
##
## ------------------------------------------
## Nim-flavored reStructuredText and Markdown
## ------------------------------------------
##
## This module implements a `reStructuredText`:idx: (RST) and
## `Markdown`:idx: parser.
## A large subset is implemented with some limitations_ and
## `Nim-specific features`_.
## Both Markdown and RST are mark-up languages whose goal is to
## typeset texts with complex structure, formatting and references
## using simple plaintext representation.
## User's manual on supported markup syntax and command line usage can be
## found in [Nim-flavored Markdown and reStructuredText](markdown_rst.html).
##
## This module is also embedded into Nim compiler; the compiler can output
## the result to HTML \[#html] or Latex \[#latex].
## * See also [Nim DocGen Tools Guide](docgen.html) for handling of
## ``.nim`` files.
## * See also [packages/docutils/rstgen module](rstgen.html) to know how to
## generate HTML or Latex strings (for embedding them into custom documents).
##
## \[#html] commands `nim doc`:cmd: for ``*.nim`` files and
## `nim rst2html`:cmd: for ``*.rst`` files
##
## \[#latex] commands `nim doc2tex`:cmd: for ``*.nim`` and
## `nim rst2tex`:cmd: for ``*.rst``.
##
## If you are new to Markdown/RST please consider reading the following:
##
## 1) `Markdown Basic Syntax`_
## 2) a long specification of Markdown: `CommonMark Spec`_
## 3) a short `quick introduction`_ to RST
## 4) an `RST reference`_: a comprehensive cheatsheet for RST
## 5) a more formal 50-page `RST specification`_.
##
## Features
## --------
##
## Supported standard RST features:
##
## * body elements
## + sections
## + transitions
## + paragraphs
## + bullet lists using \+, \*, \-
## + enumerated lists using arabic numerals or alphabet
## characters: 1. ... 2. ... *or* a. ... b. ... *or* A. ... B. ...
## + footnotes (including manually numbered, auto-numbered, auto-numbered
## with label, and auto-symbol footnotes) and citations
## + definition lists
## + field lists
## + option lists
## + indented literal blocks
## + quoted literal blocks
## + line blocks
## + simple tables
## + directives (see official documentation in `RST directives list`_):
## - ``image``, ``figure`` for including images and videos
## - ``code``
## - ``contents`` (table of contents), ``container``, ``raw``
## - ``include``
## - admonitions: "attention", "caution", "danger", "error", "hint",
## "important", "note", "tip", "warning", "admonition"
## - substitution definitions: `replace` and `image`
## + comments
## * inline markup
## + *emphasis*, **strong emphasis**,
## ``inline literals``, hyperlink references (including embedded URI),
## substitution references, standalone hyperlinks,
## internal links (inline and outline)
## + \`interpreted text\` with roles ``:literal:``, ``:strong:``,
## ``emphasis``, ``:sub:``/``:subscript:``, ``:sup:``/``:superscript:``
## (see `RST roles list`_ for description).
## + inline internal targets
##
## .. _`Nim-specific features`:
##
## Additional Nim-specific features:
##
## * directives: ``code-block`` \[cmp:Sphinx], ``title``,
## ``index`` \[cmp:Sphinx]
## * predefined roles
## - ``:nim:`` (default), ``:c:`` (C programming language),
## ``:python:``, ``:yaml:``, ``:java:``, ``:cpp:`` (C++), ``:csharp`` (C#).
## That is every language that `highlite <highlite.html>`_ supports.
## They turn on appropriate syntax highlighting in inline code.
##
## .. Note:: default role for Nim files is ``:nim:``,
## for ``*.rst`` it's currently ``:literal:``.
##
## - generic command line highlighting roles:
## - ``:cmd:`` for commands and common shells syntax
## - ``:console:`` the same for interactive sessions
## (commands should be prepended by ``$``)
## - ``:program:`` for executable names \[cmp:Sphinx]
## (one can just use ``:cmd:`` on single word)
## - ``:option:`` for command line options \[cmp:Sphinx]
## - ``:tok:``, a role for highlighting of programming language tokens
## * ***triple emphasis*** (bold and italic) using \*\*\*
## * ``:idx:`` role for \`interpreted text\` to include the link to this
## text into an index (example: `Nim index`_).
## * double slash `//` in option lists serves as a prefix for any option that
## starts from a word (without any leading symbols like `-`, `--`, `/`)::
##
## //compile compile the project
## //doc generate documentation
##
## Here the dummy `//` will disappear, while options `compile`:option:
## and `doc`:option: will be left in the final document.
##
## \[cmp:Sphinx] similar but different from the directives of
## Python `Sphinx directives`_ and `Sphinx roles`_ extensions
##
## .. _`extra features`:
##
## Optional additional features, turned on by ``options: RstParseOption`` in
## `proc rstParse`_:
##
## * emoji / smiley symbols
## * Markdown tables
## * Markdown code blocks. For them the same additional arguments as for RST
## code blocks can be provided (e.g. `test` or `number-lines`) but with
## a one-line syntax like this::
##
## ```nim test number-lines=10
## echo "ok"
## ```
## * Markdown links
## * Markdown headlines
## * Markdown block quotes
## * using ``1`` as auto-enumerator in enumerated lists like RST ``#``
## (auto-enumerator ``1`` can not be used with ``#`` in the same list)
##
## .. Note:: By default Nim has ``roSupportMarkdown`` and
## ``roSupportRawDirective`` turned **on**.
##
## .. warning:: Using Nim-specific features can cause other RST implementations
## to fail on your document.
##
## Idiosyncrasies
## --------------
##
## Currently we do **not** aim at 100% Markdown or RST compatibility in inline
## markup recognition rules because that would provide very little user value.
## This parser has 2 modes for inline markup:
##
## 1) Markdown-like mode which is enabled by `roPreferMarkdown` option
## (turned **on** by default).
##
## .. Note:: RST features like directives are still turned **on**
##
## 2) Compatibility mode which is RST rules.
##
## .. Note:: in both modes the parser interpretes text between single
## backticks (code) identically:
## backslash does not escape; the only exception: ``\`` folowed by `
## does escape so that we can always input a single backtick ` in
## inline code. However that makes impossible to input code with
## ``\`` at the end in *single* backticks, one must use *double*
## backticks::
##
## `\` -- WRONG
## ``\`` -- GOOD
## So single backticks can always be input: `\`` will turn to ` code
##
## .. Attention::
## We don't support some obviously poor design choices of Markdown (or RST).
##
## - no support for the rule of 2 spaces causing a line break in Markdown
## (use RST "line blocks" syntax for making line breaks)
##
## - interpretation of Markdown block quotes is also slightly different,
## e.g. case
##
## ::
##
## >>> foo
## > bar
## >>baz
##
## is a single 3rd-level quote `foo bar baz` in original Markdown, while
## in Nim we naturally see it as 3rd-level quote `foo` + 1st level `bar` +
## 2nd level `baz`:
##
## >>> foo
## > bar
## >>baz
##
## Limitations
## -----------
##
## * no Unicode support in character width calculations
## * body elements
## - no roman numerals in enumerated lists
## - no doctest blocks
## - no grid tables
## - some directives are missing (check official `RST directives list`_):
## ``parsed-literal``, ``sidebar``, ``topic``, ``math``, ``rubric``,
## ``epigraph``, ``highlights``, ``pull-quote``, ``compound``,
## ``table``, ``csv-table``, ``list-table``, ``section-numbering``,
## ``header``, ``footer``, ``meta``, ``class``
## - no ``role`` directives and no custom interpreted text roles
## - some standard roles are not supported (check `RST roles list`_)
## - no generic admonition support
## * inline markup
## - no simple-inline-markup
## - no embedded aliases
##
## Usage
## -----
##
## See `Nim DocGen Tools Guide <docgen.html>`_ for the details about
## `nim doc`:cmd:, `nim rst2html`:cmd: and `nim rst2tex`:cmd: commands.
##
## See `packages/docutils/rstgen module <rstgen.html>`_ to know how to
## generate HTML or Latex strings to embed them into your documents.
##
## .. _Markdown Basic Syntax: https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax
## .. _CommonMark Spec: https://spec.commonmark.org/0.30
## .. _quick introduction: https://docutils.sourceforge.io/docs/user/rst/quickstart.html
## .. _RST reference: https://docutils.sourceforge.io/docs/user/rst/quickref.html
## .. _RST specification: https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html
## .. _RST directives list: https://docutils.sourceforge.io/docs/ref/rst/directives.html
## .. _RST roles list: https://docutils.sourceforge.io/docs/ref/rst/roles.html
## .. _Nim index: https://nim-lang.org/docs/theindex.html
## .. _Sphinx directives: https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html
## .. _Sphinx roles: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html
## Choice between Markdown and RST as well as optional additional features are
## turned on by passing ``options:`` [RstParseOptions] to [proc rstParse].
import
os, strutils, rstast, dochelpers, std/enumutils, algorithm, lists, sequtils,