`_.
+A character literal that does not end in ``'`` is interpreted as ``'`` if there
+is a preceeding backtick token. There must be no whitespace between the preceeding
+backtick token and the character literal. This special rule ensures that a declaration
+like ``proc `'customLiteral`(s: string)`` is valid. See also
+`Custom Numeric Literals <#custom-numeric-literals>`_.
-Numerical constants
--------------------
-Numerical constants are of a single type and have the form::
+Numeric Literals
+----------------
+
+Numeric literals have the form::
hexdigit = digit | 'A'..'F' | 'a'..'f'
octdigit = '0'..'7'
bindigit = '0'..'1'
- HEX_LIT = '0' ('x' | 'X' ) hexdigit ( ['_'] hexdigit )*
- DEC_LIT = digit ( ['_'] digit )*
- OCT_LIT = '0' 'o' octdigit ( ['_'] octdigit )*
- BIN_LIT = '0' ('b' | 'B' ) bindigit ( ['_'] bindigit )*
+ unary_minus = '-' # See the section about unary minus
+ HEX_LIT = unary_minus? '0' ('x' | 'X' ) hexdigit ( ['_'] hexdigit )*
+ DEC_LIT = unary_minus? digit ( ['_'] digit )*
+ OCT_LIT = unary_minus? '0' 'o' octdigit ( ['_'] octdigit )*
+ BIN_LIT = unary_minus? '0' ('b' | 'B' ) bindigit ( ['_'] bindigit )*
INT_LIT = HEX_LIT
| DEC_LIT
@@ -521,7 +528,7 @@ Numerical constants are of a single type and have the form::
UINT64_LIT = INT_LIT ['\''] ('u' | 'U') '64'
exponent = ('e' | 'E' ) ['+' | '-'] digit ( ['_'] digit )*
- FLOAT_LIT = digit (['_'] digit)* (('.' digit (['_'] digit)* [exponent]) |exponent)
+ FLOAT_LIT = unary_minus? digit (['_'] digit)* (('.' digit (['_'] digit)* [exponent]) |exponent)
FLOAT32_SUFFIX = ('f' | 'F') ['32']
FLOAT32_LIT = HEX_LIT '\'' FLOAT32_SUFFIX
| (FLOAT_LIT | DEC_LIT | OCT_LIT | BIN_LIT) ['\''] FLOAT32_SUFFIX
@@ -529,12 +536,49 @@ Numerical constants are of a single type and have the form::
FLOAT64_LIT = HEX_LIT '\'' FLOAT64_SUFFIX
| (FLOAT_LIT | DEC_LIT | OCT_LIT | BIN_LIT) ['\''] FLOAT64_SUFFIX
+ CUSTOM_NUMERIC_LIT = (FLOAT_LIT | INT_LIT) '\'' CUSTOM_NUMERIC_SUFFIX
-As can be seen in the productions, numerical constants can contain underscores
+ # CUSTOM_NUMERIC_SUFFIX is any Nim identifier that is not
+ # a pre-defined type suffix.
+
+
+As can be seen in the productions, numeric literals can contain underscores
for readability. Integer and floating-point literals may be given in decimal (no
prefix), binary (prefix `0b`), octal (prefix `0o`), and hexadecimal
(prefix `0x`) notation.
+The fact that the unary minus `-` in a number literal like `-1` is considered
+to be part of the literal is a late addition to the language. The rationale is that
+an expression `-128'i8` should be valid and without this special case, this would
+be impossible -- `128` is not a valid `int8` value, only `-128` is.
+
+For the `unary_minus` rule there are further restrictions that are not covered
+in the formal grammar. For `-` to be part of the number literal its immediately
+preceeding character has to be in the
+set `{' ', '\t', '\n', '\r', ',', ';', '(', '[', '{'}`. This set was designed to
+cover most cases in a natural manner.
+
+In the following examples, `-1` is a single token:
+
+.. code-block:: nim
+
+ echo -1
+ echo(-1)
+ echo [-1]
+ echo 3,-1
+
+ "abc";-1
+
+In the following examples, `-1` is parsed as two separate tokens (as `- 1`):
+
+.. code-block:: nim
+
+ echo x-1
+ echo (int)-1
+ echo [a]-1
+ "abc"-1
+
+
There exists a literal for each numerical type that is
defined. The suffix starting with an apostrophe ('\'') is called a
`type suffix`:idx:. Literals without a type suffix are of an integer type
@@ -546,7 +590,7 @@ is optional if it is not ambiguous (only hexadecimal floating-point literals
with a type suffix can be ambiguous).
-The type suffixes are:
+The pre-defined type suffixes are:
================= =========================
Type Suffix Resulting type of literal
@@ -578,6 +622,43 @@ the bit width of the datatype, it is accepted.
Hence: 0b10000000'u8 == 0x80'u8 == 128, but, 0b10000000'i8 == 0x80'i8 == -1
instead of causing an overflow error.
+
+Custom Numeric Literals
+~~~~~~~~~~~~~~~~~~~~~~~
+
+If the suffix is not predefined, then the suffix is assumed to be a call
+to a proc, template, macro or other callable identifier that is passed the
+string containing the literal. The callable identifier needs to be declared
+with a special ``'`` prefix:
+
+.. code-block:: nim
+
+ import strutils
+ type u4 = distinct uint8 # a 4-bit unsigned integer aka "nibble"
+ proc `'u4`(n: string): u4 =
+ # The leading ' is required.
+ result = (parseInt(n) and 0x0F).u4
+
+ var x = 5'u4
+
+More formally, a custom numeric literal `123'custom` is transformed
+to r"123".`'custom` in the parsing step. There is no AST node kind that
+corresponds to this transformation. The transformation naturally handles
+the case that additional parameters are passed to the callee:
+
+.. code-block:: nim
+
+ import strutils
+ type u4 = distinct uint8 # a 4-bit unsigned integer aka "nibble"
+ proc `'u4`(n: string; moreData: int): u4 =
+ result = (parseInt(n) and 0x0F).u4
+
+ var x = 5'u4(123)
+
+Custom numeric literals are covered by the grammar rule named `CUSTOM_NUMERIC_LIT`.
+A custom numeric literal is a single token.
+
+
Operators
---------
diff --git a/doc/manual_experimental.rst b/doc/manual_experimental.rst
index cf2e0c2470..fc46a2a142 100644
--- a/doc/manual_experimental.rst
+++ b/doc/manual_experimental.rst
@@ -1283,7 +1283,7 @@ all the arguments, but also the matched operators in reverse polish notation:
echo x + y * z - x
This passes the expression `x + y * z - x` to the `optM` macro as
-an `nnkArglist` node containing::
+an `nnkArgList` node containing::
Arglist
Sym "x"
diff --git a/doc/nep1.rst b/doc/nep1.rst
index 3bae6a00bb..bdf8e0eab6 100644
--- a/doc/nep1.rst
+++ b/doc/nep1.rst
@@ -157,7 +157,8 @@ English word To use Notes
------------------- ------------ --------------------------------------
initialize initFoo initializes a value type `Foo`
new newFoo initializes a reference type `Foo`
- via `new`
+ via `new` or a value type `Foo`
+ with reference semantics.
this or self self for method like procs, e.g.:
`proc fun(self: Foo, a: int)`
rationale: `self` is more unique in English
diff --git a/doc/nimgrep.rst b/doc/nimgrep.rst
index 5b0fe0dbb7..52fc4d62a1 100644
--- a/doc/nimgrep.rst
+++ b/doc/nimgrep.rst
@@ -4,17 +4,20 @@
nimgrep User's manual
=========================
+.. default-role:: literal
+
:Author: Andreas Rumpf
-:Version: 0.9
+:Version: 1.6.0
+.. contents::
-Nimgrep is a command line tool for search&replace tasks. It can search for
+Nimgrep is a command line tool for search and replace tasks. It can search for
regex or peg patterns and can search whole directories at once. User
confirmation for every single replace operation can be requested.
Nimgrep has particularly good support for Nim's
-eccentric *style insensitivity*. Apart from that it is a generic text
-manipulation tool.
+eccentric *style insensitivity* (see option `-y` below).
+Apart from that it is a generic text manipulation tool.
Installation
@@ -30,23 +33,38 @@ And copy the executable somewhere in your `$PATH`.
Command line switches
=====================
-Usage:
- nimgrep [options] [pattern] [replacement] (file/directory)*
-Options:
- --find, -f find the pattern (default)
- --replace, -r replace the pattern
- --peg pattern is a peg
- --re pattern is a regular expression (default); extended
- syntax for the regular expression is always turned on
- --recursive process directories recursively
- --confirm confirm each occurrence/replacement; there is a chance
- to abort any time without touching the file
- --stdin read pattern from stdin (to avoid the shell's confusing
- quoting rules)
- --word, -w the match should have word boundaries (buggy for pegs!)
- --ignoreCase, -i be case insensitive
- --ignoreStyle, -y be style insensitive
- --ext:EX1|EX2|... only search the files with the given extension(s)
- --verbose be verbose: list every processed file
- --help, -h shows this help
- --version, -v shows the version
+.. include:: nimgrep_cmdline.txt
+
+Examples
+========
+
+All examples below use default PCRE Regex patterns:
+
++ To search recursively in Nim files using style-insensitive identifiers::
+
+ --recursive --ext:'nim|nims' --ignoreStyle
+ # short: -r --ext:'nim|nims' -y
+
+ .. Note:: we used `'` quotes to avoid special treatment of `|` symbol
+ for shells like Bash
+
++ To exclude version control directories (Git, Mercurial=hg, Subversion=svn)
+ from the search::
+
+ --excludeDir:'^\.git$' --excludeDir:'^\.hg$' --excludeDir:'^\.svn$'
+ # short: --ed:'^\.git$' --ed:'^\.hg$' --ed:'^\.svn$'
+
++ To search only in paths containing the `tests` sub-directory recursively::
+
+ --recursive --includeDir:'(^|/)tests($|/)'
+ # short: -r --id:'(^|/)tests($|/)'
+
+ .. Attention:: note the subtle difference between `--excludeDir` and
+ `--includeDir`: the former is applied to relative directory entries
+ and the latter is applied to the whole paths
+
++ Nimgrep can search multi-line, e.g. to find files containing `import`
+ and then `strutils` use::
+
+ 'import(.|\n)*?strutils'
+
diff --git a/doc/nimgrep_cmdline.txt b/doc/nimgrep_cmdline.txt
new file mode 100644
index 0000000000..bbcbcf530e
--- /dev/null
+++ b/doc/nimgrep_cmdline.txt
@@ -0,0 +1,114 @@
+
+Usage:
+
+* To search::
+ nimgrep [options] PATTERN [(FILE/DIRECTORY)*/-]
+* To replace::
+ nimgrep [options] PATTERN --replace REPLACEMENT (FILE/DIRECTORY)*/-
+* To list file names::
+ nimgrep [options] --filenames [PATTERN] [(FILE/DIRECTORY)*]
+
+Positional arguments, from left to right:
+1) PATTERN is either Regex (default) or Peg if `--peg` is specified.
+ PATTERN and REPLACEMENT should be skipped when `--stdin` is specified.
+2) REPLACEMENT supports `$1`, `$#` notations for captured groups in PATTERN.
+
+ .. DANGER:: `--replace` mode **DOES NOT** ask confirmation
+ unless `--confirm` is specified!
+
+3) Final arguments are a list of paths (FILE/DIRECTORY) or a standalone
+ minus `-` or not specified (empty):
+
+ * empty, current directory `.` is assumed (not with `--replace`)
+
+ .. Note:: so when no FILE/DIRECTORY/`-` is specified nimgrep
+ does **not** read the pipe, but searches files in the current
+ dir instead!
+ * `-`, read buffer once from stdin: pipe or terminal input;
+ in `--replace` mode the result is directed to stdout;
+ it's not compatible with `--stdin`, `--filenames`, or `--confirm`
+
+
+ For any given DIRECTORY nimgrep searches only its immediate files without
+ traversing sub-directories unless `--recursive` is specified.
+
+In replacement mode we require all 3 positional arguments to avoid damaging.
+
+Options:
+* Mode of operation:
+ --find, -f find the PATTERN (default)
+ --replace, -! replace the PATTERN to REPLACEMENT, rewriting the files
+ --confirm confirm each occurrence/replacement; there is a chance
+ to abort any time without touching the file
+ --filenames just list filenames. Provide a PATTERN to find it in
+ the filenames (not in the contents of a file) or run
+ with empty pattern to just list all files::
+ nimgrep --filenames # In current dir
+ nimgrep --filenames "" DIRECTORY
+ # Note empty pattern "", lists all files in DIRECTORY
+
+* Interprete patterns:
+ --peg PATTERN and PAT are Peg
+ --re PATTERN and PAT are regular expressions (default)
+ --rex, -x use the "extended" syntax for the regular expression
+ so that whitespace is not significant
+ --word, -w matches should have word boundaries (buggy for pegs!)
+ --ignoreCase, -i be case insensitive in PATTERN and PAT
+ --ignoreStyle, -y be style insensitive in PATTERN and PAT
+ .. Note:: PATERN and patterns PAT (see below in other options) are all either
+ Regex or Peg simultaneously and options `--rex`, `--word`, `--ignoreCase`,
+ and `--ignoreStyle` are applied to all of them.
+
+* File system walk:
+ --recursive, -r process directories recursively
+ --follow follow all symlinks when processing recursively
+ --ext:EX1|EX2|... only search the files with the given extension(s),
+ empty one ("--ext") means files with missing extension
+ --noExt:EX1|... exclude files having given extension(s), use empty one to
+ skip files with no extension (like some binary files are)
+ --includeFile:PAT search only files whose names contain pattern PAT
+ --excludeFile:PAT skip files whose names contain pattern PAT
+ --includeDir:PAT search only files with their whole directory path
+ containing PAT
+ --excludeDir:PAT skip directories whose name (not path)
+ contain pattern PAT
+ --if,--ef,--id,--ed abbreviations of the 4 options above
+ --sortTime, -s[:asc|desc]
+ order files by the last modification time (default: off):
+ ascending (recent files go last) or descending
+
+* Filter file content:
+ --match:PAT select files containing a (not displayed) match of PAT
+ --noMatch:PAT select files not containing any match of PAT
+ --bin:on|off|only process binary files? (detected by \0 in first 1K bytes)
+ (default: on - binary and text files treated the same way)
+ --text, -t process only text files, the same as `--bin:off`
+
+* Represent results:
+ --nocolor output will be given without any colors
+ --color[:on] force color even if output is redirected (default: auto)
+ --colorTheme:THEME select color THEME from `simple` (default),
+ `bnw` (black and white), `ack`, or `gnu` (GNU grep)
+ --count only print counts of matches for files that matched
+ --context:N, -c:N print N lines of leading context before every match and
+ N lines of trailing context after it (default N: 0)
+ --afterContext:N, -a:N
+ print N lines of trailing context after every match
+ --beforeContext:N, -b:N
+ print N lines of leading context before every match
+ --group, -g group matches by file
+ --newLine, -l display every matching line starting from a new line
+ --cols[:N] limit max displayed columns/width of output lines from
+ files by N characters, cropping overflows (default: off)
+ --cols:auto, -% calculate columns from terminal width for every line
+ --onlyAscii, -@ display only printable ASCII Latin characters 0x20-0x7E
+ substitutions: 0 -> ^@, 1 -> ^A, ... 0x1F -> ^_,
+ 0x7F -> '7F, ..., 0xFF -> 'FF
+
+* Miscellaneous:
+ --threads:N, -j:N speed up search by N additional workers (default: 0, off)
+ --stdin read PATTERN from stdin (to avoid the shell's confusing
+ quoting rules) and, if `--replace` given, REPLACEMENT
+ --verbose be verbose: list every processed file
+ --help, -h shows this help
+ --version, -v shows the version
diff --git a/koch.nim b/koch.nim
index bf7fb1e620..6f1da81654 100644
--- a/koch.nim
+++ b/koch.nim
@@ -458,8 +458,8 @@ proc temp(args: string) =
inc i
let d = getAppDir()
- var output = d / "compiler" / "nim".exe
- var finalDest = d / "bin" / "nim_temp".exe
+ let output = d / "compiler" / "nim".exe
+ let finalDest = d / "bin" / "nim_temp".exe
# 125 is the magic number to tell git bisect to skip the current commit.
var (bootArgs, programArgs) = splitArgs(args)
if "doc" notin programArgs and
@@ -483,6 +483,22 @@ proc xtemp(cmd: string) =
finally:
copyExe(d / "bin" / "nim_backup".exe, d / "bin" / "nim".exe)
+proc icTest(args: string) =
+ temp("")
+ let inp = os.parseCmdLine(args)[0]
+ let content = readFile(inp)
+ let nimExe = getAppDir() / "bin" / "nim_temp".exe
+ var i = 0
+ for fragment in content.split("#!EDIT!#"):
+ let file = inp.replace(".nim", "_temp.nim")
+ writeFile(file, fragment)
+ var cmd = nimExe & " cpp --ic:on --listcmd "
+ if i == 0:
+ cmd.add "-f "
+ cmd.add quoteShell(file)
+ exec(cmd)
+ inc i
+
proc buildDrNim(args: string) =
if not dirExists("dist/nimz3"):
exec("git clone https://github.com/zevv/nimz3.git dist/nimz3")
@@ -705,6 +721,7 @@ when isMainModule:
of "fusion":
let suffix = if latest: HeadHash else: FusionStableHash
exec("nimble install -y fusion@$#" % suffix)
+ of "ic": icTest(op.cmdLineRest)
else: showHelp()
break
of cmdEnd: break
diff --git a/lib/core/macros.nim b/lib/core/macros.nim
index 5a556fc8d6..8d6258e80e 100644
--- a/lib/core/macros.nim
+++ b/lib/core/macros.nim
@@ -78,7 +78,7 @@ type
nnkSharedTy, # 'shared T'
nnkEnumTy,
nnkEnumFieldDef,
- nnkArglist, nnkPattern
+ nnkArgList, nnkPattern
nnkHiddenTryStmt,
nnkClosure,
nnkGotoState,
diff --git a/lib/packages/docutils/rst.nim b/lib/packages/docutils/rst.nim
index b5eef76105..adc83c9af6 100644
--- a/lib/packages/docutils/rst.nim
+++ b/lib/packages/docutils/rst.nim
@@ -82,6 +82,14 @@
## * ***triple emphasis*** (bold and italic) using \*\*\*
## * ``:idx:`` role for \`interpreted text\` to include the link to this
## text into an index (example: `Nim index`_).
+## * double slash `//` in option lists serves as a prefix for any option that
+## starts from a word (without any leading symbols like `-`, `--`, `/`)::
+##
+## //compile compile the project
+## //doc generate documentation
+##
+## Here the dummy `//` will disappear, while options ``compile``
+## and ``doc`` will be left in the final document.
##
## .. [cmp:Sphinx] similar but different from the directives of
## Python `Sphinx directives`_ extensions
@@ -548,6 +556,67 @@ proc pushInd(p: var RstParser, ind: int) =
proc popInd(p: var RstParser) =
if p.indentStack.len > 1: setLen(p.indentStack, p.indentStack.len - 1)
+# Working with indentation in rst.nim
+# -----------------------------------
+#
+# Every line break has an associated tkIndent.
+# The tokenizer writes back the first column of next non-blank line
+# in all preceeding tkIndent tokens to the `ival` field of tkIndent.
+#
+# RST document is separated into body elements (B.E.), every of which
+# has a dedicated handler proc (or block of logic when B.E. is a block quote)
+# that should follow the next rule:
+# Every B.E. handler proc should finish at tkIndent (newline)
+# after its B.E. finishes.
+# Then its callers (which is `parseSection` or another B.E. handler)
+# check for tkIndent ival (without necessity to advance `p.idx`)
+# and decide themselves whether they continue processing or also stop.
+#
+# An example::
+#
+# L RST text fragment indentation
+# +--------------------+
+# 1 | | <- (empty line at the start of file) no tokens
+# 2 |First paragraph. | <- tkIndent has ival=0, and next tkWord has col=0
+# 3 | | <- tkIndent has ival=0
+# 4 |* bullet item and | <- tkIndent has ival=0, and next tkPunct has col=0
+# 5 | its continuation | <- tkIndent has ival=2, and next tkWord has col=2
+# 6 | | <- tkIndent has ival=4
+# 7 | Block quote | <- tkIndent has ival=4, and next tkWord has col=4
+# 8 | | <- tkIndent has ival=0
+# 9 | | <- tkIndent has ival=0
+# 10|Final paragraph | <- tkIndent has ival=0, and tkWord has col=0
+# +--------------------+
+# C:01234
+#
+# Here parser starts with initial `indentStack=[0]` and then calls the
+# 1st `parseSection`:
+#
+# - `parseSection` calls `parseParagraph` and "First paragraph" is parsed
+# - bullet list handler is started at reaching ``*`` (L4 C0), it
+# starts bullet item logic (L4 C2), which calls `pushInd(p, ind=2)`,
+# then calls `parseSection` (2nd call, nested) which parses
+# paragraph "bullet list and its continuation" and then starts
+# a block quote logic (L7 C4).
+# The block quote logic calls calls `pushInd(p, ind=4)` and
+# calls `parseSection` again, so a (simplified) sequence of calls now is::
+#
+# parseSection -> parseBulletList ->
+# parseSection (+block quote logic) -> parseSection
+#
+# 3rd `parseSection` finishes, block quote logic calls `popInd(p)`,
+# it returns to bullet item logic, which sees that next tkIndent has
+# ival=0 and stops there since the required indentation for a bullet item
+# is 2 and 0<2; the bullet item logic calls `popInd(p)`.
+# Then bullet list handler checks that next tkWord (L10 C0) has the
+# right indentation but does not have ``*`` so stops at tkIndent (L10).
+# - 1st `parseSection` invocation calls `parseParagraph` and the
+# "Final paragraph" is parsed.
+#
+# If a B.E. handler has advanced `p.idx` past tkIndent to check
+# whether it should continue its processing or not, and decided not to,
+# then this B.E. handler should step back (e.g. do `dec p.idx`).
+
proc initParser(p: var RstParser, sharedState: PSharedState) =
p.indentStack = @[0]
p.tok = @[]
@@ -852,6 +921,9 @@ proc isInlineMarkupEnd(p: RstParser, markup: string): bool =
if not result: return
# Rule 4:
if p.idx > 0:
+ # see bug #17260; for now `\` must be written ``\``, likewise with sequences
+ # ending in an un-escaped `\`; `\\` is legal but not `\\\` for example;
+ # for this reason we can't use `["``", "`"]` here.
if markup != "``" and prevTok(p).symbol == "\\":
result = false
@@ -1089,11 +1161,19 @@ proc parseUntil(p: var RstParser, father: PRstNode, postfix: string,
if isInlineMarkupEnd(p, postfix):
inc p.idx
break
- elif interpretBackslash:
- parseBackslash(p, father)
else:
- father.add(newLeaf(p))
- inc p.idx
+ if postfix == "`":
+ if prevTok(p).symbol == "\\" and currentTok(p).symbol == "`":
+ father.sons[^1] = newLeaf(p) # instead, we should use lookahead
+ else:
+ father.add(newLeaf(p))
+ inc p.idx
+ else:
+ if interpretBackslash:
+ parseBackslash(p, father)
+ else:
+ father.add(newLeaf(p))
+ inc p.idx
of tkAdornment, tkWord, tkOther:
father.add(newLeaf(p))
inc p.idx
@@ -1243,7 +1323,7 @@ proc parseInline(p: var RstParser, father: PRstNode) =
father.add(n)
elif isInlineMarkupStart(p, "`"):
var n = newRstNode(rnInterpretedText)
- parseUntil(p, n, "`", true)
+ parseUntil(p, n, "`", false) # bug #17260
n = parsePostfix(p, n)
father.add(n)
elif isInlineMarkupStart(p, "|"):
@@ -1901,8 +1981,9 @@ proc parseBulletList(p: var RstParser): PRstNode =
proc parseOptionList(p: var RstParser): PRstNode =
result = newRstNodeA(p, rnOptionList)
+ let col = currentTok(p).col
while true:
- if isOptionList(p):
+ if currentTok(p).col == col and isOptionList(p):
var a = newRstNode(rnOptionGroup)
var b = newRstNode(rnDescription)
var c = newRstNode(rnOptionListItem)
@@ -1925,6 +2006,7 @@ proc parseOptionList(p: var RstParser): PRstNode =
c.add(b)
result.add(c)
else:
+ dec p.idx # back to tkIndent
break
proc parseDefinitionList(p: var RstParser): PRstNode =
diff --git a/lib/pure/algorithm.nim b/lib/pure/algorithm.nim
index 0410d65ee5..f4b3f46beb 100644
--- a/lib/pure/algorithm.nim
+++ b/lib/pure/algorithm.nim
@@ -336,12 +336,12 @@ template `<-`(a, b) =
else:
copyMem(addr(a), addr(b), sizeof(T))
-proc merge[T](a, b: var openArray[T], lo, m, hi: int,
+proc mergeAlt[T](a, b: var openArray[T], lo, m, hi: int,
cmp: proc (x, y: T): int {.closure.}, order: SortOrder) =
# Optimization: If max(left) <= min(right) there is nothing to do!
# 1 2 3 4 ## 5 6 7 8
# -> O(n) for sorted arrays.
- # On random data this saves up to 40% of merge calls.
+ # On random data this saves up to 40% of mergeAlt calls.
if cmp(a[m], a[m+1]) * order <= 0: return
var j = lo
# copy a[j..m] into b:
@@ -424,7 +424,7 @@ func sort*[T](a: var openArray[T],
while s < n:
var m = n-1-s
while m >= 0:
- merge(a, b, max(m-s+1, 0), m, m+s, cmp, order)
+ mergeAlt(a, b, max(m-s+1, 0), m, m+s, cmp, order)
dec(m, s*2)
s = s*2
diff --git a/lib/pure/bitops.nim b/lib/pure/bitops.nim
index 377602e75d..4ef190c1b5 100644
--- a/lib/pure/bitops.nim
+++ b/lib/pure/bitops.nim
@@ -413,7 +413,6 @@ func fastlog2Nim(x: uint64): int {.inline.} =
import system/countbits_impl
-const arch64 = sizeof(int) == 8
const useBuiltinsRotate = (defined(amd64) or defined(i386)) and
(defined(gcc) or defined(clang) or defined(vcc) or
(defined(icl) and not defined(cpp))) and useBuiltins
diff --git a/lib/pure/collections/tables.nim b/lib/pure/collections/tables.nim
index f776423491..9387807c07 100644
--- a/lib/pure/collections/tables.nim
+++ b/lib/pure/collections/tables.nim
@@ -702,7 +702,7 @@ iterator mpairs*[A, B](t: var Table[A, B]): (A, var B) =
yield (t.data[h].key, t.data[h].val)
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator keys*[A, B](t: Table[A, B]): A =
+iterator keys*[A, B](t: Table[A, B]): lent A =
## Iterates over any key in the table `t`.
##
## See also:
@@ -723,7 +723,7 @@ iterator keys*[A, B](t: Table[A, B]): A =
yield t.data[h].key
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator values*[A, B](t: Table[A, B]): B =
+iterator values*[A, B](t: Table[A, B]): lent B =
## Iterates over any value in the table `t`.
##
## See also:
@@ -1146,7 +1146,7 @@ iterator mpairs*[A, B](t: TableRef[A, B]): (A, var B) =
yield (t.data[h].key, t.data[h].val)
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator keys*[A, B](t: TableRef[A, B]): A =
+iterator keys*[A, B](t: TableRef[A, B]): lent A =
## Iterates over any key in the table `t`.
##
## See also:
@@ -1167,7 +1167,7 @@ iterator keys*[A, B](t: TableRef[A, B]): A =
yield t.data[h].key
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator values*[A, B](t: TableRef[A, B]): B =
+iterator values*[A, B](t: TableRef[A, B]): lent B =
## Iterates over any value in the table `t`.
##
## See also:
@@ -1722,7 +1722,7 @@ iterator mpairs*[A, B](t: var OrderedTable[A, B]): (A, var B) =
yield (t.data[h].key, t.data[h].val)
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator keys*[A, B](t: OrderedTable[A, B]): A =
+iterator keys*[A, B](t: OrderedTable[A, B]): lent A =
## Iterates over any key in the table `t` in insertion order.
##
## See also:
@@ -1743,7 +1743,7 @@ iterator keys*[A, B](t: OrderedTable[A, B]): A =
yield t.data[h].key
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator values*[A, B](t: OrderedTable[A, B]): B =
+iterator values*[A, B](t: OrderedTable[A, B]): lent B =
## Iterates over any value in the table `t` in insertion order.
##
## See also:
@@ -2130,7 +2130,7 @@ iterator mpairs*[A, B](t: OrderedTableRef[A, B]): (A, var B) =
yield (t.data[h].key, t.data[h].val)
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator keys*[A, B](t: OrderedTableRef[A, B]): A =
+iterator keys*[A, B](t: OrderedTableRef[A, B]): lent A =
## Iterates over any key in the table `t` in insertion order.
##
## See also:
@@ -2151,7 +2151,7 @@ iterator keys*[A, B](t: OrderedTableRef[A, B]): A =
yield t.data[h].key
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator values*[A, B](t: OrderedTableRef[A, B]): B =
+iterator values*[A, B](t: OrderedTableRef[A, B]): lent B =
## Iterates over any value in the table `t` in insertion order.
##
## See also:
@@ -2543,7 +2543,7 @@ iterator mpairs*[A](t: var CountTable[A]): (A, var int) =
yield (t.data[h].key, t.data[h].val)
assert(len(t) == L, "the length of the table changed while iterating over it")
-iterator keys*[A](t: CountTable[A]): A =
+iterator keys*[A](t: CountTable[A]): lent A =
## Iterates over any key in the table `t`.
##
## See also:
diff --git a/lib/pure/htmlgen.nim b/lib/pure/htmlgen.nim
index 163f9303bb..89eb24bb94 100644
--- a/lib/pure/htmlgen.nim
+++ b/lib/pure/htmlgen.nim
@@ -322,7 +322,7 @@ macro html*(e: varargs[untyped]): untyped =
macro hr*(): untyped =
## Generates the HTML `hr` element.
- result = xmlCheckedTag(newNimNode(nnkArglist), "hr", commonAttr, "", true)
+ result = xmlCheckedTag(newNimNode(nnkArgList), "hr", commonAttr, "", true)
macro i*(e: varargs[untyped]): untyped =
## Generates the HTML `i` element.
diff --git a/lib/pure/json.nim b/lib/pure/json.nim
index 048ed2797e..7ee5be9c5d 100644
--- a/lib/pure/json.nim
+++ b/lib/pure/json.nim
@@ -927,7 +927,7 @@ when defined(js):
proc parseNativeJson(x: cstring): JSObject {.importjs: "JSON.parse(#)".}
- proc getVarType(x: JSObject): JsonNodeKind =
+ proc getVarType(x: JSObject, isRawNumber: var bool): JsonNodeKind =
result = JNull
case $getProtoName(x) # TODO: Implicit returns fail here.
of "[object Array]": return JArray
@@ -937,6 +937,7 @@ when defined(js):
if isSafeInteger(x):
return JInt
else:
+ isRawNumber = true
return JString
else:
return JFloat
@@ -946,13 +947,13 @@ when defined(js):
else: assert false
proc len(x: JSObject): int =
- assert x.getVarType == JArray
asm """
`result` = `x`.length;
"""
proc convertObject(x: JSObject): JsonNode =
- case getVarType(x)
+ var isRawNumber = false
+ case getVarType(x, isRawNumber)
of JArray:
result = newJArray()
for i in 0 ..< x.len:
@@ -973,7 +974,12 @@ when defined(js):
result = newJFloat(x.to(float))
of JString:
# Dunno what to do with isUnquoted here
- result = newJString($x.to(cstring))
+ if isRawNumber:
+ var value: cstring
+ {.emit: "`value` = `x`.toString();".}
+ result = newJRawNumber($value)
+ else:
+ result = newJString($x.to(cstring))
of JBool:
result = newJBool(x.to(bool))
of JNull:
@@ -1069,12 +1075,13 @@ when defined(nimFixedForwardGeneric):
dst = jsonNode.copy
proc initFromJson[T: SomeInteger](dst: var T; jsonNode: JsonNode, jsonPath: var string) =
- when T is uint|uint64:
+ when T is uint|uint64 or (not defined(js) and int.sizeof == 4):
+ verifyJsonKind(jsonNode, {JInt, JString}, jsonPath)
case jsonNode.kind
of JString:
- dst = T(parseBiggestUInt(jsonNode.str))
+ let x = parseBiggestUInt(jsonNode.str)
+ dst = cast[T](x)
else:
- verifyJsonKind(jsonNode, {JInt}, jsonPath)
dst = T(jsonNode.num)
else:
verifyJsonKind(jsonNode, {JInt}, jsonPath)
diff --git a/lib/pure/parsecfg.nim b/lib/pure/parsecfg.nim
index 23bb09ac4b..7f22a98572 100644
--- a/lib/pure/parsecfg.nim
+++ b/lib/pure/parsecfg.nim
@@ -174,6 +174,7 @@ runnableExamples:
import strutils, lexbase, streams, tables
import std/private/decode_helpers
+import std/private/since
include "system/inclrtl"
@@ -649,3 +650,8 @@ proc delSectionKey*(dict: var Config, section, key: string) =
dict.del(section)
else:
dict[section].del(key)
+
+iterator sections*(dict: Config): lent string {.since: (1, 5).} =
+ ## Iterates through the sections in the `dict`.
+ for section in dict.keys:
+ yield section
diff --git a/lib/std/private/underscored_calls.nim b/lib/std/private/underscored_calls.nim
index 6d0a99ab50..f0bcbcc741 100644
--- a/lib/std/private/underscored_calls.nim
+++ b/lib/std/private/underscored_calls.nim
@@ -39,7 +39,7 @@ proc underscoredCall(n, arg0: NimNode): NimNode =
result.add arg0
proc underscoredCalls*(result, calls, arg0: NimNode) =
- expectKind calls, {nnkArglist, nnkStmtList, nnkStmtListExpr}
+ expectKind calls, {nnkArgList, nnkStmtList, nnkStmtListExpr}
for call in calls:
if call.kind in {nnkStmtList, nnkStmtListExpr}:
diff --git a/lib/system.nim b/lib/system.nim
index 5740431259..27f9761196 100644
--- a/lib/system.nim
+++ b/lib/system.nim
@@ -1321,7 +1321,7 @@ proc delete*[T](x: var seq[T], i: Natural) {.noSideEffect.} =
## This is an `O(n)` operation.
##
## See also:
- ## * `del <#delete,seq[T],Natural>`_ for O(1) operation
+ ## * `del <#del,seq[T],Natural>`_ for O(1) operation
##
## .. code-block:: Nim
## var i = @[1, 2, 3, 4, 5]
@@ -2827,7 +2827,7 @@ when declared(initDebugger):
proc addEscapedChar*(s: var string, c: char) {.noSideEffect, inline.} =
## Adds a char to string `s` and applies the following escaping:
##
- ## * replaces any `\` by `\\`
+ ## * replaces any ``\`` by `\\`
## * replaces any `'` by `\'`
## * replaces any `"` by `\"`
## * replaces any `\a` by `\\a`
@@ -2838,7 +2838,7 @@ proc addEscapedChar*(s: var string, c: char) {.noSideEffect, inline.} =
## * replaces any `\f` by `\\f`
## * replaces any `\r` by `\\r`
## * replaces any `\e` by `\\e`
- ## * replaces any other character not in the set `{'\21..'\126'}
+ ## * replaces any other character not in the set `{\21..\126}`
## by `\xHH` where `HH` is its hexadecimal value.
##
## The procedure has been designed so that its output is usable for many
diff --git a/lib/system/cellsets.nim b/lib/system/cellsets.nim
index 779f1a91ff..a0f1fabf9c 100644
--- a/lib/system/cellsets.nim
+++ b/lib/system/cellsets.nim
@@ -7,7 +7,40 @@
# distribution, for details about the copyright.
#
-# Efficient set of pointers for the GC (and repr)
+
+#[
+
+Efficient set of pointers for the GC (and repr)
+-----------------------------------------------
+
+The GC depends on an extremely efficient datastructure for storing a
+set of pointers - this is called a `CellSet` in the source code.
+Inserting, deleting and searching are done in constant time. However,
+modifying a `CellSet` during traversal leads to undefined behaviour.
+
+All operations on a CellSet have to perform efficiently. Because a Cellset can
+become huge a hash table alone is not suitable for this.
+
+We use a mixture of bitset and hash table for this. The hash table maps *pages*
+to a page descriptor. The page descriptor contains a bit for any possible cell
+address within this page. So including a cell is done as follows:
+
+- Find the page descriptor for the page the cell belongs to.
+- Set the appropriate bit in the page descriptor indicating that the
+ cell points to the start of a memory block.
+
+Removing a cell is analogous - the bit has to be set to zero.
+Single page descriptors are never deleted from the hash table. This is not
+needed as the data structures needs to be rebuilt periodically anyway.
+
+Complete traversal is done in this way::
+
+ for each page descriptor d:
+ for each bit in d:
+ if bit == 1:
+ traverse the pointer belonging to this bit
+
+]#
when defined(gcOrc) or defined(gcArc):
type
diff --git a/lib/system/countbits_impl.nim b/lib/system/countbits_impl.nim
index d3c003ff7a..184f97bd47 100644
--- a/lib/system/countbits_impl.nim
+++ b/lib/system/countbits_impl.nim
@@ -18,6 +18,7 @@ const useGCC_builtins* = (defined(gcc) or defined(llvm_gcc) or
defined(clang)) and useBuiltins
const useICC_builtins* = defined(icc) and useBuiltins
const useVCC_builtins* = defined(vcc) and useBuiltins
+const arch64* = sizeof(int) == 8
template countBitsImpl(n: uint32): int =
# generic formula is from: https://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel
diff --git a/lib/system/gc.nim b/lib/system/gc.nim
index 1f71642662..ae29f3466f 100644
--- a/lib/system/gc.nim
+++ b/lib/system/gc.nim
@@ -12,6 +12,53 @@
# Refcounting + Mark&Sweep. Complex algorithms avoided.
# Been there, done that, didn't work.
+#[
+
+A *cell* is anything that is traced by the GC
+(sequences, refs, strings, closures).
+
+The basic algorithm is *Deferrent Reference Counting* with cycle detection.
+References on the stack are not counted for better performance and easier C
+code generation.
+
+Each cell has a header consisting of a RC and a pointer to its type
+descriptor. However the program does not know about these, so they are placed at
+negative offsets. In the GC code the type `PCell` denotes a pointer
+decremented by the right offset, so that the header can be accessed easily. It
+is extremely important that `pointer` is not confused with a `PCell`.
+
+In Nim the compiler cannot always know if a reference
+is stored on the stack or not. This is caused by var parameters.
+Consider this example:
+
+.. code-block:: Nim
+ proc setRef(r: var ref TNode) =
+ new(r)
+
+ proc usage =
+ var
+ r: ref TNode
+ setRef(r) # here we should not update the reference counts, because
+ # r is on the stack
+ setRef(r.left) # here we should update the refcounts!
+
+We have to decide at runtime whether the reference is on the stack or not.
+The generated code looks roughly like this:
+
+.. code-block:: C
+ void setref(TNode** ref) {
+ unsureAsgnRef(ref, newObj(TNode_TI, sizeof(TNode)))
+ }
+ void usage(void) {
+ setRef(&r)
+ setRef(&r->left)
+ }
+
+Note that for systems with a continuous stack (which most systems have)
+the check whether the ref is on the stack is very cheap (only two
+comparisons).
+]#
+
{.push profiler:off.}
const
diff --git a/nimdoc/rst2html/expected/rst_examples.html b/nimdoc/rst2html/expected/rst_examples.html
index 23d1920097..8253289a44 100644
--- a/nimdoc/rst2html/expected/rst_examples.html
+++ b/nimdoc/rst2html/expected/rst_examples.html
@@ -300,7 +300,7 @@ stmt = IND{>} stmt ^+ IND{=} DED # list of statements
However, this does not work. The problem is that the procedure should not only return, but return and continue after an iteration has finished. This return and continue is called a yield statement. Now the only thing left to do is to replace the proc keyword by iterator and here it is - our first iterator:
| A1 header | A2 | not fooled |
|---|
| C1 | C2 bold |
-| D1 code | | D2 |
+| D1 code \| | D2 |
| E1 | text | |
| F2 without pipe |
not in table
diff --git a/security.md b/security.md
index e8d31b1b93..72a1a3e3d6 100644
--- a/security.md
+++ b/security.md
@@ -2,7 +2,10 @@
## Supported Versions
-Security fixes are provided in new releases and bugfix releases.
+Security advisories are published at:
+https://github.com/nim-lang/security/security/advisories?state=published
+
+Security fixes are provided in new releases and in bugfix releases.
We do not backport security fixes to older releases.
@@ -10,8 +13,8 @@ We do not backport security fixes to older releases.
## Reporting a Vulnerability
-Please do not report vulnerabilities via GitHub issues.
-
-If you have discovered a vulnerability, it is best to notify us about it via
+If you have discovered a vulnerability, please notify us about it via
security@nim-lang.org in order to set up a meeting where we can discuss the next
steps.
+
+Please do not report vulnerabilities via GitHub issues.
diff --git a/testament/categories.nim b/testament/categories.nim
index acb3ee0a87..af6331dde3 100644
--- a/testament/categories.nim
+++ b/testament/categories.nim
@@ -173,7 +173,7 @@ proc gcTests(r: var TResults, cat: Category, options: string) =
test "stackrefleak"
test "cyclecollector"
- test "trace_globals"
+ testWithoutBoehm "trace_globals"
proc longGCTests(r: var TResults, cat: Category, options: string) =
when defined(windows):
@@ -448,7 +448,10 @@ proc testNimblePackages(r: var TResults; cat: Category; packageFilter: string) =
(outp, status) = execCmdEx(cmd, workingDir = workingDir2)
status == QuitSuccess
if not ok:
- addResult(r, test, targetC, "", cmd & "\n" & outp, reFailed)
+ if pkg.allowFailure:
+ inc r.passed
+ inc r.failedButAllowed
+ addResult(r, test, targetC, "", cmd & "\n" & outp, reFailed, allowFailure = pkg.allowFailure)
continue
outp
@@ -461,7 +464,7 @@ proc testNimblePackages(r: var TResults; cat: Category; packageFilter: string) =
discard tryCommand("nimble install --depsOnly -y", maxRetries = 3)
discard tryCommand(pkg.cmd, reFailed = reBuildFailed)
inc r.passed
- r.addResult(test, targetC, "", "", reSuccess)
+ r.addResult(test, targetC, "", "", reSuccess, allowFailure = pkg.allowFailure)
errors = r.total - r.passed
if errors == 0:
diff --git a/testament/important_packages.nim b/testament/important_packages.nim
index 1dd7c69cad..8a8d6ffd08 100644
--- a/testament/important_packages.nim
+++ b/testament/important_packages.nim
@@ -22,22 +22,26 @@ When this is the case, a workaround is to test this package here by adding `--pa
type NimblePackage* = object
name*, cmd*, url*: string
useHead*: bool
+ allowFailure*: bool
+ ## When true, we still run the test but the test is allowed to fail.
+ ## This is useful for packages that currently fail but that we still want to
+ ## run in CI, e.g. so that we can monitor when they start working again and
+ ## are reminded about those failures without making CI fail for unrelated PRs.
var packages*: seq[NimblePackage]
-proc pkg(name: string; cmd = "nimble test"; url = "", useHead = true) =
- packages.add NimblePackage(name: name, cmd: cmd, url: url, useHead: useHead)
+proc pkg(name: string; cmd = "nimble test"; url = "", useHead = true, allowFailure = false) =
+ packages.add NimblePackage(name: name, cmd: cmd, url: url, useHead: useHead, allowFailure: allowFailure)
-# pkg "alea"
+pkg "alea", allowFailure = true
pkg "argparse"
-when false:
- pkg "arraymancer", "nim c tests/tests_cpu.nim"
-# pkg "ast_pattern_matching", "nim c -r --oldgensym:on tests/test1.nim"
+pkg "arraymancer", "nim c tests/tests_cpu.nim", allowFailure = true
+pkg "ast_pattern_matching", "nim c -r --oldgensym:on tests/test1.nim", allowFailure = true
pkg "awk"
pkg "bigints", url = "https://github.com/Araq/nim-bigints"
pkg "binaryheap", "nim c -r binaryheap.nim"
pkg "BipBuffer"
-# pkg "blscurve" # pending https://github.com/status-im/nim-blscurve/issues/39
+pkg "blscurve", allowFailure = true # pending https://github.com/status-im/nim-blscurve/issues/39
pkg "bncurve"
pkg "brainfuck", "nim c -d:release -r tests/compile.nim"
pkg "bump", "nim c --gc:arc --path:. -r tests/tbump.nim", "https://github.com/disruptek/bump"
@@ -45,32 +49,30 @@ pkg "c2nim", "nim c testsuite/tester.nim"
pkg "cascade"
pkg "cello"
pkg "chroma"
-pkg "chronicles", "nim c -o:chr -r chronicles.nim"
-# when not defined(osx): # testdatagram.nim(560, 54): Check failed
-# pkg "chronos", "nim c -r -d:release tests/testall"
- # pending https://github.com/nim-lang/Nim/issues/17130
+pkg "chronicles", "nim c -o:chr -r chronicles.nim", allowFailure = true # pending https://github.com/status-im/nim-chronos/issues/169
+pkg "chronos", "nim c -r -d:release tests/testall", allowFailure = true # pending https://github.com/nim-lang/Nim/issues/17130
pkg "cligen", "nim c --path:. -r cligen.nim"
pkg "combparser", "nimble test --gc:orc"
pkg "compactdict"
pkg "comprehension", "nimble test", "https://github.com/alehander42/comprehension"
-# pkg "criterion" # pending https://github.com/disruptek/criterion/issues/3 (wrongly closed)
+pkg "criterion", allowFailure = true # pending https://github.com/disruptek/criterion/issues/3 (wrongly closed)
pkg "dashing", "nim c tests/functional.nim"
pkg "delaunay"
pkg "docopt"
pkg "easygl", "nim c -o:egl -r src/easygl.nim", "https://github.com/jackmott/easygl"
pkg "elvis"
-# pkg "fidget" # pending https://github.com/treeform/fidget/issues/133
+pkg "fidget"
pkg "fragments", "nim c -r fragments/dsl.nim"
pkg "fusion"
pkg "gara"
pkg "glob"
pkg "ggplotnim", "nim c -d:noCairo -r tests/tests.nim"
-# pkg "gittyup", "nimble test", "https://github.com/disruptek/gittyup"
+pkg "gittyup", "nimble test", "https://github.com/disruptek/gittyup", allowFailure = true
pkg "gnuplot", "nim c gnuplot.nim"
# pkg "gram", "nim c -r --gc:arc --define:danger tests/test.nim", "https://github.com/disruptek/gram"
# pending https://github.com/nim-lang/Nim/issues/16509
pkg "hts", "nim c -o:htss src/hts.nim"
-# pkg "httpauth"
+pkg "httpauth", allowFailure = true
pkg "illwill", "nimble examples"
pkg "inim"
pkg "itertools", "nim doc src/itertools.nim"
@@ -86,28 +88,28 @@ pkg "memo"
pkg "msgpack4nim", "nim c -r tests/test_spec.nim"
pkg "nake", "nim c nakefile.nim"
pkg "neo", "nim c -d:blas=openblas tests/all.nim"
-# pkg "nesm", "nimble tests" # notice plural 'tests'
-# pkg "nico"
+pkg "nesm", "nimble tests", allowFailure = true # notice plural 'tests'
+pkg "nico", allowFailure = true
pkg "nicy", "nim c -r src/nicy.nim"
pkg "nigui", "nim c -o:niguii -r src/nigui.nim"
pkg "nimcrypto", "nim r --path:. tests/testall.nim" # `--path:.` workaround needed, see D20210308T165435
pkg "NimData", "nim c -o:nimdataa src/nimdata.nim"
pkg "nimes", "nim c src/nimes.nim"
pkg "nimfp", "nim c -o:nfp -r src/fp.nim"
-# pkg "nimgame2", "nim c nimgame2/nimgame.nim" # XXX Doesn't work with deprecated 'randomize', will create a PR.
+pkg "nimgame2", "nim c nimgame2/nimgame.nim", allowFailure = true # XXX Doesn't work with deprecated 'randomize', will create a PR.
pkg "nimgen", "nim c -o:nimgenn -r src/nimgen/runcfg.nim"
pkg "nimlsp"
pkg "nimly", "nim c -r tests/test_readme_example.nim"
-# pkg "nimongo", "nimble test_ci"
-# pkg "nimph", "nimble test", "https://github.com/disruptek/nimph"
+pkg "nimongo", "nimble test_ci", allowFailure = true
+pkg "nimph", "nimble test", "https://github.com/disruptek/nimph", allowFailure = true
pkg "nimpy", "nim c -r tests/nimfrompy.nim"
pkg "nimquery"
pkg "nimsl"
pkg "nimsvg"
pkg "nimterop", "nimble minitest"
pkg "nimwc", "nim c nimwc.nim"
-# pkg "nimx", "nim c --threads:on test/main.nim"
-# pkg "nitter", "nim c src/nitter.nim", "https://github.com/zedeus/nitter"
+pkg "nimx", "nim c --threads:on test/main.nim", allowFailure = true
+pkg "nitter", "nim c src/nitter.nim", "https://github.com/zedeus/nitter", allowFailure = true
pkg "norm", "nim c -r tests/sqlite/trows.nim"
pkg "npeg", "nimble testarc"
pkg "numericalnim", "nim c -r tests/test_integrate.nim"
@@ -149,7 +151,7 @@ pkg "unicodedb", "nim c -d:release -r tests/tests.nim"
pkg "unicodeplus", "nim c -d:release -r tests/tests.nim"
pkg "unpack"
pkg "websocket", "nim c websocket.nim"
-# pkg "winim"
+pkg "winim", allowFailure = true
pkg "with"
pkg "ws"
pkg "yaml", "nim build"
diff --git a/testament/specs.nim b/testament/specs.nim
index 6b80fe41d3..b5e6de7c26 100644
--- a/testament/specs.nim
+++ b/testament/specs.nim
@@ -79,8 +79,6 @@ type
sortoutput*: bool
output*: string
line*, column*: int
- tfile*: string
- tline*, tcolumn*: int
exitCode*: int
msg*: string
ccodeCheck*: seq[string]
@@ -277,12 +275,6 @@ proc parseSpec*(filename: string): TSpec =
if result.msg.len == 0 and result.nimout.len == 0:
result.parseErrors.addLine "errormsg or msg needs to be specified before column"
discard parseInt(e.value, result.column)
- of "tfile":
- result.tfile = e.value
- of "tline":
- discard parseInt(e.value, result.tline)
- of "tcolumn":
- discard parseInt(e.value, result.tcolumn)
of "output":
if result.outputCheck != ocSubstr:
result.outputCheck = ocEqual
diff --git a/testament/testament.nim b/testament/testament.nim
index f45d4043af..462c0a57c5 100644
--- a/testament/testament.nim
+++ b/testament/testament.nim
@@ -66,7 +66,8 @@ proc isNimRepoTests(): bool =
type
Category = distinct string
TResults = object
- total, passed, skipped: int
+ total, passed, failedButAllowed, skipped: int
+ ## xxx rename passed to passedOrAllowedFailure
data: string
TTest = object
name: string
@@ -82,12 +83,6 @@ type
let
pegLineError =
peg"{[^(]*} '(' {\d+} ', ' {\d+} ') ' ('Error') ':' \s* {.*}"
-
- pegLineTemplate =
- peg"""
- {[^(]*} '(' {\d+} ', ' {\d+} ') '
- 'template/generic instantiation' ( ' of `' [^`]+ '`' )? ' from here' .*
- """
pegOtherError = peg"'Error:' \s* {.*}"
pegOfInterest = pegLineError / pegOtherError
@@ -165,7 +160,6 @@ proc callCompiler(cmdTemplate, filename, options, nimcache: string,
let outp = p.outputStream
var suc = ""
var err = ""
- var tmpl = ""
var x = newStringOfCap(120)
result.nimout = ""
while true:
@@ -174,9 +168,6 @@ proc callCompiler(cmdTemplate, filename, options, nimcache: string,
if x =~ pegOfInterest:
# `err` should contain the last error/warning message
err = x
- elif x =~ pegLineTemplate and err == "":
- # `tmpl` contains the last template expansion before the error
- tmpl = x
elif x.isSuccess:
suc = x
elif not running(p):
@@ -187,14 +178,7 @@ proc callCompiler(cmdTemplate, filename, options, nimcache: string,
result.output = ""
result.line = 0
result.column = 0
- result.tfile = ""
- result.tline = 0
- result.tcolumn = 0
result.err = reNimcCrash
- if tmpl =~ pegLineTemplate:
- result.tfile = extractFilename(matches[0])
- result.tline = parseInt(matches[1])
- result.tcolumn = parseInt(matches[2])
if err =~ pegLineError:
result.file = extractFilename(matches[0])
result.line = parseInt(matches[1])
@@ -229,6 +213,7 @@ proc callCCompiler(cmdTemplate, filename, options: string,
proc initResults: TResults =
result.total = 0
result.passed = 0
+ result.failedButAllowed = 0
result.skipped = 0
result.data = ""
@@ -256,16 +241,20 @@ template maybeStyledEcho(args: varargs[untyped]): untyped =
proc `$`(x: TResults): string =
- result = ("Tests passed: $1 / $3
\n" &
- "Tests skipped: $2 / $3
\n") %
- [$x.passed, $x.skipped, $x.total]
+ result = """
+Tests passed or allowed to fail: $2 / $1
+Tests failed and allowed to fail: $3 / $1
+Tests skipped: $4 / $1
+""" % [$x.total, $x.passed, $x.failedButAllowed, $x.skipped]
proc addResult(r: var TResults, test: TTest, target: TTarget,
- expected, given: string, successOrig: TResultEnum) =
+ expected, given: string, successOrig: TResultEnum, allowFailure = false) =
# test.name is easier to find than test.name.extractFilename
# A bit hacky but simple and works with tests/testament/tshould_not_work.nim
var name = test.name.replace(DirSep, '/')
name.add ' ' & $target
+ if allowFailure:
+ name.add " (allowed to fail) "
if test.options.len > 0: name.add ' ' & test.options
let duration = epochTime() - test.startTime
@@ -374,22 +363,13 @@ proc cmpMsgs(r: var TResults, expected, given: TSpec, test: TTest, target: TTarg
r.addResult(test, target, expected.msg, given.msg, reMsgsDiffer)
elif expected.nimout.len > 0 and not greedyOrderedSubsetLines(expected.nimout, given.nimout):
r.addResult(test, target, expected.nimout, given.nimout, reMsgsDiffer)
- elif expected.tfile == "" and extractFilename(expected.file) != extractFilename(given.file) and
+ elif extractFilename(expected.file) != extractFilename(given.file) and
"internal error:" notin expected.msg:
r.addResult(test, target, expected.file, given.file, reFilesDiffer)
elif expected.line != given.line and expected.line != 0 or
expected.column != given.column and expected.column != 0:
r.addResult(test, target, $expected.line & ':' & $expected.column,
- $given.line & ':' & $given.column,
- reLinesDiffer)
- elif expected.tfile != "" and extractFilename(expected.tfile) != extractFilename(given.tfile) and
- "internal error:" notin expected.msg:
- r.addResult(test, target, expected.tfile, given.tfile, reFilesDiffer)
- elif expected.tline != given.tline and expected.tline != 0 or
- expected.tcolumn != given.tcolumn and expected.tcolumn != 0:
- r.addResult(test, target, $expected.tline & ':' & $expected.tcolumn,
- $given.tline & ':' & $given.tcolumn,
- reLinesDiffer)
+ $given.line & ':' & $given.column, reLinesDiffer)
else:
r.addResult(test, target, expected.msg, given.msg, reSuccess)
inc(r.passed)
diff --git a/tests/arc/tarcmisc.nim b/tests/arc/tarcmisc.nim
index 55803085f2..82751f44cb 100644
--- a/tests/arc/tarcmisc.nim
+++ b/tests/arc/tarcmisc.nim
@@ -390,3 +390,35 @@ proc newPixelBuffer(): PixelBuffer =
discard newPixelBuffer()
+
+# bug #17199
+
+proc passSeq(data: seq[string]) =
+ # used the system.& proc initially
+ let wat = data & "hello"
+
+proc test2 =
+ let name = @["hello", "world"]
+ passSeq(name)
+ doAssert name == @["hello", "world"]
+
+static: test2() # was buggy
+test2()
+
+proc merge(x: sink seq[string], y: sink string): seq[string] =
+ newSeq(result, x.len + 1)
+ for i in 0..x.len-1:
+ result[i] = move(x[i])
+ result[x.len] = move(y)
+
+proc passSeq2(data: seq[string]) =
+ # used the system.& proc initially
+ let wat = merge(data, "hello")
+
+proc test3 =
+ let name = @["hello", "world"]
+ passSeq2(name)
+ doAssert name == @["hello", "world"]
+
+static: test3() # was buggy
+test3()
diff --git a/tests/arc/tmovebug.nim b/tests/arc/tmovebug.nim
index 888027186b..3ff1c4a0c9 100644
--- a/tests/arc/tmovebug.nim
+++ b/tests/arc/tmovebug.nim
@@ -107,6 +107,8 @@ sink
destroy
copy
destroy
+(f: 1)
+destroy
'''
"""
@@ -770,3 +772,15 @@ proc pair(): tuple[a: C, b: C] =
discard pair()
+
+# bug #17450
+proc noConsume(x: OO) {.nosinks.} = echo x
+
+proc main3 =
+ var i = 1
+ noConsume:
+ block:
+ OO(f: i)
+
+main3()
+
diff --git a/tests/astspec/tastspec.nim b/tests/astspec/tastspec.nim
index 3413d32f33..e2cfed2776 100644
--- a/tests/astspec/tastspec.nim
+++ b/tests/astspec/tastspec.nim
@@ -929,7 +929,7 @@ static:
(x & y & z) is string
ast.peelOff({nnkStmtList, nnkTypeSection}).matchAst:
- of nnkTypeDef(_, _, nnkTypeClassTy(nnkArglist, _, _, nnkStmtList)):
+ of nnkTypeDef(_, _, nnkTypeClassTy(nnkArgList, _, _, nnkStmtList)):
# note this isn't nnkConceptTy!
echo "ok"
diff --git a/tests/casestmt/tincompletecaseobject.nim b/tests/casestmt/tincompletecaseobject.nim
index 909ee4e1c3..aa5deda7a2 100644
--- a/tests/casestmt/tincompletecaseobject.nim
+++ b/tests/casestmt/tincompletecaseobject.nim
@@ -1,6 +1,6 @@
discard """
errormsg: '''
-not all cases are covered; missing: {nnkComesFrom, nnkDotCall, nnkHiddenCallConv, nnkVarTuple, nnkCurlyExpr, nnkRange, nnkCheckedFieldExpr, nnkDerefExpr, nnkElifExpr, nnkElseExpr, nnkLambda, nnkDo, nnkBind, nnkClosedSymChoice, nnkHiddenSubConv, nnkConv, nnkStaticExpr, nnkAddr, nnkHiddenAddr, nnkHiddenDeref, nnkObjDownConv, nnkObjUpConv, nnkChckRangeF, nnkChckRange64, nnkChckRange, nnkStringToCString, nnkCStringToString, nnkFastAsgn, nnkGenericParams, nnkFormalParams, nnkOfInherit, nnkImportAs, nnkConverterDef, nnkMacroDef, nnkTemplateDef, nnkIteratorDef, nnkOfBranch, nnkElifBranch, nnkExceptBranch, nnkElse, nnkAsmStmt, nnkTypeDef, nnkFinally, nnkContinueStmt, nnkImportStmt, nnkImportExceptStmt, nnkExportStmt, nnkExportExceptStmt, nnkFromStmt, nnkIncludeStmt, nnkUsingStmt, nnkBlockExpr, nnkStmtListType, nnkBlockType, nnkWith, nnkWithout, nnkTypeOfExpr, nnkObjectTy, nnkTupleTy, nnkTupleClassTy, nnkTypeClassTy, nnkStaticTy, nnkRecList, nnkRecCase, nnkRecWhen, nnkVarTy, nnkConstTy, nnkMutableTy, nnkDistinctTy, nnkProcTy, nnkIteratorTy, nnkSharedTy, nnkEnumTy, nnkEnumFieldDef, nnkArglist, nnkPattern, nnkReturnToken, nnkClosure, nnkGotoState, nnkState, nnkBreakState, nnkFuncDef, nnkTupleConstr}
+not all cases are covered; missing: {nnkComesFrom, nnkDotCall, nnkHiddenCallConv, nnkVarTuple, nnkCurlyExpr, nnkRange, nnkCheckedFieldExpr, nnkDerefExpr, nnkElifExpr, nnkElseExpr, nnkLambda, nnkDo, nnkBind, nnkClosedSymChoice, nnkHiddenSubConv, nnkConv, nnkStaticExpr, nnkAddr, nnkHiddenAddr, nnkHiddenDeref, nnkObjDownConv, nnkObjUpConv, nnkChckRangeF, nnkChckRange64, nnkChckRange, nnkStringToCString, nnkCStringToString, nnkFastAsgn, nnkGenericParams, nnkFormalParams, nnkOfInherit, nnkImportAs, nnkConverterDef, nnkMacroDef, nnkTemplateDef, nnkIteratorDef, nnkOfBranch, nnkElifBranch, nnkExceptBranch, nnkElse, nnkAsmStmt, nnkTypeDef, nnkFinally, nnkContinueStmt, nnkImportStmt, nnkImportExceptStmt, nnkExportStmt, nnkExportExceptStmt, nnkFromStmt, nnkIncludeStmt, nnkUsingStmt, nnkBlockExpr, nnkStmtListType, nnkBlockType, nnkWith, nnkWithout, nnkTypeOfExpr, nnkObjectTy, nnkTupleTy, nnkTupleClassTy, nnkTypeClassTy, nnkStaticTy, nnkRecList, nnkRecCase, nnkRecWhen, nnkVarTy, nnkConstTy, nnkMutableTy, nnkDistinctTy, nnkProcTy, nnkIteratorTy, nnkSharedTy, nnkEnumTy, nnkEnumFieldDef, nnkArgList, nnkPattern, nnkReturnToken, nnkClosure, nnkGotoState, nnkState, nnkBreakState, nnkFuncDef, nnkTupleConstr}
'''
"""
@@ -62,7 +62,7 @@ type
nnkSharedTy, # 'shared T'
nnkEnumTy,
nnkEnumFieldDef,
- nnkArglist, nnkPattern
+ nnkArgList, nnkPattern
nnkReturnToken,
nnkClosure,
nnkGotoState,
diff --git a/tests/destructor/tnewruntime_strutils.nim b/tests/destructor/tnewruntime_strutils.nim
index 8e5378f77d..9c8d419734 100644
--- a/tests/destructor/tnewruntime_strutils.nim
+++ b/tests/destructor/tnewruntime_strutils.nim
@@ -5,7 +5,8 @@ discard """
@[(input: @["KXSC", "BGMC"]), (input: @["PXFX"]), (input: @["WXRQ", "ZSCZD"])]
14
First tasks completed.
-Second tasks completed.'''
+Second tasks completed.
+test1'''
"""
import strutils, os, std / wordwrap
@@ -241,3 +242,13 @@ when true:
test_string_b.setLen new_len_b
echo "Second tasks completed."
+
+# bug #17450
+proc main =
+ var i = 1
+ echo:
+ block:
+ "test" & $i
+
+main()
+
diff --git a/tests/gc/trace_globals.nim b/tests/gc/trace_globals.nim
index d91bc5f355..f62a15692c 100644
--- a/tests/gc/trace_globals.nim
+++ b/tests/gc/trace_globals.nim
@@ -1,11 +1,20 @@
discard """
- output: '''10000000
+ output: '''
+10000000
10000000
10000000'''
"""
# bug #17085
+#[
+refs https://github.com/nim-lang/Nim/issues/17085#issuecomment-786466595
+with --gc:boehm, this warning sometimes gets generated:
+Warning: Repeated allocation of very large block (appr. size 14880768):
+May lead to memory leak and poor performance.
+nim CI now runs this test with `testWithoutBoehm` to avoid running it with --gc:boehm.
+]#
+
proc init(): string =
for a in 0..<10000000:
result.add 'c'
@@ -16,6 +25,8 @@ proc f() =
var c {.global.} = init()
echo a.len
+ # `echo` intentional according to
+ # https://github.com/nim-lang/Nim/pull/17469/files/0c9e94cb6b9ebca9da7cb19a063fba7aa409748e#r600016573
echo b.len
echo c.len
diff --git a/tests/ic/tgenerics.nim b/tests/ic/tgenerics.nim
index bc5c05f4fc..138799e853 100644
--- a/tests/ic/tgenerics.nim
+++ b/tests/ic/tgenerics.nim
@@ -1,6 +1,5 @@
discard """
output: "bar"
- disabled: "true"
"""
import tables
diff --git a/tests/iter/t16076.nim b/tests/iter/t16076.nim
new file mode 100644
index 0000000000..2eb4090688
--- /dev/null
+++ b/tests/iter/t16076.nim
@@ -0,0 +1,45 @@
+discard """
+ targets: "c js"
+"""
+
+proc main() =
+ block: # bug #17485
+ type
+ O = ref object
+ i: int
+
+ iterator t(o: O): int =
+ if o != nil:
+ yield o.i
+ yield 0
+
+ proc m =
+ var data = ""
+ for i in t(nil):
+ data.addInt i
+
+ doAssert data == "0"
+
+ m()
+
+
+ block: # bug #16076
+ type
+ R = ref object
+ z: int
+
+ var data = ""
+
+ iterator foo(x: int; y: R = nil): int {.inline.} =
+ if y == nil:
+ yield x
+ else:
+ yield y.z
+
+ for b in foo(10):
+ data.addInt b
+
+ doAssert data == "10"
+
+static: main()
+main()
diff --git a/tests/lexer/mlexerutils.nim b/tests/lexer/mlexerutils.nim
new file mode 100644
index 0000000000..eae7a00069
--- /dev/null
+++ b/tests/lexer/mlexerutils.nim
@@ -0,0 +1,9 @@
+import macros
+
+macro lispReprStr*(a: untyped): untyped = newLit(a.lispRepr)
+
+macro assertAST*(expected: string, struct: untyped): untyped =
+ var ast = newLit(struct.treeRepr)
+ result = quote do:
+ if `ast` != `expected`:
+ doAssert false, "\nGot:\n" & `ast`.indent(2) & "\nExpected:\n" & `expected`.indent(2)
\ No newline at end of file
diff --git a/tests/lexer/tcustom_numeric_literals.nim b/tests/lexer/tcustom_numeric_literals.nim
new file mode 100644
index 0000000000..17933950ac
--- /dev/null
+++ b/tests/lexer/tcustom_numeric_literals.nim
@@ -0,0 +1,185 @@
+discard """
+ targets: "c cpp js"
+"""
+
+# Test tkStrNumLit
+
+import std/[macros, strutils]
+import mlexerutils
+
+# AST checks
+
+assertAST dedent """
+ StmtList
+ ProcDef
+ AccQuoted
+ Ident "\'"
+ Ident "wrap"
+ Empty
+ Empty
+ FormalParams
+ Ident "string"
+ IdentDefs
+ Ident "number"
+ Ident "string"
+ Empty
+ Empty
+ Empty
+ StmtList
+ Asgn
+ Ident "result"
+ Infix
+ Ident "&"
+ Infix
+ Ident "&"
+ StrLit "[["
+ Ident "number"
+ StrLit "]]"""":
+ proc `'wrap`(number: string): string =
+ result = "[[" & number & "]]"
+
+assertAST dedent """
+ StmtList
+ DotExpr
+ RStrLit "-38383839292839283928392839283928392839283.928493849385935898243e-50000"
+ Ident "\'wrap"""":
+ -38383839292839283928392839283928392839283.928493849385935898243e-50000'wrap
+
+proc `'wrap`(number: string): string = "[[" & number & "]]"
+proc wrap2(number: string): string = "[[" & number & "]]"
+doAssert lispReprStr(-1'wrap) == """(DotExpr (RStrLit "-1") (Ident "\'wrap"))"""
+
+template main =
+ block: # basic suffix usage
+ template `'twrap`(number: string): untyped =
+ number.`'wrap`
+ proc extraContext(): string =
+ 22.40'wrap
+ proc `*`(left, right: string): string =
+ result = left & "times" & right
+ proc `+`(left, right: string): string =
+ result = left & "plus" & right
+
+ doAssert 1'wrap == "[[1]]"
+ doAssert -1'wrap == "[[-1]]":
+ "unable to resolve a negative integer-suffix pattern"
+ doAssert 12345.67890'wrap == "[[12345.67890]]"
+ doAssert 1'wrap*1'wrap == "[[1]]times[[1]]":
+ "unable to resolve an operator between two suffixed numeric literals"
+ doAssert 1'wrap+ -1'wrap == "[[1]]plus[[-1]]": # will generate a compiler warning about inconsistent spacing
+ "unable to resolve a negative suffixed numeric literal following an operator"
+ doAssert 1'wrap + -1'wrap == "[[1]]plus[[-1]]"
+ doAssert 1'twrap == "[[1]]"
+ doAssert extraContext() == "[[22.40]]":
+ "unable to return a suffixed numeric literal by an implicit return"
+ doAssert 0x5a3a'wrap == "[[0x5a3a]]"
+ doAssert 0o5732'wrap == "[[0o5732]]"
+ doAssert 0b0101111010101'wrap == "[[0b0101111010101]]"
+ doAssert -38383839292839283928392839283928392839283.928493849385935898243e-50000'wrap == "[[-38383839292839283928392839283928392839283.928493849385935898243e-50000]]"
+ doAssert 1234.56'wrap == "[[1234.56]]":
+ "unable to properly account for context with suffixed numeric literals"
+
+ block: # verify that the i64, f32, etc builtin suffixes still parse correctly
+ const expectedF32: float32 = 123.125
+ proc `'f9`(number: string): string = # proc starts with 'f' just like 'f32'
+ "[[" & number & "]]"
+ proc `'f32a`(number: string): string = # looks even more like 'f32'
+ "[[" & number & "]]"
+ proc `'d9`(number: string): string = # proc starts with 'd' just like the d suffix
+ "[[" & number & "]]"
+ proc `'i9`(number: string): string = # proc starts with 'i' just like 'i64'
+ "[[" & number & "]]"
+ proc `'u9`(number: string): string = # proc starts with 'u' just like 'u8'
+ "[[" & number & "]]"
+
+ doAssert 123.125f32 == expectedF32:
+ "failing to support non-quoted legacy f32 floating point suffix"
+ doAssert 123.125'f32 == expectedF32
+ doAssert 123.125e0'f32 == expectedF32
+ doAssert 1234.56'wrap == 1234.56'f9
+ doAssert 1234.56'wrap == 1234.56'f32a
+ doAssert 1234.56'wrap == 1234.56'd9
+ doAssert 1234.56'wrap == 1234.56'i9
+ doAssert 1234.56'wrap == 1234.56'u9
+ doAssert lispReprStr(1234.56'u9) == """(DotExpr (RStrLit "1234.56") (Ident "\'u9"))""":
+ "failed to properly build AST for suffix that starts with u"
+ doAssert -128'i8 == (-128).int8
+
+ block: # case checks
+ doAssert 1E2 == 100:
+ "lexer not handling upper-case exponent"
+ doAssert 1.0E2 == 100.0
+ doAssert 1e2 == 100
+ doAssert 0xdeadBEEF'wrap == "[[0xdeadBEEF]]":
+ "lexer not maintaining original case"
+ doAssert 0.1E12'wrap == "[[0.1E12]]"
+ doAssert 0.0e12'wrap == "[[0.0e12]]"
+ doAssert 0.0e+12'wrap == "[[0.0e+12]]"
+ doAssert 0.0e-12'wrap == "[[0.0e-12]]"
+ doAssert 0e-12'wrap == "[[0e-12]]"
+
+ block: # macro and template usage
+ template `'foo`(a: string): untyped = (a, 2)
+ doAssert -12'foo == ("-12", 2)
+ template `'fooplus`(a: string, b: int): untyped = (a, b)
+ doAssert -12'fooplus(2) == ("-12", 2)
+ template `'fooplusopt`(a: string, b: int = 99): untyped = (a, b)
+ doAssert -12'fooplusopt(2) == ("-12", 2)
+ doAssert -12'fooplusopt() == ("-12", 99)
+ doAssert -12'fooplusopt == ("-12", 99)
+ macro `'bar`(a: static string): untyped = newLit(a.repr)
+ doAssert -12'bar == "\"-12\""
+ macro deb(a): untyped = newLit(a.repr)
+ doAssert deb(-12'bar) == "-12'bar"
+
+ block: # bug 1 from https://github.com/nim-lang/Nim/pull/17020#issuecomment-803193947
+ macro deb1(a): untyped = newLit a.repr
+ macro deb2(a): untyped = newLit a.lispRepr
+ doAssert deb1(-12'wrap) == "-12'wrap"
+ doAssert deb1(-12'nonexistant) == "-12'nonexistant"
+ doAssert deb2(-12'nonexistant) == """(DotExpr (RStrLit "-12") (Ident "\'nonexistant"))"""
+ when false: # xxx bug:
+ # this holds:
+ doAssert deb2(-12.wrap2) == """(DotExpr (IntLit -12) (Sym "wrap2"))"""
+ doAssert deb2(-12'wrap) == """(DotExpr (RStrLit "-12") (Sym "\'wrap"))"""
+ # but instead this should hold:
+ doAssert deb2(-12.wrap2) == """(DotExpr (IntLit -12) (Ident "wrap2"))"""
+ doAssert deb2(-12'wrap) == """(DotExpr (RStrLit "-12") (Ident "\'wrap"))"""
+
+ block: # bug 2 from https://github.com/nim-lang/Nim/pull/17020#issuecomment-803193947
+ template toSuf(`'suf`): untyped =
+ let x = -12'suf
+ x
+ doAssert toSuf(`'wrap`) == "[[-12]]"
+
+ block: # bug 10 from https://github.com/nim-lang/Nim/pull/17020#issuecomment-803193947
+ proc `myecho`(a: auto): auto = a
+ template fn1(): untyped =
+ let a = "abc"
+ -12'wrap
+ template fn2(): untyped =
+ `myecho` -12'wrap
+ template fn3(): untyped =
+ -12'wrap
+ doAssert fn1() == "[[-12]]"
+ doAssert fn2() == "[[-12]]"
+ doAssert fn3() == "[[-12]]"
+
+ when false: # xxx this fails; bug 9 from https://github.com/nim-lang/Nim/pull/17020#issuecomment-803193947
+ #[
+ possible workaround: use `genAst` (https://github.com/nim-lang/Nim/pull/17426) and this:
+ let a3 = `'wrap3`("-128")
+ ]#
+ block:
+ macro metawrap(): untyped =
+ func wrap1(a: string): string = "{" & a & "}"
+ func `'wrap3`(a: string): string = "{" & a & "}"
+ result = quote do:
+ let a1 = wrap1"-128"
+ let a2 = -128'wrap3
+ metawrap()
+ doAssert a1 == "{-128}"
+ doAssert a2 == "{-128}"
+
+static: main()
+main()
diff --git a/tests/lexer/tunary_minus.nim b/tests/lexer/tunary_minus.nim
new file mode 100644
index 0000000000..87b3cb52dc
--- /dev/null
+++ b/tests/lexer/tunary_minus.nim
@@ -0,0 +1,81 @@
+discard """
+ targets: "c cpp js"
+"""
+
+# Test numeric literals and handling of minus symbol
+
+import std/[macros, strutils]
+
+import mlexerutils
+
+const one = 1
+const minusOne = `-`(one)
+
+# border cases that *should* generate compiler errors:
+assertAST dedent """
+ StmtList
+ Asgn
+ Ident "x"
+ Command
+ IntLit 4
+ IntLit -1""":
+ x = 4 -1
+assertAST dedent """
+ StmtList
+ VarSection
+ IdentDefs
+ Ident "x"
+ Ident "uint"
+ IntLit -1""":
+ var x: uint = -1
+template bad() =
+ x = 4 -1
+doAssert not compiles(bad())
+
+template main =
+ block: # check when a minus (-) is a negative sign for a literal
+ doAssert -1 == minusOne:
+ "unable to parse a spaced-prefixed negative int"
+ doAssert lispReprStr(-1) == """(IntLit -1)"""
+ doAssert -1.0'f64 == minusOne.float64
+ doAssert lispReprStr(-1.000'f64) == """(Float64Lit -1.0)"""
+ doAssert lispReprStr( -1.000'f64) == """(Float64Lit -1.0)"""
+ doAssert [-1].contains(minusOne):
+ "unable to handle negatives after square bracket"
+ doAssert lispReprStr([-1]) == """(Bracket (IntLit -1))"""
+ doAssert (-1, 2)[0] == minusOne:
+ "unable to handle negatives after parenthesis"
+ doAssert lispReprStr((-1, 2)) == """(Par (IntLit -1) (IntLit 2))"""
+ proc x(): int =
+ var a = 1;-1 # the -1 should act as the return value
+ doAssert x() == minusOne:
+ "unable to handle negatives after semi-colon"
+
+ block:
+ doAssert -0b111 == -7
+ doAssert -0xff == -255
+ doAssert -128'i8 == (-128).int8
+ doAssert $(-128'i8) == "-128"
+ doAssert -32768'i16 == int16.low
+ doAssert -2147483648'i32 == int32.low
+ when int.sizeof > 4:
+ doAssert -9223372036854775808 == int.low
+ when not defined(js):
+ doAssert -9223372036854775808 == int64.low
+
+ block: # check when a minus (-) is an unary op
+ doAssert -one == minusOne:
+ "unable to a negative prior to identifier"
+
+ block: # check when a minus (-) is a a subtraction op
+ doAssert 4-1 == 3:
+ "unable to handle subtraction sans surrounding spaces with a numeric literal"
+ doAssert 4-one == 3:
+ "unable to handle subtraction sans surrounding spaces with an identifier"
+ doAssert 4 - 1 == 3:
+ "unable to handle subtraction with surrounding spaces with a numeric literal"
+ doAssert 4 - one == 3:
+ "unable to handle subtraction with surrounding spaces with an identifier"
+
+static: main()
+main()
diff --git a/tests/macros/tmacros_issues.nim b/tests/macros/tmacros_issues.nim
index 19c706a821..a964f46ba8 100644
--- a/tests/macros/tmacros_issues.nim
+++ b/tests/macros/tmacros_issues.nim
@@ -64,7 +64,7 @@ block t7723:
block t8706:
macro varargsLen(args:varargs[untyped]): untyped =
- doAssert args.kind == nnkArglist
+ doAssert args.kind == nnkArgList
doAssert args.len == 0
result = newLit(args.len)
diff --git a/tests/notnil/tnotnil5.nim b/tests/notnil/tnotnil5.nim
new file mode 100644
index 0000000000..2dcb7f7c3e
--- /dev/null
+++ b/tests/notnil/tnotnil5.nim
@@ -0,0 +1,28 @@
+discard """
+ matrix: "--threads:on"
+"""
+
+{.experimental: "parallel".}
+{.experimental: "notnil".}
+import threadpool
+
+type
+ AO = object
+ x: int
+
+ A = ref AO not nil
+
+proc process(a: A): A =
+ return A(x: a.x+1)
+
+proc processMany(ayys: openArray[A]): seq[A] =
+ var newAs: seq[FlowVar[A]]
+
+ parallel:
+ for a in ayys:
+ newAs.add(spawn process(a))
+ for newAflow in newAs:
+ let newA = ^newAflow
+ if isNil(newA):
+ return @[]
+ result.add(newA)
diff --git a/tests/proc/t17157.nim b/tests/proc/t17157.nim
new file mode 100644
index 0000000000..020e93fce2
--- /dev/null
+++ b/tests/proc/t17157.nim
@@ -0,0 +1,6 @@
+discard """
+ errormsg: "'untyped' is only allowed in templates and macros or magic procs"
+"""
+
+template something(op: proc (v: untyped): void): void =
+ discard
diff --git a/tests/sets/t5792.nim b/tests/sets/t5792.nim
new file mode 100644
index 0000000000..efc38b1bc3
--- /dev/null
+++ b/tests/sets/t5792.nim
@@ -0,0 +1,17 @@
+discard """
+ matrix: "--gc:refc; --gc:arc"
+"""
+
+type
+ T = enum
+ a
+ b
+ c
+ U = object
+ case k: T
+ of a:
+ x: int
+ of {b, c} - {a}:
+ y: int
+
+discard U(k: b, y: 1)
diff --git a/tests/stdlib/tisolation.nim b/tests/stdlib/tisolation.nim
index 11c51fe051..c3857f483d 100644
--- a/tests/stdlib/tisolation.nim
+++ b/tests/stdlib/tisolation.nim
@@ -7,68 +7,88 @@ import std/[isolation, json]
-proc main() =
+proc main(moveZeroesOut: static bool) =
+ block:
+ type
+ Empty = ref object
+
+
+ var x = isolate(Empty())
+ discard extract(x)
+
block: # string literals
var data = isolate("string")
doAssert data.extract == "string"
- doAssert data.extract == ""
+ if moveZeroesOut:
+ doAssert data.extract == ""
block: # string literals
var data = isolate("")
doAssert data.extract == ""
- doAssert data.extract == ""
+ if moveZeroesOut:
+ doAssert data.extract == ""
block:
var src = "string"
var data = isolate(move src)
doAssert data.extract == "string"
- doAssert src.len == 0
+ if moveZeroesOut:
+ doAssert src.len == 0
block: # int literals
var data = isolate(1)
doAssert data.extract == 1
- doAssert data.extract == 0
+ if moveZeroesOut:
+ doAssert data.extract == 0
block: # float literals
var data = isolate(1.6)
doAssert data.extract == 1.6
- doAssert data.extract == 0.0
+ if moveZeroesOut:
+ doAssert data.extract == 0.0
block:
var data = isolate(@["1", "2"])
doAssert data.extract == @["1", "2"]
- doAssert data.extract == @[]
+ if moveZeroesOut:
+ doAssert data.extract == @[]
block:
var data = isolate(@["1", "2", "3", "4", "5"])
doAssert data.extract == @["1", "2", "3", "4", "5"]
- doAssert data.extract == @[]
+ if moveZeroesOut:
+ doAssert data.extract == @[]
block:
var data = isolate(@["", ""])
doAssert data.extract == @["", ""]
- doAssert data.extract == @[]
+ if moveZeroesOut:
+ doAssert data.extract == @[]
block:
var src = @["1", "2"]
var data = isolate(move src)
doAssert data.extract == @["1", "2"]
- doAssert src.len == 0
+ if moveZeroesOut:
+ doAssert src.len == 0
block:
var data = isolate(@[1, 2])
doAssert data.extract == @[1, 2]
- doAssert data.extract == @[]
+ if moveZeroesOut:
+ doAssert data.extract == @[]
block:
var data = isolate(["1", "2"])
doAssert data.extract == ["1", "2"]
- doAssert data.extract == ["", ""]
+ if moveZeroesOut:
+ doAssert data.extract == ["", ""]
block:
var data = isolate([1, 2])
doAssert data.extract == [1, 2]
- doAssert data.extract == [0, 0]
+ if moveZeroesOut:
+ doAssert data.extract == [0, 0]
block:
type
@@ -111,5 +131,5 @@ proc main() =
doAssert $x == """@[(value: "1234")]"""
-static: main()
-main()
+static: main(moveZeroesOut = false)
+main(moveZeroesOut = true)
diff --git a/tests/stdlib/tjson.nim b/tests/stdlib/tjson.nim
index ceb9efb0e2..e538baf4fa 100644
--- a/tests/stdlib/tjson.nim
+++ b/tests/stdlib/tjson.nim
@@ -300,3 +300,14 @@ block: # bug #17383
when not defined(js):
testRoundtrip(int64.high): "9223372036854775807"
testRoundtrip(uint64.high): "18446744073709551615"
+
+
+block:
+ let a = "18446744073709551615"
+ let b = a.parseJson
+ doAssert b.kind == JString
+ let c = $b
+ when defined(js):
+ doAssert c == "18446744073709552000"
+ else:
+ doAssert c == "18446744073709551615"
diff --git a/tests/stdlib/tjsonutils.nim b/tests/stdlib/tjsonutils.nim
index 5cd975cab7..63bf97704c 100644
--- a/tests/stdlib/tjsonutils.nim
+++ b/tests/stdlib/tjsonutils.nim
@@ -71,7 +71,7 @@ template fn() =
block:
let a = (int32.high, uint32.high)
testRoundtrip(a): "[2147483647,4294967295]"
- when not defined(js):
+ when int.sizeof > 4:
block:
let a = (int64.high, uint64.high)
testRoundtrip(a): "[9223372036854775807,18446744073709551615]"
diff --git a/tests/stdlib/tnetconnect.nim b/tests/stdlib/tnetconnect.nim
index b745457107..3a80df19fe 100644
--- a/tests/stdlib/tnetconnect.nim
+++ b/tests/stdlib/tnetconnect.nim
@@ -17,10 +17,11 @@ proc test() =
wrapSocket(ctx, socket)
# trying 2 sites makes it more resilent: refs #17458 this could give:
- # Error: unhandled exception: Call to 'connect' timed out. [TimeoutError]
+ # * Call to 'connect' timed out. [TimeoutError]
+ # * No route to host [OSError]
try:
fn("www.nim-lang.org")
- except TimeoutError:
+ except TimeoutError, OSError:
fn("www.google.com")
test()
diff --git a/tests/stdlib/tparsecfg.nim b/tests/stdlib/tparsecfg.nim
index 5c077bbdad..b2e57ac3d1 100644
--- a/tests/stdlib/tparsecfg.nim
+++ b/tests/stdlib/tparsecfg.nim
@@ -2,7 +2,7 @@ discard """
targets: "c js"
"""
-import parsecfg, streams
+import parsecfg, streams, sequtils
when not defined(js):
from stdtest/specialpaths import buildDir
@@ -39,19 +39,14 @@ var ss = newStringStream()
dict1.writeConfig(ss)
## Reading a configuration file.
-var dict2 = loadConfig(newStringStream(ss.data))
-var charset = dict2.getSectionValue("", "charset")
-var threads = dict2.getSectionValue("Package", "--threads")
-var pname = dict2.getSectionValue("Package", "name")
-var name = dict2.getSectionValue("Author", "name")
-var qq = dict2.getSectionValue("Author", "qq")
-var email = dict2.getSectionValue("Author", "email")
-doAssert charset == "utf-8"
-doAssert threads == "on"
-doAssert pname == "hello"
-doAssert name == "lihf8515"
-doAssert qq == "10214028"
-doAssert email == "lihaifeng@wxm.com"
+let dict2 = loadConfig(newStringStream(ss.data))
+doAssert dict2.getSectionValue("", "charset") == "utf-8"
+doAssert dict2.getSectionValue("Package", "--threads") == "on"
+doAssert dict2.getSectionValue("Package", "name") == "hello"
+doAssert dict2.getSectionValue("Author", "name") == "lihf8515"
+doAssert dict2.getSectionValue("Author", "qq") == "10214028"
+doAssert dict2.getSectionValue("Author", "email") == "lihaifeng@wxm.com"
+doAssert toSeq(dict2.sections) == @["", "Package", "Author"]
## Modifying a configuration file.
var dict3 = loadConfig(newStringStream(ss.data))
diff --git a/tests/stdlib/trstgen.nim b/tests/stdlib/trstgen.nim
index cf82cdf915..0f3890faa7 100644
--- a/tests/stdlib/trstgen.nim
+++ b/tests/stdlib/trstgen.nim
@@ -196,9 +196,15 @@ suite "RST/Markdown general":
| | F2 without pipe
not in table"""
let output1 = input1.toHtml
- doAssert output1 == """| A1 header | A2 | not fooled |
|---|
+ #[
+ TODO: `\|` inside a table cell should render as `|`
+ `|` outside a table cell should render as `\|`
+ consistently with markdown, see https://stackoverflow.com/a/66557930/1426932
+ ]#
+ doAssert output1 == """
+| A1 header | A2 | not fooled |
|---|
| C1 | C2 bold |
-| D1 code | | D2 |
+| D1 code \| | D2 |
| E1 | text | |
| F2 without pipe |
not in table
@@ -549,6 +555,18 @@ let x = 1
let output2 = input2.toHtml
doAssert "foo.bar"""
+ check """`foo\`\`bar`""".toHtml == """foo``bar"""
+ check """`foo\`bar`""".toHtml == """foo`bar"""
+ check """`\`bar`""".toHtml == """`bar"""
+ check """`a\b\x\\ar`""".toHtml == """a\b\x\\ar"""
+
+ test "inline literal":
+ check """``foo.bar``""".toHtml == """foo.bar"""
+ check """``foo\bar``""".toHtml == """foo\bar"""
+ check """``f\`o\\o\b`ar``""".toHtml == """f\`o\\o\b`ar"""
+
test "RST comments":
let input1 = """
Check that comment disappears:
@@ -1259,6 +1277,55 @@ Test1
let refline = "Ref. " & ref1 & "! and " & ref2 & ";and " & ref3 & "."
doAssert refline in output1
+ test "Option lists 1":
+ # check that "* b" is not consumed by previous bullet item because of
+ # incorrect indentation handling in option lists
+ let input = dedent """
+ * a
+ -m desc
+ -n very long
+ desc
+ * b"""
+ let output = input.toHtml
+ check(output.count("") == 2)
+ check(output.count("-mdesc | """ in output)
+ check("""-n | very long desc | """ in
+ output)
+
+ test "Option lists 2":
+ # check that 2nd option list is not united with the 1st
+ let input = dedent """
+ * a
+ -m desc
+ -n very long
+ desc
+ -d option"""
+ let output = input.toHtml
+ check(output.count("-mdesc | """ in output)
+ check("""-n | very long desc | """ in
+ output)
+ check("""-d | option | """ in
+ output)
+
+ test "Option list 3 (double /)":
+ let input = dedent """
+ * a
+ //compile compile1
+ //doc doc1
+ cont
+ -d option"""
+ let output = input.toHtml
+ check(output.count("compilecompile1 | """ in output)
+ check("""doc | doc1 cont | """ in
+ output)
+ check("""-d | option | """ in
+ output)
suite "RST/Code highlight":
test "Basic Python code highlight":
let pythonCode = """
diff --git a/tests/tuples/t7012.nim b/tests/tuples/t7012.nim
new file mode 100644
index 0000000000..32d441dddd
--- /dev/null
+++ b/tests/tuples/t7012.nim
@@ -0,0 +1,7 @@
+discard """
+ errormsg: "illegal recursion in type 'Node'"
+"""
+
+type Node[T] = tuple
+ next: ref Node[T]
+var n: Node[int]
\ No newline at end of file
diff --git a/tests/vm/t9622.nim b/tests/vm/t9622.nim
new file mode 100644
index 0000000000..214ab0f193
--- /dev/null
+++ b/tests/vm/t9622.nim
@@ -0,0 +1,30 @@
+discard """
+ targets: "c cpp"
+ matrix: "--gc:refc; --gc:arc"
+"""
+
+type
+ GlobNodeKind = enum
+ LiteralIdent,
+ Group
+
+ GlobNode = object
+ case kind: GlobNodeKind
+ of LiteralIdent:
+ value: string
+ of Group:
+ values: seq[string]
+
+ PathSegment = object
+ children: seq[GlobNode]
+
+ GlobPattern = seq[PathSegment]
+
+proc parseImpl(): GlobPattern =
+ if result.len == 0:
+ result.add PathSegment()
+ result[^1].children.add GlobNode(kind: LiteralIdent)
+
+block:
+ const pattern = parseImpl()
+ doAssert $pattern == """@[(children: @[(kind: LiteralIdent, value: "")])]"""
diff --git a/tests/vm/tvmmisc.nim b/tests/vm/tvmmisc.nim
index a485bac2e9..f542fa5608 100644
--- a/tests/vm/tvmmisc.nim
+++ b/tests/vm/tvmmisc.nim
@@ -255,6 +255,31 @@ block:
doAssert e == @[]
doAssert f == @[]
+
+block: # bug #10815
+ type
+ Opcode = enum
+ iChar, iSet
+
+ Inst = object
+ case code: Opcode
+ of iChar:
+ c: char
+ of iSet:
+ cs: set[char]
+
+ Patt = seq[Inst]
+
+
+ proc `$`(p: Patt): string =
+ discard
+
+ proc P(): Patt =
+ result.add Inst(code: iSet)
+
+ const a = P()
+ doAssert $a == ""
+
import tables
block: # bug #8007
diff --git a/tools/nimgrep.nim b/tools/nimgrep.nim
index cb46f30b87..a368c40efc 100644
--- a/tools/nimgrep.nim
+++ b/tools/nimgrep.nim
@@ -16,106 +16,7 @@ const
Version & """
(c) 2012-2020 Andreas Rumpf
-
-Usage:
-* To search:
- nimgrep [options] PATTERN [(FILE/DIRECTORY)*/-]
-* To replace:
- nimgrep [options] PATTERN --replace REPLACEMENT (FILE/DIRECTORY)*/-
-* To list file names:
- nimgrep [options] --filenames [PATTERN] [(FILE/DIRECTORY)*]
-
-Positional arguments, from left to right:
-* PATERN is either Regex (default) or Peg if --peg is specified.
- PATTERN and REPLACEMENT should be skipped when --stdin is specified.
-* REPLACEMENT supports $1, $# notations for captured groups in PATTERN.
- Note: --replace mode DOES NOT ask confirmation unless --confirm is specified!
-* Final arguments are a list of paths (FILE/DIRECTORY) or a standalone
- minus '-' (pipe) or not specified (empty). Note for the empty case: when
- no FILE/DIRECTORY/- is specified nimgrep DOES NOT read the pipe, but
- searches files in the current dir instead!
- - read buffer once from stdin: pipe or terminal input;
- in --replace mode the result is directed to stdout;
- it's not compatible with --stdin, --filenames, --confirm
- (empty) current directory '.' is assumed (not with --replace)
- For any given DIRECTORY nimgrep searches only its immediate files without
- traversing sub-directories unless --recursive is specified.
- In replacement mode all 3 positional arguments are required to avoid damaging.
-
-Options:
-* Mode of operation:
- --find, -f find the PATTERN (default)
- --replace, -! replace the PATTERN to REPLACEMENT, rewriting the files
- --confirm confirm each occurrence/replacement; there is a chance
- to abort any time without touching the file
- --filenames just list filenames. Provide a PATTERN to find it in
- the filenames (not in the contents of a file) or run
- with empty pattern to just list all files:
- nimgrep --filenames # In current directory
- nimgrep --filenames "" DIRECTORY # Note empty pattern ""
-
-* Interprete patterns:
- --peg PATTERN and PAT are Peg
- --re PATTERN and PAT are regular expressions (default)
- --rex, -x use the "extended" syntax for the regular expression
- so that whitespace is not significant
- --word, -w matches should have word boundaries (buggy for pegs!)
- --ignoreCase, -i be case insensitive in PATTERN and PAT
- --ignoreStyle, -y be style insensitive in PATTERN and PAT
- NOTE: PATERN and patterns PAT (see below in other options) are all either
- Regex or Peg simultaneously and options --rex, --word, --ignoreCase,
- --ignoreStyle are applied to all of them.
-
-* File system walk:
- --recursive, -r process directories recursively
- --follow follow all symlinks when processing recursively
- --ext:EX1|EX2|... only search the files with the given extension(s),
- empty one ("--ext") means files with missing extension
- --noExt:EX1|... exclude files having given extension(s), use empty one to
- skip files with no extension (like some binary files are)
- --includeFile:PAT search only files whose names contain pattern PAT
- --excludeFile:PAT skip files whose names contain pattern PAT
- --includeDir:PAT search only files with whole directory path containing PAT
- --excludeDir:PAT skip directories whose name (not path) contain pattern PAT
- --if,--ef,--id,--ed abbreviations of 4 options above
- --sortTime order files by the last modification time (default: off):
- -s[:asc|desc] ascending (recent files go last) or descending
-
-* Filter file content:
- --match:PAT select files containing a (not displayed) match of PAT
- --noMatch:PAT select files not containing any match of PAT
- --bin:on|off|only process binary files? (detected by \0 in first 1K bytes)
- (default: on - binary and text files treated the same way)
- --text, -t process only text files, the same as --bin:off
-
-* Represent results:
- --nocolor output will be given without any colors
- --color[:on] force color even if output is redirected (default: auto)
- --colorTheme:THEME select color THEME from 'simple' (default),
- 'bnw' (black and white) ,'ack', or 'gnu' (GNU grep)
- --count only print counts of matches for files that matched
- --context:N, -c:N print N lines of leading context before every match and
- N lines of trailing context after it (default N: 0)
- --afterContext:N,
- -a:N print N lines of trailing context after every match
- --beforeContext:N,
- -b:N print N lines of leading context before every match
- --group, -g group matches by file
- --newLine, -l display every matching line starting from a new line
- --cols[:N] limit max displayed columns/width of output lines from
- files by N characters, cropping overflows (default: off)
- --cols:auto, -% calculate columns from terminal width for every line
- --onlyAscii, -@ display only printable ASCII Latin characters 0x20-0x7E
- substitutions: 0 -> ^@, 1 -> ^A, ... 0x1F -> ^_,
- 0x7F -> '7F, ..., 0xFF -> 'FF
-* Miscellaneous:
- --threads:N, -j:N speed up search by N additional workers (default: 0, off)
- --stdin read PATTERN from stdin (to avoid the shell's confusing
- quoting rules) and, if --replace given, REPLACEMENT
- --verbose be verbose: list every processed file
- --help, -h shows this help
- --version, -v shows the version
-"""
+""" & slurp "../doc/nimgrep_cmdline.txt"
# Limitations / ideas / TODO:
# * No unicode support with --cols