stdlib final cleanup

* 'lock levels' are deprecated, now a noop

* fixes tests

.github/workflows/ci_packages.yml

@@ -4,7 +4,6 @@ on:
   push:
     branches:
       - 'devel'
-      - 'version-2-0'
       - 'version-1-6'
       - 'version-1-2'
 
@@ -28,12 +27,12 @@ jobs:
       NIM_TESTAMENT_BATCH: ${{ matrix.batch }}
     steps:
       - name: 'Checkout'
-        uses: actions/checkout@v3
+        uses: actions/checkout@v2
         with:
           fetch-depth: 2
 
       - name: 'Install node.js 16.x'
-        uses: actions/setup-node@v3
+        uses: actions/setup-node@v2
         with:
           node-version: '16.x'
 

.github/workflows/ci_publish.yml

@@ -17,12 +17,12 @@ jobs:
     runs-on: ${{ matrix.os }}
     steps:
       - name: 'Checkout'
-        uses: actions/checkout@v3
+        uses: actions/checkout@v2
         with:
           fetch-depth: 2
 
       - name: 'Install node.js 16.x'
-        uses: actions/setup-node@v3
+        uses: actions/setup-node@v2
         with:
           node-version: '16.x'
 

.gitignore

@@ -67,7 +67,6 @@ testament.db
 
 /csources
 /csources_v1
-/csources_v2
 
 /dist/
 # /lib/fusion # fusion is now unbundled; `git status` should reveal if it's there so users can act on it

azure-pipelines.yml

@@ -23,11 +23,11 @@ jobs:
         vmImage: 'ubuntu-20.04'
         CPU: amd64
       # regularly breaks, refs bug #17325
-#       Linux_i386:
-#         # on 'ubuntu-16.04' (not supported anymore anyways) it errored with:
-#         # g++-multilib : Depends: gcc-multilib (>= 4:5.3.1-1ubuntu1) but it is not going to be installed
-#         vmImage: 'ubuntu-18.04'
-#         CPU: i386
+      Linux_i386:
+        # on 'ubuntu-16.04' (not supported anymore anyways) it errored with:
+        # g++-multilib : Depends: gcc-multilib (>= 4:5.3.1-1ubuntu1) but it is not going to be installed
+        vmImage: 'ubuntu-18.04'
+        CPU: i386
       OSX_amd64:
         vmImage: 'macOS-11'
         CPU: amd64

bin/nim-gdb

@@ -14,7 +14,7 @@ else
 fi
 
 # Find out where the pretty printer Python module is
-GDB_PYTHON_MODULE_PATH="$NIM_SYSROOT/tools/debug/nim-gdb.py"
+GDB_PYTHON_MODULE_PATH="$NIM_SYSROOT/tools/nim-gdb.py"
 
 # Run GDB with the additional arguments that load the pretty printers
 # Set the environment variable `NIM_GDB` to overwrite the call to a

bin/nim-gdb.bat

@@ -3,7 +3,7 @@ for %%i in (nim.exe) do (set NIM_BIN=%%~dp$PATH:i)
 
 for %%i in ("%NIM_BIN%\..\") do (set NIM_ROOT=%%~fi)
 
-set @GDB_PYTHON_MODULE_PATH=%NIM_ROOT%\tools\debug\nim-gdb.py
+set @GDB_PYTHON_MODULE_PATH=%NIM_ROOT%\tools\nim-gdb.py
 set @NIM_GDB=gdb.exe
 
 @echo source %@GDB_PYTHON_MODULE_PATH%> wingdbcommand.txt

changelog.md

@@ -1,30 +1,273 @@
-# v2.2.0 - yyyy-mm-dd
+# v2.0.0 - yyyy-mm-dd
 
 
 ## Changes affecting backward compatibility
+- `httpclient.contentLength` default to `-1` if the Content-Length header is not set in the response, it followed Apache HttpClient(Java), http(go) and .Net HttpWebResponse(C#) behavior. Previously it raised `ValueError`.
 
+- `addr` is now available for all addressable locations,
+  `unsafeAddr` is now deprecated and an alias for `addr`.
+
+- Certain definitions from the default `system` module have been moved to
+  the following new modules:
+
+  - `std/syncio`
+  - `std/assertions`
+  - `std/formatfloat`
+  - `std/objectdollar`
+  - `std/widestrs`
+  - `std/typedthreads`
+  - `std/sysatomics`
+
+  In the future, these definitions will be removed from the `system` module,
+  and their respective modules will have to be imported to use them.
+  Currently, to make these imports required, the `-d:nimPreviewSlimSystem` option
+  may be used.
+
+- Enabling `-d:nimPreviewSlimSystem` also removes the following deprecated
+  symbols in the `system` module:
+  - Aliases with `Error` suffix to exception types that have a `Defect` suffix
+    (see [exceptions](https://nim-lang.org/docs/exceptions.html)):
+    `ArithmeticError`, `DivByZeroError`, `OverflowError`,
+    `AccessViolationError`, `AssertionError`, `OutOfMemError`, `IndexError`,
+    `FieldError`, `RangeError`, `StackOverflowError`, `ReraiseError`,
+    `ObjectAssignmentError`, `ObjectConversionError`, `FloatingPointError`,
+    `FloatOverflowError`, `FloatUnderflowError`, `FloatInexactError`,
+    `DeadThreadError`, `NilAccessError`
+  - `addQuitProc`, replaced by `exitprocs.addExitProc`
+  - Legacy unsigned conversion operations: `ze`, `ze64`, `toU8`, `toU16`, `toU32`
+  - `TaintedString`, formerly a distinct alias to `string`
+  - `PInt32`, `PInt64`, `PFloat32`, `PFloat64`, aliases to
+    `ptr int32`, `ptr int64`, `ptr float32`, `ptr float64`
+
+- Enabling `-d:nimPreviewSlimSystem` removes the import of `channels_builtin` in
+  in the `system` module.
+
+- Enabling `-d:nimPreviewCstringConversion`, `ptr char`, `ptr array[N, char]` and `ptr UncheckedArray[N, char]` don't support conversion to cstring anymore.
+
+- The `gc:v2` option is removed.
+
+- The `mainmodule` and `m` options are removed.
+
+- The `threads:on` option is now the default.
+
+- Optional parameters in combination with `: body` syntax (RFC #405) are now opt-in via
+  `experimental:flexibleOptionalParams`.
+
+- Automatic dereferencing (experimental feature) is removed.
+
+- The `Math.trunc` polyfill for targeting Internet Explorer was
+  previously included in most JavaScript output files.
+  Now, it is only included with `-d:nimJsMathTruncPolyfill`.
+  If you are targeting Internet Explorer, you may choose to enable this option
+  or define your own `Math.trunc` polyfill using the [`emit` pragma](https://nim-lang.org/docs/manual.html#implementation-specific-pragmas-emit-pragma).
+  Nim uses `Math.trunc` for the division and modulo operators for integers.
+
+- `shallowCopy` and `shallow` are removed for ARC/ORC. Use `move` when possible or combine assignment and
+`sink` for optimization purposes.
+
+- The `nimPreviewDotLikeOps` define is going to be removed or deprecated.
+
+- The `{.this.}` pragma, deprecated since 0.19, has been removed.
+- `nil` literals can no longer be directly assigned to variables or fields of `distinct` pointer types. They must be converted instead.
+  ```nim
+  type Foo = distinct ptr int
+
+  # Before:
+  var x: Foo = nil
+  # After:
+  var x: Foo = Foo(nil)
+  ```
+- Removed two type pragma syntaxes deprecated since 0.20, namely
+  `type Foo = object {.final.}`, and `type Foo {.final.} [T] = object`.
+
+- [Overloadable enums](https://nim-lang.github.io/Nim/manual.html#overloadable-enum-value-names) and Unicode Operators
+  are no longer experimental.
+
+- Removed the `nimIncrSeqV3` define.
+
+- `macros.getImpl` for `const` symbols now returns the full definition node
+  (as `nnkConstDef`) rather than the AST of the constant value.
+
+- Lock levels are deprecated, now a noop.
+
+- ORC is now the default memory management strategy. Use
+  `--mm:refc` for a transition period.
+
+- `strictEffects` are no longer experimental.
+  Use `legacy:laxEffects` to keep backward compatibility.
+
+- The `gorge`/`staticExec` calls will now return a descriptive message in the output
+  if the execution fails for whatever reason. To get back legacy behaviour use `-d:nimLegacyGorgeErrors`.
+
+- Pointer to `cstring` conversion now triggers a `[PtrToCstringConv]` warning.
+  This warning will become an error in future versions! Use a `cast` operation
+  like `cast[cstring](x)` instead.
+
+- `logging` will default to flushing all log level messages. To get the legacy behaviour of only flushing Error and Fatal messages, use `-d:nimV1LogFlushBehavior`.
+
+- Object fields now support default values, see https://nim-lang.github.io/Nim/manual.html#types-default-values-for-object-fields for details.
+
+- Using an unnamed break in a block is deprecated. This warning will become an error in future versions! Use a named block with a named break instead.
 
 ## Standard library additions and changes
 
 [//]: # "Changes:"
+- OpenSSL 3 is now supported.
+- `macros.parseExpr` and `macros.parseStmt` now accept an optional
+  filename argument for more informative errors.
+- Module `colors` expanded with missing colors from the CSS color standard.
+  `colPaleVioletRed` and `colMediumPurple` have also been changed to match the CSS color standard.
+- Fixed `lists.SinglyLinkedList` being broken after removing the last node ([#19353](https://github.com/nim-lang/Nim/pull/19353)).
+- The `md5` module now works at compile time and in JavaScript.
+- Changed `mimedb` to use an `OrderedTable` instead of `OrderedTableRef` to support `const` tables.
+- `strutils.find` now uses and defaults to `last = -1` for whole string searches,
+  making limiting it to just the first char (`last = 0`) valid.
+- `random.rand` now works with `Ordinal`s.
+- Undeprecated `os.isvalidfilename`.
+- `std/oids` now uses `int64` to store time internally (before it was int32).
+- `std/uri.Uri` dollar `$` improved, precalculates the `string` result length from the `Uri`.
+- `std/uri.Uri.isIpv6` is now exported.
+- `std/logging.ConsoleLogger` and `FileLogger` now have a `flushThreshold` attribute to set what log message levels are automatically flushed. For Nim v1 use `-d:nimFlushAllLogs` to automatically flush all message levels. Flushing all logs is the default behavior for Nim v2.
+
+
+- `std/net.IpAddress` dollar `$` improved, uses a fixed capacity for the `string` result based from the `IpAddressFamily`.
+- `std/jsfetch.newFetchOptions` now has default values for all parameters
+- `std/jsformdata` now accepts `Blob` data type.
 
 
 [//]: # "Additions:"
+- Added ISO 8601 week date utilities in `times`:
+  - Added `IsoWeekRange`, a range type for weeks in a week-based year.
+  - Added `IsoYear`, a distinct type for a week-based year in contrast to a regular year.
+  - Added a `initDateTime` overload to create a datetime from an ISO week date.
+  - Added `getIsoWeekAndYear` to get an ISO week number and week-based year from a datetime.
+  - Added `getIsoWeeksInYear` to return the number of weeks in a week-based year.
+- Added new modules which were part of `std/os`:
+  - Added `std/oserrors` for OS error reporting. Added `std/envvars` for environment variables handling.
+  - Added `std/paths`, `std/dirs`, `std/files`, `std/symlinks` and `std/appdirs`.
+- Added `sep` parameter in `std/uri` to specify the query separator.
+- Added bindings to [`Array.shift`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/shift)
+  and [`queueMicrotask`](https://developer.mozilla.org/en-US/docs/Web/API/queueMicrotask)
+  in `jscore` for JavaScript targets.
+- Added `UppercaseLetters`, `LowercaseLetters`, `PunctuationChars`, `PrintableChars` sets to `std/strutils`.
+- Added `complex.sgn` for obtaining the phase of complex numbers.
+- Added `insertAdjacentText`, `insertAdjacentElement`, `insertAdjacentHTML`,
+  `after`, `before`, `closest`, `append`, `hasAttributeNS`, `removeAttributeNS`,
+  `hasPointerCapture`, `releasePointerCapture`, `requestPointerLock`,
+  `replaceChildren`, `replaceWith`, `scrollIntoViewIfNeeded`, `setHTML`,
+  `toggleAttribute`, and `matches` to `std/dom`.
+- Added [`jsre.hasIndices`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/hasIndices)
+- Added `capacity` for `string` and `seq` to return the current capacity, see https://github.com/nim-lang/RFCs/issues/460
+- Added `openArray[char]` overloads for `std/parseutils` allowing more code reuse.
+- Added `openArray[char]` overloads for `std/unicode` allowing more code reuse.
+- Added `safe` parameter to `base64.encodeMime`.
 
 [//]: # "Deprecations:"
-
+- Deprecated `selfExe` for Nimscript.
+- Deprecated `std/sums`.
+- Deprecated `std/base64.encode` for collections of arbitrary integer element type.
+  Now only `byte` and `char` are supported.
 
 [//]: # "Removals:"
-
+- Removed deprecated module `parseopt2`.
+- Removed deprecated module `sharedstrings`.
+- Removed deprecated module `dom_extensions`.
+- Removed deprecated module `LockFreeHash`.
+- Removed deprecated module `events`.
+- Removed deprecated `oids.oidToString`.
+- Removed define `nimExperimentalAsyncjsThen` for `std/asyncjs.then` and `std/jsfetch`.
+- Removed deprecated `jsre.test` and `jsre.toString`.
+- Removed deprecated `math.c_frexp`.
+- Removed deprecated `` httpcore.`==` ``.
+- Removed deprecated `std/posix.CMSG_SPACE` and `std/posix.CMSG_LEN` that takes wrong argument types.
+- Removed deprecated `osproc.poDemon`, symbol with typo.
 
 ## Language changes
 
+- [Tag tracking](https://nim-lang.github.io/Nim/manual.html#effect-system-tag-tracking) supports the definition of forbidden tags by the `.forbids` pragma
+  which can be used to disable certain effects in proc types.
+- [Case statement macros](https://nim-lang.github.io/Nim/manual.html#macros-case-statement-macros) are no longer experimental,
+  meaning you no longer need to enable the experimental switch `caseStmtMacros` to use them.
+- Full command syntax and block arguments i.e. `foo a, b: c` are now allowed
+  for the right-hand side of type definitions in type sections. Previously
+  they would error with "invalid indentation".
+- `defined` now accepts identifiers separated by dots, i.e. `defined(a.b.c)`.
+  In the command line, this is defined as `-d:a.b.c`. Older versions can
+  use accents as in ``defined(`a.b.c`)`` to access such defines.
+- [Macro pragmas](https://nim-lang.github.io/Nim/manual.html#userminusdefined-pragmas-macro-pragmas) changes:
+  - Templates now accept macro pragmas.
+  - Macro pragmas for var/let/const sections have been redesigned in a way that works
+    similarly to routine macro pragmas. The new behavior is documented in the
+    [experimental manual](https://nim-lang.github.io/Nim/manual_experimental.html#extended-macro-pragmas).
+  - Pragma macros on type definitions can now return `nnkTypeSection` nodes as well as `nnkTypeDef`,
+    allowing multiple type definitions to be injected in place of the original type definition.
+
+    ```nim
+    import macros
+    macro multiply(amount: static int, s: untyped): untyped =
+      let name = $s[0].basename
+      result = newNimNode(nnkTypeSection)
+      for i in 1 .. amount:
+        result.add(newTree(nnkTypeDef, ident(name & $i), s[1], s[2]))
+    type
+      Foo = object
+      Bar {.multiply: 3.} = object
+        x, y, z: int
+      Baz = object
+    # becomes
+    type
+      Foo = object
+      Bar1 = object
+        x, y, z: int
+      Bar2 = object
+        x, y, z: int
+      Bar3 = object
+        x, y, z: int
+      Baz = object
+    ```
+
+- Redefining templates with the same signature implicitly was previously
+  allowed to support certain macro code. A `{.redefine.}` pragma has been
+  added to make this work explicitly, and a warning is generated in the case
+  where it is implicit. This behavior only applies to templates, redefinition
+  is generally disallowed for other symbols.
+
+- A new form of type inference called [top-down inference](https://nim-lang.github.io/Nim/manual_experimental.html#topminusdown-type-inference)
+  has been implemented for a variety of basic cases. For example, code like the following now compiles:
+
+  ```nim
+  let foo: seq[(float, byte, cstring)] = @[(1, 2, "abc")]
+  ```
+
+- `cstring` is now accepted as a selector in `case` statements, removing the
+  need to convert to `string`. On the JS backend, this is translated directly
+  to a `switch` statement.
+
+- Nim now supports `out` parameters and ["strict definitions"](https://nim-lang.github.io/Nim/manual_experimental.html#strict-definitions-and-nimout-parameters).
+- Nim now offers a [strict mode](https://nim-lang.github.io/Nim/manual_experimental.html#strict-case-objects) for `case objects`.
 
 
 ## Compiler changes
 
+- The `gc` switch has been renamed to `mm` ("memory management") in order to reflect the
+  reality better. (Nim moved away from all techniques based on "tracing".)
 
+- Defines the `gcRefc` symbol which allows writing specific code for the refc GC.
+
+- `nim` can now compile version 1.4.0 as follows: `nim c --lib:lib --stylecheck:off compiler/nim`,
+  without requiring `-d:nimVersion140` which is now a noop.
+
+- `--styleCheck`, `--hintAsError` and `--warningAsError` now only apply to the current package.
+
+- The switch `--nimMainPrefix:prefix` has been added to add a prefix to the names of `NimMain` and
+  related functions produced on the backend. This prevents conflicts with other Nim
+  static libraries.
+
+- When compiling for Release the flag `-fno-math-errno` is used for GCC.
 
 
 ## Tool changes
 
+- Nim now supports Nimble version 0.14 which added support for lock-files. This is done by
+  a simple configuration change setting that you can do yourself too. In `$nim/config/nim.cfg`
+  replace `pkgs` by `pkgs2`.

changelogs/changelog.md

@@ -0,0 +1,33 @@
+# v1.xx.x - yyyy-mm-dd
+
+## Changes affecting backward compatibility
+
+## Standard library additions and changes
+
+- `std/sharedlist` and `std/sharedtables` are now deprecated, see RFC [#433](https://github.com/nim-lang/RFCs/issues/433).
+
+### New compile flag (`-d:nimNoGetRandom`) when building `std/sysrand` to remove dependency on linux `getrandom` syscall
+
+This compile flag only affects linux builds and is necessary if either compiling on a linux kernel version < 3.17, or if code built will be executing on kernel < 3.17.
+
+On linux kernels < 3.17 (such as kernel 3.10 in RHEL7 and CentOS7), the `getrandom` syscall was not yet introduced. Without this, the `std/sysrand` module will not build properly, and if code is built on a kernel >= 3.17 without the flag, any usage of the `std/sysrand` module will fail to execute on a kernel < 3.17 (since it attempts to perform a syscall to `getrandom`, which isn't present in the current kernel). A compile flag has been added to force the `std/sysrand` module to use /dev/urandom (available since linux kernel 1.3.30), rather than the `getrandom` syscall. This allows for use of a cryptographically secure PRNG, regardless of kernel support for the `getrandom` syscall.
+
+When building for RHEL7/CentOS7 for example, the entire build process for nim from a source package would then be:
+```sh
+$ yum install devtoolset-8 # Install GCC version 8 vs the standard 4.8.5 on RHEL7/CentOS7. Alternatively use -d:nimEmulateOverflowChecks. See issue #13692 for details
+$ scl enable devtoolset-8 bash # Run bash shell with default toolchain of gcc 8
+$ sh build.sh  # per unix install instructions
+$ bin/nim c koch  # per unix install instructions
+$ ./koch boot -d:release  # per unix install instructions
+$ ./koch tools -d:nimNoGetRandom  # pass the nimNoGetRandom flag to compile std/sysrand without support for getrandom syscall
+```
+
+This is necessary to pass when building nim on kernel versions < 3.17 in particular to avoid an error of "SYS_getrandom undeclared" during the build process for stdlib (sysrand in particular).
+
+## Language changes
+
+
+## Compiler changes
+
+
+## Tool changes

changelogs/changelog_2_0_0.md

@@ -1,448 +0,0 @@
-# v2.0.0 - yyyy-mm-dd
-
-
-## Changes affecting backward compatibility
-- `httpclient.contentLength` default to `-1` if the Content-Length header is not set in the response. It follows Apache HttpClient(Java), http(go) and .Net HttpWebResponse(C#) behavior. Previously it raised `ValueError`.
-
-- `addr` is now available for all addressable locations,
-  `unsafeAddr` is now deprecated and an alias for `addr`.
-
-- Certain definitions from the default `system` module have been moved to
-  the following new modules:
-
-  - `std/syncio`
-  - `std/assertions`
-  - `std/formatfloat`
-  - `std/objectdollar`
-  - `std/widestrs`
-  - `std/typedthreads`
-  - `std/sysatomics`
-
-  In the future, these definitions will be removed from the `system` module,
-  and their respective modules will have to be imported to use them.
-  Currently, to make these imports required, the `-d:nimPreviewSlimSystem` option
-  may be used.
-
-- Enabling `-d:nimPreviewSlimSystem` also removes the following deprecated
-  symbols in the `system` module:
-  - Aliases with `Error` suffix to exception types that have a `Defect` suffix
-    (see [exceptions](https://nim-lang.github.io/Nim/exceptions.html)):
-    `ArithmeticError`, `DivByZeroError`, `OverflowError`,
-    `AccessViolationError`, `AssertionError`, `OutOfMemError`, `IndexError`,
-    `FieldError`, `RangeError`, `StackOverflowError`, `ReraiseError`,
-    `ObjectAssignmentError`, `ObjectConversionError`, `FloatingPointError`,
-    `FloatOverflowError`, `FloatUnderflowError`, `FloatInexactError`,
-    `DeadThreadError`, `NilAccessError`
-  - `addQuitProc`, replaced by `exitprocs.addExitProc`
-  - Legacy unsigned conversion operations: `ze`, `ze64`, `toU8`, `toU16`, `toU32`
-  - `TaintedString`, formerly a distinct alias to `string`
-  - `PInt32`, `PInt64`, `PFloat32`, `PFloat64`, aliases to
-    `ptr int32`, `ptr int64`, `ptr float32`, `ptr float64`
-
-- Enabling `-d:nimPreviewSlimSystem` removes the import of `channels_builtin` in
-  in the `system` module.
-
-- Enabling `-d:nimPreviewCstringConversion`, `ptr char`, `ptr array[N, char]` and `ptr UncheckedArray[N, char]` don't support conversion to cstring anymore.
-
-- The `gc:v2` option is removed.
-
-- The `mainmodule` and `m` options are removed.
-
-- The `threads:on` option is now the default.
-
-- Optional parameters in combination with `: body` syntax (RFC #405) are now opt-in via
-  `experimental:flexibleOptionalParams`.
-
-- Automatic dereferencing (experimental feature) is removed.
-
-- The `Math.trunc` polyfill for targeting Internet Explorer was
-  previously included in most JavaScript output files.
-  Now, it is only included with `-d:nimJsMathTruncPolyfill`.
-  If you are targeting Internet Explorer, you may choose to enable this option
-  or define your own `Math.trunc` polyfill using the [`emit` pragma](https://nim-lang.org/docs/manual.html#implementation-specific-pragmas-emit-pragma).
-  Nim uses `Math.trunc` for the division and modulo operators for integers.
-
-- `shallowCopy` and `shallow` are removed for ARC/ORC. Use `move` when possible or combine assignment and
-`sink` for optimization purposes.
-
-- The experimental `nimPreviewDotLikeOps` switch is going to be removed or deprecated because it didn't fullfill its promises.
-
-- The `{.this.}` pragma, deprecated since 0.19, has been removed.
-- `nil` literals can no longer be directly assigned to variables or fields of `distinct` pointer types. They must be converted instead.
-  ```nim
-  type Foo = distinct ptr int
-
-  # Before:
-  var x: Foo = nil
-  # After:
-  var x: Foo = Foo(nil)
-  ```
-- Removed two type pragma syntaxes deprecated since 0.20, namely
-  `type Foo = object {.final.}`, and `type Foo {.final.} [T] = object`.
-
-- `foo a = b` now means `foo(a = b)` rather than `foo(a) = b`. This is consistent
-  with the existing behavior of `foo a, b = c` meaning `foo(a, b = c)`.
-  This decision was made with the assumption that the old syntax was used rarely;
-  if your code used the old syntax, please be aware of this change.
-
-- [Overloadable enums](https://nim-lang.github.io/Nim/manual.html#overloadable-enum-value-names) and Unicode Operators
-  are no longer experimental.
-
-- Removed the `nimIncrSeqV3` define.
-
-- `macros.getImpl` for `const` symbols now returns the full definition node
-  (as `nnkConstDef`) rather than the AST of the constant value.
-
-- Lock levels are deprecated, now a noop.
-
-- ORC is now the default memory management strategy. Use
-  `--mm:refc` for a transition period.
-
-- `strictEffects` are no longer experimental.
-  Use `legacy:laxEffects` to keep backward compatibility.
-
-- The `gorge`/`staticExec` calls will now return a descriptive message in the output
-  if the execution fails for whatever reason. To get back legacy behaviour use `-d:nimLegacyGorgeErrors`.
-
-- Pointer to `cstring` conversion now triggers a `[PtrToCstringConv]` warning.
-  This warning will become an error in future versions! Use a `cast` operation
-  like `cast[cstring](x)` instead.
-
-- `logging` will default to flushing all log level messages. To get the legacy behaviour of only flushing Error and Fatal messages, use `-d:nimV1LogFlushBehavior`.
-
-- Redefining templates with the same signature was previously
-  allowed to support certain macro code. To do this explicitly, the
-  `{.redefine.}` pragma has been added. Note that this is only for templates.
-  Implicit redefinition of templates is now deprecated and will give an error in the future.
-
-- Using an unnamed break in a block is deprecated. This warning will become an error in future versions! Use a named block with a named break instead.
-
-- Several Standard libraries are moved to nimble packages, use `nimble` to install them:
-  - `std/punycode` => `punycode`
-  - `std/asyncftpclient` => `asyncftpclient`
-  - `std/smtp` => `smtp`
-  - `std/db_common` => `db_connector/db_common`
-  - `std/db_sqlite` => `db_connector/db_sqlite`
-  - `std/db_mysql` => `db_connector/db_mysql`
-  - `std/db_postgres` => `db_connector/db_postgres`
-  - `std/db_odbc` => `db_connector/db_odbc`
-
-- Previously, calls like `foo(a, b): ...` or `foo(a, b) do: ...` where the final argument of
-  `foo` had type `proc ()` were assumed by the compiler to mean `foo(a, b, proc () = ...)`.
-  This behavior is now deprecated. Use `foo(a, b) do (): ...` or `foo(a, b, proc () = ...)` instead.
-
-- If no exception or any exception deriving from Exception but not Defect or CatchableError given in except, a `warnBareExcept` warning will be triggered.
-
-- The experimental strictFuncs feature now disallows a store to the heap via a `ref` or `ptr` indirection.
-
-- The underscore identifier (`_`) is now generally not added to scope when
-  used as the name of a definition. While this was already the case for
-  variables, it is now also the case for routine parameters, generic
-  parameters, routine declarations, type declarations, etc. This means that the following code now does not compile:
-
-  ```nim
-  proc foo(_: int): int = _ + 1
-  echo foo(1)
-
-  proc foo[_](t: typedesc[_]): seq[_] = @[default(_)]
-  echo foo[int]()
-
-  proc _() = echo "_"
-  _()
-
-  type _ = int
-  let x: _ = 3
-  ```
-
-  Whereas the following code now compiles:
-
-  ```nim
-  proc foo(_, _: int): int = 123
-  echo foo(1, 2)
-
-  proc foo[_, _](): int = 123
-  echo foo[int, bool]()
-
-  proc foo[T, U](_: typedesc[T], _: typedesc[U]): (T, U) = (default(T), default(U))
-  echo foo(int, bool)
-
-  proc _() = echo "one"
-  proc _() = echo "two"
-
-  type _ = int
-  type _ = float
-  ```
-
-- - Added the `--legacy:verboseTypeMismatch` switch to get legacy type mismatch error messages.
-
-- The JavaScript backend now uses [BigInt](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt)
-  for 64-bit integer types (`int64` and `uint64`) by default. As this affects
-  JS code generation, code using these types to interface with the JS backend
-  may need to be updated. Note that `int` and `uint` are not affected.
-  
-  For compatibility with [platforms that do not support BigInt](https://caniuse.com/bigint)
-  and in the case of potential bugs with the new implementation, the
-  old behavior is currently still supported with the command line option
-  `--jsbigint64:off`.
-
-- The `proc` and `iterator` type classes now respectively only match
-  procs and iterators. Previously both type classes matched any of
-  procs or iterators.
-
-  ```nim
-  proc prc(): int =
-    123
-
-  iterator iter(): int =
-    yield 123
-  
-  proc takesProc[T: proc](x: T) = discard
-  proc takesIter[T: iterator](x: T) = discard
-
-  # always compiled:
-  takesProc(prc)
-  takesIter(iter)
-  # no longer compiles:
-  takesProc(iter)
-  takesIter(prc)
-  ```
-
-- The `proc` and `iterator` type classes now accept a calling convention pragma
-  (i.e. `proc {.closure.}`) that must be shared by matching proc or iterator
-  types. Previously pragmas were parsed but discarded if no parameter list
-  was given.
-
-  This is represented in the AST by an `nnkProcTy`/`nnkIteratorTy` node with
-  an `nnkEmpty` node in the place of the `nnkFormalParams` node, and the pragma
-  node in the same place as in a concrete `proc` or `iterator` type node. This
-  state of the AST may be unexpected to existing code, both due to the
-  replacement of the `nnkFormalParams` node as well as having child nodes
-  unlike other type class AST.
-
-## Standard library additions and changes
-
-[//]: # "Changes:"
-- OpenSSL 3 is now supported.
-- `macros.parseExpr` and `macros.parseStmt` now accept an optional
-  filename argument for more informative errors.
-- Module `colors` expanded with missing colors from the CSS color standard.
-  `colPaleVioletRed` and `colMediumPurple` have also been changed to match the CSS color standard.
-- Fixed `lists.SinglyLinkedList` being broken after removing the last node ([#19353](https://github.com/nim-lang/Nim/pull/19353)).
-- The `md5` module now works at compile time and in JavaScript.
-- Changed `mimedb` to use an `OrderedTable` instead of `OrderedTableRef` to support `const` tables.
-- `strutils.find` now uses and defaults to `last = -1` for whole string searches,
-  making limiting it to just the first char (`last = 0`) valid.
-- `random.rand` now works with `Ordinal`s.
-- Undeprecated `os.isvalidfilename`.
-- `std/oids` now uses `int64` to store time internally (before it was int32).
-- `std/uri.Uri` dollar `$` improved, precalculates the `string` result length from the `Uri`.
-- `std/uri.Uri.isIpv6` is now exported.
-- `std/logging.ConsoleLogger` and `FileLogger` now have a `flushThreshold` attribute to set what log message levels are automatically flushed. For Nim v1 use `-d:nimFlushAllLogs` to automatically flush all message levels. Flushing all logs is the default behavior for Nim v2.
-
-
-- `std/net.IpAddress` dollar `$` improved, uses a fixed capacity for the `string` result based from the `IpAddressFamily`.
-- `std/jsfetch.newFetchOptions` now has default values for all parameters
-- `std/jsformdata` now accepts `Blob` data type.
-
-- `std/sharedlist` and `std/sharedtables` are now deprecated, see RFC [#433](https://github.com/nim-lang/RFCs/issues/433).
-
-- New compile flag (`-d:nimNoGetRandom`) when building `std/sysrand` to remove dependency on linux `getrandom` syscall.
-
-  This compile flag only affects linux builds and is necessary if either compiling on a linux kernel version < 3.17, or if code built will be executing on kernel < 3.17.
-
-  On linux kernels < 3.17 (such as kernel 3.10 in RHEL7 and CentOS7), the `getrandom` syscall was not yet introduced. Without this, the `std/sysrand` module will not build properly, and if code is built on a kernel >= 3.17 without the flag, any usage of the `std/sysrand` module will fail to execute on a kernel < 3.17 (since it attempts to perform a syscall to `getrandom`, which isn't present in the current kernel). A compile flag has been added to force the `std/sysrand` module to use /dev/urandom (available since linux kernel 1.3.30), rather than the `getrandom` syscall. This allows for use of a cryptographically secure PRNG, regardless of kernel support for the `getrandom` syscall.
-
-  When building for RHEL7/CentOS7 for example, the entire build process for nim from a source package would then be:
-  ```sh
-  $ yum install devtoolset-8 # Install GCC version 8 vs the standard 4.8.5 on RHEL7/CentOS7. Alternatively use -d:nimEmulateOverflowChecks. See issue #13692 for details
-  $ scl enable devtoolset-8 bash # Run bash shell with default toolchain of gcc 8
-  $ sh build.sh  # per unix install instructions
-  $ bin/nim c koch  # per unix install instructions
-  $ ./koch boot -d:release  # per unix install instructions
-  $ ./koch tools -d:nimNoGetRandom  # pass the nimNoGetRandom flag to compile std/sysrand without support for getrandom syscall
-  ```
-
-  This is necessary to pass when building nim on kernel versions < 3.17 in particular to avoid an error of "SYS_getrandom undeclared" during the build process for stdlib (sysrand in particular).
-
-[//]: # "Additions:"
-- Added ISO 8601 week date utilities in `times`:
-  - Added `IsoWeekRange`, a range type for weeks in a week-based year.
-  - Added `IsoYear`, a distinct type for a week-based year in contrast to a regular year.
-  - Added a `initDateTime` overload to create a datetime from an ISO week date.
-  - Added `getIsoWeekAndYear` to get an ISO week number and week-based year from a datetime.
-  - Added `getIsoWeeksInYear` to return the number of weeks in a week-based year.
-- Added new modules which were part of `std/os`:
-  - Added `std/oserrors` for OS error reporting. Added `std/envvars` for environment variables handling.
-  - Added `std/paths`, `std/dirs`, `std/files`, `std/symlinks` and `std/appdirs`.
-  - Added `std/cmdline` for reading command line parameters.
-- Added `sep` parameter in `std/uri` to specify the query separator.
-- Added bindings to [`Array.shift`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/shift)
-  and [`queueMicrotask`](https://developer.mozilla.org/en-US/docs/Web/API/queueMicrotask)
-  in `jscore` for JavaScript targets.
-- Added `UppercaseLetters`, `LowercaseLetters`, `PunctuationChars`, `PrintableChars` sets to `std/strutils`.
-- Added `complex.sgn` for obtaining the phase of complex numbers.
-- Added `insertAdjacentText`, `insertAdjacentElement`, `insertAdjacentHTML`,
-  `after`, `before`, `closest`, `append`, `hasAttributeNS`, `removeAttributeNS`,
-  `hasPointerCapture`, `releasePointerCapture`, `requestPointerLock`,
-  `replaceChildren`, `replaceWith`, `scrollIntoViewIfNeeded`, `setHTML`,
-  `toggleAttribute`, and `matches` to `std/dom`.
-- Added [`jsre.hasIndices`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/hasIndices)
-- Added `capacity` for `string` and `seq` to return the current capacity, see https://github.com/nim-lang/RFCs/issues/460
-- Added `openArray[char]` overloads for `std/parseutils` allowing more code reuse.
-- Added `openArray[char]` overloads for `std/unicode` allowing more code reuse.
-- Added `safe` parameter to `base64.encodeMime`.
-- Added `parseutils.parseSize` - inverse to `strutils.formatSize` - to parse human readable sizes.
-
-[//]: # "Deprecations:"
-- Deprecated `selfExe` for Nimscript.
-- Deprecated `std/sums`.
-- Deprecated `std/base64.encode` for collections of arbitrary integer element type.
-  Now only `byte` and `char` are supported.
-
-[//]: # "Removals:"
-- Removed deprecated module `parseopt2`.
-- Removed deprecated module `sharedstrings`.
-- Removed deprecated module `dom_extensions`.
-- Removed deprecated module `LockFreeHash`.
-- Removed deprecated module `events`.
-- Removed deprecated `oids.oidToString`.
-- Removed define `nimExperimentalAsyncjsThen` for `std/asyncjs.then` and `std/jsfetch`.
-- Removed deprecated `jsre.test` and `jsre.toString`.
-- Removed deprecated `math.c_frexp`.
-- Removed deprecated `` httpcore.`==` ``.
-- Removed deprecated `std/posix.CMSG_SPACE` and `std/posix.CMSG_LEN` that takes wrong argument types.
-- Removed deprecated `osproc.poDemon`, symbol with typo.
-- Removed deprecated `tables.rightSize`.
-
-
-- Removed deprecated `posix.CLONE_STOPPED`.
-
-
-## Language changes
-
-- [Tag tracking](https://nim-lang.github.io/Nim/manual.html#effect-system-tag-tracking) supports the definition of forbidden tags by the `.forbids` pragma
-  which can be used to disable certain effects in proc types.
-- [Case statement macros](https://nim-lang.github.io/Nim/manual.html#macros-case-statement-macros) are no longer experimental,
-  meaning you no longer need to enable the experimental switch `caseStmtMacros` to use them.
-- Full command syntax and block arguments i.e. `foo a, b: c` are now allowed
-  for the right-hand side of type definitions in type sections. Previously
-  they would error with "invalid indentation".
-
-- Compile-time define changes:
-  - `defined` now accepts identifiers separated by dots, i.e. `defined(a.b.c)`.
-    In the command line, this is defined as `-d:a.b.c`. Older versions can
-    use accents as in ``defined(`a.b.c`)`` to access such defines.
-  - [Define pragmas for constants](https://nim-lang.github.io/Nim/manual.html#implementation-specific-pragmas-compileminustime-define-pragmas)
-    now support a string argument for qualified define names.
-
-    ```nim
-    # -d:package.FooBar=42
-    const FooBar {.intdefine: "package.FooBar".}: int = 5
-    echo FooBar # 42
-    ```
-
-    This was added to help disambiguate similar define names for different packages.
-    In older versions, this could only be achieved with something like the following:
-
-    ```nim
-    const FooBar = block:
-      const `package.FooBar` {.intdefine.}: int = 5
-      `package.FooBar`
-    ```
-  - A generic `define` pragma for constants has been added that interprets
-    the value of the define based on the type of the constant value.
-    See the [experimental manual](https://nim-lang.github.io/Nim/manual_experimental.html#generic-define-pragma)
-    for a list of supported types.
-
-- [Macro pragmas](https://nim-lang.github.io/Nim/manual.html#userminusdefined-pragmas-macro-pragmas) changes:
-  - Templates now accept macro pragmas.
-  - Macro pragmas for var/let/const sections have been redesigned in a way that works
-    similarly to routine macro pragmas. The new behavior is documented in the
-    [experimental manual](https://nim-lang.github.io/Nim/manual_experimental.html#extended-macro-pragmas).
-  - Pragma macros on type definitions can now return `nnkTypeSection` nodes as well as `nnkTypeDef`,
-    allowing multiple type definitions to be injected in place of the original type definition.
-
-    ```nim
-    import macros
-    macro multiply(amount: static int, s: untyped): untyped =
-      let name = $s[0].basename
-      result = newNimNode(nnkTypeSection)
-      for i in 1 .. amount:
-        result.add(newTree(nnkTypeDef, ident(name & $i), s[1], s[2]))
-    type
-      Foo = object
-      Bar {.multiply: 3.} = object
-        x, y, z: int
-      Baz = object
-    # becomes
-    type
-      Foo = object
-      Bar1 = object
-        x, y, z: int
-      Bar2 = object
-        x, y, z: int
-      Bar3 = object
-        x, y, z: int
-      Baz = object
-    ```
-
-- A new form of type inference called [top-down inference](https://nim-lang.github.io/Nim/manual_experimental.html#topminusdown-type-inference)
-  has been implemented for a variety of basic cases. For example, code like the following now compiles:
-
-  ```nim
-  let foo: seq[(float, byte, cstring)] = @[(1, 2, "abc")]
-  ```
-
-- `cstring` is now accepted as a selector in `case` statements, removing the
-  need to convert to `string`. On the JS backend, this is translated directly
-  to a `switch` statement.
-
-- Nim now supports `out` parameters and ["strict definitions"](https://nim-lang.github.io/Nim/manual_experimental.html#strict-definitions-and-nimout-parameters).
-- Nim now offers a [strict mode](https://nim-lang.github.io/Nim/manual_experimental.html#strict-case-objects) for `case objects`.
-
-- IBM Z architecture and macOS m1 arm64 architecture are supported.
-
-- `=wasMoved` can be overridden by users.
-
-- Tuple unpacking for variables is now treated as syntax sugar that directly
-  expands into multiple assignments. Along with this, tuple unpacking for
-  variables can now be nested.
-
-  ```nim
-  proc returnsNestedTuple(): (int, (int, int), int, int) = (4, (5, 7), 2, 3)
-
-  let (x, (_, y), _, z) = returnsNestedTuple()
-  # roughly becomes
-  let
-    tmpTup1 = returnsNestedTuple()
-    x = tmpTup1[0]
-    tmpTup2 = tmpTup1[1]
-    y = tmpTup2[1]
-    z = tmpTup1[3]
-  ```
-
-  As a result `nnkVarTuple` nodes in variable sections will no longer be
-  reflected in `typed` AST.
-
-## Compiler changes
-
-- The `gc` switch has been renamed to `mm` ("memory management") in order to reflect the
-  reality better. (Nim moved away from all techniques based on "tracing".)
-
-- Defines the `gcRefc` symbol which allows writing specific code for the refc GC.
-
-- `nim` can now compile version 1.4.0 as follows: `nim c --lib:lib --stylecheck:off compiler/nim`,
-  without requiring `-d:nimVersion140` which is now a noop.
-
-- `--styleCheck`, `--hintAsError` and `--warningAsError` now only apply to the current package.
-
-- The switch `--nimMainPrefix:prefix` has been added to add a prefix to the names of `NimMain` and
-  related functions produced on the backend. This prevents conflicts with other Nim
-  static libraries.
-
-- When compiling for Release the flag `-fno-math-errno` is used for GCC.
-
-
-## Tool changes
-
-- Nim now ships Nimble version 0.14 which added support for lock-files. Libraries are stored in `$nimbleDir/pkgs2` (it was `$nimbleDir/pkgs`). Use `nimble develop --global` to create an old style link file in the special links directory documented at https://github.com/nim-lang/nimble#nimble-develop.

compiler/ast.nim

@@ -510,7 +510,7 @@ type
     nfLastRead  # this node is a last read
     nfFirstWrite # this node is a first write
     nfHasComment # node has a comment
-    nfSkipFieldChecking # node skips field visable checking
+    nfUseDefaultField # node has a default value (object constructor)
 
   TNodeFlags* = set[TNodeFlag]
   TTypeFlag* = enum   # keep below 32 for efficiency reasons (now: 46)
@@ -706,13 +706,13 @@ type
     mNctPut, mNctLen, mNctGet, mNctHasNext, mNctNext,
 
     mNIntVal, mNFloatVal, mNSymbol, mNIdent, mNGetType, mNStrVal, mNSetIntVal,
-    mNSetFloatVal, mNSetSymbol, mNSetIdent, mNSetStrVal, mNLineInfo,
+    mNSetFloatVal, mNSetSymbol, mNSetIdent, mNSetType, mNSetStrVal, mNLineInfo,
     mNNewNimNode, mNCopyNimNode, mNCopyNimTree, mStrToIdent, mNSigHash, mNSizeOf,
     mNBindSym, mNCallSite,
     mEqIdent, mEqNimrodNode, mSameNodeType, mGetImpl, mNGenSym,
     mNHint, mNWarning, mNError,
     mInstantiationInfo, mGetTypeInfo, mGetTypeInfoV2,
-    mNimvm, mIntDefine, mStrDefine, mBoolDefine, mGenericDefine, mRunnableExamples,
+    mNimvm, mIntDefine, mStrDefine, mBoolDefine, mRunnableExamples,
     mException, mBuiltinType, mSymOwner, mUncheckedArray, mGetImplTransf,
     mSymIsInstantiationOf, mNodeId, mPrivateAccess, mZeroDefault
 
@@ -796,8 +796,6 @@ type
       ident*: PIdent
     else:
       sons*: TNodeSeq
-    when defined(nimsuggest):
-      endInfo*: TLineInfo
 
   TStrTable* = object         # a table[PIdent] of PSym
     counter*: int
@@ -894,8 +892,6 @@ type
     typ*: PType
     name*: PIdent
     info*: TLineInfo
-    when defined(nimsuggest):
-      endInfo*: TLineInfo
     owner*: PSym
     flags*: TSymFlags
     ast*: PNode               # syntax tree of proc, iterator, etc.:
@@ -935,7 +931,6 @@ type
   TTypeSeq* = seq[PType]
 
   TTypeAttachedOp* = enum ## as usual, order is important here
-    attachedWasMoved,
     attachedDestructor,
     attachedAsgn,
     attachedSink,
@@ -1081,7 +1076,7 @@ const
                                       nfIsRef, nfIsPtr, nfPreventCg, nfLL,
                                       nfFromTemplate, nfDefaultRefsParam,
                                       nfExecuteOnReload, nfLastRead,
-                                      nfFirstWrite, nfSkipFieldChecking}
+                                      nfFirstWrite, nfUseDefaultField}
   namePos* = 0
   patternPos* = 1    # empty except for term rewriting macros
   genericParamsPos* = 2
@@ -1503,7 +1498,7 @@ proc newProcNode*(kind: TNodeKind, info: TLineInfo, body: PNode,
 
 const
   AttachedOpToStr*: array[TTypeAttachedOp, string] = [
-    "=wasMoved", "=destroy", "=copy", "=sink", "=trace", "=deepcopy"]
+    "=destroy", "=copy", "=sink", "=trace", "=deepcopy"]
 
 proc `$`*(s: PSym): string =
   if s != nil:
@@ -1695,8 +1690,6 @@ proc copyNode*(src: PNode): PNode =
   of nkIdent: result.ident = src.ident
   of nkStrLit..nkTripleStrLit: result.strVal = src.strVal
   else: discard
-  when defined(nimsuggest):
-    result.endInfo = src.endInfo
 
 template transitionNodeKindCommon(k: TNodeKind) =
   let obj {.inject.} = n[]
@@ -1749,8 +1742,6 @@ template copyNodeImpl(dst, src, processSonsStmt) =
   if src == nil: return
   dst = newNode(src.kind)
   dst.info = src.info
-  when defined(nimsuggest):
-    result.endInfo = src.endInfo
   dst.typ = src.typ
   dst.flags = src.flags * PersistentNodeFlags
   dst.comment = src.comment
@@ -2056,9 +2047,6 @@ template detailedInfo*(sym: PSym): string =
 proc isInlineIterator*(typ: PType): bool {.inline.} =
   typ.kind == tyProc and tfIterator in typ.flags and typ.callConv != ccClosure
 
-proc isIterator*(typ: PType): bool {.inline.} =
-  typ.kind == tyProc and tfIterator in typ.flags
-
 proc isClosureIterator*(typ: PType): bool {.inline.} =
   typ.kind == tyProc and tfIterator in typ.flags and typ.callConv == ccClosure
 

compiler/astalgo.nim

@@ -20,6 +20,9 @@ import strutils except addf
 when defined(nimPreviewSlimSystem):
   import std/assertions
 
+when not defined(nimHasCursor):
+  {.pragma: cursor.}
+
 proc hashNode*(p: RootRef): Hash
 proc treeToYaml*(conf: ConfigRef; n: PNode, indent: int = 0, maxRecDepth: int = - 1): Rope
   # Convert a tree into its YAML representation; this is used by the
@@ -330,7 +333,7 @@ proc typeToYamlAux(conf: ConfigRef; n: PType, marker: var IntSet, indent: int,
     sonsRope = rope("null")
   elif containsOrIncl(marker, n.id):
     sonsRope = "\"$1 @$2\"" % [rope($n.kind), rope(
-        strutils.toHex(cast[int](n), sizeof(n) * 2))]
+        strutils.toHex(cast[ByteAddress](n), sizeof(n) * 2))]
   else:
     if n.len > 0:
       sonsRope = rope("[")

compiler/backendpragmas.nim

@@ -1,25 +0,0 @@
-import pragmas, options, ast, trees
-
-proc pushBackendOption(optionsStack: var seq[TOptions], options: var TOptions) =
-  optionsStack.add options
-
-proc popBackendOption(optionsStack: var seq[TOptions], options: var TOptions) =
-  options = optionsStack[^1]
-  optionsStack.setLen(optionsStack.len-1)
-
-proc processPushBackendOption*(optionsStack: var seq[TOptions], options: var TOptions,
-                           n: PNode, start: int) =
-  pushBackendOption(optionsStack, options)
-  for i in start..<n.len:
-    let it = n[i]
-    if it.kind in nkPragmaCallKinds and it.len == 2 and it[1].kind == nkIntLit:
-      let sw = whichPragma(it[0])
-      let opts = pragmaToOptions(sw)
-      if opts != {}:
-        if it[1].intVal != 0:
-          options.incl opts
-        else:
-          options.excl opts
-
-template processPopBackendOption*(optionsStack: var seq[TOptions], options: var TOptions) =
-  popBackendOption(optionsStack, options)

compiler/ccgcalls.nim

@@ -156,7 +156,7 @@ proc reifiedOpenArray(n: PNode): bool {.inline.} =
   else:
     result = true
 
-proc genOpenArraySlice(p: BProc; q: PNode; formalType, destType: PType; prepareForMutation = false): (Rope, Rope) =
+proc genOpenArraySlice(p: BProc; q: PNode; formalType, destType: PType): (Rope, Rope) =
   var a, b, c: TLoc
   initLocExpr(p, q[1], a)
   initLocExpr(p, q[2], b)
@@ -164,8 +164,6 @@ proc genOpenArraySlice(p: BProc; q: PNode; formalType, destType: PType; prepareF
   # but first produce the required index checks:
   if optBoundsCheck in p.options:
     genBoundsCheck(p, a, b, c)
-  if prepareForMutation:
-    linefmt(p, cpsStmts, "#nimPrepareStrMutationV2($1);$n", [byRefLoc(p, a)])
   let ty = skipTypes(a.t, abstractVar+{tyPtr})
   let dest = getTypeDesc(p.module, destType)
   let lengthExpr = "($1)-($2)+1" % [rdLoc(c), rdLoc(b)]
@@ -475,7 +473,6 @@ proc genClosureCall(p: BProc, le, ri: PNode, d: var TLoc) =
           discard "resetLoc(p, d)"
         pl.add(addrLoc(p.config, d))
         genCallPattern()
-        if canRaise: raiseExit(p)
       else:
         var tmp: TLoc
         getTemp(p, typ[0], tmp, needsInit=true)

compiler/ccgexprs.nim

@@ -1647,7 +1647,6 @@ proc genArrToSeq(p: BProc, n: PNode, d: var TLoc) =
     return
   if d.k == locNone:
     getTemp(p, n.typ, d)
-  initLocExpr(p, n[1], a)
   # generate call to newSeq before adding the elements per hand:
   let L = toInt(lengthOrd(p.config, n[1].typ))
   if optSeqDestructors in p.config.globalOptions:
@@ -1659,6 +1658,7 @@ proc genArrToSeq(p: BProc, n: PNode, d: var TLoc) =
     var lit = newRopeAppender()
     intLiteral(L, lit)
     genNewSeqAux(p, d, lit, L == 0)
+  initLocExpr(p, n[1], a)
   # bug #5007; do not produce excessive C source code:
   if L < 10:
     for i in 0..<L:
@@ -1704,7 +1704,7 @@ proc genNewFinalize(p: BProc, e: PNode) =
 
 proc genOfHelper(p: BProc; dest: PType; a: Rope; info: TLineInfo; result: var Rope) =
   if optTinyRtti in p.config.globalOptions:
-    let token = $genDisplayElem(MD5Digest(hashType(dest, p.config)))
+    let token = $genDisplayElem(MD5Digest(hashType(dest)))
     appcg(p.module, result, "#isObjDisplayCheck($#.m_type, $#, $#)", [a, getObjDepth(dest), token])
   else:
     # unfortunately 'genTypeInfoV1' sets tfObjHasKids as a side effect, so we
@@ -1879,13 +1879,9 @@ proc genArrayLen(p: BProc, e: PNode, d: var TLoc, op: TMagic) =
     # Bug #9279, len(toOpenArray()) has to work:
     if a.kind in nkCallKinds and a[0].kind == nkSym and a[0].sym.magic == mSlice:
       # magic: pass slice to openArray:
-      var m: TLoc
       var b, c: TLoc
-      initLocExpr(p, a[1], m)
       initLocExpr(p, a[2], b)
       initLocExpr(p, a[3], c)
-      if optBoundsCheck in p.options:
-        genBoundsCheck(p, m, b, c)
       if op == mHigh:
         putIntoDest(p, d, e, ropecg(p.module, "($2)-($1)", [rdLoc(b), rdLoc(c)]))
       else:
@@ -1947,7 +1943,7 @@ proc genSetLengthSeq(p: BProc, e: PNode, d: var TLoc) =
 
   initLoc(call, locCall, e, OnHeap)
   if not p.module.compileToCpp:
-    const setLenPattern = "($3) #setLengthSeqV2(($1)?&($1)->Sup:NIM_NIL, $4, $2)"
+    const setLenPattern = "($3) #setLengthSeqV2(&($1)->Sup, $4, $2)"
     call.r = ropecg(p.module, setLenPattern, [
       rdLoc(a), rdLoc(b), getTypeDesc(p.module, t),
       genTypeInfoV1(p.module, t.skipTypes(abstractInst), e.info)])
@@ -2126,7 +2122,7 @@ proc genSetOp(p: BProc, e: PNode, d: var TLoc, op: TMagic) =
     of mCard:
       var a: TLoc
       initLocExpr(p, e[1], a)
-      putIntoDest(p, d, e, ropecg(p.module, "#cardSet($1, $2)", [addrLoc(p.config, a), size]))
+      putIntoDest(p, d, e, ropecg(p.module, "#cardSet($1, $2)", [rdCharLoc(a), size]))
     of mLtSet, mLeSet:
       getTemp(p, getSysType(p.module.g.graph, unknownLineInfo, tyInt), i) # our counter
       initLocExpr(p, e[1], a)
@@ -2339,7 +2335,7 @@ proc genWasMoved(p: BProc; n: PNode) =
   if p.withinBlockLeaveActions > 0 and notYetAlive(n1):
     discard
   else:
-    initLocExpr(p, n1, a, {lfEnforceDeref})
+    initLocExpr(p, n1, a)
     resetLoc(p, a)
     #linefmt(p, cpsStmts, "#nimZeroMem((void*)$1, sizeof($2));$n",
     #  [addrLoc(p.config, a), getTypeDesc(p.module, a.t)])
@@ -2408,10 +2404,7 @@ proc genDispose(p: BProc; n: PNode) =
       lineCg(p, cpsStmts, ["#nimDestroyAndDispose($#)", rdLoc(a)])
 
 proc genSlice(p: BProc; e: PNode; d: var TLoc) =
-  let (x, y) = genOpenArraySlice(p, e, e.typ, e.typ.lastSon,
-    prepareForMutation = e[1].kind == nkHiddenDeref and
-                         e[1].typ.skipTypes(abstractInst).kind == tyString and
-                         p.config.selectedGC in {gcArc, gcOrc})
+  let (x, y) = genOpenArraySlice(p, e, e.typ, e.typ.lastSon)
   if d.k == locNone: getTemp(p, e.typ, d)
   linefmt(p, cpsStmts, "$1.Field0 = $2; $1.Field1 = $3;$n", [rdLoc(d), x, y])
   when false:
@@ -2779,7 +2772,7 @@ proc upConv(p: BProc, n: PNode, d: var TLoc) =
     rdMType(p, a, nilCheck, r)
     if optTinyRtti in p.config.globalOptions:
       let checkFor = $getObjDepth(dest)
-      let token = $genDisplayElem(MD5Digest(hashType(dest, p.config)))
+      let token = $genDisplayElem(MD5Digest(hashType(dest)))
       if nilCheck != "":
         linefmt(p, cpsStmts, "if ($1 && !#isObjDisplayCheck($2, $3, $4)){ #raiseObjectConversionError(); ",
                 [nilCheck, r, checkFor, token])
@@ -3203,8 +3196,6 @@ proc getDefaultValue(p: BProc; typ: PType; info: TLineInfo; result: var Rope) =
     result.add "}"
   of tyTuple:
     result.add "{"
-    if p.vccAndC and t.len == 0:
-      result.add "0"
     for i in 0..<t.len:
       if i > 0: result.add ", "
       getDefaultValue(p, t[i], info, result)
@@ -3288,7 +3279,6 @@ proc getNullValueAux(p: BProc; t: PType; obj, constOrNil: PNode,
     if constOrNil != nil:
       for i in 1..<constOrNil.len:
         if constOrNil[i].kind == nkExprColonExpr:
-          assert constOrNil[i][0].kind == nkSym, "illformed object constr; the field is not a sym"
           if constOrNil[i][0].sym.name.id == field.name.id:
             genBracedInit(p, constOrNil[i][1], isConst, field.typ, result)
             return
@@ -3334,8 +3324,6 @@ proc genConstObjConstr(p: BProc; n: PNode; isConst: bool; result: var Rope) =
 
 proc genConstSimpleList(p: BProc, n: PNode; isConst: bool; result: var Rope) =
   result.add "{"
-  if p.vccAndC and n.len == 0 and n.typ.kind == tyArray:
-    getDefaultValue(p, n.typ[1], n.info, result)
   for i in 0..<n.len:
     let it = n[i]
     if i > 0: result.add ",\n"
@@ -3345,8 +3333,6 @@ proc genConstSimpleList(p: BProc, n: PNode; isConst: bool; result: var Rope) =
 
 proc genConstTuple(p: BProc, n: PNode; isConst: bool; tup: PType; result: var Rope) =
   result.add "{"
-  if p.vccAndC and n.len == 0:
-    result.add "0"
   for i in 0..<n.len:
     let it = n[i]
     if i > 0: result.add ",\n"

compiler/ccgstmts.nim

@@ -8,6 +8,7 @@
 #
 
 # included from cgen.nim
+
 const
   RangeExpandLimit = 256      # do not generate ranges
                               # over 'RangeExpandLimit' elements
@@ -1081,7 +1082,7 @@ proc genTryCpp(p: BProc, t: PNode, d: var TLoc) =
           let memberName = if p.module.compileToCpp: "m_type" else: "Sup.m_type"
           if optTinyRtti in p.config.globalOptions:
             let checkFor = $getObjDepth(typeNode.typ)
-            appcg(p.module, orExpr, "#isObjDisplayCheck(#nimBorrowCurrentException()->$1, $2, $3)", [memberName, checkFor, $genDisplayElem(MD5Digest(hashType(typeNode.typ, p.config)))])
+            appcg(p.module, orExpr, "#isObjDisplayCheck(#nimBorrowCurrentException()->$1, $2, $3)", [memberName, checkFor, $genDisplayElem(MD5Digest(hashType(typeNode.typ)))])
           else:
             let checkFor = genTypeInfoV1(p.module, typeNode.typ, typeNode.info)
             appcg(p.module, orExpr, "#isObj(#nimBorrowCurrentException()->$1, $2)", [memberName, checkFor])
@@ -1300,7 +1301,7 @@ proc genTryGoto(p: BProc; t: PNode; d: var TLoc) =
         let memberName = if p.module.compileToCpp: "m_type" else: "Sup.m_type"
         if optTinyRtti in p.config.globalOptions:
           let checkFor = $getObjDepth(t[i][j].typ)
-          appcg(p.module, orExpr, "#isObjDisplayCheck(#nimBorrowCurrentException()->$1, $2, $3)", [memberName, checkFor,  $genDisplayElem(MD5Digest(hashType(t[i][j].typ, p.config)))])
+          appcg(p.module, orExpr, "#isObjDisplayCheck(#nimBorrowCurrentException()->$1, $2, $3)", [memberName, checkFor,  $genDisplayElem(MD5Digest(hashType(t[i][j].typ)))])
         else:
           let checkFor = genTypeInfoV1(p.module, t[i][j].typ, t[i][j].info)
           appcg(p.module, orExpr, "#isObj(#nimBorrowCurrentException()->$1, $2)", [memberName, checkFor])
@@ -1445,7 +1446,7 @@ proc genTrySetjmp(p: BProc, t: PNode, d: var TLoc) =
         let memberName = if p.module.compileToCpp: "m_type" else: "Sup.m_type"
         if optTinyRtti in p.config.globalOptions:
           let checkFor = $getObjDepth(t[i][j].typ)
-          appcg(p.module, orExpr, "#isObjDisplayCheck(#nimBorrowCurrentException()->$1, $2, $3)", [memberName, checkFor,  $genDisplayElem(MD5Digest(hashType(t[i][j].typ, p.config)))])
+          appcg(p.module, orExpr, "#isObjDisplayCheck(#nimBorrowCurrentException()->$1, $2, $3)", [memberName, checkFor,  $genDisplayElem(MD5Digest(hashType(t[i][j].typ)))])
         else:
           let checkFor = genTypeInfoV1(p.module, t[i][j].typ, t[i][j].info)
           appcg(p.module, orExpr, "#isObj(#nimBorrowCurrentException()->$1, $2)", [memberName, checkFor])
@@ -1537,7 +1538,7 @@ proc determineSection(n: PNode): TCFileSection =
   result = cfsProcHeaders
   if n.len >= 1 and n[0].kind in {nkStrLit..nkTripleStrLit}:
     let sec = n[0].strVal
-    if sec.startsWith("/*TYPESECTION*/"): result = cfsForwardTypes # TODO WORKAROUND
+    if sec.startsWith("/*TYPESECTION*/"): result = cfsTypes
     elif sec.startsWith("/*VARSECTION*/"): result = cfsVars
     elif sec.startsWith("/*INCLUDESECTION*/"): result = cfsHeaders
 
@@ -1554,14 +1555,9 @@ proc genEmit(p: BProc, t: PNode) =
     line(p, cpsStmts, s)
 
 proc genPragma(p: BProc, n: PNode) =
-  for i in 0..<n.len:
-    let it = n[i]
+  for it in n.sons:
     case whichPragma(it)
     of wEmit: genEmit(p, it)
-    of wPush:
-      processPushBackendOption(p.optionsStack, p.options, n, i+1)
-    of wPop:
-      processPopBackendOption(p.optionsStack, p.options)
     else: discard
 
 
@@ -1579,8 +1575,6 @@ proc genDiscriminantCheck(p: BProc, a, tmp: TLoc, objtype: PType,
         "#FieldDiscriminantCheck((NI)(NU)($1), (NI)(NU)($2), $3, $4);$n",
         [rdLoc(a), rdLoc(tmp), discriminatorTableName(p.module, t, field),
          lit])
-  if p.config.exc == excGoto:
-    raiseExit(p)
 
 when false:
   proc genCaseObjDiscMapping(p: BProc, e: PNode, t: PType, field: PSym; d: var TLoc) =
@@ -1605,7 +1599,7 @@ proc asgnFieldDiscriminant(p: BProc, e: PNode) =
   initLocExpr(p, e[0], a)
   getTemp(p, a.t, tmp)
   expr(p, e[1], tmp)
-  if p.inUncheckedAssignSection == 0:
+  if optTinyRtti notin p.config.globalOptions and p.inUncheckedAssignSection == 0:
     let field = dotExpr[1].sym
     genDiscriminantCheck(p, a, tmp, dotExpr[0].typ, field)
     message(p.config, e.info, warnCaseTransition)
@@ -1626,7 +1620,6 @@ proc genAsgn(p: BProc, e: PNode, fastAsgn: bool) =
     initLoc(a, locNone, le, OnUnknown)
     a.flags.incl(lfEnforceDeref)
     a.flags.incl(lfPrepareForMutation)
-    genLineDir(p, le) # it can be a nkBracketExpr, which may raise
     expr(p, le, a)
     a.flags.excl(lfPrepareForMutation)
     if fastAsgn: incl(a.flags, lfNoDeepCopy)

compiler/ccgtypes.nim

@@ -43,14 +43,16 @@ proc fillBackendName(m: BModule; s: PSym) =
     result.add rope s.itemId.item
     if m.hcrOn:
       result.add "_"
-      result.add(idOrSig(s, m.module.name.s.mangle, m.sigConflicts, m.config))
+      result.add(idOrSig(s, m.module.name.s.mangle, m.sigConflicts))
     s.loc.r = result
     writeMangledName(m.ndi, s, m.config)
 
 proc fillParamName(m: BModule; s: PSym) =
+  ## we cannot use 'sigConflicts' here since we have a BModule, not a BProc.
+  ## Fortunately C's scoping rules are sane enough so that that doesn't
+  ## cause any trouble.
   if s.loc.r == "":
     var res = s.name.s.mangle
-    res.add idOrSig(s, res, m.sigConflicts, m.config)
     # Take into account if HCR is on because of the following scenario:
     #   if a module gets imported and it has some more importc symbols in it,
     # some param names might receive the "_0" suffix to distinguish from what
@@ -67,6 +69,8 @@ proc fillParamName(m: BModule; s: PSym) =
     # and a function called in main or proxy uses `socket` as a parameter name.
     # That would lead to either needing to reload `proxy` or to overwrite the
     # executable file for the main module, which is running (or both!) -> error.
+    if m.hcrOn or isKeyword(s.name) or m.g.config.cppDefines.contains(res):
+      res.add "_0"
     s.loc.r = res.rope
     writeMangledName(m.ndi, s, m.config)
 
@@ -200,9 +204,6 @@ proc isImportedCppType(t: PType): bool =
   result = (t.sym != nil and sfInfixCall in t.sym.flags) or
            (x.sym != nil and sfInfixCall in x.sym.flags)
 
-proc isOrHasImportedCppType(typ: PType): bool =
-  searchTypeFor(typ.skipTypes({tyRef}), isImportedCppType)
-
 proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKind): Rope
 
 proc isObjLackingTypeField(typ: PType): bool {.inline.} =
@@ -305,7 +306,7 @@ proc getSimpleTypeDesc(m: BModule, typ: PType): Rope =
   else: result = ""
 
   if result != "" and typ.isImportedType():
-    let sig = hashType(typ, m.config)
+    let sig = hashType typ
     if cacheGetType(m.typeCache, sig) == "":
       m.typeCache[sig] = result
 
@@ -365,10 +366,10 @@ proc getTypeDescWeak(m: BModule; t: PType; check: var IntSet; kind: TSymKind): R
     if isImportedCppType(etB) and t.kind == tyGenericInst:
       result = getTypeDescAux(m, t, check, kind)
     else:
-      result = getTypeForward(m, t, hashType(t, m.config))
+      result = getTypeForward(m, t, hashType(t))
       pushType(m, t)
   of tySequence:
-    let sig = hashType(t, m.config)
+    let sig = hashType(t)
     if optSeqDestructors in m.config.globalOptions:
       if skipTypes(etB[0], typedescInst).kind == tyEmpty:
         internalError(m.config, "cannot map the empty seq type to a C type")
@@ -401,7 +402,7 @@ proc getSeqPayloadType(m: BModule; t: PType): Rope =
   #result = getTypeForward(m, t, hashType(t)) & "_Content"
 
 proc seqV2ContentType(m: BModule; t: PType; check: var IntSet) =
-  let sig = hashType(t, m.config)
+  let sig = hashType(t)
   let result = cacheGetType(m.typeCache, sig)
   if result == "":
     discard getTypeDescAux(m, t, check, skVar)
@@ -556,10 +557,7 @@ proc genRecordFieldsAux(m: BModule, n: PNode,
       else:
         # don't use fieldType here because we need the
         # tyGenericInst for C++ template support
-        if fieldType.isOrHasImportedCppType():
-          result.addf("$1$3 $2{};$n", [getTypeDescAux(m, field.loc.t, check, skField), sname, noAlias])
-        else:
-          result.addf("$1$3 $2;$n", [getTypeDescAux(m, field.loc.t, check, skField), sname, noAlias])
+        result.addf("$1$3 $2;$n", [getTypeDescAux(m, field.loc.t, check, skField), sname, noAlias])
   else: internalError(m.config, n.info, "genRecordFieldsAux()")
 
 proc getRecordFields(m: BModule, typ: PType, check: var IntSet): Rope =
@@ -670,7 +668,7 @@ proc resolveStarsInCppType(typ: PType, idx, stars: int): PType =
                else: result.elemType
 
 proc getOpenArrayDesc(m: BModule, t: PType, check: var IntSet; kind: TSymKind): Rope =
-  let sig = hashType(t, m.config)
+  let sig = hashType(t)
   if kind == skParam:
     result = getTypeDescWeak(m, t[0], check, kind) & "*"
   else:
@@ -694,7 +692,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKin
     # C type generation into an analysis and a code generation phase somehow.
   if t.sym != nil: useHeader(m, t.sym)
   if t != origTyp and origTyp.sym != nil: useHeader(m, origTyp.sym)
-  let sig = hashType(origTyp, m.config)
+  let sig = hashType(origTyp)
 
   defer: # defer is the simplest in this case
     if isImportedType(t) and not m.typeABICache.containsOrIncl(sig):
@@ -723,7 +721,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKin
         result = getTypeDescAux(m, et, check, kind) & star
       else:
         # no restriction! We have a forward declaration for structs
-        let name = getTypeForward(m, et, hashType(et, m.config))
+        let name = getTypeForward(m, et, hashType et)
         result = name & star
         m.typeCache[sig] = result
     of tySequence:
@@ -732,7 +730,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKin
         m.typeCache[sig] = result
       else:
         # no restriction! We have a forward declaration for structs
-        let name = getTypeForward(m, et, hashType(et, m.config))
+        let name = getTypeForward(m, et, hashType et)
         result = name & seqStar(m) & star
         m.typeCache[sig] = result
         pushType(m, et)
@@ -832,8 +830,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKin
       m.s[cfsTypes].addf("typedef $1 $2[$3];$n",
            [foo, result, rope(n)])
   of tyObject, tyTuple:
-    let tt = origTyp.skipTypes({tyDistinct})
-    if isImportedCppType(t) and tt.kind == tyGenericInst:
+    if isImportedCppType(t) and origTyp.kind == tyGenericInst:
       let cppNameAsRope = getTypeName(m, t, sig)
       let cppName = $cppNameAsRope
       var i = 0
@@ -856,7 +853,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKin
             result.add cppName.substr(chunkStart, chunkEnd)
             chunkStart = i
 
-            let typeInSlot = resolveStarsInCppType(tt, idx + 1, stars)
+            let typeInSlot = resolveStarsInCppType(origTyp, idx + 1, stars)
             addResultType(typeInSlot)
         else:
           inc i
@@ -865,9 +862,9 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKin
         result.add cppName.substr(chunkStart)
       else:
         result = cppNameAsRope & "<"
-        for i in 1..<tt.len-1:
+        for i in 1..<origTyp.len-1:
           if i > 1: result.add(" COMMA ")
-          addResultType(tt[i])
+          addResultType(origTyp[i])
         result.add("> ")
       # always call for sideeffects:
       assert t.kind != tyTuple
@@ -899,7 +896,7 @@ proc getTypeDescAux(m: BModule, origTyp: PType, check: var IntSet; kind: TSymKin
     # Don't use the imported name as it may be scoped: 'Foo::SomeKind'
     result = rope("tySet_")
     t.lastSon.typeName(result)
-    result.add $t.lastSon.hashType(m.config)
+    result.add $t.lastSon.hashType
     m.typeCache[sig] = result
     if not isImportedType(t):
       let s = int(getSize(m.config, t))
@@ -1072,7 +1069,7 @@ proc discriminatorTableName(m: BModule, objtype: PType, d: PSym): Rope =
     objtype = objtype[0].skipTypes(abstractPtrs)
   if objtype.sym == nil:
     internalError(m.config, d.info, "anonymous obj with discriminator")
-  result = "NimDT_$1_$2" % [rope($hashType(objtype, m.config)), rope(d.name.s.mangle)]
+  result = "NimDT_$1_$2" % [rope($hashType(objtype)), rope(d.name.s.mangle)]
 
 proc rope(arg: Int128): Rope = rope($arg)
 
@@ -1291,7 +1288,7 @@ proc genTypeInfo2Name(m: BModule; t: PType): Rope =
       result.add m.name.s & "."
       result.add it.sym.name.s
   else:
-    result = $hashType(it, m.config)
+    result = $hashType(it)
   result = makeCString(result)
 
 proc isTrivialProc(g: ModuleGraph; s: PSym): bool {.inline.} = getBody(g, s).len == 0
@@ -1335,14 +1332,14 @@ proc genDisplayElem(d: MD5Digest): uint32 =
     result += uint32(d[i])
     result = result shl 8
 
-proc genDisplay(m: BModule, t: PType, depth: int): Rope =
+proc genDisplay(t: PType, depth: int): Rope =
   result = Rope"{"
   var x = t
   var seqs = newSeq[string](depth+1)
   var i = 0
   while x != nil:
     x = skipTypes(x, skipPtrs)
-    seqs[i] = $genDisplayElem(MD5Digest(hashType(x, m.config)))
+    seqs[i] = $genDisplayElem(MD5Digest(hashType(x)))
     x = x[0]
     inc i
 
@@ -1351,7 +1348,7 @@ proc genDisplay(m: BModule, t: PType, depth: int): Rope =
   result.add seqs[0]
   result.add "}"
 
-proc genTypeInfoV2OldImpl(m: BModule; t, origType: PType, name: Rope; info: TLineInfo) =
+proc genTypeInfoV2Impl(m: BModule; t, origType: PType, name: Rope; info: TLineInfo) =
   cgsym(m, "TNimTypeV2")
   m.s[cfsStrData].addf("N_LIB_PRIVATE TNimTypeV2 $1;$n", [name])
 
@@ -1367,6 +1364,7 @@ proc genTypeInfoV2OldImpl(m: BModule; t, origType: PType, name: Rope; info: TLin
 
   let objDepth = if t.kind == tyObject: getObjDepth(t) else: -1
 
+
   if t.kind in {tyObject, tyDistinct} and incompleteType(t):
     localError(m.config, info, "request for RTTI generation for incomplete object: " &
               typeToString(t))
@@ -1382,7 +1380,7 @@ proc genTypeInfoV2OldImpl(m: BModule; t, origType: PType, name: Rope; info: TLin
     [name, getTypeDesc(m, t), rope(objDepth), rope(flags)])
 
   if objDepth >= 0:
-    let objDisplay = genDisplay(m, t, objDepth)
+    let objDisplay = genDisplay(t, objDepth)
     let objDisplayStore = getTempName(m)
     m.s[cfsVars].addf("static $1 $2[$3] = $4;$n", [getTypeDesc(m, getSysType(m.g.graph, unknownLineInfo, tyUInt32), skVar), objDisplayStore, rope(objDepth+1), objDisplay])
     addf(typeEntry, "$1.display = $2;$n", [name, rope(objDisplayStore)])
@@ -1392,48 +1390,6 @@ proc genTypeInfoV2OldImpl(m: BModule; t, origType: PType, name: Rope; info: TLin
   if t.kind == tyObject and t.len > 0 and t[0] != nil and optEnableDeepCopy in m.config.globalOptions:
     discard genTypeInfoV1(m, t, info)
 
-proc genTypeInfoV2Impl(m: BModule; t, origType: PType, name: Rope; info: TLineInfo) =
-  cgsym(m, "TNimTypeV2")
-  m.s[cfsStrData].addf("N_LIB_PRIVATE TNimTypeV2 $1;$n", [name])
-
-  var flags = 0
-  if not canFormAcycle(t): flags = flags or 1
-
-  var typeEntry = newRopeAppender()
-  addf(typeEntry, "N_LIB_PRIVATE TNimTypeV2 $1 = {", [name])
-  add(typeEntry, ".destructor = (void*)")
-  genHook(m, t, info, attachedDestructor, typeEntry)
-
-  let objDepth = if t.kind == tyObject: getObjDepth(t) else: -1
-
-  if t.kind in {tyObject, tyDistinct} and incompleteType(t):
-    localError(m.config, info, "request for RTTI generation for incomplete object: " &
-              typeToString(t))
-
-  addf(typeEntry, ", .size = sizeof($1), .align = (NI16) NIM_ALIGNOF($1), .depth = $2",
-    [getTypeDesc(m, t), rope(objDepth)])
-
-  if objDepth >= 0:
-    let objDisplay = genDisplay(m, t, objDepth)
-    let objDisplayStore = getTempName(m)
-    m.s[cfsVars].addf("static NIM_CONST $1 $2[$3] = $4;$n", [getTypeDesc(m, getSysType(m.g.graph, unknownLineInfo, tyUInt32), skVar), objDisplayStore, rope(objDepth+1), objDisplay])
-    addf(typeEntry, ", .display = $1", [rope(objDisplayStore)])
-  if isDefined(m.config, "nimTypeNames"):
-    var typeName: Rope
-    if t.kind in {tyObject, tyDistinct}:
-      typeName = genTypeInfo2Name(m, t)
-    else:
-      typeName = rope("NIM_NIL")
-    addf(typeEntry, ", .name = $1", [typeName])
-  add(typeEntry, ", .traceImpl = (void*)")
-  genHook(m, t, info, attachedTrace, typeEntry)
-
-  addf(typeEntry, ", .flags = $1};$n", [rope(flags)])
-  m.s[cfsVars].add typeEntry
-
-  if t.kind == tyObject and t.len > 0 and t[0] != nil and optEnableDeepCopy in m.config.globalOptions:
-    discard genTypeInfoV1(m, t, info)
-
 proc genTypeInfoV2(m: BModule, t: PType; info: TLineInfo): Rope =
   let origType = t
   # distinct types can have their own destructors
@@ -1441,7 +1397,7 @@ proc genTypeInfoV2(m: BModule, t: PType; info: TLineInfo): Rope =
 
   let prefixTI = if m.hcrOn: "(" else: "(&"
 
-  let sig = hashType(origType, m.config)
+  let sig = hashType(origType)
   result = m.typeInfoMarkerV2.getOrDefault(sig)
   if result != "":
     return prefixTI.rope & result & ")".rope
@@ -1467,10 +1423,7 @@ proc genTypeInfoV2(m: BModule, t: PType; info: TLineInfo): Rope =
     return prefixTI.rope & result & ")".rope
 
   m.g.typeInfoMarkerV2[sig] = (str: result, owner: owner)
-  if m.compileToCpp:
-    genTypeInfoV2OldImpl(m, t, origType, result, info)
-  else:
-    genTypeInfoV2Impl(m, t, origType, result, info)
+  genTypeInfoV2Impl(m, t, origType, result, info)
   result = prefixTI.rope & result & ")".rope
 
 proc openArrayToTuple(m: BModule; t: PType): PType =
@@ -1515,7 +1468,7 @@ proc genTypeInfoV1(m: BModule, t: PType; info: TLineInfo): Rope =
 
   let prefixTI = if m.hcrOn: "(" else: "(&"
 
-  let sig = hashType(origType, m.config)
+  let sig = hashType(origType)
   result = m.typeInfoMarker.getOrDefault(sig)
   if result != "":
     return prefixTI.rope & result & ")".rope

compiler/ccgutils.nim

@@ -53,7 +53,7 @@ proc hashString*(conf: ConfigRef; s: string): BiggestInt =
     a = a + (a shl 3)
     a = a xor (a shr 11)
     a = a + (a shl 15)
-    result = cast[Hash](uint(a))
+    result = cast[Hash](a)
 
 template getUniqueType*(key: PType): PType = key
 

compiler/cgen.nim

@@ -12,12 +12,10 @@
 import
   ast, astalgo, hashes, trees, platform, magicsys, extccomp, options, intsets,
   nversion, nimsets, msgs, bitsets, idents, types,
-  ccgutils, os, ropes, math, wordrecg, treetab, cgmeth,
+  ccgutils, os, ropes, math, passes, wordrecg, treetab, cgmeth,
   rodutils, renderer, cgendata, aliases,
   lowerings, tables, sets, ndi, lineinfos, pathutils, transf,
-  injectdestructors, astmsgs, modulepaths, backendpragmas
-
-import pipelineutils
+  injectdestructors, astmsgs, modulepaths
 
 when defined(nimPreviewSlimSystem):
   import std/assertions
@@ -61,12 +59,12 @@ proc findPendingModule(m: BModule, s: PSym): BModule =
     var ms = getModule(s)
     result = m.g.modules[ms.position]
 
-proc initLoc(result: var TLoc, k: TLocKind, lode: PNode, s: TStorageLoc, flags: TLocFlags = {}) =
+proc initLoc(result: var TLoc, k: TLocKind, lode: PNode, s: TStorageLoc) =
   result.k = k
   result.storage = s
   result.lode = lode
   result.r = ""
-  result.flags = flags
+  result.flags = {}
 
 proc fillLoc(a: var TLoc, k: TLocKind, lode: PNode, r: Rope, s: TStorageLoc) {.inline.} =
   # fills the loc if it is not already initialized
@@ -228,12 +226,8 @@ macro ropecg(m: BModule, frmt: static[FormatStr], args: untyped): Rope =
   result.add newCall(ident"rope", resVar)
 
 proc addIndent(p: BProc; result: var Rope) =
-  var i = result.len
-  let newLen = i + p.blocks.len
-  result.setLen newLen
-  while i < newLen:
-    result[i] = '\t'
-    inc i
+  for i in 0..<p.blocks.len:
+    result.add "\t".rope
 
 template appcg(m: BModule, c: var Rope, frmt: FormatStr,
            args: untyped) =
@@ -277,8 +271,7 @@ proc genCLineDir(r: var Rope, filename: string, line: int; conf: ConfigRef) =
         [rope(makeSingleLineCString(filename)), rope(line)])
 
 proc genCLineDir(r: var Rope, info: TLineInfo; conf: ConfigRef) =
-  if optLineDir in conf.options:
-    genCLineDir(r, toFullPath(conf, info), info.safeLineNm, conf)
+  genCLineDir(r, toFullPath(conf, info), info.safeLineNm, conf)
 
 proc freshLineInfo(p: BProc; info: TLineInfo): bool =
   if p.lastLineInfo.line != info.line or
@@ -292,7 +285,7 @@ proc genLineDir(p: BProc, t: PNode) =
 
   if optEmbedOrigSrc in p.config.globalOptions:
     p.s(cpsStmts).add("//" & sourceLine(p.config, t.info) & "\L")
-  genCLineDir(p.s(cpsStmts), t.info, p.config)
+  genCLineDir(p.s(cpsStmts), toFullPath(p.config, t.info), line, p.config)
   if ({optLineTrace, optStackTrace} * p.options == {optLineTrace, optStackTrace}) and
       (p.prc == nil or sfPure notin p.prc.flags) and t.info.fileIndex != InvalidFileIdx:
     if freshLineInfo(p, t.info):
@@ -504,7 +497,7 @@ proc constructLoc(p: BProc, loc: var TLoc, isTemp = false) =
     if not isTemp or containsGarbageCollectedRef(loc.t):
       # don't use nimZeroMem for temporary values for performance if we can
       # avoid it:
-      if not isOrHasImportedCppType(typ):
+      if not isImportedCppType(typ):
         linefmt(p, cpsStmts, "#nimZeroMem((void*)$1, sizeof($2));$n",
                 [addrLoc(p.config, loc), getTypeDesc(p.module, typ, mapTypeChooser(loc))])
     genObjectInit(p, cpsStmts, loc.t, loc, constructObj)
@@ -524,10 +517,7 @@ proc initLocalVar(p: BProc, v: PSym, immediateAsgn: bool) =
 proc getTemp(p: BProc, t: PType, result: var TLoc; needsInit=false) =
   inc(p.labels)
   result.r = "T" & rope(p.labels) & "_"
-  if p.module.compileToCpp and isOrHasImportedCppType(t):
-    linefmt(p, cpsLocals, "$1 $2{};$n", [getTypeDesc(p.module, t, skVar), result.r])
-  else:
-    linefmt(p, cpsLocals, "$1 $2;$n", [getTypeDesc(p.module, t, skVar), result.r])
+  linefmt(p, cpsLocals, "$1 $2;$n", [getTypeDesc(p.module, t, skVar), result.r])
   result.k = locTemp
   result.lode = lodeTyp t
   result.storage = OnStack
@@ -568,9 +558,6 @@ proc localVarDecl(p: BProc; n: PNode): Rope =
     if s.kind == skLet: incl(s.loc.flags, lfNoDeepCopy)
   if s.kind in {skLet, skVar, skField, skForVar} and s.alignment > 0:
     result.addf("NIM_ALIGN($1) ", [rope(s.alignment)])
-
-  genCLineDir(result, n.info, p.config)
-
   result.add getTypeDesc(p.module, s.typ, skVar)
   if s.constraint.isNil:
     if sfRegister in s.flags: result.add(" register")
@@ -588,7 +575,7 @@ proc assignLocalVar(p: BProc, n: PNode) =
   # this need not be fulfilled for inline procs; they are regenerated
   # for each module that uses them!
   let nl = if optLineDir in p.config.options: "" else: "\L"
-  let decl = localVarDecl(p, n) & (if p.module.compileToCpp and isOrHasImportedCppType(n.typ): "{};" else: ";") & nl
+  let decl = localVarDecl(p, n) & ";" & nl
   line(p, cpsLocals, decl)
 
 include ccgthreadvars
@@ -641,23 +628,7 @@ proc assignGlobalVar(p: BProc, n: PNode; value: Rope) =
         if sfVolatile in s.flags: decl.add(" volatile")
         if sfNoalias in s.flags: decl.add(" NIM_NOALIAS")
         if value != "":
-          if p.module.compileToCpp and value.startsWith "{{}":
-            # TODO: taking this branch, re"\{\{\}(,\s\{\})*\}" might be emitted, resulting in
-            # either warnings (GCC 12.2+) or errors (Clang 15, MSVC 19.3+) of C++11+ compilers **when
-            # explicit constructors are around** due to overload resolution rules in place [^0][^1][^2]
-            # *Workaround* here: have C++'s static initialization mechanism do the default init work,
-            # for us lacking a deeper knowledge of an imported object's constructors' ex-/implicitness
-            # (so far) *and yet* trying to achieve default initialization.
-            # Still, generating {}s in genConstObjConstr() just to omit them here is faaaar from ideal;
-            # need to figure out a better way, possibly by keeping around more data about the
-            # imported objects' contructors?
-            #
-            # [^0]: https://en.cppreference.com/w/cpp/language/aggregate_initialization
-            # [^1]: https://cplusplus.github.io/CWG/issues/1518.html
-            # [^2]: https://eel.is/c++draft/over.match.ctor
-            decl.addf(" $1;$n", [s.loc.r])
-          else:
-            decl.addf(" $1 = $2;$n", [s.loc.r, value])
+          decl.addf(" $1 = $2;$n", [s.loc.r, value])
         else:
           decl.addf(" $1;$n", [s.loc.r])
       else:
@@ -698,8 +669,8 @@ proc genLiteral(p: BProc, n: PNode; result: var Rope)
 proc genOtherArg(p: BProc; ri: PNode; i: int; typ: PType; result: var Rope; argsCounter: var int)
 proc raiseExit(p: BProc)
 
-proc initLocExpr(p: BProc, e: PNode, result: var TLoc, flags: TLocFlags = {}) =
-  initLoc(result, locNone, e, OnUnknown, flags)
+proc initLocExpr(p: BProc, e: PNode, result: var TLoc) =
+  initLoc(result, locNone, e, OnUnknown)
   expr(p, e, result)
 
 proc initLocExprSingleUse(p: BProc, e: PNode, result: var TLoc) =
@@ -1099,7 +1070,7 @@ proc genProcBody(p: BProc; procBody: PNode) =
 proc isNoReturn(m: BModule; s: PSym): bool {.inline.} =
   sfNoReturn in s.flags and m.config.exc != excGoto
 
-proc genProcAux*(m: BModule, prc: PSym) =
+proc genProcAux(m: BModule, prc: PSym) =
   var p = newProc(prc, m)
   var header = newRopeAppender()
   genProcHeader(m, prc, header)
@@ -1653,7 +1624,7 @@ proc registerModuleToMain(g: BModuleList; m: BModule) =
     hcrModuleMeta.addf("\t\"\"};$n", [])
     hcrModuleMeta.addf("$nN_LIB_EXPORT N_NIMCALL(void**, HcrGetImportedModules)() { return (void**)hcr_module_list; }$n", [])
     hcrModuleMeta.addf("$nN_LIB_EXPORT N_NIMCALL(char*, HcrGetSigHash)() { return \"$1\"; }$n$n",
-                          [($sigHash(m.module, m.config)).rope])
+                          [($sigHash(m.module)).rope])
     if sfMainModule in m.module.flags:
       g.mainModProcs.add(hcrModuleMeta)
       g.mainModProcs.addf("static void* hcr_handle;$N", [])
@@ -1830,9 +1801,6 @@ proc genInitCode(m: BModule) =
 
     if optStackTrace in m.initProc.options and preventStackTrace notin m.flags:
       prc.add(deinitFrame(m.initProc))
-  elif sfMainModule in m.module.flags and m.config.exc == excGoto:
-    if getCompilerProc(m.g.graph, "nimTestErrorFlag") != nil:
-      m.appcg(prc, "\t#nimTestErrorFlag();$n", [])
 
   prc.addf("}$N", [])
 
@@ -1961,7 +1929,10 @@ template injectG() {.dirty.} =
     graph.backend = newModuleList(graph)
   let g = BModuleList(graph.backend)
 
-proc setupCgen*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
+when not defined(nimHasSinkInference):
+  {.pragma: nosinks.}
+
+proc myOpen(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext {.nosinks.} =
   injectG()
   result = newModule(g, module, graph.config)
   result.idgen = idgen
@@ -2029,7 +2000,7 @@ proc addHcrInitGuards(p: BProc, n: PNode, inInitGuard: var bool) =
 
 proc genTopLevelStmt*(m: BModule; n: PNode) =
   ## Also called from `ic/cbackend.nim`.
-  if pipelineutils.skipCodegen(m.config, n): return
+  if passes.skipCodegen(m.config, n): return
   m.initProc.options = initProcOptions(m)
   #softRnl = if optLineDir in m.config.options: noRnl else: rnl
   # XXX replicate this logic!
@@ -2042,6 +2013,12 @@ proc genTopLevelStmt*(m: BModule; n: PNode) =
   else:
     genProcBody(m.initProc, transformedN)
 
+proc myProcess(b: PPassContext, n: PNode): PNode =
+  result = n
+  if b != nil:
+    var m = BModule(b)
+    genTopLevelStmt(m, n)
+
 proc shouldRecompile(m: BModule; code: Rope, cfile: Cfile): bool =
   if optForceFullMake notin m.config.globalOptions:
     if not moduleHasChanged(m.g.graph, m.module):
@@ -2107,7 +2084,7 @@ proc updateCachedModule(m: BModule) =
   cf.flags = {CfileFlag.Cached}
   addFileToCompile(m.config, cf)
 
-proc finalCodegenActions*(graph: ModuleGraph; m: BModule; n: PNode): PNode =
+proc finalCodegenActions*(graph: ModuleGraph; m: BModule; n: PNode) =
   ## Also called from IC.
   if sfMainModule in m.module.flags:
     # phase ordering problem here: We need to announce this
@@ -2118,7 +2095,7 @@ proc finalCodegenActions*(graph: ModuleGraph; m: BModule; n: PNode): PNode =
     if {optGenStaticLib, optGenDynLib, optNoMain} * m.config.globalOptions == {}:
       for i in countdown(high(graph.globalDestructors), 0):
         n.add graph.globalDestructors[i]
-  if pipelineutils.skipCodegen(m.config, n): return
+  if passes.skipCodegen(m.config, n): return
   if moduleHasChanged(graph, m.module):
     # if the module is cached, we don't regenerate the main proc
     # nor the dispatchers? But if the dispatchers changed?
@@ -2153,11 +2130,18 @@ proc finalCodegenActions*(graph: ModuleGraph; m: BModule; n: PNode): PNode =
 
       if m.g.forwardedProcs.len == 0:
         incl m.flags, objHasKidsValid
-      result = generateMethodDispatchers(graph, m.idgen)
+      let disp = generateMethodDispatchers(graph)
+      for x in disp: genProcAux(m, x.sym)
 
   let mm = m
   m.g.modulesClosed.add mm
 
+
+proc myClose(graph: ModuleGraph; b: PPassContext, n: PNode): PNode =
+  result = n
+  if b == nil: return
+  finalCodegenActions(graph, BModule(b), n)
+
 proc genForwardedProcs(g: BModuleList) =
   # Forward declared proc:s lack bodies when first encountered, so they're given
   # a second pass here
@@ -2184,3 +2168,5 @@ proc cgenWriteModules*(backend: RootRef, config: ConfigRef) =
     m.writeModule(pending=true)
   writeMapping(config, g.mapping)
   if g.generatedHeader != nil: writeHeader(g.generatedHeader)
+
+const cgenPass* = makePass(myOpen, myProcess, myClose)

compiler/cgendata.nim

@@ -86,7 +86,6 @@ type
     options*: TOptions        # options that should be used for code
                               # generation; this is the same as prc.options
                               # unless prc == nil
-    optionsStack*: seq[TOptions]
     module*: BModule          # used to prevent excessive parameter passing
     withinLoop*: int          # > 0 if we are within a loop
     splitDecls*: int          # > 0 if we are in some context for C++ that
@@ -173,7 +172,6 @@ type
 
 template config*(m: BModule): ConfigRef = m.g.config
 template config*(p: BProc): ConfigRef = p.module.g.config
-template vccAndC*(p: BProc): bool = p.module.config.cCompiler == ccVcc and p.module.config.backend == backendC
 
 proc includeHeader*(this: BModule; header: string) =
   if not this.headerFiles.contains header:

compiler/cgmeth.nim

@@ -213,7 +213,7 @@ proc sortBucket(a: var seq[PSym], relevantCols: IntSet) =
       a[j] = v
     if h == 1: break
 
-proc genDispatcher(g: ModuleGraph; methods: seq[PSym], relevantCols: IntSet; idgen: IdGenerator): PSym =
+proc genDispatcher(g: ModuleGraph; methods: seq[PSym], relevantCols: IntSet): PSym =
   var base = methods[0].ast[dispatcherPos].sym
   result = base
   var paramLen = base.typ.len
@@ -272,7 +272,7 @@ proc genDispatcher(g: ModuleGraph; methods: seq[PSym], relevantCols: IntSet; idg
   nilchecks.flags.incl nfTransf # should not be further transformed
   result.ast[bodyPos] = nilchecks
 
-proc generateMethodDispatchers*(g: ModuleGraph, idgen: IdGenerator): PNode =
+proc generateMethodDispatchers*(g: ModuleGraph): PNode =
   result = newNode(nkStmtList)
   for bucket in 0..<g.methods.len:
     var relevantCols = initIntSet()
@@ -282,4 +282,4 @@ proc generateMethodDispatchers*(g: ModuleGraph, idgen: IdGenerator): PNode =
         # if multi-methods are not enabled, we are interested only in the first field
         break
     sortBucket(g.methods[bucket].methods, relevantCols)
-    result.add newSymNode(genDispatcher(g, g.methods[bucket].methods, relevantCols, idgen))
+    result.add newSymNode(genDispatcher(g, g.methods[bucket].methods, relevantCols))

compiler/commands.nim

@@ -336,7 +336,6 @@ proc testCompileOption*(conf: ConfigRef; switch: string, info: TLineInfo): bool
   of "excessivestacktrace": result = contains(conf.globalOptions, optExcessiveStackTrace)
   of "nilseqs", "nilchecks", "taintmode": warningOptionNoop(switch)
   of "panics": result = contains(conf.globalOptions, optPanics)
-  of "jsbigint64": result = contains(conf.globalOptions, optJsBigInt64)
   else: invalidCmdLineOption(conf, passCmd1, switch, info)
 
 proc processPath(conf: ConfigRef; path: string, info: TLineInfo,
@@ -650,8 +649,6 @@ proc processSwitch*(switch, arg: string, pass: TCmdLinePass, info: TLineInfo;
   of "backend", "b":
     let backend = parseEnum(arg.normalize, TBackend.default)
     if backend == TBackend.default: localError(conf, info, "invalid backend: '$1'" % arg)
-    if backend == backendJs: # bug #21209
-      conf.globalOptions.excl {optThreadAnalysis, optThreads}
     conf.backend = backend
   of "doccmd": conf.docCmd = arg
   of "define", "d":
@@ -823,13 +820,7 @@ proc processSwitch*(switch, arg: string, pass: TCmdLinePass, info: TLineInfo;
     if conf != nil: conf.headerFile = arg
     incl(conf.globalOptions, optGenIndex)
   of "index":
-    case arg.normalize
-    of "", "on": conf.globalOptions.incl {optGenIndex}
-    of "only": conf.globalOptions.incl {optGenIndexOnly, optGenIndex}
-    of "off": conf.globalOptions.excl {optGenIndex, optGenIndexOnly}
-    else: localError(conf, info, errOnOrOffExpectedButXFound % arg)
-  of "noimportdoc":
-    processOnOffSwitchG(conf, {optNoImportdoc}, arg, pass, info)
+    processOnOffSwitchG(conf, {optGenIndex}, arg, pass, info)
   of "import":
     expectArg(conf, switch, arg, pass, info)
     if pass in {passCmd2, passPP}:
@@ -1009,9 +1000,6 @@ proc processSwitch*(switch, arg: string, pass: TCmdLinePass, info: TLineInfo;
     expectNoArg(conf, switch, arg, pass, info)
     conf.exc = low(ExceptionSystem)
     defineSymbol(conf.symbols, "noCppExceptions")
-  of "shownonexports":
-    expectNoArg(conf, switch, arg, pass, info)
-    showNonExportedFields(conf)
   of "exceptions":
     case arg.normalize
     of "cpp": conf.exc = excCpp
@@ -1066,6 +1054,29 @@ proc processSwitch*(switch, arg: string, pass: TCmdLinePass, info: TLineInfo;
   of "expandarc":
     expectArg(conf, switch, arg, pass, info)
     conf.arcToExpand[arg] = "T"
+  of "useversion":
+    expectArg(conf, switch, arg, pass, info)
+    case arg
+    of "1.0":
+      defineSymbol(conf.symbols, "NimMajor", "1")
+      defineSymbol(conf.symbols, "NimMinor", "0")
+      # old behaviors go here:
+      defineSymbol(conf.symbols, "nimOldRelativePathBehavior")
+      undefSymbol(conf.symbols, "nimDoesntTrackDefects")
+      ast.eqTypeFlags.excl {tfGcSafe, tfNoSideEffect}
+      conf.globalOptions.incl optNimV1Emulation
+    of "1.2":
+      defineSymbol(conf.symbols, "NimMajor", "1")
+      defineSymbol(conf.symbols, "NimMinor", "2")
+      conf.globalOptions.incl optNimV12Emulation
+    of "1.6":
+      defineSymbol(conf.symbols, "NimMajor", "1")
+      defineSymbol(conf.symbols, "NimMinor", "6")
+      conf.globalOptions.incl optNimV16Emulation
+    else:
+      localError(conf, info, "unknown Nim version; currently supported values are: `1.0`, `1.2`")
+    # always be compatible with 1.x.100:
+    defineSymbol(conf.symbols, "NimPatch", "100")
   of "benchmarkvm":
     processOnOffSwitchG(conf, {optBenchmarkVM}, arg, pass, info)
   of "profilevm":
@@ -1079,8 +1090,6 @@ proc processSwitch*(switch, arg: string, pass: TCmdLinePass, info: TLineInfo;
     processOnOffSwitchG(conf, {optPanics}, arg, pass, info)
     if optPanics in conf.globalOptions:
       defineSymbol(conf.symbols, "nimPanics")
-  of "jsbigint64":
-    processOnOffSwitchG(conf, {optJsBigInt64}, arg, pass, info)
   of "sourcemap": # xxx document in --fullhelp
     conf.globalOptions.incl optSourcemap
     conf.options.incl optLineDir

compiler/concepts.nim

@@ -274,7 +274,7 @@ proc matchSym(c: PContext; candidate: PSym, n: PNode; m: var MatchCon): bool =
 proc matchSyms(c: PContext, n: PNode; kinds: set[TSymKind]; m: var MatchCon): bool =
   ## Walk the current scope, extract candidates which the same name as 'n[namePos]',
   ## 'n' is the nkProcDef or similar from the concept that we try to match.
-  let candidates = searchInScopesAllCandidatesFilterBy(c, n[namePos].sym.name, kinds)
+  let candidates = searchInScopesFilterBy(c, n[namePos].sym.name, kinds)
   for candidate in candidates:
     #echo "considering ", typeToString(candidate.typ), " ", candidate.magic
     m.magic = candidate.magic

compiler/condsyms.nim

@@ -25,7 +25,7 @@ proc undefSymbol*(symbols: StringTableRef; symbol: string) =
 #  result = if isDefined(symbol): gSymbols[symbol] else: nil
 
 iterator definedSymbolNames*(symbols: StringTableRef): string =
-  for key in keys(symbols):
+  for key, val in pairs(symbols):
     yield key
 
 proc countDefinedSymbols*(symbols: StringTableRef): int =
@@ -84,24 +84,10 @@ proc initDefines*(symbols: StringTableRef) =
   defineSymbol("nimHasSignatureHashInMacro") # deadcode
   defineSymbol("nimHasDefault") # deadcode
   defineSymbol("nimMacrosSizealignof") # deadcode
-  defineSymbol("nimNoZeroExtendMagic") # deadcode
-  defineSymbol("nimMacrosGetNodeId") # deadcode
-  defineSymbol("nimFixedForwardGeneric") # deadcode
-  defineSymbol("nimToOpenArrayCString") # deadcode
-  defineSymbol("nimHasUsed") # deadcode
-  defineSymbol("nimnomagic64") # deadcode
-  defineSymbol("nimNewShiftOps") # deadcode
-  defineSymbol("nimHasCursor") # deadcode
-  defineSymbol("nimAlignPragma") # deadcode
-  defineSymbol("nimHasExceptionsQuery") # deadcode
-  defineSymbol("nimHasIsNamedTuple") # deadcode
-  defineSymbol("nimHashOrdinalFixed") # deadcode
-  defineSymbol("nimHasSinkInference") # deadcode
-  defineSymbol("nimNewIntegerOps") # deadcode
-  defineSymbol("nimHasInvariant") # deadcode
-
-
 
+  # > 0.20.0
+  defineSymbol("nimNoZeroExtendMagic")
+  defineSymbol("nimMacrosGetNodeId")
   for f in Feature:
     defineSymbol("nimHas" & $f)
 
@@ -112,18 +98,31 @@ proc initDefines*(symbols: StringTableRef) =
 
   defineSymbol("nimFixedOwned")
   defineSymbol("nimHasStyleChecks")
+  defineSymbol("nimToOpenArrayCString")
+  defineSymbol("nimHasUsed")
+  defineSymbol("nimFixedForwardGeneric")
+  defineSymbol("nimnomagic64")
+  defineSymbol("nimNewShiftOps")
+  defineSymbol("nimHasCursor")
+  defineSymbol("nimAlignPragma")
+  defineSymbol("nimHasExceptionsQuery")
+  defineSymbol("nimHasIsNamedTuple")
+  defineSymbol("nimHashOrdinalFixed")
 
   when defined(nimHasLibFFI):
     # Renaming as we can't conflate input vs output define flags; e.g. this
     # will report the right thing regardless of whether user adds
     # `-d:nimHasLibFFI` in his user config.
-    defineSymbol("nimHasLibFFIEnabled") # deadcode
+    defineSymbol("nimHasLibFFIEnabled")
 
-  defineSymbol("nimHasStacktraceMsgs") # deadcode
+  defineSymbol("nimHasSinkInference")
+  defineSymbol("nimNewIntegerOps")
+  defineSymbol("nimHasInvariant")
+  defineSymbol("nimHasStacktraceMsgs")
   defineSymbol("nimDoesntTrackDefects")
-  defineSymbol("nimHasLentIterators") # deadcode
-  defineSymbol("nimHasDeclaredMagic") # deadcode
-  defineSymbol("nimHasStacktracesModule") # deadcode
+  defineSymbol("nimHasLentIterators")
+  defineSymbol("nimHasDeclaredMagic")
+  defineSymbol("nimHasStacktracesModule")
   defineSymbol("nimHasEffectTraitsModule")
   defineSymbol("nimHasCastPragmaBlocks")
   defineSymbol("nimHasDeclaredLocs")
@@ -134,8 +133,8 @@ proc initDefines*(symbols: StringTableRef) =
   defineSymbol("nimHasCustomLiterals")
   defineSymbol("nimHasUnifiedTuple")
   defineSymbol("nimHasIterable")
-  defineSymbol("nimHasTypeofVoid") # deadcode
-  defineSymbol("nimHasDragonBox") # deadcode
+  defineSymbol("nimHasTypeofVoid")
+  defineSymbol("nimHasDragonBox")
   defineSymbol("nimHasHintAll")
   defineSymbol("nimHasTrace")
   defineSymbol("nimHasEffectsOf")
@@ -147,11 +146,6 @@ proc initDefines*(symbols: StringTableRef) =
   defineSymbol("nimHasCallsitePragma")
   defineSymbol("nimHasAmbiguousEnumHint")
 
-  defineSymbol("nimHasWarnCastSizes")
   defineSymbol("nimHasOutParams")
   defineSymbol("nimHasSystemRaisesDefect")
   defineSymbol("nimHasWarnUnnamedBreak")
-  defineSymbol("nimHasGenericDefine")
-  defineSymbol("nimHasDefineAliases")
-  defineSymbol("nimHasWarnBareExcept")
-  defineSymbol("nimHasWarnCopyHookForRefc")

compiler/depends.nim

@@ -9,7 +9,7 @@
 
 # This module implements a dependency file generator.
 
-import options, ast, ropes, pathutils, msgs, lineinfos
+import options, ast, ropes, passes, pathutils, msgs, lineinfos
 
 import modulegraphs
 
@@ -79,7 +79,7 @@ proc addDependency(c: PPassContext, g: PGen, b: Backend, n: PNode) =
   let child = nativeToUnixPath(path.dir / path.name).toNimblePath(belongsToStdlib(g.graph, n.sym))
   addDependencyAux(b, parent, child)
 
-proc addDotDependency*(c: PPassContext, n: PNode): PNode =
+proc addDotDependency(c: PPassContext, n: PNode): PNode =
   result = n
   let g = PGen(c)
   let b = Backend(g.graph.backend)
@@ -100,7 +100,10 @@ proc generateDot*(graph: ModuleGraph; project: AbsoluteFile) =
       rope(project.splitFile.name), b.dotGraph],
             changeFileExt(project, "dot"))
 
-proc setupDependPass*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
+when not defined(nimHasSinkInference):
+  {.pragma: nosinks.}
+
+proc myOpen(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext {.nosinks.} =
   var g: PGen
   new(g)
   g.module = module
@@ -109,3 +112,6 @@ proc setupDependPass*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPa
   if graph.backend == nil:
     graph.backend = Backend(dotGraph: "")
   result = g
+
+const gendependPass* = makePass(open = myOpen, process = addDotDependency)
+

compiler/docgen.nim

@@ -7,17 +7,13 @@
 #    distribution, for details about the copyright.
 #
 
-## This is the Nim documentation generator. Cross-references are generated
-## by knowing how the anchors are going to be named.
-##
-## .. importdoc:: ../docgen.md
-##
-## For corresponding users' documentation see [Nim DocGen Tools Guide].
+# This is the documentation generator. Cross-references are generated
+# by knowing how the anchors are going to be named.
 
 import
   ast, strutils, strtabs, algorithm, sequtils, options, msgs, os, idents,
   wordrecg, syntaxes, renderer, lexer,
-  packages/docutils/[rst, rstidx, rstgen, dochelpers],
+  packages/docutils/[rst, rstgen, dochelpers],
   json, xmltree, trees, types,
   typesrenderer, astalgo, lineinfos, intsets,
   pathutils, tables, nimpaths, renderverbatim, osproc, packages
@@ -95,7 +91,7 @@ type
     jEntriesFinal: JsonNode    # final JSON after RST pass 2 and rendering
     types: TStrTable
     sharedState: PRstSharedState
-    standaloneDoc: bool        # is markup (.rst/.md) document?
+    standaloneDoc: bool
     conf*: ConfigRef
     cache*: IdentCache
     exampleCounter: int
@@ -112,16 +108,13 @@ type
 proc add(dest: var ItemPre, rst: PRstNode) = dest.add ItemFragment(isRst: true, rst: rst)
 proc add(dest: var ItemPre, str: string) = dest.add ItemFragment(isRst: false, str: str)
 
-proc addRstFileIndex(d: PDoc, fileIndex: lineinfos.FileIndex): rstast.FileIndex =
-  let invalid = rstast.FileIndex(-1)
-  result = d.nimToRstFid.getOrDefault(fileIndex, default = invalid)
-  if result == invalid:
-    let fname = toFullPath(d.conf, fileIndex)
-    result = addFilename(d.sharedState, fname)
-    d.nimToRstFid[fileIndex] = result
-
 proc addRstFileIndex(d: PDoc, info: lineinfos.TLineInfo): rstast.FileIndex =
-  addRstFileIndex(d, info.fileIndex)
+  let invalid = rstast.FileIndex(-1)
+  result = d.nimToRstFid.getOrDefault(info.fileIndex, default = invalid)
+  if result == invalid:
+    let fname = toFullPath(d.conf, info)
+    result = addFilename(d.sharedState, fname)
+    d.nimToRstFid[info.fileIndex] = result
 
 proc cmpDecimalsIgnoreCase(a, b: string): int =
   ## For sorting with correct handling of cases like 'uint8' and 'uint16'.
@@ -232,7 +225,7 @@ proc attachToType(d: PDoc; p: PSym): PSym =
   if params.len > 0: check(0)
   for i in 2..<params.len: check(i)
 
-template declareClosures(currentFilename: AbsoluteFile, destFile: string) =
+template declareClosures =
   proc compilerMsgHandler(filename: string, line, col: int,
                           msgKind: rst.MsgKind, arg: string) {.gcsafe.} =
     # translate msg kind:
@@ -256,7 +249,6 @@ template declareClosures(currentFilename: AbsoluteFile, destFile: string) =
     of mwBrokenLink: k = warnRstBrokenLink
     of mwUnsupportedLanguage: k = warnRstLanguageXNotSupported
     of mwUnsupportedField: k = warnRstFieldXNotSupported
-    of mwUnusedImportdoc: k = warnRstUnusedImportdoc
     of mwRstStyle: k = warnRstStyle
     {.gcsafe.}:
       globalError(conf, newLineInfo(conf, AbsoluteFile filename, line, col), k, arg)
@@ -267,29 +259,10 @@ template declareClosures(currentFilename: AbsoluteFile, destFile: string) =
       result = getCurrentDir() / s
       if not fileExists(result): result = ""
 
-  proc docgenFindRefFile(targetRelPath: string):
-         tuple[targetPath: string, linkRelPath: string] {.gcsafe.} =
-    let fromDir = splitFile(destFile).dir  # dir where we reference from
-    let basedir = os.splitFile(currentFilename.string).dir
-    let outDirPath: RelativeFile =
-        presentationPath(conf, AbsoluteFile(basedir / targetRelPath))
-          # use presentationPath because `..` path can be be mangled to `_._`
-    result.targetPath = string(conf.outDir / outDirPath)
-    if not fileExists(result.targetPath):
-      # this can happen if targetRelPath goes to parent directory `OUTDIR/..`.
-      # Trying it, this may cause ambiguities, but allows us to insert
-      # "packages" into each other, which is actually used in Nim repo itself.
-      let destPath = fromDir / targetRelPath
-      if destPath != result.targetPath and fileExists(destPath):
-        result.targetPath = destPath
-
-    result.linkRelPath = relativePath(result.targetPath.splitFile.dir,
-                                      fromDir).replace('\\', '/')
-
-
 proc parseRst(text: string,
               line, column: int,
               conf: ConfigRef, sharedState: PRstSharedState): PRstNode =
+  declareClosures()
   result = rstParsePass1(text, line, column, sharedState)
 
 proc getOutFile2(conf: ConfigRef; filename: RelativeFile,
@@ -310,8 +283,7 @@ proc newDocumentor*(filename: AbsoluteFile; cache: IdentCache; conf: ConfigRef,
                     outExt: string = HtmlExt, module: PSym = nil,
                     standaloneDoc = false, preferMarkdown = true,
                     hasToc = true): PDoc =
-  let destFile = getOutFile2(conf, presentationPath(conf, filename), outExt, false).string
-  declareClosures(currentFilename = filename, destFile = destFile)
+  declareClosures()
   new(result)
   result.module = module
   result.conf = conf
@@ -326,7 +298,7 @@ proc newDocumentor*(filename: AbsoluteFile; cache: IdentCache; conf: ConfigRef,
   result.hasToc = hasToc
   result.sharedState = newRstSharedState(
       options, filename.string,
-      docgenFindFile, docgenFindRefFile, compilerMsgHandler, hasToc)
+      docgenFindFile, compilerMsgHandler, hasToc)
   initRstGenerator(result[], (if conf.isLatexCmd: outLatex else: outHtml),
                    conf.configVars, filename.string,
                    docgenFindFile, compilerMsgHandler)
@@ -401,7 +373,7 @@ proc newDocumentor*(filename: AbsoluteFile; cache: IdentCache; conf: ConfigRef,
       if gotten != status:
         rawMessage(conf, errGenerated, "snippet failed: cmd: '$1' status: $2 expected: $3 output: $4" % [cmd, $gotten, $status, output])
   result.emitted = initIntSet()
-  result.destFile = destFile
+  result.destFile = getOutFile2(conf, presentationPath(conf, filename), outExt, false).string
   result.thisDir = result.destFile.AbsoluteFile.splitFile.dir
 
 template dispA(conf: ConfigRef; dest: var string, xml, tex: string,
@@ -586,7 +558,7 @@ proc runAllExamples(d: PDoc) =
 
 proc quoted(a: string): string = result.addQuoted(a)
 
-proc toInstantiationInfo(conf: ConfigRef, info: TLineInfo): (string, int, int) =
+proc toInstantiationInfo(conf: ConfigRef, info: TLineInfo): auto =
   # xxx expose in compiler/lineinfos.nim
   (conf.toMsgFilename(info), info.line.int, info.col.int + ColOffset)
 
@@ -793,24 +765,21 @@ proc isVisible(d: PDoc; n: PNode): bool =
   elif n.kind == nkPragmaExpr:
     result = isVisible(d, n[0])
 
-proc getName(n: PNode): string =
+proc getName(d: PDoc, n: PNode, splitAfter = -1): string =
   case n.kind
-  of nkPostfix: result = getName(n[1])
-  of nkPragmaExpr: result = getName(n[0])
-  of nkSym: result = n.sym.renderDefinitionName
-  of nkIdent: result = n.ident.s
+  of nkPostfix: result = getName(d, n[1], splitAfter)
+  of nkPragmaExpr: result = getName(d, n[0], splitAfter)
+  of nkSym: result = esc(d.target, n.sym.renderDefinitionName, splitAfter)
+  of nkIdent: result = esc(d.target, n.ident.s, splitAfter)
   of nkAccQuoted:
-    result = "`"
-    for i in 0..<n.len: result.add(getName(n[i]))
-    result = "`"
+    result = esc(d.target, "`")
+    for i in 0..<n.len: result.add(getName(d, n[i], splitAfter))
+    result.add esc(d.target, "`")
   of nkOpenSymChoice, nkClosedSymChoice:
-    result = getName(n[0])
+    result = getName(d, n[0], splitAfter)
   else:
     result = ""
 
-proc getNameEsc(d: PDoc, n: PNode): string =
-  esc(d.target, getName(n))
-
 proc getNameIdent(cache: IdentCache; n: PNode): PIdent =
   case n.kind
   of nkPostfix: result = getNameIdent(cache, n[1])
@@ -959,13 +928,6 @@ proc symbolPriority(k: TSymKind): int =
     else: 0  # including skProc which have higher priority
     # documentation itself has even higher priority 1
 
-proc getTypeKind(n: PNode): string =
-  case n[2].kind
-  of nkEnumTy: "enum"
-  of nkObjectTy: "object"
-  of nkTupleTy: "tuple"
-  else: ""
-
 proc toLangSymbol(k: TSymKind, n: PNode, baseName: string): LangSymbol =
   ## Converts symbol info (names/types/parameters) in `n` into format
   ## `LangSymbol` convenient for ``rst.nim``/``dochelpers.nim``.
@@ -1000,7 +962,7 @@ proc toLangSymbol(k: TSymKind, n: PNode, baseName: string): LangSymbol =
       var literal = ""
       var r: TSrcGen
       initTokRender(r, genNode, {renderNoBody, renderNoComments,
-        renderNoPragmas, renderNoProcDefs, renderExpandUsing})
+        renderNoPragmas, renderNoProcDefs})
       var kind = tkEof
       while true:
         getNextTok(r, kind, literal)
@@ -1009,13 +971,17 @@ proc toLangSymbol(k: TSymKind, n: PNode, baseName: string): LangSymbol =
         if kind != tkSpaces:
           result.generics.add(literal.nimIdentNormalize)
 
-  if k == skType: result.symTypeKind = getTypeKind(n)
+  if k == skType:
+    case n[2].kind
+    of nkEnumTy: result.symTypeKind = "enum"
+    of nkObjectTy: result.symTypeKind = "object"
+    of nkTupleTy: result.symTypeKind = "tuple"
+    else: discard
 
-proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind, docFlags: DocFlags, nonExports: bool = false) =
+proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind, docFlags: DocFlags) =
   if (docFlags != kForceExport) and not isVisible(d, nameNode): return
   let
-    name = getName(nameNode)
-    nameEsc = esc(d.target, name)
+    name = getName(d, nameNode)
   var plainDocstring = getPlainDocstring(n) # call here before genRecComment!
   var result = ""
   var literal, plainName = ""
@@ -1029,7 +995,7 @@ proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind, docFlags: DocFlags, nonEx
   var r: TSrcGen
   # Obtain the plain rendered string for hyperlink titles.
   initTokRender(r, n, {renderNoBody, renderNoComments, renderDocComments,
-    renderNoPragmas, renderNoProcDefs, renderExpandUsing})
+    renderNoPragmas, renderNoProcDefs})
   while true:
     getNextTok(r, kind, literal)
     if kind == tkEof:
@@ -1042,12 +1008,9 @@ proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind, docFlags: DocFlags, nonEx
   inc(d.id)
   let
     plainNameEsc = esc(d.target, plainName.strip)
-    typeDescr =
-      if k == skType and getTypeKind(n) != "": getTypeKind(n)
-      else: k.toHumanStr
-    detailedName = typeDescr & " " & (
+    detailedName = k.toHumanStr & " " & (
         if k in routineKinds: plainName else: name)
-    uniqueName = if k in routineKinds: plainNameEsc else: nameEsc
+    uniqueName = if k in routineKinds: plainNameEsc else: name
     sortName = if k in routineKinds: plainName.strip else: name
     cleanPlainSymbol = renderPlainSymbolName(nameNode)
     complexSymbol = complexName(k, n, cleanPlainSymbol)
@@ -1061,16 +1024,11 @@ proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind, docFlags: DocFlags, nonEx
   let lineinfo = rstast.TLineInfo(
       line: nameNode.info.line, col: nameNode.info.col,
       fileIndex: addRstFileIndex(d, nameNode.info))
-  addAnchorNim(d.sharedState, external = false, refn = symbolOrId,
-               tooltip = detailedName, langSym = rstLangSymbol,
-               priority = symbolPriority(k), info = lineinfo,
-               module = addRstFileIndex(d, FileIndex d.module.position))
+  addAnchorNim(d.sharedState, refn = symbolOrId, tooltip = detailedName,
+               rstLangSymbol, priority = symbolPriority(k), info = lineinfo)
 
-  let renderFlags =
-    if nonExports: {renderNoBody, renderNoComments, renderDocComments, renderSyms,
-      renderExpandUsing, renderNonExportedFields}
-    else: {renderNoBody, renderNoComments, renderDocComments, renderSyms, renderExpandUsing}
-  nodeToHighlightedHtml(d, n, result, renderFlags, symbolOrIdEnc)
+  nodeToHighlightedHtml(d, n, result, {renderNoBody, renderNoComments,
+    renderDocComments, renderSyms}, symbolOrIdEnc)
 
   let seeSrc = genSeeSrc(d, toFullPath(d.conf, n.info), n.info.line.int)
 
@@ -1102,10 +1060,8 @@ proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind, docFlags: DocFlags, nonEx
       if e.sym.kind != skEnumField: continue
       let plain = renderPlainSymbolName(e)
       let symbolOrId = d.newUniquePlainSymbol(plain)
-      setIndexTerm(d[], ieNim, htmlFile = external, id = symbolOrId,
-                   term = plain, linkTitle = nameNode.sym.name.s & '.' & plain,
-                   linkDesc = xmltree.escape(getPlainDocstring(e).docstringSummary),
-                   line = n.info.line.int)
+      setIndexTerm(d[], external, symbolOrId, plain, nameNode.sym.name.s & '.' & plain,
+        xmltree.escape(getPlainDocstring(e).docstringSummary))
 
   d.tocSimple[k].add TocItem(
     sortName: sortName,
@@ -1120,20 +1076,25 @@ proc genItem(d: PDoc, n, nameNode: PNode, k: TSymKind, docFlags: DocFlags, nonEx
       "itemSymOrID", symbolOrId.replace(",", ",<wbr>"),
       "itemSymOrIDEnc", symbolOrIdEnc])
 
-  setIndexTerm(d[], ieNim, htmlFile = external, id = symbolOrId, term = name,
-               linkTitle = detailedName,
-               linkDesc = xmltree.escape(plainDocstring.docstringSummary),
-               line = n.info.line.int)
+  # Ironically for types the complexSymbol is *cleaner* than the plainName
+  # because it doesn't include object fields or documentation comments. So we
+  # use the plain one for callable elements, and the complex for the rest.
+  var linkTitle = changeFileExt(extractFilename(d.filename), "") & ": "
+  if n.kind in routineDefs: linkTitle.add(xmltree.escape(plainName.strip))
+  else: linkTitle.add(xmltree.escape(complexSymbol.strip))
+
+  setIndexTerm(d[], external, symbolOrId, name, linkTitle,
+    xmltree.escape(plainDocstring.docstringSummary))
   if k == skType and nameNode.kind == nkSym:
     d.types.strTableAdd nameNode.sym
 
 proc genJsonItem(d: PDoc, n, nameNode: PNode, k: TSymKind): JsonItem =
   if not isVisible(d, nameNode): return
   var
-    name = getNameEsc(d, nameNode)
+    name = getName(d, nameNode)
     comm = genRecComment(d, n)
     r: TSrcGen
-  initTokRender(r, n, {renderNoBody, renderNoComments, renderDocComments, renderExpandUsing})
+  initTokRender(r, n, {renderNoBody, renderNoComments, renderDocComments})
   result.json = %{ "name": %name, "type": %($k), "line": %n.info.line.int,
                    "col": %n.info.col}
   if comm != nil:
@@ -1318,13 +1279,12 @@ proc documentRaises*(cache: IdentCache; n: PNode) =
     if p5 != nil: n[pragmasPos].add p5
     if p6 != nil: n[pragmasPos].add p6
 
-proc generateDoc*(d: PDoc, n, orig: PNode, config: ConfigRef, docFlags: DocFlags = kDefault) =
+proc generateDoc*(d: PDoc, n, orig: PNode, docFlags: DocFlags = kDefault) =
   ## Goes through nim nodes recursively and collects doc comments.
   ## Main function for `doc`:option: command,
   ## which is implemented in ``docgen2.nim``.
   template genItemAux(skind) =
     genItem(d, n, n[namePos], skind, docFlags)
-  let showNonExports = optShowNonExportedFields in config.globalOptions
   case n.kind
   of nkPragma:
     let pragmaNode = findPragma(n, wDeprecated)
@@ -1351,20 +1311,20 @@ proc generateDoc*(d: PDoc, n, orig: PNode, config: ConfigRef, docFlags: DocFlags
       if n[i].kind != nkCommentStmt:
         # order is always 'type var let const':
         genItem(d, n[i], n[i][0],
-                succ(skType, ord(n.kind)-ord(nkTypeSection)), docFlags, showNonExports)
+                succ(skType, ord(n.kind)-ord(nkTypeSection)), docFlags)
   of nkStmtList:
-    for i in 0..<n.len: generateDoc(d, n[i], orig, config)
+    for i in 0..<n.len: generateDoc(d, n[i], orig)
   of nkWhenStmt:
     # generate documentation for the first branch only:
     if not checkForFalse(n[0][0]):
-      generateDoc(d, lastSon(n[0]), orig, config)
+      generateDoc(d, lastSon(n[0]), orig)
   of nkImportStmt:
     for it in n: traceDeps(d, it)
   of nkExportStmt:
     for it in n:
       if it.kind == nkSym:
         if d.module != nil and d.module == it.sym.owner:
-          generateDoc(d, it.sym.ast, orig, config, kForceExport)
+          generateDoc(d, it.sym.ast, orig, kForceExport)
         elif it.sym.ast != nil:
           exportSym(d, it.sym)
   of nkExportExceptStmt: discard "transformed into nkExportStmt by semExportExcept"
@@ -1377,31 +1337,9 @@ proc generateDoc*(d: PDoc, n, orig: PNode, config: ConfigRef, docFlags: DocFlags
 
 proc overloadGroupName(s: string, k: TSymKind): string =
   ## Turns a name like `f` into anchor `f-procs-all`
+  #s & " " & k.toHumanStr & "s all"
   s & "-" & k.toHumanStr & "s-all"
 
-proc setIndexTitle(d: PDoc, useMetaTitle: bool) =
-  let titleKind = if d.standaloneDoc: ieMarkupTitle else: ieNimTitle
-  let external = AbsoluteFile(d.destFile)
-    .relativeTo(d.conf.outDir, '/')
-    .changeFileExt(HtmlExt)
-    .string
-  var term, linkTitle: string
-  if useMetaTitle and d.meta[metaTitle].len != 0:
-    term = d.meta[metaTitleRaw]
-    linkTitle = d.meta[metaTitleRaw]
-  else:
-    let filename = extractFilename(d.filename)
-    term =
-      if d.standaloneDoc: filename  # keep .rst/.md extension
-      else: changeFileExt(filename, "")  # rm .nim extension
-    linkTitle =
-      if d.standaloneDoc: term  # keep .rst/.md extension
-      else: canonicalImport(d.conf, AbsoluteFile d.filename)
-  if not d.standaloneDoc:
-    linkTitle = "module " & linkTitle
-  setIndexTerm(d[], titleKind, htmlFile = external, id = "",
-               term = term, linkTitle = linkTitle)
-
 proc finishGenerateDoc*(d: var PDoc) =
   ## Perform 2nd RST pass for resolution of links/footnotes/headings...
   # copy file map `filenames` to ``rstgen.nim`` for its warnings
@@ -1414,24 +1352,7 @@ proc finishGenerateDoc*(d: var PDoc) =
       firstRst = fragment.rst
       break
   d.hasToc = d.hasToc or d.sharedState.hasToc
-  # in --index:only mode we do NOT want to load other .idx, only write ours:
-  let importdoc = optGenIndexOnly notin d.conf.globalOptions and
-                  optNoImportdoc notin d.conf.globalOptions
-  preparePass2(d.sharedState, firstRst, importdoc)
-
-  if optGenIndexOnly in d.conf.globalOptions:
-    # Top-level doc.comments may contain titles and :idx: statements:
-    for fragment in d.modDescPre:
-      if fragment.isRst:
-        traverseForIndex(d[], fragment.rst)
-    setIndexTitle(d, useMetaTitle = d.standaloneDoc)
-    # Symbol-associated doc.comments may contain :idx: statements:
-    for k in TSymKind:
-      for _, overloadChoices in d.section[k].secItems:
-        for item in overloadChoices:
-          for fragment in item.descRst:
-            if fragment.isRst:
-              traverseForIndex(d[], fragment.rst)
+  preparePass2(d.sharedState, firstRst)
 
   # add anchors to overload groups before RST resolution
   for k in TSymKind:
@@ -1441,25 +1362,13 @@ proc finishGenerateDoc*(d: var PDoc) =
           let refn = overloadGroupName(plainName, k)
           let tooltip = "$1 ($2 overloads)" % [
                       k.toHumanStr & " " & plainName, $overloadChoices.len]
-          let name = nimIdentBackticksNormalize(plainName)
-          # save overload group to ``.idx``
-          let external = d.destFile.AbsoluteFile.relativeTo(d.conf.outDir, '/').
-                         changeFileExt(HtmlExt).string
-          setIndexTerm(d[], ieNimGroup, htmlFile = external, id = refn,
-                       term = name, linkTitle = k.toHumanStr,
-                       linkDesc = "", line = overloadChoices[0].info.line.int)
-          if optGenIndexOnly in d.conf.globalOptions: continue
-          addAnchorNim(d.sharedState, external=false, refn, tooltip,
+          addAnchorNim(d.sharedState, refn, tooltip,
                        LangSymbol(symKind: k.toHumanStr,
-                                  name: name,
+                                  name: nimIdentBackticksNormalize(plainName),
                                   isGroup: true),
                        priority = symbolPriority(k),
                        # select index `0` just to have any meaningful warning:
-                       info = overloadChoices[0].info,
-                       module = addRstFileIndex(d, FileIndex d.module.position))
-
-  if optGenIndexOnly in d.conf.globalOptions:
-    return
+                       info = overloadChoices[0].info)
 
   # Finalize fragments of ``.nim`` or ``.rst`` file
   proc renderItemPre(d: PDoc, fragments: ItemPre, result: var string) =
@@ -1512,9 +1421,6 @@ proc finishGenerateDoc*(d: var PDoc) =
 
     d.jEntriesFinal.add entry.json # generates docs
 
-  setIndexTitle(d, useMetaTitle = d.standaloneDoc)
-  completePass2(d.sharedState)
-
 proc add(d: PDoc; j: JsonItem) =
   if j.json != nil or j.rst != nil: d.jEntriesPre.add j
 
@@ -1561,7 +1467,7 @@ proc generateJson*(d: PDoc, n: PNode, includeComments: bool = true) =
   else: discard
 
 proc genTagsItem(d: PDoc, n, nameNode: PNode, k: TSymKind): string =
-  result = getNameEsc(d, nameNode) & "\n"
+  result = getName(d, nameNode) & "\n"
 
 proc generateTags*(d: PDoc, n: PNode, r: var string) =
   case n.kind
@@ -1668,7 +1574,13 @@ proc genOutFile(d: PDoc, groupedToc = false): string =
   # Extract the title. Non API modules generate an entry in the index table.
   if d.meta[metaTitle].len != 0:
     title = d.meta[metaTitle]
+    let external = AbsoluteFile(d.destFile)
+      .relativeTo(d.conf.outDir, '/')
+      .changeFileExt(HtmlExt)
+      .string
+    setIndexTerm(d[], external, "", title)
   else:
+    # Modules get an automatic title for the HTML, but no entry in the index.
     title = canonicalImport(d.conf, AbsoluteFile d.filename)
   title = esc(d.target, title)
   var subtitle = ""
@@ -1707,17 +1619,11 @@ proc genOutFile(d: PDoc, groupedToc = false): string =
     code = content
   result = code
 
-proc indexFile(d: PDoc): AbsoluteFile =
-  let dir = d.conf.outDir
-  result = dir / changeFileExt(presentationPath(d.conf,
-                                                AbsoluteFile d.filename),
-                               IndexExt)
-  let (finalDir, _, _) = result.string.splitFile
-  createDir(finalDir)
-
 proc generateIndex*(d: PDoc) =
   if optGenIndex in d.conf.globalOptions:
-    let dest = indexFile(d)
+    let dir = d.conf.outDir
+    createDir(dir)
+    let dest = dir / changeFileExt(presentationPath(d.conf, AbsoluteFile d.filename), IndexExt)
     writeIndexFile(d[], dest.string)
 
 proc updateOutfile(d: PDoc, outfile: AbsoluteFile) =
@@ -1728,9 +1634,6 @@ proc updateOutfile(d: PDoc, outfile: AbsoluteFile) =
         d.conf.outFile = splitPath(d.conf.outFile.string)[1].RelativeFile
 
 proc writeOutput*(d: PDoc, useWarning = false, groupedToc = false) =
-  if optGenIndexOnly in d.conf.globalOptions:
-    d.conf.outFile = indexFile(d).relativeTo(d.conf.outDir)  # just for display
-    return
   runAllExamples(d)
   var content = genOutFile(d, groupedToc)
   if optStdout in d.conf.globalOptions:
@@ -1795,7 +1698,7 @@ proc commandDoc*(cache: IdentCache, conf: ConfigRef) =
   var ast = parseFile(conf.projectMainIdx, cache, conf)
   if ast == nil: return
   var d = newDocumentor(conf.projectFull, cache, conf, hasToc = true)
-  generateDoc(d, ast, ast, conf)
+  generateDoc(d, ast, ast)
   finishGenerateDoc(d)
   writeOutput(d)
   generateIndex(d)
@@ -1843,7 +1746,7 @@ proc commandJson*(cache: IdentCache, conf: ConfigRef) =
     let filename = getOutFile(conf, RelativeFile conf.projectName, JsonExt)
     try:
       writeFile(filename, content)
-    except IOError:
+    except:
       rawMessage(conf, errCannotOpenFile, filename.string)
 
 proc commandTags*(cache: IdentCache, conf: ConfigRef) =
@@ -1865,12 +1768,10 @@ proc commandTags*(cache: IdentCache, conf: ConfigRef) =
     let filename = getOutFile(conf, RelativeFile conf.projectName, TagsExt)
     try:
       writeFile(filename, content)
-    except IOError:
+    except:
       rawMessage(conf, errCannotOpenFile, filename.string)
 
 proc commandBuildIndex*(conf: ConfigRef, dir: string, outFile = RelativeFile"") =
-  if optGenIndexOnly in conf.globalOptions:
-    return
   var content = mergeIndexes(dir)
 
   var outFile = outFile
@@ -1888,7 +1789,7 @@ proc commandBuildIndex*(conf: ConfigRef, dir: string, outFile = RelativeFile"")
 
   try:
     writeFile(filename, code)
-  except IOError:
+  except:
     rawMessage(conf, errCannotOpenFile, filename.string)
 
 proc commandBuildIndexJson*(conf: ConfigRef, dir: string, outFile = RelativeFile"") =
@@ -1902,5 +1803,5 @@ proc commandBuildIndexJson*(conf: ConfigRef, dir: string, outFile = RelativeFile
 
   try:
     writeFile(filename, $body)
-  except IOError:
+  except:
     rawMessage(conf, errCannotOpenFile, filename.string)

compiler/docgen2.nim

@@ -11,7 +11,7 @@
 # semantic checking.
 
 import
-  options, ast, msgs, docgen, lineinfos, pathutils, packages
+  options, ast, msgs, passes, docgen, lineinfos, pathutils, packages
 
 from modulegraphs import ModuleGraph, PPassContext
 
@@ -38,21 +38,21 @@ template closeImpl(body: untyped) {.dirty.} =
     except IOError:
       discard
 
-proc closeDoc*(graph: ModuleGraph; p: PPassContext, n: PNode): PNode =
+proc close(graph: ModuleGraph; p: PPassContext, n: PNode): PNode =
   closeImpl:
     writeOutput(g.doc, useWarning, groupedToc)
 
-proc closeJson*(graph: ModuleGraph; p: PPassContext, n: PNode): PNode =
+proc closeJson(graph: ModuleGraph; p: PPassContext, n: PNode): PNode =
   closeImpl:
     writeOutputJson(g.doc, useWarning)
 
-proc processNode*(c: PPassContext, n: PNode): PNode =
+proc processNode(c: PPassContext, n: PNode): PNode =
   result = n
   var g = PGen(c)
   if shouldProcess(g):
-    generateDoc(g.doc, n, n, g.config)
+    generateDoc(g.doc, n, n)
 
-proc processNodeJson*(c: PPassContext, n: PNode): PNode =
+proc processNodeJson(c: PPassContext, n: PNode): PNode =
   result = n
   var g = PGen(c)
   if shouldProcess(g):
@@ -68,11 +68,20 @@ template myOpenImpl(ext: untyped) {.dirty.} =
   g.doc = d
   result = g
 
-proc openHtml*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
+proc myOpen(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
   myOpenImpl(HtmlExt)
 
-proc openTex*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
+proc myOpenTex(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
   myOpenImpl(TexExt)
 
-proc openJson*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
+proc myOpenJson(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
   myOpenImpl(JsonExt)
+
+const docgen2Pass* = makePass(open = myOpen, process = processNode, close = close)
+const docgen2TexPass* = makePass(open = myOpenTex, process = processNode,
+                                 close = close)
+const docgen2JsonPass* = makePass(open = myOpenJson, process = processNodeJson,
+                                  close = closeJson)
+
+proc finishDoc2Pass*(project: string) =
+  discard

compiler/evalffi.nim

@@ -77,7 +77,7 @@ proc importcSymbol*(conf: ConfigRef, sym: PSym): PNode =
       theAddr = dllhandle.symAddr(name.cstring)
     if theAddr.isNil: globalError(conf, sym.info,
       "cannot import symbol: " & name & " from " & libPathMsg)
-    result.intVal = cast[int](theAddr)
+    result.intVal = cast[ByteAddress](theAddr)
 
 proc mapType(conf: ConfigRef, t: ast.PType): ptr libffi.Type =
   if t == nil: return addr libffi.type_void
@@ -113,7 +113,7 @@ proc mapCallConv(conf: ConfigRef, cc: TCallingConvention, info: TLineInfo): TABI
 template rd(typ, p: untyped): untyped = (cast[ptr typ](p))[]
 template wr(typ, p, v: untyped): untyped = (cast[ptr typ](p))[] = v
 template `+!`(x, y: untyped): untyped =
-  cast[pointer](cast[int](x) + y)
+  cast[pointer](cast[ByteAddress](x) + y)
 
 proc packSize(conf: ConfigRef, v: PNode, typ: PType): int =
   ## computes the size of the blob
@@ -369,13 +369,13 @@ proc unpack(conf: ConfigRef, x: pointer, typ: PType, n: PNode): PNode =
       # in their unboxed representation so nothing it to be unpacked:
       result = n
     else:
-      awi(nkPtrLit, cast[int](p))
+      awi(nkPtrLit, cast[ByteAddress](p))
   of tyPtr, tyRef, tyVar, tyLent:
     let p = rd(pointer, x)
     if p.isNil:
       setNil()
     elif n == nil or n.kind == nkPtrLit:
-      awi(nkPtrLit, cast[int](p))
+      awi(nkPtrLit, cast[ByteAddress](p))
     elif n != nil and n.len == 1:
       internalAssert(conf, n.kind == nkRefTy)
       n[0] = unpack(conf, p, typ.lastSon, n[0])

compiler/extccomp.nim

@@ -328,9 +328,7 @@ proc getConfigVar(conf: ConfigRef; c: TSystemCC, suffix: string): string =
                      platform.OS[conf.target.targetOS].name & '.' &
                      CC[c].name & fullSuffix
     result = getConfigVar(conf, fullCCname)
-    if existsConfigVar(conf, fullCCname):
-      result = getConfigVar(conf, fullCCname)
-    else:
+    if result.len == 0:
       # not overridden for this cross compilation setting?
       result = getConfigVar(conf, CC[c].name & fullSuffix)
   else:
@@ -530,7 +528,7 @@ proc ccHasSaneOverflow*(conf: ConfigRef): bool =
     var exe = getConfigVar(conf, conf.cCompiler, ".exe")
     if exe.len == 0: exe = CC[conf.cCompiler].compilerExe
     # NOTE: should we need the full version, use -dumpfullversion
-    let (s, exitCode) = try: execCmdEx(exe & " -dumpversion") except IOError, OSError, ValueError: ("", 1)
+    let (s, exitCode) = try: execCmdEx(exe & " -dumpversion") except: ("", 1)
     if exitCode == 0:
       var major: int
       discard parseInt(s, major)
@@ -836,15 +834,6 @@ proc linkViaResponseFile(conf: ConfigRef; cmd: string) =
   finally:
     removeFile(linkerArgs)
 
-proc linkViaShellScript(conf: ConfigRef; cmd: string) =
-  let linkerScript = conf.projectName & "_" & "linkerScript.sh"
-  writeFile(linkerScript, cmd)
-  let shell = getEnv("SHELL")
-  try:
-    execLinkCmd(conf, shell & " " & linkerScript)
-  finally:
-    removeFile(linkerScript)
-
 proc getObjFilePath(conf: ConfigRef, f: Cfile): string =
   if noAbsolutePaths(conf): f.obj.extractFilename
   else: f.obj.string
@@ -862,20 +851,6 @@ proc displayProgressCC(conf: ConfigRef, path, compileCmd: string): string =
     else:
       result = MsgKindToStr[hintCC] % demangleModuleName(path.splitFile.name)
 
-proc preventLinkCmdMaxCmdLen(conf: ConfigRef, linkCmd: string) =
-  # Prevent linkcmd from exceeding the maximum command line length.
-  # Windows's command line limit is about 8K (8191 characters) so C compilers on
-  # Windows support a feature where the command line can be passed via ``@linkcmd``
-  # to them.
-  const MaxCmdLen = when defined(windows): 8_000 elif defined(macosx): 260_000 else: 32_000
-  if linkCmd.len > MaxCmdLen:
-    when defined(macosx):
-      linkViaShellScript(conf, linkCmd)
-    else:
-      linkViaResponseFile(conf, linkCmd)
-  else:
-    execLinkCmd(conf, linkCmd)
-
 proc callCCompiler*(conf: ConfigRef) =
   var
     linkCmd: string
@@ -952,7 +927,14 @@ proc callCCompiler*(conf: ConfigRef) =
       linkCmd = getLinkCmd(conf, mainOutput, objfiles, removeStaticFile = true)
       extraCmds = getExtraCmds(conf, mainOutput)
       if optCompileOnly notin conf.globalOptions:
-        preventLinkCmdMaxCmdLen(conf, linkCmd)
+        const MaxCmdLen = when defined(windows): 8_000 else: 32_000
+        if linkCmd.len > MaxCmdLen:
+          # Windows's command line limit is about 8K (don't laugh...) so C compilers on
+          # Windows support a feature where the command line can be passed via ``@linkcmd``
+          # to them.
+          linkViaResponseFile(conf, linkCmd)
+        else:
+          execLinkCmd(conf, linkCmd)
         for cmd in extraCmds:
           execExternalProgram(conf, cmd, hintExecuting)
   else:
@@ -1036,7 +1018,7 @@ proc changeDetectedViaJsonBuildInstructions*(conf: ConfigRef; jsonFile: Absolute
 proc runJsonBuildInstructions*(conf: ConfigRef; jsonFile: AbsoluteFile) =
   var bcache: BuildCache
   try: bcache.fromJson(jsonFile.string.parseFile)
-  except ValueError, KeyError, JsonKindError:
+  except:
     let e = getCurrentException()
     conf.quitOrRaise "\ncaught exception:\n$#\nstacktrace:\n$#error evaluating JSON file: $#" %
       [e.msg, e.getStackTrace(), jsonFile.string]
@@ -1053,7 +1035,7 @@ proc runJsonBuildInstructions*(conf: ConfigRef; jsonFile: AbsoluteFile) =
     cmds.add cmd
     prettyCmds.add displayProgressCC(conf, name, cmd)
   execCmdsInParallel(conf, cmds, prettyCb)
-  preventLinkCmdMaxCmdLen(conf, bcache.linkcmd)
+  execLinkCmd(conf, bcache.linkcmd)
   for cmd in bcache.extraCmds: execExternalProgram(conf, cmd, hintExecuting)
 
 proc genMappingFiles(conf: ConfigRef; list: CfileList): Rope =

compiler/ic/cbackend.nim

@@ -50,10 +50,7 @@ proc generateCodeForModule(g: ModuleGraph; m: var LoadedModule; alive: var Alive
     let n = unpackTree(g, m.module.position, m.fromDisk.topLevel, p)
     cgen.genTopLevelStmt(bmod, n)
 
-  let disps = finalCodegenActions(g, bmod, newNodeI(nkStmtList, m.module.info))
-  if disps != nil:
-    for disp in disps:
-      genProcAux(bmod, disp.sym)
+  finalCodegenActions(g, bmod, newNodeI(nkStmtList, m.module.info))
   m.fromDisk.backendFlags = cgen.whichInitProcs(bmod)
 
 proc replayTypeInfo(g: ModuleGraph; m: var LoadedModule; origin: FileIndex) =

compiler/ic/rodfiles.nim

@@ -24,7 +24,7 @@ when defined(nimPreviewSlimSystem):
 ## `RodFile` represents a Rod File (versioned binary format), and the
 ## associated data for common interactions such as IO and error tracking
 ## (`RodFileError`). The file format broken up into sections (`RodSection`)
-## and preceded by a header (see: `cookie`). The precise layout, section
+## and preceeded by a header (see: `cookie`). The precise layout, section
 ## ordering and data following the section are determined by the user. See
 ## `ic.loadRodFile`.
 ##

compiler/injectdestructors.nim

@@ -17,12 +17,12 @@ import
   intsets, strtabs, ast, astalgo, msgs, renderer, magicsys, types, idents,
   strutils, options, lowerings, tables, modulegraphs,
   lineinfos, parampatterns, sighashes, liftdestructors, optimizer,
-  varpartitions, aliasanalysis, dfa, wordrecg
+  varpartitions, aliasanalysis, dfa
 
 when defined(nimPreviewSlimSystem):
   import std/assertions
 
-from trees import exprStructuralEquivalent, getRoot, whichPragma
+from trees import exprStructuralEquivalent, getRoot
 
 type
   Con = object
@@ -35,7 +35,6 @@ type
     idgen: IdGenerator
     body: PNode
     otherUsage: TLineInfo
-    inUncheckedAssignSection: int
 
   Scope = object # we do scope-based memory management.
     # a scope is comparable to an nkStmtListExpr like
@@ -218,7 +217,7 @@ proc genOp(c: var Con; t: PType; kind: TTypeAttachedOp; dest, ri: PNode): PNode
   var op = getAttachedOp(c.graph, t, kind)
   if op == nil or op.ast.isGenericRoutine:
     # give up and find the canonical type instead:
-    let h = sighashes.hashType(t, c.graph.config, {CoType, CoConsiderOwned, CoDistinct})
+    let h = sighashes.hashType(t, {CoType, CoConsiderOwned, CoDistinct})
     let canon = c.graph.canonTypes.getOrDefault(h)
     if canon != nil:
       op = getAttachedOp(c.graph, canon, kind)
@@ -249,17 +248,7 @@ proc canBeMoved(c: Con; t: PType): bool {.inline.} =
 proc isNoInit(dest: PNode): bool {.inline.} =
   result = dest.kind == nkSym and sfNoInit in dest.sym.flags
 
-proc deepAliases(dest, ri: PNode): bool =
-  case ri.kind
-  of nkCallKinds, nkStmtListExpr, nkBracket, nkTupleConstr, nkObjConstr,
-     nkCast, nkConv, nkObjUpConv, nkObjDownConv:
-    for r in ri:
-      if deepAliases(dest, r): return true
-    return false
-  else:
-    return aliases(dest, ri) != no
-
-proc genSink(c: var Con; s: var Scope; dest, ri: PNode; flags: set[MoveOrCopyFlag] = {}): PNode =
+proc genSink(c: var Con; dest, ri: PNode; flags: set[MoveOrCopyFlag] = {}): PNode =
   if (c.inLoopCond == 0 and (isUnpackedTuple(dest) or IsDecl in flags or
       (isAnalysableFieldAccess(dest, c.owner) and isFirstWrite(dest, c)))) or
       isNoInit(dest):
@@ -273,14 +262,7 @@ proc genSink(c: var Con; s: var Scope; dest, ri: PNode; flags: set[MoveOrCopyFla
     else:
       # the default is to use combination of `=destroy(dest)` and
       # and copyMem(dest, source). This is efficient.
-      if deepAliases(dest, ri):
-        # consider: x = x + y, it is wrong to destroy the destination first!
-        # tmp to support self assignments
-        let tmp = c.getTemp(s, dest.typ, dest.info)
-        result = newTree(nkStmtList, newTree(nkFastAsgn, tmp, dest), newTree(nkFastAsgn, dest, ri),
-                         c.genDestroy(tmp))
-      else:
-        result = newTree(nkStmtList, c.genDestroy(dest), newTree(nkFastAsgn, dest, ri))
+      result = newTree(nkStmtList, c.genDestroy(dest), newTree(nkFastAsgn, dest, ri))
 
 proc isCriticalLink(dest: PNode): bool {.inline.} =
   #[
@@ -360,31 +342,23 @@ It is best to factor out piece of object that needs custom destructor into separ
       return
 
     # generate: if le != tmp: `=destroy`(le)
-    if c.inUncheckedAssignSection != 0:
-      let branchDestructor = produceDestructorForDiscriminator(c.graph, objType, leDotExpr[1].sym, n.info, c.idgen)
-      let cond = newNodeIT(nkInfix, n.info, getSysType(c.graph, unknownLineInfo, tyBool))
-      cond.add newSymNode(getMagicEqSymForType(c.graph, le.typ, n.info))
-      cond.add le
-      cond.add tmp
-      let notExpr = newNodeIT(nkPrefix, n.info, getSysType(c.graph, unknownLineInfo, tyBool))
-      notExpr.add newSymNode(createMagic(c.graph, c.idgen, "not", mNot))
-      notExpr.add cond
-      result.add newTree(nkIfStmt, newTree(nkElifBranch, notExpr, c.genOp(branchDestructor, le)))
+    let branchDestructor = produceDestructorForDiscriminator(c.graph, objType, leDotExpr[1].sym, n.info, c.idgen)
+    let cond = newNodeIT(nkInfix, n.info, getSysType(c.graph, unknownLineInfo, tyBool))
+    cond.add newSymNode(getMagicEqSymForType(c.graph, le.typ, n.info))
+    cond.add le
+    cond.add tmp
+    let notExpr = newNodeIT(nkPrefix, n.info, getSysType(c.graph, unknownLineInfo, tyBool))
+    notExpr.add newSymNode(createMagic(c.graph, c.idgen, "not", mNot))
+    notExpr.add cond
+    result.add newTree(nkIfStmt, newTree(nkElifBranch, notExpr, c.genOp(branchDestructor, le)))
   result.add newTree(nkFastAsgn, le, tmp)
 
 proc genWasMoved(c: var Con, n: PNode): PNode =
-  let typ = n.typ.skipTypes({tyGenericInst, tyAlias, tySink})
-  let op = getAttachedOp(c.graph, n.typ, attachedWasMoved)
-  if op != nil:
-    if sfError in op.flags:
-      c.checkForErrorPragma(n.typ, n, "=wasMoved")
-    result = genOp(c, op, n)
-  else:
-    result = newNodeI(nkCall, n.info)
-    result.add(newSymNode(createMagic(c.graph, c.idgen, "wasMoved", mWasMoved)))
-    result.add copyTree(n) #mWasMoved does not take the address
-    #if n.kind != nkSym:
-    #  message(c.graph.config, n.info, warnUser, "wasMoved(" & $n & ")")
+  result = newNodeI(nkCall, n.info)
+  result.add(newSymNode(createMagic(c.graph, c.idgen, "wasMoved", mWasMoved)))
+  result.add copyTree(n) #mWasMoved does not take the address
+  #if n.kind != nkSym:
+  #  message(c.graph.config, n.info, warnUser, "wasMoved(" & $n & ")")
 
 proc genDefaultCall(t: PType; c: Con; info: TLineInfo): PNode =
   result = newNodeI(nkCall, info)
@@ -471,7 +445,7 @@ proc ensureDestruction(arg, orig: PNode; c: var Con; s: var Scope): PNode =
     # This was already done in the sink parameter handling logic.
     result = newNodeIT(nkStmtListExpr, arg.info, arg.typ)
     let tmp = c.getTemp(s, arg.typ, arg.info)
-    result.add c.genSink(s, tmp, arg, {IsDecl})
+    result.add c.genSink(tmp, arg, {IsDecl})
     result.add tmp
     s.final.add c.genDestroy(tmp)
   else:
@@ -855,10 +829,7 @@ proc p(n: PNode; c: var Con; s: var Scope; mode: ProcessMode; tmpFlags = {sfSing
             if ri.kind != nkEmpty:
               result.add moveOrCopy(v, ri, c, s, if v.kind == nkSym: {IsDecl} else: {})
             elif ri.kind == nkEmpty and c.inLoop > 0:
-              let skipInit = v.kind == nkDotExpr and # Closure var
-                             sfNoInit in v[1].sym.flags
-              if not skipInit:
-                result.add moveOrCopy(v, genDefaultCall(v.typ, c, v.info), c, s, if v.kind == nkSym: {IsDecl} else: {})
+              result.add moveOrCopy(v, genDefaultCall(v.typ, c, v.info), c, s, if v.kind == nkSym: {IsDecl} else: {})
         else: # keep the var but transform 'ri':
           var v = copyNode(n)
           var itCopy = copyNode(it)
@@ -896,7 +867,7 @@ proc p(n: PNode; c: var Con; s: var Scope; mode: ProcessMode; tmpFlags = {sfSing
        nkTypeOfExpr, nkMixinStmt, nkBindStmt:
       result = n
 
-    of nkStringToCString, nkCStringToString, nkChckRangeF, nkChckRange64, nkChckRange:
+    of nkStringToCString, nkCStringToString, nkChckRangeF, nkChckRange64, nkChckRange, nkPragmaBlock:
       result = shallowCopy(n)
       for i in 0 ..< n.len:
         result[i] = p(n[i], c, s, normal)
@@ -904,25 +875,6 @@ proc p(n: PNode; c: var Con; s: var Scope; mode: ProcessMode; tmpFlags = {sfSing
         if mode == normal:
           result = ensureDestruction(result, n, c, s)
 
-    of nkPragmaBlock:
-      var inUncheckedAssignSection = 0
-      let pragmaList = n[0]
-      for pi in pragmaList:
-        if whichPragma(pi) == wCast:
-          case whichPragma(pi[1])
-          of wUncheckedAssign:
-            inUncheckedAssignSection = 1
-          else:
-            discard
-      result = shallowCopy(n)
-      inc c.inUncheckedAssignSection, inUncheckedAssignSection
-      for i in 0 ..< n.len:
-        result[i] = p(n[i], c, s, normal)
-      dec c.inUncheckedAssignSection, inUncheckedAssignSection
-      if n.typ != nil and hasDestructor(c, n.typ):
-        if mode == normal:
-          result = ensureDestruction(result, n, c, s)
-
     of nkHiddenSubConv, nkHiddenStdConv, nkConv:
       # we have an "ownership invariance" for all constructors C(x).
       # See the comment for nkBracket construction. If the caller wants
@@ -1021,7 +973,7 @@ proc sameLocation*(a, b: PNode): bool =
     of nkHiddenStdConv, nkHiddenSubConv: sameLocation(a[1], b)
     else: false
 
-proc genFieldAccessSideEffects(c: var Con; s: var Scope; dest, ri: PNode; flags: set[MoveOrCopyFlag] = {}): PNode =
+proc genFieldAccessSideEffects(c: var Con; dest, ri: PNode; flags: set[MoveOrCopyFlag] = {}): PNode =
   # with side effects
   var temp = newSym(skLet, getIdent(c.graph.cache, "bracketTmp"), nextSymId c.idgen, c.owner, ri[1].info)
   temp.typ = ri[1].typ
@@ -1038,7 +990,7 @@ proc genFieldAccessSideEffects(c: var Con; s: var Scope; dest, ri: PNode; flags:
   newAccess.add ri[0]
   newAccess.add tempAsNode
 
-  var snk = c.genSink(s, dest, newAccess, flags)
+  var snk = c.genSink(dest, newAccess, flags)
   result = newTree(nkStmtList, v, snk, c.genWasMoved(newAccess))
 
 proc moveOrCopy(dest, ri: PNode; c: var Con; s: var Scope, flags: set[MoveOrCopyFlag] = {}): PNode =
@@ -1056,21 +1008,21 @@ proc moveOrCopy(dest, ri: PNode; c: var Con; s: var Scope, flags: set[MoveOrCopy
   else:
     case ri.kind
     of nkCallKinds:
-      result = c.genSink(s, dest, p(ri, c, s, consumed), flags)
+      result = c.genSink(dest, p(ri, c, s, consumed), flags)
     of nkBracketExpr:
       if isUnpackedTuple(ri[0]):
         # unpacking of tuple: take over the elements
-        result = c.genSink(s, dest, p(ri, c, s, consumed), flags)
+        result = c.genSink(dest, p(ri, c, s, consumed), flags)
       elif isAnalysableFieldAccess(ri, c.owner) and isLastRead(ri, c, s):
         if aliases(dest, ri) == no:
           # Rule 3: `=sink`(x, z); wasMoved(z)
           if isAtom(ri[1]):
-            var snk = c.genSink(s, dest, ri, flags)
+            var snk = c.genSink(dest, ri, flags)
             result = newTree(nkStmtList, snk, c.genWasMoved(ri))
           else:
-            result = genFieldAccessSideEffects(c, s, dest, ri, flags)
+            result = genFieldAccessSideEffects(c, dest, ri, flags)
         else:
-          result = c.genSink(s, dest, destructiveMoveVar(ri, c, s), flags)
+          result = c.genSink(dest, destructiveMoveVar(ri, c, s), flags)
       else:
         result = c.genCopy(dest, ri, flags)
         result.add p(ri, c, s, consumed)
@@ -1082,25 +1034,25 @@ proc moveOrCopy(dest, ri: PNode; c: var Con; s: var Scope, flags: set[MoveOrCopy
         result.add p(ri, c, s, consumed)
         c.finishCopy(result, dest, isFromSink = false)
       else:
-        result = c.genSink(s, dest, p(ri, c, s, consumed), flags)
+        result = c.genSink(dest, p(ri, c, s, consumed), flags)
     of nkObjConstr, nkTupleConstr, nkClosure, nkCharLit..nkNilLit:
-      result = c.genSink(s, dest, p(ri, c, s, consumed), flags)
+      result = c.genSink(dest, p(ri, c, s, consumed), flags)
     of nkSym:
       if isSinkParam(ri.sym) and isLastRead(ri, c, s):
         # Rule 3: `=sink`(x, z); wasMoved(z)
-        let snk = c.genSink(s, dest, ri, flags)
+        let snk = c.genSink(dest, ri, flags)
         result = newTree(nkStmtList, snk, c.genWasMoved(ri))
       elif ri.sym.kind != skParam and ri.sym.owner == c.owner and
           isLastRead(ri, c, s) and canBeMoved(c, dest.typ) and not isCursor(ri):
         # Rule 3: `=sink`(x, z); wasMoved(z)
-        let snk = c.genSink(s, dest, ri, flags)
+        let snk = c.genSink(dest, ri, flags)
         result = newTree(nkStmtList, snk, c.genWasMoved(ri))
       else:
         result = c.genCopy(dest, ri, flags)
         result.add p(ri, c, s, consumed)
         c.finishCopy(result, dest, isFromSink = false)
     of nkHiddenSubConv, nkHiddenStdConv, nkConv, nkObjDownConv, nkObjUpConv, nkCast:
-      result = c.genSink(s, dest, p(ri, c, s, sinkArg), flags)
+      result = c.genSink(dest, p(ri, c, s, sinkArg), flags)
     of nkStmtListExpr, nkBlockExpr, nkIfExpr, nkCaseStmt, nkTryStmt:
       template process(child, s): untyped = moveOrCopy(dest, child, c, s, flags)
       # We know the result will be a stmt so we use that fact to optimize
@@ -1111,7 +1063,7 @@ proc moveOrCopy(dest, ri: PNode; c: var Con; s: var Scope, flags: set[MoveOrCopy
       if isAnalysableFieldAccess(ri, c.owner) and isLastRead(ri, c, s) and
           canBeMoved(c, dest.typ):
         # Rule 3: `=sink`(x, z); wasMoved(z)
-        let snk = c.genSink(s, dest, ri, flags)
+        let snk = c.genSink(dest, ri, flags)
         result = newTree(nkStmtList, snk, c.genWasMoved(ri))
       else:
         result = c.genCopy(dest, ri, flags)

compiler/installer.ini

@@ -65,12 +65,11 @@ Files: "compiler"
 Files: "doc"
 Files: "doc/html"
 Files: "tools"
-Files: "tools/debug/nim-gdb.py"
+Files: "tools/nim-gdb.py"
 Files: "nimpretty"
 Files: "testament"
 Files: "nimsuggest"
 Files: "nimsuggest/tests/*.nim"
-Files: "changelogs/*.md"
 
 [Lib]
 Files: "lib"
@@ -121,7 +120,6 @@ Files: "bin/nim"
 InstallScript: "yes"
 UninstallScript: "yes"
 Files: "bin/nim-gdb"
-Files: "build_all.sh"
 
 
 [InnoSetup]

compiler/int128.nim

@@ -33,27 +33,26 @@ template high*(t: typedesc[Int128]): Int128 = Max
 proc `$`*(a: Int128): string
 
 proc toInt128*[T: SomeInteger | bool](arg: T): Int128 =
-  {.noSideEffect.}:
-    when T is bool: result.sdata(0) = int32(arg)
-    elif T is SomeUnsignedInt:
-      when sizeof(arg) <= 4:
-        result.udata[0] = uint32(arg)
-      else:
-        result.udata[0] = uint32(arg and T(0xffffffff))
-        result.udata[1] = uint32(arg shr 32)
-    elif sizeof(arg) <= 4:
-      result.sdata(0) = int32(arg)
-      if arg < 0: # sign extend
-        result.sdata(1) = -1
-        result.sdata(2) = -1
-        result.sdata(3) = -1
+  when T is bool: result.sdata(0) = int32(arg)
+  elif T is SomeUnsignedInt:
+    when sizeof(arg) <= 4:
+      result.udata[0] = uint32(arg)
     else:
-      let tmp = int64(arg)
-      result.udata[0] = uint32(tmp and 0xffffffff)
-      result.sdata(1) = int32(tmp shr 32)
-      if arg < 0: # sign extend
-        result.sdata(2) = -1
-        result.sdata(3) = -1
+      result.udata[0] = uint32(arg and T(0xffffffff))
+      result.udata[1] = uint32(arg shr 32)
+  elif sizeof(arg) <= 4:
+    result.sdata(0) = int32(arg)
+    if arg < 0: # sign extend
+      result.sdata(1) = -1
+      result.sdata(2) = -1
+      result.sdata(3) = -1
+  else:
+    let tmp = int64(arg)
+    result.udata[0] = uint32(tmp and 0xffffffff)
+    result.sdata(1) = int32(tmp shr 32)
+    if arg < 0: # sign extend
+      result.sdata(2) = -1
+      result.sdata(3) = -1
 
 template isNegative(arg: Int128): bool =
   arg.sdata(3) < 0

compiler/jsgen.nim

@@ -31,11 +31,9 @@ implements the required case distinction.
 import
   ast, trees, magicsys, options,
   nversion, msgs, idents, types,
-  ropes, ccgutils, wordrecg, renderer,
+  ropes, passes, ccgutils, wordrecg, renderer,
   cgmeth, lowerings, sighashes, modulegraphs, lineinfos, rodutils,
-  transf, injectdestructors, sourcemap, astmsgs, backendpragmas
-
-import pipelineutils
+  transf, injectdestructors, sourcemap, astmsgs
 
 import json, sets, math, tables, intsets
 import strutils except addf
@@ -100,7 +98,6 @@ type
     prc: PSym
     globals, locals, body: Rope
     options: TOptions
-    optionsStack: seq[TOptions]
     module: BModule
     g: PGlobals
     generatedParamCopies: IntSet
@@ -264,7 +261,7 @@ proc mangleName(m: BModule, s: PSym): Rope =
       if m.config.hcrOn:
         # When hot reloading is enabled, we must ensure that the names
         # of functions and types will be preserved across rebuilds:
-        result.add(idOrSig(s, m.module.name.s, m.sigConflicts, m.config))
+        result.add(idOrSig(s, m.module.name.s, m.sigConflicts))
       else:
         result.add("_")
         result.add(rope(s.id))
@@ -571,20 +568,12 @@ proc binaryUintExpr(p: PProc, n: PNode, r: var TCompRes, op: string,
   var x, y: TCompRes
   gen(p, n[1], x)
   gen(p, n[2], y)
-  let size = n[1].typ.skipTypes(abstractRange).size
+  let trimmer = unsignedTrimmer(n[1].typ.skipTypes(abstractRange).size)
   when reassign:
     let (a, tmp) = maybeMakeTempAssignable(p, n[1], x)
-    if size == 8 and optJsBigInt64 in p.config.globalOptions:
-      r.res = "$1 = BigInt.asUintN(64, ($4 $2 $3))" % [a, rope op, y.rdLoc, tmp]
-    else:
-      let trimmer = unsignedTrimmer(size)
-      r.res = "$1 = (($5 $2 $3) $4)" % [a, rope op, y.rdLoc, trimmer, tmp]
+    r.res = "$1 = (($5 $2 $3) $4)" % [a, rope op, y.rdLoc, trimmer, tmp]
   else:
-    if size == 8 and optJsBigInt64 in p.config.globalOptions:
-      r.res = "BigInt.asUintN(64, ($1 $2 $3))" % [x.rdLoc, rope op, y.rdLoc]
-    else:
-      let trimmer = unsignedTrimmer(size)
-      r.res = "(($1 $2 $3) $4)" % [x.rdLoc, rope op, y.rdLoc, trimmer]
+    r.res = "(($1 $2 $3) $4)" % [x.rdLoc, rope op, y.rdLoc, trimmer]
   r.kind = resExpr
 
 template ternaryExpr(p: PProc, n: PNode, r: var TCompRes, magic, frmt: string) =
@@ -626,45 +615,11 @@ proc arithAux(p: PProc, n: PNode, r: var TCompRes, op: TMagic) =
     if i == 0: applyFormat(frmtA) else: applyFormat(frmtB)
 
   case op
-  of mAddI:
-    if i == 0:
-      if n[1].typ.size == 8 and optJsBigInt64 in p.config.globalOptions:
-        useMagic(p, "addInt64")
-        applyFormat("addInt64($1, $2)")
-      else:
-        applyFormat("addInt($1, $2)")
-    else:
-      applyFormat("($1 + $2)")
-  of mSubI:
-    if i == 0:
-      if n[1].typ.size == 8 and optJsBigInt64 in p.config.globalOptions:
-        useMagic(p, "subInt64")
-        applyFormat("subInt64($1, $2)")
-      else:
-        applyFormat("subInt($1, $2)")
-    else:
-      applyFormat("($1 - $2)")
-  of mMulI:
-    if i == 0:
-      if n[1].typ.size == 8 and optJsBigInt64 in p.config.globalOptions:
-        useMagic(p, "mulInt64")
-        applyFormat("mulInt64($1, $2)")
-      else:
-        applyFormat("mulInt($1, $2)")
-    else:
-      applyFormat("($1 * $2)")
-  of mDivI:
-    if n[1].typ.size == 8 and optJsBigInt64 in p.config.globalOptions:
-      useMagic(p, "divInt64")
-      applyFormat("divInt64($1, $2)", "$1 / $2")
-    else:
-      applyFormat("divInt($1, $2)", "Math.trunc($1 / $2)")
-  of mModI:
-    if n[1].typ.size == 8 and optJsBigInt64 in p.config.globalOptions:
-      useMagic(p, "modInt64")
-      applyFormat("modInt64($1, $2)", "$1 % $2")
-    else:
-      applyFormat("modInt($1, $2)", "Math.trunc($1 % $2)")
+  of mAddI: applyFormat("addInt($1, $2)", "($1 + $2)")
+  of mSubI: applyFormat("subInt($1, $2)", "($1 - $2)")
+  of mMulI: applyFormat("mulInt($1, $2)", "($1 * $2)")
+  of mDivI: applyFormat("divInt($1, $2)", "Math.trunc($1 / $2)")
+  of mModI: applyFormat("modInt($1, $2)", "Math.trunc($1 % $2)")
   of mSucc: applyFormat("addInt($1, $2)", "($1 + $2)")
   of mPred: applyFormat("subInt($1, $2)", "($1 - $2)")
   of mAddF64: applyFormat("($1 + $2)", "($1 + $2)")
@@ -673,27 +628,15 @@ proc arithAux(p: PProc, n: PNode, r: var TCompRes, op: TMagic) =
   of mDivF64: applyFormat("($1 / $2)", "($1 / $2)")
   of mShrI: applyFormat("", "")
   of mShlI:
-    let typ = n[1].typ.skipTypes(abstractVarRange)
-    if typ.size == 8:
-      if typ.kind == tyInt64 and optJsBigInt64 in p.config.globalOptions:
-        applyFormat("BigInt.asIntN(64, $1 << BigInt($2))")
-      elif typ.kind == tyUInt64 and optJsBigInt64 in p.config.globalOptions:
-        applyFormat("BigInt.asUintN(64, $1 << BigInt($2))")
-      else:
-        applyFormat("($1 * Math.pow(2, $2))")
-    else:
+    if n[1].typ.size <= 4:
       applyFormat("($1 << $2)", "($1 << $2)")
-  of mAshrI:
-    let typ = n[1].typ.skipTypes(abstractVarRange)
-    if typ.size == 8:
-      if typ.kind == tyInt64 and optJsBigInt64 in p.config.globalOptions:
-        applyFormat("BigInt.asIntN(64, $1 >> BigInt($2))")
-      elif typ.kind == tyUInt64 and optJsBigInt64 in p.config.globalOptions:
-        applyFormat("BigInt.asUintN(64, $1 >> BigInt($2))")
-      else:
-        applyFormat("Math.floor($1 / Math.pow(2, $2))")
     else:
+      applyFormat("($1 * Math.pow(2, $2))", "($1 * Math.pow(2, $2))")
+  of mAshrI:
+    if n[1].typ.size <= 4:
       applyFormat("($1 >> $2)", "($1 >> $2)")
+    else:
+      applyFormat("Math.floor($1 / Math.pow(2, $2))", "Math.floor($1 / Math.pow(2, $2))")
   of mBitandI: applyFormat("($1 & $2)", "($1 & $2)")
   of mBitorI: applyFormat("($1 | $2)", "($1 | $2)")
   of mBitxorI: applyFormat("($1 ^ $2)", "($1 ^ $2)")
@@ -751,9 +694,7 @@ proc arith(p: PProc, n: PNode, r: var TCompRes, op: TMagic) =
   of mMulU: binaryUintExpr(p, n, r, "*")
   of mDivU:
     binaryUintExpr(p, n, r, "/")
-    if optJsBigInt64 notin p.config.globalOptions and
-        n[1].typ.skipTypes(abstractRange).size == 8:
-      # bigint / already truncates
+    if n[1].typ.skipTypes(abstractRange).size == 8:
       r.res = "Math.trunc($1)" % [r.res]
   of mDivI:
     arithAux(p, n, r, op)
@@ -763,13 +704,7 @@ proc arith(p: PProc, n: PNode, r: var TCompRes, op: TMagic) =
     var x, y: TCompRes
     gen(p, n[1], x)
     gen(p, n[2], y)
-    let typ = n[1].typ.skipTypes(abstractVarRange)
-    if typ.kind == tyInt64 and optJsBigInt64 in p.config.globalOptions:
-      r.res = "BigInt.asIntN(64, BigInt.asUintN(64, $1) >> BigInt($2))" % [x.rdLoc, y.rdLoc]
-    elif typ.kind == tyUInt64 and optJsBigInt64 in p.config.globalOptions:
-      r.res = "($1 >> BigInt($2))" % [x.rdLoc, y.rdLoc]
-    else:
-      r.res = "($1 >>> $2)" % [x.rdLoc, y.rdLoc]
+    r.res = "($1 >>> $2)" % [x.rdLoc, y.rdLoc]
   of mCharToStr, mBoolToStr, mIntToStr, mInt64ToStr, mCStrToStr, mStrToStr, mEnumToStr:
     arithAux(p, n, r, op)
   of mEqRef:
@@ -789,16 +724,13 @@ proc hasFrameInfo(p: PProc): bool =
       ((p.prc == nil) or not (sfPure in p.prc.flags))
 
 proc lineDir(config: ConfigRef, info: TLineInfo, line: int): Rope =
-  "/* line $2:$3 \"$1\" */$n" % [
-    rope(toFullPath(config, info)), rope(line), rope(info.toColumn)
-  ]
+  ropes.`%`("/* line $2 \"$1\" */$n",
+         [rope(toFullPath(config, info)), rope(line)])
 
 proc genLineDir(p: PProc, n: PNode) =
   let line = toLinenumber(n.info)
   if line < 0:
     return
-  if optEmbedOrigSrc in p.config.globalOptions:
-    lineF(p, "//$1$n", [sourceLine(p.config, n.info)])
   if optLineDir in p.options or optLineDir in p.config.options:
     lineF(p, "$1", [lineDir(p.config, n.info, line)])
   if hasFrameInfo(p):
@@ -1826,25 +1758,15 @@ proc createObjInitList(p: PProc, typ: PType, excludedFieldIDs: IntSet, output: v
     createRecordVarAux(p, t.n, excludedFieldIDs, output)
     t = t[0]
 
-proc arrayTypeForElemType(conf: ConfigRef; typ: PType): string =
+proc arrayTypeForElemType(typ: PType): string =
   let typ = typ.skipTypes(abstractRange)
   case typ.kind
   of tyInt, tyInt32: "Int32Array"
   of tyInt16: "Int16Array"
   of tyInt8: "Int8Array"
-  of tyInt64:
-    if optJsBigInt64 in conf.globalOptions:
-      "BigInt64Array"
-    else:
-      ""
   of tyUInt, tyUInt32: "Uint32Array"
   of tyUInt16: "Uint16Array"
   of tyUInt8, tyChar, tyBool: "Uint8Array"
-  of tyUInt64:
-    if optJsBigInt64 in conf.globalOptions:
-      "BigUint64Array"
-    else:
-      ""
   of tyFloat32: "Float32Array"
   of tyFloat64, tyFloat: "Float64Array"
   of tyEnum:
@@ -1858,18 +1780,11 @@ proc arrayTypeForElemType(conf: ConfigRef; typ: PType): string =
 proc createVar(p: PProc, typ: PType, indirect: bool): Rope =
   var t = skipTypes(typ, abstractInst)
   case t.kind
-  of tyInt8..tyInt32, tyUInt8..tyUInt32, tyEnum, tyChar:
-    result = putToSeq("0", indirect)
-  of tyInt, tyUInt:
+  of tyInt..tyInt64, tyUInt..tyUInt64, tyEnum, tyChar:
     if $t.sym.loc.r == "bigint":
       result = putToSeq("0n", indirect)
     else:
       result = putToSeq("0", indirect)
-  of tyInt64, tyUInt64:
-    if optJsBigInt64 in p.config.globalOptions:
-      result = putToSeq("0n", indirect)
-    else:
-      result = putToSeq("0", indirect)
   of tyFloat..tyFloat128:
     result = putToSeq("0.0", indirect)
   of tyRange, tyGenericInst, tyAlias, tySink, tyOwned, tyLent:
@@ -1883,7 +1798,7 @@ proc createVar(p: PProc, typ: PType, indirect: bool): Rope =
   of tyArray:
     let length = toInt(lengthOrd(p.config, t))
     let e = elemType(t)
-    let jsTyp = arrayTypeForElemType(p.config, e)
+    let jsTyp = arrayTypeForElemType(e)
     if jsTyp.len > 0:
       result = "new $1($2)" % [rope(jsTyp), rope(length)]
     elif length > 32:
@@ -2058,11 +1973,7 @@ proc genNewSeq(p: PProc, n: PNode) =
 
 proc genOrd(p: PProc, n: PNode, r: var TCompRes) =
   case skipTypes(n[1].typ, abstractVar + abstractRange).kind
-  of tyEnum, tyInt..tyInt32, tyUInt..tyUInt32, tyChar: gen(p, n[1], r)
-  of tyInt64, tyUInt64:
-    if optJsBigInt64 in p.config.globalOptions:
-      unaryExpr(p, n, r, "", "Number($1)")
-    else: gen(p, n[1], r)
+  of tyEnum, tyInt..tyUInt64, tyChar: gen(p, n[1], r)
   of tyBool: unaryExpr(p, n, r, "", "($1 ? 1 : 0)")
   else: internalError(p.config, n.info, "genOrd")
 
@@ -2285,41 +2196,19 @@ proc genMagic(p: PProc, n: PNode, r: var TCompRes) =
       r.res = "($1).length - 1" % [x.rdLoc]
     r.kind = resExpr
   of mInc:
-    let typ = n[1].typ.skipTypes(abstractVarRange)
-    case typ.kind
-    of tyUInt..tyUInt32:
+    if n[1].typ.skipTypes(abstractRange).kind in {tyUInt..tyUInt64}:
       binaryUintExpr(p, n, r, "+", true)
-    of tyUInt64:
-      if optJsBigInt64 in p.config.globalOptions:
-        binaryExpr(p, n, r, "", "$1 = BigInt.asUintN(64, $3 + BigInt($2))", true)
-      else: binaryUintExpr(p, n, r, "+", true)
-    elif typ.kind == tyInt64 and optJsBigInt64 in p.config.globalOptions:
-      if optOverflowCheck notin p.options:
-        binaryExpr(p, n, r, "", "$1 = BigInt.asIntN(64, $3 + BigInt($2))", true)
-      else: binaryExpr(p, n, r, "addInt64", "$1 = addInt64($3, BigInt($2))", true)
     else:
       if optOverflowCheck notin p.options: binaryExpr(p, n, r, "", "$1 += $2")
       else: binaryExpr(p, n, r, "addInt", "$1 = addInt($3, $2)", true)
   of ast.mDec:
-    let typ = n[1].typ.skipTypes(abstractVarRange)
-    case typ.kind
-    of tyUInt..tyUInt32:
+    if n[1].typ.skipTypes(abstractRange).kind in {tyUInt..tyUInt64}:
       binaryUintExpr(p, n, r, "-", true)
-    of tyUInt64:
-      if optJsBigInt64 in p.config.globalOptions:
-        binaryExpr(p, n, r, "", "$1 = BigInt.asUintN(64, $3 - BigInt($2))", true)
-      else: binaryUintExpr(p, n, r, "+", true)
-    elif typ.kind == tyInt64 and optJsBigInt64 in p.config.globalOptions:
-      if optOverflowCheck notin p.options:
-        binaryExpr(p, n, r, "", "$1 = BigInt.asIntN(64, $3 - BigInt($2))", true)
-      else: binaryExpr(p, n, r, "subInt64", "$1 = subInt64($3, BigInt($2))", true)
     else:
       if optOverflowCheck notin p.options: binaryExpr(p, n, r, "", "$1 -= $2")
       else: binaryExpr(p, n, r, "subInt", "$1 = subInt($3, $2)", true)
   of mSetLengthStr:
-    binaryExpr(p, n, r, "mnewString",
-      """if ($1.length < $2) { for (var i = $3.length; i < $4; ++i) $3.push(0); }
-         else {$3.length = $4; }""")
+    binaryExpr(p, n, r, "mnewString", "($1.length = $2)")
   of mSetLengthSeq:
     var x, y: TCompRes
     gen(p, n[1], x)
@@ -2406,7 +2295,7 @@ proc genArrayConstr(p: PProc, n: PNode, r: var TCompRes) =
   ## Nim sequence maps to JS array.
   var t = skipTypes(n.typ, abstractInst)
   let e = elemType(t)
-  let jsTyp = arrayTypeForElemType(p.config, e)
+  let jsTyp = arrayTypeForElemType(e)
   if skipTypes(n.typ, abstractVarRange).kind != tySequence and jsTyp.len > 0:
     # generate typed array
     # for example Nim generates `new Uint8Array([1, 2, 3])` for `[byte(1), 2, 3]`
@@ -2487,27 +2376,7 @@ proc genConv(p: PProc, n: PNode, r: var TCompRes) =
     r.res = "(!!($1))" % [r.res]
     r.kind = resExpr
   elif toInt:
-    if src.kind in {tyInt64, tyUInt64} and optJsBigInt64 in p.config.globalOptions:
-      r.res = "Number($1)" % [r.res]
-    else:
-      r.res = "(($1) | 0)" % [r.res]
-  elif dest.kind == tyInt64 and optJsBigInt64 in p.config.globalOptions:
-    if fromInt or fromUint or src.kind in {tyBool, tyChar, tyEnum}:
-      r.res = "BigInt($1)" % [r.res]
-    elif src.kind in {tyFloat..tyFloat64}:
-      r.res = "BigInt(Math.trunc($1))" % [r.res]
-    elif src.kind == tyUInt64:
-      r.res = "BigInt.asIntN(64, $1)" % [r.res]
-  elif dest.kind == tyUInt64 and optJsBigInt64 in p.config.globalOptions:
-    if fromInt or fromUint:
-      r.res = "BigInt($1)" % [r.res]
-    elif src.kind in {tyFloat..tyFloat64}:
-      r.res = "BigInt(Math.trunc($1))" % [r.res]
-    elif src.kind == tyInt64:
-      r.res = "BigInt.asUintN(64, $1)" % [r.res]
-  elif toUint or dest.kind in tyFloat..tyFloat64:
-    if src.kind in {tyInt64, tyUInt64} and optJsBigInt64 in p.config.globalOptions:
-      r.res = "Number($1)" % [r.res]
+    r.res = "(($1) | 0)" % [r.res]
   else:
     # TODO: What types must we handle here?
     discard
@@ -2518,11 +2387,7 @@ proc upConv(p: PProc, n: PNode, r: var TCompRes) =
 proc genRangeChck(p: PProc, n: PNode, r: var TCompRes, magic: string) =
   var a, b: TCompRes
   gen(p, n[0], r)
-  let src = skipTypes(n[0].typ, abstractVarRange)
-  let dest = skipTypes(n.typ, abstractVarRange)
-  if src.kind in {tyInt64, tyUInt64} and dest.kind notin {tyInt64, tyUInt64} and optJsBigInt64 in p.config.globalOptions:
-    r.res = "Number($1)" % [r.res]
-  if optRangeCheck notin p.options or (dest.kind in {tyUInt..tyUInt64} and
+  if optRangeCheck notin p.options or (skipTypes(n.typ, abstractVar).kind in {tyUInt..tyUInt64} and
       checkUnsignedConversions notin p.config.legacyFeatures):
     discard "XXX maybe emit masking instructions here"
   else:
@@ -2616,18 +2481,17 @@ proc genProc(oldProc: PProc, prc: PSym): Rope =
   if prc.typ[0] != nil and sfPure notin prc.flags:
     resultSym = prc.ast[resultPos].sym
     let mname = mangleName(p.module, resultSym)
-    # otherwise uses "fat pointers"
-    let useRawPointer = not isIndirect(resultSym) and
+    let returnAddress = not isIndirect(resultSym) and
       resultSym.typ.kind in {tyVar, tyPtr, tyLent, tyRef, tyOwned} and
         mapType(p, resultSym.typ) == etyBaseIndex
-    if useRawPointer:
+    if returnAddress:
       resultAsgn = p.indentLine(("var $# = null;$n") % [mname])
       resultAsgn.add p.indentLine("var $#_Idx = 0;$n" % [mname])
     else:
       let resVar = createVar(p, resultSym.typ, isIndirect(resultSym))
       resultAsgn = p.indentLine(("var $# = $#;$n") % [mname, resVar])
     gen(p, prc.ast[resultPos], a)
-    if mapType(p, resultSym.typ) == etyBaseIndex:
+    if returnAddress:
       returnStmt = "return [$#, $#];$n" % [a.address, a.res]
     else:
       returnStmt = "return $#;$n" % [a.res]
@@ -2689,14 +2553,9 @@ proc genStmt(p: PProc, n: PNode) =
   if r.res != "": lineF(p, "$#;$n", [r.res])
 
 proc genPragma(p: PProc, n: PNode) =
-  for i in 0..<n.len:
-    let it = n[i]
+  for it in n.sons:
     case whichPragma(it)
     of wEmit: genAsmOrEmitStmt(p, it[1])
-    of wPush:
-      processPushBackendOption(p.optionsStack, p.options, n, i+1)
-    of wPop:
-      processPopBackendOption(p.optionsStack, p.options)
     else: discard
 
 proc genCast(p: PProc, n: PNode, r: var TCompRes) =
@@ -2714,8 +2573,6 @@ proc genCast(p: PProc, n: PNode, r: var TCompRes) =
   if toUint and (fromInt or fromUint):
     let trimmer = unsignedTrimmer(dest.size)
     r.res = "($1 $2)" % [r.res, trimmer]
-  elif toUint and src.kind in {tyInt64, tyUInt64} and optJsBigInt64 in p.config.globalOptions:
-    r.res = "Number(BigInt.asUintN($1, $2))" % [$(dest.size * 8), r.res]
   elif toInt:
     if fromInt:
       return
@@ -2731,25 +2588,6 @@ proc genCast(p: PProc, n: PNode, r: var TCompRes) =
           of 4: "0xfffffffe"
           else: ""
         r.res = "($1 - ($2 $3))" % [rope minuend, r.res, trimmer]
-    elif src.kind in {tyInt64, tyUInt64} and optJsBigInt64 in p.config.globalOptions:
-      r.res = "Number(BigInt.asIntN($1, $2))" % [$(dest.size * 8), r.res]
-  elif dest.kind == tyInt64 and optJsBigInt64 in p.config.globalOptions:
-    if fromInt or fromUint or src.kind in {tyBool, tyChar, tyEnum}:
-      r.res = "BigInt($1)" % [r.res]
-    elif src.kind in {tyFloat..tyFloat64}:
-      r.res = "BigInt(Math.trunc($1))" % [r.res]
-    elif src.kind == tyUInt64:
-      r.res = "BigInt.asIntN(64, $1)" % [r.res]
-  elif dest.kind == tyUInt64 and optJsBigInt64 in p.config.globalOptions:
-    if fromInt or fromUint:
-      r.res = "BigInt($1)" % [r.res]
-    elif src.kind in {tyFloat..tyFloat64}:
-      r.res = "BigInt(Math.trunc($1))" % [r.res]
-    elif src.kind == tyInt64:
-      r.res = "BigInt.asUintN(64, $1)" % [r.res]
-  elif dest.kind in tyFloat..tyFloat64:
-    if src.kind in {tyInt64, tyUInt64} and optJsBigInt64 in p.config.globalOptions:
-      r.res = "Number($1)" % [r.res]
   elif (src.kind == tyPtr and mapType(p, src) == etyObject) and dest.kind == tyPointer:
     r.address = r.res
     r.res = "null"
@@ -2768,17 +2606,8 @@ proc gen(p: PProc, n: PNode, r: var TCompRes) =
   of nkSym:
     genSym(p, n, r)
   of nkCharLit..nkUInt64Lit:
-    case n.typ.skipTypes(abstractVarRange).kind
-    of tyBool:
+    if n.typ.kind == tyBool:
       r.res = if n.intVal == 0: rope"false" else: rope"true"
-    of tyUInt64:
-      r.res = rope($cast[BiggestUInt](n.intVal))
-      if optJsBigInt64 in p.config.globalOptions:
-        r.res.add('n')
-    of tyInt64:
-      r.res = rope(n.intVal)
-      if optJsBigInt64 in p.config.globalOptions:
-        r.res.add('n')
     else:
       r.res = rope(n.intVal)
     r.kind = resExpr
@@ -2974,7 +2803,7 @@ proc genModule(p: PProc, n: PNode) =
   if p.config.hcrOn and n.kind == nkStmtList:
     let moduleSym = p.module.module
     var moduleLoadedVar = rope(moduleSym.name.s) & "_loaded" &
-                          idOrSig(moduleSym, moduleSym.name.s, p.module.sigConflicts, p.config)
+                          idOrSig(moduleSym, moduleSym.name.s, p.module.sigConflicts)
     lineF(p, "var $1;$n", [moduleLoadedVar])
     var inGuardedBlock = false
 
@@ -2991,11 +2820,11 @@ proc genModule(p: PProc, n: PNode) =
   if optStackTrace in p.options:
     p.body.add(frameDestroy(p))
 
-proc processJSCodeGen*(b: PPassContext, n: PNode): PNode =
+proc myProcess(b: PPassContext, n: PNode): PNode =
   ## Generate JS code for a node.
   result = n
   let m = BModule(b)
-  if pipelineutils.skipCodegen(m.config, n): return n
+  if passes.skipCodegen(m.config, n): return n
   if m.module == nil: internalError(m.config, n.info, "myProcess")
   let globals = PGlobals(m.graph.backend)
   var p = newInitProc(globals, m)
@@ -3012,7 +2841,7 @@ proc wholeCode(graph: ModuleGraph; m: BModule): Rope =
       var p = newInitProc(globals, m)
       attachProc(p, prc)
 
-  var disp = generateMethodDispatchers(graph, m.idgen)
+  var disp = generateMethodDispatchers(graph)
   for i in 0..<disp.len:
     let prc = disp[i].sym
     if not globals.generatedSyms.containsOrIncl(prc.id):
@@ -3030,7 +2859,7 @@ proc getClassName(t: PType): Rope =
   if s.loc.r != "": result = s.loc.r
   else: result = rope(s.name.s)
 
-proc finalJSCodeGen*(graph: ModuleGraph; b: PPassContext, n: PNode): PNode =
+proc myClose(graph: ModuleGraph; b: PPassContext, n: PNode): PNode =
   ## Finalize JS code generation of a Nim module.
   ## Param `n` may contain nodes returned from the last module close call.
   var m = BModule(b)
@@ -3040,14 +2869,14 @@ proc finalJSCodeGen*(graph: ModuleGraph; b: PPassContext, n: PNode): PNode =
     for i in countdown(high(graph.globalDestructors), 0):
       n.add graph.globalDestructors[i]
   # Process any nodes left over from the last call to `myClose`.
-  result = processJSCodeGen(b, n)
+  result = myProcess(b, n)
   # Some codegen is different (such as no stacktraces; see `initProcOptions`)
   # when `std/system` is being processed.
   if sfSystemModule in m.module.flags:
     PGlobals(graph.backend).inSystem = false
   # Check if codegen should continue before any files are generated.
   # It may bail early is if too many errors have been raised.
-  if pipelineutils.skipCodegen(m.config, n): return n
+  if passes.skipCodegen(m.config, n): return n
   # Nim modules are compiled into a single JS file.
   # If this is the main module, then this is the final call to `myClose`.
   if sfMainModule in m.module.flags:
@@ -3056,8 +2885,7 @@ proc finalJSCodeGen*(graph: ModuleGraph; b: PPassContext, n: PNode): PNode =
     # Generate an optional source map.
     if optSourcemap in m.config.globalOptions:
       var map: SourceMap
-      map = genSourceMap($code, outFile.string)
-      code &= "\n//# sourceMappingURL=$#.map" % [outFile.string]
+      (code, map) = genSourceMap($(code), outFile.string)
       writeFile(outFile.string & ".map", $(%map))
     # Check if the generated JS code matches the output file, or else
     # write it to the file.
@@ -3065,6 +2893,9 @@ proc finalJSCodeGen*(graph: ModuleGraph; b: PPassContext, n: PNode): PNode =
       if not writeRope(code, outFile):
         rawMessage(m.config, errCannotOpenFile, outFile.string)
 
-proc setupJSgen*(graph: ModuleGraph; s: PSym; idgen: IdGenerator): PPassContext =
+proc myOpen(graph: ModuleGraph; s: PSym; idgen: IdGenerator): PPassContext =
+  ## Create the JS backend pass context `BModule` for a Nim module.
   result = newModule(graph, s)
   result.idgen = idgen
+
+const JSgenPass* = makePass(myOpen, myProcess, myClose)

compiler/jstypes.nim

@@ -133,7 +133,7 @@ proc genTypeInfo(p: PProc, typ: PType): Rope =
       "var $1 = {size: 0,kind: $2,base: null,node: null,finalizer: null};$n" %
       [result, rope(ord(t.kind))]
     prepend(p.g.typeInfo, s)
-  of tyVar, tyLent, tyRef, tyPtr, tySequence, tyRange, tySet, tyOpenArray:
+  of tyVar, tyLent, tyRef, tyPtr, tySequence, tyRange, tySet:
     var s =
       "var $1 = {size: 0, kind: $2, base: null, node: null, finalizer: null};$n" %
               [result, rope(ord(t.kind))]

compiler/lambdalifting.nim

@@ -859,7 +859,7 @@ proc liftIterToProc*(g: ModuleGraph; fn: PSym; body: PNode; ptrType: PType;
   fn.typ.callConv = oldCC
 
 proc liftLambdas*(g: ModuleGraph; fn: PSym, body: PNode; tooEarly: var bool;
-                  idgen: IdGenerator, force: bool): PNode =
+                  idgen: IdGenerator): PNode =
   # XXX backend == backendJs does not suffice! The compiletime stuff needs
   # the transformation even when compiling to JS ...
 
@@ -868,7 +868,7 @@ proc liftLambdas*(g: ModuleGraph; fn: PSym, body: PNode; tooEarly: var bool;
 
   if body.kind == nkEmpty or (
       g.config.backend == backendJs and not isCompileTime) or
-      (fn.skipGenericOwner.kind != skModule and not force):
+      fn.skipGenericOwner.kind != skModule:
 
     # ignore forward declaration:
     result = body

compiler/lexer.nim

@@ -127,8 +127,6 @@ type
     cache*: IdentCache
     when defined(nimsuggest):
       previousToken: TLineInfo
-      tokenEnd*: TLineInfo
-      previousTokenEnd*: TLineInfo
     config*: ConfigRef
 
 proc getLineInfo*(L: Lexer, tok: Token): TLineInfo {.inline.} =
@@ -1226,10 +1224,6 @@ proc skip(L: var Lexer, tok: var Token) =
 proc rawGetTok*(L: var Lexer, tok: var Token) =
   template atTokenEnd() {.dirty.} =
     when defined(nimsuggest):
-      L.previousTokenEnd.line = L.tokenEnd.line
-      L.previousTokenEnd.col = L.tokenEnd.col
-      L.tokenEnd.line = tok.line.uint16
-      L.tokenEnd.col = getColNumber(L, L.bufpos).int16
       # we attach the cursor to the last *strong* token
       if tok.tokType notin weakTokens:
         L.previousToken.line = tok.line.uint16

compiler/liftdestructors.nim

@@ -8,7 +8,7 @@
 #
 
 ## This module implements lifting for type-bound operations
-## (``=sink``, ``=copy``, ``=destroy``, ``=deepCopy``).
+## (``=sink``, ``=``, ``=destroy``, ``=deepCopy``).
 
 import modulegraphs, lineinfos, idents, ast, renderer, semdata,
   sighashes, lowerings, options, types, msgs, magicsys, tables, ccgutils
@@ -88,8 +88,6 @@ proc defaultOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
     let call = genBuiltin(c, mDefault, "default", x)
     call.typ = t
     body.add newAsgnStmt(x, call)
-  elif c.kind == attachedWasMoved:
-    body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc genAddr(c: var TLiftCtx; x: PNode): PNode =
   if x.kind == nkHiddenDeref:
@@ -147,11 +145,6 @@ proc destructorCall(c: var TLiftCtx; op: PSym; x: PNode): PNode =
   else:
     result = destroy
 
-proc genWasMovedCall(c: var TLiftCtx; op: PSym; x: PNode): PNode =
-  result = newNodeIT(nkCall, x.info, op.typ[0])
-  result.add(newSymNode(op))
-  result.add genAddr(c, x)
-
 proc fillBodyObj(c: var TLiftCtx; n, body, x, y: PNode; enforceDefaultOp: bool) =
   case n.kind
   of nkSym:
@@ -449,20 +442,6 @@ proc considerUserDefinedOp(c: var TLiftCtx; t: PType; body, x, y: PNode): bool =
       body.add newDeepCopyCall(c, op, x, y)
       result = true
 
-  of attachedWasMoved:
-    var op = getAttachedOp(c.g, t, attachedWasMoved)
-    if op != nil and sfOverriden in op.flags:
-
-      if op.ast.isGenericRoutine:
-        # patch generic destructor:
-        op = instantiateGeneric(c, op, t, t.typeInst)
-        setAttachedOp(c.g, c.idgen.module, t, attachedWasMoved, op)
-
-      #markUsed(c.g.config, c.info, op, c.g.usageSym)
-      onUse(c.info, op)
-      body.add genWasMovedCall(c, op, x)
-      result = true
-
 proc declareCounter(c: var TLiftCtx; body: PNode; first: BiggestInt): PNode =
   var temp = newSym(skTemp, getIdent(c.g.cache, lowerings.genPrefix), nextSymId(c.idgen), c.fn, c.info)
   temp.typ = getSysType(c.g, body.info, tyInt)
@@ -545,7 +524,6 @@ proc fillSeqOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
     if canFormAcycle(t.elemType):
       # follow all elements:
       forallElements(c, t, body, x, y)
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc useSeqOrStrOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   createTypeBoundOps(c.g, c.c, t, body.info, c.idgen)
@@ -553,7 +531,7 @@ proc useSeqOrStrOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   # operation here:
   var t = t
   if t.assignment == nil or t.destructor == nil:
-    let h = sighashes.hashType(t,c.g.config, {CoType, CoConsiderOwned, CoDistinct})
+    let h = sighashes.hashType(t, {CoType, CoConsiderOwned, CoDistinct})
     let canon = c.g.canonTypes.getOrDefault(h)
     if canon != nil: t = canon
 
@@ -583,7 +561,6 @@ proc useSeqOrStrOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
       if op == nil:
         return # protect from recursion
       body.add newHookCall(c, op, x, y)
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc fillStrOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   case c.kind
@@ -599,7 +576,6 @@ proc fillStrOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
     body.add genBuiltin(c, mDestroy, "destroy", x)
   of attachedTrace:
     discard "strings are atomic and have no inner elements that are to trace"
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc cyclicType*(t: PType): bool =
   case t.kind
@@ -633,11 +609,6 @@ proc atomicRefOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   createTypeBoundOps(c.g, c.c, elemType, c.info, c.idgen)
   let isCyclic = c.g.config.selectedGC == gcOrc and types.canFormAcycle(elemType)
 
-  let isInheritableAcyclicRef = c.g.config.selectedGC == gcOrc and
-                      (not isPureObject(elemType)) and
-                      tfAcyclic in skipTypes(elemType, abstractInst+{tyOwned}-{tyTypeDesc}).flags
-  # dynamic Acyclic refs need to use dyn decRef
-
   let tmp =
     if isCyclic and c.kind in {attachedAsgn, attachedSink}:
       declareTempOf(c, body, x)
@@ -661,8 +632,6 @@ proc atomicRefOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
       cond = callCodegenProc(c.g, "nimDecRefIsLastCyclicStatic", c.info, tmp, typInfo)
     else:
       cond = callCodegenProc(c.g, "nimDecRefIsLastCyclicDyn", c.info, tmp)
-  elif isInheritableAcyclicRef:
-    cond = callCodegenProc(c.g, "nimDecRefIsLastDyn", c.info, x)
   else:
     cond = callCodegenProc(c.g, "nimDecRefIsLast", c.info, x)
   cond.typ = getSysType(c.g, x.info, tyBool)
@@ -698,8 +667,6 @@ proc atomicRefOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
         # If the ref is polymorphic we have to account for this
         body.add callCodegenProc(c.g, "nimTraceRefDyn", c.info, genAddrOf(x, c.idgen), y)
       #echo "can follow ", elemType, " static ", isFinal(elemType)
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
-
 
 proc atomicClosureOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   ## Closures are really like refs except they always use a virtual destructor
@@ -748,7 +715,6 @@ proc atomicClosureOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   of attachedDeepCopy: assert(false, "cannot happen")
   of attachedTrace:
     body.add callCodegenProc(c.g, "nimTraceRefDyn", c.info, genAddrOf(xenv, c.idgen), y)
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc weakrefOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   case c.kind
@@ -773,7 +739,6 @@ proc weakrefOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
       body.sons.insert(des, 0)
   of attachedDeepCopy: assert(false, "cannot happen")
   of attachedTrace: discard
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc ownedRefOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   var actions = newNodeI(nkStmtList, c.info)
@@ -799,7 +764,6 @@ proc ownedRefOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
     body.add genIf(c, x, actions)
   of attachedDeepCopy: assert(false, "cannot happen")
   of attachedTrace: discard
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc closureOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   if c.kind == attachedDeepCopy:
@@ -834,7 +798,6 @@ proc closureOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
         body.sons.insert(des, 0)
     of attachedDeepCopy: assert(false, "cannot happen")
     of attachedTrace: discard
-    of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc ownedClosureOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   let xx = genBuiltin(c, mAccessEnv, "accessEnv", x)
@@ -850,7 +813,6 @@ proc ownedClosureOp(c: var TLiftCtx; t: PType; body, x, y: PNode) =
     body.add genIf(c, xx, actions)
   of attachedDeepCopy: assert(false, "cannot happen")
   of attachedTrace: discard
-  of attachedWasMoved: body.add genBuiltin(c, mWasMoved, "wasMoved", x)
 
 proc fillBody(c: var TLiftCtx; t: PType; body, x, y: PNode) =
   case t.kind
@@ -966,7 +928,7 @@ proc symPrototype(g: ModuleGraph; typ: PType; owner: PSym; kind: TTypeAttachedOp
 
   result.typ = newProcType(info, nextTypeId(idgen), owner)
   result.typ.addParam dest
-  if kind notin {attachedDestructor, attachedWasMoved}:
+  if kind != attachedDestructor:
     result.typ.addParam src
 
   if kind == attachedAsgn and g.config.selectedGC == gcOrc and
@@ -1006,7 +968,7 @@ proc produceSym(g: ModuleGraph; c: PContext; typ: PType; kind: TTypeAttachedOp;
 
   let dest = result.typ.n[1].sym
   let d = newDeref(newSymNode(dest))
-  let src = if kind in {attachedDestructor, attachedWasMoved}: newNodeIT(nkSym, info, getSysType(g, info, tyPointer))
+  let src = if kind == attachedDestructor: newNodeIT(nkSym, info, getSysType(g, info, tyPointer))
             else: newSymNode(result.typ.n[2].sym)
 
   # register this operation already:
@@ -1085,7 +1047,13 @@ proc inst(g: ModuleGraph; c: PContext; t: PType; kind: TTypeAttachedOp; idgen: I
   let op = getAttachedOp(g, t, kind)
   if op != nil and op.ast != nil and op.ast.isGenericRoutine:
     if t.typeInst != nil:
-      var a = TLiftCtx(info: info, g: g, kind: kind, c: c, idgen: idgen)
+      var a: TLiftCtx
+      a.info = info
+      a.g = g
+      a.kind = kind
+      a.c = c
+      a.idgen = idgen
+
       let opInst = instantiateGeneric(a, op, t, t.typeInst)
       if opInst.ast != nil:
         patchBody(g, c, opInst.ast, info, a.idgen)
@@ -1107,7 +1075,7 @@ proc createTypeBoundOps(g: ModuleGraph; c: PContext; orig: PType; info: TLineInf
   let skipped = orig.skipTypes({tyGenericInst, tyAlias, tySink})
   if isEmptyContainer(skipped) or skipped.kind == tyStatic: return
 
-  let h = sighashes.hashType(skipped, g.config, {CoType, CoConsiderOwned, CoDistinct})
+  let h = sighashes.hashType(skipped, {CoType, CoConsiderOwned, CoDistinct})
   var canon = g.canonTypes.getOrDefault(h)
   if canon == nil:
     g.canonTypes[h] = skipped
@@ -1128,15 +1096,15 @@ proc createTypeBoundOps(g: ModuleGraph; c: PContext; orig: PType; info: TLineInf
   # bug #15122: We need to produce all prototypes before entering the
   # mind boggling recursion. Hacks like these imply we should rewrite
   # this module.
-  var generics: array[attachedWasMoved..attachedTrace, bool]
-  for k in attachedWasMoved..lastAttached:
+  var generics: array[attachedDestructor..attachedTrace, bool]
+  for k in attachedDestructor..lastAttached:
     generics[k] = getAttachedOp(g, canon, k) != nil
     if not generics[k]:
       setAttachedOp(g, idgen.module, canon, k,
           symPrototype(g, canon, canon.owner, k, info, idgen))
 
   # we generate the destructor first so that other operators can depend on it:
-  for k in attachedWasMoved..lastAttached:
+  for k in attachedDestructor..lastAttached:
     if not generics[k]:
       discard produceSym(g, c, canon, k, info, idgen)
     else:

compiler/lineinfos.nim

@@ -57,7 +57,6 @@ type
     warnRstBrokenLink = "BrokenLink",
     warnRstLanguageXNotSupported = "LanguageXNotSupported",
     warnRstFieldXNotSupported = "FieldXNotSupported",
-    warnRstUnusedImportdoc = "UnusedImportdoc",
     warnRstStyle = "warnRstStyle",
     warnCommentXIgnored = "CommentXIgnored",
     warnTypelessParam = "TypelessParam",
@@ -84,12 +83,8 @@ type
     warnPtrToCstringConv = "PtrToCstringConv",
     warnEffect = "Effect",
     warnCastSizes = "CastSizes"
-    warnImplicitTemplateRedefinition = "ImplicitTemplateRedefinition",
+    warnTemplateRedefinition = "TemplateRedefinition",
     warnUnnamedBreak = "UnnamedBreak",
-    warnStmtListLambda = "StmtListLambda",
-    warnBareExcept = "BareExcept",
-    warnCopyHookForRefc = "CopyHookForRefc",
-    warnImplicitDefaultValue = "ImplicitDefaultValue",
     warnUser = "User",
     # hints
     hintSuccess = "Success", hintSuccessX = "SuccessX",
@@ -145,7 +140,6 @@ const
     warnRstBrokenLink: "broken link '$1'",
     warnRstLanguageXNotSupported: "language '$1' not supported",
     warnRstFieldXNotSupported: "field '$1' not supported",
-    warnRstUnusedImportdoc: "importdoc for '$1' is not used",
     warnRstStyle: "RST style: $1",
     warnCommentXIgnored: "comment '$1' ignored",
     warnTypelessParam: "", # deadcode
@@ -187,12 +181,8 @@ const
     warnPtrToCstringConv: "unsafe conversion to 'cstring' from '$1'; this will become a compile time error in the future",
     warnEffect: "$1",
     warnCastSizes: "$1",
-    warnImplicitTemplateRedefinition: "template '$1' is implicitly redefined; this is deprecated, add an explicit .redefine pragma",
+    warnTemplateRedefinition: "template '$1' is implicitly redefined, consider adding an explicit .redefine pragma",
     warnUnnamedBreak: "Using an unnamed break in a block is deprecated; Use a named block with a named break instead",
-    warnStmtListLambda: "statement list expression assumed to be anonymous proc; this is deprecated, use `do (): ...` or `proc () = ...` instead",
-    warnBareExcept: "$1",
-    warnCopyHookForRefc: "Overriding `=copy` hook is not reliable for refc",
-    warnImplicitDefaultValue: "$1",
     warnUser: "$1",
     hintSuccess: "operation successful: $#",
     # keep in sync with `testament.isSuccess`

compiler/lookups.nim

@@ -15,7 +15,7 @@ when defined(nimPreviewSlimSystem):
 
 import
   intsets, ast, astalgo, idents, semdata, types, msgs, options,
-  renderer, nimfix/prettybase, lineinfos, modulegraphs, astmsgs, sets
+  renderer, nimfix/prettybase, lineinfos, modulegraphs, astmsgs
 
 proc ensureNoMissingOrUnusedSymbols(c: PContext; scope: PScope)
 
@@ -170,7 +170,6 @@ iterator allSyms*(c: PContext): (PSym, int, bool) =
   # really iterate over all symbols in all the scopes. This is expensive
   # and only used by suggest.nim.
   var isLocal = true
-
   var scopeN = 0
   for scope in allScopes(c.currentScope):
     if scope == c.topLevelScope: isLocal = false
@@ -185,17 +184,6 @@ iterator allSyms*(c: PContext): (PSym, int, bool) =
       assert s != nil
       yield (s, scopeN, isLocal)
 
-iterator uniqueSyms*(c: PContext): (PSym, int, bool) =
-  ## Like [allSyms] except only returns unique symbols (Uniqueness determined by line + name)
-  # Track seen symbols so we don't duplicate them.
-  # The int is for the symbols name, and line info is
-  # to be able to tell apart symbols with same name but on different lines
-  var seen = initHashSet[(TLineInfo, int)]()
-  for res in allSyms(c):
-    if not seen.containsOrIncl((res[0].info, res[0].name.id)):
-      yield res
-
-
 proc someSymFromImportTable*(c: PContext; name: PIdent; ambiguous: var bool): PSym =
   var marked = initIntSet()
   var symSet = OverloadableSyms
@@ -228,23 +216,6 @@ proc debugScopes*(c: PContext; limit=0, max = int.high) {.deprecated.} =
     if i == limit: return
     inc i
 
-proc searchInScopesAllCandidatesFilterBy*(c: PContext, s: PIdent, filter: TSymKinds): seq[PSym] =
-  result = @[]
-  for scope in allScopes(c.currentScope):
-    var ti: TIdentIter
-    var candidate = initIdentIter(ti, scope.symbols, s)
-    while candidate != nil:
-      if candidate.kind in filter:
-        result.add candidate
-      candidate = nextIdentIter(ti, scope.symbols)
-
-  if result.len == 0:
-    var marked = initIntSet()
-    for im in c.imports.mitems:
-      for s in symbols(im, marked, s, c.graph):
-        if s.kind in filter:
-          result.add s
-
 proc searchInScopesFilterBy*(c: PContext, s: PIdent, filter: TSymKinds): seq[PSym] =
   result = @[]
   block outer:
@@ -340,11 +311,9 @@ proc wrongRedefinition*(c: PContext; info: TLineInfo, s: string;
 # xxx pending bootstrap >= 1.4, replace all those overloads with a single one:
 # proc addDecl*(c: PContext, sym: PSym, info = sym.info, scope = c.currentScope) {.inline.} =
 proc addDeclAt*(c: PContext; scope: PScope, sym: PSym, info: TLineInfo) =
-  if sym.name.s == "_": return
   let conflict = scope.addUniqueSym(sym)
   if conflict != nil:
-    if sym.kind == skModule and conflict.kind == skModule and
-                        sym.position == conflict.position:
+    if sym.kind == skModule and conflict.kind == skModule and sym.owner == conflict.owner:
       # e.g.: import foo; import foo
       # xxx we could refine this by issuing a different hint for the case
       # where a duplicate import happens inside an include.
@@ -396,12 +365,11 @@ proc addOverloadableSymAt*(c: PContext; scope: PScope, fn: PSym) =
   if fn.kind notin OverloadableSyms:
     internalError(c.config, fn.info, "addOverloadableSymAt")
     return
-  if fn.name.s != "_":
-    let check = strTableGet(scope.symbols, fn.name)
-    if check != nil and check.kind notin OverloadableSyms:
-      wrongRedefinition(c, fn.info, fn.name.s, check.info)
-    else:
-      scope.addSym(fn)
+  let check = strTableGet(scope.symbols, fn.name)
+  if check != nil and check.kind notin OverloadableSyms:
+    wrongRedefinition(c, fn.info, fn.name.s, check.info)
+  else:
+    scope.addSym(fn)
 
 proc addInterfaceOverloadableSymAt*(c: PContext, scope: PScope, sym: PSym) =
   ## adds an overloadable symbol on the scope and the interface if appropriate
@@ -454,7 +422,7 @@ type SpellCandidate = object
   msg: string
   sym: PSym
 
-template toOrderTup(a: SpellCandidate): (int, int, string) =
+template toOrderTup(a: SpellCandidate): auto =
   # `dist` is first, to favor nearby matches
   # `depth` is next, to favor nearby enclosing scopes among ties
   # `sym.name.s` is last, to make the list ordered and deterministic among ties
@@ -477,13 +445,12 @@ proc fixSpelling(c: PContext, n: PNode, ident: PIdent, result: var string) =
     let dist = editDistance(name0, sym.name.s.nimIdentNormalize)
     var msg: string
     msg.add "\n ($1, $2): '$3'" % [$dist, $depth, sym.name.s]
+    addDeclaredLoc(msg, c.config, sym) # `msg` needed for deterministic ordering.
     list.push SpellCandidate(dist: dist, depth: depth, msg: msg, sym: sym)
 
   if list.len == 0: return
   let e0 = list[0]
-  var
-    count = 0
-    last: PIdent = nil
+  var count = 0
   while true:
     # pending https://github.com/timotheecour/Nim/issues/373 use more efficient `itemsSorted`.
     if list.len == 0: break
@@ -499,10 +466,8 @@ proc fixSpelling(c: PContext, n: PNode, ident: PIdent, result: var string) =
     elif count >= c.config.spellSuggestMax: break
     if count == 0:
       result.add "\ncandidates (edit distance, scope distance); see '--spellSuggest': "
-    if e.sym.name != last:
-      result.add e.msg
-      count.inc
-      last = e.sym.name
+    result.add e.msg
+    count.inc
 
 proc errorUseQualifier(c: PContext; info: TLineInfo; s: PSym; amb: var bool): PSym =
   var err = "ambiguous identifier: '" & s.name.s & "'"
@@ -547,16 +512,12 @@ proc errorUseQualifier*(c: PContext; info:TLineInfo; choices: PNode) =
   errorUseQualifier(c, info, candidates, prefix)
 
 proc errorUndeclaredIdentifier*(c: PContext; info: TLineInfo; name: string, extra = "") =
-  var err: string
-  if name == "_":
-    err = "the special identifier '_' is ignored in declarations and cannot be used"
-  else:
-    err = "undeclared identifier: '" & name & "'" & extra
-    if c.recursiveDep.len > 0:
-      err.add "\nThis might be caused by a recursive module dependency:\n"
-      err.add c.recursiveDep
-      # prevent excessive errors for 'nim check'
-      c.recursiveDep = ""
+  var err = "undeclared identifier: '" & name & "'" & extra
+  if c.recursiveDep.len > 0:
+    err.add "\nThis might be caused by a recursive module dependency:\n"
+    err.add c.recursiveDep
+    # prevent excessive errors for 'nim check'
+    c.recursiveDep = ""
   localError(c.config, info, errGenerated, err)
 
 proc errorUndeclaredIdentifierHint*(c: PContext; n: PNode, ident: PIdent): PSym =

compiler/main.nim

@@ -16,9 +16,9 @@ import
   std/[strutils, os, times, tables, sha1, with, json],
   llstream, ast, lexer, syntaxes, options, msgs,
   condsyms,
-  idents, extccomp,
+  sem, idents, passes, extccomp,
   cgen, nversion,
-  platform, nimconf, depends,
+  platform, nimconf, passaux, depends, vm,
   modules,
   modulegraphs, lineinfos, pathutils, vmprofiler
 
@@ -29,10 +29,12 @@ when defined(nimPreviewSlimSystem):
 import ic / [cbackend, integrity, navigator]
 from ic / ic import rodViewer
 
-import pipelines
-
 when not defined(leanCompiler):
-  import docgen
+  import jsgen, docgen, docgen2
+
+proc semanticPasses(g: ModuleGraph) =
+  registerPass g, verbosePass
+  registerPass g, semPass
 
 proc writeDepsFile(g: ModuleGraph) =
   let fname = g.config.nimcacheDir / RelativeFile(g.config.projectName & ".deps")
@@ -66,18 +68,12 @@ proc writeCMakeDepsFile(conf: ConfigRef) =
     fl.close()
 
 proc commandGenDepend(graph: ModuleGraph) =
-  setPipeLinePass(graph, GenDependPass)
-  compilePipelineProject(graph)
+  semanticPasses(graph)
+  registerPass(graph, gendependPass)
+  compileProject(graph)
   let project = graph.config.projectFull
   writeDepsFile(graph)
   generateDot(graph, project)
-
-  # dot in graphivz tool kit is required
-  let graphvizDotPath = findExe("dot")
-  if graphvizDotPath.len == 0:
-    quit("gendepend: Graphviz's tool dot is required," &
-    "see https://graphviz.org/download for downloading")
-
   execExternalProgram(graph.config, "dot -Tpng -o" &
       changeFileExt(project, "png").string &
       ' ' & changeFileExt(project, "dot").string)
@@ -91,8 +87,8 @@ proc commandCheck(graph: ModuleGraph) =
     defineSymbol(conf.symbols, "nimconfig")
   elif conf.backend == backendJs:
     setTarget(conf.target, osJS, cpuJS)
-  setPipeLinePass(graph, SemPass)
-  compilePipelineProject(graph)
+  semanticPasses(graph)  # use an empty backend for semantic checking only
+  compileProject(graph)
 
   if conf.symbolFiles != disabledSf:
     case conf.ideCmd
@@ -106,20 +102,22 @@ when not defined(leanCompiler):
   proc commandDoc2(graph: ModuleGraph; ext: string) =
     handleDocOutputOptions graph.config
     graph.config.setErrorMaxHighMaybe
+    semanticPasses(graph)
     case ext:
-    of TexExt:
-      setPipeLinePass(graph, Docgen2TexPass)
-    of JsonExt:
-      setPipeLinePass(graph, Docgen2JsonPass)
-    of HtmlExt:
-      setPipeLinePass(graph, Docgen2Pass)
+    of TexExt:  registerPass(graph, docgen2TexPass)
+    of JsonExt: registerPass(graph, docgen2JsonPass)
+    of HtmlExt: registerPass(graph, docgen2Pass)
     else: doAssert false, $ext
-    compilePipelineProject(graph)
+    compileProject(graph)
+    finishDoc2Pass(graph.config.projectName)
 
 proc commandCompileToC(graph: ModuleGraph) =
   let conf = graph.config
   extccomp.initVars(conf)
+  semanticPasses(graph)
   if conf.symbolFiles == disabledSf:
+    registerPass(graph, cgenPass)
+
     if {optRun, optForceFullMake} * conf.globalOptions == {optRun} or isDefined(conf, "nimBetterRun"):
       if not changeDetectedViaJsonBuildInstructions(conf, conf.jsonBuildInstructionsFile):
         # nothing changed
@@ -129,11 +127,7 @@ proc commandCompileToC(graph: ModuleGraph) =
   if not extccomp.ccHasSaneOverflow(conf):
     conf.symbols.defineSymbol("nimEmulateOverflowChecks")
 
-  if conf.symbolFiles == disabledSf:
-    setPipeLinePass(graph, CgenPass)
-  else:
-    setPipeLinePass(graph, SemPass)
-  compilePipelineProject(graph)
+  compileProject(graph)
   if graph.config.errorCounter > 0:
     return # issue #9933
   if conf.symbolFiles == disabledSf:
@@ -166,27 +160,33 @@ proc commandCompileToJS(graph: ModuleGraph) =
     conf.exc = excCpp
     setTarget(conf.target, osJS, cpuJS)
     defineSymbol(conf.symbols, "ecmascript") # For backward compatibility
-    setPipeLinePass(graph, JSgenPass)
-    compilePipelineProject(graph)
+    semanticPasses(graph)
+    registerPass(graph, JSgenPass)
+    compileProject(graph)
     if optGenScript in conf.globalOptions:
       writeDepsFile(graph)
 
-proc commandInteractive(graph: ModuleGraph) =
-  graph.config.setErrorMaxHighMaybe
+proc interactivePasses(graph: ModuleGraph) =
   initDefines(graph.config.symbols)
   defineSymbol(graph.config.symbols, "nimscript")
   # note: seems redundant with -d:nimHasLibFFI
   when hasFFI: defineSymbol(graph.config.symbols, "nimffi")
-  setPipeLinePass(graph, InterpreterPass)
-  compilePipelineSystemModule(graph)
+  registerPass(graph, verbosePass)
+  registerPass(graph, semPass)
+  registerPass(graph, evalPass)
+
+proc commandInteractive(graph: ModuleGraph) =
+  graph.config.setErrorMaxHighMaybe
+  interactivePasses(graph)
+  compileSystemModule(graph)
   if graph.config.commandArgs.len > 0:
-    discard graph.compilePipelineModule(fileInfoIdx(graph.config, graph.config.projectFull), {})
+    discard graph.compileModule(fileInfoIdx(graph.config, graph.config.projectFull), {})
   else:
     var m = graph.makeStdinModule()
     incl(m.flags, sfMainModule)
     var idgen = IdGenerator(module: m.itemId.module, symId: m.itemId.item, typeId: 0)
     let s = llStreamOpenStdIn(onPrompt = proc() = flushDot(graph.config))
-    discard processPipelineModule(graph, m, idgen, s)
+    processModule(graph, m, idgen, s)
 
 proc commandScan(cache: IdentCache, config: ConfigRef) =
   var f = addFileExt(AbsoluteFile mainCommandArg(config), NimExt)
@@ -241,6 +241,8 @@ proc mainCommand*(graph: ModuleGraph) =
   let conf = graph.config
   let cache = graph.cache
 
+  # In "nim serve" scenario, each command must reset the registered passes
+  clearPasses(graph)
   conf.lastCmdTime = epochTime()
   conf.searchPaths.add(conf.libpath)
 

compiler/md5_old.nim

@@ -0,0 +1,297 @@
+#
+#
+#            Nim's Runtime Library
+#        (c) Copyright 2010 Andreas Rumpf
+#
+#    See the file "copying.txt", included in this
+#    distribution, for details about the copyright.
+#
+
+# `std/md5` without VM and JavaScript support, to circumvent a bug with
+# openarrays on Nim < 1.4.
+
+when defined(nimHasStyleChecks):
+  {.push styleChecks: off.}
+
+type
+  MD5State = array[0..3, uint32]
+  MD5Block = array[0..15, uint32]
+  MD5CBits = array[0..7, uint8]
+  MD5Digest* = array[0..15, uint8]
+    ## MD5 checksum of a string, obtained with the `toMD5 proc <#toMD5,string>`_.
+  MD5Buffer = array[0..63, uint8]
+  MD5Context* {.final.} = object
+    state: MD5State
+    count: array[0..1, uint32]
+    buffer: MD5Buffer
+
+const
+  padding: array[0..63, uint8] = [
+    0x80'u8, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0
+  ]
+
+proc F(x, y, z: uint32): uint32 {.inline.} =
+  result = (x and y) or ((not x) and z)
+
+proc G(x, y, z: uint32): uint32 {.inline.} =
+  result = (x and z) or (y and (not z))
+
+proc H(x, y, z: uint32): uint32 {.inline.} =
+  result = x xor y xor z
+
+proc I(x, y, z: uint32): uint32 {.inline.} =
+  result = y xor (x or (not z))
+
+proc rot(x: var uint32, n: uint8) {.inline.} =
+  x = (x shl n) or (x shr (32'u32 - n))
+
+proc FF(a: var uint32, b, c, d, x: uint32, s: uint8, ac: uint32) =
+  a = a + F(b, c, d) + x + ac
+  rot(a, s)
+  a = a + b
+
+proc GG(a: var uint32, b, c, d, x: uint32, s: uint8, ac: uint32) =
+  a = a + G(b, c, d) + x + ac
+  rot(a, s)
+  a = a + b
+
+proc HH(a: var uint32, b, c, d, x: uint32, s: uint8, ac: uint32) =
+  a = a + H(b, c, d) + x + ac
+  rot(a, s)
+  a = a + b
+
+proc II(a: var uint32, b, c, d, x: uint32, s: uint8, ac: uint32) =
+  a = a + I(b, c, d) + x + ac
+  rot(a, s)
+  a = a + b
+
+proc encode(dest: var MD5Block, src: openArray[uint8]) =
+  var j = 0
+  for i in 0..high(dest):
+    dest[i] = uint32(ord(src[j])) or
+              uint32(ord(src[j+1])) shl 8 or
+              uint32(ord(src[j+2])) shl 16 or
+              uint32(ord(src[j+3])) shl 24
+    inc(j, 4)
+
+proc decode(dest: var openArray[uint8], src: openArray[uint32]) =
+  var i = 0
+  for j in 0..high(src):
+    dest[i] = uint8(src[j] and 0xff'u32)
+    dest[i+1] = uint8(src[j] shr 8 and 0xff'u32)
+    dest[i+2] = uint8(src[j] shr 16 and 0xff'u32)
+    dest[i+3] = uint8(src[j] shr 24 and 0xff'u32)
+    inc(i, 4)
+
+template slice(s: string | cstring, a, b): openArray[uint8] =
+  s.toOpenArrayByte(a, b)
+
+template slice(s: openArray[uint8], a, b): openArray[uint8] =
+  s.toOpenArray(a, b)
+
+proc transform(buffer: openArray[uint8], state: var MD5State) =
+  var
+    myBlock: MD5Block
+  encode(myBlock, buffer)
+  var a = state[0]
+  var b = state[1]
+  var c = state[2]
+  var d = state[3]
+  FF(a, b, c, d, myBlock[0], 7'u8, 0xD76AA478'u32)
+  FF(d, a, b, c, myBlock[1], 12'u8, 0xE8C7B756'u32)
+  FF(c, d, a, b, myBlock[2], 17'u8, 0x242070DB'u32)
+  FF(b, c, d, a, myBlock[3], 22'u8, 0xC1BDCEEE'u32)
+  FF(a, b, c, d, myBlock[4], 7'u8, 0xF57C0FAF'u32)
+  FF(d, a, b, c, myBlock[5], 12'u8, 0x4787C62A'u32)
+  FF(c, d, a, b, myBlock[6], 17'u8, 0xA8304613'u32)
+  FF(b, c, d, a, myBlock[7], 22'u8, 0xFD469501'u32)
+  FF(a, b, c, d, myBlock[8], 7'u8, 0x698098D8'u32)
+  FF(d, a, b, c, myBlock[9], 12'u8, 0x8B44F7AF'u32)
+  FF(c, d, a, b, myBlock[10], 17'u8, 0xFFFF5BB1'u32)
+  FF(b, c, d, a, myBlock[11], 22'u8, 0x895CD7BE'u32)
+  FF(a, b, c, d, myBlock[12], 7'u8, 0x6B901122'u32)
+  FF(d, a, b, c, myBlock[13], 12'u8, 0xFD987193'u32)
+  FF(c, d, a, b, myBlock[14], 17'u8, 0xA679438E'u32)
+  FF(b, c, d, a, myBlock[15], 22'u8, 0x49B40821'u32)
+  GG(a, b, c, d, myBlock[1], 5'u8, 0xF61E2562'u32)
+  GG(d, a, b, c, myBlock[6], 9'u8, 0xC040B340'u32)
+  GG(c, d, a, b, myBlock[11], 14'u8, 0x265E5A51'u32)
+  GG(b, c, d, a, myBlock[0], 20'u8, 0xE9B6C7AA'u32)
+  GG(a, b, c, d, myBlock[5], 5'u8, 0xD62F105D'u32)
+  GG(d, a, b, c, myBlock[10], 9'u8, 0x02441453'u32)
+  GG(c, d, a, b, myBlock[15], 14'u8, 0xD8A1E681'u32)
+  GG(b, c, d, a, myBlock[4], 20'u8, 0xE7D3FBC8'u32)
+  GG(a, b, c, d, myBlock[9], 5'u8, 0x21E1CDE6'u32)
+  GG(d, a, b, c, myBlock[14], 9'u8, 0xC33707D6'u32)
+  GG(c, d, a, b, myBlock[3], 14'u8, 0xF4D50D87'u32)
+  GG(b, c, d, a, myBlock[8], 20'u8, 0x455A14ED'u32)
+  GG(a, b, c, d, myBlock[13], 5'u8, 0xA9E3E905'u32)
+  GG(d, a, b, c, myBlock[2], 9'u8, 0xFCEFA3F8'u32)
+  GG(c, d, a, b, myBlock[7], 14'u8, 0x676F02D9'u32)
+  GG(b, c, d, a, myBlock[12], 20'u8, 0x8D2A4C8A'u32)
+  HH(a, b, c, d, myBlock[5], 4'u8, 0xFFFA3942'u32)
+  HH(d, a, b, c, myBlock[8], 11'u8, 0x8771F681'u32)
+  HH(c, d, a, b, myBlock[11], 16'u8, 0x6D9D6122'u32)
+  HH(b, c, d, a, myBlock[14], 23'u8, 0xFDE5380C'u32)
+  HH(a, b, c, d, myBlock[1], 4'u8, 0xA4BEEA44'u32)
+  HH(d, a, b, c, myBlock[4], 11'u8, 0x4BDECFA9'u32)
+  HH(c, d, a, b, myBlock[7], 16'u8, 0xF6BB4B60'u32)
+  HH(b, c, d, a, myBlock[10], 23'u8, 0xBEBFBC70'u32)
+  HH(a, b, c, d, myBlock[13], 4'u8, 0x289B7EC6'u32)
+  HH(d, a, b, c, myBlock[0], 11'u8, 0xEAA127FA'u32)
+  HH(c, d, a, b, myBlock[3], 16'u8, 0xD4EF3085'u32)
+  HH(b, c, d, a, myBlock[6], 23'u8, 0x04881D05'u32)
+  HH(a, b, c, d, myBlock[9], 4'u8, 0xD9D4D039'u32)
+  HH(d, a, b, c, myBlock[12], 11'u8, 0xE6DB99E5'u32)
+  HH(c, d, a, b, myBlock[15], 16'u8, 0x1FA27CF8'u32)
+  HH(b, c, d, a, myBlock[2], 23'u8, 0xC4AC5665'u32)
+  II(a, b, c, d, myBlock[0], 6'u8, 0xF4292244'u32)
+  II(d, a, b, c, myBlock[7], 10'u8, 0x432AFF97'u32)
+  II(c, d, a, b, myBlock[14], 15'u8, 0xAB9423A7'u32)
+  II(b, c, d, a, myBlock[5], 21'u8, 0xFC93A039'u32)
+  II(a, b, c, d, myBlock[12], 6'u8, 0x655B59C3'u32)
+  II(d, a, b, c, myBlock[3], 10'u8, 0x8F0CCC92'u32)
+  II(c, d, a, b, myBlock[10], 15'u8, 0xFFEFF47D'u32)
+  II(b, c, d, a, myBlock[1], 21'u8, 0x85845DD1'u32)
+  II(a, b, c, d, myBlock[8], 6'u8, 0x6FA87E4F'u32)
+  II(d, a, b, c, myBlock[15], 10'u8, 0xFE2CE6E0'u32)
+  II(c, d, a, b, myBlock[6], 15'u8, 0xA3014314'u32)
+  II(b, c, d, a, myBlock[13], 21'u8, 0x4E0811A1'u32)
+  II(a, b, c, d, myBlock[4], 6'u8, 0xF7537E82'u32)
+  II(d, a, b, c, myBlock[11], 10'u8, 0xBD3AF235'u32)
+  II(c, d, a, b, myBlock[2], 15'u8, 0x2AD7D2BB'u32)
+  II(b, c, d, a, myBlock[9], 21'u8, 0xEB86D391'u32)
+  state[0] = state[0] + a
+  state[1] = state[1] + b
+  state[2] = state[2] + c
+  state[3] = state[3] + d
+
+proc md5Init*(c: var MD5Context) {.raises: [], tags: [], gcsafe.}
+proc md5Update*(c: var MD5Context, input: openArray[uint8]) {.raises: [],
+    tags: [], gcsafe.}
+proc md5Final*(c: var MD5Context, digest: var MD5Digest) {.raises: [], tags: [], gcsafe.}
+
+proc md5Update*(c: var MD5Context, input: cstring, len: int) {.raises: [],
+    tags: [], gcsafe.} =
+  ## Updates the `MD5Context` with the `input` data of length `len`.
+  ##
+  ## If you use the `toMD5 proc <#toMD5,string>`_, there's no need to call this
+  ## function explicitly.
+  md5Update(c, input.slice(0, len - 1))
+
+
+proc toMD5*(s: string): MD5Digest =
+  ## Computes the `MD5Digest` value for a string `s`.
+  ##
+  ## **See also:**
+  ## * `getMD5 proc <#getMD5,string>`_ which returns a string representation
+  ##   of the `MD5Digest`
+  ## * `$ proc <#$,MD5Digest>`_ for converting MD5Digest to string
+  runnableExamples:
+    assert $toMD5("abc") == "900150983cd24fb0d6963f7d28e17f72"
+
+  var c: MD5Context
+  md5Init(c)
+  md5Update(c, s.slice(0, s.len - 1))
+  md5Final(c, result)
+
+proc `$`*(d: MD5Digest): string =
+  ## Converts a `MD5Digest` value into its string representation.
+  const digits = "0123456789abcdef"
+  result = ""
+  for i in 0..15:
+    add(result, digits[(d[i].int shr 4) and 0xF])
+    add(result, digits[d[i].int and 0xF])
+
+proc getMD5*(s: string): string =
+  ## Computes an MD5 value of `s` and returns its string representation.
+  ##
+  ## **See also:**
+  ## * `toMD5 proc <#toMD5,string>`_ which returns the `MD5Digest` of a string
+  runnableExamples:
+    assert getMD5("abc") == "900150983cd24fb0d6963f7d28e17f72"
+
+  var
+    c: MD5Context
+    d: MD5Digest
+  md5Init(c)
+  md5Update(c, s.slice(0, s.len - 1))
+  md5Final(c, d)
+  result = $d
+
+proc `==`*(D1, D2: MD5Digest): bool =
+  ## Checks if two `MD5Digest` values are identical.
+  for i in 0..15:
+    if D1[i] != D2[i]: return false
+  return true
+
+
+proc clearBuffer(c: var MD5Context) {.inline.} =
+  zeroMem(addr(c.buffer), sizeof(MD5Buffer))
+
+proc md5Init*(c: var MD5Context) =
+  ## Initializes an `MD5Context`.
+  ##
+  ## If you use the `toMD5 proc <#toMD5,string>`_, there's no need to call this
+  ## function explicitly.
+  c.state[0] = 0x67452301'u32
+  c.state[1] = 0xEFCDAB89'u32
+  c.state[2] = 0x98BADCFE'u32
+  c.state[3] = 0x10325476'u32
+  c.count[0] = 0'u32
+  c.count[1] = 0'u32
+  clearBuffer(c)
+
+proc writeBuffer(c: var MD5Context, index: int,
+                 input: openArray[uint8], inputIndex, len: int) {.inline.} =
+  copyMem(addr(c.buffer[index]), unsafeAddr(input[inputIndex]), len)
+
+proc md5Update*(c: var MD5Context, input: openArray[uint8]) =
+  ## Updates the `MD5Context` with the `input` data.
+  ##
+  ## If you use the `toMD5 proc <#toMD5,string>`_, there's no need to call this
+  ## function explicitly.
+  var Index = int((c.count[0] shr 3) and 0x3F)
+  c.count[0] = c.count[0] + (uint32(input.len) shl 3)
+  if c.count[0] < (uint32(input.len) shl 3): c.count[1] = c.count[1] + 1'u32
+  c.count[1] = c.count[1] + (uint32(input.len) shr 29)
+  var PartLen = 64 - Index
+  if input.len >= PartLen:
+    writeBuffer(c, Index, input, 0, PartLen)
+    transform(c.buffer, c.state)
+    var i = PartLen
+    while i + 63 < input.len:
+      transform(input.slice(i, i + 63), c.state)
+      inc(i, 64)
+    if i < input.len:
+      writeBuffer(c, 0, input, i, input.len - i)
+  elif input.len > 0:
+    writeBuffer(c, Index, input, 0, input.len)
+
+proc md5Final*(c: var MD5Context, digest: var MD5Digest) =
+  ## Finishes the `MD5Context` and stores the result in `digest`.
+  ##
+  ## If you use the `toMD5 proc <#toMD5,string>`_, there's no need to call this
+  ## function explicitly.
+  var
+    Bits: MD5CBits
+    PadLen: int
+  decode(Bits, c.count)
+  var Index = int((c.count[0] shr 3) and 0x3F)
+  if Index < 56: PadLen = 56 - Index
+  else: PadLen = 120 - Index
+  md5Update(c, padding.slice(0, PadLen - 1))
+  md5Update(c, Bits)
+  decode(digest, c.state)
+  clearBuffer(c)
+
+
+when defined(nimHasStyleChecks):
+  {.pop.} #{.push styleChecks: off.}

compiler/modulegraphs.nim

@@ -11,7 +11,7 @@
 ## represents a complete Nim project. Single modules can either be kept in RAM
 ## or stored in a rod-file.
 
-import intsets, tables, hashes, md5
+import intsets, tables, hashes, md5_old
 import ast, astalgo, options, lineinfos,idents, btrees, ropes, msgs, pathutils, packages
 import ic / [packed_ast, ic]
 
@@ -57,18 +57,6 @@ type
     sym*: PSym
     info*: TLineInfo
 
-  PipelinePass* = enum
-    NonePass
-    SemPass
-    JSgenPass
-    CgenPass
-    EvalPass
-    InterpreterPass
-    GenDependPass
-    Docgen2TexPass
-    Docgen2JsonPass
-    Docgen2Pass
-
   ModuleGraph* {.acyclic.} = ref object
     ifaces*: seq[Iface]  ## indexed by int32 fileIdx
     packed*: PackedModuleGraph
@@ -116,7 +104,6 @@ type
     cacheCounters*: Table[string, BiggestInt] # IC: implemented
     cacheTables*: Table[string, BTree[string, PNode]] # IC: implemented
     passes*: seq[TPass]
-    pipelinePass*: PipelinePass
     onDefinition*: proc (graph: ModuleGraph; s: PSym; info: TLineInfo) {.nimcall.}
     onDefinitionResolveForward*: proc (graph: ModuleGraph; s: PSym; info: TLineInfo) {.nimcall.}
     onUsage*: proc (graph: ModuleGraph; s: PSym; info: TLineInfo) {.nimcall.}

compiler/modules.nim

@@ -11,8 +11,13 @@
 
 import
   ast, magicsys, msgs, options,
-  idents, lexer, syntaxes, modulegraphs,
-  lineinfos, pathutils
+  idents, lexer, passes, syntaxes, llstream, modulegraphs,
+  lineinfos, pathutils, tables, packages
+
+when defined(nimPreviewSlimSystem):
+  import std/[syncio, assertions]
+
+import ic / replayer
 
 proc resetSystemArtifacts*(g: ModuleGraph) =
   magicsys.resetSysTypes(g)
@@ -20,12 +25,12 @@ proc resetSystemArtifacts*(g: ModuleGraph) =
 template getModuleIdent(graph: ModuleGraph, filename: AbsoluteFile): PIdent =
   getIdent(graph.cache, splitFile(filename).name)
 
-proc partialInitModule*(result: PSym; graph: ModuleGraph; fileIdx: FileIndex; filename: AbsoluteFile) =
+proc partialInitModule(result: PSym; graph: ModuleGraph; fileIdx: FileIndex; filename: AbsoluteFile) =
   let packSym = getPackage(graph, fileIdx)
   result.owner = packSym
   result.position = int fileIdx
 
-proc newModule*(graph: ModuleGraph; fileIdx: FileIndex): PSym =
+proc newModule(graph: ModuleGraph; fileIdx: FileIndex): PSym =
   let filename = AbsoluteFile toFullPath(graph.config, fileIdx)
   # We cannot call ``newSym`` here, because we have to circumvent the ID
   # mechanism, which we do in order to assign each module a persistent ID.
@@ -33,21 +38,103 @@ proc newModule*(graph: ModuleGraph; fileIdx: FileIndex): PSym =
                 name: getModuleIdent(graph, filename),
                 info: newLineInfo(fileIdx, 1, 1))
   if not isNimIdentifier(result.name.s):
-    rawMessage(graph.config, errGenerated, "invalid module name: '" & result.name.s &
-              "'; a module name must be a valid Nim identifier.")
+    rawMessage(graph.config, errGenerated, "invalid module name: " & result.name.s)
   partialInitModule(result, graph, fileIdx, filename)
   graph.registerModule(result)
 
+proc compileModule*(graph: ModuleGraph; fileIdx: FileIndex; flags: TSymFlags, fromModule: PSym = nil): PSym =
+  var flags = flags
+  if fileIdx == graph.config.projectMainIdx2: flags.incl sfMainModule
+  result = graph.getModule(fileIdx)
+
+  template processModuleAux(moduleStatus) =
+    onProcessing(graph, fileIdx, moduleStatus, fromModule = fromModule)
+    var s: PLLStream
+    if sfMainModule in flags:
+      if graph.config.projectIsStdin: s = stdin.llStreamOpen
+      elif graph.config.projectIsCmd: s = llStreamOpen(graph.config.cmdInput)
+    discard processModule(graph, result, idGeneratorFromModule(result), s)
+  if result == nil:
+    var cachedModules: seq[FileIndex]
+    result = moduleFromRodFile(graph, fileIdx, cachedModules)
+    let filename = AbsoluteFile toFullPath(graph.config, fileIdx)
+    if result == nil:
+      result = newModule(graph, fileIdx)
+      result.flags.incl flags
+      registerModule(graph, result)
+      processModuleAux("import")
+    else:
+      if sfSystemModule in flags:
+        graph.systemModule = result
+      partialInitModule(result, graph, fileIdx, filename)
+    for m in cachedModules:
+      registerModuleById(graph, m)
+      replayStateChanges(graph.packed[m.int].module, graph)
+      replayGenericCacheInformation(graph, m.int)
+  elif graph.isDirty(result):
+    result.flags.excl sfDirty
+    # reset module fields:
+    initStrTables(graph, result)
+    result.ast = nil
+    processModuleAux("import(dirty)")
+    graph.markClientsDirty(fileIdx)
+
+proc importModule*(graph: ModuleGraph; s: PSym, fileIdx: FileIndex): PSym =
+  # this is called by the semantic checking phase
+  assert graph.config != nil
+  result = compileModule(graph, fileIdx, {}, s)
+  graph.addDep(s, fileIdx)
+  # keep track of import relationships
+  if graph.config.hcrOn:
+    graph.importDeps.mgetOrPut(FileIndex(s.position), @[]).add(fileIdx)
+  #if sfSystemModule in result.flags:
+  #  localError(result.info, errAttemptToRedefine, result.name.s)
+  # restore the notes for outer module:
+  graph.config.notes =
+    if graph.config.belongsToProjectPackage(s) or isDefined(graph.config, "booting"): graph.config.mainPackageNotes
+    else: graph.config.foreignPackageNotes
+
 proc includeModule*(graph: ModuleGraph; s: PSym, fileIdx: FileIndex): PNode =
   result = syntaxes.parseFile(fileIdx, graph.cache, graph.config)
   graph.addDep(s, fileIdx)
   graph.addIncludeDep(s.position.FileIndex, fileIdx)
 
+proc connectCallbacks*(graph: ModuleGraph) =
+  graph.includeFileCallback = includeModule
+  graph.importModuleCallback = importModule
+
+proc compileSystemModule*(graph: ModuleGraph) =
+  if graph.systemModule == nil:
+    connectCallbacks(graph)
+    graph.config.m.systemFileIdx = fileInfoIdx(graph.config,
+        graph.config.libpath / RelativeFile"system.nim")
+    discard graph.compileModule(graph.config.m.systemFileIdx, {sfSystemModule})
+
 proc wantMainModule*(conf: ConfigRef) =
   if conf.projectFull.isEmpty:
     fatal(conf, gCmdLineInfo, "command expects a filename")
   conf.projectMainIdx = fileInfoIdx(conf, addFileExt(conf.projectFull, NimExt))
 
+proc compileProject*(graph: ModuleGraph; projectFileIdx = InvalidFileIdx) =
+  connectCallbacks(graph)
+  let conf = graph.config
+  wantMainModule(conf)
+  configComplete(graph)
+
+  let systemFileIdx = fileInfoIdx(conf, conf.libpath / RelativeFile"system.nim")
+  let projectFile = if projectFileIdx == InvalidFileIdx: conf.projectMainIdx else: projectFileIdx
+  conf.projectMainIdx2 = projectFile
+
+  let packSym = getPackage(graph, projectFile)
+  graph.config.mainPackageId = packSym.getPackageId
+  graph.importStack.add projectFile
+
+  if projectFile == systemFileIdx:
+    discard graph.compileModule(projectFile, {sfMainModule, sfSystemModule})
+  else:
+    graph.compileSystemModule()
+    discard graph.compileModule(projectFile, {sfMainModule})
+
 proc makeModule*(graph: ModuleGraph; filename: AbsoluteFile): PSym =
   result = graph.newModule(fileInfoIdx(graph.config, filename))
   registerModule(graph, result)

compiler/msgs.nim

@@ -128,13 +128,6 @@ proc fileInfoIdx*(conf: ConfigRef; filename: AbsoluteFile): FileIndex =
   var dummy: bool
   result = fileInfoIdx(conf, filename, dummy)
 
-proc fileInfoIdx*(conf: ConfigRef; filename: RelativeFile; isKnownFile: var bool): FileIndex =
-  fileInfoIdx(conf, AbsoluteFile expandFilename(filename.string), isKnownFile)
-
-proc fileInfoIdx*(conf: ConfigRef; filename: RelativeFile): FileIndex =
-  var dummy: bool
-  fileInfoIdx(conf, AbsoluteFile expandFilename(filename.string), dummy)
-
 proc newLineInfo*(fileInfoIdx: FileIndex, line, col: int): TLineInfo =
   result.fileIndex = fileInfoIdx
   if line < int high(uint16):
@@ -541,11 +534,10 @@ proc liMessage*(conf: ConfigRef; info: TLineInfo, msg: TMsgKind, arg: string,
     ignoreMsg = not conf.hasWarn(msg)
     if not ignoreMsg and msg in conf.warningAsErrors:
       title = ErrorTitle
-      color = ErrorColor
     else:
       title = WarningTitle
-      color = WarningColor
     if not ignoreMsg: writeContext(conf, info)
+    color = WarningColor
     inc(conf.warnCounter)
   of hintMin..hintMax:
     sev = Severity.Hint

compiler/nim.cfg

@@ -31,10 +31,6 @@ define:useStdoutAsStdmsg
   warning[ObservableStores]:off
 @end
 
-@if nimHasWarnCastSizes:
-  warning[CastSizes]:on
-@end
-
 @if nimHasWarningAsError:
   warningAsError[GcUnsafe2]:on
 @end
@@ -42,11 +38,3 @@ define:useStdoutAsStdmsg
 @if nimHasWarnUnnamedBreak:
   warningAserror[UnnamedBreak]:on
 @end
-
-@if nimHasWarnBareExcept:
-  warningAserror[BareExcept]:on
-@end
-
-@if nimHasWarnCopyHookForRefc:
-  warningAserror[CopyHookForRefc]:on
-@end

compiler/nim.nim

@@ -41,15 +41,6 @@ when defined(profiler) or defined(memProfiler):
   {.hint: "Profiling support is turned on!".}
   import nimprof
 
-proc nimbleLockExists(config: ConfigRef): bool =
-  const nimbleLock = "nimble.lock"
-  let pd = if not config.projectPath.isEmpty: config.projectPath else: AbsoluteDir(getCurrentDir())
-  if optSkipParentConfigFiles notin config.globalOptions:
-    for dir in parentDirs(pd.string, fromRoot=true, inclusive=false):
-      if fileExists(dir / nimbleLock):
-        return true
-  return fileExists(pd.string / nimbleLock)
-
 proc processCmdLine(pass: TCmdLinePass, cmd: string; config: ConfigRef) =
   var p = parseopt.initOptParser(cmd)
   var argsCount = 0
@@ -83,11 +74,6 @@ proc processCmdLine(pass: TCmdLinePass, cmd: string; config: ConfigRef) =
         config.arguments.len > 0 and config.cmd notin {cmdTcc, cmdNimscript, cmdCrun}:
       rawMessage(config, errGenerated, errArgsNeedRunOption)
 
-  if config.nimbleLockExists:
-    # disable nimble path if nimble.lock is present.
-    # see https://github.com/nim-lang/nimble/issues/1004
-    disableNimblePath(config)
-
 proc getNimRunExe(conf: ConfigRef): string =
   # xxx consider defining `conf.getConfigVar("nimrun.exe")` to allow users to
   # customize the binary to run the command with, e.g. for custom `nodejs` or `wine`.

compiler/nimeval.nim

@@ -9,16 +9,10 @@
 
 ## exposes the Nim VM to clients.
 import
-  ast, modules, condsyms,
-  options, llstream, lineinfos, vm,
+  ast, astalgo, modules, passes, condsyms,
+  options, sem, llstream, lineinfos, vm,
   vmdef, modulegraphs, idents, os, pathutils,
-  scriptconfig, std/compilesettings
-
-import pipelines
-
-
-when defined(nimPreviewSlimSystem):
-  import std/[assertions, syncio]
+  passaux, scriptconfig, std/compilesettings
 
 type
   Interpreter* = ref object ## Use Nim as an interpreter with this object
@@ -82,7 +76,7 @@ proc evalScript*(i: Interpreter; scriptStream: PLLStream = nil) =
 
   let s = if scriptStream != nil: scriptStream
           else: llStreamOpen(findFile(i.graph.config, i.scriptName), fmRead)
-  discard processPipelineModule(i.graph, i.mainModule, i.idgen, s)
+  processModule(i.graph, i.mainModule, i.idgen, s)
 
 proc findNimStdLib*(): string =
   ## Tries to find a path to a valid "system.nim" file.
@@ -115,10 +109,12 @@ proc createInterpreter*(scriptName: string;
   var conf = newConfigRef()
   var cache = newIdentCache()
   var graph = newModuleGraph(cache, conf)
-  connectPipelineCallbacks(graph)
+  connectCallbacks(graph)
   initDefines(conf.symbols)
   for define in defines:
     defineSymbol(conf.symbols, define[0], define[1])
+  registerPass(graph, semPass)
+  registerPass(graph, evalPass)
 
   for p in searchPaths:
     conf.searchPaths.add(AbsoluteDir p)
@@ -133,8 +129,7 @@ proc createInterpreter*(scriptName: string;
   if registerOps:
     vm.registerAdditionalOps() # Required to register parts of stdlib modules
   graph.vm = vm
-  setPipeLinePass(graph, EvalPass)
-  graph.compilePipelineSystemModule()
+  graph.compileSystemModule()
   result = Interpreter(mainModule: m, graph: graph, scriptName: scriptName, idgen: idgen)
 
 proc destroyInterpreter*(i: Interpreter) =
@@ -164,11 +159,13 @@ proc runRepl*(r: TLLRepl;
   defineSymbol(conf.symbols, "nimscript")
   if supportNimscript: defineSymbol(conf.symbols, "nimconfig")
   when hasFFI: defineSymbol(graph.config.symbols, "nimffi")
+  registerPass(graph, verbosePass)
+  registerPass(graph, semPass)
+  registerPass(graph, evalPass)
   var m = graph.makeStdinModule()
   incl(m.flags, sfMainModule)
   var idgen = idGeneratorFromModule(m)
 
   if supportNimscript: graph.vm = setupVM(m, cache, "stdin", graph, idgen)
-  setPipeLinePass(graph, InterpreterPass)
-  graph.compilePipelineSystemModule()
-  discard processPipelineModule(graph, m, idgen, llStreamOpenStdIn(r))
+  graph.compileSystemModule()
+  processModule(graph, m, idgen, llStreamOpenStdIn(r))

compiler/nodejs.nim

@@ -7,4 +7,4 @@ proc findNodeJs*(): string {.inline.} =
     result = findExe("node")
   if result.len == 0:
     echo "Please install NodeJS first, see https://nodejs.org/en/download"
-    raise newException(IOError, "NodeJS not found in PATH")
+    raise newException(IOError, "NodeJS not found in PATH: " & result)

compiler/optimizer.nim

@@ -16,8 +16,6 @@ import
 
 from trees import exprStructuralEquivalent
 
-import std/strutils
-
 const
   nfMarkForDeletion = nfNone # faster than a lookup table
 
@@ -112,17 +110,16 @@ proc analyse(c: var Con; b: var BasicBlock; n: PNode) =
     var reverse = false
     if n[0].kind == nkSym:
       let s = n[0].sym
-      let name = s.name.s.normalize
-      if s.magic == mWasMoved or name == "=wasmoved":
+      if s.magic == mWasMoved:
         b.wasMovedLocs.add n
         special = true
-      elif name == "=destroy":
+      elif s.name.s == "=destroy":
         if c.inFinally > 0 and (b.hasReturn or b.hasBreak):
           discard "cannot optimize away the destructor"
         else:
           c.wasMovedDestroyPair b, n
         special = true
-      elif name == "=sink":
+      elif s.name.s == "=sink":
         reverse = true
 
     if not special:

compiler/options.nim

@@ -24,7 +24,7 @@ const
   useEffectSystem* = true
   useWriteTracking* = false
   hasFFI* = defined(nimHasLibFFI)
-  copyrightYear* = "2023"
+  copyrightYear* = "2022"
 
   nimEnableCovariance* = defined(nimEnableCovariance)
 
@@ -78,8 +78,6 @@ type                          # please make sure we have under 32 options
     optThreadAnalysis,        # thread analysis pass
     optTlsEmulation,          # thread var emulation turned on
     optGenIndex               # generate index file for documentation;
-    optGenIndexOnly           # generate only index file for documentation
-    optNoImportdoc            # disable loading external documentation files
     optEmbedOrigSrc           # embed the original source in the generated code
                               # also: generate header file
     optIdeDebug               # idetools: debug mode
@@ -102,11 +100,12 @@ type                          # please make sure we have under 32 options
     optBenchmarkVM            # Enables cpuTime() in the VM
     optProduceAsm             # produce assembler code
     optPanics                 # turn panics (sysFatal) into a process termination
+    optNimV1Emulation         # emulate Nim v1.0
+    optNimV12Emulation        # emulate Nim v1.2
+    optNimV16Emulation        # emulate Nim v1.6
     optSourcemap
     optProfileVM              # enable VM profiler
     optEnableDeepCopy         # ORC specific: enable 'deepcopy' for all types.
-    optShowNonExportedFields  # for documentation: show fields that are not exported
-    optJsBigInt64             # use bigints for 64-bit integers in JS
 
   TGlobalOptions* = set[TGlobalOption]
 
@@ -194,9 +193,10 @@ type
   IdeCmd* = enum
     ideNone, ideSug, ideCon, ideDef, ideUse, ideDus, ideChk, ideChkFile, ideMod,
     ideHighlight, ideOutline, ideKnown, ideMsg, ideProject, ideGlobalSymbols,
-    ideRecompile, ideChanged, ideType, ideDeclaration, ideExpand
+    ideRecompile, ideChanged, ideType, ideDeclaration
 
   Feature* = enum  ## experimental features; DO NOT RENAME THESE!
+    implicitDeref, # deadcode; remains here for backwards compatibility
     dotOperators,
     callOperator,
     parallel,
@@ -232,7 +232,6 @@ type
       ## are not anymore.
     laxEffects
       ## Lax effects system prior to Nim 2.0.
-    verboseTypeMismatch
 
   SymbolFilesOption* = enum
     disabledSf, writeOnlySf, readOnlySf, v2Sf, stressTest
@@ -277,9 +276,6 @@ type
     scope*, localUsages*, globalUsages*: int # more usages is better
     tokenLen*: int
     version*: int
-    endLine*: uint16
-    endCol*: int
-
   Suggestions* = seq[Suggest]
 
   ProfileInfo* = object
@@ -410,11 +406,6 @@ type
     nimMainPrefix*: string
     vmProfileData*: ProfileData
 
-    expandProgress*: bool
-    expandLevels*: int
-    expandNodeResult*: string
-    expandPosition*: TLineInfo
-
 proc parseNimVersion*(a: string): NimVer =
   # could be moved somewhere reusable
   if a.len > 0:
@@ -466,7 +457,7 @@ when false:
     fn(globalOptions)
     fn(selectedGC)
 
-const oldExperimentalFeatures* = {dotOperators, callOperator, parallel}
+const oldExperimentalFeatures* = {implicitDeref, dotOperators, callOperator, parallel}
 
 const
   ChecksOptions* = {optObjCheck, optFieldCheck, optRangeCheck,
@@ -477,8 +468,7 @@ const
     optBoundsCheck, optOverflowCheck, optAssert, optWarns, optRefCheck,
     optHints, optStackTrace, optLineTrace, # consider adding `optStackTraceMsgs`
     optTrMacros, optStyleCheck, optCursorInference}
-  DefaultGlobalOptions* = {optThreadAnalysis, optExcessiveStackTrace,
-    optJsBigInt64}
+  DefaultGlobalOptions* = {optThreadAnalysis, optExcessiveStackTrace}
 
 proc getSrcTimestamp(): DateTime =
   try:
@@ -620,7 +610,7 @@ proc isDefined*(conf: ConfigRef; symbol: string): bool =
                             osQnx, osAtari, osAix,
                             osHaiku, osVxWorks, osSolaris, osNetbsd,
                             osFreebsd, osOpenbsd, osDragonfly, osMacosx, osIos,
-                            osAndroid, osNintendoSwitch, osFreeRTOS, osCrossos, osZephyr, osNuttX}
+                            osAndroid, osNintendoSwitch, osFreeRTOS, osCrossos, osZephyr}
     of "linux":
       result = conf.target.targetOS in {osLinux, osAndroid}
     of "bsd":
@@ -642,8 +632,6 @@ proc isDefined*(conf: ConfigRef; symbol: string): bool =
       result = conf.target.targetOS == osFreeRTOS
     of "zephyr":
       result = conf.target.targetOS == osZephyr
-    of "nuttx":
-      result = conf.target.targetOS == osNuttX
     of "littleendian": result = CPU[conf.target.targetCPU].endian == littleEndian
     of "bigendian": result = CPU[conf.target.targetCPU].endian == bigEndian
     of "cpu8": result = CPU[conf.target.targetCPU].bit == 8
@@ -718,24 +706,22 @@ proc getPrefixDir*(conf: ConfigRef): AbsoluteDir =
   ##  clone or using installed nim, so that these exist: `result/doc/advopt.txt`
   ## and `result/lib/system.nim`
   if not conf.prefixDir.isEmpty: result = conf.prefixDir
-  else:
-    let binParent = AbsoluteDir splitPath(getAppDir()).head
-    when defined(posix):
-      if binParent == AbsoluteDir"/usr":
-        result = AbsoluteDir"/usr/lib/nim"
-      elif binParent == AbsoluteDir"/usr/local":
-        result = AbsoluteDir"/usr/local/lib/nim"
-      else:
-        result = binParent
-    else:
-      result = binParent
+  else: result = AbsoluteDir splitPath(getAppDir()).head
 
 proc setDefaultLibpath*(conf: ConfigRef) =
   # set default value (can be overwritten):
   if conf.libpath.isEmpty:
     # choose default libpath:
     var prefix = getPrefixDir(conf)
-    conf.libpath = prefix / RelativeDir"lib"
+    when defined(posix):
+      if prefix == AbsoluteDir"/usr":
+        conf.libpath = AbsoluteDir"/usr/lib/nim"
+      elif prefix == AbsoluteDir"/usr/local":
+        conf.libpath = AbsoluteDir"/usr/local/lib/nim"
+      else:
+        conf.libpath = prefix / RelativeDir"lib"
+    else:
+      conf.libpath = prefix / RelativeDir"lib"
 
     # Special rule to support other tools (nimble) which import the compiler
     # modules and make use of them.
@@ -879,8 +865,7 @@ const stdlibDirs* = [
   "pure/concurrency",
   "pure/unidecode", "impure",
   "wrappers", "wrappers/linenoise",
-  "windows", "posix", "js",
-  "deprecated/pure"]
+  "windows", "posix", "js"]
 
 const
   pkgPrefix = "pkg/"
@@ -1008,12 +993,6 @@ proc isDynlibOverride*(conf: ConfigRef; lib: string): bool =
   result = optDynlibOverrideAll in conf.globalOptions or
      conf.dllOverrides.hasKey(lib.canonDynlibName)
 
-proc showNonExportedFields*(conf: ConfigRef) =
-  incl(conf.globalOptions, optShowNonExportedFields)
-
-proc expandDone*(conf: ConfigRef): bool =
-  result = conf.ideCmd == ideExpand and conf.expandLevels == 0 and conf.expandProgress
-
 proc parseIdeCmd*(s: string): IdeCmd =
   case s:
   of "sug": ideSug
@@ -1053,7 +1032,6 @@ proc `$`*(c: IdeCmd): string =
   of ideProject: "project"
   of ideGlobalSymbols: "globalSymbols"
   of ideDeclaration: "declaration"
-  of ideExpand: "expand"
   of ideRecompile: "recompile"
   of ideChanged: "changed"
   of ideType: "type"

compiler/parser.nim

@@ -42,7 +42,7 @@ when isMainModule or defined(nimTestGrammar):
 
     proc checkSameGrammar*() =
       doAssert sameFileContent(newGrammarText, "doc/grammar.txt"),
-              "execute 'nim r compiler/parser.nim' to keep grammar.txt up-to-date"
+              "execute 'nim r compiler.nim' to keep grammar.txt up-to-date"
   else:
     writeGrammarFile("doc/grammar.txt")
     import ".." / tools / grammar_nanny
@@ -79,7 +79,7 @@ type
     smNormal, smAllowNil, smAfterDot
 
   PrimaryMode = enum
-    pmNormal, pmTypeDesc, pmTypeDef, pmTrySimple
+    pmNormal, pmTypeDesc, pmTypeDef, pmSkipSuffix
 
 proc parseAll*(p: var Parser): PNode
 proc closeParser*(p: var Parser)
@@ -286,8 +286,7 @@ proc newIdentNodeP(ident: PIdent, p: Parser): PNode =
 
 proc parseExpr(p: var Parser): PNode
 proc parseStmt(p: var Parser): PNode
-proc parseTypeDesc(p: var Parser, fullExpr = false): PNode
-proc parseTypeDefValue(p: var Parser): PNode
+proc parseTypeDesc(p: var Parser): PNode
 proc parseParamList(p: var Parser, retColon = true): PNode
 
 proc isSigilLike(tok: Token): bool {.inline.} =
@@ -354,12 +353,6 @@ proc colcom(p: var Parser, n: PNode) =
 
 const tkBuiltInMagics = {tkType, tkStatic, tkAddr}
 
-template setEndInfo() =
-  when defined(nimsuggest):
-    result.endInfo = TLineInfo(fileIndex: p.lex.fileIdx,
-                     line: p.lex.previousTokenEnd.line,
-                     col: p.lex.previousTokenEnd.col)
-
 proc parseSymbol(p: var Parser, mode = smNormal): PNode =
   #| symbol = '`' (KEYW|IDENT|literal|(operator|'('|')'|'['|']'|'{'|'}'|'=')+)+ '`'
   #|        | IDENT | KEYW
@@ -412,17 +405,6 @@ proc parseSymbol(p: var Parser, mode = smNormal): PNode =
     # if it is a keyword:
     #if not isKeyword(p.tok.tokType): getTok(p)
     result = p.emptyNode
-  setEndInfo()
-
-proc equals(p: var Parser, a: PNode): PNode =
-  if p.tok.tokType == tkEquals:
-    result = newNodeP(nkExprEqExpr, p)
-    getTok(p)
-    #optInd(p, result)
-    result.add(a)
-    result.add(parseExpr(p))
-  else:
-    result = a
 
 proc colonOrEquals(p: var Parser, a: PNode): PNode =
   if p.tok.tokType == tkColon:
@@ -432,8 +414,14 @@ proc colonOrEquals(p: var Parser, a: PNode): PNode =
     #optInd(p, result)
     result.add(a)
     result.add(parseExpr(p))
+  elif p.tok.tokType == tkEquals:
+    result = newNodeP(nkExprEqExpr, p)
+    getTok(p)
+    #optInd(p, result)
+    result.add(a)
+    result.add(parseExpr(p))
   else:
-    result = equals(p, a)
+    result = a
 
 proc exprColonEqExpr(p: var Parser): PNode =
   #| exprColonEqExpr = expr (':'|'=' expr)?
@@ -443,14 +431,6 @@ proc exprColonEqExpr(p: var Parser): PNode =
   else:
     result = colonOrEquals(p, a)
 
-proc exprEqExpr(p: var Parser): PNode =
-  #| exprEqExpr = expr ('=' expr)?
-  var a = parseExpr(p)
-  if p.tok.tokType == tkDo:
-    result = postExprBlocks(p, a)
-  else:
-    result = equals(p, a)
-
 proc exprList(p: var Parser, endTok: TokType, result: PNode) =
   #| exprList = expr ^+ comma
   when defined(nimpretty):
@@ -458,24 +438,6 @@ proc exprList(p: var Parser, endTok: TokType, result: PNode) =
   getTok(p)
   optInd(p, result)
   # progress guaranteed
-  var a = parseExpr(p)
-  result.add(a)
-  while (p.tok.tokType != endTok) and (p.tok.tokType != tkEof):
-    if p.tok.tokType != tkComma: break
-    getTok(p)
-    optInd(p, a)
-    var a = parseExpr(p)
-    result.add(a)
-  when defined(nimpretty):
-    dec p.em.doIndentMore
-
-proc optionalExprList(p: var Parser, endTok: TokType, result: PNode) =
-  #| optionalExprList = expr ^* comma
-  when defined(nimpretty):
-    inc p.em.doIndentMore
-  getTok(p)
-  optInd(p, result)
-  # progress guaranteed
   while (p.tok.tokType != endTok) and (p.tok.tokType != tkEof):
     var a = parseExpr(p)
     result.add(a)
@@ -584,7 +546,6 @@ proc parseCast(p: var Parser): PNode =
     result.add(exprColonEqExpr(p))
   optPar(p)
   eat(p, tkParRi)
-  setEndInfo()
 
 proc setBaseFlags(n: PNode, base: NumericalBase) =
   case base
@@ -607,7 +568,6 @@ proc parseGStrLit(p: var Parser, a: PNode): PNode =
     getTok(p)
   else:
     result = a
-  setEndInfo()
 
 proc complexOrSimpleStmt(p: var Parser): PNode
 proc simpleExpr(p: var Parser, mode = pmNormal): PNode
@@ -712,7 +672,6 @@ proc parsePar(p: var Parser): PNode =
           skipComment(p, a)
   optPar(p)
   eat(p, tkParRi)
-  setEndInfo()
 
 proc identOrLiteral(p: var Parser, mode: PrimaryMode): PNode =
   #| literal = | INT_LIT | INT8_LIT | INT16_LIT | INT32_LIT | INT64_LIT
@@ -843,24 +802,25 @@ proc namedParams(p: var Parser, callee: PNode,
 proc commandParam(p: var Parser, isFirstParam: var bool; mode: PrimaryMode): PNode =
   if mode == pmTypeDesc:
     result = simpleExpr(p, mode)
-  elif not isFirstParam:
-    result = exprEqExpr(p)
   else:
     result = parseExpr(p)
-    if p.tok.tokType == tkDo:
-      result = postExprBlocks(p, result)
+  if p.tok.tokType == tkDo:
+    result = postExprBlocks(p, result)
+  elif p.tok.tokType == tkEquals and not isFirstParam:
+    let lhs = result
+    result = newNodeP(nkExprEqExpr, p)
+    getTok(p)
+    result.add(lhs)
+    result.add(parseExpr(p))
   isFirstParam = false
 
 proc commandExpr(p: var Parser; r: PNode; mode: PrimaryMode): PNode =
-  if mode == pmTrySimple:
-    result = r
-  else:
-    result = newNodeP(nkCommand, p)
-    result.add(r)
-    var isFirstParam = true
-    # progress NOT guaranteed
-    p.hasProgress = false
-    result.add commandParam(p, isFirstParam, mode)
+  result = newNodeP(nkCommand, p)
+  result.add(r)
+  var isFirstParam = true
+  # progress NOT guaranteed
+  p.hasProgress = false
+  result.add commandParam(p, isFirstParam, mode)
 
 proc isDotLike(tok: Token): bool =
   result = tok.tokType == tkOpr and tok.ident.s.len > 1 and
@@ -873,7 +833,7 @@ proc primarySuffix(p: var Parser, r: PNode,
   #|       | DOTLIKEOP optInd symbol generalizedLit?
   #|       | '[' optInd exprColonEqExprList optPar ']'
   #|       | '{' optInd exprColonEqExprList optPar '}'
-  # XXX strong spaces need to be reflected above
+  #|       | &( '`'|IDENT|literal|'cast'|'addr'|'type') expr (comma expr)* # command syntax
   result = r
 
   # progress guaranteed
@@ -884,6 +844,13 @@ proc primarySuffix(p: var Parser, r: PNode,
       # progress guaranteed
       if p.tok.strongSpaceA:
         result = commandExpr(p, result, mode)
+        # type sections allow full command syntax
+        if mode == pmTypeDef:
+          var isFirstParam = false
+          while p.tok.tokType == tkComma:
+            getTok(p)
+            optInd(p, result)
+            result.add(commandParam(p, isFirstParam, mode))
         break
       result = namedParams(p, result, nkCall, tkParRi)
       if result.len > 1 and result[1].kind == nkExprColonExpr:
@@ -924,9 +891,18 @@ proc primarySuffix(p: var Parser, r: PNode,
           # actually parsing {.push hints:off.} as {.push(hints:off).} is a sweet
           # solution, but pragmas.nim can't handle that
           result = commandExpr(p, result, mode)
+          if mode == pmTypeDef:
+            var isFirstParam = false
+            while p.tok.tokType == tkComma:
+              getTok(p)
+              optInd(p, result)
+              result.add(commandParam(p, isFirstParam, mode))
         break
     else:
       break
+  # type sections allow post-expr blocks
+  if mode == pmTypeDef:
+    result = postExprBlocks(p, result)
 
 proc parseOperators(p: var Parser, headNode: PNode,
                     limit: int, mode: PrimaryMode): PNode =
@@ -951,13 +927,9 @@ proc parseOperators(p: var Parser, headNode: PNode,
     a.add(b)
     result = a
     opPrec = getPrecedence(p.tok)
-  setEndInfo()
 
 proc simpleExprAux(p: var Parser, limit: int, mode: PrimaryMode): PNode =
-  var mode = mode
   result = primary(p, mode)
-  if mode == pmTrySimple:
-    mode = pmNormal
   if p.tok.tokType == tkCurlyDotLe and (p.tok.indent < 0 or realInd(p)) and
      mode == pmNormal:
     var pragmaExp = newNodeP(nkPragmaExpr, p)
@@ -1001,7 +973,6 @@ proc parsePragma(p: var Parser): PNode =
   when defined(nimpretty):
     dec p.em.doIndentMore
     dec p.em.keepIndents
-  setEndInfo()
 
 proc identVis(p: var Parser; allowDot=false): PNode =
   #| identVis = symbol OPR?  # postfix position
@@ -1039,9 +1010,9 @@ type
 
 proc parseIdentColonEquals(p: var Parser, flags: DeclaredIdentFlags): PNode =
   #| declColonEquals = identWithPragma (comma identWithPragma)* comma?
-  #|                   (':' optInd typeDescExpr)? ('=' optInd expr)?
+  #|                   (':' optInd typeDesc)? ('=' optInd expr)?
   #| identColonEquals = IDENT (comma IDENT)* comma?
-  #|      (':' optInd typeDescExpr)? ('=' optInd expr)?)
+  #|      (':' optInd typeDesc)? ('=' optInd expr)?)
   var a: PNode
   result = newNodeP(nkIdentDefs, p)
   # progress guaranteed
@@ -1059,7 +1030,7 @@ proc parseIdentColonEquals(p: var Parser, flags: DeclaredIdentFlags): PNode =
   if p.tok.tokType == tkColon:
     getTok(p)
     optInd(p, result)
-    result.add(parseTypeDesc(p, fullExpr = true))
+    result.add(parseTypeDesc(p))
   else:
     result.add(newNodeP(nkEmpty, p))
     if p.tok.tokType != tkEquals and withBothOptional notin flags:
@@ -1070,13 +1041,11 @@ proc parseIdentColonEquals(p: var Parser, flags: DeclaredIdentFlags): PNode =
     result.add(parseExpr(p))
   else:
     result.add(newNodeP(nkEmpty, p))
-  setEndInfo()
 
 proc parseTuple(p: var Parser, indentAllowed = false): PNode =
-  #| tupleTypeBracket = '[' optInd (identColonEquals (comma/semicolon)?)* optPar ']'
-  #| tupleType = 'tuple' tupleTypeBracket
-  #| tupleDecl = 'tuple' (tupleTypeBracket /
-  #|     COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)?)
+  #| tupleDecl = 'tuple'
+  #|     '[' optInd  (identColonEquals (comma/semicolon)?)*  optPar ']' |
+  #|     COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)?
   result = newNodeP(nkTupleTy, p)
   getTok(p)
   if p.tok.tokType == tkBracketLe:
@@ -1115,7 +1084,6 @@ proc parseTuple(p: var Parser, indentAllowed = false): PNode =
     parMessage(p, errGenerated, "the syntax for tuple types is 'tuple[...]', not 'tuple(...)'")
   else:
     result = newNodeP(nkTupleClassTy, p)
-  setEndInfo()
 
 proc parseParamList(p: var Parser, retColon = true): PNode =
   #| paramList = '(' declColonEquals ^* (comma/semicolon) ')'
@@ -1164,7 +1132,6 @@ proc parseParamList(p: var Parser, retColon = true): PNode =
   when defined(nimpretty):
     dec p.em.doIndentMore
     dec p.em.keepIndents
-  setEndInfo()
 
 proc optPragmas(p: var Parser): PNode =
   if p.tok.tokType == tkCurlyDotLe and (p.tok.indent < 0 or realInd(p)):
@@ -1185,11 +1152,9 @@ proc parseDoBlock(p: var Parser; info: TLineInfo): PNode =
     result = newProcNode(nkDo, info,
       body = result, params = params, name = p.emptyNode, pattern = p.emptyNode,
       genericParams = p.emptyNode, pragmas = pragmas, exceptions = p.emptyNode)
-  setEndInfo()
 
 proc parseProcExpr(p: var Parser; isExpr: bool; kind: TNodeKind): PNode =
   #| routineExpr = ('proc' | 'func' | 'iterator') paramListColon pragma? ('=' COMMENT? stmt)?
-  #| routineType = ('proc' | 'iterator') paramListColon pragma?
   # either a proc type or a anonymous proc
   let info = parLineInfo(p)
   let hasSignature = p.tok.tokType in {tkParLe, tkColon} and p.tok.indent < 0
@@ -1197,29 +1162,24 @@ proc parseProcExpr(p: var Parser; isExpr: bool; kind: TNodeKind): PNode =
   let pragmas = optPragmas(p)
   if p.tok.tokType == tkEquals and isExpr:
     getTok(p)
-    result = newProcNode(kind, info, body = p.emptyNode,
+    skipComment(p, result)
+    result = newProcNode(kind, info, body = parseStmt(p),
       params = params, name = p.emptyNode, pattern = p.emptyNode,
       genericParams = p.emptyNode, pragmas = pragmas, exceptions = p.emptyNode)
-    skipComment(p, result)
-    result[bodyPos] = parseStmt(p)
   else:
     result = newNodeI(if kind == nkIteratorDef: nkIteratorTy else: nkProcTy, info)
-    if hasSignature or pragmas.kind != nkEmpty:
-      if hasSignature:
-        result.add(params)
-      else: # pragmas but no param list, implies typeclass with pragmas
-        result.add(p.emptyNode)
+    if hasSignature:
+      result.add(params)
       if kind == nkFuncDef:
         parMessage(p, "func keyword is not allowed in type descriptions, use proc with {.noSideEffect.} pragma instead")
       result.add(pragmas)
-  setEndInfo()
 
 proc isExprStart(p: Parser): bool =
   case p.tok.tokType
   of tkSymbol, tkAccent, tkOpr, tkNot, tkNil, tkCast, tkIf, tkFor,
      tkProc, tkFunc, tkIterator, tkBind, tkBuiltInMagics,
      tkParLe, tkBracketLe, tkCurlyLe, tkIntLit..tkCustomLit, tkVar, tkRef, tkPtr,
-     tkEnum, tkTuple, tkObject, tkWhen, tkCase, tkOut, tkTry, tkBlock:
+     tkTuple, tkObject, tkWhen, tkCase, tkOut, tkTry, tkBlock:
     result = true
   else: result = false
 
@@ -1232,7 +1192,6 @@ proc parseSymbolList(p: var Parser, result: PNode) =
     if p.tok.tokType != tkComma: break
     getTok(p)
     optInd(p, s)
-  setEndInfo()
 
 proc parseTypeDescKAux(p: var Parser, kind: TNodeKind,
                        mode: PrimaryMode): PNode =
@@ -1240,12 +1199,8 @@ proc parseTypeDescKAux(p: var Parser, kind: TNodeKind,
   getTok(p)
   if p.tok.indent != -1 and p.tok.indent <= p.currInd: return
   optInd(p, result)
-  let isTypedef = mode == pmTypeDef and p.tok.tokType in {tkObject, tkTuple}
   if not isOperator(p.tok) and isExprStart(p):
-    if isTypedef:
-      result.add(parseTypeDefValue(p))
-    else:
-      result.add(primary(p, mode))
+    result.add(primary(p, mode))
   if kind == nkDistinctTy and p.tok.tokType == tkSymbol:
     # XXX document this feature!
     var nodeKind: TNodeKind
@@ -1259,14 +1214,11 @@ proc parseTypeDescKAux(p: var Parser, kind: TNodeKind,
     let list = newNodeP(nodeKind, p)
     result.add list
     parseSymbolList(p, list)
-  if mode == pmTypeDef and not isTypedef:
-    result = parseOperators(p, result, -1, mode)
-  setEndInfo()
 
 proc parseVarTuple(p: var Parser): PNode
 
 proc parseFor(p: var Parser): PNode =
-  #| forStmt = 'for' ((varTuple / identWithPragma) ^+ comma) 'in' expr colcom stmt
+  #| forStmt = 'for' (identWithPragma ^+ comma) 'in' expr colcom stmt
   #| forExpr = forStmt
   getTokNoInd(p)
   result = newNodeP(nkForStmt, p)
@@ -1287,7 +1239,6 @@ proc parseFor(p: var Parser): PNode =
   result.add(parseExpr(p))
   colcom(p, result)
   result.add(parseStmt(p))
-  setEndInfo()
 
 template nimprettyDontTouch(body) =
   when defined(nimpretty):
@@ -1326,25 +1277,17 @@ proc parseExpr(p: var Parser): PNode =
     nimprettyDontTouch:
       result = parseTry(p, isExpr=true)
   else: result = simpleExpr(p)
-  setEndInfo()
 
 proc parseEnum(p: var Parser): PNode
 proc parseObject(p: var Parser): PNode
 proc parseTypeClass(p: var Parser): PNode
 
 proc primary(p: var Parser, mode: PrimaryMode): PNode =
-  #| simplePrimary = SIGILLIKEOP? identOrLiteral primarySuffix*
-  #| commandStart = &('`'|IDENT|literal|'cast'|'addr'|'type'|'var'|'out'|
-  #|                  'static'|'enum'|'tuple'|'object'|'proc')
-  #| primary = simplePrimary (commandStart expr)
-  #|         / operatorB primary
-  #|         / routineExpr
-  #|         / rawTypeDesc
-  #|         / prefixOperator primary
-  # XXX strong spaces need to be reflected in commandStart
-  # command part is handled in the primarySuffix proc
-
-  # prefix operators:
+  #| primary = operatorB primary primarySuffix* |
+  #|           tupleDecl | routineExpr | enumDecl
+  #|           objectDecl | conceptDecl | ('bind' primary)
+  #|           ('var' | 'out' | 'ref' | 'ptr' | 'distinct') primary
+  #|         /  prefixOperator* identOrLiteral primarySuffix*
   if isOperator(p.tok):
     # Note 'sigil like' operators are currently not reflected in the grammar
     # and should be removed for Nim 2.0, I don't think anybody uses them.
@@ -1354,39 +1297,60 @@ proc primary(p: var Parser, mode: PrimaryMode): PNode =
     result.add(a)
     getTok(p)
     optInd(p, a)
-    const identOrLiteralKinds = tkBuiltInMagics + {tkSymbol, tkAccent, tkNil,
-      tkIntLit..tkCustomLit, tkCast, tkOut, tkParLe, tkBracketLe, tkCurlyLe}
-    if isSigil and p.tok.tokType in identOrLiteralKinds:
+    if isSigil:
+      #XXX prefix operators
       let baseInd = p.lex.currLineIndent
-      result.add(identOrLiteral(p, mode))
+      result.add(primary(p, pmSkipSuffix))
       result = primarySuffix(p, result, baseInd, mode)
     else:
       result.add(primary(p, pmNormal))
     return
 
   case p.tok.tokType
+  of tkTuple: result = parseTuple(p, mode == pmTypeDef)
   of tkProc:
     getTok(p)
-    result = parseProcExpr(p, mode != pmTypeDesc, nkLambda)
+    result = parseProcExpr(p, mode notin {pmTypeDesc, pmTypeDef}, nkLambda)
   of tkFunc:
     getTok(p)
-    result = parseProcExpr(p, mode != pmTypeDesc, nkFuncDef)
+    result = parseProcExpr(p, mode notin {pmTypeDesc, pmTypeDef}, nkFuncDef)
   of tkIterator:
     getTok(p)
-    result = parseProcExpr(p, mode != pmTypeDesc, nkIteratorDef)
+    result = parseProcExpr(p, mode notin {pmTypeDesc, pmTypeDef}, nkIteratorDef)
+  of tkEnum:
+    if mode == pmTypeDef:
+      prettySection:
+        result = parseEnum(p)
+    else:
+      result = newNodeP(nkEnumTy, p)
+      getTok(p)
+  of tkObject:
+    if mode == pmTypeDef:
+      prettySection:
+        result = parseObject(p)
+    else:
+      result = newNodeP(nkObjectTy, p)
+      getTok(p)
+  of tkConcept:
+    if mode == pmTypeDef:
+      result = parseTypeClass(p)
+    else:
+      parMessage(p, "the 'concept' keyword is only valid in 'type' sections")
   of tkBind:
-    # legacy syntax, no-op in current nim
     result = newNodeP(nkBind, p)
     getTok(p)
     optInd(p, result)
     result.add(primary(p, pmNormal))
-  of tkTuple, tkEnum, tkObject, tkConcept,
-    tkVar, tkOut, tkRef, tkPtr, tkDistinct:
-    result = parseTypeDesc(p)
+  of tkVar: result = parseTypeDescKAux(p, nkVarTy, mode)
+  of tkOut: result = parseTypeDescKAux(p, nkOutTy, mode)
+  of tkRef: result = parseTypeDescKAux(p, nkRefTy, mode)
+  of tkPtr: result = parseTypeDescKAux(p, nkPtrTy, mode)
+  of tkDistinct: result = parseTypeDescKAux(p, nkDistinctTy, mode)
   else:
     let baseInd = p.lex.currLineIndent
     result = identOrLiteral(p, mode)
-    result = primarySuffix(p, result, baseInd, mode)
+    if mode != pmSkipSuffix:
+      result = primarySuffix(p, result, baseInd, mode)
 
 proc binaryNot(p: var Parser; a: PNode): PNode =
   if p.tok.tokType == tkNot:
@@ -1401,73 +1365,17 @@ proc binaryNot(p: var Parser; a: PNode): PNode =
   else:
     result = a
 
-proc parseTypeDesc(p: var Parser, fullExpr = false): PNode =
-  #| rawTypeDesc = (tupleType | routineType | 'enum' | 'object' |
-  #|                 ('var' | 'out' | 'ref' | 'ptr' | 'distinct') typeDesc?)
-  #|                 ('not' expr)?
-  #| typeDescExpr = (routineType / simpleExpr) ('not' expr)?
-  #| typeDesc = rawTypeDesc / typeDescExpr
+proc parseTypeDesc(p: var Parser): PNode =
+  #| typeDesc = simpleExpr ('not' expr)?
   newlineWasSplitting(p)
-  if fullExpr:
-    result = simpleExpr(p, pmTypeDesc)
-  else:
-    case p.tok.tokType
-    of tkTuple:
-      result = parseTuple(p, false)
-    of tkProc:
-      getTok(p)
-      result = parseProcExpr(p, false, nkLambda)
-    of tkIterator:
-      getTok(p)
-      result = parseProcExpr(p, false, nkIteratorDef)
-    of tkEnum:
-      result = newNodeP(nkEnumTy, p)
-      getTok(p)
-    of tkObject:
-      result = newNodeP(nkObjectTy, p)
-      getTok(p)
-    of tkConcept:
-      parMessage(p, "the 'concept' keyword is only valid in 'type' sections")
-    of tkVar: result = parseTypeDescKAux(p, nkVarTy, pmTypeDesc)
-    of tkOut: result = parseTypeDescKAux(p, nkOutTy, pmTypeDesc)
-    of tkRef: result = parseTypeDescKAux(p, nkRefTy, pmTypeDesc)
-    of tkPtr: result = parseTypeDescKAux(p, nkPtrTy, pmTypeDesc)
-    of tkDistinct: result = parseTypeDescKAux(p, nkDistinctTy, pmTypeDesc)
-    else:
-      result = simpleExpr(p, pmTypeDesc)
+  result = simpleExpr(p, pmTypeDesc)
   result = binaryNot(p, result)
-  setEndInfo()
 
-proc parseTypeDefValue(p: var Parser): PNode =
-  #| typeDefValue = ((tupleDecl | enumDecl | objectDecl | conceptDecl |
-  #|                  ('ref' | 'ptr' | 'distinct') (tupleDecl | objectDecl))
-  #|                / (simpleExpr (exprEqExpr ^+ comma postExprBlocks)?))
-  #|                ('not' expr)?
-  case p.tok.tokType
-  of tkTuple: result = parseTuple(p, true)
-  of tkRef: result = parseTypeDescKAux(p, nkRefTy, pmTypeDef)
-  of tkPtr: result = parseTypeDescKAux(p, nkPtrTy, pmTypeDef)
-  of tkDistinct: result = parseTypeDescKAux(p, nkDistinctTy, pmTypeDef)
-  of tkEnum:
-    prettySection:
-      result = parseEnum(p)
-  of tkObject:
-    prettySection:
-      result = parseObject(p)
-  of tkConcept:
-    result = parseTypeClass(p)
-  else:
-    result = simpleExpr(p, pmTypeDef)
-    if p.tok.tokType != tkNot:
-      if result.kind == nkCommand:
-        var isFirstParam = false
-        while p.tok.tokType == tkComma:
-          getTok(p)
-          optInd(p, result)
-          result.add(commandParam(p, isFirstParam, pmTypeDef))
-      result = postExprBlocks(p, result)
+proc parseTypeDefAux(p: var Parser): PNode =
+  #| typeDefAux = simpleExpr ('not' expr
+  #|                         | postExprBlocks)?
+  result = simpleExpr(p, pmTypeDef)
   result = binaryNot(p, result)
-  setEndInfo()
 
 proc makeCall(n: PNode): PNode =
   ## Creates a call if the given node isn't already a call.
@@ -1481,7 +1389,7 @@ proc postExprBlocks(p: var Parser, x: PNode): PNode =
   #| postExprBlocks = ':' stmt? ( IND{=} doBlock
   #|                            | IND{=} 'of' exprList ':' stmt
   #|                            | IND{=} 'elif' expr ':' stmt
-  #|                            | IND{=} 'except' optionalExprList ':' stmt
+  #|                            | IND{=} 'except' exprList ':' stmt
   #|                            | IND{=} 'finally' ':' stmt
   #|                            | IND{=} 'else' ':' stmt )*
   result = x
@@ -1500,7 +1408,7 @@ proc postExprBlocks(p: var Parser, x: PNode): PNode =
     result = makeCall(result)
     getTok(p)
     skipComment(p, result)
-    if p.tok.tokType notin {tkOf, tkElif, tkElse, tkExcept, tkFinally}:
+    if p.tok.tokType notin {tkOf, tkElif, tkElse, tkExcept}:
       var stmtList = newNodeP(nkStmtList, p)
       stmtList.add parseStmt(p)
       # to keep backwards compatibility (see tests/vm/tstringnil)
@@ -1539,7 +1447,7 @@ proc postExprBlocks(p: var Parser, x: PNode): PNode =
           nextBlock.add parseExpr(p)
         of tkExcept:
           nextBlock = newNodeP(nkExceptBranch, p)
-          optionalExprList(p, tkColon, nextBlock)
+          exprList(p, tkColon, nextBlock)
         of tkFinally:
           nextBlock = newNodeP(nkFinally, p)
           getTok(p)
@@ -1559,10 +1467,12 @@ proc postExprBlocks(p: var Parser, x: PNode): PNode =
       parMessage(p, "expected ':'")
 
 proc parseExprStmt(p: var Parser): PNode =
-  #| exprStmt = simpleExpr postExprBlocks?
-  #|          / simplePrimary (exprEqExpr ^+ comma) postExprBlocks?
-  #|          / simpleExpr '=' optInd (expr postExprBlocks?)
-  var a = simpleExpr(p, pmTrySimple)
+  #| exprStmt = simpleExpr
+  #|          (( '=' optInd expr colonBody? )
+  #|          / ( expr ^+ comma
+  #|              postExprBlocks
+  #|            ))?
+  var a = simpleExpr(p)
   if p.tok.tokType == tkEquals:
     result = newNodeP(nkAsgn, p)
     getTok(p)
@@ -1572,23 +1482,25 @@ proc parseExprStmt(p: var Parser): PNode =
     result.add(a)
     result.add(b)
   else:
+    # simpleExpr parsed 'p a' from 'p a, b'?
     var isFirstParam = false
-    # if an expression is starting here, a simplePrimary was parsed and
-    # this is the start of a command
-    if p.tok.indent < 0 and isExprStart(p):
+    if p.tok.indent < 0 and p.tok.tokType == tkComma and a.kind == nkCommand:
+      result = a
+      while true:
+        getTok(p)
+        optInd(p, result)
+        result.add(commandParam(p, isFirstParam, pmNormal))
+        if p.tok.tokType != tkComma: break
+    elif p.tok.indent < 0 and isExprStart(p):
       result = newTreeI(nkCommand, a.info, a)
-      let baseIndent = p.currInd
       while true:
         result.add(commandParam(p, isFirstParam, pmNormal))
-        if p.tok.tokType != tkComma or
-          (p.tok.indent >= 0 and p.tok.indent < baseIndent):
-          break
+        if p.tok.tokType != tkComma: break
         getTok(p)
         optInd(p, result)
     else:
       result = a
     result = postExprBlocks(p, result)
-  setEndInfo()
 
 proc parseModuleName(p: var Parser, kind: TNodeKind): PNode =
   result = parseExpr(p)
@@ -1600,7 +1512,6 @@ proc parseModuleName(p: var Parser, kind: TNodeKind): PNode =
       getTok(p)
       result.add(a)
       result.add(parseExpr(p))
-  setEndInfo()
 
 proc parseImport(p: var Parser, kind: TNodeKind): PNode =
   #| importStmt = 'import' optInd expr
@@ -1629,7 +1540,6 @@ proc parseImport(p: var Parser, kind: TNodeKind): PNode =
       getTok(p)
       optInd(p, a)
   #expectNl(p)
-  setEndInfo()
 
 proc parseIncludeStmt(p: var Parser): PNode =
   #| includeStmt = 'include' optInd expr ^+ comma
@@ -1646,7 +1556,6 @@ proc parseIncludeStmt(p: var Parser): PNode =
     getTok(p)
     optInd(p, a)
   #expectNl(p)
-  setEndInfo()
 
 proc parseFromStmt(p: var Parser): PNode =
   #| fromStmt = 'from' expr 'import' optInd expr (comma expr)*
@@ -1667,7 +1576,6 @@ proc parseFromStmt(p: var Parser): PNode =
     getTok(p)
     optInd(p, a)
   #expectNl(p)
-  setEndInfo()
 
 proc parseReturnOrRaise(p: var Parser, kind: TNodeKind): PNode =
   #| returnStmt = 'return' optInd expr?
@@ -1689,7 +1597,6 @@ proc parseReturnOrRaise(p: var Parser, kind: TNodeKind): PNode =
     var e = parseExpr(p)
     e = postExprBlocks(p, e)
     result.add(e)
-  setEndInfo()
 
 proc parseIfOrWhen(p: var Parser, kind: TNodeKind): PNode =
   #| condStmt = expr colcom stmt COMMENT?
@@ -1714,7 +1621,6 @@ proc parseIfOrWhen(p: var Parser, kind: TNodeKind): PNode =
     colcom(p, branch)
     branch.add(parseStmt(p))
     result.add(branch)
-  setEndInfo()
 
 proc parseIfOrWhenExpr(p: var Parser, kind: TNodeKind): PNode =
   #| condExpr = expr colcom expr optInd
@@ -1739,7 +1645,6 @@ proc parseIfOrWhenExpr(p: var Parser, kind: TNodeKind): PNode =
     colcom(p, branch)
     branch.add(parseStmt(p))
     result.add(branch)
-  setEndInfo()
 
 proc parseWhile(p: var Parser): PNode =
   #| whileStmt = 'while' expr colcom stmt
@@ -1749,7 +1654,6 @@ proc parseWhile(p: var Parser): PNode =
   result.add(parseExpr(p))
   colcom(p, result)
   result.add(parseStmt(p))
-  setEndInfo()
 
 proc parseCase(p: var Parser): PNode =
   #| ofBranch = 'of' exprList colcom stmt
@@ -1797,27 +1701,24 @@ proc parseCase(p: var Parser): PNode =
 
   if wasIndented:
     p.currInd = oldInd
-  setEndInfo()
 
 proc parseTry(p: var Parser; isExpr: bool): PNode =
   #| tryStmt = 'try' colcom stmt &(IND{=}? 'except'|'finally')
-  #|            (IND{=}? 'except' optionalExprList colcom stmt)*
+  #|            (IND{=}? 'except' exprList colcom stmt)*
   #|            (IND{=}? 'finally' colcom stmt)?
   #| tryExpr = 'try' colcom stmt &(optInd 'except'|'finally')
-  #|            (optInd 'except' optionalExprList colcom stmt)*
+  #|            (optInd 'except' exprList colcom stmt)*
   #|            (optInd 'finally' colcom stmt)?
   result = newNodeP(nkTryStmt, p)
-  let parentIndent = p.currInd # isExpr
   getTok(p)
   colcom(p, result)
   result.add(parseStmt(p))
   var b: PNode = nil
-
-  while sameOrNoInd(p) or (isExpr and parentIndent <= p.tok.indent):
+  while sameOrNoInd(p) or isExpr:
     case p.tok.tokType
     of tkExcept:
       b = newNodeP(nkExceptBranch, p)
-      optionalExprList(p, tkColon, b)
+      exprList(p, tkColon, b)
     of tkFinally:
       b = newNodeP(nkFinally, p)
       getTok(p)
@@ -1826,14 +1727,12 @@ proc parseTry(p: var Parser; isExpr: bool): PNode =
     b.add(parseStmt(p))
     result.add(b)
   if b == nil: parMessage(p, "expected 'except'")
-  setEndInfo()
 
 proc parseExceptBlock(p: var Parser, kind: TNodeKind): PNode =
   result = newNodeP(kind, p)
   getTok(p)
   colcom(p, result)
   result.add(parseStmt(p))
-  setEndInfo()
 
 proc parseBlock(p: var Parser): PNode =
   #| blockStmt = 'block' symbol? colcom stmt
@@ -1844,7 +1743,6 @@ proc parseBlock(p: var Parser): PNode =
   else: result.add(parseSymbol(p))
   colcom(p, result)
   result.add(parseStmt(p))
-  setEndInfo()
 
 proc parseStaticOrDefer(p: var Parser; k: TNodeKind): PNode =
   #| staticStmt = 'static' colcom stmt
@@ -1853,7 +1751,6 @@ proc parseStaticOrDefer(p: var Parser; k: TNodeKind): PNode =
   getTok(p)
   colcom(p, result)
   result.add(parseStmt(p))
-  setEndInfo()
 
 proc parseAsm(p: var Parser): PNode =
   #| asmStmt = 'asm' pragma? (STR_LIT | RSTR_LIT | TRIPLESTR_LIT)
@@ -1870,7 +1767,6 @@ proc parseAsm(p: var Parser): PNode =
     result.add(p.emptyNode)
     return
   getTok(p)
-  setEndInfo()
 
 proc parseGenericParam(p: var Parser): PNode =
   #| genericParam = symbol (comma symbol)* (colon expr)? ('=' optInd expr)?
@@ -1906,7 +1802,6 @@ proc parseGenericParam(p: var Parser): PNode =
     result.add(parseExpr(p))
   else:
     result.add(p.emptyNode)
-  setEndInfo()
 
 proc parseGenericParamList(p: var Parser): PNode =
   #| genericParamList = '[' optInd
@@ -1925,14 +1820,12 @@ proc parseGenericParamList(p: var Parser): PNode =
     skipComment(p, a)
   optPar(p)
   eat(p, tkBracketRi)
-  setEndInfo()
 
 proc parsePattern(p: var Parser): PNode =
   #| pattern = '{' stmt '}'
   eat(p, tkCurlyLe)
   result = parseStmt(p)
   eat(p, tkCurlyRi)
-  setEndInfo()
 
 proc parseRoutine(p: var Parser, kind: TNodeKind): PNode =
   #| indAndComment = (IND{>} COMMENT)? | COMMENT?
@@ -1977,7 +1870,6 @@ proc parseRoutine(p: var Parser, kind: TNodeKind): PNode =
     #else:
     #  assert false, p.lex.config$body.info # avoids hard to track bugs, fail early.
     # Yeah, that worked so well. There IS a bug in this logic, now what?
-  setEndInfo()
 
 proc newCommentStmt(p: var Parser): PNode =
   #| commentStmt = COMMENT
@@ -2013,7 +1905,6 @@ proc parseSection(p: var Parser, kind: TNodeKind,
     result.add(defparser(p))
   else:
     parMessage(p, errIdentifierExpected, p.tok)
-  setEndInfo()
 
 proc parseEnum(p: var Parser): PNode =
   #| enumDecl = 'enum' optInd (symbol pragma? optInd ('=' optInd expr COMMENT?)? comma?)+
@@ -2060,7 +1951,6 @@ proc parseEnum(p: var Parser): PNode =
       break
   if result.len <= 1:
     parMessage(p, errIdentifierExpected, p.tok)
-  setEndInfo()
 
 proc parseObjectPart(p: var Parser): PNode
 proc parseObjectWhen(p: var Parser): PNode =
@@ -2086,7 +1976,6 @@ proc parseObjectWhen(p: var Parser): PNode =
     branch.add(parseObjectPart(p))
     flexComment(p, branch)
     result.add(branch)
-  setEndInfo()
 
 proc parseObjectCase(p: var Parser): PNode =
   #| objectBranch = 'of' exprList colcom objectPart
@@ -2128,7 +2017,6 @@ proc parseObjectCase(p: var Parser): PNode =
     if b.kind == nkElse: break
   if wasIndented:
     p.currInd = oldInd
-  setEndInfo()
 
 proc parseObjectPart(p: var Parser): PNode =
   #| objectPart = IND{>} objectPart^+IND{=} DED
@@ -2161,7 +2049,6 @@ proc parseObjectPart(p: var Parser): PNode =
       result = p.emptyNode
   else:
     result = p.emptyNode
-  setEndInfo()
 
 proc parseObject(p: var Parser): PNode =
   #| objectDecl = 'object' ('of' typeDesc)? COMMENT? objectPart
@@ -2182,7 +2069,6 @@ proc parseObject(p: var Parser): PNode =
     result.add(p.emptyNode)
   else:
     result.add(parseObjectPart(p))
-  setEndInfo()
 
 proc parseTypeClassParam(p: var Parser): PNode =
   let modifier =
@@ -2200,7 +2086,6 @@ proc parseTypeClassParam(p: var Parser): PNode =
     result.add(p.parseSymbol)
   else:
     result = p.parseSymbol
-  setEndInfo()
 
 proc parseTypeClass(p: var Parser): PNode =
   #| conceptParam = ('var' | 'out')? symbol
@@ -2244,11 +2129,10 @@ proc parseTypeClass(p: var Parser): PNode =
     result.add(p.emptyNode)
   else:
     result.add(parseStmt(p))
-  setEndInfo()
 
 proc parseTypeDef(p: var Parser): PNode =
   #|
-  #| typeDef = identVisDot genericParamList? pragma '=' optInd typeDefValue
+  #| typeDef = identVisDot genericParamList? pragma '=' optInd typeDefAux
   #|             indAndComment?
   result = newNodeP(nkTypeDef, p)
   var identifier = identVis(p, allowDot=true)
@@ -2274,26 +2158,19 @@ proc parseTypeDef(p: var Parser): PNode =
     result.info = parLineInfo(p)
     getTok(p)
     optInd(p, result)
-    result.add(parseTypeDefValue(p))
+    result.add(parseTypeDefAux(p))
   else:
     result.add(p.emptyNode)
   indAndComment(p, result)    # special extension!
-  setEndInfo()
 
 proc parseVarTuple(p: var Parser): PNode =
-  #| varTupleLhs = '(' optInd (identWithPragma / varTupleLhs) ^+ comma optPar ')'
-  #| varTuple = varTupleLhs '=' optInd expr
+  #| varTuple = '(' optInd identWithPragma ^+ comma optPar ')' '=' optInd expr
   result = newNodeP(nkVarTuple, p)
   getTok(p)                   # skip '('
   optInd(p, result)
   # progress guaranteed
-  while p.tok.tokType in {tkSymbol, tkAccent, tkParLe}:
-    var a: PNode
-    if p.tok.tokType == tkParLe:
-      a = parseVarTuple(p)
-      a.add(p.emptyNode)
-    else:
-      a = identWithPragma(p, allowDot=true)
+  while p.tok.tokType in {tkSymbol, tkAccent}:
+    var a = identWithPragma(p, allowDot=true)
     result.add(a)
     if p.tok.tokType != tkComma: break
     getTok(p)
@@ -2301,7 +2178,6 @@ proc parseVarTuple(p: var Parser): PNode =
   result.add(p.emptyNode)         # no type desc
   optPar(p)
   eat(p, tkParRi)
-  setEndInfo()
 
 proc parseVariable(p: var Parser): PNode =
   #| colonBody = colcom stmt postExprBlocks?
@@ -2314,7 +2190,6 @@ proc parseVariable(p: var Parser): PNode =
   else: result = parseIdentColonEquals(p, {withPragma, withDot})
   result[^1] = postExprBlocks(p, result[^1])
   indAndComment(p, result)
-  setEndInfo()
 
 proc parseConstant(p: var Parser): PNode =
   #| constant = (varTuple / identWithPragma) (colon typeDesc)? '=' optInd expr indAndComment
@@ -2334,7 +2209,6 @@ proc parseConstant(p: var Parser): PNode =
   result.add(parseExpr(p))
   result[^1] = postExprBlocks(p, result[^1])
   indAndComment(p, result)
-  setEndInfo()
 
 proc parseBind(p: var Parser, k: TNodeKind): PNode =
   #| bindStmt = 'bind' optInd qualifiedIdent ^+ comma
@@ -2350,7 +2224,6 @@ proc parseBind(p: var Parser, k: TNodeKind): PNode =
     getTok(p)
     optInd(p, a)
   #expectNl(p)
-  setEndInfo()
 
 proc parseStmtPragma(p: var Parser): PNode =
   #| pragmaStmt = pragma (':' COMMENT? stmt)?
@@ -2362,7 +2235,6 @@ proc parseStmtPragma(p: var Parser): PNode =
     skipComment(p, result)
     result.add a
     result.add parseStmt(p)
-  setEndInfo()
 
 proc simpleStmt(p: var Parser): PNode =
   #| simpleStmt = ((returnStmt | raiseStmt | yieldStmt | discardStmt | breakStmt
@@ -2505,7 +2377,6 @@ proc parseStmt(p: var Parser): PNode =
           if p.tok.tokType != tkSemiColon: break
           getTok(p)
           if err and p.tok.tokType == tkEof: break
-  setEndInfo()
 
 proc parseAll(p: var Parser): PNode =
   ## Parses the rest of the input stream held by the parser into a PNode.
@@ -2521,7 +2392,6 @@ proc parseAll(p: var Parser): PNode =
       getTok(p)
     if p.tok.indent != 0:
       parMessage(p, errInvalidIndentation)
-  setEndInfo()
 
 proc checkFirstLineIndentation*(p: var Parser) =
   if p.tok.indent != 0 and p.tok.strongSpaceA:
@@ -2555,7 +2425,6 @@ proc parseTopLevelStmt(p: var Parser): PNode =
       result = complexOrSimpleStmt(p)
       if result.kind == nkEmpty: parMessage(p, errExprExpected, p.tok)
       break
-  setEndInfo()
 
 proc parseString*(s: string; cache: IdentCache; config: ConfigRef;
                   filename: string = ""; line: int = 0;
@@ -2567,10 +2436,9 @@ proc parseString*(s: string; cache: IdentCache; config: ConfigRef;
   var stream = llStreamOpen(s)
   stream.lineOffset = line
 
-  var p: Parser
-  p.lex.errorHandler = errorHandler
-  openParser(p, AbsoluteFile filename, stream, cache, config)
+  var parser: Parser
+  parser.lex.errorHandler = errorHandler
+  openParser(parser, AbsoluteFile filename, stream, cache, config)
 
-  result = p.parseAll
-  closeParser(p)
-  setEndInfo()
+  result = parser.parseAll
+  closeParser(parser)

compiler/passes.nim

@@ -14,22 +14,13 @@ import
   options, ast, llstream, msgs,
   idents,
   syntaxes, modulegraphs, reorder,
-  lineinfos,
-  pipelineutils,
-  modules, pathutils, packages,
-  sem, semdata
-
-import ic/replayer
-
-export skipCodegen, resolveMod, prepareConfigNotes
+  lineinfos, pathutils, packages
 
 when defined(nimsuggest):
   import std/sha1
 
 when defined(nimPreviewSlimSystem):
-  import std/[syncio, assertions]
-
-import std/tables
+  import std/syncio
 
 type
   TPassData* = tuple[input: PNode, closeOutput: PNode]
@@ -47,6 +38,12 @@ proc makePass*(open: TPassOpen = nil,
   result.process = process
   result.isFrontend = isFrontend
 
+proc skipCodegen*(config: ConfigRef; n: PNode): bool {.inline.} =
+  # can be used by codegen passes to determine whether they should do
+  # something with `n`. Currently, this ignores `n` and uses the global
+  # error count instead.
+  result = config.errorCounter > 0
+
 const
   maxPasses = 10
 
@@ -83,6 +80,13 @@ proc processTopLevelStmt(graph: ModuleGraph, n: PNode, a: var TPassContextArray)
       if isNil(m): return false
   result = true
 
+proc resolveMod(conf: ConfigRef; module, relativeTo: string): FileIndex =
+  let fullPath = findModule(conf, module, relativeTo)
+  if fullPath.isEmpty:
+    result = InvalidFileIdx
+  else:
+    result = fileInfoIdx(conf, fullPath)
+
 proc processImplicits(graph: ModuleGraph; implicits: seq[string], nodeKind: TNodeKind,
                       a: var TPassContextArray; m: PSym) =
   # XXX fixme this should actually be relative to the config file!
@@ -96,6 +100,23 @@ proc processImplicits(graph: ModuleGraph; implicits: seq[string], nodeKind: TNod
       importStmt.add str
       if not processTopLevelStmt(graph, importStmt, a): break
 
+const
+  imperativeCode = {low(TNodeKind)..high(TNodeKind)} - {nkTemplateDef, nkProcDef, nkMethodDef,
+    nkMacroDef, nkConverterDef, nkIteratorDef, nkFuncDef, nkPragma,
+    nkExportStmt, nkExportExceptStmt, nkFromStmt, nkImportStmt, nkImportExceptStmt}
+
+proc prepareConfigNotes(graph: ModuleGraph; module: PSym) =
+  # don't be verbose unless the module belongs to the main package:
+  if graph.config.belongsToProjectPackage(module):
+    graph.config.notes = graph.config.mainPackageNotes
+  else:
+    if graph.config.mainPackageNotes == {}: graph.config.mainPackageNotes = graph.config.notes
+    graph.config.notes = graph.config.foreignPackageNotes
+
+proc moduleHasChanged*(graph: ModuleGraph; module: PSym): bool {.inline.} =
+  result = true
+  #module.id >= 0 or isDefined(graph.config, "nimBackendAssumesChange")
+
 proc processModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator;
                     stream: PLLStream): bool {.discardable.} =
   if graph.stopCompile(): return true
@@ -122,7 +143,7 @@ proc processModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator;
   while true:
     openParser(p, fileIdx, s, graph.cache, graph.config)
 
-    if (not belongsToStdlib(graph, module)) or module.name.s == "distros":
+    if not belongsToStdlib(graph, module) or (belongsToStdlib(graph, module) and module.name.s == "distros"):
       # XXX what about caching? no processing then? what if I change the
       # modules to include between compilation runs? we'd need to track that
       # in ROD files. I think we should enable this feature only
@@ -132,22 +153,43 @@ proc processModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator;
         processImplicits graph, graph.config.implicitIncludes, nkIncludeStmt, a, module
 
     checkFirstLineIndentation(p)
-    block processCode:
-      if graph.stopCompile(): break processCode
+    while true:
+      if graph.stopCompile(): break
       var n = parseTopLevelStmt(p)
-      if n.kind == nkEmpty: break processCode
-
-      # read everything, no streaming possible
-      var sl = newNodeI(nkStmtList, n.info)
-      sl.add n
-      while true:
-        var n = parseTopLevelStmt(p)
-        if n.kind == nkEmpty: break
+      if n.kind == nkEmpty: break
+      if (sfSystemModule notin module.flags and
+          ({sfNoForward, sfReorder} * module.flags != {} or
+          codeReordering in graph.config.features)):
+        # read everything, no streaming possible
+        var sl = newNodeI(nkStmtList, n.info)
         sl.add n
-      if sfReorder in module.flags or codeReordering in graph.config.features:
-        sl = reorder(graph, sl, module)
-      discard processTopLevelStmt(graph, sl, a)
-
+        while true:
+          var n = parseTopLevelStmt(p)
+          if n.kind == nkEmpty: break
+          sl.add n
+        if sfReorder in module.flags or codeReordering in graph.config.features:
+          sl = reorder(graph, sl, module)
+        discard processTopLevelStmt(graph, sl, a)
+        break
+      elif n.kind in imperativeCode:
+        # read everything until the next proc declaration etc.
+        var sl = newNodeI(nkStmtList, n.info)
+        sl.add n
+        var rest: PNode = nil
+        while true:
+          var n = parseTopLevelStmt(p)
+          if n.kind == nkEmpty or n.kind notin imperativeCode:
+            rest = n
+            break
+          sl.add n
+        #echo "-----\n", sl
+        if not processTopLevelStmt(graph, sl, a): break
+        if rest != nil:
+          #echo "-----\n", rest
+          if not processTopLevelStmt(graph, rest, a): break
+      else:
+        #echo "----- single\n", n
+        if not processTopLevelStmt(graph, n, a): break
     closeParser(p)
     if s.kind != llsStdIn: break
   closePasses(graph, a)
@@ -157,99 +199,3 @@ proc processModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator;
     # They are responsible for closing the rod files. See `cbackend.nim`.
     closeRodFile(graph, module)
   result = true
-
-proc compileModule*(graph: ModuleGraph; fileIdx: FileIndex; flags: TSymFlags, fromModule: PSym = nil): PSym =
-  var flags = flags
-  if fileIdx == graph.config.projectMainIdx2: flags.incl sfMainModule
-  result = graph.getModule(fileIdx)
-
-  template processModuleAux(moduleStatus) =
-    onProcessing(graph, fileIdx, moduleStatus, fromModule = fromModule)
-    var s: PLLStream
-    if sfMainModule in flags:
-      if graph.config.projectIsStdin: s = stdin.llStreamOpen
-      elif graph.config.projectIsCmd: s = llStreamOpen(graph.config.cmdInput)
-    discard processModule(graph, result, idGeneratorFromModule(result), s)
-  if result == nil:
-    var cachedModules: seq[FileIndex]
-    result = moduleFromRodFile(graph, fileIdx, cachedModules)
-    let filename = AbsoluteFile toFullPath(graph.config, fileIdx)
-    if result == nil:
-      result = newModule(graph, fileIdx)
-      result.flags.incl flags
-      registerModule(graph, result)
-      processModuleAux("import")
-    else:
-      if sfSystemModule in flags:
-        graph.systemModule = result
-      partialInitModule(result, graph, fileIdx, filename)
-    for m in cachedModules:
-      registerModuleById(graph, m)
-      replayStateChanges(graph.packed[m.int].module, graph)
-      replayGenericCacheInformation(graph, m.int)
-  elif graph.isDirty(result):
-    result.flags.excl sfDirty
-    # reset module fields:
-    initStrTables(graph, result)
-    result.ast = nil
-    processModuleAux("import(dirty)")
-    graph.markClientsDirty(fileIdx)
-
-proc importModule*(graph: ModuleGraph; s: PSym, fileIdx: FileIndex): PSym =
-  # this is called by the semantic checking phase
-  assert graph.config != nil
-  result = compileModule(graph, fileIdx, {}, s)
-  graph.addDep(s, fileIdx)
-  # keep track of import relationships
-  if graph.config.hcrOn:
-    graph.importDeps.mgetOrPut(FileIndex(s.position), @[]).add(fileIdx)
-  #if sfSystemModule in result.flags:
-  #  localError(result.info, errAttemptToRedefine, result.name.s)
-  # restore the notes for outer module:
-  graph.config.notes =
-    if graph.config.belongsToProjectPackage(s) or isDefined(graph.config, "booting"): graph.config.mainPackageNotes
-    else: graph.config.foreignPackageNotes
-
-proc connectCallbacks*(graph: ModuleGraph) =
-  graph.includeFileCallback = modules.includeModule
-  graph.importModuleCallback = importModule
-
-proc compileSystemModule*(graph: ModuleGraph) =
-  if graph.systemModule == nil:
-    connectCallbacks(graph)
-    graph.config.m.systemFileIdx = fileInfoIdx(graph.config,
-        graph.config.libpath / RelativeFile"system.nim")
-    discard graph.compileModule(graph.config.m.systemFileIdx, {sfSystemModule})
-
-proc compileProject*(graph: ModuleGraph; projectFileIdx = InvalidFileIdx) =
-  connectCallbacks(graph)
-  let conf = graph.config
-  wantMainModule(conf)
-  configComplete(graph)
-
-  let systemFileIdx = fileInfoIdx(conf, conf.libpath / RelativeFile"system.nim")
-  let projectFile = if projectFileIdx == InvalidFileIdx: conf.projectMainIdx else: projectFileIdx
-  conf.projectMainIdx2 = projectFile
-
-  let packSym = getPackage(graph, projectFile)
-  graph.config.mainPackageId = packSym.getPackageId
-  graph.importStack.add projectFile
-
-  if projectFile == systemFileIdx:
-    discard graph.compileModule(projectFile, {sfMainModule, sfSystemModule})
-  else:
-    graph.compileSystemModule()
-    discard graph.compileModule(projectFile, {sfMainModule})
-
-proc mySemOpen(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
-  result = preparePContext(graph, module, idgen)
-
-proc mySemClose(graph: ModuleGraph; context: PPassContext, n: PNode): PNode =
-  var c = PContext(context)
-  closePContext(graph, c, n)
-
-proc mySemProcess(context: PPassContext, n: PNode): PNode =
-  result = semWithPContext(PContext(context), n)
-
-const semPass* = makePass(mySemOpen, mySemProcess, mySemClose,
-                          isFrontend = true)

compiler/pathutils.nim

@@ -10,7 +10,7 @@
 ## Path handling utilities for Nim. Strictly typed code in order
 ## to avoid the never ending time sink in getting path handling right.
 
-import os, pathnorm, strutils
+import os, pathnorm
 
 when defined(nimPreviewSlimSystem):
   import std/[syncio, assertions]
@@ -102,52 +102,3 @@ when true:
   proc addFileExt*(x: RelativeFile; ext: string): RelativeFile {.borrow.}
 
   proc writeFile*(x: AbsoluteFile; content: string) {.borrow.}
-
-proc skipHomeDir(x: string): int =
-  when defined(windows):
-    if x.continuesWith("Users/", len("C:/")):
-      result = 3
-    else:
-      result = 0
-  else:
-    if x.startsWith("/home/") or x.startsWith("/Users/"):
-      result = 3
-    elif x.startsWith("/mnt/") and x.continuesWith("/Users/", len("/mnt/c")):
-      result = 5
-    else:
-      result = 0
-
-proc relevantPart(s: string; afterSlashX: int): string =
-  result = newStringOfCap(s.len - 8)
-  var slashes = afterSlashX
-  for i in 0..<s.len:
-    if slashes == 0:
-      result.add s[i]
-    elif s[i] == '/':
-      dec slashes
-
-template canonSlashes(x: string): string =
-  when defined(windows):
-    x.replace('\\', '/')
-  else:
-    x
-
-proc customPathImpl(x: string): string =
-  # Idea: Encode a "protocol" via "//protocol/path" which is not ambiguous
-  # as path canonicalization would have removed the double slashes.
-  # /mnt/X/Users/Y
-  # X:\\Users\Y
-  # /home/Y
-  # -->
-  # //user/
-  if not isAbsolute(x):
-    result = customPathImpl(canonSlashes(getCurrentDir() / x))
-  else:
-    let slashes = skipHomeDir(x)
-    if slashes > 0:
-      result = "//user/" & relevantPart(x, slashes)
-    else:
-      result = x
-
-proc customPath*(x: string): string =
-  customPathImpl canonSlashes x

compiler/pipelines.nim

@@ -1,294 +0,0 @@
-import sem, cgen, modulegraphs, ast, llstream, parser, msgs,
-       lineinfos, reorder, options, semdata, cgendata, modules, pathutils,
-       packages, syntaxes, depends, vm, pragmas, idents, lookups, wordrecg,
-       liftdestructors
-
-import pipelineutils
-
-when not defined(leanCompiler):
-  import jsgen, docgen2
-
-import std/[syncio, objectdollar, assertions, tables, strutils]
-import renderer
-import ic/replayer
-
-
-proc setPipeLinePass*(graph: ModuleGraph; pass: PipelinePass) =
-  graph.pipelinePass = pass
-
-proc processPipeline(graph: ModuleGraph; semNode: PNode; bModule: PPassContext): PNode =
-  case graph.pipelinePass
-  of CgenPass:
-    result = semNode
-    if bModule != nil:
-      genTopLevelStmt(BModule(bModule), result)
-  of JSgenPass:
-    when not defined(leanCompiler):
-      result = processJSCodeGen(bModule, semNode)
-  of GenDependPass:
-    result = addDotDependency(bModule, semNode)
-  of SemPass:
-    result = graph.emptyNode
-  of Docgen2Pass, Docgen2TexPass:
-    when not defined(leanCompiler):
-      result = processNode(bModule, semNode)
-  of Docgen2JsonPass:
-    when not defined(leanCompiler):
-      result = processNodeJson(bModule, semNode)
-  of EvalPass, InterpreterPass:
-    result = interpreterCode(bModule, semNode)
-  of NonePass:
-    doAssert false, "use setPipeLinePass to set a proper PipelinePass"
-
-proc processImplicitImports(graph: ModuleGraph; implicits: seq[string], nodeKind: TNodeKind,
-                      m: PSym, ctx: PContext, bModule: PPassContext, idgen: IdGenerator,
-                      ) =
-  # XXX fixme this should actually be relative to the config file!
-  let relativeTo = toFullPath(graph.config, m.info)
-  for module in items(implicits):
-    # implicit imports should not lead to a module importing itself
-    if m.position != resolveMod(graph.config, module, relativeTo).int32:
-      var importStmt = newNodeI(nodeKind, m.info)
-      var str = newStrNode(nkStrLit, module)
-      str.info = m.info
-      importStmt.add str
-      message(graph.config, importStmt.info, hintProcessingStmt, $idgen[])
-      let semNode = semWithPContext(ctx, importStmt)
-      if semNode == nil or processPipeline(graph, semNode, bModule) == nil:
-        break
-
-proc prePass(c: PContext; n: PNode) =
-  for son in n:
-    if son.kind == nkPragma:
-      for s in son:
-        var key = if s.kind in nkPragmaCallKinds and s.len > 1: s[0] else: s
-        if key.kind in {nkBracketExpr, nkCast} or key.kind notin nkIdentKinds:
-          continue
-        let ident = whichKeyword(considerQuotedIdent(c, key))
-        case ident
-        of wReorder:
-          pragmaNoForward(c, s, flag = sfReorder)
-        of wExperimental:
-          if isTopLevel(c) and s.kind in nkPragmaCallKinds and s.len == 2:
-            let name = c.semConstExpr(c, s[1])
-            case name.kind
-            of nkStrLit, nkRStrLit, nkTripleStrLit:
-              try:
-                let feature = parseEnum[Feature](name.strVal)
-                if feature == codeReordering:
-                  c.features.incl feature
-                  c.module.flags.incl sfReorder
-              except ValueError:
-                discard
-            else:
-              discard
-        else:
-          discard
-
-proc processPipelineModule*(graph: ModuleGraph; module: PSym; idgen: IdGenerator;
-                    stream: PLLStream): bool =
-  if graph.stopCompile(): return true
-  var
-    p: Parser
-    s: PLLStream
-    fileIdx = module.fileIdx
-
-  prepareConfigNotes(graph, module)
-  let ctx = preparePContext(graph, module, idgen)
-  let bModule: PPassContext =
-    case graph.pipelinePass
-    of CgenPass:
-      setupCgen(graph, module, idgen)
-    of JSgenPass:
-      when not defined(leanCompiler):
-        setupJSgen(graph, module, idgen)
-      else:
-        nil
-    of EvalPass, InterpreterPass:
-      setupEvalGen(graph, module, idgen)
-    of GenDependPass:
-      setupDependPass(graph, module, idgen)
-    of Docgen2Pass:
-      when not defined(leanCompiler):
-        openHtml(graph, module, idgen)
-      else:
-        nil
-    of Docgen2TexPass:
-      when not defined(leanCompiler):
-        openTex(graph, module, idgen)
-      else:
-        nil
-    of Docgen2JsonPass:
-      when not defined(leanCompiler):
-        openJson(graph, module, idgen)
-      else:
-        nil
-    of SemPass:
-      nil
-    of NonePass:
-      doAssert false, "use setPipeLinePass to set a proper PipelinePass"
-      nil
-
-  if stream == nil:
-    let filename = toFullPathConsiderDirty(graph.config, fileIdx)
-    s = llStreamOpen(filename, fmRead)
-    if s == nil:
-      rawMessage(graph.config, errCannotOpenFile, filename.string)
-      return false
-  else:
-    s = stream
-
-  while true:
-    syntaxes.openParser(p, fileIdx, s, graph.cache, graph.config)
-
-    if not belongsToStdlib(graph, module) or (belongsToStdlib(graph, module) and module.name.s == "distros"):
-      # XXX what about caching? no processing then? what if I change the
-      # modules to include between compilation runs? we'd need to track that
-      # in ROD files. I think we should enable this feature only
-      # for the interactive mode.
-      if module.name.s != "nimscriptapi":
-        processImplicitImports graph, graph.config.implicitImports, nkImportStmt, module, ctx, bModule, idgen
-        processImplicitImports graph, graph.config.implicitIncludes, nkIncludeStmt, module, ctx, bModule, idgen
-
-    checkFirstLineIndentation(p)
-    block processCode:
-      if graph.stopCompile(): break processCode
-      var n = parseTopLevelStmt(p)
-      if n.kind == nkEmpty: break processCode
-      # read everything, no streaming possible
-      var sl = newNodeI(nkStmtList, n.info)
-      sl.add n
-      while true:
-        var n = parseTopLevelStmt(p)
-        if n.kind == nkEmpty: break
-        sl.add n
-
-      prePass(ctx, sl)
-      if sfReorder in module.flags or codeReordering in graph.config.features:
-        sl = reorder(graph, sl, module)
-      if graph.pipelinePass != EvalPass:
-        message(graph.config, sl.info, hintProcessingStmt, $idgen[])
-      var semNode = semWithPContext(ctx, sl)
-      discard processPipeline(graph, semNode, bModule)
-
-    closeParser(p)
-    if s.kind != llsStdIn: break
-  let finalNode = closePContext(graph, ctx, nil)
-  case graph.pipelinePass
-  of CgenPass:
-    if bModule != nil:
-      let disps = finalCodegenActions(graph, BModule(bModule), finalNode)
-      if disps != nil:
-        let ctx = preparePContext(graph, module, idgen)
-        for disp in disps:
-          let retTyp = disp.sym.typ[0]
-          if retTyp != nil:
-            # todo properly semcheck the code of dispatcher?
-            createTypeBoundOps(graph, ctx, retTyp, disp.info, idgen)
-          genProcAux(BModule(bModule), disp.sym)
-        discard closePContext(graph, ctx, nil)
-  of JSgenPass:
-    when not defined(leanCompiler):
-      discard finalJSCodeGen(graph, bModule, finalNode)
-  of EvalPass, InterpreterPass:
-    discard interpreterCode(bModule, finalNode)
-  of SemPass, GenDependPass:
-    discard
-  of Docgen2Pass, Docgen2TexPass:
-    when not defined(leanCompiler):
-      discard closeDoc(graph, bModule, finalNode)
-  of Docgen2JsonPass:
-    when not defined(leanCompiler):
-      discard closeJson(graph, bModule, finalNode)
-  of NonePass:
-    doAssert false, "use setPipeLinePass to set a proper PipelinePass"
-
-  if graph.config.backend notin {backendC, backendCpp, backendObjc}:
-    # We only write rod files here if no C-like backend is active.
-    # The C-like backends have been patched to support the IC mechanism.
-    # They are responsible for closing the rod files. See `cbackend.nim`.
-    closeRodFile(graph, module)
-  result = true
-
-proc compilePipelineModule*(graph: ModuleGraph; fileIdx: FileIndex; flags: TSymFlags, fromModule: PSym = nil): PSym =
-  var flags = flags
-  if fileIdx == graph.config.projectMainIdx2: flags.incl sfMainModule
-  result = graph.getModule(fileIdx)
-
-  template processModuleAux(moduleStatus) =
-    onProcessing(graph, fileIdx, moduleStatus, fromModule = fromModule)
-    var s: PLLStream
-    if sfMainModule in flags:
-      if graph.config.projectIsStdin: s = stdin.llStreamOpen
-      elif graph.config.projectIsCmd: s = llStreamOpen(graph.config.cmdInput)
-    discard processPipelineModule(graph, result, idGeneratorFromModule(result), s)
-  if result == nil:
-    var cachedModules: seq[FileIndex]
-    result = moduleFromRodFile(graph, fileIdx, cachedModules)
-    let filename = AbsoluteFile toFullPath(graph.config, fileIdx)
-    if result == nil:
-      result = newModule(graph, fileIdx)
-      result.flags.incl flags
-      registerModule(graph, result)
-      processModuleAux("import")
-    else:
-      if sfSystemModule in flags:
-        graph.systemModule = result
-      partialInitModule(result, graph, fileIdx, filename)
-    for m in cachedModules:
-      registerModuleById(graph, m)
-      replayStateChanges(graph.packed[m.int].module, graph)
-      replayGenericCacheInformation(graph, m.int)
-  elif graph.isDirty(result):
-    result.flags.excl sfDirty
-    # reset module fields:
-    initStrTables(graph, result)
-    result.ast = nil
-    processModuleAux("import(dirty)")
-    graph.markClientsDirty(fileIdx)
-
-proc importPipelineModule(graph: ModuleGraph; s: PSym, fileIdx: FileIndex): PSym =
-  # this is called by the semantic checking phase
-  assert graph.config != nil
-  result = compilePipelineModule(graph, fileIdx, {}, s)
-  graph.addDep(s, fileIdx)
-  # keep track of import relationships
-  if graph.config.hcrOn:
-    graph.importDeps.mgetOrPut(FileIndex(s.position), @[]).add(fileIdx)
-  #if sfSystemModule in result.flags:
-  #  localError(result.info, errAttemptToRedefine, result.name.s)
-  # restore the notes for outer module:
-  graph.config.notes =
-    if graph.config.belongsToProjectPackage(s) or isDefined(graph.config, "booting"): graph.config.mainPackageNotes
-    else: graph.config.foreignPackageNotes
-
-proc connectPipelineCallbacks*(graph: ModuleGraph) =
-  graph.includeFileCallback = modules.includeModule
-  graph.importModuleCallback = importPipelineModule
-
-proc compilePipelineSystemModule*(graph: ModuleGraph) =
-  if graph.systemModule == nil:
-    connectPipelineCallbacks(graph)
-    graph.config.m.systemFileIdx = fileInfoIdx(graph.config,
-        graph.config.libpath / RelativeFile"system.nim")
-    discard graph.compilePipelineModule(graph.config.m.systemFileIdx, {sfSystemModule})
-
-proc compilePipelineProject*(graph: ModuleGraph; projectFileIdx = InvalidFileIdx) =
-  connectPipelineCallbacks(graph)
-  let conf = graph.config
-  wantMainModule(conf)
-  configComplete(graph)
-
-  let systemFileIdx = fileInfoIdx(conf, conf.libpath / RelativeFile"system.nim")
-  let projectFile = if projectFileIdx == InvalidFileIdx: conf.projectMainIdx else: projectFileIdx
-  conf.projectMainIdx2 = projectFile
-
-  let packSym = getPackage(graph, projectFile)
-  graph.config.mainPackageId = packSym.getPackageId
-  graph.importStack.add projectFile
-
-  if projectFile == systemFileIdx:
-    discard graph.compilePipelineModule(projectFile, {sfMainModule, sfSystemModule})
-  else:
-    graph.compilePipelineSystemModule()
-    discard graph.compilePipelineModule(projectFile, {sfMainModule})

compiler/pipelineutils.nim

@@ -1,26 +0,0 @@
-import ast, options, lineinfos, pathutils, msgs, modulegraphs, packages
-
-proc skipCodegen*(config: ConfigRef; n: PNode): bool {.inline.} =
-  # can be used by codegen passes to determine whether they should do
-  # something with `n`. Currently, this ignores `n` and uses the global
-  # error count instead.
-  result = config.errorCounter > 0
-
-proc resolveMod*(conf: ConfigRef; module, relativeTo: string): FileIndex =
-  let fullPath = findModule(conf, module, relativeTo)
-  if fullPath.isEmpty:
-    result = InvalidFileIdx
-  else:
-    result = fileInfoIdx(conf, fullPath)
-
-proc prepareConfigNotes*(graph: ModuleGraph; module: PSym) =
-  # don't be verbose unless the module belongs to the main package:
-  if graph.config.belongsToProjectPackage(module):
-    graph.config.notes = graph.config.mainPackageNotes
-  else:
-    if graph.config.mainPackageNotes == {}: graph.config.mainPackageNotes = graph.config.notes
-    graph.config.notes = graph.config.foreignPackageNotes
-
-proc moduleHasChanged*(graph: ModuleGraph; module: PSym): bool {.inline.} =
-  result = true
-  #module.id >= 0 or isDefined(graph.config, "nimBackendAssumesChange")

compiler/platform.nim

@@ -26,8 +26,7 @@ type
     osNone, osDos, osWindows, osOs2, osLinux, osMorphos, osSkyos, osSolaris,
     osIrix, osNetbsd, osFreebsd, osOpenbsd, osDragonfly, osCrossos, osAix, osPalmos, osQnx,
     osAmiga, osAtari, osNetware, osMacos, osMacosx, osIos, osHaiku, osAndroid, osVxWorks
-    osGenode, osJS, osNimVM, osStandalone, osNintendoSwitch, osFreeRTOS, osZephyr,
-    osNuttX, osAny
+    osGenode, osJS, osNimVM, osStandalone, osNintendoSwitch, osFreeRTOS, osZephyr, osAny
 
 type
   TInfoOSProp* = enum
@@ -194,10 +193,6 @@ const
       objExt: ".o", newLine: "\x0A", pathSep: ":", dirSep: "/",
       scriptExt: ".sh", curDir: ".", exeExt: "", extSep: ".",
       props: {ospPosix}),
-     (name: "NuttX", parDir: "..", dllFrmt: "lib$1.so", altDirSep: "/",
-      objExt: ".o", newLine: "\x0A", pathSep: ":", dirSep: "/",
-      scriptExt: ".sh", curDir: ".", exeExt: "", extSep: ".",
-      props: {ospPosix}),
      (name: "Any", parDir: "..", dllFrmt: "lib$1.so", altDirSep: "/",
       objExt: ".o", newLine: "\x0A", pathSep: ":", dirSep: "/",
       scriptExt: ".sh", curDir: ".", exeExt: "", extSep: ".",

compiler/pragmas.nim

@@ -81,8 +81,7 @@ const
     wGuard, wGoto, wCursor, wNoalias, wAlign}
   constPragmas* = declPragmas + {wHeader, wMagic,
     wGensym, wInject,
-    wIntDefine, wStrDefine, wBoolDefine, wDefine,
-    wCompilerProc, wCore}
+    wIntDefine, wStrDefine, wBoolDefine, wCompilerProc, wCore}
   paramPragmas* = {wNoalias, wInject, wGensym}
   letPragmas* = varPragmas
   procTypePragmas* = {FirstCallConv..LastCallConv, wVarargs, wNoSideEffect,
@@ -270,7 +269,7 @@ proc onOff(c: PContext, n: PNode, op: TOptions, resOptions: var TOptions) =
   if isTurnedOn(c, n): resOptions.incl op
   else: resOptions.excl op
 
-proc pragmaNoForward*(c: PContext, n: PNode; flag=sfNoForward) =
+proc pragmaNoForward(c: PContext, n: PNode; flag=sfNoForward) =
   if isTurnedOn(c, n):
     incl(c.module.flags, flag)
     c.features.incl codeReordering
@@ -364,7 +363,7 @@ proc processNote(c: PContext, n: PNode) =
     else: invalidPragma(c, n)
   else: invalidPragma(c, n)
 
-proc pragmaToOptions*(w: TSpecialWord): TOptions {.inline.} =
+proc pragmaToOptions(w: TSpecialWord): TOptions {.inline.} =
   case w
   of wChecks: ChecksOptions
   of wObjChecks: {optObjCheck}
@@ -477,16 +476,8 @@ proc processPop(c: PContext, n: PNode) =
   when defined(debugOptions):
     echo c.config $ n.info, " POP config is now ", c.config.options
 
-proc processDefineConst(c: PContext, n: PNode, sym: PSym, kind: TMagic) =
-  sym.magic = kind
-  if n.kind in nkPragmaCallKinds and n.len == 2:
-    # could also use TLib
-    n[1] = getStrLitNode(c, n)
-
-proc processDefine(c: PContext, n: PNode, sym: PSym) =
-  if sym != nil and sym.kind == skConst:
-    processDefineConst(c, n, sym, mGenericDefine)
-  elif (n.kind in nkPragmaCallKinds and n.len == 2) and (n[1].kind == nkIdent):
+proc processDefine(c: PContext, n: PNode) =
+  if (n.kind in nkPragmaCallKinds and n.len == 2) and (n[1].kind == nkIdent):
     defineSymbol(c.config.symbols, n[1].ident.s)
   else:
     invalidPragma(c, n)
@@ -513,11 +504,8 @@ proc processCompile(c: PContext, n: PNode) =
     var cf = Cfile(nimname: splitFile(src).name,
                    cname: src, obj: dest, flags: {CfileFlag.External},
                    customArgs: customArgs)
-    if not fileExists(src):
-      localError(c.config, n.info, "cannot find: " & src.string)
-    else:
-      extccomp.addExternalFileToCompile(c.config, cf)
-      recordPragma(c, it, "compile", src.string, dest.string, customArgs)
+    extccomp.addExternalFileToCompile(c.config, cf)
+    recordPragma(c, it, "compile", src.string, dest.string, customArgs)
 
   proc getStrLit(c: PContext, n: PNode; i: int): string =
     n[i] = c.semConstExpr(c, n[i])
@@ -929,8 +917,7 @@ proc singlePragma(c: PContext, sym: PSym, n: PNode, i: var int,
       of wThreadVar:
         noVal(c, it)
         incl(sym.flags, {sfThread, sfGlobal})
-      of wDeadCodeElimUnused:
-        warningDeprecated(c.config, n.info, "'{.deadcodeelim: on.}' is deprecated, now a noop")  # deprecated, dead code elim always on
+      of wDeadCodeElimUnused: discard  # deprecated, dead code elim always on
       of wNoForward: pragmaNoForward(c, it)
       of wReorder: pragmaNoForward(c, it, flag = sfReorder)
       of wMagic: processMagic(c, it, sym)
@@ -1079,7 +1066,7 @@ proc singlePragma(c: PContext, sym: PSym, n: PNode, i: var int,
           recordPragma(c, it, "error", s)
           localError(c.config, it.info, errUser, s)
       of wFatal: fatal(c.config, it.info, expectStrLit(c, it))
-      of wDefine: processDefine(c, it, sym)
+      of wDefine: processDefine(c, it)
       of wUndef: processUndef(c, it)
       of wCompile: processCompile(c, it)
       of wLink: processLink(c, it)
@@ -1226,11 +1213,11 @@ proc singlePragma(c: PContext, sym: PSym, n: PNode, i: var int,
         noVal(c, it)
         sym.flags.incl sfBase
       of wIntDefine:
-        processDefineConst(c, n, sym, mIntDefine)
+        sym.magic = mIntDefine
       of wStrDefine:
-        processDefineConst(c, n, sym, mStrDefine)
+        sym.magic = mStrDefine
       of wBoolDefine:
-        processDefineConst(c, n, sym, mBoolDefine)
+        sym.magic = mBoolDefine
       of wUsed:
         noVal(c, it)
         if sym == nil: invalidPragma(c, it)

compiler/renderer.nim

@@ -9,9 +9,10 @@
 
 # This module implements the renderer of the standard Nim representation.
 
-# 'import renderer' is so useful for debugging
-# that Nim shouldn't produce a warning for that:
-{.used.}
+when defined(nimHasUsed):
+  # 'import renderer' is so useful for debugging
+  # that Nim shouldn't produce a warning for that:
+  {.used.}
 
 import
   lexer, options, idents, strutils, ast, msgs, lineinfos
@@ -23,8 +24,7 @@ type
   TRenderFlag* = enum
     renderNone, renderNoBody, renderNoComments, renderDocComments,
     renderNoPragmas, renderIds, renderNoProcDefs, renderSyms, renderRunnableExamples,
-    renderIr, renderNonExportedFields, renderExpandUsing
-
+    renderIr
   TRenderFlags* = set[TRenderFlag]
   TRenderTok* = object
     kind*: TokType
@@ -434,24 +434,6 @@ proc lsons(g: TSrcGen; n: PNode, start: int = 0, theEnd: int = - 1): int =
   result = 0
   for i in start..n.len + theEnd: inc(result, lsub(g, n[i]))
 
-proc origUsingType(n: PNode): PSym {.inline.} =
-  ## Returns the type that a parameter references. Check with referencesUsing first
-  ## to check `n` is actually referencing a using node
-  # If the node is untyped the typ field will be nil
-  if n[0].sym.typ != nil:
-    n[0].sym.typ.sym
-  else: nil
-
-proc referencesUsing(n: PNode): bool =
-  ## Returns true if n references a using statement.
-  ## e.g. proc foo(x) # x doesn't have type or def value so it references a using
-  result = n.kind == nkIdentDefs and
-           # Sometimes the node might not have been semmed (e.g. doc0) and will be nkIdent instead
-           n[0].kind == nkSym and
-           # Templates/macros can have parameters with no type (But their orig type will be nil)
-           n.origUsingType != nil and
-           n[1].kind == nkEmpty and n[2].kind == nkEmpty
-
 proc lsub(g: TSrcGen; n: PNode): int =
   # computes the length of a tree
   if isNil(n): return 0
@@ -492,11 +474,8 @@ proc lsub(g: TSrcGen; n: PNode): int =
   of nkDo: result = lsons(g, n) + len("do__:_")
   of nkConstDef, nkIdentDefs:
     result = lcomma(g, n, 0, - 3)
-    if n.referencesUsing:
-      result += lsub(g, newSymNode(n.origUsingType)) + 2
-    else:
-      if n[^2].kind != nkEmpty: result += lsub(g, n[^2]) + 2
-      if n[^1].kind != nkEmpty: result += lsub(g, n[^1]) + 3
+    if n[^2].kind != nkEmpty: result += lsub(g, n[^2]) + 2
+    if n[^1].kind != nkEmpty: result += lsub(g, n[^1]) + 3
   of nkVarTuple:
     if n[^1].kind == nkEmpty:
       result = lcomma(g, n, 0, - 2) + len("()")
@@ -619,9 +598,7 @@ proc isHideable(config: ConfigRef, n: PNode): bool =
   # xxx compare `ident` directly with `getIdent(cache, wRaises)`, but
   # this requires a `cache`.
   case n.kind
-  of nkExprColonExpr:
-    result = n[0].kind == nkIdent and
-             n[0].ident.s.nimIdentNormalize in ["raises", "tags", "extern", "deprecated", "forbids", "stacktrace"]
+  of nkExprColonExpr: result = n[0].kind == nkIdent and n[0].ident.s in ["raises", "tags", "extern", "deprecated", "forbids"]
   of nkIdent: result = n.ident.s in ["gcsafe", "deprecated"]
   else: result = false
 
@@ -653,7 +630,7 @@ proc gcommaAux(g: var TSrcGen, n: PNode, ind: int, start: int = 0,
     inHideable = false
 
 proc gcomma(g: var TSrcGen, n: PNode, c: TContext, start: int = 0,
-            theEnd: int = -1) =
+            theEnd: int = - 1) =
   var ind: int
   if rfInConstExpr in c.flags:
     ind = g.indent + IndentWidth
@@ -1283,12 +1260,8 @@ proc gsub(g: var TSrcGen, n: PNode, c: TContext, fromStmtList = false) =
     gsub(g, n, 0)
   of nkCheckedFieldExpr, nkHiddenAddr, nkHiddenDeref, nkStringToCString, nkCStringToString:
     if renderIds in g.flags:
-      put(g, tkAddr, $n.kind)
-      put(g, tkParLe, "(")
+      putWithSpace(g, tkAddr, $n.kind)
     gsub(g, n, 0)
-    if renderIds in g.flags:
-      put(g, tkParRi, ")")
-
   of nkLambda:
     putWithSpace(g, tkProc, "proc")
     gsub(g, n, paramsPos)
@@ -1307,11 +1280,7 @@ proc gsub(g: var TSrcGen, n: PNode, c: TContext, fromStmtList = false) =
     gcomma(g, n, 0, -3)
     if n.len >= 2 and n[^2].kind != nkEmpty:
       putWithSpace(g, tkColon, ":")
-      gsub(g, n[^2], c)
-    elif n.referencesUsing and renderExpandUsing in g.flags:
-      putWithSpace(g, tkColon, ":")
-      gsub(g, newSymNode(n.origUsingType), c)
-
+      gsub(g, n, n.len - 2)
     if n.len >= 1 and n[^1].kind != nkEmpty:
       put(g, tkSpaces, Space)
       putWithSpace(g, tkEquals, "=")
@@ -1482,11 +1451,9 @@ proc gsub(g: var TSrcGen, n: PNode, c: TContext, fromStmtList = false) =
   of nkRecList:
     indentNL(g)
     for i in 0..<n.len:
-      if n[i].kind == nkIdentDefs and n[i][0].kind == nkPostfix or
-                        renderNonExportedFields in g.flags:
-        optNL(g)
-        gsub(g, n[i], c)
-        gcoms(g)
+      optNL(g)
+      gsub(g, n[i], c)
+      gcoms(g)
     dedent(g)
     putNL(g)
   of nkOfInherit:

compiler/reorder.nim

@@ -106,7 +106,6 @@ proc computeDeps(cache: IdentCache; n: PNode, declares, uses: var IntSet; topLev
     if a.kind == nkExprColonExpr and a[0].kind == nkIdent and a[0].ident.s == "pragma":
       # user defined pragma
       decl(a[1])
-      for i in 1..<n.safeLen: deps(n[i])
     else:
       for i in 0..<n.safeLen: deps(n[i])
   of nkMixinStmt, nkBindStmt: discard

compiler/scriptconfig.nim

@@ -11,10 +11,10 @@
 ## language.
 
 import
-  ast, modules, idents, condsyms,
-  options, llstream, vm, vmdef, commands,
+  ast, modules, idents, passes, condsyms,
+  options, sem, llstream, vm, vmdef, commands,
   os, times, osproc, wordrecg, strtabs, modulegraphs,
-  pathutils, pipelines
+  pathutils
 
 when defined(nimPreviewSlimSystem):
   import std/syncio
@@ -197,11 +197,13 @@ proc runNimScript*(cache: IdentCache; scriptName: AbsoluteFile;
   conf.symbolFiles = disabledSf
 
   let graph = newModuleGraph(cache, conf)
-  connectPipelineCallbacks(graph)
+  connectCallbacks(graph)
   if freshDefines: initDefines(conf.symbols)
 
   defineSymbol(conf.symbols, "nimscript")
   defineSymbol(conf.symbols, "nimconfig")
+  registerPass(graph, semPass)
+  registerPass(graph, evalPass)
 
   conf.searchPaths.add(conf.libpath)
 
@@ -216,9 +218,8 @@ proc runNimScript*(cache: IdentCache; scriptName: AbsoluteFile;
   var vm = setupVM(m, cache, scriptName.string, graph, idgen)
   graph.vm = vm
 
-  graph.setPipeLinePass(EvalPass)
-  graph.compilePipelineSystemModule()
-  discard graph.processPipelineModule(m, vm.idgen, stream)
+  graph.compileSystemModule()
+  discard graph.processModule(m, vm.idgen, stream)
 
   # watch out, "newruntime" can be set within NimScript itself and then we need
   # to remember this:

compiler/sem.nim

@@ -13,7 +13,7 @@ import
   ast, strutils, options, astalgo, trees,
   wordrecg, ropes, msgs, idents, renderer, types, platform, math,
   magicsys, nversion, nimsets, semfold, modulepaths, importer,
-  procfind, lookups, pragmas, semdata, semtypinst, sigmatch,
+  procfind, lookups, pragmas, passes, semdata, semtypinst, sigmatch,
   intsets, transf, vmdef, vm, aliases, cgmeth, lambdalifting,
   evaltempl, patterns, parampatterns, sempass2, linter, semmacrosanity,
   lowerings, plugins/active, lineinfos, strtabs, int128,
@@ -81,7 +81,7 @@ template semIdeForTemplateOrGeneric(c: PContext; n: PNode;
 proc fitNodePostMatch(c: PContext, formal: PType, arg: PNode): PNode =
   let x = arg.skipConv
   if (x.kind == nkCurly and formal.kind == tySet and formal.base.kind != tyGenericParam) or
-    (x.kind in {nkPar, nkTupleConstr}) and formal.kind notin {tyUntyped, tyBuiltInTypeClass, tyAnything}:
+    (x.kind in {nkPar, nkTupleConstr}) and formal.kind notin {tyUntyped, tyBuiltInTypeClass}:
     changeType(c, x, formal, check=true)
   result = arg
   result = skipHiddenSubConv(result, c.graph, c.idgen)
@@ -405,6 +405,9 @@ proc semExprFlagDispatched(c: PContext, n: PNode, flags: TExprFlags; expectedTyp
       evaluated = evalAtCompileTime(c, result)
       if evaluated != nil: return evaluated
 
+when not defined(nimHasSinkInference):
+  {.pragma: nosinks.}
+
 include hlo, seminst, semcall
 
 proc resetSemFlag(n: PNode) =
@@ -436,7 +439,7 @@ proc semAfterMacroCall(c: PContext, call, macroResult: PNode,
       # does not mean we expect a tyTypeDesc.
       retType = retType[0]
     case retType.kind
-    of tyUntyped, tyAnything:
+    of tyUntyped:
       # Not expecting a type here allows templates like in ``tmodulealias.in``.
       result = semExpr(c, result, flags, expectedType)
     of tyTyped:
@@ -550,17 +553,18 @@ proc pickCaseBranchIndex(caseExpr, matched: PNode): int =
   if endsWithElse:
     return caseExpr.len - 1
 
-proc defaultFieldsForTheUninitialized(c: PContext, recNode: PNode): seq[PNode]
-proc defaultNodeField(c: PContext, a: PNode, aTyp: PType): PNode
+proc defaultFieldsForTheUninitialized(c: PContext, recNode: PNode, id: var IntSet): seq[PNode]
+proc defaultNodeField(c: PContext, a: PNode, aTyp: PType, id: var IntSet): PNode
 proc defaultNodeField(c: PContext, a: PNode): PNode
+proc defaultNodeField(c: PContext, a: PNode, aTyp: PType): PNode
 
 const defaultFieldsSkipTypes = {tyGenericInst, tyAlias, tySink}
 
-proc defaultFieldsForTuple(c: PContext, recNode: PNode, hasDefault: var bool): seq[PNode] =
+proc defaultFieldsForTuple(c: PContext, recNode: PNode, id: var IntSet, hasDefault: var bool): seq[PNode] =
   case recNode.kind
   of nkRecList:
     for field in recNode:
-      result.add defaultFieldsForTuple(c, field, hasDefault)
+      result.add defaultFieldsForTuple(c, field, id, hasDefault)
   of nkSym:
     let field = recNode.sym
     let recType = recNode.typ.skipTypes(defaultFieldsSkipTypes)
@@ -569,10 +573,10 @@ proc defaultFieldsForTuple(c: PContext, recNode: PNode, hasDefault: var bool): s
       result.add newTree(nkExprColonExpr, recNode, field.ast)
     else:
       if recType.kind in {tyObject, tyArray, tyTuple}:
-        let asgnExpr = defaultNodeField(c, recNode, recNode.typ)
+        let asgnExpr = defaultNodeField(c, recNode, recNode.typ, id)
         if asgnExpr != nil:
           hasDefault = true
-          asgnExpr.flags.incl nfSkipFieldChecking
+          asgnExpr.flags.incl nfUseDefaultField
           result.add newTree(nkExprColonExpr, recNode, asgnExpr)
           return
 
@@ -582,17 +586,17 @@ proc defaultFieldsForTuple(c: PContext, recNode: PNode, hasDefault: var bool): s
                       newSymNode(getSysMagic(c.graph, recNode.info, "zeroDefault", mZeroDefault)),
                       newNodeIT(nkType, recNode.info, asgnType)
                     )
-      asgnExpr.flags.incl nfSkipFieldChecking
+      asgnExpr.flags.incl nfUseDefaultField
       asgnExpr.typ = recType
       result.add newTree(nkExprColonExpr, recNode, asgnExpr)
   else:
     doAssert false
 
-proc defaultFieldsForTheUninitialized(c: PContext, recNode: PNode): seq[PNode] =
+proc defaultFieldsForTheUninitialized(c: PContext, recNode: PNode, id: var IntSet): seq[PNode] =
   case recNode.kind
   of nkRecList:
     for field in recNode:
-      result.add defaultFieldsForTheUninitialized(c, field)
+      result.add defaultFieldsForTheUninitialized(c, field, id)
   of nkRecCase:
     let discriminator = recNode[0]
     var selectedBranch: int
@@ -604,55 +608,62 @@ proc defaultFieldsForTheUninitialized(c: PContext, recNode: PNode): seq[PNode] =
       defaultValue = newIntNode(nkIntLit#[c.graph]#, 0)
       defaultValue.typ = discriminator.typ
     selectedBranch = recNode.pickCaseBranchIndex defaultValue
-    defaultValue.flags.incl nfSkipFieldChecking
+    defaultValue.flags.incl nfUseDefaultField
     result.add newTree(nkExprColonExpr, discriminator, defaultValue)
-    result.add defaultFieldsForTheUninitialized(c, recNode[selectedBranch][^1])
+    result.add defaultFieldsForTheUninitialized(c, recNode[selectedBranch][^1], id)
   of nkSym:
     let field = recNode.sym
     let recType = recNode.typ.skipTypes(defaultFieldsSkipTypes)
     if field.ast != nil: #Try to use default value
       result.add newTree(nkExprColonExpr, recNode, field.ast)
     elif recType.kind in {tyObject, tyArray, tyTuple}:
-      let asgnExpr = defaultNodeField(c, recNode, recType)
+      let asgnExpr = defaultNodeField(c, recNode, recType, id)
       if asgnExpr != nil:
         asgnExpr.typ = recType
-        asgnExpr.flags.incl nfSkipFieldChecking
+        asgnExpr.flags.incl nfUseDefaultField
         result.add newTree(nkExprColonExpr, recNode, asgnExpr)
   else:
     doAssert false
 
-proc defaultNodeField(c: PContext, a: PNode, aTyp: PType): PNode =
+proc defaultNodeField(c: PContext, a: PNode, aTyp: PType, id: var IntSet): PNode =
   let aTypSkip = aTyp.skipTypes(defaultFieldsSkipTypes)
   if aTypSkip.kind == tyObject:
-    let child = defaultFieldsForTheUninitialized(c, aTypSkip.n)
+    if id.containsOrIncl(aTypSkip.id):
+      return
+    let child = defaultFieldsForTheUninitialized(c, aTypSkip.n, id)
     if child.len > 0:
       var asgnExpr = newTree(nkObjConstr, newNodeIT(nkType, a.info, aTypSkip))
       asgnExpr.typ = aTypSkip
       asgnExpr.sons.add child
       result = semExpr(c, asgnExpr)
   elif aTypSkip.kind == tyArray:
-    let child = defaultNodeField(c, a, aTypSkip[1])
+    let child = defaultNodeField(c, a, aTypSkip[1], id)
 
     if child != nil:
       let node = newNode(nkIntLit)
       node.intVal = toInt64(lengthOrd(c.graph.config, aTypSkip))
-      result = semExpr(c, newTree(nkCall, newSymNode(getCompilerProc(c.graph, "nimArrayWith"), a.info),
-        semExprWithType(c, child),
-        node
-          ))
+      result = semExpr(c, newTree(nkCall, newSymNode(getSysSym(c.graph, a.info, "arrayWith"), a.info),
+              semExprWithType(c, child),
+              node
+                ))
       result.typ = aTyp
   elif aTypSkip.kind == tyTuple:
     var hasDefault = false
     if aTypSkip.n != nil:
-      let children = defaultFieldsForTuple(c, aTypSkip.n, hasDefault)
+      let children = defaultFieldsForTuple(c, aTypSkip.n, id, hasDefault)
       if hasDefault and children.len > 0:
         result = newNodeI(nkTupleConstr, a.info)
         result.typ = aTyp
         result.sons.add children
         result = semExpr(c, result)
 
+proc defaultNodeField(c: PContext, a: PNode, aTyp: PType): PNode =
+  var s = initIntSet()
+  defaultNodeField(c, a, aTyp, s)
+
 proc defaultNodeField(c: PContext, a: PNode): PNode =
-  result = defaultNodeField(c, a, a.typ)
+  var s = initIntSet()
+  result = defaultNodeField(c, a, a.typ, s)
 
 include semtempl, semgnrc, semstmts, semexprs
 
@@ -666,46 +677,41 @@ proc addCodeForGenerics(c: PContext, n: PNode) =
         n.add prc.ast
   c.lastGenericIdx = c.generics.len
 
-proc preparePContext*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PContext =
-  result = newContext(graph, module)
-  result.idgen = idgen
-  result.enforceVoidContext = newType(tyTyped, nextTypeId(idgen), nil)
-  result.voidType = newType(tyVoid, nextTypeId(idgen), nil)
+proc myOpen(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext {.nosinks.} =
+  var c = newContext(graph, module)
+  c.idgen = idgen
+  c.enforceVoidContext = newType(tyTyped, nextTypeId(idgen), nil)
+  c.voidType = newType(tyVoid, nextTypeId(idgen), nil)
 
-  if result.p != nil: internalError(graph.config, module.info, "sem.preparePContext")
-  result.semConstExpr = semConstExpr
-  result.semExpr = semExpr
-  result.semTryExpr = tryExpr
-  result.semTryConstExpr = tryConstExpr
-  result.computeRequiresInit = computeRequiresInit
-  result.semOperand = semOperand
-  result.semConstBoolExpr = semConstBoolExpr
-  result.semOverloadedCall = semOverloadedCall
-  result.semInferredLambda = semInferredLambda
-  result.semGenerateInstance = generateInstance
-  result.semTypeNode = semTypeNode
-  result.instTypeBoundOp = sigmatch.instTypeBoundOp
-  result.hasUnresolvedArgs = hasUnresolvedArgs
-  result.templInstCounter = new int
+  if c.p != nil: internalError(graph.config, module.info, "sem.myOpen")
+  c.semConstExpr = semConstExpr
+  c.semExpr = semExpr
+  c.semTryExpr = tryExpr
+  c.semTryConstExpr = tryConstExpr
+  c.computeRequiresInit = computeRequiresInit
+  c.semOperand = semOperand
+  c.semConstBoolExpr = semConstBoolExpr
+  c.semOverloadedCall = semOverloadedCall
+  c.semInferredLambda = semInferredLambda
+  c.semGenerateInstance = generateInstance
+  c.semTypeNode = semTypeNode
+  c.instTypeBoundOp = sigmatch.instTypeBoundOp
+  c.hasUnresolvedArgs = hasUnresolvedArgs
+  c.templInstCounter = new int
 
-  pushProcCon(result, module)
-  pushOwner(result, result.module)
+  pushProcCon(c, module)
+  pushOwner(c, c.module)
 
-  result.moduleScope = openScope(result)
-  result.moduleScope.addSym(module) # a module knows itself
+  c.moduleScope = openScope(c)
+  c.moduleScope.addSym(module) # a module knows itself
 
   if sfSystemModule in module.flags:
     graph.systemModule = module
-  result.topLevelScope = openScope(result)
+  c.topLevelScope = openScope(c)
+  result = c
 
 proc isImportSystemStmt(g: ModuleGraph; n: PNode): bool =
   if g.systemModule == nil: return false
-  var n = n
-  if n.kind == nkStmtList:
-    for i in 0..<n.len-1:
-      if n[i].kind notin {nkCommentStmt, nkEmpty}:
-        n = n[i]
-        break
   case n.kind
   of nkImportStmt:
     for x in n:
@@ -770,7 +776,8 @@ proc recoverContext(c: PContext) =
   while getCurrOwner(c).kind != skModule: popOwner(c)
   while c.p != nil and c.p.owner.kind != skModule: c.p = c.p.next
 
-proc semWithPContext*(c: PContext, n: PNode): PNode =
+proc myProcess(context: PPassContext, n: PNode): PNode {.nosinks.} =
+  var c = PContext(context)
   # no need for an expensive 'try' if we stop after the first error anyway:
   if c.config.errorMax <= 1:
     result = semStmtAndGenerateGenerics(c, n)
@@ -791,13 +798,13 @@ proc semWithPContext*(c: PContext, n: PNode): PNode =
       #if c.config.cmd == cmdIdeTools: findSuggest(c, n)
   storeRodNode(c, result)
 
-
 proc reportUnusedModules(c: PContext) =
   for i in 0..high(c.unusedImports):
     if sfUsed notin c.unusedImports[i][0].flags:
       message(c.config, c.unusedImports[i][1], warnUnusedImportX, c.unusedImports[i][0].name.s)
 
-proc closePContext*(graph: ModuleGraph; c: PContext, n: PNode): PNode =
+proc myClose(graph: ModuleGraph; context: PPassContext, n: PNode): PNode =
+  var c = PContext(context)
   if c.config.cmd == cmdIdeTools and not c.suggestionsMade:
     suggestSentinel(c)
   closeScope(c)         # close module's scope
@@ -812,3 +819,6 @@ proc closePContext*(graph: ModuleGraph; c: PContext, n: PNode): PNode =
   popOwner(c)
   popProcCon(c)
   sealRodFile(c)
+
+const semPass* = makePass(myOpen, myProcess, myClose,
+                          isFrontend = true)

compiler/semcall.nim

@@ -10,8 +10,7 @@
 ## This module implements semantic checking for calls.
 # included from sem.nim
 
-from std/algorithm import sort
-
+from algorithm import sort
 
 proc sameMethodDispatcher(a, b: PSym): bool =
   result = false
@@ -43,7 +42,6 @@ proc initCandidateSymbols(c: PContext, headSymbol: PNode,
                           best, alt: var TCandidate,
                           o: var TOverloadIter,
                           diagnostics: bool): seq[tuple[s: PSym, scope: int]] =
-  ## puts all overloads into a seq and prepares best+alt
   result = @[]
   var symx = initOverloadIter(o, c, headSymbol)
   while symx != nil:
@@ -65,35 +63,36 @@ proc pickBestCandidate(c: PContext, headSymbol: PNode,
                        errors: var CandidateErrors,
                        diagnosticsFlag: bool,
                        errorsEnabled: bool, flags: TExprFlags) =
-  # `matches` may find new symbols, so keep track of count
-  var symCount = c.currentScope.symbols.counter
-
   var o: TOverloadIter
-  # https://github.com/nim-lang/Nim/issues/21272
-  # prevent mutation during iteration by storing them in a seq
-  # luckily `initCandidateSymbols` does just that
-  var syms = initCandidateSymbols(c, headSymbol, initialBinding, filter,
-                                  best, alt, o, diagnosticsFlag)
-  if len(syms) == 0:
-    return
-  # current overload being considered
-  var sym = syms[0].s
-  var scope = syms[0].scope
-
-  # starts at 1 because 0 is already done with setup, only needs checking
-  var nextSymIndex = 1
-  var z: TCandidate # current candidate
-  while true:
+  var sym = initOverloadIter(o, c, headSymbol)
+  var scope = o.lastOverloadScope
+  # Thanks to the lazy semchecking for operands, we need to check whether
+  # 'initCandidate' modifies the symbol table (via semExpr).
+  # This can occur in cases like 'init(a, 1, (var b = new(Type2); b))'
+  let counterInitial = c.currentScope.symbols.counter
+  var syms: seq[tuple[s: PSym, scope: int]]
+  var noSyms = true
+  var nextSymIndex = 0
+  while sym != nil:
+    if sym.kind in filter:
+      # Initialise 'best' and 'alt' with the first available symbol
+      initCandidate(c, best, sym, initialBinding, scope, diagnosticsFlag)
+      initCandidate(c, alt, sym, initialBinding, scope, diagnosticsFlag)
+      best.state = csNoMatch
+      break
+    else:
+      sym = nextOverloadIter(o, c, headSymbol)
+      scope = o.lastOverloadScope
+  var z: TCandidate
+  while sym != nil:
+    if sym.kind notin filter:
+      sym = nextOverloadIter(o, c, headSymbol)
+      scope = o.lastOverloadScope
+      continue
     determineType(c, sym)
     initCandidate(c, z, sym, initialBinding, scope, diagnosticsFlag)
-
-    # this is kinda backwards as without a check here the described
-    # problems in recalc would not happen, but instead it 100%
-    # does check forever in some cases
-    if c.currentScope.symbols.counter == symCount:
-      # may introduce new symbols with caveats described in recalc branch
+    if c.currentScope.symbols.counter == counterInitial or syms.len != 0:
       matches(c, n, orig, z)
-
       if z.state == csMatch:
         # little hack so that iterators are preferred over everything else:
         if sym.kind == skIterator:
@@ -113,36 +112,22 @@ proc pickBestCandidate(c: PContext, headSymbol: PNode,
           firstMismatch: z.firstMismatch,
           diagnostics: z.diagnostics))
     else:
-      # this branch feels like a ticking timebomb
-      # one of two bad things could happen
-      # 1) new symbols are discovered but the loop ends before we recalc
-      # 2) new symbols are discovered and resemmed forever
-      # not 100% sure if these are possible though as they would rely
-      #  on somehow introducing a new overload during overload resolution
-
       # Symbol table has been modified. Restart and pre-calculate all syms
       # before any further candidate init and compare. SLOW, but rare case.
       syms = initCandidateSymbols(c, headSymbol, initialBinding, filter,
                                   best, alt, o, diagnosticsFlag)
-
-      # reset counter because syms may be in a new order
-      symCount = c.currentScope.symbols.counter
-      nextSymIndex = 0
-
-      # just in case, should be impossible though
-      if syms.len == 0:
-        break
-
-    if nextSymIndex > high(syms):
-      # we have reached the end
+      noSyms = false
+    if noSyms:
+      sym = nextOverloadIter(o, c, headSymbol)
+      scope = o.lastOverloadScope
+    elif nextSymIndex < syms.len:
+      # rare case: retrieve the next pre-calculated symbol
+      sym = syms[nextSymIndex].s
+      scope = syms[nextSymIndex].scope
+      nextSymIndex += 1
+    else:
       break
 
-    # advance to next sym
-    sym = syms[nextSymIndex].s
-    scope = syms[nextSymIndex].scope
-    inc(nextSymIndex)
-
-
 proc effectProblem(f, a: PType; result: var string; c: PContext) =
   if f.kind == tyProc and a.kind == tyProc:
     if tfThread in f.flags and tfThread notin a.flags:
@@ -207,7 +192,7 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
   # argument in order to remove plenty of candidates. This is
   # comparable to what C# does and C# is doing fine.
   var filterOnlyFirst = false
-  if optShowAllMismatches notin c.config.globalOptions and verboseTypeMismatch in c.config.legacyFeatures:
+  if optShowAllMismatches notin c.config.globalOptions:
     for err in errors:
       if err.firstMismatch.arg > 1:
         filterOnlyFirst = true
@@ -223,10 +208,6 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
     if filterOnlyFirst and err.firstMismatch.arg == 1:
       inc skipped
       continue
-
-    if verboseTypeMismatch notin c.config.legacyFeatures:
-      candidates.add "[" & $err.firstMismatch.arg & "] "
-
     if err.sym.kind in routineKinds and err.sym.ast != nil:
       candidates.add(renderTree(err.sym.ast,
             {renderNoBody, renderNoComments, renderNoPragmas}))
@@ -236,7 +217,7 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
     candidates.add("\n")
     let nArg = if err.firstMismatch.arg < n.len: n[err.firstMismatch.arg] else: nil
     let nameParam = if err.firstMismatch.formal != nil: err.firstMismatch.formal.name.s else: ""
-    if n.len > 1 and verboseTypeMismatch in c.config.legacyFeatures:
+    if n.len > 1:
       candidates.add("  first type mismatch at position: " & $err.firstMismatch.arg)
       # candidates.add "\n  reason: " & $err.firstMismatch.kind # for debugging
       case err.firstMismatch.kind
@@ -293,28 +274,11 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
 const
   errTypeMismatch = "type mismatch: got <"
   errButExpected = "but expected one of:"
-  errExpectedPosition = "Expected one of (first mismatch at [position]):"
   errUndeclaredField = "undeclared field: '$1'"
   errUndeclaredRoutine = "attempting to call undeclared routine: '$1'"
   errBadRoutine = "attempting to call routine: '$1'$2"
   errAmbiguousCallXYZ = "ambiguous call; both $1 and $2 match for: $3"
 
-proc describeParamList(c: PContext, n: PNode, startIdx = 1; prefer = preferName): string =
-  result = "Expression: " & $n
-  for i in startIdx..<n.len:
-    result.add "\n  [" & $i & "] " & renderTree(n[i]) & ": "
-    result.add describeArg(c, n, i, startIdx, prefer)
-  result.add "\n"
-
-template legacynotFoundError(c: PContext, n: PNode, errors: CandidateErrors) =
-  let (prefer, candidates) = presentFailedCandidates(c, n, errors)
-  var result = errTypeMismatch
-  result.add(describeArgs(c, n, 1, prefer))
-  result.add('>')
-  if candidates != "":
-    result.add("\n" & errButExpected & "\n" & candidates)
-  localError(c.config, n.info, result & "\nexpression: " & $n)
-
 proc notFoundError*(c: PContext, n: PNode, errors: CandidateErrors) =
   # Gives a detailed error message; this is separated from semOverloadedCall,
   # as semOverloadedCall is already pretty slow (and we need this information
@@ -338,19 +302,17 @@ proc notFoundError*(c: PContext, n: PNode, errors: CandidateErrors) =
     if n[0].kind in nkIdentKinds:
       let ident = considerQuotedIdent(c, n[0], n).s
       localError(c.config, n.info, errUndeclaredRoutine % ident)
-    else:
+    else: 
       localError(c.config, n.info, "expression '$1' cannot be called" % n[0].renderTree)
     return
 
-  if verboseTypeMismatch in c.config.legacyFeatures:
-    legacynotFoundError(c, n, errors)
-  else:
-    let (prefer, candidates) = presentFailedCandidates(c, n, errors)
-    var result = "type mismatch\n"
-    result.add describeParamList(c, n, 1, prefer)
-    if candidates != "":
-      result.add("\n" & errExpectedPosition & "\n" & candidates)
-    localError(c.config, n.info, result)
+  let (prefer, candidates) = presentFailedCandidates(c, n, errors)
+  var result = errTypeMismatch
+  result.add(describeArgs(c, n, 1, prefer))
+  result.add('>')
+  if candidates != "":
+    result.add("\n" & errButExpected & "\n" & candidates)
+  localError(c.config, n.info, result & "\nexpression: " & $n)
 
 proc bracketNotFoundError(c: PContext; n: PNode) =
   var errors: CandidateErrors = @[]
@@ -615,7 +577,7 @@ proc tryDeref(n: PNode): PNode =
   result.add n
 
 proc semOverloadedCall(c: PContext, n, nOrig: PNode,
-                       filter: TSymKinds, flags: TExprFlags): PNode =
+                       filter: TSymKinds, flags: TExprFlags): PNode {.nosinks.} =
   var errors: CandidateErrors = @[] # if efExplain in flags: @[] else: nil
   var r = resolveOverloads(c, n, nOrig, filter, flags, errors, efExplain in flags)
   if r.state == csMatch:
@@ -630,7 +592,7 @@ proc semOverloadedCall(c: PContext, n, nOrig: PNode,
     if efExplain notin flags:
       # repeat the overload resolution,
       # this time enabling all the diagnostic output (this should fail again)
-      result = semOverloadedCall(c, n, nOrig, filter, flags + {efExplain})
+      discard semOverloadedCall(c, n, nOrig, filter, flags + {efExplain})
     elif efNoUndeclared notin flags:
       notFoundError(c, n, errors)
 

compiler/semdata.nim

@@ -75,7 +75,6 @@ type
       # overload resolution.
     efNoDiagnostics,
     efTypeAllowed # typeAllowed will be called after
-    efWantNoDefaults
 
   TExprFlags* = set[TExprFlag]
 

compiler/semexprs.nim

@@ -302,7 +302,7 @@ proc semConv(c: PContext, n: PNode; expectedType: PType = nil): PNode =
   result = newNodeI(nkConv, n.info)
 
   var targetType = semTypeNode(c, n[0], nil)
-  case targetType.skipTypes({tyDistinct}).kind
+  case targetType.kind
   of tyTypeDesc:
     internalAssert c.config, targetType.len > 0
     if targetType.base.kind == tyNone:
@@ -319,8 +319,6 @@ proc semConv(c: PContext, n: PNode; expectedType: PType = nil): PNode =
       return evaluated
     else:
       targetType = targetType.base
-  of tyAnything, tyUntyped, tyTyped:
-    localError(c.config, n.info, "illegal type conversion to '$1'" % typeToString(targetType))
   else: discard
 
   maybeLiftType(targetType, c, n[0].info)
@@ -396,8 +394,6 @@ proc semCast(c: PContext, n: PNode): PNode =
   let castedExpr = semExprWithType(c, n[1])
   if castedExpr.kind == nkClosedSymChoice:
     errorUseQualifier(c, n[1].info, castedExpr)
-  if targetType == nil:
-    localError(c.config, n.info, "Invalid usage of cast, cast requires a type to convert to, e.g., cast[int](0d).")
   if tfHasMeta in targetType.flags:
     localError(c.config, n[0].info, "cannot cast to a non concrete type: '$1'" % $targetType)
   if not isCastable(c, targetType, castedExpr.typ, n.info):
@@ -469,7 +465,6 @@ proc isOpImpl(c: PContext, n: PNode, flags: TExprFlags): PNode =
       res = t.kind == tyProc and
             t.callConv == ccClosure
     of "iterator":
-      # holdover from when `is iterator` didn't work
       let t = skipTypes(t1, abstractRange)
       res = t.kind == tyProc and
             t.callConv == ccClosure and
@@ -696,9 +691,6 @@ proc semArrayConstr(c: PContext, n: PNode, flags: TExprFlags; expectedType: PTyp
 proc fixAbstractType(c: PContext, n: PNode) =
   for i in 1..<n.len:
     let it = n[i]
-    if it == nil:
-      localError(c.config, n.info, "'$1' has nil child at index $2" % [renderTree(n, {renderNoComments}), $i])
-      return
     # do not get rid of nkHiddenSubConv for OpenArrays, the codegen needs it:
     if it.kind == nkHiddenSubConv and
         skipTypes(it.typ, abstractVar).kind notin {tyOpenArray, tyVarargs}:
@@ -744,7 +736,7 @@ proc hasUnresolvedArgs(c: PContext, n: PNode): bool =
       if hasUnresolvedArgs(c, n[i]): return true
     return false
 
-proc newHiddenAddrTaken(c: PContext, n: PNode, isOutParam: bool): PNode =
+proc newHiddenAddrTaken(c: PContext, n: PNode): PNode =
   if n.kind == nkHiddenDeref and not (c.config.backend == backendCpp or
                                       sfCompileToCpp in c.module.flags):
     checkSonsLen(n, 1, c.config)
@@ -753,17 +745,13 @@ proc newHiddenAddrTaken(c: PContext, n: PNode, isOutParam: bool): PNode =
     result = newNodeIT(nkHiddenAddr, n.info, makeVarType(c, n.typ))
     result.add n
     let aa = isAssignable(c, n)
-    let sym = getRoot(n)
     if aa notin {arLValue, arLocalLValue}:
       if aa == arDiscriminant and c.inUncheckedAssignSection > 0:
         discard "allow access within a cast(unsafeAssign) section"
-      elif strictDefs in c.features and aa == arAddressableConst and
-              sym != nil and sym.kind == skLet and isOutParam:
-        discard "allow let varaibles to be passed to out parameters"
       else:
         localError(c.config, n.info, errVarForOutParamNeededX % renderNotLValue(n))
 
-proc analyseIfAddressTaken(c: PContext, n: PNode, isOutParam: bool): PNode =
+proc analyseIfAddressTaken(c: PContext, n: PNode): PNode =
   result = n
   case n.kind
   of nkSym:
@@ -771,7 +759,7 @@ proc analyseIfAddressTaken(c: PContext, n: PNode, isOutParam: bool): PNode =
     if n.sym.typ != nil and
         skipTypes(n.sym.typ, abstractInst-{tyTypeDesc}).kind notin {tyVar, tyLent}:
       incl(n.sym.flags, sfAddrTaken)
-      result = newHiddenAddrTaken(c, n, isOutParam)
+      result = newHiddenAddrTaken(c, n)
   of nkDotExpr:
     checkSonsLen(n, 2, c.config)
     if n[1].kind != nkSym:
@@ -779,16 +767,16 @@ proc analyseIfAddressTaken(c: PContext, n: PNode, isOutParam: bool): PNode =
       return
     if skipTypes(n[1].sym.typ, abstractInst-{tyTypeDesc}).kind notin {tyVar, tyLent}:
       incl(n[1].sym.flags, sfAddrTaken)
-      result = newHiddenAddrTaken(c, n, isOutParam)
+      result = newHiddenAddrTaken(c, n)
   of nkBracketExpr:
     checkMinSonsLen(n, 1, c.config)
     if skipTypes(n[0].typ, abstractInst-{tyTypeDesc}).kind notin {tyVar, tyLent}:
       if n[0].kind == nkSym: incl(n[0].sym.flags, sfAddrTaken)
-      result = newHiddenAddrTaken(c, n, isOutParam)
+      result = newHiddenAddrTaken(c, n)
   else:
-    result = newHiddenAddrTaken(c, n, isOutParam)
+    result = newHiddenAddrTaken(c, n)
 
-proc analyseIfAddressTakenInCall(c: PContext, n: PNode, isConverter = false) =
+proc analyseIfAddressTakenInCall(c: PContext, n: PNode) =
   checkMinSonsLen(n, 1, c.config)
   const
     FakeVarParams = {mNew, mNewFinalize, mInc, ast.mDec, mIncl, mExcl,
@@ -796,15 +784,10 @@ proc analyseIfAddressTakenInCall(c: PContext, n: PNode, isConverter = false) =
       mAppendSeqElem, mNewSeq, mReset, mShallowCopy, mDeepCopy, mMove,
       mWasMoved}
 
-  template checkIfConverterCalled(c: PContext, n: PNode) =
-    ## Checks if there is a converter call which wouldn't be checked otherwise
-    # Call can sometimes be wrapped in a deref
-    let node = if n.kind == nkHiddenDeref: n[0] else: n
-    if node.kind == nkHiddenCallConv:
-      analyseIfAddressTakenInCall(c, node, true)
   # get the real type of the callee
   # it may be a proc var with a generic alias type, so we skip over them
   var t = n[0].typ.skipTypes({tyGenericInst, tyAlias, tySink})
+
   if n[0].kind == nkSym and n[0].sym.magic in FakeVarParams:
     # BUGFIX: check for L-Value still needs to be done for the arguments!
     # note sometimes this is eval'ed twice so we check for nkHiddenAddr here:
@@ -819,8 +802,6 @@ proc analyseIfAddressTakenInCall(c: PContext, n: PNode, isConverter = false) =
               discard "allow access within a cast(unsafeAssign) section"
             else:
               localError(c.config, it.info, errVarForOutParamNeededX % $it)
-        # Make sure to still check arguments for converters
-        c.checkIfConverterCalled(n[i])
     # bug #5113: disallow newSeq(result) where result is a 'var T':
     if n[0].sym.magic in {mNew, mNewFinalize, mNewSeq}:
       var arg = n[1] #.skipAddr
@@ -832,14 +813,15 @@ proc analyseIfAddressTakenInCall(c: PContext, n: PNode, isConverter = false) =
     return
   for i in 1..<n.len:
     let n = if n.kind == nkHiddenDeref: n[0] else: n
-    c.checkIfConverterCalled(n[i])
+    if n[i].kind == nkHiddenCallConv:
+      # we need to recurse explicitly here as converters can create nested
+      # calls and then they wouldn't be analysed otherwise
+      analyseIfAddressTakenInCall(c, n[i])
     if i < t.len and
         skipTypes(t[i], abstractInst-{tyTypeDesc}).kind in {tyVar}:
-      # Converters wrap var parameters in nkHiddenAddr but they haven't been analysed yet.
-      # So we need to make sure we are checking them still when in a converter call
-      if n[i].kind != nkHiddenAddr or isConverter:
-        n[i] = analyseIfAddressTaken(c, n[i].skipAddr(), isOutParam(skipTypes(t[i], abstractInst-{tyTypeDesc})))
-        
+      if n[i].kind != nkHiddenAddr:
+        n[i] = analyseIfAddressTaken(c, n[i])
+
 include semmagic
 
 proc evalAtCompileTime(c: PContext, n: PNode): PNode =
@@ -987,14 +969,6 @@ proc afterCallActions(c: PContext; n, orig: PNode, flags: TExprFlags; expectedTy
     return errorNode(c, n)
 
   result = n
-
-  when defined(nimsuggest):
-    if c.config.expandProgress:
-      if c.config.expandLevels == 0:
-        return n
-      else:
-        c.config.expandLevels -= 1
-
   let callee = result[0].sym
   case callee.kind
   of skMacro: result = semMacroExpr(c, result, orig, callee, flags, expectedType)
@@ -1005,7 +979,7 @@ proc afterCallActions(c: PContext; n, orig: PNode, flags: TExprFlags; expectedTy
     fixAbstractType(c, result)
     analyseIfAddressTakenInCall(c, result)
     if callee.magic != mNone:
-      result = magicsAfterOverloadResolution(c, result, flags, expectedType)
+      result = magicsAfterOverloadResolution(c, result, flags)
     when false:
       if result.typ != nil and
           not (result.typ.kind == tySequence and result.typ[0].kind == tyEmpty):
@@ -1292,7 +1266,15 @@ proc semSym(c: PContext, n: PNode, sym: PSym, flags: TExprFlags): PNode =
         result = newSymNode(s, n.info)
     else:
       result = newSymNode(s, n.info)
-  of skMacro, skTemplate:
+  of skMacro:
+    if efNoEvaluateGeneric in flags and s.ast[genericParamsPos].len > 0 or
+       (n.kind notin nkCallKinds and s.requiredParams > 0):
+      markUsed(c, n.info, s)
+      onUse(n.info, s)
+      result = symChoice(c, n, s, scClosed)
+    else:
+      result = semMacroExpr(c, n, n, s, flags)
+  of skTemplate:
     if efNoEvaluateGeneric in flags and s.ast[genericParamsPos].len > 0 or
        (n.kind notin nkCallKinds and s.requiredParams > 0) or
        sfCustomPragma in sym.flags:
@@ -1301,10 +1283,7 @@ proc semSym(c: PContext, n: PNode, sym: PSym, flags: TExprFlags): PNode =
       onUse(info, s)
       result = symChoice(c, n, s, scClosed)
     else:
-      case s.kind
-      of skMacro: result = semMacroExpr(c, n, n, s, flags)
-      of skTemplate: result = semTemplateExpr(c, n, s, flags)
-      else: discard # unreachable
+      result = semTemplateExpr(c, n, s, flags)
   of skParam:
     markUsed(c, n.info, s)
     onUse(n.info, s)
@@ -1353,12 +1332,6 @@ proc semSym(c: PContext, n: PNode, sym: PSym, flags: TExprFlags): PNode =
     markUsed(c, n.info, s)
     onUse(n.info, s)
     result = newSymNode(s, n.info)
-  of skModule:
-    # make sure type is None and not nil for discard checking
-    if efWantStmt in flags: s.typ = newTypeS(tyNone, c)
-    markUsed(c, n.info, s)
-    onUse(n.info, s)
-    result = newSymNode(s, n.info)
   else:
     let info = getCallLineInfo(n)
     #if efInCall notin flags:
@@ -1831,7 +1804,7 @@ proc semAsgn(c: PContext, n: PNode; mode=asgnNormal): PNode =
     result.add(n[1])
     return semExprNoType(c, result)
   of nkPar, nkTupleConstr:
-    if a.len >= 2 or a.kind == nkTupleConstr:
+    if a.len >= 2:
       # unfortunately we need to rewrite ``(x, y) = foo()`` already here so
       # that overloading of the assignment operator still works. Usually we
       # prefer to do these rewritings in transf.nim:
@@ -1844,16 +1817,11 @@ proc semAsgn(c: PContext, n: PNode; mode=asgnNormal): PNode =
   # a = b # both are vars, means: a[] = b[]
   # a = b # b no 'var T' means: a = addr(b)
   var le = a.typ
-  let assignable = isAssignable(c, a)
-  let root = getRoot(a)
-  let useStrictDefLet = root != nil and root.kind == skLet and
-                       assignable == arAddressableConst and
-                       strictDefs in c.features and isLocalSym(root)
   if le == nil:
     localError(c.config, a.info, "expression has no type")
   elif (skipTypes(le, {tyGenericInst, tyAlias, tySink}).kind notin {tyVar} and
-        assignable in {arNone, arLentValue, arAddressableConst} and not useStrictDefLet
-        ) or (skipTypes(le, abstractVar).kind in {tyOpenArray, tyVarargs} and views notin c.features):
+        isAssignable(c, a) in {arNone, arLentValue, arAddressableConst}) or (
+      skipTypes(le, abstractVar).kind in {tyOpenArray, tyVarargs} and views notin c.features):
     # Direct assignment to a discriminant is allowed!
     localError(c.config, a.info, errXCannotBeAssignedTo %
                renderTree(a, {renderNoComments}))
@@ -1866,8 +1834,6 @@ proc semAsgn(c: PContext, n: PNode; mode=asgnNormal): PNode =
         var rhsTyp = rhs.typ
         if rhsTyp.kind in tyUserTypeClasses and rhsTyp.isResolvedUserTypeClass:
           rhsTyp = rhsTyp.lastSon
-        if lhs.sym.typ.kind == tyAnything:
-          rhsTyp = rhsTyp.skipIntLit(c.idgen)
         if cmpTypes(c, lhs.typ, rhsTyp) in {isGeneric, isEqual}:
           internalAssert c.config, c.p.resultSym != nil
           # Make sure the type is valid for the result variable
@@ -1911,9 +1877,6 @@ proc semReturn(c: PContext, n: PNode): PNode =
     localError(c.config, n.info, "'return' not allowed here")
 
 proc semProcBody(c: PContext, n: PNode; expectedType: PType = nil): PNode =
-  when defined(nimsuggest):
-    if c.graph.config.expandDone():
-      return n
   openScope(c)
   result = semExpr(c, n, expectedType = expectedType)
   if c.p.resultSym != nil and not isEmptyType(result.typ):
@@ -1944,8 +1907,8 @@ proc semProcBody(c: PContext, n: PNode; expectedType: PType = nil): PNode =
     else:
       localError(c.config, c.p.resultSym.info, errCannotInferReturnType %
         c.p.owner.name.s)
-  if isIterator(c.p.owner.typ) and c.p.owner.typ[0] != nil and
-      c.p.owner.typ[0].kind == tyAnything:
+  if isInlineIterator(c.p.owner.typ) and c.p.owner.typ[0] != nil and
+      c.p.owner.typ[0].kind == tyUntyped:
     localError(c.config, c.p.owner.info, errCannotInferReturnType %
       c.p.owner.name.s)
   closeScope(c)
@@ -1971,8 +1934,6 @@ proc semYieldVarResult(c: PContext, n: PNode, restype: PType) =
             tupleConstr[i] = takeImplicitAddr(c, tupleConstr[i], e.kind == tyLent)
         else:
           localError(c.config, n[0].info, errXExpected, "tuple constructor")
-      elif e.kind == tyEmpty:
-        localError(c.config, n[0].info, errTypeExpected)
   else:
     when false:
       # XXX investigate what we really need here.
@@ -2216,13 +2177,10 @@ proc semQuoteAst(c: PContext, n: PNode): PNode =
   if ids.len > 0:
     dummyTemplate[paramsPos] = newNodeI(nkFormalParams, n.info)
     dummyTemplate[paramsPos].add getSysSym(c.graph, n.info, "untyped").newSymNode # return type
-    dummyTemplate[paramsPos].add newTreeI(nkIdentDefs, n.info, ids[0], getSysSym(c.graph, n.info, "typed").newSymNode, c.graph.emptyNode)
-    for i in 1..<ids.len:
-      let typ = semExprWithType(c, quotes[i+1], {}).typ
-      if tfTriggersCompileTime notin typ.flags and typ.kind != tyTypeDesc:
-        dummyTemplate[paramsPos].add newTreeI(nkIdentDefs, n.info, ids[i], newNodeIT(nkType, n.info, typ), c.graph.emptyNode)
-      else:
-        dummyTemplate[paramsPos].add newTreeI(nkIdentDefs, n.info, ids[i], getSysSym(c.graph, n.info, "typed").newSymNode, c.graph.emptyNode)
+    ids.add getSysSym(c.graph, n.info, "untyped").newSymNode # params type
+    ids.add c.graph.emptyNode # no default value
+    dummyTemplate[paramsPos].add newTreeI(nkIdentDefs, n.info, ids)
+
   var tmpl = semTemplateDef(c, dummyTemplate)
   quotes[0] = tmpl[namePos]
   # This adds a call to newIdentNode("result") as the first argument to the template call
@@ -2922,6 +2880,7 @@ proc semExpr(c: PContext, n: PNode, flags: TExprFlags = {}, expectedType: PType
     defer:
       if isCompilerDebug():
         echo ("<", c.config$n.info, n, ?.result.typ)
+
   template directLiteral(typeKind: TTypeKind) =
     if result.typ == nil:
       if expectedType != nil and (
@@ -2933,19 +2892,6 @@ proc semExpr(c: PContext, n: PNode, flags: TExprFlags = {}, expectedType: PType
         result.typ = getSysType(c.graph, n.info, typeKind)
 
   result = n
-  when defined(nimsuggest):
-    var expandStarted = false
-    if c.config.ideCmd == ideExpand and not c.config.expandProgress and
-        ((n.kind in {nkFuncDef, nkProcDef, nkIteratorDef, nkTemplateDef, nkMethodDef, nkConverterDef} and
-          n.info.exactEquals(c.config.expandPosition)) or
-         (n.kind in {nkCall, nkCommand} and
-          n[0].info.exactEquals(c.config.expandPosition))):
-      expandStarted = true
-      c.config.expandProgress = true
-      if c.config.expandLevels == 0:
-        c.config.expandNodeResult = $n
-        suggestQuit()
-
   if c.config.cmd == cmdIdeTools: suggestExpr(c, n)
   if nfSem in n.flags: return
   case n.kind
@@ -3273,8 +3219,3 @@ proc semExpr(c: PContext, n: PNode, flags: TExprFlags = {}, expectedType: PType
     localError(c.config, n.info, "invalid expression: " &
                renderTree(n, {renderNoComments}))
   if result != nil: incl(result.flags, nfSem)
-
-  when defined(nimsuggest):
-    if expandStarted:
-      c.config.expandNodeResult = $result
-      suggestQuit()

compiler/semfold.nim

@@ -13,7 +13,7 @@
 import
   strutils, options, ast, trees, nimsets,
   platform, math, msgs, idents, renderer, types,
-  commands, magicsys, modulegraphs, strtabs, lineinfos, wordrecg
+  commands, magicsys, modulegraphs, strtabs, lineinfos
 
 from system/memory import nimCStrLen
 
@@ -371,11 +371,6 @@ proc rangeCheck(n: PNode, value: Int128; g: ModuleGraph) =
     localError(g.config, n.info, "cannot convert " & $value &
                                     " to " & typeToString(n.typ))
 
-proc floatRangeCheck(n: PNode, value: BiggestFloat; g: ModuleGraph) =
-  if value < firstFloat(n.typ) or value > lastFloat(n.typ):
-    localError(g.config, n.info, "cannot convert " & $value &
-                                    " to " & typeToString(n.typ))
-
 proc foldConv(n, a: PNode; idgen: IdGenerator; g: ModuleGraph; check = false): PNode =
   let dstTyp = skipTypes(n.typ, abstractRange - {tyTypeDesc})
   let srcTyp = skipTypes(a.typ, abstractRange - {tyTypeDesc})
@@ -495,83 +490,6 @@ proc newSymNodeTypeDesc*(s: PSym; idgen: IdGenerator; info: TLineInfo): PNode =
   else:
     result.typ = s.typ
 
-proc foldDefine(m, s: PSym, n: PNode; idgen: IdGenerator; g: ModuleGraph): PNode =
-  var name = s.name.s
-  let prag = extractPragma(s)
-  if prag != nil:
-    for it in prag:
-      if it.kind in nkPragmaCallKinds and it.len == 2 and it[0].kind == nkIdent:
-        let word = whichKeyword(it[0].ident)
-        if word in {wStrDefine, wIntDefine, wBoolDefine, wDefine}:
-          # should be processed in pragmas.nim already
-          if it[1].kind in {nkStrLit, nkRStrLit, nkTripleStrLit}:
-            name = it[1].strVal
-  if isDefined(g.config, name):
-    let str = g.config.symbols[name]
-    case s.magic
-    of mIntDefine:
-      try:
-        result = newIntNodeT(toInt128(str.parseInt), n, idgen, g)
-      except ValueError:
-        localError(g.config, s.info,
-          "{.intdefine.} const was set to an invalid integer: '" &
-            str & "'")
-    of mStrDefine:
-      result = newStrNodeT(str, n, g)
-    of mBoolDefine:
-      try:
-        result = newIntNodeT(toInt128(str.parseBool.int), n, idgen, g)
-      except ValueError:
-        localError(g.config, s.info,
-          "{.booldefine.} const was set to an invalid bool: '" &
-            str & "'")
-    of mGenericDefine:
-      let rawTyp = s.typ
-      # pretend we don't support distinct types
-      let typ = rawTyp.skipTypes(abstractVarRange-{tyDistinct})
-      try:
-        template intNode(value): PNode =
-          let val = toInt128(value)
-          rangeCheck(n, val, g)
-          newIntNodeT(val, n, idgen, g)
-        case typ.kind
-        of tyString, tyCstring:
-          result = newStrNodeT(str, n, g)
-        of tyInt..tyInt64:
-          result = intNode(str.parseBiggestInt)
-        of tyUInt..tyUInt64:
-          result = intNode(str.parseBiggestUInt)
-        of tyBool:
-          result = intNode(str.parseBool.int)
-        of tyEnum:
-          # compile time parseEnum
-          let ident = getIdent(g.cache, str)
-          for e in typ.n:
-            if e.kind != nkSym: internalError(g.config, "foldDefine for enum")
-            let es = e.sym
-            let match =
-              if es.ast.isNil:
-                es.name.id == ident.id
-              else:
-                es.ast.strVal == str
-            if match:
-              result = intNode(es.position)
-              break
-          if result.isNil:
-            raise newException(ValueError, "invalid enum value: " & str)
-        else:
-          localError(g.config, s.info, "unsupported type $1 for define '$2'" %
-            [name, typeToString(rawTyp)])
-      except ValueError as e:
-        localError(g.config, s.info,
-          "could not process define '$1' of type $2; $3" %
-            [name, typeToString(rawTyp), e.msg])
-    else: result = copyTree(s.astdef) # unreachable
-  else:
-    result = copyTree(s.astdef)
-    if result != nil:
-      result.info = n.info
-
 proc getConstExpr(m: PSym, n: PNode; idgen: IdGenerator; g: ModuleGraph): PNode =
   result = nil
   case n.kind
@@ -591,12 +509,33 @@ proc getConstExpr(m: PSym, n: PNode; idgen: IdGenerator; g: ModuleGraph): PNode
       of mBuildOS: result = newStrNodeT(toLowerAscii(platform.OS[g.config.target.hostOS].name), n, g)
       of mBuildCPU: result = newStrNodeT(platform.CPU[g.config.target.hostCPU].name.toLowerAscii, n, g)
       of mAppType: result = getAppType(n, g)
-      of mIntDefine, mStrDefine, mBoolDefine, mGenericDefine:
-        result = foldDefine(m, s, n, idgen, g)
+      of mIntDefine:
+        if isDefined(g.config, s.name.s):
+          try:
+            result = newIntNodeT(toInt128(g.config.symbols[s.name.s].parseInt), n, idgen, g)
+          except ValueError:
+            localError(g.config, s.info,
+              "{.intdefine.} const was set to an invalid integer: '" &
+                g.config.symbols[s.name.s] & "'")
+        else:
+          result = copyTree(s.astdef)
+      of mStrDefine:
+        if isDefined(g.config, s.name.s):
+          result = newStrNodeT(g.config.symbols[s.name.s], n, g)
+        else:
+          result = copyTree(s.astdef)
+      of mBoolDefine:
+        if isDefined(g.config, s.name.s):
+          try:
+            result = newIntNodeT(toInt128(g.config.symbols[s.name.s].parseBool.int), n, idgen, g)
+          except ValueError:
+            localError(g.config, s.info,
+              "{.booldefine.} const was set to an invalid bool: '" &
+                g.config.symbols[s.name.s] & "'")
+        else:
+          result = copyTree(s.astdef)
       else:
         result = copyTree(s.astdef)
-        if result != nil:
-          result.info = n.info
     of skProc, skFunc, skMethod:
       result = n
     of skParam:
@@ -685,7 +624,7 @@ proc getConstExpr(m: PSym, n: PNode; idgen: IdGenerator; g: ModuleGraph): PNode
       n[0] = a
   of nkBracket, nkCurly:
     result = copyNode(n)
-    for son in n.items:
+    for i, son in n.pairs:
       var a = getConstExpr(m, son, idgen, g)
       if a == nil: return nil
       result.add a
@@ -709,7 +648,7 @@ proc getConstExpr(m: PSym, n: PNode; idgen: IdGenerator; g: ModuleGraph): PNode
     # tuple constructor
     result = copyNode(n)
     if (n.len > 0) and (n[0].kind == nkExprColonExpr):
-      for expr in n.items:
+      for i, expr in n.pairs:
         let exprNew = copyNode(expr) # nkExprColonExpr
         exprNew.add expr[0]
         let a = getConstExpr(m, expr[1], idgen, g)
@@ -717,7 +656,7 @@ proc getConstExpr(m: PSym, n: PNode; idgen: IdGenerator; g: ModuleGraph): PNode
         exprNew.add a
         result.add exprNew
     else:
-      for expr in n.items:
+      for i, expr in n.pairs:
         let a = getConstExpr(m, expr, idgen, g)
         if a == nil: return nil
         result.add a

compiler/semgnrc.nim

@@ -78,18 +78,14 @@ proc semGenericStmtSymbol(c: PContext, n: PNode, s: PSym,
     if macroToExpandSym(s):
       onUse(n.info, s)
       result = semTemplateExpr(c, n, s, {efNoSemCheck})
-      c.friendModules.add(s.owner.getModule)
       result = semGenericStmt(c, result, {}, ctx)
-      discard c.friendModules.pop()
     else:
       result = symChoice(c, n, s, scOpen)
   of skMacro:
     if macroToExpandSym(s):
       onUse(n.info, s)
       result = semMacroExpr(c, n, n, s, {efNoSemCheck})
-      c.friendModules.add(s.owner.getModule)
       result = semGenericStmt(c, result, {}, ctx)
-      discard c.friendModules.pop()
     else:
       result = symChoice(c, n, s, scOpen)
   of skGenericParam:
@@ -249,9 +245,7 @@ proc semGenericStmt(c: PContext, n: PNode,
         if macroToExpand(s) and sc.safeLen <= 1:
           onUse(fn.info, s)
           result = semMacroExpr(c, n, n, s, {efNoSemCheck})
-          c.friendModules.add(s.owner.getModule)
           result = semGenericStmt(c, result, flags, ctx)
-          discard c.friendModules.pop()
         else:
           n[0] = sc
           result = n
@@ -260,9 +254,7 @@ proc semGenericStmt(c: PContext, n: PNode,
         if macroToExpand(s) and sc.safeLen <= 1:
           onUse(fn.info, s)
           result = semTemplateExpr(c, n, s, {efNoSemCheck})
-          c.friendModules.add(s.owner.getModule)
           result = semGenericStmt(c, result, flags, ctx)
-          discard c.friendModules.pop()
         else:
           n[0] = sc
           result = n
@@ -501,20 +493,6 @@ proc semGenericStmt(c: PContext, n: PNode,
   of nkExprColonExpr, nkExprEqExpr:
     checkMinSonsLen(n, 2, c.config)
     result[1] = semGenericStmt(c, n[1], flags, ctx)
-  of nkObjConstr:
-    for i in 0..<n.len:
-      result[i] = semGenericStmt(c, n[i], flags, ctx)
-    if result[0].kind == nkSym:
-      let fmoduleId = getModule(result[0].sym).id
-      var isVisable = false
-      for module in c.friendModules:
-        if module.id == fmoduleId:
-          isVisable = true
-          break
-      if isVisable:
-        for i in 1..<result.len:
-          if result[i].kind == nkExprColonExpr:
-            result[i][1].flags.incl nfSkipFieldChecking
   else:
     for i in 0..<n.len:
       result[i] = semGenericStmt(c, n[i], flags, ctx)

compiler/seminst.nim

@@ -40,7 +40,7 @@ const
 
 iterator instantiateGenericParamList(c: PContext, n: PNode, pt: TIdTable): PSym =
   internalAssert c.config, n.kind == nkGenericParams
-  for a in n.items:
+  for i, a in n.pairs:
     internalAssert c.config, a.kind == nkSym
     var q = a.sym
     if q.typ.kind in {tyTypeDesc, tyGenericParam, tyStatic, tyConcept}+tyTypeClasses:
@@ -127,18 +127,11 @@ proc instantiateBody(c: PContext, n, params: PNode, result, orig: PSym) =
         if sfGenSym in param.flags:
           idTablePut(symMap, params[i].sym, result.typ.n[param.position+1].sym)
     freshGenSyms(c, b, result, orig, symMap)
-
+    
     if sfBorrow notin orig.flags: 
       # We do not want to generate a body for generic borrowed procs.
       # As body is a sym to the borrowed proc.
-      let resultType = # todo probably refactor it into a function
-        if result.kind == skMacro:
-          sysTypeFromName(c.graph, n.info, "NimNode")
-        elif not isInlineIterator(result.typ):
-          result.typ[0]
-        else:
-          nil
-      b = semProcBody(c, b, resultType)
+      b = semProcBody(c, b)
     result.ast[bodyPos] = hloBody(c, b)
     excl(result.flags, sfForward)
     trackProc(c, result, result.ast[bodyPos])
@@ -321,7 +314,7 @@ proc fillMixinScope(c: PContext) =
     p = p.next
 
 proc generateInstance(c: PContext, fn: PSym, pt: TIdTable,
-                      info: TLineInfo): PSym =
+                      info: TLineInfo): PSym {.nosinks.} =
   ## Generates a new instance of a generic procedure.
   ## The `pt` parameter is a type-unsafe mapping table used to link generic
   ## parameters to their concrete types within the generic instance.

compiler/semmagic.nim

@@ -20,8 +20,9 @@ proc addDefaultFieldForNew(c: PContext, n: PNode): PNode =
     var asgnExpr = newTree(nkObjConstr, newNodeIT(nkType, result[1].info, typ))
     asgnExpr.typ = typ
     var t = typ.skipTypes({tyGenericInst, tyAlias, tySink})[0]
+    var id = initIntSet()
     while true:
-      asgnExpr.sons.add defaultFieldsForTheUninitialized(c, t.n)
+      asgnExpr.sons.add defaultFieldsForTheUninitialized(c, t.n, id)
       let base = t[0]
       if base == nil:
         break
@@ -526,7 +527,7 @@ proc checkDefault(c: PContext, n: PNode): PNode =
     message(c.config, n.info, warnUnsafeDefault, typeToString(constructed))
 
 proc magicsAfterOverloadResolution(c: PContext, n: PNode,
-                                   flags: TExprFlags; expectedType: PType = nil): PNode =
+                                   flags: TExprFlags): PNode =
   ## This is the preferred code point to implement magics.
   ## ``c`` the current module, a symbol table to a very good approximation
   ## ``n`` the ast like it would be passed to a real macro
@@ -635,9 +636,5 @@ proc magicsAfterOverloadResolution(c: PContext, n: PNode,
     result = n
   of mPrivateAccess:
     result = semPrivateAccess(c, n)
-  of mArrToSeq:
-    result = n
-    if result.typ != nil and expectedType != nil and result.typ.kind == tySequence and expectedType.kind == tySequence and result.typ[0].kind == tyEmpty:
-      result.typ = expectedType # type inference for empty sequence # bug #21377
   else:
     result = n

compiler/semobjconstr.nim

@@ -76,7 +76,7 @@ proc semConstrField(c: PContext, flags: TExprFlags,
   let assignment = locateFieldInInitExpr(c, field, initExpr)
   if assignment != nil:
     if nfSem in assignment.flags: return assignment[1]
-    if nfSkipFieldChecking in assignment[1].flags:
+    if nfUseDefaultField in assignment[1].flags:
       discard
     elif not fieldVisible(c, field):
       localError(c.config, initExpr.info,
@@ -86,7 +86,6 @@ proc semConstrField(c: PContext, flags: TExprFlags,
     var initValue = semExprFlagDispatched(c, assignment[1], flags, field.typ)
     if initValue != nil:
       initValue = fitNodeConsiderViewType(c, field.typ, initValue, assignment.info)
-    initValue.flags.incl nfSkipFieldChecking
     assignment[0] = newSymNode(field)
     assignment[1] = initValue
     assignment.flags.incl nfSem
@@ -143,45 +142,15 @@ proc fieldsPresentInInitExpr(c: PContext, fieldsRecList, initExpr: PNode): strin
       if result.len != 0: result.add ", "
       result.add field.sym.name.s.quoteStr
 
-proc locateFieldInDefaults(sym: PSym, defaults: seq[PNode]): bool =
-  result = false
-  for d in defaults:
-    if sym.id == d[0].sym.id:
-      return true
-
 proc collectMissingFields(c: PContext, fieldsRecList: PNode,
-                          constrCtx: var ObjConstrContext, defaults: seq[PNode]
-                          ): seq[PSym] =
-    for r in directFieldsInRecList(fieldsRecList):
+                          constrCtx: var ObjConstrContext) =
+  for r in directFieldsInRecList(fieldsRecList):
+    if constrCtx.needsFullInit or
+       sfRequiresInit in r.sym.flags or
+       r.sym.typ.requiresInit:
       let assignment = locateFieldInInitExpr(c, r.sym, constrCtx.initExpr)
-      if assignment == nil and not locateFieldInDefaults(r.sym, defaults):
-        if constrCtx.needsFullInit or
-          sfRequiresInit in r.sym.flags or
-            r.sym.typ.requiresInit:
-          constrCtx.missingFields.add r.sym
-        else:
-          result.add r.sym
-
-proc collectMissingCaseFields(c: PContext, branchNode: PNode,
-                          constrCtx: var ObjConstrContext, defaults: seq[PNode]): seq[PSym] =
-  if branchNode != nil:
-    let fieldsRecList = branchNode[^1]
-    result = collectMissingFields(c, fieldsRecList, constrCtx, defaults)
-
-proc collectOrAddMissingCaseFields(c: PContext, branchNode: PNode,
-                          constrCtx: var ObjConstrContext, defaults: var seq[PNode]) =
-  let res = collectMissingCaseFields(c, branchNode, constrCtx, defaults)
-  for sym in res:
-    let asgnType = newType(tyTypeDesc, nextTypeId(c.idgen), sym.typ.owner)
-    let recTyp = sym.typ.skipTypes(defaultFieldsSkipTypes)
-    rawAddSon(asgnType, recTyp)
-    let asgnExpr = newTree(nkCall,
-          newSymNode(getSysMagic(c.graph, constrCtx.initExpr.info, "zeroDefault", mZeroDefault)),
-          newNodeIT(nkType, constrCtx.initExpr.info, asgnType)
-        )
-    asgnExpr.flags.incl nfSkipFieldChecking
-    asgnExpr.typ = recTyp
-    defaults.add newTree(nkExprColonExpr, newSymNode(sym), asgnExpr)
+      if assignment == nil:
+        constrCtx.missingFields.add r.sym
 
 proc semConstructFields(c: PContext, n: PNode, constrCtx: var ObjConstrContext,
                         flags: TExprFlags): tuple[status: InitStatus, defaults: seq[PNode]] =
@@ -197,6 +166,11 @@ proc semConstructFields(c: PContext, n: PNode, constrCtx: var ObjConstrContext,
       let fields = branch[^1]
       fieldsPresentInInitExpr(c, fields, constrCtx.initExpr)
 
+    template collectMissingFields(branchNode: PNode) =
+      if branchNode != nil:
+        let fields = branchNode[^1]
+        collectMissingFields(c, fields, constrCtx)
+
     let discriminator = n[0]
     internalAssert c.config, discriminator.kind == nkSym
     var selectedBranch = -1
@@ -314,7 +288,8 @@ proc semConstructFields(c: PContext, n: PNode, constrCtx: var ObjConstrContext,
       # When a branch is selected with a partial match, some of the fields
       # that were not initialized may be mandatory. We must check for this:
       if result.status == initPartial:
-        collectOrAddMissingCaseFields(c, branchNode, constrCtx, result.defaults)
+        collectMissingFields branchNode
+
     else:
       result.status = initNone
       let discriminatorVal = semConstrField(c, flags + {efPreferStatic},
@@ -327,7 +302,7 @@ proc semConstructFields(c: PContext, n: PNode, constrCtx: var ObjConstrContext,
         # a result:
         let defaultValue = newIntLit(c.graph, constrCtx.initExpr.info, 0)
         let matchedBranch = n.pickCaseBranch defaultValue
-        discard collectMissingCaseFields(c, matchedBranch, constrCtx, @[])
+        collectMissingFields matchedBranch
       else:
         result.status = initPartial
         if discriminatorVal.kind == nkIntLit:
@@ -337,12 +312,11 @@ proc semConstructFields(c: PContext, n: PNode, constrCtx: var ObjConstrContext,
           if matchedBranch != nil:
             let (_, defaults) = semConstructFields(c, matchedBranch[^1], constrCtx, flags)
             result.defaults.add defaults
-            collectOrAddMissingCaseFields(c, matchedBranch, constrCtx, result.defaults)
+            collectMissingFields matchedBranch
         else:
           # All bets are off. If any of the branches has a mandatory
           # fields we must produce an error:
-          for i in 1..<n.len:
-            discard collectMissingCaseFields(c, n[i], constrCtx, @[])
+          for i in 1..<n.len: collectMissingFields n[i]
   of nkSym:
     let field = n.sym
     let e = semConstrField(c, flags, field, constrCtx.initExpr)
@@ -352,13 +326,10 @@ proc semConstructFields(c: PContext, n: PNode, constrCtx: var ObjConstrContext,
       result.status = initUnknown
       result.defaults.add newTree(nkExprColonExpr, n, field.ast)
     else:
-      if efWantNoDefaults notin flags: # cannot compute defaults at the typeRightPass
-        let defaultExpr = defaultNodeField(c, n)
-        if defaultExpr != nil:
-          result.status = initUnknown
-          result.defaults.add newTree(nkExprColonExpr, n, defaultExpr)
-        else:
-          result.status = initNone
+      let defaultExpr = defaultNodeField(c, n)
+      if defaultExpr != nil:
+        result.status = initUnknown
+        result.defaults.add newTree(nkExprColonExpr, n, defaultExpr)
       else:
         result.status = initNone
   else:
@@ -374,7 +345,7 @@ proc semConstructTypeAux(c: PContext,
     result.status.mergeInitStatus status
     result.defaults.add defaults
     if status in {initPartial, initNone, initUnknown}:
-      discard collectMissingFields(c, t.n, constrCtx, result.defaults)
+      collectMissingFields c, t.n, constrCtx
     let base = t[0]
     if base == nil: break
     t = skipTypes(base, skipPtrs)
@@ -393,7 +364,7 @@ proc initConstrContext(t: PType, initExpr: PNode): ObjConstrContext =
 proc computeRequiresInit(c: PContext, t: PType): bool =
   assert t.kind == tyObject
   var constrCtx = initConstrContext(t, newNode(nkObjConstr))
-  let initResult = semConstructTypeAux(c, constrCtx, {efWantNoDefaults})
+  let initResult = semConstructTypeAux(c, constrCtx, {})
   constrCtx.missingFields.len > 0
 
 proc defaultConstructionError(c: PContext, t: PType, info: TLineInfo) =
@@ -403,7 +374,7 @@ proc defaultConstructionError(c: PContext, t: PType, info: TLineInfo) =
     assert objType != nil
   if objType.kind == tyObject:
     var constrCtx = initConstrContext(objType, newNodeI(nkObjConstr, info))
-    let initResult = semConstructTypeAux(c, constrCtx, {efWantNoDefaults})
+    let initResult = semConstructTypeAux(c, constrCtx, {})
     if constrCtx.missingFields.len > 0:
       localError(c.config, info,
         "The $1 type doesn't have a default value. The following fields must be initialized: $2." % [typeToString(t), listSymbolNames(constrCtx.missingFields)])

compiler/sempass2.nim

@@ -10,8 +10,7 @@
 import
   intsets, ast, astalgo, msgs, renderer, magicsys, types, idents, trees,
   wordrecg, strutils, options, guards, lineinfos, semfold, semdata,
-  modulegraphs, varpartitions, typeallowed, nilcheck, errorhandling, tables,
-  semstrictfuncs
+  modulegraphs, varpartitions, typeallowed, nilcheck, errorhandling, tables
 
 when defined(nimPreviewSlimSystem):
   import std/assertions
@@ -76,7 +75,7 @@ type
     guards: TModel # nested guards
     locked: seq[PNode] # locked locations
     gcUnsafe, isRecursive, isTopLevel, hasSideEffect, inEnforcedGcSafe: bool
-    isInnerProc: bool
+    hasDangerousAssign, isInnerProc: bool
     inEnforcedNoSideEffects: bool
     currOptions: TOptions
     config: ConfigRef
@@ -85,10 +84,6 @@ type
     escapingParams: IntSet
   PEffects = var TEffects
 
-const
-  errXCannotBeAssignedTo = "'$1' cannot be assigned to"
-  errLetNeedsInit = "'let' symbol requires an initialization"
-
 proc createTypeBoundOps(tracked: PEffects, typ: PType; info: TLineInfo) =
   if typ == nil: return
   when false:
@@ -102,8 +97,8 @@ proc createTypeBoundOps(tracked: PEffects, typ: PType; info: TLineInfo) =
       optSeqDestructors in tracked.config.globalOptions:
     tracked.owner.flags.incl sfInjectDestructors
 
-proc isLocalSym(a: PEffects, s: PSym): bool =
-  s.typ != nil and (s.kind in {skLet, skVar, skResult} or (s.kind == skParam and isOutParam(s.typ))) and
+proc isLocalVar(a: PEffects, s: PSym): bool =
+  s.typ != nil and (s.kind in {skVar, skResult} or (s.kind == skParam and isOutParam(s.typ))) and
     sfGlobal notin s.flags and s.owner == a.owner
 
 proc lockLocations(a: PEffects; pragma: PNode) =
@@ -178,15 +173,10 @@ proc initVar(a: PEffects, n: PNode; volatileCheck: bool) =
   let n = skipHiddenDeref(n)
   if n.kind != nkSym: return
   let s = n.sym
-  if isLocalSym(a, s):
+  if isLocalVar(a, s):
     if volatileCheck: makeVolatile(a, s)
     for x in a.init:
-      if x == s.id:
-        if strictDefs in a.c.features and s.kind == skLet:
-          localError(a.config, n.info, errXCannotBeAssignedTo %
-                    renderTree(n, {renderNoComments}
-                ))
-        return
+      if x == s.id: return
     a.init.add s.id
     if a.scopes.getOrDefault(s.id) == a.currentBlock:
       #[ Consider this case:
@@ -214,7 +204,7 @@ proc initVarViaNew(a: PEffects, n: PNode) =
     # 'x' is not nil, but that doesn't mean its "not nil" children
     # are initialized:
     initVar(a, n, volatileCheck=true)
-  elif isLocalSym(a, s):
+  elif isLocalVar(a, s):
     makeVolatile(a, s)
 
 proc warnAboutGcUnsafe(n: PNode; conf: ConfigRef) =
@@ -332,7 +322,7 @@ proc useVar(a: PEffects, n: PNode) =
   let s = n.sym
   if a.inExceptOrFinallyStmt > 0:
     incl s.flags, sfUsedInFinallyOrExcept
-  if isLocalSym(a, s):
+  if isLocalVar(a, s):
     if sfNoInit in s.flags:
       # If the variable is explicitly marked as .noinit. do not emit any error
       a.init.add s.id
@@ -341,10 +331,7 @@ proc useVar(a: PEffects, n: PNode) =
         message(a.config, n.info, warnProveInit, s.name.s)
       elif a.leftPartOfAsgn <= 0:
         if strictDefs in a.c.features:
-          if s.kind == skLet:
-            localError(a.config, n.info, errLetNeedsInit)
-          else:
-            message(a.config, n.info, warnUninit, s.name.s)
+          message(a.config, n.info, warnUninit, s.name.s)
       # prevent superfluous warnings about the same variable:
       a.init.add s.id
   useVarNoInitCheck(a, n, s)
@@ -396,7 +383,7 @@ proc addRaiseEffect(a: PEffects, e, comesFrom: PNode) =
     if sameType(a.graph.excType(aa[i]), a.graph.excType(e)): return
 
   if e.typ != nil:
-    if not isDefectException(e.typ):
+    if optNimV1Emulation in a.config.globalOptions or not isDefectException(e.typ):
       throws(a.exc, e, comesFrom)
 
 proc addTag(a: PEffects, e, comesFrom: PNode) =
@@ -491,10 +478,6 @@ proc trackTryStmt(tracked: PEffects, n: PNode) =
     let b = n[i]
     if b.kind == nkExceptBranch:
       setLen(tracked.init, oldState)
-      for j in 0..<b.len - 1:
-        if b[j].isInfixAs(): # skips initialization checks
-          assert(b[j][2].kind == nkSym)
-          tracked.init.add b[j][2].sym.id
       track(tracked, b[^1])
       for i in oldState..<tracked.init.len:
         addToIntersection(inter, tracked.init[i])
@@ -654,7 +637,7 @@ proc trackOperandForIndirectCall(tracked: PEffects, n: PNode, formals: PType; ar
   let paramType = if formals != nil and argIndex < formals.len: formals[argIndex] else: nil
   if paramType != nil and paramType.kind in {tyVar}:
     invalidateFacts(tracked.guards, n)
-    if n.kind == nkSym and isLocalSym(tracked, n.sym):
+    if n.kind == nkSym and isLocalVar(tracked, n.sym):
       makeVolatile(tracked, n.sym)
   if paramType != nil and paramType.kind == tyProc and tfGcSafe in paramType.flags:
     let argtype = skipTypes(a.typ, abstractInst)
@@ -819,7 +802,7 @@ proc passedToEffectsDelayedParam(tracked: PEffects; n: PNode) =
 ]#
 
 proc checkForSink(tracked: PEffects; n: PNode) =
-  if tracked.inIfStmt == 0 and optSinkInference in tracked.config.options:
+  if tracked.inIfStmt == 0:
     checkForSink(tracked.config, tracked.c.idgen, tracked.owner, n)
 
 proc trackCall(tracked: PEffects; n: PNode) =
@@ -926,17 +909,12 @@ proc trackCall(tracked: PEffects; n: PNode) =
         createTypeBoundOps(tracked, paramType[0], n.info)
         checkForSink(tracked, n[i])
       of tyVar:
+        tracked.hasDangerousAssign = true
         if isOutParam(paramType):
           # consider this case: p(out x, x); we want to remark that 'x' is not
           # initialized until after the call. Since we do this after we analysed the
           # call, this is fine.
           initVar(tracked, n[i].skipAddr, false)
-        if strictFuncs in tracked.c.features and not tracked.inEnforcedNoSideEffects and
-           isDangerousLocation(n[i].skipAddr, tracked.owner):
-          if sfNoSideEffect in tracked.owner.flags:
-            localError(tracked.config, n[i].info,
-              "cannot pass $1 to `var T` parameter within a strict func" % renderTree(n[i]))
-          tracked.hasSideEffect = true
       else: discard
 
 type
@@ -1047,7 +1025,7 @@ proc track(tracked: PEffects, n: PNode) =
       # bug #15038: ensure consistency
       if not hasDestructor(n.typ) and sameType(n.typ, n.sym.typ): n.typ = n.sym.typ
   of nkHiddenAddr, nkAddr:
-    if n[0].kind == nkSym and isLocalSym(tracked, n[0].sym):
+    if n[0].kind == nkSym and isLocalVar(tracked, n[0].sym):
       useVarNoInitCheck(tracked, n[0], n[0].sym)
     else:
       track(tracked, n[0])
@@ -1087,14 +1065,10 @@ proc track(tracked: PEffects, n: PNode) =
     when false: cstringCheck(tracked, n)
     if tracked.owner.kind != skMacro and n[0].typ.kind notin {tyOpenArray, tyVarargs}:
       createTypeBoundOps(tracked, n[0].typ, n.info)
-    if n[0].kind != nkSym or not isLocalSym(tracked, n[0].sym):
+    if n[0].kind != nkSym or not isLocalVar(tracked, n[0].sym):
       checkForSink(tracked, n[1])
-      if strictFuncs in tracked.c.features and not tracked.inEnforcedNoSideEffects and
-         isDangerousLocation(n[0], tracked.owner):
-        tracked.hasSideEffect = true
-        if sfNoSideEffect in tracked.owner.flags:
-          localError(tracked.config, n[0].info,
-              "cannot mutate location $1 within a strict func" % renderTree(n[0]))
+      if not tracked.hasDangerousAssign and n[0].kind != nkSym:
+        tracked.hasDangerousAssign = true
   of nkVarSection, nkLetSection:
     for child in n:
       let last = lastSon(child)
@@ -1451,9 +1425,6 @@ proc hasRealBody(s: PSym): bool =
 
 proc trackProc*(c: PContext; s: PSym, body: PNode) =
   let g = c.graph
-  when defined(nimsuggest):
-    if g.config.expandDone():
-      return
   var effects = s.typ.n[0]
   if effects.kind != nkEffectList: return
   # effects already computed?
@@ -1492,10 +1463,7 @@ proc trackProc*(c: PContext; s: PSym, body: PNode) =
      s.kind in {skProc, skFunc, skConverter, skMethod} and s.magic == mNone:
     var res = s.ast[resultPos].sym # get result symbol
     if res.id notin t.init:
-      if tfRequiresInit in s.typ[0].flags:
-        localError(g.config, body.info, "'$1' requires explicit initialization" % "result")
-      else:
-        message(g.config, body.info, warnProveInit, "result")
+      message(g.config, body.info, warnProveInit, "result")
   let p = s.ast[pragmasPos]
   let raisesSpec = effectSpec(p, wRaises)
   if not isNil(raisesSpec):
@@ -1533,9 +1501,17 @@ proc trackProc*(c: PContext; s: PSym, body: PNode) =
     effects[ensuresEffects] = ensuresSpec
 
   var mutationInfo = MutationInfo()
-  if views in c.features:
-    var partitions = computeGraphPartitions(s, body, g, {borrowChecking})
-    checkBorrowedLocations(partitions, body, g.config)
+  var hasMutationSideEffect = false
+  if {strictFuncs, views} * c.features != {}:
+    var goals: set[Goal] = {}
+    if strictFuncs in c.features: goals.incl constParameters
+    if views in c.features: goals.incl borrowChecking
+    var partitions = computeGraphPartitions(s, body, g, goals)
+    if not t.hasSideEffect and t.hasDangerousAssign:
+      t.hasSideEffect = varpartitions.hasSideEffect(partitions, mutationInfo)
+      hasMutationSideEffect = t.hasSideEffect
+    if views in c.features:
+      checkBorrowedLocations(partitions, body, g.config)
 
   if sfThread in s.flags and t.gcUnsafe:
     if optThreads in g.config.globalOptions and optThreadAnalysis in g.config.globalOptions:
@@ -1548,7 +1524,9 @@ proc trackProc*(c: PContext; s: PSym, body: PNode) =
     when false:
       listGcUnsafety(s, onlyWarning=false, g.config)
     else:
-      if c.compilesContextId == 0: # don't render extended diagnostic messages in `system.compiles` context
+      if hasMutationSideEffect:
+        localError(g.config, s.info, "'$1' can have side effects$2" % [s.name.s, g.config $ mutationInfo])
+      elif c.compilesContextId == 0: # don't render extended diagnostic messages in `system.compiles` context
         var msg = ""
         listSideEffects(msg, s, g.config, t.c)
         message(g.config, s.info, errGenerated, msg)

compiler/semstmts.nim

@@ -40,13 +40,6 @@ const
 
 proc implicitlyDiscardable(n: PNode): bool
 
-proc hasEmpty(typ: PType): bool =
-  if typ.kind in {tySequence, tyArray, tySet}:
-    result = typ.lastSon.kind == tyEmpty
-  elif typ.kind == tyTuple:
-    for s in typ.sons:
-      result = result or hasEmpty(s)
-
 proc semDiscard(c: PContext, n: PNode): PNode =
   result = n
   checkSonsLen(n, 1, c.config)
@@ -54,9 +47,7 @@ proc semDiscard(c: PContext, n: PNode): PNode =
     n[0] = semExprWithType(c, n[0])
     let sonType = n[0].typ
     let sonKind = n[0].kind
-    if isEmptyType(sonType) or hasEmpty(sonType) or
-          sonType.kind in {tyNone, tyTypeDesc} or
-          sonKind == nkTypeOfExpr:
+    if isEmptyType(sonType) or sonType.kind in {tyNone, tyTypeDesc} or sonKind == nkTypeOfExpr:
       localError(c.config, n.info, errInvalidDiscard)
     if sonType.kind == tyProc and sonKind notin nkCallKinds:
       # tyProc is disallowed to prevent ``discard foo`` to be valid, when ``discard foo()`` is meant.
@@ -114,6 +105,7 @@ proc semWhile(c: PContext, n: PNode; flags: TExprFlags): PNode =
     result.typ = n[1].typ
   elif implicitlyDiscardable(n[1]):
     result[1].typ = c.enforceVoidContext
+    result.typ = c.enforceVoidContext
 
 proc semProc(c: PContext, n: PNode): PNode
 
@@ -161,19 +153,18 @@ proc discardCheck(c: PContext, result: PNode, flags: TExprFlags) =
       if result.typ.kind == tyNone:
         localError(c.config, result.info, "expression has no type: " &
                renderTree(result, {renderNoComments}))
-      else:
-        var n = result
-        while n.kind in skipForDiscardable:
-          if n.kind == nkTryStmt: n = n[0]
-          else: n = n.lastSon
-        var s = "expression '" & $n & "' is of type '" &
-            result.typ.typeToString & "' and has to be used (or discarded)"
-        if result.info.line != n.info.line or
-            result.info.fileIndex != n.info.fileIndex:
-          s.add "; start of expression here: " & c.config$result.info
-        if result.typ.kind == tyProc:
-          s.add "; for a function call use ()"
-        localError(c.config, n.info, s)
+      var n = result
+      while n.kind in skipForDiscardable:
+        if n.kind == nkTryStmt: n = n[0]
+        else: n = n.lastSon
+      var s = "expression '" & $n & "' is of type '" &
+          result.typ.typeToString & "' and has to be used (or discarded)"
+      if result.info.line != n.info.line or
+          result.info.fileIndex != n.info.fileIndex:
+        s.add "; start of expression here: " & c.config$result.info
+      if result.typ.kind == tyProc:
+        s.add "; for a function call use ()"
+      localError(c.config, n.info, s)
 
 proc semIf(c: PContext, n: PNode; flags: TExprFlags; expectedType: PType = nil): PNode =
   result = n
@@ -219,8 +210,6 @@ proc semTry(c: PContext, n: PNode; flags: TExprFlags; expectedType: PType = nil)
       isImported = true
     elif not isException(typ):
       localError(c.config, typeNode.info, errExprCannotBeRaised)
-    elif not isDefectOrCatchableError(typ):
-      message(c.config, a.info, warnBareExcept, "catch a more precise Exception deriving from CatchableError or Defect.")
 
     if containsOrIncl(check, typ.id):
       localError(c.config, typeNode.info, errExceptionAlreadyHandled)
@@ -262,8 +251,7 @@ proc semTry(c: PContext, n: PNode; flags: TExprFlags; expectedType: PType = nil)
       elif a.len == 1:
         # count number of ``except: body`` blocks
         inc catchAllExcepts
-        message(c.config, a.info, warnBareExcept,
-                  "The bare except clause is deprecated; use `except CatchableError:` instead")
+
       else:
         # support ``except KeyError, ValueError, ... : body``
         if catchAllExcepts > 0:
@@ -420,6 +408,13 @@ proc semUsing(c: PContext; n: PNode): PNode =
     if a[^1].kind != nkEmpty:
       localError(c.config, a.info, "'using' sections cannot contain assignments")
 
+proc hasEmpty(typ: PType): bool =
+  if typ.kind in {tySequence, tyArray, tySet}:
+    result = typ.lastSon.kind == tyEmpty
+  elif typ.kind == tyTuple:
+    for s in typ.sons:
+      result = result or hasEmpty(s)
+
 proc hasUnresolvedParams(n: PNode; flags: TExprFlags): bool =
   result = tfUnresolved in n.typ.flags
   when false:
@@ -514,6 +509,13 @@ proc semVarMacroPragma(c: PContext, a: PNode, n: PNode): PNode =
       let it = pragmas[i]
       let key = if it.kind in nkPragmaCallKinds and it.len >= 1: it[0] else: it
 
+      when false:
+        let lhs = b[0]
+        let clash = strTableGet(c.currentScope.symbols, lhs.ident)
+        if clash != nil:
+          # refs https://github.com/nim-lang/Nim/issues/8275
+          wrongRedefinition(c, lhs.info, lhs.ident.s, clash.info)
+
       if isPossibleMacroPragma(c, it, key):
         # we transform ``var p {.m, rest.}`` into ``m(do: var p {.rest.})`` and
         # let the semantic checker deal with it:
@@ -560,18 +562,38 @@ proc semVarMacroPragma(c: PContext, a: PNode, n: PNode): PNode =
 
         doAssert result != nil
 
+        # since a macro pragma can set pragmas, we process these here again.
+        # This is required for SqueakNim-like export pragmas.
+        if false and result.kind in {nkVarSection, nkLetSection, nkConstSection}:
+          var validPragmas: TSpecialWords
+          case result.kind
+          of nkVarSection:
+            validPragmas = varPragmas
+          of nkLetSection:
+            validPragmas = letPragmas
+          of nkConstSection:
+            validPragmas = constPragmas
+          else:
+            # unreachable
+            discard
+          for defs in result:
+            for i in 0 ..< defs.len - 2:
+              let ex = defs[i]
+              if ex.kind == nkPragmaExpr and
+                  ex[namePos].kind == nkSym and
+                  ex[pragmaPos].kind != nkEmpty:
+                pragma(c, defs[lhsPos][namePos].sym, defs[lhsPos][pragmaPos], validPragmas)
         return result
 
-template isLocalSym(sym: PSym): bool =
-  sym.kind in {skVar, skLet} and not
-    ({sfGlobal, sfPure} * sym.flags != {} or
-      sfCompileTime in sym.flags) or
-      sym.kind in {skProc, skFunc, skIterator} and
-      sfGlobal notin sym.flags
-
 template isLocalVarSym(n: PNode): bool =
-  n.kind == nkSym and isLocalSym(n.sym)
-
+  n.kind == nkSym and 
+    (n.sym.kind in {skVar, skLet} and not 
+    ({sfGlobal, sfPure} <= n.sym.flags or
+      sfCompileTime in n.sym.flags) or
+      n.sym.kind in {skProc, skFunc, skIterator} and 
+      sfGlobal notin n.sym.flags
+      )
+  
 proc usesLocalVar(n: PNode): bool =
   for z in 1 ..< n.len:
     if n[z].isLocalVarSym:
@@ -584,57 +606,6 @@ proc globalVarInitCheck(c: PContext, n: PNode) =
   if n.isLocalVarSym or n.kind in nkCallKinds and usesLocalVar(n):
     localError(c.config, n.info, errCannotAssignToGlobal)
 
-proc makeVarTupleSection(c: PContext, n, a, def: PNode, typ: PType, symkind: TSymKind, origResult: var PNode): PNode =
-  ## expand tuple unpacking assignments into new var/let/const section
-  if typ.kind != tyTuple:
-    localError(c.config, a.info, errXExpected, "tuple")
-  elif a.len-2 != typ.len:
-    localError(c.config, a.info, errWrongNumberOfVariables)
-  var
-    tmpTuple: PSym
-    lastDef: PNode
-  let defkind = if symkind == skConst: nkConstDef else: nkIdentDefs
-  # temporary not needed if not const and RHS is tuple literal
-  # const breaks with seqs without temporary
-  let useTemp = def.kind notin {nkPar, nkTupleConstr} or symkind == skConst
-  if useTemp:
-    # use same symkind for compatibility with original section
-    tmpTuple = newSym(symkind, getIdent(c.cache, "tmpTuple"), nextSymId c.idgen, getCurrOwner(c), n.info)
-    tmpTuple.typ = typ
-    tmpTuple.flags.incl(sfGenSym)
-    lastDef = newNodeI(defkind, a.info)
-    newSons(lastDef, 3)
-    lastDef[0] = newSymNode(tmpTuple)
-    # NOTE: at the moment this is always ast.emptyNode, see parser.nim
-    lastDef[1] = a[^2]
-    lastDef[2] = def
-    tmpTuple.ast = lastDef
-    addToVarSection(c, origResult, n, lastDef)
-  result = newNodeI(n.kind, a.info)
-  for j in 0..<a.len-2:
-    let name = a[j]
-    if useTemp and name.kind == nkIdent and name.ident.s == "_":
-      # skip _ assignments if we are using a temp as they are already evaluated
-      continue
-    if name.kind == nkVarTuple:
-      # nested tuple
-      lastDef = newNodeI(nkVarTuple, name.info)
-      newSons(lastDef, name.len)
-      for k in 0..<name.len-2:
-        lastDef[k] = name[k]
-    else:
-      lastDef = newNodeI(defkind, name.info)
-      newSons(lastDef, 3)
-      lastDef[0] = name
-    lastDef[^2] = c.graph.emptyNode
-    if useTemp:
-      lastDef[^1] = newTreeIT(nkBracketExpr, name.info, typ[j], newSymNode(tmpTuple), newIntNode(nkIntLit, j))
-    else:
-      var val = def[j]
-      if val.kind == nkExprColonExpr: val = val[1]
-      lastDef[^1] = val
-    result.add(lastDef)
-
 proc semVarOrLet(c: PContext, n: PNode, symkind: TSymKind): PNode =
   var b: PNode
   result = copyNode(n)
@@ -701,37 +672,43 @@ proc semVarOrLet(c: PContext, n: PNode, symkind: TSymKind): PNode =
 
     var tup = skipTypes(typ, {tyGenericInst, tyAlias, tySink})
     if a.kind == nkVarTuple:
-      # generate new section from tuple unpacking and embed it into this one
-      let assignments = makeVarTupleSection(c, n, a, def, tup, symkind, result)
-      let resSection = semVarOrLet(c, assignments, symkind)
-      for resDef in resSection:
-        addToVarSection(c, result, n, resDef)
-    else:
-      if tup.kind == tyTuple and def.kind in {nkPar, nkTupleConstr} and
-          a.len > 3:
-        # var a, b = (1, 2)
-        message(c.config, a.info, warnEachIdentIsTuple)
+      if tup.kind != tyTuple:
+        localError(c.config, a.info, errXExpected, "tuple")
+      elif a.len-2 != tup.len:
+        localError(c.config, a.info, errWrongNumberOfVariables)
+      b = newNodeI(nkVarTuple, a.info)
+      newSons(b, a.len)
+      # keep type desc for doc generator
+      # NOTE: at the moment this is always ast.emptyNode, see parser.nim
+      b[^2] = a[^2]
+      b[^1] = def
+      addToVarSection(c, result, n, b)
+    elif tup.kind == tyTuple and def.kind in {nkPar, nkTupleConstr} and
+        a.kind == nkIdentDefs and a.len > 3:
+      message(c.config, a.info, warnEachIdentIsTuple)
 
-      for j in 0..<a.len-2:
-        if a[j].kind == nkDotExpr:
-          fillPartialObject(c, a[j], typ)
-          addToVarSection(c, result, n, a)
-          continue
-        var v = semIdentDef(c, a[j], symkind, false)
-        styleCheckDef(c, v)
-        onDef(a[j].info, v)
-        if sfGenSym notin v.flags:
-          if not isDiscardUnderscore(v): addInterfaceDecl(c, v)
+    for j in 0..<a.len-2:
+      if a[j].kind == nkDotExpr:
+        fillPartialObject(c, a[j],
+          if a.kind != nkVarTuple: typ else: tup[j])
+        addToVarSection(c, result, n, a)
+        continue
+      var v = semIdentDef(c, a[j], symkind, false)
+      styleCheckDef(c, v)
+      onDef(a[j].info, v)
+      if sfGenSym notin v.flags:
+        if not isDiscardUnderscore(v): addInterfaceDecl(c, v)
+      else:
+        if v.owner == nil: v.owner = c.p.owner
+      when oKeepVariableNames:
+        if c.inUnrolledContext > 0: v.flags.incl(sfShadowed)
         else:
-          if v.owner == nil: v.owner = c.p.owner
-        when oKeepVariableNames:
-          if c.inUnrolledContext > 0: v.flags.incl(sfShadowed)
-          else:
-            let shadowed = findShadowedVar(c, v)
-            if shadowed != nil:
-              shadowed.flags.incl(sfShadowed)
-              if shadowed.kind == skResult and sfGenSym notin v.flags:
-                message(c.config, a.info, warnResultShadowed)
+          let shadowed = findShadowedVar(c, v)
+          if shadowed != nil:
+            shadowed.flags.incl(sfShadowed)
+            if shadowed.kind == skResult and sfGenSym notin v.flags:
+              message(c.config, a.info, warnResultShadowed)
+      if a.kind != nkVarTuple:
         if def.kind != nkEmpty:
           if sfThread in v.flags: localError(c.config, def.info, errThreadvarCannotInit)
         setVarType(c, v, typ)
@@ -754,26 +731,32 @@ proc semVarOrLet(c: PContext, n: PNode, symkind: TSymKind): PNode =
         b.add copyTree(def)
         addToVarSection(c, result, n, b)
         v.ast = b
-        if def.kind == nkEmpty:
-          let actualType = v.typ.skipTypes({tyGenericInst, tyAlias,
-                                            tyUserTypeClassInst})
-          if actualType.kind in {tyObject, tyDistinct} and
-            actualType.requiresInit:
-            defaultConstructionError(c, v.typ, v.info)
-          else:
-            checkNilable(c, v)
-          # allow let to not be initialised if imported from C:
-          if v.kind == skLet and sfImportc notin v.flags and (strictDefs notin c.features or not isLocalSym(v)):
-            localError(c.config, a.info, errLetNeedsInit)
-        if sfCompileTime in v.flags:
-          var x = newNodeI(result.kind, v.info)
-          x.add result[i]
-          vm.setupCompileTimeVar(c.module, c.idgen, c.graph, x)
-        if v.flags * {sfGlobal, sfThread} == {sfGlobal}:
-          message(c.config, v.info, hintGlobalVar)
-        if {sfGlobal, sfPure} <= v.flags:
-          globalVarInitCheck(c, def)
-        suggestSym(c.graph, v.info, v, c.graph.usageSym)
+      else:
+        if def.kind in {nkPar, nkTupleConstr}: v.ast = def[j]
+        # bug #7663, for 'nim check' this can be a non-tuple:
+        if tup.kind == tyTuple: setVarType(c, v, tup[j])
+        else: v.typ = tup
+        b[j] = newSymNode(v)
+      if def.kind == nkEmpty:
+        let actualType = v.typ.skipTypes({tyGenericInst, tyAlias,
+                                          tyUserTypeClassInst})
+        if actualType.kind in {tyObject, tyDistinct} and
+           actualType.requiresInit:
+          defaultConstructionError(c, v.typ, v.info)
+        else:
+          checkNilable(c, v)
+        # allow let to not be initialised if imported from C:
+        if v.kind == skLet and sfImportc notin v.flags:
+          localError(c.config, a.info, errLetNeedsInit)
+      if sfCompileTime in v.flags:
+        var x = newNodeI(result.kind, v.info)
+        x.add result[i]
+        vm.setupCompileTimeVar(c.module, c.idgen, c.graph, x)
+      if v.flags * {sfGlobal, sfThread} == {sfGlobal}:
+        message(c.config, v.info, hintGlobalVar)
+      if {sfGlobal, sfPure} <= v.flags:
+        globalVarInitCheck(c, def)
+      suggestSym(c.graph, v.info, v, c.graph.usageSym)
 
 proc semConst(c: PContext, n: PNode): PNode =
   result = copyNode(n)
@@ -826,19 +809,23 @@ proc semConst(c: PContext, n: PNode): PNode =
       typeAllowedCheck(c, a.info, typ, skConst, typFlags)
 
     if a.kind == nkVarTuple:
-      # generate new section from tuple unpacking and embed it into this one
-      let assignments = makeVarTupleSection(c, n, a, def, typ, skConst, result)
-      let resSection = semConst(c, assignments)
-      for resDef in resSection:
-        addToVarSection(c, result, n, resDef)
-    else:
-      for j in 0..<a.len-2:
-        var v = semIdentDef(c, a[j], skConst)
-        if sfGenSym notin v.flags: addInterfaceDecl(c, v)
-        elif v.owner == nil: v.owner = getCurrOwner(c)
-        styleCheckDef(c, v)
-        onDef(a[j].info, v)
+      if typ.kind != tyTuple:
+        localError(c.config, a.info, errXExpected, "tuple")
+      elif a.len-2 != typ.len:
+        localError(c.config, a.info, errWrongNumberOfVariables)
+      b = newNodeI(nkVarTuple, a.info)
+      newSons(b, a.len)
+      b[^2] = a[^2]
+      b[^1] = def
 
+    for j in 0..<a.len-2:
+      var v = semIdentDef(c, a[j], skConst)
+      if sfGenSym notin v.flags: addInterfaceDecl(c, v)
+      elif v.owner == nil: v.owner = getCurrOwner(c)
+      styleCheckDef(c, v)
+      onDef(a[j].info, v)
+
+      if a.kind != nkVarTuple:
         setVarType(c, v, typ)
         when false:
           v.ast = def               # no need to copy
@@ -855,7 +842,12 @@ proc semConst(c: PContext, n: PNode): PNode =
         b.add a[1]
         b.add copyTree(def)
         v.ast = b
-      addToVarSection(c, result, n, b)
+      else:
+        setVarType(c, v, typ[j])
+        v.ast = if def[j].kind != nkExprColonExpr: def[j]
+                else: def[j][1]
+        b[j] = newSymNode(v)
+    addToVarSection(c, result, n, b)
   dec c.inStaticContext
 
 include semfields
@@ -1062,8 +1054,7 @@ proc semFor(c: PContext, n: PNode; flags: TExprFlags): PNode =
   result = n
   n[^2] = semExprNoDeref(c, n[^2], {efWantIterator})
   var call = n[^2]
-
-  if call.kind == nkStmtListExpr and (isTrivalStmtExpr(call) or (call.lastSon.kind in nkCallKinds and call.lastSon[0].sym.kind == skIterator)):
+  if call.kind == nkStmtListExpr and isTrivalStmtExpr(call):
     call = call.lastSon
     n[^2] = call
   let isCallExpr = call.kind in nkCallKinds
@@ -1398,7 +1389,7 @@ proc typeSectionRightSidePass(c: PContext, n: PNode) =
       rawAddSon(s.typ, newTypeS(tyNone, c))
       s.ast = a
       inc c.inGenericContext
-      var body = semTypeNode(c, a[2], s.typ)
+      var body = semTypeNode(c, a[2], nil)
       dec c.inGenericContext
       if body != nil:
         body.sym = s
@@ -1719,9 +1710,15 @@ proc semProcAnnotation(c: PContext, prc: PNode;
 
       doAssert result != nil
 
+      # since a proc annotation can set pragmas, we process these here again.
+      # This is required for SqueakNim-like export pragmas.
+      if false and result.kind in procDefs and result[namePos].kind == nkSym and
+          result[pragmasPos].kind != nkEmpty:
+        pragma(c, result[namePos].sym, result[pragmasPos], validPragmas)
+
       return result
 
-proc semInferredLambda(c: PContext, pt: TIdTable, n: PNode): PNode =
+proc semInferredLambda(c: PContext, pt: TIdTable, n: PNode): PNode {.nosinks.} =
   ## used for resolving 'auto' in lambdas based on their callsite
   var n = n
   let original = n[namePos].sym
@@ -1808,7 +1805,7 @@ proc whereToBindTypeHook(c: PContext; t: PType): PType =
 proc bindTypeHook(c: PContext; s: PSym; n: PNode; op: TTypeAttachedOp) =
   let t = s.typ
   var noError = false
-  let cond = if op in {attachedDestructor, attachedWasMoved}:
+  let cond = if op == attachedDestructor:
                t.len == 2 and t[0] == nil and t[1].kind == tyVar
              elif op == attachedTrace:
                t.len == 3 and t[0] == nil and t[1].kind == tyVar and t[2].kind == tyPointer
@@ -1884,10 +1881,6 @@ proc semOverride(c: PContext, s: PSym, n: PNode) =
     if s.magic == mAsgn: return
     incl(s.flags, sfUsed)
     incl(s.flags, sfOverriden)
-    if name == "=":
-      message(c.config, n.info, warnDeprecated, "Overriding `=` hook is deprecated; Override `=copy` hook instead")
-    elif c.config.selectedGC == gcRefc and name == "=copy":
-      message(c.config, n.info, warnCopyHookForRefc)
     let t = s.typ
     if t.len == 3 and t[0] == nil and t[1].kind == tyVar:
       var obj = t[1][0]
@@ -1925,9 +1918,6 @@ proc semOverride(c: PContext, s: PSym, n: PNode) =
   of "=trace":
     if s.magic != mTrace:
       bindTypeHook(c, s, n, attachedTrace)
-  of "=wasmoved":
-    if s.magic != mWasMoved:
-      bindTypeHook(c, s, n, attachedWasMoved)
   else:
     if sfOverriden in s.flags:
       localError(c.config, n.info, errGenerated,
@@ -2192,7 +2182,7 @@ proc semProcAux(c: PContext, n: PNode, kind: TSymKind,
         if s.kind notin {skMacro, skTemplate} and s.magic == mNone: paramsTypeCheck(c, s.typ)
 
         maybeAddResult(c, s, n)
-        let resultType =
+        let resultType = 
           if s.kind == skMacro:
             sysTypeFromName(c.graph, n.info, "NimNode")
           elif not isInlineIterator(s.typ):
@@ -2220,7 +2210,8 @@ proc semProcAux(c: PContext, n: PNode, kind: TSymKind,
     if hasProto: localError(c.config, n.info, errImplOfXexpected % proto.name.s)
     if {sfImportc, sfBorrow, sfError} * s.flags == {} and s.magic == mNone:
       # this is a forward declaration and we're building the prototype
-      if s.kind in {skProc, skFunc} and s.typ[0] != nil and s.typ[0].kind == tyAnything:
+      if s.kind in {skProc, skFunc} and s.typ[0] != nil and s.typ[0].kind == tyUntyped:
+        # `auto` is represented as `tyUntyped` at this point in compilation.
         localError(c.config, n[paramsPos][0].info, "return type 'auto' cannot be used in forward declarations")
 
       incl(s.flags, sfForward)
@@ -2306,6 +2297,7 @@ proc semMethod(c: PContext, n: PNode): PNode =
 
 proc semConverterDef(c: PContext, n: PNode): PNode =
   if not isTopLevel(c): localError(c.config, n.info, errXOnlyAtModuleScope % "converter")
+  checkSonsLen(n, bodyPos + 1, c.config)
   result = semProcAux(c, n, skConverter, converterPragmas)
   # macros can transform converters to nothing:
   if namePos >= result.safeLen: return result
@@ -2320,6 +2312,7 @@ proc semConverterDef(c: PContext, n: PNode): PNode =
   addConverterDef(c, LazySym(sym: s))
 
 proc semMacroDef(c: PContext, n: PNode): PNode =
+  checkSonsLen(n, bodyPos + 1, c.config)
   result = semProcAux(c, n, skMacro, macroPragmas)
   # macros can transform macros to nothing:
   if namePos >= result.safeLen: return result
@@ -2485,12 +2478,9 @@ proc semStmtList(c: PContext, n: PNode, flags: TExprFlags, expectedType: PType =
     else:
       n.typ = n[i].typ
       if not isEmptyType(n.typ): n.transitionSonsKind(nkStmtListExpr)
-    var m = n[i]
-    while m.kind in {nkStmtListExpr, nkStmtList} and m.len > 0: # from templates
-      m = m.lastSon
-    if m.kind in nkLastBlockStmts or
-        m.kind in nkCallKinds and m[0].kind == nkSym and
-        sfNoReturn in m[0].sym.flags:
+    if n[i].kind in nkLastBlockStmts or
+        n[i].kind in nkCallKinds and n[i][0].kind == nkSym and
+        sfNoReturn in n[i][0].sym.flags:
       for j in i + 1..<n.len:
         case n[j].kind
         of nkPragma, nkCommentStmt, nkNilLit, nkEmpty, nkState: discard

compiler/semstrictfuncs.nim

@@ -1,55 +0,0 @@
-#
-#
-#           The Nim Compiler
-#        (c) Copyright 2022 Andreas Rumpf
-#
-#    See the file "copying.txt", included in this
-#    distribution, for details about the copyright.
-#
-
-## New "strict funcs" checking. Much simpler and hopefully easier to teach than
-## the old but more advanced algorithm that can/could be found in `varpartitions.nim`.
-
-import ast, typeallowed, renderer
-from aliasanalysis import PathKinds0, PathKinds1
-from trees import getMagic
-
-proc isDangerousLocation*(n: PNode; owner: PSym): bool =
-  var n = n
-  var hasDeref = false
-  while true:
-    case n.kind
-    of nkDerefExpr, nkHiddenDeref:
-      if n[0].typ.kind != tyVar:
-        hasDeref = true
-      n = n[0]
-    of PathKinds0 - {nkDerefExpr, nkHiddenDeref}:
-      n = n[0]
-    of PathKinds1:
-      n = n[1]
-    of nkCallKinds:
-      if n.len > 1:
-        if (n.typ != nil and classifyViewType(n.typ) != noView) or getMagic(n) == mSlice:
-          # borrow from first parameter:
-          n = n[1]
-        else:
-          break
-      else:
-        break
-    else:
-      break
-  if n.kind == nkSym:
-    # dangerous if contains a pointer deref or if it doesn't belong to us:
-    result = hasDeref or n.sym.owner != owner
-    when false:
-      # store to something that belongs to a `var` parameter is fine:
-      let s = n.sym
-      if s.kind == skParam:
-        # dangerous unless a `var T` parameter:
-        result = s.typ.kind != tyVar
-      else:
-        # dangerous if contains a pointer deref or if it doesn't belong to us:
-        result = hasDeref or s.owner != owner
-  else:
-    # dangerous if it contains a pointer deref
-    result = hasDeref

compiler/semtempl.nim

@@ -642,9 +642,8 @@ proc semTemplateDef(c: PContext, n: PNode): PNode =
     # body by the absence of the sfGenSym flag:
     for i in 1..<s.typ.n.len:
       let param = s.typ.n[i].sym
-      if param.name.s != "_":
-        param.flags.incl sfTemplateParam
-        param.flags.excl sfGenSym
+      param.flags.incl sfTemplateParam
+      param.flags.excl sfGenSym
       if param.typ.kind != tyUntyped: allUntyped = false
   else:
     s.typ = newTypeS(tyProc, c)
@@ -692,7 +691,7 @@ proc semTemplateDef(c: PContext, n: PNode): PNode =
   elif not comesFromShadowscope:
     if {sfTemplateRedefinition, sfGenSym} * s.flags == {}:
       #wrongRedefinition(c, n.info, proto.name.s, proto.info)
-      message(c.config, n.info, warnImplicitTemplateRedefinition, s.name.s)
+      message(c.config, n.info, warnTemplateRedefinition, s.name.s)
     symTabReplace(c.currentScope.symbols, proto, s)
   if n[patternPos].kind != nkEmpty:
     c.patterns.add(s)

compiler/semtypes.nim

@@ -38,7 +38,7 @@ const
   errInOutFlagNotExtern = "the '$1' modifier can be used only with imported types"
 
 proc newOrPrevType(kind: TTypeKind, prev: PType, c: PContext): PType =
-  if prev == nil or prev.kind == tyGenericBody:
+  if prev == nil:
     result = newTypeS(kind, c)
   else:
     result = prev
@@ -73,7 +73,7 @@ proc semEnum(c: PContext, n: PNode, prev: PType): PType =
   rawAddSon(result, base)
   let isPure = result.sym != nil and sfPure in result.sym.flags
   var symbols: TStrTable
-  initStrTable(symbols)
+  if isPure: initStrTable(symbols)
   var hasNull = false
   for i in 1..<n.len:
     if n[i].kind == nkEmpty: continue
@@ -130,8 +130,8 @@ proc semEnum(c: PContext, n: PNode, prev: PType): PType =
     e.typ = result
     e.position = int(counter)
     let symNode = newSymNode(e)
-    if identToReplace != nil and c.config.cmd notin cmdDocLike:
-      # A hack to produce documentation for enum fields.
+    if optNimV1Emulation notin c.config.globalOptions and identToReplace != nil and
+        c.config.cmd notin cmdDocLike: # A hack to produce documentation for enum fields.
       identToReplace[] = symNode
     if e.position == 0: hasNull = true
     if result.sym != nil and sfExported in result.sym.flags:
@@ -145,7 +145,7 @@ proc semEnum(c: PContext, n: PNode, prev: PType): PType =
         addInterfaceOverloadableSymAt(c, c.currentScope, e)
       else:
         declarePureEnumField(c, e)
-    if (let conflict = strTableInclReportConflict(symbols, e); conflict != nil):
+    if isPure and (let conflict = strTableInclReportConflict(symbols, e); conflict != nil):
       wrongRedefinition(c, e.info, e.name.s, conflict.info)
     inc(counter)
   if isPure and sfExported in result.sym.flags:
@@ -222,11 +222,9 @@ proc isRecursiveType(t: PType, cycleDetector: var IntSet): bool =
 
 proc fitDefaultNode(c: PContext, n: PNode): PType =
   let expectedType = if n[^2].kind != nkEmpty: semTypeNode(c, n[^2], nil) else: nil
-  let oldType = n[^1].typ
   n[^1] = semConstExpr(c, n[^1], expectedType = expectedType)
-  n[^1].flags.incl nfSem
   if n[^2].kind != nkEmpty:
-    if expectedType != nil and oldType != expectedType:
+    if expectedType != nil:
       n[^1] = fitNodeConsiderViewType(c, expectedType, n[^1], n[^1].info)
     result = n[^1].typ
   else:
@@ -330,8 +328,6 @@ proc semRange(c: PContext, n: PNode, prev: PType): PType =
 proc semArrayIndex(c: PContext, n: PNode): PType =
   if isRange(n):
     result = semRangeAux(c, n, nil)
-  elif n.kind == nkInfix and n[0].kind == nkIdent and n[0].ident.s == "..<":
-    result = errorType(c)
   else:
     let e = semExprWithType(c, n, {efDetermineType})
     if e.typ.kind == tyFromExpr:
@@ -519,7 +515,7 @@ proc semTuple(c: PContext, n: PNode, prev: PType): PType =
         let fSym = newSymNode(field)
         if hasDefaultField:
           fSym.sym.ast = a[^1]
-          fSym.sym.ast.flags.incl nfSkipFieldChecking
+          fSym.sym.ast.flags.incl nfUseDefaultField
         result.n.add fSym
         addSonSkipIntLit(result, typ, c.idgen)
       styleCheckDef(c, a[j].info, field)
@@ -781,10 +777,9 @@ proc semRecordNodeAux(c: PContext, n: PNode, check: var IntSet, pos: var int,
   if n == nil: return
   case n.kind
   of nkRecWhen:
-    var a = copyTree(n)
     var branch: PNode = nil   # the branch to take
-    for i in 0..<a.len:
-      var it = a[i]
+    for i in 0..<n.len:
+      var it = n[i]
       if it == nil: illFormedAst(n, c.config)
       var idx = 1
       case it.kind
@@ -803,13 +798,14 @@ proc semRecordNodeAux(c: PContext, n: PNode, check: var IntSet, pos: var int,
       else: illFormedAst(n, c.config)
       if c.inGenericContext > 0:
         # use a new check intset here for each branch:
-        var newCheck: IntSet = check
+        var newCheck: IntSet
+        assign(newCheck, check)
         var newPos = pos
         var newf = newNodeI(nkRecList, n.info)
         semRecordNodeAux(c, it[idx], newCheck, newPos, newf, rectype, hasCaseFields)
         it[idx] = if newf.len == 1: newf[0] else: newf
     if c.inGenericContext > 0:
-      father.add a
+      father.add n
     elif branch != nil:
       semRecordNodeAux(c, branch, check, pos, father, rectype, hasCaseFields)
     elif father.kind in {nkElse, nkOfBranch}:
@@ -867,7 +863,7 @@ proc semRecordNodeAux(c: PContext, n: PNode, check: var IntSet, pos: var int,
       let fSym = newSymNode(f)
       if hasDefaultField:
         fSym.sym.ast = n[^1]
-        fSym.sym.ast.flags.incl nfSkipFieldChecking
+        fSym.sym.ast.flags.incl nfUseDefaultField
       if a.kind == nkEmpty: father.add fSym
       else: a.add fSym
       styleCheckDef(c, f)
@@ -981,7 +977,7 @@ proc semAnyRef(c: PContext; n: PNode; kind: TTypeKind; prev: PType): PType =
     let n = if n[0].kind == nkBracket: n[0] else: n
     checkMinSonsLen(n, 1, c.config)
     let body = n.lastSon
-    var t = if prev != nil and prev.kind != tyGenericBody and body.kind == nkObjectTy:
+    var t = if prev != nil and body.kind == nkObjectTy:
               semObjectNode(c, body, nil, prev.flags)
             else:
               semTypeNode(c, body, nil)
@@ -1310,26 +1306,14 @@ proc semProcTypeNode(c: PContext, n, genericParams: PNode,
 
     if hasDefault:
       def = a[^1]
-      if a.len > 3:
-        var msg = ""
-        for j in 0 ..< a.len - 2:
-          if msg.len != 0: msg.add(", ")
-          msg.add($a[j])
-        msg.add(" all have default value '")
-        msg.add(def.renderTree)
-        msg.add("', this may be unintentional, " &
-          "either use ';' (semicolon) or explicitly write each default value")
-        message(c.config, a.info, warnImplicitDefaultValue, msg)
       block determineType:
-        var defTyp = typ
         if genericParams != nil and genericParams.len > 0:
-          defTyp = nil
           def = semGenericStmt(c, def)
           if hasUnresolvedArgs(c, def):
             def.typ = makeTypeFromExpr(c, def.copyTree)
             break determineType
 
-        def = semExprWithType(c, def, {efDetermineType}, defTyp)
+        def = semExprWithType(c, def, {efDetermineType})
         if def.referencesAnotherParam(getCurrOwner(c)):
           def.flags.incl nfDefaultRefsParam
 
@@ -1368,10 +1352,6 @@ proc semProcTypeNode(c: PContext, n, genericParams: PNode,
 
     for j in 0..<a.len-2:
       var arg = newSymG(skParam, if a[j].kind == nkPragmaExpr: a[j][0] else: a[j], c)
-      if arg.name.s == "_":
-        arg.flags.incl(sfGenSym)
-      elif containsOrIncl(check, arg.name.id):
-        localError(c.config, a[j].info, "attempt to redefine: '" & arg.name.s & "'")
       if a[j].kind == nkPragmaExpr:
         pragma(c, arg, a[j][1], paramPragmas)
       if not hasType and not hasDefault and kind notin {skTemplate, skMacro}:
@@ -1380,11 +1360,8 @@ proc semProcTypeNode(c: PContext, n, genericParams: PNode,
         else:
           localError(c.config, a.info, "parameter '$1' requires a type" % arg.name.s)
           typ = errorType(c)
-      var nameForLift = arg.name.s
-      if sfGenSym in arg.flags:
-        nameForLift.add("`gensym" & $arg.id)
       let lifted = liftParamType(c, kind, genericParams, typ,
-                                 nameForLift, arg.info)
+                                 arg.name.s, arg.info)
       let finalType = if lifted != nil: lifted else: typ.skipIntLit(c.idgen)
       arg.typ = finalType
       arg.position = counter
@@ -1392,12 +1369,15 @@ proc semProcTypeNode(c: PContext, n, genericParams: PNode,
       inc(counter)
       if def != nil and def.kind != nkEmpty:
         arg.ast = copyTree(def)
+      if containsOrIncl(check, arg.name.id):
+        localError(c.config, a[j].info, "attempt to redefine: '" & arg.name.s & "'")
       result.n.add newSymNode(arg)
       rawAddSon(result, finalType)
       addParamOrResult(c, arg, kind)
       styleCheckDef(c, a[j].info, arg)
       onDef(a[j].info, arg)
-      a[j] = newSymNode(arg)
+      if {optNimV1Emulation, optNimV12Emulation} * c.config.globalOptions == {}:
+        a[j] = newSymNode(arg)
 
   var r: PType
   if n[0].kind != nkEmpty:
@@ -1424,7 +1404,9 @@ proc semProcTypeNode(c: PContext, n, genericParams: PNode,
             "' is only valid for macros and templates")
       # 'auto' as a return type does not imply a generic:
       elif r.kind == tyAnything:
-        discard
+        # 'p(): auto' and 'p(): untyped' are equivalent, but the rest of the
+        # compiler is hardly aware of 'auto':
+        r = newTypeS(tyUntyped, c)
       elif r.kind == tyStatic:
         # type allowed should forbid this type
         discard
@@ -1593,23 +1575,21 @@ proc maybeAliasType(c: PContext; typeExpr, prev: PType): PType =
     result = newTypeS(tyAlias, c)
     result.rawAddSon typeExpr
     result.sym = prev.sym
-    if prev.kind != tyGenericBody:
-      assignType(prev, result)
+    assignType(prev, result)
 
 proc fixupTypeOf(c: PContext, prev: PType, typExpr: PNode) =
   if prev != nil:
     let result = newTypeS(tyAlias, c)
     result.rawAddSon typExpr.typ
     result.sym = prev.sym
-    if prev.kind != tyGenericBody:
-      assignType(prev, result)
+    assignType(prev, result)
 
 proc semTypeExpr(c: PContext, n: PNode; prev: PType): PType =
   var n = semExprWithType(c, n, {efDetermineType})
   if n.typ.kind == tyTypeDesc:
     result = n.typ.base
     # fix types constructed by macros/template:
-    if prev != nil and prev.kind != tyGenericBody and prev.sym != nil:
+    if prev != nil and prev.sym != nil:
       if result.sym.isNil:
         # Behold! you're witnessing enormous power yielded
         # by macros. Only macros can summon unnamed types
@@ -1630,7 +1610,7 @@ proc semTypeExpr(c: PContext, n: PNode; prev: PType): PType =
     result = errorType(c)
 
 proc freshType(c: PContext; res, prev: PType): PType {.inline.} =
-  if prev.isNil or prev.kind == tyGenericBody:
+  if prev.isNil:
     result = copyType(res, nextTypeId c.idgen, res.owner)
     copyTypeProps(c.graph, c.idgen.module, result, res)
   else:
@@ -1759,7 +1739,7 @@ proc semProcTypeWithScope(c: PContext, n: PNode,
   if n[1].kind != nkEmpty and n[1].len > 0:
     pragma(c, s, n[1], procTypePragmas)
     when useEffectSystem: setEffectsForProcType(c.graph, result, n[1])
-  elif c.optionStack.len > 0:
+  elif c.optionStack.len > 0 and optNimV1Emulation notin c.config.globalOptions:
     # we construct a fake 'nkProcDef' for the 'mergePragmas' inside 'implicitPragmas'...
     s.ast = newTree(nkProcDef, newNodeI(nkEmpty, n.info), newNodeI(nkEmpty, n.info),
         newNodeI(nkEmpty, n.info), newNodeI(nkEmpty, n.info), newNodeI(nkEmpty, n.info))
@@ -2008,8 +1988,6 @@ proc semTypeNode(c: PContext, n: PNode, prev: PType): PType =
       let alias = maybeAliasType(c, s.typ, prev)
       if alias != nil:
         result = alias
-      elif prev.kind == tyGenericBody:
-        result = s.typ
       else:
         assignType(prev, s.typ)
         # bugfix: keep the fresh id for aliases to integral types:
@@ -2029,7 +2007,7 @@ proc semTypeNode(c: PContext, n: PNode, prev: PType): PType =
       let alias = maybeAliasType(c, t, prev)
       if alias != nil:
         result = alias
-      elif prev == nil or prev.kind == tyGenericBody:
+      elif prev == nil:
         result = t
       else:
         assignType(prev, t)
@@ -2054,27 +2032,25 @@ proc semTypeNode(c: PContext, n: PNode, prev: PType): PType =
   of nkOutTy: result = semVarOutType(c, n, prev, {tfIsOutParam})
   of nkDistinctTy: result = semDistinct(c, n, prev)
   of nkStaticTy: result = semStaticType(c, n[0], prev)
-  of nkProcTy, nkIteratorTy:
-    if n.len == 0 or n[0].kind == nkEmpty:
-      # 0 length or empty param list with possible pragmas imply typeclass
+  of nkIteratorTy:
+    if n.len == 0:
       result = newTypeS(tyBuiltInTypeClass, c)
       let child = newTypeS(tyProc, c)
-      var symKind: TSymKind
-      if n.kind == nkIteratorTy:
-        child.flags.incl tfIterator
-      if n.len > 0 and n[1].kind != nkEmpty and n[1].len > 0:
-        # typeclass with pragma
-        let symKind = if n.kind == nkIteratorTy: skIterator else: skProc
-        # dummy symbol for `pragma`:
-        var s = newSymS(symKind, newIdentNode(getIdent(c.cache, "dummy"), n.info), c)
-        s.typ = child
-        # for now only call convention pragmas supported in proc typeclass
-        pragma(c, s, n[1], {FirstCallConv..LastCallConv})
+      child.flags.incl tfIterator
       result.addSonSkipIntLit(child, c.idgen)
     else:
-      result = semProcTypeWithScope(c, n, prev, skProc)
-      if n.kind == nkIteratorTy and result.kind == tyProc:
+      result = semProcTypeWithScope(c, n, prev, skIterator)
+      if result.kind == tyProc:
         result.flags.incl(tfIterator)
+        if n.lastSon.kind == nkPragma and hasPragma(n.lastSon, wInline):
+          result.callConv = ccInline
+        else:
+          result.callConv = ccClosure
+  of nkProcTy:
+    if n.len == 0:
+      result = newConstraint(c, tyProc)
+    else:
+      result = semProcTypeWithScope(c, n, prev, skProc)
   of nkEnumTy: result = semEnum(c, n, prev)
   of nkType: result = n.typ
   of nkStmtListType: result = semStmtListType(c, n, prev)

compiler/semtypinst.nim

@@ -193,24 +193,6 @@ proc replaceObjBranches(cl: TReplTypeVars, n: PNode): PNode =
     for i in 0..<n.len:
       n[i] = replaceObjBranches(cl, n[i])
 
-proc hasValuelessStatics(n: PNode): bool =
-  # We should only attempt to call an expression that has no tyStatics
-  # As those are unresolved generic parameters, which means in the following
-  # The compiler attempts to do `T == 300` which errors since the typeclass `MyThing` lacks a parameter
-  #[
-    type MyThing[T: static int] = object
-      when T == 300:
-        a
-    proc doThing(_: MyThing)
-  ]#
-  if n.safeLen == 0:
-    n.typ == nil or n.typ.kind == tyStatic
-  else:
-    for x in n:
-      if hasValuelessStatics(x):
-        return true
-    false
-
 proc replaceTypeVarsN(cl: var TReplTypeVars, n: PNode; start=0): PNode =
   if n == nil: return
   result = copyNode(n)
@@ -235,11 +217,10 @@ proc replaceTypeVarsN(cl: var TReplTypeVars, n: PNode; start=0): PNode =
       of nkElifBranch:
         checkSonsLen(it, 2, cl.c.config)
         var cond = prepareNode(cl, it[0])
-        if not cond.hasValuelessStatics:
-          var e = cl.c.semConstExpr(cl.c, cond)
-          if e.kind != nkIntLit:
-            internalError(cl.c.config, e.info, "ReplaceTypeVarsN: when condition not a bool")
-          if e.intVal != 0 and branch == nil: branch = it[1]
+        var e = cl.c.semConstExpr(cl.c, cond)
+        if e.kind != nkIntLit:
+          internalError(cl.c.config, e.info, "ReplaceTypeVarsN: when condition not a bool")
+        if e.intVal != 0 and branch == nil: branch = it[1]
       of nkElse:
         checkSonsLen(it, 1, cl.c.config)
         if branch == nil: branch = it[0]
@@ -518,20 +499,10 @@ proc propagateFieldFlags(t: PType, n: PNode) =
 
 proc replaceTypeVarsTAux(cl: var TReplTypeVars, t: PType): PType =
   template bailout =
-    if (t.sym == nil) or (t.sym != nil and sfGeneratedType in t.sym.flags):
-      # In the first case 't.sym' can be 'nil' if the type is a ref/ptr, see
-      # issue https://github.com/nim-lang/Nim/issues/20416 for more details.
-      # Fortunately for us this works for now because partial ref/ptr types are
-      # not allowed in object construction, eg.
-      #   type
-      #     Container[T] = ...
-      #     O = object
-      #      val: ref Container
-      #
-      # In the second case only consider the recursion limit if the symbol is a
-      # type with generic parameters that have not been explicitly supplied,
-      # typechecking should terminate when generic parameters are explicitly
-      # supplied.
+    if t.sym != nil and sfGeneratedType in t.sym.flags:
+      # Only consider the recursion limit if the symbol is a type with generic
+      # parameters that have not been explicitly supplied, typechecking should
+      # terminate when generic parameters are explicitly supplied.
       if cl.recursionLimit > 100:
         # bail out, see bug #2509. But note this caching is in general wrong,
         # look at this example where TwoVectors should not share the generic

compiler/sighashes.nim

@@ -9,11 +9,10 @@
 
 ## Computes hash values for routine (proc, method etc) signatures.
 
-import ast, tables, ropes, md5, modulegraphs, options, msgs, pathutils
+import ast, tables, ropes, md5_old, modulegraphs
 from hashes import Hash
 import types
 
-
 when defined(nimPreviewSlimSystem):
   import std/assertions
 
@@ -43,7 +42,8 @@ type
     CoDistinct
     CoHashTypeInsideNode
 
-proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: ConfigRef)
+proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag])
+
 proc hashSym(c: var MD5Context, s: PSym) =
   if sfAnon in s.flags or s.kind == skGenericParam:
     c &= ":anon"
@@ -54,21 +54,20 @@ proc hashSym(c: var MD5Context, s: PSym) =
       c &= "."
       it = it.owner
 
-proc hashTypeSym(c: var MD5Context, s: PSym; conf: ConfigRef) =
+proc hashTypeSym(c: var MD5Context, s: PSym) =
   if sfAnon in s.flags or s.kind == skGenericParam:
     c &= ":anon"
   else:
     var it = s
-    c &= customPath(conf.toFullPath(s.info))
     while it != nil:
       if sfFromGeneric in it.flags and it.kind in routineKinds and
           it.typ != nil:
-        hashType c, it.typ, {CoProc}, conf
+        hashType c, it.typ, {CoProc}
       c &= it.name.s
       c &= "."
       it = it.owner
 
-proc hashTree(c: var MD5Context, n: PNode; flags: set[ConsiderFlag]; conf: ConfigRef) =
+proc hashTree(c: var MD5Context, n: PNode; flags: set[ConsiderFlag]) =
   if n == nil:
     c &= "\255"
     return
@@ -83,7 +82,7 @@ proc hashTree(c: var MD5Context, n: PNode; flags: set[ConsiderFlag]; conf: Confi
   of nkSym:
     hashSym(c, n.sym)
     if CoHashTypeInsideNode in flags and n.sym.typ != nil:
-      hashType(c, n.sym.typ, flags, conf)
+      hashType(c, n.sym.typ, flags)
   of nkCharLit..nkUInt64Lit:
     let v = n.intVal
     lowlevel v
@@ -93,9 +92,9 @@ proc hashTree(c: var MD5Context, n: PNode; flags: set[ConsiderFlag]; conf: Confi
   of nkStrLit..nkTripleStrLit:
     c &= n.strVal
   else:
-    for i in 0..<n.len: hashTree(c, n[i], flags, conf)
+    for i in 0..<n.len: hashTree(c, n[i], flags)
 
-proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: ConfigRef) =
+proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]) =
   if t == nil:
     c &= "\254"
     return
@@ -103,14 +102,14 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
   case t.kind
   of tyGenericInvocation:
     for i in 0..<t.len:
-      c.hashType t[i], flags, conf
+      c.hashType t[i], flags
   of tyDistinct:
     if CoDistinct in flags:
       if t.sym != nil: c.hashSym(t.sym)
       if t.sym == nil or tfFromGeneric in t.flags:
-        c.hashType t.lastSon, flags, conf
+        c.hashType t.lastSon, flags
     elif CoType in flags or t.sym == nil:
-      c.hashType t.lastSon, flags, conf
+      c.hashType t.lastSon, flags
     else:
       c.hashSym(t.sym)
   of tyGenericInst:
@@ -120,15 +119,15 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
       # value for each instantiation, so we hash the generic parameters here:
       let normalizedType = t.skipGenericAlias
       for i in 0..<normalizedType.len - 1:
-        c.hashType t[i], flags, conf
+        c.hashType t[i], flags
     else:
-      c.hashType t.lastSon, flags, conf
+      c.hashType t.lastSon, flags
   of tyAlias, tySink, tyUserTypeClasses, tyInferred:
-    c.hashType t.lastSon, flags, conf
+    c.hashType t.lastSon, flags
   of tyOwned:
     if CoConsiderOwned in flags:
       c &= char(t.kind)
-    c.hashType t.lastSon, flags, conf
+    c.hashType t.lastSon, flags
   of tyBool, tyChar, tyInt..tyUInt64:
     # no canonicalization for integral types, so that e.g. ``pid_t`` is
     # produced instead of ``NI``:
@@ -142,7 +141,7 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
       t.typeInst = nil
       assert inst.kind == tyGenericInst
       for i in 0..<inst.len - 1:
-        c.hashType inst[i], flags, conf
+        c.hashType inst[i], flags
       t.typeInst = inst
       return
     c &= char(t.kind)
@@ -154,7 +153,7 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
         # The user has set a specific name for this type
         c &= t.sym.loc.r
       elif CoOwnerSig in flags:
-        c.hashTypeSym(t.sym, conf)
+        c.hashTypeSym(t.sym)
       else:
         c.hashSym(t.sym)
 
@@ -173,7 +172,7 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
           # xxx instead, use a hash table to indicate we've already visited a type, which
           # would also be more efficient.
           symWithFlags.flags.excl {sfAnon, sfGenSym}
-          hashTree(c, t.n, flags + {CoHashTypeInsideNode}, conf)
+          hashTree(c, t.n, flags + {CoHashTypeInsideNode})
           symWithFlags.flags = oldFlags
         else:
           # The object has no fields: we _must_ add something here in order to
@@ -183,15 +182,15 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
     else:
       c &= t.id
     if t.len > 0 and t[0] != nil:
-      hashType c, t[0], flags, conf
+      hashType c, t[0], flags
   of tyRef, tyPtr, tyGenericBody, tyVar:
     c &= char(t.kind)
     if t.sons.len > 0:
-      c.hashType t.lastSon, flags, conf
+      c.hashType t.lastSon, flags
     if tfVarIsPtr in t.flags: c &= ".varisptr"
   of tyFromExpr:
     c &= char(t.kind)
-    c.hashTree(t.n, {}, conf)
+    c.hashTree(t.n, {})
   of tyTuple:
     c &= char(t.kind)
     if t.n != nil and CoType notin flags:
@@ -200,19 +199,19 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
         assert(t.n[i].kind == nkSym)
         c &= t.n[i].sym.name.s
         c &= ':'
-        c.hashType(t[i], flags+{CoIgnoreRange}, conf)
+        c.hashType(t[i], flags+{CoIgnoreRange})
         c &= ','
     else:
-      for i in 0..<t.len: c.hashType t[i], flags+{CoIgnoreRange}, conf
+      for i in 0..<t.len: c.hashType t[i], flags+{CoIgnoreRange}
   of tyRange:
     if CoIgnoreRange notin flags:
       c &= char(t.kind)
-      c.hashTree(t.n, {}, conf)
-    c.hashType(t[0], flags, conf)
+      c.hashTree(t.n, {})
+    c.hashType(t[0], flags)
   of tyStatic:
     c &= char(t.kind)
-    c.hashTree(t.n, {}, conf)
-    c.hashType(t[0], flags, conf)
+    c.hashTree(t.n, {})
+    c.hashType(t[0], flags)
   of tyProc:
     c &= char(t.kind)
     c &= (if tfIterator in t.flags: "iterator " else: "proc ")
@@ -222,11 +221,11 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
         let param = params[i].sym
         c &= param.name.s
         c &= ':'
-        c.hashType(param.typ, flags, conf)
+        c.hashType(param.typ, flags)
         c &= ','
-      c.hashType(t[0], flags, conf)
+      c.hashType(t[0], flags)
     else:
-      for i in 0..<t.len: c.hashType(t[i], flags, conf)
+      for i in 0..<t.len: c.hashType(t[i], flags)
     c &= char(t.callConv)
     # purity of functions doesn't have to affect the mangling (which is in fact
     # problematic for HCR - someone could have cached a pointer to another
@@ -238,10 +237,10 @@ proc hashType(c: var MD5Context, t: PType; flags: set[ConsiderFlag]; conf: Confi
     if tfVarargs in t.flags: c &= ".varargs"
   of tyArray:
     c &= char(t.kind)
-    for i in 0..<t.len: c.hashType(t[i], flags-{CoIgnoreRange}, conf)
+    for i in 0..<t.len: c.hashType(t[i], flags-{CoIgnoreRange})
   else:
     c &= char(t.kind)
-    for i in 0..<t.len: c.hashType(t[i], flags, conf)
+    for i in 0..<t.len: c.hashType(t[i], flags)
   if tfNotNil in t.flags and CoType notin flags: c &= "not nil"
 
 when defined(debugSigHashes):
@@ -258,19 +257,19 @@ when defined(debugSigHashes):
   #  select hash, type from sighashes where hash in
   # (select hash from sighashes group by hash having count(*) > 1) order by hash;
 
-proc hashType*(t: PType; conf: ConfigRef; flags: set[ConsiderFlag] = {CoType}): SigHash =
+proc hashType*(t: PType; flags: set[ConsiderFlag] = {CoType}): SigHash =
   var c: MD5Context
   md5Init c
-  hashType c, t, flags+{CoOwnerSig}, conf
+  hashType c, t, flags+{CoOwnerSig}
   md5Final c, result.MD5Digest
   when defined(debugSigHashes):
     db.exec(sql"INSERT OR IGNORE INTO sighashes(type, hash) VALUES (?, ?)",
             typeToString(t), $result)
 
-proc hashProc*(s: PSym; conf: ConfigRef): SigHash =
+proc hashProc*(s: PSym): SigHash =
   var c: MD5Context
   md5Init c
-  hashType c, s.typ, {CoProc}, conf
+  hashType c, s.typ, {CoProc}
 
   var m = s
   while m.kind != skModule: m = m.owner
@@ -316,9 +315,9 @@ proc hashOwner*(s: PSym): SigHash =
 
   md5Final c, result.MD5Digest
 
-proc sigHash*(s: PSym; conf: ConfigRef): SigHash =
+proc sigHash*(s: PSym): SigHash =
   if s.kind in routineKinds and s.typ != nil:
-    result = hashProc(s, conf)
+    result = hashProc(s)
   else:
     result = hashNonProc(s)
 
@@ -379,7 +378,7 @@ proc symBodyDigest*(graph: ModuleGraph, sym: PSym): SigHash =
 
   var c: MD5Context
   md5Init(c)
-  c.hashType(sym.typ, {CoProc}, graph.config)
+  c.hashType(sym.typ, {CoProc})
   c &= char(sym.kind)
   c.md5Final(result.MD5Digest)
   graph.symBodyHashes[sym.id] = result # protect from recursion in the body
@@ -392,12 +391,12 @@ proc symBodyDigest*(graph: ModuleGraph, sym: PSym): SigHash =
     graph.symBodyHashes[sym.id] = result
 
 proc idOrSig*(s: PSym, currentModule: string,
-              sigCollisions: var CountTable[SigHash]; conf: ConfigRef): Rope =
+              sigCollisions: var CountTable[SigHash]): Rope =
   if s.kind in routineKinds and s.typ != nil:
     # signatures for exported routines are reliable enough to
     # produce a unique name and this means produced C++ is more stable regarding
     # Nim changes:
-    let sig = hashProc(s, conf)
+    let sig = hashProc(s)
     result = rope($sig)
     #let m = if s.typ.callConv != ccInline: findPendingModule(m, s) else: m
     let counter = sigCollisions.getOrDefault(sig)

compiler/sigmatch.nim

@@ -322,32 +322,26 @@ proc argTypeToString(arg: PNode; prefer: TPreferedDesc): string =
   else:
     result = arg.typ.typeToString(prefer)
 
-template describeArgImpl(c: PContext, n: PNode, i: int, startIdx = 1; prefer = preferName) =
-  var arg = n[i]
-  if n[i].kind == nkExprEqExpr:
-    result.add renderTree(n[i][0])
-    result.add ": "
-    if arg.typ.isNil and arg.kind notin {nkStmtList, nkDo}:
-      # XXX we really need to 'tryExpr' here!
-      arg = c.semOperand(c, n[i][1])
-      n[i].typ = arg.typ
-      n[i][1] = arg
-  else:
-    if arg.typ.isNil and arg.kind notin {nkStmtList, nkDo, nkElse,
-                                          nkOfBranch, nkElifBranch,
-                                          nkExceptBranch}:
-      arg = c.semOperand(c, n[i])
-      n[i] = arg
-  if arg.typ != nil and arg.typ.kind == tyError: return
-  result.add argTypeToString(arg, prefer)
-
-proc describeArg*(c: PContext, n: PNode, i: int, startIdx = 1; prefer = preferName): string =
-  describeArgImpl(c, n, i, startIdx, prefer)
-
 proc describeArgs*(c: PContext, n: PNode, startIdx = 1; prefer = preferName): string =
   result = ""
   for i in startIdx..<n.len:
-    describeArgImpl(c, n, i, startIdx, prefer)
+    var arg = n[i]
+    if n[i].kind == nkExprEqExpr:
+      result.add renderTree(n[i][0])
+      result.add ": "
+      if arg.typ.isNil and arg.kind notin {nkStmtList, nkDo}:
+        # XXX we really need to 'tryExpr' here!
+        arg = c.semOperand(c, n[i][1])
+        n[i].typ = arg.typ
+        n[i][1] = arg
+    else:
+      if arg.typ.isNil and arg.kind notin {nkStmtList, nkDo, nkElse,
+                                           nkOfBranch, nkElifBranch,
+                                           nkExceptBranch}:
+        arg = c.semOperand(c, n[i])
+        n[i] = arg
+    if arg.typ != nil and arg.typ.kind == tyError: return
+    result.add argTypeToString(arg, prefer)
     if i != n.len - 1: result.add ", "
 
 proc concreteType(c: TCandidate, t: PType; f: PType = nil): PType =
@@ -379,16 +373,14 @@ proc concreteType(c: TCandidate, t: PType; f: PType = nil): PType =
   else:
     result = t                # Note: empty is valid here
 
-proc handleRange(c: PContext, f, a: PType, min, max: TTypeKind): TTypeRelation =
+proc handleRange(f, a: PType, min, max: TTypeKind): TTypeRelation =
   if a.kind == f.kind:
     result = isEqual
   else:
     let ab = skipTypes(a, {tyRange})
     let k = ab.kind
-    let nf = c.config.normalizeKind(f.kind)
-    let na = c.config.normalizeKind(k)
     if k == f.kind: result = isSubrange
-    elif k == tyInt and f.kind in {tyRange, tyInt..tyInt64,
+    elif k == tyInt and f.kind in {tyRange, tyInt8..tyInt64,
                                    tyUInt..tyUInt64} and
         isIntLit(ab) and getInt(ab.n) >= firstOrd(nil, f) and
                          getInt(ab.n) <= lastOrd(nil, f):
@@ -397,13 +389,9 @@ proc handleRange(c: PContext, f, a: PType, min, max: TTypeKind): TTypeRelation =
       # integer literal in the proper range; we want ``i16 + 4`` to stay an
       # ``int16`` operation so we declare the ``4`` pseudo-equal to int16
       result = isFromIntLit
-    elif a.kind == tyInt and nf == c.config.targetSizeSignedToKind:
+    elif f.kind == tyInt and k in {tyInt8..tyInt32}:
       result = isIntConv
-    elif a.kind == tyUInt and nf == c.config.targetSizeUnsignedToKind:
-      result = isIntConv
-    elif f.kind == tyInt and na in {tyInt8 .. pred(c.config.targetSizeSignedToKind)}:
-      result = isIntConv
-    elif f.kind == tyUInt and na in {tyUInt8 .. pred(c.config.targetSizeUnsignedToKind)}:
+    elif f.kind == tyUInt and k in {tyUInt8..tyUInt32}:
       result = isIntConv
     elif k >= min and k <= max:
       result = isConvertible
@@ -417,7 +405,7 @@ proc handleRange(c: PContext, f, a: PType, min, max: TTypeKind): TTypeRelation =
       result = isConvertible
     else: result = isNone
 
-proc isConvertibleToRange(c: PContext, f, a: PType): bool =
+proc isConvertibleToRange(f, a: PType): bool =
   if f.kind in {tyInt..tyInt64, tyUInt..tyUInt64} and
      a.kind in {tyInt..tyInt64, tyUInt..tyUInt64}:
     case f.kind
@@ -425,14 +413,13 @@ proc isConvertibleToRange(c: PContext, f, a: PType): bool =
     of tyInt16: result = isIntLit(a) or a.kind in {tyInt8, tyInt16}
     of tyInt32: result = isIntLit(a) or a.kind in {tyInt8, tyInt16, tyInt32}
     # This is wrong, but seems like there's a lot of code that relies on it :(
-    of tyInt, tyUInt: result = true
-    # of tyInt: result = isIntLit(a) or a.kind in {tyInt8 .. c.config.targetSizeSignedToKind}
+    of tyInt, tyUInt, tyUInt64: result = true
     of tyInt64: result = isIntLit(a) or a.kind in {tyInt8, tyInt16, tyInt32, tyInt, tyInt64}
     of tyUInt8: result = isIntLit(a) or a.kind in {tyUInt8}
     of tyUInt16: result = isIntLit(a) or a.kind in {tyUInt8, tyUInt16}
     of tyUInt32: result = isIntLit(a) or a.kind in {tyUInt8, tyUInt16, tyUInt32}
-    # of tyUInt: result = isIntLit(a) or a.kind in {tyUInt8 .. c.config.targetSizeUnsignedToKind}
-    of tyUInt64: result = isIntLit(a) or a.kind in {tyUInt8, tyUInt16, tyUInt32, tyUInt64}
+    #of tyUInt: result = isIntLit(a) or a.kind in {tyUInt8, tyUInt16, tyUInt32, tyUInt}
+    #of tyUInt64: result = isIntLit(a) or a.kind in {tyUInt8, tyUInt16, tyUInt32, tyUInt, tyUInt64}
     else: result = false
   elif f.kind in {tyFloat..tyFloat128}:
     # `isIntLit` is correct and should be used above as well, see PR:
@@ -624,9 +611,6 @@ proc procTypeRel(c: var TCandidate, f, a: PType): TTypeRelation =
     if f.len != a.len: return
     result = isEqual      # start with maximum; also correct for no
                           # params at all
-    
-    if f.flags * {tfIterator} != a.flags * {tfIterator}:
-      return isNone
 
     template checkParam(f, a) =
       result = minRel(result, procParamTypeRel(c, f, a))
@@ -1164,18 +1148,18 @@ proc typeRel(c: var TCandidate, f, aOrig: PType,
       let f = skipTypes(f, {tyRange})
       if f.kind == a.kind and (f.kind != tyEnum or sameEnumTypes(f, a)):
         result = isIntConv
-      elif isConvertibleToRange(c.c, f, a):
+      elif isConvertibleToRange(f, a):
         result = isConvertible  # a convertible to f
-  of tyInt:      result = handleRange(c.c, f, a, tyInt8, c.c.config.targetSizeSignedToKind)
-  of tyInt8:     result = handleRange(c.c, f, a, tyInt8, tyInt8)
-  of tyInt16:    result = handleRange(c.c, f, a, tyInt8, tyInt16)
-  of tyInt32:    result = handleRange(c.c, f, a, tyInt8, tyInt32)
-  of tyInt64:    result = handleRange(c.c, f, a, tyInt, tyInt64)
-  of tyUInt:     result = handleRange(c.c, f, a, tyUInt8, c.c.config.targetSizeUnsignedToKind)
-  of tyUInt8:    result = handleRange(c.c, f, a, tyUInt8, tyUInt8)
-  of tyUInt16:   result = handleRange(c.c, f, a, tyUInt8, tyUInt16)
-  of tyUInt32:   result = handleRange(c.c, f, a, tyUInt8, tyUInt32)
-  of tyUInt64:   result = handleRange(c.c, f, a, tyUInt, tyUInt64)
+  of tyInt:      result = handleRange(f, a, tyInt8, tyInt32)
+  of tyInt8:     result = handleRange(f, a, tyInt8, tyInt8)
+  of tyInt16:    result = handleRange(f, a, tyInt8, tyInt16)
+  of tyInt32:    result = handleRange(f, a, tyInt8, tyInt32)
+  of tyInt64:    result = handleRange(f, a, tyInt, tyInt64)
+  of tyUInt:     result = handleRange(f, a, tyUInt8, tyUInt32)
+  of tyUInt8:    result = handleRange(f, a, tyUInt8, tyUInt8)
+  of tyUInt16:   result = handleRange(f, a, tyUInt8, tyUInt16)
+  of tyUInt32:   result = handleRange(f, a, tyUInt8, tyUInt32)
+  of tyUInt64:   result = handleRange(f, a, tyUInt, tyUInt64)
   of tyFloat:    result = handleFloatRange(f, a)
   of tyFloat32:  result = handleFloatRange(f, a)
   of tyFloat64:  result = handleFloatRange(f, a)
@@ -1298,7 +1282,7 @@ proc typeRel(c: var TCandidate, f, aOrig: PType,
     of tyNil: result = isNone
     else: discard
   of tyOrdinal:
-    if isOrdinalType(a):
+    if isOrdinalType(a, allowEnumWithHoles = optNimV1Emulation in c.c.config.globalOptions):
       var x = if a.kind == tyOrdinal: a[0] else: a
       if f[0].kind == tyNone:
         result = isGeneric
@@ -1639,19 +1623,13 @@ proc typeRel(c: var TCandidate, f, aOrig: PType,
 
   of tyBuiltInTypeClass:
     considerPreviousT:
-      let target = f[0]
-      let targetKind = target.kind
+      let targetKind = f[0].kind
       let effectiveArgType = a.skipTypes({tyRange, tyGenericInst,
                                           tyBuiltInTypeClass, tyAlias, tySink, tyOwned})
-      if targetKind == effectiveArgType.kind:
-        if effectiveArgType.isEmptyContainer:
-          return isNone
-        if targetKind == tyProc:
-          if target.flags * {tfIterator} != effectiveArgType.flags * {tfIterator}:
-            return isNone
-          if tfExplicitCallConv in target.flags and
-              target.callConv != effectiveArgType.callConv:
-            return isNone
+      let typeClassMatches = targetKind == effectiveArgType.kind and
+                             not effectiveArgType.isEmptyContainer
+      if typeClassMatches or
+        (targetKind in {tyProc, tyPointer} and effectiveArgType.kind == tyNil):
         put(c, f, a)
         return isGeneric
       else:
@@ -1849,9 +1827,7 @@ proc typeRel(c: var TCandidate, f, aOrig: PType,
       elif f.base.kind == tyNone:
         result = isGeneric
       else:
-        let r = typeRel(c, f.base, a.base, flags)
-        if r >= isIntConv:
-          result = r
+        result = typeRel(c, f.base, a.base, flags)
 
       if result != isNone:
         put(c, f, a)
@@ -1859,9 +1835,7 @@ proc typeRel(c: var TCandidate, f, aOrig: PType,
       if tfUnresolved in f.flags:
         result = typeRel(c, prev.base, a, flags)
       elif a.kind == tyTypeDesc:
-        let r = typeRel(c, prev.base, a.base, flags)
-        if r >= isIntConv:
-          result = r
+        result = typeRel(c, prev.base, a.base, flags)
       else:
         result = isNone
 
@@ -1938,16 +1912,13 @@ proc implicitConv(kind: TNodeKind, f: PType, arg: PNode, m: TCandidate,
   result.add c.graph.emptyNode
   result.add arg
 
-proc isLValue(c: PContext; n: PNode, isOutParam = false): bool {.inline.} =
+proc isLValue(c: PContext; n: PNode): bool {.inline.} =
   let aa = isAssignable(nil, n)
   case aa
   of arLValue, arLocalLValue, arStrange:
     result = true
   of arDiscriminant:
     result = c.inUncheckedAssignSection > 0
-  of arAddressableConst:
-    let sym = getRoot(n)
-    result = strictDefs in c.features and sym != nil and sym.kind == skLet and isOutParam
   else:
     result = false
 
@@ -1988,7 +1959,7 @@ proc userConvMatch(c: PContext, m: var TCandidate, f, a: PType,
       if srca == isSubtype:
         param = implicitConv(nkHiddenSubConv, src, copyTree(arg), m, c)
       elif src.kind in {tyVar}:
-        # Analyse the converter return type.
+        # Analyse the converter return type
         param = newNodeIT(nkHiddenAddr, arg.info, s.typ[1])
         param.add copyTree(arg)
       else:
@@ -2074,7 +2045,7 @@ proc paramTypesMatchAux(m: var TCandidate, f, a: PType,
          a.n == nil and
          tfGenericTypeParam notin a.flags:
         return newNodeIT(nkType, argOrig.info, makeTypeFromExpr(c, arg))
-    elif arg.kind != nkEmpty:
+    else:
       var evaluated = c.semTryConstExpr(c, arg)
       if evaluated != nil:
         # Don't build the type in-place because `evaluated` and `arg` may point
@@ -2138,15 +2109,11 @@ proc paramTypesMatchAux(m: var TCandidate, f, a: PType,
 
   case r
   of isConvertible:
-    if f.skipTypes({tyRange}).kind in {tyInt, tyUInt}:
-      inc(m.convMatches)
     inc(m.convMatches)
     result = implicitConv(nkHiddenStdConv, f, arg, m, c)
   of isIntConv:
     # I'm too lazy to introduce another ``*matches`` field, so we conflate
     # ``isIntConv`` and ``isIntLit`` here:
-    if f.skipTypes({tyRange}).kind notin {tyInt, tyUInt}:
-      inc(m.intConvMatches)
     inc(m.intConvMatches)
     result = implicitConv(nkHiddenStdConv, f, arg, m, c)
   of isSubtype:
@@ -2169,8 +2136,6 @@ proc paramTypesMatchAux(m: var TCandidate, f, a: PType,
     elif arg.sym.kind in {skMacro, skTemplate}:
       return nil
     else:
-      if arg.sym.ast == nil:
-        return nil
       let inferred = c.semGenerateInstance(c, arg.sym, m.bindings, arg.info)
       result = newSymNode(inferred, arg.info)
     if r == isInferredConvertible:
@@ -2213,8 +2178,6 @@ proc paramTypesMatchAux(m: var TCandidate, f, a: PType,
       return arg
     elif a.kind == tyVoid and f.matchesVoidProc and argOrig.kind == nkStmtList:
       # lift do blocks without params to lambdas
-      # now deprecated
-      message(c.config, argOrig.info, warnStmtListLambda)
       let p = c.graph
       let lifted = c.semExpr(c, newProcNode(nkDo, argOrig.info, body = argOrig,
           params = nkFormalParams.newTree(p.emptyNode), name = p.emptyNode, pattern = p.emptyNode,
@@ -2420,7 +2383,7 @@ proc matchesAux(c: PContext, n, nOrig: PNode, m: var TCandidate, marker: var Int
         if argConverter.typ.kind notin {tyVar}:
           m.firstMismatch.kind = kVarNeeded
           noMatch()
-      elif not (isLValue(c, n, isOutParam(formal.typ))):
+      elif not isLValue(c, n):
         m.firstMismatch.kind = kVarNeeded
         noMatch()
 
@@ -2680,9 +2643,11 @@ proc argtypeMatches*(c: PContext, f, a: PType, fromHlo = false): bool =
     # pattern templates do not allow for conversions except from int literal
     res != nil and m.convMatches == 0 and m.intConvMatches in [0, 256]
 
+when not defined(nimHasSinkInference):
+  {.pragma: nosinks.}
 
 proc instTypeBoundOp*(c: PContext; dc: PSym; t: PType; info: TLineInfo;
-                      op: TTypeAttachedOp; col: int): PSym =
+                      op: TTypeAttachedOp; col: int): PSym {.nosinks.} =
   var m = newCandidate(c, dc.typ)
   if col >= dc.typ.len:
     localError(c.config, info, "cannot instantiate: '" & dc.name.s & "'")

compiler/sinkparameter_inference.nim

@@ -19,6 +19,7 @@ proc checkForSink*(config: ConfigRef; idgen: IdGenerator; owner: PSym; arg: PNod
     var local = p # sink parameter?
     passToSink(local)
   ]#
+  if optSinkInference notin config.options: return
   case arg.kind
   of nkSym:
     if arg.sym.kind == skParam and

compiler/sourcemap.nim

@@ -1,207 +1,383 @@
-import std/[strutils, strscans, parseutils, assertions]
+import os, strformat, strutils, tables, sets, ropes, json, algorithm
 
 type
-  Segment = object
-    ## Segment refers to a block of something in the JS output.
-    ## This could be a token or an entire line
-    original: int # Column in the Nim source
-    generated: int # Column in the generated JS
-    name: int # Index into names list (-1 for no name)
+  SourceNode* = ref object
+    line*:      int
+    column*:    int
+    source*:    string
+    name*:      string
+    children*:  seq[Child]
 
-  Mapping = object
-    ## Mapping refers to a line in the JS output.
-    ## It is made up of segments which refer to the tokens in the line
-    case inSource: bool # Whether the line in JS has Nim equivilant
-    of true:
-      file: int # Index into files list
-      line: int # 0 indexed line of code in the Nim source
-      segments: seq[Segment]
-    else: discard
+  C = enum cSourceNode, cSourceString
 
-  SourceInfo = object
-    mappings: seq[Mapping]
-    names, files: seq[string]
+  Child* = ref object
+    case kind*: C:
+    of cSourceNode:
+      node*:  SourceNode
+    of cSourceString:
+      s*:     string
 
-  SourceMap* = object
+  SourceMap* = ref object
     version*:   int
     sources*:   seq[string]
     names*:     seq[string]
     mappings*:  string
     file*:      string
+    # sourceRoot*: string
+    # sourcesContent*: string
 
-func addSegment(info: var SourceInfo, original, generated: int, name: string = "") {.raises: [].} =
-  ## Adds a new segment into the current line
-  assert info.mappings.len > 0, "No lines have been added yet"
-  var segment = Segment(original: original, generated: generated, name: -1)
-  if name != "":
-    # Make name be index into names list
-    segment.name = info.names.find(name)
-    if segment.name == -1:
-      segment.name = info.names.len
-      info.names &= name
+  SourceMapGenerator = ref object
+    file:           string
+    sourceRoot:     string
+    skipValidation: bool
+    sources:        seq[string]
+    names:          seq[string]
+    mappings:       seq[Mapping]
 
-  assert info.mappings[^1].inSource, "Current line isn't in Nim source"
-  info.mappings[^1].segments &= segment
+  Mapping* = ref object
+    source*:        string
+    original*:      tuple[line: int, column: int]
+    generated*:     tuple[line: int, column: int]
+    name*:          string
+    noSource*:      bool
+    noName*:        bool
 
-func newLine(info: var SourceInfo) {.raises: [].} =
-  ## Add new mapping which doesn't appear in the Nim source
-  info.mappings &= Mapping(inSource: false)
 
-func newLine(info: var SourceInfo, file: string, line: int) {.raises: [].} =
-  ## Starts a new line in the mappings. Call addSegment after this to add
-  ## segments into the line
-  var mapping = Mapping(inSource: true, line: line)
-  # Set file to file position. Add in if needed
-  mapping.file = info.files.find(file)
-  if mapping.file == -1:
-    mapping.file = info.files.len
-    info.files &= file
-  info.mappings &= mapping
+proc child*(s: string): Child =
+  Child(kind: cSourceString, s: s)
+
+
+proc child*(node: SourceNode): Child =
+  Child(kind: cSourceNode, node: node)
+
+
+proc newSourceNode(line: int, column: int, path: string, node: SourceNode, name: string = ""): SourceNode =
+  SourceNode(line: line, column: column, source: path, name: name, children: @[child(node)])
+
+
+proc newSourceNode(line: int, column: int, path: string, s: string, name: string = ""): SourceNode =
+  SourceNode(line: line, column: column, source: path, name: name, children: @[child(s)])
+
+
+proc newSourceNode(line: int, column: int, path: string, children: seq[Child], name: string = ""): SourceNode =
+  SourceNode(line: line, column: column, source: path, name: name, children: children)
+
+
+
+
+# debugging
+
+
+proc text*(sourceNode: SourceNode, depth: int): string =
+  let empty = "  "
+  result = &"{repeat(empty, depth)}SourceNode({sourceNode.source}:{sourceNode.line}:{sourceNode.column}):\n"
+  for child in sourceNode.children:
+    if child.kind == cSourceString:
+      result.add(&"{repeat(empty, depth + 1)}{child.s}\n")
+    else:
+      result.add(child.node.text(depth + 1))
+
+
+proc `$`*(sourceNode: SourceNode): string = text(sourceNode, 0)
 
 
 # base64_VLQ
-func encode*(values: seq[int]): string {.raises: [].} =
-  ## Encodes a series of integers into a VLQ base64 encoded string
-  # References:
-  #   - https://www.lucidchart.com/techblog/2019/08/22/decode-encoding-base64-vlqs-source-maps/
-  #   - https://github.com/rails/sprockets/blob/main/guides/source_maps.md#source-map-file
-  const
-    alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
-    shift = 5
-    continueBit = 1 shl 5
-    mask = continueBit - 1
-  for val in values:
-    # Sign is stored in first bit
-    var newVal = abs(val) shl 1
-    if val < 0:
-      newVal = newVal or 1
-    # Now comes the variable length part
-    # This is how we are able to store large numbers
-    while true:
-      # We only encode 5 bits.
-      var masked = newVal and mask
-      newVal = newVal shr shift
-      # If there is still something left
-      # then signify with the continue bit that the
-      # decoder should keep decoding
-      if newVal > 0:
-        masked = masked or continueBit
-      result &= alphabet[masked]
-      # If the value is zero then we have nothing left to encode
-      if newVal == 0:
-        break
 
-iterator tokenize*(line: string): (int, string) =
-  ## Goes through a line and splits it into Nim identifiers and
-  ## normal JS code. This allows us to map mangled names back to Nim names.
-  ## Yields (column, name). Doesn't yield anything but identifiers.
-  ## See mangleName in compiler/jsgen.nim for how name mangling is done
-  var
-    col = 0
-    token = ""
-  while col < line.len:
-    var
-      token: string
-      name: string
-    # First we find the next identifier
-    col += line.skipWhitespace(col)
-    col += line.skipUntil(IdentStartChars, col)
-    let identStart = col
-    col += line.parseIdent(token, col)
-    # Idents will either be originalName_randomInt or HEXhexCode_randomInt
-    if token.startsWith("HEX"):
-      var hex: int
-      # 3 = "HEX".len and we only want to parse the two integers after it
-      discard token[3 ..< 5].parseHex(hex)
-      name = $chr(hex)
-    elif not token.endsWith("_Idx"): # Ignore address indexes
-      # It might be in the form originalName_randomInt
-      let lastUnderscore = token.rfind('_')
-      if lastUnderscore != -1:
-        name = token[0..<lastUnderscore]
-    if name != "":
-      yield (identStart, name)
 
-func parse*(source: string): SourceInfo =
-  ## Parses the JS output for embedded line info
-  ## So it can convert those into a series of mappings
-  var
-    skipFirstLine = true
-    currColumn = 0
-    currLine = 0
-    currFile = ""
-  # Add each line as a node into the output
-  for line in source.splitLines():
-    var
-      lineNumber: int
-      linePath: string
-      column: int
-    if line.strip().scanf("/* line $i:$i \"$+\" */", lineNumber, column, linePath):
-      # When we reach the first line mappinsegmentg then we can assume
-      # we can map the rest of the JS lines to Nim lines
-      currColumn = column # Column is already zero indexed
-      currLine = lineNumber - 1
-      currFile = linePath
-      # Lines are zero indexed
-      result.newLine(currFile, currLine)
-      # Skip whitespace to find the starting column
-      result.addSegment(currColumn, line.skipWhitespace())
-    elif currFile != "":
-      result.newLine(currFile, currLine)
-      # There mightn't be any tokens so add a starting segment
-      result.addSegment(currColumn, line.skipWhitespace())
-      for jsColumn, token in line.tokenize:
-        result.addSegment(currColumn, jsColumn, token)
+let integers = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/="
+
+
+proc encode*(i: int): string =
+  result = ""
+  var n = i
+  if n < 0:
+    n = (-n shl 1) or 1
+  else:
+    n = n shl 1
+
+  var z = 0
+  while z == 0 or n > 0:
+    var e = n and 31
+    n = n shr 5
+    if n > 0:
+      e = e or 32
+
+    result.add(integers[e])
+    z += 1
+
+
+type TokenState = enum Normal, String, Ident, Mangled
+
+iterator tokenize*(line: string): (bool, string) =
+  # result = @[]
+  var state = Normal
+  var token = ""
+  var isMangled = false
+  for z, ch in line:
+    if ch.isAlphaAscii:
+      if state == Normal:
+        state = Ident
+        if token.len > 0:
+          yield (isMangled, token)
+        token = $ch
+        isMangled = false
+      else:
+        token.add(ch)
+    elif ch == '_':
+      if state == Ident:
+        state = Mangled
+        isMangled = true
+      token.add($ch)
+    elif ch != '"' and not ch.isAlphaNumeric:
+      if state in {Ident, Mangled}:
+        state = Normal
+        if token.len > 0:
+          yield (isMangled, token)
+        token = $ch
+        isMangled = false
+      else:
+        token.add($ch)
+    elif ch == '"':
+      if state != String:
+        state = String
+        if token.len > 0:
+          yield (isMangled, token)
+        token = $ch
+        isMangled = false
+      else:
+        state = Normal
+        token.add($ch)
+        if token.len > 0:
+          yield (isMangled, token)
+        isMangled = false
+        token = ""
     else:
-      result.newLine()
+      token.add($ch)
+  if token.len > 0:
+    yield (isMangled, token)
 
-func toSourceMap*(info: SourceInfo, file: string): SourceMap {.raises: [].} =
-  ## Convert from high level SourceInfo into the required SourceMap object
-  # Add basic info
-  result.version = 3
-  result.file = file
-  result.sources = info.files
-  result.names = info.names
-  # Convert nodes into mappings.
-  # Mappings are split into blocks where each block referes to a line in the outputted JS.
-  # Blocks can be separated into statements which refere to tokens on the line.
-  # Since the mappings depend on previous values we need to
-  # keep track of previous file, name, etc
-  var
-    prevFile = 0
-    prevLine = 0
-    prevName = 0
-    prevNimCol = 0
+proc parse*(source: string, path: string): SourceNode =
+  let lines = source.splitLines()
+  var lastLocation: SourceNode = nil
+  result = newSourceNode(0, 0, path, @[])
+    
+  # we just use one single parent and add all nim lines
+  # as its children, I guess in typical codegen
+  # that happens recursively on ast level
+  # we also don't have column info, but I doubt more one nim lines can compile to one js
+  # maybe in macros?
 
-  for mapping in info.mappings:
-    # We know need to encode segments with the following fields
-    # All these fields are relative to their previous values
-    # - 0: Column in generated code
-    # - 1: Index of Nim file in source list
-    # - 2: Line in Nim source
-    # - 3: Column in Nim source
-    # - 4: Index in names list
-    if mapping.inSource:
-      # JS Column is special in that it is reset after every line
-      var prevJSCol = 0
-      for segment in mapping.segments:
-        var values = @[segment.generated - prevJSCol, mapping.file - prevFile, mapping.line - prevLine, segment.original - prevNimCol]
-        # Add name field if needed
-        if segment.name != -1:
-          values &= segment.name - prevName
-          prevName = segment.name
-        prevJSCol = segment.generated
-        prevNimCol = segment.original
-        prevFile = mapping.file
-        prevLine = mapping.line
-        result.mappings &= encode(values) & ","
-      # Remove trailing ,
-      if mapping.segments.len > 0:
-        result.mappings.setLen(result.mappings.len - 1)
+  for i, originalLine in lines:
+    let line = originalLine.strip
+    if line.len == 0:
+      continue
+      
+    # this shouldn't be a problem:
+    # jsgen doesn't generate comments
+    # and if you emit // line you probably know what you're doing
+    if line.startsWith("// line"):
+      if result.children.len > 0:
+        result.children[^1].node.children.add(child(line & "\n"))
+      let pos = line.find(" ", 8)
+      let lineNumber = line[8 .. pos - 1].parseInt
+      let linePath = line[pos + 2 .. ^2] # quotes
+      
+      lastLocation = newSourceNode(
+        lineNumber,
+        0,
+        linePath,
+        @[])
+      result.children.add(child(lastLocation))
+    else:
+      var last: SourceNode
+      for token in line.tokenize():
+        var name = ""
+        if token[0]:
+          name = token[1].split('_', 1)[0]
+        
+        
+        if result.children.len > 0:
+          result.children[^1].node.children.add(
+            child(
+              newSourceNode(
+                result.children[^1].node.line,
+                0,
+                result.children[^1].node.source,
+                token[1],
+                name)))
+          last = result.children[^1].node.children[^1].node
+        else:
+          result.children.add(
+            child(
+              newSourceNode(i + 1, 0, path, token[1], name)))
+          last = result.children[^1].node
+      let nl = "\n"
+      if not last.isNil:
+        last.source.add(nl)
 
-    result.mappings &= ";"
+proc cmp(a: Mapping, b: Mapping): int =
+  var c = cmp(a.generated, b.generated)
+  if c != 0:
+    return c
 
-proc genSourceMap*(source: string, outFile: string): SourceMap =
-  let node = parse(source)
-  result = node.toSourceMap(outFile)
+  c = cmp(a.source, b.source)
+  if c != 0:
+    return c
+
+  c = cmp(a.original, b.original)
+  if c != 0:
+    return c
+
+  return cmp(a.name, b.name)
+
+
+proc index*[T](elements: seq[T], element: T): int =
+  for z in 0 ..< elements.len:
+    if elements[z] == element:
+      return z
+  return -1
+
+
+proc serializeMappings(map: SourceMapGenerator, mappings: seq[Mapping]): string =
+  var previous = Mapping(generated: (line: 1, column: 0), original: (line: 0, column: 0), name: "", source: "")
+  var previousSourceId = 0
+  var previousNameId = 0
+  var next = ""
+  var nameId = 0
+  var sourceId = 0
+  result = ""
+
+  for z, mapping in mappings:
+    next = ""
+
+    if mapping.generated.line != previous.generated.line:
+      previous.generated.column = 0
+
+      while mapping.generated.line != previous.generated.line:
+        next.add(";")
+        previous.generated.line += 1
+
+    else:
+      if z > 0:
+        if cmp(mapping, mappings[z - 1]) == 0:
+          continue
+        next.add(",")
+
+    next.add(encode(mapping.generated.column - previous.generated.column))
+    previous.generated.column = mapping.generated.column
+
+    if not mapping.noSource and mapping.source.len > 0:
+      sourceId = map.sources.index(mapping.source)
+      next.add(encode(sourceId - previousSourceId))
+      previousSourceId = sourceId
+      next.add(encode(mapping.original.line - 1 - previous.original.line))
+      previous.original.line = mapping.original.line - 1
+      next.add(encode(mapping.original.column - previous.original.column))
+      previous.original.column = mapping.original.column
+
+      if not mapping.noName and mapping.name.len > 0:
+        nameId = map.names.index(mapping.name)
+        next.add(encode(nameId - previousNameId))
+        previousNameId = nameId
+
+    result.add(next)
+
+
+proc gen*(map: SourceMapGenerator): SourceMap =
+  var mappings = map.mappings.sorted do (a: Mapping, b: Mapping) -> int:
+    cmp(a, b)
+  result = SourceMap(
+    file: map.file,
+    version: 3,
+    sources: map.sources[0..^1],
+    names: map.names[0..^1],
+    mappings: map.serializeMappings(mappings))
+
+
+
+proc addMapping*(map: SourceMapGenerator, mapping: Mapping) =
+  if not mapping.noSource and mapping.source notin map.sources:
+    map.sources.add(mapping.source)
+
+  if not mapping.noName and mapping.name.len > 0 and mapping.name notin map.names:
+    map.names.add(mapping.name)
+
+  # echo "map ", mapping.source, " ", mapping.original, " ", mapping.generated, " ", mapping.name
+  map.mappings.add(mapping)
+
+
+proc walk*(node: SourceNode, fn: proc(line: string, original: SourceNode)) =
+  for child in node.children:
+    if child.kind == cSourceString and child.s.len > 0:
+      fn(child.s, node)
+    else:
+      child.node.walk(fn)
+
+
+proc toSourceMap*(node: SourceNode, file: string): SourceMapGenerator =
+  var map = SourceMapGenerator(file: file, sources: @[], names: @[], mappings: @[])
+
+  var generated = (line: 1, column: 0)
+  var sourceMappingActive = false
+  var lastOriginal = SourceNode(source: "", line: -1, column: 0, name: "", children: @[])
+
+  node.walk do (line: string, original: SourceNode):
+    if original.source.endsWith(".js"):
+      # ignore it
+      discard
+    else:
+      if original.line != -1:
+        if lastOriginal.source != original.source or
+           lastOriginal.line != original.line or
+           lastOriginal.column != original.column or
+           lastOriginal.name != original.name:
+          map.addMapping(
+            Mapping(
+              source: original.source,
+              original: (line: original.line, column: original.column),
+              generated: (line: generated.line, column: generated.column),
+              name: original.name))
+
+        lastOriginal = SourceNode(
+          source: original.source,
+          line: original.line,
+          column: original.column,
+          name: original.name,
+          children: lastOriginal.children)
+        sourceMappingActive = true
+      elif sourceMappingActive:
+        map.addMapping(
+          Mapping(
+            noSource: true,
+            noName: true,
+            generated: (line: generated.line, column: generated.column),
+            original: (line: -1, column: -1)))
+        lastOriginal.line = -1
+        sourceMappingActive = false
+
+    for z in 0 ..< line.len:
+      if line[z] in Newlines:
+        generated.line += 1
+        generated.column = 0
+
+        if z == line.len - 1:
+          lastOriginal.line = -1
+          sourceMappingActive = false
+        elif sourceMappingActive:
+          map.addMapping(
+            Mapping(
+              source: original.source,
+              original: (line: original.line, column: original.column),
+              generated: (line: generated.line, column: generated.column),
+              name: original.name))
+      else:
+        generated.column += 1
+    
+  map
+
+
+proc genSourceMap*(source: string, outFile: string): (Rope, SourceMap) =
+  let node = parse(source, outFile)
+  let map = node.toSourceMap(file = outFile)
+  ((&"{source}\n//# sourceMappingURL={outFile}.map").rope, map.gen)
 

compiler/suggest.nim

@@ -36,7 +36,7 @@ import algorithm, sets, prefixmatches, parseutils, tables
 from wordrecg import wDeprecated, wError, wAddr, wYield
 
 when defined(nimsuggest):
-  import tables, pathutils # importer
+  import passes, tables, pathutils # importer
 
 const
   sep = '\t'
@@ -120,9 +120,7 @@ proc getTokenLenFromSource(conf: ConfigRef; ident: string; info: TLineInfo): int
 proc symToSuggest*(g: ModuleGraph; s: PSym, isLocal: bool, section: IdeCmd, info: TLineInfo;
                   quality: range[0..100]; prefix: PrefixMatch;
                   inTypeContext: bool; scope: int;
-                  useSuppliedInfo = false,
-                  endLine: uint16 = 0,
-                  endCol = 0): Suggest =
+                  useSuppliedInfo = false): Suggest =
   new(result)
   result.section = section
   result.quality = quality
@@ -178,8 +176,6 @@ proc symToSuggest*(g: ModuleGraph; s: PSym, isLocal: bool, section: IdeCmd, info
                       else:
                         getTokenLenFromSource(g.config, s.name.s, infox)
   result.version = g.config.suggestVersion
-  result.endLine = endLine
-  result.endCol = endCol
 
 proc `$`*(suggest: Suggest): string =
   result = $suggest.section
@@ -220,12 +216,6 @@ proc `$`*(suggest: Suggest): string =
         result.add(sep)
         result.add($suggest.prefix)
 
-  if (suggest.version == 3 and suggest.section in {ideOutline, ideExpand}):
-    result.add(sep)
-    result.add($suggest.endLine)
-    result.add(sep)
-    result.add($suggest.endCol)
-
 proc suggestResult*(conf: ConfigRef; s: Suggest) =
   if not isNil(conf.suggestionResultHook):
     conf.suggestionResultHook(s)
@@ -299,7 +289,7 @@ proc suggestField(c: PContext, s: PSym; f: PNode; info: TLineInfo; outputs: var
                               s.getQuality, pm, c.inTypeContext > 0, 0))
 
 template wholeSymTab(cond, section: untyped) {.dirty.} =
-  for (item, scopeN, isLocal) in uniqueSyms(c):
+  for (item, scopeN, isLocal) in allSyms(c):
     let it = item
     var pm: PrefixMatch
     if cond:
@@ -372,7 +362,7 @@ proc suggestOperations(c: PContext, n, f: PNode, typ: PType, outputs: var Sugges
 
 proc suggestEverything(c: PContext, n, f: PNode, outputs: var Suggestions) =
   # do not produce too many symbols:
-  for (it, scopeN, isLocal) in uniqueSyms(c):
+  for (it, scopeN, isLocal) in allSyms(c):
     var pm: PrefixMatch
     if filterSym(it, f, pm):
       outputs.add(symToSuggest(c.graph, it, isLocal = isLocal, ideSug, n.info,
@@ -690,7 +680,7 @@ proc suggestSentinel*(c: PContext) =
   inc(c.compilesContextId)
   var outputs: Suggestions = @[]
   # suggest everything:
-  for (it, scopeN, isLocal) in uniqueSyms(c):
+  for (it, scopeN, isLocal) in allSyms(c):
     var pm: PrefixMatch
     if filterSymNoOpr(it, nil, pm):
       outputs.add(symToSuggest(c.graph, it, isLocal = isLocal, ideSug,

compiler/transf.nim

@@ -31,7 +31,7 @@ type
   TransformBodyFlag* = enum
     dontUseCache, useCache
 
-proc transformBody*(g: ModuleGraph; idgen: IdGenerator, prc: PSym, flag: TransformBodyFlag, force = false): PNode
+proc transformBody*(g: ModuleGraph; idgen: IdGenerator, prc: PSym, flag: TransformBodyFlag): PNode
 
 import closureiters, lambdalifting
 
@@ -50,7 +50,6 @@ type
     module: PSym
     transCon: PTransCon      # top of a TransCon stack
     inlining: int            # > 0 if we are in inlining context (copy vars)
-    isIntroducingNewLocalVars: bool  # true if we are in `introducingNewLocalVars` (don't transform yields)
     contSyms, breakSyms: seq[PSym]  # to transform 'continue' and 'break'
     deferDetected, tooEarly: bool
     graph: ModuleGraph
@@ -69,6 +68,7 @@ proc newTransNode(kind: TNodeKind, n: PNode,
                   sons: int): PNode {.inline.} =
   var x = newNodeIT(kind, n.info, n.typ)
   newSeq(x.sons, sons)
+  x.typ = n.typ
 #  x.flags = n.flags
   result = x
 
@@ -451,9 +451,7 @@ proc transformYield(c: PTransf, n: PNode): PNode =
     result.add(c.transCon.forLoopBody)
   else:
     # we need to introduce new local variables:
-    c.isIntroducingNewLocalVars = true # don't transform yields when introducing new local vars
     result.add(introduceNewLocalVars(c, c.transCon.forLoopBody))
-    c.isIntroducingNewLocalVars = false
 
   for idx in 0 ..< result.len:
     var changeNode = result[idx]
@@ -1039,7 +1037,7 @@ proc transform(c: PTransf, n: PNode): PNode =
     else:
       result = transformSons(c, n)
   of nkYieldStmt:
-    if c.inlining > 0 and not c.isIntroducingNewLocalVars:
+    if c.inlining > 0:
       result = transformYield(c, n)
     else:
       result = transformSons(c, n)
@@ -1148,7 +1146,7 @@ template liftDefer(c, root) =
   if c.deferDetected:
     liftDeferAux(root)
 
-proc transformBody*(g: ModuleGraph; idgen: IdGenerator; prc: PSym; flag: TransformBodyFlag, force = false): PNode =
+proc transformBody*(g: ModuleGraph; idgen: IdGenerator; prc: PSym; flag: TransformBodyFlag): PNode =
   assert prc.kind in routineKinds
 
   if prc.transformedBody != nil:
@@ -1158,7 +1156,7 @@ proc transformBody*(g: ModuleGraph; idgen: IdGenerator; prc: PSym; flag: Transfo
   else:
     prc.transformedBody = newNode(nkEmpty) # protects from recursion
     var c = openTransf(g, prc.getModule, "", idgen)
-    result = liftLambdas(g, prc, getBody(g, prc), c.tooEarly, c.idgen, force)
+    result = liftLambdas(g, prc, getBody(g, prc), c.tooEarly, c.idgen)
     result = processTransf(c, result, prc)
     liftDefer(c, result)
     result = liftLocalsIfRequested(prc, result, g.cache, g.config, c.idgen)

compiler/types.nim

@@ -753,13 +753,8 @@ proc firstOrd*(conf: ConfigRef; t: PType): Int128 =
     assert(t.n.kind == nkRange)
     result = getOrdValue(t.n[0])
   of tyInt:
-    if conf != nil:
-      case conf.target.intSize
-      of 8: result = toInt128(0x8000000000000000'i64)
-      of 4: result = toInt128(-2147483648)
-      of 2: result = toInt128(-32768)
-      of 1: result = toInt128(-128)
-      else: discard
+    if conf != nil and conf.target.intSize == 4:
+      result = toInt128(-2147483648)
     else:
       result = toInt128(0x8000000000000000'i64)
   of tyInt8: result =  toInt128(-128)
@@ -802,29 +797,6 @@ proc firstFloat*(t: PType): BiggestFloat =
     internalError(newPartialConfigRef(), "invalid kind for firstFloat(" & $t.kind & ')')
     NaN
 
-proc targetSizeSignedToKind*(conf: ConfigRef): TTypeKind =
-  case conf.target.intSize
-  of 8: result = tyInt64
-  of 4: result = tyInt32
-  of 2: result = tyInt16
-  else: discard
-
-proc targetSizeUnsignedToKind*(conf: ConfigRef): TTypeKind =
-  case conf.target.intSize
-  of 8: result = tyUInt64
-  of 4: result = tyUInt32
-  of 2: result = tyUInt16
-  else: discard
-
-proc normalizeKind*(conf: ConfigRef, k: TTypeKind): TTypeKind =
-  case k
-  of tyInt:
-    result = conf.targetSizeSignedToKind()
-  of tyUInt:
-    result = conf.targetSizeUnsignedToKind()
-  else:
-    result = k
-
 proc lastOrd*(conf: ConfigRef; t: PType): Int128 =
   case t.kind
   of tyBool: result = toInt128(1'u)
@@ -836,13 +808,7 @@ proc lastOrd*(conf: ConfigRef; t: PType): Int128 =
     assert(t.n.kind == nkRange)
     result = getOrdValue(t.n[1])
   of tyInt:
-    if conf != nil:
-      case conf.target.intSize
-      of 8: result = toInt128(0x7FFFFFFFFFFFFFFF'u64)
-      of 4: result = toInt128(0x7FFFFFFF)
-      of 2: result = toInt128(0x00007FFF)
-      of 1: result = toInt128(0x0000007F)
-      else: discard
+    if conf != nil and conf.target.intSize == 4: result = toInt128(0x7FFFFFFF)
     else: result = toInt128(0x7FFFFFFFFFFFFFFF'u64)
   of tyInt8: result = toInt128(0x0000007F)
   of tyInt16: result = toInt128(0x00007FFF)
@@ -1686,7 +1652,7 @@ proc isTupleRecursive(t: PType, cycleDetector: var IntSet): bool =
   of tyTuple:
     var cycleDetectorCopy: IntSet
     for i in 0..<t.len:
-      cycleDetectorCopy = cycleDetector
+      assign(cycleDetectorCopy, cycleDetector)
       if isTupleRecursive(t[i], cycleDetectorCopy):
         return true
   of tyAlias, tyRef, tyPtr, tyGenericInst, tyVar, tyLent, tySink,
@@ -1721,18 +1687,6 @@ proc isDefectException*(t: PType): bool =
     t = skipTypes(t[0], abstractPtrs)
   return false
 
-proc isDefectOrCatchableError*(t: PType): bool =
-  var t = t.skipTypes(abstractPtrs)
-  while t.kind == tyObject:
-    if t.sym != nil and t.sym.owner != nil and
-        sfSystemModule in t.sym.owner.flags and
-        (t.sym.name.s == "Defect" or
-        t.sym.name.s == "CatchableError"):
-      return true
-    if t[0] == nil: break
-    t = skipTypes(t[0], abstractPtrs)
-  return false
-
 proc isSinkTypeForParam*(t: PType): bool =
   # a parameter like 'seq[owned T]' must not be used only once, but its
   # elements must, so we detect this case here:

compiler/typesrenderer.nim

@@ -64,8 +64,9 @@ proc renderType(n: PNode, toNormalize: bool): string =
       result = "ptr"
   of nkProcTy:
     assert n.len != 1
-    if n.len > 1 and n[0].kind == nkFormalParams:
+    if n.len > 1:
       let params = n[0]
+      assert params.kind == nkFormalParams
       assert params.len > 0
       result = "proc("
       for i in 1..<params.len: result.add(renderType(params[i], toNormalize) & ',')

compiler/vm.nim

@@ -10,10 +10,10 @@
 ## This file implements the new evaluation engine for Nim code.
 ## An instruction is 1-3 int32s in memory, it is a register based VM.
 
-import semmacrosanity
+
 import
   std/[strutils, tables, parseutils],
-  msgs, vmdef, vmgen, nimsets, types,
+  msgs, vmdef, vmgen, nimsets, types, passes,
   parser, vmdeps, idents, trees, renderer, options, transf,
   gorgeimpl, lineinfos, btrees, macrocacheimpl,
   modulegraphs, sighashes, int128, vmprofiler
@@ -32,6 +32,8 @@ const
 when hasFFI:
   import evalffi
 
+when not defined(nimHasCursor):
+  {.pragma: cursor.}
 
 proc stackTraceAux(c: PCtx; x: PStackFrame; pc: int; recursionLimit=100) =
   if x != nil:
@@ -531,6 +533,9 @@ template maybeHandlePtr(node2: PNode, reg: TFullReg, isAssign2: bool): bool =
   else:
     false
 
+when not defined(nimHasSinkInference):
+  {.pragma: nosinks.}
+
 template takeAddress(reg, source) =
   reg.nodeAddr = addr source
   GC_ref source
@@ -1271,7 +1276,7 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
             let ast = a.sym.ast.shallowCopy
             for i in 0..<a.sym.ast.len:
               ast[i] = a.sym.ast[i]
-            ast[bodyPos] = transformBody(c.graph, c.idgen, a.sym, useCache, force=true)
+            ast[bodyPos] = transformBody(c.graph, c.idgen, a.sym, useCache)
             ast.copyTree()
     of opcSymOwner:
       decodeB(rkNode)
@@ -1351,7 +1356,7 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
       let prc = if not isClosure: bb.sym else: bb[0].sym
       if prc.offset < -1:
         # it's a callback:
-        c.callbacks[-prc.offset-2](
+        c.callbacks[-prc.offset-2].value(
           VmArgs(ra: ra, rb: rb, rc: rc, slots: cast[ptr UncheckedArray[TFullReg]](addr regs[0]),
                  currentException: c.currentExceptionA,
                  currentLineInfo: c.debug[pc])
@@ -1408,9 +1413,6 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
         for i in 1..rc-1:
           let node = regs[rb+i].regToNode
           node.info = c.debug[pc]
-          if prc.typ[i].kind notin {tyTyped, tyUntyped}:
-            node.annotateType(prc.typ[i], c.config)
-
           macroCall.add(node)
         var a = evalTemplate(macroCall, prc, genSymOwner, c.config, c.cache, c.templInstCounter, c.idgen)
         if a.kind == nkStmtList and a.len == 1: a = a[0]
@@ -1606,7 +1608,7 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
     of opcRepr:
       decodeB(rkNode)
       createStr regs[ra]
-      regs[ra].node.strVal = renderTree(regs[rb].regToNode, {renderNoComments, renderDocComments, renderNonExportedFields})
+      regs[ra].node.strVal = renderTree(regs[rb].regToNode, {renderNoComments, renderDocComments})
     of opcQuit:
       if c.mode in {emRepl, emStaticExpr, emStaticStmt}:
         message(c.config, c.debug[pc], hintQuitCalled)
@@ -1677,7 +1679,7 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
         rb = instr.regB
         rc = instr.regC
         idx = int(regs[rb+rc-1].intVal)
-        callback = c.callbacks[idx]
+        callback = c.callbacks[idx].value
         args = VmArgs(ra: ra, rb: rb, rc: rc, slots: cast[ptr UncheckedArray[TFullReg]](addr regs[0]),
                 currentException: c.currentExceptionA,
                 currentLineInfo: c.debug[pc])
@@ -1855,7 +1857,7 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
       if regs[rb].node.kind != nkSym:
         stackTrace(c, tos, pc, "node is not a symbol")
       else:
-        regs[ra].node.strVal = $sigHash(regs[rb].node.sym, c.config)
+        regs[ra].node.strVal = $sigHash(regs[rb].node.sym)
     of opcSlurp:
       decodeB(rkNode)
       createStr regs[ra]
@@ -1893,7 +1895,7 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
       var error: string
       let ast = parseString(regs[rb].node.strVal, c.cache, c.config,
                             regs[rc].node.strVal, 0,
-                            proc (conf: ConfigRef; info: TLineInfo; msg: TMsgKind; arg: string) =
+                            proc (conf: ConfigRef; info: TLineInfo; msg: TMsgKind; arg: string) {.nosinks.} =
                               if error.len == 0 and msg <= errMax:
                                 error = formatMsg(conf, info, msg, arg))
       if error.len > 0:
@@ -1908,7 +1910,7 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
       var error: string
       let ast = parseString(regs[rb].node.strVal, c.cache, c.config,
                             regs[rc].node.strVal, 0,
-                            proc (conf: ConfigRef; info: TLineInfo; msg: TMsgKind; arg: string) =
+                            proc (conf: ConfigRef; info: TLineInfo; msg: TMsgKind; arg: string) {.nosinks.} =
                               if error.len == 0 and msg <= errMax:
                                 error = formatMsg(conf, info, msg, arg))
       if error.len > 0:
@@ -1932,24 +1934,14 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
       of 1: # getLine
         regs[ra].node = newIntNode(nkIntLit, n.info.line.int)
       of 2: # getColumn
-        regs[ra].node = newIntNode(nkIntLit, n.info.col.int)
+        regs[ra].node = newIntNode(nkIntLit, n.info.col)
       else:
         internalAssert c.config, false
       regs[ra].node.info = n.info
       regs[ra].node.typ = n.typ
-    of opcNCopyLineInfo:
+    of opcNSetLineInfo:
       decodeB(rkNode)
       regs[ra].node.info = regs[rb].node.info
-    of opcNSetLineInfoLine:
-      decodeB(rkNode)
-      regs[ra].node.info.line = regs[rb].intVal.uint16
-    of opcNSetLineInfoColumn:
-      decodeB(rkNode)
-      regs[ra].node.info.col = regs[rb].intVal.int16
-    of opcNSetLineInfoFile:
-      decodeB(rkNode)
-      regs[ra].node.info.fileIndex =
-        fileInfoIdx(c.config, RelativeFile regs[rb].node.strVal)
     of opcEqIdent:
       decodeBC(rkInt)
       # aliases for shorter and easier to understand code below
@@ -2074,6 +2066,12 @@ proc rawExecute(c: PCtx, start: int, tos: PStackFrame): TFullReg =
         dest.ident = regs[rb].node.ident
       else:
         stackTrace(c, tos, pc, errFieldXNotFound & "ident")
+    of opcNSetType:
+      decodeB(rkNode)
+      let b = regs[rb].node
+      internalAssert c.config, b.kind == nkSym and b.sym.kind == skType
+      internalAssert c.config, regs[ra].node != nil
+      regs[ra].node.typ = b.sym.typ
     of opcNSetStrVal:
       decodeB(rkNode)
       var dest = regs[ra].node
@@ -2309,7 +2307,7 @@ proc setupGlobalCtx*(module: PSym; graph: ModuleGraph; idgen: IdGenerator) =
   else:
     refresh(PCtx graph.vm, module, idgen)
 
-proc setupEvalGen*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext =
+proc myOpen(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassContext {.nosinks.} =
   #var c = newEvalContext(module, emRepl)
   #c.features = {allowCast, allowInfiniteLoops}
   #pushStackFrame(c, newStackFrame())
@@ -2318,7 +2316,7 @@ proc setupEvalGen*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PPassC
   setupGlobalCtx(module, graph, idgen)
   result = PCtx graph.vm
 
-proc interpreterCode*(c: PPassContext, n: PNode): PNode =
+proc myProcess(c: PPassContext, n: PNode): PNode =
   let c = PCtx(c)
   # don't eval errornous code:
   if c.oldErrorCount == c.config.errorCounter:
@@ -2328,12 +2326,14 @@ proc interpreterCode*(c: PPassContext, n: PNode): PNode =
     result = n
   c.oldErrorCount = c.config.errorCounter
 
+proc myClose(graph: ModuleGraph; c: PPassContext, n: PNode): PNode =
+  result = myProcess(c, n)
+
+const evalPass* = makePass(myOpen, myProcess, myClose)
+
 proc evalConstExprAux(module: PSym; idgen: IdGenerator;
                       g: ModuleGraph; prc: PSym, n: PNode,
                       mode: TEvalMode): PNode =
-  when defined(nimsuggest):
-    if g.config.expandDone():
-      return n
   #if g.config.errorCounter > 0: return n
   let n = transformExpr(g, idgen, module, n)
   setupGlobalCtx(module, g, idgen)

compiler/vmconv.nim

@@ -1,6 +1,4 @@
-import ast, idents, lineinfos, astalgo
-import vmdef
-import std/times
+import ast
 
 template elementType*(T: typedesc): typedesc =
   typeof(block:
@@ -45,12 +43,3 @@ proc toLit*[T](a: T): PNode =
   else:
     static: doAssert false, "not yet supported: " & $T # add as needed
 
-proc toTimeLit*(a: Time, c: PCtx, obj: PNode, info: TLineInfo): PNode =
-  # probably refactor it into `toLit` in the future
-  result = newTree(nkObjConstr)
-  result.add(newNode(nkEmpty)) # can be changed to a symbol according to PType
-  for k, ai in fieldPairs(a):
-    let reti = newNode(nkExprColonExpr)
-    reti.add newSymNode(lookupInRecord(obj, getIdent(c.cache, k)), info)
-    reti.add ai.toLit
-    result.add reti

compiler/vmdef.nim

@@ -10,7 +10,7 @@
 ## This module contains the type definitions for the new evaluation engine.
 ## An instruction is 1-3 int32s in memory, it is a register based VM.
 
-import std/[tables, strutils]
+import tables
 
 import ast, idents, options, modulegraphs, lineinfos
 
@@ -127,7 +127,7 @@ type
     opcNGetSize,
 
     opcNSetIntVal,
-    opcNSetFloatVal, opcNSetSymbol, opcNSetIdent, opcNSetStrVal,
+    opcNSetFloatVal, opcNSetSymbol, opcNSetIdent, opcNSetType, opcNSetStrVal,
     opcNNewNimNode, opcNCopyNimNode, opcNCopyNimTree, opcNDel, opcGenSym,
 
     opcNccValue, opcNccInc, opcNcsAdd, opcNcsIncl, opcNcsLen, opcNcsAt,
@@ -141,8 +141,7 @@ type
     opcNError,
     opcNWarning,
     opcNHint,
-    opcNGetLineInfo, opcNCopyLineInfo, opcNSetLineInfoLine,
-    opcNSetLineInfoColumn, opcNSetLineInfoFile
+    opcNGetLineInfo, opcNSetLineInfo,
     opcEqIdent,
     opcStrToIdent,
     opcGetImpl,
@@ -259,8 +258,7 @@ type
     traceActive*: bool
     loopIterations*: int
     comesFromHeuristic*: TLineInfo # Heuristic for better macro stack traces
-    callbacks*: seq[VmCallback]
-    callbackIndex*: Table[string, int]
+    callbacks*: seq[tuple[key: string, value: VmCallback]]
     errorFlag*: string
     cache*: IdentCache
     config*: ConfigRef
@@ -293,7 +291,7 @@ proc newCtx*(module: PSym; cache: IdentCache; g: ModuleGraph; idgen: IdGenerator
   PCtx(code: @[], debug: @[],
     globals: newNode(nkStmtListExpr), constants: newNode(nkStmtList), types: @[],
     prc: PProc(blocks: @[]), module: module, loopIterations: g.config.maxLoopIterationsVM,
-    comesFromHeuristic: unknownLineInfo, callbacks: @[], callbackIndex: initTable[string, int](), errorFlag: "",
+    comesFromHeuristic: unknownLineInfo, callbacks: @[], errorFlag: "",
     cache: cache, config: g.config, graph: g, idgen: idgen)
 
 proc refresh*(c: PCtx, module: PSym; idgen: IdGenerator) =
@@ -302,18 +300,9 @@ proc refresh*(c: PCtx, module: PSym; idgen: IdGenerator) =
   c.loopIterations = c.config.maxLoopIterationsVM
   c.idgen = idgen
 
-proc reverseName(s: string): string =
-  result = newStringOfCap(s.len)
-  let y = s.split('.')
-  for i in 1..y.len:
-    result.add y[^i]
-    if i != y.len:
-      result.add '.'
-
 proc registerCallback*(c: PCtx; name: string; callback: VmCallback): int {.discardable.} =
   result = c.callbacks.len
-  c.callbacks.add(callback)
-  c.callbackIndex[reverseName(name)] = result
+  c.callbacks.add((name, callback))
 
 const
   firstABxInstr* = opcTJmp

compiler/vmgen.nim

@@ -32,7 +32,7 @@ when defined(nimPreviewSlimSystem):
   import std/assertions
 
 import
-  strutils, ast, types, msgs, renderer, vmdef, trees,
+  strutils, ast, types, msgs, renderer, vmdef,
   intsets, magicsys, options, lowerings, lineinfos, transf, astmsgs
 
 from modulegraphs import getBody
@@ -1313,6 +1313,9 @@ proc genMagic(c: PCtx; n: PNode; dest: var TDest; m: TMagic) =
   of mNSetIdent:
     unused(c, n, dest)
     genBinaryStmt(c, n, opcNSetIdent)
+  of mNSetType:
+    unused(c, n, dest)
+    genBinaryStmt(c, n, opcNSetType)
   of mNSetStrVal:
     unused(c, n, dest)
     genBinaryStmt(c, n, opcNSetStrVal)
@@ -1332,19 +1335,7 @@ proc genMagic(c: PCtx; n: PNode; dest: var TDest; m: TMagic) =
     of "copyLineInfo":
       internalAssert c.config, n.len == 3
       unused(c, n, dest)
-      genBinaryStmt(c, n, opcNCopyLineInfo)
-    of "setLine":
-      internalAssert c.config, n.len == 3
-      unused(c, n, dest)
-      genBinaryStmt(c, n, opcNSetLineInfoLine)
-    of "setColumn":
-      internalAssert c.config, n.len == 3
-      unused(c, n, dest)
-      genBinaryStmt(c, n, opcNSetLineInfoColumn)
-    of "setFile":
-      internalAssert c.config, n.len == 3
-      unused(c, n, dest)
-      genBinaryStmt(c, n, opcNSetLineInfoFile)
+      genBinaryStmt(c, n, opcNSetLineInfo)
     else: internalAssert c.config, false
   of mNHint:
     unused(c, n, dest)
@@ -1829,7 +1820,7 @@ proc getNullValueAux(t: PType; obj: PNode, result: PNode; conf: ConfigRef; currP
     let field = newNodeI(nkExprColonExpr, result.info)
     field.add(obj)
     let value = getNullValue(obj.sym.typ, result.info, conf)
-    value.flags.incl nfSkipFieldChecking
+    value.flags.incl nfUseDefaultField
     field.add(value)
     result.add field
     doAssert obj.sym.position == currPosition
@@ -1909,19 +1900,6 @@ proc genVarSection(c: PCtx; n: PNode) =
           c.genAdditionalCopy(a[2], opcWrDeref, tmp, 0, val)
           c.freeTemp(val)
           c.freeTemp(tmp)
-        elif not importcCondVar(s) and not (s.typ.kind == tyProc and s.typ.callConv == ccClosure) and
-                sfPure notin s.flags: # fixes #10938
-          # there is a pre-existing issue with closure types in VM
-          # if `(var s: proc () = default(proc ()); doAssert s == nil)` works for you;
-          # you might remove the second condition.
-          # the problem is that closure types are tuples in VM, but the types of its children
-          # shouldn't have the same type as closure types.
-          let tmp = c.genx(a[0], {gfNodeAddr})
-          let sa = getNullValue(s.typ, a.info, c.config)
-          let val = c.genx(sa)
-          c.genAdditionalCopy(sa, opcWrDeref, tmp, 0, val)
-          c.freeTemp(val)
-          c.freeTemp(tmp)
       else:
         setSlot(c, s)
         if a[2].kind == nkEmpty:
@@ -2024,29 +2002,25 @@ proc genTupleConstr(c: PCtx, n: PNode, dest: var TDest) =
 
 proc genProc*(c: PCtx; s: PSym): int
 
-proc toKey(s: PSym): string =
+proc matches(s: PSym; x: string): bool =
+  let y = x.split('.')
   var s = s
-  while s != nil:
-    result.add s.name.s
-    if s.owner != nil:
-      if sfFromGeneric in s.flags:
-        s = s.owner.owner
-      else:
-        s = s.owner
-      result.add "."
-    else:
-      break
+  for i in 1..y.len:
+    if s == nil or (y[^i].cmpIgnoreStyle(s.name.s) != 0 and y[^i] != "*"):
+      return false
+    s = if sfFromGeneric in s.flags: s.owner.owner else: s.owner
+    while s != nil and s.kind == skPackage and s.owner != nil: s = s.owner
+  result = true
 
 proc procIsCallback(c: PCtx; s: PSym): bool =
   if s.offset < -1: return true
-  let key = toKey(s)
-  if c.callbackIndex.contains(key):
-    let index = c.callbackIndex[key]
-    doAssert s.offset == -1
-    s.offset = -2 - index
-    result = true
-  else:
-    result = false
+  var i = -2
+  for key, value in items(c.callbacks):
+    if s.matches(key):
+      doAssert s.offset == -1
+      s.offset = i
+      return true
+    dec i
 
 proc gen(c: PCtx; n: PNode; dest: var TDest; flags: TGenFlags = {}) =
   when defined(nimCompilerStacktraceHints):
@@ -2067,10 +2041,7 @@ proc gen(c: PCtx; n: PNode; dest: var TDest; flags: TGenFlags = {}) =
       genLit(c, n, dest)
     of skConst:
       let constVal = if s.astdef != nil: s.astdef else: s.typ.n
-      if dontInlineConstant(n, constVal):
-        genLit(c, constVal, dest)
-      else:
-        gen(c, constVal, dest)
+      gen(c, constVal, dest)
     of skEnumField:
       # we never reach this case - as of the time of this comment,
       # skEnumField is folded to an int in semfold.nim, but this code

compiler/vmops.nim

@@ -36,8 +36,7 @@ from std/osproc import nil
 when defined(nimPreviewSlimSystem):
   import std/syncio
 else:
-  from std/formatfloat import addFloatRoundtrip, addFloatSprintf
-
+  from std/formatfloat import addFloatRoundtrip, addFloatSprintf 
 
 # There are some useful procs in vmconv.
 import vmconv, vmmarshal
@@ -142,40 +141,39 @@ proc staticWalkDirImpl(path: string, relative: bool): PNode =
   for k, f in walkDir(path, relative):
     result.add toLit((k, f))
 
-from std / compilesettings import SingleValueSetting, MultipleValueSetting
+when defined(nimHasInvariant):
+  from std / compilesettings import SingleValueSetting, MultipleValueSetting
 
-proc querySettingImpl(conf: ConfigRef, switch: BiggestInt): string =
-  {.push warning[Deprecated]:off.}
-  case SingleValueSetting(switch)
-  of arguments: result = conf.arguments
-  of outFile: result = conf.outFile.string
-  of outDir: result = conf.outDir.string
-  of nimcacheDir: result = conf.getNimcacheDir().string
-  of projectName: result = conf.projectName
-  of projectPath: result = conf.projectPath.string
-  of projectFull: result = conf.projectFull.string
-  of command: result = conf.command
-  of commandLine: result = conf.commandLine
-  of linkOptions: result = conf.linkOptions
-  of compileOptions: result = conf.compileOptions
-  of ccompilerPath: result = conf.cCompilerPath
-  of backend: result = $conf.backend
-  of libPath: result = conf.libpath.string
-  of gc: result = $conf.selectedGC
-  of mm: result = $conf.selectedGC
-  {.pop.}
+  proc querySettingImpl(conf: ConfigRef, switch: BiggestInt): string =
+    case SingleValueSetting(switch)
+    of arguments: result = conf.arguments
+    of outFile: result = conf.outFile.string
+    of outDir: result = conf.outDir.string
+    of nimcacheDir: result = conf.getNimcacheDir().string
+    of projectName: result = conf.projectName
+    of projectPath: result = conf.projectPath.string
+    of projectFull: result = conf.projectFull.string
+    of command: result = conf.command
+    of commandLine: result = conf.commandLine
+    of linkOptions: result = conf.linkOptions
+    of compileOptions: result = conf.compileOptions
+    of ccompilerPath: result = conf.cCompilerPath
+    of backend: result = $conf.backend
+    of libPath: result = conf.libpath.string
+    of gc: result = $conf.selectedGC
+    of mm: result = $conf.selectedGC
 
-proc querySettingSeqImpl(conf: ConfigRef, switch: BiggestInt): seq[string] =
-  template copySeq(field: untyped): untyped =
-    for i in field: result.add i.string
+  proc querySettingSeqImpl(conf: ConfigRef, switch: BiggestInt): seq[string] =
+    template copySeq(field: untyped): untyped =
+      for i in field: result.add i.string
 
-  case MultipleValueSetting(switch)
-  of nimblePaths: copySeq(conf.nimblePaths)
-  of searchPaths: copySeq(conf.searchPaths)
-  of lazyPaths: copySeq(conf.lazyPaths)
-  of commandArgs: result = conf.commandArgs
-  of cincludes: copySeq(conf.cIncludes)
-  of clibs: copySeq(conf.cLibs)
+    case MultipleValueSetting(switch)
+    of nimblePaths: copySeq(conf.nimblePaths)
+    of searchPaths: copySeq(conf.searchPaths)
+    of lazyPaths: copySeq(conf.lazyPaths)
+    of commandArgs: result = conf.commandArgs
+    of cincludes: copySeq(conf.cIncludes)
+    of clibs: copySeq(conf.cLibs)
 
 proc stackTrace2(c: PCtx, msg: string, n: PNode) =
   stackTrace(c, PStackFrame(prc: c.prc.sym, comesFrom: 0, next: nil), c.exceptionInstr, msg, n.info)
@@ -255,12 +253,13 @@ proc registerAdditionalOps*(c: PCtx) =
     wrap2si(readLines, ioop)
     systemop getCurrentExceptionMsg
     systemop getCurrentException
-    registerCallback c, "stdlib.osdirs.staticWalkDir", proc (a: VmArgs) {.nimcall.} =
+    registerCallback c, "stdlib.*.staticWalkDir", proc (a: VmArgs) {.nimcall.} =
       setResult(a, staticWalkDirImpl(getString(a, 0), getBool(a, 1)))
-    registerCallback c, "stdlib.compilesettings.querySetting", proc (a: VmArgs) =
-      setResult(a, querySettingImpl(c.config, getInt(a, 0)))
-    registerCallback c, "stdlib.compilesettings.querySettingSeq", proc (a: VmArgs) =
-      setResult(a, querySettingSeqImpl(c.config, getInt(a, 0)))
+    when defined(nimHasInvariant):
+      registerCallback c, "stdlib.compilesettings.querySetting", proc (a: VmArgs) =
+        setResult(a, querySettingImpl(c.config, getInt(a, 0)))
+      registerCallback c, "stdlib.compilesettings.querySettingSeq", proc (a: VmArgs) =
+        setResult(a, querySettingSeqImpl(c.config, getInt(a, 0)))
 
     if defined(nimsuggest) or c.config.cmd == cmdCheck:
       discard "don't run staticExec for 'nim suggest'"
@@ -283,12 +282,6 @@ proc registerAdditionalOps*(c: PCtx) =
       stackTrace2(c, "isExported() requires a symbol. '$#' is of kind '$#'" % [$n, $n.kind], n)
     setResult(a, sfExported in n.sym.flags)
 
-  registerCallback c, "stdlib.macrocache.hasKey", proc (a: VmArgs) =
-    let
-      table = getString(a, 0)
-      key = getString(a, 1)
-    setResult(a, table in c.graph.cacheTables and key in c.graph.cacheTables[table])
-
   registerCallback c, "stdlib.vmutils.vmTrace", proc (a: VmArgs) =
     c.config.isVmTrace = getBool(a, 0)
 
@@ -335,9 +328,8 @@ proc registerAdditionalOps*(c: PCtx) =
     registerCallback c, "stdlib.osproc.execCmdEx", proc (a: VmArgs) {.nimcall.} =
       let options = getNode(a, 1).fromLit(set[osproc.ProcessOption])
       a.setResult osproc.execCmdEx(getString(a, 0), options).toLit
-    registerCallback c, "stdlib.times.getTimeImpl", proc (a: VmArgs) =
-      let obj = a.getNode(0).typ.n
-      setResult(a, times.getTime().toTimeLit(c, obj, a.currentLineInfo))
+    registerCallback c, "stdlib.times.getTime", proc (a: VmArgs) {.nimcall.} =
+      setResult(a, times.getTime().toLit)
 
   proc getEffectList(c: PCtx; a: VmArgs; effectIndex: int) =
     let fn = getNode(a, 0)
@@ -381,10 +373,6 @@ proc registerAdditionalOps*(c: PCtx) =
     let x = a.getFloat(1)
     addFloatSprintf(p.strVal, x)
 
-  registerCallback c, "stdlib.strutils.formatBiggestFloat", proc(a: VmArgs) =
-    setResult(a, formatBiggestFloat(a.getFloat(0), FloatFormatMode(a.getInt(1)),
-                                    a.getInt(2), chr(a.getInt(3))))
-
   wrapIterator("stdlib.envvars.envPairsImplSeq"): envPairs()
 
   registerCallback c, "stdlib.marshal.toVM", proc(a: VmArgs) =

compiler/wordrecg.nim

@@ -130,7 +130,20 @@ const
     wFor, wIf, wReturn, wStatic, wTemplate, wTry, wWhile, wUsing}
 
 
-from std/enumutils import genEnumCaseStmt
-from strutils import normalize
-proc findStr*[T: enum](a, b: static[T], s: string, default: T): T =
-  genEnumCaseStmt(T, s, default, ord(a), ord(b), normalize)
+const enumUtilsExist = compiles:
+  import std/enumutils
+
+when enumUtilsExist:
+  from std/enumutils import genEnumCaseStmt
+  from strutils import normalize
+  proc findStr*[T: enum](a, b: static[T], s: string, default: T): T =
+    genEnumCaseStmt(T, s, default, ord(a), ord(b), normalize)
+
+else:
+  from strutils import cmpIgnoreStyle
+  proc findStr*[T: enum](a, b: static[T], s: string, default: T): T {.deprecated.} =
+    # used for compiler bootstrapping only
+    for i in a..b:
+      if cmpIgnoreStyle($i, s) == 0:
+        return i
+    result = default

config/build_config.txt

@@ -1,5 +1,5 @@
 nim_comment="key-value pairs for windows/posix bootstrapping build scripts"
-nim_csourcesDir=csources_v2
-nim_csourcesUrl=https://github.com/nim-lang/csources_v2.git
+nim_csourcesDir=csources_v1
+nim_csourcesUrl=https://github.com/nim-lang/csources_v1.git
 nim_csourcesBranch=master
-nim_csourcesHash=86742fb02c6606ab01a532a0085784effb2e753e
+nim_csourcesHash=561b417c65791cd8356b5f73620914ceff845d10

config/nim.cfg

@@ -39,7 +39,7 @@ arm64.linux.gcc.linkerexe = "aarch64-linux-gnu-gcc"
 riscv32.linux.gcc.exe = "riscv64-linux-gnu-gcc"
 riscv32.linux.gcc.linkerexe = "riscv64-linux-gnu-gcc"
 riscv64.linux.gcc.exe = "riscv64-linux-gnu-gcc"
-riscv64.linux.gcc.linkerexe = "riscv64-linux-gnu-gcc"
+riscv64.linux.gcc.linkerexe = "arm-linux-gnueabihf-gcc"
 
 # For OpenWRT, you will also need to adjust PATH to point to your toolchain.
 mips.linux.gcc.exe = "mips-openwrt-linux-gcc"
@@ -100,11 +100,6 @@ nimblepath="$home/.nimble/pkgs/"
   gcc.options.always %= "${gcc.options.always} -fsanitize=null -fsanitize-undefined-trap-on-error"
 @end
 
-# Turn off threads support when compiling for bare-metal targets (--os:any)
-@if any:
-  threads:off
-@end
-
 @if unix and mingw:
   # Cross compile for Windows from Linux/OSX using MinGW
   i386.windows.gcc.exe = "i686-w64-mingw32-gcc"
@@ -186,9 +181,10 @@ nimblepath="$home/.nimble/pkgs/"
 
 gcc.maxerrorsimpl = "-fmax-errors=3"
 
-@if freebsd or netbsd:
+@if freebsd:
   tlsEmulation:off
 @elif bsd:
+  # at least NetBSD has problems with thread local storage:
   tlsEmulation:on
 @end
 

copying.txt

@@ -1,7 +1,7 @@
 =====================================================
 Nim -- a Compiler for Nim. https://nim-lang.org/
 
-Copyright (C) 2006-2023 Andreas Rumpf. All rights reserved.
+Copyright (C) 2006-2022 Andreas Rumpf. All rights reserved.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

doc/advopt.txt

@@ -4,11 +4,8 @@ Advanced commands:
   //compileToOC, objc       compile project to Objective C code
   //js                      compile project to Javascript
   //e                       run a Nimscript file
-  //md2html                 convert a Markdown file to HTML
-                            use `--docCmd:skip` to skip compiling snippets
   //rst2html                convert a reStructuredText file to HTML
                             use `--docCmd:skip` to skip compiling snippets
-  //md2tex                  convert a Markdown file to LaTeX
   //rst2tex                 convert a reStructuredText file to LaTeX
   //doc2tex                 extract the documentation to a LaTeX file
   //jsondoc                 extract the documentation to a json file
@@ -133,9 +130,7 @@ Advanced options:
                             select which memory management to use; default is 'orc'
   --exceptions:setjmp|cpp|goto|quirky
                             select the exception handling implementation
-  --index:on|off|only       docgen: turn index file generation? (`only` means
-                            not generate output files like HTML)
-  --noImportdoc:on|off      disable loading documentation ``.idx`` files?
+  --index:on|off            turn index file generation on|off
   --putenv:key=value        set an environment variable
   --NimblePath:PATH         add a path for Nimble support
   --noNimblePath            deactivate the Nimble path
@@ -169,10 +164,9 @@ Advanced options:
                             enable experimental language feature
   --legacy:$2
                             enable obsolete/legacy language feature
+  --useVersion:1.0|1.2|1.6  emulate Nim version X of the Nim compiler, for testing
   --benchmarkVM:on|off      turn benchmarking of VM code with cpuTime() on|off
   --profileVM:on|off        turn compile time VM profiler on|off
-  --sinkInference:on|off    turn sink parameter inference on|off (default: off)
+  --sinkInference:on|off    turn sink parameter inference on|off (default: on)
   --panics:on|off           turn panics into process terminations (default: off)
   --deepcopy:on|off         enable 'system.deepCopy' for ``--mm:arc|orc``
-  --jsbigint64:on|off       toggle the use of [BigInt](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt)
-                            for 64-bit integers on the JavaScript backend (default: on)

doc/astspec.txt

@@ -1348,7 +1348,7 @@ Generic parameters are treated in the type, not the ``proc`` itself.
 Concrete syntax:
 
   ```nim
-  type MyProc[T] = proc(x: T) {.nimcall.}
+  type MyProc[T] = proc(x: T)
   ```
 
 AST:
@@ -1363,8 +1363,7 @@ AST:
     nnkProcTy( # behaves like a procedure declaration from here on
       nnkFormalParams(
         # ...
-      ),
-      nnkPragma(nnkIdent("nimcall"))
+      )
     )
   )
   ```
@@ -1372,37 +1371,6 @@ AST:
 The same syntax applies to ``iterator`` (with ``nnkIteratorTy``), but
 *does not* apply to ``converter`` or ``template``.
 
-Type class versions of these nodes generally share the same node kind but
-without any child nodes. The ``tuple`` type class is represented by
-``nnkTupleClassTy``, while a ``proc`` or ``iterator`` type class with pragmas
-has an ``nnkEmpty`` node in place of the ``nnkFormalParams`` node of a
-concrete ``proc`` or ``iterator`` type node.
-
-  ```nim
-  type TypeClass = proc {.nimcall.} | ref | tuple
-  ```
-
-AST:
-
-  ```nim
-  nnkTypeDef(
-    nnkIdent("TypeClass"),
-    nnkEmpty(),
-    nnkInfix(
-      nnkIdent("|"),
-      nnkProcTy(
-        nnkEmpty(),
-        nnkPragma(nnkIdent("nimcall"))
-      ),
-      nnkInfix(
-        nnkIdent("|"),
-        nnkRefTy(),
-        nnkTupleClassTy()
-      )
-    )
-  )
-  ```
-
 Mixin statement
 ---------------
 

doc/docgen.md

@@ -9,7 +9,6 @@
 .. include:: rstcommon.rst
 .. contents::
 
-.. importdoc:: markdown_rst.md, compiler/docgen.nim
 
 Introduction
 ============
@@ -44,7 +43,6 @@ Generate HTML documentation for a whole project:
   # or `$nimcache/htmldocs` with `--usenimcache` which avoids clobbering your sources;
   # and likewise without `--project`.
   # Adding `-r` will open in a browser directly.
-  # Use `--showNonExports` to show non-exported fields of an exported type.
   ```
 
 Documentation Comments
@@ -105,64 +103,6 @@ won't influence RST formatting.
      ## Paragraph.
      ```
 
-Structuring output directories
-------------------------------
-
-Basic directory for output is set by `--outdir:OUTDIR`:option: switch,
-by default `OUTDIR` is ``htmldocs`` sub-directory in the directory of
-the processed file.
-
-There are 2 basic options as to how generated HTML output files are stored:
-
-1) complex hierarchy when docgen-compiling with `--project`:option:,
-   which follows directory structure of the project itself.
-   So `nim doc`:cmd: replicates project's directory structure
-   inside `--outdir:OUTDIR`:option: directory.
-   `--project`:option: is well suited for projects that have 1 main module.
-   File name clashes are impossible in this case.
-
-2) flattened structure, where user-provided script goes through all
-   needed input files and calls commands like `nim doc`:cmd:
-   with `--outdir:OUTDIR`:option: switch, thus putting all HTML (and
-   ``.idx``) files into 1 directory.
-
-   .. Important:: Make sure that you don't have files with same base name
-     like ``x.nim`` and ``x.md`` in the same package, otherwise you'll
-     have name conflict for ``x.html``.
-
-   .. Tip:: To structure your output directories and avoid file name
-     clashes you can split your project into
-     different *packages* -- parts of your repository that are
-     docgen-compiled with different `--outdir:OUTDIR`:option: options.
-
-     An example of such strategy is Nim repository itself which has:
-
-     * its stdlib ``.nim`` files from different directories and ``.md``
-       documentation from ``doc/`` directory are all docgen-compiled
-       into `--outdir:web/upload/<version>/`:option: directory
-     * its ``.nim`` files from ``compiler/`` directory are docgen-compiled
-       into `--outdir:web/upload/<version>/compiler/`:option: directory.
-       Interestingly, it's compiled with complex hierarchy using
-       `--project`:option: switch.
-
-     Contents of ``web/upload/<version>`` are then deployed into Nim's
-     Web server.
-
-     This output directory structure allows to work correctly with files like
-     ``compiler/docgen.nim`` (implementation) and ``doc/docgen.md`` (user
-     documentation) in 1 repository.
-
-
-Index files
------------
-
-Index (``.idx``) files are used for 2 different purposes:
-
-1. easy cross-referencing between different ``.nim`` and/or ``.md`` / ``.rst``
-   files described in [Nim external referencing]
-2. creating a whole-project index for searching of symbols and keywords,
-   see [Buildindex command].
-
 
 Document Types
 ==============
@@ -286,46 +226,13 @@ Note that the `jsondoc`:option: command outputs its JSON without pretty-printing
 while `jsondoc0`:option: outputs pretty-printed JSON.
 
 
-Simple documentation links
-==========================
+Referencing Nim symbols: simple documentation links
+===================================================
 
-It's possible to use normal Markdown/RST syntax to *manually*
-reference Nim symbols using HTML anchors, however Nim has an *automatic*
-facility that makes referencing inside ``.nim`` and ``.md/.rst`` files and
-between them easy and seamless.
-The point is that such links will be resolved automatically
-by `nim doc`:cmd: (or `md2html`:option:, or `jsondoc`:option:,
-or `doc2tex`:option:, ...). And, unlike manual links, such automatic
-links **check** that their target exists -- a warning is emitted for
-any broken link, so you avoid broken links in your project.
-
-Nim treats both ``.md/.rst`` files and ``.nim`` modules (their doc comment
-part) as *documents* uniformly.
-Hence all directions of referencing are equally possible having the same syntax:
-
-1. ``.md/rst`` -> itself (internal). See [Markup local referencing].
-2. ``.md/rst`` -> external ``.md/rst``. See [Markup external referencing].
-   To summarize, referencing in `.md`/`.rst` files was already described in
-   [Nim-flavored Markdown and reStructuredText]
-   (particularly it described usage of index files for referencing),
-   while in this document we focus on Nim-specific details.
-3. ``.md/rst`` -> external ``.nim``. See [Nim external referencing].
-4. ``.nim`` -> itself (internal). See [Nim local referencing].
-5. ``.nim`` -> external ``.md/rst``. See [Markup external referencing].
-6. ``.nim`` -> external ``.nim``. See [Nim external referencing].
-
-To put it shortly, local referencing always works out of the box,
-external referencing requires to use ``.. importdoc:: <file>``
-directive to import `file` and to ensure that the corresponding
-``.idx`` file was generated.
-
-
-Nim local referencing
----------------------
-
-You can reference Nim identifiers from Nim documentation comments
-inside their ``.nim`` file (or inside a ``.rst`` file included from
-a ``.nim``).
+You can reference Nim identifiers from Nim documentation comments, currently
+only inside their ``.nim`` file (or inside a ``.rst`` file included from
+a ``.nim``). The point is that such links will be resolved automatically
+by `nim doc`:cmd: (or `nim jsondoc`:cmd: or `nim doc2tex`:cmd:).
 This pertains to any exported symbol like `proc`, `const`, `iterator`, etc.
 Syntax for referencing is basically a normal RST one: addition of
 underscore `_` to a *link text*.
@@ -498,143 +405,6 @@ recognized fine:
          ...
        ## Ref. `CopyFlag enum`_
 
-Nim external referencing
-------------------------
-
-Just like for [Markup external referencing], which saves markup anchors,
-the Nim symbols are also saved in ``.idx`` files, so one needs
-to generate them beforehand, and they should be loaded by
-an ``.. importdoc::`` directive. Arguments to ``.. importdoc::`` is a
-comma-separated list of Nim modules or Markdown/RST documents.
-
-`--index:only`:option: tells Nim to only generate ``.idx`` file and
-do **not** attempt to generate HTML/LaTeX output.
-For ``.nim`` modules there are 2 alternatives to work with ``.idx`` files:
-
-1. using [Project switch] implies generation of ``.idx`` files,
-   however, if ``importdoc`` is called on upper modules as its arguments,
-   their ``.idx`` are not yet created. Thus one should generate **all**
-   required ``.idx`` first:
-     ```cmd
-     nim doc --project --index:only <main>.nim
-     nim doc --project <main>.nim
-     ```
-2. or run `nim doc --index:only <module.nim>`:cmd: command for **all** (used)
-   Nim modules in your project. Then run `nim doc <module.nim>` on them for
-   output HTML generation.
-
-   .. Warning:: A mere `nim doc --index:on`:cmd: may fail on an attempt to do
-      ``importdoc`` from another module (for which ``.idx`` was not yet
-      generated), that's why `--index:only`:option: shall be used instead.
-
-   For ``.md``/``.rst`` markup documents point 2 is the only option.
-
-Then, you can freely use something like this in ``your_module.nim``:
-
-  ```nim
-  ## .. importdoc::  user_manual.md, another_module.nim
-
-  ...
-  ## Ref. [some section from User Manual].
-
-  ...
-  ## Ref. [proc f]
-  ## (assuming you have a proc `f` in ``another_module``).
-  ```
-
-and compile it by `nim doc`:cmd:. Note that link text will
-be automatically prefixed by the module name of symbol,
-so you will see something like "Ref. [another_module: proc f](#)"
-in the generated output.
-
-It's also possible to reference a whole module by prefixing or
-suffixing full canonical module name with "module":
-
-    Ref. [module subdir/name] or [subdir/name module].
-
-Markup documents as a whole can be referenced just by their title
-(or by their file name if the title was not set) without any prefix.
-
-.. Tip:: During development process the stage of ``.idx`` files generation
-  can be done only *once*, after that you use already generated ``.idx``
-  files while working with a document *being developed* (unless you do
-  incompatible changes to *referenced* documents).
-
-.. Hint:: After changing a *referenced* document file one may need
-  to regenerate its corresponding ``.idx`` file to get correct results.
-  Of course, when referencing *internally* inside any given ``.nim`` file,
-  it's not needed, one can even immediately use any freshly added anchor
-  (a document's own ``.idx`` file is not used for resolving its internal links).
-
-If an ``importdoc`` directive fails to find a ``.idx``, then an error
-is emitted.
-
-In case of such compilation failures please note that:
-
-* **all** relative paths, given to ``importdoc``, relate to insides of
-  ``OUTDIR``, and **not** project's directory structure.
-
-* ``importdoc`` searches for ``.idx`` in `--outdir:OUTDIR`:option: directory
-  (``htmldocs`` by default) and **not** around original modules, so:
-
-  .. Tip:: look into ``OUTDIR`` to understand what's going on.
-
-* also keep in mind that ``.html`` and ``.idx`` files should always be
-  output to the same directory, so check this and, if it's not true, check
-  that both runs *with* and *without* `--index:only`:option: have all
-  other options the same.
-
-To summarize, for 2 basic options of [Structuring output directories]
-compilation options are different:
-
-1) complex hierarchy with `--project`:option: switch.
-
-   As the **original** project's directory structure is replicated in
-   `OUTDIR`, all passed paths are related to this structure also.
-
-   E.g. if a module ``path1/module.nim`` does
-   ``.. importdoc:: path2/another.nim`` then docgen tries to load file
-   ``OUTDIR/path1/path2/another.idx``.
-
-   .. Note:: markup documents are just placed into the specified directory
-     `OUTDIR`:option: by default (i.e. they are **not** affected by
-     `--project`:option:), so if you have ``PROJECT/doc/manual.md``
-     document and want to use complex hirearchy (with ``doc/``),
-     compile it with `--docroot`:option:\:
-       ```cmd
-       # 1st stage
-       nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \
-            --index:only PROJECT/doc/manual.md
-       ...
-       # 2nd stage
-       nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \
-                         PROJECT/doc/manual.md
-       ```
-
-     Then the output file will be placed as ``OUTDIR/doc/manual.idx``.
-     So if you have ``PROJECT/path1/module.nim``, then ``manual.md`` can
-     be referenced as ``../doc/manual.md``.
-
-2) flattened structure.
-
-   E.g. if a module ``path1/module.nim`` does
-   ``.. importdoc:: path2/another.nim`` then docgen tries to load
-   ``OUTDIR/path2/another.idx``, so the path ``path1``
-   does not matter and providing ``path2`` can be useful only
-   in the case it contains another package that was placed there
-   using `--outdir:OUTDIR/path2`:option:.
-
-   The links' text will be prefixed as ``another: ...`` in both cases.
-
-   .. Warning:: Again, the same `--outdir:OUTDIR`:option: option should
-     be provided to both `doc --index:only`:option: /
-     `md2html --index:only`:option: and final generation by
-     `doc`:option:/`md2html`:option: inside 1 package.
-
-To temporarily disable ``importdoc``, e.g. if you don't need
-correct link resolution at the moment, use a `--noImportdoc`:option: switch
-(only warnings about unresolved links will be generated for external references).
-
 Related Options
 ===============
 
@@ -664,21 +434,10 @@ index file is line-oriented (newlines have to be escaped). Each line
 represents a tab-separated record of several columns, the first two mandatory,
 the rest optional. See the [Index (idx) file format] section for details.
 
-.. Note:: `--index`:option: switch only affects creation of ``.idx``
-  index files, while user-searchable Index HTML file is created by
-  `buildIndex`:option: commmand.
-
-Buildindex command
-------------------
-
 Once index files have been generated for one or more modules, the Nim
-compiler command `nim buildIndex directory`:cmd: can be run to go over all the index
+compiler command `buildIndex directory` can be run to go over all the index
 files in the specified directory to generate a [theindex.html](theindex.html)
-file:
-
-  ```cmd
-  nim buildIndex -o:path/to/htmldocs/theindex.html path/to/htmldocs
-  ```
+file.
 
 See source switch
 -----------------
@@ -809,22 +568,10 @@ references so they can be later concatenated into a big index file with
 the file format in detail.
 
 Index files are line-oriented and tab-separated (newline and tab characters
-have to be escaped). Each line represents a record with 6 fields.
-The content of these columns is:
+have to be escaped). Each line represents a record with at least two fields
+but can have up to four (additional columns are ignored). The content of these
+columns is:
 
-0. Discriminator tag denoting type of the index entry, allowed values are:
-   `markupTitle`
-   : a title for ``.md``/``.rst`` document
-   `nimTitle`
-   : a title of ``.nim`` module
-   `heading`
-   : heading of sections, can be both in Nim and markup files
-   `idx`
-   : terms marked with :idx: role
-   `nim`
-   : a Nim symbol
-   `nimgrp`
-   : a Nim group for overloadable symbols like `proc`s
 1. Mandatory term being indexed. Terms can include quoting according to
    Nim's rules (e.g. \`^\`).
 2. Base filename plus anchor hyperlink (e.g. ``algorithm.html#*,int,SortOrder``).
@@ -834,20 +581,29 @@ The content of these columns is:
    not for an API symbol but for a TOC entry.
 4. Optional title or description of the hyperlink. Browsers usually display
    this as a tooltip after hovering a moment over the hyperlink.
-5. A line number of file where the entry was defined.
 
-The index generation tools differentiate between documentation
-generated from ``.nim`` files and documentation generated from ``.md`` or
-``.rst`` files by tag `nimTitle` or `markupTitle` in the 1st line of
-the ``.idx`` file.
+The index generation tools try to differentiate between documentation
+generated from ``.nim`` files and documentation generated from ``.txt`` or
+``.rst`` files. The former are always closely related to source code and
+consist mainly of API entries. The latter are generic documents meant for
+human reading.
 
-.. TODO Normal symbols are added to the index with surrounding whitespaces removed. An
-  exception to this are the table of content (TOC) entries. TOC entries are added to
-  the index file with their third column having as much prefix spaces as their
-  level is in the TOC (at least 1 character). The prefix whitespace helps to
-  filter TOC entries from API or text symbols. This is important because the
-  amount of spaces is used to replicate the hierarchy for document TOCs in the
-  final index, and TOC entries found in ``.nim`` files are discarded.
+To differentiate both types (documents and APIs), the index generator will add
+to the index of documents an entry with the title of the document. Since the
+title is the topmost element, it will be added with a second field containing
+just the filename without any HTML anchor.  By convention, this entry without
+anchor is the *title entry*, and since entries in the index file are added as
+they are scanned, the title entry will be the first line. The title for APIs
+is not present because it can be generated concatenating the name of the file
+to the word **Module**.
+
+Normal symbols are added to the index with surrounding whitespaces removed. An
+exception to this are the table of content (TOC) entries. TOC entries are added to
+the index file with their third column having as much prefix spaces as their
+level is in the TOC (at least 1 character). The prefix whitespace helps to
+filter TOC entries from API or text symbols. This is important because the
+amount of spaces is used to replicate the hierarchy for document TOCs in the
+final index, and TOC entries found in ``.nim`` files are discarded.
 
 
 Additional resources
@@ -859,8 +615,6 @@ Additional resources
   [Markdown and RST markup languages](markdown_rst.html), which also
   contains the list of implemented features of these markup languages.
 
-* the implementation is in [module compiler/docgen].
-
 The output for HTML and LaTeX comes from the ``config/nimdoc.cfg`` and
 ``config/nimdoc.tex.cfg`` configuration files. You can add and modify these
 files to your project to change the look of the docgen output.

doc/grammar.txt

@@ -28,9 +28,7 @@ operatorB = OP0 | OP1 | OP2 | OP3 | OP4 | OP5 | OP6 | OP7 | OP8 | OP9 |
 symbol = '`' (KEYW|IDENT|literal|(operator|'('|')'|'['|']'|'{'|'}'|'=')+)+ '`'
        | IDENT | KEYW
 exprColonEqExpr = expr (':'|'=' expr)?
-exprEqExpr = expr ('=' expr)?
 exprList = expr ^+ comma
-optionalExprList = expr ^* comma
 exprColonEqExprList = exprColonEqExpr (comma exprColonEqExpr)* (comma)?
 qualifiedIdent = symbol ('.' optInd symbol)?
 setOrTableConstr = '{' ((exprColonEqExpr comma)* | ':' ) '}'
@@ -62,26 +60,25 @@ primarySuffix = '(' (exprColonEqExpr comma?)* ')'
       | DOTLIKEOP optInd symbol generalizedLit?
       | '[' optInd exprColonEqExprList optPar ']'
       | '{' optInd exprColonEqExprList optPar '}'
+      | &( '`'|IDENT|literal|'cast'|'addr'|'type') expr (comma expr)* # command syntax
 pragma = '{.' optInd (exprColonEqExpr comma?)* optPar ('.}' | '}')
 identVis = symbol OPR?  # postfix position
 identVisDot = symbol '.' optInd symbol OPR?
 identWithPragma = identVis pragma?
 identWithPragmaDot = identVisDot pragma?
 declColonEquals = identWithPragma (comma identWithPragma)* comma?
-                  (':' optInd typeDescExpr)? ('=' optInd expr)?
+                  (':' optInd typeDesc)? ('=' optInd expr)?
 identColonEquals = IDENT (comma IDENT)* comma?
-     (':' optInd typeDescExpr)? ('=' optInd expr)?)
-tupleTypeBracket = '[' optInd (identColonEquals (comma/semicolon)?)* optPar ']'
-tupleType = 'tuple' tupleTypeBracket
-tupleDecl = 'tuple' (tupleTypeBracket /
-    COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)?)
+     (':' optInd typeDesc)? ('=' optInd expr)?)
+tupleDecl = 'tuple'
+    '[' optInd  (identColonEquals (comma/semicolon)?)*  optPar ']' |
+    COMMENT? (IND{>} identColonEquals (IND{=} identColonEquals)*)?
 paramList = '(' declColonEquals ^* (comma/semicolon) ')'
 paramListArrow = paramList? ('->' optInd typeDesc)?
 paramListColon = paramList? (':' optInd typeDesc)?
 doBlock = 'do' paramListArrow pragma? colcom stmt
 routineExpr = ('proc' | 'func' | 'iterator') paramListColon pragma? ('=' COMMENT? stmt)?
-routineType = ('proc' | 'iterator') paramListColon pragma?
-forStmt = 'for' ((varTuple / identWithPragma) ^+ comma) 'in' expr colcom stmt
+forStmt = 'for' (identWithPragma ^+ comma) 'in' expr colcom stmt
 forExpr = forStmt
 expr = (blockExpr
       | ifExpr
@@ -90,32 +87,25 @@ expr = (blockExpr
       | forExpr
       | tryExpr)
       / simpleExpr
-simplePrimary = SIGILLIKEOP? identOrLiteral primarySuffix*
-commandStart = &('`'|IDENT|literal|'cast'|'addr'|'type'|'var'|'out'|
-                 'static'|'enum'|'tuple'|'object'|'proc')
-primary = simplePrimary (commandStart expr)
-        / operatorB primary
-        / routineExpr
-        / rawTypeDesc
-        / prefixOperator primary
-rawTypeDesc = (tupleType | routineType | 'enum' | 'object' |
-                ('var' | 'out' | 'ref' | 'ptr' | 'distinct') typeDesc?)
-                ('not' expr)?
-typeDescExpr = (routineType / simpleExpr) ('not' expr)?
-typeDesc = rawTypeDesc / typeDescExpr
-typeDefValue = ((tupleDecl | enumDecl | objectDecl | conceptDecl |
-                 ('ref' | 'ptr' | 'distinct') (tupleDecl | objectDecl))
-               / (simpleExpr (exprEqExpr ^+ comma postExprBlocks)?))
-               ('not' expr)?
+primary = operatorB primary primarySuffix* |
+          tupleDecl | routineExpr | enumDecl
+          objectDecl | conceptDecl | ('bind' primary)
+          ('var' | 'out' | 'ref' | 'ptr' | 'distinct') primary
+        /  prefixOperator* identOrLiteral primarySuffix*
+typeDesc = simpleExpr ('not' expr)?
+typeDefAux = simpleExpr ('not' expr
+                        | postExprBlocks)?
 postExprBlocks = ':' stmt? ( IND{=} doBlock
                            | IND{=} 'of' exprList ':' stmt
                            | IND{=} 'elif' expr ':' stmt
-                           | IND{=} 'except' optionalExprList ':' stmt
+                           | IND{=} 'except' exprList ':' stmt
                            | IND{=} 'finally' ':' stmt
                            | IND{=} 'else' ':' stmt )*
-exprStmt = simpleExpr postExprBlocks?
-         / simplePrimary (exprEqExpr ^+ comma) postExprBlocks?
-         / simpleExpr '=' optInd (expr postExprBlocks?)
+exprStmt = simpleExpr
+         (( '=' optInd expr colonBody? )
+         / ( expr ^+ comma
+             postExprBlocks
+           ))?
 importStmt = 'import' optInd expr
               ((comma expr)*
               / 'except' optInd (expr ^+ comma))
@@ -149,10 +139,10 @@ caseStmt = 'case' expr ':'? COMMENT?
             (IND{>} ofBranches DED
             | IND{=} ofBranches)
 tryStmt = 'try' colcom stmt &(IND{=}? 'except'|'finally')
-           (IND{=}? 'except' optionalExprList colcom stmt)*
+           (IND{=}? 'except' exprList colcom stmt)*
            (IND{=}? 'finally' colcom stmt)?
 tryExpr = 'try' colcom stmt &(optInd 'except'|'finally')
-           (optInd 'except' optionalExprList colcom stmt)*
+           (optInd 'except' exprList colcom stmt)*
            (optInd 'finally' colcom stmt)?
 blockStmt = 'block' symbol? colcom stmt
 blockExpr = 'block' symbol? colcom stmt
@@ -185,10 +175,9 @@ objectDecl = 'object' ('of' typeDesc)? COMMENT? objectPart
 conceptParam = ('var' | 'out')? symbol
 conceptDecl = 'concept' conceptParam ^* ',' (pragma)? ('of' typeDesc ^* ',')?
               &IND{>} stmt
-typeDef = identVisDot genericParamList? pragma '=' optInd typeDefValue
+typeDef = identVisDot genericParamList? pragma '=' optInd typeDefAux
             indAndComment?
-varTupleLhs = '(' optInd (identWithPragma / varTupleLhs) ^+ comma optPar ')'
-varTuple = varTupleLhs '=' optInd expr
+varTuple = '(' optInd identWithPragma ^+ comma optPar ')' '=' optInd expr
 colonBody = colcom stmt postExprBlocks?
 variable = (varTuple / identColonEquals) colonBody? indAndComment
 constant = (varTuple / identWithPragma) (colon typeDesc)? '=' optInd expr indAndComment

doc/lib.md

@@ -46,18 +46,14 @@ Core
 * [bitops](bitops.html)
   Provides a series of low-level methods for bit manipulation.
 
-* [compilesettings](compilesettings.html)
-  Querying the compiler about diverse configuration settings from code.
-
 * [cpuinfo](cpuinfo.html)
-  Procs to determine the number of CPUs / cores.
-
-* [effecttraits](effecttraits.html)
-  Access to the inferred .raises effects
-  for Nim's macro system.
+  This module implements procs to determine the number of CPUs / cores.
 
 * [endians](endians.html)
-  Helpers that deal with different byte orders.
+  This module contains helpers that deal with different byte orders.
+
+* [lenientops](lenientops.html)
+  Provides binary operators for mixed integer/float expressions for convenience.
 
 * [locks](locks.html)
   Locks and condition variables for Nim.
@@ -75,10 +71,10 @@ Core
   Provides (unsafe) access to Nim's run-time type information.
 
 * [typetraits](typetraits.html)
-  Compile-time reflection procs for working with types.
+  This module defines compile-time reflection procs for working with types.
 
 * [volatile](volatile.html)
-  Code for generating volatile loads and stores,
+  This module contains code for generating volatile loads and stores,
   which are useful in embedded and systems programming.
 
 
@@ -86,24 +82,24 @@ Algorithms
 ----------
 
 * [algorithm](algorithm.html)
-  Some common generic algorithms like sort or binary search.
+  This module implements some common generic algorithms like sort or binary search.
 
 * [enumutils](enumutils.html)
-  Additional functionality for the built-in `enum` type.
+  This module adds functionality for the built-in `enum` type.
 
 * [sequtils](sequtils.html)
-  Operations for the built-in `seq` type
+  This module implements operations for the built-in `seq` type
   which were inspired by functional programming languages.
 
 * [setutils](setutils.html)
-  Additional functionality for the built-in `set` type.
+  This module adds functionality for the built-in `set` type.
 
 
 Collections
 -----------
 
 * [critbits](critbits.html)
-  A *crit bit tree* which is an efficient
+  This module implements a *crit bit tree* which is an efficient
   container for a sorted set of strings, or a sorted mapping of strings.
 
 * [deques](deques.html)
@@ -126,19 +122,9 @@ Collections
 * [packedsets](packedsets.html)
   Efficient implementation of a set of ordinals as a sparse bit set.
 
-* [ropes](ropes.html)
-  A *rope* data type.
-  Ropes can represent very long strings efficiently;
-  in particular, concatenation is done in O(1) instead of O(n).
-
 * [sets](sets.html)
   Nim hash set support.
 
-* [strtabs](strtabs.html)
-  The `strtabs` module implements an efficient hash table that is a mapping
-  from strings to strings. Supports a case-sensitive, case-insensitive and
-  style-insensitive modes.
-
 * [tables](tables.html)
   Nim hash table support. Contains tables, ordered tables, and count tables.
 
@@ -150,55 +136,60 @@ String handling
   Utilities for `cstring` handling.
 
 * [editdistance](editdistance.html)
-  An algorithm to compute the edit distance between two
+  This module contains an algorithm to compute the edit distance between two
   Unicode strings.
 
 * [encodings](encodings.html)
   Converts between different character encodings. On UNIX, this uses
   the `iconv` library, on Windows the Windows API.
 
-* [formatfloat](formatfloat.html)
-  Formatting floats as strings.
+* [parseutils](parseutils.html)
+  This module contains helpers for parsing tokens, numbers, identifiers, etc.
 
-* [objectdollar](objectdollar.html)
-  A generic `$` operator to convert objects to strings.
+* [pegs](pegs.html)
+  This module contains procedures and operators for handling PEGs.
 
 * [punycode](punycode.html)
   Implements a representation of Unicode with the limited ASCII character subset.
 
+* [ropes](ropes.html)
+  This module contains support for a *rope* data type.
+  Ropes can represent very long strings efficiently;
+  in particular, concatenation is done in O(1) instead of O(n).
+
 * [strbasics](strbasics.html)
-  Some high performance string operations.
+  This module provides some high performance string operations.
 
 * [strformat](strformat.html)
   Macro based standard string interpolation/formatting. Inspired by
-  Python's f-strings.\
-  **Note:** if you need templating, consider using Nim
-  [Source Code Filters (SCF)](filters.html).
+  Python's f-strings.
 
 * [strmisc](strmisc.html)
-  Uncommon string handling operations that do not
-  fit with the commonly used operations in [strutils](strutils.html).
+  This module contains uncommon string handling operations that do not
+  fit with the commonly used operations in strutils.
 
 * [strscans](strscans.html)
-  A `scanf` macro for convenient parsing of mini languages.
+  This module contains a `scanf` macro for convenient parsing of mini languages.
+
+* [strtabs](strtabs.html)
+  The `strtabs` module implements an efficient hash table that is a mapping
+  from strings to strings. Supports a case-sensitive, case-insensitive and
+  style-insensitive modes.
 
 * [strutils](strutils.html)
-  Common string handling operations like changing
+  This module contains common string handling operations like changing
   case of a string, splitting a string into substrings, searching for
   substrings, replacing substrings.
 
 * [unicode](unicode.html)
-  Support for handling the Unicode UTF-8 encoding.
+  This module provides support to handle the Unicode UTF-8 encoding.
 
 * [unidecode](unidecode.html)
   It provides a single proc that does Unicode to ASCII transliterations.
   Based on Python's Unidecode module.
 
-* [widestrs](widestrs.html)
-  Nim support for C/C++'s wide strings.
-
 * [wordwrap](wordwrap.html)
-  An algorithm for word-wrapping Unicode strings.
+  This module contains an algorithm to wordwrap a Unicode string.
 
 
 Time handling
@@ -214,17 +205,8 @@ Time handling
 Generic Operating System Services
 ---------------------------------
 
-* [appdirs](appdirs.html)
-  Helpers for determining special directories used by apps.
-
-* [cmdline](cmdline.html)
-  System facilities for reading command line parameters.
-
-* [dirs](dirs.html)
-  Directory handling.
-
 * [distros](distros.html)
-  Basics for OS distribution ("distro") detection
+  This module implements the basics for OS distribution ("distro") detection
   and the OS's native package manager.
   Its primary purpose is to produce output for Nimble packages,
   but it also contains the widely used **Distribution** enum
@@ -232,19 +214,14 @@ Generic Operating System Services
   See [packaging](packaging.html) for hints on distributing Nim using OS packages.
 
 * [dynlib](dynlib.html)
-  Accessing symbols from shared libraries.
+  This module implements the ability to access symbols from shared libraries.
 
-* [envvars](envvars.html)
-  Environment variable handling.
-
-* [exitprocs](exitprocs.html)
-  Adding hooks to program exit.
-
-* [files](files.html)
-  File handling.
+* [marshal](marshal.html)
+  Contains procs for serialization and deserialization of arbitrary Nim
+  data structures.
 
 * [memfiles](memfiles.html)
-  Support for memory-mapped files (Posix's `mmap`)
+  This module provides support for memory-mapped files (Posix's `mmap`)
   on the different operating systems.
 
 * [os](os.html)
@@ -252,52 +229,35 @@ Generic Operating System Services
   reading command line arguments, working with directories, running shell
   commands, etc.
 
-* [oserrors](oserrors.html)
-  OS error reporting.
-
 * [osproc](osproc.html)
   Module for process communication beyond `os.execShellCmd`.
 
-* [paths](paths.html)
-  Path handling.
-
-* [reservedmem](reservedmem.html)
-  Utilities for reserving portions of the
-  address space of a program without consuming physical memory.
-
 * [streams](streams.html)
-  A stream interface and two implementations thereof:
+  This module provides a stream interface and two implementations thereof:
   the `FileStream` and the `StringStream` which implement the stream
   interface for Nim file objects (`File`) and strings. Other modules
   may provide other implementations for this standard stream interface.
 
-* [symlinks](symlinks.html)
-  Symlink handling.
-
-* [syncio](syncio.html)
-  Various synchronized I/O operations.
-
 * [terminal](terminal.html)
-  A module to control the terminal output (also called *console*).
+  This module contains a few procedures to control the *terminal*
+  (also called *console*). The implementation simply uses ANSI escape
+  sequences and does not depend on any other module.
   
 * [tempfiles](tempfiles.html)
-  Some utilities for generating temporary path names and
-  creating temporary files and directories.
+  This module provides some utils to generate temporary path names and
+  create temporary files and directories.
 
 
 Math libraries
 --------------
 
 * [complex](complex.html)
-  Complex numbers and relevant mathematical operations.
+  This module implements complex numbers and relevant mathematical operations.
 
 * [fenv](fenv.html)
   Floating-point environment. Handling of floating-point rounding and
   exceptions (overflow, zero-divide, etc.).
 
-* [lenientops](lenientops.html)
-  Binary operators for mixed integer/float expressions for convenience.
-
 * [math](math.html)
   Mathematical operations like cosine, square root.
 
@@ -305,11 +265,14 @@ Math libraries
   Fast and tiny random number generator.
 
 * [rationals](rationals.html)
-  Rational numbers and relevant mathematical operations.
+  This module implements rational numbers and relevant mathematical operations.
 
 * [stats](stats.html)
   Statistical analysis.
 
+* [sums](sums.html)
+  Accurate summation functions.
+
 * [sysrand](sysrand.html)
   Cryptographically secure pseudorandom number generator.
 
@@ -317,92 +280,80 @@ Math libraries
 Internet Protocols and Support
 ------------------------------
 
-* [async](async.html)
-  Exports `asyncmacro` and `asyncfutures` for native backends, and `asyncjs` on the JS backend. 
-
 * [asyncdispatch](asyncdispatch.html)
-  An asynchronous dispatcher for IO operations.
+  This module implements an asynchronous dispatcher for IO operations.
 
 * [asyncfile](asyncfile.html)
-  An asynchronous file reading and writing using `asyncdispatch`.
+  This module implements asynchronous file reading and writing using
+  `asyncdispatch`.
 
 * [asyncftpclient](asyncftpclient.html)
-  An asynchronous FTP client using the `asyncnet` module.
+  This module implements an asynchronous FTP client using the `asyncnet`
+  module.
 
 * [asynchttpserver](asynchttpserver.html)
-  An asynchronous HTTP server using the `asyncnet` module.
-
-* [asyncmacro](asyncmacro.html)
-  `async` and `multisync` macros for `asyncdispatch`.
+  This module implements an asynchronous HTTP server using the `asyncnet`
+  module.
 
 * [asyncnet](asyncnet.html)
-  Asynchronous sockets based on the `asyncdispatch` module.
+  This module implements asynchronous sockets based on the `asyncdispatch`
+  module.
 
 * [asyncstreams](asyncstreams.html)
-  `FutureStream` - a future that acts as a queue.
+  This module provides `FutureStream` - a future that acts as a queue.
 
 * [cgi](cgi.html)
-  Helpers for CGI applications.
+  This module implements helpers for CGI applications.
 
 * [cookies](cookies.html)
-  Helper procs for parsing and generating cookies.
+  This module contains helper procs for parsing and generating cookies.
 
 * [httpclient](httpclient.html)
-  A simple HTTP client with support for both synchronous
+  This module implements a simple HTTP client which supports both synchronous
   and asynchronous retrieval of web pages.
 
 * [mimetypes](mimetypes.html)
-  A mimetypes database.
+  This module implements a mimetypes database.
 
 * [nativesockets](nativesockets.html)
-  A low-level sockets API.
+  This module implements a low-level sockets API.
 
 * [net](net.html)
-  A high-level sockets API.
+  This module implements a high-level sockets API. It replaces the
+  `sockets` module.
 
 * [selectors](selectors.html)
-  A selector API with backends specific to each OS.
-  Supported OS primitives: `epoll`, `kqueue`, `poll`, and `select` on Windows.
-
-* [smtp](smtp.html)
-  A simple SMTP client with support for both synchronous and asynchronous operation.
-
-* [socketstreams](socketstreams.html)
-  An implementation of the streams interface for sockets.
+  This module implements a selector API with backends specific to each OS.
+  Currently, epoll on Linux and select on other operating systems.
 
 * [uri](uri.html)
-  Functions for working with URIs and URLs.
+  This module provides functions for working with URIs.
 
 
 Threading
 ---------
 
-* [isolation](isolation.html)
-  The `Isolated[T]` type for
-  safe construction of isolated subgraphs that can be
-  passed efficiently to different channels and threads.
-
-* [tasks](tasks.html)
-  Basic primitives for creating parallel programs.
+* [threads](threads.html)
+  Basic Nim thread support.
 
 * [threadpool](threadpool.html)
   Implements Nim's [spawn](manual_experimental.html#parallel-amp-spawn).
 
-* [typedthreads](typedthreads.html)
-  Basic Nim thread support.
-
 
 Parsers
 -------
 
 * [htmlparser](htmlparser.html)
-  HTML document parser that creates a XML tree representation.
+  This module parses an HTML document and creates its XML tree representation.
 
 * [json](json.html)
   High-performance JSON parser.
 
+* [jsonutils](jsonutils.html)
+  This module implements a hookable (de)serialization for arbitrary types.
+
 * [lexbase](lexbase.html)
-  A low-level module that implements an extremely efficient buffering
+  This is a low-level module that implements an extremely efficient buffering
   scheme for lexers and parsers. This is used by the diverse parsing modules.
 
 * [parsecfg](parsecfg.html)
@@ -416,7 +367,7 @@ Parsers
   The `parsecsv` module implements a simple high-performance CSV parser.
 
 * [parsejson](parsejson.html)
-  A JSON parser. It is used and exported by the [json](json.html) module, but can also be used in its own right.
+  This module implements a JSON parser. It is used and exported by the [json](json.html) module, but can also be used in its own right.
 
 * [parseopt](parseopt.html)
   The `parseopt` module implements a command line option parser.
@@ -424,18 +375,12 @@ Parsers
 * [parsesql](parsesql.html)
   The `parsesql` module implements a simple high-performance SQL parser.
 
-* [parseutils](parseutils.html)
-  Helpers for parsing tokens, numbers, identifiers, etc.
-
 * [parsexml](parsexml.html)
   The `parsexml` module implements a simple high performance XML/HTML parser.
   The only encoding that is supported is UTF-8. The parser has been designed
   to be somewhat error-correcting, so that even some "wild HTML" found on the
   web can be parsed with it.
 
-* [pegs](pegs.html)
-  Procedures and operators for handling PEGs.
-
 
 Docutils
 --------
@@ -446,14 +391,14 @@ Docutils
   The interface supports one language nested in another.
 
 * [packages/docutils/rst](rst.html)
-  A reStructuredText parser. A large subset
+  This module implements a reStructuredText parser. A large subset
   is implemented. Some features of the markdown wiki syntax are also supported.
 
 * [packages/docutils/rstast](rstast.html)
-  An AST for the reStructuredText parser.
+  This module implements an AST for the reStructuredText parser.
 
 * [packages/docutils/rstgen](rstgen.html)
-  A generator of HTML/Latex from reStructuredText.
+  This module implements a generator of HTML/Latex from reStructuredText.
 
 
 XML Processing
@@ -464,17 +409,14 @@ XML Processing
   contains a macro for XML/HTML code generation.
 
 * [xmlparser](xmlparser.html)
-  XML document parser that creates a XML tree representation.
+  This module parses an XML document and creates its XML tree representation.
 
 
 Generators
 ----------
 
-* [genasts](genasts.html)
-  AST generation using captured variables for macros.
-
 * [htmlgen](htmlgen.html)
-  A simple XML and HTML code
+  This module implements a simple XML and HTML code
   generator. Each commonly used HTML tag has a corresponding macro
   that generates a string with its HTML representation.
 
@@ -483,13 +425,14 @@ Hashing
 -------
 
 * [base64](base64.html)
-  A Base64 encoder and decoder.
+  This module implements a Base64 encoder and decoder.
 
 * [hashes](hashes.html)
-  Efficient computations of hash values for diverse Nim types.
+  This module implements efficient computations of hash values for diverse
+  Nim types.
 
 * [md5](md5.html)
-  The MD5 checksum algorithm.
+  This module implements the MD5 checksum algorithm.
 
 * [oids](oids.html)
   An OID is a global ID that consists of a timestamp,
@@ -497,54 +440,33 @@ Hashing
   produce a globally distributed unique ID.
 
 * [sha1](sha1.html)
-  The SHA-1 checksum algorithm.
-
-
-Serialization
--------------
-
-* [jsonutils](jsonutils.html)
-  Hookable (de)serialization for arbitrary types
-  using JSON.
-
-* [marshal](marshal.html)
-  Contains procs for serialization and deserialization of arbitrary Nim
-  data structures.
+  This module implements the SHA-1 checksum algorithm.
 
 
 Miscellaneous
 -------------
 
-* [assertions](assertions.html)
-  Assertion handling.
-
 * [browsers](browsers.html)
-  Procs for opening URLs with the user's default
+  This module implements procs for opening URLs with the user's default
   browser.
 
 * [colors](colors.html)
-  Color handling.
+  This module implements color handling for Nim.
 
 * [coro](coro.html)
-  Experimental coroutines in Nim.
-
-* [decls](decls.html)
-  Syntax sugar for some declarations.
+  This module implements experimental coroutines in Nim.
 
 * [enumerate](enumerate.html)
-  `enumerate` syntactic sugar based on Nim's macro system.
-
-* [importutils](importutils.html)
-  Utilities related to import and symbol resolution.
+  This module implements `enumerate` syntactic sugar based on Nim's macro system.
 
 * [logging](logging.html)
-  A simple logger.
+  This module implements a simple logger.
 
 * [segfaults](segfaults.html)
   Turns access violations or segfaults into a `NilAccessDefect` exception.
 
 * [sugar](sugar.html)
-  Nice syntactic sugar based on Nim's macro system.
+  This module implements nice syntactic sugar based on Nim's macro system.
 
 * [unittest](unittest.html)
   Implements a Unit testing DSL.
@@ -553,14 +475,11 @@ Miscellaneous
   Decode variable-length integers that are compatible with SQLite.
 
 * [with](with.html)
-  The `with` macro for easy function chaining.
-
-* [wrapnils](wrapnils.html)
-  Allows evaluating expressions safely against nil dereferences.
+  This module implements the `with` macro for easy function chaining.
 
 
-Modules for the JavaScript backend
-----------------------------------
+Modules for the JS backend
+--------------------------
 
 * [asyncjs](asyncjs.html)
   Types and macros for writing asynchronous procedures in JavaScript.
@@ -578,15 +497,9 @@ Modules for the JavaScript backend
   The wrapper of core JavaScript functions. For most purposes, you should be using
   the `math`, `json`, and `times` stdlib modules instead of this module.
 
-* [jsfetch](jshttp.html)
-  Wrapper for `fetch`.
-
 * [jsffi](jsffi.html)
   Types and macros for easier interaction with JavaScript.
 
-* [jsre](jsre.html)
-  Regular Expressions for the JavaScript target.
-
 
 Impure libraries
 ================
@@ -595,29 +508,21 @@ Regular expressions
 -------------------
 
 * [re](re.html)
-  Procedures and operators for handling regular
+  This module contains procedures and operators for handling regular
   expressions. The current implementation uses PCRE.
 
-* [nre](nre.html)
-
-  Many help functions for handling regular expressions.
-  The current implementation uses PCRE.
 
 Database support
 ----------------
 
-* [db_mysql](db_mysql.html)
-  A higher level MySQL database wrapper. The same interface is implemented
-  for other databases too.
-
-* [db_odbc](db_odbc.html)
-  A higher level ODBC database wrapper. The same interface is implemented
-  for other databases too.
-
 * [db_postgres](db_postgres.html)
   A higher level PostgreSQL database wrapper. The same interface is implemented
   for other databases too.
 
+* [db_mysql](db_mysql.html)
+  A higher level MySQL database wrapper. The same interface is implemented
+  for other databases too.
+
 * [db_sqlite](db_sqlite.html)
   A higher level SQLite database wrapper. The same interface is implemented
   for other databases too.
@@ -627,7 +532,7 @@ Generic Operating System Services
 ---------------------------------
 
 * [rdstdin](rdstdin.html)
-  Code for reading user input from stdin.
+  This module contains code for reading from stdin.
 
 
 Wrappers
@@ -641,7 +546,7 @@ Windows-specific
 ----------------
 
 * [winlean](winlean.html)
-  Wrapper for a small subset of the Win32 API.
+  Contains a wrapper for a small subset of the Win32 API.
 * [registry](registry.html)
   Windows registry support.
 
@@ -650,7 +555,7 @@ UNIX specific
 -------------
 
 * [posix](posix.html)
-  Wrapper for the POSIX standard.
+  Contains a wrapper for the POSIX standard.
 * [posix_utils](posix_utils.html)
   Contains helpers for the POSIX standard or specialized for Linux and BSDs.
 
@@ -665,14 +570,14 @@ Regular expressions
 Database support
 ----------------
 
+* [postgres](postgres.html)
+  Contains a wrapper for the PostgreSQL API.
 * [mysql](mysql.html)
-  Wrapper for the mySQL API.
+  Contains a wrapper for the mySQL API.
+* [sqlite3](sqlite3.html)
+  Contains a wrapper for the SQLite 3 API.
 * [odbcsql](odbcsql.html)
   interface to the ODBC driver.
-* [postgres](postgres.html)
-  Wrapper for the PostgreSQL API.
-* [sqlite3](sqlite3.html)
-  Wrapper for the SQLite 3 API.
 
 
 Network Programming and Internet Protocols

doc/manual.md

@@ -3076,25 +3076,12 @@ when they are declared. The only exception to this is if the `{.importc.}`
 pragma (or any of the other `importX` pragmas) is applied, in this case the
 value is expected to come from native code, typically a C/C++ `const`.
 
-Special identifier `_` (underscore)
------------------------------------
-
-The identifier `_` has a special meaning in declarations.
-Any definition with the name `_` will not be added to scope, meaning the
-definition is evaluated, but cannot be used. As a result the name `_` can be
-indefinitely redefined.
-
-  ```nim
-  let _ = 123
-  echo _ # error
-  let _ = 456 # compiles
-  ```
 
 Tuple unpacking
 ---------------
 
-In a `var`, `let` or `const` statement tuple unpacking can be performed.
-The special identifier `_` can be used to ignore some parts of the tuple:
+In a `var` or `let` statement tuple unpacking can be performed. The special
+identifier `_` can be used to ignore some parts of the tuple:
 
   ```nim
   proc returnsTuple(): (int, int, int) = (4, 2, 3)
@@ -3102,35 +3089,6 @@ The special identifier `_` can be used to ignore some parts of the tuple:
   let (x, _, z) = returnsTuple()
   ```
 
-This is treated as syntax sugar for roughly the following:
-
-  ```nim
-  let
-    tmpTuple = returnsTuple()
-    x = tmpTuple[0]
-    z = tmpTuple[2]
-  ```
-
-For `var` or `let` statements, if the value expression is a tuple literal,
-each expression is directly expanded into an assignment without the use of
-a temporary variable.
-
-  ```nim
-  let (x, y, z) = (1, 2, 3)
-  # becomes
-  let
-    x = 1
-    y = 2
-    z = 3
-  ```
-
-Tuple unpacking can also be nested:
-
-  ```nim
-  proc returnsNestedTuple(): (int, (int, int), int, int) = (4, (5, 7), 2, 3)
-
-  let (x, (_, y), _, z) = returnsNestedTuple()
-  ```
 
 
 Const section
@@ -3616,8 +3574,8 @@ Example:
   var y = if x > 8: 9 else: 10
   ```
 
-An `if` expression always results in a value, so the `else` part is
-required. `elif` parts are also allowed.
+An if expression always results in a value, so the `else` part is
+required. `Elif` parts are also allowed.
 
 When expression
 ---------------
@@ -3816,6 +3774,15 @@ every time the function is called.
   proc foo(a: int, b: int = 47): int
   ```
 
+Just as the comma propagates the types from right to left until the
+first parameter or until a semicolon is hit, it also propagates the
+default value starting from the parameter declared with it.
+
+  ```nim
+  # Both a and b are optional with 47 as their default values.
+  proc foo(a, b: int = 47): int
+  ```
+
 Parameters can be declared mutable and so allow the proc to modify those
 arguments, by using the type modifier `var`.
 
@@ -4155,7 +4122,7 @@ the operator is in scope (including if it is private).
   ```
 
 Type bound operators are:
-`=destroy`, `=copy`, `=sink`, `=trace`, `=deepcopy`, `=wasMoved`.
+`=destroy`, `=copy`, `=sink`, `=trace`, `=deepcopy`.
 
 These operations can be *overridden* instead of *overloaded*. This means that
 the implementation is automatically lifted to structured types. For instance,
@@ -4806,8 +4773,8 @@ Example:
       echo "overflow!"
     except ValueError, IOError:
       echo "catch multiple exceptions!"
-    except CatchableError:
-      echo "Catchable exception!"
+    except:
+      echo "Unknown exception!"
     finally:
       close(f)
   ```
@@ -4819,6 +4786,9 @@ listed in an `except` clause, the corresponding statements are executed.
 The statements following the `except` clauses are called
 `exception handlers`:idx:.
 
+The empty `except`:idx: clause is executed if there is an exception that is
+not listed otherwise. It is similar to an `else` clause in `if` statements.
+
 If there is a `finally`:idx: clause, it is always executed after the
 exception handlers.
 
@@ -4836,11 +4806,11 @@ Try can also be used as an expression; the type of the `try` branch then
 needs to fit the types of `except` branches, but the type of the `finally`
 branch always has to be `void`:
 
-  ```nim test
+  ```nim
   from std/strutils import parseInt
 
   let x = try: parseInt("133a")
-          except ValueError: -1
+          except: -1
           finally: echo "hi"
   ```
 
@@ -4848,9 +4818,8 @@ branch always has to be `void`:
 To prevent confusing code there is a parsing limitation; if the `try`
 follows a `(` it has to be written as a one liner:
 
-  ```nim test
-  from std/strutils import parseInt
-  let x = (try: parseInt("133a") except ValueError: -1)
+  ```nim
+  let x = (try: parseInt("133a") except: -1)
   ```
 
 
@@ -4898,7 +4867,7 @@ error message from `e`, and for such situations, it is enough to use
   ```nim
   try:
     # ...
-  except CatchableError:
+  except:
     echo getCurrentExceptionMsg()
   ```
 
@@ -5086,7 +5055,7 @@ An empty `raises` list (`raises: []`) means that no exception may be raised:
     try:
       unsafeCall()
       result = true
-    except CatchableError:
+    except:
       result = false
   ```
 
@@ -5467,9 +5436,9 @@ type class           matches
 ==================   ===================================================
 `object`             any object type
 `tuple`              any tuple type
+
 `enum`               any enumeration
 `proc`               any proc type
-`iterator`           any iterator type
 `ref`                any `ref` type
 `ptr`                any `ptr` type
 `var`                any `var` type
@@ -5488,7 +5457,7 @@ more complex type classes:
 
   ```nim
   # create a type class that will match all tuple and object types
-  type RecordType = (tuple or object)
+  type RecordType = tuple or object
 
   proc printFields[T: RecordType](rec: T) =
     for key, value in fieldPairs(rec):
@@ -5529,17 +5498,6 @@ as `type constraints`:idx: of the generic type parameter:
   onlyIntOrString("xy", 50) # invalid as 'T' cannot be both at the same time
   ```
 
-`proc` and `iterator` type classes also accept a calling convention pragma
-to restrict the calling convention of the matching `proc` or `iterator` type.
-
-  ```nim
-  proc onlyClosure[T: proc {.closure.}](x: T) = discard
-
-  onlyClosure(proc() = echo "hello") # valid
-  proc foo() {.nimcall.} = discard
-  onlyClosure(foo) # type mismatch
-  ```
-
 
 Implicit generics
 -----------------
@@ -5548,7 +5506,7 @@ A type class can be used directly as the parameter's type.
 
   ```nim
   # create a type class that will match all tuple and object types
-  type RecordType = (tuple or object)
+  type RecordType = tuple or object
 
   proc printFields(rec: RecordType) =
     for key, value in fieldPairs(rec):
@@ -8091,24 +8049,6 @@ used. To see if a value was provided, `defined(FooBar)` can be used.
 The syntax `-d:flag`:option: is actually just a shortcut for
 `-d:flag=true`:option:.
 
-These pragmas also accept an optional string argument for qualified
-define names.
-
-  ```nim
-  const FooBar {.intdefine: "package.FooBar".}: int = 5
-  echo FooBar
-  ```
-
-  ```cmd
-  nim c -d:package.FooBar=42 foobar.nim
-  ```
-
-This helps disambiguate define names in different packages.
-
-See also the [generic `define` pragma](manual_experimental.html#generic-define-pragma)
-for a version of these pragmas that detects the type of the define based on
-the constant value.
-
 User-defined pragmas
 ====================
 
@@ -8484,12 +8424,23 @@ This is only useful if the program is compiled as a dynamic library via the
 `--app:lib`:option: command-line option.
 
 
+
 Threads
 =======
 
-The `--threads:on`:option: command-line switch is enabled by default. The [typedthreads module](typedthreads.html) module then contains several threading primitives. See [spawn](manual_experimental.html#parallel-amp-spawn) for
+To enable thread support the `--threads:on`:option: command-line switch needs to
+be used. The [system module](system.html) module then contains several threading primitives.
+See the [channels](channels_builtin.html) modules
+for the low-level thread API. There are also high-level parallelism constructs
+available. See [spawn](manual_experimental.html#parallel-amp-spawn) for
 further details.
 
+Nim's memory model for threads is quite different than that of other common
+programming languages (C, Pascal, Java): Each thread has its own (garbage
+collected) heap, and sharing of memory is restricted to global variables. This
+helps to prevent race conditions. GC efficiency is improved quite a lot,
+because the GC never has to stop other threads and see what they reference.
+
 The only way to create a thread is via `spawn` or
 `createThread`. The invoked proc must not use `var` parameters nor must
 any of its parameters contain a `ref` or `closure` type. This enforces

doc/manual_experimental.md

@@ -65,29 +65,6 @@ However, a `void` type cannot be inferred in generic code:
 The `void` type is only valid for parameters and return types; other symbols
 cannot have the type `void`.
 
-Generic `define` pragma
-=======================
-
-Aside the [typed define pragmas for constants](manual.html#implementation-specific-pragmas-compileminustime-define-pragmas),
-there is a generic `{.define.}` pragma that interprets the value of the define
-based on the type of the constant value.
-
-  ```nim
-  const foo {.define: "package.foo".} = 123
-  const bar {.define: "package.bar".} = false
-  ```
-
-  ```cmd
-  nim c -d:package.foo=456 -d:package.bar foobar.nim
-  ```
-
-The following types are supported:
-
-* `string` and `cstring`
-* Signed and unsigned integer types
-* `bool`
-* Enums
-
 Top-down type inference
 =======================
 
@@ -542,7 +519,8 @@ Since version 1.4, a stricter definition of "side effect" is available.
 In addition to the existing rule that a side effect is calling a function
 with side effects, the following rule is also enforced:
 
-A store to the heap via a `ref` or `ptr` indirection is not allowed.
+Any mutation to an object does count as a side effect if that object is reachable
+via a parameter that is not declared as a `var` parameter.
 
 For example:
 
@@ -562,14 +540,17 @@ For example:
       it = it.ri
 
   func mut(n: Node) =
-    var it = n
-    while it != nil:
-      it.data = "yeah" # forbidden mutation
-      it = it.ri
-
+    let m = n # is the statement that connected the mutation to the parameter
+    m.data = "yeah" # the mutation is here
+    # Error: 'mut' can have side effects
+    # an object reachable from 'n' is potentially mutated
   ```
 
 
+The algorithm behind this analysis is described in
+the [view types algorithm][Algorithm].
+
+
 View types
 ==========
 
@@ -1878,20 +1859,6 @@ before it is used. Thus the following is valid:
 
 In this example every path does set `s` to a value before it is used.
 
-  ```nim
-  {.experimental: "strictDefs".}
-
-  proc test(cond: bool) =
-    let s: seq[string]
-    if cond:
-      s = @["y"]
-    else:
-      s = @[]
-  ```
-
-With `experimental: "strictDefs"`, `let` statements are allowed to not have an initial value, but every path should set `s` to a value before it is used.
-
-
 `out` parameters
 ----------------