Fix and tests for issues #26147 case A.
See individual commit messages.
Checked on the full test suite: no new failure with respect to devel as
baseline.
`propagateToOwner` aborted with
ast.nim(1314, 9) `mask <= o2.flags` IC bug: sealed type missing
propagated flags
whenever a consumer module propagated `tfHasAsgn`/`tfHasOwned` into a
type another module had already sealed. The assert encoded the hope that
a producer's derivation is complete at seal time. It is not: these flags
are DERIVED -- a fixpoint over kind, MM config, elements and the
attached-op table -- so a consumer can legitimately discover one later.
The trigger in the wild is a generic alias, `Channel[TMsg] {.gcsafe.} =
RawChannel`, whose object only acquires `tfHasAsgn` at the first
`Channel[T]` instantiation, in another module and (under IC) another
process. Any program that opens a channel crashed the compiler under
`nim ic` as soon as a `=copy` hook was declared for the alias
(nim-lang/Nim#26182).
Partition the flags instead. `derivedTypeFlags` names the bookkeeping
bits -- tfHasAsgn, tfHasOwned, tfHasGCedMem, tfCheckedForDestructor --
and `ast.inclDerived` / `ast.exclDerived` write exactly those without
the `Sealed` assert. The exemption is sound: none of them is in
`eqTypeFlags`, the `typekeys` content key or the NIF name, so they
cannot rename a type, move it in the cache or change `sameType`; and
`ast2nif.writeType` emits a `SymUse` for any type the current module
does not own, so a consumer's write can never reach a NIF. They stay
serialized, so a reload starts from the producer's derivation and a
consumer only ever adds to it.
`tfGenericHasDestructor` is deliberately NOT in the set: it is an alias
for `tfExplicitCallConv`, and exempting it would exempt a real proc-type
property.
This also gives the four existing `flagsImpl` writes a name. They wrote
the raw field precisely to dodge this assert -- liftdestructors' one
carried `# ^ XXX Breaks IC!` -- and are now `inclDerived`/`exclDerived`
calls, so the exemption is typed, asserted against `derivedTypeFlags`,
and greppable. `-d:icDerivedBarrier` reports every derived-flag write
onto a sealed type; a full `nim ic` build of a channel program produces
25, of which 24 are the `tfCheckedForDestructor` writes that were
already happening silently.
The Microsoft-hosted Azure agents are 2-core; the GitHub-hosted runners
are 4-core (3-core on macOS arm64). Measured on an identical `koch boot
-d:release` against our own Docs CI, the GitHub runners are ~2.3x
faster:
Linux boot 8.2 min -> 4.5 min
Windows boot 11.4 min -> 4.9 min
macOS boot 4.2 min -> 1.7 min
csources 2.0 min -> 0.8 min
`.github/workflows/ci_main.yml` keeps the same six jobs, the same runner
images, the same dependency installation and the same `ci/funs.sh` entry
points, so this is a move, not a redesign.
Differences forced by the platform:
* `[skip ci]` is handled natively by GitHub, so the `nimIsCiSkip` step
and the `skipci` variable that gated every step are gone. `nimIsCiSkip`
stays in `ci/funs.sh` for the version branches.
* `SYSTEM_ACCESSTOKEN` is gone: `testament/azure.nim` activates on
`TF_BUILD`, which is unset here, so it no-ops. This also removes a flake
source, where a failure to create the Azure test run cancelled an
otherwise green job.
* `concurrency: cancel-in-progress` replaces Azure's `pr.autoCancel`.
* `NIM_TESTAMENT_BATCH` defaults to `_` explicitly: a matrix-derived env
var is set to the empty string rather than left unset, so `getEnv`'s
default would not have applied.
`disabled: "azure"` was the only way to skip a test on the main
pipeline, so add `disabled: "github"` (`isGithubActions`) to replace it;
`azure` is kept as deprecated, alongside `travis` and `appveyor`.
Two manual steps remain: disabling the Azure pipeline definition, and
pointing the required status checks at the new job names.
---------
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
…s crashes codegen
fixes#26176
It follows the same method of the C backend. Don't insert "this" in
JavaScript backend If no variables are captured by closure functions.
`--app:lib` currently produces an ELF-style name when targeting iOS:
```console
$ nim c --app:lib --os:macosx --cpu:arm64 mylib.nim # -> libmylib.dylib
$ nim c --app:lib --os:ios --cpu:arm64 mylib.nim # -> libmylib.so
```
iOS is Darwin: its shared libraries are Mach-O `.dylib`, loaded by the
same dyld as on macOS. `compiler/platform.nim` already encodes this for
the `MacOSX` entry (`lib$1.dylib`); the `iOS` entry is the only Darwin
row still carrying the ELF name, so the same source yields a differently
named artifact depending on which Apple platform it is built for.
This PR adds a new Nim compiler command and introduces some improvements
to the docgen suite in general.
1. Adds `nim book`, the new command that takes a directory with
Markdown/ReST files and generates a navigatable, searchable, Nim-first
documentation site.
2. Refactors the default nimdoc.cfg, specifically the part marked with
"needs to be refactored." Code duplication was removed, new overridable
variables were added, quirky logic with the "Group by" switch display
was fixed.
Here's a live demo of a `nim book` produced book:
https://moigagoo.github.io/nim-chronos/
The original mdBook-powered version:
https://status-im.github.io/nim-chronos/
Related to this PR but valuable on their own:
1. `.. include::` directive has received several improvements:
- You can now include code from line to line, merged:
https://github.com/nim-lang/Nim/pull/26130
- You can now include code with syntax highlighting, merged:
https://github.com/nim-lang/Nim/pull/26146
2. `.. admonition::` directive (and its derivatives like `warning`,
`error`, etc.) got new useful functions:
- You can now set a title to your admonitions, open:
https://github.com/nim-lang/Nim/pull/26159
- You can make admonitions collapsible (useful when you need to include
a large chunk if code), open: https://github.com/nim-lang/Nim/pull/26159
Supersedes #26120 (https://github.com/nim-lang/Nim/pull/26120), which
fixed the reported case but left the `tfShallow` half of the same
condition open.
Since #25860 array literals are materialized into temporaries so that
`lent` results keep valid backing storage. In an async proc such a
temporary is lifted into the closure environment, and the environment is
filled with an `nkFastAsgn`, i.e. without `needToCopy`. `genGenericAsgn`
then emitted `genericShallowAssign` from the static const array, and for
the `string` elements that ends in `unsureAsgnRef` ->
`incRef(usrToCell(literal))` on memory that has no GC header:
proc f() {.async.} =
for ip in ["::1", "2001:db8::", "::"]:
await sleepAsync(1)
`OnStatic` sources must therefore always take the `genericAssign` path.
Note that the guard has to dominate the `tfShallow` test as well, not
just the `needToCopy` one: a `{.shallow.}` destination assigned from a
`const` crashes in exactly the same way. This is the precedence
`genOptAsgnTuple` and `genOptAsgnObject` already use, so
`genGenericAsgn` now agrees with its two siblings instead of
contradicting them.
refc only; the other GCs do not reference count in `unsureAsgnRef`.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…ult is never destroyed
fixes#26094
Conceptually, the new lowering for `destination = raisingCall()` is:
```nim
var tmp: T
try:
tmp = raisingCall()
let value = tmp
wasMoved(tmp)
destination = value
finally:
destroy(tmp)
```
So whether `raisingCall` Succeeds or not, `tmp` is destroyed
---------
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
fixes#26152
PR #26086 introduced {base, off, len} view wrappers for var openArray
arguments to preserve write-through semantics. This caused imported JS
pattern calls such as #.sort(#) to emit invalid object-literal syntax
instead of invoking the underlying array method.
Skip the view wrapper when generating arguments for imported pattern
calls, while retaining it for regular Nim procedures.
ref #26144
The C backend must not use `sfNeverRaises` to remove exception checks
from
virtual method calls. The flag describes only the selected base method
body,
while a vtable override may raise a catchable exception.
This change makes `canRaiseDisp` conservative for `skMethod` symbols and
adds a
regression test covering an exception raised by a child method invoked
through a
base reference.
Fixes#11797.
Imported scalar and pointer aliases inherit their external C spelling,
but
receive a different Nim symbol. Signature hashing previously used that
symbol
identity, so aliases that emit exactly the same C type could produce
different
backend names for tuples, sequences, and other generic types.
For example, `cint` and `type CIntAlias = cint` both emit `int`, but
`seq[cint]` and `seq[CIntAlias]` could be emitted as incompatible C
structs.
The Nim type checker nevertheless permits assignments and calls between
them,
causing the generated C or C++ compilation to fail.
This changes the backend hash to use the external type spelling when
available.
A symbol-based fallback remains for imported types without a resolved
spelling.
The change deliberately does not collapse imported types into their
underlying
Nim builtin. Types such as `pid_t`, imported pointers with qualifiers,
and
platform typedefs may require distinct backend representations.
## NIF and incremental compilation
This does not change NIF serialization, NIF type keys, or the IC cache
format.
The bug is in backend type-name generation. An IC regression test is
included
to ensure that the corrected backend identity is preserved when
compilation
passes through the NIF pipeline.
## Tests
The regressions cover:
- tuple and sequence assignments between an imported type and its alias
- cross-module sequence parameters and mutation
- C and C++ backends
- NIF-backed incremental compilation
Existing C-type tests were also run under C/C++, refc, and ARC.
## Remaining scope
This does not solve the broader question of compatibility between
imported and
builtin types that have different backend identities, such as
`seq[cdouble]` and `seq[float]`. That remains tracked by #19374.
Grinding a small figdraw-based program under `nim ic` and diffing its
output against the classic backend surfaced eight bugs, four of which
silently produced a wrong binary rather than an error.
Frontend / build graph (`deps.nim`):
* Dead `when`-guarded imports were compiled anyway. `when someStrdefine
== "x": import y` is `cvUnknown` to the scanner, which conservatively
keeps the edge — right for an edge, but it also gave `y` its own `nim m`
rule, so a build died on a package the user never installed because they
never selected that backend. Track which edges are speculative and drop
a speculative subtree that cannot compile; if the guard was in fact
live, the discovery fixpoint puts the node back with the honest `cannot
open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so
nothing re-fires and `nim ic` relinked a stale binary while `nim c`
reported `cannot open file`. Report an unresolvable import from a
non-speculatively reached module during the graph scan.
* Macro-generated imports were discovered once and then forgotten.
Discovery only ran after a failure and the graph is re-derived
statically every run, so on a warm build the discovered module had no
rules at all and editing it changed nothing. Seed the graph from the
`.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from
file mtimes and never looks at a rule's command line, so `-d:foo=bar` /
`--mm:` / `--threads:` regenerated the build file with the new switches
and re-fired zero rules. Reify the configuration as a file and make it
an input of every rule.
* Command-line switches never reached the children: they replay the
project's config files, never the driver's argv, so `nim ic --opt:speed`
produced a byte-identical debug binary (likewise `--panics`,
`--experimental`, `--passC`). Forward the driver's switches, minus the
ones that must differ per child.
Artifacts and codegen:
* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw
the rule as satisfied on the next run: `nim ic` then reported success
for a program that does not compile, and generated code from
error-bearing AST (or hit an internal error in `ccgexprs`). Never
persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives
on the module symbol, which `moduleFromNifFile` rebuilds from scratch,
so `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a
module-level `block: let h = openHandle()` never ran `=destroy`. Persist
the flag as a `(modflags)` record. `injectdestructors` also has to
tolerate the `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym
node is written as a bare NIF `SymUse` token, which has nowhere to put
node flags, so the frontend's move analysis never reached the backend:
EVERY first assignment to a destructor-bearing local compiled as
`=sink`, i.e. `=destroy` on still-zeroed memory followed by a copy, and
no read was ever a move. Wrap a sym use in `(nflags ...)` when it
carries persistent node flags.
In the JS backend `toOpenArray` used `slice` (a copy), so writes through
a `var openArray` parameter silently vanished.
This emits `subarray` (a live shared-buffer view) for homogeneous
numeric arrays, otherwise such parameters are passed as a `{base, off,
len}` view that always aliases the caller's storage. Sliced seq/array
args become `{base, off, len}`, whole values `{base, off:0, len}`,
re-slices rebase.
Un-skips the JS guard in tests/openarray/topenarray.nim
Fixes#15952.
fixes#26015
Fixes imported global variables with codegenDecl being emitted as
definitions instead of extern declarations.
A variable’s codegenDecl format should customize its definition in the
owning module. Other modules referencing the variable must emit a normal
declaration:
```c
extern NI variable;
```
After the variable-declaration builder refactor, genVarPrototype passed
Extern visibility to addVar. However, the sfCodegenDecl branch returned
before applying that visibility. This caused importing modules to emit
another tentative definition, resulting in duplicate-symbol linker
errors.
The fix restores the previous distinction between the custom definition
and cross-module prototypes. It also adds C and C++ regression coverage
for both direct access and access through an inline procedure.
follows up https://github.com/nim-lang/Nim/pull/24423
fixes#26123
`cast[T](x)` is a transparent path expression for compiler analysis.
Previously, move/alias analysis could fail to see a later use through a
cast and incorrectly mark the source as moved, causing the issue’s
segmentation fault.
for views,
https://nim-lang.org/docs/manual_experimental.html#view-types-path-expressions:
A cast expression cast[T](e) is a path expression.
It also affects skipConvDfa, isAnalysableFieldAccess, and aliases. And I
might narrow it down for the two cases above mentioned if it causes
problems
fixes#25992
```nim
type
Foo = object
case kind: bool
of true:
a: ref Bar # 8 bytes (pointer)
of false:
b: int # 4 bytes
```
specializeResetT for b emits accessor.b = 0 — writes 4 bytes
But the union is 8 bytes wide (sized by the largest branch)
The remaining 4 bytes where a used to live are untouched
Those stale bytes could contain a heap pointer the GC traces → crash
Add nimZeroMem after specializeResetN for case objects to clear the
entire union including unused branch bytes.
We defer copying the AST during generic instantiation until a cache miss
and fetch from cache based on bindings. On miss, we fall back to the old
logic and populate the cache.
This gains us a roughly 36% reduction in memory usage during bootstrap,
from `799.242MiB` down to `508.672MiB` on my machine.
For another point of reference, nimbus-eth2 goes from `10.5GB` memory
usage to `7.5GB`.
Independent companion to #26090 which together with this one yields a
bit under 20% faster compiles (or at least `--compileOnly` bootstraps)
on ORC.
…haracterization with--warning:ResultUsed:on
fixes#26062
> A return statement with no expression is shorthand for return result.
> ResultUsed: Warn about the usage of the built-in result variable.
> A procedure that does not have any return statement and does not use
the special result variable returns the value of its last expression.
`unapplyBlockContext` reset `inEnforcedGcSafe` and
`inEnforcedNoSideEffects` to false whenever a `{.cast(gcsafe).}` or
`{.cast(noSideEffect).}` block ended. When such a block is nested inside
another block of the same cast, the inner block's exit switched
enforcement back on for the rest of the enclosing block, so statements
lexically inside the outer cast were rejected.
The fix saves both flags in `PragmaBlockContext` when the block context
is created and restores the saved values on exit. That matches how every
other piece of block state there (`locked`, `exc`, `tags`, `forbids`) is
already handled; these two bools were the only ones reset to a constant
instead of restored.
No change for non-nested blocks: entering from the non-enforced state
saves false, so exit still clears the flag. A statement after the outer
block is still rejected as before.
Test covers nested `cast(gcsafe)` and nested `cast(noSideEffect)`.
`tests/effects` passes unchanged (49/49, same as stock).
Boostrap without linking is (conservatively) 17% faster on my machine.
Test setup runs the compiler compiling itself under ORC in release mode
from a clean cache and no C compilation (`--compileOnly`.)
Multiple samples are gathered before judging a potential optimization.
The test suite passes locally before push and bootstrap is tested with
strict views (mostly as a sanity check).
Tests are fair in the sense that they compile the same git worktree and
produce identical C output.
Refc will see less or no benefit.
Changes:
- Save tree traversals in `considerGenSyms` when no mappings exist
- Inline `maybeSkipDistinct` into `typeRel` for a safe cursor
- Only skip to static when there is a static type to reach in
`paramTypesMatchAux`
- Disable overflow checks for hashes
- Disable bounds checks for `nextIdentIter`
- Use cursor annotation when judged safe
- Use lent annotation for ast accessors
Spiritual companion to #26084
## Summary
- nimpretty with non-default \--indent\ (e.g. 3 or 10) produced invalid
indentation in if/block/try expression regions because layouter kept the
original column when \keepIndents > 0\ and ignored \indWidth\.
- Rebase the column onto \indWidth\ using the relative offset from the
enclosing block baseline (\indentStack[^1]\).
## Root cause
\parser.nim\'s \
imprettyDontTouch\ template sets \keepIndents\ for if/block/try
expressions. layouter in the \keepIndents > 0\ branch used \ ok.indent\
(source column) directly as \indentLevel\ without scaling by \indWidth\,
so lines in these regions kept the original column and misaligned with
the rest of the file when \--indent\ differed from source indent width.
## Fix
\\\
im
em.indentLevel = em.indentStack.high * em.indWidth +
(tok.indent - em.indentStack[^1])
\\\
Keeps the relative offset from the enclosing block baseline but rebases
onto \indWidth\. At default \--indent:2\ the offset equals \indWidth\,
so output is unchanged (backwards compatible).
## Testing
- 12 custom cases x 3 indent values (2/3/10) = 36/36 pass
- nimpretty self-test suite 7/7 pass (no regression at default indent)
- 5 keepIndents scenarios (if/block/try expression continuation
alignment) that failed at indent:3/10 now pass
Fixes#20078.
fixes#25942fixes#25938
After a successful match to a concrete static T, normalizes an empty
static container literal to the formal payload type before binding it.
This prevents `static[set[empty]]({})` from leaking into the
instantiated proc body.
fixes#26045;
fixes#26046
The fix isolates compiler option state while semantically checking each
when nimvm branch.
compiler/semexprs.nim:2745 snapshots the option stack, compiler options,
diagnostics settings, and enabled features. It analyzes one branch and
restores that state in finally. Both the nimvm and else branches use
this function.
This prevents:
```nim
when nimvm:
{.push overflowChecks: off.}
```
from disabling overflow checks in following runtime code. It also means
a {.pop.} in the opposite branch correctly reports that it has no
corresponding {.push.}.
