`rotateLeft` and `rotatedLeft` raised `DivByZeroDefect` for empty
containers because their whole-container overloads computed `dist mod
arg.len` before checking for an empty input.
Handle empty inputs explicitly:
- `rotateLeft` returns `0` and leaves the container unchanged.
- `rotatedLeft` returns an empty sequence.
Add a regression test covering both overloads.
The slice overloads are intentionally left unchanged because zero-length
slice semantics need separate consideration.
Signed-off-by: cuishuang <imcusg@gmail.com>
The Microsoft-hosted Azure agents are 2-core; the GitHub-hosted runners
are 4-core (3-core on macOS arm64). Measured on an identical `koch boot
-d:release` against our own Docs CI, the GitHub runners are ~2.3x
faster:
Linux boot 8.2 min -> 4.5 min
Windows boot 11.4 min -> 4.9 min
macOS boot 4.2 min -> 1.7 min
csources 2.0 min -> 0.8 min
`.github/workflows/ci_main.yml` keeps the same six jobs, the same runner
images, the same dependency installation and the same `ci/funs.sh` entry
points, so this is a move, not a redesign.
Differences forced by the platform:
* `[skip ci]` is handled natively by GitHub, so the `nimIsCiSkip` step
and the `skipci` variable that gated every step are gone. `nimIsCiSkip`
stays in `ci/funs.sh` for the version branches.
* `SYSTEM_ACCESSTOKEN` is gone: `testament/azure.nim` activates on
`TF_BUILD`, which is unset here, so it no-ops. This also removes a flake
source, where a failure to create the Azure test run cancelled an
otherwise green job.
* `concurrency: cancel-in-progress` replaces Azure's `pr.autoCancel`.
* `NIM_TESTAMENT_BATCH` defaults to `_` explicitly: a matrix-derived env
var is set to the empty string rather than left unset, so `getEnv`'s
default would not have applied.
`disabled: "azure"` was the only way to skip a test on the main
pipeline, so add `disabled: "github"` (`isGithubActions`) to replace it;
`azure` is kept as deprecated, alongside `travis` and `appveyor`.
Two manual steps remain: disabling the Azure pipeline definition, and
pointing the required status checks at the new job names.
---------
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
wrapWords(splitLongWords = true) dropped the separator before a word it
had to split, fusing it with the word before it. Flush lastSep first,
the same as the branch that handles a word which fits.
Updates twordwrap's longlongwordRes fixture, which was pinned against
the old behavior at three points where a source line break normalizes to
a synthetic space.
…s crashes codegen
fixes#26176
It follows the same method of the C backend. Don't insert "this" in
JavaScript backend If no variables are captured by closure functions.
Fixes#26173.
## What breaks today
`encodings.convert` allocates its output buffer as `newString(s.len)`,
i.e. the
same size as the input. Any conversion that expands — which is most
non-ASCII to
UTF-8 — therefore hits `E2BIG` and then **resumes** the same `iconv_t`
where it
stopped.
That is fine for stateless encodings, but the shift state of a stateful
encoding
is not guaranteed to survive a short write, and on macOS it does not.
After the
`E2BIG` the converter behaves as if it were back in the initial state,
so the
tail of an ISO-2022-JP string is emitted as raw bytes:
```nim
import std/[encodings, unicode]
proc repeatedA(n: int): string =
result = "\x1B\x24\x42"
for _ in 0 ..< n: result.add "\x24\x22" # あ
result.add "\x1B\x28\x42"
let c = open("UTF-8", "ISO-2022-JP")
for n in 1 .. 10:
echo n, " chars in -> ", c.convert(repeatedA(n)).runeLen, " chars out"
```
```
6 chars in -> 6 chars out
7 chars in -> 8 chars out <-- wrong
10 chars in -> 12 chars out <-- wrong
```
Nothing is raised. The text is just wrong, and only in the tail, so
short test
strings pass and real data does not. The failure correlates exactly with
whether
the output buffer has to grow:
| chars | input bytes | expected output bytes | growth needed | result |
|---|---|---|---|---|
| 6 | 18 | 18 | no | correct |
| 7 | 20 | 21 | **yes** | corrupted |
EUC-JP, which has the same 2-to-3 byte expansion but no shift state, is
correct
at every length — so it is the statefulness, not the growth ratio, that
matters.
The issue has a raw-C-API reproducer showing the state loss happens
inside
`iconv` and is not an artifact of the Nim string handling.
## What this PR does
Two commits, because they are two separate defects:
**1. `fixes #26173; don't resume iconv after a short write`**
Reset the converter with `iconv(c, nil, nil, nil, nil)` and redo the
whole
conversion into a larger buffer instead of resuming. Adds a regression
test that
fails on `devel` and passes with the fix.
**2. `fix out-of-bounds write in encodings.convert on a full output
buffer`**
The `EILSEQ`/`EINVAL` branch does `dst[0] = src[0]` and `dec(outLen)`
without
checking there is room, so a full output buffer writes one byte past the
end and
underflows `outLen` (a `csize_t`). Guarded with `outLen > 0`, letting
the
buffer-growth path handle the full-buffer case.
This is a latent bug independent of #26173, found while working on it —
happy to
split it into its own PR if that is preferred.
## Testing
`tests/stdlib/tencodings.nim` gains coverage for ISO-2022-JP across the
buffer
growth boundary (1..64 chars, plus a string with several ASCII/JIS state
switches) and a stateless EUC-JP case that also crosses the boundary.
- Fails on `devel` at `tencodings.nim(124)` without the fix
- Passes with the fix under both `--mm:refc` and `--mm:orc`
- Existing assertions in the file are unaffected
Verified on macOS 15 / arm64. The Windows path (`convertWin`) does not
use
`iconv` and is untouched; ISO-2022-JP is already in `nameToCodePage` as
50220, so
the new test exercises that path there too.
Fixes#25200.
`xtensa-esp-elf-gcc` (the ESP-IDF toolchain for the Xtensa
ESP32/ESP32-S2/ESP32-S3) defines `int32_t` as `long int`, exactly like
`arm-none-eabi-gcc` and `riscv32-unknown-elf-gcc` do:
```console
$ xtensa-esp32s3-elf-gcc -dM -E -x c /dev/null | grep -E '__INT32_TYPE__|__SIZEOF_(INT|LONG)__|__xtensa__|__unix__'
#define __SIZEOF_INT__ 4
#define __SIZEOF_LONG__ 4
#define __xtensa__ 1
#define __INT32_TYPE__ long int
```
So with `--cpu:esp`, `NI`/`NI32` become `long int` while `nimAddInt`
still expands to `__builtin_sadd_overflow`, which expects `int*`:
```
error: passing argument 3 of '__builtin_sadd_overflow' from incompatible pointer type [-Wincompatible-pointer-types]
note: expected 'int *' but argument is of type 'NI32 *' {aka 'long int *'}
```
This is a warning on GCC 13 and below, but [a hard error since GCC
14](https://gcc.gnu.org/gcc-14/porting_to.html#incompatible-pointer-types)
— the log in #25200 shows the build failing outright with the
`esp-15.1.0` toolchain.
This applies the same fix that #23835 (arm-none-eabi), #24553 (riscv32)
and #26121 (limiting arm to non-`__unix__`) already applied, now for
Xtensa. The `!defined(__unix__)` guard mirrors #26121: the bare-metal
ESP toolchain does not define `__unix__` (verified above), so it is
unaffected, while a hypothetical `xtensa-*-linux` target keeps the
current `int` behaviour rather than being switched blindly.
### Verification
Compiling Nim-generated C for `--cpu:esp --os:standalone` with
`xtensa-esp32s3-elf-gcc 12.2.0` and `-Werror=incompatible-pointer-types`
(to reproduce the GCC 14+ default):
| `lib/nimbase.h` | exit code | `incompatible pointer` diagnostics |
|---|---|---|
| devel | 1 | 1 |
| this PR | 0 | 0 |
No effect on any other target: the change is inside the `NIM_INTBITS ==
32` branch and is gated on `__xtensa__`.
`--app:lib` currently produces an ELF-style name when targeting iOS:
```console
$ nim c --app:lib --os:macosx --cpu:arm64 mylib.nim # -> libmylib.dylib
$ nim c --app:lib --os:ios --cpu:arm64 mylib.nim # -> libmylib.so
```
iOS is Darwin: its shared libraries are Mach-O `.dylib`, loaded by the
same dyld as on macOS. `compiler/platform.nim` already encodes this for
the `MacOSX` entry (`lib$1.dylib`); the `iOS` entry is the only Darwin
row still carrying the ELF name, so the same source yields a differently
named artifact depending on which Apple platform it is built for.
This PR adds a new Nim compiler command and introduces some improvements
to the docgen suite in general.
1. Adds `nim book`, the new command that takes a directory with
Markdown/ReST files and generates a navigatable, searchable, Nim-first
documentation site.
2. Refactors the default nimdoc.cfg, specifically the part marked with
"needs to be refactored." Code duplication was removed, new overridable
variables were added, quirky logic with the "Group by" switch display
was fixed.
Here's a live demo of a `nim book` produced book:
https://moigagoo.github.io/nim-chronos/
The original mdBook-powered version:
https://status-im.github.io/nim-chronos/
Related to this PR but valuable on their own:
1. `.. include::` directive has received several improvements:
- You can now include code from line to line, merged:
https://github.com/nim-lang/Nim/pull/26130
- You can now include code with syntax highlighting, merged:
https://github.com/nim-lang/Nim/pull/26146
2. `.. admonition::` directive (and its derivatives like `warning`,
`error`, etc.) got new useful functions:
- You can now set a title to your admonitions, open:
https://github.com/nim-lang/Nim/pull/26159
- You can make admonitions collapsible (useful when you need to include
a large chunk if code), open: https://github.com/nim-lang/Nim/pull/26159
Supersedes #26120 (https://github.com/nim-lang/Nim/pull/26120), which
fixed the reported case but left the `tfShallow` half of the same
condition open.
Since #25860 array literals are materialized into temporaries so that
`lent` results keep valid backing storage. In an async proc such a
temporary is lifted into the closure environment, and the environment is
filled with an `nkFastAsgn`, i.e. without `needToCopy`. `genGenericAsgn`
then emitted `genericShallowAssign` from the static const array, and for
the `string` elements that ends in `unsureAsgnRef` ->
`incRef(usrToCell(literal))` on memory that has no GC header:
proc f() {.async.} =
for ip in ["::1", "2001:db8::", "::"]:
await sleepAsync(1)
`OnStatic` sources must therefore always take the `genericAssign` path.
Note that the guard has to dominate the `tfShallow` test as well, not
just the `needToCopy` one: a `{.shallow.}` destination assigned from a
`const` crashes in exactly the same way. This is the precedence
`genOptAsgnTuple` and `genOptAsgnObject` already use, so
`genGenericAsgn` now agrees with its two siblings instead of
contradicting them.
refc only; the other GCs do not reference count in `unsureAsgnRef`.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…ult is never destroyed
fixes#26094
Conceptually, the new lowering for `destination = raisingCall()` is:
```nim
var tmp: T
try:
tmp = raisingCall()
let value = tmp
wasMoved(tmp)
destination = value
finally:
destroy(tmp)
```
So whether `raisingCall` Succeeds or not, `tmp` is destroyed
---------
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Fixes for a few crashes, including a runtime underflow, I stumbled upon
while trying some more _inventive_ patterns.
* `primary()`: unary `*` applied to an operand that can match empty
input (!>.*), 'a'?) now pegError()s at pattern-parse time instead of
AssertionDefect "unreachable" from `*`
* `getCharSet()`: unknown builtin/malformed escapes inside charsets
([^\n], [z-\n]) propagate `tkInvalid` instead of reading
`tok.literal[len-1]` of an empty string (IndexDefect)
* rawMatch pkCapture: `{}` with no previous capture is a no-op instead
of a runtime underflow defect
fixes#26152
PR #26086 introduced {base, off, len} view wrappers for var openArray
arguments to preserve write-through semantics. This caused imported JS
pattern calls such as #.sort(#) to emit invalid object-literal syntax
instead of invoking the underlying array method.
Skip the view wrapper when generating arguments for imported pattern
calls, while retaining it for regular Nim procedures.
ref #26144
The C backend must not use `sfNeverRaises` to remove exception checks
from
virtual method calls. The flag describes only the selected base method
body,
while a vtable override may raise a catchable exception.
This change makes `canRaiseDisp` conservative for `skMethod` symbols and
adds a
regression test covering an exception raised by a child method invoked
through a
base reference.
Fixes#11797.
Imported scalar and pointer aliases inherit their external C spelling,
but
receive a different Nim symbol. Signature hashing previously used that
symbol
identity, so aliases that emit exactly the same C type could produce
different
backend names for tuples, sequences, and other generic types.
For example, `cint` and `type CIntAlias = cint` both emit `int`, but
`seq[cint]` and `seq[CIntAlias]` could be emitted as incompatible C
structs.
The Nim type checker nevertheless permits assignments and calls between
them,
causing the generated C or C++ compilation to fail.
This changes the backend hash to use the external type spelling when
available.
A symbol-based fallback remains for imported types without a resolved
spelling.
The change deliberately does not collapse imported types into their
underlying
Nim builtin. Types such as `pid_t`, imported pointers with qualifiers,
and
platform typedefs may require distinct backend representations.
## NIF and incremental compilation
This does not change NIF serialization, NIF type keys, or the IC cache
format.
The bug is in backend type-name generation. An IC regression test is
included
to ensure that the corrected backend identity is preserved when
compilation
passes through the NIF pipeline.
## Tests
The regressions cover:
- tuple and sequence assignments between an imported type and its alias
- cross-module sequence parameters and mutation
- C and C++ backends
- NIF-backed incremental compilation
Existing C-type tests were also run under C/C++, refc, and ARC.
## Remaining scope
This does not solve the broader question of compatibility between
imported and
builtin types that have different backend identities, such as
`seq[cdouble]` and `seq[float]`. That remains tracked by #19374.
Grinding a small figdraw-based program under `nim ic` and diffing its
output against the classic backend surfaced eight bugs, four of which
silently produced a wrong binary rather than an error.
Frontend / build graph (`deps.nim`):
* Dead `when`-guarded imports were compiled anyway. `when someStrdefine
== "x": import y` is `cvUnknown` to the scanner, which conservatively
keeps the edge — right for an edge, but it also gave `y` its own `nim m`
rule, so a build died on a package the user never installed because they
never selected that backend. Track which edges are speculative and drop
a speculative subtree that cannot compile; if the guard was in fact
live, the discovery fixpoint puts the node back with the honest `cannot
open file`.
* Deleting a still-imported module went unnoticed: no mtime moves, so
nothing re-fires and `nim ic` relinked a stale binary while `nim c`
reported `cannot open file`. Report an unresolvable import from a
non-speculatively reached module during the graph scan.
* Macro-generated imports were discovered once and then forgotten.
Discovery only ran after a failure and the graph is re-derived
statically every run, so on a warm build the discovered module had no
rules at all and editing it changed nothing. Seed the graph from the
`.s.deps` sidecars up front.
* Config changes invalidated nothing. nifmake decides staleness from
file mtimes and never looks at a rule's command line, so `-d:foo=bar` /
`--mm:` / `--threads:` regenerated the build file with the new switches
and re-fired zero rules. Reify the configuration as a file and make it
an input of every rule.
* Command-line switches never reached the children: they replay the
project's config files, never the driver's argv, so `nim ic --opt:speed`
produced a byte-identical debug binary (likewise `--panics`,
`--experimental`, `--passC`). Forward the driver's switches, minus the
ones that must differ per child.
Artifacts and codegen:
* A failed `nim m` still wrote its `.s.bif` and cookies, so nifmake saw
the rule as satisfied on the next run: `nim ic` then reported success
for a program that does not compile, and generated code from
error-bearing AST (or hit an internal error in `ccgexprs`). Never
persist an artifact when `errorCounter > 0`.
* Top-level destructors were never injected. `sfInjectDestructors` lives
on the module symbol, which `moduleFromNifFile` rebuilds from scratch,
so `genTopLevelStmt` skipped `injectDestructorCalls` entirely: a
module-level `block: let h = openHandle()` never ran `=destroy`. Persist
the flag as a `(modflags)` record. `injectdestructors` also has to
tolerate the `nkReplayAction` entries the loader prepends to `topLevel`.
* `nfFirstWrite` / `nfLastRead` were dropped by the serializer. A sym
node is written as a bare NIF `SymUse` token, which has nowhere to put
node flags, so the frontend's move analysis never reached the backend:
EVERY first assignment to a destructor-bearing local compiled as
`=sink`, i.e. `=destroy` on still-zeroed memory followed by a copy, and
no read was ever a move. Wrap a sym use in `(nflags ...)` when it
carries persistent node flags.
In the JS backend `toOpenArray` used `slice` (a copy), so writes through
a `var openArray` parameter silently vanished.
This emits `subarray` (a live shared-buffer view) for homogeneous
numeric arrays, otherwise such parameters are passed as a `{base, off,
len}` view that always aliases the caller's storage. Sliced seq/array
args become `{base, off, len}`, whole values `{base, off:0, len}`,
re-slices rebase.
Un-skips the JS guard in tests/openarray/topenarray.nim
Fixes#15952.
#23835 tried to do this, but it also switched to `long int` on the GNU
ABI, where it's actually just `int`.
Fixes#26111
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
fixes#26015
Fixes imported global variables with codegenDecl being emitted as
definitions instead of extern declarations.
A variable’s codegenDecl format should customize its definition in the
owning module. Other modules referencing the variable must emit a normal
declaration:
```c
extern NI variable;
```
After the variable-declaration builder refactor, genVarPrototype passed
Extern visibility to addVar. However, the sfCodegenDecl branch returned
before applying that visibility. This caused importing modules to emit
another tentative definition, resulting in duplicate-symbol linker
errors.
The fix restores the previous distinction between the custom definition
and cross-module prototypes. It also adds C and C++ regression coverage
for both direct access and access through an inline procedure.
follows up https://github.com/nim-lang/Nim/pull/24423
Specialize `rawAlloc` for alignment (cherry-picked from the other PR).
This cuts the frame of the normal unaligned path down enough to regain
the performance lost from loading the cold page in #26110
Also cleans up `MemRegion` a bit, the regressions are either gone or
were measurement errors.
fixes#26123
`cast[T](x)` is a transparent path expression for compiler analysis.
Previously, move/alias analysis could fail to see a later use through a
cast and incorrectly mark the source as moved, causing the issue’s
segmentation fault.
for views,
https://nim-lang.org/docs/manual_experimental.html#view-types-path-expressions:
A cast expression cast[T](e) is a path expression.
It also affects skipConvDfa, isAnalysableFieldAccess, and aliases. And I
might narrow it down for the two cases above mentioned if it causes
problems
Issue popped up when using `fromJson` into an array but the JSON passed
is an object
```nim
import std/[jsonutils, json]
let data = parseJson """
{"key": "value"}
"""
var foo: seq[int]
foo.fromJson(data)
echo foo #> @[0]
```
Basically the `setLen` would set the size to be equal to the number of
keys, but `getElems` just returns an empty array if the JSON isn't an
array which lead to it just creating zero'd items in the seq without
letting the user know.
Felt adding the checks was better than just skipping the `setLen` since
it lets the user know that there is a problem with the JSON
fixes#25992
```nim
type
Foo = object
case kind: bool
of true:
a: ref Bar # 8 bytes (pointer)
of false:
b: int # 4 bytes
```
specializeResetT for b emits accessor.b = 0 — writes 4 bytes
But the union is 8 bytes wide (sized by the largest branch)
The remaining 4 bytes where a used to live are untouched
Those stale bytes could contain a heap pointer the GC traces → crash
Add nimZeroMem after specializeResetN for case objects to clear the
entire union including unused branch bytes.
We defer copying the AST during generic instantiation until a cache miss
and fetch from cache based on bindings. On miss, we fall back to the old
logic and populate the cache.
This gains us a roughly 36% reduction in memory usage during bootstrap,
from `799.242MiB` down to `508.672MiB` on my machine.
For another point of reference, nimbus-eth2 goes from `10.5GB` memory
usage to `7.5GB`.
Independent companion to #26090 which together with this one yields a
bit under 20% faster compiles (or at least `--compileOnly` bootstraps)
on ORC.
fixes#26088
in https://github.com/nim-lang/Nim/pull/25772, `beginStores` requires
`newLen` to be passed for setting up the new length. So `streams.nim`
must make the string length exactly newLen.
…haracterization with--warning:ResultUsed:on
fixes#26062
> A return statement with no expression is shorthand for return result.
> ResultUsed: Warn about the usage of the built-in result variable.
> A procedure that does not have any return statement and does not use
the special result variable returns the value of its last expression.
`unapplyBlockContext` reset `inEnforcedGcSafe` and
`inEnforcedNoSideEffects` to false whenever a `{.cast(gcsafe).}` or
`{.cast(noSideEffect).}` block ended. When such a block is nested inside
another block of the same cast, the inner block's exit switched
enforcement back on for the rest of the enclosing block, so statements
lexically inside the outer cast were rejected.
The fix saves both flags in `PragmaBlockContext` when the block context
is created and restores the saved values on exit. That matches how every
other piece of block state there (`locked`, `exc`, `tags`, `forbids`) is
already handled; these two bools were the only ones reset to a constant
instead of restored.
No change for non-nested blocks: entering from the non-enforced state
saves false, so exit still clears the flag. A statement after the outer
block is still rejected as before.
Test covers nested `cast(gcsafe)` and nested `cast(noSideEffect)`.
`tests/effects` passes unchanged (49/49, same as stock).
