Nested transformBody/liftLambdas passes used a fresh DetectionPass, so
getEnvTypeForOwner could allocate a duplicate PType for the same owner
while :envP already referenced the inner pass type. When addClosureParam
saw cp.typ != t, it errored.
If both types are env objects for the same routine owner, reuse cp.typ
and sync ownerToType.
Adds regression test tests/iter/t21242_nested_closure_in_iter.nim.
This fixes type resolution for `iterable[T]`.
I want to proceed with RFC
[#562](https://github.com/nim-lang/RFCs/issues/562) and this is the main
blocker for composability.
Fixes#22098 and, arguably, #19206
```nim
import std/strutils
template collect[T](it: iterable[T]): seq[T] =
block:
var res: seq[T] = @[]
for x in it:
res.add x
res
const text = "a b c d"
let words = text.split.collect()
doAssert words == @[ "a", "b", "c", "d" ]
```
In cases like `strutils.split`, where both proc and iterator overload
exists, the compiler resolves to the `func` overload causing a type
mismatch.
The old mode resolved `text.split` to `seq[string]` before the
surrounding `iterable[T]` requirement was applied, so the argument no
longer matched this template.
It should be noted that, compared to older sequtils templates,
composable chains based on `iterable[T]` require an iterator-producing
expression, e.g. `"foo".items.iterableTmpl()` rather than just
`"foo".iterableTmpl()`. This is actually desirable: it keeps the
iteration boundary explicit and makes iterable-driven templates
intentionally not directly interchangeable with older
untyped/loosely-typed templates like those in `sequtils`, whose internal
iterator setup we have zero control over (e.g. hard-coding adapters like
`items`).
Also, I noticed in `semstmts` that anonymous iterators are always
`closure`, which is not that surprising if you think about it, but still
I added a paragraph to the manual.
Regarding implementation:
From what I gathered, the root cause is that `semOpAux` eagerly
pre-types all arguments with plain flags before overload resolution
begins, so by the time `prepareOperand` processes `split` against the
`iterable[T]`, the wrong overload has already won.
The fix touches a few places:
- `prepareOperand` in `sigmatch.nim`:
When `formal.kind == tyIterable` and the argument was already typed as
something else, it's re-semchecked with the
`efPreferIteratorForIterable` flag. The recheck is limited to direct
calls (`a[0].kind in {nkIdent, nkAccQuoted, nkSym, nkOpenSym}`) to avoid
recursing through `semIndirectOp`/`semOpAux` again.
- `iteratorPreference` field `TCandidate`, checked before
`genericMatches` in `cmpCandidates`, gives the iterator overload a win
without touching the existing iterator heuristic used by `for` loops.
**Limitations:**
The implementation is still flag-driven rather than purely
formal-driven, so the behaviour is a bit too broad `efWantIterable` can
cause iterator results to be wrapped as `tyIterable` in
iterable-admitting contexts, not only when `iterable[T]` match is being
processed.
`iterable[T]` still does not accept closure iterator values such
as`iterator(): T {.closure.}`. It only matches the compiler's internal
`tyIterable`, not arbitrary iterator-typed values.
The existing iterator-preference heuristic is still in place, because
when I tried to remove it, some loosely-related regressions happened. In
particular, ordinary iterator-admitting contexts and iterator chains
still rely on early iterator preference during semchecking, before the
compiler has enough surrounding context to distinguish between
value/iterator producing overloads. Full heuristic removal would require
a broader refactor of dot-chain/intermediate-expression semchecking,
which is just too much for me ATM. This PR narrows only the
tyIterable-specific cases.
**Future work:**
Rework overload resolution to preserve additional information of
matching iterator overloads for calls up to the point where the
iterator-requiring context is established, to avoid re-sem in
`prepareOperand`.
Currently there's no good channel to store that information. Nodes can
get rewritten, TCandidate doesn't live long enough, storing in Context
or some side-table raises the question how to properly key that info.
fixes#25682
This pull request introduces a fix to the Nim compiler's assignment code
generation logic to better handle statement list expressions, and adds
regression tests to ensure correct behavior when assigning to object
fields via templates. The changes address a specific bug (#25682)
related to assignments using templates with side effects in static
contexts.
**Compiler code generation improvements:**
* Updated the `genAsgn` procedure in `compiler/vmgen.nim` to properly
handle assignments where the left-hand side is a `nkStmtListExpr`
(statement list expression), ensuring all statements except the last are
executed before the assignment occurs.
**Regression tests for assignment semantics:**
* Added new test blocks in `tests/vm/tvmmisc.nim` to verify that
template-based assignments to object fields work as expected in static
contexts, specifically testing for bug #25682.
```nim
template compute(body: untyped): int =
block:
body
let x = compute:
var sum = 0
for i in 1..10: sum += i
sum
echo x
```
supersedes https://github.com/nim-lang/Nim/pull/25653
which in
02893e2f4c
```nim
of nkSym:
genSingleVar(p, it.sym, newSymNode(it.sym), it.sym.astdef)
```
A new branch for `nkSym` is added, though more changes might be needed
if `nkSym` is handled specifically
fixes#25677;
fixes#25678
This pull request introduces both a bug fix to the type checking logic
in the compiler and new test cases for lent types involving procedures
and tables. The most significant change is a refinement in how type
flags are handled for procedure and function types in the compiler,
which improves correctness in type allowance checks. Additionally, the
test suite is expanded to cover more complex scenarios with lent types
and table lookups.
**Compiler improvements:**
* Refined the handling of type flags in `typeAllowedAux` for procedure
and function types by introducing `innerFlags`, which removes certain
flags (`taObjField`, `taTupField`, `taIsOpenArray`) before recursing
into parameter and return types. This ensures more accurate type
checking and prevents inappropriate flag propagation.
**Testing enhancements:**
* Added new test blocks in `tests/lent/tlents.nim` to cover lent
procedure types stored in objects and used as table values, including a
function that retrieves such procedures from a table by key.
* Introduced a test case for an object containing a lent procedure
field, ensuring correct behavior when accessing and using these fields.
Fixes bug #25674.
`replace` read `s[i+1]` for a CRLF pair without ensuring `i+1 <
s.len()`, so a value ending in a lone `\\c` (quoted in `writeConfig`)
raised `IndexDefect`.
- Fix: only treat `\\c\\l` when the following character exists.
- Test: `tests/stdlib/tparsecfg.nim` block bug #25674 — fails before
fix, passes after.
This pull request includes a few targeted changes across the codebase,
primarily focusing on improving symbol locality detection in the
compiler, adding a utility function for integer division and modulus,
and simplifying a test case.
- **Compiler Improvements**
* Improved the `isLocalSym` function in `compiler/ast2nif.nim` to more
accurately determine if a symbol is local by checking that the symbol's
owner is not a module.
- **Utility Function Addition**
* Added a new `divmod` procedure in `tests/ic/tmiscs.nim` that returns
both the quotient and remainder of integer division, along with a usage
example.
- **Test Simplification**
* Simplified the `showMeters` test in `tests/ic/tconverter.nim` by
removing a floating-point assertion, leaving only an output statement.
------------------------------------------------------------------------------------------------------------------
```nim
proc divmod(a, b: int): (int, int) =
(a div b, a mod b)
let (q, r) = divmod(17, 5)
echo q
echo r
```
gives `Error: unhandled exception: local symbol 'tmpTuple.0' not found
in localSyms. [AssertionDefect]`
`makeVarTupleSection` uses a temp of which the globalness and localness
is not specified. Turning it a global variable for top level scope broke
some Nim programs. So I think it's better to check the owner of the
symbol
```nim
if useTemp:
# use same symkind for compatibility with original section
let temp = newSym(symkind, getIdent(c.cache, "tmpTuple"), c.idgen, getCurrOwner(c), n.info)
```
fixes#25620
This pull request includes a fix to the type key generation logic in the
compiler and updates to a test file to cover additional language
features. The most important changes are summarized below:
### Compiler logic fix
* In `compiler/typekeys.nim`, the `typeKey` procedure was updated to
iterate over all elements in `t.sonsImpl` starting from index 0 instead
of 1, ensuring that all type sons are considered during type key
generation.
### Test suite improvements
* The test file `tests/ic/tenum.nim` was renamed to
`tests/ic/tmiscs.nim`, and its output expectations were updated to
reflect the new test cases.
* Added new test cases to `tests/ic/tmiscs.nim` to cover sink and move
semantics, including the definition of a `BigObj` type and a `consume`
procedure that demonstrates moving and consuming large objects.
```nim
# Sink and move semantics
type
BigObj = object
data: seq[int]
proc consume(x: sink BigObj) =
echo x.data.len
var b = BigObj(data: @[1, 2, 3, 4, 5])
consume(move b)
```
gives
```
error: passing 'tySequence__qwqHTkRvwhrRyENtudHQ7g' (aka 'struct tySequence__qwqHTkRvwhrRyENtudHQ7g') to parameter of incompatible type 'tySequence__cTyVHeHOWk5jStsToosJ8Q' (aka 'struct tySequence__cTyVHeHOWk5jStsToosJ8Q')
84 | eqdestroy___sysma2dyk_u75((*dest_p0).data);
```
follows up https://github.com/nim-lang/Nim/pull/25614
`getTypeImpl` and friends were always putting `nkEmpty` in the default
value field which meant the default values couldn't be introspected.
This copies the default AST so it can be seen in the returned object
On simple code like:
```nim
type Foo = object
case x: range[0..7]
of 0..2:
a: string
else:
b: string
var foo = Foo()
{.cast(uncheckedAssign).}:
foo.x = 5
```
The compiler tries to generate a destructor for the variant fields by
checking if the discrim is equal to the old one, but the type is not
skipped when looking for an `==` operator in system, so any
discriminator with type `range`/`distinct`/etc crashes with:
```
(10, 9) Error: can't find magic equals operator for type kind tyRange
```
This is fixed by just skipping abstract types.
fixes#25262
```nim
if constraint != nil and constraint.kind == tyTypeDesc:
n[i].typ = e.typ
else:
n[i].typ = e.typ.skipTypes({tyTypeDesc})
```
at least when `constraint` is a typedesc, it should not skip
`tyTypeDesc`
```nim
if arg.kind != tyTypeDesc:
arg = makeTypeDesc(m.c, arg)
```
Wrappers literals into typedesc, which can cause problems. Though, it
doesn't seem to be necessary
1. A trailing `$` at the end of a replacement string could read out of
bounds via `how[i + 1]`; this now raises `ValueError` instead.
2. Numeric capture parsing used `id += (id * 10) + digit` instead of `id
= (id * 10) + digit`, so multi-digit refs were parsed incorrectly (e.g.
`$12` resolved as capture 13 instead of 12).
4. Unterminated named replacement syntax (e.g. `${foo)` is now rejected
with ValueError instead of being accepted and parsed inconsistently.
Found and fixed by GPT 5.3 Codex.
This PR allows passing the defining type to generic types in the right
side in a type definition like this:
```nim
type
Foo = object
x: Option[Foo]
```
I think generic types should be instanciated after all given arguments
are semchecked,
because generic types can access information about them.
(for example, `Option[T]` in std/option checks if `T` is a pointer like
type)
But in this case, need to instanciate `Option[Foo]` before type of
`Foo.x` is determined.
fixes#25005
In `semTypeIdent`, when resolving a typedesc parameter inside a generic
instantiation, the code took a shortcut: it returned the symbol of the
element type (`bound = result.typ.elementType.sym`). However, for
generic types like `RpcResponse[T] = ref object`, the instantiated
object type (e.g., `RpcResponse:ObjectType[string]`) is a copy with a
new type ID but still points to the same symbol as the uninstantiated
generic body type. That symbol's .typ refers to the original
uninstantiated type, which still contains unresolved generic params `T`
Adds configurable parser modes to std/parseopt module. **Take two.**
Initially solved the issue of not being able to pass arguments to short
options as you do with most everyday CLI programs, but reading the tests
made me add more features so that some of the behaviour could be changed
and here we are.
**`std/parseopt` now supports three parser modes** via an optional
`mode` parameter in `initOptParser` and `getopt`.
Three modes are provided:
- `NimMode` (default, fully backward compatible),
- `LaxMode` (POSIX-inspired with relaxed short option handling),
- `GnuMode` (stricter GNU-style conventions).
The new modes are marked as experimental in the documentation.
The parser behaviour is controlled by a new `ParserRules` enum, which
provides granular feature flags that modes are built from. This makes it
possible for users with specific requirements to define custom rule sets
by importing private symbols, this is mentioned but clearly marked as
unsupported.
**Backward compatibility:**
The default mode preserves existing behaviour completely, with a single
exception: `allowWhitespaceAfterColon` is deprecated.
Now, `allowWhitespaceAfterColon` doesn't make much sense as a single
tuning knob. The `ParserRule.prSepAllowDelimAfter` controls this now.
As `allowWhitespaceAfterColon` had a default, most calls never mention
it so they will silently migrate to the new `initOptParser` overload. To
cover cases when the proc param was used at call-site, I added an
overload, which modifies the default parser mode to reflect the required
`allowWhitespaceAfterColon` value. Should be all smooth for most users,
except the deprecation warning.
The only thing I think can be classified as the breaking change is a
surprising **bug** of the old parser:
```nim
let p = initOptParser("-n 10 -m20 -k= 30 -40", shortNoVal = {'v'})
# ^-disappears
```
This is with the aforementioned `allowWhitespaceAfterColon` being true
by default, of course. In this case the `30` token is skipped
completely. I don't think that's right, so it's fixed.
Things I still don't like about how the old parser and the new default
mode behave:
1. **Parser behaviour is controlled by an emptiness of two containers**.
This is an interesting approach. It's also made more interesting because
the `shortNoVal`/`longNoVal` control both the namesakes, but *and also
how their opposites (value-taking opts) work*.
---
**Edit:**
2. `shortNoVal` is not mandatory:
```nim
let p = initOptParser(@["-a=foo"], shortNoVal = {'a'})
# Nim, Lax parses as: (cmdShortOption, "a", "foo")
# GnuMode parses as: (cmdShortOption, "a", "=foo")
```
In this case, even though the user specified `a` as no no-val, parser
ignores it, relying only on the syntax to decide the kind of the
argument. This is especially problematic with the modes that don't use
the rule `prShortAllowSep` (GnuMode), in this case the provided input is
twice invalid, regardless of the `shortNoVal`.
With the current parser architecture, parsing it this way **is
inevitable**, though. We don't have any way to signal the error state
detected with the input, so the user is expected to validate the input
for mistakes.
Bundling positional arguments is nonsensical and short option can't use
the separator character, so `[cmd "a", arg "=foo"]` and `[cmd "a", cmd
"=", cmd "f"...]` are both out of the question **and** would complicate
validating, requiring keeping track of a previous argument. Hope I'm
clear enough on the issue.
**Future work:**
1. Looks like the new modes are already usable, but from the discussions
elsewhere it looks like we might want to support special-casing
multi-digit short options (`-XX..`) to allow numerical options greater
than 9. This complicates bundling, though, so requires a bit of thinking
through.
2. Signaling error state?
---------
Co-authored-by: Andreas Rumpf <araq4k@proton.me>
fixes#25475
```nim
var x: array[0..1, int] = [0, 1]
var y: array[4'u..5'u, int] = [0, 3]
echo x == y
```
sigmatch treats array compatibility by element type + length, not by the
index (range) type. Perhaps backend should do the same check
This fixes two issues with impotc'ed types.
1. Passing an importc'ed inherited object to where superclass is
expected emitted `v.Sup` previously. Now it emits `v`, similar to cpp
codegen.
2. Casting between different nim types that resolve to the same C type
previously was done like `*(T*)&v`, now it is just `v`.
fixes https://github.com/nim-lang/Nim/issues/25457
Small chunks allocate memory in fixed-size cells. Each cell is
positioned at exact multiples of the cell size from the chunk's data
start, which makes it much harder to support alignment
```nim
sysAssert c.size == size, "rawAlloc 6"
if c.freeList == nil:
sysAssert(c.acc.int + smallChunkOverhead() + size <= SmallChunkSize,
"rawAlloc 7")
result = cast[pointer](cast[int](addr(c.data)) +% c.acc.int)
inc(c.acc, size)
```
See also https://github.com/nim-lang/Nim/pull/12926
While using big trunk, each allocation gets its own chunk
First performance numbers:
time tests/arc/torcbench -- YRC
true peak memory: true
real 0m0,163s
user 0m0,161s
sys 0m0,002s
time tests/arc/torcbench -- ORC
true peak memory: true
real 0m0,107s
user 0m0,104s
sys 0m0,003s
So it's 1.6x slower. But it's threadsafe and provably correct. (Lean and
model checking via TLA+ used.)
Of course there is always the chance that the implementation is wrong
and doesn't match the model.
`hashType` proc returned the same hash value from different instanced
generics types like `D[int64]` and `D[F]`.
That caused the struct type with wrong field types.
object/tuple type size check code is generated when it is compiled with
`-d:checkAbi` option.
