ref https://github.com/nim-lang/Nim/pull/24686
With this PR
```nim
import std/streams
proc foo() =
var name = newStringStream("2r2")
raise newException(ValueError, "sh")
try:
foo()
except:
discard
echo 123
```
this example no longer leaks
---------
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
(cherry picked from commit 510ac84518)
Testament now retries a test by a specified amount if it fails in any
way other than an invalid spec. This is to deal with the flaky GC tests
on Windows CI that fail in many different ways, from the linker randomly
erroring, segfaults, etc.
Unfortunately I couldn't do this cleanly in testament's current code.
The proc `addResult`, which is the "final" proc called in a test run's
lifetime, is now wrapped in a proc `finishTest` that returns a bool
`true` if the test failed and has to be retried. This result is
propagated up from `cmpMsgs` and `compilerOutputTests` until it reaches
`testSpecHelper`, which handles these results by recursing if the test
has to be retried. Since calling `testSpecHelper` means "run this test
with one given configuration", this means every single matrix
option/target etc. receive an equal amount of retries each.
The result of `finishTest` is ignored in cases where it's known that it
won't be retried due to passing, being skipped, having an invalid spec
etc. It's also ignored in `testNimblePackages` because it's not
necessary for those specific tests yet and similar retry behavior is
already implemented for part of it.
This was a last resort for the flaky GC tests but they've been a problem
for years at this point, they give us more work to do and turn off
contributors. Ideally GC tests failing should mark as "needs review" in
the CI rather than "failed" but I don't know if Github supports
something like this.
(cherry picked from commit 720d0aee5c)
Flaky not just due to recent ubuntu 24/GCC 14 upgrades, windows fails as
well, assuming the issue is with badssl or it's just not worth testing
here.
(cherry picked from commit 5f056f87b2)
fixes#8758, fixes#10828, fixes#12172, fixes#21610, fixes#23803,
fixes#24633, fixes#24634, succeeds #24085
We simply track the symbol ID of every traversed `var`/`let` definition
in `vmgen`, then these symbols are always considered evaluable in the
current `vmgen` context. The set of symbols is reset before every
generation, but both tests worked properly without doing this including
the nested `const`, so maybe it's already done in some way I'm not
seeing.
(cherry picked from commit a5cc33c1d3)
ref https://en.cppreference.com/w/cpp/error/exception/what
> Pointer to a null-terminated string with explanatory information. The
pointer is guaranteed to be valid at least until the exception object
from which it is obtained is destroyed, or until a non-const member
function on the exception object is called.
The pointer is only valid before `CStdException as e` is destroyed
Old examples are broken on macOS arm64
```
/Users/blue/Desktop/nimony/test4.nim(38) test4
/Users/blue/Desktop/nimony/test4.nim(26) fn
/Users/blue/.choosenim/toolchains/nim-#devel/lib/std/assertions.nim(41) failedAssertImpl
/Users/blue/.choosenim/toolchains/nim-#devel/lib/std/assertions.nim(36) raiseAssert
/Users/blue/.choosenim/toolchains/nim-#devel/lib/system/fatal.nim(53) sysFatal
Error: unhandled exception: /Users/blue/Desktop/nimony/test4.nim(26, 3) `$b == "foo2"` [AssertionDefect]
```
(cherry picked from commit e6f6c369ff)
fixes#24631
[Object
constructors](793baf34ff/compiler/semobjconstr.nim (L462)),
[casts](793baf34ff/compiler/semexprs.nim (L494))
and [type
conversions](793baf34ff/compiler/semexprs.nim (L419))
copy their type nodes verbatim instead of producing semchecked type
nodes. This causes a crash in transf when an untyped expression in the
type node has `nil` type. To deal with this, don't try to transform the
type node in these expressions at all. I couldn't reproduce the problem
with type conversion nodes though so those are unchanged in transf.
(cherry picked from commit 6d59680217)
With some inputs larger than `BiggestUInt.high`, `parseBiggestUInt` proc
in `parseutils.nim` fails to detect overflow and returns random value.
This is because `rawParseUInt` try to detects overflow with `if prev >
res:` but it doesn't detects the overflow from multiplication.
It is possible that `x *= 10` causes overflow and resulting value is
larger than original value.
Here is example values larger than `BiggestUInt.high` but
`parseBiggestUInt` returns without detecting overflow:
```
22751622367522324480000000
41404969074137497600000000
20701551093035827200000000000000000
22546225502460313600000000000000000
204963831854661632000000000000000000
```
Following code search for values larger than `BiggestUInt.high` and
`parseBiggestUInt` cannot detect overflow:
```nim
import std/[strutils]
const
# Increase this to extend search range
NBits = 34'u
NBitsMax1 = 1'u shl NBits
NBitsMax = NBitsMax1 - 1'u
# Increase this when there are too many results and want to see only larger result.
MinMultiply10 = 14
var nfound = 0
for i in (NBitsMax div 10'u + 1'u) .. NBitsMax:
var
x = i
n10 = 0
for j in 0 ..< NBits:
let px = x
x = (x * 10'u) and NBitsMax
if x < px:
break
inc n10
if n10 >= MinMultiply10:
echo "i = ", i
echo "uint: ", (i shl (64'u - NBits)), '0'.repeat n10
inc nfound
if nfound > 15:
break
echo "found: ", nfound
```
(cherry picked from commit 95b1dda1db)
fixes#17958
In `transf`, conversions in subscript expressions are skipped (with
`skipConv`'s rules). This is because array indexing can produce
conversions to the range type that is the array's index type, which
causes a `RangeDefect` rather than an `IndexDefect` (and also
`--rangeChecks` and `--indexChecks` are both considered). However this
causes problems when explicit conversions are used, between types of
different bitsizes, because those also get skipped.
To fix this, we only skip the conversion if:
* it's a hidden (implicit) conversion
* it's a range check conversion (produces `nkChckRange`)
* the subscript is on an array type and the result type of the
conversion has the same bounds as the array index type
And `skipConv` rules also still apply (int/float classification).
Another idea would be to prevent the implicit conversion to the array
index type from being generated. But there is no good way to do this:
matching to the base type instead prevents types like `uint32` from
implicitly converting (i.e. it can convert to `range[0..3]` but not
`int`), and analyzing whether this is an array bound check is easier in
`transf`, since `sigmatch` just produces a type conversion.
The rules for skipping the conversion could also receive some other
tweaks: We could add a rule that changing bitsizes also doesn't skip the
conversion, but this breaks the `uint32` case. We could simplify it to
only removing implicit skips to specifically fix#17958, but this is
wrong in general.
We could also add something like `nkChckIndex` that generates index
errors instead but this is weird when it doesn't have access to the
collection type and it might be overkill.
(cherry picked from commit 76c5f16ac5)
fixes#24626
`createTypeboundOps` in sempass2 is called when generating destructors
for types including for explicit destructor calls, however it blocks
destructors from getting generated in a `nodestroy` proc. This causes
issues when a destructor is explicitly called in a `nodestroy` proc. To
fix this, allow destructors to get generated only for explicit
destructor calls in nodestroy procs.
(cherry picked from commit 793baf34ff)
fixes#24472
Excluding variables which are initialized in the nimvm branch so that
they won't interfere the other branch
(cherry picked from commit e7f48cdd5c)
fixes#22523
There were 2 problems with the code in `sameType` for
`dcEqIgnoreDistinct`:
1. The code that skipped `{tyDistinct, tyGenericInst}` only ran if the
given types had different kinds. This is fixed by always performing this
skip.
2. The code block below that checks if `tyGenericInst`s have different
values still ran for `dcEqIgnoreDistinct` since it checks if the given
types are generic insts, not the skipped types (and also only the 1st
given type). This is fixed by only invoking this block for `dcEq`;
`dcEqOrDistinctOf` (which is unused) also skips the first given type.
Arguably there is another issue here that `skipGenericAlias` only ever
skips 1 type.
These combined fix the issue (`T` is `GenericInst(V, 1, distinct int)`
and `D[0]` is `GenericInst(D, 0, distinct int)`).
(cherry picked from commit b0e6d28782)
fixes#24097
For `nkConv` addresses where the conversion is between 2 types that are
equal between backends, treat assignments the same as assignments to the
argument of the conversion. In the VM this seems to be in `genAsgn` and
`genAsgnPatch`, as evidenced by the special logic for `nkDerefExpr` etc.
This doesn't handle ranges after #24037 because `sameBackendType` is
used and not `sameBackendTypeIgnoreRange`. This is so this is
backportable without #24037 and another PR can be opened that implements
it for ranges and adds tests as well. We can also merge
`sameBackendTypeIgnoreRange` with `sameBackendType` since it doesn't
seem like anything that uses it would be affected (only cycle checks and
the VM), but then we still have to add tests.
(cherry picked from commit 1fbb67ffe9)
fixes#24449
The standalone `seq` type is a `tyBuiltInTypeClass` with a single child
of kind `tySequence`, which itself has no children. This is also the
case for most other `tyBuiltInTypeClass` kinds. However this can cause a
crash in sigmatch when calling `isEmptyContainer` on this child type,
which expects the sequence type to have children. This check was added
in #5557 to prevent empty collections like `@[]` from matching their
respective typeclass, but it's not useful when matching against another
typeclass (which is done here to resolve an ambiguity). So to avoid the
crash, this empty container check is disabled when matching against
another typeclass.
(cherry picked from commit 96043bdbb7)
fixes#14067, fixes#15004, fixes#19019
Anonymous procs are [added to
scope](8091d76306/compiler/semstmts.nim (L2466))
with the name `:anonymous`. This means that if they have the same
signature in a scope, they can consider each other as redefinitions. To
prevent this, mark their symbols as `sfGenSym` so they do not get added
to scope or cause any name conflicts. The commented out `and not isAnon`
check wouldn't work because `isAnon` would not be true if the proc is
being resemmed, in which case the name field in the proc AST would have
the symbol of the anonymous proc rather than being empty.
There is a separate problem of default values in generic/normal procs
not opening new scopes which is partially responsible for #19019.
(cherry picked from commit 3e47725c08)
fixes#24319
`byRefLoc` (`mapType`) requires the Loc `a` to have the right type.
Without `lfEnforceDeref`, it produces the wrong type for `deref (var
array)`, which may come from `mitems`.
(cherry picked from commit 0347536ff2)
Caught in https://github.com/metagn/applicates, I'm not sure which
commit causes this but it's also in the 2.0 branch (but not 2.0.2), so
it's not any recent PRs.
If a proc has a static parameter with type `static Foo[T]`, then another
parameter with type `static Bar[T, U]`, the generic instantiation for
`Bar` doesn't match `U` which has type `tyGenericParam`, but matches `T`
since it has type `tyTypeDesc`. The reason is that `concreteType`
returns the type itself for `tyTypeDesc` if `c.isNoCall` (i.e. matching
a generic invocation), but returns `nil` for `tyGenericParam`. I'm
guessing `tyGenericParam` is received here because of #22618, but that
doesn't explain why `T` is still `tyTypeDesc`. I'm not sure.
Regardless, we can just copy the behavior for `tyTypeDesc` to
`tyGenericParam` and also return the type itself when `c.isNoCall`. This
feels like it defeats the purpose of `concreteType` but the way it's
used doesn't make sense without it (generic param can't match another
generic param?). Alternatively we could loosen the `if concrete == nil:
return isNone` checks in some places for specific conditions, whether
`c.isNoCall` or `c.inGenericContext == 0` (though this would need
(cherry picked from commit 24e5b21c90)
fixes#13979
Fixes templates in generics that use identifiers that aren't defined
yet, giving an early `undeclared identifier` error, by just marking
template bodies as in a mixin context in `semgnrc`.
(cherry picked from commit bf865fa75a)
fixes#19737
As in the diff, `semResolvedCall` sets the return type of a call to a
proc to the type of the call. But in the case of the [subscript
magic](https://nim-lang.org/docs/system.html#%5B%5D%2CT%2CI), this type
is the first generic param which is also supposed to be the type of the
first argument, but this is invalid, the correct type is the element
type eventually given by `semSubscript`. Some lines above also [prevent
the subscript magics from instantiating their
params](dda638c1ba/compiler/semcall.nim (L699))
so this type ends up being an unresolved generic param.
Since the type of the node is not `nil`, `prepareOperand` doesn't try to
type it again, and this unresolved generic param type ends up being the
final type of the node. To prevent this, we just never set the type of
the node if we encountered a subscript magic.
Maybe we could also rename the generic parameters of the subscript
magics to stuff like `DummyT`, `DummyI` if we want this to be easier to
debug in the future.
(cherry picked from commit 04da0a6028)
fixes#19848
Not sure why this wasn't the case already. The `if cl.allowMetaTypes:
return` line below for `tyFromExpr` [was added 10 years
ago](d5798b43de).
Hopefully it was just negligence?
(cherry picked from commit 1befb8d4a3)
fixes#19819, fixes#23339
Since #22029 `tyFromExpr` does not match anything in overloading, so
generic bodies can know which call expressions to delay until the type
can be evaluated. However generic type invocations also run overloading
to check for generic constraints even in generic bodies. To prevent them
from failing early from the overload not matching, pretend that
`tyFromExpr` matches. This mirrors the behavior of the compiler in more
basic cases like:
```nim
type
Foo[T: int] = object
x: T
Bar[T] = object
y: Foo[T]
```
Unfortunately this case doesn't respect the constraint (#21181, some
other bugs) but `tyFromExpr` should easily use the same principle when
it does.
(cherry picked from commit 31ee75f10e)
fixes#22153
It's a problem for refc because you cannot index a nil string: i.e.
`[""]` is `{((NimStringDesc*) NIM_NIL)}` which cannot be indexed
(cherry picked from commit 9bb7e53e7f)
fixes#24305, refs #23807
Since #23014 `nkHiddenAddr` is produced to fast assign array elements in
iterators. However the array access inside this `nkHiddenAddr` can get
folded at compile time, generating invalid code. In #23807, compile time
folding of regular `addr` expressions was changed to be prevented in
`transf` but `nkHiddenAddr` was not updated alongside it.
The method for preventing folding in `addr` in #23807 was also faulty,
it should only trigger on the immediate child node of the address rather
than all nodes nested inside it. This caused a regression as outlined in
[this
comment](https://github.com/nim-lang/Nim/pull/24322#issuecomment-2419560182).
To fix both issues, `addr` and `nkHiddenAddr` now both shallowly prevent
constant folding for their immediate children.
(cherry picked from commit 52cf7dfde0)
(cherry picked from commit 7ad7ee03e5c0adb6832cbae10a62de7b68ef6fa5)
fixes#24258
It uses conditionals to guard against ill formed AST to produce better
error messages, rather than crashing
(cherry picked from commit 8b39b2df7d)
fixes#24338
When unrolling each iteration of a `fields` iterator, the compiler only
opens a new scope for semchecking, but doesn't generate a node that
signals to the codegen that a new scope should be created. This causes
issues for reused template instantiations that reuse variable symbols
between each iteration, which causes the codegen to generate multiple
declarations for them in the same scope (regardless of `inject` or
`gensym`). To fix this, we wrap the unrolled iterations in an `if true:
body` node, which both opens a new scope and doesn't interfere with
`break`.
(cherry picked from commit ca5df9ab25)
fixes#24296fixes#24295
Templates use `expectedType` for type inference. It's justified that
when templates don't have an actual return type, i.e., `untyped` etc.
When the return type of templates is specified, we should not infer the
type
```nim
template g(): string = ""
let c: cstring = g()
```
In this example, it is not reasonable to annotate the templates
expression with the `cstring` type before the `fitNode` check with its
specified return type.
(cherry picked from commit 80e6b35721)
fixes#24021
The field checking for case object branches at some point generates a
negated set `contains` check for the object discriminator. For enum
types, this tries to generate a complement set and convert to a
`contains` check in that instead. It obtains this type from the type of
the element node in the `contains` check.
`buildProperFieldCheck` creates the element node by changing a field
access expression like `foo.z` into `foo.kind`. In order to do this, it
copies the node `foo.z` and sets the field name in the node to the
symbol `kind`. But when copying the node, the type of the original
`foo.z` is retained. This means that the complement is performed on the
type of the accessed field rather than the type of the discriminator,
which causes problems when the accessed field is also an enum.
To fix this, we properly set the type of the copied node to the type of
the kind field. An alternative is just to make a new node instead.
A lot of text for a single line change, I know, but this part of the
codebase could use more explanation.
(cherry picked from commit 1bebc236bd)
fixes#24228, refs #22022
As described in
https://github.com/nim-lang/Nim/issues/24228#issuecomment-2392462221,
instantiating generic routines inside `typeof` causes all code inside to
be treated as being in a typeof context, and thus preventing compile
time proc folding, causing issues when code is generated for the
instantiated routine. Now, instantiated generic procs are treated as
never being inside a `typeof` context.
This is probably an arbitrary special case and more issues with the
`typeof` behavior from #22022 are likely. Ideally this behavior would be
removed but it's necessary to accomodate the current [proc `declval` in
the package `stew`](https://github.com/status-im/nim-stew/pull/190), at
least without changes to `compileTime` that would either break other
code (making it not eagerly fold by default) or still require a change
in stew (adding an option to disable the eager folding).
Alternatively we could also make the eager folding opt-in only for
generic compileTime procs so that #22022 breaks nothing whatsoever, but
a universal solution would be better. Edit: Done in #24230 via
experimental switch
(cherry picked from commit ea9811a4d2)
fixes#24269, refs #20095
Instead of checking the package of the *used sym* to determine whether a
stylecheck should trigger, we check the package of the lineinfo instead.
Before #20095 this checked for the current compilation context module
instead which caused issues with generic procs, but the lineinfo should
more closely match the AST.
I figured this might cause issues with includes etc but the foreign
package test specifically tests for an include and passes, so maybe the
package determining logic accounts for this already. This still might
not be the correct logic, I'm not too familiar with the package handling
in the compiler.
Package PRs, both merged:
- json_rpc: https://github.com/status-im/nim-json-rpc/pull/226
- json_serialization:
https://github.com/status-im/nim-json-serialization/pull/99
(cherry picked from commit aaf6c408c6)
Followup to #24154, packages aren't ready for macos 14 (M1/ARM CPU) yet
and it seems to be preview on azure, so upgrade to macos 13 for now.
Macos 12 gives a warning:
```
You are using macOS 12.
We (and Apple) do not provide support for this old version.
It is expected behaviour that some formulae will fail to build in this old version.
It is expected behaviour that Homebrew will be buggy and slow.
Do not create any issues about this on Homebrew's GitHub repositories.
Do not create any issues even if you think this message is unrelated.
Any opened issues will be immediately closed without response.
Do not ask for help from Homebrew or its maintainers on social media.
You may ask for help in Homebrew's discussions but are unlikely to receive a response.
Try to figure out the problem yourself and submit a fix as a pull request.
We will review it but may or may not accept it.
```
(cherry picked from commit 4a63186cda)
