Commit Graph

6594 Commits

Author SHA1 Message Date
metagn
05c74d6844 always reinstantiate nominal values of generic instantiations (#24425)
fixes #22479, fixes #24374, depends on #24429 and #24430

When instantiating generic types which directly have nominal types
(object, distinct, ref/ptr object but not enums[^1]) as their values,
the nominal type is now copied (in the case of ref objects, its child as
well) so that it receives a fresh ID and `typeInst` field. Previously
this only happened if it contained any generic types in its structure,
as is the case for all other types.

This solves #22479 and #24374 by virtue of the IDs being unique, which
is what destructors check for. Technically types containing generic
param fields work for the same reason. There is also the benefit that
the `typeInst` field is correct. However issues like #22445 aren't
solved because the compiler still uses structural object equality checks
for inheritance etc. which could be removed in a later PR.

Also fixes a pre-existing issue where destructors bound to object types
with generic fields would not error when attempting to define a user
destructor after the fact, but the error message doesn't show where the
implicit destructor was created now since it was only created for
another instance. To do this, a type flag is used that marks the generic
type symbol when a generic instance has a destructor created. Reusing
`tfCheckedForDestructor` for this doesn't work.

Maybe there is a nicer design that isn't an overreliance on the ID
mechanism, but the shortcomings of `tyGenericInst` are too ingrained in
the compiler to use for this. I thought about maybe adding something
like `tyNominalGenericInst`, but it's really much easier if the nominal
type itself directly contains the information of its generic parameters,
or at least its "symbol", which the design is heading towards.

[^1]: See [this
test](21420d8b09/lib/std/enumutils.nim (L102))
in enumutils. The field symbols `b0`/`b1` always have the uninstantiated
type `B` because enum fields don't expect to be generic, so no generic
instance of `B` matches its own symbols. Wouldn't expect anyone to use
generic enums but maybe someone does.
2024-11-16 10:48:01 +01:00
metagn
75b512bc6a prevent codegen of inactive case fields in VM object constructor nodes (#24442)
fixes #17571

Objects in the VM are represented as object constructor nodes that
contain every single field, including ones in different case branches.
This is so that every field has a unique invariant index in the object
constructor that can be written to and read from. However when
converting this node back into semantic code, fields from inactive case
branches can remain in the constructor which causes bad codegen,
generating assignments to fields from other case branches.

To fix this, fields from inactive branches are now detected in
`semmacrosanity.annotateType` (called in `fixupTypeAfterEval`) and
marked to prevent the codegen of their assignments. In #24441 these
fields were excluded from the resulting node, but this causes issues
when the node is directly supposed to go back into the VM, for example
as `const` values. I don't know if this is the only case where this
happens, so I wasn't sure about how to keep that implementation working.
2024-11-16 10:43:58 +01:00
metagn
e239968b80 fix wrong error for iterators with no body and pragma macro (#24440)
fixes #16413

`semIterator` checks if the original iterator passed to it has no body,
but it should check the processed node created by `semProcAux`.
2024-11-15 22:52:38 +01:00
ringabout
cc696f18c0 adds some test cases (#24436)
closes #24043
closes #24045
2024-11-15 19:31:12 +08:00
ringabout
21420d8b09 fixes #24402; Memory leak under Arc/Orc on inline iterators with nested seq (#24419)
fixes #24402

```nim
iterator myPairsInline*[T](twoDarray: seq[seq[T]]): (int, seq[T]) {.inline.} =
  for indexValuePair in twoDarray.pairs:
    yield indexValuePair

proc innerTestTotalMem() =
  var my2dArray: seq[seq[int32]] = @[]

  # fill with some data...
  for i in 0'i32..100:
    var z = @[i, i+1]
    my2dArray.add z

  for oneDindex, innerArray in myPairsInline(my2dArray):
    discard

innerTestTotalMem()
```

In `for oneDindex, innerArray in myPairsInline(my2dArray)`, `oneDindex`
and `innerArray` becomes `cursors` because they satisfy the criterion of
`isSimpleIteratorVar`. On the one hand, it is not correct to have them
point to the temporary generated by tuple unpacking, which left the
memory of the temporary uncleaned up. On the other hand, we don't need
to generate a temporary for a symbol node when unpacking the tuple.
2024-11-12 22:57:31 +01:00
metagn
511ab72342 fix subtype match of generic object types (#24430)
split from #24425

Matching `tyGenericBody` performs a match on the last child of the
generic body, in this case the uninstantied `tyObject` type. If the
object contains no generic fields, this ends up being the same type as
all instantiated ones, but if it does, a new type is created. This fails
the `sameObjectTypes` check that subtype matching for object types uses.
To fix this, also consider that the pattern type could be the generic
uninstantiated object type of the matched type in subtype matching.
2024-11-12 14:31:59 +01:00
metagn
45e21ce8f1 fix jsonutils macro with generic case object (#24429)
split from #24425

The added test did not work previously. The result of `getTypeImpl` is
the uninstantiated AST of the original type symbol, and the macro
attempts to use this type for the result. To fix the issue, the provided
`typedesc` argument is used instead.
2024-11-12 14:31:08 +01:00
metagn
76c5f16ac5 stricter skip for conversions in array indices in transf (#24424)
fixes #17958

In `transf`, conversions in subscript expressions are skipped (with
`skipConv`'s rules). This is because array indexing can produce
conversions to the range type that is the array's index type, which
causes a `RangeDefect` rather than an `IndexDefect` (and also
`--rangeChecks` and `--indexChecks` are both considered). However this
causes problems when explicit conversions are used, between types of
different bitsizes, because those also get skipped.

To fix this, we only skip the conversion if:

* it's a hidden (implicit) conversion
* it's a range check conversion (produces `nkChckRange`)
* the subscript is on an array type and the result type of the
conversion has the same bounds as the array index type

And `skipConv` rules also still apply (int/float classification).

Another idea would be to prevent the implicit conversion to the array
index type from being generated. But there is no good way to do this:
matching to the base type instead prevents types like `uint32` from
implicitly converting (i.e. it can convert to `range[0..3]` but not
`int`), and analyzing whether this is an array bound check is easier in
`transf`, since `sigmatch` just produces a type conversion.

The rules for skipping the conversion could also receive some other
tweaks: We could add a rule that changing bitsizes also doesn't skip the
conversion, but this breaks the `uint32` case. We could simplify it to
only removing implicit skips to specifically fix #17958, but this is
wrong in general.

We could also add something like `nkChckIndex` that generates index
errors instead but this is weird when it doesn't have access to the
collection type and it might be overkill.
2024-11-11 10:48:28 +01:00
Sam
1fddb61b3b Fixes #24369 (#24370)
Hope this fixes #24369, happy for any feedback on the PR.
2024-11-10 17:16:07 +01:00
metagn
3e47725c08 gensym anonymous proc symbols (#24422)
fixes #14067, fixes #15004, fixes #19019

Anonymous procs are [added to
scope](8091d76306/compiler/semstmts.nim (L2466))
with the name `:anonymous`. This means that if they have the same
signature in a scope, they can consider each other as redefinitions. To
prevent this, mark their symbols as `sfGenSym` so they do not get added
to scope or cause any name conflicts. The commented out `and not isAnon`
check wouldn't work because `isAnon` would not be true if the proc is
being resemmed, in which case the name field in the proc AST would have
the symbol of the anonymous proc rather than being empty.

There is a separate problem of default values in generic/normal procs
not opening new scopes which is partially responsible for #19019.
2024-11-09 12:33:23 +01:00
metagn
45b8434c7d skip tyAlias in generic alias checks [backport:2.0] (#24417)
fixes #24415

Since #23978 (which is in 2.0), all generic types that alias to another
type now insert a `tyAlias` layer in their value. However the
`skipGenericAlias` etc code which `sigmatch` uses is not updated for
this, so `tyAlias` is now skipped in these.

The relevant code in sigmatch is:
67ad1ae159/compiler/sigmatch.nim (L1668-L1673)

This behavior is also suspicious IMO, not skipping a structural
`tyGenericInst` alias can be useful for code like #10220, but this is
currently arbitrarily decided based on "depth" and whether the alias is
to another `tyGenericInst` type or not. Maybe in the future we could
enforce the use of a nominal type.
2024-11-08 08:36:52 +01:00
metagn
67ad1ae159 fix standalone explicit generic procs with unresolved arguments (#24404)
fixes issue described in https://forum.nim-lang.org/t/12579

In #24065 explicit generic parameter matching was made to fail matches
on arguments with unresolved types in generic contexts (the sigmatch
diff, following #24010), similar to what is done for regular calls since
#22029. However unlike regular calls, a failed match in a generic
context for a standalone explicit generic instantiation did not convert
the expression into one with `tyFromExpr` type, which means it would
error immediately given any unresolved parameter. This is now done to
fix the issue.

For explicit generic instantiations on single non-overloaded symbols, a
successful match is still instantiated. For multiple overloads (i.e.
symchoice), if any of the overloads fail the match, the entire
expression is considered untyped and any instantiations are not used, so
as to not void overloads that would match later. This means even
symchoices without unresolved arguments aren't instantiated, which may
be too restrictive, but it could also be too lenient and we might need
to make symchoice instantiations always untyped. The behavior for
symchoice is not sound anyway given it causes #9997 so this is something
to consider for a redesign.

Diff follows #24276.
2024-11-06 10:54:03 +01:00
ringabout
c71de10608 fixes strictdefs with when nimvm (#24409)
ref https://github.com/nim-lang/Nim/pull/24225
related https://github.com/nim-lang/Nim/pull/24306

> Code in branches must not affect semantics of the code that follows
the
`when nimvm` statement. E.g. it must not define symbols that are used in
  the following code.

The test shouldn't have passed when
https://github.com/nim-lang/Nim/pull/24306
would be implemented somehow. Some third packages have already misused
`when nimvm` by defining symbols in the other branch of `when nimvm`.

e.g. in https://github.com/status-im/nim-unittest2/pull/34

```nim
when nimvm:
  discard
else:
  let suiteName {.inject.} = nameParam

use(suiteName)
```
2024-11-05 15:14:57 +01:00
metagn
5f056f87b2 disable all badssl tests indefinitely (#24403)
Flaky not just due to recent ubuntu 24/GCC 14 upgrades, windows fails as
well, assuming the issue is with badssl or it's just not worth testing
here.
2024-11-03 14:56:20 +01:00
Phil Krylov
46bb47a444 std/parsesql: Fix JOIN parsing (#22890)
This commit fixes/adds tests for and fixes several issues with `JOIN`
operator parsing:

- For OUTER joins, LEFT | RIGHT | FULL specifier is not optional
```nim
doAssertRaises(SqlParseError): discard parseSql("""
SELECT id FROM a
OUTER JOIN b
ON a.id = b.id
""")
```

- For NATURAL JOIN and CROSS JOIN, ON and USING clauses are forbidden
```nim
doAssertRaises(SqlParseError): discard parseSql("""
SELECT id FROM a
CROSS JOIN b
ON a.id = b.id
""")
```

- JOIN should parse as part of FROM, not after WHERE
```nim
doAssertRaises(SqlParseError): discard parseSql("""
SELECT id FROM a
WHERE a.id IS NOT NULL
INNER JOIN b
ON a.id = b.id
""")
```

- LEFT JOIN should parse
```nim
doAssert $parseSql("""
SELECT id FROM a
LEFT JOIN b
ON a.id = b.id
""") == "select id from a left join b on a.id = b.id;"
```

- NATURAL JOIN should parse
```nim
doAssert $parseSql("""
SELECT id FROM a
NATURAL JOIN b
""") == "select id from a natural join b;"
```

- USING should parse
```nim
doAssert $parseSql("""
SELECT id FROM a
JOIN b
USING (id)
""") == "select id from a join b using (id );"
```

- Multiple JOINs should parse
```nim
doAssert $parseSql("""
SELECT id FROM a
JOIN b
ON a.id = b.id
LEFT JOIN c
USING (id)
""") == "select id from a join b on a.id = b.id left join c using (id );"
```
2024-11-02 07:58:19 +01:00
ringabout
5e56f0a356 fixes #24378; supportsCopyMem can fail from macro context with tuples (#24383)
fixes #24378

```nim
type Win = typeof(`body`)
doAssert not supportsCopyMem((int, Win))
```

`semAfterMacroCall` doesn't skip the children aliases types in the tuple
typedesc construction while the normal program seem to skip the aliases
types somewhere

`(int, Win)` is kept as `(int, alias string)` instead of expected `(int,
string)`
2024-10-30 22:58:39 +01:00
ringabout
1576563775 disable Test on aarch64 (#24389)
ref https://github.com/nim-lang/Nim/issues/24287
2024-10-30 20:17:19 +01:00
metagn
4091576ab7 implement generic default values for object fields (#24384)
fixes #21941, fixes #23594
2024-10-30 08:58:04 +01:00
ringabout
815bbf0e73 fixes #23545; C compiler error when default initializing an object field function (#24375)
fixes #23545
2024-10-29 08:08:35 +01:00
ringabout
031ad957ba fixes #24359; VM problem: dest register is not set with const-bound proc (#24364)
fixes #24359

follow up https://github.com/nim-lang/Nim/pull/11076

It should not try to evaluate the const proc if the proc doesn't have a
return value.
2024-10-26 20:49:07 +02:00
metagn
40fc2d0e76 include static types in type bound ops (#24366)
refs https://github.com/nim-lang/Nim/pull/24315#discussion_r1816332587
2024-10-26 17:49:02 +02:00
ringabout
aa90d00caf fixes #18081; fixes #18079; fixes #18080; nested ref/deref'd types (#24335)
fixes #18081;
fixes https://github.com/nim-lang/Nim/issues/18080
fixes #18079

reverts https://github.com/nim-lang/Nim/pull/20738

It is probably more reasonable to use the type node from `nkObjConstr`
since it is barely changed unlike the external type, which is
susceptible to code transformation e.g. `addr(deref objconstr)`.
2024-10-25 22:36:19 +02:00
ringabout
2af602a5c8 deprecate NewFinalize with the ref T finalizer (#24354)
pre-existing issues:

```nim
block:
  type
    FooObj = object
      data: int
    Foo = ref ref FooObj


  proc delete(self: Foo) =
    echo self.data

  var s: Foo
  new(s, delete)
```
it crashed with arc/orc in 1.6.x and 2.x.x

```nim
block:
  type
    Foo = ref int


  proc delete(self: Foo) =
    echo self[]

  var s: Foo
  new(s, delete)
```

The simple fix is to add a type restriction for the type `T` for arc/orc
versions
```nim
  proc new*[T: object](a: var ref T, finalizer: proc (x: T) {.nimcall.})
```
2024-10-25 22:35:26 +02:00
metagn
d303c289fa consider calls as complex openarray assignment to iterator params (#24333)
fixes #13417, fixes #19703

When passing an expression to an `openarray` iterator parameter: If the
expression is a statement list (considered "complex"), it's assigned in
a non-deep-copying way to a temporary variable first, then this variable
is used as a parameter. If it's not a statement list, i.e. a call or a
symbol, the parameter is substituted directly with the given expression.
In the case of calls, this results in the call potentially being
executed more than once, or can cause redefined variables in the
codegen.

To fix this, calls are also considered as "complex" assignments to
openarrays, as long as the return type of the call is not `openarray` as
the generated assignment in that case has issues/is unimplemented
(caused a segfault [here in
datamancer](47ba4d81bf/src/datamancer/dataframe.nim (L1580))).

As for why creating a temporary isn't the default only with exceptions
for things like `nkSym`, the "non-deep-copying" way of assignment
apparently still causes arrays to be copied according to a comment in
the code. I'm not sure to what extent this is true: if it still happens
on ARC/ORC, if it happens for every array length, or if we can fix it by
passing arrays by reference. Otherwise, a more general way to assign to
openarrays might be needed, but I'm not sure if the compiler can easily
do this.
2024-10-25 22:13:22 +02:00
ringabout
294b1566e7 fixes #23952; Size/Signedness issues with unordered enums (#24356)
fixes #23952

It reorders `type Foo = enum A, B = -1` to `type Foo = enum B = -1, A`
so that `firstOrd` etc. continue to work.
2024-10-25 23:03:17 +08:00
metagn
2864830941 implement type bound operation RFC (#24315)
closes https://github.com/nim-lang/RFCs/issues/380, fixes #4773, fixes
#14729, fixes #16755, fixes #18150, fixes #22984, refs #11167 (only some
comments fixed), refs #12620 (needs tiny workaround)

The compiler gains a concept of root "nominal" types (i.e. objects,
enums, distincts, direct `Foo = ref object`s, generic versions of all of
these). Exported top-level routines in the same module as the nominal
types that their parameter types derive from (i.e. with
`var`/`sink`/`typedesc`/generic constraints) are considered attached to
the respective type, as the RFC states. This happens for every argument
regardless of placement.

When a call is overloaded and overload matching starts, for all
arguments in the call that already have a type, we add any operation
with the same name in the scope of the root nominal type of each
argument (if it exists) to the overload match. This also happens as
arguments gradually get typed after every overload match. This restricts
the considered overloads to ones attached to the given arguments, as
well as preventing `untyped` arguments from being forcefully typed due
to unrelated overloads. There are some caveats:

* If no overloads with a name are in scope, type bound ops are not
triggered, i.e. if `foo` is not declared, `foo(x)` will not consider a
type bound op for `x`.
* If overloads in scope do not have enough parameters up to the argument
which needs its type bound op considered, then type bound ops are also
not added. For example, if only `foo()` is in scope, `foo(x)` will not
consider a type bound op for `x`.

In the cases of "generic interfaces" like `hash`, `$`, `items` etc. this
is not really a problem since any code using it will have at least one
typed overload imported. For arbitrary versions of these though, as in
the test case for #12620, a workaround is to declare a temporary
"template" overload that never matches:

```nim
# neither have to be exported, just needed for any use of `foo`:
type Placeholder = object
proc foo(_: Placeholder) = discard
```

I don't know what a "proper" version of this could be, maybe something
to do with the new concepts.

Possible directions:

A limitation with the proposal is that parameters like `a: ref Foo` are
not attached to any type, even if `Foo` is nominal. Fixing this for just
`ptr`/`ref` would be a special case, parameters like `seq[Foo]` would
still not be attached to `Foo`. We could also skip any *structural* type
but this could produce more than one nominal type, i.e. `(Foo, Bar)`
(not that this is hard to implement, it just might be unexpected).

Converters do not use type bound ops, they still need to be in scope to
implicitly convert. But maybe they could also participate in the nominal
type consideration: if `Generic[T] = distinct T` has a converter to `T`,
both `Generic` and `T` can be considered as nominal roots.

The other restriction in the proposal, being in the same scope as the
nominal type, could maybe be worked around by explicitly attaching to
the type, i.e.: `proc foo(x: T) {.attach: T.}`, similar to class
extensions in newer OOP languages. The given type `T` needs to be
obtainable from the type of the given argument `x` however, i.e.
something like `proc foo(x: ref T) {.attach: T.}` doesn't work to fix
the `ref` issue since the compiler never obtains `T` from a given `ref
T` argument. Edit: Since the module is queried now, this is likely not
possible.

---------

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2024-10-25 11:26:42 +02:00
ringabout
baf3695c76 closes #19984; adds a test case (#24349)
closes #19984
2024-10-23 16:05:52 +08:00
metagn
ca5df9ab25 wrap fields iterations in if true scope [backport] (#24343)
fixes #24338

When unrolling each iteration of a `fields` iterator, the compiler only
opens a new scope for semchecking, but doesn't generate a node that
signals to the codegen that a new scope should be created. This causes
issues for reused template instantiations that reuse variable symbols
between each iteration, which causes the codegen to generate multiple
declarations for them in the same scope (regardless of `inject` or
`gensym`). To fix this, we wrap the unrolled iterations in an `if true:
body` node, which both opens a new scope and doesn't interfere with
`break`.
2024-10-22 19:56:37 +02:00
metagn
041098e882 clean up stdlib with --jsbigint64 (#24255)
refs #6978, refs #6752, refs #21613, refs #24234

The `jsNoInt64`, `whenHasBigInt64`, `whenJsNoBigInt64` templates are
replaced with bool constants to use with `when`. Weird that I didn't do
this in the first place.

The `whenJsNoBigInt64` template was also slightly misleading. The first
branch was compiled for both no bigint64 on JS as well as on C/C++. It
seems only `trandom` depended on this by mistake.

The workaround for #6752 added in #6978 to `times` is also removed with
`--jsbigint64:on`, but #24233 was also encountered with this, so this PR
depends on #24234.
2024-10-19 16:40:28 +02:00
Jake Leahy
93c24fe1c5 Fix quoted idents in ctags (#24317)
Running `ctags` on files with quoted symbols (e.g. `$`) would list \`
instead of the full ident. Issue was the result getting reassigned at
the end to a \` instead of appending
2024-10-19 16:39:15 +02:00
metagn
ae9287c4f3 symmetric difference operation for sets via xor (#24286)
closes https://github.com/nim-lang/RFCs/issues/554

Adds a symmetric difference operation to the language bitset type. This
maps to a simple `xor` operation on the backend and thus is likely
faster than the current alternatives, namely `(a - b) + (b - a)` or `a +
b - a * b`. The compiler VM implementation of bitsets already
implemented this via `symdiffSets` but it was never used.

The standalone binary operation is added to `setutils`, named
`symmetricDifference` in line with [hash
sets](https://nim-lang.org/docs/sets.html#symmetricDifference%2CHashSet%5BA%5D%2CHashSet%5BA%5D).
An operator version `-+-` and an in-place version like `toggle` as
described in the RFC are also added, implemented as trivial sugar.
2024-10-19 10:07:00 +02:00
metagn
0a058a6b8f better errors for standalone explicit generic instantiations (#24276)
refs #8064, refs #24010

Error messages for standalone explicit generic instantiations are
revamped. Failing standalone explicit generic instantiations now only
error after overloading has finished and resolved to the default `[]`
magic (this means `[]` can now be overloaded for procs but this isn't
necessarily intentional, in #24010 it was documented that it isn't
possible). The error messages for failed instantiations are also no
longer a simple `cannot instantiate: foo` message, instead they now give
the same type mismatch error message as overloads with mismatching
explicit generic parameters.

This is now possible due to the changes in #24010 that delegate all
explicit generic proc instantiations to overload resolution. Old code
that worked around this is now removed. `maybeInstantiateGeneric` could
maybe also be removed in favor of just `explicitGenericSym`, the `result
== n` case is due to `explicitGenericInstError` which is only for niche
cases.

Also, to cause "ambiguous identifier" error messages when the explicit
instantiation is a symchoice and the expression context doesn't allow
symchoices, we semcheck the sym/symchoice created by
`explicitGenericSym` with the given expression flags.

#8064 isn't entirely fixed because the error message is still misleading
for the original case which does `values[1]`, as a consequence of
#12664.
2024-10-18 19:06:42 +02:00
Yuriy Glukhov
5fa96ef270 Fixes #3824, fixes #19154, and hopefully #24094. Re-applies #23787. (#24316)
The first commit reverts the revert of #23787.
The second fixes lambdalifting in convolutedly nested
closures/closureiters. This is considered to be the reason of #24094,
though I can't tell for sure, as I was not able to reproduce #24094 for
complicated but irrelevant reasons. Therefore I ask @jmgomez, @metagn or
anyone who could reproduce it to try it again with this PR.

I would suggest this PR to not be squashed if possible, as the history
is already messy enough.

Some theory behind the lambdalifting fix:
- A closureiter that captures anything outside its body will always have
`:up` in its env. This property is now used as a trigger to lift any
proc that captures such a closureiter.
- Instantiating a closureiter involves filling out its `:up`, which was
previously done incorrectly. The fixed algorithm is to use "current" env
if it is the owner of the iter declaration, or traverse through `:up`s
of env param until the common ancestor is found.

---------

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
2024-10-18 10:36:41 +02:00
metagn
52cf7dfde0 shallow fold prevention for addr, nkHiddenAddr (#24322)
fixes #24305, refs #23807

Since #23014 `nkHiddenAddr` is produced to fast assign array elements in
iterators. However the array access inside this `nkHiddenAddr` can get
folded at compile time, generating invalid code. In #23807, compile time
folding of regular `addr` expressions was changed to be prevented in
`transf` but `nkHiddenAddr` was not updated alongside it.

The method for preventing folding in `addr` in #23807 was also faulty,
it should only trigger on the immediate child node of the address rather
than all nodes nested inside it. This caused a regression as outlined in
[this
comment](https://github.com/nim-lang/Nim/pull/24322#issuecomment-2419560182).

To fix both issues, `addr` and `nkHiddenAddr` now both shallowly prevent
constant folding for their immediate children.
2024-10-18 07:37:05 +02:00
ringabout
0347536ff2 fixes #24319; move doesn't work well with (deref (var array)) (#24321)
fixes #24319

`byRefLoc` (`mapType`) requires the Loc `a` to have the right type.
Without `lfEnforceDeref`, it produces the wrong type for `deref (var
array)`, which may come from `mitems`.
2024-10-18 10:56:37 +08:00
ringabout
8be82c36c9 fixes #18896; fixes #20886; importc types alias doesn't work with distinct (#24313)
fixes #18896
fixes #20886

```nim
type
  PFile {.importc: "FILE*", header: "<stdio.h>".} = distinct pointer
    # import C's FILE* type; Nim will treat it as a new pointer type
```
This is an excerpt from the Nim manual. In the old Nim versions, it
produces a void pointer type instead of the `FILE*` type that should
have been generated. Because these C types tend to be opaque and adapt
to changes on different platforms. It might affect the portability of
Nim on various OS, i.e. `csource_v2` cannot build on the apline platform
because of `Time` relies on Nim types instead of the `time_t` type.

ref https://github.com/nim-lang/Nim/pull/18851
2024-10-16 20:49:31 +02:00
ringabout
922f7dfd71 closes #19585; adds a test case for #21648 (#24310)
closes #19585
follow up #21648
2024-10-15 09:19:46 +08:00
ringabout
8b39b2df7d fixes #24258; compiler crash on len of varargs[untyped] (#24307)
fixes #24258

It uses conditionals to guard against ill formed AST to produce better
error messages, rather than crashing
2024-10-14 17:43:12 +02:00
metagn
6df050d6d2 only generate first field for default value of union (#24303)
fixes #20653
2024-10-14 17:07:57 +02:00
ringabout
80e6b35721 templates/macros use no expected types when return types are specified (#24298)
fixes #24296 
fixes #24295

Templates use `expectedType` for type inference. It's justified that
when templates don't have an actual return type, i.e., `untyped` etc.

When the return type of templates is specified, we should not infer the
type

```nim
template g(): string = ""

let c: cstring = g()
```
In this example, it is not reasonable to annotate the templates
expression with the `cstring` type before the `fitNode` check with its
specified return type.
2024-10-13 19:54:30 +02:00
metagn
720d0aee5c add retries to testament, use it for GC tests (#24279)
Testament now retries a test by a specified amount if it fails in any
way other than an invalid spec. This is to deal with the flaky GC tests
on Windows CI that fail in many different ways, from the linker randomly
erroring, segfaults, etc.

Unfortunately I couldn't do this cleanly in testament's current code.
The proc `addResult`, which is the "final" proc called in a test run's
lifetime, is now wrapped in a proc `finishTest` that returns a bool
`true` if the test failed and has to be retried. This result is
propagated up from `cmpMsgs` and `compilerOutputTests` until it reaches
`testSpecHelper`, which handles these results by recursing if the test
has to be retried. Since calling `testSpecHelper` means "run this test
with one given configuration", this means every single matrix
option/target etc. receive an equal amount of retries each.

The result of `finishTest` is ignored in cases where it's known that it
won't be retried due to passing, being skipped, having an invalid spec
etc. It's also ignored in `testNimblePackages` because it's not
necessary for those specific tests yet and similar retry behavior is
already implemented for part of it.

This was a last resort for the flaky GC tests but they've been a problem
for years at this point, they give us more work to do and turn off
contributors. Ideally GC tests failing should mark as "needs review" in
the CI rather than "failed" but I don't know if Github supports
something like this.
2024-10-12 22:48:44 +02:00
metagn
1bebc236bd fix type of reconstructed kind field node in field checking analysis [backport] (#24290)
fixes #24021

The field checking for case object branches at some point generates a
negated set `contains` check for the object discriminator. For enum
types, this tries to generate a complement set and convert to a
`contains` check in that instead. It obtains this type from the type of
the element node in the `contains` check.

`buildProperFieldCheck` creates the element node by changing a field
access expression like `foo.z` into `foo.kind`. In order to do this, it
copies the node `foo.z` and sets the field name in the node to the
symbol `kind`. But when copying the node, the type of the original
`foo.z` is retained. This means that the complement is performed on the
type of the accessed field rather than the type of the discriminator,
which causes problems when the accessed field is also an enum.

To fix this, we properly set the type of the copied node to the type of
the kind field. An alternative is just to make a new node instead.

A lot of text for a single line change, I know, but this part of the
codebase could use more explanation.
2024-10-12 21:20:21 +02:00
metagn
aaf6c408c6 make linter use lineinfo to check originating package (#24270)
fixes #24269, refs #20095

Instead of checking the package of the *used sym* to determine whether a
stylecheck should trigger, we check the package of the lineinfo instead.
Before #20095 this checked for the current compilation context module
instead which caused issues with generic procs, but the lineinfo should
more closely match the AST.

I figured this might cause issues with includes etc but the foreign
package test specifically tests for an include and passes, so maybe the
package determining logic accounts for this already. This still might
not be the correct logic, I'm not too familiar with the package handling
in the compiler.

Package PRs, both merged:

- json_rpc: https://github.com/status-im/nim-json-rpc/pull/226
- json_serialization:
https://github.com/status-im/nim-json-serialization/pull/99
2024-10-11 12:00:05 +02:00
metagn
9c85f4fd07 fix deref/addr pair deleting assignment location in C++ (#24280)
fixes #24274

The code in the `if` branch replaces the current destination `d` with a
new one. But the location `d` can be an assignment location, in which
case the provided expression isn't generated. To fix this, don't trigger
this code for when the location already exists. An alternative would be
to call `putIntoDest` in this case as is done below.
2024-10-11 10:36:40 +02:00
dlesnoff
e9a4d096ab std/math: Add ^ overload for float32 and float64 (#20898)
I have added a new overload of `^` for float exponents.
Is two overloads for `float32` and `float64` better than just one
overload with `SomeFloat` type ?
I guess this would not work with `SomeFloat`, as `pow` is not defined
for `float`.

Another remark. Maybe we should catch exponents with 0.5 and call `sqrt`
instead ?

---------

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
Co-authored-by: metagn <metagngn@gmail.com>
2024-10-10 20:30:40 +03:00
metagn
2f904535d0 don't allow instantiations resolving to generic body types (#24273)
fixes #24091, refs #24092

Any instantiations resolving to a generic body type now gives an error.
Due to #24092, this does not error in cases like matching against `type
M` in generics because generic body type symbols are just not
instantiated. But this prevents parameters with type `type M` from being
used, although there doesn't seem to be any code which does this. Just
in case such code exists, we still allow `typedesc` types resolving to
generic body types.
2024-10-10 15:35:51 +02:00
metagn
67ea754b7f remove conflicting default call in tables.getOrDefault (#24265)
fixes #23587

As explained in the issue, `getOrDefault` has a parameter named
`default` that can be a proc after generic instantiation. But the
parameter having a proc type [overrides all other
overloads](f73e03b132/compiler/semexprs.nim (L1203))
including the magic `system.default` overload and causes a compile error
if the proc doesn't match the normal use of `default`. To fix this, the
`result = default(B)` initializer call is removed because it's not
needed, `result` is always set in `getOrDefaultImpl` when a default
value is provided.

This is still a suspicious behavior of the compiler but `tables` working
has a higher priority.
2024-10-09 18:20:43 +02:00
metagn
d72b848d17 process non-language pragma nodes in generics (#24254)
fixes #18649, refs #24183

Same as in #24183 for templates, we now process pragma nodes in generics
so that macro symbols are captured and the pragma arguments are checked,
but ignoring language pragma keywords.

A difference is that we cannot process call nodes as is, we have to
process their children individually so that the early untyped
macro/template instantiation in generics does not kick in.
2024-10-07 23:18:45 +02:00
metagn
c73eedfe6e give int literals matched type on generic match (#24234)
fixes #24233

Integer literals with type `int` can match `int64` with a generic match.
Normally this would generate an conversion via `isFromIntLit`, but when
it matches with a generic match (`isGeneric`) the node is left alone and
continues to have type `int` (related to #4858, but separate; since
`isFromIntLit > isGeneric` it doesn't propagate). This did not cause
problems on the C backend up to this point because either the compiler
generated a cast when generating the C code or it was implicitly casted
in the C code itself. On the JS backend however, we need to generate
`int64` and `int` values differently, so we copy the integer literal and
give it the matched type now instead.

This is somewhat risky even if CI passes but it's required to make the
times module work without [this
workaround](7dfadb8b4e/lib/pure/times.nim (L219-L238))
on `--jsbigint64:on` (the default).

CI exposed an issue: When matching an int literal to a generic parameter
in a generic instantiation, the literal is only treated like a value if
it has `int literal` type, but if it has the type `int`, it gets
transformed into literally the type `int` (#12664, #13906), which breaks
the tests t14193 and t12938. To deal with this, we don't give it the
type `int` if we are in a generic instantiation and preserve the `int
literal` type.
2024-10-07 11:40:44 +02:00
metagn
911cef1621 process non-language pragma nodes in templates (#24183)
fixes #24186 

When encountering pragma nodes in templates, if it's a language pragma,
we don't process the name, and only any values if they exist. If it's
not a language pragma, we process the full node. Previously only the
values of colon expressions were processed.

To make this simpler, `whichPragma` is patched to consider bracketed
hint/warning etc pragmas like `{.hint[HintName]: off.}` as being a
pragma of kind `wHint` rather than an invalid pragma which would have to
be checked separately. From looking at the uses of `whichPragma` this
doesn't seem like it would cause problems.

Generics have [the same
problem](a27542195c/compiler/semgnrc.nim (L619))
(causing #18649), but to make it work we need to make sure the
templates/macros don't get evaluated or get evaluated correctly (i.e.
passing the proc node as the final argument), either with #23094 or by
completely disabling template/macro evaluation when processing the
pragma node, which would also cover `{.pragma.}` templates.
2024-10-07 11:39:26 +02:00