TODO:
- [ ] test writing of .nif files
- [x] implement loading of fields in PType/PSym that might not have been
loaded
- [ ] implement interface logic
- [ ] implement pragma "replays"
- [ ] implement special logic for `converter`
- [ ] implement special logic for `method`
- [ ] test the logic holds up for `export`
- [ ] implement logic to free the memory of PSym/PType if memory
pressure is high
- [ ] implement logic to close memory mapped files if too many are open.
---------
Co-authored-by: demotomohiro <gpuppur@gmail.com>
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Jacek Sieka <arnetheduck@gmail.com>
issue 1 - statics in the type:
This probably only handles simple cases. It's probably too accepting
only comparing the base, but that should only affect candidate selection
I think.
issue 2 - `tyArray` of length 3:
This is just a work around since I couldn't get the fix right in
previous PR
issue 3 - shadowing:
The part in `concepts.nim` that iterates candidates does not consider
imported idents if at least once module level ident matches. It does not
have to match in any other way then name.
EDIT: 2 more
issue 4 - composite typeclasses:
when declared in both the concept and the `proc` can cause problems
issue 5 - recursion:
simple recursion and scenarios where more than one concepts recurse
together (only tested two)
fixes#18649, refs #24183
Same as in #24183 for templates, we now process pragma nodes in generics
so that macro symbols are captured and the pragma arguments are checked,
but ignoring language pragma keywords.
A difference is that we cannot process call nodes as is, we have to
process their children individually so that the early untyped
macro/template instantiation in generics does not kick in.
alternative to #24101#23892 changed the opensym experimental switch so that it has to be
enabled in the context of the generic/template declarations capturing
the symbols, not the context of the instantiation of the
generics/templates. This was to be in line with where the compiler gives
the warnings and changes behavior in a potentially breaking way.
However `results` [depends on the old
behavior](71d404b314/results.nim (L1428)),
so that the callers of the macros provided by results always take
advantage of the opensym behavior. To accomodate this, we change the
behavior of the old experimental option that `results` uses,
`genericsOpenSym`, so that ignores the information of whether or not
symbols are intentionally opened and always gives the opensym behavior
as long as it's enabled at instantiation time. This should keep
`results` working as is. However this differs from the normal opensym
switch in that it doesn't generate `nnkOpenSym`.
Before it was just a generics-only version of `openSym` along with
`templateOpenSym` which was only for templates. So `templateOpenSym` is
removed along with this change, but no one appears to have used it.
fixes#13979
Fixes templates in generics that use identifiers that aren't defined
yet, giving an early `undeclared identifier` error, by just marking
template bodies as in a mixin context in `semgnrc`.
fixes#15314, fixes#24002
The OpenSym behavior first added to generics in #23091 now also applies
to templates, since templates can also capture symbols that are meant to
be replaced by local symbols if the context imports symbols with the
same name, as in the issue #24002. The experimental switch
`templateOpenSym` is added to enable this behavior for templates only,
and the experimental switch `openSym` is added to enable it for both
templates and generics, and the documentation now mainly mentions this
switch.
Additionally the logic for `nkOpenSymChoice` nodes that were previously
wrapped in `nkOpenSym` now apply to all `nkOpenSymChoice` nodes, and so
these nodes aren't wrapped in `nkOpenSym` anymore. This means
`nkOpenSym` can only have children of kind `nkSym` again, so it is more
in line with the structure of symchoice nodes. As for why they aren't
merged with `nkOpenSymChoice` nodes yet, we need some way to signal that
the node shouldn't become ambiguous if other options exist at
instantiation time, we already captured a symbol at the beginning and
another symbol can only replace it if it's closer in scope and
unambiguous.
fixes#23898, supersedes #23966 and #23990
Since #20631 ambiguous type symbols in templates are rejected outright,
now we generate a symchoice for type nodes if they're ambiguous, a
generalization of what was done in #22375. This is done for generics as
well. Symchoices also handle type symbols better now, ensuring their
type is a `typedesc` type; this probably isn't necessary for everything
to work but it makes the logic more robust.
Similar to #23989, we have to prepare for the fact that ambiguous type
symbols behave differently than normal type symbols and either error
normally or relegate to other routine symbols if the symbol is being
called. Generating a symchoice emulates this behavior, `semExpr` will
find the type symbol first, but since the symchoice has other symbols,
it will count as an ambiguous type symbol.
I know it seems spammy to carry around an ambiguity flag everywhere, but
in the future when we have something like #23104 we could just always
generate a symchoice, and the symchoice itself would carry the info of
whether the first symbol was ambiguous. But this could harm compiler
performance/memory use, it might be better to generate it only when we
have to, which in the case of type symbols is only when they're
ambiguous.
fixes#23406, closes#23854, closes#23855 (test code of both compiles
but separate issue exists), refs #23432, follows #23411
In generic bodies, previously all regular `nkCall` nodes like `foo(a,
b)` were directly treated as generic statements and delayed immediately,
but other call kinds like `a.foo(b)`, `foo a, b` etc underwent
typechecking before making sure they have to be delayed, as implemented
in #22029. Since the behavior for `nkCall` was slightly buggy (as in
#23406), the behavior for all call kinds is now to call `semTypeExpr`.
However the vast majority of calls in generic bodies out there are
`nkCall`, and while there isn't a difference in the expected behavior,
this exposes many issues with the implementation started in #22029 given
how much more code uses it now. The portion of these issues that CI has
caught are fixed in this PR but it's possible there are more.
1. Deref expressions, dot expressions and calls to dot expressions now
handle and propagate `tyFromExpr`. This is most of the changes in
`semexprs`.
2. For deref expressions to work in `typeof`, a new type flag
`tfNonConstExpr` is added for `tyFromExpr` that calls `semExprWithType`
with `efInTypeof` on the expression instead of `semConstExpr`. This type
flag is set for every `tyFromExpr` type of a node that `prepareNode`
encounters, so that the node itself isn't evaluated at compile time when
just trying to get the type of the node.
3. Unresolved `static` types matching `static` parameters is now treated
the same as unresolved generic types matching `typedesc` parameters in
generic type bodies, it causes a failed match which delays the call
instantiation.
4. `typedesc` parameters now reject all types containing unresolved
generic types like `seq[T]`, not just generic param types by themselves.
(using `containsGenericType`)
5. `semgnrc` now doesn't leave generic param symbols it encounters in
generic type contexts as just identifiers, and instead turns them into
symbol nodes. Normally in generic procs, this isn't a problem since the
generic param symbols will be provided again at instantiation time (and
in fact creating symbol nodes causes issues since `seminst` doesn't
actually instantiate proc body node types).
But generic types can try to be instantiated early in `sigmatch` which
will give an undeclared identifier error when the param is not provided.
Nodes in generic types (specifically in `tyFromExpr` which should be the
only use for `semGenericStmt`) undergo full generic type instantiation
with `prepareNode`, so there is no issue of these symbols remaining as
uninstantiated generic types.
6. `prepareNode` now has more logic for which nodes to avoid
instantiating.
Subscripts and subscripts turned into calls to `[]` by `semgnrc` need to
avoid instantiating the first operand, since it may be a generic body
type like `Generic` in an expression like `Generic[int]`.
Dot expressions cannot instantiate their RHS as it may be a generic proc
symbol or even an undeclared identifier for generic param fields, but
have to instantiate their LHS, so calls and subscripts need to still
instantiate their first node if it's a dot expression.
This logic still isn't perfect and needs the same level of detail as in
`semexprs` for which nodes can be left as "untyped" for overloading/dot
exprs/subscripts to handle, but should handle the majority of cases.
Also the `efDetermineType` requirement for which calls become
`tyFromExpr` is removed and as a result `efDetermineType` is entirely
unused again.
refs https://github.com/nim-lang/Nim/pull/23873#discussion_r1687995060,
fixes#23386, fixes#23385, supersedes #23572
Turns the `nfOpenSym` node flag implemented in #23091 and extended in
#23102 and #23873, into a node kind `nkOpenSym` that forms a unary node
containing either `nkSym` or `nkOpenSymChoice`. Since this affects
macros working on generic proc AST, the node kind is now only generated
when the experimental switch `genericsOpenSym` is enabled, and a new
node flag `nfDisabledOpenSym` is set to the `nkSym` or `nkOpenSymChoice`
when the switch is not enabled so that we can give a warning.
Now that the experimental switch has more reasonable semantics, we
define `nimHasGenericsOpenSym2`.
actually fixes#23865 following up #23873
In the handling of `nkIdent` in `semExpr`, the compiler looks for the
closest symbol with the name and [checks the symbol
kind](6126a0bf46/compiler/semexprs.nim (L3171))
to also consider the overloads if the symbol kind is overloadable. But
it treats the normally overloadable template/macro/module sym kinds the
same as non-overloadable symbols, just calling `semSym` on it. We need
to mirror this behavior in `semOpenSym`; we treat the captured symchoice
as a fresh identifier, so if the symbol we find is a
template/macro/module, we use that symbol immediately as opposed to
waiting for overloads.
fixes#23865
The node flag `nfOpenSym` implemented in #23091 for sym nodes is now
also implemented for open symchoices. This means the intended behavior
is still achieved when multiple overloads are in scope to be captured,
so the issue is fixed. The code for the flag is documented and moved
into a helper proc and the experimental switch is now enabled for the
compiler test suite.
refs #23091, especially post merge comments
Unsure if `experimental` and `bind` are the perfect constructs to use
but they seem to get the job done here. Symbol nodes do not get marked
`nfOpenSym` if the `bind` statement is used for their symbol, and
`nfOpenSym` nodes do not get replaced by new local symbols if the
experimental switch is not enabled in the local context (meaning it also
works with `push experimental`). However this incurs a warning as the
fact that the node is marked `nfOpenSym` means we did not `bind` it, so
we might want to do that or turn on the experimental switch if we didn't
intend to bind it.
The experimental switch name is arbitrary and could be changed.
---------
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
fixes#22605, separated from #22744
This marks symbol captures in macro calls in generic contexts as
`nfOpenSym`, which means if there is a new symbol in the local
instantiatied body during instantiation time, this symbol replaces the
captured symbol. We have to be careful not to consider symbols outside
of the instantiation body during instantiation, because this will leak
symbols from the instantiation context scope rather than the original
declaration scope. This is done by checking if the local context owner
(maybe should be the symbol of the proc currently getting instantiated
instead? not sure how to get this) is the same as or a parent owner of
the owner of the replacement candidate symbol.
This solution is distinct from the symchoice mechanisms which we
originally assumed had to be related, if this assumption was wrong it
would explain why this solution took so long to arrive at.
* always force open generic dot field symbols?
fixes#21724 but might break code
* alternative, should fix CI
* other alternative, add test for previous CI failure
* not needed
* make sure call doesn't compile too
* ok actual second test
* ok final actual correct test
* apply performance idea
* don't make fromDotExpr static
* refactoring in preparation for better, simpler name mangling that works with IC flawlessly
* use new disamb field
* see if this makes tests green
* make tests green again
* alias syntax fixes, improvements and tests
* even better, cannot use alias syntax with generics
* more type tests, improve comment
* fix again
* consistent error message + make t5167_5 work
* more comments, remove {.noalias.}
* Change `styleCheck` to ignore foreign packages
* Symbols from foreign packages are now ignored.
* Fixed `styleCheck` violations in `compiler` package.
* Added symbol ownership to custom annotation pragmas.
* Minor refactors to cleanup style check callsites.
* Minor internal documentation of reasons why a symbol isn't checked.
Style violations were fixed in the compiler after thet were exposed by
the changes. The compiler wouldn't compile otherwise.
Symbol ownership for custom pragma annotations is needed for checking
the annotation's style. A NPE was raised otherwise.
Fixes#10201
See also nim-lang/RFCs#456
* Fix a misunderstanding about excluding field style checks
I had refactored the callsites of `styleCheckUse` to apply the DRY
principle, but I misunderstood the field access handling in a template
as a general case. This corrects it.
* Fix some `styleCheck` violations in `compiler/evalffi`
The violations were exposed in CI when the compiler was built with
libffi.
* Removed some uneeded transitionary code
* Add changelog entry
Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
* cleanups
* ast.nim: cleanups
* IC: no more sym.tab field, stored externally in the module graph
* nimble compiles again
* rodfiles: store bitwidth of integers and the endianness in the cookie because we serialize 'int' directly
* rodfiles: added compilerproc and export sections
* rodfiles: added all the missing sections
* rodfiles: track the missing information
* IC: architecture for lazy loading of proc bodies
* make tests green again
* completed the lazy loading of proc bodies
* symbol lookup integration, part 1
* symbol lookup integration, part 2
* symbol lookup integration, part 3
* make tcompilerapi work again
* rodfiles: fixed config change handling
* reworked ID handling
* the packed AST now has its own ID mechanism
* basic serialization code works
* extract rodfiles to its own module
* rodfiles: store and compare configs
* rodfiles: store dependencies
* store config at the end
* precise dependency tracking
* dependency tracking for rodfiles
* completed loading of PSym, PType, etc
* removed dead code
* bugfix: do not realloc seqs when taking addr into an element
* make IC opt-in for now
* makes tcompilerapi green again
* final cleanups
Co-authored-by: Andy Davidoff <github@andy.disruptek.com>
* added ic specific Nim code; WIP
* make the symbol import mechanism lazy; WIP
* ensure that modules can be imported multiple times
* ambiguity checking
* handle converters and TR macros properly
* make 'enum' test category green again
* special logic for semi-pure enums
* makes nimsuggest tests green again
* fixes nimdata
* makes nimpy green again
* makes more important packages work
* refactoring: idents don't need inheritance
* refactoring: adding an IdGenerator (part 1)
* refactoring: adding an IdGenerator (part 2)
* refactoring: adding an IdGenerator (part 3)
* refactoring: adding an IdGenerator (part 4)
* refactoring: adding an IdGenerator (part 5)
* refactoring: adding an IdGenerator (part 5)
* IdGenerator must be a ref type; hello world works again
* make bootstrapping work again
* progress: add back the 'exactReplica' ideas
* added back the missing exactReplica hacks
* make tcompilerapi work again
* make important packages green
* attempt to fix the build for 32 bit machines (probably need a better solution here)
