fixes#24887 (really just this [1 line
commit](632c7b3397)
would have been enough to fix the issue but it would ignore the general
problem)
When a type definition is encountered where the symbol already has a
type (not a forward type), the type is left alone (not reset to
`tyForward`) and the RHS is handled differently: The RHS is still
semchecked, but the type of the symbol is not updated, and nominal type
nodes are ignored entirely (specifically if they are the same kind as
the symbol's existing type but this restriction is not really needed).
If the existing type of the symbol is an enum and and the RHS has a
nominal enum type node, the enum fields of the existing type are added
to scope rather than creating a new type from the RHS and adding its
symbols instead.
The goal is to prevent any incompatible nominal types from being
generated during resem as in #24887. But it also restricts what macros
can do if they generate type section AST, for example if we have:
```nim
type Foo = int
```
and a macro modifies the type section while keeping the symbol node for
`Foo` like:
```nim
type Foo = float
```
Then the type of `Foo` will still remain `int`, while it previously
became `float`. While we could maybe allow this and make it so only
nominal types cannot be changed, it gets even more complex when
considering generic params and whether or not they get updated. So to
keep it as simple as possible the rule is that the symbol type does not
change, but maybe this behavior was useful for macros.
Only nominal type nodes are ignored for semchecking on the RHS, so that
cases like this do not cause a regression:
```nim
template foo(): untyped =
proc bar() {.inject.} = discard
int
type Foo = foo()
bar() # normally works
```
However this specific code exposed a problem with forward type handling:
---
In specific cases, when the type section is undergoing the final pass,
if the type fits some overly general criteria (it is not an object,
enum, alias or a sink type and its node is not a nominal type node), the
entire RHS is semchecked for a 2nd time as a standalone type (with `nil`
prev) and *maybe* reassigned to the new semchecked type, depending on
its type kind. (for some reason including nominal types when we excluded
them before?) This causes a redefinition error if the RHS defines a
symbol.
This code goes all the way back to the first commit and I could not find
the reason why it was there, but removing it showed a failure in
`thard_tyforward`: If a generic forward type is invoked, it is left as
an unresolved `tyGenericInvocation` on the first run. Semchecking it
again at the end turns it into a `tyGenericInst`. So my understanding is
that it exists to handle these loose forward types, but it is way too
general and there is a similar mechanism `c.skipTypes` which is supposed
to do the same thing but doesn't.
So this is no longer done, and `c.skipTypes` is revamped (and renamed):
It is now a list of types and the nodes that are supposed to evaluate to
them, such that types needing to be updated later due to containing
forward types are added to it along with their nodes. When finishing the
type section, these types are reassigned to the semchecked value of
their nodes so that the forward types in them are fully resolved. The
"reassigning" here works due to updating the data inside the type
pointer directly, and is how forward types work by themselves normally
(`tyForward` types are modified in place as `s.typ`).
For example, as mentioned before, generic invocations of forward types
are first created as `tyGenericInvocation` and need to become
`tyGenericInst` later. So they are now added to this list along with
their node. Object types with forward types as their base types also
need to be updated later to check that the base type is correct/inherit
fields from it: For this the entire object type and its node are added
to the list. Similarly, any case where whether a component type is
`tyGenericInst` or `tyGenericInvocation` matters also needs to cascade
this (`set` does presumably to check the instantiated type).
This is not complete: Generic invocations with forward types only check
that their base type is a forward type, but not any of their arguments,
which causes #16754 and #24133. The generated invocations also need to
cascade properly: `Foo[Bar[ForwardType]]` for example would see that
`Bar[ForwardType]` is a generic invocation and stay as a generic
invocation itself, but it might not queue itself to be updated later.
Even if it did, only the entire type `Foo[Bar[ForwardType]]` needs to be
queued, updating `Bar[ForwardType]` by itself would be redundant or it
would not change anything at all. But these can be done later.
(cherry picked from commit 525d64fe88)
fixes#24179
The original fix made it so calls to `skError`/`skUnknown` (in this case
`->`, for some reason `sugar` couldn't be imported) returned an error
node, however this breaks tsug_accquote for some reason I don't
understand (it even parses as `tsug_accquote.discard`) so I've just
added a guard based on the stacktrace.
fixes#16376
The way the compiler handled generic proc instantiations in calls (like
`foo[int](...)`) up to this point was to instantiate `foo[int]`, create
a symbol for the instantiated proc (or a symchoice for multiple procs
excluding ones with mismatching generic param counts), then perform
overload resolution on this symbol/symchoice. The exception to this was
when the called symbol was already a symchoice node, in which case it
wasn't instantiated and overloading was called directly ([these
lines](b7b1313d21/compiler/semexprs.nim (L3366-L3371))).
This has several problems:
* Templates and macros can't create instantiated symbols, so they
couldn't participate in overloaded explicit generic instantiations,
causing the issue #16376.
* Every single proc that can be instantiated with the given generic
params is fully instantiated including the body. #9997 is about this but
isn't fixed here since the instantiation isn't in a call.
The way overload resolution handles explicit instantiations by itself is
also buggy:
* It doesn't check constraints.
* It allows only partially providing the generic parameters, which makes
sense for implicit generics, but can cause ambiguity in overloading.
Here is how this PR deals with these problems:
* Overload resolution now always handles explicit generic instantiations
in calls, in `initCandidate`, as long as the symbol resolves to a
routine symbol.
* Overload resolution now checks the generic params for constraints and
correct parameter count (ignoring implicit params). If these don't
match, the entire overload is considered as not matching and not
instantiated.
* Special error messages are added for mismatching/missing/extra generic
params. This is almost all of the diff in `semcall`.
* Procs with matching generic parameters now instantiate only the type
of the signature in overload resolution, not the proc itself, which also
works for templates and macros.
Unfortunately we can't entirely remove instantiations because overload
resolution can't handle some cases with uninstantiated types even though
it's resolved in the binding (see the last 2 blocks in
`texplicitgenerics`). There are also some instantiation issues with
default params that #24005 didn't fix but I didn't want this to become
the 3rd huge generics PR in a row so I didn't dive too deep into trying
to fix them. There is still a minor instantiation fix in `semtypinst`
though for subscripts in calls.
Additional changes:
* Overloading of `[]` wasn't documented properly, it somewhat is now
because we need to mention the limitation that it can't be done for
generic procs/types.
* Tests can now enable the new type mismatch errors with just
`-d:testsConciseTypeMismatch` in the command.
Package PRs:
- using fork for now:
[combparser](https://github.com/PMunch/combparser/pull/7) (partial
generic instantiation)
- merged: [cligen](https://github.com/c-blake/cligen/pull/233) (partial
generic instantiation but non-overloaded + template)
- merged: [neo](https://github.com/andreaferretti/neo/pull/56) (trying
to instantiate template with no generic param)
Currently, I don't have syntax highlighting (+ no/wrong
jump-to-definition) for some import statement forms, namely:
- `import module/name/with/(slashes)`
- `import (mod) as alias`
- `import basemod/[ (sub1), (sub2) ]`
With this patch, highlight/def will work for the regions indicated by
parentheses.
Currently pragmas just fall through to `suggestSentinel` and show
everything which isn't very useful. Now it filters for symbols that
could be pragmas (templates with `{.pragma.}`, macros, user pragmas) and
only shows them
Currently when using `use` with nimsuggest on an enum field, it doesn't
return the definition of the field.
Breaks renaming in IDEs since it will replace all the usages, but not
the declaration
Currently the suggestions create a lot of noise when creating enum
fields. I don't see any way of a macro creating fields (when called
inside an enum) so it should be safe to not show suggestions
---------
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* Add tests
Also test if exported all tuple fields works. This seems like a hacky solution so will try and dive further to find a better solution
* Always suggest tuple fields if it passes the filter
If the tuple we are accessing is in scope then all the fields will also be in scope
* Update tests so line numbers are correct
* fixes#19795; remove parse pipeline
* isScript
* fixes nimscriptapi
* don't touch reorder
* check script
* fixes tests
* it seems implicit imports of system cause troubles
* access the first child of `nkStmtList`
* ignore comments
* minor messages
* perhaps increases hloLoopDetector
* the module is a stmtList, which changes the errors
* fixes nimdoc
* fixes tlinter
* fixes nim secret tests
* fixes arc_misc
* fixes nim secret tests again
* safe; fixes one more test
* GlobalError is the root cause too
* fixes parsing errors
* put emit types to the cfsForwardTypes section
* fixes#11852; `{.push checks:off}` now works in procs
* disable navigator
* fixes nimdoc
* add tests for JS
* fixes nimsuggest
* Implemented level based macro expand functionality
- it can handle single macro call or expand whole function/proc/etc and it
- In addition, I have altered the parser to provide the endInfo for the node.
The usefulness of the `endInfo` is not limited to the `expandMacro`
functionality but also it is useful for `ideOutline` functionality and I have
altered the ideOutline functionality to use `endInfo`. Note `endInfo` most of
the time is lost during the AST transformation thus in `nimsuggest.nim` I am
using freshly parsed tree to get the location information.
* Make sure we stop expanding correctly
* Test CI
* Fix tv3_outline.nim
* better error messages for dot operators [backport]
fixes#13063
* also fixes#7777
* fix#6981 and #9831 too
* fix
* minor improvement
* sus test fixes
* make test multiplatform lol
* fix nimsuggest test, extra improvements
* Fix/improve handling of forward declarations in nimsuggest
- ideUse now works fine when invoked on the implementation
- implemented ideDeclaration to make cover lsp feature textDocument/declaration
- fixed performance issue related to deduplicating symbols. Now the
deduplication happens after the symbols are filtered. As a alternative we might
change the way cached symbols are stored(e. g. use set).
- I also fixed the way globalSymbols work. Now it will sort the responses based
on the match location to make sure that the results are sorted in user friendly way.
* Update nimsuggest/nimsuggest.nim
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* system refactor, move out 600 lines
* compilation, slice, backwardsindex, misc_num moved out of system
* some procs/types moved into arithmetics, basic_types
* system no longer depends on syncio
* some procs moved around to fit with their surroundings
* make exceptions an import, old ops to misc_num
* move instantiationInfo back
* move back nim version, fix windows echo
* include compilation
* better docs for imported modules, fix unsigned ops
also remove ze, ze64, toU8, toU16, toU32 with nimPreviewSlimSystem
* fix terminal
* workaround IC test & weird csize bug, changelog
* move NimMajor etc back to compilation, rebase for CI
* try ic fix
* form single `indices`, slim out TaintedString, try fix IC
* fix CI, update changelog, addQuitProc
* fix CI
* try fix CI
* actually fix CI finally hopefully
* Update lib/system/compilation.nim
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
* update kochdocs
* hopefully fix csize uses for slimsystem
* fix tquit
Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
- apparently TLineInfo's implementation of `==` ignores the column. After I fixed
the code to use exact TLineInfo comparison I fixed several other issues hidden
by that issue.
- Replaced `tuple[sym, info]` with `SymInfoPair`
* Implement type command
- this will be mapped to textDocument/typeDefinition in LSP protocol. It will be
very useful for `nim` in particular because typically most of the time the type
is inferred.
* Update nimsuggest/nimsuggest.nim
Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
* Initial implementation of nimsuggest v3
Rework `nimsuggest` to use caching to make usage of ide commands more efficient.
Previously, all commands no matter what the state of the process is were causing
clean build. In the context of Language Server Protocol(LSP) and lsp clients
this was causing perf issues and overall instability. Overall, the goal of v3 is
to fit to LSP Server needs
- added two new commands:
- `recompile` to do clean compilation
- `changed` which can be used by the IDEs to notify that a particular file has been changed.
The later can be utilized when using LSP file watches.
- `globalSymbols` - searching global references
- added `segfaults` dependency to allow fallback to clean build when incremental
fails. I wish the error to be propagated to the client so we can work on fixing
the incremental build failures (typically hitting pointer)
- more efficient rebuild flow. ATM incremental rebuild is triggered when the
command needs that(i. e. it is global) while the commands that work on the
current source rebuild only it
Things missing in this PR:
- Documentation
- Extensive unit testing.
Although functional I still see this more as a POC that this approach can work.
Next steps:
- Implement `sug` request.
- Rework/extend the protocol to allow better client/server communication.
Ideally we will need push events, diagnostics should be restructored to allow
per file notifications, etc.
- implement v3 test suite.
- better logging
* Add tests for v3 and implement ideSug
* Remove typeInstCache/procInstCache cleanup
* Add ideChkFile command
* Avoid contains call when adding symbol info
* Remove log
* Remove segfaults
* improvements to $(SomeInteger) and addInt
* remove mIntToStr, mInt64ToStr
* improvements
* fix tests/pragmas/tinjectstmt.nim; the diff is harmless, cgen code is identical with -d:danger or debug mode
* rm tests/system/tstrmantle.nim
* revert compiler/jsgen.nim for -d:nimVersion140
* improvements to hint:processing
* fix tests; do not show hintProcessing for nimscript unless given -d:nimHintProcessingNims
* fix trunner and avoid need for -d:nimHintProcessingNims
* fix some tests
* basic stability improvements; refs nimsuggest
* fixed dot operator recursive loop & macro suggest
* hacky fix for run away dot operator sem check
Committing this mostly to make the issue more clear. Perhaps get better
feedback.
* semExprWithType seems like a better place to check
* fixed error messages const case expressions
* Clean-up test
* stopped the dot operator madness
No longer get infinite recursion when seming broken code with a dot
operator macro like in jsffi.
Co-authored-by: Araq <rumpf_a@web.de>
* penalizes the quality score of deprecated symbols
* uses quality more pervasively in order to reflect deprecation impact
* impacts both sug and con
additional notes:
* linux i386 CI was failing
* this is because the suggested results differ slightly in their sort
* 64 bit tables.getOrDefault:441 was returned, while 32 bit returned 422
* for now simply removing the last line is good enough
nimsuggest outline should account for includes, now it does:
- the module prefix will be of the module doing the including
- the filename will be of the module that was included
- adds a test case for it
A number of nimsuggest tests were disabled for various reasons, sometimes due
to brittleness. These tests have been fixed where needed and most have are now
enabled -- details below. The updates are meant to provide better regression
coverage for future nimsuggest improvements. To avoid brittleness some tests
were refactored.
Impact:
* test coverage has now increased
* faster execution of the test suite
* tests are less likely to break due to stdlib changes
Re-enabled Test & Test Description:
* `tchk1.nim`: check (chk) via nimsuggest works at end of file
* `tdot4.nim`: prioritize already used completion
* `tinclude.nim`: definition lookup (def) with includes
* `tstrutils.nim` -> `tdef2.nim`: test template definition lookup (def)
* `tsug_regression.nim`: regression test for [nimsuggest #52](https://github.com/nim-lang/nimsuggest/issues/52)
* `ttemplate_highlight.nim`: per the file name
* `twithin_macro_prefix.nim`: suggest within a macro with a prefix
Tests Not Re-Enabled:
* `twithin_macro.nim` still disabled as it doesn't provide a good test signal
* EPC highlight tests remain disabled -- requires out of scope tester changes
Additional Notes:
* todos added in comments for follow-up work
