Commit Graph

5197 Commits

Author SHA1 Message Date
Arne Döring
1102b8ac6e StringStream and parseJson, parseCfg, parseSql et al for the vm (#10746) 2019-02-28 22:57:57 +01:00
Andreas Rumpf
728ff1004a gc:destructors: progress 2019-02-28 22:39:24 +01:00
alaviss
38aec488b7 compiler/commands: keep extension for output file (#10755)
Fixes #10747
2019-02-28 14:47:15 +01:00
alaviss
1b542e7320 compiler/[main,docgen]: don't put generated doc in subfolder (#10754)
* compiler/[main,docgen]: don't put generated doc in subfolder

Fixes regression caused by ca4b971bc8.

foo.nim will now generates foo.html instead of foo/foo.html

* compiler/[main,docgen]: map -o to -outdir for project

This reinstantiate the old behavior of -o for project-wide docgen
2019-02-28 14:46:35 +01:00
zah
ca4b971bc8 Initial version of the hot-code reloading support for native targets (#10729)
* squashed work by Zahary

* squashing a ton of useful history... otherwise rebasing on top of upstream Nim after commit 82c009a2cb would be impossible.

* Code review changes; Working test suite (without code reloading enabled)

* - documentation
- implemented the HCR test - almost works...
- fix the issue on Unix where for executable targets the source file for the main module of a project in nimcache was being overwritten with the binary itself (and thus the actual source code was lost)
- fixing embedded paths to shared objects on unix (the "lib" prefix was being prepended to the entire path instead of just the filename)
- other fixes
- removing unnecessary includes since that file is already included in chcks.nim which is in turn included in system.nim (and previously was getting imported in chcks.nim but then system.nim improts something... and that breaks HCR (perhaps it could be fixed but it would be nice not to import anything in system))

* fix for clang & C++ - explicitly casting a function pointer to void*
more stable mangling of parameter names when HCR is on
the length of the static arrays in the DatInit functions is now part of the name of the variables, so when they get resized they get also recreated
more stable mangling for inline functions - no longer depends on the module which first used them
work on the new complicated HCR test - turned surprisingly complex - WIP
test now successfully passes even when re-running `koch test` (previously when the nimcache wasn't cold that lead to errors)
better documentation
calling setStackBottomWith for PreMain
passes over the HcrInit/DatInit/Init calls of all modules are now in the proper order (first all of one type, then all of the next). Also typeinfo globals are registered (created) in a single pass before the DatInit pass (because of the way generic instantiations are handled)
Fix the test suite execution on macOs
fix for being able to query the program arguments when using HCR on posix!
other fixes

* Bugfix: Fix a compilation error in C++ mode when a function pointer
is converted to a raw pointer

* basic documentation for the new hot code reloading semantics

* Add change log entry

* Don't re-execute the top-level statements while reloading JS code

* fix a number of tests broken in a recent bugfix

* Review changes

* Added {.executeOnReload.} pragma that indicates top-level statements
  that should be executed on each reload. To make this work, I've modified
  the way the `if (hcr_init_) {...}` guards are produced in the init code.
  This still needs more work as the new guards seem to be inserted within
  the previously generated guards.

  This change also removes the need for `lastRegistedGlobal` in nimhcr.

* Implemented the `signatureHash` magic and the `hasModuleChanged` API
  depending on it (the actual logic is not imlemented yet).

* Add the "hcr" prefix to all HCR-related symbols in the system module.
  Added a new `hotcodereloading` module exporting the high-level API to
  the user.

  Besides being more hygienic, this was also required in order to make
  it possible to use macros in the high-level API. Without the split,
  `system` would have to import `macros`, which was going to produce
  the well-known init problems.

* Attempted to solve the "GC markers problem".

  Crashes were expected with the previous code, because the GC markers
  were compiled as normal procs are registered in the GC. When their
  module is unloaded, dangling pointers will remain in the GC tables.
  To solve this issue, I don't register any GC markers when HCR is on,
  but I add them to the HCR globals metadata and I use a single marker
  registed in nimhcr during the initialization of the system module that
  will be responsible for marking all globals.

* fix a compilation error

* - implemented the hasModuleChanged functionality
- tuples can be returned and broken into different vars in global scope
- added comments for the closnig scopes of the if statements in the init proc
- the new executeOnReload pragma works now!
- other fixes

* finally! fixing this hack in a proper way - declaring the destructor out of line (out of the class body) - we no longer need to forward-declare popCurrentExceptionEx

* Force full module parsing

This is a temporary hack that breaks some tests. I'll investigate
later how these can be fixed.

* tuples are now properly handled when global!

* these comments mess up the codegen in debug mode when $n is not actually a new line (or something like that) - these labels are intended only for GOTO labels anyway...

* "solved" the issue with the .pdb locks on windows when a binary is being debugged and hot code reloading is used at the same time

* fixes after rebasing...

* small fixes for the test

* better handling of globals! no more compiler crashes for locals with the global pragma, also simplified code around loops in global scope which have local vars (actually globals)

* we can now use the global pragma even for ... globals!

* the right output

* lets try those boehm GC tests

* after the test is ran it will be at its starting state - no git modifications

* clarification in the docs

* removed unnecessary line directives for forward declarations of functions - they were causing trouble with hot code reloading when no semantic change propagates to the main module but a line directive got changed and thus the main module had to be recompiled since the .c code had changed

* fixed bug! was inserting duplicate keys into the table and later was removing only 1 copy of all the duplicates (after a few reloads)

* no longer breaking into DatInit code when not supposed to

* fixes after rebasing

* yet more fixes after rebasing

* Update jssys.nim

* Rework the HCR path-handling logic

After reviewing the code more carefully, I've noticed that the old logic
will be broken when the user overrides the '--out:f' compiler option.

Besides fixing this issues, I took the opportunity to implement the
missing '--outdir:d' option.

Other changes:

* ./koch test won't overwrite any HCR and RTL builds located in nim/lib
* HCR and RTL are compiled with --threads:on by default

* Clean up the globals registration logic

* Handle non-flattened top-level stmtlists in JS as well

* The HCR is not supported with the Boehm GC yet

Also fixes some typos and the expected output of the HCR integration test

* The GC marker procs are now properly used as trampolines

* Fix the HCR integration test in release builds

* Fix ./koch tools

* this forward declaration doesn't seem to be necessary, and in fact breaks HCR because a 2nd function pointer is emitted for this externed/rtl func

* the forward declaration I removed in the last commit was actually necessary

* Attempt to make all tests green

* Fix tgenscript

* BAT file for running the HCR integration test on Windows [skip ci]

* Fix the docgen tests

* A final fix for Travis (hopefully)
2019-02-26 15:48:55 +01:00
Arne Döring
287206f993 minor fix for debug on symbols (#10742) 2019-02-26 15:45:01 +01:00
Andreas Rumpf
84f0a33bf0 make typeToString sane for sequence again 2019-02-25 19:01:01 +01:00
Andreas Rumpf
bf4e688ca3 make nimsuggest aware of tyOwned 2019-02-25 16:37:09 +01:00
deech
68a82f100e Fixes #10727. (#10728) 2019-02-25 11:21:14 +01:00
LemonBoy
f39aa1b40b discard destroys its argument in-place (#9478) 2019-02-23 14:03:32 +01:00
Neelesh Chandola
9884129058 Tuple unpacking now works for for vars (#10152)
* Tuple unpacking now works for `for` vars
* Give error if length of tuple vars != length of tuple
* Fix error message showing wrong tuple length
* unpacking now works now for mutable items
* Update changelog
2019-02-23 14:01:34 +01:00
LemonBoy
e89aaaeaab Open a new scope for static: expr blocks (#10649)
Bring this in line with how plain blocks are analysed and avoids codegen
errors if one references variables defined in such a block.
2019-02-23 12:05:07 +01:00
LemonBoy
71df1b060b Tighten the conversion from tyRange to scalar types (#10495)
* Tighten the conversion from tyRange to scalar types.

Introduce the `isIntConv` rule for unsigned types.
Do not allow mixed-signedness conversions between ranges and scalar types.

* More json adjustments
2019-02-23 11:52:52 +01:00
Andreas Rumpf
7ec187a8e1 make the VM aware of tyOwned 2019-02-23 11:31:31 +01:00
Timothee Cour
adbabf145c FFI at CT (#10150)
* enable FFI at CT
* rename useFFI=>nimHasLibFFI; improve formatting rawExecute traceCode
* disable libffi on windows (works for win32, not yet win64)
2019-02-23 11:31:01 +01:00
Arne Döring
30ab7e6bdd fixes #10678 (#10681) 2019-02-23 10:58:40 +01:00
Andreas Rumpf
ea409fb15a first steps in implementing 'owned' pointers; undocumented, do not use 2019-02-23 08:44:08 +01:00
Andreas Rumpf
721bf7188b code cleanup: there is no tyOptRef 2019-02-22 19:42:11 +01:00
Andreas Rumpf
15c208cd29 it's spelt callsite 2019-02-22 18:28:49 +01:00
Andreas Rumpf
a4543c1aac gc:destructors: make strutils compile (but still crashes) 2019-02-22 12:06:30 +01:00
Andreas Rumpf
417d27c544 gc:destructors: progress 2019-02-21 12:19:06 +01:00
Andreas Rumpf
43d570c178 gc:destructors: make system/repr compile 2019-02-21 07:57:29 +01:00
Araq
6c10d331db gc:destructors: slightly more stuff compiles 2019-02-20 14:29:18 +01:00
Andreas Rumpf
c025786b75 gc:destructors: sink parameters need to be destroyed properly 2019-02-20 02:04:58 +01:00
Andreas Rumpf
7e404c670d gc:destructors: add first test program 2019-02-19 23:32:41 +01:00
Araq
f31d1fa98b fixes #10195 2019-02-19 16:25:57 +01:00
Arne Döring
6b9a139f7f error message for accidental use of macro (#10490)
* error message for accidental use of macro
2019-02-19 12:21:42 +01:00
Araq
257965e105 fixes 10697 [backport] 2019-02-19 11:56:14 +01:00
Andreas Rumpf
2deb1e354f fixes #10702 (#10705)
* --define:nimQuirky exception handling for Nim; in preparation of a blog post
* make it work with latest system.nim
* make code more readable
* fixes #10702
2019-02-18 15:23:05 +01:00
LemonBoy
cf32d61fa5 Prevent crash on pragma templates w/ generics (#10685)
* Prevent crash on pragma templates w/ generics

* Remove incorrect call to pragma reconversion

`semOverloadedCall` may return a node with more elements than the
original nkCall node had (implicit and/or explicit generics).
2019-02-18 12:17:00 +01:00
Araq
b081eb4d6d fixes #10651 2019-02-14 15:55:38 +01:00
Timothee Cour
9913cef872 fix typo in codeReordering error msg (#10667)
* fix typo in codeReordering

* reorder=>codeReordering
2019-02-14 09:52:15 +01:00
alaviss
3520253a09 compiler/types: correctly generates signature for non-proc types (#10658)
This makes signatures generated for nimsuggest correctly distinguish
template/macro/converter from proc.
2019-02-13 08:07:58 +01:00
Arne Döring
6870345cd2 32 bit fixes (#10608) 2019-02-12 22:00:31 +01:00
Arne Döring
0e5d4f1036 reduce debug output (#10638) 2019-02-12 08:27:38 +01:00
LemonBoy
1d1360496f Fix codegen problem with strict C++ compilers (#10639)
Since tyCString is convertible to a tyPointer we must be extra careful
to emit a cast to (void*) in order to appease clang++.

Reported by masnagam on the Nim forum.
2019-02-12 08:23:24 +01:00
LemonBoy
5fd9bf9cd5 Propagate tfGcSafe flag to generic instantiations (#10620)
Fixes a nasty endless loop in the generic instantiation phase.
2019-02-10 22:07:50 +01:00
Timothee Cour
4910608394 revive #10228 (fix #9880) (#10610)
* Make index out of bounds more useful by including the 'bounds'.
* fixes #9880 index out of bounds (remaining cases); revives #10228
* change err msg to: `index 3 not in 0 .. 1`
2019-02-10 22:07:11 +01:00
Andreas Rumpf
f0e6becc23 fixes #10547 2019-02-10 17:03:32 +01:00
LemonBoy
0c4d1fd10b Do not walk into type sub-nodes for cast/conv expr (#10616) 2019-02-09 19:55:26 +01:00
Andreas Rumpf
c6fdf93c7f fixes #6955 2019-02-09 11:18:21 +01:00
Andreas Rumpf
3c909ae8ff fixes #10606 2019-02-09 08:18:33 +01:00
Arne Döring
594c2576e6 Print missing case labels (#10600) 2019-02-09 07:36:30 +01:00
Andreas Rumpf
444f2231c9 make tests green again 2019-02-08 16:54:12 +01:00
LemonBoy
aa6e40abe6 Fix wrong result in tuple assignment (#9340)
Fixes #9177
2019-02-08 12:24:03 +01:00
Araq
4ed7507cb2 error messages can have spaces, don't be dyslexic 2019-02-08 12:18:42 +01:00
LemonBoy
710cfcecd3 Rework exception handling in the VM (#10544)
* Rework exception handling in the VM

Make the safepoint handling more precise and less forgiving.
The new code is clearer and more commented.
Perform cleanup on `return`.
The no-exception-thrown case in a try block should be slightly faster
since we don't parse the whole set of exceptions every time.
More tests.

* Fix silly error that broke a few tests

* Testament doesn't like files having the same name

* Remove test case that failed compilation to js
2019-02-08 11:57:47 +01:00
LemonBoy
631a8ab57f Fix edge case in type hashing (#10601) [backport]
Empty types introduced by a template produced the same hash of the
"clean" type sharing the same name.
2019-02-08 09:56:32 +01:00
LemonBoy
6b88ce3384 Stop useless suggestion of unsafeAddr (#10598)
Fixes #10594
2019-02-08 08:59:38 +01:00
alaviss
2fdf816332 compiler/[msgs, suggest]: improve highlighter accuracy (#10496)
Previously the compiler would generate suggestions based on the symbol
identifier length, but that might not reflect the actual representation
of it within the actual source code.

This commit implements a simple source scanner for the suggest module to
address the problem outlined above.

Fixes nim-lang/nimsuggest#24
2019-02-07 17:08:15 +01:00