mirror of
https://github.com/nim-lang/Nim.git
synced 2026-05-03 20:44:46 +00:00
Extend and document compiler debugging utilities (#19841)
* Add two debugutils procs that native debuggers can break on use to execute commands when code of interest is being compiled. * Add GDB and LLDB programs to disable and enable breakpoints and watchpoints when code of interest is being compiled. * Extend the `intern.rst` docs regarding debugging the compiler. Co-authored-by: quantimnot <quantimnot@users.noreply.github.com>
This commit is contained in:
quantimnot
@@ -54,3 +54,19 @@ proc isCompilerDebug*(): bool =
|
||||
{.undef(nimCompilerDebug).}
|
||||
echo 'x'
|
||||
conf0.isDefined("nimCompilerDebug")
|
||||
|
||||
proc enteringDebugSection*() {.exportc, dynlib.} =
|
||||
## Provides a way for native debuggers to enable breakpoints, watchpoints, etc
|
||||
## when code of interest is being compiled.
|
||||
##
|
||||
## Set your debugger to break on entering `nimCompilerIsEnteringDebugSection`
|
||||
## and then execute a desired command.
|
||||
discard
|
||||
|
||||
proc exitingDebugSection*() {.exportc, dynlib.} =
|
||||
## Provides a way for native debuggers to disable breakpoints, watchpoints, etc
|
||||
## when code of interest is no longer being compiled.
|
||||
##
|
||||
## Set your debugger to break on entering `exitingDebugSection`
|
||||
## and then execute a desired command.
|
||||
discard
|
||||
|
||||
222
doc/intern.rst
222
doc/intern.rst
@@ -79,55 +79,216 @@ Set the compilation timestamp with the `SOURCE_DATE_EPOCH` environment variable.
|
||||
koch boot # or `./build_all.sh`
|
||||
|
||||
|
||||
Developing the compiler
|
||||
=======================
|
||||
Debugging the compiler
|
||||
======================
|
||||
|
||||
To create a new compiler for each run, use `koch temp`:cmd:\:
|
||||
|
||||
Bisecting for regressions
|
||||
-------------------------
|
||||
|
||||
There are often times when there is a bug that is caused by a regression in the
|
||||
compiler or stdlib. Bisecting the Nim repo commits is a usefull tool to identify
|
||||
what commit introduced the regression.
|
||||
|
||||
Even if it's not known whether a bug is caused by a regression, bisection can reduce
|
||||
debugging time by ruling it out. If the bug is found to be a regression, then you
|
||||
focus on the changes introduced by that one specific commit.
|
||||
|
||||
`koch temp`:cmd: returns 125 as the exit code in case the compiler
|
||||
compilation fails. This exit code tells `git bisect`:cmd: to skip the
|
||||
current commit:
|
||||
|
||||
.. code:: cmd
|
||||
|
||||
koch temp c test.nim
|
||||
git bisect start bad-commit good-commit
|
||||
git bisect run ./koch temp -r c test-source.nim
|
||||
|
||||
`koch temp`:cmd: creates a debug build of the compiler, which is useful
|
||||
to create stacktraces for compiler debugging.
|
||||
You can also bisect using custom options to build the compiler, for example if
|
||||
you don't need a debug version of the compiler (which runs slower), you can replace
|
||||
`./koch temp`:cmd: by explicit compilation command, see `Bootstrapping the compiler`_.
|
||||
|
||||
You can of course use GDB or Visual Studio to debug the
|
||||
compiler (via `--debuginfo --lineDir:on`:option:). However, there
|
||||
are also lots of procs that aid in debugging:
|
||||
|
||||
Building an instrumented compiler
|
||||
---------------------------------
|
||||
|
||||
Considering that a useful method of debugging the compiler is inserting debug
|
||||
logging, or changing code and then observing the outcome of a testcase, it is
|
||||
fastest to build a compiler that is instrumented for debugging from an
|
||||
existing release build. `koch temp`:cmd: provides a convenient method of doing
|
||||
just that.
|
||||
|
||||
By default running `koch temp`:cmd: will build a lean version of the compiler
|
||||
with `-d:debug`:option: enabled. The compiler is written to `bin/nim_temp` by
|
||||
default. A lean version of the compiler lacks JS and documentation generation.
|
||||
|
||||
`bin/nim_temp` can be directly used to run testcases, or used with testament
|
||||
with `testament --nim:bin/nim_temp r tests/category/tsometest`:cmd:.
|
||||
|
||||
`koch temp`:cmd: will build the temporary compiler with the `-d:debug`:option:
|
||||
enabled. Here are compiler options that are of interest for debugging:
|
||||
|
||||
* `-d:debug`:option:\: enables `assert` statements and stacktraces and all
|
||||
runtime checks
|
||||
* `--opt:speed`:option:\: build with optimizations enabled
|
||||
* `--debugger:native`:option:\: enables `--debuginfo --lineDir:on`:option: for using
|
||||
a native debugger like GDB, LLDB or CDB
|
||||
* `-d:nimDebug`:option: cause calls to `quit` to raise an assertion exception
|
||||
* `-d:nimDebugUtils`:option:\: enables various debugging utilities;
|
||||
see `compiler/debugutils`
|
||||
* `-d:stacktraceMsgs -d:nimCompilerStacktraceHints`:option:\: adds some additional
|
||||
stacktrace hints; see https://github.com/nim-lang/Nim/pull/13351
|
||||
* `-u:leanCompiler`:option:\: enable JS and doc generation
|
||||
|
||||
Another method to build and run the compiler is directly through `koch`:cmd:\:
|
||||
|
||||
.. code:: cmd
|
||||
|
||||
koch temp [options] c test.nim
|
||||
|
||||
# (will build with js support)
|
||||
koch temp [options] js test.nim
|
||||
|
||||
# (will build with doc support)
|
||||
koch temp [options] doc test.nim
|
||||
|
||||
Debug logging
|
||||
-------------
|
||||
|
||||
"Printf debugging" is still the most appropriate way to debug many problems
|
||||
arising in compiler development. The typical usage of breakpoints to debug
|
||||
the code is often less practical, because almost all of the code paths in the
|
||||
compiler will be executed hundreds of times before a particular section of the
|
||||
tested program is reached where the newly developed code must be activated.
|
||||
|
||||
To work-around this problem, you'll typically introduce an if statement in the
|
||||
compiler code detecting more precisely the conditions where the tested feature
|
||||
is being used. One very common way to achieve this is to use the `mdbg` condition,
|
||||
which will be true only in contexts, processing expressions and statements from
|
||||
the currently compiled main module:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
# dealing with PNode:
|
||||
echo renderTree(someNode)
|
||||
debug(someNode) # some JSON representation
|
||||
# inside some compiler module
|
||||
if mdbg:
|
||||
debug someAstNode
|
||||
|
||||
# dealing with PType:
|
||||
Using the `isCompilerDebug`:nim: condition along with inserting some statements
|
||||
into the testcase provides more granular logging:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
# compilermodule.nim
|
||||
if isCompilerDebug():
|
||||
debug someAstNode
|
||||
|
||||
# testcase.nim
|
||||
proc main =
|
||||
{.define(nimCompilerDebug).}
|
||||
let a = 2.5 * 3
|
||||
{.undef(nimCompilerDebug).}
|
||||
|
||||
Logging can also be scoped to a specific filename as well. This will of course
|
||||
match against every module with that name.
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
if `??`(conf, n.info, "module.nim"):
|
||||
debug(n)
|
||||
|
||||
The above examples also makes use of the `debug`:nim: proc, which is able to
|
||||
print a human-readable form of an arbitrary AST tree. Other common ways to print
|
||||
information about the internal compiler types include:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
# pretty print PNode
|
||||
|
||||
# pretty prints the Nim ast
|
||||
echo renderTree(someNode)
|
||||
|
||||
# pretty prints the Nim ast, but annotates symbol IDs
|
||||
echo renderTree(someNode, {renderIds})
|
||||
|
||||
# pretty print ast as JSON
|
||||
debug(someNode)
|
||||
|
||||
# print as YAML
|
||||
echo treeToYaml(config, someNode)
|
||||
|
||||
|
||||
# pretty print PType
|
||||
|
||||
# print type name
|
||||
echo typeToString(someType)
|
||||
|
||||
# pretty print as JSON
|
||||
debug(someType)
|
||||
|
||||
# dealing with PSym:
|
||||
# print as YAML
|
||||
echo typeToYaml(config, someType)
|
||||
|
||||
|
||||
# pretty print PSym
|
||||
|
||||
# print the symbol's name
|
||||
echo symbol.name.s
|
||||
|
||||
# pretty print as JSON
|
||||
debug(symbol)
|
||||
|
||||
# pretty prints the Nim ast, but annotates symbol IDs:
|
||||
echo renderTree(someNode, {renderIds})
|
||||
if `??`(conf, n.info, "temp.nim"):
|
||||
# only output when it comes from "temp.nim"
|
||||
echo renderTree(n)
|
||||
if `??`(conf, n.info, "temp.nim"):
|
||||
# why does it process temp.nim here?
|
||||
writeStackTrace()
|
||||
# print as YAML
|
||||
echo symToYaml(config, symbol)
|
||||
|
||||
|
||||
# pretty print TLineInfo
|
||||
lineInfoToStr(lineInfo)
|
||||
|
||||
|
||||
# print the structure of any type
|
||||
repr(someVar)
|
||||
|
||||
Here are some other helpful utilities:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
# how did execution reach this location?
|
||||
writeStackTrace()
|
||||
|
||||
These procs may not already be imported by the module you're editing.
|
||||
You can import them directly for debugging:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
from astalgo import debug
|
||||
from types import typeToString
|
||||
from renderer import renderTree
|
||||
from msgs import `??`
|
||||
|
||||
Native debugging
|
||||
----------------
|
||||
|
||||
Stepping through the compiler with a native debugger is a very powerful tool to
|
||||
both learn and debug it. However, there is still the need to constrain when
|
||||
breakpoints are triggered. The same methods as in `Debug logging`_ can be applied
|
||||
here when combined with calls to the debug helpers `enteringDebugSection()`:nim:
|
||||
and `exitingDebugSection()`:nim:.
|
||||
|
||||
#. Compile the temp compiler with `--debugger:native -d:nimDebugUtils`:option:
|
||||
#. Set your desired breakpoints or watchpoints.
|
||||
#. Configure your debugger:
|
||||
* GDB: execute `source tools/compiler.gdb` at startup
|
||||
* LLDB execute `command source tools/compiler.lldb` at startup
|
||||
#. Use one of the scoping helpers like so:
|
||||
|
||||
.. code-block:: nim
|
||||
|
||||
if isCompilerDebug():
|
||||
enteringDebugSection()
|
||||
else:
|
||||
exitingDebugSection()
|
||||
|
||||
A caveat of this method is that all breakpoints and watchpoints are enabled or
|
||||
disabled. Also, due to a bug, only breakpoints can be constrained for LLDB.
|
||||
|
||||
The compiler's architecture
|
||||
===========================
|
||||
@@ -152,23 +313,6 @@ for the type definitions. The `macros <macros.html>`_ module contains many
|
||||
examples how the AST represents each syntactic structure.
|
||||
|
||||
|
||||
Bisecting for regressions
|
||||
=========================
|
||||
|
||||
`koch temp`:cmd: returns 125 as the exit code in case the compiler
|
||||
compilation fails. This exit code tells `git bisect`:cmd: to skip the
|
||||
current commit:
|
||||
|
||||
.. code:: cmd
|
||||
|
||||
git bisect start bad-commit good-commit
|
||||
git bisect run ./koch temp -r c test-source.nim
|
||||
|
||||
You can also bisect using custom options to build the compiler, for example if
|
||||
you don't need a debug version of the compiler (which runs slower), you can replace
|
||||
`./koch temp`:cmd: by explicit compilation command, see `Bootstrapping the compiler`_.
|
||||
|
||||
|
||||
Runtimes
|
||||
========
|
||||
|
||||
|
||||
@@ -38,6 +38,9 @@ options:
|
||||
unless you are debugging the compiler.
|
||||
-d:nimUseLinenoise Use the linenoise library for interactive mode
|
||||
(not needed on Windows).
|
||||
-d:leanCompiler Produce a compiler without JS codegen or
|
||||
documentation generator in order to use less RAM
|
||||
for bootstrapping.
|
||||
|
||||
After compilation is finished you will hopefully end up with the nim
|
||||
compiler in the `bin` directory. You can add Nim's `bin` directory to
|
||||
|
||||
39
tools/compiler.gdb
Normal file
39
tools/compiler.gdb
Normal file
@@ -0,0 +1,39 @@
|
||||
# create a breakpoint on `debugutils.enteringDebugSection`
|
||||
define enable_enteringDebugSection
|
||||
break -function enteringDebugSection
|
||||
# run these commands once breakpoint enteringDebugSection is hit
|
||||
command
|
||||
# enable all breakpoints and watchpoints
|
||||
enable
|
||||
# continue execution
|
||||
cont
|
||||
end
|
||||
end
|
||||
|
||||
# create a breakpoint on `debugutils.exitingDebugSection` named exitingDebugSection
|
||||
define enable_exitingDebugSection
|
||||
break -function exitingDebugSection
|
||||
# run these commands once breakpoint exitingDebugSection is hit
|
||||
command
|
||||
# disable all breakpoints and watchpoints
|
||||
disable
|
||||
# but enable the enteringDebugSection breakpoint
|
||||
enable_enteringDebugSection
|
||||
# continue execution
|
||||
cont
|
||||
end
|
||||
end
|
||||
|
||||
# some commands can't be set until the process is running, so set an entry breakpoint
|
||||
break -function NimMain
|
||||
# run these commands once breakpoint NimMain is hit
|
||||
command
|
||||
# disable all breakpoints and watchpoints
|
||||
disable
|
||||
# but enable the enteringDebugSection breakpoint
|
||||
enable_enteringDebugSection
|
||||
# no longer need this breakpoint
|
||||
delete -function NimMain
|
||||
# continue execution
|
||||
cont
|
||||
end
|
||||
40
tools/compiler.lldb
Normal file
40
tools/compiler.lldb
Normal file
@@ -0,0 +1,40 @@
|
||||
# create a breakpoint on `debugutils.enteringDebugSection` named enteringDebugSection
|
||||
breakpoint set -n 'enteringDebugSection' -N enteringDebugSection
|
||||
# run these commands once breakpoint enteringDebugSection is hit
|
||||
breakpoint command add enteringDebugSection
|
||||
# enable all breakpoints
|
||||
breakpoint enable
|
||||
# enable all watchpoints
|
||||
# watchpoint enable # FIXME: not currently working for unknown reason
|
||||
# continue execution
|
||||
continue
|
||||
DONE
|
||||
|
||||
# create a breakpoint on `debugutils.exitingDebugSection` named exitingDebugSection
|
||||
breakpoint set -n 'exitingDebugSection' -N exitingDebugSection
|
||||
# run these commands once breakpoint exitingDebugSection is hit
|
||||
breakpoint command add exitingDebugSection
|
||||
# disable all breakpoints
|
||||
breakpoint disable
|
||||
# disable all watchpoints
|
||||
# watchpoint disable # FIXME: not currently working for unknown reason
|
||||
breakpoint enable enteringDebugSection
|
||||
# continue execution
|
||||
continue
|
||||
DONE
|
||||
|
||||
# some commands can't be set until the process is running, so set an entry breakpoint
|
||||
breakpoint set -n NimMain -N NimMain
|
||||
# run these commands once breakpoint NimMain is hit
|
||||
breakpoint command add NimMain
|
||||
# disable all breakpoints
|
||||
breakpoint disable
|
||||
# disable all watchpoints
|
||||
# watchpoint disable # FIXME: not currently working for unknown reason
|
||||
# enable the enteringDebugSection breakpoint though
|
||||
breakpoint enable enteringDebugSection
|
||||
# no longer need this breakpoint
|
||||
breakpoint delete NimMain
|
||||
# continue execution
|
||||
continue
|
||||
DONE
|
||||
Reference in New Issue
Block a user