mirror of
https://github.com/nim-lang/Nim.git
synced 2026-01-20 03:27:11 +00:00
123 lines
14 KiB
HTML
123 lines
14 KiB
HTML
<?xml version="1.0" encoding="utf-8" ?>
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<!-- This file is generated by Nim. -->
|
|
<html xmlns="https://www.w3.org/1999/xhtml" xml:lang="en" lang="en" data-theme="auto">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>Incremental Compilation (IC)</title>
|
|
|
|
<!-- Google fonts -->
|
|
<link href='https://fonts.googleapis.com/css?family=Lato:400,600,900' rel='stylesheet' type='text/css'/>
|
|
<link href='https://fonts.googleapis.com/css?family=Source+Code+Pro:400,500,600' rel='stylesheet' type='text/css'/>
|
|
|
|
<!-- Favicon -->
|
|
<link rel="shortcut icon" href="data:image/x-icon;base64,AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AAAAAAUAAAAF////AP///wD///8A////AP///wD///8A////AP///wD///8A////AAAAAAIAAABbAAAAlQAAAKIAAACbAAAAmwAAAKIAAACVAAAAWwAAAAL///8A////AP///wD///8A////AAAAABQAAADAAAAAYwAAAA3///8A////AP///wD///8AAAAADQAAAGMAAADAAAAAFP///wD///8A////AP///wAAAACdAAAAOv///wD///8A////AP///wD///8A////AP///wD///8AAAAAOgAAAJ3///8A////AP///wAAAAAnAAAAcP///wAAAAAoAAAASv///wD///8A////AP///wAAAABKAAAAKP///wAAAABwAAAAJ////wD///8AAAAAgQAAABwAAACIAAAAkAAAAJMAAACtAAAAFQAAABUAAACtAAAAkwAAAJAAAACIAAAAHAAAAIH///8A////AAAAAKQAAACrAAAAaP///wD///8AAAAARQAAANIAAADSAAAARf///wD///8AAAAAaAAAAKsAAACk////AAAAADMAAACcAAAAnQAAABj///8A////AP///wAAAAAYAAAAGP///wD///8A////AAAAABgAAACdAAAAnAAAADMAAAB1AAAAwwAAAP8AAADpAAAAsQAAAE4AAAAb////AP///wAAAAAbAAAATgAAALEAAADpAAAA/wAAAMMAAAB1AAAAtwAAAOkAAAD/AAAA/wAAAP8AAADvAAAA3gAAAN4AAADeAAAA3gAAAO8AAAD/AAAA/wAAAP8AAADpAAAAtwAAAGUAAAA/AAAA3wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAADfAAAAPwAAAGX///8A////AAAAAEgAAADtAAAAvwAAAL0AAADGAAAA7wAAAO8AAADGAAAAvQAAAL8AAADtAAAASP///wD///8A////AP///wD///8AAAAAO////wD///8A////AAAAAIcAAACH////AP///wD///8AAAAAO////wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A////AP///wD///8A//8AAP//AAD4HwAA7/cAAN/7AAD//wAAoYUAAJ55AACf+QAAh+EAAAAAAADAAwAA4AcAAP5/AAD//wAA//8AAA=="/>
|
|
<link rel="icon" type="image/png" sizes="32x32" href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAABmJLR0QA/wD/AP+gvaeTAAAACXBIWXMAAA3XAAAN1wFCKJt4AAAAB3RJTUUH4QQQEwksSS9ZWwAAAk1JREFUWMPtll2ITVEUx39nn/O7Y5qR8f05wtCUUr6ZIS++8pEnkZInPImneaCQ5METNdOkeFBKUhMPRIkHKfEuUZSUlGlKPN2TrgfncpvmnntnmlEyq1Z7t89/rf9a6+y99oZxGZf/XeIq61EdtgKXgdXA0xrYAvBjOIF1AI9zvjcC74BSpndrJPkBWDScTF8Aa4E3wDlgHbASaANmVqlcCnwHvgDvgVfAJ+AikAAvgfVZwLnSVZHZaOuKoQi3ZOMi4NkYkpe1p4J7A8BpYAD49hfIy/oqG0+hLomiKP2L5L+1ubn5115S+3OAn4EnwBlgMzCjyt6ZAnQCJ4A7wOs88iRJHvw50HoujuPBoCKwHWiosy8MdfZnAdcHk8dxXFJ3VQbQlCTJvRBCGdRbD4M6uc5glpY3eAihpN5S5w12diSEcCCEcKUO4ljdr15T76ur1FDDLIQQ3qv71EdDOe3Kxj3leRXyk+pxdWnFWod6Wt2bY3de3aSuUHcPBVimHs7mK9WrmeOF6lR1o9qnzskh2ar2qm1qizpfXaPeVGdlmGN5pb09qMxz1Xb1kLqgzn1RyH7JUXW52lr5e/Kqi9qpto7V1atuUzfnARrV7jEib1T76gG2qxdGmXyiekkt1GswPTtek0aBfJp6YySGBfWg2tPQ0FAYgf1stUfdmdcjarbYJEniKIq6gY/Aw+zWHAC+p2labGpqiorFYgGYCEzN7oQdQClN07O1/EfDyGgC0ALMBdYAi4FyK+4H3gLPsxfR1zRNi+NP7nH5J+QntnXe5B5mpfQAAAAASUVORK5CYII=">
|
|
|
|
<!-- CSS -->
|
|
<link rel="stylesheet" type="text/css" href="nimdoc.out.css?v=2.3.1">
|
|
|
|
<!-- JS -->
|
|
<script type="text/javascript" src="dochack.js?v=2.3.1"></script>
|
|
</head>
|
|
<body>
|
|
<div class="document" id="documentId">
|
|
<div class="container">
|
|
<h1 class="title">Incremental Compilation (IC)</h1>
|
|
<p>The <tt class="docutils literal"><span class="pre">nim ic</span></tt> command provides incremental compilation support for Nim projects, allowing faster rebuilds by reusing previously compiled intermediate representations of modules that haven't changed.</p>
|
|
|
|
<h1 id="overview">Overview</h1><p>Incremental compilation works by decomposing the compilation process into several stages:</p>
|
|
<ol class="simple"><li><strong>Parsing</strong> - Source files are parsed into an abstract syntax tree (AST)</li>
|
|
<li><strong>Semantic Analysis</strong> - Symbols are resolved and type checking is performed</li>
|
|
<li><strong>Code Generation</strong> - Platform-specific code is generated from the analyzed AST</li>
|
|
<li><strong>Linking</strong> - The generated code is linked into an executable</li>
|
|
</ol>
|
|
<p>The IC mechanism caches the results of earlier stages in NIF files (Nim intermediate format): <tt class="docutils literal"><span class="pre">.p.nif</span></tt> (parsed), <tt class="docutils literal"><span class="pre">.deps.nif</span></tt> (dependencies), and <tt class="docutils literal"><span class="pre">.nif</span></tt> (semantically analyzed). When recompiling, only modules that have changed need to be reprocessed through the semantic analysis and code generation stages, significantly reducing compilation time for large projects.</p>
|
|
|
|
<h1 id="nif-file-format">NIF File Format</h1><p>NIF (Nim Intermediate Format) files are text-based files that use a Lisp-like syntax. They employ a hybrid format where byte offsets into the text are used for efficient access, making them simultaneously human-readable and machine-efficient. The text representation is particularly valuable for debugging and introspection.</p>
|
|
<p>Each <tt class="docutils literal"><span class="pre">.nim</span></tt> module produces its own <tt class="docutils literal"><span class="pre">.nif</span></tt> file during compilation. The NIF format contains:</p>
|
|
<ul class="simple"><li><strong>Header</strong> - Version information (e.g., <tt class="docutils literal"><span class="pre">(.nif26)</span></tt>)</li>
|
|
<li><strong>Dependencies</strong> - List of source files and dependencies</li>
|
|
<li><strong>Interface</strong> - Exported symbols and their indices</li>
|
|
<li><strong>Body</strong> - The intermediate representation of the module's code in Lisp-like syntax</li>
|
|
</ul>
|
|
<p>The NIF format is designed specifically for Nim and allows efficient serialization and deserialization of the compiler's intermediate representation while remaining readable and debuggable by tools and developers.</p>
|
|
|
|
<h1 id="the-nim-ic-switch">The <tt class="docutils literal"><span class="pre">nim ic</span></tt> Switch</h1><p>The <tt class="docutils literal"><span class="pre">nim ic</span></tt> command initiates incremental compilation for a project. It automatically manages the build process by:</p>
|
|
<ol class="simple"><li>Parsing all source files into <tt class="docutils literal"><span class="pre">.nif</span></tt> format (using the <tt class="docutils literal"><span class="pre">nifler</span></tt> tool)</li>
|
|
<li>Performing semantic analysis on modified modules</li>
|
|
<li>Generating code only for modules with changes or dependencies on changed modules</li>
|
|
<li>Generating a build file (in NIFMake format) that orchestrates the compilation</li>
|
|
<li>Executing the build file through <tt class="docutils literal"><span class="pre">nifmake</span></tt></li>
|
|
</ol>
|
|
|
|
<h2 id="prerequisites">Prerequisites</h2><ul class="simple"><li><strong>nifler</strong> - Tool for parsing Nim source files into NIF format. The <tt class="docutils literal"><span class="pre">nim ic</span></tt> command uses <tt class="docutils literal"><span class="pre">nifler parse --deps</span></tt> to generate both parsed files (<tt class="docutils literal"><span class="pre">.p.nif</span></tt>) and dependency files (<tt class="docutils literal"><span class="pre">.deps.nif</span></tt>).</li>
|
|
<li><strong>nifmake</strong> - Build orchestration tool that follows dependencies and executes the build rules defined in <tt class="docutils literal"><span class="pre">.build.nif</span></tt> files.</li>
|
|
</ul>
|
|
<p>If these tools are not available, <tt class="docutils literal"><span class="pre">nim ic</span></tt> will display instructions on how to obtain them.</p>
|
|
|
|
<h1 id="key-modules-for-ic-logic">Key Modules for IC Logic</h1><p>The primary modules in the compiler that handle incremental compilation logic are:</p>
|
|
<ul class="simple"><li><strong>deps.nim</strong> - Dependency analysis and build file generation. Contains the <tt class="docutils literal"><span class="pre">commandIc</span></tt> procedure which is the main entry point for the <tt class="docutils literal"><span class="pre">nim ic</span></tt> command. This module orchestrates the incremental compilation process, handling dependency traversal (via <tt class="docutils literal"><span class="pre">nifler deps</span></tt>), build rule generation, and build file creation. The build file is written to <tt class="docutils literal"><span class="pre">nifcache/</span></tt> directory. This module also explicitly models <tt class="docutils literal"><span class="pre">system.nim</span></tt> as a dependency of all modules.</li>
|
|
<li><strong>ast2nif.nim</strong> - Core mapping between AST and NIF.</li>
|
|
</ul>
|
|
|
|
<h1 id="code-logic-amp-debugging"><strong>Code, Logic & Debugging</strong></h1><p>This section focuses on the compiler-side code paths, the logic you will inspect while debugging IC, and a pragmatic manual workflow for bug hunting using local invocations such as <tt class="docutils literal"><span class="pre">nim m --nimcache:nifcache</span></tt>.</p>
|
|
<p>Core places to inspect</p>
|
|
<ul class="simple"><li><strong>`compiler/deps.nim`</strong>: generates the NIF-based build file and implements <tt class="docutils literal"><span class="pre">commandIc</span></tt> (entry point for <tt class="docutils literal"><span class="pre">nim ic</span></tt>). Look for how build rules are emitted (calls to the NIF builder) and how inputs/outputs are wired.</li>
|
|
<li><strong>`compiler/modulegraphs.nim`</strong> and <strong>`compiler/pipelines.nim`</strong>: dependency graph and compilation pipeline integration — useful when a module is rebuilt unexpectedly.</li>
|
|
</ul>
|
|
<p>Understanding the NIF text</p>
|
|
<ul class="simple"><li>NIF files are human-readable; open the per-module <tt class="docutils literal"><span class="pre">.nif</span></tt> files in <tt class="docutils literal"><span class="pre">nifcache/</span></tt> to inspect parsed ASTs, dependency lists and interface tables.</li>
|
|
<li>Because NIF uses textual nodes and byte offsets, tools can quickly seek to positions in the file — but for debugging you usually only need to read the file top-to-bottom.</li>
|
|
</ul>
|
|
<p>Manual bug-hunting workflow</p>
|
|
<ul class="simple"><li><p>Prepare a clean nimcache directory (relative to your project):</p>
|
|
<p><pre class="listing">mkdir -p nifcache</pre></p>
|
|
</li>
|
|
<li><p>Parse/semantic-check a single module and write NIF/sem artifacts:</p>
|
|
<p><pre class="listing">nim m --nimcache:nifcache path/to/module.nim</pre></p>
|
|
<ul class="simple"><li><tt class="docutils literal"><span class="pre">nim m</span></tt> runs the compiler up to the semantic checking stage for the specified module and emits intermediate cache files into <tt class="docutils literal"><span class="pre">nifcache/</span></tt>.</li>
|
|
<li>Use this to reproduce and isolate failures in the semantic stage.</li>
|
|
</ul>
|
|
</li>
|
|
<li><p>Inspect the generated files for that module under <tt class="docutils literal"><span class="pre">nifcache/</span></tt> (look for <tt class="docutils literal"><span class="pre">.nif</span></tt>, sem/parsed artifacts). Because NIF is text-based you can open and grep it directly:</p>
|
|
<p><pre class="listing">sed -n '1,200p' nifcache/ModuleName.nif
|
|
grep -n "someSymbol" -n nifcache/ModuleName.nif</pre></p>
|
|
</li>
|
|
<li>To reproduce a full incremental compilation of the project, generate the build file and run it (<tt class="docutils literal"><span class="pre">nim ic</span></tt> automates this). The build file is generated in <tt class="docutils literal"><span class="pre">nifcache/</span></tt> directory. To debug an individual build step, run the command that the build file would execute manually:<ul class="simple"><li>Parsing step: <tt class="docutils literal"><span class="pre">nifler parse --deps input.nim</span></tt> (produces <tt class="docutils literal"><span class="pre">.p.nif</span></tt> and <tt class="docutils literal"><span class="pre">.deps.nif</span></tt>)</li>
|
|
<li>Semantic step: <tt class="docutils literal"><span class="pre">nim m --nimcache:nifcache input.nim</span></tt> (produces <tt class="docutils literal"><span class="pre">.nif</span></tt>)</li>
|
|
<li>Code generation: <tt class="docutils literal"><span class="pre">nim nifc --nimcache:nifcache input.nim</span></tt> (produces executable)</li>
|
|
</ul>
|
|
</li>
|
|
<li><p>Force a cache invalidation for a single module by removing its NIF/sem artifact and re-running the semantic step:</p>
|
|
<p><pre class="listing">rm nifcache/ModuleName.nif
|
|
nim m --nimcache:nifcache path/to/ModuleName.nim</pre></p>
|
|
</li>
|
|
<li>When investigating incorrect replayed state (pragmas, <tt class="docutils literal"><span class="pre">{.compile: ...}</span></tt>): inspect the replay actions in <tt class="docutils literal"><span class="pre">compiler/ic/replayer.nim</span></tt> and open the module's NIF to find the <tt class="docutils literal"><span class="pre">toReplay</span></tt>/action entries that will be executed during reload.</li>
|
|
</ul>
|
|
<p>Tips for efficient debugging</p>
|
|
<ul class="simple"><li>Use <tt class="docutils literal"><span class="pre">--path:...</span></tt> flags when invoking <tt class="docutils literal"><span class="pre">nim m</span></tt> to emulate the exact search paths used in your project, e.g. <tt class="docutils literal"><span class="pre">--path:lib --path:vendor</span></tt>.</li>
|
|
<li>Compare two successive <tt class="docutils literal"><span class="pre">.nif</span></tt> files with <tt class="docutils literal"><span class="pre">diff</span></tt> to see what changed and why a module was rebuilt.</li>
|
|
</ul>
|
|
<p>Where to change behavior</p>
|
|
<ul class="simple"><li>Cache invalidation decisions and build-rule emission are implemented in <tt class="docutils literal"><span class="pre">compiler/deps.nim</span></tt>. When investigating surprising rebuilds, instrument those modules to log the footprint/hash/comparison outcome.</li>
|
|
</ul>
|
|
|
|
<h1 id="see-also">See also</h1><ul class="simple"><li><tt class="docutils literal"><span class="pre">nif-spec</span></tt> - NIF format specification (text format and node grammar): <a class="reference external" href="../nifspec/doc/nif-spec.md">nifspec/doc/nif-spec.md</a></li>
|
|
</ul>
|
|
|
|
|
|
|
|
<div class="twelve-columns footer">
|
|
<span class="nim-sprite"></span>
|
|
<br>
|
|
<small style="color: var(--hint);">Made with Nim. Generated: 2026-01-16 14:35:48 UTC</small>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<script defer data-domain="nim-lang.org" src="https://plausible.io/js/plausible.js"></script>
|
|
|
|
</body>
|
|
</html>
|