mirrors/neovim
1
0
Fork 0
mirror of https://github.com/neovim/neovim.git synced 2025-09-06 03:18:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: Treesitter (#13260)

Browse Source 
* doc & fixes: Generate treesitter docs

* fixup to treesitter-core

* docs(treesitter): fix docs for most functions

Co-authored-by: Thomas Vigouroux <tomvig38@gmail.com>
This commit is contained in:
TJ DeVries
2021-05-01 05:19:48 -07:00
committed by GitHub
parent ca6107cfbc
commit 27da5511a0
10 changed files with 775 additions and 298 deletions
Download Patch File Download Diff File Expand all files Collapse all files

563
runtime/doc/treesitter.txt
View File

@@ -102,14 +102,14 @@ tsnode:field({name}) *tsnode:field()*
tsnode:child_count() *tsnode:child_count()* tsnode:child_count() *tsnode:child_count()*
Get the node's number of children. Get the node's number of children.
tsnode:child({index}) *tsnode:child()* tsnode:child({index}) *tsnode:child()*
Get the node's child at the given {index}, where zero represents the Get the node's child at the given {index}, where zero represents the
first child. first child.
tsnode:named_child_count() *tsnode:named_child_count()* tsnode:named_child_count() *tsnode:named_child_count()*
Get the node's number of named children. Get the node's number of named children.
tsnode:named_child({index}) *tsnode:named_child()* tsnode:named_child({index}) *tsnode:named_child()*
Get the node's named child at the given {index}, where zero represents Get the node's named child at the given {index}, where zero represents
the first named child. the first named child.
@@ -146,7 +146,7 @@ tsnode:has_error() *tsnode:has_error()*
tsnode:sexpr() *tsnode:sexpr()* tsnode:sexpr() *tsnode:sexpr()*
Get an S-expression representing the node as a string. Get an S-expression representing the node as a string.
tsnode:id() *tsnode:id()* tsnode:id() *tsnode:id()*
Get an unique identier for the node inside its own tree. Get an unique identier for the node inside its own tree.
No guarantees are made about this identifer's internal representation, No guarantees are made about this identifer's internal representation,
@@ -156,16 +156,16 @@ tsnode:id() *tsnode:id()*
NB: the id is not guaranteed to be unique for nodes from different trees. NB: the id is not guaranteed to be unique for nodes from different trees.
tsnode:descendant_for_range({start_row}, {start_col}, {end_row}, {end_col}) tsnode:descendant_for_range({start_row}, {start_col}, {end_row}, {end_col})
*tsnode:descendant_for_range()* *tsnode:descendant_for_range()*
Get the smallest node within this node that spans the given range of Get the smallest node within this node that spans the given range of
(row, column) positions (row, column) positions
tsnode:named_descendant_for_range({start_row}, {start_col}, {end_row}, {end_col}) tsnode:named_descendant_for_range({start_row}, {start_col}, {end_row}, {end_col})
*tsnode:named_descendant_for_range()* *tsnode:named_descendant_for_range()*
Get the smallest named node within this node that spans the given Get the smallest named node within this node that spans the given
range of (row, column) positions range of (row, column) positions
Query methods *lua-treesitter-query* Query *lua-treesitter-query*
Tree-sitter queries are supported, with some limitations. Currently, the only Tree-sitter queries are supported, with some limitations. Currently, the only
supported match predicate is `eq?` (both comparing a capture against a string supported match predicate is `eq?` (both comparing a capture against a string
@@ -178,65 +178,6 @@ and predicates. A `capture` allows you to associate names with a specific
node in a pattern. A `predicate` adds arbitrary metadata and conditional data node in a pattern. A `predicate` adds arbitrary metadata and conditional data
to a match. to a match.
vim.treesitter.parse_query({lang}, {query})
*vim.treesitter.parse_query()*
Parse {query} as a string. (If the query is in a file, the caller
should read the contents into a string before calling).
Returns a `Query` (see |lua-treesitter-query|) object which can be used to
search nodes in the syntax tree for the patterns defined in {query}
using `iter_*` methods below. Exposes `info` and `captures` with
additional information about the {query}.
- `captures` contains the list of unique capture names defined in
{query}.
-` info.captures` also points to `captures`.
- `info.patterns` contains information about predicates.
query:iter_captures({node}, {bufnr}, {start_row}, {end_row})
*query:iter_captures()*
Iterate over all captures from all matches inside {node}.
{bufnr} is needed if the query contains predicates, then the caller
must ensure to use a freshly parsed tree consistent with the current
text of the buffer. {start_row} and {end_row} can be used to limit
matches inside a row range (this is typically used with root node
as the node, i e to get syntax highlight matches in the current
viewport). When omitted the start and end row values are used from
the given node.
The iterator returns three values, a numeric id identifying the capture,
the captured node, and metadata from any directives processing the match.
The following example shows how to get captures by name:
>
for id, node, metadata in query:iter_captures(tree:root(), bufnr, first, last) do
local name = query.captures[id] -- name of the capture in the query
-- typically useful info about the node:
local type = node:type() -- type of the captured node
local row1, col1, row2, col2 = node:range() -- range of the capture
... use the info here ...
end
<
query:iter_matches({node}, {bufnr}, {start_row}, {end_row})
*query:iter_matches()*
Iterate over all matches within a node. The arguments are the same as
for |query:iter_captures()| but the iterated values are different:
an (1-based) index of the pattern in the query, a table mapping
capture indices to nodes, and metadata from any directives processing the match.
If the query has more than one pattern the capture table might be sparse,
and e.g. `pairs()` method should be used over `ipairs`.
Here an example iterating over all captures in every match:
>
for pattern, match, metadata in cquery:iter_matches(tree:root(), bufnr, first, last) do
for id, node in pairs(match) do
local name = query.captures[id]
-- `node` was captured by the `name` capture in the match
local node_data = metadata[id] -- Node level metadata
... use the info here ...
end
end
Treesitter Query Predicates *lua-treesitter-predicates* Treesitter Query Predicates *lua-treesitter-predicates*
When writing queries for treesitter, one might use `predicates`, that is, When writing queries for treesitter, one might use `predicates`, that is,
@@ -298,28 +239,6 @@ Here is a list of built-in directives:
`({capture_id}, {start_row}, {start_col}, {end_row}, {end_col}, {key?})` `({capture_id}, {start_row}, {start_col}, {end_row}, {end_col}, {key?})`
The default key is "offset". The default key is "offset".
*vim.treesitter.query.add_predicate()*
vim.treesitter.query.add_predicate({name}, {handler})
This adds a predicate with the name {name} to be used in queries.
{handler} should be a function whose signature will be : >
handler(match, pattern, bufnr, predicate)
<
*vim.treesitter.query.list_predicates()*
vim.treesitter.query.list_predicates()
This lists the currently available predicates to use in queries.
*vim.treesitter.query.add_directive()*
vim.treesitter.query.add_directive({name}, {handler})
This adds a directive with the name {name} to be used in queries.
{handler} should be a function whose signature will be : >
handler(match, pattern, bufnr, predicate, metadata)
Handlers can set match level data by setting directly on the metadata object `metadata.key = value`
Handlers can set node level data by using the capture id on the metadata table
`metadata[capture_id].key = value`
Treesitter syntax highlighting (WIP) *lua-treesitter-highlight* Treesitter syntax highlighting (WIP) *lua-treesitter-highlight*
NOTE: This is a partially implemented feature, and not usable as a default NOTE: This is a partially implemented feature, and not usable as a default
@@ -364,92 +283,434 @@ identical identifiers, highlighting both as |hl-WarningMsg|: >
((binary_expression left: (identifier) @WarningMsg.left right: (identifier) @WarningMsg.right) ((binary_expression left: (identifier) @WarningMsg.left right: (identifier) @WarningMsg.right)
(eq? @WarningMsg.left @WarningMsg.right)) (eq? @WarningMsg.left @WarningMsg.right))
<
Treesitter language injection (WIP) *lua-treesitter-language-injection* ==============================================================================
Lua module: vim.treesitter *lua-treesitter-core*
NOTE: This is a partially implemented feature, and not usable as a default get_parser({bufnr}, {lang}, {opts}) *get_parser()*
solution yet. What is documented here is a temporary interface intended Gets the parser for this bufnr / ft combination.
for those who want to experiment with this feature and contribute to
its development.
Languages can have nested languages within them, for example javascript inside If needed this will create the parser. Unconditionnally attach
HTML. We can "inject" a treesitter parser for a child language by configuring the provided callback
injection queries. Here is an example of Javascript and CSS injected into
HTML. >
local query = [[ Parameters: ~
(script_element (raw_text) @javascript) {bufnr} The buffer the parser should be tied to
(style_element (raw_text) @css) {lang} The filetype of this parser
]]; {opts} Options object to pass to the created language
tree
local parser = vim.treesitter.get_parser(nil, nil, { Return: ~
injections = {html = query} The parser
})
parser:parse() get_string_parser({str}, {lang}, {opts}) *get_string_parser()*
Gets a string parser
Any capture will be treated as the node treesitter will use for the injected Parameters: ~
language. The capture name will be used as the language. There are a couple {str} The string to parse
reserved captures that do not have this behavior {lang} The language of this string
{opts} Options to pass to the created language tree
`@language`
This will use a nodes text content as the language to be injected.
`@content` ==============================================================================
This will use the captured nodes content as the injected content. Lua module: vim.treesitter.language *treesitter-language*
`@combined` inspect_language({lang}) *inspect_language()*
This will combine all matches of a pattern as one single block of content. Inspects the provided language.
By default, each match of a pattern is treated as it's own block of content
and parsed independent of each other.
`@<language>` Inspecting provides some useful informations on the language
Any other capture name will be treated as both the language and the content. like node names, ...
`@_<name>` Parameters: ~
Any capture with a leading "_" will not be treated as a language and will have {lang} The language.
no special processing and is useful for capturing nodes for directives.
Injections can be configured using `directives` instead of using capture require_language({lang}, {path}, {silent}) *require_language()*
names. Here is an example of a directive that resolves the language based on a Asserts that the provided language is installed, and
buffer variable instead of statically in the query. > optionally provide a path for the parser
local query = require("vim.treesitter.query") Parsers are searched in the `parser` runtime directory.
query.add_directive("inject-preprocessor!", function(_, bufnr, _, _, data) Parameters: ~
local success, lang = pcall(vim.api.nvim_buf_get_var, bufnr, "css_preprocessor") {lang} The language the parser should parse
{path} Optional path the parser is located at
{silent} Don't throw an error if language not found
data.language = success and lang or "css"
end)
Here is the same HTML query using this directive. > ==============================================================================
Lua module: vim.treesitter.query *treesitter-query*
local query = [[ add_directive({name}, {handler}, {force}) *add_directive()*
(script_element (raw_text) @javascript) Adds a new directive to be used in queries
(style_element
((raw_text) @content
(#inject-preprocessor!)))
]];
local parser = vim.treesitter.get_parser(nil, nil, { Parameters: ~
injections = {html = query} {name} the name of the directive, without leading #
}) {handler} the handler function to be used signature will
be (match, pattern, bufnr, predicate)
parser:parse() add_predicate({name}, {handler}, {force}) *add_predicate()*
Adds a new predicate to be used in queries
The following properties can be attached to the metadata object provided to Parameters: ~
the directive. {name} the name of the predicate, without leading #
{handler} the handler function to be used signature will
be (match, pattern, bufnr, predicate)
`language` get_node_text({node}, {source}) *get_node_text()*
Same as the language capture. Gets the text corresponding to a given node
`content` Parameters: ~
A list of ranges or nodes to inject as content. These ranges and/or nodes will {node} the node
be treated as combined source and will be parsed within the same context. This {bsource} The buffer or string from which the node is
differs from the `@content` capture which only captures a single node as extracted
content. This can also be a single number that references a captured node.
`combined` get_query({lang}, {query_name}) *get_query()*
Same as the combined capture. Returns the runtime query {query_name} for {lang}.
Parameters: ~
{lang} The language to use for the query
{query_name} The name of the query (i.e. "highlights")
Return: ~
The corresponding query, parsed.
*get_query_files()*
get_query_files({lang}, {query_name}, {is_included})
Gets the list of files used to make up a query
Parameters: ~
{lang} The language
{query_name} The name of the query to load
{is_included} Internal parameter, most of the time left
as `nil`
list_predicates() *list_predicates()*
TODO: Documentation
parse_query({lang}, {query}) *parse_query()*
Parse {query} as a string. (If the query is in a file, the
caller should read the contents into a string before calling).
Returns a `Query` (see |lua-treesitter-query|) object which
can be used to search nodes in the syntax tree for the
patterns defined in {query} using `iter_*` methods below.
Exposes `info` and `captures` with additional information about the {query}.
• `captures` contains the list of unique capture names defined
in {query}. - `info.captures` also points to `captures` .
• `info.patterns` contains information about predicates.
Parameters: ~
{lang} The language
{query} A string containing the query (s-expr syntax)
Return: ~
The query
*Query:iter_captures()*
Query:iter_captures({self}, {node}, {source}, {start}, {stop})
Iterate over all captures from all matches inside {node}
{source} is needed if the query contains predicates, then the
caller must ensure to use a freshly parsed tree consistent
with the current text of the buffer (if relevent). {start_row}
and {end_row} can be used to limit matches inside a row range
(this is typically used with root node as the node, i e to get
syntax highlight matches in the current viewport). When
omitted the start and end row values are used from the given
node.
The iterator returns three values, a numeric id identifying
the capture, the captured node, and metadata from any
directives processing the match. The following example shows
how to get captures by name:
>
for id, node, metadata in query:iter_captures(tree:root(), bufnr, first, last) do
local name = query.captures[id] -- name of the capture in the query
-- typically useful info about the node:
local type = node:type() -- type of the captured node
local row1, col1, row2, col2 = node:range() -- range of the capture
... use the info here ...
end
<
Parameters: ~
{node} The node under which the search will occur
{source} The source buffer or string to exctract text
from
{start} The starting line of the search
{stop} The stopping line of the search (end-exclusive)
{self}
Return: ~
The matching capture id
The captured node
*Query:iter_matches()*
Query:iter_matches({self}, {node}, {source}, {start}, {stop})
Iterates the matches of self on a given range.
Iterate over all matches within a node. The arguments are the
same as for |query:iter_captures()| but the iterated values
are different: an (1-based) index of the pattern in the query,
a table mapping capture indices to nodes, and metadata from
any directives processing the match. If the query has more
than one pattern the capture table might be sparse, and e.g.
`pairs()` method should be used over `ipairs` . Here an
example iterating over all captures in every match:
>
for pattern, match, metadata in cquery:iter_matches(tree:root(), bufnr, first, last) do
for id, node in pairs(match) do
local name = query.captures[id]
-- `node` was captured by the `name` capture in the match
local node_data = metadata[id] -- Node level metadata
... use the info here ...
end
end
<
Parameters: ~
{node} The node under which the search will occur
{source} The source buffer or string to search
{start} The starting line of the search
{stop} The stopping line of the search (end-exclusive)
{self}
Return: ~
The matching pattern id
The matching match
set_query({lang}, {query_name}, {text}) *set_query()*
Sets the runtime query {query_name} for {lang}
This allows users to override any runtime files and/or
configuration set by plugins.
Parameters: ~
{lang} string: The language to use for the query
{query_name} string: The name of the query (i.e.
"highlights")
{text} string: The query text (unparsed).
==============================================================================
Lua module: vim.treesitter.highlighter *treesitter-highlighter*
new({tree}, {opts}) *highlighter.new()*
Creates a new highlighter using
Parameters: ~
{tree} The language tree to use for highlighting
{opts} Table used to configure the highlighter
• queries: Table to overwrite queries used by the
highlighter
TSHighlighter:destroy({self}) *TSHighlighter:destroy()*
Removes all internal references to the highlighter
Parameters: ~
{self}
TSHighlighter:get_query({self}, {lang}) *TSHighlighter:get_query()*
Gets the query used for
Parameters: ~
{lang} A language used by the highlighter.
{self}
==============================================================================
Lua module: vim.treesitter.languagetree *treesitter-languagetree*
LanguageTree:add_child({self}, {lang}) *LanguageTree:add_child()*
Adds a child language to this tree.
If the language already exists as a child, it will first be
removed.
Parameters: ~
{lang} The language to add.
{self}
LanguageTree:children({self}) *LanguageTree:children()*
Returns a map of language to child tree.
Parameters: ~
{self}
LanguageTree:contains({self}, {range}) *LanguageTree:contains()*
Determines wether This goes down the tree to recursively check childs.
Parameters: ~
{range} is contained in this language tree
Parameters: ~
{range} A range, that is a `{ start_line, start_col,
end_line, end_col }` table.
{self}
LanguageTree:destroy({self}) *LanguageTree:destroy()*
Destroys this language tree and all its children.
Any cleanup logic should be performed here. Note, this DOES
NOT remove this tree from a parent. `remove_child` must be called on the parent to remove it.
Parameters: ~
{self}
*LanguageTree:for_each_child()*
LanguageTree:for_each_child({self}, {fn}, {include_self})
Invokes the callback for each LanguageTree and it's children
recursively
Parameters: ~
{fn} The function to invoke. This is invoked
with arguments (tree: LanguageTree, lang:
string)
{include_self} Whether to include the invoking tree in
the results.
{self}
LanguageTree:for_each_tree({self}, {fn}) *LanguageTree:for_each_tree()*
Invokes the callback for each treesitter trees recursively.
Note, this includes the invoking language tree's trees as
well.
Parameters: ~
{fn} The callback to invoke. The callback is invoked
with arguments (tree: TSTree, languageTree:
LanguageTree)
{self}
LanguageTree:included_regions({self}) *LanguageTree:included_regions()*
Gets the set of included regions
Parameters: ~
{self}
LanguageTree:invalidate({self}, {reload}) *LanguageTree:invalidate()*
Invalidates this parser and all its children
Parameters: ~
{self}
LanguageTree:is_valid({self}) *LanguageTree:is_valid()*
Determines whether this tree is valid. If the tree is invalid, `parse()` must be called to get the an updated tree.
Parameters: ~
{self}
LanguageTree:lang({self}) *LanguageTree:lang()*
Gets the language of this tree node.
Parameters: ~
{self}
*LanguageTree:language_for_range()*
LanguageTree:language_for_range({self}, {range})
Gets the appropriate language that contains
Parameters: ~
{range} A text range, see |LanguageTree:contains|
{self}
LanguageTree:parse({self}) *LanguageTree:parse()*
Parses all defined regions using a treesitter parser for the
language this tree represents. This will run the injection
query for this language to determine if any child languages
should be created.
Parameters: ~
{self}
LanguageTree:register_cbs({self}, {cbs}) *LanguageTree:register_cbs()*
Registers callbacks for the parser
Parameters: ~
{cbs} An `nvim_buf_attach` -like table argument with the following keys : `on_bytes` : see `nvim_buf_attach` , but this will be called after the parsers callback. `on_changedtree` : a callback that will be called every time the
tree has syntactical changes. it will only be
passed one argument, that is a table of the ranges
(as node ranges) that changed. `on_child_added` : emitted when a child is added to the tree. `on_child_removed` : emitted when a child is removed from the tree.
{self}
LanguageTree:remove_child({self}, {lang}) *LanguageTree:remove_child()*
Removes a child language from this tree.
Parameters: ~
{lang} The language to remove.
{self}
*LanguageTree:set_included_regions()*
LanguageTree:set_included_regions({self}, {regions})
Sets the included regions that should be parsed by this
parser. A region is a set of nodes and/or ranges that will be
parsed in the same context.
For example, `{ { node1 }, { node2} }` is two separate
regions. This will be parsed by the parser in two different
contexts... thus resulting in two separate trees.
`{ { node1, node2 } }` is a single region consisting of two
nodes. This will be parsed by the parser in a single
context... thus resulting in a single tree.
This allows for embedded languages to be parsed together
across different nodes, which is useful for templating
languages like ERB and EJS.
Note, this call invalidates the tree and requires it to be
parsed again.
Parameters: ~
{regions} A list of regions this tree should manage and
parse.
{self}
LanguageTree:source({self}) *LanguageTree:source()*
Returns the source content of the language tree (bufnr or
string).
Parameters: ~
{self}
LanguageTree:trees({self}) *LanguageTree:trees()*
Returns all trees this language tree contains. Does not
include child languages.
Parameters: ~
{self}
new({source}, {lang}, {opts}) *languagetree.new()*
Represents a single treesitter parser for a language. The
language can contain child languages with in its range, hence
the tree.
Parameters: ~
{source} Can be a bufnr or a string of text to
parse
{lang} The language this tree represents
{opts} Options table
{opts.injections} A table of language to injection query
strings. This is useful for overriding
the built-in runtime file searching for
the injection language query per
language.
==============================================================================
Lua module: vim.treesitter.health *treesitter-health*
check_health() *check_health()*
TODO: Documentation
list_parsers() *list_parsers()*
Lists the parsers currently installed
Return: ~
A list of parsers
vim:tw=78:ts=8:ft=help:norl: vim:tw=78:ts=8:ft=help:norl:

38
runtime/lua/vim/treesitter.lua
View File

@@ -25,12 +25,12 @@ setmetatable(M, {
}) })
--- Creates a new parser. --- Creates a new parser.
-- ---
-- It is not recommended to use this, use vim.treesitter.get_parser() instead. --- It is not recommended to use this, use vim.treesitter.get_parser() instead.
-- ---
-- @param bufnr The buffer the parser will be tied to --- @param bufnr The buffer the parser will be tied to
-- @param lang The language of the parser --- @param lang The language of the parser
-- @param opts Options to pass to the language tree --- @param opts Options to pass to the created language tree
function M._create_parser(bufnr, lang, opts) function M._create_parser(bufnr, lang, opts)
language.require_language(lang) language.require_language(lang)
if bufnr == 0 then if bufnr == 0 then
@@ -41,10 +41,12 @@ function M._create_parser(bufnr, lang, opts)
local self = LanguageTree.new(bufnr, lang, opts) local self = LanguageTree.new(bufnr, lang, opts)
---@private
local function bytes_cb(_, ...) local function bytes_cb(_, ...)
self:_on_bytes(...) self:_on_bytes(...)
end end
---@private
local function detach_cb(_, ...) local function detach_cb(_, ...)
if parsers[bufnr] == self then if parsers[bufnr] == self then
parsers[bufnr] = nil parsers[bufnr] = nil
@@ -52,6 +54,7 @@ function M._create_parser(bufnr, lang, opts)
self:_on_detach(...) self:_on_detach(...)
end end
---@private
local function reload_cb(_, ...) local function reload_cb(_, ...)
self:_on_reload(...) self:_on_reload(...)
end end
@@ -64,15 +67,15 @@ function M._create_parser(bufnr, lang, opts)
end end
--- Gets the parser for this bufnr / ft combination. --- Gets the parser for this bufnr / ft combination.
-- ---
-- If needed this will create the parser. --- If needed this will create the parser.
-- Unconditionnally attach the provided callback --- Unconditionnally attach the provided callback
-- ---
-- @param bufnr The buffer the parser should be tied to --- @param bufnr The buffer the parser should be tied to
-- @param ft The filetype of this parser --- @param lang The filetype of this parser
-- @param opts Options object to pass to the parser --- @param opts Options object to pass to the created language tree
-- ---
-- @returns The parser --- @returns The parser
function M.get_parser(bufnr, lang, opts) function M.get_parser(bufnr, lang, opts)
opts = opts or {} opts = opts or {}
@@ -92,6 +95,11 @@ function M.get_parser(bufnr, lang, opts)
return parsers[bufnr] return parsers[bufnr]
end end
--- Gets a string parser
---
--- @param str The string to parse
--- @param lang The language of this string
--- @param opts Options to pass to the created language tree
function M.get_string_parser(str, lang, opts) function M.get_string_parser(str, lang, opts)
vim.validate { vim.validate {
str = { str, 'string' }, str = { str, 'string' },

4
runtime/lua/vim/treesitter/health.lua
View File

@@ -1,10 +1,14 @@
local M = {} local M = {}
local ts = vim.treesitter local ts = vim.treesitter
--- Lists the parsers currently installed
---
---@return A list of parsers
function M.list_parsers() function M.list_parsers()
return vim.api.nvim_get_runtime_file('parser/*', true) return vim.api.nvim_get_runtime_file('parser/*', true)
end end
--- Performs a healthcheck for treesitter integration
function M.check_health() function M.check_health()
local report_info = vim.fn['health#report_info'] local report_info = vim.fn['health#report_info']
local report_ok = vim.fn['health#report_ok'] local report_ok = vim.fn['health#report_ok']

22
runtime/lua/vim/treesitter/highlighter.lua
View File

@@ -70,11 +70,13 @@ TSHighlighter.hl_map = {
["include"] = "Include", ["include"] = "Include",
} }
---@private
local function is_highlight_name(capture_name) local function is_highlight_name(capture_name)
local firstc = string.sub(capture_name, 1, 1) local firstc = string.sub(capture_name, 1, 1)
return firstc ~= string.lower(firstc) return firstc ~= string.lower(firstc)
end end
---@private
function TSHighlighterQuery.new(lang, query_string) function TSHighlighterQuery.new(lang, query_string)
local self = setmetatable({}, { __index = TSHighlighterQuery }) local self = setmetatable({}, { __index = TSHighlighterQuery })
@@ -99,10 +101,12 @@ function TSHighlighterQuery.new(lang, query_string)
return self return self
end end
---@private
function TSHighlighterQuery:query() function TSHighlighterQuery:query()
return self._query return self._query
end end
---@private
--- Get the hl from capture. --- Get the hl from capture.
--- Returns a tuple { highlight_name: string, is_builtin: bool } --- Returns a tuple { highlight_name: string, is_builtin: bool }
function TSHighlighterQuery:_get_hl_from_capture(capture) function TSHighlighterQuery:_get_hl_from_capture(capture)
@@ -116,6 +120,11 @@ function TSHighlighterQuery:_get_hl_from_capture(capture)
end end
end end
--- Creates a new highlighter using @param tree
---
--- @param tree The language tree to use for highlighting
--- @param opts Table used to configure the highlighter
--- - queries: Table to overwrite queries used by the highlighter
function TSHighlighter.new(tree, opts) function TSHighlighter.new(tree, opts)
local self = setmetatable({}, TSHighlighter) local self = setmetatable({}, TSHighlighter)
@@ -165,12 +174,14 @@ function TSHighlighter.new(tree, opts)
return self return self
end end
--- Removes all internal references to the highlighter
function TSHighlighter:destroy() function TSHighlighter:destroy()
if TSHighlighter.active[self.bufnr] then if TSHighlighter.active[self.bufnr] then
TSHighlighter.active[self.bufnr] = nil TSHighlighter.active[self.bufnr] = nil
end end
end end
---@private
function TSHighlighter:get_highlight_state(tstree) function TSHighlighter:get_highlight_state(tstree)
if not self._highlight_states[tstree] then if not self._highlight_states[tstree] then
self._highlight_states[tstree] = { self._highlight_states[tstree] = {
@@ -182,24 +193,31 @@ function TSHighlighter:get_highlight_state(tstree)
return self._highlight_states[tstree] return self._highlight_states[tstree]
end end
---@private
function TSHighlighter:reset_highlight_state() function TSHighlighter:reset_highlight_state()
self._highlight_states = {} self._highlight_states = {}
end end
---@private
function TSHighlighter:on_bytes(_, _, start_row, _, _, _, _, _, new_end) function TSHighlighter:on_bytes(_, _, start_row, _, _, _, _, _, new_end)
a.nvim__buf_redraw_range(self.bufnr, start_row, start_row + new_end + 1) a.nvim__buf_redraw_range(self.bufnr, start_row, start_row + new_end + 1)
end end
---@private
function TSHighlighter:on_detach() function TSHighlighter:on_detach()
self:destroy() self:destroy()
end end
---@private
function TSHighlighter:on_changedtree(changes) function TSHighlighter:on_changedtree(changes)
for _, ch in ipairs(changes or {}) do for _, ch in ipairs(changes or {}) do
a.nvim__buf_redraw_range(self.bufnr, ch[1], ch[3]+1) a.nvim__buf_redraw_range(self.bufnr, ch[1], ch[3]+1)
end end
end end
--- Gets the query used for @param lang
---
--- @param lang A language used by the highlighter.
function TSHighlighter:get_query(lang) function TSHighlighter:get_query(lang)
if not self._queries[lang] then if not self._queries[lang] then
self._queries[lang] = TSHighlighterQuery.new(lang) self._queries[lang] = TSHighlighterQuery.new(lang)
@@ -208,6 +226,7 @@ function TSHighlighter:get_query(lang)
return self._queries[lang] return self._queries[lang]
end end
---@private
local function on_line_impl(self, buf, line) local function on_line_impl(self, buf, line)
self.tree:for_each_tree(function(tstree, tree) self.tree:for_each_tree(function(tstree, tree)
if not tstree then return end if not tstree then return end
@@ -251,6 +270,7 @@ local function on_line_impl(self, buf, line)
end, true) end, true)
end end
---@private
function TSHighlighter._on_line(_, _win, buf, line, _) function TSHighlighter._on_line(_, _win, buf, line, _)
local self = TSHighlighter.active[buf] local self = TSHighlighter.active[buf]
if not self then return end if not self then return end
@@ -258,6 +278,7 @@ function TSHighlighter._on_line(_, _win, buf, line, _)
on_line_impl(self, buf, line) on_line_impl(self, buf, line)
end end
---@private
function TSHighlighter._on_buf(_, buf) function TSHighlighter._on_buf(_, buf)
local self = TSHighlighter.active[buf] local self = TSHighlighter.active[buf]
if self then if self then
@@ -265,6 +286,7 @@ function TSHighlighter._on_buf(_, buf)
end end
end end
---@private
function TSHighlighter._on_win(_, _win, buf, _topline) function TSHighlighter._on_win(_, _win, buf, _topline)
local self = TSHighlighter.active[buf] local self = TSHighlighter.active[buf]
if not self then if not self then

20
runtime/lua/vim/treesitter/language.lua
View File

@@ -3,12 +3,12 @@ local a = vim.api
local M = {} local M = {}
--- Asserts that the provided language is installed, and optionally provide a path for the parser --- Asserts that the provided language is installed, and optionally provide a path for the parser
-- ---
-- Parsers are searched in the `parser` runtime directory. --- Parsers are searched in the `parser` runtime directory.
-- ---
-- @param lang The language the parser should parse --- @param lang The language the parser should parse
-- @param path Optional path the parser is located at --- @param path Optional path the parser is located at
-- @param silent Don't throw an error if language not found --- @param silent Don't throw an error if language not found
function M.require_language(lang, path, silent) function M.require_language(lang, path, silent)
if vim._ts_has_language(lang) then if vim._ts_has_language(lang) then
return true return true
@@ -37,10 +37,10 @@ function M.require_language(lang, path, silent)
end end
--- Inspects the provided language. --- Inspects the provided language.
-- ---
-- Inspecting provides some useful informations on the language like node names, ... --- Inspecting provides some useful informations on the language like node names, ...
-- ---
-- @param lang The language. --- @param lang The language.
function M.inspect_language(lang) function M.inspect_language(lang)
M.require_language(lang) M.require_language(lang)
return vim._ts_inspect_language(lang) return vim._ts_inspect_language(lang)

160
runtime/lua/vim/treesitter/languagetree.lua
View File

@@ -5,16 +5,16 @@ local language = require'vim.treesitter.language'
local LanguageTree = {} local LanguageTree = {}
LanguageTree.__index = LanguageTree LanguageTree.__index = LanguageTree
-- Represents a single treesitter parser for a language. --- Represents a single treesitter parser for a language.
-- The language can contain child languages with in its range, --- The language can contain child languages with in its range,
-- hence the tree. --- hence the tree.
-- ---
-- @param source Can be a bufnr or a string of text to parse --- @param source Can be a bufnr or a string of text to parse
-- @param lang The language this tree represents --- @param lang The language this tree represents
-- @param opts Options table --- @param opts Options table
-- @param opts.injections A table of language to injection query strings. --- @param opts.injections A table of language to injection query strings.
-- This is useful for overriding the built-in runtime file --- This is useful for overriding the built-in runtime file
-- searching for the injection language query per language. --- searching for the injection language query per language.
function LanguageTree.new(source, lang, opts) function LanguageTree.new(source, lang, opts)
language.require_language(lang) language.require_language(lang)
opts = opts or {} opts = opts or {}
@@ -50,7 +50,7 @@ function LanguageTree.new(source, lang, opts)
return self return self
end end
-- Invalidates this parser and all its children --- Invalidates this parser and all its children
function LanguageTree:invalidate(reload) function LanguageTree:invalidate(reload)
self._valid = false self._valid = false
@@ -64,38 +64,38 @@ function LanguageTree:invalidate(reload)
end end
end end
-- Returns all trees this language tree contains. --- Returns all trees this language tree contains.
-- Does not include child languages. --- Does not include child languages.
function LanguageTree:trees() function LanguageTree:trees()
return self._trees return self._trees
end end
-- Gets the language of this tree layer. --- Gets the language of this tree node.
function LanguageTree:lang() function LanguageTree:lang()
return self._lang return self._lang
end end
-- Determines whether this tree is valid. --- Determines whether this tree is valid.
-- If the tree is invalid, `parse()` must be called --- If the tree is invalid, `parse()` must be called
-- to get the an updated tree. --- to get the an updated tree.
function LanguageTree:is_valid() function LanguageTree:is_valid()
return self._valid return self._valid
end end
-- Returns a map of language to child tree. --- Returns a map of language to child tree.
function LanguageTree:children() function LanguageTree:children()
return self._children return self._children
end end
-- Returns the source content of the language tree (bufnr or string). --- Returns the source content of the language tree (bufnr or string).
function LanguageTree:source() function LanguageTree:source()
return self._source return self._source
end end
-- Parses all defined regions using a treesitter parser --- Parses all defined regions using a treesitter parser
-- for the language this tree represents. --- for the language this tree represents.
-- This will run the injection query for this language to --- This will run the injection query for this language to
-- determine if any child languages should be created. --- determine if any child languages should be created.
function LanguageTree:parse() function LanguageTree:parse()
if self._valid then if self._valid then
return self._trees return self._trees
@@ -169,9 +169,10 @@ function LanguageTree:parse()
return self._trees, changes return self._trees, changes
end end
-- Invokes the callback for each LanguageTree and it's children recursively --- Invokes the callback for each LanguageTree and it's children recursively
-- @param fn The function to invoke. This is invoked with arguments (tree: LanguageTree, lang: string) ---
-- @param include_self Whether to include the invoking tree in the results. --- @param fn The function to invoke. This is invoked with arguments (tree: LanguageTree, lang: string)
--- @param include_self Whether to include the invoking tree in the results.
function LanguageTree:for_each_child(fn, include_self) function LanguageTree:for_each_child(fn, include_self)
if include_self then if include_self then
fn(self, self._lang) fn(self, self._lang)
@@ -182,10 +183,12 @@ function LanguageTree:for_each_child(fn, include_self)
end end
end end
-- Invokes the callback for each treesitter trees recursively. --- Invokes the callback for each treesitter trees recursively.
-- Note, this includes the invoking language tree's trees as well. ---
-- @param fn The callback to invoke. The callback is invoked with arguments --- Note, this includes the invoking language tree's trees as well.
-- (tree: TSTree, languageTree: LanguageTree) ---
--- @param fn The callback to invoke. The callback is invoked with arguments
--- (tree: TSTree, languageTree: LanguageTree)
function LanguageTree:for_each_tree(fn) function LanguageTree:for_each_tree(fn)
for _, tree in ipairs(self._trees) do for _, tree in ipairs(self._trees) do
fn(tree, self) fn(tree, self)
@@ -196,9 +199,11 @@ function LanguageTree:for_each_tree(fn)
end end
end end
-- Adds a child language to this tree. --- Adds a child language to this tree.
-- If the language already exists as a child, it will first be removed. ---
-- @param lang The language to add. --- If the language already exists as a child, it will first be removed.
---
--- @param lang The language to add.
function LanguageTree:add_child(lang) function LanguageTree:add_child(lang)
if self._children[lang] then if self._children[lang] then
self:remove_child(lang) self:remove_child(lang)
@@ -212,8 +217,9 @@ function LanguageTree:add_child(lang)
return self._children[lang] return self._children[lang]
end end
-- Removes a child language from this tree. --- Removes a child language from this tree.
-- @param lang The language to remove. ---
--- @param lang The language to remove.
function LanguageTree:remove_child(lang) function LanguageTree:remove_child(lang)
local child = self._children[lang] local child = self._children[lang]
@@ -225,10 +231,11 @@ function LanguageTree:remove_child(lang)
end end
end end
-- Destroys this language tree and all its children. --- Destroys this language tree and all its children.
-- Any cleanup logic should be performed here. ---
-- Note, this DOES NOT remove this tree from a parent. --- Any cleanup logic should be performed here.
-- `remove_child` must be called on the parent to remove it. --- Note, this DOES NOT remove this tree from a parent.
--- `remove_child` must be called on the parent to remove it.
function LanguageTree:destroy() function LanguageTree:destroy()
-- Cleanup here -- Cleanup here
for _, child in ipairs(self._children) do for _, child in ipairs(self._children) do
@@ -236,23 +243,23 @@ function LanguageTree:destroy()
end end
end end
-- Sets the included regions that should be parsed by this parser. --- Sets the included regions that should be parsed by this parser.
-- A region is a set of nodes and/or ranges that will be parsed in the same context. --- A region is a set of nodes and/or ranges that will be parsed in the same context.
-- ---
-- For example, `{ { node1 }, { node2} }` is two separate regions. --- For example, `{ { node1 }, { node2} }` is two separate regions.
-- This will be parsed by the parser in two different contexts... thus resulting --- This will be parsed by the parser in two different contexts... thus resulting
-- in two separate trees. --- in two separate trees.
-- ---
-- `{ { node1, node2 } }` is a single region consisting of two nodes. --- `{ { node1, node2 } }` is a single region consisting of two nodes.
-- This will be parsed by the parser in a single context... thus resulting --- This will be parsed by the parser in a single context... thus resulting
-- in a single tree. --- in a single tree.
-- ---
-- This allows for embedded languages to be parsed together across different --- This allows for embedded languages to be parsed together across different
-- nodes, which is useful for templating languages like ERB and EJS. --- nodes, which is useful for templating languages like ERB and EJS.
-- ---
-- Note, this call invalidates the tree and requires it to be parsed again. --- Note, this call invalidates the tree and requires it to be parsed again.
-- ---
-- @param regions A list of regions this tree should manage and parse. --- @param regions A list of regions this tree should manage and parse.
function LanguageTree:set_included_regions(regions) function LanguageTree:set_included_regions(regions)
-- TODO(vigoux): I don't think string parsers are useful for now -- TODO(vigoux): I don't think string parsers are useful for now
if type(self._source) == "number" then if type(self._source) == "number" then
@@ -281,16 +288,18 @@ function LanguageTree:set_included_regions(regions)
self:invalidate() self:invalidate()
end end
-- Gets the set of included regions --- Gets the set of included regions
function LanguageTree:included_regions() function LanguageTree:included_regions()
return self._regions return self._regions
end end
-- Gets language injection points by language. --- Gets language injection points by language.
-- This is where most of the injection processing occurs. ---
-- TODO: Allow for an offset predicate to tailor the injection range --- This is where most of the injection processing occurs.
-- instead of using the entire nodes range. ---
-- @private --- TODO: Allow for an offset predicate to tailor the injection range
--- instead of using the entire nodes range.
--- @private
function LanguageTree:_get_injections() function LanguageTree:_get_injections()
if not self._injection_query then return {} end if not self._injection_query then return {} end
@@ -395,12 +404,14 @@ function LanguageTree:_get_injections()
return result return result
end end
---@private
function LanguageTree:_do_callback(cb_name, ...) function LanguageTree:_do_callback(cb_name, ...)
for _, cb in ipairs(self._callbacks[cb_name]) do for _, cb in ipairs(self._callbacks[cb_name]) do
cb(...) cb(...)
end end
end end
---@private
function LanguageTree:_on_bytes(bufnr, changed_tick, function LanguageTree:_on_bytes(bufnr, changed_tick,
start_row, start_col, start_byte, start_row, start_col, start_byte,
old_row, old_col, old_byte, old_row, old_col, old_byte,
@@ -425,24 +436,26 @@ function LanguageTree:_on_bytes(bufnr, changed_tick,
new_row, new_col, new_byte) new_row, new_col, new_byte)
end end
---@private
function LanguageTree:_on_reload() function LanguageTree:_on_reload()
self:invalidate(true) self:invalidate(true)
end end
---@private
function LanguageTree:_on_detach(...) function LanguageTree:_on_detach(...)
self:invalidate(true) self:invalidate(true)
self:_do_callback('detach', ...) self:_do_callback('detach', ...)
end end
--- Registers callbacks for the parser --- Registers callbacks for the parser
-- @param cbs An `nvim_buf_attach`-like table argument with the following keys : --- @param cbs An `nvim_buf_attach`-like table argument with the following keys :
-- `on_bytes` : see `nvim_buf_attach`, but this will be called _after_ the parsers callback. --- `on_bytes` : see `nvim_buf_attach`, but this will be called _after_ the parsers callback.
-- `on_changedtree` : a callback that will be called every time the tree has syntactical changes. --- `on_changedtree` : a callback that will be called every time the tree has syntactical changes.
-- it will only be passed one argument, that is a table of the ranges (as node ranges) that --- it will only be passed one argument, that is a table of the ranges (as node ranges) that
-- changed. --- changed.
-- `on_child_added` : emitted when a child is added to the tree. --- `on_child_added` : emitted when a child is added to the tree.
-- `on_child_removed` : emitted when a child is removed from the tree. --- `on_child_removed` : emitted when a child is removed from the tree.
function LanguageTree:register_cbs(cbs) function LanguageTree:register_cbs(cbs)
if not cbs then return end if not cbs then return end
@@ -467,6 +480,7 @@ function LanguageTree:register_cbs(cbs)
end end
end end
---@private
local function tree_contains(tree, range) local function tree_contains(tree, range)
local start_row, start_col, end_row, end_col = tree:root():range() local start_row, start_col, end_row, end_col = tree:root():range()
local start_fits = start_row < range[1] or (start_row == range[1] and start_col <= range[2]) local start_fits = start_row < range[1] or (start_row == range[1] and start_col <= range[2])
@@ -479,6 +493,11 @@ local function tree_contains(tree, range)
return false return false
end end
--- Determines wether @param range is contained in this language tree
---
--- This goes down the tree to recursively check childs.
---
--- @param range A range, that is a `{ start_line, start_col, end_line, end_col }` table.
function LanguageTree:contains(range) function LanguageTree:contains(range)
for _, tree in pairs(self._trees) do for _, tree in pairs(self._trees) do
if tree_contains(tree, range) then if tree_contains(tree, range) then
@@ -489,6 +508,9 @@ function LanguageTree:contains(range)
return false return false
end end
--- Gets the appropriate language that contains @param range
---
--- @param range A text range, see |LanguageTree:contains|
function LanguageTree:language_for_range(range) function LanguageTree:language_for_range(range)
for _, child in pairs(self._children) do for _, child in pairs(self._children) do
if child:contains(range) then if child:contains(range) then

153
runtime/lua/vim/treesitter/query.lua
View File

@@ -8,6 +8,7 @@ Query.__index = Query
local M = {} local M = {}
---@private
local function dedupe_files(files) local function dedupe_files(files)
local result = {} local result = {}
local seen = {} local seen = {}
@@ -22,6 +23,7 @@ local function dedupe_files(files)
return result return result
end end
---@private
local function safe_read(filename, read_quantifier) local function safe_read(filename, read_quantifier)
local file, err = io.open(filename, 'r') local file, err = io.open(filename, 'r')
if not file then if not file then
@@ -32,6 +34,11 @@ local function safe_read(filename, read_quantifier)
return content return content
end end
--- Gets the list of files used to make up a query
---
--- @param lang The language
--- @param query_name The name of the query to load
--- @param is_included Internal parameter, most of the time left as `nil`
function M.get_query_files(lang, query_name, is_included) function M.get_query_files(lang, query_name, is_included)
local query_path = string.format('queries/%s/%s.scm', lang, query_name) local query_path = string.format('queries/%s/%s.scm', lang, query_name)
local lang_files = dedupe_files(a.nvim_get_runtime_file(query_path, true)) local lang_files = dedupe_files(a.nvim_get_runtime_file(query_path, true))
@@ -79,6 +86,7 @@ function M.get_query_files(lang, query_name, is_included)
return query_files return query_files
end end
---@private
local function read_query_files(filenames) local function read_query_files(filenames)
local contents = {} local contents = {}
@@ -103,19 +111,20 @@ local explicit_queries = setmetatable({}, {
--- ---
--- This allows users to override any runtime files and/or configuration --- This allows users to override any runtime files and/or configuration
--- set by plugins. --- set by plugins.
---@param lang string: The language to use for the query ---
---@param query_name string: The name of the query (i.e. "highlights") --- @param lang string: The language to use for the query
---@param text string: The query text (unparsed). --- @param query_name string: The name of the query (i.e. "highlights")
--- @param text string: The query text (unparsed).
function M.set_query(lang, query_name, text) function M.set_query(lang, query_name, text)
explicit_queries[lang][query_name] = M.parse_query(lang, text) explicit_queries[lang][query_name] = M.parse_query(lang, text)
end end
--- Returns the runtime query {query_name} for {lang}. --- Returns the runtime query {query_name} for {lang}.
-- ---
-- @param lang The language to use for the query --- @param lang The language to use for the query
-- @param query_name The name of the query (i.e. "highlights") --- @param query_name The name of the query (i.e. "highlights")
-- ---
-- @return The corresponding query, parsed. --- @return The corresponding query, parsed.
function M.get_query(lang, query_name) function M.get_query(lang, query_name)
if explicit_queries[lang][query_name] then if explicit_queries[lang][query_name] then
return explicit_queries[lang][query_name] return explicit_queries[lang][query_name]
@@ -129,12 +138,23 @@ function M.get_query(lang, query_name)
end end
end end
--- Parses a query. --- Parse {query} as a string. (If the query is in a file, the caller
-- --- should read the contents into a string before calling).
-- @param language The language ---
-- @param query A string containing the query (s-expr syntax) --- Returns a `Query` (see |lua-treesitter-query|) object which can be used to
-- --- search nodes in the syntax tree for the patterns defined in {query}
-- @returns The query --- using `iter_*` methods below.
---
--- Exposes `info` and `captures` with additional information about the {query}.
--- - `captures` contains the list of unique capture names defined in
--- {query}.
--- -` info.captures` also points to `captures`.
--- - `info.patterns` contains information about predicates.
---
--- @param lang The language
--- @param query A string containing the query (s-expr syntax)
---
--- @returns The query
function M.parse_query(lang, query) function M.parse_query(lang, query)
language.require_language(lang) language.require_language(lang)
local self = setmetatable({}, Query) local self = setmetatable({}, Query)
@@ -147,8 +167,9 @@ end
-- TODO(vigoux): support multiline nodes too -- TODO(vigoux): support multiline nodes too
--- Gets the text corresponding to a given node --- Gets the text corresponding to a given node
-- @param node the node ---
-- @param bufnr the buffer from which the node is extracted. --- @param node the node
--- @param bsource The buffer or string from which the node is extracted
function M.get_node_text(node, source) function M.get_node_text(node, source)
local start_row, start_col, start_byte = node:start() local start_row, start_col, start_byte = node:start()
local end_row, end_col, end_byte = node:end_() local end_row, end_col, end_byte = node:end_()
@@ -200,6 +221,7 @@ local predicate_handlers = {
["match?"] = (function() ["match?"] = (function()
local magic_prefixes = {['\\v']=true, ['\\m']=true, ['\\M']=true, ['\\V']=true} local magic_prefixes = {['\\v']=true, ['\\m']=true, ['\\M']=true, ['\\V']=true}
---@private
local function check_magic(str) local function check_magic(str)
if string.len(str) < 2 or magic_prefixes[string.sub(str,1,2)] then if string.len(str) < 2 or magic_prefixes[string.sub(str,1,2)] then
return str return str
@@ -282,10 +304,10 @@ local directive_handlers = {
} }
--- Adds a new predicate to be used in queries --- Adds a new predicate to be used in queries
-- ---
-- @param name the name of the predicate, without leading # --- @param name the name of the predicate, without leading #
-- @param handler the handler function to be used --- @param handler the handler function to be used
-- signature will be (match, pattern, bufnr, predicate) --- signature will be (match, pattern, bufnr, predicate)
function M.add_predicate(name, handler, force) function M.add_predicate(name, handler, force)
if predicate_handlers[name] and not force then if predicate_handlers[name] and not force then
error(string.format("Overriding %s", name)) error(string.format("Overriding %s", name))
@@ -295,10 +317,10 @@ function M.add_predicate(name, handler, force)
end end
--- Adds a new directive to be used in queries --- Adds a new directive to be used in queries
-- ---
-- @param name the name of the directive, without leading # --- @param name the name of the directive, without leading #
-- @param handler the handler function to be used --- @param handler the handler function to be used
-- signature will be (match, pattern, bufnr, predicate) --- signature will be (match, pattern, bufnr, predicate)
function M.add_directive(name, handler, force) function M.add_directive(name, handler, force)
if directive_handlers[name] and not force then if directive_handlers[name] and not force then
error(string.format("Overriding %s", name)) error(string.format("Overriding %s", name))
@@ -312,14 +334,17 @@ function M.list_predicates()
return vim.tbl_keys(predicate_handlers) return vim.tbl_keys(predicate_handlers)
end end
---@private
local function xor(x, y) local function xor(x, y)
return (x or y) and not (x and y) return (x or y) and not (x and y)
end end
---@private
local function is_directive(name) local function is_directive(name)
return string.sub(name, -1) == "!" return string.sub(name, -1) == "!"
end end
---@private
function Query:match_preds(match, pattern, source) function Query:match_preds(match, pattern, source)
local preds = self.info.patterns[pattern] local preds = self.info.patterns[pattern]
@@ -358,7 +383,7 @@ function Query:match_preds(match, pattern, source)
return true return true
end end
--- Applies directives against a match and pattern. ---@private
function Query:apply_directives(match, pattern, source, metadata) function Query:apply_directives(match, pattern, source, metadata)
local preds = self.info.patterns[pattern] local preds = self.info.patterns[pattern]
@@ -380,6 +405,7 @@ end
--- Returns the start and stop value if set else the node's range. --- Returns the start and stop value if set else the node's range.
-- When the node's range is used, the stop is incremented by 1 -- When the node's range is used, the stop is incremented by 1
-- to make the search inclusive. -- to make the search inclusive.
---@private
local function value_or_node_range(start, stop, node) local function value_or_node_range(start, stop, node)
if start == nil and stop == nil then if start == nil and stop == nil then
local node_start, _, node_stop, _ = node:range() local node_start, _, node_stop, _ = node:range()
@@ -389,15 +415,36 @@ local function value_or_node_range(start, stop, node)
return start, stop return start, stop
end end
--- Iterates of the captures of self on a given range. --- Iterate over all captures from all matches inside {node}
-- ---
-- @param node The node under which the search will occur --- {source} is needed if the query contains predicates, then the caller
-- @param buffer The source buffer to search --- must ensure to use a freshly parsed tree consistent with the current
-- @param start The starting line of the search --- text of the buffer (if relevent). {start_row} and {end_row} can be used to limit
-- @param stop The stopping line of the search (end-exclusive) --- matches inside a row range (this is typically used with root node
-- --- as the node, i e to get syntax highlight matches in the current
-- @returns The matching capture id --- viewport). When omitted the start and end row values are used from the given node.
-- @returns The captured node ---
--- The iterator returns three values, a numeric id identifying the capture,
--- the captured node, and metadata from any directives processing the match.
--- The following example shows how to get captures by name:
---
--- <pre>
--- for id, node, metadata in query:iter_captures(tree:root(), bufnr, first, last) do
--- local name = query.captures[id] -- name of the capture in the query
--- -- typically useful info about the node:
--- local type = node:type() -- type of the captured node
--- local row1, col1, row2, col2 = node:range() -- range of the capture
--- ... use the info here ...
--- end
--- </pre>
---
--- @param node The node under which the search will occur
--- @param source The source buffer or string to exctract text from
--- @param start The starting line of the search
--- @param stop The stopping line of the search (end-exclusive)
---
--- @returns The matching capture id
--- @returns The captured node
function Query:iter_captures(node, source, start, stop) function Query:iter_captures(node, source, start, stop)
if type(source) == "number" and source == 0 then if type(source) == "number" and source == 0 then
source = vim.api.nvim_get_current_buf() source = vim.api.nvim_get_current_buf()
@@ -406,6 +453,7 @@ function Query:iter_captures(node, source, start, stop)
start, stop = value_or_node_range(start, stop, node) start, stop = value_or_node_range(start, stop, node)
local raw_iter = node:_rawquery(self.query, true, start, stop) local raw_iter = node:_rawquery(self.query, true, start, stop)
---@private
local function iter() local function iter()
local capture, captured_node, match = raw_iter() local capture, captured_node, match = raw_iter()
local metadata = {} local metadata = {}
@@ -425,14 +473,35 @@ function Query:iter_captures(node, source, start, stop)
end end
--- Iterates the matches of self on a given range. --- Iterates the matches of self on a given range.
-- ---
-- @param node The node under which the search will occur --- Iterate over all matches within a node. The arguments are the same as
-- @param buffer The source buffer to search --- for |query:iter_captures()| but the iterated values are different:
-- @param start The starting line of the search --- an (1-based) index of the pattern in the query, a table mapping
-- @param stop The stopping line of the search (end-exclusive) --- capture indices to nodes, and metadata from any directives processing the match.
-- --- If the query has more than one pattern the capture table might be sparse,
-- @returns The matching pattern id --- and e.g. `pairs()` method should be used over `ipairs`.
-- @returns The matching match --- Here an example iterating over all captures in every match:
---
--- <pre>
--- for pattern, match, metadata in cquery:iter_matches(tree:root(), bufnr, first, last) do
--- for id, node in pairs(match) do
--- local name = query.captures[id]
--- -- `node` was captured by the `name` capture in the match
---
--- local node_data = metadata[id] -- Node level metadata
---
--- ... use the info here ...
--- end
--- end
--- </pre>
---
--- @param node The node under which the search will occur
--- @param source The source buffer or string to search
--- @param start The starting line of the search
--- @param stop The stopping line of the search (end-exclusive)
---
--- @returns The matching pattern id
--- @returns The matching match
function Query:iter_matches(node, source, start, stop) function Query:iter_matches(node, source, start, stop)
if type(source) == "number" and source == 0 then if type(source) == "number" and source == 0 then
source = vim.api.nvim_get_current_buf() source = vim.api.nvim_get_current_buf()

76
scripts/gen_vimdoc.py
View File

@@ -186,6 +186,48 @@ CONFIG = {
'module_override': {}, 'module_override': {},
'append_only': [], 'append_only': [],
}, },
'treesitter': {
'mode': 'lua',
'filename': 'treesitter.txt',
'section_start_token': '*lua-treesitter-core*',
'section_order': [
'treesitter.lua',
'language.lua',
'query.lua',
'highlighter.lua',
'languagetree.lua',
'health.lua',
],
'files': ' '.join([
os.path.join(base_dir, 'runtime/lua/vim/treesitter.lua'),
os.path.join(base_dir, 'runtime/lua/vim/treesitter/'),
]),
'file_patterns': '*.lua',
'fn_name_prefix': '',
'section_name': {},
'section_fmt': lambda name: (
'Lua module: vim.treesitter'
if name.lower() == 'treesitter'
else f'Lua module: vim.treesitter.{name.lower()}'),
'helptag_fmt': lambda name: (
'*lua-treesitter-core*'
if name.lower() == 'treesitter'
else f'*treesitter-{name.lower()}*'),
'fn_helptag_fmt': lambda fstem, name: (
f'*{name}()*'
if name != 'new'
else f'*{fstem}.{name}()*'),
# 'fn_helptag_fmt': lambda fstem, name: (
# f'*vim.treesitter.{name}()*'
# if fstem == 'treesitter'
# else (
# '*vim.lsp.client*'
# # HACK. TODO(justinmk): class/structure support in lua2dox
# if 'lsp.client' == f'{fstem}.{name}'
# else f'*vim.lsp.{fstem}.{name}()*')),
'module_override': {},
'append_only': [],
}
} }
param_exclude = ( param_exclude = (
@@ -666,15 +708,6 @@ def extract_from_xml(filename, target, width):
annotations = filter(None, map(lambda x: annotation_map.get(x), annotations = filter(None, map(lambda x: annotation_map.get(x),
annotations.split())) annotations.split()))
if not fmt_vimhelp:
pass
else:
fstem = '?'
if '.' in compoundname:
fstem = compoundname.split('.')[0]
fstem = CONFIG[target]['module_override'].get(fstem, fstem)
vimtag = CONFIG[target]['fn_helptag_fmt'](fstem, name)
params = [] params = []
type_length = 0 type_length = 0
@@ -695,17 +728,37 @@ def extract_from_xml(filename, target, width):
if fmt_vimhelp and param_type.endswith('*'): if fmt_vimhelp and param_type.endswith('*'):
param_type = param_type.strip('* ') param_type = param_type.strip('* ')
param_name = '*' + param_name param_name = '*' + param_name
type_length = max(type_length, len(param_type)) type_length = max(type_length, len(param_type))
params.append((param_type, param_name)) params.append((param_type, param_name))
# Handle Object Oriented style functions here.
# We make sure they have "self" in the parameters,
# and a parent function
if return_type.startswith('function') \
and len(return_type.split(' ')) >= 2 \
and any(x[1] == 'self' for x in params):
split_return = return_type.split(' ')
name = f'{split_return[1]}:{name}'
c_args = [] c_args = []
for param_type, param_name in params: for param_type, param_name in params:
c_args.append((' ' if fmt_vimhelp else '') + ( c_args.append((' ' if fmt_vimhelp else '') + (
'%s %s' % (param_type.ljust(type_length), param_name)).strip()) '%s %s' % (param_type.ljust(type_length), param_name)).strip())
if not fmt_vimhelp:
pass
else:
fstem = '?'
if '.' in compoundname:
fstem = compoundname.split('.')[0]
fstem = CONFIG[target]['module_override'].get(fstem, fstem)
vimtag = CONFIG[target]['fn_helptag_fmt'](fstem, name)
prefix = '%s(' % name prefix = '%s(' % name
suffix = '%s)' % ', '.join('{%s}' % a[1] for a in params suffix = '%s)' % ', '.join('{%s}' % a[1] for a in params
if a[0] not in ('void', 'Error')) if a[0] not in ('void', 'Error'))
if not fmt_vimhelp: if not fmt_vimhelp:
c_decl = '%s %s(%s);' % (return_type, name, ', '.join(c_args)) c_decl = '%s %s(%s);' % (return_type, name, ', '.join(c_args))
signature = prefix + suffix signature = prefix + suffix
@@ -774,7 +827,9 @@ def extract_from_xml(filename, target, width):
xrefs.clear() xrefs.clear()
fns = collections.OrderedDict(sorted(fns.items())) fns = collections.OrderedDict(sorted(
fns.items(),
key=lambda key_item_tuple: key_item_tuple[0].lower()))
deprecated_fns = collections.OrderedDict(sorted(deprecated_fns.items())) deprecated_fns = collections.OrderedDict(sorted(deprecated_fns.items()))
return (fns, deprecated_fns) return (fns, deprecated_fns)
@@ -1002,6 +1057,7 @@ def main(config, args):
title, helptag, section_doc = sections.pop(filename) title, helptag, section_doc = sections.pop(filename)
except KeyError: except KeyError:
msg(f'warning: empty docs, skipping (target={target}): {filename}') msg(f'warning: empty docs, skipping (target={target}): {filename}')
msg(f' existing docs: {sections.keys()}')
continue continue
i += 1 i += 1
if filename not in CONFIG[target]['append_only']: if filename not in CONFIG[target]['append_only']:

21
scripts/lua2dox.lua
View File

@@ -491,6 +491,27 @@ function TLua2DoX_filter.readfile(this,AppStamp,Filename)
end end
end end
-- Big hax
if string.find(fn, ":") then
-- TODO: We need to add a first parameter of "SELF" here
-- local colon_place = string.find(fn, ":")
-- local name = string.sub(fn, 1, colon_place)
fn = fn:gsub(":", ".", 1)
outStream:writeln("/// @param self")
local paren_start = string.find(fn, "(", 1, true)
local paren_finish = string.find(fn, ")", 1, true)
-- Nothing in between the parens
local comma
if paren_finish == paren_start + 1 then
comma = ""
else
comma = ", "
end
fn = string.sub(fn, 1, paren_start) .. "self" .. comma .. string.sub(fn, paren_start + 1)
end
-- add vanilla function -- add vanilla function
outStream:writeln(fn_type .. 'function ' .. fn .. '{}') outStream:writeln(fn_type .. 'function ' .. fn .. '{}')
end end

16
src/nvim/lua/vim.lua
View File

@@ -400,7 +400,10 @@ do
wfw = true; winbl = true; winblend = true; winfixheight = true; wfw = true; winbl = true; winblend = true; winfixheight = true;
winfixwidth = true; winhighlight = true; winhl = true; wrap = true; winfixwidth = true; winhighlight = true; winhl = true; wrap = true;
} }
--@private
local function new_buf_opt_accessor(bufnr) local function new_buf_opt_accessor(bufnr)
--@private
local function get(k) local function get(k)
if window_options[k] then if window_options[k] then
return a.nvim_err_writeln(k.." is a window option, not a buffer option") return a.nvim_err_writeln(k.." is a window option, not a buffer option")
@@ -410,23 +413,34 @@ do
end end
return a.nvim_buf_get_option(bufnr or 0, k) return a.nvim_buf_get_option(bufnr or 0, k)
end end
--@private
local function set(k, v) local function set(k, v)
if window_options[k] then if window_options[k] then
return a.nvim_err_writeln(k.." is a window option, not a buffer option") return a.nvim_err_writeln(k.." is a window option, not a buffer option")
end end
return a.nvim_buf_set_option(bufnr or 0, k, v) return a.nvim_buf_set_option(bufnr or 0, k, v)
end end
return make_meta_accessor(get, set) return make_meta_accessor(get, set)
end end
vim.bo = new_buf_opt_accessor(nil) vim.bo = new_buf_opt_accessor(nil)
--@private
local function new_win_opt_accessor(winnr) local function new_win_opt_accessor(winnr)
--@private
local function get(k) local function get(k)
if winnr == nil and type(k) == "number" then if winnr == nil and type(k) == "number" then
return new_win_opt_accessor(k) return new_win_opt_accessor(k)
end end
return a.nvim_win_get_option(winnr or 0, k) return a.nvim_win_get_option(winnr or 0, k)
end end
local function set(k, v) return a.nvim_win_set_option(winnr or 0, k, v) end
--@private
local function set(k, v)
return a.nvim_win_set_option(winnr or 0, k, v)
end
return make_meta_accessor(get, set) return make_meta_accessor(get, set)
end end
vim.wo = new_win_opt_accessor(nil) vim.wo = new_win_opt_accessor(nil)