Merge pull request #31400 from vanaigr/decor-provider-range

feat(decor): add range-based highlighting
perf: add on_range in treesitter highlighting
2025-09-03 18:08:16 +00:00 · 2025-08-29 10:33:15 +02:00 · 2025-08-28 08:22:38 -05:00 · 2025-08-28 08:22:38 -05:00
19 changed files with 797 additions and 199 deletions
--- a/runtime/doc/api.txt
+++ b/runtime/doc/api.txt
@@ -3290,11 +3290,12 @@ nvim_set_decoration_provider({ns_id}, {opts})
    Note: this function should not be called often. Rather, the callbacks
    themselves can be used to throttle unneeded callbacks. the `on_start`
    callback can return `false` to disable the provider until the next redraw.
-    Similarly, return `false` in `on_win` will skip the `on_line` calls for
-    that window (but any extmarks set in `on_win` will still be used). A
-    plugin managing multiple sources of decoration should ideally only set one
-    provider, and merge the sources internally. You can use multiple `ns_id`
-    for the extmarks set/modified inside the callback anyway.
+    Similarly, return `false` in `on_win` will skip the `on_line` and
+    `on_range` calls for that window (but any extmarks set in `on_win` will
+    still be used). A plugin managing multiple sources of decoration should
+    ideally only set one provider, and merge the sources internally. You can
+    use multiple `ns_id` for the extmarks set/modified inside the callback
+    anyway.

    Note: doing anything other than setting extmarks is considered
    experimental. Doing things like changing options are not explicitly
@@ -3302,8 +3303,8 @@ nvim_set_decoration_provider({ns_id}, {opts})
    consumption). Doing `vim.rpcnotify` should be OK, but `vim.rpcrequest` is
    quite dubious for the moment.

-    Note: It is not allowed to remove or update extmarks in `on_line`
-    callbacks.
+    Note: It is not allowed to remove or update extmarks in `on_line` or
+    `on_range` callbacks.

    Attributes: ~
        Lua |vim.api| only
@@ -3326,6 +3327,14 @@ nvim_set_decoration_provider({ns_id}, {opts})
                 • on_line: called for each buffer line being redrawn. (The
                   interaction with fold lines is subject to change) >
                    ["line", winid, bufnr, row]
+<
+                 • on_range: called for each buffer range being redrawn. Range
+                   is end-exclusive and may span multiple lines. Range bounds
+                   point to the first byte of a character. An end position of
+                   the form (lnum, 0), including (number of lines, 0), is
+                   valid and indicates that EOL of the preceding line is
+                   included. >
+                    ["range", winid, bufnr, begin_row, begin_col, end_row, end_col]
 <
                 • on_end: called at the end of a redraw cycle >
                    ["end", tick]
--- a/runtime/doc/news.txt
+++ b/runtime/doc/news.txt
@@ -51,7 +51,7 @@ These changes may require adaptations in your config or plugins.

 API

-• todo
+• Decoration provider has `on_range()` callback.

 BUILD

@@ -118,6 +118,7 @@ TREESITTER
  `metadata[capture_id].offset`. The offset will be applied in
  |vim.treesitter.get_range()|, which should be preferred over reading
  metadata directly for retrieving node ranges.
+• |Query:iter_captures()| supports specifying starting and ending columns.

 TUI

--- a/runtime/doc/treesitter.txt
+++ b/runtime/doc/treesitter.txt
@@ -1522,7 +1522,7 @@ them to parse text. See |vim.treesitter.query.parse()| for a working example.
      • {has_combined_injections}  (`boolean`) whether the query contains
                                   combined injections
      • {query}                    (`TSQuery`) userdata query object
-      • {iter_captures}            (`fun(self: vim.treesitter.Query, node: TSNode, source: integer|string, start: integer?, stop: integer?, opts: table?): fun(end_line: integer?): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch, TSTree`)
+      • {iter_captures}            (`fun(self: vim.treesitter.Query, node: TSNode, source: integer|string, start_row: integer?, end_row: integer?, opts: table?): fun(end_line: integer?, end_col: integer?): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch, TSTree`)
                                   See |Query:iter_captures()|.
      • {iter_matches}             (`fun(self: vim.treesitter.Query, node: TSNode, source: integer|string, start: integer?, stop: integer?, opts: table?): fun(): integer, table<integer, TSNode[]>, vim.treesitter.query.TSMetadata, TSTree`)
                                   See |Query:iter_matches()|.
@@ -1693,7 +1693,7 @@ parse({lang}, {query})                          *vim.treesitter.query.parse()*
      • |vim.treesitter.query.get()|

                                                       *Query:iter_captures()*
-Query:iter_captures({node}, {source}, {start}, {stop}, {opts})
+Query:iter_captures({node}, {source}, {start_row}, {end_row}, {opts})
    Iterates over all captures from all matches in {node}.

    {source} is required if the query contains predicates; then the caller
@@ -1725,22 +1725,26 @@ Query:iter_captures({node}, {source}, {start}, {stop}, {opts})
        contained predicates.

    Parameters: ~
-      • {node}    (`TSNode`) under which the search will occur
-      • {source}  (`integer|string`) Source buffer or string to extract text
-                  from
-      • {start}   (`integer?`) Starting line for the search. Defaults to
-                  `node:start()`.
-      • {stop}    (`integer?`) Stopping line for the search (end-exclusive).
-                  Defaults to `node:end_()`.
-      • {opts}    (`table?`) Optional keyword arguments:
-                  • max_start_depth (integer) if non-zero, sets the maximum
-                    start depth for each match. This is used to prevent
-                    traversing too deep into a tree.
-                  • match_limit (integer) Set the maximum number of
-                    in-progress matches (Default: 256).
+      • {node}       (`TSNode`) under which the search will occur
+      • {source}     (`integer|string`) Source buffer or string to extract
+                     text from
+      • {start_row}  (`integer?`) Starting line for the search. Defaults to
+                     `node:start()`.
+      • {end_row}    (`integer?`) Stopping line for the search (end-inclusive,
+                     unless `stop_col` is provided). Defaults to
+                     `node:end_()`.
+      • {opts}       (`table?`) Optional keyword arguments:
+                     • max_start_depth (integer) if non-zero, sets the maximum
+                       start depth for each match. This is used to prevent
+                       traversing too deep into a tree.
+                     • match_limit (integer) Set the maximum number of
+                       in-progress matches (Default: 256).
+                     • start_col (integer) Starting column for the search.
+                     • end_col (integer) Stopping column for the search
+                       (end-exclusive).

    Return: ~
-        (`fun(end_line: integer?): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch, TSTree`)
+        (`fun(end_line: integer?, end_col: integer?): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch, TSTree`)
        capture id, capture node, metadata, match, tree

                                                        *Query:iter_matches()*
--- a/runtime/lua/vim/_meta/api.lua
+++ b/runtime/lua/vim/_meta/api.lua
@@ -2128,7 +2128,7 @@ function vim.api.nvim_set_current_win(window) end
 --- Note: this function should not be called often. Rather, the callbacks
 --- themselves can be used to throttle unneeded callbacks. the `on_start`
 --- callback can return `false` to disable the provider until the next redraw.
--- Similarly, return `false` in `on_win` will skip the `on_line` calls
+--- Similarly, return `false` in `on_win` will skip the `on_line` and `on_range` calls
 --- for that window (but any extmarks set in `on_win` will still be used).
 --- A plugin managing multiple sources of decoration should ideally only set
 --- one provider, and merge the sources internally. You can use multiple `ns_id`
@@ -2140,7 +2140,7 @@ function vim.api.nvim_set_current_win(window) end
 --- Doing `vim.rpcnotify` should be OK, but `vim.rpcrequest` is quite dubious
 --- for the moment.
 ---
--- Note: It is not allowed to remove or update extmarks in `on_line` callbacks.
+--- Note: It is not allowed to remove or update extmarks in `on_line` or `on_range` callbacks.
 ---
 --- @param ns_id integer Namespace id from `nvim_create_namespace()`
 --- @param opts vim.api.keyset.set_decoration_provider Table of callbacks:
@@ -2162,6 +2162,14 @@ function vim.api.nvim_set_current_win(window) end
 ---   ```
 ---     ["line", winid, bufnr, row]
 ---   ```
+--- - on_range: called for each buffer range being redrawn.
+---   Range is end-exclusive and may span multiple lines. Range
+---   bounds point to the first byte of a character. An end position
+---   of the form (lnum, 0), including (number of lines, 0), is valid
+---   and indicates that EOL of the preceding line is included.
+---   ```
+---     ["range", winid, bufnr, begin_row, begin_col, end_row, end_col]
+---   ```
 --- - on_end: called at the end of a redraw cycle
 ---   ```
 ---     ["end", tick]
--- a/runtime/lua/vim/_meta/api_keysets.lua
+++ b/runtime/lua/vim/_meta/api_keysets.lua
@@ -382,6 +382,7 @@ error('Cannot require a meta file')
 --- @field on_buf? fun(_: "buf", bufnr: integer, tick: integer)
 --- @field on_win? fun(_: "win", winid: integer, bufnr: integer, toprow: integer, botrow: integer): boolean?
 --- @field on_line? fun(_: "line", winid: integer, bufnr: integer, row: integer): boolean?
+--- @field on_range? fun(_: "range", winid: integer, bufnr: integer, start_row: integer, start_col: integer, end_row: integer, end_col: integer): boolean?
 --- @field on_end? fun(_: "end", tick: integer)
 --- @field _on_hl_def? fun(_: "hl_def")
 --- @field _on_spell_nav? fun(_: "spell_nav")
--- a/runtime/lua/vim/treesitter/_meta/misc.lua
+++ b/runtime/lua/vim/treesitter/_meta/misc.lua
@@ -80,8 +80,6 @@ function TSQueryCursor:next_match() end

 --- @param node TSNode
 --- @param query TSQuery
--- @param start integer?
--- @param stop integer?
--- @param opts? { max_start_depth?: integer, match_limit?: integer}
+--- @param opts? { start_row: integer, start_col: integer, end_row: integer, end_col: integer, max_start_depth?: integer, match_limit?: integer }
 --- @return TSQueryCursor
-function vim._create_ts_querycursor(node, query, start, stop, opts) end
+function vim._create_ts_querycursor(node, query, opts) end
--- a/runtime/lua/vim/treesitter/highlighter.lua
+++ b/runtime/lua/vim/treesitter/highlighter.lua
@@ -1,10 +1,11 @@
 local api = vim.api
 local query = vim.treesitter.query
 local Range = require('vim.treesitter._range')
+local cmp_lt = Range.cmp_pos.lt

 local ns = api.nvim_create_namespace('nvim.treesitter.highlighter')

---@alias vim.treesitter.highlighter.Iter fun(end_line: integer|nil): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch
+---@alias vim.treesitter.highlighter.Iter fun(end_line: integer|nil, end_col: integer|nil): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch, TSTree

 ---@class (private) vim.treesitter.highlighter.Query
 ---@field private _query vim.treesitter.Query?
@@ -57,6 +58,7 @@ end
 ---@class (private) vim.treesitter.highlighter.State
 ---@field tstree TSTree
 ---@field next_row integer
+---@field next_col integer
 ---@field iter vim.treesitter.highlighter.Iter?
 ---@field highlighter_query vim.treesitter.highlighter.Query
 ---@field prev_marks MarkInfo[]
@@ -233,6 +235,7 @@ function TSHighlighter:prepare_highlight_states(win, srow, erow)
    table.insert(self._highlight_states[win], {
      tstree = tstree,
      next_row = 0,
+      next_col = 0,
      iter = nil,
      highlighter_query = hl_query,
      prev_marks = {},
@@ -331,27 +334,36 @@ end
 ---Queues the remainder if the mark continues after the line.
 ---@param m MarkInfo
 ---@param buf integer
---@param line integer
+---@param range_start_row integer
+---@param range_start_col integer
+---@param range_end_row integer
+---@param range_end_col integer
 ---@param next_marks MarkInfo[]
-local function add_mark(m, buf, line, next_marks)
+local function add_mark(
+  m,
+  buf,
+  range_start_row,
+  range_start_col,
+  range_end_row,
+  range_end_col,
+  next_marks
+)
  local cur_start_l = m.start_line
  local cur_start_c = m.start_col
-  if cur_start_l < line then
-    cur_start_l = line
-    cur_start_c = 0
+  if cmp_lt(cur_start_l, cur_start_c, range_start_row, range_start_col) then
+    cur_start_l = range_start_row
+    cur_start_c = range_start_col
  end

  local cur_opts = m.opts
-  if cur_opts.end_line >= line + 1 then
+  if cmp_lt(range_end_row, range_end_col, cur_opts.end_line, cur_opts.end_col) then
    cur_opts = vim.deepcopy(cur_opts, true)
-    cur_opts.end_line = line + 1
-    cur_opts.end_col = 0
+    cur_opts.end_line = range_end_row
+    cur_opts.end_col = range_end_col
    table.insert(next_marks, m)
  end

-  local empty = cur_opts.end_line < cur_start_l
-    or (cur_opts.end_line == cur_start_l and cur_opts.end_col <= cur_start_c)
-  if cur_start_l <= line and not empty then
+  if cmp_lt(cur_start_l, cur_start_c, cur_opts.end_line, cur_opts.end_col) then
    api.nvim_buf_set_extmark(buf, ns, cur_start_l, cur_start_c, cur_opts)
  end
 end
@@ -359,17 +371,44 @@ end
 ---@param self vim.treesitter.highlighter
 ---@param win integer
 ---@param buf integer
---@param line integer
+---@param range_start_row integer
+---@param range_start_col integer
+---@param range_end_row integer
+---@param range_end_col integer
 ---@param on_spell boolean
 ---@param on_conceal boolean
-local function on_line_impl(self, win, buf, line, on_spell, on_conceal)
-  self._conceal_checked[line] = self._conceal_line and true or nil
+local function on_range_impl(
+  self,
+  win,
+  buf,
+  range_start_row,
+  range_start_col,
+  range_end_row,
+  range_end_col,
+  on_spell,
+  on_conceal
+)
+  if self._conceal_line then
+    range_start_col = 0
+    if range_end_col ~= 0 then
+      range_end_row = range_end_row + 1
+      range_end_col = 0
+    end
+  end
+  for i = range_start_row, range_end_row - 1 do
+    self._conceal_checked[i] = self._conceal_line or nil
+  end
  self:for_each_highlight_state(win, function(state)
    local root_node = state.tstree:root()
-    local root_start_row, _, root_end_row, _ = root_node:range()
+    ---@type { [1]: integer, [2]: integer, [3]: integer, [4]: integer }
+    local root_range = { root_node:range() }

-    -- Only consider trees that contain this line
-    if root_start_row > line or root_end_row < line then
+    if
+      not Range.intercepts(
+        root_range,
+        { range_start_row, range_start_col, range_end_row, range_end_col }
+      )
+    then
      return
    end

@@ -378,85 +417,114 @@ local function on_line_impl(self, win, buf, line, on_spell, on_conceal)
    local next_marks = {}

    for _, mark in ipairs(state.prev_marks) do
-      add_mark(mark, buf, line, next_marks)
+      add_mark(
+        mark,
+        buf,
+        range_start_row,
+        range_start_col,
+        range_end_row,
+        range_end_col,
+        next_marks
+      )
    end

-    if state.iter == nil or state.next_row < line then
+    local next_row = state.next_row
+    local next_col = state.next_col
+
+    if state.iter == nil or cmp_lt(next_row, next_col, range_start_row, range_start_col) then
      -- Mainly used to skip over folds

      -- TODO(lewis6991): Creating a new iterator loses the cached predicate results for query
      -- matches. Move this logic inside iter_captures() so we can maintain the cache.
-      state.iter =
-        state.highlighter_query:query():iter_captures(root_node, self.bufnr, line, root_end_row + 1)
+      state.iter = state.highlighter_query:query():iter_captures(
+        root_node,
+        self.bufnr,
+        range_start_row,
+        root_range[3],
+        { start_col = range_start_col, end_col = root_range[4] }
+      )
    end

    local captures = state.highlighter_query:query().captures

-    while line >= state.next_row do
-      local capture, node, metadata, match = state.iter(line)
-
-      local outer_range = { root_end_row + 1, 0, root_end_row + 1, 0 }
-      if node then
-        outer_range = vim.treesitter.get_range(node, buf, metadata and metadata[capture])
+    while cmp_lt(next_row, next_col, range_end_row, range_end_col) do
+      local capture, node, metadata, match = state.iter(range_end_row, range_end_col)
+      if not node then
+        next_row = math.huge
+        next_col = math.huge
+        break
+      end
+
+      local outer_range = vim.treesitter.get_range(node, buf, metadata and metadata[capture])
+      if cmp_lt(next_row, next_col, outer_range[1], outer_range[2]) then
+        next_row = outer_range[1]
+        next_col = outer_range[2]
+      end
+
+      if not capture then
+        break
      end
-      local outer_range_start_row = outer_range[1]

      for _, range in ipairs(tree_region) do
        local intersection = Range.intersection(range, outer_range)
        if intersection then
          local start_row, start_col, end_row, end_col = Range.unpack4(intersection)

-          if capture then
-            local hl = state.highlighter_query:get_hl_from_capture(capture)
+          local hl = state.highlighter_query:get_hl_from_capture(capture)

-            local capture_name = captures[capture]
+          local capture_name = captures[capture]

-            local spell, spell_pri_offset = get_spell(capture_name)
+          local spell, spell_pri_offset = get_spell(capture_name)

-            -- The "priority" attribute can be set at the pattern level or on a particular capture
-            local priority = (
-              tonumber(metadata.priority or metadata[capture] and metadata[capture].priority)
-              or vim.hl.priorities.treesitter
-            ) + spell_pri_offset
+          -- The "priority" attribute can be set at the pattern level or on a particular capture
+          local priority = (
+            tonumber(metadata.priority or metadata[capture] and metadata[capture].priority)
+            or vim.hl.priorities.treesitter
+          ) + spell_pri_offset

-            -- The "conceal" attribute can be set at the pattern level or on a particular capture
-            local conceal = metadata.conceal or metadata[capture] and metadata[capture].conceal
+          -- The "conceal" attribute can be set at the pattern level or on a particular capture
+          local conceal = metadata.conceal or metadata[capture] and metadata[capture].conceal

-            local url = get_url(match, buf, capture, metadata)
+          local url = get_url(match, buf, capture, metadata)

-            if hl and end_row >= line and not on_conceal and (not on_spell or spell ~= nil) then
-              local opts = {
-                end_line = end_row,
-                end_col = end_col,
-                hl_group = hl,
-                ephemeral = true,
-                priority = priority,
-                conceal = conceal,
-                spell = spell,
-                url = url,
-              }
-              local mark = { start_line = start_row, start_col = start_col, opts = opts }
-              add_mark(mark, buf, line, next_marks)
-            end
+          if hl and not on_conceal and (not on_spell or spell ~= nil) then
+            local opts = {
+              end_line = end_row,
+              end_col = end_col,
+              hl_group = hl,
+              ephemeral = true,
+              priority = priority,
+              conceal = conceal,
+              spell = spell,
+              url = url,
+            }
+            local mark = { start_line = start_row, start_col = start_col, opts = opts }
+            add_mark(
+              mark,
+              buf,
+              range_start_row,
+              range_start_col,
+              range_end_row,
+              range_end_col,
+              next_marks
+            )
+          end

-            if
-              (metadata.conceal_lines or metadata[capture] and metadata[capture].conceal_lines)
-              and #api.nvim_buf_get_extmarks(buf, ns, { start_row, 0 }, { start_row, 0 }, {}) == 0
-            then
-              api.nvim_buf_set_extmark(buf, ns, start_row, 0, {
-                end_line = end_row,
-                conceal_lines = '',
-              })
-            end
+          if
+            (metadata.conceal_lines or metadata[capture] and metadata[capture].conceal_lines)
+            and #api.nvim_buf_get_extmarks(buf, ns, { start_row, 0 }, { start_row, 0 }, {}) == 0
+          then
+            api.nvim_buf_set_extmark(buf, ns, start_row, 0, {
+              end_line = end_row,
+              conceal_lines = '',
+            })
          end
        end
      end
-
-      if outer_range_start_row > line then
-        state.next_row = outer_range_start_row
-      end
    end

+    state.next_row = next_row
+    state.next_col = next_col
    state.prev_marks = next_marks
  end)
 end
@@ -464,14 +532,17 @@ end
 ---@private
 ---@param win integer
 ---@param buf integer
---@param line integer
-function TSHighlighter._on_line(_, win, buf, line, _)
+---@param br integer
+---@param bc integer
+---@param er integer
+---@param ec integer
+function TSHighlighter._on_range(_, win, buf, br, bc, er, ec, _)
  local self = TSHighlighter.active[buf]
  if not self then
    return
  end

-  on_line_impl(self, win, buf, line, false, false)
+  on_range_impl(self, win, buf, br, bc, er, ec, false, false)
 end

 ---@private
@@ -490,9 +561,7 @@ function TSHighlighter._on_spell_nav(_, win, buf, srow, _, erow, _)
  local highlight_states = self._highlight_states[win]
  self:prepare_highlight_states(win, srow, erow)

-  for row = srow, erow do
-    on_line_impl(self, win, buf, row, true, false)
-  end
+  on_range_impl(self, win, buf, srow, 0, erow, 0, true, false)
  self._highlight_states[win] = highlight_states
 end

@@ -510,7 +579,7 @@ function TSHighlighter._on_conceal_line(_, win, buf, row)
  local highlight_states = self._highlight_states[win]
  self.tree:parse({ row, row })
  self:prepare_highlight_states(win, row, row)
-  on_line_impl(self, win, buf, row, false, true)
+  on_range_impl(self, win, buf, row, 0, row + 1, 0, false, true)
  self._highlight_states[win] = highlight_states
 end

@@ -554,7 +623,7 @@ end

 api.nvim_set_decoration_provider(ns, {
  on_win = TSHighlighter._on_win,
-  on_line = TSHighlighter._on_line,
+  on_range = TSHighlighter._on_range,
  _on_spell_nav = TSHighlighter._on_spell_nav,
  _on_conceal_line = TSHighlighter._on_conceal_line,
 })
--- a/runtime/lua/vim/treesitter/query.lua
+++ b/runtime/lua/vim/treesitter/query.lua
@@ -4,6 +4,7 @@
 local api = vim.api
 local language = require('vim.treesitter.language')
 local memoize = vim.func._memoize
+local cmp_ge = require('vim.treesitter._range').cmp_pos.ge

 local MODELINE_FORMAT = '^;+%s*inherits%s*:?%s*([a-z_,()]+)%s*$'
 local EXTENDS_FORMAT = '^;+%s*extends%s*$'
@@ -951,18 +952,20 @@ end
 ---
 ---@param node TSNode under which the search will occur
 ---@param source (integer|string) Source buffer or string to extract text from
---@param start? integer Starting line for the search. Defaults to `node:start()`.
---@param stop? integer Stopping line for the search (end-exclusive). Defaults to `node:end_()`.
+---@param start_row? integer Starting line for the search. Defaults to `node:start()`.
+---@param end_row? integer Stopping line for the search (end-inclusive, unless `stop_col` is provided). Defaults to `node:end_()`.
 ---@param opts? table Optional keyword arguments:
 ---   - max_start_depth (integer) if non-zero, sets the maximum start depth
 ---     for each match. This is used to prevent traversing too deep into a tree.
 ---   - match_limit (integer) Set the maximum number of in-progress matches (Default: 256).
+---   - start_col (integer) Starting column for the search.
+---   - end_col (integer) Stopping column for the search (end-exclusive).
 ---
---@return (fun(end_line: integer|nil): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch, TSTree):
+---@return (fun(end_line: integer|nil, end_col: integer|nil): integer, TSNode, vim.treesitter.query.TSMetadata, TSQueryMatch, TSTree):
 ---        capture id, capture node, metadata, match, tree
 ---
 ---@note Captures are only returned if the query pattern of a specific capture contained predicates.
-function Query:iter_captures(node, source, start, stop, opts)
+function Query:iter_captures(node, source, start_row, end_row, opts)
  opts = opts or {}
  opts.match_limit = opts.match_limit or 256

@@ -970,17 +973,24 @@ function Query:iter_captures(node, source, start, stop, opts)
    source = api.nvim_get_current_buf()
  end

-  start, stop = value_or_node_range(start, stop, node)
+  start_row, end_row = value_or_node_range(start_row, end_row, node)

  local tree = node:tree()
-  local cursor = vim._create_ts_querycursor(node, self.query, start, stop, opts)
+  local cursor = vim._create_ts_querycursor(node, self.query, {
+    start_row = start_row,
+    start_col = opts.start_col or 0,
+    end_row = end_row,
+    end_col = opts.end_col or 0,
+    max_start_depth = opts.max_start_depth,
+    match_limit = opts.match_limit or 256,
+  })

  -- For faster checks that a match is not in the cache.
  local highest_cached_match_id = -1
  ---@type table<integer, vim.treesitter.query.TSMetadata>
  local match_cache = {}

-  local function iter(end_line)
+  local function iter(end_line, end_col)
    local capture, captured_node, match = cursor:next_capture()

    if not capture then
@@ -1005,9 +1015,22 @@ function Query:iter_captures(node, source, start, stop, opts)
        local predicates = processed_pattern.predicates
        if not self:_match_predicates(predicates, pattern_i, captures, source) then
          cursor:remove_match(match_id)
-          if end_line and captured_node:range() > end_line then
+
+          local row, col = captured_node:range()
+
+          local outside = false
+          if end_line then
+            if end_col then
+              outside = cmp_ge(row, col, end_line, end_col)
+            else
+              outside = row > end_line
+            end
+          end
+
+          if outside then
            return nil, captured_node, nil, nil
          end
+
          return iter(end_line) -- tail call: try next match
        end

@@ -1072,7 +1095,14 @@ function Query:iter_matches(node, source, start, stop, opts)
  start, stop = value_or_node_range(start, stop, node)

  local tree = node:tree()
-  local cursor = vim._create_ts_querycursor(node, self.query, start, stop, opts)
+  local cursor = vim._create_ts_querycursor(node, self.query, {
+    start_row = start,
+    start_col = 0,
+    end_row = stop,
+    end_col = 0,
+    max_start_depth = opts.max_start_depth,
+    match_limit = opts.match_limit or 256,
+  })

  local function iter()
    local match = cursor:next_match()
--- a/src/nvim/api/extmark.c
+++ b/src/nvim/api/extmark.c
@@ -1004,7 +1004,7 @@ void nvim_buf_clear_namespace(Buffer buffer, Integer ns_id, Integer line_start,
 /// Note: this function should not be called often. Rather, the callbacks
 /// themselves can be used to throttle unneeded callbacks. the `on_start`
 /// callback can return `false` to disable the provider until the next redraw.
-/// Similarly, return `false` in `on_win` will skip the `on_line` calls
+/// Similarly, return `false` in `on_win` will skip the `on_line` and `on_range` calls
 /// for that window (but any extmarks set in `on_win` will still be used).
 /// A plugin managing multiple sources of decoration should ideally only set
 /// one provider, and merge the sources internally. You can use multiple `ns_id`
@@ -1016,7 +1016,7 @@ void nvim_buf_clear_namespace(Buffer buffer, Integer ns_id, Integer line_start,
 /// Doing `vim.rpcnotify` should be OK, but `vim.rpcrequest` is quite dubious
 /// for the moment.
 ///
-/// Note: It is not allowed to remove or update extmarks in `on_line` callbacks.
+/// Note: It is not allowed to remove or update extmarks in `on_line` or `on_range` callbacks.
 ///
 /// @param ns_id  Namespace id from |nvim_create_namespace()|
 /// @param opts  Table of callbacks:
@@ -1038,6 +1038,14 @@ void nvim_buf_clear_namespace(Buffer buffer, Integer ns_id, Integer line_start,
 ///               ```
 ///                 ["line", winid, bufnr, row]
 ///               ```
+///             - on_range: called for each buffer range being redrawn.
+///               Range is end-exclusive and may span multiple lines. Range
+///               bounds point to the first byte of a character. An end position
+///               of the form (lnum, 0), including (number of lines, 0), is valid
+///               and indicates that EOL of the preceding line is included.
+///               ```
+///                 ["range", winid, bufnr, begin_row, begin_col, end_row, end_col]
+///               ```
 ///             - on_end: called at the end of a redraw cycle
 ///               ```
 ///                 ["end", tick]
@@ -1061,6 +1069,7 @@ void nvim_set_decoration_provider(Integer ns_id, Dict(set_decoration_provider) *
    { "on_buf", &opts->on_buf, &p->redraw_buf },
    { "on_win", &opts->on_win, &p->redraw_win },
    { "on_line", &opts->on_line, &p->redraw_line },
+    { "on_range", &opts->on_range, &p->redraw_range },
    { "on_end", &opts->on_end, &p->redraw_end },
    { "_on_hl_def", &opts->_on_hl_def, &p->hl_def },
    { "_on_spell_nav", &opts->_on_spell_nav, &p->spell_nav },
--- a/src/nvim/api/keysets_defs.h
+++ b/src/nvim/api/keysets_defs.h
@@ -18,6 +18,8 @@ typedef struct {
  LuaRefOf(("win" _, Integer winid, Integer bufnr, Integer toprow, Integer botrow),
           *Boolean) on_win;
  LuaRefOf(("line" _, Integer winid, Integer bufnr, Integer row), *Boolean) on_line;
+  LuaRefOf(("range" _, Integer winid, Integer bufnr, Integer start_row, Integer start_col,
+            Integer end_row, Integer end_col), *Boolean) on_range;
  LuaRefOf(("end" _, Integer tick)) on_end;
  LuaRefOf(("hl_def" _)) _on_hl_def;
  LuaRefOf(("spell_nav" _)) _on_spell_nav;
--- a/src/nvim/decoration.c
+++ b/src/nvim/decoration.c
@@ -513,7 +513,7 @@ static void decor_state_pack(DecorState *state)
  state->future_begin = fut_beg;
 }

-bool decor_redraw_line(win_T *wp, int row, DecorState *state)
+void decor_redraw_line(win_T *wp, int row, DecorState *state)
 {
  decor_state_pack(state);

@@ -527,7 +527,11 @@ bool decor_redraw_line(win_T *wp, int row, DecorState *state)
  state->row = row;
  state->col_until = -1;
  state->eol_col = -1;
+}

+// Checks if there are (likely) more decorations on the current line.
+bool decor_has_more_decorations(DecorState *state, int row)
+{
  if (state->current_end != 0 || state->future_begin != (int)kv_size(state->ranges_i)) {
    return true;
  }
--- a/src/nvim/decoration_defs.h
+++ b/src/nvim/decoration_defs.h
@@ -149,6 +149,7 @@ typedef struct {
  LuaRef redraw_buf;
  LuaRef redraw_win;
  LuaRef redraw_line;
+  LuaRef redraw_range;
  LuaRef redraw_end;
  LuaRef hl_def;
  LuaRef spell_nav;
--- a/src/nvim/decoration_provider.c
+++ b/src/nvim/decoration_provider.c
@@ -25,7 +25,7 @@ static kvec_t(DecorProvider) decor_providers = KV_INITIAL_VALUE;

 #define DECORATION_PROVIDER_INIT(ns_id) (DecorProvider) \
  { ns_id, kDecorProviderDisabled, LUA_NOREF, LUA_NOREF, \
-    LUA_NOREF, LUA_NOREF, LUA_NOREF, \
+    LUA_NOREF, LUA_NOREF, LUA_NOREF, LUA_NOREF, \
    LUA_NOREF, LUA_NOREF, -1, false, false, 0 }

 static void decor_provider_error(DecorProvider *provider, const char *name, const char *msg)
@@ -189,6 +189,30 @@ void decor_providers_invoke_line(win_T *wp, int row)
  decor_state.running_decor_provider = false;
 }

+void decor_providers_invoke_range(win_T *wp, int start_row, int start_col, int end_row, int end_col)
+{
+  decor_state.running_decor_provider = true;
+  for (size_t i = 0; i < kv_size(decor_providers); i++) {
+    DecorProvider *p = &kv_A(decor_providers, i);
+    if (p->state == kDecorProviderActive && p->redraw_range != LUA_NOREF) {
+      MAXSIZE_TEMP_ARRAY(args, 6);
+      ADD_C(args, WINDOW_OBJ(wp->handle));
+      ADD_C(args, BUFFER_OBJ(wp->w_buffer->handle));
+      ADD_C(args, INTEGER_OBJ(start_row));
+      ADD_C(args, INTEGER_OBJ(start_col));
+      ADD_C(args, INTEGER_OBJ(end_row));
+      ADD_C(args, INTEGER_OBJ(end_col));
+      if (!decor_provider_invoke((int)i, "range", p->redraw_range, args, true)) {
+        // return 'false' or error: skip rest of this window
+        kv_A(decor_providers, i).state = kDecorProviderWinDisabled;
+      }
+
+      hl_check_ns();
+    }
+  }
+  decor_state.running_decor_provider = false;
+}
+
 /// For each provider invoke the 'buf' callback for a given buffer.
 ///
 /// @param      buf       Buffer
@@ -272,6 +296,7 @@ void decor_provider_clear(DecorProvider *p)
  NLUA_CLEAR_REF(p->redraw_buf);
  NLUA_CLEAR_REF(p->redraw_win);
  NLUA_CLEAR_REF(p->redraw_line);
+  NLUA_CLEAR_REF(p->redraw_range);
  NLUA_CLEAR_REF(p->redraw_end);
  NLUA_CLEAR_REF(p->spell_nav);
  NLUA_CLEAR_REF(p->conceal_line);
--- a/src/nvim/drawline.c
+++ b/src/nvim/drawline.c
@@ -1142,6 +1142,9 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b
  // Not drawing text when line is concealed or drawing filler lines beyond last line.
  const bool draw_text = !concealed && (lnum != buf->b_ml.ml_line_count + 1);

+  int decor_provider_end_col;
+  bool check_decor_providers = false;
+
  if (col_rows == 0 && draw_text) {
    // To speed up the loop below, set extra_check when there is linebreak,
    // trailing white space and/or syntax processing to be done.
@@ -1163,14 +1166,7 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b
      }
    }

-    decor_providers_invoke_line(wp, lnum - 1);  // may invalidate wp->w_virtcol
-    validate_virtcol(wp);
-
-    has_decor = decor_redraw_line(wp, lnum - 1, &decor_state);
-
-    if (has_decor) {
-      extra_check = true;
-    }
+    check_decor_providers = true;

    // Check for columns to display for 'colorcolumn'.
    wlv.color_cols = wp->w_buffer->terminal ? NULL : wp->w_p_cc_cols;
@@ -1466,22 +1462,22 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b

  // 'nowrap' or 'wrap' and a single line that doesn't fit: Advance to the
  // first character to be displayed.
-  const int start_col = wp->w_p_wrap
-                        ? (startrow == 0 ? wp->w_skipcol : 0)
-                        : wp->w_leftcol;
+  const int start_vcol = wp->w_p_wrap
+                         ? (startrow == 0 ? wp->w_skipcol : 0)
+                         : wp->w_leftcol;

  if (has_foldtext) {
-    wlv.vcol = start_col;
-  } else if (start_col > 0 && col_rows == 0) {
+    wlv.vcol = start_vcol;
+  } else if (start_vcol > 0 && col_rows == 0) {
    char *prev_ptr = ptr;
    CharSize cs = { 0 };

    CharsizeArg csarg;
    CSType cstype = init_charsize_arg(&csarg, wp, lnum, line);
-    csarg.max_head_vcol = start_col;
+    csarg.max_head_vcol = start_vcol;
    int vcol = wlv.vcol;
    StrCharInfo ci = utf_ptr2StrCharInfo(ptr);
-    while (vcol < start_col && *ci.ptr != NUL) {
+    while (vcol < start_vcol && *ci.ptr != NUL) {
      cs = win_charsize(cstype, vcol, ci.ptr, ci.chr.value, &csarg);
      vcol += cs.width;
      prev_ptr = ci.ptr;
@@ -1518,23 +1514,23 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b
    // - the visual mode is active, or
    // - drawing a fold
    // the end of the line may be before the start of the displayed part.
-    if (wlv.vcol < start_col && (wp->w_p_cuc
-                                 || wlv.color_cols
-                                 || virtual_active(wp)
-                                 || (VIsual_active && wp->w_buffer == curwin->w_buffer)
-                                 || has_fold)) {
-      wlv.vcol = start_col;
+    if (wlv.vcol < start_vcol && (wp->w_p_cuc
+                                  || wlv.color_cols
+                                  || virtual_active(wp)
+                                  || (VIsual_active && wp->w_buffer == curwin->w_buffer)
+                                  || has_fold)) {
+      wlv.vcol = start_vcol;
    }

    // Handle a character that's not completely on the screen: Put ptr at
    // that character but skip the first few screen characters.
-    if (wlv.vcol > start_col) {
+    if (wlv.vcol > start_vcol) {
      wlv.vcol -= charsize;
      ptr = prev_ptr;
    }

-    if (start_col > wlv.vcol) {
-      wlv.skip_cells = start_col - wlv.vcol - head;
+    if (start_vcol > wlv.vcol) {
+      wlv.skip_cells = start_vcol - wlv.vcol - head;
    }

    // Adjust for when the inverted text is before the screen,
@@ -1588,6 +1584,23 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b
    }
  }

+  if (check_decor_providers) {
+    int const col = (int)(ptr - line);
+    decor_provider_end_col = decor_providers_setup(endrow - startrow,
+                                                   start_vcol == 0,
+                                                   lnum,
+                                                   col,
+                                                   wp);
+    line = ml_get_buf(wp->w_buffer, lnum);
+    ptr = line + col;
+  }
+
+  decor_redraw_line(wp, lnum - 1, &decor_state);
+  if (!has_decor && decor_has_more_decorations(&decor_state, lnum - 1)) {
+    has_decor = true;
+    extra_check = true;
+  }
+
  // Correct highlighting for cursor that can't be disabled.
  // Avoids having to check this for each character.
  if (wlv.fromcol >= 0) {
@@ -1642,6 +1655,18 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b

    bool did_decrement_ptr = false;

+    // Get next chunk of extmark highlights if previous approximation was smaller than needed.
+    if (check_decor_providers && (int)(ptr - line) >= decor_provider_end_col) {
+      int const col = (int)(ptr - line);
+      decor_provider_end_col = invoke_range_next(wp, lnum, col, 100);
+      line = ml_get_buf(wp->w_buffer, lnum);
+      ptr = line + col;
+      if (!has_decor && decor_has_more_decorations(&decor_state, lnum - 1)) {
+        has_decor = true;
+        extra_check = true;
+      }
+    }
+
    // Skip this quickly when working on the text.
    if (draw_cols) {
      if (cul_screenline) {
@@ -2740,7 +2765,7 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b
      // Highlight 'cursorcolumn' & 'colorcolumn' past end of the line.

      // check if line ends before left margin
-      wlv.vcol = MAX(wlv.vcol, start_col + wlv.col - win_col_off(wp));
+      wlv.vcol = MAX(wlv.vcol, start_vcol + wlv.col - win_col_off(wp));
      // Get rid of the boguscols now, we want to draw until the right
      // edge for 'cursorcolumn'.
      wlv.col -= wlv.boguscols;
@@ -2762,7 +2787,7 @@ int win_line(win_T *wp, linenr_T lnum, int startrow, int endrow, int col_rows, b

      if (((wp->w_p_cuc
            && wp->w_virtcol >= vcol_hlc(wlv) - eol_hl_off
-            && wp->w_virtcol < view_width * (ptrdiff_t)(wlv.row - startrow + 1) + start_col
+            && wp->w_virtcol < view_width * (ptrdiff_t)(wlv.row - startrow + 1) + start_vcol
            && lnum != wp->w_cursor.lnum)
           || wlv.color_cols || wlv.line_attr_lowprio || wlv.line_attr
           || wlv.diff_hlf != 0 || wp->w_buffer->terminal)) {
@@ -3180,3 +3205,50 @@ static void wlv_put_linebuf(win_T *wp, const winlinevars_T *wlv, int endcol, boo
  ScreenGrid *g = grid_adjust(grid, &row, &coloff);
  grid_put_linebuf(g, row, coloff, startcol, endcol, clear_width, bg_attr, 0, wlv->vcol - 1, flags);
 }
+
+static int decor_providers_setup(int rows_to_draw, bool draw_from_line_start, linenr_T lnum,
+                                 colnr_T col, win_T *wp)
+{
+  // Approximate the number of bytes that will be drawn.
+  // Assume we're dealing with 1-cell ascii and ignore
+  // the effects of 'linebreak', 'breakindent', etc.
+  int rem_vcols;
+  if (wp->w_p_wrap) {
+    int width = wp->w_view_width - win_col_off(wp);
+    int width2 = width + win_col_off2(wp);
+
+    int first_row_width = draw_from_line_start ? width : width2;
+    rem_vcols = first_row_width + (rows_to_draw - 1) * width2;
+  } else {
+    rem_vcols = wp->w_view_height - win_col_off(wp);
+  }
+
+  // Call it here since we need to invalidate the line pointer anyway.
+  decor_providers_invoke_line(wp, lnum - 1);
+  validate_virtcol(wp);
+
+  return invoke_range_next(wp, lnum, col, rem_vcols + 1);
+}
+
+/// @return New begin column, or INT_MAX.
+static int invoke_range_next(win_T *wp, int lnum, colnr_T begin_col, colnr_T col_off)
+{
+  char const *const line = ml_get_buf(wp->w_buffer, lnum);
+  int const line_len = ml_get_buf_len(wp->w_buffer, lnum);
+  col_off = MAX(col_off, 1);
+
+  colnr_T new_col;
+  if (col_off <= line_len - begin_col) {
+    int end_col = begin_col + col_off;
+    end_col += mb_off_next(line, line + end_col);
+    decor_providers_invoke_range(wp, lnum - 1, begin_col, lnum - 1, end_col);
+    validate_virtcol(wp);
+    new_col = end_col;
+  } else {
+    decor_providers_invoke_range(wp, lnum - 1, begin_col, lnum, 0);
+    validate_virtcol(wp);
+    new_col = INT_MAX;
+  }
+
+  return new_col;
+}
--- a/src/nvim/lua/treesitter.c
+++ b/src/nvim/lua/treesitter.c
@@ -1361,34 +1361,46 @@ static int tslua_push_querycursor(lua_State *L)

  TSQuery *query = query_check(L, 2);
  TSQueryCursor *cursor = ts_query_cursor_new();
+
+  if (lua_gettop(L) >= 3 && !lua_isnil(L, 3)) {
+    luaL_argcheck(L, lua_istable(L, 3), 3, "table expected");
+  }
+
+  lua_getfield(L, 3, "start_row");
+  uint32_t start_row = (uint32_t)luaL_checkinteger(L, -1);
+  lua_pop(L, 1);
+
+  lua_getfield(L, 3, "start_col");
+  uint32_t start_col = (uint32_t)luaL_checkinteger(L, -1);
+  lua_pop(L, 1);
+
+  lua_getfield(L, 3, "end_row");
+  uint32_t end_row = (uint32_t)luaL_checkinteger(L, -1);
+  lua_pop(L, 1);
+
+  lua_getfield(L, 3, "end_col");
+  uint32_t end_col = (uint32_t)luaL_checkinteger(L, -1);
+  lua_pop(L, 1);
+
+  ts_query_cursor_set_point_range(cursor, (TSPoint){ start_row, start_col },
+                                  (TSPoint){ end_row, end_col });
+
+  lua_getfield(L, 3, "max_start_depth");
+  if (!lua_isnil(L, -1)) {
+    uint32_t max_start_depth = (uint32_t)luaL_checkinteger(L, -1);
+    ts_query_cursor_set_max_start_depth(cursor, max_start_depth);
+  }
+  lua_pop(L, 1);
+
+  lua_getfield(L, 3, "match_limit");
+  if (!lua_isnil(L, -1)) {
+    uint32_t match_limit = (uint32_t)luaL_checkinteger(L, -1);
+    ts_query_cursor_set_match_limit(cursor, match_limit);
+  }
+  lua_pop(L, 1);
+
  ts_query_cursor_exec(cursor, query, node);

-  if (lua_gettop(L) >= 3) {
-    uint32_t start = (uint32_t)luaL_checkinteger(L, 3);
-    uint32_t end = lua_gettop(L) >= 4 ? (uint32_t)luaL_checkinteger(L, 4) : MAXLNUM;
-    ts_query_cursor_set_point_range(cursor, (TSPoint){ start, 0 }, (TSPoint){ end, 0 });
-  }
-
-  if (lua_gettop(L) >= 5 && !lua_isnil(L, 5)) {
-    luaL_argcheck(L, lua_istable(L, 5), 5, "table expected");
-    lua_pushnil(L);  // [dict, ..., nil]
-    while (lua_next(L, 5)) {
-      // [dict, ..., key, value]
-      if (lua_type(L, -2) == LUA_TSTRING) {
-        char *k = (char *)lua_tostring(L, -2);
-        if (strequal("max_start_depth", k)) {
-          uint32_t max_start_depth = (uint32_t)lua_tointeger(L, -1);
-          ts_query_cursor_set_max_start_depth(cursor, max_start_depth);
-        } else if (strequal("match_limit", k)) {
-          uint32_t match_limit = (uint32_t)lua_tointeger(L, -1);
-          ts_query_cursor_set_match_limit(cursor, match_limit);
-        }
-      }
-      // pop the value; lua_next will pop the key.
-      lua_pop(L, 1);  // [dict, ..., key]
-    }
-  }
-
  TSQueryCursor **ud = lua_newuserdata(L, sizeof(*ud));  // [node, query, ..., udata]
  *ud = cursor;
  lua_getfield(L, LUA_REGISTRYINDEX, TS_META_QUERYCURSOR);  // [node, query, ..., udata, meta]
--- a/test/benchmark/decor_spec.lua
+++ b/test/benchmark/decor_spec.lua
@@ -8,7 +8,7 @@ describe('decor perf', function()
  it('can handle long lines', function()
    Screen.new(100, 101)

-    local result = exec_lua [==[
+    local result = exec_lua(function()
      local ephemeral_pattern = {
        { 0, 4, 'Comment', 11 },
        { 0, 3, 'Keyword', 12 },
@@ -61,7 +61,7 @@ describe('decor perf', function()
          return true
        end,
        on_line = function()
-            add_pattern(ephemeral_pattern, true)
+          add_pattern(ephemeral_pattern, true)
        end,
      })

@@ -69,16 +69,16 @@ describe('decor perf', function()

      local total = {}
      local provider = {}
-      for i = 1, 100 do
+      for _ = 1, 100 do
        local tic = vim.uv.hrtime()
-        vim.cmd'redraw!'
+        vim.cmd 'redraw!'
        local toc = vim.uv.hrtime()
        table.insert(total, toc - tic)
        table.insert(provider, pe - ps)
      end

      return { total, provider }
-    ]==]
+    end)

    local total, provider = unpack(result)
    table.sort(total)
@@ -137,4 +137,39 @@ describe('decor perf', function()
    )
    print('\nTotal ' .. res)
  end)
+
+  it('can handle long lines with treesitter highlighting', function()
+    Screen.new(100, 51)
+
+    local result = exec_lua(function()
+      local long_line = 'local a = { ' .. ('a = 5, '):rep(2000) .. '}'
+      vim.api.nvim_buf_set_lines(0, 0, 0, false, { long_line })
+      vim.api.nvim_win_set_cursor(0, { 1, 0 })
+      vim.treesitter.start(0, 'lua')
+
+      local total = {}
+      for _ = 1, 50 do
+        local tic = vim.uv.hrtime()
+        vim.cmd 'redraw!'
+        local toc = vim.uv.hrtime()
+        table.insert(total, toc - tic)
+      end
+
+      return { total }
+    end)
+
+    local total = unpack(result)
+    table.sort(total)
+
+    local ms = 1 / 1000000
+    local res = string.format(
+      'min, 25%%, median, 75%%, max:\n\t%0.1fms,\t%0.1fms,\t%0.1fms,\t%0.1fms,\t%0.1fms',
+      total[1] * ms,
+      total[1 + math.floor(#total * 0.25)] * ms,
+      total[1 + math.floor(#total * 0.5)] * ms,
+      total[1 + math.floor(#total * 0.75)] * ms,
+      total[#total] * ms
+    )
+    print('\nTotal ' .. res)
+  end)
 end)
--- a/test/benchmark/treesitter_spec.lua
+++ b/test/benchmark/treesitter_spec.lua
@@ -1,10 +1,11 @@
 local n = require('test.functional.testnvim')()
+local Screen = require('test.functional.ui.screen')

 local clear = n.clear
 local exec_lua = n.exec_lua

 describe('treesitter perf', function()
-  setup(function()
+  before_each(function()
    clear()
  end)

@@ -47,4 +48,144 @@ describe('treesitter perf', function()
      return vim.uv.hrtime() - start
    ]]
  end)
+
+  local function test_long_line(_pos, _wrap, _line, grid)
+    local screen = Screen.new(20, 11)
+
+    local result = exec_lua(function(...)
+      local pos, wrap, line = ...
+
+      vim.api.nvim_buf_set_lines(0, 0, 0, false, { line })
+      vim.api.nvim_win_set_cursor(0, pos)
+      vim.api.nvim_set_option_value('wrap', wrap, { win = 0 })
+
+      vim.treesitter.start(0, 'lua')
+
+      local total = {}
+      for _ = 1, 100 do
+        local tic = vim.uv.hrtime()
+        vim.cmd 'redraw!'
+        local toc = vim.uv.hrtime()
+        table.insert(total, toc - tic)
+      end
+
+      return { total }
+    end, _pos, _wrap, _line)
+
+    screen:expect({ grid = grid or '' })
+
+    local total = unpack(result)
+    table.sort(total)
+
+    local ms = 1 / 1000000
+    local res = string.format(
+      'min, 25%%, median, 75%%, max:\n\t%0.2fms,\t%0.2fms,\t%0.2fms,\t%0.2fms,\t%0.2fms',
+      total[1] * ms,
+      total[1 + math.floor(#total * 0.25)] * ms,
+      total[1 + math.floor(#total * 0.5)] * ms,
+      total[1 + math.floor(#total * 0.75)] * ms,
+      total[#total] * ms
+    )
+    print('\nTotal ' .. res)
+  end
+
+  local long_line = 'local a = { ' .. ('a = 5, '):rep(500) .. '}'
+  it('can redraw the beginning of a long line with wrapping', function()
+    local grid = [[
+      {15:^local} {25:a} {15:=} {16:{} {25:a} {15:=} {26:5}{16:,} {25:a}|
+       {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} |
+      {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,}|
+       {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}|
+      {16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} |
+      {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=}|
+       {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} |
+      {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a}|
+       {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} |
+      {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,}|
+                          |
+    ]]
+    test_long_line({ 1, 0 }, true, long_line, grid)
+  end)
+
+  it('can redraw the middle of a long line with wrapping', function()
+    local grid = [[
+      {1:<<<}{26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} |
+      {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,}|
+       {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}|
+      {16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} |
+      {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=}|
+       {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} |
+      {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a}|
+       {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} |
+      {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,}|
+       {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a}^ {15:=} {26:5}|
+                          |
+    ]]
+    test_long_line({ 1, math.floor(#long_line / 2) }, true, long_line, grid)
+  end)
+
+  it('can redraw the end of a long line with wrapping', function()
+    local grid = [[
+      {1:<<<}{25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=}|
+       {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} |
+      {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a}|
+       {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} |
+      {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,}|
+       {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}|
+      {16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} |
+      {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=}|
+       {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {25:a} |
+      {15:=} {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {16:^}}       |
+                          |
+    ]]
+    test_long_line({ 1, #long_line - 1 }, true, long_line, grid)
+  end)
+
+  it('can redraw the beginning of a long line without wrapping', function()
+    local grid = [[
+      {15:^local} {25:a} {15:=} {16:{} {25:a} {15:=} {26:5}{16:,} {25:a}|
+                          |
+      {1:~                   }|*8
+                          |
+    ]]
+    test_long_line({ 1, 0 }, false, long_line, grid)
+  end)
+
+  it('can redraw the middle of a long line without wrapping', function()
+    local grid = [[
+      {16:,} {25:a} {15:=} {26:5}{16:,} {25:a}^ {15:=} {26:5}{16:,} {25:a} {15:=} |
+                          |
+      {1:~                   }|*8
+                          |
+    ]]
+    test_long_line({ 1, math.floor(#long_line / 2) }, false, long_line, grid)
+  end)
+
+  it('can redraw the end of a long line without wrapping', function()
+    local grid = [[
+      {26:5}{16:,} {25:a} {15:=} {26:5}{16:,} {16:^}}         |
+                          |
+      {1:~                   }|*8
+                          |
+    ]]
+    test_long_line({ 1, #long_line - 1 }, false, long_line, grid)
+  end)
+
+  local long_line_mb = 'local a = { ' .. ('À = 5, '):rep(500) .. '}'
+  it('can redraw the middle of a long line with multibyte characters', function()
+    local grid = [[
+      {1:<<<}{26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} |
+      {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,}|
+       {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}|
+      {16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} |
+      {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=}|
+       {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} |
+      {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À}|
+       {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} |
+      {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,}|
+       {25:À} {15:=} {26:5}{16:,} {25:À} {15:=} {26:5}{16:,} {25:À}^ {15:=} {26:5}|
+                          |
+    ]]
+    test_long_line({ 1, math.floor(#long_line_mb / 2) }, true, long_line_mb, grid)
+  end)
 end)
--- a/test/functional/treesitter/query_spec.lua
+++ b/test/functional/treesitter/query_spec.lua
@@ -747,6 +747,47 @@ void ui_refresh(void)
    eq({ { 1, 10, 1, 13 } }, ret)
  end)

+  it('iter_captures supports columns', function()
+    local txt = table.concat({
+      'int aaa = 1, bbb = 2;',
+      'int foo = 1, bar = 2;',
+      'int baz = 3, qux = 4;',
+      'int ccc = 1, ddd = 2;',
+    }, '\n')
+
+    local function test(opts)
+      local parser = vim.treesitter.get_string_parser(txt, 'c')
+
+      local nodes = {}
+      local query = vim.treesitter.query.parse('c', '((identifier) @foo)')
+      local root = assert(parser:parse()[1]:root())
+      local iter = query:iter_captures(root, txt, 1, 2, opts)
+
+      while true do
+        local capture, node = iter()
+        if not capture then
+          break
+        end
+        table.insert(nodes, { node:range() })
+      end
+
+      return nodes
+    end
+
+    local ret
+    ret = exec_lua(test, { start_col = 7, end_col = 13 })
+    eq({ { 1, 13, 1, 16 }, { 2, 4, 2, 7 } }, ret)
+
+    ret = exec_lua(test, { start_col = 7 })
+    eq({ { 1, 13, 1, 16 } }, ret)
+
+    ret = exec_lua(test, { end_col = 13 })
+    eq({ { 1, 4, 1, 7 }, { 1, 13, 1, 16 }, { 2, 4, 2, 7 } }, ret)
+
+    ret = exec_lua(test, {})
+    eq({ { 1, 4, 1, 7 }, { 1, 13, 1, 16 } }, ret)
+  end)
+
  it('fails to load queries', function()
    local function test(exp, cquery)
      eq(exp, pcall_err(exec_lua, "vim.treesitter.query.parse('c', ...)", cquery))
--- a/test/functional/ui/decorations_spec.lua
+++ b/test/functional/ui/decorations_spec.lua
@@ -28,39 +28,43 @@ local function setup_provider(code)
  ]]) .. [[
    api.nvim_set_decoration_provider(_G.ns1, {
      on_start = on_do; on_buf = on_do;
-      on_win = on_do; on_line = on_do;
+      on_win = on_do; on_line = on_do; on_range = on_do;
      on_end = on_do; _on_spell_nav = on_do;
    })
    return _G.ns1
  ]])
 end

+local function setup_screen(screen)
+  screen:set_default_attr_ids {
+    [1] = { bold = true, foreground = Screen.colors.Blue },
+    [2] = { foreground = Screen.colors.Grey100, background = Screen.colors.Red },
+    [3] = { foreground = Screen.colors.Brown },
+    [4] = { foreground = Screen.colors.Blue1 },
+    [5] = { foreground = Screen.colors.Magenta },
+    [6] = { bold = true, foreground = Screen.colors.Brown },
+    [7] = { background = Screen.colors.Gray90 },
+    [8] = { bold = true, reverse = true },
+    [9] = { reverse = true },
+    [10] = { italic = true, background = Screen.colors.Magenta },
+    [11] = { foreground = Screen.colors.Red, background = tonumber('0x005028') },
+    [12] = { foreground = tonumber('0x990000') },
+    [13] = { background = Screen.colors.LightBlue },
+    [14] = { background = Screen.colors.WebGray, foreground = Screen.colors.DarkBlue },
+    [15] = { special = Screen.colors.Blue, undercurl = true },
+    [16] = { special = Screen.colors.Red, undercurl = true },
+    [17] = { foreground = Screen.colors.Red },
+    [18] = { bold = true, foreground = Screen.colors.SeaGreen },
+    [19] = { bold = true },
+  }
+end
+
 describe('decorations providers', function()
  local screen ---@type test.functional.ui.screen
  before_each(function()
    clear()
    screen = Screen.new(40, 8)
-    screen:set_default_attr_ids {
-      [1] = { bold = true, foreground = Screen.colors.Blue },
-      [2] = { foreground = Screen.colors.Grey100, background = Screen.colors.Red },
-      [3] = { foreground = Screen.colors.Brown },
-      [4] = { foreground = Screen.colors.Blue1 },
-      [5] = { foreground = Screen.colors.Magenta },
-      [6] = { bold = true, foreground = Screen.colors.Brown },
-      [7] = { background = Screen.colors.Gray90 },
-      [8] = { bold = true, reverse = true },
-      [9] = { reverse = true },
-      [10] = { italic = true, background = Screen.colors.Magenta },
-      [11] = { foreground = Screen.colors.Red, background = tonumber('0x005028') },
-      [12] = { foreground = tonumber('0x990000') },
-      [13] = { background = Screen.colors.LightBlue },
-      [14] = { background = Screen.colors.WebGray, foreground = Screen.colors.DarkBlue },
-      [15] = { special = Screen.colors.Blue, undercurl = true },
-      [16] = { special = Screen.colors.Red, undercurl = true },
-      [17] = { foreground = Screen.colors.Red },
-      [18] = { bold = true, foreground = Screen.colors.SeaGreen },
-      [19] = { bold = true },
-    }
+    setup_screen(screen)
  end)

  local mulholland = [[
@@ -110,12 +114,19 @@ describe('decorations providers', function()
      { 'start', 4 },
      { 'win', 1000, 1, 0, 6 },
      { 'line', 1000, 1, 0 },
+      { 'range', 1000, 1, 0, 0, 1, 0 },
      { 'line', 1000, 1, 1 },
+      { 'range', 1000, 1, 1, 0, 2, 0 },
      { 'line', 1000, 1, 2 },
+      { 'range', 1000, 1, 2, 0, 3, 0 },
      { 'line', 1000, 1, 3 },
+      { 'range', 1000, 1, 3, 0, 4, 0 },
      { 'line', 1000, 1, 4 },
+      { 'range', 1000, 1, 4, 0, 5, 0 },
      { 'line', 1000, 1, 5 },
+      { 'range', 1000, 1, 5, 0, 6, 0 },
      { 'line', 1000, 1, 6 },
+      { 'range', 1000, 1, 6, 0, 7, 0 },
      { 'end', 4 },
    }

@@ -137,6 +148,7 @@ describe('decorations providers', function()
      { 'buf', 1, 5 },
      { 'win', 1000, 1, 0, 6 },
      { 'line', 1000, 1, 6 },
+      { 'range', 1000, 1, 6, 0, 7, 0 },
      { 'end', 5 },
    }
  end)
@@ -206,9 +218,13 @@ describe('decorations providers', function()
      { 'start', 5 },
      { 'win', 1000, 1, 0, 3 },
      { 'line', 1000, 1, 0 },
+      { 'range', 1000, 1, 0, 0, 1, 0 },
      { 'line', 1000, 1, 1 },
+      { 'range', 1000, 1, 1, 0, 2, 0 },
      { 'line', 1000, 1, 2 },
+      { 'range', 1000, 1, 2, 0, 3, 0 },
      { 'line', 1000, 1, 3 },
+      { 'range', 1000, 1, 3, 0, 4, 0 },
      { 'end', 5 },
    }

@@ -806,6 +822,126 @@ describe('decorations providers', function()
    ]])
  end)

+  it('on_range is invoked on all visible characters', function()
+    clear()
+    screen = Screen.new(20, 4)
+    setup_screen(screen)
+
+    local function record()
+      exec_lua(function()
+        _G.p_min = { math.huge, math.huge }
+        _G.p_max = { -math.huge, -math.huge }
+        function _G.pos_gt(a, b)
+          return a[1] > b[1] or (a[1] == b[1] and a[2] > b[2])
+        end
+        function _G.pos_lt(a, b)
+          return a[1] < b[1] or (a[1] == b[1] and a[2] < b[2])
+        end
+      end)
+      setup_provider [[
+        local function on_do(kind, _, bufnr, br, bc, er, ec)
+          if kind == 'range' then
+            local b = { br, bc }
+            local e = { er, ec }
+            if _G.pos_gt(_G.p_min, b) then
+              _G.p_min = b
+            end
+            if _G.pos_lt(_G.p_max, e) then
+              _G.p_max = e
+            end
+          end
+        end
+      ]]
+    end
+    local function check(min, max)
+      local p_min = exec_lua('return _G.p_min')
+      assert(
+        p_min[1] < min[1] or (p_min[1] == min[1] and p_min[2] <= min[2]),
+        'minimum position ' .. vim.inspect(p_min) .. ' should be before the first char'
+      )
+      local p_max = exec_lua('return _G.p_max')
+      assert(
+        p_max[1] > max[1] or (p_max[1] == max[1] and p_max[2] >= max[2]),
+        'maximum position ' .. vim.inspect(p_max) .. ' should be on or after the last char'
+      )
+    end
+
+    -- Multiple lines.
+    exec_lua([[
+      local lines = { ('a'):rep(40), ('b'):rep(40), ('c'):rep(40) }
+      vim.api.nvim_buf_set_lines(0, 0, -1, true, lines)
+      vim.api.nvim_win_set_cursor(0, { 2, 0 })
+    ]])
+    record()
+    screen:expect([[
+      ^bbbbbbbbbbbbbbbbbbbb|
+      bbbbbbbbbbbbbbbbbbbb|
+      ccccccccccccccccc{1:@@@}|
+                          |
+    ]])
+    check({ 1, 0 }, { 2, 21 })
+
+    -- One long line.
+    exec_lua([[
+      local lines = { ('a'):rep(100) }
+      vim.api.nvim_buf_set_lines(0, 0, -1, true, lines)
+      vim.api.nvim_win_set_cursor(0, { 1, 70 })
+    ]])
+    record()
+    screen:expect([[
+      {1:<<<}aaaaaaaaaaaaaaaaa|
+      aaaaaaaaaaaaaaaaaaaa|
+      aaaaaaaaaa^aaaaaaaaaa|
+                          |
+    ]])
+    check({ 0, 20 }, { 0, 81 })
+
+    -- Multibyte characters.
+    exec_lua([[
+      local lines = { ('\195\162'):rep(100) }
+      vim.api.nvim_buf_set_lines(0, 0, -1, true, lines)
+      vim.api.nvim_win_set_cursor(0, { 1, 70 * 2 })
+    ]])
+    record()
+    screen:expect([[
+      {1:<<<}âââââââââââââââââ|
+      ââââââââââââââââââââ|
+      ââââââââââ^ââââââââââ|
+                          |
+    ]])
+    check({ 0, 20 * 2 }, { 0, 81 * 2 })
+
+    -- Tabs.
+    exec_lua([[
+      local lines = { 'a' .. ('\t'):rep(100) }
+      vim.api.nvim_buf_set_lines(0, 0, -1, true, lines)
+      vim.api.nvim_win_set_cursor(0, { 1, 39 })
+    ]])
+    record()
+    screen:expect([[
+      {1:<<<}                 |
+                          |
+                 ^         |
+                          |
+    ]])
+    check({ 0, 33 }, { 0, 94 })
+
+    -- One long line without wrapping.
+    command('set nowrap')
+    exec_lua([[
+      local lines = { ('a'):rep(50) .. ('b'):rep(50) }
+      vim.api.nvim_buf_set_lines(0, 0, -1, true, lines)
+      vim.api.nvim_win_set_cursor(0, { 1, 50 })
+    ]])
+    record()
+    screen:expect([[
+      aaaaaaaaaa^bbbbbbbbbb|
+      {1:~                   }|*2
+                          |
+    ]])
+    check({ 0, 40 }, { 0, 60 })
+  end)
+
  it('can add new providers during redraw #26652', function()
    setup_provider [[
    local ns = api.nvim_create_namespace('test_no_add')
Author	SHA1	Message	Date
bfredl	772f1966a3	Merge pull request #31400 from vanaigr/decor-provider-range feat(decor): add range-based highlighting	2025-08-29 10:33:15 +02:00
vanaigr	5edbabdbec	perf: add on_range in treesitter highlighting	2025-08-28 08:22:38 -05:00
vanaigr	118e7e7111	test: add treesitter long lines benchmark	2025-08-28 08:22:38 -05:00