feat(provider): detect venv python via "pynvim-python" tool #35273

Problem:
Detection of the pynvim module is currently done by finding the first
Python interpreter in the `PATH` and checking if it can import pynvim.
This has several problems:
- Activation of an unrelated Python virtual environment will break
  automatic detection, unless pynvim is also installed in that
  environment.
- Installing pynvim to the expected location is difficult. User
  installation into the system-wide or user-wide Python site area is now
  deprecated.  On Ubuntu 24.04 with Python 3.12, for example, the
  command `pip install --user pynvim` now fails with the error message
  `error: externally-managed-environment`.
- Users may create a dedicated virtual environment in which to install
  pynvim, but Nvim won't detect it; instead, they must either activate
  it before launching Nvim (which interferes with the user of other
  virtual environments) or else hard-code the variable
  `g:python3_host_prog` in their `init.vim` to the path of the correct
  Python interpreter.  Neither option is desirable.

Solution:
Expose pynvim's Python interpreter on the `PATH` under the
name `pynvim-python`.  Typical user-flow:

1. User installs either uv or pipx.
2. User installs pynvim via:
   ```
   uv tool install --upgrade pynvim
   # Or:
   pipx install --upgrade pynvim
   ```

With corresponding changes in pynvim https://github.com/neovim/pynvim/issues/593
the above user-flow is all that's needed for Nvim to detect the
installed location of pynvim, even if an unrelated Python virtual
environments is activated.  It uses standard Python tooling to automate
the necessary creation of a Python virtual environment for pyenv and the
publication of `pynvim-python` to a directory on `PATH`.

INSTALL.md

@@ -66,11 +66,13 @@ Several Neovim GUIs are available from scoop (extras): [scoop.sh/#/apps?q=neovim
   You can then copy your spell files over (for English, located
   [here](https://github.com/vim/vim/blob/master/runtime/spell/en.utf-8.spl) and
   [here](https://github.com/vim/vim/blob/master/runtime/spell/en.utf-8.sug));
-- For Python plugins you need the `pynvim` module. "Virtual envs" are recommended. After activating the virtual env do `pip install pynvim` (in *both*). Edit your `init.vim` so that it contains the path to the env's Python executable:
-    ```vim
-    let g:python3_host_prog='C:/Users/foo/Envs/neovim3/Scripts/python.exe'
+- For Python plugins you need the `pynvim` module. Installation via uv
+  (https://docs.astral.sh/uv/) is recommended; the `--upgrade` switch ensures
+  installation of the latest version:
     ```
-    - Run `:checkhealth` and read `:help provider-python`.
+    uv tool install --upgrade pynvim
+    ```
+    - Run `:checkhealth` and read `:help provider-python` for more details.
 - **init.vim ("vimrc"):** If you already have Vim installed you can copy `%userprofile%\_vimrc` to `%userprofile%\AppData\Local\nvim\init.vim` to use your Vim config with Neovim.
 
 

runtime/doc/faq.txt

@@ -187,20 +187,19 @@ Run |:checkhealth| in Nvim for automatic diagnosis.
 
 Other hints:
 
-- The python `neovim` module was renamed to `pynvim` (long ago).
-- If you're using pyenv or virtualenv for the `pynvim` module
-    https://pypi.org/project/pynvim/, you must set `g:python3_host_prog` to
-    the virtualenv's interpreter path.
-- Read |provider-python|.
+- Read |provider-python| to learn how to install `pynvim`.
 - Be sure you have the latest version of the `pynvim` Python module: >bash
 
-    python -m pip install setuptools
-    python -m pip install --upgrade pynvim
-    python3 -m pip install --upgrade pynvim
+    uv tool install --upgrade pynvim
 <
+  See |provider-python| for other installation options.
+- If you're manually creating a Python virtual environment for the `pynvim` module
+    https://pypi.org/project/pynvim/, you must set `g:python3_host_prog` to
+    the virtualenv's interpreter path.
 - Try with `nvim -u NORC` to make sure your config (|init.vim|) isn't causing a
     problem. If you get `E117: Unknown function`, that means there's a runtime
     issue: |faq-runtime|.
+- The python `neovim` module was renamed to `pynvim` (long ago).
 
 
 :CHECKHEALTH REPORTS E5009: INVALID $VIMRUNTIME ~

runtime/doc/news.txt

@@ -277,6 +277,11 @@ PLUGINS
 • Customize :checkhealth by handling a `FileType checkhealth` event.
   |health-usage|
 
+• Simplify Python provider setup to a single step: `uv tool install pynvim`
+  Nvim will detect the plugin's location without user configuration, even if
+  unrelated Python virtual environments are activated.
+  |provider-python|
+
 STARTUP
 
 • todo

runtime/doc/provider.txt

@@ -36,21 +36,18 @@ itself).
 
 For Python 3 plugins:
 1. Make sure Python 3.9+ is available in your $PATH.
-2. Install the module (try "python" if "python3" is missing): >bash
-   python3 -m pip install --user --upgrade pynvim
+2. Install either uv (https://docs.astral.sh/uv/) or pipx
+   (https://pipx.pypa.io/stable/).
+3. Install the module: >bash
+   uv tool install --upgrade pynvim
+   # or:
+   pipx install --upgrade pynvim
 
-The pip `--upgrade` flag ensures that you get the latest version even if
+The `--upgrade` flag ensures that you get the latest version even if
 a previous version was already installed.
 
 See also |python-virtualenv|.
 
-Note: The old "neovim" module was renamed to "pynvim".
-https://github.com/neovim/neovim/wiki/Following-HEAD#20181118
-If you run into problems, uninstall _both_ then install "pynvim" again: >bash
-  python -m pip uninstall neovim pynvim
-  python -m pip install --user --upgrade pynvim
-
-
 PYTHON PROVIDER CONFIGURATION ~
 						*g:python3_host_prog*
 Command to start Python 3 (executable, not directory). Setting this makes
@@ -65,20 +62,18 @@ To disable Python 3 support: >vim
 
 PYTHON VIRTUALENVS ~
 						*python-virtualenv*
-If you plan to use per-project virtualenvs often, you should assign one
-virtualenv for Nvim and hard-code the interpreter path via
-|g:python3_host_prog| so that the "pynvim" package is not required
-for each virtualenv.
-
-Example using pyenv: >bash
-    pyenv install 3.4.4
-    pyenv virtualenv 3.4.4 py3nvim
-    pyenv activate py3nvim
-    python3 -m pip install pynvim
-    pyenv which python  # Note the path
-The last command reports the interpreter path, add it to your init.vim: >vim
-    let g:python3_host_prog = '/path/to/py3nvim/bin/python'
+Using pynvim 0.6.0+ installed via uv or pipx, Nvim will automatically detect
+pynvim even if other Python virtual environments are activated (technical
+note: via the "pynvim-python" global python tool).  For older pynvim (or older
+Neovim), where detection involved finding the first Python interpreter and
+checking if it could import pynvim, automatic detection would fail when
+another virtual environment is active.  Upgrading to the latest pynvim is the
+recommended solution to this; but if that's not an option, then you can set
+the variable |g:python3_host_prog| in `init.vim` to point to the full path to
+the Python interpreter where `pynvim` is installed, e.g.: >vim
 
+  let g:python3_host_prog = '/path/to/pynvim-venv/bin/python'
+<
 See also: https://github.com/zchee/deoplete-jedi/wiki/Setting-up-Python-for-Neovim
 
 ==============================================================================

runtime/lua/vim/provider/python.lua

@@ -83,6 +83,10 @@ function M.detect_by_module(module)
     return vim.fn.exepath(vim.fn.expand(python_exe, true)), nil
   end
 
+  if vim.fn.executable('pynvim-python') == 1 then
+    return 'pynvim-python'
+  end
+
   local errors = {}
   for _, exe in ipairs(python_candidates) do
     local error = check_for_module(exe, module)