Problem: Lockfile can become out of sync with what is actually installed
on disk when user performs (somewhat reasonable) manual actions like:
- Delete lockfile and expect it to regenerate.
- Delete plugin directory without `vim.pack.del()`.
- Manually edit lock data in a bad way.
Solution: Synchronize lockfile data with installed plugins on every
lockfile read. In particular:
1. Install immediately all missing plugins with valid lock data.
This helps with "manually delete plugin directory" case by
prompting user to figure out how to properly delete a plugin.
2. Repair lock data for properly installed plugins.
This helps with "manually deleted lockfile", "manually edited
lockfile in an unexpected way", "installation terminated due to
timeout" cases.
3. Remove unrepairable corrupted lock data and their plugins. This
includes bad lock data for missing plugins and any lock data
for corrupted plugins (right now this only means that plugin
path is not a directory, but can be built upon).
Step 1 also improves usability in case there are lazy loaded plugins
that are rarely loaded (like on `FileType` event, for example):
- Previously starting with config+lockfile on a new machine only
installs rare `vim.pack.add()` plugin after it is called (while
an entry in lockfile would still be present). This could be
problematic if there is no Internet connection, for example.
- Now all plugins from the lockfile are installed before actually
executing the first `vim.pack.add()` call in 'init.lua'. And later
they are only loaded on a rare `vim.pack.add()` call.
---
Synchronizing lockfile on its every read makes it work more robustly
if other `vim.pack` functions are called without any `vim.pack.add()`.
---
Performance for a regular startup (good lockfile, everything is
installed) is not affected and usually even increased. The bottleneck
in this area is figuring out which plugins need to be installed.
Previously the check was done by `vim.uv.fs_stat()` for every plugin
in `vim.pack.add()`. Now it is replaced with a single `vim.fs.dir()`
traversal during lockfile sync while later using lockfile data to
figure out if plugin needs to be installed.
The single `vim.fs.dir` approach scales better than `vim.uv.fs_stat`,
but might be less performant if there are many plugins that will be
not loaded via `vim.pack.add()` during startup.
Rough estimate of how long the same steps (read lockfile and normalize
plugin array) take with a single `vim.pack.add()` filled with 43
plugins benchmarking:
- Before commit: ~700 ms
- After commit: ~550 ms
Problem: Currently it is possible to have plugin in a "partial install"
state when `git clone` was successfull but `git checkout` was not.
This was done to not checkout default branch by default in these
situations (for security reasons).
The problem is that it adds complexity when both dealing with lockfile
(plugin's `rev` might be `nil`) and in how `src` and `version` are
treated (wrong `src` - no plugin on disk; wrong `version` - "partial"
plugin on disk).
Solution: Treat plugin as "installed" if both `git clone` and
`git checkout` are successful, while ensuring that not installed
plugins are not on disk and in lockfile.
This also means that if in 'init.lua' there is a `vim.pack.add()` with
bad `version`, for first install there will be an informative error
about it BUT next session will also try to install it. The solution is
the same - adjust `version` beforehand.
Problem: No example workflow of how to revert after a bad update.
Solution: Add example workflow of how to revert after a bad update.
In future this might be improved by utilizing other `vim.pack`
features or via a dedicated function (like `vim.pack.restore()` that
restores all installed plugins to a state from the lockfile).
Problem: Changing `src` of an existing plugin cleanly requires manual
`vim.pack.del()` prior to executing `vim.pack.add()` with a new `src`.
Solution: Autodetect `src` change for an existing plugin (by comparing
against lockfile data). If different - properly delete immediately and
treat this as new plugin installation.
Alternative solution might be to update `origin` remote in the
installed plugin after calling `vim.pack.update()`. Although, doable,
this 1) requires more code; and 2) works only for Git plugins (which
might be not the only type of plugins in the future). Automatic
"delete and clean install" feels more robust.
Problem: Plain `vim.pack.add()` calls (with default `opts.load`) does
not fully work if called inside 'plugin/' runtime directory. In
particular, 'plugin/' files of newly added plugins are not sourced.
This is because `opts.load` is `false` during the whole startup, which
means `:packadd!` is used (modify 'runtimepath' but not force source
newly added 'plugin/' files).
This use case is common due to users organizing their config as
separate files in '~/.config/nvim/plugin/'.
Solution: Use newly added `v:vim_did_init` to decide default `opts.load`
value instead of `v:vim_did_enter`.
Problem: Current requirement is Git>=2.36 as `--also-filter-submodules`
flag for `git clone` was introduced there. This is problematic since
default Git version on Ubuntu 22.04 is 2.34.
Solution: Relax minimal Git version to be (at least) 2.0 by selectively
applying necessary flags based on the current Git version.
As 2.0.0 was released in 2014-05-28 (almost the same age as Neovim
project itself), it is reasonable to drop any mention and checks on
minimal version altogether.
Problem: In some areas plugin's revision is named "state". This might be
confusing for the users.
Solution: Consistently use "revision" to indicate "plugin's state on
disk".
Problem: Inside `PackChanged[Pre]` callbacks it might be useful to tell
if the affected plugin is active or not. It is already possible via
extra `vim.pack.get({ 'plug-name' })[1].active`, but it is not quite
user-friendly for something that might be needed frequently in real
world use cases.
Solution: Supply extra `active` event data field.
Use double sentence spacing and wrap lines at 'textwidth'. Code
examples and tables were not wrapped unless this had already been done
locally.
closes: vim/vim#18453c58f91c035
Fix incorrect docs in :h ModeChanged.
Cherry-pick :h bufnr() changes from patch 8.1.2080.
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Problem: No way to granularly operate on plugins when inside
confirmation buffer.
Solution: Implement code actions for in-process LSP that act on "plugin
at cursor":
- Update (if has updates).
- Skip updating (if has updates).
- Delete.
Activate via default `gra` or `vim.lsp.buf.code_action()`.
Problem: Installing plugin always pulls latest `version` changes
(usually from the default branch or "latest version tag"). It is more
robust to prefer initial installation to use the latest recorded
(i.e. "working") revision.
Solution: Prefer using revision from the lockfile (if present) during
install. The extra `update()` will pull the latest changes.
Problem: Running `update()` by default doesn't include not active
plugins, because there was no way to get relevant `version` to get
updates from. This might be a problem in presence of lazy loaded
plugins, i.e. ones that can be "not *yet* active" but still needed to
be updated.
Solution: Include not active plugins by default since their `version` is
tracked via lockfile.
Problem: The revision data is returned behind `opts.info` flag because
it required extra Git calls. With lockfile it is not the case.
Solution: Use lockfile to always set `rev` field in output of `get()`.
Problem: Some use cases require or benefit from persistent on disk
storage of plugin data (a.k.a. "lockfile"):
1. Allow `update()` to act on not-yet-active plugins. Currently if
`add()` is not yet called, then plugin's version is unknown
and `update()` can't decide where to look for changes.
2. Efficiently know plugin's dependencies without having to read
'pkg.json' files on every load for every plugin. This is for the
future, after there is `packspec` support (or other declaration of
dependencies on plugin's side).
3. Allow initial install to check out the exact latest "working" state
for a reproducible setup. Currently it pulls the latest available
`version.`
4. Ensure that all declared plugins are installed, even if lazy loaded.
So that later `add()` does not trigger auto-install (when there
might be no Internet connection, for example) and there is no issues
with knowing which plugins are used in the config (so even never
loaded rare plugins are still installed and can be updated).
5. Allow `add()` to detect if plugin's spec has changed between
Nvim sessions and act accordingly. I.e. either set new `src` as
origin or enforce `version.` This is not critical and can be done
during `update()`, but it might be nice to have.
Solution: Add lockfile in JSON format that tracks (adds, updtes,
removes) necessary data for described use cases. Here are the required
data that enables each point:
1. `name` -> `version` map.
2. `name` -> `dependencies` map.
3. `name` -> `rev` map. Probably also requires `name` -> `src` map
to ensure that commit comes from correct origin.
4. `name` -> `src` map. It would be good to also track the order,
but that might be too many complications and redundant together
with point 2.
5. Map from `name` to all relevant spec fields. I.e. `name` -> `src`
and `name` -> `version` for now. Storing data might be too much,
but can be discussed, of course.
This commit only adds lockfile tracking without implementing actual
use cases. It is stored in user's config directory and is suggested to
be tracked via version control.
Example of a lockfile:
```json
{
# Extra nesting to more future proof.
"plugins": {
"plug-a": {
"ref": "abcdef1"
"src": "https://github.com/user/plug-a",
# No `version` means it was `nil` (infer default branch later)
},
"plug-b": {
"dependencies": ["plugin-a", "plug-c"],
"src": "https://github.com/user/plug-b",
"ref": "bcdefg2",
# Enclose string `version` in quotes
"version": "'dev'"
},
"plug-c": {
"src": "https://github.com/user/plug-c",
"ref": "cdefgh3",
# Store `vim.version.Range` via its `tostring()` output
"version": ">=0.0.0",
}
}
}
```
Problem:
Force resolve `spec.version` overrides the information about whether
a user supplied `version` or not. Knowing it might be useful in some use
cases (like comparing to previously set `spec` to detect if it has
changed).
Solution:
Do not resolve `spec.version`. This also improves speed when triggering
events and calling `get()`.
- Place default branch first when listing all branches.
- Use correct terminology in `get_hash` helper.
- Do not return `{ '' }` if there are no tags.
Problem:
There is no way to get more information about installed plugins, like
current revision or default branch (necessary if resolving default
`spec.version` manually). As computing Git data migth take some time,
also allow `get()` to limit output to only necessary set of plugins.
Solution:
- introduce arguments to `get(names, opts)`, which follows other
`vim.pack` functions. Plugin extra info is returned by default and
should be opt-out via `opts.info = false`.
- Examples:
- Get current revision: `get({ 'plug-name' })[1].rev`
- Get default branch: `get({ 'plug_name' })[1].branches[1]`
- `update()` and `del()` act on plugins in the same order their names
are supplied. This is less surprising.
- default `opts.info` to `true` since this simplifies logic for the
common user, while still leaving the door open for a faster `get()` if
needed.
Problem: Some use cases might lead to `vim.pack.add()` failing to
`:packadd` a plugin because of missing entry in 'packpath'. Like with
`nvim --clean` or manually setting `$XDG_DATA_HOME` during startup.
Solution: Document it. A more proactive approach can be ensuring correct
'packpath' entry, but it is currently somewhat verbose to do (due to
having to adjust for Windows using `\` in 'packpath' entries).
Problem:
The load function in opts was difficult to use if you wished to
customize based on the plugin being loaded.
You could get the name, but without some way to mark a spec, that was of
limited usefulness unless you wanted to hardcode a list of names in the
function, or write a wrapper around the whole thing
Solution:
Allow users to provide an arbitrary data field in plugin specs so that
they may receive info as to how to handle that plugin in load, get() and
events, and act upon it
Co-authored-by: BirdeeHub <birdee@localhost>
Co-authored-by: Evgeni Chasnovski <evgeni.chasnovski@gmail.com>
Problem: No way to skip install confirmation in `add()`. Having install
confirmation by default is a more secure design. However, users are
usually aware of the fact that plugin will be installed and there is
currently no way to skip confirmation.
Plus it can introduce inconvenience on the clean config initialization
if it is modularized with many `vim.pack.add()` calls (leads to
confirming installation many times in a row).
Solution: Add `opts.confirm` option that can skip install confirmation.
Problem: No way to have full control over how plugin is loaded.
Although `:packadd!` has small side effects (only adds plugin
directory to 'runtimepath'; and maybe its 'after/' subdirectory), it
still has side effects. For example, 'plugin/' directories are still
loaded during startup (as part of `:h load-plugins`).
Solution: Allow function `opts.load` that has full control over how
plugin is loaded.
Problem: the `load=true` in `vim.pack.add()` means that `:packadd` is
executed even during startup. This leads to force source of 'plugin/',
which breaks the intended loading order (`:h load-plugins`) and
results into sourcing them twice. This also makes it ignore
`--noplugin` argument.
Using `:packadd!` during startup is more appropriate, while `:packadd`
afterwards is still more favorable to actually force 'plugin/' source
(as there is no pre-defined mechanism that will load them later).
Solution: have `load=false` default during startup, `true` - afterwards.
Problem: Both `PackChangedPre` and `PackChanged` contain |event-data|
with plugin's `spec`. It looks like a good idea to have all its
triggers contain the same format across all kinds ("install",
"update", "delete"). There are several choices:
- Have it be as verbatim as supplied to `vim.pack.add()`, i.e. can
be either string or table. A bit too ambiguous.
- Have it be table with `src` and `name` inferred. This requires
less work for "install", but more work for "update" and "delete"
(since they use `vim.pack.get()` which already infers default
`version`).
- Have it be table with *all* defaults made explicit. This looks
like the best approach, but requires extra care to only infer
default `version` when needed (i.e. avoid inferring during regular
load) because it is costly in terms of startup time.
This might also introduce inconsistency when dealing with
lockfile(s) as information there should be as close to what user
supplied as possible. Address that when dealing with lockfile.
Solution: Ensure explicit `version` in all events where possible.
