Problem: nil error possible if a loaded buffer hasn't been drawn in a window:
```vim
lua vim.lsp.document_color.is_enabled() -- Load module
badd foo
call bufload('foo')
colo default
```
Solution: Skip _buf_refresh branch also if bufstates[bufnr] is nil.
d976834864