NVIDIA drivers take two seconds to switch between X and text-mode VT

With nvidia-375 or nvidia-378 (and possibly earlier) and a GTX 1050 Ti on Ubuntu 16.04.1, switching between an X server and a text-mode VT (e.g. with ctrl-alt-f3) takes two seconds. This delay happens when switching in either direction. This is somewhat annoying when working with text-mode VTs and it doesn’t happen with open-source drivers for other graphics cards. (I try to work as root in a text-mode VT, not in my X server, because X’s security model allows any connected application to sniff keystrokes.) Any improvements to the switch time would be really appreciated, though I realize this is probably low-priority.

Edit: UEFI boot, no CSM, grub-efi-amd64, 3840x2160 resolution on both X and text-mode VT

The delay also happens when switching between two X servers, but not when switching between two text-mode VTs.

It switched from X driver to a VGA text mode since these Nvidia drivers don’t support framebuffer.

same here but with bios