nvidiafb is loading, but it should be blacklisted.
Please look for nvidia blacklist files:
grep -r nvidia /etc/modprobe.d /lib/modprobe.d
if you do find a file with the entry: blacklist nvidia, that entry should be removed.
if you do NOT find a file with the entry blacklist nvidiafb, you should create one (ie /etc/modprobe.de/blacklist-nvidiafb.conf) with exactly that content.
If any of this applies, then run sudo update-initramfs -u and reboot.