Understanding nvidia-drm.modeset=1 (NVIDIA Linux driver modesetting)

I have a desktop PC with the only GPU which is NVIDIA’s 1660 Ti.

I’ve enabled modesetting via

/etc/modprobe.d/nvidia.conf                                                                                                                                                                                                          
options nvidia-drm modeset=1

However I don’t feel or see any difference. My text terminal is still running its initial EFI resolution (I guess it’s 720p or something - hard to say), it takes a few seconds to switch between an X.org session and text console.

/proc/fb
0 simple

and dmesg says:

Console: colour dummy device 80x25
printk: console [tty0] enabled
pci 0000:08:00.0: vgaarb: setting as boot VGA device
pci 0000:08:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
pci 0000:08:00.0: vgaarb: bridge control possible
vgaarb: loaded
simple-framebuffer simple-framebuffer.0: framebuffer at 0xe1000000, 0x300000 bytes
simple-framebuffer simple-framebuffer.0: format=a8r8g8b8, mode=1024x768x32, linelength=4096
fbcon: Deferring console take-over
simple-framebuffer simple-framebuffer.0: fb0: simplefb registered!
fbcon: Taking over console
Console: switching to colour frame buffer device 128x48
systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
nvidia: loading out-of-tree module taints kernel.
nvidia: module license 'NVIDIA' taints kernel.
nvidia-nvlink: Nvlink Core is being initialized, major device number 248
nvidia 0000:08:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
NVRM: loading NVIDIA UNIX x86_64 Kernel Module  510.47.03  Mon Jan 24 22:58:54 UTC 2022
nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  510.47.03  Mon Jan 24 22:51:43 UTC 2022
[drm] [nvidia-drm] [GPU ID 0x00000800] Loading driver
caller os_map_kernel_space.part.0+0x6d/0x90 [nvidia] mapping multiple BARs
[drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:08:00.0 on minor 0

The only difference is that I now cannot unload the nvidia kernel module which was possible before.

What am I doing wrong or misunderstanding? I thought the text console would switch to my native monitor resolution and switching between it and X.org will be near instant.

The kernel seemingly supports all the required features:

$ grep CONFIG_DRM config | egrep -v "^#"
CONFIG_DRM=y
CONFIG_DRM_KMS_HELPER=y
CONFIG_DRM_FBDEV_EMULATION=y
CONFIG_DRM_FBDEV_OVERALLOC=300
CONFIG_DRM_PANEL=y
CONFIG_DRM_BRIDGE=y
CONFIG_DRM_PANEL_BRIDGE=y
CONFIG_DRM_PANEL_ORIENTATION_QUIRKS=y

I’m not using initrd/initramfs.

Found this post: Kernel mode setting does not work in CONSOLE mode under linux - #4 by aplattner is it still relevant? It’s kinda sad if it is.

And then there’s SIMPLEDRM which should work and I’m getting lost: 510.39.01 on 5.16.0 kernel green screen - #19 by BCS

It’s not enabled for my Linux config.

(Talking about x86 here, keep in mind other arches exist)
The console/VT is still the old linux framebuffer console (fbcon)
https://www.kernel.org/doc/html/latest/fb/fbcon.html
which runs on a simple framebuffer device (fbdev).
The fbdev would usually be provided by the bios’ vga or efi framebuffer (over some in-kernel fb driver).
DRM drivers have the possibility to provide a drm based fbdev emulation, like in i915:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/i915/display/intel_fbdev.c?h=v5.17-rc5
Which can be turned on/off by CONFIG_DRM_FBDEV_EMULATION=y
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/i915/display/intel_fbdev.h?h=v5.17-rc5
NB: the kconfig setting does not provide a general drm fbdev emulation but only enables it in the gpu specific drm driver, if available. You’ll then see in dmesg:

fb0: switching to inteldrmfb from EFI VGA

The nvidia drm driver does not provide a drm fbdev emulation so the fbcon will stay on e.g. efifb.
In general, there are more-or-less ongoing works/rfcs over the past decade or so to get rid of fbdev/fbcon/CONFIG_VT altogether and replace them with something drm only. Some recent discussion, which provides a good insight:
https://patchwork.kernel.org/project/linux-fbdev/cover/20210827100027.1577561-1-javierm@redhat.com/#24417083

simpledrm is rather the opposite of this, it provides a minimal drm driver running on the bios provided framebuffer (EFI/VGA) to have a fallback driver for wayland in case no gpu specific drm driver is available. See it as a vesa driver for wayland.

1 Like

Setting modeset=1 doesn’t actually install a framebuffer console. All it really does is enable the DRIVER_MODESET capability flag in the nvidia-drm devices so that DRM clients can use the various modesetting APIs. In addition to allowing clients that talk to the low-level DRM interface to work, it’s also necessary for some PRIME-related interoperability features.

The downside, if you want to call it that, is that loading nvidia-drm with modeset=1 causes it to configure and initialize all GPUs immediately rather than waiting for a client to open the /dev/nvidia* device files. This means that some options that require a userspace application to configure them before the GPUs are initialized won’t work if they were already configured by nvidia-drm. The big example at the moment is SLI Mosaic, which is enabled by the X driver if /etc/X11/xorg.conf says it should be.

3 Likes

Which should also be noted in the Mosaic chapter of the manual since often this happens:
https://forums.developer.nvidia.com/t/i-can-no-longer-can-set-up-a-system-with-cloned-mirrored-monitors-using-2-quadro-k420-cards/204359?u=generix
It’s noted in the drm kms chapter but nobody that’s affected finds it there.

1 Like

@aplattner

Thanks!

Are there any plans (I know I know you cannot talk about the future) to enable fb emulation in NVIDIA drivers, so that I could run my text console at a native resolution?

BTW, I’ve still noticed a single positive change: resume is now quite faster. Previously it took 2-3 seconds, now I get my Xorg’s unlock screen almost instantly after I resume. I wonder if it’s real or I’m imagining things.

Can’t mark two replies as answers so I’ll just say big thank you!

Not currently, since installing a framebuffer console pins the kernel modules so they can’t be unloaded. We still (for better or worse) support shutting down GPU clients, unloading the kernel modules, installing a different driver, and starting everything back up without rebooting. If nvidia-drm is always in use because it’s driving a framebuffer console, then that won’t work.

It’s theoretically possible to remove the framebuffer console device in order to unload nvidia-drm.ko, but there’s no mechanism for the system to switch back to something like simplefb or efifb so the console just goes blank if you do that.

2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.