460.73.01, Failed to acquire modesetting permission

Graphics / Linux Linux
1

Xorg server output
(WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
(II) NVIDIA(0): Creating default Display subsection in Screen section
“Default Screen Section” for depth/fbbpp 24/32
(==) NVIDIA(0): Depth 24, (==) framebuffer bpp 32
(==) NVIDIA(0): RGB weight 888
(==) NVIDIA(0): Default visual is TrueColor
(==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
(**) NVIDIA(0): Enabling 2D acceleration
(II) NVIDIA: The X server supports PRIME Render Offload.
(–) NVIDIA(0): Valid display device(s) on GPU-0 at PCI:1:0:0
(–) NVIDIA(0): DFP-0
(–) NVIDIA(0): DFP-1 (boot)
(–) NVIDIA(0): DFP-2
(–) NVIDIA(0): DFP-3
(–) NVIDIA(0): DFP-4
(II) NVIDIA(0): NVIDIA GPU GeForce GTX 1050 Ti (GP107-A) at PCI:1:0:0 (GPU-0)
(–) NVIDIA(0): Memory: 4194304 kBytes
(–) NVIDIA(0): VideoBIOS: 86.07.39.00.8b
(II) NVIDIA(0): Detected PCI Express Link width: 16X
(EE) NVIDIA(GPU-0): Failed to acquire modesetting permission.
(EE) NVIDIA(0): Failing initialization of X screen
(II) UnloadModule: “nvidia”

How do debug the problem
‘Failed to acquire modesetting permission.’

I installed nvidia drivers by hands
unpack installation kit
NVIDIA-Linux-x86_64-460.73.01.run -x
cd NVIDIA-Linux-x86_64-460.73.01
cd kernel
make
su - # log into root
make modules_install
modprobe nvidia

then I copy libs by hands
/usr/lib64/xorg/modules/drivers/
rename to be version specific, then I might change only link for another version of driver

nvidia_drv.so → nvidia_drv.so.460.73.01

/usr/lib64/xorf/modules/extension/nvidia
libglxserver_nvidia.so → libglxserver_nvidia.so.460.73.01
libglxserver_nvidia.so.460.73.01
libglx.so → libglxserver_nvidia.so.460.73.01

/usr/lib64/
libnvidia-glcore.so.460.73.01 → /usr/lib64/nvidia/460.73.01/libnvidia-glcore.so.460.7
libnvidia-tls.so.460.73.01 → /usr/lib64/nvidia/460.73.01/libnvidia-tls.so.460.73.01
sitting in X as startx with mate-desktop.

/usr/lib64/nvidia/460.73.01
All other libs from nvidia installation kit
but I don’t set LD_LIBRARY_PATH here while
because for starting xorg I expect it is enough libs cited above.

Instead of restart xorg I’m trying xorg audio specific config in order to enable audio for nvidia hdmi and as test to load
nvidia module into xorg server.

cat xorg.conf.nvidia.audio
Section “ServerLayout”
Identifier “Layout0”
Option “AutoAddDevices” “false”
Option “AutoAddGPU” “false”
Option “IsolateDevice” “PCI:01:00:0”
# this is nvidia audio device
# Option “IsolateDevice” “PCI:01:00:1”
EndSection

Section “Device”
Identifier “DiscreteNvidia”
Driver “nvidia”
VendorName “NVIDIA Corporation”
EndSection

Section “Files”
ModulePath “/usr/lib64/xorg/modules/extensions/nvidia,/usr/lib64/xorg/modules,/usr/lib64/modules/extensions,/usr/lib64/xorg/modules/input”
EndSection

This scheme worked in old version of driver.
It even worked for version 460.73.01
But it fails after I tried 465.19.01 from cuda 11.3 then revert back to version 460.73.01.

Reason why I don’t use nvidia-installer because it might override
some libs with the same name what are in Fedora Core 32
breaks my existed xorg config and startx stops working.
After then I need to revert back such libs to libfs from FC32 distribution. It is pain to compare libs and their sizes in order to understand what was overwritten.
That is why I prefer to install nvidia drivers by hands.

I do googling but not found something reasonable about
‘Failed to acquire modesetting permission’.
what is the root cause and hence what might fix it.

If nvidia xorg extension would work ok then after startx
there’s no need for second xorg server for audio over nvidia hdmi (instead of monitor on hdmi is AV receiver) and might to probe PRIME.
In past I used bumblebee.

cat /proc/cmdline
BOOT_IMAGE=(hd2,gpt3)/vmlinuz-5.11.15-100.fc32.x86_64 root=UUID=4461fbcc-73f1-4905-816b-2082bea70cc6 ro quiet rd.driver.blacklist=nouveau nvidia-drm.modeset=1

journalctl -b | grep -i nvidia
Apr 24 17:38:23 nzasf kernel: Command line: BOOT_IMAGE=(hd2,gpt3)/vmlinuz-5.11.15-100.fc32.x86_64 root=UUID=4461fbcc-73f1-4905-816b-2082bea70cc6 ro quiet rd.driver.blacklist=nouveau nvidia-drm.modeset=1
Apr 24 17:38:23 nzasf kernel: Kernel command line: BOOT_IMAGE=(hd2,gpt3)/vmlinuz-5.11.15-100.fc32.x86_64 root=UUID=4461fbcc-73f1-4905-816b-2082bea70cc6 ro quiet rd.driver.blacklist=nouveau nvidia-drm.modeset=1
Apr 24 17:38:23 nzasf dracut-cmdline[276]: Using kernel command line parameters: BOOT_IMAGE=(hd2,gpt3)/vmlinuz-5.11.15-100.fc32.x86_64 root=UUID=4461fbcc-73f1-4905-816b-2082bea70cc6 ro quiet rd.driver.blacklist=nouveau nvidia-drm.modeset=1
Apr 24 17:38:25 nzasf kernel: nvidia: loading out-of-tree module taints kernel.
Apr 24 17:38:25 nzasf kernel: nvidia: module license ‘NVIDIA’ taints kernel.
Apr 24 17:38:25 nzasf kernel: nvidia: module verification failed: signature and/or required key missing - tainting kernel
Apr 24 17:38:25 nzasf kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 235
Apr 24 17:38:25 nzasf kernel: nvidia 0000:01:00.0: enabling device (0000 → 0003)
Apr 24 17:38:25 nzasf kernel: nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
Apr 24 17:38:25 nzasf kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 460.73.01 Thu Apr 1 21:40:36 UTC 2021
Apr 24 17:38:25 nzasf kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 460.73.01 Thu Apr 1 21:32:31 UTC 2021
Apr 24 17:38:25 nzasf kernel: [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
Apr 24 17:38:25 nzasf kernel: input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input17
Apr 24 17:38:25 nzasf kernel: input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input18
Apr 24 17:38:25 nzasf kernel: input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input19
Apr 24 17:38:25 nzasf kernel: input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input20
Apr 24 17:38:25 nzasf kernel: input: HDA NVidia HDMI/DP,pcm=10 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input21
Apr 24 17:38:25 nzasf kernel: input: HDA NVidia HDMI/DP,pcm=11 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input22
Apr 24 17:38:25 nzasf kernel: input: HDA NVidia HDMI/DP,pcm=12 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input23
Apr 24 17:38:27 nzasf kernel: [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 1
Apr 24 18:14:17 nzasf kernel: nvidia_uvm: module uses symbols from proprietary module nvidia, inheriting taint.
Apr 24 18:14:17 nzasf kernel: nvidia-uvm: Loaded the UVM driver, major device number 511.
Apr 24 18:35:09 nzasf dbus-daemon[5033]: [session uid=0 pid=5031] Activating service name=‘ca.desrt.dconf’ requested by ‘:1.0’ (uid=0 pid=5026 comm="pluma --new-window id.nz.devtalknvidia " label=“unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023”)

lsmod | grep nvidia
nvidia_uvm 1146880 0
nvidia_drm 65536 1
nvidia_modeset 1232896 1 nvidia_drm
nvidia 34168832 2 nvidia_uvm,nvidia_modeset
drm_kms_helper 286720 2 nvidia_drm,i915
drm 622592 13 drm_kms_helper,nvidia_drm,i915

dmesg | grep -i nvidia
[ 0.000000] Command line: BOOT_IMAGE=(hd2,gpt3)/vmlinuz-5.11.15-100.fc32.x86_64 root=UUID=4461fbcc-73f1-4905-816b-2082bea70cc6 ro quiet rd.driver.blacklist=nouveau nvidia-drm.modeset=1
[ 0.152978] Kernel command line: BOOT_IMAGE=(hd2,gpt3)/vmlinuz-5.11.15-100.fc32.x86_64 root=UUID=4461fbcc-73f1-4905-816b-2082bea70cc6 ro quiet rd.driver.blacklist=nouveau nvidia-drm.modeset=1
[ 8.965346] nvidia: loading out-of-tree module taints kernel.
[ 8.965361] nvidia: module license ‘NVIDIA’ taints kernel.
[ 8.972457] nvidia: module verification failed: signature and/or required key missing - tainting kernel
[ 8.999180] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[ 9.001666] nvidia 0000:01:00.0: enabling device (0000 → 0003)
[ 9.001775] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
[ 9.232553] NVRM: loading NVIDIA UNIX x86_64 Kernel Module 460.73.01 Thu Apr 1 21:40:36 UTC 2021
[ 9.246631] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 460.73.01 Thu Apr 1 21:32:31 UTC 2021
[ 9.262757] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
[ 9.263069] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input17
[ 9.263121] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input18
[ 9.263170] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input19
[ 9.263216] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input20
[ 9.263248] input: HDA NVidia HDMI/DP,pcm=10 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input21
[ 9.263275] input: HDA NVidia HDMI/DP,pcm=11 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input22
[ 9.263301] input: HDA NVidia HDMI/DP,pcm=12 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input23
[ 10.330015] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 1
[ 2160.262552] nvidia_uvm: module uses symbols from proprietary module nvidia, inheriting taint.
[ 2160.294892] nvidia-uvm: Loaded the UVM driver, major device number 511.

2

I have compiled cuda 11.3 example
./deviceQueryDrv
and it is still working
export LD_LIBRARY_PATH=/usr/lib64/nvidia/460.73.01
/deviceQueryDrv
./deviceQueryDrv Starting…

CUDA Device Query (Driver API) statically linked version
Detected 1 CUDA Capable device(s)

Device 0: “GeForce GTX 1050 Ti”
CUDA Driver Version: 11.2
CUDA Capability Major/Minor version number: 6.1
Total amount of global memory: 4040 MBytes (4236312576 bytes)
( 6) Multiprocessors, (128) CUDA Cores/MP: 768 CUDA Cores
GPU Max Clock rate: 1418 MHz (1.42 GHz)
Memory Clock rate: 3504 Mhz
Memory Bus Width: 128-bit
L2 Cache Size: 1048576 bytes
Max Texture Dimension Sizes 1D=(131072) 2D=(131072, 65536) 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Texture alignment: 512 bytes
Maximum memory pitch: 2147483647 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Concurrent kernel execution: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device supports Managed Memory: Yes
Device supports Compute Preemption: Yes
Supports Cooperative Kernel Launch: Yes
Supports MultiDevice Co-op Kernel Launch: Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
Result = PASS

3

Cancelled. It seems issue concerns correct xorg.conf.

cat /etc/X11/xorg.conf.d/nvidia_offline.conf

#try also without ver2 , not sure it is necessary
# but I cite working config
#ver 2
Section “ServerLayout”
Identifier “layout”
Screen 0 “intel”
Inactive “nvidia”
EndSection

Section “Module”
Load “dri2”
Load “glamoregl”
EndSection

Section “Device”
Identifier “intel”
Driver “intel”
Option “AccelMethod” “glamor”
EndSection

Section “Device”
Identifier “nvidia”
Driver “nvidia”
BusID “PCI:01:00:0”
# nvidia audio device
BusID “PCI:01:00:1”
Option “AccelMethod” “glamor”
EndSection

# ver 2
Section “Screen”
Identifier “intel”
Device “intel”
EndSection

xrandr --listproviders
Providers: number : 2
Provider 0: id: 0x49 cap: 0xb, Source Output, Sink Output, Sink Offload crtcs: 4 outputs: 6 associated providers: 1 name:Intel
Provider 1: id: 0x224 cap: 0x2, Sink Output crtcs: 4 outputs: 5 associated providers: 1 name:NVIDIA-G0

export _NV_PRIME_RENDER_OFFLOAD=1
export __GLX_VENDOR_LIBRARY_NAME=nvidia
export _NV_PRIME_RENDER_OFFLOAD_PROVIDER=‘NVIDIA-G0’
glxinfo | grep vendor
server glx vendor string: NVIDIA Corporation
client glx vendor string: NVIDIA Corporation
OpenGL vendor string: NVIDIA Corporation

export __GLX_VENDOR_LIBRARY_NAME=intel
glxinfo | grep vendor
server glx vendor string: SGI
client glx vendor string: Mesa Project and SGI
OpenGL vendor string: Intel