Ubuntu 22.04: OBS using llvmpipe instead of Nvidia GPU

So I have a headless server and use xrdp to connect to the system. I tried running OBS both using the apt install and flatpak way but both show llvmpipe instead of the Nvidia driver being used in the logs which results in high CPU usage and frame drops obviously.

OBS logs show:

10:44:10.997: Using EGL/X11
10:44:10.997: CPU Name: Intel Core Processor (Broadwell, no TSX, IBRS)
10:44:10.997: CPU Speed: 2793.436MHz
10:44:10.997: Physical Cores: 1, Logical Cores: 2
10:44:10.997: Physical Memory: 3905MB Total, 430MB Free
10:44:10.997: Kernel Version: Linux 5.15.0-88-generic
10:44:10.997: Flatpak Branch: stable
10:44:10.997: Flatpak Arch: x86_64
10:44:10.997: Flatpak Runtime: runtime/org.kde.Platform/x86_64/6.5
10:44:10.997: App Extensions:
10:44:10.997:  - com.obsproject.Studio.Locale
10:44:10.997: Runtime Extensions:
10:44:10.997:  - org.freedesktop.Platform.GL.default
10:44:10.997:  - org.freedesktop.Platform.GL.nvidia-525-125-06
10:44:10.997:  - org.freedesktop.Platform.openh264
10:44:10.997:  - org.gtk.Gtk3theme.Greybird
10:44:10.997:  - org.kde.PlatformTheme.QGnomePlatform
10:44:10.997:  - org.kde.WaylandDecoration.QAdwaitaDecorations
10:44:10.997:  - org.kde.WaylandDecoration.QGnomePlatform-decoration
10:44:10.997:  - org.kde.Platform.Locale
10:44:10.997:  - org.freedesktop.Platform.GL.default
10:44:10.997: Flatpak Framework Version: 1.14.4
10:44:10.997: Desktop Environment: ubuntu:GNOME
10:44:10.997: Session Type: x11
10:44:10.997: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.21.1
10:44:10.998: Qt Version: 6.5.3 (runtime), 6.5.3 (compiled)
10:44:10.998: Portable mode: false
10:44:11.029: OBS 30.0.0 (linux)
10:44:11.029: ---------------------------------
10:44:11.029: ---------------------------------
10:44:11.029: audio settings reset:
10:44:11.029: 	samples per sec: 48000
10:44:11.029: 	speakers:        2
10:44:11.029: 	max buffering:   960 milliseconds
10:44:11.029: 	buffering type:  dynamically increasing
10:44:11.033: ---------------------------------
10:44:11.033: Initializing OpenGL...
10:44:11.083: Loading up OpenGL on adapter Mesa llvmpipe (LLVM 15.0.7, 256 bits)
10:44:11.083: OpenGL loaded successfully, version 4.5 (Core Profile) Mesa 23.1.9 (git-acb5ee278a), shading language 4.50
10:44:11.117: ---------------------------------
10:44:11.117: video settings reset:
10:44:11.117: 	base resolution:   1920x1080
10:44:11.117: 	output resolution: 1920x1080
10:44:11.117: 	downscale filter:  Bicubic
10:44:11.117: 	fps:               60/1
10:44:11.117: 	format:            NV12
10:44:11.117: 	YUV mode:          Rec. 709/Partial
10:44:11.117: NV12 texture support not available
10:44:11.117: P010 texture support not available
10:44:11.129: Audio monitoring device:
10:44:11.129: 	name: Default
10:44:11.129: 	id: default
10:44:11.129: ---------------------------------
10:44:11.146: No AJA devices found, skipping loading AJA UI plugin
10:44:11.146: Failed to initialize module 'aja-output-ui.so'
10:44:11.172: No AJA devices found, skipping loading AJA plugin
10:44:11.172: Failed to initialize module 'aja.so'
10:44:11.177: Failed to load 'en-US' text for module: 'decklink-captions.so'
10:44:11.184: Failed to load 'en-US' text for module: 'decklink-output-ui.so'
10:44:11.188: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
10:44:11.188: Failed to initialize module 'decklink.so'
10:44:11.381: [pipewire] No captures available
10:44:11.410: [obs-browser]: Version 2.22.1
10:44:11.410: [obs-browser]: CEF Version 103.0.5060.134 (runtime), 103.0.0-5060-shared-textures.2594+g17f8588+chromium-103.0.5060.134 (compiled)
10:44:11.418: NVENC supported
10:44:11.418: VAAPI: Failed to initialize display in vaapi_device_h264_supported
10:44:11.418: FFmpeg VAAPI H264 encoding not supported
10:44:11.418: VAAPI: Failed to initialize display in vaapi_device_hevc_supported
10:44:11.418: FFmpeg VAAPI HEVC encoding not supported
10:44:11.473: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.3.3 | RPC Version: 1)
10:44:11.473: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.5.3 | Qt version (run-time): 6.5.3
10:44:11.473: [obs-websocket] [obs_module_load] Linked ASIO Version: 102800
10:44:11.478: [obs-websocket] [obs_module_load] Module loaded.
10:44:11.494: ---------------------------------
10:44:11.494:   Loaded Modules:
10:44:11.494:     text-freetype2.so
10:44:11.494:     rtmp-services.so
10:44:11.494:     obs-x264.so
10:44:11.494:     obs-websocket.so
10:44:11.494:     obs-webrtc.so
10:44:11.494:     obs-vst.so
10:44:11.494:     obs-transitions.so
10:44:11.494:     obs-qsv11.so
10:44:11.494:     obs-outputs.so
10:44:11.494:     obs-libfdk.so
10:44:11.494:     obs-filters.so
10:44:11.494:     obs-ffmpeg.so
10:44:11.494:     obs-browser.so
10:44:11.494:     linux-v4l2.so
10:44:11.494:     linux-pulseaudio.so
10:44:11.494:     linux-pipewire.so
10:44:11.494:     linux-jack.so
10:44:11.494:     linux-capture.so
10:44:11.494:     image-source.so
10:44:11.494:     frontend-tools.so
10:44:11.494:     decklink-output-ui.so
10:44:11.494:     decklink-captions.so
10:44:11.494: ---------------------------------
10:44:11.494: QWidget::setTabOrder: 'first' and 'second' must be in the same window
10:44:11.494: ==== Startup complete ===============================================
10:44:11.520: All scene data cleared
10:44:11.520: ------------------------------------------------
10:44:11.522: pulse-input: Server name: 'pulseaudio 15.99.1'
10:44:11.522: pulse-input: Audio format: s16le, 44100 Hz, 2 channels
10:44:11.522: pulse-input: Started recording from 'auto_null.monitor' (default)
10:44:11.522: [Loaded global audio device]: 'Desktop Audio'
10:44:11.523: [Media Source 'BGLoop']: settings:
10:44:11.523: 	input:                   /home/stream/Downloads/pexels_videos_1093662 (1080p).mp4
10:44:11.523: 	input_format:            (null)
10:44:11.523: 	speed:                   100
10:44:11.523: 	is_looping:              yes
10:44:11.523: 	is_linear_alpha:         no
10:44:11.523: 	is_hw_decoding:          yes
10:44:11.523: 	is_clear_on_media_end:   yes
10:44:11.523: 	restart_on_activate:     yes
10:44:11.523: 	close_when_inactive:     no
10:44:11.523: 	full_decode:             no
10:44:11.523: 	ffmpeg_options:          
10:44:11.524: Switched to scene 'Scene'
10:44:11.524: ------------------------------------------------
10:44:11.524: Loaded scenes:
10:44:11.524: - scene 'Scene':
10:44:11.524:     - source: 'BGLoop' (ffmpeg_source)
10:44:11.524: ------------------------------------------------

nvidia-smi shows:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A16-1Q       On   | 00000000:06:00.0  On |                    0 |
| N/A   N/A    P0    N/A /  N/A |    307MiB /  1024MiB |      2%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1109      G   /usr/lib/xorg/Xorg                108MiB |
|    0   N/A  N/A      1538      G   /usr/lib/xorg/Xorg                 69MiB |
|    0   N/A  N/A      3412      C   obs                               129MiB |
+-----------------------------------------------------------------------------+

Not really sure what to test anymore as it seems that everything “should” be fine but it’s not.

xrdp per default uses a virtual Xserver, not the real one. You could use x11vnc to connect to the real server on the nvidia gpu and pipe it to xrdp.

Oh, that’s a great hint! I will need to try that. Thanks for the help! I think the main problem then is to get this running as the server is headless. Not sure if that is possible.

Looking again at the nvidia-smi output shows the gpu type “A16-1Q” telling you’re inside a vm running on a vGPU system so you should rather use the vGPU specific software for remote access and check with the vGPU forum
https://forums.developer.nvidia.com/c/nvidia-virtual-gpu-forums/general-discussion/337

The vgpu docs state the same, use x11vnc:
https://docs.nvidia.com/grid/10.0/grid-vgpu-user-guide/index.html#configuring-x11vnc-checking-gpu-linux-server