Vulkan PRIME fail on dual NVIDIA gpu system

Graphics / Linux Linux
1

Hello All.
On my dual NVIDIA GPU system I can successfully use PRIME on OpenGL applications.
However, Vulkan has problems.

System Summary:
OS: Gentoo
DE: LXQT
GPUs: NVIDIA GeForce GT 1030 (this drives the monitors), NVIDIA GeForce GTX 1660 TI (this is for Prime use)

nvidia-bug-report.log.gz (1.0 MB)

When trying to run vkcube an error message is created.

$ vkcube
Selected GPU 0: NVIDIA GeForce GTX 1660 Ti, type: 2
Could not find both graphics and present queues

Although this is the GPU I would like to be used, it is not successful.

Running this same command for the GT 1030 is completely successful.

$ vkcube --gpu_number 1
Selected GPU 1: NVIDIA GeForce GT 1030, type: 2

I have attempted both of these command variations with and without the __NV_PRIME_RENDER_OFFLOAD=1 environment variable and got the same results.
Testing OpenGL using glxgears works perfectly for both cards. Which I verified using watch nvidia-smi on a separate terminal. So it appears to be some conflict between Vulkan and PRIME.

I realize that a two NVIDIA gpu system is rare, but I am completely lost on how to diagnose this failure.

2

It’s been a while that this was opened, but I have the same issue with a Thunderbolt external GPU.

3

@giovanni.beltrame

I too am using dual GTX 1080Ti’s in NVIDIA2NVIDIA Prime for a 7-display setup.

for whatever reason I haven’t needed to understand yet, only 1 GPU is able to successfully connect to the ‘graphics queue’ it likes to complain about.

I was able to solve the issue by installing this project, which just adds an additional Vulkan layer that enables choosing a specific GPU for each process.

I wrote a wrapper script that I use to launch games/wine/vkcube:

#!/usr/bin/env bash

# vkgpu.bash
# 
# wrapper to launch vulkan using correct gpu
# on dual Nvidia gpu system using Nvidia2Nvidia PRIME
# NOTE: will only work with exactly 2 dGPUs unless
# gpu number is passed manually
#
# usage: vkgpu.bash [gpuid] <vulkan executable>
# gpuid is 0 or 1
# vulkan executable is path to app to launch
#

builtin declare -x ENABLE_DEVICE_CHOOSER_LAYER=1
if [ $1 = 0 ] || [ $1 = 1 ]; then
    VULKAN_DEVICE_INDEX=${1}
    builtin shift
else
    VULKAN_DEVICE_INDEX=$(</sys/devices/pci0000:00/0000:00:01.1/0000:02:00.0/boot_vga)
    # adjust this to the sysfs path of your gpu1 + /boot_vga
    # if gpu0 is the gpu in use, this value will be 0
    # and if gpu1 is in use, this will be 1, so it
    # ends up being exactly the value needed 
fi

builtin echo Running ${@} using vulkan on GPU${VULKAN_DEVICE_INDEX}
builtin exec "${@}" 2>&1 &

You can also export those values in your ~/.bashrc so all apps launch with the correct GPU. If you do this, you can still override it on a per-command basis by prefixing the launch command with VULKAN_DEVICE_INDEX=“”, e.g.

VULKAN_DEVICE_INDEX=0 /usr/bin/wine explorer

Hope it helps, because this issue drove me nuts for months until I finally figured it out. Cheers!

4

Can confirm still happens with two gpus: NVIDIA-SMI 565.57.01
DGPU: 2070 Super
Thunderbolt gpu: 4080