How to offload game rendering to second Nvidia gpu?

I currently have a RTX 2070 Super and a RTX 3090. I would like to use the 2070 as the source for my displays and have it be used to do all of the desktop environment rendering. As for the 3090, I want to use it to render games to the display that the 2070 is connected to. I looked into this and PRIME seems to be what I am looking for. I tried setting it up but it doesn’t work for anything vulkan based and when I do prime-run vkcube it just gives the message: Selected GPU 0: NVIDIA GeForce RTX 3090, type: DiscreteGpu Could not find both graphics and present queues Is Nvidia to Nvidia prime possible with vulkan?

https://forums.developer.nvidia.com/t/vulkan-prime-fail-on-dual-nvidia-gpu-system/213084/3?u=generix

I tried this, and vulkan still selects the 2070 rather than the 3090. If I try selecting the gpu with --gpu_number it again gives the same error about graphics and present queues.

prime-run is a wrapper around this:

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia command

Try doing this instead of prime-run (replacing 1 with whatever number corresponds to your GPU), while also passing in the vulkan --gpu-number parameter. E.g.

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia myfavoritegame --gpu-number=1

or the steam equivalent

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia %command% --gpu-number=1

I haven’t tested this myself.

Just tried it, steam games will open a process on the 3090 but a second instance opens on the 2070 and all of the rendering happens on the 2070 (3090 gpu clock stays low and power usage is low). vkcube gives the same error.

I do not know then, perhaps another will chime in.

Either this or this should work

https://docs.mesa3d.org/envvars.html#envvar-MESA_VK_DEVICE_SELECT

Second one needs this package on Arch, don’t know the other distros.

https://archlinux.org/packages/extra/x86_64/vulkan-mesa-layers/

MESA_VK_DEVICE_SELECT=list vulkaninfo

After getting pci adress of it:

MESA_VK_DEVICE_SELECT="xx:yz" as an env var.

Running:

MESA_VK_DEVICE_SELECT=list vulkaninfo

returns:

  GPU 0: 10de:1e84 "NVIDIA GeForce RTX 2070 SUPER" discrete GPU 0000:04:00.0
  GPU 1: 10de:2204 "NVIDIA GeForce RTX 3090" discrete GPU 0000:0a:00.0

Running:

vulkaninfo --summary

returns:

==========
VULKANINFO
==========

Vulkan Instance Version: 1.3.269


Instance Extensions: count = 23
-------------------------------
VK_EXT_acquire_drm_display             : extension revision 1
VK_EXT_acquire_xlib_display            : extension revision 1
VK_EXT_debug_report                    : extension revision 10
VK_EXT_debug_utils                     : extension revision 2
VK_EXT_direct_mode_display             : extension revision 1
VK_EXT_display_surface_counter         : extension revision 1
VK_EXT_surface_maintenance1            : extension revision 1
VK_EXT_swapchain_colorspace            : extension revision 4
VK_KHR_device_group_creation           : extension revision 1
VK_KHR_display                         : extension revision 23
VK_KHR_external_fence_capabilities     : extension revision 1
VK_KHR_external_memory_capabilities    : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2         : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2       : extension revision 1
VK_KHR_portability_enumeration         : extension revision 1
VK_KHR_surface                         : extension revision 25
VK_KHR_surface_protected_capabilities  : extension revision 1
VK_KHR_wayland_surface                 : extension revision 6
VK_KHR_xcb_surface                     : extension revision 6
VK_KHR_xlib_surface                    : extension revision 6
VK_LUNARG_direct_driver_loading        : extension revision 1

Instance Layers: count = 14
---------------------------
VK_LAYER_FROG_gamescope_wsi_x86_64 Gamescope WSI (XWayland Bypass) Layer (x86_64) 1.3.221  version 1
VK_LAYER_INTEL_nullhw              INTEL NULL HW                                  1.1.73   version 1
VK_LAYER_KHRONOS_validation        Khronos Validation Layer                       1.3.268  version 1
VK_LAYER_LUNARG_api_dump           LunarG API dump layer                          1.3.250  version 2
VK_LAYER_LUNARG_monitor            Execution Monitoring Layer                     1.3.250  version 1
VK_LAYER_LUNARG_screenshot         LunarG image capture layer                     1.3.250  version 1
VK_LAYER_MESA_device_select        Linux device selection layer                   1.3.211  version 1
VK_LAYER_MESA_overlay              Mesa Overlay layer                             1.3.211  version 1
VK_LAYER_NV_optimus                NVIDIA Optimus layer                           1.3.260  version 1
VK_LAYER_VALVE_steam_fossilize_32  Steam Pipeline Caching Layer                   1.3.207  version 1
VK_LAYER_VALVE_steam_fossilize_64  Steam Pipeline Caching Layer                   1.3.207  version 1
VK_LAYER_VALVE_steam_overlay_32    Steam Overlay Layer                            1.3.207  version 1
VK_LAYER_VALVE_steam_overlay_64    Steam Overlay Layer                            1.3.207  version 1
VK_LAYER_VKBASALT_post_processing  a post processing layer                        1.3.223  version 1

Devices:
========
GPU0:
	apiVersion         = 1.3.260
	driverVersion      = 545.29.6.0
	vendorID           = 0x10de
	deviceID           = 0x1e84
	deviceType         = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
	deviceName         = NVIDIA GeForce RTX 2070 SUPER
	driverID           = DRIVER_ID_NVIDIA_PROPRIETARY
	driverName         = NVIDIA
	driverInfo         = 545.29.06
	conformanceVersion = 1.3.6.0
	deviceUUID         = b5291141-3ff6-f8bf-c85d-2f8e52fc144d
	driverUUID         = 468717c2-5245-5d6d-9401-72b8a4ff98e5
GPU1:
	apiVersion         = 1.3.260
	driverVersion      = 545.29.6.0
	vendorID           = 0x10de
	deviceID           = 0x2204
	deviceType         = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
	deviceName         = NVIDIA GeForce RTX 3090
	driverID           = DRIVER_ID_NVIDIA_PROPRIETARY
	driverName         = NVIDIA
	driverInfo         = 545.29.06
	conformanceVersion = 1.3.6.0
	deviceUUID         = b57a16e3-308d-9773-fddb-b863910a9c41
	driverUUID         = 468717c2-5245-5d6d-9401-72b8a4ff98e5

Running:

MESA_VK_DEVICE_SELECT=10de:2204 vkcube

returns:

Selected GPU 0: NVIDIA GeForce RTX 3090, type: DiscreteGpu
Could not find both graphics and present queues

The same happens when using vkdevicechooser. Could the issue be that vulkan only creates queues for the driver (which both gpus are using the same driver). I’m not too familiar with how vulkan works, I’m just trying to best make sense of the behavior.