Dual GPU only showing mouse cursor on second display

Using Ubuntu 18.04 on a system with dual GPU (integrated ASpeed vga + Nvidia RTX 5000).
I am only able to see the desktop (and application windows) on one display, while the second display only shows the mouse cursor.

nvidia-smi
Sun Jan 3 15:59:09 2021
±----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02 Driver Version: 450.80.02 CUDA Version: 11.0 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Quadro RTX 5000 Off | 00000000:65:00.0 On | Off |
| 33% 29C P5 27W / 230W | 211MiB / 16125MiB | 22% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1497 G /usr/lib/xorg/Xorg 109MiB |
| 0 N/A N/A 15444 G /usr/bin/nvidia-settings 35MiB |
±----------------------------------------------------------------------------+

glxinfo |grep render
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
direct rendering: Yes
GLX_MESA_multithread_makecurrent, GLX_MESA_query_renderer,
GLX_EXT_visual_rating, GLX_MESA_copy_sub_buffer, GLX_MESA_query_renderer,
Extended renderer info (GLX_MESA_query_renderer):
OpenGL renderer string: llvmpipe (LLVM 10.0.0, 256 bits)
GL_ARB_conditional_render_inverted, GL_ARB_conservative_depth,
GL_MESA_ycbcr_texture, GL_NV_conditional_render, GL_NV_depth_clamp,
GL_ARB_conditional_render_inverted, GL_ARB_conservative_depth,
GL_NV_conditional_render, GL_NV_depth_clamp, GL_NV_fog_distance,
GL_EXT_polygon_offset_clamp, GL_EXT_read_format_bgra, GL_EXT_render_snorm,
GL_MESA_shader_integer_functions, GL_NV_conditional_render,
GL_OES_element_index_uint, GL_OES_fbo_render_mipmap,
direct rendering: Yes
GLX_MESA_multithread_makecurrent, GLX_MESA_query_renderer,
OpenGL renderer string: Quadro RTX 5000/PCIe/SSE2
GL_ARB_conditional_render_inverted, GL_ARB_conservative_depth,
GL_NVX_conditional_render, GL_NVX_gpu_memory_info, GL_NVX_nvenc_interop,
GL_NV_compute_shader_derivatives, GL_NV_conditional_render,
GL_NV_parameter_buffer_object2, GL_NV_path_rendering,
GL_NV_path_rendering_shared_edge, GL_NV_pixel_data_range,
GL_NV_stereo_view_rendering, GL_NV_texgen_reflection,
GL_ARB_compute_variable_group_size, GL_ARB_conditional_render_inverted,
GL_NVX_conditional_render, GL_NVX_gpu_memory_info, GL_NVX_nvenc_interop,
GL_NV_compute_shader_derivatives, GL_NV_conditional_render,
GL_NV_parameter_buffer_object2, GL_NV_path_rendering,
GL_NV_path_rendering_shared_edge, GL_NV_pixel_data_range,
GL_NV_stereo_view_rendering, GL_NV_texgen_reflection,
GL_EXT_multisample_compatibility, GL_EXT_multisampled_render_to_texture,
GL_EXT_multisampled_render_to_texture2,
GL_EXT_raster_multisample, GL_EXT_render_snorm, GL_EXT_robustness,
GL_NV_conditional_render, GL_NV_conservative_raster,
GL_NV_packed_float_linear, GL_NV_path_rendering,
GL_NV_path_rendering_shared_edge, GL_NV_pixel_buffer_object,
GL_NV_shadow_samplers_cube, GL_NV_stereo_view_rendering,
GL_OES_element_index_uint, GL_OES_fbo_render_mipmap,
GL_OVR_multiview_multisampled_render_to_texture

sudo lshw -c video
*-display
description: VGA compatible controller
product: ASPEED Graphics Family
vendor: ASPEED Technology, Inc.
physical id: 0
bus info: pci@0000:05:00.0
version: 41
width: 32 bits
clock: 33MHz
capabilities: pm msi vga_controller bus_master cap_list rom
configuration: driver=ast latency=0
resources: irq:18 memory:a4000000-a7ffffff memory:a8000000-a801ffff ioport:2000(size=128) memory:c0000-dffff
*-display
description: VGA compatible controller
product: NVIDIA Corporation
vendor: NVIDIA Corporation
physical id: 0
bus info: pci@0000:65:00.0
version: a1
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
configuration: driver=nvidia latency=0
resources: iomemory:102f0-102ef iomemory:102f0-102ef irq:71 memory:df000000-dfffffff memory:102fe0000000-102fefffffff memory:102ff0000000-102ff1ffffff ioport:b000(size=128) memory:e0000000-e007ffff

Please connect both monitors to the Quadro and try disabling the aspeed graphics in bios.

I cannot disable the on-board, just set the priority.
My goal is to use the onboard for one display (will display only the UI) and leave the Nvidia card to render to several displays with processed live image from a camera.

That won’t really work that way. Better forget about it and use only the nvidia.
You now have two Xscreens, one on the aspeed :0.0 and one on :0.1 (or maybe :1.0/:1.1)
You can start applications (or a window manager) on the Nvidia screen, e.g.
DISPLAY=:0.1 glxgears
or
DISPLAY=:1.1 glxgears
But having an application display one window on one screen and other windows on another won’t be possible that way.
You could tuck the screens together using Xinerama but then you would lose all hw accel on the nvidia, rendering it useless.

Thank you for the quick responses.
So can I have 1 application render to the on-board connected display and have another application render to multiple displays connected to the Nvidia card?

Yes, that’s possible by prepending the DISPLAY number.