M60 shows available vGPUs only on one GPU

Hi,

Setup: RHEV 4.2 on RHEL 7.5

We have an Nvidia M60 that shows available vGPU instances on only one of the GPUs on the hypervisor. We only have a few VMs configured to use vGPUs, so it shouldn’t account for why no instances are available. The guide mentions that 0 instances may appear if there is an ‘instance of another vGPU type already exists on the physical GPU’. What other vGPU type could exist on the GPU, and how would I tell?

The logs shows that there are vGPU instances on both GPUs (00:86:00:0 and 00:87:00:0)

Mar 6 12:20:53 dev1 kernel: nvidia: loading out-of-tree module taints kernel.
Mar 6 12:20:53 dev1 kernel: WARNING: module ‘nvidia’ built without retpoline-enabled compiler, may affect Spectre v2 mitigation
Mar 6 12:20:53 dev1 kernel: nvidia: module license ‘NVIDIA’ taints kernel.
Mar 6 12:20:53 dev1 kernel: nvidia: module verification failed: signature and/or required key missing - tainting kernel
Mar 6 12:20:53 dev1 kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 238
Mar 6 12:20:53 dev1 kernel: nvidia 0000:86:00.0: enabling device (0100 -> 0103)
Mar 6 12:20:53 dev1 kernel: nvidia 0000:87:00.0: enabling device (0100 -> 0103)
Mar 6 12:20:54 dev1 kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 410.91 Wed Dec 19 05:57:36 CST 2018 (using threaded interrupts)
Mar 6 12:20:54 dev1 systemd: Starting NVIDIA vGPU Daemon…
Mar 6 12:20:54 dev1 nvidia-vgpud: Verbose syslog connection opened
Mar 6 12:20:54 dev1 nvidia-vgpud: Started (1562)
Mar 6 12:20:54 dev1 nvidia-vgpud: Global settings:
Mar 6 12:20:54 dev1 nvidia-vgpud: Size: 16#012Version 1
Mar 6 12:20:54 dev1 nvidia-vgpud: Homogeneous vGPUs: 1
Mar 6 12:20:54 dev1 nvidia-vgpud: vGPU types: 203
Mar 6 12:20:54 dev1 nvidia-vgpud:
Mar 6 12:20:54 dev1 nvidia-vgpud: pciId of gpu [0]: 0:86:0:0
Mar 6 12:20:54 dev1 nvidia-vgpud: pciId of gpu [1]: 0:87:0:0
Mar 6 12:20:54 dev1 systemd: Starting NVIDIA vGPU Manager Daemon…
Mar 6 12:20:54 dev1 systemd: Started NVIDIA vGPU Daemon.
Mar 6 12:20:54 dev1 systemd: Started NVIDIA vGPU Manager Daemon.
Mar 6 12:20:56 dev1 nvidia-vgpu-mgr[1619]: notice: vgpu_mgr_log: nvidia-vgpu-mgr daemon started
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Physical GPU:
Mar 6 12:20:56 dev1 nvidia-vgpud: PciID: 0x0000 / 0x0086 / 0x0000 / 0x0000
Mar 6 12:20:56 dev1 nvidia-vgpud: Size: 52#012Version 1
Mar 6 12:20:56 dev1 nvidia-vgpud: DevID: 0x10de / 0x13f2 / 0x10de / 0x0000
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported vGPUs count: 13
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xb: max 16
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xb: GRID M60-0B Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x1176
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1a000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x6000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xc: max 16
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xc: GRID M60-0Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114c
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1a000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x6000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xd: max 8
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xd: GRID M60-1A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11ae
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x38000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x8000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xe: max 8
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xe: GRID M60-1B Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x1177
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x38000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x8000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xf: max 8
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xf: GRID M60-1Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114d
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x38000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x8000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x10: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x10: GRID M60-2A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11af
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x11: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x11: GRID M60-2B Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x117d
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xd2: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xd2: GRID M60-2B4 Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x12ec
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x12: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x12: GRID M60-2Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114e
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x13: max 2
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x13: GRID M60-4A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11b0
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0xec000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x14000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x14: max 2
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x14: GRID M60-4Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114f
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0xec000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x14000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x15: max 1
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x15: GRID M60-8A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11b1
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1dc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x24000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x16: max 1
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x16: GRID M60-8Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x1150
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1dc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x24000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 kernel: nvidia 0000:86:00.0: MDEV: Registered
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Physical GPU:
Mar 6 12:20:56 dev1 nvidia-vgpud: PciID: 0x0000 / 0x0087 / 0x0000 / 0x0000
Mar 6 12:20:56 dev1 nvidia-vgpud: Size: 52#012Version 1
Mar 6 12:20:56 dev1 nvidia-vgpud: DevID: 0x10de / 0x13f2 / 0x10de / 0x0000
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported vGPUs count: 13
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xb: max 16
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xb: GRID M60-0B Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x1176
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1a000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x6000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xc: max 16
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xc: GRID M60-0Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114c
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1a000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x6000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xd: max 8
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xd: GRID M60-1A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11ae
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x38000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x8000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xe: max 8
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xe: GRID M60-1B Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x1177
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x38000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x8000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xf: max 8
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xf: GRID M60-1Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114d
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x38000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x8000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x10: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x10: GRID M60-2A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11af
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x11: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x11: GRID M60-2B Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x117d
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 2
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0xd2: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0xd2: GRID M60-2B4 Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x12ec
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x2d
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 2560, height 1600
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-PC,2.0;GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x12: max 4
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x12: GRID M60-2Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114e
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x74000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0xc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x13: max 2
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x13: GRID M60-4A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11b0
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0xec000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x14000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x14: max 2
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x14: GRID M60-4Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x114f
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0xec000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x14000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x15: max 1
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x15: GRID M60-8A Class: NVS
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x11b1
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1dc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x24000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x0
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 1
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 1280, height 1024
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-Apps,3.0
Mar 6 12:20:56 dev1 nvidia-vgpud:
Mar 6 12:20:56 dev1 nvidia-vgpud: Supported VGPU 0x16: max 1
Mar 6 12:20:56 dev1 nvidia-vgpud: VGPU Type 0x16: GRID M60-8Q Class: Quadro
Mar 6 12:20:56 dev1 nvidia-vgpud: DevId: 0x10de / 0x13f2 / 0x10de / 0x1150
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer: 0x1dc000000
Mar 6 12:20:56 dev1 nvidia-vgpud: Mappable video size: 0x400000
Mar 6 12:20:56 dev1 nvidia-vgpud: Framebuffer reservation: 0x24000000
Mar 6 12:20:56 dev1 nvidia-vgpud: FRL configuration: 0x3c
Mar 6 12:20:56 dev1 nvidia-vgpud: CUDA enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Multi vGPU supported: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Encoder Capacity: 0x64
Mar 6 12:20:56 dev1 nvidia-vgpud: BAR1 Length: 0x100
Mar 6 12:20:56 dev1 nvidia-vgpud: Frame Rate Limiter enabled: 0x1
Mar 6 12:20:56 dev1 nvidia-vgpud: Number of Displays: 4
Mar 6 12:20:56 dev1 nvidia-vgpud: Display: width 4096, height 2160
Mar 6 12:20:56 dev1 nvidia-vgpud: License: GRID-Virtual-WS,2.0;Quadro-Virtual-DWS,5.0;GRID-Virtual-WS-Ext,2.0
Mar 6 12:20:56 dev1 kernel: nvidia 0000:87:00.0: MDEV: Registered
Mar 6 12:20:56 dev1 nvidia-vgpud: PID file unlocked.
Mar 6 12:20:56 dev1 nvidia-vgpud: PID file closed.
Mar 6 12:20:56 dev1 nvidia-vgpud: Shutdown (1562)

However, on each GPU device sysfs entry:

cat /sys/class/mdev_bus/0000:86:00.0/mdev_supported_types/nvidia-*/available_instances
0
0
0
0
0
0
0
0
0
0
0
0
0

cat /sys/class/mdev_bus/0000:87:00.0/mdev_supported_types/nvidia-*/available_instances
16
16
8
8
8
4
4
4
2
2
4
1
1

I have a suspicion this is preventing vGPUs from working in the VMs I have created (both Linux and Windows).

Any idea what could be going on here?

Thnaks for any help,

Cam

You cannot mix vGPU types on the same engine, hence I’d check what profiles are being allocated. Your license manager might help reveal this or see if "nvidia-smi vgpu" helps.