Hi，
I test vGPU in KVM recently, and meet some problems confused me.
I first try on PVE, and then use kvm in Ubuntu22.04, both meet trouble.
my server info：
intel x99 motherboard
2* Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
2T NVME
4* Nvidia-P40 gpu
according to pve doc，I use the following configuration：
| pve-manager | kernel | vGPU Software Branch | NVIDIA Host drivers |
|---|---|---|---|
| 8.1.4 | 6.5.11-8-pve | 16.3 | 535.154.02 |
and vGPU16.3 for kvm supports P40
after install the vGPU driver in PVE, I meet 3 big problems：
- ssh and command
nvdia-smistuck for a long time, even kill the host - there are some block log I do not know why
- nvidia-vgpu-mgr server use more than one hour to start
bellow is a complete log from boot to vgpud and mgr server start:
Jul 09 18:32:50 pve kernel: Linux version 6.5.11-8-pve (build@proxmox) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT_DYNAMIC PMX 6.5.11-8 (2024-01-30T12:27Z) ()
Jul 09 18:32:50 pve kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-6.5.11-8-pve root=/dev/mapper/pve-root ro initcall_blacklist=nvidiafb_init quiet intel_iommu=on iommu=pt
Jul 09 18:32:50 pve kernel: KERNEL supported cpus:
Jul 09 18:32:50 pve kernel: Intel GenuineIntel
Jul 09 18:32:50 pve kernel: AMD AuthenticAMD
Jul 09 18:32:50 pve kernel: Hygon HygonGenuine
Jul 09 18:32:50 pve kernel: Centaur CentaurHauls
Jul 09 18:32:50 pve kernel: zhaoxin Shanghai
Jul 09 18:32:50 pve kernel: BIOS-provided physical RAM map:
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x0000000000100000-0x00000000782e7fff] usable
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x00000000782e8000-0x0000000079043fff] reserved
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x0000000079044000-0x00000000792d5fff] usable
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x00000000792d6000-0x0000000079f53fff] ACPI NVS
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x0000000079f54000-0x000000007bcd8fff] reserved
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x000000007bcd9000-0x000000007bd20fff] type 20
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x000000007bd21000-0x000000007bd21fff] usable
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x000000007bd22000-0x000000007bda7fff] reserved
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x000000007bda8000-0x000000007bffffff] usable
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x000000007c000000-0x000000008fffffff] reserved
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
Jul 09 18:32:50 pve kernel: BIOS-e820: [mem 0x0000000100000000-0x000000407fffffff] usable
Jul 09 18:32:50 pve kernel: NX (Execute Disable) protection: active
Jul 09 18:32:50 pve kernel: efi: EFI v2.4 by American Megatrends
Jul 09 18:32:50 pve kernel: efi: ESRT=0x7bcd5498 ACPI=0x7970e000 ACPI 2.0=0x7970e000 SMBIOS=0xf05e0 SMBIOS 3.0=0xf0600 MOKvar=0x7bb9a000
Jul 09 18:32:50 pve kernel: efi: Remove mem30: MMIO range=[0x80000000-0x8fffffff] (256MB) from e820 map
Jul 09 18:32:50 pve kernel: e820: remove [mem 0x80000000-0x8fffffff] reserved
Jul 09 18:32:50 pve kernel: efi: Not removing mem31: MMIO range=[0xfed1c000-0xfed44fff] (164KB) from e820 map
Jul 09 18:32:50 pve kernel: efi: Remove mem32: MMIO range=[0xff000000-0xffffffff] (16MB) from e820 map
Jul 09 18:32:50 pve kernel: e820: remove [mem 0xff000000-0xffffffff] reserved
Jul 09 18:32:50 pve kernel: secureboot: Secure boot disabled
Jul 09 18:32:50 pve kernel: SMBIOS 3.0.0 present.
Jul 09 18:32:50 pve kernel: DMI: INTEL X99/X99, BIOS 5.11 03/08/2024
Jul 09 18:32:50 pve kernel: tsc: Fast TSC calibration using PIT
Jul 09 18:32:50 pve kernel: tsc: Detected 2394.519 MHz processor
Jul 09 18:32:50 pve kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
Jul 09 18:32:50 pve kernel: e820: remove [mem 0x000a0000-0x000fffff] usable
Jul 09 18:32:50 pve kernel: last_pfn = 0x4080000 max_arch_pfn = 0x400000000
Jul 09 18:32:50 pve kernel: MTRR map: 5 entries (3 fixed + 2 variable; max 23), built from 10 variable MTRRs
Jul 09 18:32:50 pve kernel: x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
Jul 09 18:32:50 pve kernel: x2apic: enabled by BIOS, switching to x2apic ops
Jul 09 18:32:50 pve kernel: last_pfn = 0x7c000 max_arch_pfn = 0x400000000
Jul 09 18:32:50 pve kernel: found SMP MP-table at [mem 0x000fd030-0x000fd03f]
Jul 09 18:32:50 pve kernel: esrt: Reserving ESRT space from 0x000000007bcd5498 to 0x000000007bcd54d0.
Jul 09 18:32:50 pve kernel: Using GB pages for direct mapping
Jul 09 18:32:50 pve kernel: secureboot: Secure boot disabled
Jul 09 18:32:50 pve kernel: RAMDISK: [mem 0x30c85000-0x34639fff]
Jul 09 18:32:50 pve kernel: ACPI: Early table checksum verification disabled
Jul 09 18:32:50 pve kernel: ACPI: RSDP 0x000000007970E000 000024 (v02 ALASKA)
Jul 09 18:32:50 pve kernel: ACPI: XSDT 0x000000007970E088 000094 (v01 ALASKA A M I 01072009 AMI 00010013)
Jul 09 18:32:50 pve kernel: ACPI: FACP 0x0000000079741748 00010C (v05 ALASKA A M I 01072009 AMI 00010013)
Jul 09 18:32:50 pve kernel: ACPI: DSDT 0x000000007970E1B8 03358B (v02 ALASKA A M I 01072009 INTL 20091013)
Jul 09 18:32:50 pve kernel: ACPI: FACS 0x0000000079F52F80 000040
Jul 09 18:32:50 pve kernel: ACPI: APIC 0x0000000079741858 000374 (v03 ALASKA A M I 01072009 AMI 00010013)
Jul 09 18:32:50 pve kernel: ACPI: FPDT 0x0000000079741BD0 000044 (v01 ALASKA A M I 01072009 AMI 00010013)
Jul 09 18:32:50 pve kernel: ACPI: FIDT 0x0000000079741C18 00009C (v01 ALASKA A M I 01072009 AMI 00010013)
Jul 09 18:32:50 pve kernel: ACPI: MCFG 0x0000000079741CB8 00003C (v01 ALASKA A M I 01072009 MSFT 00000097)
Jul 09 18:32:50 pve kernel: ACPI: UEFI 0x0000000079741CF8 000042 (v01 ALASKA A M I 01072009 00000000)
Jul 09 18:32:50 pve kernel: ACPI: HPET 0x0000000079741D40 000038 (v01 ALASKA A M I 00000001 INTL 20091013)
Jul 09 18:32:50 pve kernel: ACPI: WDDT 0x0000000079741D78 000040 (v01 ALASKA A M I 00000000 INTL 20091013)
Jul 09 18:32:50 pve kernel: ACPI: SSDT 0x0000000079741DB8 017089 (v02 ALASKA PmMgt 00000001 INTL 20120913)
Jul 09 18:32:50 pve kernel: ACPI: NITR 0x0000000079758E48 000071 (v02 ALASKA A M I 00000001 INTL 20091013)
Jul 09 18:32:50 pve kernel: ACPI: SSDT 0x0000000079758EC0 002652 (v02 ALASKA SpsNm 00000002 INTL 20120913)
Jul 09 18:32:50 pve kernel: ACPI: SSDT 0x000000007975B518 000064 (v02 ALASKA SpsNvs 00000002 INTL 20120913)
Jul 09 18:32:50 pve kernel: ACPI: PRAD 0x000000007975B580 000102 (v02 ALASKA A M I 00000002 INTL 20120913)
Jul 09 18:32:50 pve kernel: ACPI: DMAR 0x000000007975B688 000108 (v01 ALASKA A M I 00000001 INTL 20091013)
Jul 09 18:32:50 pve kernel: ACPI: Reserving FACP table memory at [mem 0x79741748-0x79741853]
Jul 09 18:32:50 pve kernel: ACPI: Reserving DSDT table memory at [mem 0x7970e1b8-0x79741742]
Jul 09 18:32:50 pve kernel: ACPI: Reserving FACS table memory at [mem 0x79f52f80-0x79f52fbf]
Jul 09 18:32:50 pve kernel: ACPI: Reserving APIC table memory at [mem 0x79741858-0x79741bcb]
Jul 09 18:32:50 pve kernel: ACPI: Reserving FPDT table memory at [mem 0x79741bd0-0x79741c13]
Jul 09 18:32:50 pve kernel: ACPI: Reserving FIDT table memory at [mem 0x79741c18-0x79741cb3]
Jul 09 18:32:50 pve kernel: ACPI: Reserving MCFG table memory at [mem 0x79741cb8-0x79741cf3]
Jul 09 18:32:50 pve kernel: ACPI: Reserving UEFI table memory at [mem 0x79741cf8-0x79741d39]
Jul 09 18:32:50 pve kernel: ACPI: Reserving HPET table memory at [mem 0x79741d40-0x79741d77]
Jul 09 18:32:50 pve kernel: ACPI: Reserving WDDT table memory at [mem 0x79741d78-0x79741db7]
Jul 09 18:32:50 pve kernel: ACPI: Reserving SSDT table memory at [mem 0x79741db8-0x79758e40]
Jul 09 18:32:50 pve kernel: ACPI: Reserving NITR table memory at [mem 0x79758e48-0x79758eb8]
Jul 09 18:32:50 pve kernel: ACPI: Reserving SSDT table memory at [mem 0x79758ec0-0x7975b511]
Jul 09 18:32:50 pve kernel: ACPI: Reserving SSDT table memory at [mem 0x7975b518-0x7975b57b]
Jul 09 18:32:50 pve kernel: ACPI: Reserving PRAD table memory at [mem 0x7975b580-0x7975b681]
Jul 09 18:32:50 pve kernel: ACPI: Reserving DMAR table memory at [mem 0x7975b688-0x7975b78f]
Jul 09 18:32:50 pve kernel: Setting APIC routing to cluster x2apic.
Jul 09 18:32:50 pve kernel: No NUMA configuration found
Jul 09 18:32:50 pve kernel: Faking a node at [mem 0x0000000000000000-0x000000407fffffff]
Jul 09 18:32:50 pve kernel: NODE_DATA(0) allocated [mem 0x407ffd5000-0x407fffffff]
Jul 09 18:32:50 pve kernel: Zone ranges:
Jul 09 18:32:50 pve kernel: DMA [mem 0x0000000000001000-0x0000000000ffffff]
Jul 09 18:32:50 pve kernel: DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
Jul 09 18:32:50 pve kernel: Normal [mem 0x0000000100000000-0x000000407fffffff]
Jul 09 18:32:50 pve kernel: Device empty
Jul 09 18:32:50 pve kernel: Movable zone start for each node
Jul 09 18:32:50 pve kernel: Early memory node ranges
Jul 09 18:32:50 pve kernel: node 0: [mem 0x0000000000001000-0x000000000009ffff]
Jul 09 18:32:50 pve kernel: node 0: [mem 0x0000000000100000-0x00000000782e7fff]
Jul 09 18:32:50 pve kernel: node 0: [mem 0x0000000079044000-0x00000000792d5fff]
Jul 09 18:32:50 pve kernel: node 0: [mem 0x000000007bd21000-0x000000007bd21fff]
Jul 09 18:32:50 pve kernel: node 0: [mem 0x000000007bda8000-0x000000007bffffff]
Jul 09 18:32:50 pve kernel: node 0: [mem 0x0000000100000000-0x000000407fffffff]
Jul 09 18:32:50 pve kernel: Initmem setup node 0 [mem 0x0000000000001000-0x000000407fffffff]
Jul 09 18:32:50 pve kernel: On node 0, zone DMA: 1 pages in unavailable ranges
Jul 09 18:32:50 pve kernel: On node 0, zone DMA: 96 pages in unavailable ranges
Jul 09 18:32:50 pve kernel: On node 0, zone DMA32: 3420 pages in unavailable ranges
Jul 09 18:32:50 pve kernel: On node 0, zone DMA32: 10827 pages in unavailable ranges
Jul 09 18:32:50 pve kernel: On node 0, zone DMA32: 134 pages in unavailable ranges
Jul 09 18:32:50 pve kernel: On node 0, zone Normal: 16384 pages in unavailable ranges
Jul 09 18:32:50 pve kernel: ACPI: PM-Timer IO Port: 0x408
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x08] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x0a] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x0c] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x10] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x12] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x14] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x16] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x18] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x1a] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x1c] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x20] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x22] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x24] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x26] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x28] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x2a] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x2c] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x30] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x32] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x34] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x36] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x38] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x3a] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x3c] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x09] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x0b] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x0d] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x11] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x13] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x15] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x17] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x19] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x1b] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x1d] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x21] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x23] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x25] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x27] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x29] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x2b] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x2d] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x31] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x33] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x35] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x37] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x39] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x3b] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: ACPI: LAPIC_NMI (acpi_id[0x3d] high edge lint[0x1])
Jul 09 18:32:50 pve kernel: IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23
Jul 09 18:32:50 pve kernel: IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47
Jul 09 18:32:50 pve kernel: IOAPIC[2]: apic_id 3, version 32, address 0xfec40000, GSI 48-71
Jul 09 18:32:50 pve kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Jul 09 18:32:50 pve kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Jul 09 18:32:50 pve kernel: ACPI: Using ACPI (MADT) for SMP configuration information
Jul 09 18:32:50 pve kernel: ACPI: HPET id: 0x8086a701 base: 0xfed00000
Jul 09 18:32:50 pve kernel: TSC deadline timer available
Jul 09 18:32:50 pve kernel: smpboot: Allowing 56 CPUs, 0 hotplug CPUs
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000fffff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x782e8000-0x79043fff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x792d6000-0x79f53fff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x79f54000-0x7bcd8fff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x7bcd9000-0x7bd20fff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x7bd22000-0x7bda7fff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x7c000000-0x7fffffff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0x80000000-0xfed1bfff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0xfed1c000-0xfed44fff]
Jul 09 18:32:50 pve kernel: PM: hibernation: Registered nosave memory: [mem 0xfed45000-0xffffffff]
Jul 09 18:32:50 pve kernel: [mem 0x80000000-0xfed1bfff] available for PCI devices
Jul 09 18:32:50 pve kernel: Booting paravirtualized kernel on bare hardware
Jul 09 18:32:50 pve kernel: clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
Jul 09 18:32:50 pve kernel: setup_percpu: NR_CPUS:8192 nr_cpumask_bits:56 nr_cpu_ids:56 nr_node_ids:1
Jul 09 18:32:50 pve kernel: percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u262144
Jul 09 18:32:50 pve kernel: pcpu-alloc: s221184 r8192 d28672 u262144 alloc=1*2097152
Jul 09 18:32:50 pve kernel: pcpu-alloc: [0] 00 01 02 03 04 05 06 07 [0] 08 09 10 11 12 13 14 15
Jul 09 18:32:50 pve kernel: pcpu-alloc: [0] 16 17 18 19 20 21 22 23 [0] 24 25 26 27 28 29 30 31
Jul 09 18:32:50 pve kernel: pcpu-alloc: [0] 32 33 34 35 36 37 38 39 [0] 40 41 42 43 44 45 46 47
Jul 09 18:32:50 pve kernel: pcpu-alloc: [0] 48 49 50 51 52 53 54 55
Jul 09 18:32:50 pve kernel: Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.5.11-8-pve root=/dev/mapper/pve-root ro initcall_blacklist=nvidiafb_init quiet intel_iommu=on iommu=pt
Jul 09 18:32:50 pve kernel: blacklisting initcall nvidiafb_init
Jul 09 18:32:50 pve kernel: DMAR: IOMMU enabled
Jul 09 18:32:50 pve kernel: Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-6.5.11-8-pve", will be passed to user space.
Jul 09 18:32:50 pve kernel: random: crng init done
Jul 09 18:32:50 pve kernel: printk: log_buf_len individual max cpu contribution: 4096 bytes
Jul 09 18:32:50 pve kernel: printk: log_buf_len total cpu_extra contributions: 225280 bytes
Jul 09 18:32:50 pve kernel: printk: log_buf_len min size: 262144 bytes
Jul 09 18:32:50 pve kernel: printk: log_buf_len: 524288 bytes
Jul 09 18:32:50 pve kernel: printk: early log buf free: 248336(94%)
Jul 09 18:32:50 pve kernel: Dentry cache hash table entries: 16777216 (order: 15, 134217728 bytes, linear)
Jul 09 18:32:50 pve kernel: Inode-cache hash table entries: 8388608 (order: 14, 67108864 bytes, linear)
Jul 09 18:32:50 pve kernel: Fallback order for Node 0: 0
Jul 09 18:32:50 pve kernel: Built 1 zonelists, mobility grouping on. Total pages: 66029747
Jul 09 18:32:50 pve kernel: Policy zone: Normal
Jul 09 18:32:50 pve kernel: mem auto-init: stack:all(zero), heap alloc:on, heap free:off
Jul 09 18:32:50 pve kernel: software IO TLB: area num 64.
Jul 09 18:32:50 pve kernel: Memory: 263650664K/268312008K available (20480K kernel code, 3583K rwdata, 12756K rodata, 4624K init, 18244K bss, 4661084K reserved, 0K cma-reserved)
Jul 09 18:32:50 pve kernel: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=56, Nodes=1
Jul 09 18:32:50 pve kernel: Kernel/User page tables isolation: enabled
Jul 09 18:32:50 pve kernel: ftrace: allocating 52819 entries in 207 pages
Jul 09 18:32:50 pve kernel: ftrace: allocated 207 pages with 6 groups
Jul 09 18:32:50 pve kernel: Dynamic Preempt: voluntary
Jul 09 18:32:50 pve kernel: rcu: Preemptible hierarchical RCU implementation.
Jul 09 18:32:50 pve kernel: rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=56.
Jul 09 18:32:50 pve kernel: Trampoline variant of Tasks RCU enabled.
Jul 09 18:32:50 pve kernel: Rude variant of Tasks RCU enabled.
Jul 09 18:32:50 pve kernel: Tracing variant of Tasks RCU enabled.
Jul 09 18:32:50 pve kernel: rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
Jul 09 18:32:50 pve kernel: rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=56
Jul 09 18:32:50 pve kernel: NR_IRQS: 524544, nr_irqs: 1688, preallocated irqs: 16
Jul 09 18:32:50 pve kernel: rcu: srcu_init: Setting srcu_struct sizes based on contention.
Jul 09 18:32:50 pve kernel: Console: colour dummy device 80x25
Jul 09 18:32:50 pve kernel: printk: console [tty0] enabled
Jul 09 18:32:50 pve kernel: ACPI: Core revision 20230331
Jul 09 18:32:50 pve kernel: clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns
Jul 09 18:32:50 pve kernel: APIC: Switch to symmetric I/O mode setup
Jul 09 18:32:50 pve kernel: DMAR: Host address width 46
Jul 09 18:32:50 pve kernel: DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
Jul 09 18:32:50 pve kernel: DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
Jul 09 18:32:50 pve kernel: DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1
Jul 09 18:32:50 pve kernel: DMAR: dmar1: reg_base_addr c7ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
Jul 09 18:32:50 pve kernel: DMAR: RMRR base: 0x0000007bbaf000 end: 0x0000007bbbefff
Jul 09 18:32:50 pve kernel: DMAR: ATSR flags: 0x0
Jul 09 18:32:50 pve kernel: DMAR: RHSA base: 0x000000c7ffc000 proximity domain: 0x0
Jul 09 18:32:50 pve kernel: DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x1
Jul 09 18:32:50 pve kernel: DMAR-IR: IOAPIC id 3 under DRHD base 0xfbffc000 IOMMU 0
Jul 09 18:32:50 pve kernel: DMAR-IR: IOAPIC id 1 under DRHD base 0xc7ffc000 IOMMU 1
Jul 09 18:32:50 pve kernel: DMAR-IR: IOAPIC id 2 under DRHD base 0xc7ffc000 IOMMU 1
Jul 09 18:32:50 pve kernel: DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000
Jul 09 18:32:50 pve kernel: DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
Jul 09 18:32:50 pve kernel: DMAR-IR: Enabled IRQ remapping in x2apic mode
Jul 09 18:32:50 pve kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Jul 09 18:32:50 pve kernel: clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2283fdd66f9, max_idle_ns: 440795314208 ns
Jul 09 18:32:50 pve kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 4789.03 BogoMIPS (lpj=9578076)
Jul 09 18:32:50 pve kernel: CPU0: Thermal monitoring enabled (TM1)
Jul 09 18:32:50 pve kernel: process: using mwait in idle threads
Jul 09 18:32:50 pve kernel: Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
Jul 09 18:32:50 pve kernel: Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
Jul 09 18:32:50 pve kernel: Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
Jul 09 18:32:50 pve kernel: Spectre V2 : Mitigation: Retpolines
Jul 09 18:32:50 pve kernel: Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
Jul 09 18:32:50 pve kernel: Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
Jul 09 18:32:50 pve kernel: Spectre V2 : Enabling Restricted Speculation for firmware calls
Jul 09 18:32:50 pve kernel: Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
Jul 09 18:32:50 pve kernel: Spectre V2 : User space: Mitigation: STIBP via prctl
Jul 09 18:32:50 pve kernel: Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
Jul 09 18:32:50 pve kernel: MDS: Vulnerable: Clear CPU buffers attempted, no microcode
Jul 09 18:32:50 pve kernel: TAA: Vulnerable: Clear CPU buffers attempted, no microcode
Jul 09 18:32:50 pve kernel: MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode
Jul 09 18:32:50 pve kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Jul 09 18:32:50 pve kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Jul 09 18:32:50 pve kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Jul 09 18:32:50 pve kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
Jul 09 18:32:50 pve kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
Jul 09 18:32:50 pve kernel: Freeing SMP alternatives memory: 44K
Jul 09 18:32:50 pve kernel: pid_max: default: 57344 minimum: 448
Jul 09 18:32:50 pve kernel: LSM: initializing lsm=lockdown,capability,yama,apparmor,integrity
Jul 09 18:32:50 pve kernel: Yama: becoming mindful.
Jul 09 18:32:50 pve kernel: AppArmor: AppArmor initialized
Jul 09 18:32:50 pve kernel: Mount-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
Jul 09 18:32:50 pve kernel: Mountpoint-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
Jul 09 18:32:50 pve kernel: smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz (family: 0x6, model: 0x4f, stepping: 0x1)
Jul 09 18:32:50 pve kernel: RCU Tasks: Setting shift to 6 and lim to 1 rcu_task_cb_adjust=1.
Jul 09 18:32:50 pve kernel: RCU Tasks Rude: Setting shift to 6 and lim to 1 rcu_task_cb_adjust=1.
Jul 09 18:32:50 pve kernel: RCU Tasks Trace: Setting shift to 6 and lim to 1 rcu_task_cb_adjust=1.
Jul 09 18:32:50 pve kernel: Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver.
Jul 09 18:32:50 pve kernel: ... version: 3
Jul 09 18:32:50 pve kernel: ... bit width: 48
Jul 09 18:32:50 pve kernel: ... generic registers: 4
Jul 09 18:32:50 pve kernel: ... value mask: 0000ffffffffffff
Jul 09 18:32:50 pve kernel: ... max period: 00007fffffffffff
Jul 09 18:32:50 pve kernel: ... fixed-purpose events: 3
Jul 09 18:32:50 pve kernel: ... event mask: 000000070000000f
Jul 09 18:32:50 pve kernel: signal: max sigframe size: 1776
Jul 09 18:32:50 pve kernel: Estimated ratio of average max frequency by base frequency (times 1024): 1280
Jul 09 18:32:50 pve kernel: rcu: Hierarchical SRCU implementation.
Jul 09 18:32:50 pve kernel: rcu: Max phase no-delay instances is 1000.
Jul 09 18:32:50 pve kernel: NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
Jul 09 18:32:50 pve kernel: smp: Bringing up secondary CPUs ...
Jul 09 18:32:50 pve kernel: smpboot: x86: Booting SMP configuration:
Jul 09 18:32:50 pve kernel: .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 #23 #24 #25 #26 #27
Jul 09 18:32:50 pve kernel: smpboot: CPU 14 Converting physical 0 to logical die 1
Jul 09 18:32:50 pve kernel: #28 #29 #30 #31 #32 #33 #34 #35 #36 #37 #38 #39 #40 #41 #42 #43 #44 #45 #46 #47 #48 #49 #50 #51 #52 #53 #54 #55
Jul 09 18:32:50 pve kernel: MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.
Jul 09 18:32:50 pve kernel: TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details.
Jul 09 18:32:50 pve kernel: MMIO Stale Data CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details.
Jul 09 18:32:50 pve kernel: smp: Brought up 1 node, 56 CPUs
Jul 09 18:32:50 pve kernel: smpboot: Max logical packages: 2
Jul 09 18:32:50 pve kernel: smpboot: Total of 56 processors activated (268186.12 BogoMIPS)
Jul 09 18:32:50 pve kernel: devtmpfs: initialized
Jul 09 18:32:50 pve kernel: x86/mm: Memory block size: 2048MB
Jul 09 18:32:50 pve kernel: ACPI: PM: Registering ACPI NVS region [mem 0x792d6000-0x79f53fff] (13099008 bytes)
Jul 09 18:32:50 pve kernel: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
Jul 09 18:32:50 pve kernel: futex hash table entries: 16384 (order: 8, 1048576 bytes, linear)
Jul 09 18:32:50 pve kernel: pinctrl core: initialized pinctrl subsystem
Jul 09 18:32:50 pve kernel: PM: RTC time: 10:32:46, date: 2024-07-09
Jul 09 18:32:50 pve kernel: NET: Registered PF_NETLINK/PF_ROUTE protocol family
Jul 09 18:32:50 pve kernel: DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations
Jul 09 18:32:50 pve kernel: DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
Jul 09 18:32:50 pve kernel: DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
Jul 09 18:32:50 pve kernel: audit: initializing netlink subsys (disabled)
Jul 09 18:32:50 pve kernel: audit: type=2000 audit(1720521166.100:1): state=initialized audit_enabled=0 res=1
Jul 09 18:32:50 pve kernel: thermal_sys: Registered thermal governor 'fair_share'
Jul 09 18:32:50 pve kernel: thermal_sys: Registered thermal governor 'bang_bang'
Jul 09 18:32:50 pve kernel: thermal_sys: Registered thermal governor 'step_wise'
Jul 09 18:32:50 pve kernel: thermal_sys: Registered thermal governor 'user_space'
Jul 09 18:32:50 pve kernel: thermal_sys: Registered thermal governor 'power_allocator'
Jul 09 18:32:50 pve kernel: EISA bus registered
Jul 09 18:32:50 pve kernel: cpuidle: using governor ladder
Jul 09 18:32:50 pve kernel: cpuidle: using governor menu
Jul 09 18:32:50 pve kernel: acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
Jul 09 18:32:50 pve kernel: PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
Jul 09 18:32:50 pve kernel: PCI: not using MMCONFIG
Jul 09 18:32:50 pve kernel: PCI: Using configuration type 1 for base access
Jul 09 18:32:50 pve kernel: kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
Jul 09 18:32:50 pve kernel: HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
Jul 09 18:32:50 pve kernel: HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
Jul 09 18:32:50 pve kernel: HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
Jul 09 18:32:50 pve kernel: HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
Jul 09 18:32:50 pve kernel: ACPI: Added _OSI(Module Device)
Jul 09 18:32:50 pve kernel: ACPI: Added _OSI(Processor Device)
Jul 09 18:32:50 pve kernel: ACPI: Added _OSI(3.0 _SCP Extensions)
Jul 09 18:32:50 pve kernel: ACPI: Added _OSI(Processor Aggregator Device)
Jul 09 18:32:50 pve kernel: ACPI: 4 ACPI AML tables successfully acquired and loaded
Jul 09 18:32:50 pve kernel: ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
Jul 09 18:32:50 pve kernel: ACPI: Interpreter enabled
Jul 09 18:32:50 pve kernel: ACPI: PM: (supports S0 S3 S4 S5)
Jul 09 18:32:50 pve kernel: ACPI: Using IOAPIC for interrupt routing
Jul 09 18:32:50 pve kernel: PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
Jul 09 18:32:50 pve kernel: [Firmware Info]: PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] not reserved in ACPI motherboard resources
Jul 09 18:32:50 pve kernel: PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved as EfiMemoryMappedIO
Jul 09 18:32:50 pve kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
Jul 09 18:32:50 pve kernel: PCI: Ignoring E820 reservations for host bridge windows
Jul 09 18:32:50 pve kernel: ACPI: Enabled 5 GPEs in block 00 to 3F
Jul 09 18:32:50 pve kernel: ACPI: PCI Root Bridge [UNC1] (domain 0000 [bus ff])
Jul 09 18:32:50 pve kernel: acpi PNP0A03:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3]
Jul 09 18:32:50 pve kernel: acpi PNP0A03:02: _OSC: platform does not support [SHPCHotplug LTR DPC]
Jul 09 18:32:50 pve kernel: acpi PNP0A03:02: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability]
Jul 09 18:32:50 pve kernel: PCI host bridge to bus 0000:ff
Jul 09 18:32:50 pve kernel: pci_bus 0000:ff: root bus resource [bus ff]
Jul 09 18:32:50 pve kernel: pci 0000:ff:08.0: [8086:6f80] type 00 class 0x088000
Jul 09 18:32:50 pve kernel: pci 0000:ff:08.2: [8086:6f32] type 00 class 0x110100
..........
was skipped because of an unmet condition check (ConditionPathExists=!/usr/bin/dbus-daemon).
Jul 09 18:32:52 pve systemd[1]: Starting ksmtuned.service - Kernel Samepage Merging (KSM) Tuning Daemon...
Jul 09 18:32:52 pve systemd[1]: Started lxcfs.service - FUSE filesystem for LXC.
Jul 09 18:32:52 pve systemd[1]: Starting nvidia-vgpu-mgr.service - NVIDIA vGPU Manager Daemon...
Jul 09 18:32:52 pve lxcfs[1113]: Running constructor lxcfs_init to reload liblxcfs
Jul 09 18:32:52 pve systemd[1]: Starting nvidia-vgpud.service - NVIDIA vGPU Daemon...
Jul 09 18:32:52 pve systemd[1]: proxmox-boot-cleanup.service - Clean up bootloader next-boot setting was skipped because of an unmet condition check (ConditionPathExists=/etc/kernel/next-boot-pin).
Jul 09 18:32:52 pve dbus-daemon[1109]: [system] AppArmor D-Bus mediation is enabled
Jul 09 18:32:52 pve systemd[1]: Starting pve-lxc-syscalld.service - Proxmox VE LXC Syscall Daemon...
Jul 09 18:32:52 pve nvidia-vgpud[1123]: Verbose syslog connection opened
Jul 09 18:32:52 pve nvidia-vgpud[1123]: Started (1123)
Jul 09 18:32:52 pve systemd[1]: Starting qmeventd.service - PVE Qemu Event Daemon...
Jul 09 18:32:52 pve kernel: NVRM: GPU at 0000:03:00.0 has software scheduler ENABLED with policy BEST_EFFORT.
Jul 09 18:32:52 pve systemd[1]: Starting smartmontools.service - Self Monitoring and Reporting Technology (SMART) Daemon...
Jul 09 18:32:52 pve smartd[1127]: smartd 7.3 2022-02-28 r5338 [x86_64-linux-6.5.11-8-pve] (local build)
Jul 09 18:32:52 pve lxcfs[1113]: mount namespace: 5
Jul 09 18:32:52 pve lxcfs[1113]: hierarchies:
Jul 09 18:32:52 pve lxcfs[1113]: 0: fd: 6: cpuset,cpu,io,memory,hugetlb,pids,rdma,misc
Jul 09 18:32:52 pve lxcfs[1113]: Kernel supports pidfds
Jul 09 18:32:52 pve lxcfs[1113]: Kernel supports swap accounting
Jul 09 18:32:52 pve lxcfs[1113]: api_extensions:
Jul 09 18:32:52 pve lxcfs[1113]: - cgroups
Jul 09 18:32:52 pve lxcfs[1113]: - sys_cpu_online
Jul 09 18:32:52 pve lxcfs[1113]: - proc_cpuinfo
Jul 09 18:32:52 pve lxcfs[1113]: - proc_diskstats
Jul 09 18:32:52 pve lxcfs[1113]: - proc_loadavg
Jul 09 18:32:52 pve lxcfs[1113]: - proc_meminfo
Jul 09 18:32:52 pve lxcfs[1113]: - proc_stat
Jul 09 18:32:52 pve lxcfs[1113]: - proc_swaps
Jul 09 18:32:52 pve lxcfs[1113]: - proc_uptime
Jul 09 18:32:52 pve lxcfs[1113]: - proc_slabinfo
Jul 09 18:32:52 pve lxcfs[1113]: - shared_pidns
Jul 09 18:32:52 pve lxcfs[1113]: - cpuview_daemon
Jul 09 18:32:52 pve lxcfs[1113]: - loadavg_daemon
Jul 09 18:32:52 pve lxcfs[1113]: - pidfds
Jul 09 18:32:52 pve smartd[1127]: Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
Jul 09 18:32:52 pve lxcfs[1113]: Ignoring invalid max threads value 4294967295 > max (100000).
Jul 09 18:32:52 pve smartd[1127]: Opened configuration file /etc/smartd.conf
Jul 09 18:32:52 pve smartd[1127]: Drive: DEVICESCAN, implied '-a' Directive on line 21 of file /etc/smartd.conf
Jul 09 18:32:52 pve smartd[1127]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Jul 09 18:32:52 pve smartd[1127]: Device: /dev/nvme0, opened
Jul 09 18:32:52 pve systemd[1]: Starting systemd-logind.service - User Login Management...
Jul 09 18:32:52 pve nvidia-vgpud[1123]: Global settings:
Jul 09 18:32:52 pve nvidia-vgpud[1123]: Size: 16
Version 1
Jul 09 18:32:52 pve nvidia-vgpud[1123]: Homogeneous vGPUs: 1
Jul 09 18:32:52 pve nvidia-vgpud[1123]: vGPU types: 586
Jul 09 18:32:52 pve nvidia-vgpud[1123]:
Jul 09 18:32:52 pve smartd[1127]: Device: /dev/nvme0, ZHITAI Ti600 2TB, S/N:ZTA602TAB2337648R5, FW:ZTA23001, 2.00 TB
Jul 09 18:32:52 pve systemd[1]: Started watchdog-mux.service - Proxmox VE watchdog multiplexer.
Jul 09 18:32:52 pve smartd[1127]: Device: /dev/nvme0, is SMART capable. Adding to "monitor" list.
Jul 09 18:32:52 pve smartd[1127]: Device: /dev/nvme0, state read from /var/lib/smartmontools/smartd.ZHITAI_Ti600_2TB-ZTA602TAB2337648R5.nvme.state
Jul 09 18:32:52 pve smartd[1127]: Monitoring 0 ATA/SATA, 0 SCSI/SAS and 1 NVMe devices
Jul 09 18:32:52 pve systemd[1]: Starting zfs-share.service - ZFS file system shares...
Jul 09 18:32:52 pve smartd[1127]: Device: /dev/nvme0, state written to /var/lib/smartmontools/smartd.ZHITAI_Ti600_2TB-ZTA602TAB2337648R5.nvme.state
Jul 09 18:32:52 pve systemd[1]: Started zfs-zed.service - ZFS Event Daemon (zed).
Jul 09 18:32:52 pve systemd[1]: Started dbus.service - D-Bus System Message Bus.
Jul 09 18:32:52 pve systemd[1]: Started pve-lxc-syscalld.service - Proxmox VE LXC Syscall Daemon.
Jul 09 18:32:52 pve watchdog-mux[1138]: Watchdog driver 'Software Watchdog', version 0
Jul 09 18:32:52 pve systemd[1]: Started smartmontools.service - Self Monitoring and Reporting Technology (SMART) Daemon.
Jul 09 18:32:52 pve systemd[1]: Started ksmtuned.service - Kernel Samepage Merging (KSM) Tuning Daemon.
Jul 09 18:32:52 pve systemd[1]: Started nvidia-vgpu-mgr.service - NVIDIA vGPU Manager Daemon.
Jul 09 18:32:52 pve systemd[1]: Started nvidia-vgpud.service - NVIDIA vGPU Daemon.
Jul 09 18:32:52 pve kernel: softdog: initialized. soft_noboot=0 soft_margin=60 sec soft_panic=0 (nowayout=0)
Jul 09 18:32:52 pve kernel: softdog: soft_reboot_cmd=<not set> soft_active_on_boot=0
Jul 09 18:32:52 pve zed[1144]: ZFS Event Daemon 2.2.2-pve1 (PID 1144)
Jul 09 18:32:52 pve systemd[1]: Started qmeventd.service - PVE Qemu Event Daemon.
Jul 09 18:32:52 pve systemd[1]: Finished zfs-share.service - ZFS file system shares.
Jul 09 18:32:52 pve systemd[1]: Reached target zfs.target - ZFS startup target.
Jul 09 18:32:52 pve zed[1144]: Processing events since eid=0
Jul 09 18:32:52 pve systemd[1]: Mounted run-rpc_pipefs.mount - RPC Pipe File System.
Jul 09 18:32:52 pve systemd[1]: Reached target rpc_pipefs.target.
Jul 09 18:32:52 pve systemd[1]: rpc-gssd.service - RPC security service for NFS client and server was skipped because of an unmet condition check (ConditionPathExists=/etc/krb5.keytab).
Jul 09 18:32:52 pve systemd[1]: Reached target nfs-client.target - NFS client services.
Jul 09 18:32:52 pve kernel: RPC: Registered named UNIX socket transport module.
Jul 09 18:32:52 pve kernel: RPC: Registered udp transport module.
Jul 09 18:32:52 pve kernel: RPC: Registered tcp transport module.
Jul 09 18:32:52 pve kernel: RPC: Registered tcp-with-tls transport module.
Jul 09 18:32:52 pve kernel: RPC: Registered tcp NFSv4.1 backchannel transport module.
Jul 09 18:32:52 pve systemd[1]: e2scrub_reap.service: Deactivated successfully.
Jul 09 18:32:52 pve systemd[1]: Finished e2scrub_reap.service - Remove Stale Online ext4 Metadata Check Snapshots.
Jul 09 18:32:52 pve systemd-logind[1135]: Watching system buttons on /dev/input/event1 (Power Button)
Jul 09 18:32:52 pve systemd-logind[1135]: Watching system buttons on /dev/input/event0 (Power Button)
Jul 09 18:32:52 pve systemd-logind[1135]: Watching system buttons on /dev/input/event3 (Logitech USB Keyboard)
Jul 09 18:32:52 pve systemd-logind[1135]: Watching system buttons on /dev/input/event4 (Logitech USB Keyboard)
Jul 09 18:32:52 pve systemd-logind[1135]: New seat seat0.
Jul 09 18:32:52 pve systemd[1]: Started systemd-logind.service - User Login Management.
Jul 09 18:32:52 pve systemd[1]: Finished pvebanner.service - Proxmox VE Login Banner.
Jul 09 18:32:53 pve kernel: vmbr0: port 1(ens11f1) entered blocking state
Jul 09 18:32:53 pve kernel: vmbr0: port 1(ens11f1) entered disabled state
Jul 09 18:32:53 pve kernel: ixgbe 0000:01:00.1 ens11f1: entered allmulticast mode
Jul 09 18:32:53 pve kernel: ixgbe 0000:01:00.1 ens11f1: entered promiscuous mode
Jul 09 18:32:53 pve kernel: pps pps0: new PPS source ptp0
Jul 09 18:32:53 pve kernel: ixgbe 0000:01:00.1: registered PHC device on ens11f1
Jul 09 18:32:54 pve nvidia-vgpud[1123]: pciId of gpu [0]: 0:3:0:0
Jul 09 18:32:54 pve nvidia-vgpud[1123]: pciId of gpu [1]: 0:4:0:0
Jul 09 18:32:54 pve nvidia-vgpud[1123]: pciId of gpu [2]: 0:81:0:0
Jul 09 18:32:54 pve nvidia-vgpud[1123]: pciId of gpu [3]: 0:82:0:0
Jul 09 18:32:54 pve kernel: NVRM: GPU at 0000:04:00.0 has software scheduler ENABLED with policy BEST_EFFORT.
Jul 09 18:32:54 pve systemd[1]: Finished networking.service - Network initialization.
Jul 09 18:32:54 pve systemd[1]: Reached target network.target - Network.
Jul 09 18:32:54 pve systemd[1]: Reached target network-online.target - Network is Online.
Jul 09 18:32:54 pve systemd[1]: Starting chrony.service - chrony, an NTP client/server...
Jul 09 18:32:54 pve systemd[1]: Started lxc-monitord.service - LXC Container Monitoring Daemon.
Jul 09 18:32:54 pve systemd[1]: Starting lxc-net.service - LXC network bridge setup...
Jul 09 18:32:54 pve systemd[1]: open-iscsi.service - Login to default iSCSI targets was skipped because no trigger condition checks were met.
Jul 09 18:32:54 pve systemd[1]: Reached target remote-fs-pre.target - Preparation for Remote File Systems.
Jul 09 18:32:54 pve systemd[1]: Reached target remote-fs.target - Remote File Systems.
Jul 09 18:32:54 pve systemd[1]: Reached target pve-storage.target - PVE Storage Target.
Jul 09 18:32:54 pve systemd[1]: Starting postfix@-.service - Postfix Mail Transport Agent (instance -)...
Jul 09 18:32:54 pve systemd[1]: Starting rbdmap.service - Map RBD devices...
Jul 09 18:32:54 pve systemd[1]: Starting rpc-statd-notify.service - Notify NFS peers of a restart...
Jul 09 18:32:54 pve systemd[1]: Starting ssh.service - OpenBSD Secure Shell server...
Jul 09 18:32:54 pve systemd[1]: systemd-pcrphase.service - TPM2 PCR Barrier (User) was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
Jul 09 18:32:54 pve sm-notify[1245]: Version 2.6.2 starting
Jul 09 18:32:54 pve systemd[1]: Starting systemd-user-sessions.service - Permit User Sessions...
Jul 09 18:32:54 pve systemd[1]: Finished rbdmap.service - Map RBD devices.
Jul 09 18:32:54 pve systemd[1]: Started rpc-statd-notify.service - Notify NFS peers of a restart.
Jul 09 18:32:54 pve systemd[1]: Finished blk-availability.service - Availability of block devices.
Jul 09 18:32:54 pve systemd[1]: Finished systemd-user-sessions.service - Permit User Sessions.
Jul 09 18:32:54 pve chronyd[1263]: chronyd version 4.3 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 -DEBUG)
Jul 09 18:32:54 pve chronyd[1263]: Frequency -37.233 +/- 0.674 ppm read from /var/lib/chrony/chrony.drift
Jul 09 18:32:54 pve chronyd[1263]: Using right/UTC timezone to obtain leap second data
Jul 09 18:32:54 pve chronyd[1263]: Loaded seccomp filter (level 1)
Jul 09 18:32:55 pve systemd[1]: Started getty@tty1.service - Getty on tty1.
Jul 09 18:32:55 pve systemd[1]: Reached target getty.target - Login Prompts.
Jul 09 18:32:55 pve systemd[1]: Started chrony.service - chrony, an NTP client/server.
Jul 09 18:32:55 pve systemd[1]: Finished lxc-net.service - LXC network bridge setup.
Jul 09 18:32:55 pve systemd[1]: Reached target time-sync.target - System Time Synchronized.
Jul 09 18:32:55 pve systemd[1]: Started apt-daily.timer - Daily apt download activities.
Jul 09 18:32:55 pve systemd[1]: Started apt-daily-upgrade.timer - Daily apt upgrade and clean activities.
Jul 09 18:32:55 pve systemd[1]: Started dpkg-db-backup.timer - Daily dpkg database backup timer.
Jul 09 18:32:55 pve systemd[1]: Started e2scrub_all.timer - Periodic ext4 Online Metadata Check for All Filesystems.
Jul 09 18:32:55 pve systemd[1]: Started fstrim.timer - Discard unused blocks once a week.
Jul 09 18:32:55 pve systemd[1]: Started logrotate.timer - Daily rotation of log files.
Jul 09 18:32:55 pve systemd[1]: Started man-db.timer - Daily man-db regeneration.
Jul 09 18:32:55 pve systemd[1]: Started pve-daily-update.timer - Daily PVE download activities.
Jul 09 18:32:55 pve systemd[1]: Reached target timers.target - Timer Units.
Jul 09 18:32:55 pve systemd[1]: Starting lxc.service - LXC Container Initialization and Autoboot Code...
Jul 09 18:32:55 pve systemd[1]: Starting rrdcached.service - LSB: start or stop rrdcached...
Jul 09 18:32:55 pve sshd[1309]: Server listening on 0.0.0.0 port 22.
Jul 09 18:32:55 pve sshd[1309]: Server listening on :: port 22.
Jul 09 18:32:55 pve systemd[1]: Started ssh.service - OpenBSD Secure Shell server.
Jul 09 18:32:55 pve postfix[1314]: Postfix is using backwards-compatible default settings
Jul 09 18:32:55 pve postfix[1314]: See http://www.postfix.org/COMPATIBILITY_README.html for details
Jul 09 18:32:55 pve postfix[1314]: To disable backwards compatibility use "postconf compatibility_level=3.6" and "postfix reload"
Jul 09 18:32:55 pve audit[1319]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/usr/bin/lxc-start" pid=1319 comm="apparmor_parser"
Jul 09 18:32:55 pve audit[1328]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default" pid=1328 comm="apparmor_parser"
Jul 09 18:32:55 pve audit[1328]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-cgns" pid=1328 comm="apparmor_parser"
Jul 09 18:32:55 pve audit[1328]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-with-mounting" pid=1328 comm="apparmor_parser"
Jul 09 18:32:55 pve audit[1328]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-with-nesting" pid=1328 comm="apparmor_parser"
Jul 09 18:32:55 pve systemd[1]: Finished lxc.service - LXC Container Initialization and Autoboot Code.
Jul 09 18:32:55 pve rrdcached[1311]: rrdcached started.
Jul 09 18:32:55 pve systemd[1]: Started rrdcached.service - LSB: start or stop rrdcached.
Jul 09 18:32:55 pve systemd[1]: Starting pve-cluster.service - The Proxmox VE cluster filesystem...
Jul 09 18:32:55 pve pmxcfs[1348]: [main] notice: resolved node name 'pve' to '192.168.10.231' for default node IP address
Jul 09 18:32:55 pve pmxcfs[1348]: [main] notice: resolved node name 'pve' to '192.168.10.231' for default node IP address
Jul 09 18:32:55 pve kernel: NVRM: GPU at 0000:81:00.0 has software scheduler ENABLED with policy BEST_EFFORT.
Jul 09 18:32:55 pve postfix/postfix-script[1435]: starting the Postfix mail system
Jul 09 18:32:55 pve postfix/master[1437]: daemon started -- version 3.7.9, configuration /etc/postfix
Jul 09 18:32:55 pve systemd[1]: Started postfix@-.service - Postfix Mail Transport Agent (instance -).
Jul 09 18:32:55 pve systemd[1]: Starting postfix.service - Postfix Mail Transport Agent...
Jul 09 18:32:55 pve systemd[1]: Finished postfix.service - Postfix Mail Transport Agent.
Jul 09 18:32:56 pve systemd[1]: Started pve-cluster.service - The Proxmox VE cluster filesystem.
Jul 09 18:32:56 pve systemd[1]: corosync.service - Corosync Cluster Engine was skipped because of an unmet condition check (ConditionPathExists=/etc/corosync/corosync.conf).
Jul 09 18:32:56 pve systemd[1]: Started cron.service - Regular background program processing daemon.
Jul 09 18:32:56 pve cron[1445]: (CRON) INFO (pidfile fd = 3)
Jul 09 18:32:56 pve systemd[1]: Starting pve-firewall.service - Proxmox VE firewall...
Jul 09 18:32:56 pve cron[1445]: (CRON) INFO (Running @reboot jobs)
Jul 09 18:32:56 pve systemd[1]: Starting pvedaemon.service - PVE API Daemon...
Jul 09 18:32:56 pve systemd[1]: Starting pvestatd.service - PVE Status Daemon...
Jul 09 18:32:56 pve pvestatd[1452]: starting server
Jul 09 18:32:57 pve systemd[1]: Started pvestatd.service - PVE Status Daemon.
Jul 09 18:32:57 pve pve-firewall[1453]: starting server
Jul 09 18:32:57 pve systemd[1]: Started pve-firewall.service - Proxmox VE firewall.
Jul 09 18:32:57 pve kernel: bpfilter: Loaded bpfilter_umh pid 1457
Jul 09 18:32:57 pve unknown: Started bpfilter
Jul 09 18:32:57 pve kernel: ixgbe 0000:01:00.1 ens11f1: NIC Link is Up 1 Gbps, Flow Control: None
Jul 09 18:32:57 pve kernel: vmbr0: port 1(ens11f1) entered blocking state
Jul 09 18:32:57 pve kernel: vmbr0: port 1(ens11f1) entered forwarding state
Jul 09 18:32:57 pve pvedaemon[1482]: starting server
Jul 09 18:32:57 pve pvedaemon[1482]: starting 3 worker(s)
Jul 09 18:32:57 pve pvedaemon[1482]: worker 1483 started
Jul 09 18:32:57 pve pvedaemon[1482]: worker 1484 started
Jul 09 18:32:57 pve pvedaemon[1482]: worker 1485 started
Jul 09 18:32:57 pve systemd[1]: Started pvedaemon.service - PVE API Daemon.
Jul 09 18:32:57 pve systemd[1]: Starting pve-ha-crm.service - PVE Cluster HA Resource Manager Daemon...
Jul 09 18:32:57 pve systemd[1]: Starting pveproxy.service - PVE API Proxy Server...
Jul 09 18:32:58 pve pve-ha-crm[1491]: starting server
Jul 09 18:32:58 pve pve-ha-crm[1491]: status change startup => wait_for_quorum
Jul 09 18:32:58 pve systemd[1]: Started pve-ha-crm.service - PVE Cluster HA Resource Manager Daemon.
Jul 09 18:32:58 pve pveproxy[1492]: starting server
Jul 09 18:32:58 pve pveproxy[1492]: starting 3 worker(s)
Jul 09 18:32:58 pve pveproxy[1492]: worker 1493 started
Jul 09 18:32:58 pve pveproxy[1492]: worker 1494 started
Jul 09 18:32:58 pve pveproxy[1492]: worker 1495 started
Jul 09 18:32:58 pve systemd[1]: Started pveproxy.service - PVE API Proxy Server.
Jul 09 18:32:58 pve systemd[1]: Starting pve-ha-lrm.service - PVE Local HA Resource Manager Daemon...
Jul 09 18:32:58 pve systemd[1]: Starting spiceproxy.service - PVE SPICE Proxy Server...
Jul 09 18:32:59 pve spiceproxy[1498]: starting server
Jul 09 18:32:59 pve spiceproxy[1498]: starting 1 worker(s)
Jul 09 18:32:59 pve spiceproxy[1498]: worker 1499 started
Jul 09 18:32:59 pve systemd[1]: Started spiceproxy.service - PVE SPICE Proxy Server.
Jul 09 18:32:59 pve pve-ha-lrm[1500]: starting server
Jul 09 18:32:59 pve pve-ha-lrm[1500]: status change startup => wait_for_agent_lock
Jul 09 18:32:59 pve systemd[1]: Started pve-ha-lrm.service - PVE Local HA Resource Manager Daemon.
Jul 09 18:32:59 pve systemd[1]: Starting pve-guests.service - PVE guests...
Jul 09 18:33:00 pve pve-guests[1502]: <root@pam> starting task UPID:pve:000005DF:00000612:668D11DC:startall::root@pam:
Jul 09 18:33:00 pve pve-guests[1502]: <root@pam> end task UPID:pve:000005DF:00000612:668D11DC:startall::root@pam: OK
Jul 09 18:33:00 pve systemd[1]: Finished pve-guests.service - PVE guests.
Jul 09 18:33:00 pve systemd[1]: Starting pvescheduler.service - Proxmox VE scheduler...
Jul 09 18:33:01 pve pvescheduler[1505]: starting server
Jul 09 18:33:01 pve systemd[1]: Started pvescheduler.service - Proxmox VE scheduler.
Jul 09 18:33:01 pve systemd[1]: Reached target multi-user.target - Multi-User System.
Jul 09 18:33:01 pve systemd[1]: Reached target graphical.target - Graphical Interface.
Jul 09 18:33:01 pve systemd[1]: Starting systemd-update-utmp-runlevel.service - Record Runlevel Change in UTMP...
Jul 09 18:33:01 pve systemd[1]: systemd-update-utmp-runlevel.service: Deactivated successfully.
Jul 09 18:33:01 pve systemd[1]: Finished systemd-update-utmp-runlevel.service - Record Runlevel Change in UTMP.
Jul 09 18:33:01 pve systemd[1]: Startup finished in 38.187s (firmware) + 8.319s (loader) + 4.966s (kernel) + 11.513s (userspace) = 1min 2.988s.
Jul 09 18:33:03 pve chronyd[1263]: Selected source 211.68.71.118 (2.debian.pool.ntp.org)
Jul 09 18:33:03 pve chronyd[1263]: System clock TAI offset set to 37 seconds
Jul 09 18:33:07 pve kernel: kvm[1511]: memfd_create() called without MFD_EXEC or MFD_NOEXEC_SEAL set
Jul 09 18:33:07 pve kernel: kvm_intel: L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.
Jul 09 18:33:21 pve systemd[1]: systemd-fsckd.service: Deactivated successfully.
Jul 09 18:34:25 pve pvedaemon[1483]: <root@pam> successful auth for user 'root@pam'
Jul 09 18:36:49 pve kernel: INFO: task nvidia-vgpud:1123 blocked for more than 120 seconds.
Jul 09 18:36:49 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:36:49 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:36:49 pve kernel: task:nvidia-vgpud state:D stack:0 pid:1123 ppid:1 flags:0x00000002
Jul 09 18:36:49 pve kernel: Call Trace:
Jul 09 18:36:49 pve kernel: <TASK>
Jul 09 18:36:49 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:36:49 pve kernel: ? __kmem_cache_alloc_node+0x1aa/0x360
Jul 09 18:36:49 pve kernel: ? os_alloc_mem+0xdd/0x100 [nvidia]
Jul 09 18:36:49 pve kernel: schedule+0x63/0x110
Jul 09 18:36:49 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:36:49 pve kernel: __down_common+0x111/0x210
Jul 09 18:36:49 pve kernel: __down+0x1d/0x30
Jul 09 18:36:49 pve kernel: down+0x54/0x80
Jul 09 18:36:49 pve kernel: nvidia_frontend_open+0x29/0xb0 [nvidia]
Jul 09 18:36:49 pve kernel: chrdev_open+0xcb/0x250
Jul 09 18:36:49 pve kernel: ? fsnotify_perm.part.0+0x83/0x200
Jul 09 18:36:49 pve kernel: ? __pfx_chrdev_open+0x10/0x10
Jul 09 18:36:49 pve kernel: do_dentry_open+0x220/0x530
Jul 09 18:36:49 pve kernel: vfs_open+0x33/0x50
Jul 09 18:36:49 pve kernel: path_openat+0xb1c/0x1180
Jul 09 18:36:49 pve kernel: ? chacha_block_generic+0x6d/0xc0
Jul 09 18:36:49 pve kernel: ? _get_random_bytes+0xcf/0x1b0
Jul 09 18:36:49 pve kernel: do_filp_open+0xaf/0x170
Jul 09 18:36:49 pve kernel: do_sys_openat2+0xb3/0xe0
Jul 09 18:36:49 pve kernel: __x64_sys_openat+0x6c/0xa0
Jul 09 18:36:49 pve kernel: do_syscall_64+0x5b/0x90
Jul 09 18:36:49 pve kernel: ? do_symlinkat+0xd6/0x150
Jul 09 18:36:49 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:36:49 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:36:49 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:36:49 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:36:49 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:36:49 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:36:49 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:36:49 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:36:49 pve kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Jul 09 18:36:49 pve kernel: RIP: 0033:0x7f0423bedf01
Jul 09 18:36:49 pve kernel: RSP: 002b:00007fff7d7ef520 EFLAGS: 00000202 ORIG_RAX: 0000000000000101
Jul 09 18:36:49 pve kernel: RAX: ffffffffffffffda RBX: 0000000000080002 RCX: 00007f0423bedf01
Jul 09 18:36:49 pve kernel: RDX: 0000000000080002 RSI: 00007fff7d7ef5b0 RDI: 00000000ffffff9c
Jul 09 18:36:49 pve kernel: RBP: 00007fff7d7ef5b0 R08: 0000000000000000 R09: 0000000000000064
Jul 09 18:36:49 pve kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007fff7d7ef660
Jul 09 18:36:49 pve kernel: R13: 00000000c1d00008 R14: 00000000d0040802 R15: 00000000c1d00008
Jul 09 18:36:49 pve kernel: </TASK>
Jul 09 18:36:49 pve kernel: INFO: task nv_queue:1230 blocked for more than 120 seconds.
Jul 09 18:36:49 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:36:49 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:36:49 pve kernel: task:nv_queue state:D stack:0 pid:1230 ppid:2 flags:0x00004000
Jul 09 18:36:49 pve kernel: Call Trace:
Jul 09 18:36:49 pve kernel: <TASK>
Jul 09 18:36:49 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:36:49 pve kernel: ? _nv010522rm+0xd0/0x250 [nvidia]
Jul 09 18:36:49 pve kernel: schedule+0x63/0x110
Jul 09 18:36:49 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:36:49 pve kernel: __down_common+0x111/0x210
Jul 09 18:36:49 pve kernel: ? finish_task_switch.isra.0+0x85/0x2c0
Jul 09 18:36:49 pve kernel: __down+0x1d/0x30
Jul 09 18:36:49 pve kernel: down+0x54/0x80
Jul 09 18:36:49 pve kernel: os_acquire_mutex+0x3c/0x70 [nvidia]
Jul 09 18:36:49 pve kernel: _nv042338rm+0x10/0x40 [nvidia]
Jul 09 18:36:49 pve kernel: ? _nv013205rm+0x64d/0x7d0 [nvidia]
Jul 09 18:36:49 pve kernel: ? _nv043295rm+0x122/0x180 [nvidia]
Jul 09 18:36:49 pve kernel: ? _nv048990rm+0xeb/0x260 [nvidia]
Jul 09 18:36:49 pve kernel: ? rm_execute_work_item+0x5e/0x130 [nvidia]
Jul 09 18:36:49 pve kernel: ? os_execute_work_item+0x6c/0x90 [nvidia]
Jul 09 18:36:49 pve kernel: ? _main_loop+0x82/0x140 [nvidia]
Jul 09 18:36:49 pve kernel: ? __pfx__main_loop+0x10/0x10 [nvidia]
Jul 09 18:36:49 pve kernel: ? kthread+0xf2/0x120
Jul 09 18:36:49 pve kernel: ? __pfx_kthread+0x10/0x10
Jul 09 18:36:49 pve kernel: ? ret_from_fork+0x47/0x70
Jul 09 18:36:49 pve kernel: ? __pfx_kthread+0x10/0x10
Jul 09 18:36:49 pve kernel: ? ret_from_fork_asm+0x1b/0x30
Jul 09 18:36:49 pve kernel: </TASK>
Jul 09 18:36:49 pve kernel: INFO: task (agetty):1251 blocked for more than 120 seconds.
Jul 09 18:36:49 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:36:49 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:36:49 pve kernel: task:(agetty) state:D stack:0 pid:1251 ppid:1 flags:0x00000002
Jul 09 18:36:49 pve kernel: Call Trace:
Jul 09 18:36:49 pve kernel: <TASK>
Jul 09 18:36:49 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:36:49 pve kernel: ? hrtimer_try_to_cancel+0x87/0x120
Jul 09 18:36:49 pve kernel: ? schedule_hrtimeout_range_clock+0xc4/0x130
Jul 09 18:36:49 pve kernel: schedule+0x63/0x110
Jul 09 18:36:49 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:36:49 pve kernel: __down_common+0x111/0x210
Jul 09 18:36:49 pve kernel: __down+0x1d/0x30
Jul 09 18:36:49 pve kernel: down+0x54/0x80
Jul 09 18:36:49 pve kernel: console_lock+0x25/0x80
Jul 09 18:36:49 pve kernel: con_install+0x21/0x130
Jul 09 18:36:49 pve kernel: tty_init_dev.part.0+0x4e/0x280
Jul 09 18:36:49 pve kernel: tty_open+0x48d/0x6f0
Jul 09 18:36:49 pve kernel: chrdev_open+0xcb/0x250
Jul 09 18:36:49 pve kernel: ? fsnotify_perm.part.0+0x83/0x200
Jul 09 18:36:49 pve kernel: ? __pfx_chrdev_open+0x10/0x10
Jul 09 18:36:49 pve kernel: do_dentry_open+0x220/0x530
Jul 09 18:36:49 pve kernel: vfs_open+0x33/0x50
Jul 09 18:36:49 pve kernel: path_openat+0xb1c/0x1180
Jul 09 18:36:49 pve kernel: do_filp_open+0xaf/0x170
Jul 09 18:36:49 pve kernel: do_sys_openat2+0xb3/0xe0
Jul 09 18:36:49 pve kernel: __x64_sys_openat+0x6c/0xa0
Jul 09 18:36:49 pve kernel: do_syscall_64+0x5b/0x90
Jul 09 18:36:49 pve kernel: ? irqentry_exit_to_user_mode+0x17/0x20
Jul 09 18:36:49 pve kernel: ? irqentry_exit+0x43/0x50
Jul 09 18:36:49 pve kernel: ? exc_page_fault+0x94/0x1b0
Jul 09 18:36:49 pve kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Jul 09 18:36:49 pve kernel: RIP: 0033:0x7f25a2116f80
Jul 09 18:36:49 pve kernel: RSP: 002b:00007ffdb0e85060 EFLAGS: 00000293 ORIG_RAX: 0000000000000101
Jul 09 18:36:49 pve kernel: RAX: ffffffffffffffda RBX: 0000000000080902 RCX: 00007f25a2116f80
Jul 09 18:36:49 pve kernel: RDX: 0000000000080902 RSI: 0000555822eee780 RDI: 00000000ffffff9c
Jul 09 18:36:49 pve kernel: RBP: 0000555822eee780 R08: 0000000000000000 R09: 00007ffdb0e85150
Jul 09 18:36:49 pve kernel: R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000080902
Jul 09 18:36:49 pve kernel: R13: 0000555822eee780 R14: 00007ffdb0e85680 R15: 0000555822ee9510
Jul 09 18:36:49 pve kernel: </TASK>
Jul 09 18:37:13 pve pvedaemon[1484]: <root@pam> successful auth for user 'root@pam'
Jul 09 18:38:50 pve kernel: INFO: task nvidia-vgpud:1123 blocked for more than 241 seconds.
Jul 09 18:38:50 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:38:50 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:38:50 pve kernel: task:nvidia-vgpud state:D stack:0 pid:1123 ppid:1 flags:0x00000002
Jul 09 18:38:50 pve kernel: Call Trace:
Jul 09 18:38:50 pve kernel: <TASK>
Jul 09 18:38:50 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:38:50 pve kernel: ? __kmem_cache_alloc_node+0x1aa/0x360
Jul 09 18:38:50 pve kernel: ? os_alloc_mem+0xdd/0x100 [nvidia]
Jul 09 18:38:50 pve kernel: schedule+0x63/0x110
Jul 09 18:38:50 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:38:50 pve kernel: __down_common+0x111/0x210
Jul 09 18:38:50 pve kernel: __down+0x1d/0x30
Jul 09 18:38:50 pve kernel: down+0x54/0x80
Jul 09 18:38:50 pve kernel: nvidia_frontend_open+0x29/0xb0 [nvidia]
Jul 09 18:38:50 pve kernel: chrdev_open+0xcb/0x250
Jul 09 18:38:50 pve kernel: ? fsnotify_perm.part.0+0x83/0x200
Jul 09 18:38:50 pve kernel: ? __pfx_chrdev_open+0x10/0x10
Jul 09 18:38:50 pve kernel: do_dentry_open+0x220/0x530
Jul 09 18:38:50 pve kernel: vfs_open+0x33/0x50
Jul 09 18:38:50 pve kernel: path_openat+0xb1c/0x1180
Jul 09 18:38:50 pve kernel: ? chacha_block_generic+0x6d/0xc0
Jul 09 18:38:50 pve kernel: ? _get_random_bytes+0xcf/0x1b0
Jul 09 18:38:50 pve kernel: do_filp_open+0xaf/0x170
Jul 09 18:38:50 pve kernel: do_sys_openat2+0xb3/0xe0
Jul 09 18:38:50 pve kernel: __x64_sys_openat+0x6c/0xa0
Jul 09 18:38:50 pve kernel: do_syscall_64+0x5b/0x90
Jul 09 18:38:50 pve kernel: ? do_symlinkat+0xd6/0x150
Jul 09 18:38:50 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:38:50 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:38:50 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:38:50 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:38:50 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:38:50 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:38:50 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:38:50 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:38:50 pve kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Jul 09 18:38:50 pve kernel: RIP: 0033:0x7f0423bedf01
Jul 09 18:38:50 pve kernel: RSP: 002b:00007fff7d7ef520 EFLAGS: 00000202 ORIG_RAX: 0000000000000101
Jul 09 18:38:50 pve kernel: RAX: ffffffffffffffda RBX: 0000000000080002 RCX: 00007f0423bedf01
Jul 09 18:38:50 pve kernel: RDX: 0000000000080002 RSI: 00007fff7d7ef5b0 RDI: 00000000ffffff9c
Jul 09 18:38:50 pve kernel: RBP: 00007fff7d7ef5b0 R08: 0000000000000000 R09: 0000000000000064
Jul 09 18:38:50 pve kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007fff7d7ef660
Jul 09 18:38:50 pve kernel: R13: 00000000c1d00008 R14: 00000000d0040802 R15: 00000000c1d00008
Jul 09 18:38:50 pve kernel: </TASK>
Jul 09 18:38:50 pve kernel: INFO: task nv_queue:1230 blocked for more than 241 seconds.
Jul 09 18:38:50 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:38:50 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:38:50 pve kernel: task:nv_queue state:D stack:0 pid:1230 ppid:2 flags:0x00004000
Jul 09 18:38:50 pve kernel: Call Trace:
Jul 09 18:38:50 pve kernel: <TASK>
Jul 09 18:38:50 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:38:50 pve kernel: ? _nv010522rm+0xd0/0x250 [nvidia]
Jul 09 18:38:50 pve kernel: schedule+0x63/0x110
Jul 09 18:38:50 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:38:50 pve kernel: __down_common+0x111/0x210
Jul 09 18:38:50 pve kernel: ? finish_task_switch.isra.0+0x85/0x2c0
Jul 09 18:38:50 pve kernel: __down+0x1d/0x30
Jul 09 18:38:50 pve kernel: down+0x54/0x80
Jul 09 18:38:50 pve kernel: os_acquire_mutex+0x3c/0x70 [nvidia]
Jul 09 18:38:50 pve kernel: _nv042338rm+0x10/0x40 [nvidia]
Jul 09 18:38:50 pve kernel: ? _nv013205rm+0x64d/0x7d0 [nvidia]
Jul 09 18:38:50 pve kernel: ? _nv043295rm+0x122/0x180 [nvidia]
Jul 09 18:38:50 pve kernel: ? _nv048990rm+0xeb/0x260 [nvidia]
Jul 09 18:38:50 pve kernel: ? rm_execute_work_item+0x5e/0x130 [nvidia]
Jul 09 18:38:50 pve kernel: ? os_execute_work_item+0x6c/0x90 [nvidia]
Jul 09 18:38:50 pve kernel: ? _main_loop+0x82/0x140 [nvidia]
Jul 09 18:38:50 pve kernel: ? __pfx__main_loop+0x10/0x10 [nvidia]
Jul 09 18:38:50 pve kernel: ? kthread+0xf2/0x120
Jul 09 18:38:50 pve kernel: ? __pfx_kthread+0x10/0x10
Jul 09 18:38:50 pve kernel: ? ret_from_fork+0x47/0x70
Jul 09 18:38:50 pve kernel: ? __pfx_kthread+0x10/0x10
Jul 09 18:38:50 pve kernel: ? ret_from_fork_asm+0x1b/0x30
Jul 09 18:38:50 pve kernel: </TASK>
Jul 09 18:38:50 pve kernel: INFO: task (agetty):1251 blocked for more than 241 seconds.
Jul 09 18:38:50 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:38:50 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:38:50 pve kernel: task:(agetty) state:D stack:0 pid:1251 ppid:1 flags:0x00000002
Jul 09 18:38:50 pve kernel: Call Trace:
Jul 09 18:38:50 pve kernel: <TASK>
Jul 09 18:38:50 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:38:50 pve kernel: ? hrtimer_try_to_cancel+0x87/0x120
Jul 09 18:38:50 pve kernel: ? schedule_hrtimeout_range_clock+0xc4/0x130
Jul 09 18:38:50 pve kernel: schedule+0x63/0x110
Jul 09 18:38:50 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:38:50 pve kernel: __down_common+0x111/0x210
Jul 09 18:38:50 pve kernel: __down+0x1d/0x30
Jul 09 18:38:50 pve kernel: down+0x54/0x80
Jul 09 18:38:50 pve kernel: console_lock+0x25/0x80
Jul 09 18:38:50 pve kernel: con_install+0x21/0x130
Jul 09 18:38:50 pve kernel: tty_init_dev.part.0+0x4e/0x280
Jul 09 18:38:50 pve kernel: tty_open+0x48d/0x6f0
Jul 09 18:38:50 pve kernel: chrdev_open+0xcb/0x250
Jul 09 18:38:50 pve kernel: ? fsnotify_perm.part.0+0x83/0x200
Jul 09 18:38:50 pve kernel: ? __pfx_chrdev_open+0x10/0x10
Jul 09 18:38:50 pve kernel: do_dentry_open+0x220/0x530
Jul 09 18:38:50 pve kernel: vfs_open+0x33/0x50
Jul 09 18:38:50 pve kernel: path_openat+0xb1c/0x1180
Jul 09 18:38:50 pve kernel: do_filp_open+0xaf/0x170
Jul 09 18:38:50 pve kernel: do_sys_openat2+0xb3/0xe0
Jul 09 18:38:50 pve kernel: __x64_sys_openat+0x6c/0xa0
Jul 09 18:38:50 pve kernel: do_syscall_64+0x5b/0x90
Jul 09 18:38:50 pve kernel: ? irqentry_exit_to_user_mode+0x17/0x20
Jul 09 18:38:50 pve kernel: ? irqentry_exit+0x43/0x50
Jul 09 18:38:50 pve kernel: ? exc_page_fault+0x94/0x1b0
Jul 09 18:38:50 pve kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Jul 09 18:38:50 pve kernel: RIP: 0033:0x7f25a2116f80
Jul 09 18:38:50 pve kernel: RSP: 002b:00007ffdb0e85060 EFLAGS: 00000293 ORIG_RAX: 0000000000000101
Jul 09 18:38:50 pve kernel: RAX: ffffffffffffffda RBX: 0000000000080902 RCX: 00007f25a2116f80
Jul 09 18:38:50 pve kernel: RDX: 0000000000080902 RSI: 0000555822eee780 RDI: 00000000ffffff9c
Jul 09 18:38:50 pve kernel: RBP: 0000555822eee780 R08: 0000000000000000 R09: 00007ffdb0e85150
Jul 09 18:38:50 pve kernel: R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000080902
Jul 09 18:38:50 pve kernel: R13: 0000555822eee780 R14: 00007ffdb0e85680 R15: 0000555822ee9510
Jul 09 18:38:50 pve kernel: </TASK>
Jul 09 18:40:51 pve kernel: INFO: task nvidia-vgpud:1123 blocked for more than 362 seconds.
Jul 09 18:40:51 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:40:51 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:40:51 pve kernel: task:nvidia-vgpud state:D stack:0 pid:1123 ppid:1 flags:0x00000002
Jul 09 18:40:51 pve kernel: Call Trace:
Jul 09 18:40:51 pve kernel: <TASK>
Jul 09 18:40:51 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:40:51 pve kernel: ? __kmem_cache_alloc_node+0x1aa/0x360
Jul 09 18:40:51 pve kernel: ? os_alloc_mem+0xdd/0x100 [nvidia]
Jul 09 18:40:51 pve kernel: schedule+0x63/0x110
Jul 09 18:40:51 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:40:51 pve kernel: __down_common+0x111/0x210
Jul 09 18:40:51 pve kernel: __down+0x1d/0x30
Jul 09 18:40:51 pve kernel: down+0x54/0x80
Jul 09 18:40:51 pve kernel: nvidia_frontend_open+0x29/0xb0 [nvidia]
Jul 09 18:40:51 pve kernel: chrdev_open+0xcb/0x250
Jul 09 18:40:51 pve kernel: ? fsnotify_perm.part.0+0x83/0x200
Jul 09 18:40:51 pve kernel: ? __pfx_chrdev_open+0x10/0x10
Jul 09 18:40:51 pve kernel: do_dentry_open+0x220/0x530
Jul 09 18:40:51 pve kernel: vfs_open+0x33/0x50
Jul 09 18:40:51 pve kernel: path_openat+0xb1c/0x1180
Jul 09 18:40:51 pve kernel: ? chacha_block_generic+0x6d/0xc0
Jul 09 18:40:51 pve kernel: ? _get_random_bytes+0xcf/0x1b0
Jul 09 18:40:51 pve kernel: do_filp_open+0xaf/0x170
Jul 09 18:40:51 pve kernel: do_sys_openat2+0xb3/0xe0
Jul 09 18:40:51 pve kernel: __x64_sys_openat+0x6c/0xa0
Jul 09 18:40:51 pve kernel: do_syscall_64+0x5b/0x90
Jul 09 18:40:51 pve kernel: ? do_symlinkat+0xd6/0x150
Jul 09 18:40:51 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:40:51 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:40:51 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:40:51 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:40:51 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:40:51 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:40:51 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:40:51 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:40:51 pve kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Jul 09 18:40:51 pve kernel: RIP: 0033:0x7f0423bedf01
Jul 09 18:40:51 pve kernel: RSP: 002b:00007fff7d7ef520 EFLAGS: 00000202 ORIG_RAX: 0000000000000101
Jul 09 18:40:51 pve kernel: RAX: ffffffffffffffda RBX: 0000000000080002 RCX: 00007f0423bedf01
Jul 09 18:40:51 pve kernel: RDX: 0000000000080002 RSI: 00007fff7d7ef5b0 RDI: 00000000ffffff9c
Jul 09 18:40:51 pve kernel: RBP: 00007fff7d7ef5b0 R08: 0000000000000000 R09: 0000000000000064
Jul 09 18:40:51 pve kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007fff7d7ef660
Jul 09 18:40:51 pve kernel: R13: 00000000c1d00008 R14: 00000000d0040802 R15: 00000000c1d00008
Jul 09 18:40:51 pve kernel: </TASK>
Jul 09 18:40:51 pve kernel: INFO: task nv_queue:1230 blocked for more than 362 seconds.
Jul 09 18:40:51 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:40:51 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:40:51 pve kernel: task:nv_queue state:D stack:0 pid:1230 ppid:2 flags:0x00004000
Jul 09 18:40:51 pve kernel: Call Trace:
Jul 09 18:40:51 pve kernel: <TASK>
Jul 09 18:40:51 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:40:51 pve kernel: ? _nv010522rm+0xd0/0x250 [nvidia]
Jul 09 18:40:51 pve kernel: schedule+0x63/0x110
Jul 09 18:40:51 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:40:51 pve kernel: __down_common+0x111/0x210
Jul 09 18:40:51 pve kernel: ? finish_task_switch.isra.0+0x85/0x2c0
Jul 09 18:40:51 pve kernel: __down+0x1d/0x30
Jul 09 18:40:51 pve kernel: down+0x54/0x80
Jul 09 18:40:51 pve kernel: os_acquire_mutex+0x3c/0x70 [nvidia]
Jul 09 18:40:51 pve kernel: _nv042338rm+0x10/0x40 [nvidia]
Jul 09 18:40:51 pve kernel: ? _nv013205rm+0x64d/0x7d0 [nvidia]
Jul 09 18:40:51 pve kernel: ? _nv043295rm+0x122/0x180 [nvidia]
Jul 09 18:40:51 pve kernel: ? _nv048990rm+0xeb/0x260 [nvidia]
Jul 09 18:40:51 pve kernel: ? rm_execute_work_item+0x5e/0x130 [nvidia]
Jul 09 18:40:51 pve kernel: ? os_execute_work_item+0x6c/0x90 [nvidia]
Jul 09 18:40:51 pve kernel: ? _main_loop+0x82/0x140 [nvidia]
Jul 09 18:40:51 pve kernel: ? __pfx__main_loop+0x10/0x10 [nvidia]
Jul 09 18:40:51 pve kernel: ? kthread+0xf2/0x120
Jul 09 18:40:51 pve kernel: ? __pfx_kthread+0x10/0x10
Jul 09 18:40:51 pve kernel: ? ret_from_fork+0x47/0x70
Jul 09 18:40:51 pve kernel: ? __pfx_kthread+0x10/0x10
Jul 09 18:40:51 pve kernel: ? ret_from_fork_asm+0x1b/0x30
Jul 09 18:40:51 pve kernel: </TASK>
Jul 09 18:40:51 pve kernel: INFO: task (agetty):1251 blocked for more than 362 seconds.
Jul 09 18:40:51 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:40:51 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:40:51 pve kernel: task:(agetty) state:D stack:0 pid:1251 ppid:1 flags:0x00000002
Jul 09 18:40:51 pve kernel: Call Trace:
Jul 09 18:40:51 pve kernel: <TASK>
Jul 09 18:40:51 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:40:51 pve kernel: ? hrtimer_try_to_cancel+0x87/0x120
Jul 09 18:40:51 pve kernel: ? schedule_hrtimeout_range_clock+0xc4/0x130
Jul 09 18:40:51 pve kernel: schedule+0x63/0x110
Jul 09 18:40:51 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:40:51 pve kernel: __down_common+0x111/0x210
Jul 09 18:40:51 pve kernel: __down+0x1d/0x30
Jul 09 18:40:51 pve kernel: down+0x54/0x80
Jul 09 18:40:51 pve kernel: console_lock+0x25/0x80
Jul 09 18:40:51 pve kernel: con_install+0x21/0x130
Jul 09 18:40:51 pve kernel: tty_init_dev.part.0+0x4e/0x280
Jul 09 18:40:51 pve kernel: tty_open+0x48d/0x6f0
Jul 09 18:40:51 pve kernel: chrdev_open+0xcb/0x250
Jul 09 18:40:51 pve kernel: ? fsnotify_perm.part.0+0x83/0x200
Jul 09 18:40:51 pve kernel: ? __pfx_chrdev_open+0x10/0x10
Jul 09 18:40:51 pve kernel: do_dentry_open+0x220/0x530
Jul 09 18:40:51 pve kernel: vfs_open+0x33/0x50
Jul 09 18:40:51 pve kernel: path_openat+0xb1c/0x1180
Jul 09 18:40:51 pve kernel: do_filp_open+0xaf/0x170
Jul 09 18:40:51 pve kernel: do_sys_openat2+0xb3/0xe0
Jul 09 18:40:51 pve kernel: __x64_sys_openat+0x6c/0xa0
Jul 09 18:40:51 pve kernel: do_syscall_64+0x5b/0x90
Jul 09 18:40:51 pve kernel: ? irqentry_exit_to_user_mode+0x17/0x20
Jul 09 18:40:51 pve kernel: ? irqentry_exit+0x43/0x50
Jul 09 18:40:51 pve kernel: ? exc_page_fault+0x94/0x1b0
Jul 09 18:40:51 pve kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Jul 09 18:40:51 pve kernel: RIP: 0033:0x7f25a2116f80
Jul 09 18:40:51 pve kernel: RSP: 002b:00007ffdb0e85060 EFLAGS: 00000293 ORIG_RAX: 0000000000000101
Jul 09 18:40:51 pve kernel: RAX: ffffffffffffffda RBX: 0000000000080902 RCX: 00007f25a2116f80
Jul 09 18:40:51 pve kernel: RDX: 0000000000080902 RSI: 0000555822eee780 RDI: 00000000ffffff9c
Jul 09 18:40:51 pve kernel: RBP: 0000555822eee780 R08: 0000000000000000 R09: 00007ffdb0e85150
Jul 09 18:40:51 pve kernel: R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000080902
Jul 09 18:40:51 pve kernel: R13: 0000555822eee780 R14: 00007ffdb0e85680 R15: 0000555822ee9510
Jul 09 18:40:51 pve kernel: </TASK>
Jul 09 18:42:44 pve chronyd[1263]: Selected source 139.199.214.202 (2.debian.pool.ntp.org)
Jul 09 18:42:52 pve kernel: INFO: task nvidia-vgpud:1123 blocked for more than 483 seconds.
Jul 09 18:42:52 pve kernel: Tainted: P OE 6.5.11-8-pve #1
Jul 09 18:42:52 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 09 18:42:52 pve kernel: task:nvidia-vgpud state:D stack:0 pid:1123 ppid:1 flags:0x00000002
Jul 09 18:42:52 pve kernel: Call Trace:
Jul 09 18:42:52 pve kernel: <TASK>
Jul 09 18:42:52 pve kernel: __schedule+0x3fd/0x1450
Jul 09 18:42:52 pve kernel: ? __kmem_cache_alloc_node+0x1aa/0x360
Jul 09 18:42:52 pve kernel: ? os_alloc_mem+0xdd/0x100 [nvidia]
Jul 09 18:42:52 pve kernel: schedule+0x63/0x110
Jul 09 18:42:52 pve kernel: schedule_timeout+0x157/0x170
Jul 09 18:42:52 pve kernel: __down_common+0x111/0x210
Jul 09 18:42:52 pve kernel: __down+0x1d/0x30
Jul 09 18:42:52 pve kernel: down+0x54/0x80
Jul 09 18:42:52 pve kernel: nvidia_frontend_open+0x29/0xb0 [nvidia]
Jul 09 18:42:52 pve kernel: chrdev_open+0xcb/0x250
Jul 09 18:42:52 pve kernel: ? fsnotify_perm.part.0+0x83/0x200
Jul 09 18:42:52 pve kernel: ? __pfx_chrdev_open+0x10/0x10
Jul 09 18:42:52 pve kernel: do_dentry_open+0x220/0x530
Jul 09 18:42:52 pve kernel: vfs_open+0x33/0x50
Jul 09 18:42:52 pve kernel: path_openat+0xb1c/0x1180
Jul 09 18:42:52 pve kernel: ? chacha_block_generic+0x6d/0xc0
Jul 09 18:42:52 pve kernel: ? _get_random_bytes+0xcf/0x1b0
Jul 09 18:42:52 pve kernel: do_filp_open+0xaf/0x170
Jul 09 18:42:52 pve kernel: do_sys_openat2+0xb3/0xe0
Jul 09 18:42:52 pve kernel: __x64_sys_openat+0x6c/0xa0
Jul 09 18:42:52 pve kernel: do_syscall_64+0x5b/0x90
Jul 09 18:42:52 pve kernel: ? do_symlinkat+0xd6/0x150
Jul 09 18:42:52 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:42:52 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:42:52 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:42:52 pve kernel: ? exit_to_user_mode_prepare+0x39/0x190
Jul 09 18:42:52 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:42:52 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:42:52 pve kernel: ? syscall_exit_to_user_mode+0x37/0x60
Jul 09 18:42:52 pve kernel: ? do_syscall_64+0x67/0x90
Jul 09 18:42:52 pve kernel: entry_SYSCALL_64_after_hwframe+0x6e/0xd8
Jul 09 18:42:52 pve kernel: RIP: 0033:0x7f0423bedf01
Jul 09 18:42:52 pve kernel: RSP: 002b:00007fff7d7ef520 EFLAGS: 00000202 ORIG_RAX: 0000000000000101
Jul 09 18:42:52 pve kernel: RAX: ffffffffffffffda RBX: 0000000000080002 RCX: 00007f0423bedf01
Jul 09 18:42:52 pve kernel: RDX: 0000000000080002 RSI: 00007fff7d7ef5b0 RDI: 00000000ffffff9c
Jul 09 18:42:52 pve kernel: RBP: 00007fff7d7ef5b0 R08: 0000000000000000 R09: 0000000000000064
Jul 09 18:42:52 pve kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007fff7d7ef660
Jul 09 18:42:52 pve kernel: R13: 00000000c1d00008 R14: 00000000d0040802 R15: 00000000c1d00008
Jul 09 18:42:52 pve kernel: </TASK>
Jul 09 18:42:52 pve kernel: Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
Jul 09 18:48:33 pve systemd[1]: Starting systemd-tmpfiles-clean.service - Cleanup of Temporary Directories...
Jul 09 18:48:33 pve systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.
Jul 09 18:48:33 pve systemd[1]: Finished systemd-tmpfiles-clean.service - Cleanup of Temporary Directories.
Jul 09 18:48:33 pve systemd[1]: run-credentials-systemd\x2dtmpfiles\x2dclean.service.mount: Deactivated successfully.
Jul 09 18:49:26 pve pvedaemon[1484]: <root@pam> successful auth for user 'root@pam'
Jul 09 18:53:13 pve pvedaemon[1485]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:04:27 pve pvedaemon[1484]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:06:06 pve pveproxy[1495]: worker exit
Jul 09 19:06:06 pve pveproxy[1492]: worker 1495 finished
Jul 09 19:06:06 pve pveproxy[1492]: starting 1 worker(s)
Jul 09 19:06:06 pve pveproxy[1492]: worker 5931 started
Jul 09 19:08:13 pve pvedaemon[1485]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:08:22 pve kernel: NVRM: GPU at 0000:82:00.0 has software scheduler ENABLED with policy BEST_EFFORT.
Jul 09 19:14:45 pve pvedaemon[1483]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:17:01 pve CRON[7401]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
Jul 09 19:17:01 pve CRON[7402]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
Jul 09 19:17:01 pve CRON[7401]: pam_unix(cron:session): session closed for user root
Jul 09 19:24:13 pve pvedaemon[1483]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:29:46 pve pvedaemon[1485]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:34:22 pve pveproxy[1493]: worker exit
Jul 09 19:34:22 pve pveproxy[1492]: worker 1493 finished
Jul 09 19:34:22 pve pveproxy[1492]: starting 1 worker(s)
Jul 09 19:34:22 pve pveproxy[1492]: worker 9713 started
Jul 09 19:34:42 pve pveproxy[1494]: worker exit
Jul 09 19:34:42 pve pveproxy[1492]: worker 1494 finished
Jul 09 19:34:42 pve pveproxy[1492]: starting 1 worker(s)
Jul 09 19:34:42 pve pveproxy[1492]: worker 9756 started
Jul 09 19:38:01 pve pvedaemon[1483]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:38:03 pve sshd[10207]: Accepted password for root from 192.168.10.23 port 56082 ssh2
Jul 09 19:38:03 pve sshd[10207]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0)
Jul 09 19:38:03 pve systemd-logind[1135]: New session 2 of user root.
Jul 09 19:38:03 pve systemd[1]: Created slice user-0.slice - User Slice of UID 0.
Jul 09 19:38:03 pve systemd[1]: Starting user-runtime-dir@0.service - User Runtime Directory /run/user/0...
Jul 09 19:38:03 pve systemd[1]: Finished user-runtime-dir@0.service - User Runtime Directory /run/user/0.
Jul 09 19:38:03 pve systemd[1]: Starting user@0.service - User Manager for UID 0...
Jul 09 19:38:03 pve (systemd)[10212]: pam_unix(systemd-user:session): session opened for user root(uid=0) by (uid=0)
Jul 09 19:38:03 pve systemd[10212]: Queued start job for default target default.target.
Jul 09 19:38:03 pve systemd[10212]: Created slice app.slice - User Application Slice.
Jul 09 19:38:03 pve systemd[10212]: Reached target paths.target - Paths.
Jul 09 19:38:03 pve systemd[10212]: Reached target timers.target - Timers.
Jul 09 19:38:03 pve systemd[10212]: Listening on dirmngr.socket - GnuPG network certificate management daemon.
Jul 09 19:38:03 pve systemd[10212]: Listening on gpg-agent-browser.socket - GnuPG cryptographic agent and passphrase cache (access for web browsers).
Jul 09 19:38:03 pve systemd[10212]: Listening on gpg-agent-extra.socket - GnuPG cryptographic agent and passphrase cache (restricted).
Jul 09 19:38:03 pve systemd[10212]: Listening on gpg-agent-ssh.socket - GnuPG cryptographic agent (ssh-agent emulation).
Jul 09 19:38:03 pve systemd[10212]: Listening on gpg-agent.socket - GnuPG cryptographic agent and passphrase cache.
Jul 09 19:38:03 pve systemd[10212]: Reached target sockets.target - Sockets.
Jul 09 19:38:03 pve systemd[10212]: Reached target basic.target - Basic System.
Jul 09 19:38:03 pve systemd[10212]: Reached target default.target - Main User Target.
Jul 09 19:38:03 pve systemd[10212]: Startup finished in 120ms.
Jul 09 19:38:03 pve systemd[1]: Started user@0.service - User Manager for UID 0.
Jul 09 19:38:03 pve systemd[1]: Started session-2.scope - Session 2 of User root.
Jul 09 19:38:03 pve sshd[10207]: pam_env(sshd:session): deprecated reading of user environment enabled
Jul 09 19:38:03 pve sshd[10210]: Accepted password for root from 192.168.10.23 port 56083 ssh2
Jul 09 19:38:03 pve sshd[10210]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0)
Jul 09 19:38:04 pve systemd-logind[1135]: New session 4 of user root.
Jul 09 19:38:04 pve systemd[1]: Started session-4.scope - Session 4 of User root.
Jul 09 19:38:04 pve sshd[10210]: pam_env(sshd:session): deprecated reading of user environment enabled
Jul 09 19:38:24 pve sshd[10207]: pam_unix(sshd:session): session closed for user root
Jul 09 19:38:24 pve sshd[10207]: pam_systemd(sshd:session): Failed to release session: Interrupted system call
Jul 09 19:38:24 pve systemd-logind[1135]: Session 2 logged out. Waiting for processes to exit.
Jul 09 19:38:24 pve sshd[10210]: pam_unix(sshd:session): session closed for user root
Jul 09 19:38:24 pve systemd-logind[1135]: Session 4 logged out. Waiting for processes to exit.
Jul 09 19:38:24 pve systemd[1]: session-4.scope: Deactivated successfully.
Jul 09 19:38:24 pve systemd-logind[1135]: Removed session 4.
Jul 09 19:38:28 pve sshd[10305]: Accepted password for root from 192.168.10.23 port 56103 ssh2
Jul 09 19:38:28 pve sshd[10305]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0)
Jul 09 19:38:28 pve systemd-logind[1135]: New session 5 of user root.
Jul 09 19:38:28 pve systemd[1]: Started session-5.scope - Session 5 of User root.
Jul 09 19:38:28 pve sshd[10305]: pam_env(sshd:session): deprecated reading of user environment enabled
Jul 09 19:38:29 pve sshd[10307]: Accepted password for root from 192.168.10.23 port 56104 ssh2
Jul 09 19:38:29 pve sshd[10307]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0)
Jul 09 19:38:29 pve systemd-logind[1135]: New session 6 of user root.
Jul 09 19:38:29 pve systemd[1]: Started session-6.scope - Session 6 of User root.
Jul 09 19:38:29 pve sshd[10307]: pam_env(sshd:session): deprecated reading of user environment enabled
Jul 09 19:40:13 pve pvedaemon[1485]: <root@pam> successful auth for user 'root@pam'
Jul 09 19:43:19 pve systemd[1]: session-2.scope: Deactivated successfully.
Jul 09 19:43:19 pve systemd-logind[1135]: Removed session 2.
Jul 09 19:43:19 pve nvidia-vgpu-mgr[1118]: notice: vmiop_env_log: nvidia-vgpu-mgr daemon started
I need help，anyone knows the reasons that stuck the vGPU installing？
Thanks