Tesla T4 on a Zynq Ultrascale+ (aarch64)

I’m trying to get the aarch64 drivers working on a Zynq Ultrascale+. I can build and load the drivers, and they see the Tesla T4, but when I run nvidia-smi, it fails to initialize the device, with the kernel giving an “RmInitAdapter failed! (0x63:0xffff:2344)”. I have tried several different versions of Petalinux and different versions of the drivers, and I see similar failures.

The following dmesg output is from a setup using Petalinux 2021.2 and driver version 515.43.04, on a ZCU102 board. This is with the following nvidia.ko module options:
NVreg_EnableMSI=0
NVreg_ResmanDebugLevel=1

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 5.10.0-xilinx-v2021.2 (oe-user@oe-host) (aarch64-xilinx-linux-gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.35.1) #1 SMP Tue Oct 12 09:30:57 UTC 2021
[    0.000000] Machine model: xlnx,zynqmp
[    0.000000] earlycon: cdns0 at MMIO 0x00000000ff000000 (options '115200n8')
[    0.000000] printk: bootconsole [cdns0] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] cma: Reserved 256 MiB at 0x000000006dc00000
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000087fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000007fefffff]
[    0.000000]   node   0: [mem 0x0000000800000000-0x000000087fffffff]
[    0.000000] Zeroed struct page in unavailable ranges: 256 pages
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000087fffffff]
[    0.000000] On node 0 totalpages: 1048320
[    0.000000]   DMA32 zone: 8188 pages used for memmap
[    0.000000]   DMA32 zone: 0 pages reserved
[    0.000000]   DMA32 zone: 524032 pages, LIFO batch:63
[    0.000000]   Normal zone: 8192 pages used for memmap
[    0.000000]   Normal zone: 524288 pages, LIFO batch:63
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 22 pages/cpu s50968 r8192 d30952 u90112
[    0.000000] pcpu-alloc: s50968 r8192 d30952 u90112 alloc=22*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: ARM erratum 845719
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1031940
[    0.000000] Kernel command line:  earlycon console=ttyPS0,115200 clk_ignore_unused root=/dev/mmcblk0p2 rw rootwait
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x0000000069c00000-0x000000006dc00000] (64MB)
[    0.000000] Memory: 3763080K/4193280K available (13952K kernel code, 984K rwdata, 3928K rodata, 2112K init, 588K bss, 168056K reserved, 262144K cma-reserved)
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Adjusting CPU interface base to 0x00000000f902f000
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x31c/0x524 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 99.99MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x170f8de2d3, max_idle_ns: 440795206112 ns
[    0.000003] sched_clock: 56 bits at 99MHz, resolution 10ns, wraps every 4398046511101ns
[    0.008255] Console: colour dummy device 80x25
[    0.012395] Calibrating delay loop (skipped), value calculated using timer frequency.. 199.98 BogoMIPS (lpj=399960)
[    0.022756] pid_max: default: 32768 minimum: 301
[    0.027481] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.034705] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.043508] rcu: Hierarchical SRCU implementation.
[    0.047476] EFI services will not be available.
[    0.051879] smp: Bringing up secondary CPUs ...
[    0.056602] Detected VIPT I-cache on CPU1
[    0.056640] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[    0.057023] Detected VIPT I-cache on CPU2
[    0.057047] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[    0.057412] Detected VIPT I-cache on CPU3
[    0.057435] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[    0.057486] smp: Brought up 1 node, 4 CPUs
[    0.091703] SMP: Total of 4 processors activated.
[    0.096375] CPU features: detected: 32-bit EL0 Support
[    0.101479] CPU features: detected: CRC32 instructions
[    0.106617] CPU: All CPU(s) started at EL2
[    0.110662] alternatives: patching kernel code
[    0.116152] devtmpfs: initialized
[    0.122702] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.128051] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.141412] pinctrl core: initialized pinctrl subsystem
[    0.141898] DMI not present or invalid.
[    0.144992] NET: Registered protocol family 16
[    0.150269] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.156331] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.164138] audit: initializing netlink subsys (disabled)
[    0.169598] audit: type=2000 audit(0.108:1): state=initialized audit_enabled=0 res=1
[    0.177185] cpuidle: using governor menu
[    0.181133] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.187860] ASID allocator initialised with 65536 entries
[    0.193275] Serial: AMBA PL011 UART driver
[    0.216921] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.217986] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.224642] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.231303] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    1.283271] cryptd: max_cpu_qlen set to 1000
[    1.307615] DRBG: Continuing without Jitter RNG
[    1.386741] raid6: neonx8   gen()  2138 MB/s
[    1.454805] raid6: neonx8   xor()  1597 MB/s
[    1.522869] raid6: neonx4   gen()  2188 MB/s
[    1.590931] raid6: neonx4   xor()  1566 MB/s
[    1.658994] raid6: neonx2   gen()  2079 MB/s
[    1.727055] raid6: neonx2   xor()  1431 MB/s
[    1.795131] raid6: neonx1   gen()  1775 MB/s
[    1.863184] raid6: neonx1   xor()  1219 MB/s
[    1.931263] raid6: int64x8  gen()  1437 MB/s
[    1.999323] raid6: int64x8  xor()   771 MB/s
[    2.067386] raid6: int64x4  gen()  1558 MB/s
[    2.135450] raid6: int64x4  xor()   815 MB/s
[    2.203534] raid6: int64x2  gen()  1397 MB/s
[    2.271581] raid6: int64x2  xor()   743 MB/s
[    2.339663] raid6: int64x1  gen()  1032 MB/s
[    2.407714] raid6: int64x1  xor()   517 MB/s
[    2.407752] raid6: using algorithm neonx4 gen() 2188 MB/s
[    2.411704] raid6: .... xor() 1566 MB/s, rmw enabled
[    2.416641] raid6: using neon recovery algorithm
[    2.421660] iommu: Default domain type: Translated 
[    2.426292] SCSI subsystem initialized
[    2.429809] libata version 3.00 loaded.
[    2.429925] usbcore: registered new interface driver usbfs
[    2.435272] usbcore: registered new interface driver hub
[    2.440544] usbcore: registered new device driver usb
[    2.445588] mc: Linux media interface: v0.10
[    2.449792] videodev: Linux video capture interface: v2.00
[    2.455265] EDAC MC: Ver: 3.0.0
[    2.458708] zynqmp-ipi-mbox mailbox@ff990400: Registered ZynqMP IPI mbox with TX/RX channels.
[    2.466994] FPGA manager framework
[    2.470302] Advanced Linux Sound Architecture Driver Initialized.
[    2.476535] Bluetooth: Core ver 2.22
[    2.479802] NET: Registered protocol family 31
[    2.484205] Bluetooth: HCI device and connection manager initialized
[    2.490522] Bluetooth: HCI socket layer initialized
[    2.495365] Bluetooth: L2CAP socket layer initialized
[    2.500388] Bluetooth: SCO socket layer initialized
[    2.505473] clocksource: Switched to clocksource arch_sys_counter
[    2.511426] VFS: Disk quotas dquot_6.6.0
[    2.515218] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    2.526163] NET: Registered protocol family 2
[    2.526668] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    2.534855] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    2.542886] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    2.550333] TCP: Hash tables configured (established 32768 bind 32768)
[    2.556512] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    2.563175] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    2.570352] NET: Registered protocol family 1
[    2.574803] RPC: Registered named UNIX socket transport module.
[    2.580415] RPC: Registered udp transport module.
[    2.585076] RPC: Registered tcp transport module.
[    2.589746] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    2.596734] PCI: CLS 0 bytes, default 64
[    2.625558] Initialise system trusted keyrings
[    2.625683] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    2.631455] NFS: Registering the id_resolver key type
[    2.635797] Key type id_resolver registered
[    2.639937] Key type id_legacy registered
[    2.643934] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    2.650596] jffs2: version 2.2. (NAND) (SUMMARY)  \xc2\xa9 2001-2006 Red Hat, Inc.
[    2.693227] NET: Registered protocol family 38
[    2.693270] xor: measuring software checksum speed
[    2.700963]    8regs           :  2363 MB/sec
[    2.704628]    32regs          :  2798 MB/sec
[    2.709575]    arm64_neon      :  2380 MB/sec
[    2.709761] xor: using function: 32regs (2798 MB/sec)
[    2.714785] Key type asymmetric registered
[    2.718850] Asymmetric key parser 'x509' registered
[    2.723714] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    2.731049] io scheduler mq-deadline registered
[    2.735545] io scheduler kyber registered
[    2.741375] ps_pcie_dma init()
[    2.767537] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    2.769269] Serial: AMBA driver
[    2.773264] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    2.781935] brd: module loaded
[    2.786310] loop: module loaded
[    2.786999] mtdoops: mtd device (mtddev=name/number) must be supplied
[    2.791641] libphy: Fixed MDIO Bus: probed
[    2.795583] tun: Universal TUN/TAP device driver, 1.6
[    2.799379] CAN device driver interface
[    2.803737] usbcore: registered new interface driver asix
[    2.808513] usbcore: registered new interface driver ax88179_178a
[    2.814552] usbcore: registered new interface driver cdc_ether
[    2.820343] usbcore: registered new interface driver net1080
[    2.825966] usbcore: registered new interface driver cdc_subset
[    2.831851] usbcore: registered new interface driver zaurus
[    2.837398] usbcore: registered new interface driver cdc_ncm
[    2.843757] usbcore: registered new interface driver uas
[    2.848304] usbcore: registered new interface driver usb-storage
[    2.854842] rtc_zynqmp ffa60000.rtc: registered as rtc0
[    2.859446] rtc_zynqmp ffa60000.rtc: setting system clock to 2022-06-06T17:33:33 UTC (1654536813)
[    2.868288] i2c /dev entries driver
[    2.873129] usbcore: registered new interface driver uvcvideo
[    2.877422] USB Video Class driver (1.1.1)
[    2.881947] Bluetooth: HCI UART driver ver 2.3
[    2.885902] Bluetooth: HCI UART protocol H4 registered
[    2.891003] Bluetooth: HCI UART protocol BCSP registered
[    2.896293] Bluetooth: HCI UART protocol LL registered
[    2.901384] Bluetooth: HCI UART protocol ATH3K registered
[    2.906759] Bluetooth: HCI UART protocol Three-wire (H5) registered
[    2.913007] Bluetooth: HCI UART protocol Intel registered
[    2.918354] Bluetooth: HCI UART protocol QCA registered
[    2.923557] usbcore: registered new interface driver bcm203x
[    2.929179] usbcore: registered new interface driver bpa10x
[    2.934713] usbcore: registered new interface driver bfusb
[    2.940165] usbcore: registered new interface driver btusb
[    2.945625] usbcore: registered new interface driver ath3k
[    2.951117] EDAC MC: ECC not enabled
[    2.954721] EDAC DEVICE0: Giving out device to module zynqmp-ocm-edac controller zynqmp_ocm: DEV ff960000.memory-controller (INTERRUPT)
[    2.967046] sdhci: Secure Digital Host Controller Interface driver
[    2.972844] sdhci: Copyright(c) Pierre Ossman
[    2.977166] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.983063] ledtrig-cpu: registered to indicate activity on CPUs
[    2.988770] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
[    2.995200] zynqmp_firmware_probe Platform Management API v1.1
[    3.000958] zynqmp_firmware_probe Trustzone version v1.0
[    3.024341] zynqmp-pinctrl firmware:zynqmp-firmware:pinctrl: zynqmp pinctrl initialized
[    3.061506] securefw securefw: securefw probed

[    3.061773] alg: No test for xilinx-zynqmp-aes (zynqmp-aes)
[    3.065963] zynqmp_aes firmware:zynqmp-firmware:zynqmp-aes: AES Successfully Registered
[    3.074074] alg: No test for xilinx-keccak-384 (zynqmp-keccak-384)
[    3.080214] alg: No test for xilinx-zynqmp-rsa (zynqmp-rsa)
[    3.085739] usbcore: registered new interface driver usbhid
[    3.091116] usbhid: USB HID core driver
[    3.097602] ARM CCI_400_r1 PMU driver probed
[    3.098044] fpga_manager fpga0: Xilinx ZynqMP FPGA Manager registered
[    3.105975] usbcore: registered new interface driver snd-usb-audio
[    3.112472] pktgen: Packet Generator for packet performance testing. Version: 2.75
[    3.119845] Initializing XFRM netlink socket
[    3.123537] NET: Registered protocol family 10
[    3.128282] Segment Routing with IPv6
[    3.131687] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    3.137738] NET: Registered protocol family 17
[    3.141819] NET: Registered protocol family 15
[    3.146231] can: controller area network core
[    3.150572] NET: Registered protocol family 29
[    3.154959] can: raw protocol
[    3.157899] can: broadcast manager protocol
[    3.162054] can: netlink gateway - max_hops=1
[    3.166451] Bluetooth: RFCOMM TTY layer initialized
[    3.171227] Bluetooth: RFCOMM socket layer initialized
[    3.176334] Bluetooth: RFCOMM ver 1.11
[    3.180049] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[    3.185322] Bluetooth: BNEP filters: protocol multicast
[    3.190519] Bluetooth: BNEP socket layer initialized
[    3.195443] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[    3.201330] Bluetooth: HIDP socket layer initialized
[    3.206376] 9pnet: Installing 9P2000 support
[    3.210512] Key type dns_resolver registered
[    3.214923] registered taskstats version 1
[    3.218807] Loading compiled-in X.509 certificates
[    3.224486] Btrfs loaded, crc32c=crc32c-generic
[    3.236487] ff000000.serial: ttyPS0 at MMIO 0xff000000 (irq = 54, base_baud = 6249375) is a xuartps
[    3.245508] printk: console [ttyPS0] enabled
[    3.249803] printk: bootconsole [cdns0] disabled
[    3.259201] ff010000.serial: ttyPS1 at MMIO 0xff010000 (irq = 55, base_baud = 6249375) is a xuartps
[    3.272624] of-fpga-region fpga-full: FPGA Region probed
[    3.278526] nwl-pcie fd0e0000.pcie: host bridge /axi/pcie@fd0e0000 ranges:
[    3.285423] nwl-pcie fd0e0000.pcie:      MEM 0x00e0000000..0x00efffffff -> 0x00e0000000
[    3.293431] nwl-pcie fd0e0000.pcie:      MEM 0x0600000000..0x07ffffffff -> 0x0600000000
[    3.301510] nwl-pcie fd0e0000.pcie: Link is UP
[    3.306094] nwl-pcie fd0e0000.pcie: PCI host bridge to bus 0000:00
[    3.312271] pci_bus 0000:00: root bus resource [bus 00-ff]
[    3.317759] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xefffffff]
[    3.324628] pci_bus 0000:00: root bus resource [mem 0x600000000-0x7ffffffff pref]
[    3.332133] pci 0000:00:00.0: [10ee:d021] type 01 class 0x060400
[    3.338196] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot
[    3.345537] pci 0000:01:00.0: [10de:1eb8] type 00 class 0x030200
[    3.351558] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00ffffff]
[    3.357837] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0fffffff 64bit pref]
[    3.365069] pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x01ffffff 64bit pref]
[    3.372322] pci 0000:01:00.0: Enabling HDA controller
[    3.377466] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    3.383592] pci 0000:01:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x2 link at 0000:00:00.0 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)
[    3.399887] pci 0000:00:00.0: BAR 9: assigned [mem 0x600000000-0x617ffffff 64bit pref]
[    3.407802] pci 0000:00:00.0: BAR 8: assigned [mem 0xe0000000-0xe0ffffff]
[    3.414592] pci 0000:01:00.0: BAR 1: assigned [mem 0x600000000-0x60fffffff 64bit pref]
[    3.422517] pci 0000:01:00.0: BAR 3: assigned [mem 0x610000000-0x611ffffff 64bit pref]
[    3.430440] pci 0000:01:00.0: BAR 0: assigned [mem 0xe0000000-0xe0ffffff]
[    3.437223] pci 0000:00:00.0: PCI bridge to [bus 01-0c]
[    3.442452] pci 0000:00:00.0:   bridge window [mem 0xe0000000-0xe0ffffff]
[    3.449234] pci 0000:00:00.0:   bridge window [mem 0x600000000-0x617ffffff 64bit pref]
[    3.457227] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[    3.463398] pcieport 0000:00:00.0: PME: Signaling with IRQ 59
[    3.469610] xilinx-zynqmp-dma fd500000.dma: ZynqMP DMA driver Probe success
[    3.476792] xilinx-zynqmp-dma fd510000.dma: ZynqMP DMA driver Probe success
[    3.483970] xilinx-zynqmp-dma fd520000.dma: ZynqMP DMA driver Probe success
[    3.491142] xilinx-zynqmp-dma fd530000.dma: ZynqMP DMA driver Probe success
[    3.498314] xilinx-zynqmp-dma fd540000.dma: ZynqMP DMA driver Probe success
[    3.505494] xilinx-zynqmp-dma fd550000.dma: ZynqMP DMA driver Probe success
[    3.512660] xilinx-zynqmp-dma fd560000.dma: ZynqMP DMA driver Probe success
[    3.519836] xilinx-zynqmp-dma fd570000.dma: ZynqMP DMA driver Probe success
[    3.527072] xilinx-zynqmp-dma ffa80000.dma: ZynqMP DMA driver Probe success
[    3.534241] xilinx-zynqmp-dma ffa90000.dma: ZynqMP DMA driver Probe success
[    3.541415] xilinx-zynqmp-dma ffaa0000.dma: ZynqMP DMA driver Probe success
[    3.548585] xilinx-zynqmp-dma ffab0000.dma: ZynqMP DMA driver Probe success
[    3.555755] xilinx-zynqmp-dma ffac0000.dma: ZynqMP DMA driver Probe success
[    3.562925] xilinx-zynqmp-dma ffad0000.dma: ZynqMP DMA driver Probe success
[    3.570098] xilinx-zynqmp-dma ffae0000.dma: ZynqMP DMA driver Probe success
[    3.577263] xilinx-zynqmp-dma ffaf0000.dma: ZynqMP DMA driver Probe success
[    3.584671] spi_master spi0: cannot find modalias for /axi/spi@ff0f0000/flash@0
[    3.591979] spi_master spi0: Failed to create SPI device for /axi/spi@ff0f0000/flash@0
[    3.601763] macb ff0e0000.ethernet: Not enabling partial store and forward
[    3.609298] libphy: MACB_mii_bus: probed
[    3.613968] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 38 (ee:d4:a1:bb:ac:11)
[    3.624195] xilinx-axipmon ffa00000.perf-monitor: Probed Xilinx APM
[    3.630756] xilinx-axipmon fd0b0000.perf-monitor: Probed Xilinx APM
[    3.637292] xilinx-axipmon fd490000.perf-monitor: Probed Xilinx APM
[    3.643827] zynqmp_pll_disable() clock disable failed for apll_int, ret = -13
[    3.651061] xilinx-axipmon ffa10000.perf-monitor: Probed Xilinx APM
[    3.657764] cdns-i2c ff020000.i2c: 400 kHz mmio ff020000 irq 40
[    3.664137] cdns-i2c ff030000.i2c: 400 kHz mmio ff030000 irq 41
[    3.670458] cdns-wdt fd4d0000.watchdog: Xilinx Watchdog Timer with timeout 60s
[    3.677955] cdns-wdt ff150000.watchdog: Xilinx Watchdog Timer with timeout 10s
[    3.685636] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 1199880 KHz, changing to: 1199999 KHz
[    3.699581] of_cfs_init
[    3.702051] of_cfs_init: OK
[    3.704959] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    3.730342] mmc0: SDHCI controller on ff170000.mmc [ff170000.mmc] using ADMA 64-bit
[    3.771442] mmc0: new high speed SDHC card at address 5048
[    3.777296] mmcblk0: mmc0:5048 SD32G 28.9 GiB 
[    3.783272]  mmcblk0: p1 p2
[    3.839775] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    3.846309] clk: Not disabling unused clocks
[    3.850815] ALSA device list:
[    3.853777]   No soundcards found.
[    3.857449] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[    3.866059] cfg80211: failed to load regulatory.db
[    3.882746] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[    3.890867] VFS: Mounted root (ext4 filesystem) on device 179:2.
[    3.898760] devtmpfs: mounted
[    3.902294] Freeing unused kernel memory: 2112K
[    3.913516] Run /sbin/init as init process
[    3.917606]   with arguments:
[    3.917609]     /sbin/init
[    3.917611]   with environment:
[    3.917613]     HOME=/
[    3.917616]     TERM=linux
[    4.251539] random: fast init done
[    4.457192] udevd[253]: starting version 3.2.9
[    4.479758] random: udevd: uninitialized urandom read (16 bytes read)
[    4.487296] random: udevd: uninitialized urandom read (16 bytes read)
[    4.493767] random: udevd: uninitialized urandom read (16 bytes read)
[    4.530798] udevd[254]: starting eudev-3.2.9
[    4.672359] mali: loading out-of-tree module taints kernel.
[    7.240328] nvidia: module license 'NVIDIA' taints kernel.
[    7.245822] Disabling lock debugging due to kernel taint
[    7.292984] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[    7.357597] nvidia-nvlink: Nvlink Core is being initialized, major device number 241
[    7.365464] NVRM: CPUID: unknown implementer/part 0x41/0xd03.
[    7.372102] The NVIDIA GPU driver for AArch64 has not been qualified on this CPU
               and therefore it is not recommended or intended for use in any production
               environment.
[    7.391963] NVRM: probing 0x10de 0x1eb8, class 0x30200
[    7.397120] nvidia 0000:01:00.0: enabling device (0000 -> 0002)
[    7.403054] NVRM: GPU 0000:01:00.0: ATS not supported by this GPU. Disabling ATS support for all the GPUs in the system!
[    7.414036] NVRM: PCI:0000:01:00.0 (10de:1eb8): BAR0 @ 0xe0000000 (16MB)
[    7.420735] NVRM: PCI:0000:01:00.0 (10de:1eb8): BAR1 @ 0x600000000 (256MB)
[    7.469802] NVRM: loading NVIDIA UNIX aarch64 Kernel Module  515.43.04  Tue Apr 26 16:23:32 UTC 2022
[    7.586607] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  515.43.04  Tue Apr 26 15:45:20 UTC 2022
[    7.753792] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
[    7.759889] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 0
[    7.794324] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
[    9.154410] macb ff0e0000.ethernet eth0: PHY [ff0e0000.ethernet-ffffffff:0c] driver [TI DP83867] (irq=POLL)
[    9.154422] macb ff0e0000.ethernet eth0: configuring for phy/rgmii-id link mode
[   10.039270] random: crng init done
[   10.039282] random: 7 urandom warning(s) missed due to ratelimiting
[   12.211132] macb ff0e0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control tx
[   12.211162] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   60.332235] NVRM: nvidia_open...
[   60.332244] NVRM: nvidia_ctl_open
[   60.332260] NVRM: ioctl(0xd2, 0xe686a9c8, 0x48)
[   60.332321] NVRM: ioctl(0xd6, 0xe686aa60, 0x8)
[   60.332327] NVRM: ioctl(0xc8, 0x8c842980, 0x900)
[   60.332339] NVRM: ioctl(0x2b, 0xe686ab10, 0x20)
[   60.338698] NVRM: ioctl(0x2b, 0xe686aab8, 0x28)
[   60.339030] NVRM: ioctl(0x2b, 0xe686aab8, 0x28)
[   60.339068] NVRM: ioctl(0x2a, 0xe686a098, 0x20)
[   60.339104] NVRM: ioctl(0x2a, 0xe686a098, 0x20)
[   60.339130] NVRM: ioctl(0x2a, 0xe686a098, 0x20)
[   60.339152] NVRM: ioctl(0x2a, 0xe686a098, 0x20)
[   60.339262] NVRM: nvidia_open...
[   60.339269] NVRM: GPU 0000:01:00.0: Opening GPU with minor number 0
[   60.339515] NVRM: GPU 0000:01:00.0: RmInitAdapter
[   60.339521] NVRM: GPU 0000:01:00.0: RmSetupRegisters for 0x10de:0x1eb8
[   60.339525] NVRM: GPU 0000:01:00.0: pci config info:
[   60.339532] NVRM: GPU 0000:01:00.0:    registers look  like: 000000002b7d0555 00000000eef9bce6NVRM: GPU 0000:01:00.0:    fb        looks like: 00000000ce180de4 00000000595cdb8aNVRM: GPU 0000:01:00.0: Successfully mapped framebuffer and registers
[   60.339560] NVRM: GPU 0000:01:00.0: final mappings:
[   60.339565] NVRM: GPU 0000:01:00.0:     regs: 000000002b7d0555 00000000eef9bce6 0x00000000dabde85d
[   63.054304] NVRM: VM: nv_alloc_pages: 1 pages
[   63.054312] NVRM: VM:    contig 1  cache_type 1
[   63.054322] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   63.054347] NVRM: VM: nv_alloc_pages:3454: 0x00000000557260f5, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x000000005a67d0be
[   63.054774] NVRM: VM: nv_alloc_pages: 129 pages
[   63.054778] NVRM: VM:    contig 0  cache_type 0
[   63.054807] NVRM: VM: 158628136: 129 pages
[   63.054956] NVRM: VM: nv_alloc_pages:3454: 0x000000003ae49260, 129 page(s), count = 1, flags = 0x00000004, page_table = 0x0000000067f83c0a
[   63.055175] NVRM: VM: nv_alloc_pages: 1 pages
[   63.055179] NVRM: VM:    contig 1  cache_type 0
[   63.055183] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   63.055191] NVRM: VM: nv_alloc_pages:3454: 0x0000000019fbd577, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000dffc46d5
[   63.055201] NVRM: VM: nv_alloc_pages: 1 pages
[   63.055204] NVRM: VM:    contig 1  cache_type 1
[   63.055208] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   63.055216] NVRM: VM: nv_alloc_pages:3454: 0x000000000fa0033f, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000e839d098
[   63.055229] NVRM: VM: nv_alloc_pages: 1 pages
[   63.055232] NVRM: VM:    contig 1  cache_type 0
[   63.055236] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   63.055244] NVRM: VM: nv_alloc_pages:3454: 0x00000000d4ceeffc, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000f9e3661b
[   63.065650] NVRM: Chipset not recognized (vendor ID 0x10ee, device ID 0xd021)
[   63.065655] The NVIDIA GPU driver for AArch64 has not been qualified on this platform
               and therefore it is not recommended or intended for use in any production
               environment.
[   63.179542] NVRM: VM: nv_alloc_pages: 10 pages
[   63.179547] NVRM: VM:    contig 1  cache_type 1
[   63.179556] NVRM: VM: nv_alloc_contig_pages: 10 pages
[   63.179576] NVRM: VM: nv_alloc_pages:3454: 0x000000000e0a5a6d, 10 page(s), count = 1, flags = 0x0000000d, page_table = 0x000000007bb6c199
[   63.179588] NVRM: VM: nv_alloc_pages: 1 pages
[   63.179592] NVRM: VM:    contig 1  cache_type 1
[   63.179596] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   63.179604] NVRM: VM: nv_alloc_pages:3454: 0x000000006c7c61a6, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x0000000072d6aaa4
[   63.181209] NVRM: VM: nv_alloc_pages: 1 pages
[   63.181213] NVRM: VM:    contig 1  cache_type 0
[   63.181219] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   63.181227] NVRM: VM: nv_alloc_pages:3454: 0x000000000e9e03c4, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x000000007167538d
[   63.181401] NVRM: VM: nv_alloc_pages: 1 pages
[   63.181405] NVRM: VM:    contig 1  cache_type 0
[   63.181409] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   63.181417] NVRM: VM: nv_alloc_pages:3454: 0x00000000145c7b08, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000fc024806
[   63.181454] NVRM: VM: nv_alloc_pages: 9812 pages
[   63.181458] NVRM: VM:    contig 0  cache_type 0
[   63.183078] NVRM: VM: 158628136: 9812 pages
[   63.193982] NVRM: VM: nv_alloc_pages:3454: 0x000000002752a247, 9812 page(s), count = 1, flags = 0x00000004, page_table = 0x000000005474fdbd
[   63.226767] NVRM: VM: nv_alloc_pages: 16 pages
[   63.226776] NVRM: VM:    contig 1  cache_type 0
[   63.226788] NVRM: VM: nv_alloc_contig_pages: 16 pages
[   63.226816] NVRM: VM: nv_alloc_pages:3454: 0x0000000029b3dc7f, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000724d741e
[   63.226895] NVRM: VM: nv_alloc_pages: 16 pages
[   63.226899] NVRM: VM:    contig 1  cache_type 0
[   63.226910] NVRM: VM: nv_alloc_contig_pages: 16 pages
[   63.226930] NVRM: VM: nv_alloc_pages:3454: 0x0000000034236dfa, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000f7c0e638
[   63.227465] NVRM: VM: nv_free_pages: 0xa
[   63.227473] NVRM: VM: nv_free_pages:3477: 0x000000000e0a5a6d, 10 page(s), count = 1, flags = 0x0000000d, page_table = 0x000000007bb6c199
[   63.227476] NVRM: VM: nv_free_contig_pages: 10 pages
[   63.227490] NVRM: VM: nv_free_pages: 0x1
[   63.227496] NVRM: VM: nv_free_pages:3477: 0x000000006c7c61a6, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x0000000072d6aaa4
[   63.227499] NVRM: VM: nv_free_contig_pages: 1 pages
[   63.227511] NVRM: VM: nv_free_pages: 0x1
[   63.227517] NVRM: VM: nv_free_pages:3477: 0x0000000019fbd577, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000dffc46d5
[   63.227520] NVRM: VM: nv_free_contig_pages: 1 pages
[   63.227530] NVRM: VM: nv_free_pages: 0x1
[   63.227536] NVRM: VM: nv_free_pages:3477: 0x000000000fa0033f, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000e839d098
[   63.227540] NVRM: VM: nv_free_contig_pages: 1 pages
[   63.227549] NVRM: VM: nv_free_pages: 0x1
[   63.227555] NVRM: VM: nv_free_pages:3477: 0x00000000d4ceeffc, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000f9e3661b
[   63.227558] NVRM: VM: nv_free_contig_pages: 1 pages
[   63.230480] NVRM: VM: nv_free_pages: 0x2654
[   63.230487] NVRM: VM: nv_free_pages:3477: 0x000000002752a247, 9812 page(s), count = 1, flags = 0x00000004, page_table = 0x000000005474fdbd
[   63.230491] NVRM: VM: nv_free_system_pages: 9812 pages
[   63.238802] NVRM: VM: nv_free_pages: 0x1
[   63.238811] NVRM: VM: nv_free_pages:3477: 0x00000000145c7b08, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000fc024806
[   63.238814] NVRM: VM: nv_free_contig_pages: 1 pages
[   63.238840] NVRM: VM: nv_free_pages: 0x10
[   63.238846] NVRM: VM: nv_free_pages:3477: 0x0000000029b3dc7f, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000724d741e
[   63.238849] NVRM: VM: nv_free_contig_pages: 16 pages
[   63.238872] NVRM: VM: nv_free_pages: 0x10
[   63.238878] NVRM: VM: nv_free_pages:3477: 0x0000000034236dfa, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000f7c0e638
[   63.238882] NVRM: VM: nv_free_contig_pages: 16 pages
[   63.238943] NVRM: VM: nv_free_pages: 0x81
[   63.238949] NVRM: VM: nv_free_pages:3477: 0x000000003ae49260, 129 page(s), count = 1, flags = 0x00000004, page_table = 0x0000000067f83c0a
[   63.238953] NVRM: VM: nv_free_system_pages: 129 pages
[   63.239075] NVRM: VM: nv_free_pages: 0x1
[   63.239081] NVRM: VM: nv_free_pages:3477: 0x000000000e9e03c4, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x000000007167538d
[   63.239085] NVRM: VM: nv_free_contig_pages: 1 pages
[   63.239135] NVRM: VM: nv_free_pages: 0x1
[   63.239141] NVRM: VM: nv_free_pages:3477: 0x00000000557260f5, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x000000005a67d0be
[   63.239144] NVRM: VM: nv_free_contig_pages: 1 pages
[   63.239282] NVRM: GPU 0000:01:00.0: Tearing down registers
[   63.239292] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x63:0xffff:2344)
[   63.247218] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   63.247399] NVRM: nvidia_open...
[   63.247407] NVRM: GPU 0000:01:00.0: Opening GPU with minor number 0
[   63.247645] NVRM: GPU 0000:01:00.0: RmInitAdapter
[   63.247651] NVRM: GPU 0000:01:00.0: RmSetupRegisters for 0x10de:0x1eb8
[   63.247655] NVRM: GPU 0000:01:00.0: pci config info:
[   63.247662] NVRM: GPU 0000:01:00.0:    registers look  like: 000000002b7d0555 00000000eef9bce6NVRM: GPU 0000:01:00.0:    fb        looks like: 00000000ce180de4 00000000595cdb8aNVRM: GPU 0000:01:00.0: Successfully mapped framebuffer and registers
[   63.247684] NVRM: GPU 0000:01:00.0: final mappings:
[   63.247690] NVRM: GPU 0000:01:00.0:     regs: 000000002b7d0555 00000000eef9bce6 0x0000000038a047f0
[   64.317487] NVRM: VM: nv_alloc_pages: 1 pages
[   64.317497] NVRM: VM:    contig 1  cache_type 1
[   64.317505] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   64.317524] NVRM: VM: nv_alloc_pages:3454: 0x00000000d2916125, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x0000000022aa024a
[   64.317936] NVRM: VM: nv_alloc_pages: 129 pages
[   64.317941] NVRM: VM:    contig 0  cache_type 0
[   64.317966] NVRM: VM: 158628136: 129 pages
[   64.318118] NVRM: VM: nv_alloc_pages:3454: 0x00000000122be365, 129 page(s), count = 1, flags = 0x00000004, page_table = 0x0000000067f83c0a
[   64.318334] NVRM: VM: nv_alloc_pages: 1 pages
[   64.318338] NVRM: VM:    contig 1  cache_type 0
[   64.318342] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   64.318350] NVRM: VM: nv_alloc_pages:3454: 0x000000002a4c79a1, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x000000005d7a5605
[   64.318360] NVRM: VM: nv_alloc_pages: 1 pages
[   64.318363] NVRM: VM:    contig 1  cache_type 1
[   64.318367] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   64.318375] NVRM: VM: nv_alloc_pages:3454: 0x0000000069e280b9, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000ddbf6f86
[   64.318388] NVRM: VM: nv_alloc_pages: 1 pages
[   64.318391] NVRM: VM:    contig 1  cache_type 0
[   64.318396] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   64.318403] NVRM: VM: nv_alloc_pages:3454: 0x00000000699c178f, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x0000000020aefcc9
[   64.433328] NVRM: VM: nv_alloc_pages: 10 pages
[   64.433334] NVRM: VM:    contig 1  cache_type 1
[   64.433344] NVRM: VM: nv_alloc_contig_pages: 10 pages
[   64.433365] NVRM: VM: nv_alloc_pages:3454: 0x00000000769189a5, 10 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000eb08753d
[   64.433378] NVRM: VM: nv_alloc_pages: 1 pages
[   64.433381] NVRM: VM:    contig 1  cache_type 1
[   64.433385] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   64.433394] NVRM: VM: nv_alloc_pages:3454: 0x00000000264c8531, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x0000000093e892be
[   64.435074] NVRM: VM: nv_alloc_pages: 1 pages
[   64.435078] NVRM: VM:    contig 1  cache_type 0
[   64.435083] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   64.435092] NVRM: VM: nv_alloc_pages:3454: 0x000000007cc25ff6, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000f9840d32
[   64.435266] NVRM: VM: nv_alloc_pages: 1 pages
[   64.435270] NVRM: VM:    contig 1  cache_type 0
[   64.435274] NVRM: VM: nv_alloc_contig_pages: 1 pages
[   64.435282] NVRM: VM: nv_alloc_pages:3454: 0x000000006975bc37, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000e0e3c8b3
[   64.435316] NVRM: VM: nv_alloc_pages: 9812 pages
[   64.435320] NVRM: VM:    contig 0  cache_type 0
[   64.436958] NVRM: VM: 158628136: 9812 pages
[   64.447848] NVRM: VM: nv_alloc_pages:3454: 0x00000000b6d42932, 9812 page(s), count = 1, flags = 0x00000004, page_table = 0x00000000ac3556b6
[   64.480782] NVRM: VM: nv_alloc_pages: 16 pages
[   64.480791] NVRM: VM:    contig 1  cache_type 0
[   64.480803] NVRM: VM: nv_alloc_contig_pages: 16 pages
[   64.480831] NVRM: VM: nv_alloc_pages:3454: 0x000000008412cd26, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000890d6067
[   64.480907] NVRM: VM: nv_alloc_pages: 16 pages
[   64.480911] NVRM: VM:    contig 1  cache_type 0
[   64.480918] NVRM: VM: nv_alloc_contig_pages: 16 pages
[   64.480936] NVRM: VM: nv_alloc_pages:3454: 0x0000000042e4020d, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x000000002ea683b5
[   64.481796] NVRM: VM: nv_free_pages: 0xa
[   64.481804] NVRM: VM: nv_free_pages:3477: 0x00000000769189a5, 10 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000eb08753d
[   64.481807] NVRM: VM: nv_free_contig_pages: 10 pages
[   64.481821] NVRM: VM: nv_free_pages: 0x1
[   64.481827] NVRM: VM: nv_free_pages:3477: 0x00000000264c8531, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x0000000093e892be
[   64.481831] NVRM: VM: nv_free_contig_pages: 1 pages
[   64.481843] NVRM: VM: nv_free_pages: 0x1
[   64.481849] NVRM: VM: nv_free_pages:3477: 0x000000002a4c79a1, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x000000005d7a5605
[   64.481852] NVRM: VM: nv_free_contig_pages: 1 pages
[   64.481862] NVRM: VM: nv_free_pages: 0x1
[   64.481868] NVRM: VM: nv_free_pages:3477: 0x0000000069e280b9, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000ddbf6f86
[   64.481871] NVRM: VM: nv_free_contig_pages: 1 pages
[   64.481879] NVRM: VM: nv_free_pages: 0x1
[   64.481885] NVRM: VM: nv_free_pages:3477: 0x00000000699c178f, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x0000000020aefcc9
[   64.481888] NVRM: VM: nv_free_contig_pages: 1 pages
[   64.484762] NVRM: VM: nv_free_pages: 0x2654
[   64.484769] NVRM: VM: nv_free_pages:3477: 0x00000000b6d42932, 9812 page(s), count = 1, flags = 0x00000004, page_table = 0x00000000ac3556b6
[   64.484772] NVRM: VM: nv_free_system_pages: 9812 pages
[   64.492970] NVRM: VM: nv_free_pages: 0x1
[   64.492979] NVRM: VM: nv_free_pages:3477: 0x000000006975bc37, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000e0e3c8b3
[   64.492983] NVRM: VM: nv_free_contig_pages: 1 pages
[   64.493008] NVRM: VM: nv_free_pages: 0x10
[   64.493015] NVRM: VM: nv_free_pages:3477: 0x000000008412cd26, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000890d6067
[   64.493018] NVRM: VM: nv_free_contig_pages: 16 pages
[   64.493037] NVRM: VM: nv_free_pages: 0x10
[   64.493043] NVRM: VM: nv_free_pages:3477: 0x0000000042e4020d, 16 page(s), count = 1, flags = 0x00000005, page_table = 0x000000002ea683b5
[   64.493046] NVRM: VM: nv_free_contig_pages: 16 pages
[   64.493111] NVRM: VM: nv_free_pages: 0x81
[   64.493117] NVRM: VM: nv_free_pages:3477: 0x00000000122be365, 129 page(s), count = 1, flags = 0x00000004, page_table = 0x0000000067f83c0a
[   64.493120] NVRM: VM: nv_free_system_pages: 129 pages
[   64.493243] NVRM: VM: nv_free_pages: 0x1
[   64.493249] NVRM: VM: nv_free_pages:3477: 0x000000007cc25ff6, 1 page(s), count = 1, flags = 0x00000005, page_table = 0x00000000f9840d32
[   64.493253] NVRM: VM: nv_free_contig_pages: 1 pages
[   64.493302] NVRM: VM: nv_free_pages: 0x1
[   64.493308] NVRM: VM: nv_free_pages:3477: 0x00000000d2916125, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x0000000022aa024a
[   64.493312] NVRM: VM: nv_free_contig_pages: 1 pages
[   64.493449] NVRM: GPU 0000:01:00.0: Tearing down registers
[   64.493460] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x63:0xffff:2344)
[   64.501402] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   64.501586] NVRM: ioctl(0xd1, 0xe6869f80, 0xc)
[   64.501650] NVRM: ioctl(0x2a, 0xe6867bf8, 0x20)
[   64.503656] NVRM: ioctl(0x2a, 0xe686a278, 0x20)
[   64.503690] NVRM: ioctl(0x29, 0xe686ad00, 0x10)
[   64.503788] NVRM: GPU 0000:00:00.0: nvidia_close on GPU with minor number 255
[   64.503792] NVRM: nvidia_ctl_close

Any ideas?

nvidia-bug-report.log.gz (37.6 KB)