[aarch64] [arm64] RmInitAdapter failed! (0x25:0x54:1214) driver fails to activate the display on arm64

Good Morning,

I’ve been trying to get two different cards working on an arm64 board without any uefi services.
Environment

Component Version
GPU Nvidia GTX 645, GTX 960
DRIVERS NVIDIA-Linux-aarch64-465.24.02
CPU Rockchip rk3566
ARCH aarch64/arm64
BOOTLOADER [U-Boot, no ACPI or EFI services
OS Ubuntu 20.04
KERNEL Linux-Next 5.12.0-rc6-next-20210409

Delightfully, there are a few recent changes in linux-next that breaks building the driver without some work.
I’ve tried both in a VFIO emulator and bare metal, and the result is the same.
Running in an emulator with UEFI x86 Option Rom emulator provides a basic framebuffer, but the driver times out in exactly the same fashion when the kernel inits.
It’s really interesting because X reports a valid display before the driver dies, so at some parts of the card are talking correctly.

This issue is also seen on the RPi4 when attempting to enumerate the card.
Also Nouveau times out reading the card as well.

I believe that something isn’t being set up right, and I’d like to solve this issue.

nvidia-bug-report.log.gz (539.9 KB)

[16761.572454] nvidia-nvlink: Nvlink Core is being initialized, major device number 238
[16761.575804] nvidia 0000:01:00.0: assign IRQ: got 89
[16761.576413] NVRM: probing 0x10de 0x11c4, class 0x30000
[16761.576930] nvidia 0000:01:00.0: enabling device (0000 → 0003)
[16761.577528] nvidia 0000:01:00.0: enabling bus mastering
[16761.578005] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
[16761.579148] NVRM: PCI:0000:01:00.0 (10de:11c4): BAR0 @ 0x301000000 (16MB)
[16761.579768] NVRM: PCI:0000:01:00.0 (10de:11c4): BAR1 @ 0x308000000 (128MB)
[16763.104717] NVRM: PBI is not supported for GPU 0000:01:00.0
[16763.105712] NVRM: loading NVIDIA UNIX aarch64 Kernel Module 465.24.02 Thu Apr 8 18:11:15 UTC 2021
[16811.534047] NVRM: nvidia_open…
[16811.534408] NVRM: nvidia_ctl_open
[16811.534849] NVRM: ioctl(0xd2, 0xcb412b18, 0x48)
[16811.535603] NVRM: ioctl(0xc8, 0x846e0d20, 0x900)
[16811.536118] NVRM: ioctl(0x2b, 0xcb412c60, 0x20)
[16812.290864] NVRM: ioctl(0x2a, 0xcb412c10, 0x20)
[16812.291639] NVRM: ioctl(0x2a, 0xcb412c10, 0x20)
[16812.292316] NVRM: ioctl(0x2a, 0xcb412a50, 0x20)
[16812.293952] NVRM: nvidia_open…
[16812.294281] NVRM: GPU 0000:01:00.0: Opening GPU with minor number 0
[16812.295773] NVRM: GPU 0000:01:00.0: RmInitAdapter
[16812.296208] NVRM: GPU 0000:01:00.0: RmSetupRegisters for 0x10de:0x11c4
[16812.296792] NVRM: GPU 0000:01:00.0: pci config info:
[16812.297309] NVRM: GPU 0000:01:00.0: registers look like: 0000000019cf44f7 000000003da941ffNVRM: GPU 0000:01:00.0: fb looks like: 00000000296c5e64 0000000044081fa0NVRM: GPU 0000:01:00.0: Successfully mapped framebuffer and registers
[16812.299349] NVRM: GPU 0000:01:00.0: final mappings:
[16812.299791] NVRM: GPU 0000:01:00.0: regs: 0000000019cf44f7 000000003da941ff 0x000000009d695f33
[16812.658607] NVRM: GPU 0000:01:00.0: OS reports GPU is not primary VGA
[16812.659207] NVRM: GPU 0000:01:00.0: is not primary VGA
[16812.670917] NVRM: VM: nv_alloc_pages: 1 pages
[16812.671329] NVRM: VM: contig 1 cache_type 1
[16812.671742] NVRM: VM: nv_alloc_contig_pages: 1 pages
[16812.672194] NVRM: VM: nv_alloc_pages:3415: 0x00000000038a7bff, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000b0c3a652
[16812.673409] NVRM: VM: nv_alloc_pages: 1 pages
[16812.673804] NVRM: VM: contig 1 cache_type 1
[16812.674216] NVRM: VM: nv_alloc_contig_pages: 1 pages
[16812.674668] NVRM: VM: nv_alloc_pages:3415: 0x0000000019e39878, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x000000001b290aea
[16812.678029] NVRM: GPU 0000:01:00.0: Tearing down registers
[16812.678767] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x25:0x54:1214)
[16812.679502] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[16814.204722] NVRM: PBI is not supported for GPU 0000:01:00.0
[16814.205387] NVRM: nvidia_open…
[16814.205697] NVRM: GPU 0000:01:00.0: Opening GPU with minor number 0
[16814.206913] NVRM: GPU 0000:01:00.0: RmInitAdapter
[16814.207340] NVRM: GPU 0000:01:00.0: RmSetupRegisters for 0x10de:0x11c4
[16814.207920] NVRM: GPU 0000:01:00.0: pci config info:
[16814.208364] NVRM: GPU 0000:01:00.0: registers look like: 0000000019cf44f7 000000003da941ffNVRM: GPU 0000:01:00.0: fb looks like: 00000000296c5e64 0000000044081fa0NVRM: GPU 0000:01:00.0: Successfully mapped framebuffer and registers
[16814.210465] NVRM: GPU 0000:01:00.0: final mappings:
[16814.210905] NVRM: GPU 0000:01:00.0: regs: 0000000019cf44f7 000000003da941ff 0x0000000042c3bbeb
[16814.277800] NVRM: GPU 0000:01:00.0: OS reports GPU is not primary VGA
[16814.278397] NVRM: GPU 0000:01:00.0: is not primary VGA
[16814.281789] NVRM: VM: nv_alloc_pages: 1 pages
[16814.282199] NVRM: VM: contig 1 cache_type 1
[16814.282611] NVRM: VM: nv_alloc_contig_pages: 1 pages
[16814.283064] NVRM: VM: nv_alloc_pages:3415: 0x0000000018ae3ea6, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x000000009bc2e117
[16814.284221] NVRM: VM: nv_alloc_pages: 1 pages
[16814.284613] NVRM: VM: contig 1 cache_type 1
[16814.285023] NVRM: VM: nv_alloc_contig_pages: 1 pages
[16814.285531] NVRM: VM: nv_alloc_pages:3415: 0x000000002b27f8a7, 1 page(s), count = 1, flags = 0x0000000d, page_table = 0x00000000ffae2fea
[16814.288727] NVRM: GPU 0000:01:00.0: Tearing down registers
[16814.289561] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x25:0x54:1214)
[16814.290293] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[16815.815590] NVRM: PBI is not supported for GPU 0000:01:00.0
[16815.816231] NVRM: ioctl(0xd1, 0xcb412950, 0xc)
[16815.901785] NVRM: GPU 0000:00:00.0: nvidia_close on GPU with minor number 255
[16815.902443] NVRM: nvidia_ctl_close
[17181.707835] fuse: init (API version 7.33)