EVGA RTX2080Ti doesn't work on ubuntu 18.04 or 20.04 when driver version newer than 470.63.01

EVGA RTX2080Ti, Device ID is 10de:1e07 (rev a1)
Ubuntu 18.04/Ubuntu 20.04.
CPU is i7-9700, Chipset is intel C246.
The detail spec, please check this link. https://www.neousys-tech.com/en/product/application/edge-ai-gpu-computing/nuvo-8108gc-intel-9th-gen-nvidia-rtx-2080-250w-gpu-computing-platform#specification

Below is the error message,
Version 470.103.01 Failed
[ 7.785664] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 7.785697] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 8.387460] e1000e: enp0s31f6 NIC Link is Up 100 Mbps Full Duplex, Flow Control: Rx/Tx
[ 8.387462] e1000e 0000:00:1f.6 enp0s31f6: 10/100 speed: disabling TSO
[ 8.387568] IPv6: ADDRCONF(NETDEV_CHANGE): enp0s31f6: link becomes ready
[ 8.455556] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 8.455668] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 9.297761] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 9.297782] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 9.955571] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 9.955663] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 11.062134] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 11.062240] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 11.721474] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 11.721513] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 12.377315] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 12.377350] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 13.036952] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1253)
[ 13.036976] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
495.29.05 Failed
[ 8.065335] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1479)
[ 8.065368] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 8.065375] BUG: unable to handle page fault for address: 000000000000291c
[ 8.065377] #PF: supervisor read access in kernel mode
[ 8.065378] #PF: error_code(0x0000) - not-present page
[ 8.065379] PGD 0 P4D 0
[ 8.065380] Oops: 0000 [#1] SMP NOP
510.47.03 Failed
[ 10.028336] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x26:0x56:1463)
[ 10.028400] BUG: unable to handle page fault for address: 0000000000002a04
[ 10.028402] #PF: supervisor read access in kernel mode
[ 10.028403] #PF: error_code(0x0000) - not-present page
[ 10.028403] PGD 0 P4D 0
[ 10.028405] Oops: 0000 [#1] SMP NOPTI
[ 10.028406] CPU: 3 PID: 1273 Comm: nv_queue Tainted: P OE 5.4.0-42-generic #46~18.04.1-Ubuntu
[ 10.028407] Hardware name: Neousys Technology Inc. Nuvo-8108GC Series/NVS-8108, BIOS Build210825 08/25/2021
[ 10.028663] RIP: 0010:_nv009917rm+0x38/0xc0 [nvidia]
[ 10.028663] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 0
[ 10.028665] Code: 88 f0 01 48 8b bb 68 01 00 00 e8 d3 55 4d 00 85 c0 74 0f 48 83 c4 08 5b 41 5c c3 0f 1f 80 00 00 00 00 44 89 e7 e8 38 0b be ff 90 04 2a 00 00 83 fa 01 74 2f 80 b8 0c 05 00 00 00 74 12 80 b8
[ 10.028666] RSP: 0018:ffffb8370103fde0 EFLAGS: 00010246
[ 10.028667] RAX: 0000000000000000 RBX: ffff8f89321f0808 RCX: 0000000000000000
[ 10.028668] RDX: ffffb8370103fe0c RSI: 0000000000000000 RDI: 0000000000000000
[ 10.028668] RBP: ffff8f891ab5e000 R08: ffffd836ffac7c80 R09: ffffb8370103fe10
[ 10.028669] R10: ffffb8370103fe90 R11: 0000000000000001 R12: 0000000000000000
[ 10.028669] R13: ffffb8370103fed0 R14: ffff8f89321ebe08 R15: ffff8f89329f2f80
[ 10.028670] FS: 0000000000000000(0000) GS:ffff8f894c0c0000(0000) knlGS:0000000000000000
[ 10.028671] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 10.028671] CR2: 0000000000002a04 CR3: 0000000e6b40a006 CR4: 00000000003606e0
[ 10.028672] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 10.028672] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

At the same computer hardware, same OS system, RTX A5000 works well.
If i switch to Supermicro X9SCA-F-O motherboard which uses Xeon E3 v2 CPU, use same OS system, RTX2080Ti works well with all version driver.
It looks like on particular motherboard, RTX2080Ti doesn’t work with newer version driver.

RTX3070 has same issue, upload report
nvidia-bug-report495.log.gz (181.0 KB)
Uploading: nvidia-bug-report510.log.gz…

(random internet user here)

Have you already tried to look for a BIOS update for the failing system? Just a random part that might impact device compatibility and is able to be updated.

Or tried a different slot? I realize that might be the only x16 slot, but that still might change the result and be a new data point.

And I assume everything else is already updated on it to the latest releases, like Linux kernel version. I know some people that are hesitant to update regularly.

Same issue here. Palit GamingPro RTX 2080, Neousys Nuvo 6108GC motherboard. All three tested drivers give the following dmesg (470, 510, 515):

NVRM GPU 0000:01:00.0: RmInitAdapter failed! (0x26:0x56:1463)

On drivers 510 and 515, all processes that touches the gpu hangs in the D status.