NVIDIA Corporation AD102GL [L20]: loading /lib/firmware/nvidia/580.95.05/gsp_ga10x.bin failed with error -4

We encountered the following kernel errors when using NVIDIA L20 GPUs behind a PCIe Gen5 switch.

Multiple L20 (AD102) devices exhibit similar failures. Total 8 x L20 (AD102) cards are populated under the switch.

GPU 0000:1a:00.0 becomes non‑functional after these errors.

[14114.032724] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0xffff:2661)

[14114.036199] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14116.468580] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14116.472050] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14118.904576] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14118.908125] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14121.340581] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14121.344137] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14123.772557] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14123.776131] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14126.208603] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14126.212221] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14126.231811] nvidia 0000:98:00.0: loading /lib/firmware/nvidia/580.95.05/gsp_ga10x.bin failed with error -4

[14126.231819] nvidia 0000:98:00.0: Direct firmware load for nvidia/580.95.05/gsp_ga10x.bin failed with error -4

[14130.140604] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14130.142846] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14136.428566] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14136.432030] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14138.864607] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14138.868185] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14141.300560] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14141.304109] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14143.724559] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14143.726757] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14146.160562] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14146.164166] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14148.327596] nvidia 0000:1a:00.0: loading /lib/firmware/nvidia/580.95.05/gsp_ga10x.bin failed with error -4

[14148.327604] nvidia 0000:1a:00.0: Direct firmware load for nvidia/580.95.05/gsp_ga10x.bin failed with error -4

[14208.444203] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x24:0x72:1590)

[14208.444211] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

[14219.540575] NVRM: GPU 0000:1a:00.0: RmInitAdapter failed! (0x62:0x65:2661)

[14219.543871] NVRM: GPU 0000:1a:00.0: rm_init_adapter failed, device minor number 3

nvidia-bug-report.log.gz (7.5 MB)