Hi all,
In GB300 compute tray the following error occurs and not moving once it is taken place…Any one can explain and why does that occurred? Has any one come across the following error in booting time. Not moving. And how to investigate and how to fix it?
I am the Nvidia Technical Support, about your issue on the GB300 compute tray, the error message of “Synchronous exception at 0x0000000000000” is too simple, it might be related to the BF3 card, please help to confirm the information as follows:
this error message is seen on the bootup process of the cutomer tray, and it hung here with this error message, is it correct?
the BF3 HCA card is on the NIC mode or the DPU mode?
the compute tray is load the OS from its local disk? or load the OS from the BCM server via the PXE?
And please try the operation as follows:
power off the compute tray, wait for 5-10 minutes, plugout it (wait for 1 minute) and plugin it, wait for 10-15 minutes (let the liquid be stable)
then check if it can bootup correctly into the OS?
Please share the above information related firstly, if there is any question, please contact us.
this error message is seen on the bootup process of the cutomer tray, and it hung here with this error message, is it correct?
-Yes, it is certainly.
the BF3 HCA card is on the NIC mode or the DPU mode?
-How to check and determine it is on NIC or DPU mode?
the compute tray is load the OS from its local disk? or load the OS from the BCM server via the PXE?
-Does it make difference?
And please try the operation as follows:
power off the compute tray, wait for 5-10 minutes, plugout it (wait for 1 minute) and plugin it, wait for 10-15 minutes (let the liquid be stable)
Is that performing power drain for 2-3 mins ?
then check if it can bootup correctly into the OS?
How to check that?
Just according to the simple message of “Synchronous exception at 0x0000000000000”, we cannot judge it because of the BF3 hardware issue at 100%, it needs check more information, so it is better to get some other information on the OS.
But now according to your information, this compute tray seem be hung and cannot enter the OS.
So for this loading OS issue, it might be related:
the BF3 really has the hardware issue, which impact the OS load;
if your compute tray is loading the OS image via BF3 from the BCM server, then it also mingt be related BF3 configure error or the BCM server configure error, which impact the OS load;
So please confirm:
if your compute tray is loading the OS from the local disk, then we ignore BF3 configue or the BCM configure, please just re-seat this compute tray, and check if it can enter the OS correctly (you can see the tip to input the username and password from the screen, or you can ssh login it);
if your compute tray is loading the OS form the remote BCM server via the BF3 card, usually the BF3 needs to configue to NIC mode.
About how to check the BF3 NIC mode, please check it as follows:
reboot this compute tray, then enter the BIOS
click “Device Manager”
click “Network Device List”
then it will show a few MAC address
click any one MAC address, then click “Nvidia Network Adapter - xx:xx:xx:xx:xx:xx”, then check if it shows “Chip Type BlueField-3”, if it has no such information, then this MAC address is not for the BF3, then please go back and select another MAC address; if it shows the “Chip Type BlueField-3”, then this is the BF3 card, and click the “BlueField Internal Cpu Configuration”, and check the “Internal Cpu Offload Engine”: Disabled (it means the NIC mode); Enabled (it means the DPU mode).
Please check, if there is any question, pllease contact us.