We tried the loopback test for Mellanox cards, there are two cards with two ports each. Each port from a Ethernet cards are shorted with SFP28 DAC Cable. In Ubuntu OS, the ports are correctly updating as UP and DOWN.
To check the packet transmission we assigned the static ip and try to send the packets using ping command. The packets are transmitting but not receiving.
Card: NVIDIA® ConnectX®-6 Lx
Cable: 25G SFP+ DAC Cable - Twinax SFP Cable
Let me know anybody have a suggestion
Welcome, and thank you for posting your inquiry to the NVIDIA Developer Forums!
With both ports connected in the same system, I assume you are assigning each interface an IP which is on the same subnet.
In Linux, when multiple interfaces on a node belong to the same subnet, it can lead to unexpected network behavior (as Linux hosts may respond to incoming packets via a different interface than the packet came in on).
In order to work around this situation, it is possible to use advanced routing and rules on the host side to guarantee that the Linux kernel will not accept traffic on a given interface if the destination IP is not defined on the incoming interface.
Examples of this type of configuration can be found on the web, for example:
But our recommendation is to use 2 separate systems to perform network testing and throughput measurements, as this type of test case is more representative of real-world capabilities.
If you are still unable to make this work after reviewing the above material (and similar articles around the web), we recommend engaging our Enterprise Support team by creating a support case.
NVIDIA Enterprise Experience