Only seeing about 160MB/s on file transfers between two Ubuntu desktops with Mellanox DAC cables and Mellanox cards.

Infrastructure & Networking Software And Drivers
, , ,
1

Using a Mellanox DAC cable and two CX311A cards to directly connect two Ubuntu 18.04 desktops. My RAID array can handle writing at over 500MB/s but on file transfers between the PCs, I’m seeing about 150 to 170MB/s when I should be getting about 500MB/s. Cards are installed in PCI x16 slots. Any idea what could be wrong?

2

Hi Gene,

Can you validate that your servers and cards are tuned as per our tuning guide —> https://community.mellanox.com/s/article/performance-tuning-for-mellanox-adapters

Can you please provide following information:

  1. What is type of traffic? (TCP,UDP etc)
  2. #ibv_devinfo​
  3. P/N of cable?
  4. #[ofed_info -s]​

Thanks,

Namrata.