Ib_write_bw on cx5 interface fails

Infrastructure & Networking Software And Drivers
#1

I am trying to run ib_write_bw on 2 cx5 cards i.e. 4 ports parallely but it fails with

Completion with error at client

Failed status 12: wr_id 0 syndrom 0x81

scnt=128, ccnt=128

Error occurred while running infinitely! aborting …

#2

Hello Sanchi,

Are both kernels and drivers the same version?

Have you tried the “-d” option to specify the devices to use on both the sender and the receiver?

example: -x is gid index “command: show_gids”

root@server1:~# ib_send_bw -x 3 -d mlx5_0

root@server2:~# ib_send_bw -x 5 -d mlx5_0 10.0.0.2 --run_infinitely

many thanks,

~Steve

#3

Hi Steve,

Thanks for your response.

I have a code where I am using ib_write_bw command using posix_spawn.

When I run that below command are actually executing.

root 128901 118460 0 03:03 pts/3 00:00:00 /usr/bin/ib_write_bw -p 45980 -d mlx5_3 --run_infinitely

root 128961 118460 21 03:03 pts/3 00:00:05 /usr/bin/ib_write_bw -p 45980 -d mlx5_3 1.1.3.1 -s 65536 --run_infinitely

root 129021 118465 0 03:03 pts/3 00:00:00 /usr/bin/ib_write_bw -p 45979 -d mlx5_2 --run_infinitely

root 129088 118465 23 03:03 pts/3 00:00:05 /usr/bin/ib_write_bw -p 45979 -d mlx5_2 1.1.2.1 -s 65536 --run_infinitely

root 129722 118458 0 03:04 pts/3 00:00:00 /usr/bin/ib_write_bw -p 45990 -d mlx5_3 --run_infinitely

root 129733 118458 20 03:04 pts/3 00:00:02 /usr/bin/ib_write_bw -p 45990 -d mlx5_3 1.1.3.2 -s 65536 --run_infinitely

root 129921 118464 1 03:04 pts/3 00:00:00 /usr/bin/ib_write_bw -p 45989 -d mlx5_2 --run_infinitely

root 129943 118464 15 03:04 pts/3 00:00:00 /usr/bin/ib_write_bw -p 45989 -d mlx5_2 1.1.2.2 -s 65536 --run_infinitely

and it fails on one or 2 ports .

Thanks

Sanchi

#4

Hi Sanchi

Could you open a case with Mellanox support?

Many thanks -

Steve

#5

Hi Sanchi,

Unfortunately ib_write_bw can’t be used on multiple ports.

Many thanks,

~Steve