Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?

Infrastructure & Networking Software And Drivers Mellanox OFED
1

Hello.

I have Ubuntu 14.04. I have Mellanox Technologies MT27500 Family [ConnectX-3] HCAs. I was able to pass all the tests like ibping and ib_rdma_lat.

Now I’m running MPI tests to verify that InfiniBand is working. How can I actually see that there is traffic going through the HCA’s port?

root@node3:~/ibdump-1.0.5-4-rpms/ibdump-1.0.5-4.x86_64-rhel6/usr/bin# ./ibdump ------------------------------------------------ IB device : "mlx4_0" IB port : 1 Dump file : sniffer.pcap Sniffer WQEs (max burst size) : 4096 ------------------------------------------------ Initiating resources ... searching for IB devices in host Warning: memory access to device mlx4_0 failed: Input/output error. Warning: Fallback on IO: much slower, and unsafe if device in use. -E- Failed to open mst device mlx4_0: No such file or directory -E- failed to create resources

I’ve tried to execute the ibdump utility (from all the RPMs that were in the download). But it fails. with this:

But there is actually the mlx4_0 device. As I said all the tests from the infiniband-diags package are passed.

Is ibdump supported in MLNX_OFED 2 in Ubuntu 14.04? If no then what alternatives can I use?

Thank you.

2

Now, it works also for me with ibdump 2.0.0-5 (the one included in MLNX_OFED 2.0-3.0.0).

So it seems there is a regression in ibdump 2.0.0-9 (the one included in MLNX_OFED_LINUX-2.3-2.0.0)

I can see my network trace in wireshark with no problem. that’s great…

Thanks a lot

3

I have only tried with MLNX_OFED and it worked nicely for me. I was able to capture and open the files in wireshark.

I haven’t tried the OFED library.

4

I was able to capture IB traffic with the ConnectX-3 adapters in Ubuntu 12.04, by using ibdump 2.0.0 from the Mellanox OFED RPMs.

I just converted the rpm package to deb using alien.

5

Hi Grigory,

ibdump 1.0.5.4 doesn’t support ConnectX-3, and currently there is no ibdump version that supports Ubuntu/ Debian.

Currently there is no alternative for IB traffic on Ubuntu Machines.

I suggest to check with the community or support@mellanox.com mailto:support@mellanox.com in the coming future ( next few months) for any development around this feature

6

Hi,

I try to capture the RDMA traffic also with ibdump, but Ibdump 2.0 doesn’t work on my host.

OS: SUSE (SLES11SP3), library OFED-3.12-1-rc3, Mellanox Connect X-3 NIC.

ibdump --mem-mode 65536

Initiating resources …

searching for IB devices in host

Port active_mtu=4096

MR was registered with addr=0x6197e0, lkey=0xe8011900, rkey=0xe8011900, flags=0x1

Device : “mlx4_0”

Physical port : 1

Link layer : Infiniband

Dump file : sniffer.pcap

Sniffer WQEs (max burst size) : 4096

-E- Failed to set sniffer mode. ibv_create_flow failed: File exists

It creates an empty ‘sniffer.pcap’ file, and that’s all. No capture file.

Does ibdump work with OFED library? I haven’t installed the MLNX_OFED lib.Is it required?

thanks

Eric

7

I’ve uninstalled OFED and installed MLNX_OFED_LINUX-2.3-2.0.0 on my SUSE host, but ibdump does not work better.

ibdump

Initiating resources …

searching for IB devices in host

Port active_mtu=4096

MR was registered with addr=0x6198f0, lkey=0x88011300, rkey=0x88011300, flags=0x1

Device : “mlx4_0”

Physical port : 1

Link layer : Infiniband

Dump file : sniffer.pcap

Sniffer WQEs (max burst size) : 4096

-E- Failed to set sniffer mode. ibv_create_flow failed: Function not implemented

any idea about the cause of this “function not implemented” ?

Thanks

Eric

8

It was working for me on MLNX_OFED 2.0-3.0.0 on Ubuntu 12.04.

Now that I upgraded to the latest version of the drivers (2.3-2.0.0), I got the same error as the one you got.

However, if I use the ibdump package included in the MLNX_OFED 2.0-3.0.0, it works even with the latest drivers (2.3-2.0.0)!