passthru MT27500 ConnectX-3 SR-IOV via kvm-qemu cause host Redhat 6.4 crash

Hi there,

When I tried to passthrough MT27500 ConnectX-3 SR-IOV via kvm-qemu, it always cause the host, RedHat 6.4, crash.

Host: IBM dx360 m4

Redhat 6.4, default kernel

kvm-qemu version is 2.5

Mellanox OFED driver 3.1-1.1.0.1

firmware version 2.35.5100

list:

20:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

20:00.1 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function]

20:00.2 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function]

20:00.3 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function]

20:00.4 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function]

I tried many times, it always cause host crash. Did I make some apparent mistakes?

Please advice and many thanks in advance!

Hi Noa,

Thanks for your reply. Due to the RHEL 6.4, this driver, 3.1-1.1.0.1, is latest suitable one. The 3.3… driver doesn’t support RHEL 6.4. I was not at the scene temporarily, when I am back, I will try to catch the messages.

Cheers,

Delsun

HI Delsun

I suggest you check your core dump file for detail information of the crash.

Thanks

Do you found any special report when it crash ?

Thanks

Are you use IB or Etherent

And if you configure SR-IOV following following guide ?

HowTo Configure SR-IOV for ConnectX-3 with KVM (Ethernet) https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-3-with-kvm--ethernet-x

HowTo Configure SR-IOV for ConnectX-3 with KVM (InfiniBand) https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-3-with-kvm--infiniband-x

Thansk

I’m using IB. Yes, I really configured it as HowTo Configure SR-IOV for ConnectX-3 with KVM (InfiniBand) https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-3-with-kvm--infiniband-x that’s why we see the

20:00.1 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function]

right?

Zhang Suo,

Thanks. I didn’t find anything about crash. Maybe I should say I don’t know what I should look. Do you have any ideas about this? Where can I find logs/ error files etc… Which files are helpful to this issue?

On the other side, I always doubt there are some wrong settings. If you also think so, which setting should I check?

BTW, I found this post

SR-IOV issues with Mellanox CX-3 with RH6.4 KVM

but in this post, the host did not crash, the problem is just the guest cannot start…and this post mentioned the kernel should upgrade…

Cheers

Hi,

You’re using relatively old driver and FW. Any reason for that? If no, can you upgrade them?

Also, please attach dmesg output (you can use netconsole to keep it logged on a different server, or keep another shell opened and tail /var/log/messages).

Hi Delsun

Can you let us know if you have any update for this issue?

Thansk

Zhang Suo,

Thanks for your concern. I was not at the scene temporarily, when I am back, I will update this.

Cheers,

Delsun

Hi Delsun Foo

Sorry for late because I am on business trip these days.

I don’t know if your issue is solved , but,

From the file

mlx4_core 0000:1b:00.0: Bad or unreadable EEPROM on port 1

Do you try othere card?

Thanks

Original message from @Delsun Foo​