onboard ethernet connection conflict

I am working on a project that uses two Dell computers running ubuntu 16.04 LTS to pass data between systems using RDMA. Each system uses a connectX-5 adapter.

That part is working great.

The problem is that the onboard ethernet for the system stops working when I boot with the Mellanox cards plugged in. As soon as I unplug them, there is no more conflict and I can connect to the internet. after I plug them back in, things will work, but following another reboot, the connection is lost. The onboard ethernet shows up fine with lspci, I assume it is some type of driver conflict, but can’t seem to track it down at all.

Has anyone else run into this problem?

Any suggestions on how to go about debugging?