according to the https://docs.nvidia.com/networking/pages/viewpage.action?pageId=64306551#IPoverInfiniBand(IPoIB)-EnhancedIPoIB , try to change datagram in /sys/class/net/ib/mode , but im failed .Im setting 'SET_IPOIB_CM=yes'and /etc/init.d/openibd restart ,then found which in the mode is datagram . Use echo connected > /sys/class/net/ib/mode ,also failed, i need help .