NV p100 and usb display card can not work together

My system is ubuntu 14.04, for the hardware, it has both NV p100 GPU and usb display card, but I find the usb display card will not work. If I forbid the NV p100 GPU driver or unplug/plug the usb display card, the usb display card will work and the screen can output image output. so I think this issue is due to NV P100 Linux driver has conflict with the USB displaylink drivers. can you give some suggestion about how to fix it?