Some more observations. One problem seems to be that nvidia-p2p is not loaded. When trying to load this module manually it fails with “exports duplicate symbol” owned module nividia. I did the an experiment to unload the nvidia module. It is used by the graphical system so it must be disabled first.
sudo systemctl set-default multi-user.target
sudo reboot
*LOGIN AFTER REBOOT*
sudo modprobe -r nvidia
sudo modprobe nvidia-p2p
sudo insmod gdrdrv.ko
And hey i can load my module! Even the module built locally on the Jetson can be loaded. I have no means to actually verify p2p functionality at this stage.
WARNING! Doing this seems to kill the DisplayPort output and you can only access it with ssh even after reboot!
You can restore the system DisplayPort output with:
sudo systemctl set-default graphical.target
sudo reboot