I can’t answer your question, but there is a common subtle point to remote connections and X11. People think of X as a graphics system, and it is, but it is also an interface to a GPU even if you don’t use it for graphics.
X has the ability to forward to another X server, and when it does that, it forwards events, not resulting computations. The GPU on the server doing the display is the one which does the processing regardless of which machine the program runs on. If you run and display entirely on the Jetson, then you can never go wrong because CUDA and driver on its GPU will always match. If you run on PC and display on PC, you are also in good shape.
If you run on Jetson and display on PC, then you found the quirk. PC’s CUDA/GPU/drivers are what the program running on Jetson will require, but most likely the program was compiled for the Jetson’s CUDA version. You could compile the program to make available both CUDA versions with some advanced effort (I haven’t done it, but there is an “advanced” setup section in the nsight edition of eclipse which allows this). You’d still be using the PC’s GPU though, which may not be what you want. If you use “ssh -X” or “ssh -Y”, then you can be certain X11 events are being forwarded and there is an attempt to offload GPU work from Jetson to PC.
If you run a virtual desktop on the Jetson, and display to a virtual desktop client on the PC, then you get all you wanted…it runs on the Jetson, uses the Jetson’s GPU, can work even when the Jetson doesn’t have a monitor, and can be viewed in real time on the PC. There have been several threads about virtual desktop setup, and there may even be a suitable virtual desktop server setup from the Jetson’s Ubuntu tool box…I haven’t set this up myself though. This even works to display from Windows.