Any time an event goes to an X server the GPU driver of that X server is what implements the result. When you use “ssh -Y” or “ssh -X” you are forwarding events from Jetson to PC…thus the PC is what handles OpenGL and CUDA for that case (and thus the PC GPU does GPU side of the work, but CPU work is done on the Jetson). If you just ssh as usual, or log in at a text console, and then “export DISPLAY=…”, then the display which DISPLAY refers to is the one used for both CPU and GPU.
In most cases the DISPLAY variable will name the display of the local system. This is an example of “:0” and “:1”…these would execute on the Jetson (“:0” is the first display, “:1” is the second display, so on…default is “:0” on a TX2, but Xavier starts with “:1”). For a local DISPLAY to be valid (and if you ssh without “-Y” or “-X” and set “export DISPLAY=:0”, then this is local to the Jetson) the same user who exports DISPLAY must be logged in to the GUI of that machine.
If you ssh without forwarding to a Jetson, and “export DISPLAY=:0”, then all CUDA and OpenGL is running on the Jetson. A failure to display would not be due to basic environment settings and would require other debugging. You’d need to say what application you started, the command line, and anything special or custom about the system setup.
Note that when an X server is started the instance designation (e.g., “:0”) can be manually set. More than one X server can run. An example would be having two graphics cards running completely independent displays rather than having two monitors combined as a desktop. One display would likely be “:0”, and the other “:1”. If the same user logs in to both instances, then the one used to display would depend on which one is exported in “export DISPLAY=…”.
A virtual X server works just like a real server…you could have a real display on “:0”, and also have a virtual server…virtual servers, simply by tradition, are usually started at “:10”. A virtual server can replace a real display. It just so happens that a virtual client can be used over the network to see the virtual desktop. Anything remote which talks to a virtual X server is using a custom protocol and no X event forwarding takes place. With a virtual X server the Jetson will always be the one using its GPU for any CUDA or OpenGL application. To forward X events you’d need another computer running X…Windows can’t do this without some sort of emulation…but a virtual desktop client can run on any platform and talk to a Linux desktop (Windows, Mac, Linux, FreeBSD, so on).
Your desktop PC without GPU would be incapable of using “ssh -X” or “ssh -Y”…there no server to forward to. Note that you could run X with an alternate video card (perhaps an integrated Intel GPU), and anything specific to NVIDIA would fail, e.g., CUDA. Commands not relying on the specific graphics might work, e.g., xterm doesn’t require OpenGL so it might work (no hardware acceleration, it’d be entirely software rendering).
Lesson: If you don’t want to run a display on a Jetson, but want to run CUDA or anything with GPU hardware acceleration, then add a virtual desktop server. Ignore port forwarding via “ssh -Y” or “ssh -X”.