Some explanation of X servers might help here. The gist is that ssh forwards “events”, and those events are processed by the GPU of the host PC instead of the Jetson. Virtual servers process events directly on the Jetson with its GPU, and forward the results of those events to the PC. For more detail:
https://forums.developer.nvidia.com/t/video-output-does-not-show-up-because-opengl-failed-to-create-x11-window-imagenet-camera-failed-to-create-opengl-display/68681/7
To put that in a more intuitive context relative to your issue, consider that if your program either uses CUDA or directly rendered OpenGL, then the libraries and APIs of the host PC must have that software for ssh forwarding, and that software must be of the correct revision/version. For the case of a virtual desktop, that same CUDA or OpenGL software must run on the Jetson itself, and the only thing the PC needs is the virtual desktop software. ssh forwarding from a Jetson implies you are no longer using its GPU; virtual desktops running on a Jetson implies the Jetson is using its GPU.