I have to point to a distinction between ssh forwarding of X (the X11 server uses the X protocol…previously known as the X protocol 😁 …sorry, had to joke about twitter, which changed its name to X, but the X protocol was there long before twitter changed its name), and virtual desktops. The details of what hurts or improves performance is quite different.
For more detail I’ll refer to these URLs:
- https://forums.developer.nvidia.com/t/headless-configurations-with-remote-opengl-accelerated-desktop-jetson-xavier/264784/16
- https://forums.developer.nvidia.com/t/video-output-does-not-show-up-because-opengl-failed-to-create-x11-window-imagenet-camera-failed-to-create-opengl-display/68681/7
If you are using ssh forwarding, then the X server is forwarding events, not graphics. The graphics performance is mainly due to the machine which is displaying the graphics. In the case of a virtual desktop, the end display system has some influence on this, but not nearly as much.
Consider the case of an OpenGL accelerated application. Or a CUDA accelerated application. Both use the GPU. Not all CUDA uses the X display system interface to the GPU, but some applications do. When you forward events, then it is up to the display computer to process the graphics (or CUDA) calls (assuming it is CUDA going through the X server as an API to reach the GPU). If the desktop PC has all of the requirements, and is using a 4090 Ti GPU, then all of a sudden your Jetson seems to be a mighty powerhouse of performance. On the other hand, if your desktop has integrated graphics, or worse yet, is software rendering (and CUDA with a non-NVIDIA GPU will software render), then the Jetson will appear to perform terribly bad. Not only that, in both cases, the Jetson GPU load will show as low.
The reverse situation, where the GPU is used on the Jetson to display locally will imply that all of the performance greatness or sorrow belongs solely to the Jetson.
There is a third case, that all is rendered on the Jetson, but the video buffer goes to a virtual network device on another computer. That’s a virtual desktop. This latter case implies forwarding the graphics of the Jetson’s work, and not the events which make up the instructions of how to render. The network does limit how much graphical data goes through, and the end display computer can limit how fast this can be rendered, but virutal video frame compression tends to mean most of the performance is a result of the Jetson, and not the remote display computer.
You might even find applications which run on the Jetson, but won’t work when forwarded to the host PC over ssh. For example, if the application running on the Jetson only works with a given CUDA release or given version of OpenGL, and the desktop PC doing the display has the wrong version, then forwarding won’t work. If you used a virtual desktop, then there will never be a compatibility issue from that since events are not used to construct a result. The virtual server is indistinguishable to the Jetson in comparison to a local server with a physical monitor attached.
Make sure you know where the work load is actually distributed. I’m not the guy to ask about virtual desktop installation, but if you really want to be certain, or if you want to be portable across operating systems, then you need to look up virtual desktops.
Someone else could comment on VNC or TurboVNC or other virtual desktop servers and clients.