Challenges with Visualizing GUI Apps and Processing Pointcloud/RGB Data over SSH (slow SSH experience)

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hello everyone,

I’ve been encountering significant difficulties when trying to visualize GUI applications and handle pointcloud/RGB data remotely via SSH. The primary issue is the extremely slow performance, making visualization tasks almost impractical.

To give a detailed picture of my setup and the issues I’ve faced, here’s a rundown of the hardware I’ve been using:

  • Nvidia Nano 4GB Module on Nvidia dev board
  • Nvidia Nano 4GB Module on an Auvidea JN30D Carrier Board
  • Xavier NX 8GB Module on an Auvidea JNX30D Carrier Board
  • Xavier NX 16GB Module also on a JNX30D Carrier Board
  • Intel Realsense D435i Depth Camera
  • Intel Realsense T265 Tracking Camera
  • TP-Link UE330 USB 3.0 Hub (used for connecting the cameras and a Wi-Fi dongle)

In my attempts, I tested all these boards with the mentioned sensors. My objective was to visualize GUIs (such as the Realsense SDK) or transmit video streams using OpenCV. Unfortunately, regardless of the configuration, the performance was consistently slow. This sluggishness persists with SSH connections, both over Wi-Fi and Ethernet (though Ethernet is marginally better).

As an additional troubleshooting step, I powered the USB hub externally instead of relying on power from the Jetson boards. However, this didn’t lead to any noticeable improvement. By the way, there is no issues downloading large packages.

At this point, the only feasible method to work effectively with all the sensors is by directly connecting an HDMI cable and using a monitor for visualization. I’m reaching out to see if anyone else has faced similar challenges or has any suggestions on how to improve the SSH-based visualization performance.

Thanks in advance for any advice or insights!
Selim

2

I have to point to a distinction between ssh forwarding of X (the X11 server uses the X protocol…previously known as the X protocol 😁 …sorry, had to joke about twitter, which changed its name to X, but the X protocol was there long before twitter changed its name), and virtual desktops. The details of what hurts or improves performance is quite different.

For more detail I’ll refer to these URLs:

If you are using ssh forwarding, then the X server is forwarding events, not graphics. The graphics performance is mainly due to the machine which is displaying the graphics. In the case of a virtual desktop, the end display system has some influence on this, but not nearly as much.

Consider the case of an OpenGL accelerated application. Or a CUDA accelerated application. Both use the GPU. Not all CUDA uses the X display system interface to the GPU, but some applications do. When you forward events, then it is up to the display computer to process the graphics (or CUDA) calls (assuming it is CUDA going through the X server as an API to reach the GPU). If the desktop PC has all of the requirements, and is using a 4090 Ti GPU, then all of a sudden your Jetson seems to be a mighty powerhouse of performance. On the other hand, if your desktop has integrated graphics, or worse yet, is software rendering (and CUDA with a non-NVIDIA GPU will software render), then the Jetson will appear to perform terribly bad. Not only that, in both cases, the Jetson GPU load will show as low.

The reverse situation, where the GPU is used on the Jetson to display locally will imply that all of the performance greatness or sorrow belongs solely to the Jetson.

There is a third case, that all is rendered on the Jetson, but the video buffer goes to a virtual network device on another computer. That’s a virtual desktop. This latter case implies forwarding the graphics of the Jetson’s work, and not the events which make up the instructions of how to render. The network does limit how much graphical data goes through, and the end display computer can limit how fast this can be rendered, but virutal video frame compression tends to mean most of the performance is a result of the Jetson, and not the remote display computer.

You might even find applications which run on the Jetson, but won’t work when forwarded to the host PC over ssh. For example, if the application running on the Jetson only works with a given CUDA release or given version of OpenGL, and the desktop PC doing the display has the wrong version, then forwarding won’t work. If you used a virtual desktop, then there will never be a compatibility issue from that since events are not used to construct a result. The virtual server is indistinguishable to the Jetson in comparison to a local server with a physical monitor attached.

Make sure you know where the work load is actually distributed. I’m not the guy to ask about virtual desktop installation, but if you really want to be certain, or if you want to be portable across operating systems, then you need to look up virtual desktops.

Someone else could comment on VNC or TurboVNC or other virtual desktop servers and clients.

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.