How do I use a video sink through VNC/X tunnel and fix "No EGL Display"?

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hello,

I am unable to open an autovideosink when I connect to the Xavier remotely using VNC or X-tunnelling through ssh (ssh -X). In both cases, it returns “No EGL Display”, even though I can pop open X-applications with no problem. I tried every video sink present in the default image.

This works locally and over VNC/ssh-X:

gtk-launch-1.0 videotestsrc ! ximagesink

This only works locallly:

gst-launch-1.0 filesrc location=small.mp4 ! decodebin ! videoconvert ! ximagesink

And complains “Could not get EGL display connection”.

I thought it was perhaps due to the decode/convert stages using NVVRAM causing the problem. It is odd that it isn’t working on VNC, at least I can see how ssh-X might fail to have an EGL driver on the X system.

I cannot unset DISPLAY the way other comments in this forum have suggested: I actually want to render the output.

Thoughts?

Thanks,
Peter

2

You should know that “ssh -X” (or “ssh -Y”) probably does not do what you think it does. I’ll post a couple of related URLs, YMMV:

The gist is that ssh forwarding is very different versus a desktop which is virtual with the desktop itself being forwarded. ssh forwarding forwards events, and the events are what cause various graphics or mouse or keyboard actions to have meaning. If those events include EGL requirements, then the system actually providing the display is the one which needs that support. If you do not forward, then the Jetson needs EGL support; once you forward, it is the other computer which provides the GPU and EGL support.

If you want the requirement to be part of the Jetson, and not be offloaded to the PC, then it implies your software producing X events must all run and be interpreted on the Jetson without “forwarding” of events to a different computer. This is virtual desktop software, e.g., Vino, which has this function. A purely software virtual desktop still runs on the Jetson, and events are still interpreted by the “local” desktop, but then the desktop itself has the result of events forwarded instead of the actual event. The remote desktop client understands the forwarded desktop rather than the events.

VNC or other remote desktops, if set up correctly to run on the Jetson, but merely forward to a client on a PC will do the job. If there is an EGL requirement, then it would be the VNC virtual desktop (not the client) which would need EGL support. Try VNC without “-X” (I don’t use VNC, I probably should, but as is I’m not sure how one would make sure VNC is forwarding results instead of ssh forwarding events).

3

I see, I read your other posts. I need a remote desktop that is rendering ON the CUDA device, not forwarding X events to my local VNC Server. Let me experiment with Gnome and Vino. I thought I was doing that running vncserver on the Xavier, but apparently that isn’t running the EGL driver or something… I can understand why ssh -Y on macOS fails, but not vncserver on the Xavier. (Up this point I had been sinking to UDP and connected to that port on my mac to view the examples!)

Thanks for explaining.

Solved: I launched /usr/lib/vino/vino-server from the Xavier (I use mwm not gnome), and connected to that via macOS VNC. It works but it is unbearably slow and crashes frequently. I will resume using UDPSINK on the Xavier board.

4

@anon73843961
vnc typically works mapping entirely the display content of the embedded device [ jetson] as long as there is a hardware display connected to the jetson.
Also there is alternative way that might be used:

scenario 4. Accessing Jetson_NX graphical applications over vlc & x2x. From a Linux client.
It is a hybrid method, in certain cases it might require to use few computers including windows computer and linux computer. The former will display the video output and the latter will act as keyboard/mouse controller.
0. Make sure a hardware display is connected to Jetson_NX
At Linux computer[client] execute:

sudo apt install x2x

Connect to Jetson_NX over ssh terminal, navigate to the target folder

ssh ***jetsonNX***@*JetsonNX_ipaddress* -p *portnumber*
cd gst-rtsp-server-1.14.1/
cd examples

Export the DISPLAY output variable so that the hardware display connected to jetson will be the target source of the execution.

For DisplayPort connected display

export DISPLAY=:2

For a HDMI connected display the value will be 0 or 1 in the step above. To detect the type of the display connection ‘xrandr’ can be executed:

xrandr

Further a resolution of the streamed content can be adjusted e.g. with

xrandr --fb 1280x960

Start streaming server

./test-launch "ximagesrc use-damage=0 ! nvvidconv ! nvv4l2h265enc ! h265parse ! video/x-h265, stream-format=byte-stream ! rtph265pay name=pay0 pt=96 "

Connect a vlc player e.g. from local windows computer]
Network streaming source to specify : rtsp://ipaddress_ofJetsonNX:8554/test
If the connection doesn’t establish and in the terminal from step 4 there are messages about BLOCKING mode that means that export DISPLAY variable needs to be changed. Then the streamer server needs to be restarted and vlc playing will need to be tried again.
After there is a visual confirmation from the step 5 the only steps required to mount local keyboard & mouse to control these from local linux host is:

ssh -X JetsonNX@IP_address -p portnumber'x2x -east -to :0'

However, once the connection establishes it will be required to move the local cursor to the east of the display so it will extend to the vlc displayed display. Keystrokes will now go to the jetson display that is being streamed. To regain control to the local display - move the mouse out of the borders of the jetson display.
Known issues:
Sometimes if 0 display can not be opened 1 value might need to be used in the command above.

The defined method is just streaming of the display + separate controls
It also works over the Internet e.g. with ngrok tunnel

1 Like
5

The only VNC server that supports rendering OpenGL is x11vnc, it is hardware accelerated and support opengl because is two servers,.one is graphic interface, that is rendered again with opengl.

If can be installed in Jetsons, will work. I tried time ago and didn’t, but I didn’t lose more time with it, I had planed try again but is not a priority yet.

Try a let us know, I have it working in RPI4 streaming RVIZ rendered in opengl.

6

@fpsychosis : Strange: I started x11vnc -display :0, connected to it via macOS+RealVNC, but when I run, say, gst-launch-1.0 filesrc ... ! decodebin ! autovideosink, it doesn’t fail with EGL error, however there’s no output window showing the video. Display is set to :0, and there’s no error messages, just no video.

7

@fpsychosis
as per my experience, plain VNC [ vino]
works to deliver image of local jetson hardware display to remote vnc client
so if the jetson has rviz on its display - it will be displayed at the client, as the vnc in that case just mirrors the content without rendering
The downside - is that display/ hdmi emulator needs to be connected to jetson in order to get vino server useable

8

@Andrey1984
Wow, I’m impressed with this solution! However, it is very … heavy: it appears to create an mpeg rtsp stream of the desktop? Woof. I’m looking for something a bit more native. :)
Cheers,
Peter

9

@anon73843961
yes, it does create a stream [rtsp or udp] of the desktops content
controls ar attached with x2x