Problem running gstreamer via ssh / X11

[marving1]

Hello everyone!

I’m having a TX2 flashed with JetPack 4.4 DP and a quite ‘old’ Microsoft LifeCam VX-800. When I’m connected directly on the TX2 (no ssh) I have no problems in running gstreamer like this:

gst-launch-1.0 v4l2src device=/dev/video1 ! xvimagesink with a relatively good performance.

However if I want to do it remotely via ssh I have a lot of trouble get it working. On my host (win10) I’m running Xming server and set a display variable like this:
set DISPLAY=127.0.0.1:0.0 and then connect to my TX2 ssh -Y tx2@192.168.0.150 I’m able to confirm that the forwarding works with commands like xclock or xeyes, but the gst-launch command above unfortunateley throws this error:

tx2@tx2-desktop:~$ gst-launch-1.0 v4l2src device=/dev/video1 ! xvimagesink
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Could not initialise Xv output
Additional debug info:
xvimagesink.c(1773): gst_xv_image_sink_open (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
XVideo extension is not available
Setting pipeline to NULL ...
Freeing pipeline ...

I also read about adding videoconvert before xvimagesink, but that doesn’t help either :/
Same error persists:

tx2@tx2-desktop:~$ gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,framerate=30/1,width=640,height=480 ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Could not initialise Xv output
Additional debug info:
xvimagesink.c(1773): gst_xv_image_sink_open (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
XVideo extension is not available
Setting pipeline to NULL ...
Freeing pipeline ...

Here’s my v4l2-ctl -d /dev/video1 --list-formats-ext if that helps:

tx2@tx2-desktop:~$ v4l2-ctl -d /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUYV 4:2:2
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 352x288
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 320x240
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 176x144
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 160x120
                        Interval: Discrete 0.033s (30.000 fps)

Any help is appreciated :-)

[DaneLLL]

Hi,
In remote login, we configure

nvidia@nvidia-desktop:~$ export DISPLAY=:1
nvidia@nvidia-desktop:~$ xrandr
Screen 0: minimum 8 x 8, current 1920 x 1080, maximum 32767 x 32767
HDMI-0 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 530mm x 300mm
   1920x1080     60.00*+
   1280x1024     75.03    60.00
   1152x864      75.00
   1024x768      75.03    60.01
   800x600       75.00    60.32
   720x400       70.04
   640x480       75.00    59.94

Please give it a try. If you hit

nvidia@nvidia-desktop:~$ xrandr
No protocol specified
Can't open display :0

Please try

nvidia@nvidia-desktop:~$ export DISPLAY=:0

[marving1]

Hi DaneLLL,

I’m still getting an error and it tells me HDMI-0 disconnected primary

tx2@tx2-desktop:~$ env | grep DISPLAY
DISPLAY=localhost:10.0
tx2@tx2-desktop:~$ export DISPLAY=:1
tx2@tx2-desktop:~$ xrandr
Can't open display :1
tx2@tx2-desktop:~$ export DISPLAY=:0
tx2@tx2-desktop:~$ xrandr
Screen 0: minimum 8 x 8, current 640 x 480, maximum 32767 x 32767
HDMI-0 disconnected primary (normal left inverted right x axis y axis)

However if I try to run the gstreamer command I can see the camera is working (the green LED is indicating that), but I guess since HDMI-0 is disconnected that’s the reason I cannot see any picture and it stucks there:

tx2@tx2-desktop:~$ gst-launch-1.0 v4l2src device=/dev/video1 ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Btw. - not sure if it is important, but I just realized/saw this error message directly after login to the Jetson TX2:

[...]
Xlib:  extension "NV-GLX" missing on display "localhost:10.0".
Xlib:  extension "NV-GLX" missing on display "localhost:10.0".
Xlib:  extension "NV-GLX" missing on display "localhost:10.0".
tx2@tx2-desktop:~$

Thanks a lot! :-)

[DaneLLL]

Hi,
Your source supports video/x-raw,format=YUY2,width=640,height=480,framerate=30/1. If you cannot see video preview, please try to save to a file.

$ gst-launch-10. v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=a.mp4

[marving1]

Hi,

saving to a file works fine. However, is it possible to get the ‘livestream’ as a preview as it is possible without X11 on the device itself?
I might want to process the stream later on and therefore it is a little bit uncomfortable to save it to a file before :/

Thanks!

[DaneLLL]

Hi,

@marving1

However, is it possible to get the ‘livestream’ as a preview as it is possible without X11 on the device itself?

Not understand the request. Looks like you don’t have TV connected to TX2. How to see the video preview?

[marving1]

Well, I’d like to see the preview on my laptop which is connected to the device via ssh/x11.
I just figured out that it works when I use ximagesink in combination with videoconvert instead of xvimagesink.

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,framerate=30/1,width=640,height=480 ! videoconvert ! ximagesink

The only downside is, that the performance is extreme bad. I can basically see every frame updating with a delay of like 4-5 seconds.

[DaneLLL]

Hi,
You may try nveglglessink.

$ gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! nvegltransform ! nveglglessink

[marving1]

Hi DaneLLL,

first of all thanks a lot for being so patient - I appreciate that a lot! :)

Unfortunately it tells me that it cannot get an EGL display connection:

tx2@tx2-desktop:~$ gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! nvegltransform ! nveglglessink
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED ...

Using winsys: x11
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...

[DaneLLL]

Hi,
Sorry I thought it is native display and did not realize it is ssh/x11 forwarding. We don’t support it in nveglglessink. You would need to use ximagesink.

One possible solution is to use RTSP. Please refer to a valid usecase:

Problems minimizing latency and maximizing quality for rtsp and mpeg-ts.

We do not observe long latency in streaming with nvcamerasrc: Server $ ./test-launch "nvcamerasrc ! video/x-raw(memory:NVMM),width=640,height=480 ! tee name=t t. ! queue ! nvoverlaysink overlay-x=50 overlay-y=50 overlay-w=640 overlay-h=480 t. ! queue ! omxh264enc ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96" Client $ export RTSP_PATH=rtsp://127.0.0.1:8554/test $ gst-launch-1.0 rtspsrc location="$RTSP_PATH" latency=500 ! rtph264depay ! h264parse ! omxh264dec ! nvoverlaysink ov…