Which videosink for Jetson TX2 in EGLFS?

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,

I’m using Nvidia’s ubuntu on TX2.

I turn OFF X11 as follow:
`sudo systemctl stop gdm # disable desktop manager
sudo loginctl terminate-seat seat0

sudo -s
cd /sys/class/graphics/fb0
echo 4 > blank # Blanks monitor for changing
echo 0x0 > device/win_mask # Clears current window setting.
echo 0x3f > device/win_mask # Assigns all 6 overlay windows => it failed. Dunno why.
echo 0 > blank # Unblank display.

gst-launch-1.0 filesrc location= ‘/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_1080p_h264.mp4’ ! qtdemux ! h264parse ! nvv4l2decoder ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0 -e # it works
…

gst-launch-1.0 filesrc location= ‘/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_1080p_h264.mp4’ ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-4.0/samples/configs/deepstream-app/config_infer_primary.txt ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0 -e # FAILED!
…
Failed to set plane
Failed to display frame buffer
ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: Internal data stream error.
…

I tried also nvoverlaysink but it failed and it’s deprecated anyway.

What is wrong?

2

Not sure, but maybe for deepstream it would expect new buffer API.
You may try to add option bufapi-version=1 to decoder options:

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder bufapi-version=1 ! nvvideoconvert ! m.sink_0     nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-4.0/samples/configs/deepstream-app/config_infer_primary.txt ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0 -e
3

Hi,

Thank you for the reply.

I tried the command you suggest and various pipeline using bufapi-version=1.
The problem persists and bufapi-version seems to have no impact.

4

I can’t say more… I just tried your pipeline with bufapi-version=1 in decoder with my Xavier NX R32.4.4 and DS5, and it seemed to work, at least displaying fine the video.

5

OK thank you for your support.

I’m with TX2 RS32.3.1 and DS4.
I will try an upgrade as soon as possible.

6

Also note that I didn’t disable gdm and X, not tweaked the fb. I just switched to a virtual console with Alt-Ctl-F3 and tried there as I use to do for testing with nvdrmvideosink.
Not sure how much CUDA can work without X, nor in that case how much TRT can work without CUDA. Someone better skilled may advise for this.

7

Thanks for the note.
I did some addtional tests according to your inputs.

First, following commands are to validate some working pipelines. It does not perform what I want to do since I want to use hardware overlays. It only shows that decoder and neural network are working fine.

sudo chvt 1  # X11 desktop
gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder bufapi-version=1 ! nvvideoconvert ! xvimagesink -e  # OK on VT1

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder bufapi-version=1 ! nvvideoconvert ! m.sink_0     nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-4.0/samples/configs/deepstream-app/config_infer_primary.txt ! nvvideoconvert ! xvimagesink -e  # OK on VT1

Following commands are to use nvdrmvideosink and its built-in hardware overlay support. I tried to configure my ubuntu as Honey_Patouceul (except I’m still using deepstream 4.0).

# add NAutoVTs=6 to /etc/systemd/logind.conf
sudo chvt 2  # no desktop, X11 still running (ps -e | grep X)

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder bufapi-version=1 ! nvvideoconvert ! m.sink_0     nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-4.0/samples/configs/deepstream-app/config_infer_primary.txt ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0 -e  # NOK on VT2
# Failed to set plane 
# Failed to display frame buffer
# 0:00:05.679210968  9943   0x55b7104d90 WARN                 nvinfer gstnvinfer.cpp:1830:gst_nvinfer_output_loop:<nvinfer0> error: Internal data stream error.
# 0:00:05.679443064  9943   0x55b7104d90 WARN                 nvinfer gstnvinfer.cpp:1830:gst_nvinfer_output_loop:<nvinfer0> error: streaming stopped, reason error (-5)
# ERROR: from element /GstPipeline:pipeline0/GstNvInfer:nvinfer0: Internal data stream error.

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder bufapi-version=1 ! nvvideoconvert ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0 -e  # NOK on VT2
# Failed to set plane 
# Failed to display frame buffer
# ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: Internal data stream error.

Gstreamer’s errors does not seem to be the right one. I guess “failed to set plane” error from nvdrmvideosink breaks the pipeline. Then qtdemux and nvinfer generate errors but it’s not the root cause.

8

Not sure, but again do you have X server running or not ?
Without it CUDA may not be fully available, so wouldn’t TRT, so wouldn’t nvinfer model using TRT.
Just a thought, again someone more skilled may better advise.

9

I fixed the previous message to be more clear about X11.

Please note:

  • X11 server is still running after “sudo chvt 2”.
  • X11 server is no longer running after “sudo loginctl terminate-seat seat0”

I check X11 using “ps -e | grep X”.

So, nvdrmvideosink does not work with or without running X server.

11

Hi,
We now support nveglglessink and nvoverlaysink in DeepStream SDK. Please try the two sinks. And you can refer to this pipeline:

12

Hi,

@Honey_Patouceul

Without it CUDA may not be fully available, so wouldn’t TRT, so wouldn’t nvinfer model using TRT.

I do not understand this limitation. Do you have any detailed infos about it?

The following pipeline works on my TX2 (yocto / meta-tegra):

gst-launch-1.0 filesrc location= '/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4' ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvinfer config-file-path= /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/config_infer_primary.txt ! nvdsosd ! nvoverlaysink -e # OK on EGLFS, deepstream 5, l4t-r32.4.3

@DaneLLL

We now support nveglglessink and nvoverlaysink in DeepStream SDK. Please try the two sinks.

I’m trying to make it work using EGLFS. It seems nveglglessink should only work using X11 or wayland according to winsys attribute. This element fails every time I try to run a pipeline with.

The following documentation tells nvoverlaysink is deprecated :
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/accelerated_gstreamer.html

Half of the docmentation show examples with deprecated elements. This is confusing.
nvoverlaysink works better than nvdrmvideosink for my use case. Pretty weird. Thank you.

nvdsosd was failing with the following set: nvdrmvideosink + DS4 + l4t-r32.1.2 + EGLFS.
nvdsosd works with the following set: nvoverlaysink + DS5 + l4t-r32.4.3 + EGLFS.
I do not know exactly what make the job.

13

Hi,
We plan to deprecate the sinks and it may go through several releases. For backward compatibility, it is not a right-away deprecation. On DS5.0.1, nveglglessink and nvoverlaysink are valid. Please refer to the sink group:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_ref_app_deepstream.html#sink-group

14

This is a useful link. Thank you.