Using nv_omx_videomixer with nv_omx_hdmi_videosink in Jetson TK1

Autonomous Machines Jetson & Embedded Systems Jetson TK1
1

Hi,

My goal is to run 2 videos in non-windowed mode. I managed to get this working as below:

gst-launch -e -v videomixer name=mix sink_0::xpos=1000 sink_1::xpos=0 ! queue ! nvvidconv ! video/x-nv-yuv, format=\(fourcc\)NV12 ! \
nv_omx_hdmi_videosink overlay-x=100 overlay-y=100 overlay-w=1800 overlay-h=450 \
filesrc location=test2.mp4 ! qtdemux name=demux1 demux1.video_00 ! queue ! nv_omx_h264dec ! "video/x-nv-yuv" ! queue ! nvvidconv ! queue ! mix.sink_0 \
filesrc location=test3.mp4 ! qtdemux name=demux2 demux2.video_00 ! queue ! nv_omx_h264dec ! "video/x-nv-yuv" ! queue ! nvvidconv ! queue ! mix.sink_1

The problem with this approach is that it takes higher CPU and frame loss happens if other tasks are done (for example, opening a chromium browser). I believe this is because of using the videomixer and the conversions done in between.

So, I started to use the omx videomixer with 1 video file before bringing in 2 videos as below, hoping to reduce CPU usage/conversions:

gst-launch -e -v filesrc location=test2.mp4 ! qtdemux name=demux1 demux1.video_00 ! nv_omx_h264dec ! video/x-nv-yuv ! \
nvvidconv ! video/x-nv-yuv, format=\(fourcc\)I420 ! nv_omx_videomixer name=mix ! video/x-nv-yuv, format=\(fourcc\)I420 ! \
nv_omx_hdmi_videosink overlay-x=100 overlay-y=100 overlay-w=600 overlay-h=250

and, I get the below error:

Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingSetting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...

I am a beginner, so, please take it easy. Any help to guid me to move forward would be greatly appreciated. Thanks

2

I searched for some examples using omx videomixer as below and I still see the same error. Has anyone been able to use nv_omx_videomixer successfully?

gst-launch videotestsrc ! 'video/x-raw-yuv, width=(int)640, height=(int)480, format=(fourcc)YUY2' ! nvvidconv ! 'video/x-nv-yuv, width=(int)640, height=(int)480, format=(fourcc)I420' ! nv_omx_videomixer ! nv_gl_eglimagesink
3

I wrote a simple python player that uses 4 independent pipelines to decode and render videos. The code assumes xvimagesink is used for the rendering part and that seems to work on my PC.

EDIT: I modified it to explicitly use xvimagesink.

I haven’t yet had the time to test this on Jetson but if you want to try it out:

[url]https://github.com/kulve/gst-multiwindow[/url]

4

Thanks for your reply. I tried your code and it worked ok in Jeston, but the CPU usage was still high and the videos lagged slightly behind each other.

I am specifically looking to use nv_omx_videomixer so that I can use nv_omx_hdmi_videosink for windowless playback.