Video playback: Full-screen mode vs. Windowed mode

NickHuang · November 26, 2014, 6:40am

Hi everyone, we followed L4T_Jetson_TK1_Multimedia_User_Guide_V1.2.pdf and found some problems when building our customized media player on Jetson TK1. Here’s the comparison of full-screen mode and windowed mode when playing the same 2160p 30fps h264 encoded video:

Full-screen Mode*:
CPU usage: 5%
comment: very smooth

Windowed Mode**:
CPU usage: 70%
comment: showing “WARNING: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: A lot of buffers are being dropped.”

We want to make our media player under windowed mode. Does anyone have any idea what’s the difference between HDMI Overlay Sink and Xvimagesink, or what’s the right way to construct pipeline? Thanks.

*pipeline: gst-launch-0.10 filesrc location=<filename.mp4> ! qtdemux name=demux
demux.video_00 ! queue ! nv_omx_h264dec ! nv_omx_hdmi_videosink –v -e

**pipeline: gst-launch-0.10 filesrc location=<filename.mp4> ! qtdemux name=demux
demux.video_00 ! queue ! nv_omx_h264dec ! ‘video/x-nv-yuv’ ! nvvidconv
! xvimagesink –v -e

kulve · November 26, 2014, 10:57am

Try this:

gst-launch-1.0 -v -e uridecodebin uri=file:///path/to/the/video.mp4 ! nveglglessink

NickHuang · November 26, 2014, 12:00pm

Hi kulve,
I tried the pipeline you mentioned but it shows

ubuntu@tegra-ubuntu:~/Videos$ gst-launch-1.0 -v -e uridecodebin uri=file:///home/ubuntu/Videos/2160p.mp4 ! nveglglessink
Setting pipeline to PAUSED ...
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: source = "\(GstFileSrc\)\ source"
Pipeline is PREROLLING ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = video/quicktime, variant=(string)iso
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = NULL
Missing element: H.264 decoder
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)5.1, profile=(string)high, codec_data=(buffer)01640033ffe1001a67640033acd300f0010fb0110000030001000003003c0f18313801000468cebcb0, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1'.
Additional debug info:
gsturidecodebin.c(930): unknown_type_cb (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:sink_0: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:src_0: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
Missing element: MPEG-4 AAC decoder
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2'.
Additional debug info:
gsturidecodebin.c(930): unknown_type_cb (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0: Your GStreamer installation is missing a plug-in.
Additional debug info:
gstdecodebin2.c(3928): gst_decode_bin_expose (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0:
no suitable plugins found
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Caught SIGSEGV
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstAacParse:aacparse0.GstPad:src: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstAacParse:aacparse0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
#0  __libc_do_syscall ()
#1  0xb6d82ec2 in pthread_join (threadid=<optimized out>, thread_return=0x0)
#2  0xb6dfc6a6 in ?? () from /lib/arm-linux-gnueabihf/libglib-2.0.so.0
Spinning.  Please run 'gdb gst-launch-1.0 2727' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

and ‘gdb gst-launch-1.0 2727’ shows that

Loaded symbols for /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstaudioparsers.so
__libc_do_syscall () at ../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:43
43	../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S: No such file or directory.

Did I miss anything? Thanks for your help!

NickHuang · November 26, 2014, 12:38pm

Oops… it seems that I didn’t install gstreamer-1.0 :p. I fixed it by
sudo apt-get install gstreamer1.0*

Now I can play H264 2160p 30fps video smoothly in windowed mode with 20% CPU usage by using the pipeline kulve mentioned.

Thank you, kulve!

rpaliwal_nvidia · November 27, 2014, 6:01am

Hi Nick,
If you only want to play videos at particular location with fixed size, you can do so with nv_omx_hdmi_videosink too by specifying the correct coordinates to the properties:
overlay-x (x coordinate)
overlay-y (y coordinate)
overlay-h (window height)
overlay-w (window width)

If you want windowed playback, nveglglessink is the way to go.
Regards,
Rahool

mark_pei · March 26, 2015, 6:55am

hi rpaliwal_nvidia,

i want to play videos at particular location with fixed size, and i can do so with nv_omx_hdmi_videosink

code:
gst-lauch-0.10
filesrc location=“filename.mov”
! qtdemux
! queue
! nv_omx_h264dec
! nv_omx_hdmi_videosink overlay-x=280 overlay-y=200 overlay-w=640 overlay-h=480
sync=false

that code is ok with one video window
if i want play many video windows, maybe 4 video windows or 8 video windows
how i want to do it?

thanks
Mark Pei

kulve · March 26, 2015, 9:18am

I think you should be able to run multiple gst-launch-0.10 commands with different overlay coordinates to nv_omx_hdmi_videosink.

EDIT: please don’t discuss about one issue in multiple threads.

mark_pei · March 27, 2015, 7:13am

hi kulve,

i also try it with different overlay coordinates to nv_omx_hdmi_videosink, but it not run

https://devtalk.nvidia.com/default/topic/821241/tk1-gstreamer-h264/#4494837

Topic		Replies	Views
TK1 Gstreamer H264 Jetson TK1	6	3483	April 7, 2015
Failed delayed linking pad video of GstFlvDemux named demux to some pad of GstH264Parse named h264parse0 Jetson Xavier NX gstreamer	11	2317	March 2, 2022
using gstreamer to transcode/downscale 1080i videos on Jetson TX2 Jetson TX2	19	3772	October 18, 2021
VLC not working on L4T R27.0.1? Jetson TX2	18	9400	October 18, 2021
Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1) DeepStream SDK	15	1568	October 12, 2021
Accelerated Video Playback on Jetson TK1 Jetson TK1	7	5408	November 17, 2015
Using gst-launch-1.0 for webcam live streaming between Ubuntu PC (host) and Nvidia Jetson TX2(target) Jetson TX2	10	6414	October 18, 2021
revising streaming with Xavier Jetson AGX Xavier	12	1252	October 18, 2021
ACCELERATED GSTREAMER USER GUIDE RESULTS Jetson TX1	9	4037	October 18, 2021
Problem with C++ gstreamer pipeline Jetson Nano gstreamer	7	3195	September 19, 2021

Video playback: Full-screen mode vs. Windowed mode

Related topics