GStreamer nvoverlaysink not accepting UHD video

Maxbns · September 14, 2017, 12:41pm

It seems that the GStreamer plugin nvoverlaysink is not accepting videos in the 3840x2160 (UHD) resolution.

For testing I have 2 videos:
1800p_h265.mp4: 1080p (as the name says)
2160p60_h265.mp4: 2160p UHD
Both at ~60fps, converted with libx265 (current package version in Ubuntu 16.04).

I’m using gstreamer 1.12.0 built with instructions from the NVidia Accelerated GStreamer User Guide r28.1.

First test (working):

Playing 1080p video with nvoverlaysink

./gst-launch-1.0 filesrc=../../../1080p_h265.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h265parse ! omxh265dec ! nvoverlaysink

Second test (failing):

Playing 2160p60 video with nvoverlaysink

./gst-launch-1.0 filesrc=../../../2160p60_h265.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h265parse ! omxh265dec ! nvoverlaysink

Third test (working):

Playing 2160p60 video with nvglglessink

./gst-launch-1.0 filesrc location=../../../2160p60_h265.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h265parse ! omxh265dec ! nvglglessink

This last test works, proving that decode is not an issue, but it doesn’t override other windows (so Gnome left bar and menus are still visible, making it impossible to scale the video perfectly to the screen).

Logs:

Log for working 1080p test (first):

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
NvMMLiteOpen : Block : BlockType = 279 
TVMR: NvMMLiteTVMRDecBlockOpen: 7907: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 279 
TVMR: cbBeginSequence: 1223: BeginSequence  1920x1088, bVPR = 0
TVMR: LowCorner Frequency = 180000 
TVMR: cbBeginSequence: 1622: DecodeBuffers = 7, pnvsi->eCodec = 10, codec = 9 
TVMR: cbBeginSequence: 1693: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 1694: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1762: ColorFormat : 5 
TVMR: cbBeginSequence:1776 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1904: SurfaceLayout = 3
TVMR: cbBeginSequence: 2005: NumOfSurfaces = 14, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 2007: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 1920x1088 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 1920, nFrameHeight = 1088 
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
TVMR: FrameRate = 60 
TVMR: NVDEC LowCorner Freq = (360000 * 1024) 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: FrameRate = 60.000240 
TVMR: NvMMLiteTVMRDecDoWork: 6768: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5723: Processing of EOS 
TVMR: FrameRate = 60.000240 
TVMR: TVMRBufferProcessing: 5800: Processing of EOS Done
Got EOS from element "pipeline0".
Execution ended after 0:00:30.033531417
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6369: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 6179: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 6219: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 8105: Done 
Setting pipeline to NULL ...
Freeing pipeline ...

Log for failing 2160p test (second):

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
NvMMLiteOpen : Block : BlockType = 279 
TVMR: NvMMLiteTVMRDecBlockOpen: 7907: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 279 
TVMR: cbBeginSequence: 1223: BeginSequence  3840x2160, bVPR = 0
TVMR: LowCorner Frequency = 345000 
TVMR: cbBeginSequence: 1622: DecodeBuffers = 7, pnvsi->eCodec = 10, codec = 9 
TVMR: cbBeginSequence: 1693: Display Resolution : (3840x2160) 
TVMR: cbBeginSequence: 1694: Display Aspect Ratio : (3840x2160) 
TVMR: cbBeginSequence: 1762: ColorFormat : 5 
TVMR: cbBeginSequence:1776 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1904: SurfaceLayout = 3
TVMR: cbBeginSequence: 2005: NumOfSurfaces = 14, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 16 LumaBitDepth = 10, ChromaBitDepth = 10, ChromaFormat = 5
TVMR: cbBeginSequence: 2007: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 3840x2160 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 3840, nFrameHeight = 2160 
ERROR: from element /GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0: Internal data stream error.
Additional debug info:
/dvs/git/dirty/git-master_linux/external/gstreamer/gst-omx/omx/gstomxvideodec.c(2877): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0:
stream stopped, reason not-negotiated
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
TVMR: TVMRFrameStatusReporting: 6369: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 6179: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 6219: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 8105: Done 
Freeing pipeline ...

Log for working 2160p nvglglessink test (third):

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
NvMMLiteOpen : Block : BlockType = 279 
TVMR: NvMMLiteTVMRDecBlockOpen: 7907: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 279 
TVMR: cbBeginSequence: 1223: BeginSequence  3840x2160, bVPR = 0
TVMR: LowCorner Frequency = 345000 
TVMR: cbBeginSequence: 1622: DecodeBuffers = 7, pnvsi->eCodec = 10, codec = 9 
TVMR: cbBeginSequence: 1693: Display Resolution : (3840x2160) 
TVMR: cbBeginSequence: 1694: Display Aspect Ratio : (3840x2160) 
TVMR: cbBeginSequence: 1762: ColorFormat : 5 
TVMR: cbBeginSequence:1776 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1904: SurfaceLayout = 3
TVMR: cbBeginSequence: 2005: NumOfSurfaces = 14, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 16 LumaBitDepth = 10, ChromaBitDepth = 10, ChromaFormat = 5
TVMR: cbBeginSequence: 2007: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 3840x2160 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 3840, nFrameHeight = 2160 
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
TVMR: FrameRate = 59 
TVMR: NVDEC LowCorner Freq = (576000 * 1024) 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: FrameRate = 59.940180 
TVMR: NvMMLiteTVMRDecDoWork: 6768: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5723: Processing of EOS 
TVMR: TVMRBufferProcessing: 5800: Processing of EOS Done
Got EOS from element "pipeline0".
Execution ended after 0:02:38.598207313
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6369: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 6179: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 6219: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 8105: Done 
Setting pipeline to NULL ...
Freeing pipeline ...

Maxbns · September 14, 2017, 12:57pm

Enabling additional logs, it seems that the 10-bit color depth is the issue (Unsupported format I420_10LE, which prevents pad creation between the decoder and final sink).

I’m trying again with 8-bit 2160p and updating ASAP.

DaneLLL · September 15, 2017, 1:47am

Hi Maxbns, does your output device have 4K support? A 4K TV connecting to TX2 via HDMI?

Maxbns · September 30, 2017, 4:58pm

It’s working with a video encoded with 8-bit color depth.

However, I noticed that using nvglglessink element renders much deeper blacks for the same video.

For that reason, I switched entirely to nvglglessink, which I patched to always turn full screen (no border - no system menu or anything). The result is much better. I can provide links to achieve that if needed.

Anyone has an idea why this happens? Does the X server or other software component modifies the video to adapt it to the monitor or something? (it must be something that is not done by nvoverlaysink which maps directly to HDMI output…)

carolyuu · October 2, 2017, 3:28am

Hi Maxbns,

We can play 4K video via 4K TV with nvoverlaysink command.
The 4K video need to play through 4K TV.
You can check your TV resolution with below command:

cat /sys/class/graphics/fb0/mode

It should show “U:3840x2160p-60”

Topic		Replies	Views
Gstreamer v4l2src 4Kp60 source encoding & preview problem -> Framerate is poor. Jetson TX2	2	1667	October 18, 2021
Gstreamer transcoding performance issue Jetson TX1	9	4919	October 18, 2021
Video scaling with GStreamer element 'videoscale' and 'nvdrmvideosink set-mode=1' Jetson TX2 gstreamer , video	18	8623	March 5, 2022
Playback error Jetson TX2 gstreamer	10	878	October 18, 2021
do deinterlace with TX1 Jetson TX1	19	2282	October 18, 2021
GstQTDemux:qtdemux0: streaming stopped, reason not-negotiated (-4) DeepStream SDK gstreamer , nvbugs	8	6066	March 12, 2021
[TX1] 4k60 video EGLImage readback stuttering Jetson TX1	2	544	February 14, 2018
Gstreamer and v4l2loopback Jetson Nano gstreamer	12	5709	October 15, 2021
Wonky coupling of GStreamer 1.0 NVMM plugins Jetson TK1	1	629	June 29, 2017
Why does omxh264enc to v4l2sink fail? Jetson TX2	6	2011	October 18, 2021

GStreamer nvoverlaysink not accepting UHD video

Related topics