omxh264dec nvvidconv performance problem.

Hi there,

I tested the following Gstreamer pipeline to handle 4k H.264 video stream, it is not so bad to use autovideosink as receiver, the video data’s bitrate is around 5000Kbps.

gst-launch-1.0 -v udpsrc ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264delay ! h264parse ! omxh264dec ! autovideosink false=true

and it is just a test pipeline, since I am using openGL to handle video data, and the openGL texture format is in “RGBA” format right now, so “nvvidcov”(NV12->RGBA) is used after “omxh264dec” to push the final raw video data using “appsink”, we finally dump video buffer from “appsink” to our custom openGL’s texture, we test this method using “filesrc” to read a local file, and it seems not bad, the file data is at 4096x1716@24fps.

gst-launch-1.0 -v filesrc location=*.mp4 ! qtdemux ! queue ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=(string)RGBA" ! appsink

the problem is when I move the source of above pipeline to “udpsrc” to handle “3840x2160@30fps”, the omxh264dec’s decoding frame rate drop to around 17fps, I am sure that “omxh264dec” can handle 4k@30 from my first test pipeline, and “nvvidconv” can also handle “4096x1716@24fps” without any problem from my second pipeline, do any one have the same problem or any good solution to my problem, thanks in advance, I also post my final pipeline with debug information below.

gst-launch-1.0 -v  udpsrc ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264delay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=(string)RGBA" ! appsink
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingNvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 6736: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteTVMRDecSetAttribute:: Error status reporting set to 1
TVMR: cbBeginSequence: 803: BeginSequence  3840x2160, bVPR = 0
TVMR: cbBeginSequence: 1147: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1206: Display Resolution : (3840x2160) 
TVMR: cbBeginSequence: 1207: Display Aspect Ratio : (3840x2160) 
TVMR: cbBeginSequence: 1247: ColorFormat : 5 
TVMR: cbBeginSequence:1261 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1429: SurfaceLayout = 3
TVMR: cbBeginSequence: 1474: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8
Allocating new output: 3840x2160 (x 12), ThumbnailMode = 0
TVMR: FrameRate = 1890 
TVMR: NVDEC LowCorner Freq = (576000 * 1024) 
---> TVMR: Video-conferencing detected !!!!!!!!!
TVMR: FrameRate = 18.425402 
TVMR: FrameRate = 17.721981 
TVMR: FrameRate = 17.178087 
TVMR: FrameRate = 17.182632 
TVMR: FrameRate = 19.416684 
TVMR: FrameRate = 17.247594

Hello, usaarizona:
for the pipeline:

gst-launch-1.0 -v  udpsrc ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264delay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=(string)RGBA" ! appsink

The low frame-rate may come from many reasons, like network, appsink, etc.

would you please replace the appsink by ‘filesink location=/dev/null’
and then check the framerate?

I tested in my side with similar pipeline (source comes from on-board camera, and streamed by 1080p30fps H264), the frame-rate is about 30. Paste log for reference.

gst-launch-1.0 -e udpsrc address=x.x.x.x port=5004   caps='application/x-rtp, media=(string)video, encoding-name=(string)H264, payload=(int)96'   ! rtph264depay ! queue ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=(string)RGBA" ! filesink location=/dev/null -e
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7382: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1046: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 0.000000
TVMR: LowCorner Frequency = 180000 
TVMR: cbBeginSequence: 1421: DecodeBuffers = 14, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1480: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 1481: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1521: ColorFormat : 5 
TVMR: cbBeginSequence:1532 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1659: SurfaceLayout = 3
TVMR: cbBeginSequence: 1735: NumOfSurfaces = 18, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 20), ThumbnailMode = 0
TVMR: FrameRate = 44 
TVMR: NVDEC LowCorner Freq = (264000 * 1024) 
---> TVMR: Video-conferencing detected !!!!!!!!!
TVMR: FrameRate = 30.036164 
TVMR: FrameRate = 30.007232 
TVMR: FrameRate = 29.998860 
TVMR: FrameRate = 29.724219 
TVMR: FrameRate = 29.801640 
^Chandling interrupt.
Interrupt: Stopping pipeline ...
EOS on shutdown enabled -- Forcing EOS on the pipeline
Waiting for EOS...
TVMR: NvMMLiteTVMRDecDoWork: 6283: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5275: Processing of EOS 
TVMR: TVMRBufferProcessing: 5350: Processing of EOS Done
Got EOS from element "pipeline0".
EOS received - stopping pipeline...
Execution ended after 0:00:22.845192836
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 5882: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5700: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5732: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7542: Done 
Setting pipeline to NULL ...
Freeing pipeline ...

br
ChenJian