Gstreamer transcoding performance issue

Yar1k · December 3, 2016, 4:11pm

Hello.

I use gstreamer to test the performance of tegra’s (tx1) video subsystem.

I’ve managed to transcode two FHD H264 streams to H265 simultaneously on stock system image. If I add the third one, the error message appears and fps drops. The message is about the reference frame not decoded from DPB. Tergastats says that the load is below of the half of the maximum by CPU and mem controller. The single stream is processed like this:

udpsrc uri='some multicast group' ! tsdemux ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! fakesink or filesink

By the way omxmpeg2videodec produces errors and the wrong frames on interlaced input (progressive not tested).

I’ve flashed the system image from jetpack 2.3.1 and have installed additional plugins for gstreamer. It can’t even decode single stream without DPB errors and the FPS drops now on fresh system.

What could be the reason of this?
How to fix it?

I’ve also read about new multimedia API but it seems that there is no MPEG2 support.

MPEG2 is not supported now, isn't it?

Thank you.

Yar1k · December 5, 2016, 11:08am

I’ve dumped H264 elementary stream from multicast source to file and have tried to play it with gst-play, gst-launch and the sample application from Multimedia API.

The sample from Multimedia API plays it fine.
Gst-play shows one image per 10 seconds or so.

Gst-launch plays ok with this pipeline:

filesrc location='some_es.h264' ! h264parse ! omxh264dec ! nvvidconv ! videorate ! 'video/x-raw, framerate=50/1' ! nveglglessink

It behaves the same way as gst-play if there is no videorate in pipeline.

The framerate counter shows some weird numbers but there is no message about the reference in DPB was never decoded. Here are the output logs:
gst-launch multicast:

ubuntu@tegra-ubuntu:~/tegra_multimedia_api/samples/00_video_decode$ gst-launch-1.0 udpsrc uri='udp://225.101.15.16:1234' ! tsdemux ! h264parse ! omxh264dec ! nvvidconv ! nveglglessink
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 180000 
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1649: ColorFormat : 5 
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1761: Layout 3: Interlaced Output Resolution : (1920x1088) 
TVMR: InterlaceBufferAlloc !!!!!!!!!!!!!!!!!!!!! 
TVMR: TVMRInit: 100: frameWidth = 1920, frameHeight = 1088 
TVMR: TVMRInit: 114: deinterlaceType = 2
TVMR: DeinterlaceThread: 827: DeinterlaceThread is created 
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 4, InteraceStream = 1, InterlaceEnabled = 1, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 6), ThumbnailMode = 0
TVMR: FrameRate = 50 
TVMR: NVDEC LowCorner Freq = (300000 * 1024) 
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
---> TVMR: Video-conferencing detected !!!!!!!!!
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
...
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
TVMR: FrameRate = 2.837210 
reference in DPB was never decoded
reference in DPB was never decoded
reference in DPB was never decoded
ERROR: from element /GstPipeline:pipeline0/GstEglGlesSink:eglglessink0: Output window was closed
Additional debug info:
/dvs/git/dirty/git-master_linux/external/gstreamer/gst-nveglglessink/ext/eglgles/gsteglglessink.c(820): gst_eglglessink_event_thread (): /GstPipeline:pipeline0/GstEglGlesSink:eglglessink0
Execution ended after 0:06:12.945089854
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6067: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5885: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5917: Closing TVMR Frame Delivery Thread -------------
TVMR: DeinterlaceThread: 903: Closing Deinterlace Thread 
TVMR: NvMMLiteTVMRDecBlockClose: 7740: Done 
Setting pipeline to NULL ...
Freeing pipeline ...

video_decode dumped ES:

ubuntu@tegra-ubuntu:~/tegra_multimedia_api/samples/00_video_decode$ ./video_decode /home/ubuntu/dump.h264 H264
Set governor to performance before enabling profiler
Failed to query video capabilities: Bad address
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
Failed to query video capabilities: Bad address
Starting decoder capture loop thread
TVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 0 
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1649: ColorFormat : 5 
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1790: SurfaceLayout = 3
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 6, InteraceStream = 1, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Video Resolution: 1920x1080
libv4l2_nvvidconv (0):(765) (INFO) : Allocating (10) OUTPUT PLANE BUFFERS Layout=1
libv4l2_nvvidconv (0):(775) (INFO) : Allocating (10) CAPTURE PLANE BUFFERS Layout=0
Query and set capture successful
TVMR: NvMMLiteTVMRDecDoWork: 6466: NVMMLITE_TVMR: EOS detected
Input file read complete
TVMR: FrameRate = 25.000000 
TVMR: FrameRate = 25.000000 
TVMR: FrameRate = 25.000000 
TVMR: FrameRate = 25.000000 
TVMR: FrameRate = 25.000000 
TVMR: FrameRate = 25.000000 
TVMR: FrameRate = 25.000000 
TVMR: FrameRate = 25.000000 
TVMR: TVMRBufferProcessing: 5444: Processing of EOS 
TVMR: TVMRBufferProcessing: 5519: Processing of EOS Done
Exiting decoder capture loop thread
TVMR: TVMRFrameStatusReporting: 6067: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5885: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5917: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7740: Done 
App run was successful

gst-play dumped ES:

ubuntu@tegra-ubuntu:~/tegra_multimedia_api/samples/00_video_decode$ gst-play-1.0 /home/ubuntu/dump.h264 
Press 'k' to see a list of keyboard shortcuts.
Now playing /home/ubuntu/dump.h264
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingNvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 180000 
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1649: ColorFormat : 5 
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1761: Layout 3: Interlaced Output Resolution : (1920x1088) 
TVMR: InterlaceBufferAlloc !!!!!!!!!!!!!!!!!!!!! 
TVMR: TVMRInit: 100: frameWidth = 1920, frameHeight = 1088 
TVMR: TVMRInit: 114: deinterlaceType = 2
TVMR: DeinterlaceThread: 827: DeinterlaceThread is created 
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 4, InteraceStream = 1, InterlaceEnabled = 1, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 6), ThumbnailMode = 0
TVMR: FrameRate = 150.001500 
TVMR: FrameRate = -428.577551 
TVMR: FrameRate = 750.018750 
TVMR: FrameRate = 25        
TVMR: NVDEC LowCorner Freq = (150000 * 1024) 
TVMR: FrameRate = 250.000000 
TVMR: FrameRate = 375.009375 
TVMR: FrameRate = 300.003000 
TVMR: FrameRate = -200.000000 
0:00:01.0 / 0:01:20.5

gst-launch dumped ES:

ubuntu@tegra-ubuntu:~/tegra_multimedia_api/samples/00_video_decode$ gst-launch-1.0 filesrc location='/home/ubuntu/dump.h264' ! h264parse ! omxh264dec ! nvvidconv ! videorate ! 'video/x-raw, framerate=50/1' ! nveglglessink
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingPipeline is PREROLLING ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 180000 
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1649: ColorFormat : 5 
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1761: Layout 3: Interlaced Output Resolution : (1920x1088) 
TVMR: InterlaceBufferAlloc !!!!!!!!!!!!!!!!!!!!! 
TVMR: TVMRInit: 100: frameWidth = 1920, frameHeight = 1088 
TVMR: TVMRInit: 114: deinterlaceType = 2
TVMR: DeinterlaceThread: 827: DeinterlaceThread is created 
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 4, InteraceStream = 1, InterlaceEnabled = 1, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 6), ThumbnailMode = 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
TVMR: FrameRate = 150.001500 
TVMR: FrameRate = -428.577551 
TVMR: FrameRate = 750.018750 
TVMR: FrameRate = 25 
TVMR: NVDEC LowCorner Freq = (150000 * 1024) 
TVMR: FrameRate = 250.000000 
TVMR: FrameRate = 375.009375 
TVMR: FrameRate = 300.003000 
TVMR: FrameRate = -200.000000 
TVMR: FrameRate = 130.435917 
TVMR: NvMMLiteTVMRDecDoWork: 6466: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5444: Processing of EOS 
TVMR: TVMRBufferProcessing: 5519: Processing of EOS Done
Got EOS from element "pipeline0".
Execution ended after 0:00:37.999986604
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6067: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5885: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5917: Closing TVMR Frame Delivery Thread -------------
TVMR: DeinterlaceThread: 903: Closing Deinterlace Thread 
TVMR: NvMMLiteTVMRDecBlockClose: 7740: Done 
Setting pipeline to NULL ...
Freeing pipeline ...

DaneLLL · December 6, 2016, 2:00am

Hi Yar1k,
As omxh264dec and omxh265enc are HW engines, you can check the freq of NVDEC and MSENC in tegrastats. Three transcoding streams can be too heavy for the engines.

For #2, could you describe the steps and the issue in detail one more time? Not clearly understand what the issue is. If it is possible, please share some_es.h264 also.

For omxmpeg2videodec, please also share your test stream if possible. I think we don’t support interlaced input but would like to double confirm.

And MPEG2 is not supported in MM API.

Yar1k · December 6, 2016, 1:07pm

I suppose TX1 can transcode even 4 streams. Data sheet says about decoding 240fps 1080p H264 and encoding 120fps 1080p H265. Decoder and Encoder are separate modules on diagram. Maybe ISP or VIC is the issue.

I have flashed the fresh ubuntu image(16.04) on target board during Jetpack installation process, have installed gstreamer plugins via apt and have tried to launch the same transcoding pipeline. The result is that the single stream transcoding pipeline drops frames and produces “DPB not decoed” errors just like three transcoding streams on stock ubuntu(14.04) image. So I have tried to just decode and display the stream which I wanted to transcode before. The result is in output log of gst-launch: fps=2 and “reference in DPB was never decoded”. I’ve decided to check that HW decoder is working OK by dumping ES from TS and playing it both on GST-OMX and MM API. It seems that the problems is in gst or omx.

MPEG2 decoder produces correct frames on fresh ubuntu now but the playback speed is floating. I guess it’s a gstreamer issue because totem player which uses SW decoder and Gst framework have this problem too.

GStreamer 1.8.2

The samples are here:

DaneLLL · December 7, 2016, 7:05am

Hi Yar1k,
I tried to run 4 transcodings:

ubuntu@tegra-ubuntu:~$ gst-launch-1.0 filesrc location=some_es.h264 ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! 'video/x-h265,stream-format=(string)byte-stream' ! filesink location=a.h265 & gst-launch-1.0 filesrc location=some_es1.h264 ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! 'video/x-h265,stream-format=(string)byte-stream' ! filesink location=a1.h265 & gst-launch-1.0 filesrc location=some_es2.h264 ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! 'video/x-h265,stream-format=(string)byte-stream' ! filesink location=a2.h265 & gst-launch-1.0 filesrc location=some_es3.h264 ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! 'video/x-h265,stream-format=(string)byte-stream' ! filesink location=a3.h265
[1] 5218
[2] 5219
[3] 5220
Setting pipeline to PAUSED ...
Setting pipeline to PAUSED ...
Setting pipeline to PAUSED ...
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitInside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingPipeline is PREROLLING ...
Pipeline is PREROLLING ...
Pipeline is PREROLLING ...
NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen
NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
TVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 180000
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingNvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080)
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080)
TVMR: cbBeginSequence: 1649: ColorFormat : 5
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
Pipeline is PREROLLING ...
TVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 180000
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0
NvMMLiteBlockCreate : Block : BlockType = 261
TVMR: cbBeginSequence: 1761: Layout 3: Interlaced Output Resolution : (1920x1088)
TVMR: InterlaceBufferAlloc !!!!!!!!!!!!!!!!!!!!!
TVMR: TVMRInit: 100: frameWidth = 1920, frameHeight = 1088
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080)
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080)
TVMR: cbBeginSequence: 1649: ColorFormat : 5
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1761: Layout 3: Interlaced Output Resolution : (1920x1088)
TVMR: InterlaceBufferAlloc !!!!!!!!!!!!!!!!!!!!!
TVMR: TVMRInit: 100: frameWidth = 1920, frameHeight = 1088
TVMR: TVMRInit: 114: deinterlaceType = 2
TVMR: DeinterlaceThread: 827: DeinterlaceThread is created
TVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 180000
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 4, InteraceStream = 1, InterlaceEnabled = 1, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 6), ThumbnailMode = 0
TVMR: TVMRInit: 114: deinterlaceType = 2
TVMR: DeinterlaceThread: 827: DeinterlaceThread is created
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080)
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080)
TVMR: cbBeginSequence: 1649: ColorFormat : 5
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 4, InteraceStream = 1, InterlaceEnabled = 1, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 6), ThumbnailMode = 0
TVMR: cbBeginSequence: 1761: Layout 3: Interlaced Output Resolution : (1920x1088)
TVMR: InterlaceBufferAlloc !!!!!!!!!!!!!!!!!!!!!
TVMR: TVMRInit: 100: frameWidth = 1920, frameHeight = 1088
TVMR: TVMRInit: 114: deinterlaceType = 2
TVMR: DeinterlaceThread: 827: DeinterlaceThread is created
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 4, InteraceStream = 1, InterlaceEnabled = 1, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 6), ThumbnailMode = 0
NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7580: NvMMLiteBlockOpen
NvMMLiteBlockCreate : Block : BlockType = 261
Framerate set to : 25 at NvxVideoEncoderSetParameterTVMR: cbBeginSequence: 1166: BeginSequence  1920x1088, bVPR = 0, fFrameRate = 25.000000
TVMR: LowCorner Frequency = 180000
TVMR: cbBeginSequence: 1545: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0
NvMMLiteOpen : Block : BlockType = 8
Framerate set to : 25 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8
TVMR: cbBeginSequence: 1606: Display Resolution : (1920x1080)
TVMR: cbBeginSequence: 1607: Display Aspect Ratio : (1920x1080)
TVMR: cbBeginSequence: 1649: ColorFormat : 5
TVMR: cbBeginSequence:1660 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1761: Layout 3: Interlaced Output Resolution : (1920x1088)
TVMR: InterlaceBufferAlloc !!!!!!!!!!!!!!!!!!!!!
TVMR: TVMRInit: 100: frameWidth = 1920, frameHeight = 1088
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
TVMR: TVMRInit: 114: deinterlaceType = 2
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
TVMR: cbBeginSequence: 1868: NumOfSurfaces = 4, InteraceStream = 1, InterlaceEnabled = 1, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
Allocating new output: 1920x1088 (x 6), ThumbnailMode = 0
TVMR: DeinterlaceThread: 827: DeinterlaceThread is created
Framerate set to : 25 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
Framerate set to : 25 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
lv 20, tier 1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
lv 20, tier 1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
lv 20, tier 1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
lv 20, tier 1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock


TVMR: FrameRate = 150.001500
TVMR: FrameRate = 150.001500
TVMR: FrameRate = 150.001500
TVMR: FrameRate = 150.001500
TVMR: FrameRate = -428.577551
TVMR: FrameRate = -428.577551
TVMR: FrameRate = -428.577551
TVMR: FrameRate = -428.577551
TVMR: FrameRate = 750.018750
TVMR: FrameRate = 750.018750
TVMR: FrameRate = 25
TVMR: NVDEC LowCorner Freq = (150000 * 1024)
TVMR: FrameRate = 750.018750
TVMR: FrameRate = 25
TVMR: NVDEC LowCorner Freq = (150000 * 1024)
TVMR: FrameRate = 750.018750
TVMR: FrameRate = 250.000000
TVMR: FrameRate = 25
TVMR: NVDEC LowCorner Freq = (150000 * 1024)
TVMR: FrameRate = 25
TVMR: NVDEC LowCorner Freq = (150000 * 1024)
TVMR: FrameRate = 250.000000
TVMR: FrameRate = 250.000000
TVMR: FrameRate = 250.000000
TVMR: FrameRate = 375.009375
TVMR: FrameRate = 375.009375
TVMR: FrameRate = 375.009375
TVMR: FrameRate = 375.009375
TVMR: FrameRate = 300.003000
TVMR: FrameRate = 300.003000
TVMR: FrameRate = 300.003000
TVMR: FrameRate = 300.003000
TVMR: FrameRate = -200.000000
TVMR: FrameRate = -200.000000
TVMR: FrameRate = -200.000000
TVMR: FrameRate = -200.000000
TVMR: FrameRate = 130.435917
TVMR: NvMMLiteTVMRDecDoWork: 6466: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5444: Processing of EOS
TVMR: TVMRBufferProcessing: 5519: Processing of EOS Done
Got EOS from element "pipeline0".
Execution ended after 0:00:27.650336088
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6067: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5885: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5917: Closing TVMR Frame Delivery Thread -------------
TVMR: DeinterlaceThread: 903: Closing Deinterlace Thread
TVMR: NvMMLiteTVMRDecBlockClose: 7740: Done
Setting pipeline to NULL ...
Freeing pipeline ...
ubuntu@tegra-ubuntu:~$
ubuntu@tegra-ubuntu:~$
ubuntu@tegra-ubuntu:~$ TVMR: FrameRate = 130.435917
TVMR: NvMMLiteTVMRDecDoWork: 6466: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5444: Processing of EOS
TVMR: TVMRBufferProcessing: 5519: Processing of EOS Done
Got EOS from element "pipeline0".
Execution ended after 0:00:28.575044439
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6067: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5885: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5917: Closing TVMR Frame Delivery Thread -------------
TVMR: DeinterlaceThread: 903: Closing Deinterlace Thread
TVMR: NvMMLiteTVMRDecBlockClose: 7740: Done
Setting pipeline to NULL ...
Freeing pipeline ...
TVMR: NvMMLiteTVMRDecDoWork: 6466: NVMMLITE_TVMR: EOS detected
TVMR: FrameRate = 130.435917
TVMR: TVMRBufferProcessing: 5444: Processing of EOS
TVMR: TVMRBufferProcessing: 5519: Processing of EOS Done
Got EOS from element "pipeline0".
Execution ended after 0:00:28.823468480
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6067: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5885: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5917: Closing TVMR Frame Delivery Thread -------------
TVMR: DeinterlaceThread: 903: Closing Deinterlace Thread
TVMR: NvMMLiteTVMRDecBlockClose: 7740: Done
Setting pipeline to NULL ...
Freeing pipeline ...
TVMR: NvMMLiteTVMRDecDoWork: 6466: NVMMLITE_TVMR: EOS detected
TVMR: FrameRate = 130.435917
TVMR: TVMRBufferProcessing: 5444: Processing of EOS
TVMR: TVMRBufferProcessing: 5519: Processing of EOS Done
Got EOS from element "pipeline0".
Execution ended after 0:00:29.008143820
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 6067: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5885: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5917: Closing TVMR Frame Delivery Thread -------------
TVMR: DeinterlaceThread: 903: Closing Deinterlace Thread
TVMR: NvMMLiteTVMRDecBlockClose: 7740: Done
Setting pipeline to NULL ...
Freeing pipeline ...

[1]   Done                    gst-launch-1.0 filesrc location=some_es.h264 ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! 'video/x-h265,stream-format=(string)byte-stream' ! filesink location=a.h265
[2]-  Done                    gst-launch-1.0 filesrc location=some_es1.h264 ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! 'video/x-h265,stream-format=(string)byte-stream' ! filesink location=a1.h265
[3]+  Done                    gst-launch-1.0 filesrc location=some_es2.h264 ! h264parse ! omxh264dec ! nvvidconv ! omxh265enc ! 'video/x-h265,stream-format=(string)byte-stream' ! filesink location=a2.h265
ubuntu@tegra-ubuntu:~$

some_es.h264 is a 32-second content, so the performance seems good enough:
Execution ended after 0:00:27.650336088
Execution ended after 0:00:28.575044439
Execution ended after 0:00:28.823468480
Execution ended after 0:00:29.008143820

I think the error you see is probably because of the network bandwidth. I check some_es.h264 via JM decoder:

00040( I | P )   81   471    18                             4:2:0      42
00038( B | B )   77   472    22                             4:2:0      48
00037( B | B )   75   473    22                             4:2:0      50
00039( B | B )   79   474    22                             4:2:0      60
00044( P | P )   89   475    18                             4:2:0      49
00042( B | B )   85   476    22                             4:2:0      44
00041( B | B )   83   477    22                             4:2:0      47
00043( B | B )   87   478    22                             4:2:0      47
00048( P | P )   97   479    18                             4:2:0      54

There are three B frames between I and P / P and P frames. If one of the I/P frame is incomplete and not decoded successfully, the three B frames cannot be decoded as well. Since 4 transcodings with filesrc work fine, the issue should be from the udpsrc. Possibly the network bandwidth is limited and some packets are dropped.

Yar1k · December 7, 2016, 9:19am

Hello DaneLLL,
It’s nice to see this.
What’s about MPEG2? Is it played fine on your side?
Maybe I should open another thread, but are there any plans about MPEG2 support in MM API?
Thank you.

DaneLLL · December 8, 2016, 2:57am

Hi Yar1k,
On my side, it is played fine by running
$ gst-launch-1.0 filesrc location=some_es.mpv ! mpegvideoparse ! omxmpeg2videodec ! nvoverlaysink

Will update you about supporting MPEG2 decoding in MM API.

DaneLLL · December 8, 2016, 8:50am

Hi Yar1k,
Confirm that we don’t have plan to support MPEG2 decoding in MM API.

Yar1k · December 8, 2016, 3:36pm

Hello DaneLLL,
This is sad. Thank you for your support.

Topic		Replies	Views
Gstreamer decode live video stream with latency Jetson TX2	6	4120	October 18, 2021
do deinterlace with TX1 Jetson TX1	19	2280	October 18, 2021
Gstreamer v4l2src 4Kp60 source encoding & preview problem -> Framerate is poor. Jetson TX2	2	1667	October 18, 2021
Problem using gstreamer with opencv and cuda on TX2 with Jetpack 4.3 Jetson TX2 rtsp , opencv , cuda , gstreamer	21	3301	October 18, 2021
Why does omxh264enc to v4l2sink fail? Jetson TX2	6	2010	October 18, 2021
using gstreamer to transcode/downscale 1080i videos on Jetson TX2 Jetson TX2	19	3779	October 18, 2021
H.264 decoding failure Jetson Nano gstreamer	12	3692	October 18, 2021
TX2 H264 RTSP Stream decoding issues Jetson TX2	27	11557	October 18, 2021
the cpu usage cannot down (use cuda decode) Jetson TX1	29	11487	October 18, 2021
Gstreamer issue in NX: Nano is faster than NX! Jetson Xavier NX gstreamer	16	1981	October 18, 2021

Gstreamer transcoding performance issue

Related topics