VP8 to H264 transcoding problem

IvanGolob · June 11, 2019, 11:00am

Hi,

I have to transcode a RTP VP8 encoded stream to a H264 stream. With this pipeline I have a “Floating point exception”:

gst-launch-1.0 udpsrc address=127.0.0.1 port=6789 ! application/x-rtp,encoding-name=VP8 ! rtpvp8depay ! queue ! omxvp8dec ! nvvidconv ! omxh264enc ! fakesink async=0 sync=0

Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 278 
TVMR: NvMMLiteTVMRDecBlockOpen: 5107: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 278 
TVMR: cbBeginSequence: 584: BeginSequence  1920x1088, bVPR = 0
TVMR: cbBeginSequence: 826: DecodeBuffers = 4 
TVMR: cbBeginSequence: 850: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 851: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1015: SurfaceLayout = 3
TVMR: cbBeginSequence: 1045: NumOfSurfaces = 8, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1 
Allocating new output: 1920x1088 (x 10), ThumbnailMode = 0
Framerate set to : 0 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
Floating point exception

Note that the vp8 decoding works, with this pipeline I can see the stream:

gst-launch-1.0 udpsrc address=127.0.0.1 port=6789 ! application/x-rtp,encoding-name=VP8 ! rtpvp8depay ! omxvp8dec ! nveglglessink async=0 sync=0

Can you help me?

Thank you, best regards.

Ivan

DaneLLL · June 12, 2019, 2:54am

Hi,
Please utilize videoparse plugin:
[url]https://devtalk.nvidia.com/default/topic/1045295/jetson-tk1/omxh264enc/post/5304172/#5304172[/url]

IvanGolob · June 12, 2019, 9:24am

Hi DaneLLL,

thank you, but I still have problems:

if I put the plugin just after the omxvp8dec I have the same error:

gst-launch-1.0 udpsrc address=127.0.0.1 port=6789 ! application/x-rtp,encoding-name=VP8,clock-rate=90000,payload=126 ! rtpvp8depay ! queue ! omxvp8dec ! videoparse format=2 ! nvvidconv ! omxh264enc ! fakesink async=0 sync=0
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 278 
TVMR: NvMMLiteTVMRDecBlockOpen: 5107: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 278 
TVMR: cbBeginSequence: 584: BeginSequence  1920x1088, bVPR = 0
TVMR: cbBeginSequence: 826: DecodeBuffers = 4 
TVMR: cbBeginSequence: 850: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 851: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1015: SurfaceLayout = 3
TVMR: cbBeginSequence: 1045: NumOfSurfaces = 8, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1 
Allocating new output: 1920x1088 (x 10), ThumbnailMode = 0
Framerate set to : 0 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
Floating point exception

If I put it just after nvvidconv I still have errors:

gst-launch-1.0 udpsrc address=127.0.0.1 port=6789 ! application/x-rtp,encoding-name=VP8,clock-rate=90000,payload=126 ! rtpvp8depay ! queue ! omxvp8dec ! nvvidconv ! videoparse format=2 ! omxh264enc ! fakesink async=0 sync=0
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 278 
TVMR: NvMMLiteTVMRDecBlockOpen: 5107: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 278 
TVMR: cbBeginSequence: 584: BeginSequence  1920x1088, bVPR = 0
TVMR: cbBeginSequence: 826: DecodeBuffers = 4 
TVMR: cbBeginSequence: 850: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 851: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1015: SurfaceLayout = 3
TVMR: cbBeginSequence: 1045: NumOfSurfaces = 8, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1 
Allocating new output: 1920x1088 (x 10), ThumbnailMode = 0
ERROR: from element /GstPipeline:pipeline0/GstOMXVP8Dec-omxvp8dec:omxvp8dec-omxvp8dec0: Internal data stream error.
Additional debug info:
/dvs/git/dirty/git-master_linux/external/gstreamer/gst-omx/omx/gstomxvideodec.c(2779): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXVP8Dec-omxvp8dec:omxvp8dec-omxvp8dec0:
stream stopped, reason not-negotiated
Execution ended after 0:00:01.098222405
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 3883: Closing TVMR Frame Status Thread -------------
TVMR: TVMRFrameDelivery: 3739: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMDecTVMRDestroyParser: 4290: NvAvpClose
TVMR: NvMMLiteTVMRDecBlockClose: 5263: Done 
Setting pipeline to NULL ...
Freeing pipeline ...

The same happens also if I replace “videoparse format=2” with “videoconvert ! video/x-raw,format=I420 ! videoparse format=2”.

I which way I have to use this plugin?

Thank you, best regards.

Ivan

DaneLLL · June 12, 2019, 9:38am

Hi,
Please share your pipeline of running udp server. Should be similar to
[url]https://devtalk.nvidia.com/default/topic/1027423/jetson-tx2/gstreamer-issue-on-tx2/post/5225972/#5225972[/url]

Also suggest you consider Jetson Nano. It has latest r32.1 release and competitive price.

IvanGolob · June 12, 2019, 9:56am

Hi,

I can reproduce the error also with this udp server pipeline:

gst-launch-1.0 videotestsrc ! capsfilter caps=video/x-raw,width=1920,height=1080,framerate=30/1 ! nvvidconv ! omxvp8enc ! rtpvp8pay ! udpsink host=127.0.0.1 port=5000 &

I can see the stream with:

gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! application/x-rtp,encoding-name=VP8 ! rtpvp8depay ! omxvp8dec ! nveglglessink async=0 sync=0

But I have the “Floating point exception” with the transcode pipeline:

gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! application/x-rtp,encoding-name=VP8 ! rtpvp8depay ! queue ! omxvp8dec ! videoparse format=2 ! nvvidconv ! omxh264enc ! fakesink async=0 sync=0
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 278 
TVMR: NvMMLiteTVMRDecBlockOpen: 5107: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 278 
TVMR: cbBeginSequence: 584: BeginSequence  1920x1088, bVPR = 0
TVMR: cbBeginSequence: 826: DecodeBuffers = 4 
TVMR: cbBeginSequence: 850: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 851: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1015: SurfaceLayout = 3
TVMR: cbBeginSequence: 1045: NumOfSurfaces = 8, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1 
Allocating new output: 1920x1088 (x 10), ThumbnailMode = 0
Framerate set to : 0 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
Floating point exception

gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! application/x-rtp,encoding-name=VP8 ! rtpvp8depay ! queue ! omxvp8dec ! nvvidconv ! videoparse format=2 ! omxh264enc ! fakesink async=0 sync=0
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 278 
TVMR: NvMMLiteTVMRDecBlockOpen: 5107: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 278 
TVMR: cbBeginSequence: 584: BeginSequence  1920x1088, bVPR = 0
TVMR: cbBeginSequence: 826: DecodeBuffers = 4 
TVMR: cbBeginSequence: 850: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 851: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1015: SurfaceLayout = 3
TVMR: cbBeginSequence: 1045: NumOfSurfaces = 8, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1 
Allocating new output: 1920x1088 (x 10), ThumbnailMode = 0
ERROR: from element /GstPipeline:pipeline0/GstOMXVP8Dec-omxvp8dec:omxvp8dec-omxvp8dec0: Internal data stream error.
Additional debug info:
/dvs/git/dirty/git-master_linux/external/gstreamer/gst-omx/omx/gstomxvideodec.c(2779): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXVP8Dec-omxvp8dec:omxvp8dec-omxvp8dec0:
stream stopped, reason not-negotiated
Execution ended after 0:00:00.080877802
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 3883: Closing TVMR Frame Status Thread -------------
TVMR: TVMRFrameDelivery: 3739: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMDecTVMRDestroyParser: 4290: NvAvpClose
TVMR: NvMMLiteTVMRDecBlockClose: 5263: Done 
Setting pipeline to NULL ...
Freeing pipeline ...

Thank you, best regards.

Ivan

DaneLLL · June 13, 2019, 2:08am

Hi,
Somehow I cannot run your decoding pipeline on r21.5. I am able to run

$ gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! application/x-rtp ! rtpvp8depay ! vp8dec ! videoparse format=2 ! omxh264enc ! fakesink

FYI.

IvanGolob · June 13, 2019, 6:51am

Hi,
I am using gstreamer 1.8 and not the r21.5 one version, so maybe this is the reason why you can’t run the decoding pipeline on “original” r21.5 (maybe due to the rtpvp8depay plugin version).

Anyway, will I have a hardware-accelerated transcoding with “vp8dec”? In the “Linux Driver Package Multimedia User Guide” it is suggested to use “omxvp8dec” with “nvvidconv” and “omxh264enc”.

Thank you, best regards.

Ivan

DaneLLL · June 13, 2019, 9:30am

Hi,
Please try

gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! application/x-rtp,encoding-name=VP8 ! rtpvp8depay ! queue ! omxvp8dec ! <b>videoparse format=23 width=1920 height=1080</b> ! omxh264enc ! fakesink async=0 sync=0

vp8dec is a software deocder. In default Gstreamer 1.4.2, ‘rtpvp8depay ! omxvp8dec’ cannot be linked.

IvanGolob · June 13, 2019, 2:21pm

Hi,
thank you, with this pipeline the transcoding is ok, but I don’t think it is hardware-accelerated. With both pipelines (udp-server and transcoding) I have high CPU usage (50%). Note that with the udp-server pipeline alone the CPU usage is about 20%.

Thank you, best regards.

Ivan

DaneLLL · June 14, 2019, 1:38am

Hi,
This is an optimal pipeline for TK1. Both omxvp8enc and omxvp8dec are hardware-accelerated. You may also consider other Jetson platforms to acquire good performance.