Tegra 3 Gstreamer

Developer Tools Other Tools Tegra Tools (closed)
1

Hello All,

I am having an issue attempting to get the NVIDIA accelerated h264 gstreamer plugins running. Specifically the gstreamer pipelines I create to do h264, h263 or mpeg4 hardware encoding on the Tegra 3 all get stuck at the pre-loading step. Below is a more detailed description of my issue, I would greatly appreciate any help I could get on this issue.

Details:

System:
Carma Dev Kit: Embedded Systems Developer Kits & Modules from NVIDIA Jetson
-Tegra 3
-Ubuntu 12.04 with CUDA 5.0
-Cardhu Tegra Linux extra Codecs R-16 (Jetson Linux Archive | NVIDIA Developer)

Problem:
When I run the pipeline “gst-launch -v videotestsrc ! ffmpegcolorspace ! video/x-raw-yuv ! nv_omx_h264enc ! rtph264pay ! udpsink host=127.0.0.1 port=7899” on the Carma Dev kit it pauses and permanently hangs on the pre-loading step. If I replace “nv_omx_h264enc” with “x264enc”, the stream works, so the pipeline structure appears to be correct. When I turn on gstreamer debug messages I am not getting any errors( I can send the output of this in a follow on email).

In addition to having a problem with the h264 codec, I am also unable to get mpeg4 or h263 running. If in the example above I replace "nv_omx_h264enc ! rtph264pay" with "nv_omx_h264enc ! rtph263pay" or "ffmpegcolorspace ! nv_omx_mpeg4enc" I also get stuck on the pre-rolling phase.

As an important side not I am able to get the mpjeg codec to work.  If I use "ffmpegcolorspace ! nv_omx_jpegenc" the pipeline runs correctly and I can receive video.

Thanks for the help,

Chris

2

Hi Chris,

Can you please provide the (console logs; don’t enable GST_DEBUG) of the following pipelines

  1. gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! fakesink silent=0 —> Note that in this case, fakesink element will dump log for every buffer that passes through it. Please only provide first 5-6 dumps (assuming the pipeline goes past the pre-rolling stage)

  2. gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! rtph264pay ! fakesink silent=1

  3. gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! rtph264pay ! udpsink

Also, “…! nv_omx_h264enc ! rtph263pay !..” will never work, as you are trying to packetize h264 frames with h263 rtp payloader. Same holds true for mpeg4enc. You need to use rtpmp4vpay element.

As an important side not I am able to get the mpjeg codec to work. If I use “ffmpegcolorspace !
nv_omx_jpegenc” the pipeline runs correctly and I can receive video

Can you please elaborate here as to what was the exact pipeline you were trying?

Thanks,
Vikram

3

Vikram,

Thanks for the response, I’ll reply below in the same numbered format. Sorry about “…! nv_omx_h264enc ! rtph263pay !..” it is a typo and should be “! nv_omx_h263enc ! rtph263pay !”. Additionally “ffmpegcolorspace ! nv_omx_mpeg4enc” is also a typo and should be “nv_omx_mpeg4enc ! rtpmp4vpay”. Anyway the gstreamer command outputs…

  1. gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! fakesink silent=0 ):
    ubuntu@seco-gpu-devkit:~$ sudo gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! fakesink silent=0
    [sudo] password for ubuntu:
    NvxLiteH264DecoderInit : Opening TVMR H264 block
    NvxLiteH264DecoderInit : Opening TVMR H264 block
    Setting pipeline to PAUSED …
    /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    Pipeline is PREROLLING …
    /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstOmxH264Enc:omxh264enc0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstOmxH264Enc:omxh264enc0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2

  2. gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! rtph264pay ! fakesink silent=1 ):
    ubuntu@seco-gpu-devkit:~$ sudo gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! rtph264pay ! fakesink silent=1
    NvxLiteH264DecoderInit : Opening TVMR H264 block
    NvxLiteH264DecoderInit : Opening TVMR H264 block
    Setting pipeline to PAUSED …
    /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    Pipeline is PREROLLING …
    /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstOmxH264Enc:omxh264enc0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstOmxH264Enc:omxh264enc0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2

  3. gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! rtph264pay ! udpsink ):
    ubuntu@seco-gpu-devkit:~$ sudo gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv’ ! nv_omx_h264enc ! rtph264pay ! udpsink
    NvxLiteH264DecoderInit : Opening TVMR H264 block
    NvxLiteH264DecoderInit : Opening TVMR H264 block
    Setting pipeline to PAUSED …
    /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    Pipeline is PREROLLING …
    /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstOmxH264Enc:omxh264enc0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2
    /GstPipeline:pipeline0/GstOmxH264Enc:omxh264enc0.GstPad:sink: caps = video/x-raw-yuv, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(fourcc)YUY2, color-matrix=(string)sdtv, chroma-site=(string)mpeg2

  4. Can you please elaborate here as to what was the exact pipeline you were trying? ):
    gst-launch videotestsrc ! ffmpegcolorspace ! nv_omx_jpegenc ! rtpjpegpay ! udpsink port= host=<ip_addr> sync=false

-Chris

4

duplicate

5

duplicate

6

Hi Chris,

We are not able to reproduce this issue on R16.3 as on Jetson Linux Archive | NVIDIA Developer.
Also, can you try with I420 format? i.e

gst-launch -v videotestsrc ! ffmpegcolorspace ! ‘video/x-raw-yuv, format=(fourcc)I420’ ! nv_omx_h264enc ! fakesink silent=0

7

Hmmm…

The main difference seems to be the install process. I have been using SECOs ubuntu 12.04 build with cuda 5.0 along with their own custom install process for the CARMA dev. Given that it works for you I am going to take another look at the install.

8

Not sure if that helps anything, just for the reference, I managed to get gstreamer decode H.264 on Nexus 7 with Plasma Active using something like this:

gst-launch-0.10 filesrc location=$HOME/sintel_trailer-720p.mp4 ! qtdemux name=demuxer \ demuxer. ! nv_omx_aacdec ! queue ! audioconvert ! audioresample ! autoaudiosink \ demuxer. ! queue ! nv_omx_h264dec ! nvxvimagesink