installing missing plugin ffmpeg for gstreamer

Hello,

I have a h264 video file which I am trying to read through gstreamer on jetson Xavier as follows:

gst-launch-1.0 filesrc location=/home/lalita/Downloads/training-20190708T173445Z-001/training/0/0.h264 ! h264parse ! ffdec_h264 ! xvimagesink

However, it gives an error that there is no element ffdec_h264. I have installed gstreamer as per the instructions provided in the manual. Could anyone please help me to install the missing plugin ffmpeg?

Thanks

It is not supposed to be used on a jetson platform.

Use omxh264dec or nvv4l2decoder. nvv4l2decoder is preferred.

@TakenoriSato
Thanks for the reply. I used nvv4l2decoder instead of ffdec_h264 but it gave me another error that it cannot be linked with xvimagesink so I tried another h264 pipeline as follows using which I could play the video file:

$ gst-launch-1.0 filesrc location=/home/petehouston/projects/sample/data.h264 ! h264parse ! avdec_h264 ! xvimagesink

My question is, are there any resources available from which I can go through to get an idea which plugin is to be used where and building an application using deepstream?

Since its my first time using it, the user guide and manuals seem to be a reference material but it doesn’t provide hands on learning. Also, I have found that they are not updated to the latest frameworks.

I also referred gstreamer user guide and used the following pipeline for decoding h264 video:

gst-launch-1.0 filesrc location=/home/nvidia/Downloads/0.h264 \qtdemux ! queue ! h264parse ! nvv4l2decoder ! nv3dsink -e

with nv3dsink it says that the element is not found and with other plugins it says can’t be linked. Can someone please clear out the doubt regarding linking the plugins?

Thanks

This is a question about GStreamer in general. A plugin guide of GStreamer is available, but when it comes to building a pipeline, your best friend would be gst-inspect.

For example, nvv4l2decoder is inspected as follow.

$ gst-inspect-1.0 nvv4l2decoder
Factory Details:
  Rank                     primary + 11 (267)
  Long-name                NVIDIA v4l2 video decoder
  Klass                    Codec/Decoder/Video
  Description              Decode video streams via V4L2 API
  Author                   Nicolas Dufresne <nicolas.dufresne@collabora.com>, Viranjan Pagar <vpagar@nvidia.com>

Plugin Details:
  Name                     nvvideo4linux2
  Description              Nvidia elements for Video 4 Linux
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideo4linux2.so
  Version                  1.14.0
  License                  LGPL
  Source module            nvvideo4linux2
  Binary package           nvvideo4linux2
  Origin URL               http://nvidia.com/

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstNvV4l2VideoDec
                               +----nvv4l2decoder

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
  
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      image/jpeg
      video/x-h264
          stream-format: { (string)byte-stream }
              alignment: { (string)au }
      video/x-h265
          stream-format: { (string)byte-stream }
              alignment: { (string)au }
      video/mpeg
            mpegversion: 4
           systemstream: false
                 parsed: true
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/mpeg
            mpegversion: [ 1, 2 ]
           systemstream: false
                 parsed: true
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/x-divx
            divxversion: [ 4, 5 ]
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/x-vp8
      video/x-vp9
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "nvv4l2decoder0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  device              : Device location
                        flags: readable
                        String. Default: "/dev/nvhost-nvdec"
  device-name         : Name of the device
                        flags: Opening in BLOCKING MODE 
readable
                        String. Default: ""
  device-fd           : File descriptor of the device
                        flags: readable
                        Integer. Range: -1 - 2147483647 Default: -1 
  output-io-mode      : Output side I/O mode (matches sink pad)
                        flags: readable, writable
                        Enum "GstNvV4l2DecOutputIOMode" Default: 0, "auto"
                           (0): auto             - GST_V4L2_IO_AUTO
                           (2): mmap             - GST_V4L2_IO_MMAP
                           (3): userptr          - GST_V4L2_IO_USERPTR
  capture-io-mode     : Capture I/O mode (matches src pad)
                        flags: readable, writable
                        Enum "GstNvV4l2DecCaptureIOMode" Default: 0, "auto"
                           (0): auto             - GST_V4L2_IO_AUTO
                           (2): mmap             - GST_V4L2_IO_MMAP
  extra-controls      : Extra v4l2 controls (CIDs) for the device
                        flags: readable, writable
                        Boxed pointer of type "GstStructure"
  skip-frames         : Type of frames to skip during decoding
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Enum "SkipFrame" Default: 0, "decode_all"
                           (0): decode_all       - Decode all frames
                           (1): decode_non_ref   - Decode non-ref frames
                           (2): decode_key       - decode key frames
  drop-frame-interval : Interval to drop the frames,ex: value of 5 means every 5th frame will be given by decoder, rest all dropped
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 30 Default: 0 
  num-extra-surfaces  : Additional number of surfaces in addition to min decode surfaces given by the v4l2 driver
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 24 Default: 1 
  disable-dpb         : Set to disable DPB buffer for low latency
                        flags: readable, writable
                        Boolean. Default: false
  enable-full-frame   : Whether or not the data is full framed
                        flags: readable, writable
                        Boolean. Default: false
  enable-frame-type-reporting: Set to enable frame type reporting
                        flags: readable, writable
                        Boolean. Default: false
  enable-error-check  : Set to enable error check
                        flags: readable, writable
                        Boolean. Default: false
  enable-max-performance: Set to enable max performance
                        flags: readable, writable
                        Boolean. Default: false
  mjpeg               : Set to open MJPEG block
                        flags: readable, writable
                        Boolean. Default: false
  bufapi-version      : Set to use new buf API
                        flags: readable, writable
                        Boolean. Default: false

And, xvimagesink.

$ gst-inspect-1.0 xvimagesink
Factory Details:
  Rank                     primary (256)
  Long-name                Video sink
  Klass                    Sink/Video
  Description              A Xv based videosink
  Author                   Julien Moutte <julien@moutte.net>

Plugin Details:
  Name                     xvimagesink
  Description              XFree86 video output plugin using Xv extension
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstxvimagesink.so
  Version                  1.14.1
  License                  LGPL
  Source module            gst-plugins-base
  Source release date      2018-05-17
  Binary package           GStreamer Base Plugins (Ubuntu)
  Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-base1.0

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstXvImageSink

Implemented Interfaces:
  GstNavigation
  GstVideoOverlay
  GstColorBalance

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
              framerate: [ 0/1, 2147483647/1 ]
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "xvimagesink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  contrast            : The contrast of the video
                        flags: readable, writable
                        Integer. Range: -1000 - 1000 Default: 0 
  brightness          : The brightness of the video
                        flags: readable, writable
                        Integer. Range: -1000 - 1000 Default: 0 
  hue                 : The hue of the video
                        flags: readable, writable
                        Integer. Range: -1000 - 1000 Default: 0 
  saturation          : The saturation of the video
                        flags: readable, writable
                        Integer. Range: -1000 - 1000 Default: 0 
  display             : X Display name
                        flags: readable, writable
                        String. Default: null
  synchronous         : When enabled, runs the X display in synchronous mode. (unrelated to A/V sync, used only for debugging)
                        flags: readable, writable
                        Boolean. Default: false
  pixel-aspect-ratio  : The pixel aspect ratio of the device
                        flags: readable, writable
                        String. Default: null
  force-aspect-ratio  : When enabled, scaling will respect original aspect ratio
                        flags: readable, writable
                        Boolean. Default: true
  handle-events       : When enabled, XEvents will be selected and handled
                        flags: readable, writable
                        Boolean. Default: true
  device              : The number of the video adaptor
                        flags: readable, writable
                        String. Default: "0"
  device-name         : The name of the video adaptor
                        flags: readable
                        String. Default: null
  handle-expose       : When enabled, the current frame will always be drawn in response to X Expose events
                        flags: readable, writable
                        Boolean. Default: true
  double-buffer       : Whether to double-buffer the output
                        flags: readable, writable
                        Boolean. Default: false
  autopaint-colorkey  : Whether to autofill overlay with colorkey
                        flags: readable, writable
                        Boolean. Default: true
  colorkey            : Color to use for the overlay mask
                        flags: readable, writable
                        Integer. Range: -2147483648 - 2147483647 Default: 526352 
  draw-borders        : Draw black borders to fill unused area in force-aspect-ratio mode
                        flags: readable, writable
                        Boolean. Default: true
  window-width        : Width of the window
                        flags: readable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  window-height       : Height of the window
                        flags: readable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  render-rectangle    : The render rectangle ('<x, y, width, height>')
                        flags: writable
                        GstValueArray of GValues of type "gint" Write only

An element of GStremer has zero or more sink(input) and source(output) pads. And if one element can be linked to another is up to their capabilities. If one capability matches to another one, it is linked. Otherwise, it is not. This gets complicated if a pad is dynamically created. But for now, just consider a static pad(Availability: Always).

Now, take a look at the sink’s capability of xvimagesink, which is:

video/x-raw
              framerate: [ 0/1, 2147483647/1 ]
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

And let’s compare it with the source’s capability of nvv4l2decoder.

video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

This looks the same, except for the memory. nvv4l2decoder passes an output buffer in NVMM, which means on a GPU memory here. But xvimagesink needs a buffer on a main memory. That’s why your pipeline fails.

This is a hardware accelerated version of your pipeline.

$ gst-launch-1.0 filesrc location=/home/lalita/Downloads/training-20190708T173445Z-001/training/0/0.h264 ! h264parse ! nvv4l2decoder ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=RGBA" ! nvegltransform ! nveglglessink sync=False

Try inspecting unknown elements, and see how they are connected with the gst-launch. Sometimes it doesn’t work as expected even if they look identical from capabilities.