bandwidth of h264 decoder

Hi

I try to composite 8 h264 30fps fullHD streams from ethernet port to 1 frame and render it.
When input data transfer rate is high(more than 280Mbits), stream’s image become fuzzy.
I think the problem is receiving ability of h264 decoder. My question is blew.
1,What is max data transfer rate h264 decoder can receive?
2,Is there any way to drop frame rate before h264 decoder to reduce data transfer.
3,Is there any way to increase decoder receiving bandwidth. I tried using 2 ethernet ports, but it has no effect.

Hi

If disable-dpb=1 (disable DPB buffer).fuzzy problem is solved.
but I do not want to use disable-dpb for after-image around moving object

Hi,
The capability is listed in
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fsoftware_features_jetson_nano.html%23wwpID0EYHA

If it is an issue in decoder bandwidth, you should see slow decoding, not fuzzy images. Every decoded image shall be correct, although it cannot achieve desire frame rate. Per your description, it looks to be an issue that hardware decoder cannot decode the stream correctly. Please share a h264 stream that we can reproduce fuzzy images by running 00_video_decode.

Hi DaneLLL

To reproduce the problem, please use a pc conected to jetson-nano to send udp data.
execute the flowing shell script and programme

PC(ip adress 192.168.2.1)
1, run “/test/problem_str_release/chport_all.sh” create 8 port udp streaming data
2, run “/test/problem_str_release/runall.sh [ethernet interface name]” send udp streaming data to jetson-nano

jetson-nano (ip adress 192.168.2.10)
3, run “/test/rspstream_display/rspstream” receive streaming data from pc and render it

test.zip (52.8 MB)

Hi,
Could you try 120Mbps in total? It is hardware limitation:
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fsoftware_features_jetson_nano.html%23wwpID0EYHA

Hi

Thank you for your replay.
It is difficult to reduce udp sender bit rate.
Can I reduce frame rate before h264 decoder in jetson-nano

Hi,
Please try to configure the following properties:

skip-frames         : Type of frames to skip during decoding
                    flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                    Enum "SkipFrame" Default: 0, "decode_all"
                       (0): decode_all       - Decode all frames
                       (1): decode_non_ref   - Decode non-ref frames
                       (2): decode_key       - decode key frames
drop-frame-interval : Interval to drop the frames,ex: value of 5 means every 5th frame will be given by decoder, rest all dropped
                    flags: readable, writable, changeable only in NULL or READY state
                    Unsigned Integer. Range: 0 - 30 Default: 0

Hi DaneLLL

I cannot find drop-frame-interva.

dpkg -l | grep gstreamer
ii  gir1.2-gstreamer-1.0:arm64                 1.14.4-1~ubuntu18.04.1                          arm64        GObject introspection data for the GStreamer library
ii  gstreamer1.0-alsa:arm64                    1.14.1-1ubuntu1~ubuntu18.04.2                   arm64        GStreamer plugin for ALSA
ii  gstreamer1.0-clutter-3.0:arm64             3.0.26-1                                        arm64        Clutter PLugin for GStreamer 1.0
ii  gstreamer1.0-gl:arm64                      1.14.1-1ubuntu1~ubuntu18.04.2                   arm64        GStreamer plugins for GL
ii  gstreamer1.0-libav:arm64                   1.14.1-1~ubuntu18.04.1                          arm64        libav plugin for GStreamer
ii  gstreamer1.0-packagekit                    1.1.9-1ubuntu2.18.04.4                          arm64        GStreamer plugin to install codecs using PackageKit
ii  gstreamer1.0-plugins-bad:arm64             1.14.1-1ubuntu1~ubuntu18.04.1                   arm64        GStreamer plugins from the "bad" set
ii  gstreamer1.0-plugins-base:arm64            1.14.1-1ubuntu1~ubuntu18.04.2                   arm64        GStreamer plugins from the "base" set
ii  gstreamer1.0-plugins-base-apps             1.14.1-1ubuntu1~ubuntu18.04.2                   arm64        GStreamer helper programs from the "base" set
ii  gstreamer1.0-plugins-good:arm64            1.14.1-1ubuntu1~ubuntu18.04.1                   arm64        GStreamer plugins from the "good" set
ii  gstreamer1.0-plugins-ugly:arm64            1.14.1-1~ubuntu18.04.1                          arm64        GStreamer plugins from the "ugly" set
ii  gstreamer1.0-pulseaudio:arm64              1.14.1-1ubuntu1~ubuntu18.04.1                   arm64        GStreamer plugin for PulseAudio
ii  gstreamer1.0-tools                         1.14.1-1~ubuntu18.04.2                          arm64        Tools for use with GStreamer
ii  gstreamer1.0-x:arm64                       1.14.1-1ubuntu1~ubuntu18.04.2                   arm64        GStreamer plugins for X11 and Pango
ii  libgstreamer-gl1.0-0:arm64                 1.14.4-1ubuntu1.1~ubuntu18.04.1                 arm64        GStreamer GL libraries
ii  libgstreamer-plugins-bad1.0-0:arm64        1.14.1-1ubuntu1~ubuntu18.04.1                   arm64        GStreamer libraries from the "bad" set
ii  libgstreamer-plugins-base1.0-0:arm64       1.14.4-1ubuntu1.1~ubuntu18.04.1                 arm64        GStreamer libraries from the "base" set
ii  libgstreamer-plugins-base1.0-dev:arm64     1.14.4-1ubuntu1.1~ubuntu18.04.1                 arm64        GStreamer development files for libraries from the "base" set
ii  libgstreamer-plugins-good1.0-0:arm64       1.14.1-1ubuntu1~ubuntu18.04.1                   arm64        GStreamer development files for libraries from the "good" set
ii  libgstreamer1.0-0:arm64                    1.14.4-1~ubuntu18.04.1                          arm64        Core GStreamer libraries and elements
ii  libgstreamer1.0-dev:arm64                  1.14.4-1~ubuntu18.04.1                          arm64        GStreamer core development files
ii  libreoffice-avmedia-backend-gstreamer      1:6.0.7-0ubuntu0.18.04.5                        arm64        GStreamer backend for LibreOffice
 head -n 1 /etc/nv_tegra_release
# R32 (release), REVISION: 1.0, GCID: 14531094, BOARD: t210ref, EABI: aarch64, DATE: Wed Mar 13 07:46:13 UTC 2019
gst-inspect-1.0 omxh264dec
Factory Details:
  Rank                     primary + 10 (266)
  Long-name                OpenMAX H.264 Video Decoder
  Klass                    Codec/Decoder/Video
  Description              Decode H.264 video streams
  Author                   Sebastian Dröge <sebastian.droege@collabora.co.uk>

Plugin Details:
  Name                     omx
  Description              GStreamer OpenMAX Plug-ins
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstomx.so
  Version                  1.0.0.1
  License                  LGPL
  Source module            gst-omx
  Source release date      2014-02-08
  Binary package           GStreamer source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstOMXVideoDec
                               +----GstOMXH264Dec
                                     +----GstOMXH264Dec-omxh264dec

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h264
                 parsed: true
              alignment: au
          stream-format: byte-stream
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "omxh264dec-omxh264dec0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  use-omxdec-res      : Omx decoder resolution to be used(for debugging purpose)
                        flags: readable, writable
                        Boolean. Default: false
  full-frame          : Whether or not the data is full framed
                        flags: readable, writable
                        Boolean. Default: false
  disable-dpb         : Set to disable H.264 DPB buffer for low latency
                        flags: readable, writable
                        Boolean. Default: false
  skip-frames         : Which type of frames to skip during decoding
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Enum "H264SkipFrame" Default: 0, "DECODE_ALL_Frames"
                           (0): DECODE_ALL_Frames - GST_OMX_DECODE_ALL_FRAMES
                           (1): SKIP_NON_REF_FRAMES - GST_OMX_DECODE_SKIP_NON_REF_FRAMES
                           (2): DECODE_KEY_FRAMES - GST_OMX_DECODE_KEY_FRAMES
  output-buffers      : The number of OMX decoder output buffers (0 = auto)
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 16 Default: 0
  enable-error-check  : Set to enable on the fly decode error check
                        flags: readable, writable
                        Boolean. Default: false
  enable-low-outbuffer: Set to enable low output hardware buffer allocation
                        flags: readable, writable
                        Boolean. Default: false
  disable-dvfs        : Set to disable dvfs
                        flags: readable, writable
                        Boolean. Default: false
  enable-frame-type-reporting: Set to enable frame type reporting
                        flags: readable, writable
                        Boolean. Default: false

In my case, input h264 streaming data includes I-frame only.
I think skip-frames does not work.

Hi, pleas use nvv4l2decoder plugin.

Hi

I’m afraid, drop-frame-interva also cannot be found.

gst-inspect-1.0 nvv4l2decoder
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                NVIDIA v4l2 video decoder
  Klass                    Codec/Decoder/Video
  Description              Decode video streams via V4L2 API
  Author                   Nicolas Dufresne <nicolas.dufresne@collabora.com>, Viranjan Pagar <vpagar@nvidia.com>

Plugin Details:
  Name                     nvvideo4linux2
  Description              Nvidia elements for Video 4 Linux
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideo4linux2.so
  Version                  1.14.0
  License                  LGPL
  Source module            nvvideo4linux2
  Binary package           nvvideo4linux2
  Origin URL               http://nvidia.com/

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstNvV4l2VideoDec
                               +----nvv4l2decoder

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h264
          stream-format: { (string)byte-stream }
              alignment: { (string)au }
      video/x-h265
          stream-format: { (string)byte-stream }
              alignment: { (string)au }
      video/mpeg
            mpegversion: 4
           systemstream: false
                 parsed: true
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/mpeg
            mpegversion: [ 1, 2 ]
           systemstream: false
                 parsed: true
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/x-vp8
      video/x-vp9
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "nvv4l2decoder0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  device              : Device location
                        flags: readable
                        String. Default: "/dev/nvhost-nvdec"
  device-name         : Name of the device
                        flags: Opening in BLOCKING MODE
readable
                        String. Default: ""
  device-fd           : File descriptor of the device
                        flags: readable
                        Integer. Range: -1 - 2147483647 Default: -1
  output-io-mode      : Output side I/O mode (matches sink pad)
                        flags: readable, writable
                        Enum "GstNvV4l2IOMode" Default: 0, "auto"
                           (0): auto             - GST_V4L2_IO_AUTO
                           (2): mmap             - GST_V4L2_IO_MMAP
                           (5): dmabuf-import    - GST_V4L2_IO_DMABUF_IMPORT
  capture-io-mode     : Capture I/O mode (matches src pad)
                        flags: readable, writable
                        Enum "GstNvV4l2IOMode" Default: 0, "auto"
                           (0): auto             - GST_V4L2_IO_AUTO
                           (2): mmap             - GST_V4L2_IO_MMAP
                           (5): dmabuf-import    - GST_V4L2_IO_DMABUF_IMPORT
  extra-controls      : Extra v4l2 controls (CIDs) for the device
                        flags: readable, writable
                        Boxed pointer of type "GstStructure"
  skip-frames         : Type of frames to skip during decoding
                        flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
                        Enum "SkipFrame" Default: 0, "decode_all"
                           (0): decode_all       - Decode all frames
                           (1): decode_non_ref   - Decode non-ref frames
                           (2): decode_key       - decode key frames
  disable-dpb         : Set to disable DPB buffer for low latency
                        flags: readable, writable
                        Boolean. Default: false
  enable-full-frame   : Whether or not the data is full framed
                        flags: readable, writable
                        Boolean. Default: false
  enable-frame-type-reporting: Set to enable frame type reporting
                        flags: readable, writable
                        Boolean. Default: false
  enable-error-check  : Set to enable error check
                        flags: readable, writable
                        Boolean. Default: false
  enable-max-performance: Set to enable max performance
                        flags: readable, writable
                        Boolean. Default: false

Hi,
The properties are implemented in r32.2. If you do not use r32.2, please upgrade and try.

Hi

I have tried skip-frames and drop-frame-interva properties.
The properties are not effective on reducing bit rate before H264decoder.
Is there any way I can try?

Hi,
Spec is 120Mbps, and 280Mbps is way above the limitations. It may not be able to overcome it in software solutions. Suggest you consider to lower the total data rate.