Frame rate drop on video encoding

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi,

I am encoding 6 streams simultaneously 720p@60 with the following gstreamer pipeline:

v4l2src-> capsfilter -> intervideosink -> intervideosrc-> nvvidconv-> capsfilter-> queue-> omxh264enc-> h264parse-> queue-> mpegpsmux-> queue-> filesink

After 5 minutes I get a drop of frames. That is, I have a callback function after the h264parse elment and there is an increasing difference to the previous frame of ~60[mSec], while during the normal operation it was ~16[mSec] as expected.

RAM 4496/6839MB (lfb 15x4MB) CPU [64%@1727,85%@1881,84%@1881,66%@1728,65%@1727,70%@1726] EMC_FREQ 21%@1600 GR3D_FREQ 35%@140 MSENC 294 APE 158 MTS fg 0% bg 0% BCPU@57.5C MCPU@57.5C GPU@56C PLL@57.5C Tboard@55C Tdiode@56.75C PMIC@100C thermal@56.9C VDD_IN 8375/8434 VDD_CPU 1659/1729 VDD_GPU 237/237 VDD_SOC 1753/1753 VDD_WIFI 0/0 VDD_DDR 2269/2288

Upon the frame rate drop the CPU utilization also drops.

I’m using the R28.2.

Do you have any suggestions how to debug this issue?

Regards.

2

Hi,
Please apply the patch and rebuild, replace libgstomx.so.

Source code of gstomx is in https://developer.nvidia.com/embedded/dlc/sources-r2821

3

DaneLLL, hi,

I’ve tried to make the sources without any patches and failed.

I’ve followed the README.txt file in the gst-omx1 directory:

  1. Untar the package and enter the dir
  2. export NOCONFIGURE=true
  3. export GST_EGL_LIBS="-lgstnvegl-1.0 -lEGL -lX11 -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0"
  4. sudo ln -s /usr/lib/aarch64-linux-gnu/libgstnvegl-1.0.so.0 /usr/lib/aarch64-linux-gnu/libgstnvegl-1.0.so
  5. ./autogen.sh
  6. ./configure --with-omx-target=tegra
  7. make <<<< failed here
  8. make install
....
checking for GST_PLUGINS_BASE... yes
configure: using GStreamer Base Plugins in /usr/lib/aarch64-linux-gnu/gstreamer-1.0
checking for GST_EGL... configure: error: Package requirements (gstreamer-egl-1.0) were not met:

No package 'gstreamer-egl-1.0' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables GST_EGL_CFLAGS
and GST_EGL_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

Do you have any suggestions?

4

Hi,
gst-omx package depends on gst-egl package. Please build gst-egl and then gst-omx.

5

Hi,

I have tried but I failue the ./configure stage, I assume that I need to provide some flags, but I don’t know which, as they are not provided in the README.txt file

If you need more informarion I’ll run it again. Please assist.

6

Hi,
There is a README in gstegl. Have you followed it to build gstegl?

7

Hi,
Yes I have, but I failed to identify the correct flags for ARM as I did for the omx:
./configure --with-omx-target=tegra, in the egl case it was not correct.

8

Hi,
Please check https://devtalk.nvidia.com/default/topic/1042881

9

DaneLLL, hi,

I have finally compiled the modified drivers and rerun the application. When each channel encodes @8Mbps it works OK, once I increase the encoding to 10Mbps the callbacks are slow again after awhile.

Any suggestions?

10

Hi,
Please run ‘sudo tegrastats’ to confirm NVENC(or called MSENC) runs at max clocks always. If it is, encoder keeps at maximum performance and there might be limitation in memory bandwidth or file IO(saving to storage). you may use ‘fpsdisplaysink video-sink=fakesink text-overlay=false’ to eliminate file IO for further clarification.

11

DaneLLL,

I have replace the filesink with fakesink, I uess that this eliminates the disk access. and yet after ~1minute the frame rate that I get is dropped. If I increase the bitrate (12Mbps, 20Mbps, …) the drop in frames happens much quickly.
Following is the tegrastats print out when I have frames drop:

sudo ./tegrastats
[sudo] password for nvidia: 
RAM 4068/6839MB (lfb 499x4MB) CPU [0%@1579,0%@1573,0%@1574,0%@1594,0%@1607,0%@1611] EMC_FREQ 13%@1600 GR3D_FREQ 14%@140 MSENC 115 APE 158 MTS fg 0% bg 1% BCPU@53.5C MCPU@53.5C GPU@52C PLL@53.5C Tboard@51C Tdiode@51.75C PMIC@100C thermal@52.9C VDD_IN 6661/6661 VDD_CPU 1046/1046 VDD_GPU 95/95 VDD_SOC 1520/1520 VDD_WIFI 0/0 VDD_DDR 1779/1779
RAM 4068/6839MB (lfb 498x4MB) CPU [53%@806,76%@1688,74%@1687,61%@806,61%@806,55%@806] EMC_FREQ 13%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@51.5C PLL@53.5C Tboard@51C Tdiode@51.75C PMIC@100C thermal@52.7C VDD_IN 6614/6637 VDD_CPU 998/1022 VDD_GPU 95/95 VDD_SOC 1520/1520 VDD_WIFI 0/0 VDD_DDR 1779/1779
RAM 4068/6839MB (lfb 498x4MB) CPU [57%@805,75%@1574,74%@1574,64%@806,61%@806,59%@805] EMC_FREQ 13%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@51.5C PLL@53.5C Tboard@51C Tdiode@51.75C PMIC@100C thermal@52.7C VDD_IN 6637/6637 VDD_CPU 998/1014 VDD_GPU 95/95 VDD_SOC 1473/1504 VDD_WIFI 0/0 VDD_DDR 1798/1785
RAM 4068/6839MB (lfb 498x4MB) CPU [57%@806,76%@1718,74%@1706,59%@806,59%@806,60%@806] EMC_FREQ 13%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@52C PLL@53.5C Tboard@51C Tdiode@51.75C PMIC@100C thermal@52.7C VDD_IN 6661/6643 VDD_CPU 998/1010 VDD_GPU 95/95 VDD_SOC 1473/1496 VDD_WIFI 0/0 VDD_DDR 1760/1779
RAM 4069/6839MB (lfb 497x4MB) CPU [62%@1420,80%@1934,80%@1935,66%@1420,68%@1419,63%@1421] EMC_FREQ 15%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 1% bg 0% BCPU@54C MCPU@54C GPU@52C PLL@54C Tboard@51C Tdiode@51.75C PMIC@100C thermal@52.7C VDD_IN 7745/6863 VDD_CPU 1662/1140 VDD_GPU 237/123 VDD_SOC 1566/1510 VDD_WIFI 0/0 VDD_DDR 2047/1832
RAM 4070/6839MB (lfb 497x4MB) CPU [55%@806,75%@1958,73%@1956,56%@806,57%@806,58%@805] EMC_FREQ 14%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@52C PLL@53.5C Tboard@51C Tdiode@52C PMIC@100C thermal@53.2C VDD_IN 6823/6856 VDD_CPU 1141/1140 VDD_GPU 142/126 VDD_SOC 1520/1512 VDD_WIFI 0/0 VDD_DDR 1837/1833
RAM 4070/6839MB (lfb 497x4MB) CPU [53%@960,75%@1575,74%@1574,53%@960,55%@959,59%@959] EMC_FREQ 13%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@52C PLL@53.5C Tboard@51C Tdiode@52C PMIC@100C thermal@52.9C VDD_IN 6614/6822 VDD_CPU 998/1120 VDD_GPU 95/122 VDD_SOC 1473/1506 VDD_WIFI 0/0 VDD_DDR 1779/1825
RAM 4070/6839MB (lfb 497x4MB) CPU [52%@960,76%@1855,74%@1841,54%@960,58%@959,58%@959] EMC_FREQ 13%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@52C PLL@53.5C Tboard@51C Tdiode@52C PMIC@100C thermal@52.9C VDD_IN 6566/6790 VDD_CPU 998/1104 VDD_GPU 95/118 VDD_SOC 1473/1502 VDD_WIFI 0/0 VDD_DDR 1760/1817
RAM 4070/6839MB (lfb 497x4MB) CPU [53%@806,76%@1690,73%@1687,59%@806,56%@806,54%@806] EMC_FREQ 13%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@51.5C PLL@53.5C Tboard@51C Tdiode@51.75C PMIC@100C thermal@52.9C VDD_IN 6590/6767 VDD_CPU 998/1093 VDD_GPU 95/116 VDD_SOC 1520/1504 VDD_WIFI 0/0 VDD_DDR 1798/1815
RAM 4070/6839MB (lfb 497x4MB) CPU [57%@960,74%@1806,74%@1810,58%@959,60%@960,60%@959] EMC_FREQ 13%@1600 GR3D_FREQ 0%@140 MSENC 115 APE 158 MTS fg 0% bg 0% BCPU@53.5C MCPU@53.5C GPU@52C PLL@53.5C Tboard@51C Tdiode@51.75C PMIC@100C thermal@52.7C VDD_IN 6614/6752 VDD_CPU 998/1083 VDD_GPU 95/113 VDD_SOC 1473/1501 VDD_WIFI 0/0 VDD_DDR 1779/1811

Any suggestions?

12

Hi,
You should see MSENC keeps at 1113 always. 115(MHz) looks low. Could you check again if libstomx.so is correctly rebuilt and replaced?

13

DaneLLL,

Thank you very much. Upon compiling the driver libgstomx.so was not copied to the directory where it supposed to be utilized, so I’ve manually copied it. We have at the moment MSENC (883 MHz).

Is for decoding I need to compile a driver too?

14

Hi,
The parameter bSetMaxEncClock is specific to encoders. For running decoders, no change is required.

15

DaneLLL, hi,

The problem is that although the encoder works OK the MPEG wrapper that I use mpegpsmux somehow fails to create a file that UBUNTU can play (a Windows station plays the file OK). When I change the muxer to mpegtsmux It plays OK. By “Plays OK” I mean playing on an Ubuntu standard player. Are there any configurations used within the modified omxh264enc that might cause this behaviour.

Notes:

  1. The problem was identified when I tried to encode the recorded video stream by linking mpegpsdemux dynamically and failed
  2. I have copied a file that have encoded prior to the omxh264enc modification and it played OK on the same station. I guess the problem is the connectivity between the encoder and the muxer.
  3. I saw that the patch included "ignore of the insert_sps_pps flag", could it influence in any way?
  4. Windows Media Player does not provide sound while VLC does, and prior to the modification it was vice-versa

Any suggestions?

16

Does anyone might know what might gone wrong between the modified omxh264enc and the muxpsmux?

Thanks

17

Hi,
Not sure but mpegpsmux seems to be for vintage MPEG1/2 stream. For H264, would suggest use transport stream, or matroskamux(mkv, webm)

18

Hi,

I understand, but just prior to the modifications the encoder and the decoder were working OK.