Frames dropped with high VIC load

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hardware: TX2
Software: L4T 32.7.4
PowerMode: MAXN, all VI/ISP/NVCSI clocks maxed out

I am running 3 pipelines for 3 image sensors.
Pipeline1:
Sensor1(4k@30) ->nvarguscamerasrc → tee
|-> nvvideoconvert(1280x720) → nvv4l2h264enc → h264parse → rtph264pay → udpsink
|-> nvvideoconvert → jpegenc → multifilesink (ondemand)
|-> nvstreammux → nvvideoconvert ->nvinfer → nvtracker → nvdsanalytics → …
|-> nvvideoconvert → clockoverlay → nvvideoconvert → nvv4l2h264enc → h264parse → splitmuxsink

Pipeline 2&3:
Sensor2/3(4k@30) ->nvarguscamerasrc → tee
|-> nvvideoconvert(1280x720) → nvv4l2h264enc → h264parse → rtph264pay → udpsink
|-> nvvideoconvert → jpegenc → multifilesink (ondemand)

Problem description:

  1. When I have all three pipelines running, I noticed frames dropped in the JPEG branch by the discontinuity of the frame number in the metadata. in the order of 2~3 frames per second.
  2. If I stop either of pipeline 2 or 3, the remaining two pipelines would stop having such issue.
  3. If I stop the nvinfer branch or h264 branch in the first pipeline, the frame drop would also go away.
  4. When frame drop happens: I noticed high VIC load(always ~99%)

RAM 3845/7849MB (lfb 185x4MB) SWAP 0/3924MB (cached 0MB) CPU [57%@2031,0%@1728,0%@1728,56%@2029,59%@2028,57%@2026] EMC_FREQ 35%@1866 GR3D_FREQ 46%@1300 NVENC 345 VIC_FREQ 99%@1024 APE 150 MTS fg 0% bg 0% PLL@58C MCPU@58C PMIC@50C Tboard@51C GPU@58C BCPU@58C thermal@58C Tdiode@55C VDD_SYS_GPU 1410/1410 VDD_SYS_SOC 4569/4569 VDD_4V0_WIFI 38/38 VDD_IN 11348/11348 VDD_SYS_CPU 1459/1459 VDD_SYS_DDR 2913/2913

  1. When I kill pipeline 2 or 3, VIC load drops to 60~90%. So this appears to be correlated with VIC load.

Question:

  1. How can I be sure this is really caused by VIC overload?
  2. If so, how can I mitigate this issue?

Thanks!

2

Hi @hwang4

I have a couple of suggestions:

  1. Could you try adding maxperf-enable=1 to the nvv4l2h264enc instances?
  2. Could you try adding queue elements before and after each tee (on both branches)?

Regards,

Enrique Ramirez
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com
Website: www.ridgerun.com

4

Thanks for the reply, Enrique.

  1. I already have queue before/after the tee.
  2. I tried the maxperf-enable, it’s not making any difference.
5

I made some changes to only enable the RTSP branch when there’s active session.
That appears to fix the problem for now; but still would like to know if VIC is the bottleneck and how we can potentially improve the max performance.
Thanks.

6

It looks like the VIC could be the bottleneck. I see multiple nvvidconv, could you try using the nvjpegenc instead of “nvvideoconvert + jpegenc”?

Regards,
-Enrique

7

Thanks gain, Enrique.
I have changed jpegenc to nvjpegenc. that should help a little.

However this pipeline will cause segmentation fault.

gst-launch-1.0 nvarguscamerasrc bufapi-version=1 num-buffers=1 ! nvjpegenc ! filesink location=/tmp/test.jpg

If I change to old api: bufapi-version=0 it will work fine.
So the nvjpegenc is not compatible with new buffer API, but I didn’t see this mentioned anywhere.

8

hello hwang4,

did you work with DeepStream SDK? it’s due to nvjpegenc is not a DeepStream plugin.

Or, you may use software encoder to encode mjpeg with nvvideoconvert.
$ gst-launch-1.0 -e nvarguscamerasrc bufapi-version=true ... ! nvvideoconvert ! 'video/x-raw, format=(string)NV12, width=1920, height=1080, framerate=30/1' ! jpegenc ! filesink location=file.mjpeg

Or… you can encode H264 or H265 streams with the pipeline:
$ gst-launch-1.0 -e nvarguscamerasrc bufapi-version=true ... ! nvvideoconvert ! queue ! nvv4l2h264enc ! h264parse ! filesink location=file.264

9

Thanks for the reply, Jerry.

I did see the note that nvv4l2jpegenc is not compatible with DeepStream SDK. However in such a simple pipeline, only other plugin is nvarguscamerasrc, and I thought it’s not part of deepstream, but l4t-camera, right?

10

hello hwang4,

nvarguscamerasrc it’s only plugins.
you may see-also deepstream samples, i.e. /opt/nvidia/deepstream/deepstream/samples/