Deepstream-app freezes when runs with "ds-example" enabled

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello, I am trying to run deepstream-app with ds-example enabled. For this, I have added a config group to the source30_1080p_dec_infer-resnet_tiled_display_int8.txt as the following:

[ds-example]
enable=1
full-frame=0
processing-width=640
processing-height=480
gpu-id=0
unique-id=15

I have also dropped some frames in cost of computational demand for batch processing by setting

[source0] 
drop-frame-interval=4
[source1]
drop-frame-interval=4

But I’m still getting 11FPS at maximum avg. per channel on 30 streams 720p each with Xavier.

When ds-example disabled, the deepstream-app runs perfectly on 30FPS for multiple streams on both Xavier and Nano. But when enabled, the deepstream-app runs very slowly (2~5 FPS) lags severely! I can’t find any explanation for this behavior, or is it expected?

2

Hi,
It is expected. Please check get_converted_mat() in

deepstream_sdk_v4.0.1_jetson\sources\gst-plugins\gst-dsexample\gstdsexample.cpp

There is code of converting RGBA to BGR via OpenCV. You may customize the code to your usecase accordingly.

3

@DaneLLL what do you mean by “its expected”? Are you saying that converting from RGBA to BGR with OpenCV is what slows processing down from 30fps down to the 11 the original poster mentioned?

So using OpenCV anywhere in your pipeline sounds like a real performance killer then?

4

Hi,

Yes. You can remove following code in gstdsexample.cpp and do comparison.

// Use openCV to remove padding and convert RGBA to BGR. Can be skipped if
// algorithm can handle padded RGBA data.
in_mat =
  cv::Mat (dsexample->processing_height, dsexample->processing_width,
  CV_8UC4, dsexample->inter_buf->surfaceList[0].mappedAddr.addr[0],
  dsexample->inter_buf->surfaceList[0].pitch);

cv::cvtColor (in_mat, *dsexample->cvmat, CV_RGBA2BGR);

We have NvBufSurf APIs in

deepstream_sdk_v4.0.1_jetson\sources\includes\nvbufsurface.h
deepstream_sdk_v4.0.1_jetson\sources\includes\nvbufsurftransform.h

You can leverage these APIs to have better performance in cropping/downscaling/flipping/compositing surfaces.