Can deepstream plugins (nvvideoconvert, nvinfer etc) support CUDA streams?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

**• Hardware Platform: RTXA4000
**• DeepStream Version: 6.0
**• TensorRT Version: 8.0.1
**• NVIDIA GPU Driver Version: 470.94
**• Issue Type: Question regarding CUDA streams with Deepstream plugins

Is it possible to use CUDA streams from custom source and sink plugins to process transfer plugins like nvvideoconvert and nvinfer?

We are looking to process each buffer with the same cuda stream throughout the pipeline while conconcurrently processing the next buffer with a different stream. For example:

stream1, buffer1:  nvcustomsource > ... > nvvideoconvert > nvinfer > ... > nvcustomsink
stream2, buffer2:                   nvcustomsource > ... > nvvideoconvert > nvinfer > ... > nvcustomsink
stream3, buffer3:                                    nvcustomsource > ... > nvvideoconvert > nvinfer > ... > nvcustomsink
3

Please refer to the appsrc sample /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-appsrc-test

4

Hi, I do not see any cuda stream usage in deepstream-appsrc-test.

The only place in sample_apps where I see cuda stream usage is deepstream-3d-action-recognition, is this what you mean?

We are looking for the flexibility of deepstream without compromising performance, our existing app uses cuda streams for performance.

Suppose for example we are porting our application which uses opencv’s cuda::cvtColor, as you see below the last argument is an optional cuda stream. In deepstream it will use the nvvideoconvert plugin. Can we provide the plugin a cuda stream in the same way we do in opencv? What about other gpu-accelerated plugins in deepstream?

void cv::cuda::cvtColor ( InputArray src,
OutputArray dst,
int code,
int dcn = 0,
Stream & stream = Stream::Null()
)
5

The cv::cuda::cvtColor format is not compatible to DeepStream, you need to transform the data to the format DeepStream knows.

6

As I mentioned in deepstream we will replace cv::cuda::cvtColor with nvvideoconvert. As shown the cv::cuda::cvtColor function allows a cuda stream to process the color conversion.

Can deepstream allow the same, that is can deepstream allow a cuda stream to process the color conversion?

Will the cuda extension be able to support this?

CudaExtension — DeepStream 6.1 Release documentation (nvidia.com)

7

The cv::cuda::cvtColor data format is not compatible to DeepStream. You can not use this interface to replace nvvideoconvert.

The color conversion inside nvvideoconvert is already GPU accelerated.

This is cuda extension, but not DeepStream extension.

8

Yes the color conversion inside nvvideoconvert is GPU accelerated, but can it use cuda streams?

9

No. Since the data format is not compatible.

10

@Fiona.Chen

I believe I understand now why the mentioned plugins won’t work with the CUDA extensions.

The CUDA extension is for NVIDIA Graph eXecution Framework (GXF) and not GStreamer, but they are documented to work with deepstream.

Deepstream can work with the GXF backend replacing the gstreamer one.

Am I understanding this correctly? If so we would seek to port our pipelines to GXF. This is used for a medical device in development.

11

DeepStream has its own data format based on HW. Even DeepStream supports CUDA memory, but the data format is not compatible to opencv cuda. There is no interface to support it now.

12

The question is not about opencv its about deepstream. I only used opencv as an example because its able to work with cuda streams.

We are looking to get our deepstream pipelines to work with cuda streams and the cuda extensions.

13

DeepStream supports CUDA memory. There is sample of CUDA operation in the sample codes of /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-dsexample

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.