Approaches to run TensorRT inference and VPI image functions simultaneously in GPU?

[liellplane]

Hello, I have an application which grabs images from a camera

Each image will be processed by VPI CUDA and output to gstreamer

Less frequently, an image is to be analysed by TensorRT yolo (inference time longer than several frames from camera) - with the result informing the parameters of transform to be conducted in the previous task

What strategy should I take for this ? The problem is that the images should be output at same framerate as input and I cannot wait for the inference to finish if its inline

thanks for any advice

[AastaLLL]

Hi,

What kind of VPI process do you need for the input image?

Based on your use case, Deepstream should be a useful library for you:

DeepStream SDK

Thanks.

[liellplane]

Hi AastaLLL and thanks for your response

I will only be using the warp perspective VPI, I am thinking maybe this does not need CUDA and can be done on the CPU if it can be done in realtime? Deepstream looks very interesting, are you suggesting it is flexible enough to handle asynchronous processes that occasionally inform the other? EG - an inference working at 1 fps can update an input parameter to a warp perspective which is happening at 30fps?

[AastaLLL]

Hi,

Warping is a relatively complicated task so it’s recommended to give it a try.
In some cases, the CPU backend can be 10 times slower than the CUDA backend.

https://docs.nvidia.com/vpi/algo_persp_warp.html

Deepstream can do inference periodically and generate the intermediate data with the tracker (detection case).
It will be a good direction to try if the above testing cannot reach 30fps.

Thanks.