How to run spatio-temporal models with Deepstream?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 7.1
• NVIDIA GPU Driver Version (valid for GPU only) 440+
• Issue Type( questions, new requirements, bugs) Question

I would like to perform activity recognition using deepstream. We want to make use of a pose network as secondary gie and then batch the patches pertaining to an object temporally and pass it to a spatio-temporal model (like lstm).

How do we achieve the above pipeline using deepstream?

3

Not sure if you have checked GitHub - NVIDIA-AI-IOT/trt_pose: Real-time pose estimation accelerated with NVIDIA TensorRT?

4

Hi @bcao thanks for the information. I looked at the repository. However, the repository runs the pose estimation standalone and do not use any temporal models like (lstm and rnns), and not integrated into the deepstream pipeline.

My question is more on integration. Like taking the model from trt_pose/other temporal models and infer using nvinfer plugin directly.
print
One solution, I can think of is using the pose estimation model as a library which is called from a custom plugin.

5

Please refer GitHub - NVIDIA-AI-IOT/deepstream_pose_estimation: This is a sample DeepStream application to demonstrate a human pose estimation pipeline.

7

What about 3D conv neural networks that use prev frames to do inference?

8

Hi abrar.shahriar,

Please help to open a new topic if it’s still an issue.

Thanks

1 Like