Run model with consecutive frames input on deepstream?

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU) : T4
**• DeepStream Version : 5.1
**• TensorRT Version 7.2
**• NVIDIA GPU Driver Version (valid for GPU only) : 418.67
**• Issue Type: questions/requirement
**• Requirement details :

I have a model which works on consectutive three frames from video stream. The model self can be converted to trt model and be exectuted successfully on TensorRT/Nvinfer in Deepstream.

0 INPUT kFLOAT input.1 3x448x448
1 INPUT kFLOAT input.4 3x448x448
2 INPUT kFLOAT input.7 3x448x448

Is there a solution to input multiple frames to a model with deepstream? Or shall I go with an appsink solution where I try to execute the onnx model within that one.

Sorry, what’s exactly is your model’s input, Just 3 frames from one input stream, I mean the 0th, 1st, 2nd as the model’s input?

Yes, three frames from one stream. Can I modify the nvdsinfer to achieve this?

Hey, could you provide more info about your model?

Here is the graph for the model.

Thanks, so how to do the preprocess for the model, just resize the 3 consecutive frames to 3x488x488(RGB or BGR) and put them to the 3 inputs of the model?

Yes. Resize three consecutive frames to 3x448x448

Hey customer, we had discussed this internally and had a solution to solve the problem. Would you mind to share your model with us and how to run the model, we would like to test our solution before publishing it to you.

I create a project on github which can run the model. The trt model can be downloaded from google drive. The postprocess can be output with a message broker. I did not yet upload the script which plot the figure. Is this sufficient for you to verify your solution?

Thanks, I will check it in some time.
Also we had another suggestion for you:
1.create a customized plugin to stitch the 3 frames to one frame, and then you can add another input wrapper layer before current 3 input layers, you can split the frame to 3 frames again and put them to the corresponding inputs.