How to inference with a multi-input model that requires two streams of images

Please provide complete information as applicable to your setup.

• Hardware Platform: Jetson
• DeepStream Version : 6.3
• JetPack Version : 5.1.2
• TensorRT Version : 8.5.2.2
• Issue Type : questions

I have a multi-input model that requires two streams of images as input. I’m trying to modify NvDsInferInitializeInputLayers, but it doesn’t seem to work. How should I modify gst-nvinfer plugin and nvinfer library?
image

You can use our nvdspreprocess to customize your own tensor. You can refer to our deepstream-pose-classification demo.

In this demo, there are multiple fixed inputs that are non-image based. Are there any examples that involve multiple image inputs?

We don’t have a demo like that yet. But you can consider using multiple preprocesses to meet your needs.

...nvdspreprocess(image for the 1st input layer)-> \
nvdspreprocess(image for the 2nd input lyaer)->nvinfer->.....
1 Like

I have rewritten the nvdspreprocess_lib. How do I configure a PGIE in the deepstream-app configuration file to correspond to multiple nvdspreprocess instances? My model has two input tensors, input0 and input1. Do these two tensors require separate nvdspreprocess config files? The documentation for deepstream-app appears to provides a simple use case for nvdspreprocess.


Currently, deepstream-app does not support multiple pre-process configurations. Could you just write a simple demo, like ourdeepstream-test1, instead of the deepstream-app?