Engine IO configs are mismatching

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) NVIDIA GeForce RTX 4070
• DeepStream Version 7.0
• TensorRT Version 8.6.1
• NVIDIA GPU Driver Version (valid for GPU only) 560.35.03
• Issue Type( questions)

The following is the expected IO configs of the engines to work on follow-up logics.

deepstream-1     | INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:612 [Implicit Engine Info]: layers num: 4
deepstream-1     | 0   INPUT  kFLOAT input           3x640x640
deepstream-1     | 1   OUTPUT kFLOAT boxes           25200x4
deepstream-1     | 2   OUTPUT kFLOAT scores          25200x1
deepstream-1     | 3   OUTPUT kFLOAT classes         25200x1

But for a new GPU based generated engine contains the following IO configs and we were unable to get the detections out.

INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:612 [Implicit Engine Info]: layers num: 5
deepstream-1     | 0   INPUT  kFLOAT x               3x640x640
deepstream-1     | 1   OUTPUT kFLOAT onnx::Sigmoid_1226 3x80x80x10
deepstream-1     | 2   OUTPUT kFLOAT onnx::Sigmoid_1590 3x40x40x10
deepstream-1     | 3   OUTPUT kFLOAT onnx::Sigmoid_1953 3x20x20x10
deepstream-1     | 4   OUTPUT kFLOAT 2267            25200x10

How to solve this?

3

Are you saying that you are using the same model, the same config file, the same pipeline to generate different engine IO on different Gpus?

4

Yeah, I have generated an engine file from an onnx file in another GPU with the same configs and the same deepstream pipeline

5

The engine files generated by the models cannot be shared between different gpus. When you change the GPU, you should regenerate the engine file.

6

Yeah, I have regenerated the engine in my new GPU device and there is this output logs.

INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:612 [Implicit Engine Info]: layers num: 5
deepstream-1     | 0   INPUT  kFLOAT x               3x640x640
deepstream-1     | 1   OUTPUT kFLOAT onnx::Sigmoid_1226 3x80x80x10
deepstream-1     | 2   OUTPUT kFLOAT onnx::Sigmoid_1590 3x40x40x10
deepstream-1     | 3   OUTPUT kFLOAT onnx::Sigmoid_1953 3x20x20x10
deepstream-1     | 4   OUTPUT kFLOAT 2267            25200x10
7

And in the previous GPU, I got the logs as follows,

deepstream-1     | INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:612 [Implicit Engine Info]: layers num: 4
deepstream-1     | 0   INPUT  kFLOAT input           3x640x640
deepstream-1     | 1   OUTPUT kFLOAT boxes           25200x4
deepstream-1     | 2   OUTPUT kFLOAT scores          25200x1
deepstream-1     | 3   OUTPUT kFLOAT classes         25200x1

In the newly generated engine I am able to run the pipeline for the detection but I do not get the predictions from it.

8

Could you attach your model and config file to us? Also could you attach the specific type of the two GPUs?

9

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.