Description
Hi everyone,
I have two basic script to infer TensorRT model to Python and C++ and I see some difference between C++ and Python inference.
When I infer the model in Python, I see expected results and the all results look consistent. The outputs are labels, masks and detection’s. But when I infer the model in C++, I cannot see correct results. The detection and mask results looks okay but the labels looks different and wrong.
You can see my problem in following videos:
- Python: https://youtu.be/rBS3KIzIzLI
- C++: https://youtu.be/s00_xcu4fns
If you check the videos, you can see the only difference is the label names. Also when I print the outputs on my terminal, I see exact same numbers for detection’s but the numbers is different for labels.
Following image shows the visualization of ONNX model outputs:
You can see the datatype is float for masks and detection’s. And int64 for labels. So, I am thinking I am using wrong data type for labels. I tried to use int, uint16_t, uint32_t, uint64_t, long, long long and long int but any of them did not give correct result. In the videos I used int.
I shared the scripts which I used under the Relevant Files title.
The following code block shows the part I used for label results.
cudaMalloc(&outputLabelsBuffer, batchSize * outputLabelsSize * sizeof(int));
// ...
std::vector<int> outputLabels(batchSize * outputLabelsSize);
cudaMemcpy(outputLabels.data(), outputLabelsBuffer, batchSize * outputLabelsSize * sizeof(int),
cudaMemcpyDeviceToHost);
So, my problem is I cannot see the exact same results in Python and C++, and the only different part is label results. Is there any idea on my problem?
Environment
TensorRT Version: 8.6.1
GPU Type: RTX 3090
Nvidia Driver Version: 550.67
CUDA Version: 11.8
CUDNN Version: 8.9
Operating System + Version: Ubuntu 20.04.6 LTS focal x86_64
Python Version (if applicable): 3.8.10
Relevant Files
Python script I used: RTMDet TensorRT Python Deploy · GitHub
C++ script I used: RTMDet TensorRT Python Deploy · GitHub
TensorRT model: end2end.engine - Google Drive