Issue with Inferencing with TensorRT on Python

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.0.1
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.2.3.1
• NVIDIA GPU Driver Version (valid for GPU only) 11.4
• Issue Type( questions, new requirements, bugs)

When I was trying to inference a model on TensorRT python, I got the following error:

Following is the log of the execution:

06/30/2022-17:15:43] [TRT] [I] [MemUsageChange] Init CUDA: CPU +450, GPU +0, now: CPU 663, GPU 1388 (MiB)
[06/30/2022-17:15:43] [TRT] [I] Loaded engine size: 60 MiB
[06/30/2022-17:15:44] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.5 but loaded cuBLAS/cuBLAS LT 110.9.2
[06/30/2022-17:15:44] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +871, GPU +378, now: CPU 1595, GPU 1828 (MiB)
[06/30/2022-17:15:44] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +127, GPU +58, now: CPU 1722, GPU 1886 (MiB)
[06/30/2022-17:15:44] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +59, now: CPU 0, GPU 59 (MiB)
[06/30/2022-17:15:44] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.5 but loaded cuBLAS/cuBLAS LT 110.9.2
[06/30/2022-17:15:44] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +10, now: CPU 1661, GPU 1878 (MiB)
[06/30/2022-17:15:44] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +1, GPU +8, now: CPU 1662, GPU 1886 (MiB)
[06/30/2022-17:15:45] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +307, now: CPU 0, GPU 366 (MiB)
check:2
binding dims=(3, 500, 500)
binding dims=(1, 500, 500)
[06/30/2022-17:15:45] [TRT] [E] 3: [executionContext.cpp::enqueueV2::304] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::enqueueV2::304, condition: !mEngine.hasImplicitBatchDimension()
)

Request your help in resolving the issue

2

Hi,
This looks more related to TensorRT. We are moving this post to the TensorRT forum to get better help.
Thank you.

6

Hi,

We recommend you please try on the latest TensorRT version 8.4 GA, if you still face this issue, please share with us the issue repro ONNX model and script for better debugging.

Thank you.

7

I tried the same but I get the same issue

8

Could you please share with us the issue repro ONNX model and script for better debugging.

9

The following is the test script. Unfortunately, I cannot give the repo ONNX/TRT model due to some constraints
trt_test_2.py (2.4 KB)
model_with_plugins_2.py (6.7 KB)

trt_test_2.py is the main file. Input shape of the model (1,500,500). Output_shape = (3,500,500). The name ‘Yolo’ may appear in files but the model is not object detection

10

Hi,

Could you please refer to Run PeopleNet with tensorrt - #21 by carlos.alvare

Thank you.

11

I will refer and get back. Thank you