Hello, I made an ONNX model with dynamic dimensions of input [batch,3,height,width] by repository:
After that I started trtexec with the following parameters:
trtexec --onnx=YOLOv8-dbs-dbd.onnx --saveEngine=YOLOv8-dbs-dbd.trt --explicitBatch=12 --minShapes=images:1x3x480x480 --optShapes=images:6x3x512x512 --maxShapes=images:12x3x640x640 --memPoolSize=workspace:3000
After that, I got a TensorRT engine from my model with dynamic dimensions and the batch size, then I start allocating CUDA memory, for example, for the size of 480 pixels. And I get an error
7: [shapeMachine.cpp::executeContinuation::887] Error Code 7: Internal Error (Add_430: dimensions not compatible for elementwise. Condition '==' violated: 5376 != 4725. Instruction: CHECK_EQUAL 5376 4725.)
The code swears at 430 Add node. It looks like this:
Why does it not work for me with dimension 480, if the model has dynamic dimensions at the input? And how to work properly with CUDA memory in general, if the model is with dynamic shapes?
TensorRT Version: 9.0.1
NVIDIA GPU: RTX3060
NVIDIA Driver Version: 535
CUDA Version: 11.1
CUDNN Version: 8.0.4
Python Version (if applicable): 3.8
link to ONNX model: YOLOv8-dbs-dbd.onnx
link to TensorRT model: YOLOv8-dbs-dbd.trt
Model is correct. I checked by check_model.py
Text output from the terminal after using the trtexec I added to my GoogleDrive:
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
- validating your model with the below snippet
filename = yourONNXmodel
model = onnx.load(filename)
2) Try running your model with trtexec command.
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
@AakankshaS I’ve already checked it. I wrote about it above :)
We are unable to reproduce this issue on the latest TensorRT versions 8.61 and 9.0.1.
Please make sure you installed correct version of the TensorRT.
&&&& PASSED TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=YOLOv8-dbs-dbd.onnx --saveEngine=YOLOv8-dbs-dbd.trt --explicitBatch=12 --minShapes=images:1x3x480x480 --optShapes=images:6x3x512x512 --maxShapes=images:12x3x640x640 --verbose