Error in engine file


If you just change the file to an engine file to use tensorrt, you can predict, but why can’t you predict if you change the file to engine after training?


TensorRT Version: 10.0.1
GPU Type: GeForce RTX 2080 SUPER
Nvidia Driver Version: 551.86
CUDA Version: 11.2
CUDNN Version: v8.2.1
Operating System + Version: window 11 pro
Python Version (if applicable): 3.8.0
TensorFlow Version (if applicable): 2.5.0
PyTorch Version (if applicable): 1.8.0
Baremetal or Container (if container which image + tag):

Relevant Files

WARNING ⚠️ Unable to automatically guess model task, assuming ‘task=detect’. Explicitly define task for your model, i.e. ‘task=detect’, ‘segment’, ‘classify’,‘pose’ or ‘obb’.
Loading C:\psi\mono\egg100\egg1000\project_4cls\train9\weights\best.engine for TensorRT inference…
[05/02/2024-13:33:15] [TRT] [I] Loaded engine size: 20 MiB
[05/02/2024-13:33:15] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +4, now: CPU 0, GPU 23 (MiB)

[05/02/2024-13:33:16] [TRT] [E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2037] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2037, condition: engineDims.d[i] == dims.d[i] Static dimension mismatch while setting input shape.)
Traceback (most recent call last):
File “”, line 44, in
results = tensorrt_model(“C:/psi/mono/egg100/egg1000/egg_sha2_flipped3/1/2690811_sha2.bmp”)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\”, line 176, in call
return self.predict(source, stream, **kwargs)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\”, line 452, in predict
return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\”, line 168, in call
return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\torch\autograd\”, line 44, in generator_context
response = gen.send(None)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\”, line 255, in stream_inference
self.results = self.postprocess(preds, im, im0s)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\models\yolo\detect\”, line 25, in postprocess
preds = ops.non_max_suppression(
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\utils\”, line 220, in non_max_suppression
xc = prediction[:, 4:mi].amax(1) > conf_thres # candidates
RuntimeError: operation does not have an identity

Steps To Reproduce

Load the exported TensorRT model

tensorrt_model = YOLO(‘C:/psi/mono/egg100/egg1000/project_4cls/train9/weights/best.engine’)

Run inference

results = tensorrt_model(“C:/psi/mono/egg100/egg1000/egg_sha2_flipped3/1/2690811_sha2.bmp”)

for result in results:
highest_class = result.probs.top1


Hi @darkeagle333 ,
can you please share with us the reproducible files with us?