Description
If you just change the yolov8s-cls.pt file to an engine file to use tensorrt, you can predict, but why can’t you predict if you change the best.pt file to engine after training?
Environment
TensorRT Version: 10.0.1
GPU Type: GeForce RTX 2080 SUPER
Nvidia Driver Version: 551.86
CUDA Version: 11.2
CUDNN Version: v8.2.1
Operating System + Version: window 11 pro
Python Version (if applicable): 3.8.0
TensorFlow Version (if applicable): 2.5.0
PyTorch Version (if applicable): 1.8.0
Baremetal or Container (if container which image + tag):
Relevant Files
WARNING ⚠️ Unable to automatically guess model task, assuming ‘task=detect’. Explicitly define task for your model, i.e. ‘task=detect’, ‘segment’, ‘classify’,‘pose’ or ‘obb’.
Loading C:\psi\mono\egg100\egg1000\project_4cls\train9\weights\best.engine for TensorRT inference…
[05/02/2024-13:33:15] [TRT] [I] Loaded engine size: 20 MiB
[05/02/2024-13:33:15] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +4, now: CPU 0, GPU 23 (MiB)
[05/02/2024-13:33:16] [TRT] [E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2037] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2037, condition: engineDims.d[i] == dims.d[i] Static dimension mismatch while setting input shape.)
Traceback (most recent call last):
File “pt2engine.py”, line 44, in
results = tensorrt_model(“C:/psi/mono/egg100/egg1000/egg_sha2_flipped3/1/2690811_sha2.bmp”)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\model.py”, line 176, in call
return self.predict(source, stream, **kwargs)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\model.py”, line 452, in predict
return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\predictor.py”, line 168, in call
return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\torch\autograd\grad_mode.py”, line 44, in generator_context
response = gen.send(None)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\engine\predictor.py”, line 255, in stream_inference
self.results = self.postprocess(preds, im, im0s)
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\models\yolo\detect\predict.py”, line 25, in postprocess
preds = ops.non_max_suppression(
File “C:\Users\User\anaconda3\envs\yolov8_cls\lib\site-packages\ultralytics\utils\ops.py”, line 220, in non_max_suppression
xc = prediction[:, 4:mi].amax(1) > conf_thres # candidates
RuntimeError: operation does not have an identity
Steps To Reproduce
Load the exported TensorRT model
tensorrt_model = YOLO(‘C:/psi/mono/egg100/egg1000/project_4cls/train9/weights/best.engine’)
Run inference
results = tensorrt_model(“C:/psi/mono/egg100/egg1000/egg_sha2_flipped3/1/2690811_sha2.bmp”)
for result in results:
highest_class = result.probs.top1
print(highest_class)