Converting Yolo model to TensorRT format without ONNX conversion

Robotics & Edge Computing Jetson & Embedded Systems Jetson Projects
, , , , ,
1

Hi,
I am trying to convert a Yolov8s model to TensortRT without converting it to ONNX format first.
The related tools like torch_tensorrt and triton (with nvinferserver for DS) did not work as they struggle with the dynamic input shapes of the Yolo model which is not compatible with the torch.jit converter as well.
Is there any method that can do the conversion from Yolo model (directly) to TensorRT or can deploy a version of Yolo model directly in Deepstream?

These are the logs of the torch_tensorrt conversion and triton attempts (I aligned two terminals in one picture because of the media attachment limit) :

image
image1831×998 296 KB

pip list versions:
tensorrt 8.5.2.2
torch 2.0.1
torch-tensorrt 1.4.0
torchaudio 2.4.1+cu118
torchvision 0.15.2+cu117
triton 2.0.0

Additional context

  • PyTorch Version (e.g., 1.0): 2.0.1
  • OS (e.g., Linux): Linux
  • How you installed PyTorch ( conda , pip , source): pip
  • Build command you used (if compiling from source): NA
  • Python version: 3.8
  • CUDA/cuDNN version: CUDA 11.8, cuDNN 8.7.0
  • GPU models and configuration: Tesla T4
  • Any other relevant information: GPU Driver Version: 565.57.01
3

Dear @nvd-user ,
No. TensorRT accepts only ONNX format as input. Do you see any issue with ONNX conversion?

1 Like
5

BTW, issue is related with dGPU, and should open at Latest Deep Learning (Training & Inference)/TensorRT topics - NVIDIA Developer Forums if further questions. Thanks

1 Like