I want to convert my trained model and optimize inference with TensorRT 8.0.
For this I use the following conversion flow: Pytorch → ONNX → TensorRT
The ONNX model can be successfully runned with onxxruntime-gpu, but failed with conversion from ONNX to TensorRT with trtexec.
From debugging, I have found the problem place which is related with following original Pytorch code:
def sample_points(points, idx):
idx = idx.view(-1).unsqueeze(1)
index = idx.expand(-1, points.size(-1))
res = torch.gather(points, 0, index)
return res.reshape(points.size(0), -1, points.size(1))
This sample_points function is used as intermediate operation inside NN. Seems like problem is connected that I use torch.size to initialize dimension and produces non-static, dynamic input data for torch.gather.
Does anyone know how to solve this problem? I very need your help. Many thanks!
TensorRT Version 8.0:
GPU Type NVIDIA RTX 2080 Ti
Nvidia Driver Version 450.5106
CUDA Version 11.0
Operating System + Version Ubuntu 18.04 LTS
Python Version 3.6.10
PyTorch Version 1.9.0