Hi, community. I converted my pytorch model with custom layer from pytroch to tensorrt through torch2trt (GitHub - NVIDIA-AI-IOT/torch2trt: An easy to use PyTorch to TensorRT converter).
So I write custom plugin for tensorrt and custom converter for torch2trt.
To keep it simple I took lenet and convert it to tensorrt and after that measured the error like this:
# create some regular pytorch model... model = LeNet.eval().cuda() # create example data x = torch.ones((1, 1, 32, 32)).cuda() # convert to TensorRT feeding sample data as input model_trt = torch2trt(model, [x]) y = model(x) y_trt = model_trt(x) # check the output against PyTorch print(torch.max(torch.abs(y - y_trt)))
And the error is very small like 4e-7/
After that I tried to convert lenet with my custom layers and the error is like 0.018
But when I load weights and measure the error it becomes even higher.
model_path = 'models/lenet5_mnist.pt' lenet5_model.load_state_dict(torch.load(model_path)) lenet5_model.eval().cuda() # create example data x = torch.ones((1, 1, 32, 32)).cuda() # convert to TensorRT feeding sample data as input model_trt = torch2trt(lenet5_model, [x]) y = lenet5_model(x) print(torch.max(torch.abs(y - y_trt)))
And the error is around 2.5:
tensor(2.5567, device='cuda:0', grad_fn=<MaxBackward1>)
I’ve tried inference on several images and make sure that there is a great drop in accuracy.
I’m trying to debug this and wanted to print architecture.
for pytorch model I can simple write:
but when I try on wrapped tensorrt model:
it outputs the only:
Tried to check model weights, for torch is ok:
But tensorrt outputs smth unreadable:
So my question is how to check model graph, weights etc in tensorrt?
Or may be some tips how to debug my convertation?