Jetson Nano onnx inference issue

Robotics & Edge Computing Jetson Systems Jetson Nano
1

I am trying to inference autoencoder.onnx file with trtexec. so i convert autoencoder.onnx to autoencoder.trt file by using command “trtexec --onnx=autoencoder.onnx --saveEngine=autoencoder.trt”. and i inferenced trt file by using command “trtexec --loadEngine=autoencdoer.trt --loadInputs=input:input_image1.npy --exportOutput=output_image1.json”. i also tried different from “input_image1.npy” to “input_image5.npy”. but when i recover image from “output_image1.json” to “output_image5.json”, output image is not same with input image and those 5 output are same values. how can i slove this problems? i already checked onnx file.

4

Dear @21800806,
Could you share the reproducible steps and files?

5

autoencoder.zip (742.5 KB)

this is my zip file including onnx, input npy files.

1.i converted autoencoder file from pytorch to onnx on my desktop.
WITH COMMAND:
BATCH_SIZE=1
dummy_input=torch.randn(BATCH_SIZE,1,28*28)
torch.save(model.state_dict(), ‘model_weights.pth’)
prm= torch.load(“model_weights.pth”,map_location=“cpu”)
model.load_state_dict(prm)
model.eval()
torch.onnx.export(model, dummy_input, “autoencoder.onnx”,opset_version=11, verbose=True, input_names=[‘input’],output_names=[‘output’])

  1. I bring onnx files to Jetson nano and inference onnx model
    WITH COMMAND:
    trtexec --onnx=autoencoder.onnx --saveEngine=autoencoder.trt
    trtexec --loadEngine=autoencoder.trt --loadInputs=input:input_image1.npy --exportOutput=output_image1.json

  2. recover output_image1.json file

ENVIRONMENTS
Jetson Nano Developer Kit - Jetpack 4.6.4
CUDA:10.2.
TensorRT: 8.2.

DESKTOP ENVIRONMENTS
python 3.11.5
pytorch 2.1.2
onnx 1.15.0
GPU: RTX 4090

6

Dear @21800806,
Does this issue still need support?

7

Yes, Thank you to email!. I don’t know how to resolve this problem so I was waiting for your advices.

2024년 2월 27일 (화) 오후 5:51, SivaRamaKrishnaNV via NVIDIA Developer Forums <notifications@nvidia.discoursemail.com>님이 작성:

8

Dear @21800806,
output image is not same with input image

How did you generate input data from input image and how did you generate output image from TRT model output? Does this need any pre/postprocessing of data needed before/after feeding to TRT? Did you verify already if the ONNX model is working as expected before ONNX-> TRT conversion?