Aside from having the .onnx file what else is required to load my custom ResNet model on the Xavier?
I also tried loading my custom model using the image-console binary but saw an error.
This was the command I used: ./imagenet-console --model=k2onnx_exp52_resnet50.onnx --input_blob=input_0 --output_blob=output_0 --labels=defect_labels.txt S9370.jpg_defect_0.png
Is there any example code demonstrating the link above (for Python Jetson Inference) but using a custom ResNet model?
I have searched the forums and web without any luck for this specific example of using a custom model with the Jetson Inference API.
Thanks for the reply!
When I use your Python program with my custom model I do see the TensorRT ERRORS: Network has dynamic or shape inputs, but no optimization profile has been defined, Network validation failed. I understand you’re saying this is because of “dynamic shape” but I’m not sure what to do to fix the problem. I saw the TesnorRT WARNING that my “ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32”. Is this an issue? There is also TypeError: ‘NoneType’ object is not iterable on line 67 in PrepareEngine for binding in engine:
and so I am also unsure how to set EXPLICIT_BATCH to static shape.
How would I generate an ‘optimization profile’ (not sure what this is precisely) to be defined assuming this is what resolves the “Network dynamic or shape inputs” error?
Just to re-iterate for what it’s worth, all that I would like to do is deploy my custom ResNet50 on the Jetson Xavier, ideally in Python for image classification, which seems like it should be possible. The boilerplate code you posted is a significant help for that!
The custom model is trained using TensorFlow 2.3.1.
I am using TensorFlow’s ResNet50 with imagenet weights, image data with dimensions 224x224x3, and vary the batch-size between 1 and 32. The loss function is categorical_crossentropy. Here is a snippet of the model definition:
Thanks for all of your help, you have been amazing!
The script you mentioned using onnx_graphsurgeon seems to have worked without error. Since my model used a BS = 32 I used the onnx_graphsurgeon to rewrite the BS to the ONNX model directly.
I have also confirmed my model exported properly from TensorFlow and the tf2onnx built the onnx model.
I am now seeing a new error when I run the Inference(engine) function in the original Python program you provided. The screenshot is attached below. It seems like the input image shape possibly does not match the input dimensions of the model. Since my model was generated using TensorFlow’s ResNet50 and used imagenet weights then the dimensions should be 224x224x3. Is there any advice you can provide for debugging this ValueError? Thanks!!
Thank you, I tried modifying the batchsize to 32 but now there is another error show below in the image. Please let me know if you have any suggestions resolving this and I will continue to debug it from my end.