TAO classification_pyt

Hello,

I’m running the entire classification process in the notebook and I’m getting the onnx model. After resolving the error “Initializer onnx::MatMul_3870 appears in graph inputs and will not be treated as a constant value/weight.” with remove_initializer_from_input.py,

I’m unable to correctly classify images presented to the camera.

I substituted the TAO onnx model with the “resnet18-v1-7” model from onnx, and my program works fine. It classifies everything I present to it.

I checked the model’s input:
input_1, Shape: [‘batch’, 3, 224, 224]

and the image format I’m passing to it.
(np.shape(frame_normalized)
1, 3, 224, 224)

So I’m wondering if the problem is known, and what solution I could implement.

Thank’s

Sylvain Bolduc

Hi @sylvain.bolduc I’ve moved this to the Tao forum so the right team can help you out.

Best,

Sophie

1 Like

Which notebook did you run? Can you share logs or .ipynb? The notebook should run smoothly instead of error.

Hello,
I created a share point with the fully executed notebook, along with all the useful files.

TAO_classification

I understand that the model gives a slightly higher than 50% correct answer in the evaluation. But when I try to use it, it only gives one answer. Everything is a dog. But when I use the Resnet model and its classes, the little program works well. The TAO philosophy appeals to me a lot because I’m not an AI expert, but I’d like to use it for simple applications, as does the idea of ​​working with pre-trained networks and finetuning them. Thank you.

Can you run with the default spec file? The default epoch is 40. But you only set to 5 epochs. That may be a reason to the “slightly higher than 50% correct answer”. Please run with default tao_tutorials/notebooks/tao_launcher_starter_kit/classification_pyt/specs/train_cats_dogs.yaml at main · NVIDIA/tao_tutorials · GitHub and retry.

More, all the source code in the notebook can be found in two githubs(pytorch project and deploy project).
For example, tao model classification_pyt inference is using tao_pytorch_backend/nvidia_tao_pytorch/cv/classification/scripts/inference.py at dc07b02eb78c2eb868315107892b466496e55a0f · NVIDIA/tao_pytorch_backend · GitHub.

And tao deploy classification_pyt inference is using tao_deploy/nvidia_tao_deploy/cv/classification_pyt/scripts/inference.py at main · NVIDIA/tao_deploy · GitHub.

1 Like