Problem with running Keras model with the Jetbot

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello,

I currently have a pretrained keras model for a tracker with the .h5 format that’s around 3GB. This model works great loading and running on the computer but I wish to use it on the Jetson Nano aswell, together with the camera.

I’ve tried to load the model simply by doing "model = keras.models.load_model(‘model.h5’) and using that, but the memory appears to be an issue. I wonder if there is any preoptimization steps that I could perform or different alternative or if I need to make my model smaller?

I only wish to use the model for inference on nano, not any training.

Thanks
Alexander

3

Hi,

3GB model sounds too heavy to use on the Nano.
Maybe you can try to convert it into onnx and remove the training layer to see if helps first.

Thanks.

4

Hi AastaLLL, thank you for your response, I will look into onnx. I am not exactly sure what you mean by the training layer? All layers are still in the model during inference right? Just with frozen weights.

I will also try to create a smaller model but I’m worried about its performance. Would an alternative work with storing the model on a USB-stick inserted into the jetson nano and calling model.load(‘model.h5’) from there?

Thanks.

5

Hi,

There are some layers used only for training. Ex. layers for calculating backward loss.
You can remove them directly for inference.

To add an extra storage can increase swap memory which is accessible for CPU.
However, it won’t increase the memory amount for GPU.

Thanks.