FailedPredictionError and ResourceExhasutedError on running keras-retinanet on Jetson Nano

Hi.

I am trying to run keras-retinanet (https://github.com/fizyr/keras-retinanet) on my Jetson Nano board. First, I installed keras-retinanet on the board itself without any hassle but when I tried to run inference, it resulted in FailedPredictionError. I did try with TF_CUDNN_USE_AUTOTUNE=0 as an environmental variable, unfortunately, that did not work as well.

The next thing I tried was to get the weights from retinanet model in .h5 format, then convert it to its TRT variant. I am not sure if I am doing that in the right way, but I am trying to run inference using the TRT graph, it is causing FailedPredictionError.

Retinanet has been super successful in recent object detection tasks over two-stage detectors and single-shot detectors. This is why I thought, running retinanet on a Jetson Nano would be damn cool.

Here’s the link (https://www.dropbox.com/sh/5u1a1sxd6c51rl0/AADj_zcPj0X-IWM8gaTAxE7Ta?dl=0) which contains the script that converts .h5 weights to a TRT graph, the relevant error trace, and an example notebook with which I am trying to run the inference using the TRT graph.

Any help would be immensely appreciated.

Hi

ResourceExhasutedError indicates that you are running out of memory.
Please help to monitor the system status with tegrastats at the same time.

sudo tegrastats

Please check the RAM information to confirm if this is a resource issue or not.

Thanks.

Thanks for your reply. Even the following line of code takes forever to run on the Nano:

import tensorflow as tf

def get_frozen_graph(graph_file):
    """Read Frozen Graph file from disk."""
    with tf.gfile.FastGFile(graph_file, "rb") as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
    return graph_def

# The TensorRT inference graph file downloaded from Colab or your local machine.
pb_fname = "trt_graph.pb"
trt_graph = get_frozen_graph(pb_fname)

And I was able to execute the rest of the cells in the Predictions.ipynbnotebook apart from the last one. That causes resource exhaustion (stats taken by runningsudo tegrastats`).

Hi,

May I know the total size of your model. It looks like your are running out of the memory.
To check this, you can try to add some swap space to see if helps.

Thanks.