Optimize TF-TRT models on Jetson Nano to improve inference timing and efficiency

Hi,
I have executed a TF-TRT model(FP16) for image classification on Jetson Nano.
For more information see https://github.com/NVIDIA-AI-IOT/tf_trt_models/blob/master/examples/classification/classification.ipynb

I used an inception-ResNet-v2 model for the prediction. The timing I got for the inference was about 140ms. In addition that takes around ~3.5GiB Memory and ~5.2GiB Swap.

Can I get support in order to optimize the timing and efficiency of my system? the details are:

Framework: Tensorflow TensorRT
Architecture: inception-ResNet-v2
inference: ~140 msec.
Memory: ~3.5GiB
Swap: ~5.2GiB

I would like to know if I can get the support to improve it or if I can get literature and examples on how to take advantage of the hardware as much as.

Thanks.

Hi,

Swap may have some bad impact on the performance.
Is your model can be inference without adding the swap?

More, it’s recommended to use pure TensorRT to have a better performance.
Here is a tutorial for your reference: https://github.com/NVIDIA-AI-IOT/tf_to_trt_image_classification

Thanks.

Hi AastaLLL,

Thank your recommendations.

I tried a tutorial: https://github.com/NVIDIA-AI-IOT/tf_to_trt_image_classification on my Jetson Nano.

I got this error when built this project

git clone --recursive https://github.com/NVIDIA-Jetson/tf_to_trt_image_classification.git
cd tf_to_trt_image_classification
mkdir build
cd build
cmake …
make

– Configuring done
– Generating done
– Build files have been written to: /home/nano/prj/tf_to_trt_image_classification/build
[ 25%] Building CXX object src/CMakeFiles/uff_to_plan.dir/uff_to_plan.cpp.o
/home/nano/prj/tf_to_trt_image_classification/src/uff_to_plan.cpp: In function ‘int main(int, char**)’:
/home/nano/prj/tf_to_trt_image_classification/src/uff_to_plan.cpp:71:79: error: no matching function for call to ‘nvuffparser::IUffParser::registerInput(const char*, nvinfer1::DimsCHW)’
parser->registerInput(inputName.c_str(), DimsCHW(3, inputHeight, inputWidth));
^
How can I fix this?

Thanks.

Hi,

Please check the patch for TensorRT 5.x here:
https://github.com/NVIDIA-AI-IOT/tf_to_trt_image_classification/pull/40

Thanks.