Overview
I am trying to run a tensorflow
model to do predictions on the Orin NX 8GB platform. I have Nvidia’s tensorflow==2.12.0+nv23.6
installed so it is using the integrated GPU. This has been confirmed via tegrastats
showing non-zero GPU usage when the model runs.
As a baseline, it seems like Linux and normal processes consume about 1.4GB of memory.
After the first call to model.predict()
, I see memory use jump to 6GB and then it never seems to reduce. I suspect the extra memory is from the GPU since it doesn’t track to a specific process (at least exactly). See below htop
screenshot :
38.5% MEM is ~2.8GB. With the 1.4GB baseline I measured, that leaves about 2GB unaccounted for from the 6.21GB shown utilized above.
My Setup
- An Orin NX 8GB with L4T R35.4.1 (the minimal filesystem with no desktop)
- Python 3.8 that L4T provides
tensorflow==2.12.0+nv23.6
from Nvidialibcudnn.so.8.6.0
- model is loaded from a
.hdf5
file if that matters
Questions
-
Is this a known issue with Nvidia’s
tensorflow
implementation for Jetson…that the GPU does not free up memory once allocated? If so, is there a way to limit the memory or tell it to behave differently? -
From other posts, I’ve seen recommendations to use
tensorrt
rather than normaltensorflow
. I am fairly new to regulartensorflow
so am not sure how difficult switching is. Is this a path worth pursuing?