Hi,
I am developing a system using a model developed in the PyTorch environment. I noticed that loading the model into the Jetson Nano memory uses a lot of memory.
I have checked several solutions of different possibilities to load the model:
-
Jetson Nano after startup consumes on average 654 MB of ram (LXDE Desktop version)
-
After loading the MobileNetV2 model directly from the Trochvision library, the RAM usage increases to 782 MB on average - the loaded model consumes about 128 MB.
-
After loading the custom mobilenetV2 model in the Pytorch environment the memory usage increases to 2373.78 MB. The model consumes about 1719 MB.
-
In the case of loading the same model mobilenetV2 converted to TRT using Torch2TRT the RAM consumption is 3079 MB - the model occupies about 2425 MB of memory.
(I was inspired by this tutorial: torch2trt/conversion.ipynb at master · NVIDIA-AI-IOT/torch2trt · GitHub and torch2trt/live_demo.ipynb at master · NVIDIA-AI-IOT/torch2trt · GitHub ) -
In the case of converting the model to ONNX and then generating the trt engine, the memory consumption increases to 3313 MB - the model occupies 2659 MB.
(I followed this tutorial: TensorRT/4. Using PyTorch through ONNX.ipynb at master · NVIDIA/TensorRT · GitHub )
To check the memory I used:
sudo tegrastats
Is it possible to reduce the ram consumption associated with loading the model? Is there only 800MB of RAM left after loading the optimal trt model?
I would appreciate help if I am doing something wrong.