Does training deep learning mode on fp16 uses less memory?

i am trying to use a unet trained whit img size of 640x360 on my 2060 in 32fp, but when i try to load it on the nano i get

OOM when allocating tensor with shape…

so as the title say, does training deep learning mode on fp16 uses less memory?

Hi,

Which framework do you use?
For TensorRT, yes, the fp16 will use less memory than the fp32 mode.
However, this may not be true for other frameworks since they may pre-allocate some working space memory.


https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#mixed_precision
Using precision lower than FP32 reduces memory usage, allowing the deployment of larger networks. Data transfers take less time, and compute performance increases, especially on GPUs with Tensor Core support for that precision.

Thanks.

thanks for the answer, im using tensorflow, i never used tensorRT is to much diferent to tf?

Hi,

In Jetson, TensorFlow tends to occupy much more memory than TensorRT.
So it’s recommended to use TensorRT to save memory.

Thanks.