Please tell me how to improve out of memory in sensorflow in orin?
We have a total of 11G of SWAP memory. However, I am getting the following error and would appreciate if you could tell me how to improve it. Thank you in advance.
2023-11-11 21:04:07.706483: W tensorflow/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 165979200 exceeds 10% of free system memory.
2023-11-11 21:04:08.387216: W tensorflow/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 2028265824 exceeds 10% of free system memory.
2023-11-11 21:04:27.170980: W tensorflow/tsl/framework/bfc_allocator.cc:485] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.89GiB (rounded to 2028265984)requested by op EagerConst
If the cause is memory fragmentation maybe the environment variable ‘TF_GPU_ALLOCATOR=cuda_malloc_async’ will improve the situation.
Current allocation summary follows.
Current allocation summary follows.
2023-11-11 21:04:27.171515: W tensorflow/tsl/framework/bfc_allocator.cc:497] ***************____________________________________________________________________________________
Please specify the “training set” and the “number of epochs” as two arguments
n_epoch = 100
Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run _EagerConst: Dst tensor is not initialized.