Hi,
TensorFlow might occupy much more CUDA memory for workspace.
Could you help to monitor the system status via tegrastats first?
$ sudo tegrastats
If the issue is cased by the OOM, please try the following suggestion to see if helps first.
TF v1.15.x: Resource Exhausted Error - #3 by AastaLLL
TF v2.x: Jetson Nano running out of memory: ResourceExhaustedError: OOM when allocating tensor with shape[3,3,512,1024 - #5 by AastaLLL
Thanks.