Tensorflow2.8 and p100 gpu memory

Description

We lately upgraded tensorflow 2.3 to tensorflow2.8.
We encounter serious memory issues. We get out of memory.
With the same script we can put x11 in the gpu virtual memory.
Can we use tensorflow2.8 with p100 gpu card?

Environment

TensorRT Version: None
GPU Type: P100
Nvidia Driver Version: 460.27.04
CUDA Version: 11.2
CUDNN Version: 8.2.1.32
Operating System + Version: Ubuntu 18.04.4 LTS
Python Version (if applicable): 3.10
TensorFlow Version (if applicable): 2.8
PyTorch Version (if applicable): NA
Baremetal or Container (if container which image + tag): NA

Attaching simple script that works runs tensorflow2.3 and does not work on tensorflow2.8.
sample_resnet50_train_test.py (2.4 KB)

Hi,

Your issue looks more related to Tensorflow.
We recommend you to please reach out to the Tensorflow forum to get better help.

Thank you.