nvidia-smi not showing any running process while training the model

installed on cpu

nvidia driver version 430

cuda 10.1

mesh-tensorflow 0.0.5
tensorflow 1.14.0
tensorflow-datasets 1.1.0
tensorflow-estimator 1.14.0
tensorflow-gpu 1.12.0
tensorflow-metadata 0.14.0
tensorflow-probability 0.7.0
tensorflow-tensorboard 1.5.1

:~$ nvidia-smi
Wed Aug 21 16:03:06 2019
±----------------------------------------------------------------------------+
| NVIDIA-SMI 430.40 Driver Version: 430.40 CUDA Version: 10.1 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce RTX 2080 Off | 00000000:01:00.0 Off | N/A |
| 41% 28C P8 1W / 225W | 2MiB / 7982MiB | 0% Default |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
±----------------------------------------------------------------------------+

It looks like you have both tensorflow and tensorflow-gpu installed. tensorflow-gpu is not a side-package, but provides a full version of TF. With both installed, it is likely that the cpu-only tensorflow installation is taking precedence over tensorflow-gpu, and as a result, you are running on the CPU only.

Did you solve this issue? I’ve got exactly the same problem. I’ve only installed tensorflow-gpu.