Profiling tensorflow 2.0

AI & Data Science Deep Learning (Training & Inference) Frameworks
1

I would like to profile my model for tensorflow 2.0 with MultiWorkerMirroredStrategy. What us the best way to see all the GPU activity and other information?

2

For low-level comprehensive profiling you can use Nsight Systems https://developer.nvidia.com/nsight-systems.
If you are using the NGC Docker containers, NVTX annotations are added around each node in the TF graph, to make reading the profile easier.

Alternatively, if you are using tf.keras, you can collect timeline data with TensorFlow’s built in profiler. https://www.tensorflow.org/tensorboard/tensorboard_profiling_keras.