How to use DLProf plugin with Tensorboard?

I’m using Ubuntu 18 and Tensorflow 2.5.0

I want to be able to see a visualization of the GPU utilization, Tensor Core Kernel efficiency, etc. For example, to see Figures 5-11 like in Profiling and Optimizing Deep Neural Networks with DLProf and PyProf | NVIDIA Developer Blog, and slides 18 and 24 in https://tigress-web.princeton.edu/~jdh4/how_to_profile_with_dlprof_may_2021.pdf

I followed the steps in this tutorial: TensorFlow Profiler: Profile model performance  |  TensorBoard

However, I don’t see the “DLProf” option anywhere in Tensorboard

It’s because I need to install DLprof, right?

I tried

pip3 install nvidia-pyindex
pip3 install nvidia-dlprof
docker pull nvcr.io/nvidia/tensorflow:21.07-tf2-py3

But then when I try

 docker run --gpus all -it --rm --network=host -v $PWD:/workspace/tf2 nvcr.io/nvidia/tensorflow:21.07-tf2-py3

I get an error that says

ERROR: This container was built for NVIDIA Driver Release 470.42 or later, but version 455.32.00 was detected and compatibility mode is UNAVAILABLE

If I try dlprof like on slide 16 of that tigress-web.princeton.edu link

dlprof --mode=tensorflow2 --key_node=sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits --iter_start=100 --iter_stop=200 python3 myPython.py

it is able to generate .sqlite and .qdrep files, but when I try dlprofviewer nsys_profile.sqlite, I get an error saying

the DLProf database is version 'UNKNOWN` BUT DLProf is version '1.3.0'. Please downgrade DLProf to match the database version

can anyone help?

Hi,
Please refer to below links related custom plugin implementation and sample:
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMnistCoordConvAC

While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.

Thanks!

that link doesn’t work

Hi @asfadsfsadfadsf,

This doesn’t look like a TensorRT issue, Request you to post your concern on related forum to get better help.
https://forums.developer.nvidia.com/

Thank you.