Jetson Orin Nano (Tensor Cores utilisation in Tensorflow)

On my jetson orin nano how can i check via GUI or command line command to verify if tensor cores is been used during tf.float16 matrix-multiply operations?

Using Jetpack 5.1.2

Python 3.8.10

From pip3 freeze
tensorboard==2.12.3
tensorboard-data-server==0.7.2
tensorflow==2.12.0+nv23.6
tensorflow-cpu-aws==2.13.1
tensorflow-estimator==2.12.0
tensorflow-io-gcs-filesystem==0.35.0
tensorflow-probability==0.21.0
tensorrt==8.5.2.2
termcolor==2.4.0

code snippets
tf.keras.mixed_precision.set_global_policy(policy="mixed_float16");

tf16_var = tf.Variable(initial_value=np.ndarray(shape=(32,32),dtype=np.float16,buffer=np.random.uniform(low=0.174,high=0.453,size=32*32).reshape((32,32))),dtype=tf.float16);
tf16_var_1 = tf.Variable(initial_value=np.ndarray(shape=(32,64),dtype=np.float16,buffer=np.random.uniform(low=0.174,high=0.453,size=32*64).reshape((32,64))),dtype=tf.float16);


with tf.device(device_name="/GPU:0"):
    tf.matmul(a=tf16_var,b=tf16_var_1);
print("performed tensor16 on tensor_cores");

Hi,

Please try Nsight System which can profile Tensor Core utilization.

https://docs.nvidia.com/nsight-systems/UserGuide/index.html

Thanks.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.