How do we know is Sparsity feature is enable on system?

As we know Sparsity was enabled on 3rd Generation of Tensor Cores, but how do we know is this feature be enable when we do any Inferencing or benchmarking?

Hi boon.hai.ng.

I think this question might be addressed better when asked in the TensorRT forums.

It would also be very helpful if you could describe with a bit more detail what kind of project you have in mind. That way people in this forum could suggest how to implement code paths that will make use of the sparsity features.

Thanks!

Hi Maskus,
Thanks to your advise here. Currently I’m going to install the dGPU A10 into my system, to run trtexec AI inference using Container. Based on data sheet (https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a10/pdf/a10-datasheet.pdf) shared for A10 dGPU card, it’s used 3rd Gen of tensor cores which has Sparsity feature enabled and it’s provided faster performance compared to previous Gen AI Inference. So, how do we validate the A10 dGPU card is running on Sparsity enable? How to difference is the Sparsity turn on or not on A10 dGPU card. Below is the sample result got from trtexec result output with batch size. please advise and thank you.

JETPACK 4.6
Batch Size || Latency (ms) || Throughput (QPS)
64 || 8.76 || 11,032.80
128 || 17.39 || 11,015.00

Hi,

Please refer following,