Does Jetson Xavier NX 16 GB support sparse tensor?

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

I have Pytorch model and I pruned my model. The number of weights after pruning is same as before pruning, but there are many zero weights (unstructured pruning). I converted .pt model to .onnx and .engine model. I want to know: Does Jetson Xavier NX 16 GB support sparse tensor as in my case to accelerate calculation?

I read the blog Accelerating Inference with Sparsity Using the NVIDIA Ampere Architecture and NVIDIA TensorRT | NVIDIA Technical Blog, it seems that A100 with Ampere architeture support sparse tensor.
Thanks.

2

Hi,

It should be supported. However, we are moving this post to the Jetson Xavier NX forum to get better help.
For more info please refer,

Thank you.

1 Like
3

@spolisetty
What do we need to do to Jetson NX support sparse tensors?
I mean that, I have model weights with many zeros weights and I do nothing, Jetson NX can speed up inference. Or do i need convert (save) weights in the format of sparse tensor to Jetson NX can speed up inference?

7

Hi,

How do you convert the model into TensorRT engine?
If trtexec binary is used, please try it with --sparsity flag:

Thanks.

1 Like
8

@AastaLLL
Thanks. I used Tensorrt Python API. How can set sparsity flag when using Tensorrt Python API.

Could you tell me more detailed what happen when setting flag sparsity?

9

Hi,

This can be set with below API:
https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Core/BuilderConfig.html?highlight=sparse#tensorrt.BuilderFlag

The more info about sparsity can be found below:

Thanks.

10

Thanks for the information.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.