Sparse convolution using tensorrt

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

image

As I understand sparsity inference using tensorrt we need following process.

  1. Use pytorch to find optimal pth(dense network) results after learning.
  2. Reproduce the pth file through sparse relearning using ASP in apex.
  3. Convert the reproduced pth to onnx.
  4. Convert onnx to tensorrt plan file (.trt).

./workspace/TensorRT/build/out/trtexec \
–onnx=/workspace/TensorRT/model/resnext101_32x8d_pyt_torchvision_sparse.onnx \ –saveEngine=/workspace/TensorRT/model/resnext101_engine.trt
–explicitBatch
–sparsity=enable
–fp16

  1. inference the plan file using tensorrt.

I have a question here.
Does tensorrt inference using 2-bit indices (shown in the figure above) information in addition to sparse matrix data?
I don’t think pth, onnx, trt files are structures containing 2 bit indices information. How can tensorrt use the 2 bit indices information which is shown in the picture above?

2

Hi,

Please refer to the following docs, hope they are helpful. Please let us know if your query is still not answered.

Thank you.

5

Thank you!

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.