TensorRT getBindingVectorizedDim API

Description

I just would like to know more details about the getBindingVectorized() function of ICudaEngine class. Not quite understand about the description in the C++ code “Return the dimension index that the buffer is vectorized.”. I also experimented with it a little bit, and it always returns -1. Thanks.
A clear and concise description of the bug or issue.

Environment

TensorRT Version: 7.2.3.4
GPU Type: RTX3070
Nvidia Driver Version: 465.27
CUDA Version: 11.2
CUDNN Version: 8.1
Operating System + Version: Windows 10
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

Hi,
Please check the below link, as they might answer your concerns
https://docs.nvidia.com/deeplearning/tensorrt/api/index.html
Thanks!

Hi NVES, Thanks for your reply, I actually took a look at that page and C++ API docs before I posted this question because I didn’t find any specific information regarding getBindingVectorizedDim API other than “Return the dimension index that the buffer is vectorized.”. Can you please elaborate more on this or give me an example to illustrate? Thanks a lot!

This is for tensorrt vectorized layout, it is always -1 for linear layout. Please refer,
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#reformat-free-support

Thanks a lot! That totally makes sense.

1 Like