Question about Batch Matmul


Hi, all. Hope you guys are doing well.
I’m a newbie in TensorRT and trying to learn how to add batch matmul in TensorRT.
Is there any example code that I can follow?
It seems like this function IMatrixMultiplyLayer is the one to use, but I’m not sure yet.

Thank you!!


TensorRT Version:
GPU Type: V100
Nvidia Driver Version:
CUDA Version: 11.1
CUDNN Version:
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered


Currently we do not have sample for IMatrixMultiplyLayer. Please refer Layers — NVIDIA TensorRT Standard Python API Documentation 8.2.0 documentation or TensorRT: nvinfer1::IMatrixMultiplyLayer Class Reference for more details.

Thank you.