ONNX to TensorRT conversion

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello, I am trying to run from ONNX to tensorRT while doing that conversion I was getting this error context = engine.create_execution_context()

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

5

Hi Team - We were getting below error screenshots when we trying ONNX model to trt conversion. Please guide us how to proceed.

1.We are using Mask2formertransformer decoder in our panoptic segmentation model

  1. We are able to convert all the individual layers to trt but when we run as a whole we are getting error.

MagnaMask2Former

  1. MagnaMask2FormerPixelLevelModule (Torch2ONNX: Success / ONNX2TRT: Success)
  2. Mask2FormerTransformerModule (Torch2ONNX: Success / ONNX2TRT: Failed)
    a. Mask2FormerSinePositionEmbedding (Torch2ONNX: ??? / ONNX2TRT: ???)
    b. Mask2FormerMaskedAttentionDecoder (Torch2ONNX: ??? / ONNX2TRT: ???)
    i. Mask2FormerMaskedAttentionDecoderLayer (Torch2ONNX: ??? / ONNX2TRT: ???)
  3. Mask2FormerAttention (Torch2ONNX: ??? / ONNX2TRT: ???)
  4. MultiheadAttention (Torch2ONNX: ??? / ONNX2TRT: ???)
    ii. Mask2FormerMaskPredictor (Torch2ONNX: ??? / ONNX2TRT: ???)
  5. Mask2FormerMLPPredictionHead (Torch2ONNX: ??? / ONNX2TRT: ???)
    a. Mask2FormerPredictionBlock (Torch2ONNX: ??? / ONNX2TRT: ???)

MicrosoftTeams-image
MicrosoftTeams-image771×329 156 KB

6

Hi,

We recommend you open a new topic with the issue repro ONNX model and complete verbose logs.

Thank you.