[TensorRT] ERROR: Unsupported data format, expect 5D numpy, List of NCHWs, List of CHWs, List of List of CHWs Single NCHW or Single CHW

Hi all, the situation is this, I used a gluoncv model and converted it to an onnx file, from onnx file I created an engine file that is TensorRT optimised. The error arises when I try to make an inference with the engine by inputting an example image to test.

When I converted the gluoncv model to an onnx file, I set the input shape = (1, 32, 32, 3)
This is the link to the model: https://gluon-cv.mxnet.io/build/examples_classification/demo_cifar10.html#

I built the engine by following the instructions for importing from onnx using python with this link:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#build_engine_python

Subsequently I tried to make an inference by loading the test image into the engine model. The test image was transform based on the gluoncv example requirements, it is here where the error of unsupported data format arises. I am unsure of what to edit/adjust/transform the image so that it suits the engine requirements. Pls help! Am unsure if error arises in the onnx file or the engine file. Thanks!

PS. I am using Jetson Tx2 to run the inference

Hi,

Can you provide the following information so we can better help?
Provide details on the platforms you are using:
o Linux distro and version
o Nvidia driver version
o CUDA version
o CUDNN version
o Python version [if using python]
o Tensorflow version
o TensorRT version

Also, if possible please share the script to reproduce the issue.

Thanks

Thank you for replying, I am using Jetson Tx2 with jetpack 4.3. Below are additional details.
Linux distro and version:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION=“Ubuntu 18.04.4 LTS”
NAME=“Ubuntu”
VERSION=“18.04.4 LTS (Bionic Beaver)”

Nvidia driver version

CUDA version:
10.0.326

cudNN version:
7.6.3

Python 3.6.9

TensorRT 6.0.1

Mxnet: 1.6.0

Gluoncv: 0.7.0

Code (to create onnx file):
‘’’
import mxnet as mx
import numpy as np
from mxnet.contrib import onnx as onnx_mxnet
import logging
import gluoncv as gcv
from gluoncv.utils import export_block

logging.basicConfig(level=logging.INFO)

net = gcv.model_zoo.get_model(‘cifar_resnet110_v1’, classes=10, pretrained=True)
export_block(‘cifar_resnet110_v1’, net, preprocess=True)

import glob
print(glob.glob(’.json’) + glob.glob(’.params’))

We will use the downloaded pre-trained model files (sym, params) and define input variables.

Downloaded input symbol and params files

sym = ‘./cifar_resnet110_v1-symbol.json’
params = ‘./cifar_resnet110_v1-0000.params’

input_shape = (1, 32, 32, 3)

Path of the output file

onnx_file = ‘./cifar_resnet110_v1.onnx’

Invoke export model API. It returns path of the converted onnx model

converted_model_path = onnx_mxnet.export_model(sym, params, [input_shape], np.float32, onnx_file)

from onnx import checker
import onnx

Load onnx model

model_proto = onnx.load_model(converted_model_path)

Check if converted ONNX protobuf is valid

checker.check_graph(model_proto.graph)
‘’’

Code (to create tensorrt engine file):
‘’’
import tensorrt.legacy as trt
import numpy as np
import time
from mxnet import gluon, nd, image
from mxnet.gluon.data.vision import transforms
from gluoncv import utils
import glob

model_path = ‘/home/sapphire/Desktop/cifar_resnet110_v1.onnx’

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

def build_engine(model_path):
with trt.Builder(TRT_LOGGER) as builder,
builder.create_network() as network,
trt.OnnxParser(network, TRT_LOGGER) as parser:
builder.max_workspace_size = 1<<25
builder.max_batch_size = 1
with open(model_path, “rb”) as f:
parser.parse(f.read())
engine = builder.build_cuda_engine(network)
return engine

engine = build_engine(model_path)

trt.utils.write_engine_to_file(‘cifar_resnet110_v1.engine’, engine.serialize())
glob.glob("*.engine")

print(‘done’)
‘’’

Code (to make inference with engine):
‘’’
from tensorrt.legacy.lite import Engine
from tensorrt.legacy.infer import LogSeverity
import numpy as np
from mxnet import gluon, nd, image
from mxnet.gluon.data.vision import transforms
from gluoncv import utils
import tensorrt.legacy as trt
import pycuda.autoinit

TRT_LOGGER =trt.Logger(trt.Logger.WARNING)

url = ‘https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/classification/plane-draw.jpeg
im_fname = utils.download(url)
img = image.imread(im_fname)

def transform(img):
transform_fn = transforms.Compose([
transforms.Resize(32),
transforms.CenterCrop(32),
transforms.ToTensor(),
transforms.Normalize([0.4914, 0.4822, 0.4465], [0.2023, 0.1994, 0.2010])
])
return transform_fn(img)

engine = Engine(PLAN=‘cifar_resnet110_v1.engine’)

img = transform(img)

pred = engine.infer(img.expand_dims(axis=0))
print(pred)

class_names = [‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,
‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’]
ind = nd.argmax(pred, axis=1).astype(‘int’)
print(‘The input picture is classified as [%s], with probability %.3f.’% (class_names[ind.asscalar()], nd.softmax(pred)[0][ind].asscalar()))
‘’’

I am still trying to understand how to utilise tensorrt, so pls I would greatly appreciate any feedback and advice if I am using it wrongly. Thank you

Hi,

Can you try running the script on TRT 7?
Also, you can use “trtexec” command line tool to understand performance and possibly locate bottlenecks.
Please find the below links for your reference:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#trtexec
https://github.com/NVIDIA/TensorRT/blob/release/6.0/samples/opensource/trtexec/README.md

In case issue persist, please share the error log as well and ONNX model so we can help better.

Thanks