TensorRT 7.0.0.11 TRT_LOGGER Error

AI & Data Science Deep Learning (Training & Inference)
,
1

Hi,

I’m trying to run an Onnx model in tensorRT and one error occured on creating the engine.

Error:

TypeError Traceback (most recent call last)
in
1 TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
----> 2 trt_runtime = trt.Runtime(TRT_LOGGER)
3
4 def build_engine(onnx_path):
5

TypeError: pybind11::init(): factory function returned nullptr

Code:

import argparse
from onnx import ModelProto
import tensorrt as trt

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt_runtime = trt.Runtime(TRT_LOGGER)

def build_engine(onnx_path):

with trt.Builder(TRT_LOGGER) as builder, builder.create_network(1) as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
    builder.max_workspace_size = (256 << 20)
    with open(onnx_path, 'rb') as model:
        parser.parse(model.read())
    network.get_input(0).shape = shape
    engine = builder.build_cuda_engine(network)
    return engine

def save_engine(engine, file_name):
buf = engine.serialize()
with open(file_name, ‘wb’) as f:
f.write(buf)
def load_engine(trt_runtime, plan_path):
with open(engine_path, ‘rb’) as f:
engine_data = f.read()
engine = trt_runtime.deserialize_cuda_engine(engine_data)
return engine

engine_name = ‘retinaface.plan’
onnx_path = “FaceDetector.onnx”
batch_size = 1

model = ModelProto()
with open(onnx_path, “rb”) as f:
model.ParseFromString(f.read())

d0 = model.graph.input[0].type.tensor_type.shape.dim[1].dim_value
d1 = model.graph.input[0].type.tensor_type.shape.dim[2].dim_value
d2 = model.graph.input[0].type.tensor_type.shape.dim[3].dim_value
shape = [batch_size , d0, d1 ,d2]
engine = build_engine(onnx_path, shape= shape)
save_engine(engine, engine_name)

Machine:

IBM Power AI AC922 on Docker “ibmcom/powerai:1.7.0-all-ubuntu18.04-py37” - linux/ppc64le

Conda 4.8.1
Python 3.7
Ubuntu18.04 - ppc64le
GPU Nvidia V100
Cuda 10.2
cuDNN 7.6.5
ONNX 1.6.0
TensorRT 7.0.0.11

Does anyone know how to fix this error?

Cheers,
Kevin