Problem converting ONNX model to TensorRT Engine for SSD Mobilenet V2

Hi everyone,
I have trained SSD Mobilenet v2 model on my dataset. It got successfully converted to ONNX but, during converting ONNX model to TensorRT engine, it throws error due to unsupported datatype UINT8.
Is their any work around to generate tensorrt engine? If I have to retrain the model with supported datatype, how to change datatype of model from uint8 to supported one?

TensorRT version: 7.1

Thank You

because TRT cannot support UINT8 datatype. It means your model already used the uint8 datatype
https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/FoundationalTypes/DataType.html

Can I change UINT8 to INT32 in my model?

Hi,

What kind of frameworks do you use for training SSD Mobilenet v2?
If it is TensorFlow, would you mind to give this GitHub a try?

Thanks.

Hi @AastaLLL,

After executing main.py from suggested Github repository, I got the following error:

OSError: libnvinfer.so.5: cannot open shared object file: No such file or directory

My tensorrt version is 7.1

Hi,

The FlattenConcat is by default installed in the TensorRT 7.1.
So please comment the plugin library link here to skip the error:

Thanks.

Hi @AastaLLL,

Thank you for reply.

This Github Repo. worked for me for converting uff file to .bin engine file.
Still I am unable to convert onnx file to TensorRT engine file due to UINT8 datatype used in SSD MobileNet V2. Is there any way to generate engine file from onnx file for SSD MobileNet V2 by changing datatype from UINT8 to another one?

Hi,

UINT8 isn’t supported by TensorRT.

But we don’t meet this issue for SSD Mobilenet v2 before.
May I know how do you train and serialize your model?

Thanks.

Hi @AastaLLL,

I have trained the model on my desktop with Tensorflow-1.15 and generated the frozen graph. Then I transferred this frozen graph to the Jetson nano and converted it to Onnx model using following command:

python3 -m tf2onnx.convert --saved-model saved_model/ --output onnx_model.onnx --opset 11 --fold_const

And then converted ONNX to TensorRT engine through following code:

def build_engine(onnx_path, shape = [1,300,300,3]):

   """
   This is the function to create the TensorRT engine
   Args:
      onnx_path : Path to onnx_file. 
      shape : Shape of the input of the ONNX file. 
  """
   with trt.Builder(TRT_LOGGER) as builder, builder.create_network(1) as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
       builder.max_workspace_size = (256 << 20)
       with open(onnx_path, 'rb') as model:
           parser.parse(model.read())
       network.get_input(0).shape = shape
       engine = builder.build_cuda_engine(network)
       return engine

def save_engine(engine, file_name):
   buf = engine.serialize()
   with open(file_name, 'wb') as f:
       f.write(buf)
def load_engine(trt_runtime, plan_path):
   with open(engine_path, 'rb') as f:
       engine_data = f.read()
   engine = trt_runtime.deserialize_cuda_engine(engine_data)
   return engine

Hi,

Would you mind to share the pb and onnx file with us so we can check it?
If you are not comfortable to share it public, please pass the link via direct message.

Thanks.