ssd-mobile-net-v3 to tensorRT

can i convert ssd-mobilenet-v3-small/ssd-mobilenet-v3-large (https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md) pretrained tensorflow model to TensorRT Model.

i tried using this link(GitHub - NVIDIA-AI-IOT/tf_trt_models: TensorFlow models accelerated with NVIDIA TensorRT).but got error .

def global_pool(input_tensor, pool_op=tf.compat.v2.nn.avg_pool2d): AttributeError: module ‘tensorflow._api.v1.compat’ has no attribute ‘v2’

but it successfully work for ssd_mobilenet_v1_coco,ssd_inception_v2_coco

Hi,

Try converting your model to ONNX instead using tf2onnx and then convert to TensorRT using ONNX parser. Any layer that are not supported needs to be replaced by custom plugin.
https://github.com/onnx/tensorflow-onnx
https://github.com/onnx/onnx-tensorrt/blob/master/operators.md

You can also convert the model to TRT using TF-TRT and serialize it to a .plan file.
Save model file using TF-TRT 2.0
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#worflow-with-savedmodel

Thanks

1 Like

Hi,

I converted successfully ssd-mobilenet-v3-large to tensorrt uff file. You can try it. Hope it will help you.

import graphsurgeon as gs
import tensorflow as tf

Input = gs.create_node("Input",
    op="Placeholder",
    dtype=tf.float32,
    shape=[1, 3, 320, 320])

PriorBox = gs.create_plugin_node(name="GridAnchor", op="GridAnchor_TRT",
    numLayers=6,
    minSize=0.2,
    maxSize=0.95,
    aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
    variance=[0.1,0.1,0.2,0.2],
    featureMapShapes=[20, 10, 5, 3, 2, 1])

NMS = gs.create_plugin_node(name="NMS", op="NMS_TRT",
    shareLocation=1,
    varianceEncodedInTarget=0,
    backgroundLabelId=0,
    confidenceThreshold=1e-8,
    nmsThreshold=0.6,
    topK=100,
    keepTopK=100,
    numClasses=3,
    inputOrder=[0, 2, 1],
    confSigmoid=1,
    isNormalized=1)

concat_priorbox = gs.create_node(name="concat_priorbox", op="ConcatV2", dtype=tf.float32, axis=2)
concat_box_loc = gs.create_plugin_node("concat_box_loc", op="FlattenConcat_TRT", dtype=tf.float32, axis=1, ignoreBatch=0)
concat_box_conf = gs.create_plugin_node("concat_box_conf", op="FlattenConcat_TRT", dtype=tf.float32, axis=1, ignoreBatch=0)

namespace_plugin_map = {
    "MultipleGridAnchorGenerator": PriorBox,
    "Postprocessor": NMS,
    "Preprocessor": Input,
    "ToFloat": Input,
    "image_tensor": Input,
    "Concatenate": concat_priorbox,
    "concat": concat_box_loc,
    "concat_1": concat_box_conf
}

def preprocess(dynamic_graph):
    # Now create a new graph by collapsing namespaces
    dynamic_graph.collapse_namespaces(namespace_plugin_map)
    # Remove the outputs, so we just have a single output node (NMS).
    dynamic_graph.remove(dynamic_graph.graph_outputs, remove_exclusive_dependencies=False)
    # Disconnect the Input node from NMS, as it expects to have only 3 inputs.
    dynamic_graph.find_nodes_by_op("NMS_TRT")[0].input.remove("Input")`Preformatted text`

convert-to-uff --input-file frozen_inference_graph.pb -O NMS -p config.py

If you have issue with Cast operation, checkout tensorflow models to branch 1.12.0 (from 1.13.0, tensorflow will Cast operation automatically to trained model) , add some SSD Mobilenet V3 stuffs, then export frozen model and convert it to uff file.

2 Likes

Hello @michaelnguyen1195
I have two questions
1: I have trained the ssd-mobilenet-v1 on a custom dataset
how can I convert that model to a TensorRT engine ?!
2:I have converted the yolov5 model to onnx and convert the .onnx model to tensorRT engine (.trt file),
now
how can I inference(make prediction, draw bounding box) ?!