Convert YOLOV3 to TF- UFF - tennsorRT on Jetson Xavier

I am trying to convert YOLOv3 model to UFF, to run on jetson Xavier using TensorRT, but I need support with the steps involved. Note that I have Jetpack 4.1 setup on Jetson Xavier that has TensorRT 5.0.3 on it.

Here is the steps that I performed so far:

  1. From the original network, I froze the model to get the .pb file. Done on the Xavier using the following command :
    python3 hdf5_to_protobuf.py --input_model="./model_data/yolo.h5" --output_model="./yolov3.pb"

  2. Convert the .pb file to a UFF file using the convert-to-uff.py command from TensorRT package. The resulting UFF file is still platform independent, so this step can also be done on Xavier:
    convert-to-uff --input-file yolov3.pb -O NMS -p config.py
    When executing this command with NMS option the converted file is so small (around 450kb), however when running the command :
    convert-to-uff --input-file yolov3.pb -p config.py
    the same size of file is obtained from the .pb file

  3. The last step should include building a TensorRT engine from the UFF file that I obtained. Based on my understanding, this step must be done on the Xavier itself, since the resulting serialized engine is platform specific. How to proceed at this stage?

below I am including the scripts that I am passing to uff converter to obtain the uff file:

Config.py :

import graphsurgeon as gs
import tensorflow as tf

Input = gs.create_node(“Input”,
op=“Placeholder”,
dtype=tf.float32,
shape=[1, 3, 300, 300])
PriorBox = gs.create_plugin_node(name=“GridAnchor”, op=“GridAnchor_TRT”,
numLayers=6,
minSize=0.2,
maxSize=0.95,
aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
variance=[0.1,0.1,0.2,0.2],
featureMapShapes=[19, 10, 5, 3, 2, 1])
NMS = gs.create_plugin_node(name=“NMS”, op=“NMS_TRT”,
shareLocation=1,
varianceEncodedInTarget=0,
backgroundLabelId=0,
confidenceThreshold=1e-8,
nmsThreshold=0.6,
topK=100,
keepTopK=100,
numClasses=91,
inputOrder=[0, 2, 1],
confSigmoid=1,
isNormalized=1,
scoreConverter=“SIGMOID”)
concat_priorbox = gs.create_node(name=“concat_priorbox”, op=“ConcatV2”, dtype=tf.float32, axis=2)
concat_box_loc = gs.create_plugin_node(“concat_box_loc”, op=“FlattenConcat_TRT”, dtype=tf.float32, axis=1, ignoreBatch=0)
concat_box_conf = gs.create_plugin_node(“concat_box_conf”, op=“FlattenConcat_TRT”, dtype=tf.float32, axis=1, ignoreBatch=0)

namespace_plugin_map = {
“MultipleGridAnchorGenerator”: PriorBox,
“Postprocessor”: NMS,
“Preprocessor”: Input,
“ToFloat”: Input,
“image_tensor”: Input,
“MultipleGridAnchorGenerator/Concatenate”: concat_priorbox,
“concat”: concat_box_loc,
“concat_1”: concat_box_conf
}

def preprocess(dynamic_graph):
# Now create a new graph by collapsing namespaces
dynamic_graph.collapse_namespaces(namespace_plugin_map)
# Remove the outputs, so we just have a single output node (NMS).
dynamic_graph.remove(dynamic_graph.graph_outputs, remove_exclusive_dependencies=False)

What are the minimum changes that I have to make in order to get YOLOv3 running on the Xavier?

I am also tyring to implement the deepstream implementation in a seperate project hoping to get similar results since they did almost the same conversion but the code is crashing when I try to build the nbgstiva-app source but this I will leave it for a different thread

could you please help?

Hi,

It’s recommended to use trt-yolo-app of this tutorial since YOLO requires some plugin implementation.
https://github.com/vat-nvidia/deepstream-plugins

Currently, TensorRT Python API is not available for Jetson yet.
This limitation will avoid you to run a plugin layer with python implementation.

Another alternative is to use TF-TRT.
You can check the tutorial here: https://github.com/NVIDIA-AI-IOT/tf_trt_models
If the YOLO .pb file is able to run on TensorFlow, you can switch to use TF-TRT.

However, the simplest way should still be trt-yolo-app.

Thanks.

Hi AastaLL,

I know you are trying to answer me on the other thread about trt-yolo-app, however I am replying also here jut to align what I did so far with your responses, this is the error that I am getting now when executing the repo

nvidia@jetson:~deepstream-plugins$ trt-yolo-app --flagfile=/home/nvidia/Projects/deepstream-plugins/config/deepstream-app_yolo_config.txt
Using previously generated plan file located at sources/lib/models/yolov3-kFLOAT-batch1.engine
Loading TRT Engine...
ERROR: The engine plan file is incompatible with this version of TensorRT, expecting 5.0.3.2got 5.0.2.6, please rebuild.
Loading Complete!
trt-yolo-app: yolo.cpp:100: Yolo::Yolo(uint): Assertion `m_Engine != nullptr' failed.
Aborted (core dumped)

I know that my jetson Xavier has 5.0.3 TensorRT version installed on it, but from where the 5.0.2.6 build is coming, is it the original build provided by the repo and how can I change it?

However, I didn’t try to implement https://github.com/NVIDIA-AI-IOT/tf_trt_models yet, but I will wait for your response first regarding the trt-yolo-app.

Thank you so much for your support.

Regards,
Raed

Hi,

Please check the original topic for the suggestion:
https://devtalk.nvidia.com/default/topic/1043648/jetson-tx2/building-nvyolo-plugin/post/5309040/#5309040

Thanks.