I am trying to convert YOLOv3 model to UFF, to run on jetson Xavier using TensorRT, but I need support with the steps involved. Note that I have Jetpack 4.1 setup on Jetson Xavier that has TensorRT 5.0.3 on it.
Here is the steps that I performed so far:
-
From the original network, I froze the model to get the .pb file. Done on the Xavier using the following command :
python3 hdf5_to_protobuf.py --input_model=“./model_data/yolo.h5” --output_model=“./yolov3.pb” -
Convert the .pb file to a UFF file using the convert-to-uff.py command from TensorRT package. The resulting UFF file is still platform independent, so this step can also be done on Xavier:
convert-to-uff --input-file yolov3.pb -O NMS -p config.py
When executing this command with NMS option the converted file is so small (around 450kb), however when running the command :
convert-to-uff --input-file yolov3.pb -p config.py
the same size of file is obtained from the .pb file -
The last step should include building a TensorRT engine from the UFF file that I obtained. Based on my understanding, this step must be done on the Xavier itself, since the resulting serialized engine is platform specific. How to proceed at this stage?
below I am including the scripts that I am passing to uff converter to obtain the uff file:
Config.py :
import graphsurgeon as gs
import tensorflow as tf
Input = gs.create_node(“Input”,
op=“Placeholder”,
dtype=tf.float32,
shape=[1, 3, 300, 300])
PriorBox = gs.create_plugin_node(name=“GridAnchor”, op=“GridAnchor_TRT”,
numLayers=6,
minSize=0.2,
maxSize=0.95,
aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
variance=[0.1,0.1,0.2,0.2],
featureMapShapes=[19, 10, 5, 3, 2, 1])
NMS = gs.create_plugin_node(name=“NMS”, op=“NMS_TRT”,
shareLocation=1,
varianceEncodedInTarget=0,
backgroundLabelId=0,
confidenceThreshold=1e-8,
nmsThreshold=0.6,
topK=100,
keepTopK=100,
numClasses=91,
inputOrder=[0, 2, 1],
confSigmoid=1,
isNormalized=1,
scoreConverter=“SIGMOID”)
concat_priorbox = gs.create_node(name=“concat_priorbox”, op=“ConcatV2”, dtype=tf.float32, axis=2)
concat_box_loc = gs.create_plugin_node(“concat_box_loc”, op=“FlattenConcat_TRT”, dtype=tf.float32, axis=1, ignoreBatch=0)
concat_box_conf = gs.create_plugin_node(“concat_box_conf”, op=“FlattenConcat_TRT”, dtype=tf.float32, axis=1, ignoreBatch=0)
namespace_plugin_map = {
“MultipleGridAnchorGenerator”: PriorBox,
“Postprocessor”: NMS,
“Preprocessor”: Input,
“ToFloat”: Input,
“image_tensor”: Input,
“MultipleGridAnchorGenerator/Concatenate”: concat_priorbox,
“concat”: concat_box_loc,
“concat_1”: concat_box_conf
}
def preprocess(dynamic_graph):
# Now create a new graph by collapsing namespaces
dynamic_graph.collapse_namespaces(namespace_plugin_map)
# Remove the outputs, so we just have a single output node (NMS).
dynamic_graph.remove(dynamic_graph.graph_outputs, remove_exclusive_dependencies=False)
What are the minimum changes that I have to make in order to get YOLOv3 running on the Xavier?
I am also tyring to implement the deepstream implementation in a seperate project hoping to get similar results since they did almost the same conversion but the code is crashing when I try to build the nbgstiva-app source but this I will leave it for a different thread
could you please help?