jetson-inference with custom model

stefano.maccarana · October 8, 2019, 3:47pm

Hello,
I have my tensorflow model .pb (MobileNetV2) for classification and I want to use it with my Jetson Nano.
From jetson-inference samples I dint’t understand how to do it.

Can someone please help me?

Thanks
Stefano

AastaLLL · October 14, 2019, 3:49am

Hi,

Do you want to convert the .pb file into TensorRT engine, and inference it with pure TensorRT?
If yes, please check this tutorial for the steps in detail:
[url]https://github.com/AastaNV/TRT_object_detection[/url]

Thanks.

stefano.maccarana · October 14, 2019, 6:53am

Yes, I trained a classificator based on MobilenetV2 with kersa, then converted it in tensorflow .pb file.

Now I want to do inference on my Jetson Nano in C/C++.

I have to do the same for a SSD MobilnetV2?

Thanks

AastaLLL · October 25, 2019, 7:09am

YES.

You can follow the GitHub shared in comment#2 directly.

stefano.maccarana · October 25, 2019, 12:16pm

Thanks, but I’m encountering some problem…

calling python3 main.py ./image1.jpg

I get:

2019-10-25 14:09:57.359005: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /usr/lib/python3.6/dist-packages/uff/converters/tensorflow/conversion_helpers.py:18: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.

WARNING:tensorflow:From /usr/lib/python3.6/dist-packages/graphsurgeon/_utils.py:2: The name tf.NodeDef is deprecated. Please use tf.compat.v1.NodeDef instead.

WARNING:tensorflow:From /usr/lib/python3.6/dist-packages/graphsurgeon/StaticGraph.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING: To create TensorRT plugin nodes, please use the `create_plugin_node` function instead.
NOTE: UFF has been tested with TensorFlow 1.12.0. Other versions are not guaranteed to work
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.3
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 300
      }
      dim {
        size: 300
      }
    }
  }
}
]
=========================================

Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
WARNING:tensorflow:From /usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:179: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

Warning: No conversion function registered for layer: GridAnchor_TRT yet.
Converting GridAnchor as custom op: GridAnchor_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_loc as custom op: FlattenConcat_TRT
Traceback (most recent call last):
  File "main.py", line 31, in <module>
    uff_model = uff.from_tensorflow(dynamic_graph.as_graph_def(), model.output_name, output_filename='tmp.uff')
  File "/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 181, in from_tensorflow
    debug_mode=debug_mode)
  File "/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py", line 94, in convert_tf2uff_graph
    uff_graph, input_replacements, debug_mode=debug_mode)
  File "/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py", line 79, in convert_tf2uff_node
    op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes, debug_mode=debug_mode)
  File "/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py", line 47, in convert_layer
    return cls.registry_[op](name, tf_node, inputs, uff_graph, **kwargs)
  File "/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter_functions.py", line 21, in convert_placeholder
    dtype = tf2uff.convert_tf2numpy_dtype(tf_node.attr['dtype'].type)
  File "/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py", line 103, in convert_tf2numpy_dtype
    return tf.as_dtype(dtype).as_numpy_dtype
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py", line 717, in as_dtype
    (type_value,))
TypeError: Cannot convert value 0 to a TensorFlow DType.

both with MobilenetV1 and V2

AastaLLL · November 5, 2019, 8:19am

Hi,

Please remember to update the graphsurgeon converter which mentioned here:
[url]GitHub - AastaNV/TRT_object_detection: Python sample for referencing object detection model with TensorRT

Thanks.

stefano.maccarana · November 5, 2019, 3:44pm

Thanks. I solved [with coco ssd mnv1 and mnv2] updating graphsurgeon converter.

Now, with my SSD MobileNetV2 model I’m getting this error:

[TensorRT] INFO: Detected 1 input and 2 output network tensors.
python3: nmsPlugin.cpp:139: virtual void nvinfer1::plugin::DetectionOutput::configureWithFormat(const nvinfer1::Dims*, int, const nvinfer1::Dims*, int, nvinfer1::DataType, nvinfer1::PluginFormat, int): Assertion `numPriors * numLocClasses * 4 == inputDims[param.inputOrder[0]].d[0]' failed.
Aborted (core dumped)

In order to try to convert my model I created model_ssd_mobilenet_v2_custom.py (modifying model_ssd_mobilenet_v2_coco_2018_03_29.py) with:

dims = [1,150,150]

(I used 1 channel images for training)

PriorBox = gs.create_plugin_node(
        name="GridAnchor",
        op="GridAnchor_TRT",
        minSize=0.1,   ---> instead of 0.2
        maxSize=0.95,
        aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
        variance=[0.1,0.1,0.2,0.2],
        featureMapShapes=[19, 10, 5, 3, 2, 1],
        numLayers=6
    )

and, since I have 9 classes, I put 10 as numClasses

NMS = gs.create_plugin_node(
        name="NMS",
        op="NMS_TRT",
        shareLocation=1,
        varianceEncodedInTarget=0,
        backgroundLabelId=0,
        confidenceThreshold=1e-8,
        nmsThreshold=0.6,
        topK=100,
        keepTopK=100,
        numClasses=10,
        inputOrder=[1, 0, 2],
        confSigmoid=1,
        isNormalized=1
    )

The tmp.uff file is generated but I’m not able to use it.

I’ve uploaded here custom.zip - Google Drive my TF model and my modified scripts.

Can you please help me?
Thanks.

AastaLLL · November 19, 2019, 8:24am

Hi,

We are checking your model internally.
Will update information with you later.

Thanks.

stefano.maccarana · November 19, 2019, 8:29am

Thank you very much

AastaLLL · December 6, 2019, 8:04am

Hi,

Sorry for keeping you waiting.

We recently found that the dimension of model output is beyond our expectation.
Not sure if this is related to a customized input dimension.

Your model

numPriors : 1917
loc_data  : 2160 1 1
conf_data : 5400 1 1
prior_data: 2 7668 1

However, we expect the dimension follow this rule:

loc_data  : [numPriors*numLocClasses*4, 1, 1]
conf_data : [numPriors*numClasses, 1, 1]
prior_data: [2, numPriors*4, 1]

Did you change the prior number or update other parameter?
Is this output dimension under your expectation?

Thanks.

Topic		Replies	Views
Converting Tensorflow .pb models to TensorRT Models to run on the Nano Jetson Nano tensorrt , tensorflow	9	905	October 18, 2021
[ASK] How to make tensor RT engine from frozen graph tensor flow? Jetson Nano	14	2088	October 14, 2021
Problem with SSD Mobilenet TensorRT conversion Jetson Nano tensorrt	7	962	October 15, 2021
Tensorflow model to tensorrf uff model convert error Jetson TX2	10	3780	October 18, 2021
how to write config.py for converting ssd-mobilenetv2 to uff format Jetson Nano	19	6877	October 14, 2021
Getting error while converting custom model using faster rcnn resnet 50 to tensor rt engine using tensor rt 5.0 TensorRT	17	3466	September 26, 2020
Object Detection in Jetson Nano Jetson Nano	8	2677	October 18, 2021
Problems with SSD Mobilenet v2 UFF Jetson Nano ssd	35	7934	October 18, 2021
Convert SSD-Mobilenet to UFF Jetson Nano	13	1828	October 14, 2021
Tensorflow not running, shows low memory warning ! and program stops Jetson Nano	2	765	November 8, 2019

jetson-inference with custom model

Related topics