Exporting Tensorflow models to Jetson Nano

harryxdllll · September 11, 2020, 2:01pm

Hi,

We are training different neural networks using Tensorflow Object Detection API on Tensorflow 1.15 and 2.3 with pretrained models. The whole process is running on the computer with Tesla GPU. After exporting the frozen inference graph, we are converting it to the UFF or ONNX format on computer with Tesla and then loading it into TensorRT on Jetson Nano. Unfortunatelly with UFF files we have config file only for SSD Mobilenet v2 and when we to load the ONNX graph to Jetson Nano, TensorRT says that it has unsupported data type (UINT8). We did not find any way to get the FP16, FP32 or INT8 precision, which would be supported on the Jetson.

My questions are (depending on the way we choose):

frozen_inference_graph.pb → UFF → TensorRT
Is there any actual tutorial/list of steps with apropriate config files on how to export the pretrained models into UFF
files? For example for ResNet 50 or 101. Those are being parsed correctly by TensorRT on the Jetson Nano.
frozen_inference_graph.pb → ONNX → TensorRT
How can we get the proper precision (other than UINT8) that would be supported on Jetson Nano with TensorRT?

adityabalpande66 · September 11, 2020, 3:36pm

This is quite interesting!
Could you please tell me a bit about this idea…

Andrey1984 · September 11, 2020, 3:56pm

I also have got two pb files, that I am just curious how to get to TRT format
Thanks

AastaLLL · September 14, 2020, 3:45am

Hi,

There are some config.py example in the TensorRT sample folder:

/usr/src/tensorrt/samples/sampleUffFasterRCNN/config.py
/usr/src/tensorrt/samples/sampleUffMaskRCNN/converted/config.py
/usr/src/tensorrt/samples/sampleUffSSD/config.py

The interface to map a .pb model into .uff is called graphsurgeon.
You can find the detail document here:
https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/graphsurgeon/graphsurgeon.html

For INT8 issue, please noticed that INT8 operation requires > 7.x GPU architecture.
Nano’s architecture is 5.3 which cannot support INT8 inference.

Thanks.

Elviron · September 16, 2020, 10:17am

For INT8 issue, please noticed that INT8 operation requires > 7.x GPU architecture.
Nano’s architecture is 5.3 which cannot support INT8 inference.

Ok, so Jetson Nano supports FP16 and FP32 only. But how to force that precision in the output ONNX model?

After exporting our trained model to frozen inference graph we are doing something like:

python3 -m tf2onnx.convert --saved-model model/saved_model/ --opset 11 --output model.onnx

And no matter what, we always get the UINT8 format, which is unsupported.

harryxdllll · September 21, 2020, 7:26am

@AastaLLL
Thanks for replay.
Ok so now we use convert-to-uff
python3 convert-to-uff frozen_ssd_mobilenet.pb -O NMS -p config.py (/usr/src/tensorrt/samples/sampleUffSSD/config.py)

Loading frozen_ssd_mobilenet.pb
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/uff/converters/tensorflow/conversion_helpers.py:274: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

NOTE: UFF has been tested with TensorFlow 1.15.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.9
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 300
      }
      dim {
        size: 300
      }
    }
  }
}
]
=========================================

Using output node NMS
Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:226: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_conf as custom op: FlattenConcat_TRT
Warning: No conversion function registered for layer: GridAnchor_TRT yet.
Converting GridAnchor as custom op: GridAnchor_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_loc as custom op: FlattenConcat_TRT
DEBUG [/usr/local/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:143] Marking ['NMS', 'NMS'] as outputs
No. nodes: 662
UFF Output written to frozen_ssd_mobilenet.uff

In next step we are trying to load model using jetson-inference but we get some errors:

[TRT]    TensorRT version 7.1.3
[TRT]    loading NVIDIA plugins...
[TRT]    Registered plugin creator - ::GridAnchor_TRT version 1
[TRT]    Registered plugin creator - ::NMS_TRT version 1
[TRT]    Registered plugin creator - ::Reorg_TRT version 1
[TRT]    Registered plugin creator - ::Region_TRT version 1
[TRT]    Registered plugin creator - ::Clip_TRT version 1
[TRT]    Registered plugin creator - ::LReLU_TRT version 1
[TRT]    Registered plugin creator - ::PriorBox_TRT version 1
[TRT]    Registered plugin creator - ::Normalize_TRT version 1
[TRT]    Registered plugin creator - ::RPROI_TRT version 1
[TRT]    Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT]    Could not register plugin creator -  ::FlattenConcat_TRT version 1
[TRT]    Registered plugin creator - ::CropAndResize version 1
[TRT]    Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT]    Registered plugin creator - ::Proposal version 1
[TRT]    Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT]    Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT]    Registered plugin creator - ::Split version 1
[TRT]    Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT]    detected model format - UFF  (extension '.uff')
[TRT]    desired precision specified for GPU: FP16
[TRT]    native precisions detected for GPU:  FP32, FP16
[TRT]    attempting to open engine cache file /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.7103.GPU.FP16.engine
[TRT]    cache file not found, profiling network model on device GPU
[TRT]    device GPU, loading /home/administrator/Desktop/LinuxAPP/ /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT]    UFFParser: Parsing Input[Op: Input].
[TRT]    UFFParser: Input -> [3,300,300]
[TRT]    UFFParser: Applying order forwarding to: Input
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/weights[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/weights -> [3,3,3,32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/weights
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/Conv2D[Op: Conv]. Inputs: Input, FeatureExtractor/MobilenetV2/Conv/weights
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/Conv2D -> [32,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/Conv2D
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3[Op: BatchNorm]. Inputs: FeatureExtractor/MobilenetV2/Conv/Conv2D, FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma, FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta, FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean, FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3 -> [32,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/Relu6[Op: Activation]. Inputs: FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3
[TRT]    Setting dynamic range for (Unnamed Layer* 7) [Activation]_output to [0,6]
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/Conv/Relu6 -> [32,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/Relu6
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise_weights[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise_weights -> [3,3,32,1]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise_weights
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise[Op: Conv]. Inputs: FeatureExtractor/MobilenetV2/Conv/Relu6, FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise_weights
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise -> [32,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/gamma[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/gamma -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/gamma
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/beta[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/beta -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/beta
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_mean[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_mean -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_mean
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_variance[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_variance -> [32]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_variance
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/FusedBatchNormV3[Op: BatchNorm]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/depthwise, FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/gamma, FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/beta, FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_mean, FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/moving_variance
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/FusedBatchNormV3 -> [32,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/FusedBatchNormV3
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/depthwise/Relu6[Op: Activation]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/FusedBatchNormV3
[TRT]    Setting dynamic range for (Unnamed Layer* 15) [Activation]_output to [0,6]
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/Relu6 -> [32,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/Relu6
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/project/weights[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/project/weights -> [1,1,32,16]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/project/weights
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/project/Conv2D[Op: Conv]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv/depthwise/Relu6, FeatureExtractor/MobilenetV2/expanded_conv/project/weights
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/project/Conv2D -> [16,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/project/Conv2D
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/gamma[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/gamma -> [16]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/gamma
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/beta[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/beta -> [16]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/beta
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_mean[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_mean -> [16]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_mean
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_variance[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_variance -> [16]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_variance
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/FusedBatchNormV3[Op: BatchNorm]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv/project/Conv2D, FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/gamma, FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/beta, FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_mean, FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/moving_variance
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/FusedBatchNormV3 -> [16,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/FusedBatchNormV3
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv/output[Op: Identity]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/FusedBatchNormV3
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv/output -> [16,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv/output
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/weights[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/weights -> [1,1,16,96]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/weights
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/Conv2D[Op: Conv]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv/output, FeatureExtractor/MobilenetV2/expanded_conv_1/expand/weights
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/Conv2D -> [96,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/Conv2D
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/gamma[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/gamma -> [96]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/gamma
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/beta[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/beta -> [96]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/beta
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_mean[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_mean -> [96]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_mean
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_variance[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_variance -> [96]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_variance
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/FusedBatchNormV3[Op: BatchNorm]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/Conv2D, FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/gamma, FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/beta, FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_mean, FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/moving_variance
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/FusedBatchNormV3 -> [96,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/FusedBatchNormV3
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/expand/Relu6[Op: Activation]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/FusedBatchNormV3
[TRT]    Setting dynamic range for (Unnamed Layer* 31) [Activation]_output to [0,6]
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/Relu6 -> [96,150,150]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/expanded_conv_1/expand/Relu6
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/expanded_conv_1/depthwise/depthwise_weights[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/expanded_conv_1/depthwise/depthwise_weights -> [3,3,96,1]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/FusedBatchNormV3[Op: BatchNorm]. Inputs: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/Conv2D, FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/gamma, FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/beta, FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/moving_mean, FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/moving_variance
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/FusedBatchNormV3 -> [64,2,2]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/FusedBatchNormV3
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/Relu6[Op: Activation]. Inputs: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/FusedBatchNormV3
[TRT]    Setting dynamic range for (Unnamed Layer* 541) [Activation]_output to [0,6]
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/Relu6 -> [64,2,2]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/Relu6
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/weights[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/weights -> [3,3,64,128]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/weights
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Conv2D[Op: Conv]. Inputs: FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/Relu6, FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/weights
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Conv2D -> [128,1,1]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Conv2D
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/gamma[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/gamma -> [128]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/gamma
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/beta[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/beta -> [128]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/beta
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_mean[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_mean -> [128]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_mean
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_variance[Op: Const].
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_variance -> [128]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_variance
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/FusedBatchNormV3[Op: BatchNorm]. Inputs: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Conv2D, FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/gamma, FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/beta, FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_mean, FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/moving_variance
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/FusedBatchNormV3 -> [128,1,1]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/FusedBatchNormV3
[TRT]    UFFParser: Parsing FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Relu6[Op: Activation]. Inputs: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/FusedBatchNormV3
[TRT]    Setting dynamic range for (Unnamed Layer* 549) [Activation]_output to [0,6]
[TRT]    UFFParser: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Relu6 -> [128,1,1]
[TRT]    UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Relu6
[TRT]    UFFParser: Parsing BoxPredictor_5/BoxEncodingPredictor/weights[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/BoxEncodingPredictor/weights -> [1,1,128,24]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/BoxEncodingPredictor/weights
[TRT]    UFFParser: Parsing BoxPredictor_5/BoxEncodingPredictor/Conv2D[Op: Conv]. Inputs: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Relu6, BoxPredictor_5/BoxEncodingPredictor/weights
[TRT]    UFFParser: BoxPredictor_5/BoxEncodingPredictor/Conv2D -> [24,1,1]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/BoxEncodingPredictor/Conv2D
[TRT]    UFFParser: Parsing BoxPredictor_5/BoxEncodingPredictor/biases[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/BoxEncodingPredictor/biases -> [24]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/BoxEncodingPredictor/biases
[TRT]    UFFParser: Parsing BoxPredictor_5/BoxEncodingPredictor/BiasAdd[Op: Binary]. Inputs: BoxPredictor_5/BoxEncodingPredictor/Conv2D, BoxPredictor_5/BoxEncodingPredictor/biases
[TRT]    UFFParser: BoxPredictor_5/BoxEncodingPredictor/BiasAdd -> [24,1,1]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/BoxEncodingPredictor/BiasAdd
[TRT]    UFFParser: Parsing BoxPredictor_5/Shape[Op: Shape]. Inputs: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/Relu6
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/Shape
[TRT]    UFFParser: Parsing BoxPredictor_5/strided_slice/stack[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/strided_slice/stack -> [1]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/strided_slice/stack
[TRT]    UFFParser: Parsing BoxPredictor_5/strided_slice/stack_1[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/strided_slice/stack_1 -> [1]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/strided_slice/stack_1
[TRT]    UFFParser: Parsing BoxPredictor_5/strided_slice/stack_2[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/strided_slice/stack_2 -> [1]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/strided_slice/stack_2
[TRT]    UFFParser: Parsing BoxPredictor_5/strided_slice[Op: StridedSlice]. Inputs: BoxPredictor_5/Shape, BoxPredictor_5/strided_slice/stack, BoxPredictor_5/strided_slice/stack_1, BoxPredictor_5/strided_slice/stack_2
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/strided_slice
[TRT]    UFFParser: Parsing BoxPredictor_5/Reshape/shape/1[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/Reshape/shape/1 -> []
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/Reshape/shape/1
[TRT]    UFFParser: Parsing BoxPredictor_5/Reshape/shape/2[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/Reshape/shape/2 -> []
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/Reshape/shape/2
[TRT]    UFFParser: Parsing BoxPredictor_5/Reshape/shape/3[Op: Const].
[TRT]    UFFParser: BoxPredictor_5/Reshape/shape/3 -> []
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/Reshape/shape/3
[TRT]    UFFParser: Parsing BoxPredictor_5/Reshape/shape[Op: Stack]. Inputs: BoxPredictor_5/strided_slice, BoxPredictor_5/Reshape/shape/1, BoxPredictor_5/Reshape/shape/2, BoxPredictor_5/Reshape/shape/3
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/Reshape/shape
[TRT]    UFFParser: Parsing BoxPredictor_5/Reshape[Op: Reshape]. Inputs: BoxPredictor_5/BoxEncodingPredictor/BiasAdd, BoxPredictor_5/Reshape/shape
[TRT]    UFFParser: BoxPredictor_5/Reshape -> [6,1,4]
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_5/Reshape
[TRT]    UFFParser: Parsing concat_box_loc[Op: _FlattenConcat_TRT]. Inputs: BoxPredictor_0/Reshape, BoxPredictor_1/Reshape, BoxPredictor_2/Reshape, BoxPredictor_3/Reshape, BoxPredictor_4/Reshape, BoxPredictor_5/Reshape
[TRT]    UFFParser: Did not find plugin field entry ignoreBatch in the Registered Creator for layer concat_box_loc
[TRT]    UFFParser: Did not find plugin field entry axis in the Registered Creator for layer concat_box_loc
[TRT]    UFFParser: Parsing Squeeze[Op: Squeeze]. Inputs: concat_box_loc
[TRT]    UFFParser: Squeeze -> [7668,1,1]
[TRT]    UFFParser: Applying order forwarding to: Squeeze
[TRT]    UFFParser: Parsing GridAnchor[Op: _GridAnchor_TRT].
[TRT]    UFFParser: Parsing concat_priorbox[Op: Concat]. Inputs: GridAnchor
[TRT]    UFFParser: concat_priorbox -> [2,7668,1]
[TRT]    UFFParser: Applying order forwarding to: concat_priorbox
[TRT]    UFFParser: Parsing Concatenate/concat[Op: Concat]. Inputs: concat_priorbox, GridAnchor
[TRT]    Parameter check failed at: ../builder/Layers.h::setAxis::381, condition: axis >= 0 && axis < Dims::MAX_DIMS
[TRT]    Concatenate/concat: all concat input tensors must have the same dimensions except on the concatenation axis (0), but dimensions mismatched at index 1. Input 0 shape: [2,7668,1], Input 1 shape: [2,4332,1]
[TRT]    UFFParser: Concatenate/concat -> []
[TRT]    UFFParser: Applying order forwarding to: Concatenate/concat
[TRT]    UFFParser: Parsing BoxPredictor_0/ClassPredictor/weights[Op: Const].
[TRT]    Concatenate/concat: all concat input tensors must have the same dimensions except on the concatenation axis (0), but dimensions mismatched at index 1. Input 0 shape: [2,7668,1], Input 1 shape: [2,4332,1]
[TRT]    UFFParser: BoxPredictor_0/ClassPredictor/weights -> []
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_0/ClassPredictor/weights
[TRT]    UFFParser: Parsing BoxPredictor_0/ClassPredictor/Conv2D[Op: Conv]. Inputs: FeatureExtractor/MobilenetV2/expanded_conv_13/expand/Relu6, BoxPredictor_0/ClassPredictor/weights
[TRT]    Concatenate/concat: all concat input tensors must have the same dimensions except on the concatenation axis (0), but dimensions mismatched at index 1. Input 0 shape: [2,7668,1], Input 1 shape: [2,4332,1]
[TRT]    UFFParser: BoxPredictor_0/ClassPredictor/Conv2D -> []
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_0/ClassPredictor/Conv2D
[TRT]    UFFParser: Parsing BoxPredictor_0/ClassPredictor/biases[Op: Const].
[TRT]    Concatenate/concat: all concat input tensors must have the same dimensions except on the concatenation axis (0), but dimensions mismatched at index 1. Input 0 shape: [2,7668,1], Input 1 shape: [2,4332,1]
[TRT]    UFFParser: BoxPredictor_0/ClassPredictor/biases -> []
[TRT]    UFFParser: Applying order forwarding to: BoxPredictor_0/ClassPredictor/biases
[TRT]    UFFParser: Parsing BoxPredictor_0/ClassPredictor/BiasAdd[Op: Binary]. Inputs: BoxPredictor_0/ClassPredictor/Conv2D, BoxPredictor_0/ClassPredictor/biases
[TRT]    Concatenate/concat: all concat input tensors must have the same dimensions except on the concatenation axis (0), but dimensions mismatched at index 1. Input 0 shape: [2,7668,1], Input 1 shape: [2,4332,1]
[TRT]    UffParser: Parser error: BoxPredictor_0/ClassPredictor/BiasAdd: The input to the Scale Layer is required to have a minimum of 3 dimensions.
[TRT]    failed to parse UFF model '/usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff'
[TRT]    device GPU, failed to load networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT]    detectNet -- failed to initialize.
Segmentation fault (core dumped)[log_jetson.txt|attachment](upload://iTKVN2IZ69RjNIOJE34fizZYTZs.txt) (224.1 KB)

UPDATE
So we just try to run config.py from this post. It’s working for mobilenet_v2
https://forums.developer.nvidia.com/t/how-adapt-tensorflow-object-detection-for-custom-dataset-to-deepstream-5-0/145706/15?u=harryxdllll

Can you guide as what we should change in the config file to export ssd_resnet_50_fpn_coco?

harryxdllll · September 22, 2020, 7:29am

Let me share log from convert_to_uff.py ( ssd_resnet_50_fpn_coco):

Loading frozen_ssd_resnet50.pb
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/uff/bin/../../uff/converters/tensorflow/conversion_helpers.py:274: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

NOTE: UFF has been tested with TensorFlow 1.15.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.9
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 320
      }
      dim {
        size: 320
      }
    }
  }
}
]
=========================================

Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/uff/bin/../../uff/converters/tensorflow/converter.py:226: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_conf as custom op: FlattenConcat_TRT
Warning: No conversion function registered for layer: Split yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_4/Scale/split as custom op: Split
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid/Tile as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_2/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Cast as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Cast_3 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/range_1 as custom op: Range
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_1/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Cast_2 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/range as custom op: Range
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_4/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_4/Cast as custom op: Cast
Warning: No conversion function registered for layer: Split yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_3/Scale/split as custom op: Split
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid/Tile as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_2/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Cast as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Cast_2 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/range_1 as custom op: Range
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_3/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Meshgrid_1/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_3/range as custom op: Range
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_3/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_3/Cast as custom op: Cast
Warning: No conversion function registered for layer: Split yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_2/Scale/split as custom op: Split
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid/Tile as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_2/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Cast as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Cast_2 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/range_1 as custom op: Range
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_3/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Meshgrid_1/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_2/range as custom op: Range
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_2/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_2/Cast as custom op: Cast
Warning: No conversion function registered for layer: Split yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_1/Scale/split as custom op: Split
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid/Tile as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_2/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Cast as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Cast_2 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/range_1 as custom op: Range
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_3/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Meshgrid_1/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator_1/range as custom op: Range
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_1/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates_1/Cast as custom op: Cast
Warning: No conversion function registered for layer: Split yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates/Scale/split as custom op: Split
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid/Tile as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_2/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/Tile as custom op: Tile
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/ExpandedShape_1/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/ExpandedShape_1/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/ExpandedShape_1/Slice as custom op: Slice
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/ExpandedShape/Slice_1 as custom op: Slice
Warning: No conversion function registered for layer: Fill yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/ExpandedShape/ones as custom op: Fill
Warning: No conversion function registered for layer: Slice yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/ExpandedShape/Slice as custom op: Slice
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Cast as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Cast_2 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/range_1 as custom op: Range
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_3/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Tile yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Meshgrid_1/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting MultiscaleGridAnchorGenerator/GridAnchorGenerator/range as custom op: Range
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting MultiscaleGridAnchorGenerator/ToNormalizedCoordinates/Cast as custom op: Cast
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_loc as custom op: FlattenConcat_TRT
DEBUG [/usr/local/lib/python3.6/dist-packages/uff/bin/../../uff/converters/tensorflow/converter.py:143] Marking ['NMS'] as outputs
No. nodes: 1932
UFF Output written to sample_ssd_resnet50.uff

And when we run test by trtexec it’s acour error:

[TRT] UffParser: Validator error: MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/Tile: Unsupported operation _Tile

&&&& RUNNING TensorRT.trtexec # ../../targets/x86_64-linux-gnu/bin/trtexec --uff=./sample_ssd_resnet50.uff --uffInput=Input,3,320,320 --output=NMS
[09/22/2020-07:21:49] [I] === Model Options ===
[09/22/2020-07:21:49] [I] Format: UFF
[09/22/2020-07:21:49] [I] Model: ./sample_ssd_resnet50.uff
[09/22/2020-07:21:49] [I] Uff Inputs Layout: NCHW
[09/22/2020-07:21:49] [I] Input: Input,3,320,320
[09/22/2020-07:21:49] [I] Output: NMS
[09/22/2020-07:21:49] [I] === Build Options ===
[09/22/2020-07:21:49] [I] Max batch: 1
[09/22/2020-07:21:49] [I] Workspace: 16 MB
[09/22/2020-07:21:49] [I] minTiming: 1
[09/22/2020-07:21:49] [I] avgTiming: 8
[09/22/2020-07:21:49] [I] Precision: FP32
[09/22/2020-07:21:49] [I] Calibration:
[09/22/2020-07:21:49] [I] Safe mode: Disabled
[09/22/2020-07:21:49] [I] Save engine:
[09/22/2020-07:21:49] [I] Load engine:
[09/22/2020-07:21:49] [I] Builder Cache: Enabled
[09/22/2020-07:21:49] [I] NVTX verbosity: 0
[09/22/2020-07:21:49] [I] Inputs format: fp32:CHW
[09/22/2020-07:21:49] [I] Outputs format: fp32:CHW
[09/22/2020-07:21:49] [I] Input build shapes: model
[09/22/2020-07:21:49] [I] Input calibration shapes: model
[09/22/2020-07:21:49] [I] === System Options ===
[09/22/2020-07:21:49] [I] Device: 0
[09/22/2020-07:21:49] [I] DLACore:
[09/22/2020-07:21:49] [I] Plugins:
[09/22/2020-07:21:49] [I] === Inference Options ===
[09/22/2020-07:21:49] [I] Batch: 1
[09/22/2020-07:21:49] [I] Input inference shapes: model
[09/22/2020-07:21:49] [I] Iterations: 10
[09/22/2020-07:21:49] [I] Duration: 3s (+ 200ms warm up)
[09/22/2020-07:21:49] [I] Sleep time: 0ms
[09/22/2020-07:21:49] [I] Streams: 1
[09/22/2020-07:21:49] [I] ExposeDMA: Disabled
[09/22/2020-07:21:49] [I] Spin-wait: Disabled
[09/22/2020-07:21:49] [I] Multithreading: Disabled
[09/22/2020-07:21:49] [I] CUDA Graph: Disabled
[09/22/2020-07:21:49] [I] Skip inference: Disabled
[09/22/2020-07:21:49] [I] Inputs:
[09/22/2020-07:21:49] [I] === Reporting Options ===
[09/22/2020-07:21:49] [I] Verbose: Disabled
[09/22/2020-07:21:49] [I] Averages: 10 inferences
[09/22/2020-07:21:49] [I] Percentile: 99
[09/22/2020-07:21:49] [I] Dump output: Disabled
[09/22/2020-07:21:49] [I] Profile: Disabled
[09/22/2020-07:21:49] [I] Export timing to JSON file:
[09/22/2020-07:21:49] [I] Export output to JSON file:
[09/22/2020-07:21:49] [I] Export profile to JSON file:
[09/22/2020-07:21:49] [I]
[09/22/2020-07:21:50] [E] [TRT] UffParser: Validator error: MultiscaleGridAnchorGenerator/GridAnchorGenerator_4/Meshgrid_3/Tile: Unsupported operation _Tile
[09/22/2020-07:21:50] [E] Failed to parse uff file
[09/22/2020-07:21:50] [E] Parsing model failed
[09/22/2020-07:21:50] [E] Engine creation failed
[09/22/2020-07:21:50] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # ../../targets/x86_64-linux-gnu/bin/trtexec --uff=./sample_ssd_resnet50.uff --uffInput=Input,3,320,320 --output=NMS

AastaLLL · October 14, 2020, 3:54am

Hi, harryxdllll

The error indicates some non-supported operation (ex. Tile, Slice, …).
To run these operations with TensorRT, you will need to enable them as a plugin layer.

Please find the following document for the plugin implementation details:

Thanks.

AastaLLL · October 14, 2020, 3:59am

Hi, Elviron

The root cause is onnx expects input image to be INT8 but TensorRT use Float32.
To solve this issue, you can modify the input data format of ONNX with our graphsurgeon API directly.

1. Install ONNX Graphsurgeon API

$ sudo apt-get install python3-pip libprotobuf-dev protobuf-compiler
$ git clone https://github.com/NVIDIA/TensorRT.git
$ cd TensorRT/tools/onnx-graphsurgeon/
$ make install

2. Modify your model

import onnx_graphsurgeon as gs
import onnx
import numpy as np

graph = gs.import_onnx(onnx.load("model.onnx"))
for inp in graph.inputs:
    inp.dtype = np.float32

onnx.save(gs.export_onnx(graph), "updated_model.onnx")

Thanks.

Elviron · October 15, 2020, 2:32pm

Thanks, it works! However, now we have another problem. We have onnx version 1.7.0 (1.6.0 gave the same results), tensorflow 1.15.3 and tf2onnx 1.6.3. On every our network (SSD_mobilenet_v2, SSD_inception_v2, SSD_resnet_50) we have the same error when trying to parse ONNX model to TRT engine by using detectNet from jetson-inference:

...
[TRT]    ModelImporter.cpp:179: MultipleGridAnchorGenerator/Meshgrid_8/Reshape [                                                                                                                                                             Reshape] outputs: [MultipleGridAnchorGenerator/Meshgrid_8/Reshape:0 -> ()],
[TRT]    ModelImporter.cpp:103: Parsing node: MultipleGridAnchorGenerator/Meshgr                                                                                                                                                             id_8/Tile [Tile]
[TRT]    ModelImporter.cpp:119: Searching for input: MultipleGridAnchorGenerator                                                                                                                                                             /Meshgrid_8/Reshape:0
[TRT]    ModelImporter.cpp:119: Searching for input: MultipleGridAnchorGenerator                                                                                                                                                             /Meshgrid_8/Tile__774:0
[TRT]    ModelImporter.cpp:125: MultipleGridAnchorGenerator/Meshgrid_8/Tile [Til                                                                                                                                                             e] inputs: [MultipleGridAnchorGenerator/Meshgrid_8/Reshape:0 -> ()], [MultipleGr                                                                                                                                                             idAnchorGenerator/Meshgrid_8/Tile__774:0 -> (-1)],
[TRT]    ImporterContext.hpp:141: Registering layer: MultipleGridAnchorGenerator                                                                                                                                                             /Meshgrid_8/Tile for ONNX node: MultipleGridAnchorGenerator/Meshgrid_8/Tile
[TRT]    ImporterContext.hpp:116: Registering tensor: MultipleGridAnchorGenerato                                                                                                                                                             r/Meshgrid_8/Tile:0 for ONNX tensor: MultipleGridAnchorGenerator/Meshgrid_8/Tile                                                                                                                                                             :0
[TRT]    ModelImporter.cpp:179: MultipleGridAnchorGenerator/Meshgrid_8/Tile [Til                                                                                                                                                             e] outputs: [MultipleGridAnchorGenerator/Meshgrid_8/Tile:0 -> ()],
[TRT]    ModelImporter.cpp:103: Parsing node: MultipleGridAnchorGenerator/stack_                                                                                                                                                             5_Unsqueeze__775 [Unsqueeze]
[TRT]    ModelImporter.cpp:119: Searching for input: MultipleGridAnchorGenerator                                                                                                                                                             /Meshgrid_8/Tile:0
[TRT]    ModelImporter.cpp:125: MultipleGridAnchorGenerator/stack_5_Unsqueeze__7                                                                                                                                                             75 [Unsqueeze] inputs: [MultipleGridAnchorGenerator/Meshgrid_8/Tile:0 -> ()],
ERROR: onnx2trt_utils.cpp:188 In function convertAxis:
[8] Assertion failed: axis >= 0 && axis < nbDims
[TRT]    failed to parse ONNX model 'networks/updated_model.onnx'
[TRT]    device GPU, failed to load networks/updated_model.onnx
[TRT]    detectNet -- failed to initialize.
detectnet:  failed to load detectNet model

When we change the tf2onnx version to 1.7.1, we have another issue:

...
[TRT]    ModelImporter.cpp:103: Parsing node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533 [Unsqueeze]
[TRT]    ModelImporter.cpp:119: Searching for input: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Reshape:0
[TRT]    ModelImporter.cpp:125: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533 [Unsqueeze] inputs: [Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Reshape:0 -> (-1)],
[TRT]    onnx2trt_utils.cpp:1793: Original shape: (_,), unsqueezing to: (_, _, _)
[TRT]    ImporterContext.hpp:141: Registering layer: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533 for ONNX node: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533
[TRT]    ImporterContext.hpp:116: Registering tensor: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533:0 for ONNX tensor: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533:0
[TRT]    ModelImporter.cpp:179: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533 [Unsqueeze] outputs: [Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533:0 -> (-1, -1, -1)],
[TRT]    ModelImporter.cpp:103: Parsing node: NonMaxSuppression__1536 [NonMaxSuppression]
[TRT]    ModelImporter.cpp:119: Searching for input: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1531:0
[TRT]    ModelImporter.cpp:119: Searching for input: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533:0
[TRT]    ModelImporter.cpp:119: Searching for input: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1535:0
[TRT]    ModelImporter.cpp:119: Searching for input: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores_1/iou_threshold:0
[TRT]    ModelImporter.cpp:119: Searching for input: Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores_1/score_threshold:0
[TRT]    ModelImporter.cpp:125: NonMaxSuppression__1536 [NonMaxSuppression] inputs: [Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1531:0 -> (-1, -1, -1)], [Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1533:0 -> (-1, -1, -1)], [Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores/NonMaxSuppressionV5__1535:0 -> ()], [Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores_1/iou_threshold:0 -> ()], [Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression_with_scores_1/score_threshold:0 -> ()],
[TRT]    ModelImporter.cpp:135: No importer registered for op: NonMaxSuppression. Attempting to import as plugin.
[TRT]    builtin_op_importers.cpp:3659: Searching for plugin: NonMaxSuppression, plugin_version: 1, plugin_namespace:
[TRT]    INVALID_ARGUMENT: getPluginCreator could not find plugin NonMaxSuppression version 1
ERROR: builtin_op_importers.cpp:3661 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[TRT]    failed to parse ONNX model 'networks/updated_model.onnx'
[TRT]    device GPU, failed to load networks/updated_model.onnx
[TRT]    detectNet -- failed to initialize.
detectnet:  failed to load detectNet model

What can we do about it?

AastaLLL · October 23, 2020, 6:40am

Hi,

Good to know it works now.

The second issue looks related to the output.
Would you mind to check the onnx model with trtexec and share the log with us?

/usr/src/tensorrt/bin/trtexec --onnx=[your/model] --verbose

Thanks.

Elviron · October 23, 2020, 7:19am

Ok, I’m sending you the full log.

trtexec.log (569,7 KB)

I don’t like the line “[10/23/2020-09:00:53] [V] [TRT] ModelImporter.cpp:202: Adding network input: image_tensor:0 with dtype: float32, dimensions: (-1, -1, -1, 3)”, but can this be the cause? How can I put proper dimensions to the model?

AastaLLL · October 27, 2020, 4:47am

Hi,

The model uses dynamic input, and it’s recommended to set it to fix if you don’t need it.
You can specify the dimension directly through the ONNX Graphsurgeon API shared above.

For example:

...
input = graph.inputs[0]
input.shape = [1, 3, 512, 512]
...

Thanks.

bnascimento · October 27, 2020, 11:28am

Hi @AastaLLL thank you for your help so far. I am followinf this issue thread and I am stuck at exactly the same place as @Elviron to create a tensorRT file from ONNX.

I picked up a tensorflow model .pb (object detection), turned into .onnx with

python3 -m tf2onnx.convert --saved-model /tmp/export/saved_model/ --output /tmp/export/saved_model/model.onnx --opset 11

applied the above graph surgeon patch on .onnx file to cast the input tensor to 32f and applied a fixed size.

However when I try to turn it into a tensorRT file withonnx2trt /tmp/export/saved_model/updated_model.onnx -o /tmp/export/saved_model/model.trt

I got:

[2020-10-27 11:19:46 WARNING] /opt/onnx-tensorrt/onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[2020-10-27 11:19:46 WARNING] /opt/onnx-tensorrt/onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[2020-10-27 11:19:46 ERROR] INVALID_ARGUMENT: getPluginCreator could not find plugin NonMaxSuppression version 1
While parsing node number 371 [NonMaxSuppression → “NonMaxSuppression__995:0”]:
ERROR: /opt/onnx-tensorrt/builtin_op_importers.cpp:3777 In function importFallbackPluginImporter:
[8] Assertion failed: creator && “Plugin not found, are the plugin name, version, and namespace correct?”

Any ideas how to get past this?
Thanks

AastaLLL · October 28, 2020, 4:15am

Hi,

[2020-10-27 11:19:46 ERROR] INVALID_ARGUMENT: getPluginCreator could not find plugin NonMaxSuppression version 1

This indicates that the required NonMaxSuppression layer is not supported by the TensorRT currently.
You can find the TensorRT support matrix here and onnx2trt’s here.

To fix this, you can integrate the NonMaxSuppression implementation to our TensorRT plugin and add it to the ModelImporter.

Thanks.

bnascimento · October 28, 2020, 8:34am

Hi again @AastaLLL , thanks for reaching out. Would you advice to replicate the tf’s version of nms into tensorrt instead of using any of the existing nms plugins that tensorRT has already on TensorRT/plugin/nmsPlugin at master · NVIDIA/TensorRT · GitHub or TensorRT/plugin/batchedNMSPlugin at master · NVIDIA/TensorRT · GitHub ?
So for I’ve been unsucessful with the last idea. Do you have a clear example how to integrate a plugin via ModelImporter?

Thanks

AastaLLL · October 29, 2020, 4:53am

Hi,

I think the NonMaxSuppression used in the TensorFlow is different from the batchedNMSPlugin in the TensorRT.
You will need to integrate the TensorFlow implementation to TensorRT plugin first.

If you want to use batchedNMSPlugin, you can add the support to onnx parser builtin_op_importers.cpp directly.
Please check InstanceNormalization implementation as an example:

github.com

onnx/onnx-tensorrt/blob/eb559b6cdd1ec2169d64c0112fab9b564d8d503b/builtin_op_importers.cpp#L1635


      
              ShapedWeights scaleWeights = ctx->createTempWeights(::ONNX_NAMESPACE::TensorProto_DataType_FLOAT, dims);
              std::fill(static_cast<float*>(scaleWeights.values), static_cast<float*>(scaleWeights.values) + scaleWeights.count(),
                  scale);
              // Finally add the scale layer.
              auto layer = ctx->network()->addScale(
                  tensor, nvinfer1::ScaleMode::kCHANNEL, shiftWeights, scaleWeights, nvinfer1::Weights{});
              ctx->registerLayer(layer, node.name());
              RETURN_FIRST_OUTPUT(layer);
          }
          
          
DEFINE_BUILTIN_OP_IMPORTER(InstanceNormalization)
          {
              // Scales and biases must be initializers
              ASSERT(inputs.at(1).is_weights(), ErrorCode::kUNSUPPORTED_NODE);
              ASSERT(inputs.at(2).is_weights(), ErrorCode::kUNSUPPORTED_NODE);
              nvinfer1::ITensor* tensorPtr = &convertToTensor(inputs.at(0), ctx);
              int nbDims = tensorPtr->getDimensions().nbDims;
              ASSERT(nbDims >= 3 && nbDims <= 4 && "TensorRT only supports InstanceNormalization on 3D or 4D tensors!",
                  ErrorCode::kUNSUPPORTED_NODE);
              auto scale_weights = inputs.at(1).weights();
              auto bias_weights = inputs.at(2).weights();

Thanks.

bnascimento · October 29, 2020, 9:52am

Yeah that confirms also our findings, that batchedNMSPlugin is expecting tensors with different dimensions then the ones we have from parsed onnx model. Instead of the batchedNMSPlugin you think we can make this work via the NMSPlugin instead? (TensorRT/plugin/nmsPlugin at master · NVIDIA/TensorRT · GitHub)

Elviron · October 29, 2020, 9:59am

@AastaLLL thanks for your advices! So we have to implement the NonMaxSuppression plugin by our own in order to start our models working.
Is this necessary for every model? If not, can you provide us some list of models which should work in this workflow (TensorFlow → Frozen Graph → ONNX → TensorRT engine) without that kind of modifications?
If none of the models works, then it seems we will have to go the @bnascimento way.

AastaLLL · November 9, 2020, 6:54am

Hi, both

Sorry for the late reply.
You can use the NMSPlugin for the TensorFlow NonMaxSuppression operation.

The TensorRT plugin is already implemented and merged in the default library.
But the mapping doesn’t be added into the onnx-tensorr parser and leads to a non-registered error.

As mentioned above, you can add the mapping for NMSPlugin <-> NonMaxSuppression in builtin_op_importers.cpp and rebuild the onnx parser.

Thanks.

Topic		Replies	Views
Cannot convert a tensorflow model to UFF Jetson TX2	13	2333	October 18, 2021
Process killed during tensorrt conversion on Jetson orin NX (8 GB) Jetson Orin NX tensorrt	15	687	April 30, 2024
Convert Tensoflow V2 model ( pb format) to UFF Jetson Nano tensorflow	7	1332	October 18, 2021
Segmentation fault occurs while running jetson-inference example application Jetson Nano jetson-inference	2	300	October 15, 2021
I am trying to convert the ONNX SSD mobilnet v3 model into TensorRT Engine. I am getting the below error Jetson TX2 tensorrt , tensorflow	24	3679	February 17, 2022
I do not get any performance improvement after using TensorRT provider for object detection model Jetson Nano tensorrt , onnx	7	1389	July 12, 2022
Deploy Object Detection TF-TRT INT8 with DS Triton DeepStream SDK inference-server-triton	16	1299	October 12, 2021
Convert custom Tensorflow model to TensorRT Jetson Nano	12	5084	October 14, 2021
Model inferencing with TensorRT on Jetson (TX2) Jetson TX2	4	943	October 18, 2021
Where is the right tutorial for NANO detection implementation? Jetson Nano	9	896	October 14, 2021

Exporting Tensorflow models to Jetson Nano

1. Install ONNX Graphsurgeon API

2. Modify your model

Related topics