Converting sampleUffSSD for different tensorflow models

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello I’ve been trying to convert my object_detection ssd model from tensorflow 1.8,
I’ve converted the layers using the following python script. All the parameters are carefully matched to the tensorflow model.

import graphsurgeon as gs
import tensorflow as tf

Input = gs.create_node(
	"Input",
	op="Placeholder",
	dtype=tf.float32,
	shape=[1, 3, 1600, 1600])
    
PriorBox = gs.create_node(
	"PriorBox",
	numLayers=6,
	minScale=0.1,
	maxScale=0.2,
	aspectRatios=[1.0, 2.0, 0.5],
	layerVariances=[0.1, 0.1, 0.2, 0.2],
	featureMapShapes=[200, 100, 50, 25, 13, 6])
    
NMS = gs.create_node(
	"NMS",
	numClasses=13,
	iouThreshold=0.6,
	scoreThreshold=1e-8,
	maxDetectionsPerClass=100,
	maxTotalDetections=100,
	scoreConverter="SIGMOID")

concat_box_loc = gs.create_node("concat_box_loc")
concat_box_conf = gs.create_node("concat_box_conf")
concat_priorbox = gs.create_node("concat_priorbox", dtype=tf.float32, axis=2)

namespace_plugin_map = {
    "Postprocessor": NMS,
    "Preprocessor": Input,
    "concat": concat_box_loc,
    "concat_1": concat_box_conf,
	"Concatenate/concat": concat_priorbox,
	"MultipleGridAnchorGenerator": PriorBox,
}

namespace_remove = {
    "ToFloat",
    "image_tensor",
    "Preprocessor/map/TensorArrayStack_1/TensorArrayGatherV3",
}

def preprocess(dynamic_graph):
    # remove the unrelated or error layers
    dynamic_graph.remove(
		dynamic_graph.find_nodes_by_path(namespace_remove), 
		remove_exclusive_dependencies=False)

    # Now create a new graph by collapsing namespaces
    dynamic_graph.collapse_namespaces(namespace_plugin_map)
	
    # Remove the outputs, so we just have a single output node (NMS).
    dynamic_graph.remove(dynamic_graph.graph_outputs, remove_exclusive_dependencies=False)

    # Remove the Squeeze to avoid "Assertion `isPlugin(layerName)' failed"
    Squeeze = dynamic_graph.find_node_inputs_by_name(dynamic_graph.graph_outputs[0], 'Squeeze')
    dynamic_graph.forward_inputs(Squeeze)

which gives me the following result

Loading resnet_w_frozen_inference_graph.pb
Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS yet.
Converting as custom op NMS NMS
name: "NMS"
op: "NMS"
input: "concat_box_loc"
input: "concat_priorbox"
input: "concat_box_conf"
attr {
  key: "iouThreshold_u_float"
  value {
    f: 0.6000000238418579
  }
}
attr {
  key: "maxDetectionsPerClass_u_int"
  value {
    i: 100
  }
}
attr {
  key: "maxTotalDetections_u_int"
  value {
    i: 100
  }
}
attr {
  key: "numClasses_u_int"
  value {
    i: 13
  }
}
attr {
  key: "scoreConverter_u_str"
  value {
    s: "SIGMOID"
  }
}
attr {
  key: "scoreThreshold_u_float"
  value {
    f: 9.99999993922529e-09
  }
}

Warning: No conversion function registered for layer: concat_box_conf yet.
Converting as custom op concat_box_conf concat_box_conf
name: "concat_box_conf"
op: "concat_box_conf"
input: "BoxPredictor_0/Reshape_1"
input: "BoxPredictor_1/Reshape_1"
input: "BoxPredictor_2/Reshape_1"
input: "BoxPredictor_3/Reshape_1"
input: "BoxPredictor_4/Reshape_1"
input: "BoxPredictor_5/Reshape_1"

Warning: No conversion function registered for layer: concat_priorbox yet.
Converting as custom op concat_priorbox concat_priorbox
name: "concat_priorbox"
op: "concat_priorbox"
input: "PriorBox"
attr {
  key: "axis_u_int"
  value {
    i: 2
  }
}
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}

Warning: No conversion function registered for layer: PriorBox yet.
Converting as custom op PriorBox PriorBox
name: "PriorBox"
op: "PriorBox"
input: "Const"
attr {
  key: "aspectRatios_u_flist"
  value {
    list {
      f: 1.0
      f: 2.0
      f: 0.5
    }
  }
}
attr {
  key: "featureMapShapes_u_ilist"
  value {
    list {
      i: 200
      i: 100
      i: 50
      i: 25
      i: 13
      i: 6
    }
  }
}
attr {
  key: "layerVariances_u_flist"
  value {
    list {
      f: 0.10000000149011612
      f: 0.10000000149011612
      f: 0.20000000298023224
      f: 0.20000000298023224
    }
  }
}
attr {
  key: "maxScale_u_float"
  value {
    f: 0.20000000298023224
  }
}
attr {
  key: "minScale_u_float"
  value {
    f: 0.10000000149011612
  }
}
attr {
  key: "numLayers_u_int"
  value {
    i: 6
  }
}
Warning: No conversion function registered for layer: concat_box_loc yet.
Converting as custom op concat_box_loc concat_box_loc
name: "concat_box_loc"
op: "concat_box_loc"
input: "BoxPredictor_0/Reshape"
input: "BoxPredictor_1/Reshape"
input: "BoxPredictor_2/Reshape"
input: "BoxPredictor_3/Reshape"
input: "BoxPredictor_4/Reshape"
input: "BoxPredictor_5/Reshape"

No. nodes: 292
UFF Output written to resnet_w_frozen_inference_graph.pb.uff

As you can see I’m interested in bigger images 1600x1600, i’ve changed the code to handle that and even started the optimization process, I’m however stuck at this point

INFO: Formats and tactics selection completed in 11.507 seconds.
INFO: After reformat layers: 57 layers
INFO: Block size 1073741824
INFO: Block size 81920000
INFO: Block size 30835456
INFO: Block size 5546240
INFO: Block size 5120000
INFO: Block size 2720000
INFO: Block size 2080000
INFO: Block size 680192
INFO: Block size 640000
INFO: Block size 170240
INFO: Block size 160000
INFO: Block size 46080
INFO: Block size 40192
INFO: Block size 9984
INFO: Block size 4608
INFO: Total Activation Memory: 1203714816
sample_uff_ssd: NvPluginSSD.cu:795: virtual void nvinfer1::plugin::DetectionOutput::configure(const nvinfer1::Dims*, int, const nvinfer1::Dims*, int, int): Assertion `numPriors*numLocClasses*4 == inputDims[param.inputOrder[0]].d[0]' failed.

and i would like to see what’s under the hood to fix my problem.
I have a good understanding of the code but I can’t find a way to print any useful message from the detection plugin.

Any suggestions on how to solve this, Is there any way to check the dimensions passed ?

2

It should be caused by your feature map size. would you mind to share your config file in tensorflow and in tensorrt.

3

My tensorrt config is on the first post,

my tensorflow config is as follows

model {
  ssd {
    num_classes: 12
    image_resizer {
      fixed_shape_resizer {
        height: 1600
        width: 1600
        resize_method: AREA
      }
    }
    feature_extractor {
      type: "ssd_resnet_w_v1"
      depth_multiplier: 1.0
      min_depth: 0
      conv_hyperparams {
        regularizer {
          l2_regularizer {
            weight: 0.000500000023749
          }
        }
        initializer {
          truncated_normal_initializer {
            mean: 0.0
            stddev: 0.0299999993294
          }
        }
        activation: NONE
      }
    }
    box_coder {
      faster_rcnn_box_coder {
        y_scale: 10.0
        x_scale: 10.0
        height_scale: 5.0
        width_scale: 5.0
      }
    }
    matcher {
      argmax_matcher {
        matched_threshold: 0.5
        unmatched_threshold: 0.5
        ignore_thresholds: false
        negatives_lower_than_unmatched: true
        force_match_for_each_row: true
      }
    }
    similarity_calculator {
      iou_similarity {
      }
    }
    box_predictor {
      convolutional_box_predictor {
        conv_hyperparams {
          regularizer {
            l2_regularizer {
              weight: 0.000500000023749
            }
          }
          initializer {
            truncated_normal_initializer {
              mean: 0.0
              stddev: 0.0299999993294
            }
          }
          activation: NONE
        }
        min_depth: 40
        max_depth: 40
        num_layers_before_predictor: 1
        use_dropout: false
        dropout_keep_probability: 0.800000011921
        kernel_size: 3
        box_code_size: 4
        apply_sigmoid_to_scores: false
      }
    }
    anchor_generator {
      ssd_anchor_generator {
        num_layers: 6
        min_scale: 0.0196299999952
        max_scale: 0.17745000124
        aspect_ratios: 1.0
        aspect_ratios: 2.0
        aspect_ratios: 0.5
        reduce_boxes_in_lowest_layer: false
        height_stride: 8
        height_stride: 16
        height_stride: 32
        height_stride: 64
        height_stride: 128
        height_stride: 256
        width_stride: 8
        width_stride: 16
        width_stride: 32
        width_stride: 64
        width_stride: 128
        width_stride: 256
      }
    }
    post_processing {
      batch_non_max_suppression {
        score_threshold: 0.10000000149
        iou_threshold: 0.40000000596
        max_detections_per_class: 100
        max_total_detections: 400
      }
      score_converter: SOFTMAX
    }
    normalize_loss_by_num_matches: true
    loss {
      localization_loss {
        weighted_smooth_l1 {
          anchorwise_output: true
        }
      }
      classification_loss {
        weighted_softmax {
          anchorwise_output: true
        }
      }
      hard_example_miner {
        num_hard_examples: 300
        iou_threshold: 0.699999988079
        loss_type: CLASSIFICATION
        max_negatives_per_positive: 3
        min_negatives_per_image: 0
      }
      classification_weight: 1.0
      localization_weight: 1.0
    }
  }
}
eval_config {
  num_visualizations: 50
  num_examples: 60000
  max_evals: 0
  use_moving_averages: false
}
eval_input_reader {
  label_map_path: "/media/nikolas/Data1/tf_converted/label_map.pbtxt"
  shuffle: false
  num_epochs: 1
  num_readers: 1
  tf_record_input_reader {
    input_path: "/media/nikolas/Data1/tf_converted/test.record"
  }
}
4

I’ve converted my model to a serialized engine.
When i run the inception_v2 model i get some results
when i run mine i get all zeros.
I know the model works since i evaluated it in tensorflow, the question is, is there any utility to check out the weights and/or the convertion ?

5

the feature maps should be:
featureMapShapes=[100,50,25,13,7,4])