Post processing FasterRCNN output in python

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello @Morganh
I need your help.
After training and exporting the model using tlt-train faster_rcnn and tlt-export faster_rcnn, I converted the model using tlt-converter v7.1 on Xavier NX as below.

./tlt-converter ../exported_models/frcnn_resnet18_fp16.etlt -k <my-key> -d 3,480,640 -o dense_class_td/Softmax,dense_regress_td/BiasAdd,proposal -e trt.fp16.engine -m 1 -t fp16

Then after loading the engine using tensorrt python API, I am allocating buffer and doing inference as done in this example to get the output.

print(detection_out)

[array([ 0.        ,  0.42495573,  0.04938519, ...,  0.19440129,
        0.10163826,  0.22853875], dtype=float32), array([ 0.05993652,  0.89257812, -1.24902344, ...,  1.51464844,
       -1.39941406, -0.53710938], dtype=float32), array([  1.08480453e-05,   1.00000000e+00,   1.41859055e-05,
         1.00000000e+00,   2.38418579e-07,   1.00000000e+00,
         2.38418579e-07,   1.00000000e+00,   4.76837158e-07,
         1.00000000e+00,   1.69873238e-05,   1.00000000e+00,
         1.78813934e-07,   1.00000000e+00,   1.65343285e-04,
  ....
...
         1.00000000e+00,   1.19209290e-07,   1.00000000e+00], dtype=float32)]

Now, I want to extract bbox, class_labels, and score. How can I do that? It seems I can’t find the relevant document.
Thanks in advance.

3

For faster-rcnn post-processing, please refer to https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/tree/master/nvdsinfer_customparser_frcnn_tlt.

1 Like
4

Thanks, @Morganh. Can you please tell me more about dense_class_td/Softmax, dense_regress_td/BiasAdd, proposal?

5

It is Comma-separated list of output blob names.

For classification use: predictions/Softmax.
For DetectNet_v2: output_bbox/BiasAdd,output_cov/Sigmoid
For FasterRCNN: dense_class_td/Softmax,dense_regress_td/BiasAdd, proposal
For SSD, DSSD, RetinaNet: NMS
For YOLOv3: BatchedNMS
For MaskRCNN: generate_detections, mask_head/mask_fcn_logits/BiasAdd
1 Like
6