Low precision but good mAPn many false positives for same object(Faster RCNN)

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hi ! I’m doing insect familly recognition using Faster RCNN (dataset of 3500 images per class) and I’m surprised that my precision is very low compared to the mAP per class. If I look at the inference images made, I can see that most images have more than one familly prediction per butterfly. So this confirms the low precision. How can I improve this ?
Here are some files :

Here are my specs files :
default_spec_resnet18_custom.txt (4.1 KB)
frcnn_tfrecords_kitti_trainval_custom.txt (355 Bytes)
output.txt (75.8 KB)

And some images and their inference labels:
4585-noctuidae.txt (136 Bytes)

4585-noctuidae
4585-noctuidae512×512 54.5 KB

15-noctuidae
15-noctuidae512×512 43.5 KB

15-noctuidae.txt (200 Bytes)

(Note that the name of the file represents the true label)
Thanks for your help !

3

Could you please try a larger backbone?

4

More, please set
a lower classifier_nms_max_boxes, and
a larger classifier_nms_overlap_threshold, for example,0.5.
a larger object_confidence_thres

5

The problem is that I need to run deepstream with this model on a jetson nano, so a larger backbone will cause less FPS on deepstream right ? I’ve already tried ResNet 50 but with less epoch, it takes way longer to train.

6

Thanks ! I’ve already trained this night with only two classes (more easy to recognise). What I can conclude after this experiment is that after 100 epochs on a ResNet18, the precision starts to rise slowly until 20% for each object. I’ll now try with your parameters.

7

Seems that the insect class is similar, in order to distinguish insect classes well, some tips here:

  1. Add more training data
  2. Try a larger backbone
  3. Try other detection architecture, for example, ssd
8

Yes that’s the goal here. The goal is to find their big familly even they have different sub famillies. But what is strange is that even more distinguishable insects had super low precision (classification of coleopters and flies)

9

For lower precision, you can try to set a lower classifier_nms_max_boxes. By default is 100. You can modify the spec and run evaluation. In your previous result folder, I think there should be some existing tlt models. Run evaluation against it directly without training.

10

Indeed, I have plenty of tlt models, and precision is now around 0.9 with the parameters suggested (still Faster RCNN and Resnet18)
classifier_nms_max_boxes 100 → 25
object_confidence_threshol 0.0001 → 0.9
classifier_nms_overlap_threshold 0.3 → 0.5
Thank you so much !