Detectnet_v2(resnet50) low accuracy on 2 class dataset

[Morganh]

@pddarrell

Does the ubiquitous result of “0” mean something is not set up correctly? - zero is difficult to achieve, isn’t it?

I am afraid yes. Please check

1. Are the labels correct? Especially for the “healthy” class.
2. More, how many images for these two classes separately?
3. Set to the same
   initial_weight: 10.0
   weight_target: 10.0
4. Can you share latest training spec?

[pddarrell]

@Morganh

Are the labels correct? Especially for the “healthy” class.

“healthy” image and label files have a “0” as the first digit:
They start at “000001” and are not entirely consecutive e.g:

000016.txt (76 Bytes)

“damage” image and label files have a “1” as the first digit:

100268.txt (74 Bytes)

I believe the labels are 100% correct. I hand-labelled them and I have checked them very carefully.

Blockquote

In train/val: 531 “healthy”, 531 “damage”
In test: 10 “healthy”, 10 “damage”

Blockquote
currently the settings in “cost_function_config” are:

cost_function_config {
target_classes {
name: “damage”
class_weight: 8.0
coverage_foreground_weight: 0.0500000007451
objectives {
name: “cov”
initial_weight: 1.0
weight_target: 1.0
}
objectives {
name: “bbox”
initial_weight: 10.0
weight_target: 1.0
}
}
target_classes {
name: “healthy”
class_weight: 1.0
coverage_foreground_weight: 0.0500000007451
objectives {
name: “cov”
initial_weight: 1.0
weight_target: 1.0
}
objectives {
name: “bbox”
initial_weight: 10.0
weight_target: 10.0
}
}
To be 100% sure: do you want me to reset the initial_weight and weight_target of both classes to 10.0?

Blockquote
detectnet_v2_train_resnet50_kitti.txt (4.2 KB)
This is the latest training spec before I reset the initial_weight and weight_target of both classes to 10.0.

[Morganh]

For healthy class, from the label file,

healthy 0.00 0 0.00 0.0 0.0 3648.0 1417.0 0.00 0.00 0.00 0.00 0.00 0.00 0.00

It is 1920x746 image, why the bbox is (0.0 0.0 3648.0 1417.0) ?

[pddarrell]

@Morganh

It is 1920x746 image, why the bbox is (0.0 0.0 3648.0 1417.0) ?

image size of 000016.jpg is 3648 x 1417. I guess the forum formatting has resized the image that you are seeing. The label is correct.

[pddarrell]

@pddarrell

Does the notebook allow the confidence threshold to be adjusted? Maybe it is a confidence issue?

Problem solved. I had the coverage_threshold for “healthy” in postprocessing_config set too high. Interestingly, when I lowered it the first time the average precision of “damage” improved and “healthy” stayed at “0”. It encouraged me to lower it further and, finally, I got a non-zero value for “healthy” too.

These experiments are only run for 10 epochs, so I am hopeful that further improvements are possible at ~ 120 epochs.

Basically I had not fully appreciated what “mean average precision” was doing and what it was affected by.
I found this blog helpful:
https://machinethink.net/blog/object-detection

Thank you for being a great help.