The first class is always not detected in inference

Accelerated Computing Intelligent Video Analytics TAO Toolkit
#1

• Hardware (A30)
• Network Type (Mask_rcnn)
• TLT Version (/tao-toolkit-tf:v3.22.05-tf1.15.5-py3)
• Training spec file(If have, please share here)
maskrcnn_train_resnet50.txt (2.4 KB)

I’m training with attached config file.
Altogether 22 classes, so added background so total 23 classes in training.
The instance label file is as shown in below.
instance_label_singpilot.txt (406 Bytes)

single_broken_white_line is the dash line as shown in the following image.

5
51280×720 86.8 KB

That was in the training.
But in inference, it was never detected. What could be wrong?

#2

How can I find clue why the first class in label file is not detected?

#3

How about other classes? Are they correct?

#4

Yes they are correctly detected. Only the first class is not detected.

#5

How can I find where is the issue?

#6

You are running “tao mask_rcnn inference” to test, right?

#7

Yes I’m using mask inference command.

#8

In your shared image, I find several single_broken_white lines, for example, do you mean below can not be detected?

image

#9

The green box is detection for solid line.
Our mask labelling for dash line is like below in red.

dash
All images with those dash lines have no detection at all.

Is it because of mask config? Which one should I change?

maskrcnn_config {
    nlayers: 50
    arch: "resnet"
    freeze_bn: False
    #freeze_blocks: "[0,1]"
    gt_mask_size: 112
        
    # Region Proposal Network
    rpn_positive_overlap: 0.7
    rpn_negative_overlap: 0.3
    rpn_batch_size_per_im: 256
    rpn_fg_fraction: 0.5
    rpn_min_size: 0.

    # Proposal layer.
    batch_size_per_im: 512
    fg_fraction: 0.25
    fg_thresh: 0.5
    bg_thresh_hi: 0.5
    bg_thresh_lo: 0.

    # Faster-RCNN heads.
    fast_rcnn_mlp_head_dim: 1024
    bbox_reg_weights: "(10., 10., 5., 5.)"

    # Mask-RCNN heads.
    include_mask: True
    mrcnn_resolution: 28

    # training
    train_rpn_pre_nms_topn: 2000
    train_rpn_post_nms_topn: 1000
    train_rpn_nms_threshold: 0.7

    # evaluation
    test_detections_per_image: 100
    test_nms: 0.5
    test_rpn_pre_nms_topn: 1000
    test_rpn_post_nms_topn: 1000
    test_rpn_nms_thresh: 0.7

    # model architecture
    min_level: 2
    max_level: 6
    num_scales: 1
    aspect_ratios: "[(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]"
    anchor_scale: 8

    # localization loss
    rpn_box_loss_weight: 1.0
    fast_rcnn_box_loss_weight: 1.0
    mrcnn_weight_loss_mask: 1.0
}
#10

Got it. Could you set a lower threshold and retry?
See MaskRCNN — TAO Toolkit 4.0 documentation

  • -t, --threshold: The threshold for drawing a bbox (default: 0.6)
#11

This is the command used. threshold is 0.5

mask_rcnn inference -i /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/data/FishEyes_Training/val -o /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/maskrcnn_annotated_images -e /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/specs/maskrcnn_train_resnet50.txt -m /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/experiment_dir_unpruned/model.step-165000.tlt -l /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/specs/instance_label_singpilot.txt -t 0.5 -k nvidia_tlt --include_mask --gpu_index 2

#12

Could you share instance_label_singpilot.txt?
And please set to a lower threshold and retry. For quick experiment, you can test against less images.

#13

It was shared in the first post. I am testing now with 0.3.

#14

Tested with threshold 0.3.
Still not detected.

1500
15001280×720 89.9 KB

#15

The -l is the directory of predicted labels in COCO format. It is a json file. Could you double check?

#16

That could be the reason for N/A class for those detected classes. But no detection means I am referring to no bounding box.
For training, i have json file.

#17

Could you try t=0.1 ?

#18

This is 0.1, still same.

100
1001280×720 84.8 KB

#19

OK, please change the “-i” argument to use some of the training images.
We just want to narrow down to check if it works via some training images.

#20

Training image also not detected. Is it mask configuration? But mask size for dash line is similar to solid line.

1
11280×720 84.5 KB