The first class is always not detected in inference

[edit_or]

The green box is detection for solid line.
Our mask labelling for dash line is like below in red.

All images with those dash lines have no detection at all.

Is it because of mask config? Which one should I change?

maskrcnn_config {
    nlayers: 50
    arch: "resnet"
    freeze_bn: False
    #freeze_blocks: "[0,1]"
    gt_mask_size: 112
        
    # Region Proposal Network
    rpn_positive_overlap: 0.7
    rpn_negative_overlap: 0.3
    rpn_batch_size_per_im: 256
    rpn_fg_fraction: 0.5
    rpn_min_size: 0.

    # Proposal layer.
    batch_size_per_im: 512
    fg_fraction: 0.25
    fg_thresh: 0.5
    bg_thresh_hi: 0.5
    bg_thresh_lo: 0.

    # Faster-RCNN heads.
    fast_rcnn_mlp_head_dim: 1024
    bbox_reg_weights: "(10., 10., 5., 5.)"

    # Mask-RCNN heads.
    include_mask: True
    mrcnn_resolution: 28

    # training
    train_rpn_pre_nms_topn: 2000
    train_rpn_post_nms_topn: 1000
    train_rpn_nms_threshold: 0.7

    # evaluation
    test_detections_per_image: 100
    test_nms: 0.5
    test_rpn_pre_nms_topn: 1000
    test_rpn_post_nms_topn: 1000
    test_rpn_nms_thresh: 0.7

    # model architecture
    min_level: 2
    max_level: 6
    num_scales: 1
    aspect_ratios: "[(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]"
    anchor_scale: 8

    # localization loss
    rpn_box_loss_weight: 1.0
    fast_rcnn_box_loss_weight: 1.0
    mrcnn_weight_loss_mask: 1.0
}

[Morganh]

Got it. Could you set a lower threshold and retry?
See MaskRCNN — TAO Toolkit 4.0 documentation

• -t, --threshold: The threshold for drawing a bbox (default: 0.6)

[edit_or]

This is the command used. threshold is 0.5

mask_rcnn inference -i /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/data/FishEyes_Training/val -o /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/maskrcnn_annotated_images -e /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/specs/maskrcnn_train_resnet50.txt -m /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/experiment_dir_unpruned/model.step-165000.tlt -l /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/specs/instance_label_singpilot.txt -t 0.5 -k nvidia_tlt --include_mask --gpu_index 2

[Morganh]

Could you share instance_label_singpilot.txt?
And please set to a lower threshold and retry. For quick experiment, you can test against less images.

[edit_or]

It was shared in the first post. I am testing now with 0.3.

[edit_or]

Tested with threshold 0.3.
Still not detected.

[Morganh]

@edit_or

It was shared in the first post

The -l is the directory of predicted labels in COCO format. It is a json file. Could you double check?

[edit_or]

That could be the reason for N/A class for those detected classes. But no detection means I am referring to no bounding box.
For training, i have json file.

[Morganh]

Could you try t=0.1 ?

[edit_or]

This is 0.1, still same.

[Morganh]

@edit_or

-i /workspace/Nyan/cv_samples_v1.3.0/mask_rcnn/data/FishEyes_Training/val

OK, please change the “-i” argument to use some of the training images.
We just want to narrow down to check if it works via some training images.

[edit_or]

Training image also not detected. Is it mask configuration? But mask size for dash line is similar to solid line.

[Morganh]

Suggest to double check the label of each training image. To check the “single_broken_white_line” class.

• segmentation value
• bbox value

If possible, could you share the training json file?

[edit_or]

I used the same json file and trained with mmdetection maskrcnn, detection is well for dash lines.

[Morganh]

More, need to check the id and category id, etc.

[edit_or]

train_annotation_coco.json (2.2 MB)

yes, thanks
id": 0, “name”: “single_broken_white_line”

[Morganh]

The reason is found.

“categories”: [{“id”: 0, “name”: “single_broken_white_line”}, {“id”: 1, “name”: “continuous_white_line”}, {“id”: 2, “name”: “parallel_continuous_white_line”}, {“id”: 3, “name”: “single_continuous_yellow_line”}, {“id”: 4, “name”: “parallel_continuous_yellow_line”}, {“id”: 5, “name”: “stop_line”}, {“id”: 6, “name”: “hatched_road_marking”}, {“id”: 7, “name”: “stop_text”}, {“id”: 8, “name”: “straight_arrow”}, {“id”: 9, “name”: “left_arrow”}, {“id”: 10, “name”: “right_arrow”}, {“id”: 11, “name”: “straight_left_arrow”}, {“id”: 12, “name”: “straight_right_arrow”}, {“id”: 13, “name”: “straight_left_right_arrow”}, {“id”: 14, “name”: “yellow_box”}, {“id”: 15, “name”: “yellow_hatched”}, {“id”: 16, “name”: “black_white_carb”}, {“id”: 17, “name”: “hump”}, {“id”: 18, “name”: “jagged_line”}, {“id”: 19, “name”: “light_truck”}, {“id”: 20, “name”: “heavy_vehicle”}, {“id”: 21, “name”: “corner_carb”}]}

In categories dict, the id should start from 1 instead of 0. That means please change to

“categories”: [{“id”: 1, “name”: “single_broken_white_line”}, {“id”: 2, “name”: “continuous_white_line”}, {“id”: 3, “name”: “parallel_continuous_white_line”} …

In annotations dict, the category_id should start from 1 instead of 0. That means please change each image’s “category_id” as well. To match above-mentioned “categories” dict.

Sorry for the inconvenient. Currently, the dataset_convert does not help customer check this. We will improve this to do value checking during tfrecords generation via dataset_convert to avoid such issues.

[edit_or]

@Morganh

1 instead of 0

Ic, thanks. Let me update.

[Morganh]

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Thanks for that. It is a bit heavy work.

• Change the id in categories dict
• Also change the category_id of each image.