Dealing with Class Imbalance in SegFormer Semantic Segmentation

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc) T4
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc) SegFormer
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here) TAO 5.5.0
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

I have SegFormer working with my semantic segmentation use case. I’ve got two classes - background and cables. The problem is during training I end up with nearly 100% accuracy for background and close to 0% for cables. My cables only take up 0.4% of the pixels in the image so there is a huge class imbalance. Research suggests the class imbalance may be the reason for the accuracy issue. Is there a recommended way to configure SegFormer to deal with this? One suggestion was to use a different loss function but I’m not sure how to configure that.

Any help would be appreciated.

Duane Harkness
Rendered.ai

1 Like
2

That means the training does not converge yet.

It is not the reason. The network can work for this case.

Please make sure the mask images are expected. See https://docs.nvidia.com/tao/tao-toolkit/text/data_annotation_format.html#semantic-segmentation-mask-format.
For color/ RGB input images, each mask image is a single-channel or three-channel image with the size equal to the input image. Every pixel in the mask must have an integer value that represents the segmentation class label_id.

Please check every pixel of the mask image.