Fine-Tune the TAO v5.5.0 Mask2former Instance segmentation model on a custom dataset

[dhara8877]

• Hardware (x86_ Linux ubuntu jammy 22.04, RTX 3090)
• Network Type (Mask2Former)
• Training spec file(spec.txt (1.9 KB))

Can you help me to fine tune my Mask2former Instance segmentation model on a custom dataset

Dataset has 3880 - training samples and 970 - validation samples(The annotations are in coco format),
I have 5 classes (in that 2 classes have extremely more labels comparative to other 3 classes).

My Image file dimension are width: 2208 and height: 1242 in .png format

Can you just specify any online augmentation techniques such as flips , rotation, etc. which can be included in augmentation config to handle class imbalance, because i don’t see anything like that in the docs.

Below i share my training graphs for visualization can help out to get an accuracy upto 97 to 99%

[Morganh]

@dhara8877

Can you just specify any online augmentation techniques such as flips , rotation, etc. which can be included in augmentation config to handle class imbalance, because i don’t see anything like that in the docs.

For augmentation, you can refer to tao_pytorch_backend/nvidia_tao_pytorch/cv/mask2former/dataloader/datasets.py at dc07b02eb78c2eb868315107892b466496e55a0f · NVIDIA/tao_pytorch_backend · GitHub and tao_pytorch_backend/nvidia_tao_pytorch/cv/mask2former/dataloader/augmentations.py at dc07b02eb78c2eb868315107892b466496e55a0f · NVIDIA/tao_pytorch_backend · GitHub. You can run inside the 5.5.0 docker directly and find/modify these codes if needed.

[dhara8877]

Can you mention any fine-tuning in my spec file , that I need to change from the insights of my training graphs that I shared above, which will eventually increase my accuracy to more than 97%

[Morganh]

@dhara8877

I need to change from the insights of my training graphs that I shared above, which will eventually increase my accuracy to more than 97%

From your latest result, the mIOU is about 68%. Do you mean the target mIOU is above 97%?

[dhara8877]

Yes, I meant mIoU

[Morganh]

Please try to modify below and run experiments.

1. Change train_min_size to 1024 and crop_size to [1024,1024].
2. Set higher num_object_queries in case there are lots of objects in each image.

More, you can try
3. swinL . Refer to tao_pytorch_backend/nvidia_tao_pytorch/cv/mask2former/experiment_specs/spec_coco.yaml at dc07b02eb78c2eb868315107892b466496e55a0f · NVIDIA/tao_pytorch_backend · GitHub.
4. Train against the 2 classes which have extremely more labels
5. Make dataset more balance

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks