Black Mask Images While Converting COCO to UNET

[tahseenabbas983]

Hey,
I want to train UNET-18 using tao on my custom dataset. I have my dataset in COCO format with jpg images and i went through this post where i found the following command to convert the COCO Dataset Format to UNET

!tao model unet dataset_convert -f "/workspace/tao-experiments/data/new_merged_annotations.json" -r "/workspace/tao-experiments/results"

The issue is when I execute this command it generates the mask images with black screen only and in logs it shows the status as PASS.

The mask image is just a black image without any mask in it.
Can you help me how can i resolve this issue and create the exact masks to train the UNET.

[david9xqqb]

Are you doing grayscale foreground/background images? In my experience, for grayscale, I have only been able to create these two class models where one class has a mask value 0 and the another typically 255.

Can you share the coco_json_file ?

Notice that the default values are for this case. If your original masks are multiclass and / or your actual images are RGB you need to make changes for that!

Cheers!

[tahseenabbas983]

You can find the coco JSON file here https://drive.google.com/file/d/1x-yahDMH7vYqnJJqsuMZQqD9SVEGsg6v/view?usp=sharing
I am not sure about the grayscale foreground/background images, I have RGB images along with it with a single category. Can you please guide me about the changes so I can proceed to the training?

Also, I tried it on a sample of the coco dataset, for that data, it was generating masks properly.
Thanks for the reply…!

[david9xqqb]

In the dataset convert link you posted there is a mention of a configuration json file that look like this…

dataset_config {
    dataset: "custom"
    augment: True
    resize_padding: True
    resize_method: BILINEAR
    augmentation_config {
      spatial_augmentation {
        hflip_probability : 0.5
        vflip_probability : 0.5
        crop_and_resize_prob : 0.5
      }
      brightness_augmentation {
        delta: 0.2
      }
    }

    input_image_type: "grayscale"
    train_images_path:"/workspace/tao-experiments/data/unet/isbi/images/train"
    train_masks_path:"/workspace/tao-experiments/data/unet/isbi/masks/train"
    val_images_path:"/workspace/tao-experiments/data/unet/isbi/images/val"
    val_masks_path:"/workspace/tao-experiments/data/unet/isbi/masks/val"
    test_images_path:"/workspace/tao-experiments/data/unet/isbi/images/test"
    data_class_config {
      target_classes {
      name: "foreground"
      mapping_class: "foreground"
      label_id: 0
    }
    target_classes {
      name: "background"
      mapping_class: "background"
      label_id: 1
    }
  }
}

While I use unet frequently, I’ve never converted from COCO format. But in order to get good results from Unet, every pixel must have a mask class. So I think you need COCO annotations for every pixel.

Also, I am not an expert in COCO standard, but I recommend you check the COCO annotations in the JSON file you shared, since they seemed to me to be bounding boxes and not segmentation.

How did you generate the COCO annotations?

Can you share a sample image?

Also, note that you are training with very large images, and will need enough GPU memory. With 20GB I run out of memory with 1024 X 704 images with batch size 4

[tahseenabbas983]

I got to know something that the mask images being generated had two different values 0 and 1. when we see them in images they look complete black after that I changed the non-zero pixels’ value to 255 and i got the mask as i wanted. Then, I proceeded as per notebook and in visualization part i got the images along with the mask. For now i have started the training and will get back to you when it is completed.
Thanks for you feedback.

[Morganh]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Thanks for the info. More info can be found in Data Annotation Format - NVIDIA Docs.