Training Custom Model with EfficientDet_tf2 - Objects Not Detected After Training with Custom Dataset

AI & Data Science Deep Learning (Training & Inference) Frameworks
1

We followed the link provided below to train a custom model using EfficientDet_tf2: TAO Toolkit Getting Started | NVIDIA NGC

We successfully trained the model using the COCO dataset provided in the EfficientDet_tf2 notebook. During inference, the model correctly detects objects.

However, when we train the model with our own data, the training completes successfully, but it does not detect any objects.

Here are the details of our custom dataset training:

  1. Number of images for training: 1396
  2. We created annotation files as mentioned in the following link: Data Annotation Format - NVIDIA Docs The only change we made is setting the segmentation field to blank [ ] since we only have bounding boxes. While creating TF records, we set include_masks to False.
  3. Training time: 1 hour
  4. num_epochs: 300
  5. num_examples_per_epoch: 349
  6. learning_rate: 0.8
  7. image_size: ‘512x512’
  8. num_classes: 2
  9. Actual image resolution is 2592x1944 and are approximately 1.6MB in size
  10. Instance type: AWS g4dn.2xlarge, which has 4 GPUs

Apart from the above details, we kept almost all settings as they were.

We would appreciate any insights or suggestions regarding the issue we are facing. Thank you.
train.json (2.4 MB)

2

hey when I try to run tao efficientdet_tf2 data_convert but it cannot find efficientdet_tf2 giving this error: /bin/bash not found efficientdet_tf2, do you have any idea?

4

Please try efficientdet_tf2 notebook from:
wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/tao-getting-started/versions/4.0.1/zip -O getting_started_v4.0.1.zip
unzip -u getting_started_v4.0.1.zip -d ./getting_started_v4.0.1 && rm -rf getting_started_v4.0.1.zip && cd ./getting_started_v4.0.1

Make sure that nvidia-tao is installed with “pip3 install nvidia-tao”.

And its “dataset_convert” not “data_convert”