Training DeformableDETR with custom dataset

• Hardware V100
• Network Type - Detection using Deformable DETR
• TLT Version 3.0

I am training DeformableDETR for the first time and have some questions relating to the information provided here: DeformableDETR - NVIDIA Docs

  1. Data input - it appears that the data size is modified during the augmentation. Does it mean that images may be stretched max 1330px? I wanted to find out if we can use 640px x 640px input images. If possible, we would like to use even smaller images for the training. OR should we provide larger images to allow the augmentation? If so, what is the recommended input image size?

  2. we are going to have 3 SGIE detectors. For SGIE, it will obviously crop teh PGIE and then run SGIE model only on that part of the image. With that, I am assuming that our ROI will be even smaller. So, would it, in fact, help me to train SGIE models on smaller images e.g., 32px x 32px. This question is more on the usability side but I’d really appreciate if you could please provide some info.

Thank you so much for your help.

The 1333 is default random_resize_max_size. You can change it. And also, user can set different scales, train_random_resize , train_random_crop_min , train_random_crop_max. So, it is possible to use 640px x 640px input images or smaller.

1 Like

Thanks Morgan,

Just to double confirm…In terms of the solution practicality and how it is all handled by the Deformable DETR…is it ok and practical to go with 640px 640px? Whats the normal case scenario?

For user case, you can refer to the notebook. notebooks/tao_launcher_starter_kit/deformable_detr/deformable_detr.ipynb

wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/tao-getting-started/versions/4.0.2/zip
1 Like

My question was more so on practicality…is it normally considered practical to use 640x640 input images on deformable detr?

Also, the notebook link you gave seems to be for the detectnet. Is there any specific notebook for DeformableDETR?

Sorry, for this one.
notebooks/tao_launcher_starter_kit/deformable_detr/deformable_detr.ipynb

It is ok to use 640x640. But it is hard to say which one is the best. Usually for different dataset, experiments are expected.

1 Like

Thanks Morgan :)

One additional question regarding the dataset. I can see that the coco dataset format is accepted. Is there a way to use polygon annotation instead of the rectangle for DeformableDETR?

To add a bit of context here…i wanted to know if i can provide input images with segmentation annotation and as output receive instance segmentation. I understand that the DETR by default supports bounding boxes, however, just wanted to know if Deformable DETR has that extra layer to support instance segmentation.

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Currently, just default bounding box format from COCO is supported.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.