Low batch size during training

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Just notices a strange thing. Why the BS is so small during training? With Faster RCNN (resnet50-101) I can only fit 1 image on 2080TI with res=320. And for some small detectors like SSD/Yolo with MobileNet the BS is about 16-32, which is also quite small for such simple models.

Keep in mind that I unfroze all blocks and here is my preprocessing config.

augmentation_config {
  preprocessing {
    output_image_width: 320
    output_image_height: 320
    output_image_channel: 3
    crop_right: 320
    crop_bottom: 320
    min_bbox_width: 1.0
    min_bbox_height: 1.0
  }
3

For bs in Faster-rcnn, refer to Faster RCNN ResNet-101 Problems - #10 by Morganh
ResNet101 is a huge backbone and can not fit into a single GPU with a large batch size like 16.

For SSD/Yolo, the bs can be set above 32.