Data Aumentation techniques in Nvidia Tao

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc) : A40
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc) : Detectnet_v2
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here): nvidia/tao/tao-toolkit: 5.5.0-pyt
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

I have question regarding data augementation techniques availabel in tao.

Basically I have three questions:

a.) Available Data Augmentation techniques in Tao
b.)How we can select/configure Data Augmentation techniques while training a model using Tao .
c.) How can we integrate custom data Augmentation technique to tao that is not available in tao by default.

The default Augmentation config I can see for DetectNet V2 is as below:

augmentation_config {
  preprocessing {
    output_image_width: 1248
    output_image_height: 384
    min_bbox_width: 1.0
    min_bbox_height: 1.0
    output_image_channel: 3
  }
  spatial_augmentation {
    hflip_probability: 0.5
    zoom_min: 1.0
    zoom_max: 1.0
    translate_max_x: 8.0
    translate_max_y: 8.0
  }
  color_augmentation {
    hue_rotation_max: 25.0
    saturation_shift_max: 0.20000000298
    contrast_scale_max: 0.10000000149
    contrast_center: 0.5
  }
}

This is also specified here in the DetectnetV2 Docs: DetectNet_v2 - NVIDIA Docs

2

The TAO Toolkit supports both offline and online data augmentation techniques. Offline augmentation includes:

  • Spatial Augmentation: Rotate, Resize, Translate, Shear, Flip.
  • Color Space Augmentation: Hue Rotation, Brightness offset, Contrast shift.
  • Image Blur: Gaussian blur.
    Refer to Offline Data Augmentation - NVIDIA Docs.

Online augmentation allows for customization of spatial and color augmentations during training, similar to offline methods but applied randomly based on the data loader’s distribution.

To configure data online augmentation in TAO detectnet_v2, you can follow these steps:

  • Configure the training spec file to include online augmentation parameters.
  • During training, TAO will apply these augmentations randomly to the data

Additional: Integrating Custom Data Augmentation Techniques

To integrate custom data augmentation techniques not available in TAO by default:

  1. Modify the TAO Docker Environment:
  1. Customize the Training Pipeline:
  • Modify the training scripts to include your custom augmentation functions.
  • Ensure these functions are applied to the data before it is fed into the model.

This approach requires modifying the TAO codebase or integrating your custom augmentations within the TAO Docker environment. The code is in tao_tensorflow1_backend/nvidia_tao_tf1/cv/detectnet_v2 at main · NVIDIA/tao_tensorflow1_backend · GitHub. They are also available inside the tao docker.(Under /usr/xxx)