Tao toolkit Augmentation not working

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc) : Jetson
• Network Type: Detectnet_v2
• TLT Version: NA
• Training spec file
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

2024-04-16 08:31:33.582805: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
/usr/local/lib/python3.6/dist-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.26.5) or chardet (3.0.4) doesn’t match a supported version!
2024-04-16 08:31:35,530 [INFO] iva.augment.spec_handler.spec_loader: Merging specification from aug_spec.txt
2024-04-16 08:31:35,532 [INFO] iva.augment.scripts.augment: Serializing augmentation spec
spatial_config {
rotation_config {
angle: 90.0
units: “degrees”
shear_config {
flip_config {
translation_config {
color_config {
hue_saturation_config {
saturation_shift: 1.0
contrast_config {
brightness_config {
blur_config {
std: 1.0
output_image_width: 960
output_image_height: 544
output_image_channel: 3
image_extension: “jpg”
dataset_config {
image_path: “images”
label_path: “labels”

2024-04-16 08:31:35,580 [INFO] iva.augment.scripts.augment: Listing files in the dataset.
2024-04-16 08:31:35,951 [INFO] iva.augment.scripts.augment: Preparing file fetchers for augmentation.
0%| | 8/4249 [00:00<00:58, 71.89it/s]
Augmentation run failed with error: All tensors in a batch must have the same number of dimensions

All my images are of same dimension and each has its own augmentation file, i have been able to run the training using the same files. I am using the following command :

augment -a aug_spec.txt -o /workspace/datasets/augmented -d /workspace/datasets/DS_data_merged -g 0

it works sometimes on other folders of the images but i cant pinpoint why

May I know that how many images and how many labels separately?

685 images and labels. It doesnt matter though, it fails even if i try 100 odd images, max i can give is like 10 images. the resolution is 960*544 and labels are in kitti format

You mentioned that it works sometimes on other folders. Can you check the difference between the folders?
Maybe the failed folder contains additional files which are unexpected?

Also, can you run below to check if there is some other other kind of image? RGBA?
$ file /your/image/folder/*

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.