Deformable_detr training with custom dataset :: TFrecords generating for {'person': 1, 'face': 2, 'bag': 3} class

[neuroSparK]

Please provide the following information when requesting support.

• TLT Version 5.0

I am trying to create TFrecords for a large amount of data for TAO toolkit detection training. I am following the tutorial here Deformable DETR - NVIDIA Docs
Here is my spec file

input_source: /workspace/tao-experiments/data/seq.txt
results_dir: /workspace/tao-experiments/sharded
image_dir_name: images
label_dir_name: labels
num_shards: 32
num_partitions: 1

But when I try to run the command !tao model deformable_detr convert -e $SPECS_DIR/data2TFrec.yaml it shows

Starting DDETR/DINO dataset convert
/workspace/tao-experiments//data/LocationOD2

Category mapping: 
 {'person': 1, 'face': 2, 'bag': 3}
Generating partitions
Writing partition 0, shard 0
Writing partition 0, shard 0
/usr/local/lib/python3.8/dist-packages/nvidia_tao_pytorch/cv/deformable_detr/utils/converter.py:457: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default.
 labels = np.genfromtxt(label_file, dtype=None).tolist()

I am surprised to see the

Category mapping: 
 {'person': 1, 'face': 2, 'bag': 3}

as my custom dataset images and corresponding label files have 7 vehicle categories rather than the {‘person’: 1, ‘face’: 2, ‘bag’: 3} , Where is there categories coming? How can I set class mapping for my custom categories?

[Morganh]

@neuroSparK

Starting DDETR/DINO dataset convert
/workspace/tao-experiments//data/LocationOD2

Category mapping: 
 {'person': 1, 'face': 2, 'bag': 3}

Please try to add mapping_path: xxx.json in the spec file.
This is based on
https://github.com/NVIDIA/tao_pytorch_backend/blob/main/nvidia_tao_pytorch/cv/deformable_detr/scripts/convert.py#L43 and https://github.com/NVIDIA/tao_pytorch_backend/blob/main/nvidia_tao_pytorch/cv/deformable_detr/utils/converter.py#L119.

mapping_path (str): Path to a JSON file containing the class mapping.

[neuroSparK]

@Morganh Thanks for the insight. I have created the classmap.json file like this:

{
  "1" : "bicycle",
  "2" : "motorcycle",
  "3" : "car",
  "4" : "bus",
  "5" : "truck",
  "6" : "threewheeler",
  "7" : "nonmotorized"  
}

However, when I execute !tao model dino convert -e $SPECS_DIR/spec.yaml where the spec.yaml is like this

input_source: /workspace/tao-experiments/data/seq.txt
results_dir: /workspace/tao-experiments/sharded
image_dir_name: images
label_dir_name: labels
num_shards: 4
num_partitions: 1
mapping_path: /workspace/tao-experiments/data/classmap.json

I see the tao convert output shows the category in random order/reverse order

Creating output directory /workspace/tao-experiments/sharded/Location8
Load category mapping from /workspace/tao-experiments/data/bdvehiclenet_category_map.json
Category mapping: 
 {'truck': 1, 'threewheeler': 2, 'nonmotorized': 3, 'motorcycle': 4, 'car': 5, 'bus': 6, 'bicycle': 7}

I am confused why it’s happening. The mapping is not documented in the TAO documentation thus creating confusion.

[Morganh]

@neuroSparK

{
  "1" : "bicycle",
  "2" : "motorcycle",
  "3" : "car",
  "4" : "bus",
  "5" : "truck",
  "6" : "threewheeler",
  "7" : "nonmotorized"  
}

Please generate the json file according to https://github.com/NVIDIA/tao_pytorch_backend/blob/main/nvidia_tao_pytorch/cv/deformable_detr/utils/misc.py#L28-L44. For your case, it is as below.

{
  "bicycle" : "bicycle",
  "motorcycle" : "motorcycle",
  "car" : "car",
  "bus" : "bus",
  "truck" : "truck",
  "threewheeler" : "threewheeler",
  "nonmotorized" : "nonmotorized"
}

Your result is expected. It will be sorted and also reverse. More can be found in https://github.com/NVIDIA/tao_pytorch_backend/blob/main/nvidia_tao_pytorch/cv/deformable_detr/utils/misc.py#L62

[neuroSparK]

@Morganh thanks. I have changed the classmap accordingly but still getting the output as bellow:

Category mapping: 
 {'truck': 1, 'threewheeler': 2, 'nonmotorized': 3, 'motorcycle': 4, 'car': 5, 'bus': 6, 'bicycle': 7}

Is it normal? If so, then isn’t is ambiguous to see truck as the first category where’s truck is my 5th category? I assume it won’t affect training, but I have to follow this classmap during inferencing. Am I right?

[Morganh]

It is normal to get this result. It is in reverse alphabetic order.