Class Mapping in Detectnet_v2

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello everyone, I spent a couple of days trying to train a model with detectnet but it always returned “average precision = 0.0”. The dataset consists of 960x544 images (all images have the same dimension). The dataset contains two classes that, when training the model, are mapped to only one. The label classes are “hipopotamo_agua” and “hipopotamo_tierra”, which are mapped to “hippo”. An important point is that there is no object directly labeled with the label “hippo” in the dataset.

If the configuration of “dataset_config” is the following:

dataset_config {
  data_sources: {
    tfrecords_path: "/workspace/tlt-experiments/cemzoo_hippo_bkp/tf_records/*"
    image_directory_path: "/workspace/tlt-experiments/cemzoo_hippo_bkp/dataset/trainval"
  }
  image_extension: "jpg"
  target_class_mapping {
      key: 'hipopotamo_tierra'
      value: 'hippo'
  }
  target_class_mapping {
      key: 'hipopotamo_agua'
      value: 'hippo'
  }
  validation_fold: 0
}

Average precision is equal to zero throughout the training. However, in one of the tests configure “dataset_config” as follows:

dataset_config {
  data_sources: {
    tfrecords_path: "/workspace/tlt-experiments/cemzoo_hippo_bkp/tf_records/*"
    image_directory_path: "/workspace/tlt-experiments/cemzoo_hippo_bkp/dataset/trainval"
  }
  image_extension: "jpg"
  target_class_mapping {
      key: 'hipopotamo_tierra'
      value: 'hippo'
  }
  target_class_mapping {
      key: 'hipopotamo_agua'
      value: 'hippo'
  }
  target_class_mapping {
      key: 'hippo'
      value: 'hippo'
  }
  validation_fold: 0
}

I finally “average precision” started to rise. In the TLT sample code the classes “van” and “car” are mapped to “car”, but in this case in the dataset the class “car” does exist. In my case the class “hippo” does not exist originally in the dataset labels.
My query: Is this the correct way to configure dataset_config ?, I have to do the mapping from hippo (key) to hippo (value) even though I don’t have “hippo” tags in the dataset?

1 Like
3

Could you please show the full log when you run tlt-dataset-convert?

From user guide,

target_class_mapping : This parameter maps the class names in the tfrecords to the target class to be trained in the network. An element is defined for every source class to target class mapping. This field was included with the intention of grouping similar class objects under one umbrella. For eg: car, van, heavy_truck etc may be grouped under automobile. The “key” field is the value of the class name in the tfrecords file, and “value” field corresponds to the value that the network is expected to learn.

The class names key in the target_class_mapping must be identical to the one shown in the dataset converter log, so that the correct classes are picked up for training.

4

Thanks for your response @Morganh. The full log is:

Converting Tfrecords for trainval dataset
2020-12-03 15:50:43.025548: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
Using TensorFlow backend.
2020-12-03 15:50:47,309 - iva.detectnet_v2.dataio.build_converter - INFO - Instantiating a kitti converter
2020-12-03 15:50:47,315 - iva.detectnet_v2.dataio.kitti_converter_lib - INFO - Num images in
Train: 827 Val: 206
2020-12-03 15:50:47,316 - iva.detectnet_v2.dataio.kitti_converter_lib - INFO - Validation data in partition 0. Hence, while choosing the validationset during training choose validation_fold 0.
2020-12-03 15:50:47,318 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 0
WARNING:tensorflow:From /home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataio/dataset_converter_lib.py:142: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

2020-12-03 15:50:47,318 - tensorflow - WARNING - From /home/vpraveen/.cache/dazel/_dazel_vpraveen/715c8bafe7816f3bb6f309cd506049bb/execroot/ai_infra/bazel-out/k8-py3-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataio/dataset_converter_lib.py:142: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

/usr/local/lib/python3.6/dist-packages/iva/detectnet_v2/dataio/kitti_converter_lib.py:273: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default.
2020-12-03 15:50:47,571 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 1
2020-12-03 15:50:47,689 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 2
2020-12-03 15:50:47,906 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 3
2020-12-03 15:50:48,080 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 4
2020-12-03 15:50:48,184 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 5
2020-12-03 15:50:48,246 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 6
2020-12-03 15:50:48,386 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 7
2020-12-03 15:50:48,512 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 8
2020-12-03 15:50:48,640 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 9
2020-12-03 15:50:48,742 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO -
Wrote the following numbers of objects:
b’hipopotamo_agua’: 198
b’hipopotamo_tierra’: 53

2020-12-03 15:50:48,742 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 0
2020-12-03 15:50:49,128 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 1
2020-12-03 15:50:49,552 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 2
2020-12-03 15:50:49,911 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 3
2020-12-03 15:50:50,390 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 4
2020-12-03 15:50:50,758 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 5
2020-12-03 15:50:51,160 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 6
2020-12-03 15:50:51,698 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 7
2020-12-03 15:50:52,042 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 8
2020-12-03 15:50:52,544 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 1, shard 9
2020-12-03 15:50:53,061 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO -
Wrote the following numbers of objects:
b’hipopotamo_agua’: 839
b’hipopotamo_tierra’: 209

2020-12-03 15:50:53,061 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Cumulative object statistics
2020-12-03 15:50:53,061 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO -
Wrote the following numbers of objects:
b’hipopotamo_agua’: 1037
b’hipopotamo_tierra’: 262

2020-12-03 15:50:53,062 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Class map.
Label in GT: Label in tfrecords file
b’hipopotamo_agua’: b’hipopotamo_agua’
b’hipopotamo_tierra’: b’hipopotamo_tierra’
For the dataset_config in the experiment_spec, please use labels in the tfrecords file, while writing the classmap.

2020-12-03 15:50:53,062 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Tfrecords generation complete.

5

Please try

dataset_config {
  data_sources: {
    tfrecords_path: "/workspace/tlt-experiments/cemzoo_hippo_bkp/tf_records/*"
    image_directory_path: "/workspace/tlt-experiments/cemzoo_hippo_bkp/dataset/trainval"
  }
  image_extension: "jpg"
  target_class_mapping {
      key: 'hipopotamo_agua'
      value: 'hippo'
  }
  target_class_mapping {
      key: 'hipopotamo_tierra'
      value: 'hippo'
  }
  validation_fold: 0
}
6

Ohh, I just noticed that I made a mistake writing the post, I put repeat twice “hippopotamus_water”. However my original training file was written as you are recommending.

dataset_config {
data_sources: {
tfrecords_path: “/workspace/tlt-experiments/cemzoo_hippo_bkp/tf_records/*”
image_directory_path: “/workspace/tlt-experiments/cemzoo_hippo_bkp/dataset/trainval”
}
image_extension: “jpg”
target_class_mapping {
key: ‘hipopotamo_agua’
value: ‘hippo’
}
target_class_mapping {
key: ‘hipopotamo_tierra’
value: ‘hippo’
}
validation_fold: 0
}

After adding the “third” mapping (hippo-> hippo) it started to have average precision greater than zero.