YOLOV4 AP all zero in custom dataset

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc) Titan XP
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc) Yolov4
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here) TAO(TLT3)
• Training spec file(If have, please share here)
yolo_v4_train_cspdarknet53_kitti.txt (3.3 KB)

• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

I was training with my own dataset in KITTI format and converted to tfrecord by running !tao yolo_v4 dataset_convert in the jupyter notebook. In the training phase, I set the checkpoint to every epoch and it prints out the AP across classes. I saw all zeros as APs and as well as the APs in evaluation phase like below:

Producing predictions: 100%|████████████████| 2725/2725 [05:57<00:00,  7.63it/s]
Start to calculate AP for each class
*******************************
ambulance     AP    0.0
bicycle       AP    0.0
bus           AP    0.0
car           AP    0.0
carwithtrailersAP    0.0
escooter      AP    0.0
fireengine    AP    0.0
industrialvehicleAP    0.0
motorcycle    AP    0.0
pedestrian    AP    0.0
pickuptruck   AP    0.0
policecar     AP    0.0
schoolbus     AP    0.0
semitruck     AP    0.0
trafficcone   AP    0.0
wheelchair    AP    0.0
workvan       AP    0.0
              mAP   0.0
*******************************

And in visualizing predicted labels, I didn’t see bounding boxes in the images.

Please help me. Thanks!

3

Could you please share one label file?
I want to check the class name in it.

4

I took a look at those label files as output of inference, and they are all blank files.

Here is an example how my original kitti label looks for train/val/test.
video32_2_350.txt (281 Bytes)
I build tfrecord from there.

5

Refer to SSD: custom tlt training result in AP:0 for all my classes - #6 by Morganh

Please modify your spec, set to lowercase class name.
For example,
target_class_mapping {
key: “Car”
value: “Car”
}

to

target_class_mapping {
key: “car”
value: “car”
}

6

Ok. Do I need to set the names to lower case in kitti format label files as well?

7

Not needed. Just try to modify your spec and then run
!tao yolo_v4 evaluate xxx

to check if it works.

8

Hi Morgan,

I changed them all to lower cases but the APs are still 0.0 as the same. I wonder if there is anything wrong with the tfrecord I created?

9

Can you share the latest training spec?
More, did you ever save the log of tfrecords generation? If yes, please share as well.

10

Yes here is the latest config.
yolo_v4_train_cspdarknet53_kitti.txt (3.3 KB)

Unfortunately, I lost the log of my training. I will resume from my latest model for another couple of epochs and post it here for sharing soon.

11

Not the log of training. Just the log of tfrecords generation when you run " !tao yolo_v4 dataset_convert "

12

Sorry for misunderstanding. Below is the log for converting tfrecord:

2021-12-03 15:02:11,321 [INFO] root: Registry: ['nvcr.io']
2021-12-03 15:02:11,468 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.5-py3
2021-12-03 15:02:11,724 [WARNING] tlt.components.docker_handler.docker_handler: 
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the "user":"UID:GID" in the
DockerOptions portion of the "/home/dwang/.tao_mounts.json" file. You can obtain your
users UID and GID by using the "id -u" and "id -g" commands on the
terminal.
Using TensorFlow backend.
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
2021-12-03 23:02:20,243 [INFO] iva.detectnet_v2.dataio.build_converter: Instantiating a kitti converter
2021-12-03 23:02:20,244 [INFO] root: Instantiating a kitti converter
2021-12-03 23:02:20,244 [INFO] root: Generating partitions
2021-12-03 23:02:20,535 [INFO] iva.detectnet_v2.dataio.kitti_converter_lib: Num images in
Train: 84396	Val: 13738
2021-12-03 23:02:20,535 [INFO] root: Num images in
Train: 84396	Val: 13738
2021-12-03 23:02:20,535 [INFO] iva.detectnet_v2.dataio.kitti_converter_lib: Validation data in partition 0. Hence, while choosing the validationset during training choose validation_fold 0.
2021-12-03 23:02:20,535 [INFO] root: Validation data in partition 0. Hence, while choosing the validationset during training choose validation_fold 0.
2021-12-03 23:02:20,619 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 0
2021-12-03 23:02:20,619 [INFO] root: Writing partition 0, shard 0
WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataio/dataset_converter_lib.py:161: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

2021-12-03 23:02:20,619 [WARNING] tensorflow: From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataio/dataset_converter_lib.py:161: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

/usr/local/lib/python3.6/dist-packages/iva/detectnet_v2/dataio/kitti_converter_lib.py:297: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default.
2021-12-03 23:02:21,997 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 1
2021-12-03 23:02:21,997 [INFO] root: Writing partition 0, shard 1
2021-12-03 23:02:23,404 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 2
2021-12-03 23:02:23,404 [INFO] root: Writing partition 0, shard 2
2021-12-03 23:02:24,778 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 3
2021-12-03 23:02:24,778 [INFO] root: Writing partition 0, shard 3
2021-12-03 23:02:26,145 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 4
2021-12-03 23:02:26,145 [INFO] root: Writing partition 0, shard 4
2021-12-03 23:02:27,506 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 5
2021-12-03 23:02:27,507 [INFO] root: Writing partition 0, shard 5
2021-12-03 23:02:28,878 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 6
2021-12-03 23:02:28,878 [INFO] root: Writing partition 0, shard 6
2021-12-03 23:02:30,246 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 7
2021-12-03 23:02:30,246 [INFO] root: Writing partition 0, shard 7
2021-12-03 23:02:31,620 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 8
2021-12-03 23:02:31,620 [INFO] root: Writing partition 0, shard 8
2021-12-03 23:02:32,996 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 9
2021-12-03 23:02:32,996 [INFO] root: Writing partition 0, shard 9
2021-12-03 23:02:34,375 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: 
Wrote the following numbers of objects:
b'pedestrian': 8035
b'car': 97990
b'bicycle': 1842
b'workvan': 2135
b'trafficcone': 4623
b'pickuptruck': 5747
b'bus': 951
b'semitruck': 497
b'industrialvehicle': 520
b'schoolbus': 95
b'carwithtrailers': 151
b'motorcycle': 204
b'escooter': 22
b'wheelchair': 17
b'policecar': 34
b'fireengine': 10
b'ambulance': 13

2021-12-03 23:02:34,376 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 0
2021-12-03 23:02:34,376 [INFO] root: Writing partition 1, shard 0
2021-12-03 23:02:42,819 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 1
2021-12-03 23:02:42,819 [INFO] root: Writing partition 1, shard 1
2021-12-03 23:02:51,242 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 2
2021-12-03 23:02:51,243 [INFO] root: Writing partition 1, shard 2
2021-12-03 23:02:59,677 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 3
2021-12-03 23:02:59,677 [INFO] root: Writing partition 1, shard 3
2021-12-03 23:03:08,268 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 4
2021-12-03 23:03:08,268 [INFO] root: Writing partition 1, shard 4
2021-12-03 23:03:16,739 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 5
2021-12-03 23:03:16,739 [INFO] root: Writing partition 1, shard 5
2021-12-03 23:03:25,312 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 6
2021-12-03 23:03:25,312 [INFO] root: Writing partition 1, shard 6
2021-12-03 23:03:33,871 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 7
2021-12-03 23:03:33,871 [INFO] root: Writing partition 1, shard 7
2021-12-03 23:03:42,368 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 8
2021-12-03 23:03:42,368 [INFO] root: Writing partition 1, shard 8
2021-12-03 23:03:51,033 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 9
2021-12-03 23:03:51,033 [INFO] root: Writing partition 1, shard 9
2021-12-03 23:03:59,628 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: 
Wrote the following numbers of objects:
b'pedestrian': 51409
b'car': 607200
b'bus': 5790
b'pickuptruck': 35549
b'trafficcone': 29750
b'carwithtrailers': 974
b'workvan': 13109
b'semitruck': 3011
b'bicycle': 11484
b'industrialvehicle': 3136
b'motorcycle': 1341
b'schoolbus': 580
b'policecar': 188
b'fireengine': 69
b'ambulance': 99
b'wheelchair': 97
b'escooter': 94
b'w': 1

2021-12-03 23:03:59,629 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Cumulative object statistics
2021-12-03 23:03:59,629 [INFO] root: Cumulative object statistics
2021-12-03 23:03:59,629 [INFO] root: {
    "pedestrian": 59444,
    "car": 705190,
    "bicycle": 13326,
    "workvan": 15244,
    "trafficcone": 34373,
    "pickuptruck": 41296,
    "bus": 6741,
    "semitruck": 3508,
    "industrialvehicle": 3656,
    "schoolbus": 675,
    "carwithtrailers": 1125,
    "motorcycle": 1545,
    "escooter": 116,
    "wheelchair": 114,
    "policecar": 222,
    "fireengine": 79,
    "ambulance": 112,
    "w": 1
}
2021-12-03 23:03:59,629 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: 
Wrote the following numbers of objects:
b'pedestrian': 59444
b'car': 705190
b'bicycle': 13326
b'workvan': 15244
b'trafficcone': 34373
b'pickuptruck': 41296
b'bus': 6741
b'semitruck': 3508
b'industrialvehicle': 3656
b'schoolbus': 675
b'carwithtrailers': 1125
b'motorcycle': 1545
b'escooter': 116
b'wheelchair': 114
b'policecar': 222
b'fireengine': 79
b'ambulance': 112
b'w': 1

2021-12-03 23:03:59,629 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Class map. 
Label in GT: Label in tfrecords file 
b'Pedestrian': b'pedestrian'
b'Car': b'car'
b'Bicycle': b'bicycle'
b'WorkVan': b'workvan'
b'TrafficCone': b'trafficcone'
b'PickupTruck': b'pickuptruck'
b'Bus': b'bus'
b'SemiTruck': b'semitruck'
b'IndustrialVehicle': b'industrialvehicle'
b'SchoolBus': b'schoolbus'
b'CarWithTrailers': b'carwithtrailers'
b'Motorcycle': b'motorcycle'
b'eScooter': b'escooter'
b'WheelChair': b'wheelchair'
b'PoliceCar': b'policecar'
b'FireEngine': b'fireengine'
b'Ambulance': b'ambulance'
b'w': b'w'
2021-12-03 23:03:59,629 [INFO] root: Class map. 
Label in GT: Label in tfrecords file 
b'Pedestrian': b'pedestrian'
b'Car': b'car'
b'Bicycle': b'bicycle'
b'WorkVan': b'workvan'
b'TrafficCone': b'trafficcone'
b'PickupTruck': b'pickuptruck'
b'Bus': b'bus'
b'SemiTruck': b'semitruck'
b'IndustrialVehicle': b'industrialvehicle'
b'SchoolBus': b'schoolbus'
b'CarWithTrailers': b'carwithtrailers'
b'Motorcycle': b'motorcycle'
b'eScooter': b'escooter'
b'WheelChair': b'wheelchair'
b'PoliceCar': b'policecar'
b'FireEngine': b'fireengine'
b'Ambulance': b'ambulance'
b'w': b'w'
For the dataset_config in the experiment_spec, please use labels in the tfrecords file, while writing the classmap.

2021-12-03 23:03:59,630 [INFO] root: For the dataset_config in the experiment_spec, please use labels in the tfrecords file, while writing the classmap.

2021-12-03 23:03:59,630 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Tfrecords generation complete.
2021-12-03 23:03:59,630 [INFO] root: TFRecords generation complete.
2021-12-03 15:04:00,619 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
!tao yolo_v4 dataset_convert 
!tao yolo_v4 dataset_convert -d $SPECS_DIR/yolo_v4_tfrecords_kitti_val.txt \
                             -o $DATA_DOWNLOAD_DIR/val/tfrecords/val
2021-12-03 15:04:17,839 [INFO] root: Registry: ['nvcr.io']
2021-12-03 15:04:17,991 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.5-py3
2021-12-03 15:04:18,106 [WARNING] tlt.components.docker_handler.docker_handler: 
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the "user":"UID:GID" in the
DockerOptions portion of the "/home/dwang/.tao_mounts.json" file. You can obtain your
users UID and GID by using the "id -u" and "id -g" commands on the
terminal.
Using TensorFlow backend.
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
2021-12-03 23:04:26,878 [INFO] iva.detectnet_v2.dataio.build_converter: Instantiating a kitti converter
2021-12-03 23:04:26,878 [INFO] root: Instantiating a kitti converter
2021-12-03 23:04:26,878 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Creating output directory /workspace/tao-experiments/data/val/tfrecords
2021-12-03 23:04:26,878 [INFO] root: Generating partitions
2021-12-03 23:04:26,911 [INFO] iva.detectnet_v2.dataio.kitti_converter_lib: Num images in
Train: 9377	Val: 1526
2021-12-03 23:04:26,911 [INFO] root: Num images in
Train: 9377	Val: 1526
2021-12-03 23:04:26,911 [INFO] iva.detectnet_v2.dataio.kitti_converter_lib: Validation data in partition 0. Hence, while choosing the validationset during training choose validation_fold 0.
2021-12-03 23:04:26,911 [INFO] root: Validation data in partition 0. Hence, while choosing the validationset during training choose validation_fold 0.
2021-12-03 23:04:26,920 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 0
2021-12-03 23:04:26,920 [INFO] root: Writing partition 0, shard 0
WARNING:tensorflow:From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataio/dataset_converter_lib.py:161: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

2021-12-03 23:04:26,920 [WARNING] tensorflow: From /root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/detectnet_v2/dataio/dataset_converter_lib.py:161: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

/usr/local/lib/python3.6/dist-packages/iva/detectnet_v2/dataio/kitti_converter_lib.py:297: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default.
2021-12-03 23:04:28,618 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 1
2021-12-03 23:04:28,618 [INFO] root: Writing partition 0, shard 1
2021-12-03 23:04:30,230 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 2
2021-12-03 23:04:30,230 [INFO] root: Writing partition 0, shard 2
2021-12-03 23:04:31,796 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 3
2021-12-03 23:04:31,796 [INFO] root: Writing partition 0, shard 3
2021-12-03 23:04:33,388 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 4
2021-12-03 23:04:33,388 [INFO] root: Writing partition 0, shard 4
2021-12-03 23:04:35,014 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 5
2021-12-03 23:04:35,014 [INFO] root: Writing partition 0, shard 5
2021-12-03 23:04:36,813 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 6
2021-12-03 23:04:36,813 [INFO] root: Writing partition 0, shard 6
2021-12-03 23:04:38,400 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 7
2021-12-03 23:04:38,400 [INFO] root: Writing partition 0, shard 7
2021-12-03 23:04:40,186 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 8
2021-12-03 23:04:40,187 [INFO] root: Writing partition 0, shard 8
2021-12-03 23:04:41,729 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 0, shard 9
2021-12-03 23:04:41,729 [INFO] root: Writing partition 0, shard 9
2021-12-03 23:04:43,265 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: 
Wrote the following numbers of objects:
b'car': 10689
b'trafficcone': 488
b'pickuptruck': 612
b'workvan': 236
b'pedestrian': 886
b'bicycle': 194
b'semitruck': 45
b'bus': 93
b'industrialvehicle': 59
b'carwithtrailers': 14
b'motorcycle': 16
b'schoolbus': 8
b'wheelchair': 3
b'escooter': 4
b'policecar': 3
b'ambulance': 1

2021-12-03 23:04:43,265 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 0
2021-12-03 23:04:43,265 [INFO] root: Writing partition 1, shard 0
2021-12-03 23:04:50,371 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 1
2021-12-03 23:04:50,372 [INFO] root: Writing partition 1, shard 1
2021-12-03 23:04:59,289 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 2
2021-12-03 23:04:59,289 [INFO] root: Writing partition 1, shard 2
2021-12-03 23:05:05,520 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 3
2021-12-03 23:05:05,520 [INFO] root: Writing partition 1, shard 3
2021-12-03 23:05:11,935 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 4
2021-12-03 23:05:11,935 [INFO] root: Writing partition 1, shard 4
2021-12-03 23:05:18,288 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 5
2021-12-03 23:05:18,288 [INFO] root: Writing partition 1, shard 5
2021-12-03 23:05:24,682 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 6
2021-12-03 23:05:24,682 [INFO] root: Writing partition 1, shard 6
2021-12-03 23:05:33,082 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 7
2021-12-03 23:05:33,083 [INFO] root: Writing partition 1, shard 7
2021-12-03 23:05:39,848 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 8
2021-12-03 23:05:39,848 [INFO] root: Writing partition 1, shard 8
2021-12-03 23:05:46,333 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Writing partition 1, shard 9
2021-12-03 23:05:46,333 [INFO] root: Writing partition 1, shard 9
2021-12-03 23:05:52,937 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: 
Wrote the following numbers of objects:
b'car': 68141
b'semitruck': 334
b'workvan': 1394
b'pickuptruck': 4067
b'pedestrian': 5628
b'trafficcone': 3551
b'bus': 657
b'industrialvehicle': 346
b'bicycle': 1265
b'fireengine': 9
b'schoolbus': 73
b'motorcycle': 157
b'carwithtrailers': 112
b'policecar': 20
b'ambulance': 9
b'escooter': 12
b'wheelchair': 7

2021-12-03 23:05:52,937 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Cumulative object statistics
2021-12-03 23:05:52,937 [INFO] root: Cumulative object statistics
2021-12-03 23:05:52,938 [INFO] root: {
    "car": 78830,
    "trafficcone": 4039,
    "pickuptruck": 4679,
    "workvan": 1630,
    "pedestrian": 6514,
    "bicycle": 1459,
    "semitruck": 379,
    "bus": 750,
    "industrialvehicle": 405,
    "carwithtrailers": 126,
    "motorcycle": 173,
    "schoolbus": 81,
    "wheelchair": 10,
    "escooter": 16,
    "policecar": 23,
    "ambulance": 10,
    "fireengine": 9
}
2021-12-03 23:05:52,938 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: 
Wrote the following numbers of objects:
b'car': 78830
b'trafficcone': 4039
b'pickuptruck': 4679
b'workvan': 1630
b'pedestrian': 6514
b'bicycle': 1459
b'semitruck': 379
b'bus': 750
b'industrialvehicle': 405
b'carwithtrailers': 126
b'motorcycle': 173
b'schoolbus': 81
b'wheelchair': 10
b'escooter': 16
b'policecar': 23
b'ambulance': 10
b'fireengine': 9

2021-12-03 23:05:52,938 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Class map. 
Label in GT: Label in tfrecords file 
b'Car': b'car'
b'TrafficCone': b'trafficcone'
b'PickupTruck': b'pickuptruck'
b'WorkVan': b'workvan'
b'Pedestrian': b'pedestrian'
b'Bicycle': b'bicycle'
b'SemiTruck': b'semitruck'
b'Bus': b'bus'
b'IndustrialVehicle': b'industrialvehicle'
b'CarWithTrailers': b'carwithtrailers'
b'Motorcycle': b'motorcycle'
b'SchoolBus': b'schoolbus'
b'WheelChair': b'wheelchair'
b'eScooter': b'escooter'
b'PoliceCar': b'policecar'
b'Ambulance': b'ambulance'
b'FireEngine': b'fireengine'
2021-12-03 23:05:52,938 [INFO] root: Class map. 
Label in GT: Label in tfrecords file 
b'Car': b'car'
b'TrafficCone': b'trafficcone'
b'PickupTruck': b'pickuptruck'
b'WorkVan': b'workvan'
b'Pedestrian': b'pedestrian'
b'Bicycle': b'bicycle'
b'SemiTruck': b'semitruck'
b'Bus': b'bus'
b'IndustrialVehicle': b'industrialvehicle'
b'CarWithTrailers': b'carwithtrailers'
b'Motorcycle': b'motorcycle'
b'SchoolBus': b'schoolbus'
b'WheelChair': b'wheelchair'
b'eScooter': b'escooter'
b'PoliceCar': b'policecar'
b'Ambulance': b'ambulance'
b'FireEngine': b'fireengine'
For the dataset_config in the experiment_spec, please use labels in the tfrecords file, while writing the classmap.

2021-12-03 23:05:52,938 [INFO] root: For the dataset_config in the experiment_spec, please use labels in the tfrecords file, while writing the classmap.

2021-12-03 23:05:52,938 [INFO] iva.detectnet_v2.dataio.dataset_converter_lib: Tfrecords generation complete.
2021-12-03 23:05:52,938 [INFO] root: TFRecords generation complete.
2021-12-03 15:05:53,937 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
13

I think I will retrain the model with the updated class name in config. I can see the loss starting to decaying from a higher value again.

14

Yes, please train and check whether there is AP value instead of 0.

15

Thanks! I double-checked the result with corrected label map. APs are no longer zeros.

17

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.