Training error on resnet18

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

I am trying to convert KITTI dataset to TFrecords, converter throwing out an error, any pointers?

Total number images 640818

root@24dfc5c16956:/workspace/tlt_model# tlt-dataset-convert -d kitti_car -o hello/test
Using TensorFlow backend.
2019-10-20 10:10:08,770 - iva.detectnet_v2.dataio.build_converter - INFO - Instantiating a kitti converter
2019-10-20 10:10:09,905 - iva.detectnet_v2.dataio.kitti_converter_lib - INFO - Num images in
Train: 90548	Val: 22637
2019-10-20 10:10:09,906 - iva.detectnet_v2.dataio.kitti_converter_lib - INFO - Validation data in partition 0. Hence, while choosing the validationset during training choose validation_fold 0.
2019-10-20 10:10:09,956 - iva.detectnet_v2.dataio.dataset_converter_lib - INFO - Writing partition 0, shard 0
Traceback (most recent call last):
  File "/usr/local/bin/tlt-dataset-convert", line 10, in <module>
    sys.exit(main())
  File "./detectnet_v2/scripts/dataset_convert.py", line 64, in main
  File "./detectnet_v2/dataio/dataset_converter_lib.py", line 74, in convert
  File "./detectnet_v2/dataio/dataset_converter_lib.py", line 108, in _write_partitions
  File "./detectnet_v2/dataio/dataset_converter_lib.py", line 149, in _write_shard
  File "./detectnet_v2/dataio/kitti_converter_lib.py", line 169, in _create_example_proto
  File "./detectnet_v2/dataio/kitti_converter_lib.py", line 272, in _add_targets
TypeError: object of type 'int' has no len()

spec file

root@24dfc5c16956:/workspace/tlt_model# cat kitti_car 
kitti_config {
  root_directory_path:"/workspace/car"
  image_dir_name:"/workspace/uploaded_images"
  label_dir_name:"/workspace/car/bbox"
  image_extension:".png"
  partition_mode:"random"
  num_partitions:2
  val_split:20
  num_shards:10
}
image_directory_path:"/workspace/car"

Data format 

cat CAR123_roi_2.txt
1 176 1 242 77
1 282 0 355 63

I will try this format and let you know
https://docs.nvidia.com/metropolis/TLT/tlt-getting-started-guide/index.html#label_file

2

Hi RaviKiranK,
The object detection apps in TLT expect data in KITTI file format. I saw you have fixed the problem and closed this topic, right?

3

Hi Morganh,

I have both PNG and JPG files, can I use both formats? 

root@24dfc5c16956:/workspace/tlt_model# cat kitti_tuktuk 
kitti_config {
  root_directory_path:"/workspace/tuktuk"
  image_dir_name:"/workspace/uploaded_images"
  label_dir_name:"/workspace/tlt_model/NV_TLT/tuktuk_dataset/labels"
  image_extension:".png"
  image_extension:".jpg"
  partition_mode:"random"
  num_partitions:2
  val_split:20
  num_shards:10
}
image_directory_path:"/workspace/tuktuk"
4

Hi Morganh,

Can you confirm it?
if it accepts both formats

5

Hi RaviKiranK,
Firstly, from the tlt doc, for the SSD and DetectNet_v2 apps, as mentioned in Data input for object detection, require KITTI format data to be converted to TFRecords.

The sum of the total number of elements per object is 15.The kitti format looks like this. Here is a sample text file:
car 0.00 0 -1.58 587.01 173.33 614.12 200.12 1.65 1.67 3.64 -0.65 1.71 46.70 -1.59

But yours is as below.
cat CAR123_roi_2.txt
1 176 1 242 77

So, could you please change to kitti format? Thanks.

6

Hi Morgah,

we changed the data format we created TFrecords. I just want to make sure I created records for .PNG and JPG format

7

Glad to hear that you fix the original issue.
For your last question,please refer to https://devtalk.nvidia.com/default/topic/1065238/transfer-learning-toolkit/tensor-reshape-error-when-evaluating-a-detectnet_v2-model/post/5394680/#5394680 to check if both png and jpg are converted into tfrecord.
I have never tried two kinds of format yet.

8

Hi Morganh,

Can you add support for adding both png /jpg/jpeg ? Our dataset is 3.4 million images, we will loose 1/3rd

9

Hi RaviKiranK,
Sorry,the “tlt-dataset-convert” does not support mixed extension format. You can transfer one kind of extension to another in order to unify the image format.