Yolov4 for different input size, pretrained model weights for the different sizes

Hi Morganh,

The training started well and anchor points were generated. However, the only problem we faced is that it is not accepting JPEG images. At the step where it generates validation dataset out of training dataset, it kept telling that “Total 0 samples in kitti training dataset” despite having 800 images there. We tried everything, set JPEG in every spec file. Still the same issue. We converted our images to PNG and reuploaded and it worked. Now, the only problem is that the PNG converted from JPEG is increasing file sizes to 9-fold. Have you come across this issue before? any suggestions?

Could you share the training spec file which will run into error?
What it the extension of your training images?

Hi Morganh,

Here are the settings for all three files.

config file:

random_seed: 42
yolov4_config {
  big_anchor_shape: "[(114.94, 60.67), (159.06, 114.59), (297.59, 176.38)]"
  mid_anchor_shape: "[(42.99, 31.91), (79.57, 31.75), (56.80, 56.93)]"
  small_anchor_shape: "[(15.60, 13.88), (30.25, 20.25), (20.67, 49.63)]"
  box_matching_iou: 0.25
  matching_neutral_box_iou: 0.5
  arch: "resnet"
  nlayers: 18
  arch_conv_blocks: 2
  loss_loc_weight: 1
  loss_neg_obj_weights: 1
  loss_class_weights: 1
  label_smoothing: 0.0
  big_grid_xy_extend: 0.05
  mid_grid_xy_extend: 0.1
  small_grid_xy_extend: 0.2
  freeze_bn: false
  #freeze_blocks: 0
  force_relu: false
}
training_config {
  batch_size_per_gpu: 8
  num_epochs: 80
  enable_qat: false
  checkpoint_interval: 10
  learning_rate {
    soft_start_cosine_annealing_schedule {
      min_learning_rate: 1e-7
      max_learning_rate: 1e-4
      soft_start: 0.3
    }
  }
  regularizer {
    type: L1
    weight: 3e-5
  }
  optimizer {
    adam {
      epsilon: 1e-7
      beta1: 0.9
      beta2: 0.999
      amsgrad: false
    }
  }
  pretrain_model_path: "EXPERIMENT_DIR/pretrained_resnet18/pretrained_object_detection_vresnet18/resnet_18.hdf5"
}
eval_config {
  average_precision_mode: SAMPLE
  batch_size: 8
  matching_iou_threshold: 0.5
}
nms_config {
  confidence_threshold: 0.001
  clustering_iou_threshold: 0.5
  force_on_cpu: true
  top_k: 200
}
augmentation_config {
  hue: 0.1
  saturation: 1.5
  exposure:1.5
  vertical_flip:0
  horizontal_flip: 0.5
  jitter: 0.3
  output_width: 1248
  output_height: 384
  output_channel: 3
  randomize_input_shape_period: 0
  mosaic_prob: 0
  mosaic_min_ratio:0.2
}
dataset_config {
  data_sources: {
      tfrecords_path: "/workspace/tao-experiments/data/training/tfrecords/train*"
      image_directory_path: "/workspace/tao-experiments/data/training"
  }
  include_difficult_in_training: true
  image_extension: "jpeg"
  target_class_mapping {
      key: "cow"
      value: "cow"
  }
  validation_data_sources: {
      tfrecords_path: "/workspace/tao-experiments/data/val/tfrecords/val*"
      image_directory_path: "/workspace/tao-experiments/data/val"
  }
}

kitti_val:

kitti_config {
  root_directory_path: "/workspace/tao-experiments/data/val"
  image_dir_name: "image"
  label_dir_name: "label"
  image_extension: ".jpeg"
  partition_mode: "random"
  num_partitions: 2
  val_split: 14
  num_shards: 10
}
image_directory_path: "/workspace/tao-experiments/data/val"

kitti_train:

kitti_config {
  root_directory_path: "/workspace/tao-experiments/data/training"
  image_dir_name: "image_2"
  label_dir_name: "label_2"
  image_extension: ".jpeg"
  partition_mode: "random"
  num_partitions: 2
  val_split: 14
  num_shards: 10
}
image_directory_path: "/workspace/tao-experiments/data/training"

The tfrecord generator can support .jpeg files. See DetectNet_v2 — TAO Toolkit 3.22.05 documentation
And also YOLO_v4 can support training on jpeg files. See https://docs.nvidia.com/tao/tao-toolkit/text/object_detection/yolo_v4.html#input-requirement
Suggest you to double check. Or to narrow down, please use sequence format dataset directly instead of tfrecords files.

Hi Morganh, gave it a million tries, somehow it just won’t accept any other format.

Any way to use coco dataset instead of kitti to train the yolo? if yes, how can i go about it? is there any example jupyter notebook available for that?

Could you share the latest error log?

  • tfrecord generation log
  • training error log

Hi Morganh, its same… at the time of tfrecords creation it says there are 0 samples in kitti training datasets.

I was wondering if its ok to somehow bring tfrecords converted outside. Any suggestions?

Please share the tfrecord spec file and also the full command and full log.

Hi Morganh,

I could not solve my png/jpg issue but got the results we wanted to by being able to convert to a png file without increasing the file size. Now I was able to export the mode with 93% mAP which is a great start (I believe) with a small subset of my data. When I export the model, I am getting everything I need except for prototxt. It is probably because prototxt is only for caffemodel. My questions is, will I be able to use following example to run inference on Jetson Xavier? deepstream_python_apps/apps/deepstream-test2 at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub (it asks for protoxtx)

I found some github examples that are for Tao Toolkit + deepstream, however, the code is in c++ and i am looking for something that is: Python and also has an example with upto 3 sgie models.

Any suggestions?

My other question is, the Deepstream examples I mentioned above include only caffe models. Does this mean that natively Deepstream supports Caffee models only?

Also, if anyone else here looking to convert Yolo dataset to Kitti dataset, please use Voxel51. It works like a charm. I used another solution on github and wasted 3 precious days just to find out that the labels were converted wrong.

Could you create a new topic? The original question is already fixed.

sure will do thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.