DSSD resume error

[Morganh]

Please use a new result folder and retry.

[Morganh]

More , could you change pred_num_channels as below?
pred_num_channels: 0

[glscsvg]

I set pred_num_channels: 0,and create new folder (folder1)

[Morganh]

Could you share your latest training spec file?

[glscsvg]

latest training file

dssd_train_resnet18_kitti.txt (1.6 KB)

[Morganh]

Could you share the full command and full log?

[glscsvg]

output.txt (535.4 KB)

[Morganh]

Hi,
Could you try to train with SSD network to check if there is the same error?
Please refer to the SSD training spec file inside jupyter notebook.

[glscsvg]

I tried with kitti dataset and my own dataset but got error

custom dataset (original image size 500x500):
ssd (copy).ipynb (90.1 KB)

with kiiti dataset:

ssd.ipynb (191.2 KB)

[Morganh]

For ssd or dssd, please

• use the smallest input-size to avoid gpu memory issue as much as possible

• also change tfrecord format to sequence format.
  data_sources: {
  tfrecords_path: “/workspace/tao-experiments/Data/kitti_data/tfrecords/kitti_train*”
  }

  to

   data_sources: {
      label_directory_path: "/workspace/tao-experiments/Data/kitti_data/training/label"
      image_directory_path: "/workspace/tao-experiments/Data/kitti_data/training/image"
   }
  

[glscsvg]

yes, this way the training continued from where it left off

[Morganh]

So, issue is gone, right?

[glscsvg]

Yes, no error occurred