Custom Training TrafficCamNet


I am trying to train DetectNet_v2 - ResNet18 on custom data using TrafficCamNet as pretrained model.

I download the weight file using below command:

!ngc registry model download-version nvidia/tlt_trafficcamnet:unpruned_v1.0 --dest .

Then I used the downloaded file resnet18_trafficcamnet.tlt
API_KEY: tlt_encode

The problem is, tlt_train command keeps on writing bunch of stuffs on the console but did not started the training.
I am using DGX1 with all 8 GPUs, and waiting from around 5 Hours but still no clue.

Here is my spec_file:
train.txt (10.7 KB)

Here is tlt_train console output (7MB):

Please help what went wrong.

Could you set smaller epoch and try again to check if it can run?
For you case, set epochs 4000 to 10.

Sorry for the delayed update, I left the command running over night and finally it started training.
Though I’m not sure why it took so long.

Thanks for the info.