Model Training Accuracy Issue

[CostGazelle]

Hi,
We are trying to detect foreign objects by using custom model. The model has 5 classes and for each class we have taken 48 images. We have trained our ssd model by using jetson-train github repo for around 1200 epoch. However at 1200 epoch our loss was around 1.64. My questions are:

1. Would it have decreased if we had trained the model more?
2. How many images should be in the dataset for each class to get a well-trained model?
3. How much epoch do you suggest us to train our model? I mean should we train our model up until the loss decreases around 0.1 or 0.01 assuming that it will reach that threshold at some point?
4. We have tested our model by showing the objects that it is trained with to the camera and confidence was around %90-100. However, when we show another object that the model didn’t see before the confidence was decreased around %65. Is there a possibility of overfitting to be occured in our model? If so how can we detect it and how can we prevent our model to be overfitted?

Thanks.

[dusty_nv]

Hi @CostGazelle, yes for a robust model, you need a lot more images for each object (it’s hard to say how many for sure - I would start with collecting a few hundred per object, then re-train your model and try again, and collect more if needed).

Training your model for that many epochs will likely result in overfitting - I don’t typically run train_ssd.py for more than 100 epochs (but then again my models/datasets are typically smaller examples).

Recently I added the --validation-mean-ap flag to train_ssd.py which will compute the per-class accuracies at the end of each training epoch, which will tell you if the model has balanced performance across your object classes (and may tell you which classes you need to collect more data for)

[CostGazelle]

Hi, @dusty_nv Thank you for the informations.