Failing custom object detection

[conradj]

Hello!

I trained a custom object detection model, following Hello AI World. When I use the trained model for object detection on images of my training set, I get hardly any detections (~99% false negatives). Could you point me into the direction about which things to check first for better performance?

Thank you!

Training data: 1’000 images, 3 object classes (overfitting is currently no issue in the application)

Trained for 30 epochs, batch size 4 (default parameters).

[ankithsingh]

hi @conradj

Can you please give more information like what steps you have followed and whether your custom dataset have images which are clear enough so that the models can process it with some ease

Thank you

[conradj]

Hi @ankithsingh,

I had an existing, labelled dataset that i exported in Pascal VOC format. For training file generation, i followed this instruction: training ssd-mobilenet from custom dataset · Issue #789 · dusty-nv/jetson-inference · GitHub

For the training and onnx conversion, i followed the Hello AI World instructions: jetson-inference/pytorch-collect-detection.md at master · dusty-nv/jetson-inference · GitHub

The pictures should be clear enough, as they were already used to train a Mobilenet SSD network (I have to retrain because it is formatted to be used with other hardware and i cannot convert it).

Thank you!

[dusty_nv]

Hi @conradj, can you try deleting the .engine file from your model’s folder? Then TensorRT will re-generate the engine, in case you had re-exported the ONNX and it was using old version of model.

Also can you check if your PyTorch .pth model checkpoint is able to detect objects ok? You can use run_ssd_example.py to try the model on the PyTorch side.

[conradj]

Hello @dusty_nv,

thank you very much for the input. The conversion from .pth to .onnx seems to be the problem, as it strongly affects the model performance. Is this expected or can i debug this somehow?

[dusty_nv]

If you have exported the ONNX multiple times during the various phases of training, have you tried deleting the *.engine file from your model’s folder? This will have TensorRT regenerate it from your latest ONNX model the next time you run detectnet program.