Other Models for object detection

[MaxMarth]

Hello everyone!
i do object detection by collecting my own datasets.
There are 5 traffic signs to be detected.
when i use the SSD-v2 model with epochs=10 and 60 pictures for each sign, the trained model can distinguish between left turn and right turn, but the confidence level is about 50%.
and then i use the SSD-v2 model with 120 pictures for each with epochs=1 and epochs=10, it seems that in both cases the trained models cannot distinguish between turn left and turn right.

i think maybe an overfitting just happens. But as you can see a blue bottle, which has the similar color to most of the turn left and turn right traffic signs, also be detected as turn left. (i think maybe the SSD-v2 model just take the color pixels as features for detection, i dont know if it’s ture.)
So my questions is that, should i take a trade off between number of dataset and epochs or should i try some new detection models?

[AastaLLL]

Hi,

May I know how you train your model?

Basically, it’s recommended to enlarge the database diversity.
So the detector can distinguish the sign with more clues other than colors.

Thansk.

[MaxMarth]

i follow this to train the model:

dusty-nv/jetson-inference/blob/master/docs/pytorch-collect-detection.md

use camera-capture and ssd.

[dusty_nv]

Hi @MaxMarth, in addition to increasing the number of images in your dataset like @AastaLLL suggested, I would also train your models for more epochs - I typically train these for at least 30 epochs.

[MaxMarth]

Thank you @dusty_nv!
We are trying to increase the number of images in our dataset.
Now we have 2 questions:
First is that the jetbot always crash with the reason “no more memory to allocate”, we use jetson nano 4GB.
And then we need to shut down the command window and restart. Is it possible to avoide it?
Second is that we want to use an open source dataset availabe online with 50,000 images to train, but the image form is PPM and for our jetbot is JPEG. So i want to ask if we need to transfer the images before we use it or this dataset cannot be used anyway.

[dusty_nv]

@MaxMarth

First is that the jetbot always crash with the reason “no more memory to allocate”, we use jetson nano 4GB.

Have you mounted additional swap, disabled ZRAM, and disabled the desktop GUI?

• https://github.com/dusty-nv/jetson-inference/blob/master/docs/pytorch-transfer-learning.md#mounting-swap
• https://github.com/dusty-nv/jetson-inference/blob/master/docs/pytorch-transfer-learning.md#disabling-the-desktop-gui

Also, have you tried running train_ssd.py with --batch-size=1 --workers=0 ?

@MaxMarth

Second is that we want to use an open source dataset availabe online with 50,000 images to train, but the image form is PPM and for our jetbot is JPEG

Here are the image extensions that the dataset loader supports: https://github.com/dusty-nv/pytorch-ssd/blob/3f9ba554e33260c8c493a927d7c4fdaa3f388e72/vision/datasets/voc_dataset.py#L163

So I would recommend either adding PPM to there, or converting them to JPG.

[MaxMarth]

Hi @dusty_nv ,
follow your recommendation i convert all .ppm images into .jpeg images.
If we want to use the dataset to train, the annotations are also important.
Our jetbot use .xml type of annotations, but in the GTSRB(https://benchmark.ini.rub.de/) the annotations are provided in CSV files like follows:

So my question is that: are there any possibility to transform .csv into .xml? Or if we can just use jpeg images and then manual mark them to get annotations and then train the model? For example camera-capture tool with image sequence.

[dusty_nv]

@MaxMarth

So my question is that: are there any possibility to transform .csv into .xml? Or if we can just use jpeg images and then manual mark them to get annotations and then train the model? For example camera-capture tool with image sequence.

Hi @MaxMarth, you can either find or create some script that will convert your CSV into a dataset in the Pascal VOC format, or use CVAT.org to re-label your images and export them as Pascal VOC.

[MaxMarth]

Hi @dusty_nv ,
i go to CVAT.org and label the turn left and the turn right images.
And i upload them to jetbot and want to use train_ssd.py, but there’s an error:

I dont know why.
And follows are the annotations we get from CVAT.org:

and from camera-capture on jetbot:

You can see there’s a difference. I want to know if this will be a problem.
Also i want to ask, is it possible that we use train_ssd.py on our own computers and get .path file, then we upload it to jetbot?

[dusty_nv]

@MaxMarth

And i upload them to jetbot and want to use train_ssd.py, but there’s an error:

The way that CVAT formats the XML files is fine for Pascal VOC and train_ssd.py, however it appears that some of the annotations in your dataset are invalid and unfortunately causes this exception. To figure out which XML file is causing it, try the following:

• uncomment this line of code https://github.com/dusty-nv/pytorch-ssd/blob/3f9ba554e33260c8c493a927d7c4fdaa3f388e72/vision/datasets/voc_dataset.py#L76
  (if you are running this in the container, modify this file inside the container using nano/vi text editor)
• run train_ssd.py with --batch-size=1 --debug-steps=1 --num-workers=1

It will then print out the file/image info while loading it, and the last one to get printed out before the exception is the one that causes the error. Look into that XML file for anything strange, or remove it from your ImageSets files so it’s no longer used.

If you have problems, you can upload your dataset to google drive or somewhere, and I can download it and take a look for you.

@MaxMarth

is it possible that we use train_ssd.py on our own computers and get .path file

Yea you can run train_ssd.py on a PC (ideally one with an NVIDIA GPU so it’s fast) as long as you have PyTorch installed and the other dependencies. What I do is run it on my Ubuntu laptop in the PyTorch NGC container. You can export the ONNX on your PC and copy that ONNX and labels.txt over to your Jetson and run it with jetson-inference.

[MaxMarth]

Hi @dusty_nv ,
Now i want to use other DNN models in order to do object detection. Then i can make a comparison. Are DNN-models from bellow:

all can be used on jetbot and create onnx file?

[dusty_nv]

Those ones in the Model Downloader are already integrated with jetson-inference and can be run using the command-line arguments from this table:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md#pre-trained-detection-models-available

They are some UFF, caffe, and ONNX models. But they are each already supported in jetson-inference.

[MaxMarth]

Thanks for your advice, @dusty_nv!
Since i download for example ssd-v1 model ssd-v2, and i have in ./ssd/vision/ssd mobilenetv1_ssd.py, mobilenet_v2_ssd_lite.py and so on. How can i make a change to use ssd-v2 model to train? Because i find that the .path file in ./ssd/models is mobilenet-v1-ssd-mp-0_675.path.
Should i write something when i use command: python3 train_ssd.py --dataset-type=voc --data=data/ --model-dir=models/ ?

[dusty_nv]

Hi @MaxMarth, you can refer to the readme from the original upstream repo about the different models: https://github.com/qfgaohao/pytorch-ssd

The ssd-mobilenet-v1 model is the only one that is tested/working with ONNX export, so that is the one that I use.

[MaxMarth]

Hi @dusty_nv , if i use MobileNetV2 SSD Lite and exchange [mobilenet-v1-ssd-mp-0_675.pth] into [mb2-ssd-lite-mp-0_686.pth], in this way there will be no onnx output as you said. Then how could i test on jetbot? Cause in video we use “detectnet --model=models/traffic/ssd-mobilenet.onnx” to test the trained model.

[dusty_nv]

@MaxMarth

if i use MobileNetV2 SSD Lite and exchange [mobilenet-v1-ssd-mp-0_675.pth] into [mb2-ssd-lite-mp-0_686.pth], in this way there will be no onnx output as you said. Then how could i test on jetbot? Cause in video we use “detectnet --model=models/traffic/ssd-mobilenet.onnx” to test the trained model.

Hi @MaxMarth, the upstream pytorch-ssd repo explains that the mobilenet-v2-ssd-lite model isn’t compatible with ONNX:

The above MobileNetV2 SSD-Lite model is not ONNX-Compatible, as it uses Relu6 which is not supported by ONNX.

Hence it won’t work with detectnet/detectnet.py, so please just use the SSD-Mobilenet-v1 instead (which is the version that is working with ONNX). It will have given similar performance to -v2 anyways.

[MaxMarth]

Hi, @dusty_nv
after training i get model.pth, which i wanna use in roadfollowing + object detection. but first i must define the model(e.g.
model = torchvision.models.resnet18(pretrained=False)
) to load the model.path. when i load the model.path:
model.load_state_dict(torch.load(‘model.pth’)),
it obviously doesnt match e.g. resnet18.
i wanna find the right neural network under package torchvision.models, but after checking the official site von package torchvision.models, there is no SSD-mobilenetV1. So how can i do to load the trained model here to be combined with RoadFollowing?

[dusty_nv]

I believe this is a similar discussion to what we are having in:

How to create our own ssd_mobilenet_v1_coco.engine file?

The jetson.inference.detectNet interface (aka detectnet.py) is the recommended way to use the ssd-mobilenet.onnx trained with train_ssd.py. Otherwise you may need to make code changes to JetBot/ect to support the pre/post-processing that model expects. To run trtexec, try this: $ cd /usr/src/tensorrt/bin $ ./trtexec