Jetson Nano - train_ssd.py example doesn't detect anything

Hey,
I’ve been searching around for quite a while now, but can’t find any solution. I’ve been working with a couple of demos with pre-trained networks and they are working fine for me.
The next logical step seemed to train my own objects and see if it would detect them.

I followed the tutorial on Jetson AI Fundamentals - S3E5 - Training Object Detection Models - YouTube and created close too 1000 training images.
First piece I noticed is that the files that the camera-capture tool creates an ImageSets/Main/train.txt whereas the train_ssd.py expects the file to be called default.txt. I could fix that by copying the file over to the right file name.

Then I let it train for about 30 epochs which took quite a while but eventually finished.

Lastly I exported the onnx and tried to run it with the detectnet command:

detectnet --model=models/mymodel/ssd-mobilenet.onnx --labels=models/mymodel/labels.txt \
          --input-blob=input_0 --output-cvg=scores --output-bbox=boxes /dev/video1

It would run it’s optimisations in the beginning and finally show the camera image, but didn’t detect my object at all. So how can I find out what went wrong during my training?

I’m sorry I’m very new to the topic, so any documentation on how I can go at debugging would be great.
Thank you very much for helping!

Hi @moritz3, if you run detectnet with --threshold=0.25 argument, do detected objects appear? The default is --threshold=0.5.

You could also try the run_ssd_example.py script, which runs the detection from the PyTorch side, to see if there is some difference in detection output.

How many different classes are in your dataset? Are the objects you are trying to detect very tiny, or is the subject matter otherwise challenging (i.e. varying backgrounds or camera angles / lighting conditions)? If so, you may want to let the model train for more epochs - I believe you can restart it with using the --pretrained argument to train_ssd.py

Did you get an error and it would only work when you copied the file to default.txt? It should work with both - I added an option that looked for default.txt because the CVAT annotation tool saves it as default.txt.

Hey, thank you for your answer!

The error I received was:

2021-02-15 17:22:37 - Stored labels into file models/camp/labels.txt.
2021-02-15 17:22:37 - Train dataset size: 985
2021-02-15 17:22:37 - Prepare Validation datasets.
Traceback (most recent call last):
  File "train_ssd.py", line 243, in <module>
    target_transform=target_transform, is_test=True)
  File "/home/moritz/git/jetson-inference/python/training/detection/ssd/vision/datasets/voc_dataset.py", line 33, in __init__
    raise IOError("missing ImageSet file {:s}".format(image_sets_file))
TypeError: unsupported format string passed to PosixPath.__format__

Also I just tried with a threshold of .3 and could indeed get results! I’ll try running with more epochs next!

Thanks, at least I’m on the right track now and likely need to improve my training data. It does have a lot of changing backgrounds and different camera exposure.

Cheers
Moritz

OK great, glad you got it working! I will look into that error you got.

I don’t know if I should open a new thread, but since I got the detectnet working with the example code, I wanted to switch over to utilizing cv2 for my image rendering.

I’m getting my camera picture via cv2.VideoCapture() and then convert the frame for the jetson inference functions:

img = cv2.cvtColor( frame, cv2.COLOR_BGR2RGB ).astype( np.uint8 )
Jframe = jetson.utils.cudaFromNumpy( img )

detections = net.Detect( Jframe )
print("detected {:d} objects in image".format(len(detections)))

When I output the new Jframe with jetson.utils.videoOutput() → Render() it looks like a proper video to me; I’ve also compared the type of image data and it shows as rgb8 in the example detectnet.py and my code.

Yet I don’t get any results where in the detectnet.py I do get results. For testing I’m pointing my webcam at the same picture on my monitor.
Is there anything I’m still missing?

Here’s a screenshot of the output directly from cv2 (bottom) and after converting it via cudaFromNumpy() (top):

Thanks for taking a look.

Hmm. In your customized application using cv2, are you still setting the threshold=0.3 when creating detectNet object?

https://github.com/dusty-nv/jetson-inference/blob/cf756333ab1876d90f107f54e7a3c5e58c3ddd5a/python/examples/detectnet.py#L51

I instanciate my network like this:

net = jetson.inference.detectNet( 'ssd-mobilenet-v1'
, [
    '--model=models/mymodel/ssd-mobilenet.onnx',
    '--input-blob=input_0',
    '--output-cvg=scores',
    '--output-bbox=boxes',
    '--labels=models/mymodel/labels.txt',
    '--threshold=0.25'
] )

Then I just checked the example code again and found that the threshold needs to be in the function argument:

net =  jetson.inference.detectNet( 'ssd-mobilnet-v1', [ '--model=...' ], threshold=0.25 )

Have it working now! Thank you again. Sometimes I really just see my mistakes when I post them to a forum :D

I have had the same problem after collecting data with the camera,and it only works after saving train.txt as default.txt