Train custom object detectio model

Hello
I’m using Jetson Nano.
I need to train custom object detectors for a real time webcam application.
What API/Technology… is best to use in order to run the training on my Jetson Nano?
Please suggest me something.
Thanks in advance!
Ryan

Hi,

Please noticed that Nano is an edge device so we don’t recommend to apply training on it.
You will get a much better training performance on a desktop GPU.

A possible case for Nano is transfer learning.
Please check this document for information:

Thanks.

Thanks for the reply.
I did this tutorial, which is for Image Classification. The tutorial is great and the Inference library does a great job.
I didn’t find a tutorial on object detection though. Do you know where I can find one?
Thanks again

Hi @rsamvelyan, see this post: DIGITS or somthing else

For PyTorch training of object detection, I’ve been preparing the tutorial. You can find the code here: https://github.com/dusty-nv/pytorch-ssd

thanks a lot! I’m going to do this tutorial

And is there an example on how to load the custom trained model into detectnet-camera.py which is in the examplea folder of Jetson-Inference?

I tried to load different pre-trained models, by using --model and and specifying the labels file but it always gives me an error.
Do I need to specify something under --network too?

Dusty, first of all thanks a lot for all your contribution!

The tutorial for PyTorch you built requires Caffe. I tried compiling it from source on my Jetson Nano, it troughs c++ errors… total nightmare

Can I install DIGITS on a computer without an NVIDIA GPU? Or can I install it on my Jetson Nano?

What are my options for transfer learning? I’m totally lost.

Can I use the train.py in the jetson-inference/python/training/detection for transfer learning on my custom data-set?
Any specific and complete tutorial on that one, just like the cat_dog example?

At this point, all I can get to work is inference with detectnet on the pre-trained models that came with the installation… :(

Thanks again!!

Hi,

1.
You can find the instruction to load custom model below:
Although this is for imagenet with C++, the procedure for detectnet-camera.py should be similar.

2.
DIGITs cannot install on the environment without desktop GPU.
Since it need NCCL support, which requires a desktop GPU.

To install pyTorch on Jetson, please follow this topic:

To install Caffe on Jetson, please follow this tutorial:
https://jkjung-avt.github.io/caffe-on-tx2/

Thanks.

Hello AastaLLL
Thanks for the reply
I have my custom ONNX model, which is a resnet-18 model, fine tuned with Pytorch with the sample python script provided with Jetson Inference. Then I converted it to ONNX
First of all, I just made a copy for myself, and renamed it with my-detectnet-camera.py
This is what I use to call the detectnet-camera.py:

python3 my-detectnet-camera.py --model=resnet18_smarties_224x224.onnx --class_labels=labels.txt --input_blob=data

The GPU engine file is successfully created: resnet18_smarties_224x224.onnx.1.1.GPU.FP16.engine

Gstreamer starts and my webcam begins to work. I’m showing around 80FPS, but absolutely no detection. Not one single object gets detected, no bounding boxes.

This is what I get in the terminal while the webcam is up and running:
[TRT] detectNet::Detect() – ONNX – coord (nan, nan) (nan, nan) image 640x480
[cuda] cudaDetectionOverlay((float4*)input, (float4*)output, width, height, detections, numDetections, (float4*)mClassColors[1])
[cuda] invalid configuration argument (error 9) (hex 0x09)
[cuda] /home/ryan/jetson-inference/c/detectNet.cpp:945
[TRT] detectNet::Detect() – failed to render overlay
detected 1 objects in image
jetson.inference – PyDetection_Dealloc()
<detectNet.Detection object>
– ClassID: 0
– Confidence: 1
– Left: nan
– Top: nan
– Right: nan
– Bottom: nan
– Width: nan
– Height: nan
– Area: nan
– Center: (nan, nan)

These are all the parameters I can specify:
optional arguments:
-h, --help show this help message and exit
–network NETWORK pre-trained model to load (see below for options)
–overlay OVERLAY detection overlay flags (e.g. --overlay=box,labels,conf)
valid combinations are: ‘box’, ‘labels’, ‘conf’, ‘none’
–threshold THRESHOLD
minimum detection threshold to use
–camera CAMERA index of the MIPI CSI camera to use (e.g. CSI camera 0)
or for VL42 cameras, the /dev/video device to use.
by default, MIPI CSI camera 0 will be used.
–width WIDTH desired width of camera stream (default is 1280 pixels)
–height HEIGHT desired height of camera stream (default is 720 pixels)

detectNet arguments:
–network NETWORK pre-trained model to load, one of the following:
* ssd-mobilenet-v1
* ssd-mobilenet-v2 (default)
* ssd-inception-v2
* pednet
* multiped
* facenet
* coco-airplane
* coco-bottle
* coco-chair
* coco-dog
–model MODEL path to custom model to load (caffemodel, uff, or onnx)
–prototxt PROTOTXT path to custom prototxt to load (for .caffemodel only)
–class_labels LABELS path to text file containing the labels for each class
–threshold THRESHOLD minimum threshold for detection (default is 0.5)
–input_blob INPUT name of the input layer (default is ‘data’)
–output_cvg COVERAGE name of the coverge output layer (default is ‘coverage’)
–output_bbox BOXES name of the bounding output layer (default is ‘bboxes’)
–mean_pixel PIXEL mean pixel value to subtract from input (default is 0.0)
–batch_size BATCH maximum batch size (default is 1)
–alpha ALPHA overlay alpha blending value, range 0-255 (default: 120)
–profile enable layer profiling in TensorRT

usage: my-detectnet-camera.py [-h] [–network NETWORK] [–overlay OVERLAY]
[–threshold THRESHOLD] [–camera CAMERA]
[–width WIDTH] [–height HEIGHT]

Locate objects in a live camera stream using an object detection DNN.

optional arguments:
-h, --help show this help message and exit
–network NETWORK pre-trained model to load (see below for options)
–overlay OVERLAY detection overlay flags (e.g. --overlay=box,labels,conf)
valid combinations are: ‘box’, ‘labels’, ‘conf’, ‘none’
–threshold THRESHOLD
minimum detection threshold to use
–camera CAMERA index of the MIPI CSI camera to use (e.g. CSI camera 0)
or for VL42 cameras, the /dev/video device to use.
by default, MIPI CSI camera 0 will be used.
–width WIDTH desired width of camera stream (default is 1280 pixels)
–height HEIGHT desired height of camera stream (default is 720 pixels)

detectNet arguments:
–network NETWORK pre-trained model to load, one of the following:
* ssd-mobilenet-v1
* ssd-mobilenet-v2 (default)
* ssd-inception-v2
* pednet
* multiped
* facenet
* coco-airplane
* coco-bottle
* coco-chair
* coco-dog
–model MODEL path to custom model to load (caffemodel, uff, or onnx)
–prototxt PROTOTXT path to custom prototxt to load (for .caffemodel only)
–class_labels LABELS path to text file containing the labels for each class
–threshold THRESHOLD minimum threshold for detection (default is 0.5)
–input_blob INPUT name of the input layer (default is ‘data’)
–output_cvg COVERAGE name of the coverge output layer (default is ‘coverage’)
–output_bbox BOXES name of the bounding output layer (default is ‘bboxes’)
–mean_pixel PIXEL mean pixel value to subtract from input (default is 0.0)
–batch_size BATCH maximum batch size (default is 1)
–alpha ALPHA overlay alpha blending value, range 0-255 (default: 120)
–profile enable layer profiling in TensorRT

I’ve even tried with caffe models taken from the network folder provided with Jetson Inference ans still no luck!

I can’t figure out what exactly is that I am doing wrong here?

Thanks!

Hi @rsamvelyan, resnet-18 is an image classification model, not object detection - so it wouldn’t work with detectNet. You should instead load resnet-18 with the imageNet programs.

If you want to train your own object detection model, I have been working on new part of Hello AI World tutorial for that. You can see the PyTorch training code awhile here: https://github.com/dusty-nv/pytorch-ssd

You’re roght Dusty. Yesterday I realized that I used the wron model.
Nevertheless, for some reason, I haven’t beem able to properly load no other model. Even when loading the same mobilenetv2 I’m getting messages such as wrong class id or something like that.
I was wonderif there is a specific example on how to properly load an ONNX object detection model into detectnet-camera.py?

Hi,

You can find an imageNet with ONNX model sample here:

The steps for detection model should be similar.
Thanks.