How to include new parameters in ROS Deep Learning App

AI & Data Science Deep Learning (Training & Inference) Frameworks
,
1

Hi @dusty_nv
I have flashed Jetson Xavier NX with latest sd-cad image and performed :

  1. Jetson-inference test along with ROS Deep Learning Application tests from :
    GitHub - dusty-nv/jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
    GitHub - dusty-nv/ros_deep_learning: Deep learning inference nodes for ROS / ROS2 with support for NVIDIA Jetson and TensorRT

  2. I have UART based TI mmWave RADAR AWR 1442 supported, but the ROS-Deep Learning application supports Overlay Flags : Box, Label, Confidence.
    To include the Radar Range and velocity values Iā€™ve faced error such as:
    roswork/src/ros_deep_learning/src/node_detectnet.cpp:120:155: error: ā€˜struct detectNet::Detectionā€™ has no member named ā€˜Rangeā€™
    ROS_INFO(ā€œobject %i class #%u (%s) confidence=%f range=%imtr velocity=%fm/sā€, n, det->ClassID, net->GetClassDesc(det->ClassID), det->Confidence, det->Range, det->Velocity);

Clearly, the structure needs support for new variables and ways to fill data for the same.
Can you provide any insights of how this can be done ?
Any references to fuse radar values with Detection data would be really helpful.

Thanks
Yash Jain

2

Hi @dusty_nv, @nluehr

I had faced the issue of Detection not significant under some conditions:

  1. Iā€™m using the IMX219 Camera with practically tested Field of Vision = 100 degrees.In some cases when the object to detetct is darker, the camera failed to detect person, can we improve the performance by setting it to 1080p ?

  2. I created a data buffer to store array[num_detetctions], to process the values but when array size went cross 50 the ROS application gave error :

[detectnet-2] process has died [pid 10440, exit code -11, cmd /home/mistral/workspace/roswork/devel/lib/ros_deep_learning/detectnet /detectnet/image_in:=/video_source/raw __name:=detectnet __log:=/home/mistral/.ros/log/38145380-174a-11eb-937d-48b02d077326/detectnet-2.log].
log file: /home/mistral/.ros/log/38145380-174a-11eb-937d-48b02d077326/detectnet-2*.log
^C[static_tf_map_to_base_radar_link-6] killing on exit

  1. What would be the best way to modify labels.txt in Networks to reduce the labels to detect in Mobilenet V2 model ?

Thanks for any help on thisā€¦

3

Hi Yesh, you may want to create your own new structure which contains your range/velocity values, and a detectNet.Detection result. When you get the detectNet.Detection results back, loop through them and fill out your new structure with the radar data included.

The camera image gets downsampled to 300x300 before being input to the SSD-Mobilenet model, so I donā€™t think higher camera resolution would make a huge difference. However perhaps you an increase the exposure somehow so that the image is not so dark? Or perhaps a low-light camera is needed. Or the model needs to be trained on dark objects.

For classes that you want to ignore, if you change the class name in labels.txt to void, it will be ignored.

1 Like