How to process and show a lidar point-cloud, alongside with rectangular boxes on the detected objects?

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.0
[V] DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version

Target Operating System
[V] Linux

Hardware Platform
[V] NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)

SDK Manager Version

Host Machine Version
[V] native Ubuntu 18.04

Hi. I wrote a custom lidar sensor plugin for app “sample_lidar_replay” in the NVIDIA driveworks SDK. It works. Looks something like that:

Now I would like to show some square boxes on the cars and objects, so the view will look like that:

I watched the webinar at:

but didn’t see there how to process and draw the rectangular boxes on the detected objects of a lidar point cloud.

Is there any sample in driveworks sdk that can use a custom SAL lidar plugin, and show the pointcloud with rectangular boxes on the detected objects?

Dear @Sunny127,
We don’t have DW sample demonstrating the asked use case.

Can you maybe provide some help or links regarding how can we do it ourselves?

Dear @Sunny127,
You need to run a object detection DNN to get bounding box info and draw over point cloud visualization.

Can you provide some links to nvidia drive AGX API functions / nvidia samples / nvidia webinars, on how to do these things?

Dear @Sunny127,
It should be a fusion of outputs from two sensors like Lidar and Camera. Please see DNN object detection/tracker sample to know APIs used to integrate a DNN and perform object detection using cameras. Once you get bounding boxes using your DNN, You can overlap the bounding box data as rectangle using dwRenderEngine_render API. Also, please check dwRender module to know list of APIs used for various purpose and grep for usage in DW samples.

1 Like