How to process and show a lidar point-cloud, alongside with rectangular boxes on the detected objects?

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.0
[V] DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
[V] Linux
QNX
other

Hardware Platform
[V] NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
[V] 1.6.1.8175
1.6.0.8170
other

Host Machine Version
[V] native Ubuntu 18.04
other

Hi. I wrote a custom lidar sensor plugin for app “sample_lidar_replay” in the NVIDIA driveworks SDK. It works. Looks something like that:

sample_lidar_replay
sample_lidar_replay1250×976 258 KB

Now I would like to show some square boxes on the cars and objects, so the view will look like that:

rectangles_on_pcl
rectangles_on_pcl836×863 64.8 KB

I watched the webinar at:
https://developer.nvidia.com/video/point-cloud-processing-driveworks

but didn’t see there how to process and draw the rectangular boxes on the detected objects of a lidar point cloud.

Is there any sample in driveworks sdk that can use a custom SAL lidar plugin, and show the pointcloud with rectangular boxes on the detected objects?

3

Dear @Sunny127,
We don’t have DW sample demonstrating the asked use case.

4

Can you maybe provide some help or links regarding how can we do it ourselves?

5

Dear @Sunny127,
You need to run a object detection DNN to get bounding box info and draw over point cloud visualization.

6

Can you provide some links to nvidia drive AGX API functions / nvidia samples / nvidia webinars, on how to do these things?

7

Dear @Sunny127,
It should be a fusion of outputs from two sensors like Lidar and Camera. Please see DNN object detection/tracker sample to know APIs used to integrate a DNN and perform object detection using cameras. Once you get bounding boxes using your DNN, You can overlap the bounding box data as rectangle using dwRenderEngine_render API. Also, please check dwRender module to know list of APIs used for various purpose and grep for usage in DW samples.

1 Like