How to pass data from one module to other module for further processing?

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.10.0
[1] DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[1] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
[1] DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
[1] other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description
Hi team, I went through some image processing samples such as video reactifier, color correction, etc. So these all are the image processing steps that are performed at first. I have some questions regarding this.

  1. If I want to take a video input, process it(color correction, reactification, etc) and further give it to a DNN model for object detection, how can I do this in order to create a ADAS pipeline?
  2. How can I pass one module (video reactification) to another module (object detection)?

In object detection sample, input video is already preprocessed, but what if I want to process raw video first and then perform other operations?
Please advice!

2

You need a robotics application middleware to composite multiple modules.

The middleware provided in driveworks5 is CGF, the sample is in DriveWorks SDK Reference: Mini Pipeline Sample

You can use other compatible opensource robotics middleware such as ros2, which is not supported by official, see How to install ros2 foxy/humble on Nvidia DRIVE OS 6.0.5 - #3 by lizhensheng

3

dear @lizhensheng
IPC will not work here?

4

Dear @akshay.tupkar ,

Is it like each module runs in a separate process? How about using the DW module APIs one after the other in same process like SAL APIs → Image processing APIs → dwDNN APIs ?

5

yes @SivaRamaKrishnaNV
What I intend to do is, first I will take a input, then, I want to perform some image processing steps on the frames such as noise reduction and color correction. Once I am done with processing, I will feed this to dnn module. So my question is how can we combine two separate modules(color correction for video frames and then object detection module)?

7

You can call the APIs one after the other and feed the output image from one module to another(SAL → Image processing → DNN). Please see if you can leverage from refer to sample objector(feeds SAL camera frames to DNN module) and color correction sample(feeds SAL camera frames to Image processing module).