Incorrect output from dwDataConditioner API

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.7.1.8928
other

Host Machine Version
native Ubuntu 18.04
other

Hi Team,
I am trying to preprocess the image for inferencing using dwDataConditioner APIs.

The input image size is 1280 * 720.I also have a detection file(generated from the previous stage of object detection) which contains corresponding bounding boxes (vehicles) for the input image.

I would like to perform below operations using dwDataConditioner to pre-process the input (individual vehicles) for the next stage -

Step1 .Crop the ROI(bounding box for the vehicle) from image.
Step2. Resize cropped ROI to desired height and width(224*224).
Step3. Normalize the image at Step2 with given mean and std. deviation values.

Essentially, I would like to extract the individual vehicles detected from previous stage and input the cropped & pre-processed vehicle image to the next detector for brake light detection.

We are setting ROI parameters in below data structure to define the ROI in input image.
dwRect m_detectionRegion

However the resized output image data at Step2 is invalid

code_snip_dataconditioner.txt (2.0 KB)
. Please find the below code snippet from my experiment.

Please suggest a way to address this issue.

Dear @jyotsna.patil,
FIrstly, could you check if the RGB array values of input image(contents of inputImageCUDA variable ) are same as expected before feeding into Dataconditioner? Also, did you check the dataconditioner tutorial?

Hi @SivaRamaKrishnaNV ,

The RGB array values of input image(contents of inputImageCUDA variable ) are same as expected before feeding into Dataconditioner i.e.

inputImageCUDA->dptr[0]–> R plane data of input image
inputImageCUDA->dptr[1]–> G plane data of input image
inputImageCUDA->dptr[2]–> B plane data of input image

Also checked the dataconditioner tutorial link.

Dear @jyotsna.patil,
Per tutorial, splitPlanes should be set as false for interleaved and true for planar. Does that help?

Hi @SivaRamaKrishnaNV ,

I changed the split planes to true , but this did not help.

However, I am able to get the resized output image as (224*224) and it is observed that there are some extra rows information being added(in y direction) other than the defined ROI . Following are the ROI dimension values.

m_detectionRegion.width = 613;
m_detectionRegion.height = 555;
m_detectionRegion.x = 352;
m_detectionRegion.y = 83;

inputBlobSize.width = 224;
inputBlobSize.height = 224;
inputBlobSize.channels = 3;
inputBlobSize.batchsize = 1;

May I know the sequence in which data conditioner does the pre-process operations ?

E.g.

1.Detect ROI in input image
2.Resize the ROI to input blob size
3.Normalize the resized image

Dear @jyotsna.patil,
it is observed that there are some extra rows information being added(in y direction) other than the defined ROI

Does that mean the destination image has image data other than ROI? The preprocessing operations are expected to happen in ROI region only.

Could you provide repro code(fills image data buffer via file or video and apply preprocessing and verify output) on our side to confirm if there any issue with data conditioner?

Hi @SivaRamaKrishnaNV ,

With below configuration the vehicle image is cropped correctly with defined ROI & resized to 224*224 dimension.

m_detectionRegion.width = 613;
m_detectionRegion.height = 555;
m_detectionRegion.x = 352;
m_detectionRegion.y = 83;

inputBlobSize.width = 224;
inputBlobSize.height = 224;
inputBlobSize.channels = 3;
inputBlobSize.batchsize = 1;

ignoreAspectRatio = true,

However, I would like to know

  1. Which interpolation method does Data Conditioner use to resize the image(nearest neighbour/linear/bicubic etc.) ?.

  2. Does Data Conditioner use Vision accelerator/GPU for execution?.

Dear @jyotsna.patil,

  1. linear
  2. GPU

Hi @SivaRamaKrishnaNV ,
Continuing on the same topic, if I want to use the Vision Accelerator to do similar pre-processing, are there any DriveWorks APIs that would allow us to do so? Or do we have to use DriveOS based NvMedia APIs?

Also for similar pre-processing tasks (scaling, normalizing etc.) would you advise using GPU or Vision accelerators to get fast compute times?
Thanks!

Dear @dhruva.ungrupulithaya,
Note that resize/scaling etc operation can not be performed by PVA. Please check the supported operations on PVA at https://docs.nvidia.com/drive/drive-os-5.2.6.0L/drive-os/index.html#page/DRIVE_OS_Linux_SDK_NGC_Development_Guide/NvMedia/nvmedia_concept_vpi.html#wwpID0E0LP0HA.

Also, please check DW PVA samples at DriveWorks SDK Reference: Samples