DataConditioner

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

  1. My model input is 2x3x100x100 format. i.e 2 is camera images from left and right.
  2. I stacked two planar images together but now how to make DataConditioner normalize on these two stacked planar images.
  3. Is it ok to write a cuda kernel to convert DataConditioners’s NHWC data to NCHW and then pass it to dNN? or is there any api in driveworks that provides NHWC to NCHW conversion
3

Dear @arkos,
Dataconditioner module has doPerPlaneMeanNormalization parameter. See if it helps.
The input to model seems to be NCHW format. If you set splitPlanes to true, The output of dataconditioner will be NCHW or it will be NHWC.

Let me know if you see any issues.

4

Dear @SivaRamaKrishnaNV,

I looked into those parameters doPerPlaneMeanNormalization and they will normalize on planar RGB images. But my architecture is little different.

  1. My model input is 2x3x100x100 (planar RGB image from tow cameras stacked) , 2x3x3, 2x3x1 (camera params).

  2. I don’t think dwDataConditioner is capable of performing normalization on stacked images. dwconditioner can normalize single image, but it accepts batch of images which is not my case.
    Reference https://docs.nvidia.com/drive/driveworks-3.5/group__dataconditioner__group.html#gaf5e9ef46996aaeda482814d6886039ea
    “numImages cannot exceed the batchsize set at initialization time. If numImages is higher than 1, the resultant output image is batched, and it can then be given to the corresponding DNN.”

  3. My proposal to adapt to the driveworks architecture. I same same Image transformation for both cameras.
    a. Perform ImageTransformation on two images respectively. RGB interleaved
    b. Perform two seprate normalization of the above images using dwDataConditioner_prepareDataRaw() and split tha planes (now each conditioner output is planar). Call this function two times with two images from point a.
    c. Stack the two conditioner output vertically and pass it to DNN. Not sure if the DNN api will have any problem with it.

5

Dear @arkos,
just to confirm, are you using DRIVE AGX Xavier platform?

Could you clarify what is 2x3x3, 2x3x1 here?

Do you want to do normalization combing the both images? If so, that is not supported, as you understand, only possible at per image level.

Just trying to understand, Does it look like batch of images here? From Step b, each output is a NCHW . How does it look like after Step c?

6

Dear @SivaRamaKrishnaNV

just to confirm, are you using DRIVE AGX Xavier platform?

Drive AGX Orin, DRIVE AGX Orin Developer Kit (940-63710-0010-200), DRIVE OS 6.0.8.1

Could you clarify what is 2x3x3, 2x3x1 here?

These are just floating point value for camera paramters. So it means to the model, I will be passing the stacked images as well as their camera params.
e.g. 2x3x3 → 2 represents number of cameras. it means it holds both cameras intrinsic. Similarly for 2x3x1. It represents both cameras params.

Just trying to understand, Does it look like batch of images here? From Step b, each output is a NCHW . How does
it look like after Step c?

b.) I wanted to perform dataConditioner of both the images separately. The individual outputs are in CHW format.
c.) I vertically combine both the above dataconditioner outputs and make it 2xCxHxW.

image
image421×673 14.5 KB

Then I pass this stacked image along with other camera params to dNN.

7

Dear @arkos,
The output from data conditioner can be feed into DNN Infer API(dwDNN_inferRaw). Inference is performed on batch size 2.

8

Dear @SivaRamaKrishnaNV

I don’t want to run the two images in a batch. My models batch size is 1.

B x N x C x H x W.

B = 1 (Batch size)
N = 2(number of camera images)
C = 3
H =100
W = 100

This means my model runs on single batch, and it takes two images vertically stacked.

Is there any api or code to stack two dwImageCUDA images?

Thanks
Arko

9

Dear @arkos,
Here N refers to batch size.
Does your model requires to input buffers? If so, you can fill 2 input buffer data pointers and use batch size as 1 in dwDNN_inferRaw

10

Dear @arkos,
Do you have any queries further?

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.