how to get an RGB buffer .

Hello ,

I have question
Does dwImageCUDA contain RGB image .

I am looking to utilize Nvidia hardware isp not softisp. will I be able to get RGB output.

thanks
nk

Dear nk,
There is no inbuilt function to Extract RGB from RGBA. you need to access them byte by byte to extra individual channels.
The RGBA stored format is given at file:///usr/local/driveworks/doc/nvdwx_html/group__image__group.html#gga40e445bbc92b6c60197dd57fc9ca602da2fad36b6d1b6c257083f5ff6afcb7f07

Hello ,

I think I did not ask correct question.

I need RGBA buffer using nvidia hardware isp not soft isp.

so by looking at block-diagram ( image signal processor) were it shows output from IPP(ISP) goes to dwStreamer and ouput from dwstreamer is in dwImagecuda which is YUV8 image.

Now looking at this set I picked up example.

/driveworks/samples/src/dnn/sample_object_tracker

will this be the buffer to look at


Question 1


void on Process()
{
dwImageCUDA *rgbaImage;
CHECK_DW_ERROR(dwImage_getCUDA(&rgbaImage, m_imageRGBA));
}

does this buffer contain RGBA image —> m_imageRGBA.


Question 2


As this example sample_object_tracker is for multiple cameras.
do i need to call on Process() four time to have four camera output buffers.

at present myunderstanding is buffer from each camera is sent to m_imageRGBA in a pipe.so I have to read it in a loop for 4.


Question 3


Also i would like to know
m_streamerCUDA2GL.reset(new SimpleImageStreamer<>(displayProperties, DW_IMAGE_GL, 1000, m_sdk));

do I need this funtion to get m_imageRGBA image

thanks
nk

Dear nk,
Question 1:
m_imageRGBA is handle to dwImage.
rgbaImage is dwCUDAImage which contains RGBA Image. You can check access CUDAImage buffer at imageBuffer.dptr[0] assuming the image is a single planar.

Question 2:
This sample is designed work with single camera. You can see m_camera variable corresponding to it. IF you want to use multiple cameras, you can check creating 4 camera objects and replicating the same process for each camera.

Question 3:
It sets the output image type as GL. I think Ans to Question 1 helps. You need to use dwImage_create to create a handle to Image.

Hello Siva,
thanks for your reply that was helpful


Question 1


do we have any sample application were I can see how the process is being done like dwImage_create which will be handle to image or any specific documentation which defines how to access the cameras and how to use camera buffers.


Question 2


is RGBA a single planar , I believe raw data should be single planar .
the way i understan is dwCUDAImage will always have multiplanar data as it is end o/p from isp .

can you confirm on this imageBuffer.dptr[0] will it be RGBA or raw data.

thanks
nk

Dear nk,
For Question1 :
You can check sample_object_detector workflow to know how to accessing cameras and create Image handles. You can check generateImage() function in sample_image_streamer_multi to know how to access/modify cudaImage buffers.
Question 2: It will contain RGBA buffer

Hello,

I need to print the contents of the buffer rawImageCUDA. how can I do that
dwImageHandle_t rawImageCUDA;
CHECK_DW_ERROR(dwSensorCamera_getImage(&rawImageCUDA, DW_CAMERA_OUTPUT_CUDA_RAW_UINT16, frameHandle));

I would like to see whether the buffer has any data in it or not.

thanks
nk

Dear xmnav,
Have you checked data at dptr? You can copy the dptr CUDA buffer to host using cudaMemcpy function and then verify it on host. Please check https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY.html#group__CUDART__MEMORY_1gc263dbe6574220cc776b45438fc351e8. Else, you can pass that buffer as a parameter to CUDA kernel and try printing the values inside a CUDA kernel.

hello Siva,

I have few questions for you .

is there an sample which uses dwImage_getNvMedia using gsml camera . can you point me which sample it can be.

  1. Does any driveworks sample code use hardware ISP. if yes can you please point me which one it is.

  2. in case if dwImage_getNvMedia when using this api does it invoke hardware isp or software isp.

  3. i was looking at code sample src\sensors\camera_gmsl. does this code sample go through hardware isp? if yes can you please point to me which is the output buffer after hardware ISP output.

thanks
xmnav

Hi xmnav,

For 1, you can try to search in /usr/local/driveworks/.
For 2, why are you eager for hardware isp? If for the RGBA buffer, it was converted by cuda/vic(hardware video image compositor).

Dear xmnav,
You can check samples/image/image_common/utils.cu to know the usage of dwImage_getNvMedia. Please note that, dwImage_getNvMedia just gives NvMedia Image from the image handle. It does not invoke any ISP. If you wish to know the usage of softISP, please check sample_camera_gsml_raw.

Hello Vicky/siva,
I am already using the samples and trying to understand them.
When you say why I am eager to use hardware isp the answer lies in your question that means I need output from hardware isp which I wish to provide to my algorithms.

Vicky when I look at samples code like sample_camera_gmsl or same_camer_multiple_gsml. I am really getting issues to understand which one is the ouput from hardware isp and how can I use that buffer.

Will this code fragment get me the buffer

dwImageCUDA* rgbaImage;
dwImageHandle_t m_imageRGBA;

memcpy(RGBAarray,(char*)(rgbaImage->data[0]),rgbaImage->prop.width * rgbaImage->prop.height * sizeof(char));
filename = std::to_string(count)+ std::string(“_Image.txt”);
FILE *fp=fopen(filename.c_str(),“w+”);
fprintf(fp, “%s”, RGBAarray);
fclose(fp);
count++;

or this will

memcpy(RGBAarray,(char*)(m_imageRGBA.data[0]),m_imageRGBA.prop.width * m_imageRGBA.prop.height * sizeof(char));
filename = std::to_string(count)+ std::string(“_Image.txt”);
FILE *fp=fopen(filename.c_str(),“w+”);
fprintf(fp, “%s”, RGBAarray);
fclose(fp);
count++;

i hope you guys can help .

thanks
xmnav

Dear xmnav,
Please check using cudaMemcpy(https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY.html#group__CUDART__MEMORY_1gc263dbe6574220cc776b45438fc351e8) with rgbaImage as source.

Siva,

I was trying to use code from your code fragment.

https://devtalk.nvidia.com/default/topic/1037487/driveworks/problem-with-extracting-image-data-array-of-rgba-format-from-dwimagecpu/post/5271198/#5271198.

i picked memcpy from this post.

xmnav

Dear xmnav,
Please note that in that post you mentioned, both source and destination buffers are host buffers where as in your case you want to copy data from dwImageCUDA to host, so you need to use cudaMemcpy function. Please let us know if you have any other issues