GetcudaImage error:

I am using driveworks showing images and doing other things. In one thread I capture cuda image use interface dwSensorCamera_getImage(&image_handle, DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8, frame_handle).
In another thread I get image from the handle “CHECK_DW_ERROR(dwImage_getCUDA(&rgba_image, frame_data.frame_handle))”, if the thread is much slower than the capture, the procedure will crash and print error like “Driveworks exception thrown: DW_CUDA_ERROR: Call failed cuEGLStreamConsumerAcquireFrame : unspecified launch failure”.
How can I solve this problem? Thanks!

Hi yxiang,

In our documentation you do find more information on how to proceed if you look for Image Processing->Image.

To answer your problem your calls are not thread safe, so what you can do to “solve this problem” is simply using streamers as here:

// Thread 1:
dwImageStreamer_initialize(...);
while(true)
{
    // CODE: Get an image to stream
    dwImageStreamer_producerSend(...);
    dwImageStreamer_producerReturn(TIMEOUT);
}
dwImageStreamer_release(...);
// Thread 2:
// CODE: If the streamer has been initialized on thread 1
while(true)
{
    dwImageStreamer_consumerReceive(TIMEOUT);
    // CODE: Use the streamed image
    dwImageStreamer_consumerReturn(...);
}

Besides you do find some samples on how to use our streamers in the DW package as well.

All clear?

Best regards,
Fabian

OK, very thanks. I will have a try.

And another question, I used the below codes getting cuda image and did something with the image
while(true)
{
dwSensorCamera_readFrame(&frame_handle, camera_idx, 500000, sal_sensor_);
CHECK_DW_ERROR(dwSensorCamera_getImage(&image_handle, DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8, frame_handle));
CHECK_DW_ERROR(dwImage_getCUDA(&rgba_image, image_handle));
Process(rgba_image); //maybe cost 100ms
}
when I run the procedure, the warnings were always printed:
[14-6-2019 10:21:6] CameraGMSL: Frame: 627 CaptureTimestamp: 2252328673 - ICP DROP in camera 0
[14-6-2019 10:21:6] CameraGMSL: Frame: 627 CaptureTimestamp: 2252328699 - ICP DROP in camera 1
[14-6-2019 10:21:6] CameraGMSL: Frame: 612 CaptureTimestamp: 2252348693 - ICP DROP in camera 2
[14-6-2019 10:21:6] CameraGMSL: Frame: 612 CaptureTimestamp: 2252348720 - ICP DROP in camera 3
I think maybe it get the next image waited so long time.
How can I solve this problem, avoid the warnings printed?
Thanks!

Hi, I tried to use your method but it does not work. I used producer-consumer mode to make thread safe.

//Thread 1:
while(true)
{
    dwSensorCamera_readFrame();
    dwSensorCamera_getImage(.., DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8, ..);
    dwImage_copyConvert(frames_produced_,...);
}
//Thread 2:
while(true)
{
    dwImage_getCUDA(dwImageCUDA, frames_consumed_);
    //use the image
}

As I think call the function dwSensorCamera_getImage using type DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8, we can get image with cuda type, do not need to use dwImageStreamer_producerSend(cuda_stream) and dwImageStreamer_consumerReceive(cuda_stream) to stream the image to the cuda domain. Is it right?
Very thanks!

Regarding the ICP frame drops, what is the FPS you are recording with and did you correctly calculate the reading and writing speeds of the frames according to the storage you are using?

You can test using a lower FPS to do a first verification and to calculations afterwards too…

Fabian

The camera FPS is 30. Every process on the image cost 100ms, the processing speed is slower than capturing speed.
I test a small case that, in the Process() function I just sleep 30ms, the speed is faster than capturing speed, there is no warning.

The frames are coming with 30Hz aka 33ms, so processing an image of 100ms costs will make you drop frames.

Most importantly is the thread getting the images from the camera.

Fabian

So how to avoid the warning?
And anonther problem, I use single-thread like as above

while(true)
{
    dwSensorCamera_readFrame(&frame_handle, index, 66000, sensor_);
    dwSensorCamera_getImage(&image_handle, DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8, frame_handle);
    dwImage_getCUDA(&image, image_handle);
    Process(image);//100ms
}

I found that the timeout is too long, about 500ms, every time the image I got is not the newest, is later than 500ms. How did it happen, how can I solve this problem?

Thanks!

How exactly are you measuring the timeout and did you use the CameraGMSL sample to verify your findings already?

Fabian

On linux terminal, I print the system timestamp always, and everytime dwSensorCamera_readFrame() I also get the system timestamp and set the timestamp to the got image and show it on image.
Make the camera capture the printed timestamp on terminal, so on the image we can see the captured printed timestamp and the timestamp in image.
Add also when show the image, I can see the timestamp obviously.
If I just capture the image then show it, do nothing processing, the timeout is much slower.

Hi Fabian,
I also test the sample sample_camera_gmsl, in source code
CHECK_DW_ERROR(dwRenderer_renderTexture(imageGL->tex, imageGL->target, m_renderer));
usleep(80000);
I sleep 80ms everytime stream the image to GL domin, the same issues appear.
The “icp drop…” warning always print, and obviously that the timeout is larger.

Can you perhaps specify more information like what camera model you are using and which exact platform, so I can raise an internal bug for that?

Fabian

Camera type is ar0231, the platform is px2 AutoChauffeur P2379, the SDK version is 5.0.10.3.
Any solutions for the issues?
Thanks!

I would please you to strictly follow the use of our image streamer definitions and try again. We have a sample for exactly your purpose which is called image_streamer_multi. Make use of TIMEOUT as parameter and actually use the streamers as everything else is not thread-safe and needs to be run sequentially instead.

The solution for your problem is actually fetching images more frequently since your are dropping a lot of them out of the IPP pipeline.
It also makes sense to see 500ms old frames since by FIFO approach they are still in the queue to be fetched next.

So by optimizing your process() method by using more efficient algorithms, you are able to fetch much more frequently which does not let images drop at the end of the queue.

Fabian

Hi Fabian,
As you said, using the interface dwSensorCamera_readFrame(), the images are put into buffer, a queue, then dwSensorCamera_getImage() get image from the queue? Is it right?
So is there a interface that we can get image in real-time, not the buffer?
Thanks!

Hi,

Why don’t you use the interface we provide you? I strongly believe you can enhance your Process() to fit all requirements.

The ICP drop come from the IPP Pipeline: If the capture buffer is full, the ICP will drop the next acquired frame. That is, in case you get ICP drops, this means you are not returning images fast enough:
The camera captures at 33ms and it takes you 100 ms to return images, of course you will fill the buffer completely.

So whatever you read it is from the front of the buffer since it follows FIFO.

For me, it seems you are doing something very inefficient during Process() because you are more than 300% slower than the frequency of the camera. Also from coding principles POV please use more efficient algorithms since this solves your main issues.

Fabian

Hi Fabian,
I understand all you said. But in fact our program is huge, we do many things in Process() so it costs about 100ms, we can not increase efficiency in Process() at present.

Okay, let’s have the 100ms as fix assumption for Process() then. Then what I can think of is either

  1. reading from the other side of the buffer - size to be known and always full,
  2. draining the buffer so that it compensates t_process - t_camera_frequency or
  3. decreasing the buffer size to 1, so it only contains recent images.
  4. I will update you shortly.

    Fabian

Hi Fabian,
It is the thing I want to try but have not solved it.
Waiting for your reply.
And a small question, is there any interfaces I can using copy dwCameraFrameHandle_t?
Very thanks!