dwCameraFrameHandle_t multithreaded use

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hello!

We have PX2 AutoChauffeur platform setup with 7 Sekonix SF3324-100 cameras. We wanted our software architecture to have an individual thread for each of the cameras, where the individual thread will send the acquired images to the next “worker” thread. The worker threads then receive the camera image from their respective camera thread and the processed images/data are pipelined in the same manner onwards.

The DriveWorks context is shared amoung all of the threads, SAL and SAL_Sensor handles are shared amoung the camera threads.

The question is - can we acquire the dwCameraFrameHandle_t at the camera thread and then send it to the next thread, in which we use dwSensorCamera_getImage() function to acquire the actual image? We have tried this and we are getting sporadic behaviour out of this solution. Sometimes the program runs just fine (it always works fine when we are using less than 3 cameras), but at other times we are getting errors like these:

[7-11-2019 6:16:12] Driveworks exception thrown: DW_BUFFER_FULL: CameraSynchronous: Out of Frames in pool. Return frames earlier or increase pool size.
...
[7-11-2019 6:16:12] Driveworks exception thrown: DW_INVALID_ARGUMENT: Cannot cast handle, given instance is a nullptr
...
[7-11-2019 6:16:13] Driveworks exception thrown: DW_FAILURE: Camera getImage: failed streaming for DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8

Could someone explain, why are we running into these issues? If it is needed, I can provide a minimal working example (I just need to create one first).

What I found out from this post here - https://devtalk.nvidia.com/default/topic/1055394/driveworks/getcudaimage-error-/1 - is that it seems like I should use dwImageStreamer, but could someone explain why we cannot do it as I’ve explained previously? We are returning the frame at the worker thread side and the worker thread is waiting to receive the dwCameraFrameHandle_t before it starts to work on it.

I would be grateful if you could point me where I should look for answers or how to approach this issue.

Thank you!

2

Anyone?

3

Dear driver123,
DW APIs are not thread safe. So you need to take care of thread safety issues in your application.
Using ImageStramer sets a pipeline for consumer and receiver where consumer continuosly receives frames from producer and process it. I think, this is what your use case need.

4

Dear SivaRamaKrishna,

Thank you for your answer!

Then does this mean that ImageStreamer takes care of thread safety issues? Also, can I send multiple images through the streamer without returning them immediately?

5

Yes. DW APIs are not thread safety. ImageStreamer in DW is an implemention over EGLStream(https://www.khronos.org/registry/EGL/extensions/KHR/EGL_KHR_stream.txt). Producer keep the images in a buffer and consumer consumes it and return it to buffer again. So you need to return the frame in order to get it removed from buffer. If you do not return it lead to overflow of buffer.