dwImage_copyConvert() Fails

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

Hello Nvidia Team,

I have a basic pipeline to get the camera images, the problem I’m having is that only works for a few seconds, during those seconds I’m able to see the camera images, but suddenly I start getting this error:

Module_id 47 Severity 2 : Error registering the buffers
Module_id 47 Severity 2 : Non-safety mode run time buffer registration failed
Module_id 47 Severity 2 : Error in video to output surface conversion
Module_id 48 Severity 2 : Failed YUV to RGB conversion 1
dwImage_copyConvert() Failed

I haven’t seen this error messages before and I don’t understand could be wrong in the code. Below a snippet of the pipeline:

      dwCameraFrameHandle_t frameHandle;
      dwContext_getCurrentTime(&frame_capture_time, context_handle);

      result = dwSensorCamera_readFrameNew(&frameHandle,
                                          2000000, // read timeout (in micro sec)
                                          camera_set.sensor_port);
      if (result != DW_SUCCESS) {
        NODELET_ERROR_STREAM("Cannot read frame: " << dwGetStatusName(result));
        continue;
      }

      dwImageHandle_t image_handle_original;
      dwImageHandle_t image_handle;

      result = dwSensorCamera_getImage(&image_handle_original, 
                                        DW_CAMERA_OUTPUT_NATIVE_PROCESSED, 
                                        frameHandle);
      if (result != DW_SUCCESS) {
        std::cerr << "Cannot get raw image: " << dwGetStatusName(result) << std::endl;
        continue;
      }
    
      result = dwImage_create(&image_handle, camera_set.cameraImageProperties, context_handle);
      if (result != DW_SUCCESS) {
        std::cout << "dwImage_create() Failed" << std::endl;
      }

      result = dwImage_copyConvert(image_handle, image_handle_original, context_handle);
      if (result != DW_SUCCESS) {
        std::cout << "dwImage_copyConvert() Failed " << std::endl;
      }

      result = dwImage_getNvMedia(&camera_set.u_camera.current_image_ptr, image_handle);
      if (result != DW_SUCCESS) {
        std::cout << "dwImage_getNvMedia() Failed" << std::endl;

      // don't need the frame handle anymore
      result = dwSensorCamera_returnFrame(&frameHandle);
      if (result != DW_SUCCESS) {
        std::cout << "dwSensorCamera_returnFrame() Failed" << std::endl;
      }
3

Dear @jber4282,
Could you help to confirm the DW version? Are you modifying any of the DW sample(like sample_camera) for your use case? So the dwImage_copyConvert() worked for few images initially? If so, could you print counter(or a statement) to know how many frames processed successfully.
Also, could you check/print the return value of result = dwImage_copyConvert(image_handle, image_handle_original, context_handle);

4

Thanks @SivaRamaKrishnaNV ,

I’m using DRIVE OS Linux 5.2.0 and DriveWorks 3.5, the output of the return result is:
dwImage_copyConvert() Failed: DW_NVMEDIA_ERROR

By printing the counter i saw that 252 were processed successfully.

The pipeline should work in theory, I read the frame, get the image, create an image where to store the rgb image, then convert the yuv image to rbg and store it in the image handle created before, finally releasing the frame. Not sure what internal buffer creating the issue.

5

Dear @jber4282,
Could you feed a 1 or 2 sec video and check if the workflow is working correct? Also, I am wondering if there is any memory leakage of user buffers causing the issue. Could you double check that too. If it does not help, please attach sample file to repro this issue with video as sample input.

6

Hi @SivaRamaKrishnaNV

I think I have an idea of what’s the problem, but still I don’t know how to solve it. I have two threads one that runs the camera pipeline, as described before, that pipeline feeds dwImageNvMediapointers to a buffer. The other thread processes those dwImageNvMedia pointers. After some experimentation I found out that I need to release the image_handle which stores the result of dwImage_copyConvert (this is where there error is generated), nevertheless, if I release the image_handle the image processing thread is unable to process the dwImageNvMedia pointers in the buffer, somehow the dwImageNvMedia and the image_handle are connected. From the documentation:
Note that any modification to the image retrieved will modify the content of the original handle
now, is there anyway to release the image_handle through the dwImageNvMedia ?

7

Hi @SivaRamaKrishnaNV, any update about this issue?

8

Dear @jber4282,
Sorry for missing your update. It is not possible to release image_handle via dwImageNvMedia structure. It looks like a multi threading issue. Could you share the DW sample code completely with input video via private message or post. So that I can take a look to debug at my end.

9

Dear @jber4282,
Do you still have issue with dwImage_copyConvert()?

10

Hi @SivaRamaKrishnaNV
yes, there is a buffer problem, the code was sent to you via email as requested.

11

Any update on your end @SivaRamaKrishnaNV ?

12

Dear @jber4282,
My apologies for missing this update. I remember seeing a mix of ROS + DW implementation and it was difficult to narrow down the root cause in code inspection. Is it possible to share a simple reproducible DW code as compilable sample? I would want to reproduce this issue on my target.

Could you please file a bug and update bug ID here to follow up. Please login to NVIDIA DRIVE - Autonomous Vehicle Development Platforms | NVIDIA Developer with your credentials. Please check MyAccount->MyBugs-> Submit a new bug to file bug