Error dwSensorCamera_getImage() with the DW_CAMERA_OUTPUT_CUDA_RAW_UINT16

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.10.0
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description
In both the “camera.gmsl” and “camera.virtual” cases, we want to perform image output using dwSensorCamera_getImage() with the DW_CAMERA_OUTPUT_CUDA_RAW_UINT16 format.

In the case of “camera.gmsl” using Case.1, it completes successfully without any issues.
In the case of “camera.virtual” using Case.2, it crashes with dwSensorCamera_getImage.
According to the interface specifications, there should be no issues, but we would like to know why it crashes.

Case.1: ./sample_camera --rig=rig_gmsl.json --write-file=output.raw --offscreen=0
Case.2: ./sample_camera --rig=rig_virtual.json --offscreen=0

The code for dwSensorCamera_getImage is as follows:

dwCameraOutputType outputType = DW_CAMERA_OUTPUT_CUDA_RAW_UINT16;
dwImageHandle_t tmp[1] = {DW_NULL_HANDLE};
CHECK_DW_ERROR(dwSensorCamera_getImage(&tmp[0], outputType, frame[0]));

rig files.
rig.zip (1.6 KB)

Logs
To investigate the issue of the program crashing without any error message, we added code to output the property information before calling dwSensorCamera_getImage.
The outputted property information includes dwCameraProperties and dwImageProperties.
When comparing the output between Step 1 and Step 2, we noticed a difference in the memoryLayout property, although the reason for this difference is unknown.

—Case.1 property logs—
m_camera[0]->dwCameraProperties
cameraType:99 outputTypes:279 rawFormat:0
m_camera[0]->dwImageProperties->dwImageMetaData
format:4000 height:2160 memoryLayout:1 type:2 width:3840
meta.flags:68 meta.frameSequenceNumber:0 meta.rawProperties.msbPosition:15
meta.rawProperties.rawBitDataType:12 meta.rawProperties.rawFormatDescriptor:33

—Case.2 property logs—
m_camera[0]->dwCameraProperties
cameraType:99 outputTypes:279 rawFormat:0
m_camera[0]->dwImageProperties->dwImageMetaData
format:4000 height:2160 memoryLayout:0 type:2 width:3840
meta.flags:68 meta.frameSequenceNumber:0 meta.rawProperties.msbPosition:15
meta.rawProperties.rawBitDataType:12 meta.rawProperties.rawFormatDescriptor:33

Dear @Natsuha_Shishido ,
Is the camera model listed in DRIVE AGX Orin Sensors & Accessories | NVIDIA Developer ?
Do you notice display coming up in case 1?

Dear @SivaRamaKrishnaNV
We are using the IMX728EVB-MSH-SMM17 camera, which is not listed in the supported camera list.
However, we have also tried with the IMX728EVB-MLH-SMM2 (which is listed), and we are experiencing the same issue.

Yes, in case 1, the display output is confirmed.

Dear @Natsuha_Shishido ,
Can you check DriveWorks SDK Reference: Camera Replay Sample to replay the recorded data.

Dear @SivaRamaKrishnaNV

We borrowed the Orin(DRIVE OS 6.0.8.1) from another project, and tested to see if the same error occurs.
However, the error did not occur.
Could this be a specific issue with 6.0.6?

【Logs】

From the logs, two differences have been found between the 6.0.8 environment and the 6.0.6 environment:

・The log displays “Raw video with version” as 13 in the 6.0.8 environment, whereas it is 12 in the 6.0.6 environment.
・The value of dwCameraProperties.rawFormat is ‘7200: DW_CAMERA_RAW_FORMAT_GRBG’ in the 6.0.8 environment,
whereas it is ‘0: DW_CAMERA_RAW_FORMAT_UNKNOWN’ in the 6.0.6 environment.

—Case.1 property logs(DRIVE OS 6.0.8.1)—
m_camera[0]->dwCameraProperties
cameraType:99 outputTypes:279 rawFormat:7200
m_camera[0]->dwImageProperties->dwImageMetaData
format:4000 height:2160 memoryLayout:1 type:2 width:3840
meta.flags:68 meta.frameSequenceNumber:0 meta.rawProperties.msbPosition:15
meta.rawProperties.rawBitDataType:12 meta.rawProperties.rawFormatDescriptor:33

—Case.2 property logs(DRIVE OS 6.0.8.1)—
m_camera[0]->dwCameraProperties
cameraType:99 outputTypes:279 rawFormat:7200
m_camera[0]->dwImageProperties->dwImageMetaData
format:4000 height:2160 memoryLayout:0 type:2 width:3840
meta.flags:68 meta.frameSequenceNumber:0 meta.rawProperties.msbPosition:15
meta.rawProperties.rawBitDataType:12 meta.rawProperties.rawFormatDescriptor:33

Dear @SivaRamaKrishnaNV

In the sample_camera_replay code, the ‘DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8’ format is specified when calling ‘dwSensorCamera_getImage’.
Our inquiry pertains to the behavior when specifying the ‘DW_CAMERA_OUTPUT_CUDA_RAW_UINT16’ format and calling ‘dwSensorCamera_getImage’.

For your reference, we have already confirmed that when DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8 is specified for format in RAW format video replay, the dwSensorCamera_getImage completes without any issues.

Yes. We recommend to use newer release. Can you test the same on DRIVE OS 6.0.10?

Dear @SivaRamaKrishnaNV

We want to run it on DRIVE OS 6.0.6, Is there any method available?

Unfortunately we don’t have patch release on top of devzone release. It seems 6.0.8.1 has the fix for your issue as it is working.