Stereo GMSL Camera Rectification and Disparity Calculation on Drive PX 2

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hi,

We are trying to use GMSL stereo camera on Drive Px 2 for stereo rectification and disparity calculation (similar to ZED stereo usb camera module (/usr/local/driveworks/samples/src/stereo/camera/camera_usb), which was provided by NVIDIA). We have modified the code for GMSL stereo camera. Its working fine with our calibration file and recorded our own GMSL video as an input, but if we are trying to run live stereo rectification and disparity with GMSL on Drive PX 2, its throwing an error is as shown below:
GenericSimpleFormatConverter: Unsupported input type

Our software versions on Drive PX 2:
DRIVE: 5.0.5.0bL
Aurix: 4.02.02.00

We have modified the code in camera_usb module, is as shown below:
//#######################################################################################
std::unique_ptr StereoApp::initFromCamera()
{
std::cout << "initFromCamera() " << std::endl;
dwSensorParams params;

std::string parameterString = "camera-type=" + m_args.get("camera-type");
parameterString += ",csi-port=" + m_args.get("csi-port");
parameterString += ",slave=" + m_args.get("slave");
parameterString += ",serialize=false,output-format=raw,camera-count=4";
std::string cameraMask[4] = {"0001", "0010", "0100", "1000"};
uint32_t cameraIdx = std::stoi(m_args.get("camera-index"));
if(cameraIdx < 0 || cameraIdx > 3){
    std::cerr << "Error: camera index must be 0, 1, 2 or 3" << std::endl;
    return false;
}
parameterString += ",camera-mask=" + cameraMask[cameraIdx];

params.parameters           = parameterString.c_str();
params.protocol             = "camera.gmsl";

return initInput(params);

}

I think stereo rectification and disparity modules are taking as cuda image format as an input. Therefore, we are trying to convert raw image to cuda image format to feed to stereo module. Based on DriveNet module raw image code, we have considered the “RawSimpleCamera” instead of “SimpleCamera” is as follows:
std::unique_ptr camera(new RawSimpleCamera(params, m_sal, m_context, cudaStream, DW_CAMERA_PROCESSED_IMAGE));
instead of
std::unique_ptr camera(new SimpleCamera(m_imageProperties, params, m_sal, m_context));

Is there any alternate solution to feed live image to stereo module? Or are you planning to include sample stereo module for live GMSL camera in future DriveWorks software?

Thanks!

Narsimlu

2

Dear Narsimlu,
Cold you please tell which camera are you using?
Could you confirm if you are modifying stereoRectifier sample or camera_usb sample?[b]

I think stereo rectification and disparity modules are taking as cuda image format as an input.[/b]

Yes.

Are you getting GenericSimpleFormatConverter: Unsupported input type error even after you add raw to CUDA image code?

3

Dear SivaRamaKrishna,

Please find below responses:

Cold you please tell which camera are you using?

ar0231-rccb (GMSL camera)

Could you confirm if you are modifying stereoRectifier sample or camera_usb sample?

We modified sample “camera_usb” for GMSL camera (path in DriveWorks: /usr/local/driveworks/samples/src/stereo/camera/camera_usb). Also, modified “initFromCamera” function for GMSL camera initialization in “stereo_common.cpp” file at path /usr/local/driveworks/samples/src/stereo/stereo_common.cpp

Are you getting GenericSimpleFormatConverter: Unsupported input type error even after you add raw to CUDA image code?

Yes, we referred sample DriveNet for GMSL camera initialization and raw image to CUDA image.

4