SGBM VisionWorks - Black Disparity Image

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hi, I’m trying to use the VisionWorks tool to calculate the disparity map of a stereo vision.
I read the documentation and I wrote a short code to get the map but the results are not good, so I decided to try with the images that OpenCV provides and do a test with those but the result is an all black image of disparity.
Sorry if I put external links but I could only upload one photo in the post, anyway they are the aloeL.jpg and aloeR.jpg images of the OpenCV stereo_match example
Left:
https://ibb.co/t2dw0mT
Right:
https://ibb.co/7gYQ04B
This is the result

disparity
disparity1282×1110 268 Bytes

These are the parameters of the function:

 nvxuSemiGlobalMatching(
            context_,
            left_gray,
            right_gray,
            disparity_short,
            0,                  // min_disparity
            64,                 // max_disparity
            8,                  // P1
            109,                // P2
            5,                  // sad
            0,                  // ct_win_size
            1,                  // hc_win_size
            31,                 // bt_clip_value
            32000,              // max_diff
            0,                  // uniqueness_ratio
            7,                 // scanlines_mask
            2);                //flags

For now I use OpenCV to show it, convert vx_image to Mat and then use these lines of code to show it:

cv::normalize(disp,disp_8,0,255,cv::NORM_MINMAX,CV_8U);
cv::imshow("Disp",disp_8);
cv::waitKey(0);
3

Hi,

Would you mind to share the complete code with us?

The disparity map should be a GPU buffer.
So you need to memcpy it back to CPU first.

Thanks.

4

Here, some pieces of code were in some functions recalled but I copied and connected the various parts in the right way should be like this.

cv::Mat frame_left,frame_right,frame_gray_left_,frame_gray_right_,disp,disp_8;

frame_left = cv::imread("aloeL.jpg",cv::IMREAD_COLOR);
frame_right = cv::imread("aloeR.jpg",cv::IMREAD_COLOR);

cv::cvtColor(frame_left,frame_gray_left_,cv::COLOR_BGR2GRAY);
cv::cvtColor(frame_right,frame_gray_right_,cv::COLOR_BGR2GRAY);

vx_image gray_left = nvx_cv::createVXImageFromCVMat(context_, frame_gray_left_);
vx_image gray_right = nvx_cv::createVXImageFromCVMat(context_, frame_gray_right_);

vx_uint32 width,height;
vxQueryImage(gray_left,VX_IMAGE_WIDTH,&width,sizeof(width));
vxQueryImage(gray_left,VX_IMAGE_HEIGHT,&height,sizeof(height));

vx_image disparity = vxCreateImage(context_,width,height,VX_DF_IMAGE_S16);
nvxuSemiGlobalMatching(context_,gray_left,gray_right,disparity,0,64,8,109,5,0,1,31,32000,0,7,2);

nvx_cv::VXImageToCVMatMapper mapVXtoCV = nvx_cv::VXImageToCVMatMapper(disparity);    
disp = mapVXtoCV.getMat();

cv::normalize(disp,disp_8,0,255,cv::NORM_MINMAX,CV_8U);
cv::imshow("Disp",disp_8);
cv::waitKey(0);
5

Hi,

The default disparity is represented in Q11.4 fixed point format.
You will need a post-processing as below to convert disparity from fixed point to grayscale:

vx_image disparity_U8 = vxCreateImage(context, width,height, VX_DF_IMAGE_U8);
...
vx_int32 shift = 4;
vx_scalar s_shift = vxCreateScalar(context, VX_TYPE_INT32, &shift);
convert_depth_node_ = vxConvertDepthNode(main_graph_, disparity, disparity_U8, VX_CONVERT_POLICY_SATURATE, s_shift);

Thanks.

6

Thanks for your answer, I need time to rewrite the code using the graphs because I wasn’t using them in the code above, please wait for my news.

In the meantime I have these questions:

  • If I use other images that come from my stereo vision the disparity that comes back to me using the code above is not all black.(If you want I can upload the images).

  • Your solution uses a node to do the conversion, while I use a VisionWorks single call function in the code above, is there a way to do the conversion without using nodes?

I was going to set the code with graphs and nodes so these are just questions of curiosity.

[EDIT UPDATE]
Thanks for waiting, when I try to launch the graph with input these two images give me this error:

terminate called after throwing an instance of ‘std::runtime_error’
what(): vxProcessGraph(main_graph_) failure [status = -10] in file /home/nvidia/Desktop/stereoDepth/src/stereo_match_graph.cpp line 37
Aborted (core dumped)

debugging gives me an error on the input parameters of the SGBM node “VX_ERROR_INVALID_PARAMETERS”,
but I didn’t change anything in the VisionWorks code, not even the SGBM node parameters.
I tried the images captured by my stereo vision and I have no errors, I guess the problem is related to these two images, just out of curiosity can you understand why?

7

If there isn’t answer close topic .

8

Hi,

Do you meet the error when using /usr/share/visionworks/sources/demos/stereo_matching/?
Or the error comes from your implementation?

If it’s customized, could you share the source with us for reproducing?

Thanks.

9

When I use the example in the VisionWorks folder I have no errors.
The code is above, where simply I read these two images with OpenCV I convert them in vx_image and I execute SGBM. Same code with other two images doesn’t give me errors, actually it doesn’t give me errors but the output is an all black screen.

10

Hi,

Sorry for the late update.

Could you try if ovxio::Render can show the disparity correctly?
If yes, the issue may occur when converting the disparity into the cvMat image.

If the issue goes on, could you share a complete source so we can reproduce this directly?

Thanks.