# Disparity map with VPI in jetson Xavier AGX

**URL:** <https://forums.developer.nvidia.com/t/disparity-map-with-vpi-in-jetson-xavier-agx/170988>\
**Category:** Jetson AGX Xavier\
**Tags:** vpi\
**Created:** [March 15, 2021, 5:13pm UTC](https://forums.developer.nvidia.com/t/disparity-map-with-vpi-in-jetson-xavier-agx/170988 "2021-03-15T17:13:04Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sej](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@sej](https://forums.developer.nvidia.com/u/sej)\
**Post date:** [March 15, 2021, 5:13pm UTC](https://forums.developer.nvidia.com/t/disparity-map-with-vpi-in-jetson-xavier-agx/170988/1 "2021-03-15T17:13:04Z")

</div>

Hi,  
I’m starting to work with VPI and I am trying to test the StereoDisparityEstimator function.  
My code is something as the following:

```
CHECK_STATUS(vpiStreamCreate(VPI_BACKEND_CUDA|VPI_BACKEND_PVA| VPI_BACKEND_VIC, &streamLeft));
CHECK_STATUS(vpiStreamCreate(VPI_BACKEND_CUDA|VPI_BACKEND_PVA| VPI_BACKEND_VIC, &streamRight));
CHECK_STATUS(vpiCreateStereoDisparityEstimator(VPI_BACKEND_CUDA, 1280, 720, VPI_IMAGE_FORMAT_U16, NULL, &payload));
     
     VPIStereoDisparityEstimatorParams params;
     params.windowSize = 5;
     params.maxDisparity = 64;
     VPIConvertImageFormatParams cvtParams;
     vpiInitConvertImageFormatParams(&cvtParams);
     while(true) {
       capL >> matL;
       capR >> matR;
           
      remap(matL, matL, Left_Stereo_Map_x, Left_Stereo_Map_y, cv::INTER_LINEAR, cv::BORDER_CONSTANT, cv::Scalar());
       remap(matR, matR, Right_Stereo_Map_x, Right_Stereo_Map_y, cv::INTER_LINEAR, cv::BORDER_CONSTANT, cv::Scalar());
       cv::cvtColor(matL, matL, COLOR_BGR2GRAY);
       cv::cvtColor(matR, matR, COLOR_BGR2GRAY);
       matL.convertTo(matL, CV_16UC1);
       matR.convertTo(matR, CV_16UC1);
       leftVpi = mat2VPI(leftVpi, matL);
       rightVpi = mat2VPI(rightVpi, matR);
     
       CHECK_STATUS(vpiSubmitStereoDisparityEstimator(streamLeft, VPI_BACKEND_CUDA, payload, leftVpi, rightVpi, disparity, NULL, &params));
       CHECK_STATUS(vpiStreamSync(streamLeft));
     
     // vpi to opencv
       VPIImageData outData;
       CHECK_STATUS(vpiImageLock(disparity, VPI_LOCK_READ, &outData));
     
       cv::Mat outFrame(outData.planes[0].height, outData.planes[0].width, CV_16UC1, outData.planes[0].data, outData.planes[0].pitchBytes);
     
       double min, max;
       minMaxLoc(outFrame, &min, &max);
       outFrame.convertTo(outFrame, CV_8UC1, 255.0 / (max - min), -min);
       CHECK_STATUS(vpiImageUnlock(disparity));
    }

```

After that I show the image and my results are very bad. All the images are like the following image:

 ![1](https://global.discourse-cdn.com/nvidia/original/3X/8/d/8d20773e81f03fc337aa450d26f03975fdd77d6f.png)

I don’t know what I’m doing wrong, any ideas?  
Thanks in advance.

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [March 16, 2021, 5:51am UTC](https://forums.developer.nvidia.com/t/disparity-map-with-vpi-in-jetson-xavier-agx/170988/3 "2021-03-16T05:51:16Z")

</div>

Hi,

Could you check if you feed the left and right image correctly?  
And please adjust the searching range based on the view difference between images.

Thanks.

---

<div class="post-metadata">

**Author:** ![sej](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@sej](https://forums.developer.nvidia.com/u/sej)\
**Post date:** [March 16, 2021, 9:19am UTC](https://forums.developer.nvidia.com/t/disparity-map-with-vpi-in-jetson-xavier-agx/170988/4 "2021-03-16T09:19:12Z")

</div>

Hi, thanks for your answer.  
I’m sure the left and right image are correct, I switched between them just in case, but I got the same results.  
And I added the following code to check if the correction is correct:

```
      cv::Mat imgRectify = Mat::zeros(matL.rows, matL.cols*2+10, matL.type());
    
      matL.copyTo(imgRectify(Range::all(), Range(0, matL.cols)));
      matR.copyTo(imgRectify(Range::all(), Range(matR.cols+10, matR.cols*2+10)));
      
      //If it gets too large to fit on the screen, it is scaled down, by 2, to fit.
      if(imgRectify.cols > 1920){
          resize(imgRectify, imgRectify, Size(imgRectify.cols/2, imgRectify.rows/2));
      } 
      //To draw the lines in the rectified image
      for(int j = 0; j < imgRectify.rows; j += 16){
          Point p1 = Point(0,j);
          Point p2 = Point(imgRectify.cols*2,j);
          line(imgRectify, p1, p2, CV_RGB(255,0,0));
      }
      imshow("Rectified image", imgRectify);

```

With that I can see the image well remapped.

And sorry but I’m not sure I undestood your request, can you explain to me what you mean with “And please adjust the searching range based on the view difference between images.”?

Thanks.

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [March 18, 2021, 6:30am UTC](https://forums.developer.nvidia.com/t/disparity-map-with-vpi-in-jetson-xavier-agx/170988/5 "2021-03-18T06:30:25Z")

</div>

Hi,

Sorry the for the nonclear statement.

There are some parameters used in the VPI Stereo Disparity Estimator.  
Could you check if it need some update based on your stereo pair?  
[https://docs.nvidia.com/vpi/group\_\_VPI\_\_StereoDisparityEstimator.html#structVPIStereoDisparityEstimatorParams](https://docs.nvidia.com/vpi/group __VPI__ StereoDisparityEstimator.html#structVPIStereoDisparityEstimatorParams)

Thanks.

---

<div class="post-metadata">

**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\
**Post date:** [October 18, 2021, 6:20pm UTC](https://forums.developer.nvidia.com/t/disparity-map-with-vpi-in-jetson-xavier-agx/170988/7 "2021-10-18T18:20:06Z")

</div>