Stereo disparity block matching using Vision Accelerator

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
#1

• Hardware Platform (Jetson / GPU) Jetson AGX Xavier
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only) 4.4

I read about AGX Xavier Vision Accelerator description mentioned that it can be used for stereo disparity block matching. Questions:

  1. Do we have a demo in DeepStream SDK to show how it works?
  2. does Vision Accelerator support cuda opencv, for example: cv::cuda::StereoBM? do we have example demo source code for enable Vision Accelerator when calling cv::cuda::createStereoBM?

Thank you very much for your help.

#3

No DeepStream for now,

You can check doc - https://docs.nvidia.com/vpi/algo_stereo_disparity.html and vpi code under /opt/nvidia/vpi

#4

@mchi

Thank you for the information.

Questions:

  1. what’s the relationship between VPI and deepstream?
  2. can we use VPI in the deepstream plugin development? for example, develop a deepstream transform-ip plugin calling VPI stereo interface using CUDA-EGL API to perform inplace computation without copying out the image?

Thank you.

#5

DeepStrean is a SDK that provides many GST plugins to operate NVIDIA hw accelerators and also provides many typical reference sample for many typical smart-city pipeline/use case. VPI includes a set of APIs that provides to operate VIC/CUDA/PVA, it’s relative low level API.

can we use VPI in the deepstream plugin development?
Yes, you can check the VPI API introduction in the link

#7

@mchi

Thank you for the info. I run the vpi_sample_02_stereo_disparity and I trace the code it sets the maxDisparity=64. But why the output disparity.png has pixel value (disparity) ranging from 0 to 255? should it be ranging from 0 to 64?

I check the code again it has a line to scale the min/max disparity to 255, then I print out the min/max for a testing left/right image: min=0 and max=2016, how come? shouldn’t the max be 64?

Please help to explain the output disparity how all the above associates to the maxDisparity=64. Thanks a lot.

#8

another question regarding maxDisparity for vpi_sample_02_stereo_disparity: can we set a higher than 64? for example 128?

When I set it to 128, and recompile, then I got all flat disparity output (all in max for entire scene). If I set it back to 64 or lower (say 32, or 16), then the disparity looks normal again. Does this imply the maxDisparity is limited to 64? Please advise. Thanks.

#10

Hi @ynjiun,
Sorry for late! What backend are you using?
Could you share me the repo?

Thanks!

#11

No problem. I am using docker running on ubuntu 18.04 LTS with 1080Ti GPU for development. Once ready, I will port it over to AGX Xavier. When running I set cuda as the backend:

VPIBackend backendType = VPI_BACKEND_CUDA;

Thanks.