• Hardware Platform (Jetson / GPU) Jetson AGX Xavier • DeepStream Version 5.0 • JetPack Version (valid for Jetson only) 4.4
I read about AGX Xavier Vision Accelerator description mentioned that it can be used for stereo disparity block matching. Questions:
Do we have a demo in DeepStream SDK to show how it works?
does Vision Accelerator support cuda opencv, for example: cv::cuda::StereoBM? do we have example demo source code for enable Vision Accelerator when calling cv::cuda::createStereoBM?
what’s the relationship between VPI and deepstream?
can we use VPI in the deepstream plugin development? for example, develop a deepstream transform-ip plugin calling VPI stereo interface using CUDA-EGL API to perform inplace computation without copying out the image?
DeepStrean is a SDK that provides many GST plugins to operate NVIDIA hw accelerators and also provides many typical reference sample for many typical smart-city pipeline/use case. VPI includes a set of APIs that provides to operate VIC/CUDA/PVA, it’s relative low level API.
can we use VPI in the deepstream plugin development?
Yes, you can check the VPI API introduction in the link
Thank you for the info. I run the vpi_sample_02_stereo_disparity and I trace the code it sets the maxDisparity=64. But why the output disparity.png has pixel value (disparity) ranging from 0 to 255? should it be ranging from 0 to 64?
I check the code again it has a line to scale the min/max disparity to 255, then I print out the min/max for a testing left/right image: min=0 and max=2016, how come? shouldn’t the max be 64?
Please help to explain the output disparity how all the above associates to the maxDisparity=64. Thanks a lot.
another question regarding maxDisparity for vpi_sample_02_stereo_disparity: can we set a higher than 64? for example 128?
When I set it to 128, and recompile, then I got all flat disparity output (all in max for entire scene). If I set it back to 64 or lower (say 32, or 16), then the disparity looks normal again. Does this imply the maxDisparity is limited to 64? Please advise. Thanks.
No problem. I am using docker running on ubuntu 18.04 LTS with 1080Ti GPU for development. Once ready, I will port it over to AGX Xavier. When running I set cuda as the backend:
Then modify the params.maxDisparity = 128;
main.cpp line 154: params.maxDisparity = 128; //modified code
make
./vpi_sample_02_stereo_disparity cuda flc.jpg frc.jpg
mv disparity_cuda.png disparity_cuda_128.png
About can not change maxDisparity to value other than 64, reproed the issue, and we have internal bug 200680267 to track this issue.
I run the vpi_sample_02_stereo_disparity and I trace the code it sets the maxDisparity=64. But why the output disparity.png has pixel value (disparity) ranging from 0 to 255? should it be ranging from 0 to 64?
→ finally the disparity output convert from VPI_IMAGE_FORMAT_U16 to CV_8UC1,
cvOut.convertTo(cvOut, CV_8UC1, 255.0 / (max - min), -min); //from here
with each pixel 8 bits, that’s why the pixel value ranging from 0-255
I check the code again it has a line to scale the min/max disparity to 255, then I print out the min/max for a testing left/right image: min=0 and max=2016, how come? shouldn’t the max be 64?
Please help to explain the output disparity how all the above associates to the maxDisparity=64. Thanks a lot.
I check the code again it has a line to scale the min/max disparity to 255, then I print out the min/max for a testing left/right image: min=0 and max=2016, how come? shouldn’t the max be 64?
Please help to explain the output disparity how all the above associates to the maxDisparity=64. Thanks a lot.
–> Will check internally, and get back to you.
the output is fixed-point, so you have to divide it by 2^5, the resulting float is bounded by 64.
In order to get the disparity to be in 0 to 63 (inclusive) range, divide output disparity by 32 and store it in a float.
