Questions about VisionWorks SFM

I’m using Visionworks SFM for visual odometer application right now, and have some questions:

  1. I found that the performance of nvxSfmFindFundamentalMatNode is poorer than the findFundamentalMat in OpenCV. It takes almost 20ms, but the OpenCV function just takes about 1ms (I wrote an OpenVX extra node by wrapping the findFundamentalMat and compared them). And If I use findFundamentalMat, I will get a very good triangulation result after nvxSfmTwoViewTriangulationNode, otherwise the result is very poor or even wrong. Is it just because I don’t know how to tune the nvxSfmFindFundamentalMatNode?

  2. The performance of nvxSfmTwoViewTriangulationNode is very good if I use findFundamentalMat. I can always get very good triangulated point cloud. The problem is that the node’s input only accepts the pose of the second view and assuming the first view is at the origin. So is there any clue which algorithm it uses, then I can implement a new one which accepts tow view poses.

  3. Does the 1.5.3 VisionWorks on Windows have SFM extension?

  4. Since VPI is recommended now, but I cannot find any SFM related algorithms in the latest VPI. Will it add the SFM?

Thanks!

Hi,

1.
Have you maximized the device clocks first?

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

2.
Based on the information in /usr/share/visionworks-sfm/sources/samples/sfm/sfm_user_guide.md , here are the related papers:

  • Multiple View Geometry in Computer Vision, Richard Hartley and Andrew Zisserman, 2003
  • Robust Scale Estimation in Real-Time Monocular SFM for Autonomous Driving, Song, Chandraker, 2014

3.
Yes, please check https://developer.nvidia.com/embedded/visionworks-1-5-3

4.
SFM is not in our road map yet.

Thanks.

Thank you very much! But about the third question, I can only find the “NVIDIA VisionWorks” in “Program Files” after windows installation, and cannot find any SFM related nodes in headers. Did I miss something?

Here is a demo video of my OVX based SFM.
https://youtu.be/rCH_Duoqm8U

Cool. Thanks for sharing this.

Two more questions:

  1. The algorithm used in nvxSfmTwoViewTriangulationNode, is Iterative linear least square, or simply no Iterative?

  2. Since the triangulation is a linear least square question, does the implementation use streamed of batched cusolver function?