Accuracy of NV-VPI sparse Optical Flow on Orin

[Appletree]

We evaluated the sparse optical flow by the following method on Orin:

Here is the results of three test datasets by the same PyrLK parameters:

These results show that vpiSubmitOpticalFlowPyrLK() systemically has larger errors comparing with its peers: cv::cudaSparsePyrLkOpticalFlow() and cv::calcOpticalFlowPyrLK(). Our question is that is it normal or we made some mistakes on parameter settings?

Thanks,

[TomNVIDIA]

Moving this to the Jetson category for visibility.

Jetson & Embedded Systems

[AastaLLL]

Hi,

Do you have sources/instructions to reproduce this issue?
So we can check this with our internal team?

Thanks.

[Appletree]

Hi Tom, Thanks for reply. Yes, i just created a public repo at:

GitHub - shunguang/opt-flow-evaluation

you can check out our source code and datasets to repeat this work

[AastaLLL]

Hi,

Just want to confirm the environment as well.
Which JetPack version do you use? Is it JetPack 5.1.2?

Thanks.

[Appletree]

nvidia@nvdia-desktop:~$ sudo apt-cache show nvidia-jetpack

Package: nvidia-jetpack

Version: 5.0.2-b231

Architecture: arm64

Maintainer: NVIDIA Corporation

Installed-Size: 194

Depends: nvidia-jetpack-runtime (= 5.0.2-b231), nvidia-jetpack-dev (= 5.0.2-b231)

Homepage: Jetson - Embedded AI Computing Platform | NVIDIA Developer

Priority: standard

Section: metapackages

Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_5.0.2-b231_arm64.deb

Size: 29304

SHA256: b1268b2cb969e677163f291967bc7542371a29d536379df3f7dfa1f247ff3fab

SHA1: 7ff288a771b83eec8f80a41ccf0eec490f32e10a

MD5sum: 5cc57807b33630d8edb249e53daf58ed

Description: NVIDIA Jetpack Meta Package

Description-md5: ad1462289bdbc54909ae109d1d32c0a

[AastaLLL]

Hi,

Would you mind testing this on the latest JetPack 5.1.2?
Usually, enhancement won’t backport to the previous release.

Thanks,

[Appletree]

Does 5.1.2 update nv-vpi libs?

[Appletree]

Hi Aasta, Thanks for the suggestion, I am wondering if you have NVIDIA internal nv-vpi sparse optical flow accuracy test report?
Best wishes!

[Appletree]

Finally I had a chance to update my JetPack to 5.1.2, and redo the process. The results are identical with the previously posted plots.

[AastaLLL]

Hi,

Thanks for the testing.
We will check this issue with our internal team and get back to you.

[AastaLLL]

Hi,

We don’t guarantee that our result will be aligned with the OpenCV’s output.

However, there are several parameters in the Pyramidal LK Optical Flow.
Have you tried to tune them?

For example, how many pyramid levels are you using for VPI? Is it the same as OpenCV?
Thanks.

[Appletree]

Yes, I used the same parameters as that in opencv PryLK algorithm. Do you have an internal evaluation document to share? I am just curious how do you engineers evaluate it? Thanks a lot!

[AastaLLL]

Hi,

We cannot share internal evaluations and reports here.
But we can check if any info for you with our internal team.

Thanks.