VPI StereoDisparity hangs after first frame with OFA | PVA | VIC backend on JetPack 7.0 / VPI 4.0.0

Robotics & Edge Computing Jetson Systems Jetson Thor
1

Test Environment

  • Platform: Jetson AGX Thor Dev-Kit
  • JetPack version: 7.0
  • VPI version: 4.0.0
  • Camera: Stereo Camera
  • Camera resolution: 2560 x 1984
  • VPI StereoDisparity backends tested:vpi.Backend.OFA | vpi.Backend.PVA | vpi.Backend.VIC\

Hello,

I am currently testing VPI StereoDisparity with a stereo camera, and I would like to ask about an issue I observed when using the `OFA | PVA | VIC` backend.

I attached the pipeline structure image for reference.

image
image1156×427 84.4 KB

========================================

Issue

With the `OFA | PVA | VIC` backend, the first frame is processed successfully.

The disparity image and confidence image can be displayed with OpenCV `imshow`.

However, after the first frame is displayed, the application does not proceed to the next frame. It appears to hang or freeze in the live loop.

As a cross test, I changed the backend to OFA only:

BACKEND = vpi.Backend.OFA

With the OFA-only backend, the live loop continues normally. However, confidence map output is not available in this mode.

Therefore, I suspect that the issue may be related to the OFA | PVA | VIC backend path, especially the PVA / confidence map side.

===============================================

Related Code

BACKEND = vpi.Backend.OFA | vpi.Backend.PVA | vpi.Backend.VIC
conf_map = None  # Also tested with confidence map enabled.
stream = vpi.Stream()

disp_bl = vpi.stereodisp(
    left_bl,
    right_bl,
    backend=BACKEND,
    stream=stream,
    out_confmap=conf_map,
    conftype=vpi.ConfidenceType.RELATIVE,
    window=WINDOW,
    maxdisp=MAX_DISP,
    confthreshold=CONF_THRESH,
    quality=QUALITY,
    p1=P1,
    p2=P2,
    p2alpha=P2ALPHA,
    numpasses=NUMPASSES,
)
stream.sync()

===================================

Question

Could you please help confirm whether I am using the VPI StereoDisparity API incorrectly, or whether this behavior is a known issue in VPI 4.0?

If this is related to a known VPI 4.0 issue, could you also let me know if there is any recommended workaround or planned fix?

I attached the related test code for reference.

Thank you.

5

Dear @MDS_Su ,
I see this is known issue listed in VPI 4.0 release notes and got fixed in VPI 4.1.3(VPI - Vision Programming Interface: Release Notes)
Can you please check with Jetpack 7.2

6

Hello,

Thank you for sharing the information.

I have reviewed the related details. Currently, we do not have a camera driver for JetPack 7.2, so it is difficult for us to proceed with the related testing at this moment.

We will perform the test on JetPack 7.2 once the camera driver becomes available.

Thank you for your support.