Stereo camera opencv segmentation fault during exit

I have a simple demo to show the disparity map, using a Jetson Nano 4GB B01:

    import numpy as np
    import cv2

    left_camera = cv2.VideoCapture("nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1024, height=(int)768,format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")
    right_camera = cv2.VideoCapture("nvarguscamerasrc sensor-id=1 ! video/x-raw(memory:NVMM), width=(int)1024, height=(int)768,format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")
    stereo = cv2.StereoBM_create(numDisparities=80, blockSize=25)
    while(True):
        ret, left_frame = left_camera.read()
        ret, right_frame = right_camera.read()
        left_frame = cv2.cvtColor(left_frame, cv2.COLOR_BGR2GRAY)
        right_frame = cv2.cvtColor(right_frame, cv2.COLOR_BGR2GRAY)
        disparity = stereo.compute(left_frame, right_frame)
        disparity = cv2.normalize(disparity, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)
        cv2.imshow('disparity', disparity)
        cv2.imshow('left', left_frame)
        cv2.imshow('right', right_frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    right_camera.release()
    left_camera.release()
    cv2.destroyAllWindows()

It works until I exit (Keyboard q), when it gives a segmentation fault. Here is the end of the log + stack trace:

    GST_ARGUS: Cleaning up
    [Thread 0x7f90ff91f0 (LWP 22846) exited]
    [Thread 0x7f737fe1f0 (LWP 22848) exited]
    
    Thread 14 "argus_thread" received signal SIGSEGV, Segmentation fault.
    [Switching to Thread 0x7f73fff1f0 (LWP 22847)]
    0x0000007faeb34508 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
    (gdb) bt
    #0  0x0000007faeb34508 in  () at /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
    #1  0x0000007faebb64a0 in  () at /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
    #2  0x0000007fb2e887f0 in  () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvarguscamerasrc.so
    #3  0x0000007fb2e883d4 in  () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvarguscamerasrc.so
    #4  0x0000007fb2e88570 in  () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvarguscamerasrc.so
    #5  0x0000007fb7e17088 in start_thread (arg=0x7f917f97bf) at pthread_create.c:463
    #6  0x0000007fb7f0bffc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

I am using the following Waveshare stereo camera: IMX219-83 Stereo Camera - Waveshare Wiki

I am using JetPack SDK version 4.5.

    cat /etc/nv_tegra_release 
    # R32 (release), REVISION: 5.0, GCID: 25531747, BOARD: t210ref, EABI: aarch64, DATE: Fri Jan 15 22:55:35 UTC 2021

Hi,
For more information, do you observe the issue in launching single camera? Would like to know if it is specific to two-camera case.

Hi DaneLLL,
I created another testcase with a single camera and the segmentation fault continues to happen.
Here is my modified testcase:

    import numpy as np
    import cv2
    
    left_camera = cv2.VideoCapture("nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")

    stereo = cv2.StereoBM_create(numDisparities=80, blockSize=25)

    while(True):
        ret, left_frame = left_camera.read()
        left_frame = cv2.cvtColor(left_frame, cv2.COLOR_BGR2GRAY)
        disparity = stereo.compute(left_frame, left_frame)
        disparity = cv2.normalize(disparity, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)
        cv2.imshow('disparity', disparity)
        cv2.imshow('left', left_frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    left_camera.release()
    cv2.destroyAllWindows()

I could only get it to work without a segmentation fault when I commented out the StereoBM and the disparity map calculation code (which is not a possible solution).

Hi,
Please check if you can run this:
OpenCV Video Capture with GStreamer doesn't work on ROS-melodic - #3 by DaneLLL
See if it works in showing video preview.

Hi DaneLLL,
It works showing the video preview with the code in the link you shared.
Please notice that the code I shared also shows the video preview and the disparity map. The segmentation fault happens when exiting the program.

You may monitor argus messages in syslog with:

tail -f /var/log/syslog

and report here.
I’d suspect a reference to frame from stereoBM not being released. Not sure how its destructor is called from python.
Can you add a cv::waitKey(at_least_one_frame_period) before capture release and check if this improves ?

Thank you Honey_Patouceul.
I added cv2.waitKey(1000) before left_camera.release() came and still got a segmentation fault.
Here is the output from syslog:

    Feb 22 12:08:50 gss-nano systemd[1]: Stopped Argus daemon.
    Feb 22 12:08:54 gss-nano systemd[1]: Started Argus daemon.
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: === NVIDIA Libargus Camera Service (0.97.3)=== Listening for connections...=== python3[19104]: Connection established (7F9B7E01D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: ---- imager: No override file found. ----
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: LSC: LSC surface is not based on full res!
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: ---- imager: No override file found. ----
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: LSC: LSC surface is not based on full res!
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: === python3[19104]: CameraProvider initialized (0x7f94b72110)LSC: LSC surface is not based on full res!
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: NvIspAfConfigParamsSanityCheck: Error: positionWorkingHigh is not larger than positionWorkingLow positionWorkingHigh = 0, positionWorkingLow = 0
    Feb 22 12:09:06 gss-nano nvargus-daemon[19058]: message repeated 2 times: [ NvIspAfConfigParamsSanityCheck: Error: positionWorkingHigh is not larger than positionWorkingLow positionWorkingHigh = 0, positionWorkingLow = 0]
    Feb 22 12:09:07 gss-nano nvargus-daemon[19058]: LSC: LSC surface is not based on full res!
    Feb 22 12:09:07 gss-nano nvargus-daemon[19058]: NvIspAfConfigParamsSanityCheck: Error: positionWorkingHigh is not larger than positionWorkingLow positionWorkingHigh = 0, positionWorkingLow = 0
    Feb 22 12:09:07 gss-nano nvargus-daemon[19058]: message repeated 2 times: [ NvIspAfConfigParamsSanityCheck: Error: positionWorkingHigh is not larger than positionWorkingLow positionWorkingHigh = 0, positionWorkingLow = 0]
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: === python3[19104]: Connection closed (7F9B7E01D0)=== python3[19104]: WARNING: CameraProvider was not destroyed before client connection terminated.=== python3[19104]:          The client may have abnormally terminated. Destroying CameraProvider...=== python3[19104]: CameraProvider destroyed (0x7f94b72110)=== python3[19104]: WARNING: Cleaning up 2 outstanding requests...=== python3[19104]: WARNING: Cleaning up 2 outstanding stream settings...=== python3[19104]: WARNING: Cleaning up 1 outstanding queues...=== python3[19104]: WARNING: Cleaning up 2 outstanding sessions...SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function returnFrame(), line 372)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function getBuffer(), line 434)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/components/CaptureContainerImpl.cpp, function assignAllBuffersFromStream(), line 230)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/components/stages/CCDataSetupStage.cpp, function doHandleRequest(), line 68)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 992)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function returnFrame(), line 372)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function getBuffer(), line 434)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/components/CaptureContainerImpl.cpp, function assignAllBuffersFromStream(), line 230)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/components/stages/CCDataSetupStage.cpp, function doHandleRequest(), line 68)
    Feb 22 12:09:12 gss-nano nvargus-daemon[19058]: SCF: Error InvalidState:  (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
    Feb 22 12:09:17 gss-nano nvargus-daemon[19058]: waitForIdleLocked remaining request 200
    Feb 22 12:09:17 gss-nano nvargus-daemon[19058]: waitForIdleLocked remaining request 199
    Feb 22 12:09:17 gss-nano nvargus-daemon[19058]: SCF: Error Timeout: waitForIdle() timed out (in src/api/Session.cpp, function waitForIdleLocked(), line 922)
    Feb 22 12:09:17 gss-nano nvargus-daemon[19058]: (Argus) Error Timeout:  (propagating from src/api/CaptureSessionImpl.cpp, function destroy(), line 166)
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: waitForIdleLocked remaining request 200
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: waitForIdleLocked remaining request 199
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: SCF: Error Timeout: waitForIdle() timed out (in src/api/Session.cpp, function waitForIdleLocked(), line 922)
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: SCF: Error Timeout:  (propagating from src/api/Session.cpp, function abortCaptures(), line 888)
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: SCF: Error Timeout:  (propagating from src/api/Session.cpp, function shutdown(), line 401)
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: PowerServiceCore:handleRequests: timePassed = 5034
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: SCF: Error Timeout:  (propagating from src/api/Session.cpp, function shutdown(), line 501)
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: SCF: Error Timeout:  (propagating from src/api/CameraDriver.cpp, function deleteSession(), line 627)
    Feb 22 12:09:22 gss-nano nvargus-daemon[19058]: (Argus) Error Timeout:  (propagating from src/api/CaptureSessionImpl.cpp, function destroy(), line 191)

Hi,
Do you hit segment fault with this sample:
OpenCV Video Capture with GStreamer doesn't work on ROS-melodic - #3 by DaneLLL
Or only happens when you call StereoBM functions?

Hi DaneLLL,
It only happens when I call StereoBM.

I mentioned before that the seg fault happened when exiting the program. It actually happens during the call to release() method from cv2.VideoCapture.

I’ve just checked from a XavierNX running R32.5 with a opencv-4.4 build.
It works fine in my case, but I need to set gstreamer backend for VideoCapture:

left_camera = cv2.VideoCapture("nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink", cv2.CAP_GSTREAMER)

otherwise it segfaults. However if it works and only crashes on release it should be a different issue.

Which version of opencv is installed in python ?

print(cv2.__version__)

I am using OpenCV version 4.1.1.
I added the cv2.CAP_GSTREAMER argument, but still getting a segmentation fault.

Hi,
Please check if the issue happens with videotestsrc:

videotestsrc is-live=1 ! video/x-raw, width=(int)640, height=(int)480,format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM),format=NV12 ! nvvidconv  ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink

If it is confirmed to be an issue in nvarguscamerasrc, please download the source code to add prints for further debugging. We don’t test the OpenCV function and it might be an race condition.

The source code is in
https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/sources/T210/public_sources.tbz2