GStreamer Error depending on camera resolution mode IMX477

Hello,

I’m a beginner in the Jetson Nano Universe trying to set up a camera monitoring system. Therefore I have purchased the Arducam IMX477 Camera for Jetson. I am using the example python-code from the Arducam website to create a GStreamer pipeline which I can then use to capture the video with opencv.

def gstreamer_pipeline(
    capture_width=4320,
    capture_height=3040,
    display_width=1920,
    display_height=1080,
    framerate=30,
    flip_method=0,
):
    return (
        "nvarguscamerasrc ! "
        "video/x-raw(memory:NVMM), "
        "width=(int)%d, height=(int)%d, "
        "format=(string)NV12, framerate=(fraction)%d/1 ! "
        "nvvidconv flip-method=%d ! "
        "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
        "videoconvert ! "
        "video/x-raw, format=(string)BGR ! appsink max-buffers=1 drop=true"
        % (
            capture_width,
            capture_height,
            framerate,
            flip_method,
            display_width,
            display_height,
        )
    )

The camera supports the following different resolution/framerate modes:

GST_ARGUS: Available Sensor modes :
GST_ARGUS: 4032 x 3040 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 2592 x 1944 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 2560 x 1440 FR = 40.000000 fps Duration = 25000000 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

Using the short HDMI cable that was delivered with the camera I was able to display the video frames with all of the sensor modes. However, I will be using a significantly longer cable (5 meters) for my actual project. Unfortunately I cannot use the modes 4032x3040 and 2560x1440 with the longer cable which is annoying since I would like to use the highest resolution setting. Instead I’m getting the following error:

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 3
Output Stream W = 2560 H = 1440
seconds to Run = 0
Frame Rate = 40.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module nvarguscamerasrc0 reported: CANCELLED
(python3:16464): GStreamer-CRITICAL **: 15:42:36.346: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
GST_ARGUS: Cleaning up
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 266)
(Argus) Error EndOfFile: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 340)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 357)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 368)
(Argus) Error EndOfFile: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error EndOfFile: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 87)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 87)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 87)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 87)
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (886) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 87

First I thought this could be an issue with latency/buffer, but that wouldn’t explain why 2592 x 1944 works while 2560x1440 doesn’t.
Has someone already had a similar issue or knows what solution for the problem could be? I would be grateful for any hint.

Cheers!

2592x1944 mode is slowest fps (30). This might be your limitiation. You may also try other modes with 30 fps only:

gst-launch-1.0 -v nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=2560,height=1440,framerate=30/1' ! nvvidconv ! xvimagesink

Could you check with v4l2-ctl to confirm this mode is working on your device.

Looks like this sensor only have 4032x3040@30 and 1920x1080@60,

nvidia@nvidia-desktop:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'RG10'
        Name        : 10-bit Bayer RGRG/GBGB
                Size: Discrete 4032x3040
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.017s (60.000 fps)

1 Like

Thanks for the quick answers!
@Honey_Patouceul : Somehow I cannot set the framerate manually; for your code example it just automatically detects the 2560x1440 mode and sets the framerate to the corresponding 40 fps …

@ShaneCCC v4l2-ctl confirms all four modes:

v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 4032x3040
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 2592x1944
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 2560x1440
			Interval: Discrete 0.025s (40.000 fps)

The mode I am most interested in is 4032x3040 30 fps which is definitely working with the short HDMI cable. But it does not with the longer cable. Also, lagging is quite extreme in general, even with the working modes. Could the issue maybe be the buffer since the code seems to fail because of:

nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…