Nvarguscamerasrc issues, related to OpenCV

From your experiment any reason need to use nvoverlaysink?

There’s no reason, but the symptoms are the same as those shown by OpenCV, so I’m thinking that those two are related in implementation, by having some conversion from EGL to NVMM that’s not working. Also they don’t seem to allow the stream to be freed.

1 Like

Sorry for the late response, have you managed to get issue resolved or still need the support? Thanks

I haven’t resolved it, and i’m currently trying to get Argus to work. I will check again with a clean install in a few days.

1 Like

Any result can be shared? Thanks

I don’t have physical access to a Jetson right now, but my code with Argus can be seen here: Argus : STATUS_DISCONNECTED when trying to acquire frame - #5 by ShaneCCC
It’s not working.
I’ll make a tutorial for eveything (CMake setup, includes and usage) once I get something working.

I have done a fresh install on my jetson : I have compiled opencv 4.5 myself, same as before. When starting a capture with nvarguscamerasrc, this happens :

robotronik@damnson:~/jetsonMV/build$ ./Robotronikaruco -c
/dev/gpiochip0 at 5;� is attached to
/dev/tegra_camera_ctrl
vi-output, imx219 7-0010 at platform:54080000.vi:0 is attached to
/dev/video0
[ERROR:0@0.477] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration//dev/gpiochip0.yaml' in read mode
Failed to read camera parameters for ../calibration//dev/gpiochip0
[ERROR:0@0.477] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration/vi-output, imx219 7-0010.yaml' in read mode
Failed to read camera parameters for ../calibration/vi-output, imx219 7-0010
Starting calibration of camera index0
Opening device at "nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink" with API id 1800
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0@1.857] global /home/robotronik/opencv/modules/videoio/src/cap_gstreamer.cpp (1412) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
CONSUMER: Done Success
(Argus) Error InvalidState: Argus client is exiting with 2 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 357)
WARNING Argus: 6 client objects still exist during shutdown:
	548155724480 (0x7f24000ee0)
	548160724704 (0x7f24000c70)
	548160725056 (0x7f24000fe0)
	548160730736 (0x7f240010f0)
	548160732032 (0x7f24002fe0)
	548160740120 (0x7f240030f8)
robotronik@damnson:~/jetsonMV/build$ ./Robotronikaruco -c
/dev/gpiochip0 at �p� is attached to
/dev/tegra_camera_ctrl
vi-output, imx219 7-0010 at platform:54080000.vi:0 is attached to
/dev/video0
[ERROR:0@0.380] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration//dev/gpiochip0.yaml' in read mode
Failed to read camera parameters for ../calibration//dev/gpiochip0
[ERROR:0@0.380] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration/vi-output, imx219 7-0010.yaml' in read mode
Failed to read camera parameters for ../calibration/vi-output, imx219 7-0010
Starting calibration of camera index0
Opening device at "nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink" with API id 1800
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: ERROR OCCURRED
[ WARN:0@2.552] global /home/robotronik/opencv/modules/videoio/src/cap_gstreamer.cpp (2409) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module nvarguscamerasrc0 reported: CANCELLED
GST_ARGUS: Cleaning up
^C
robotronik@damnson:~/jetsonMV/build$ systemctl restart nvargus-daemon.service 
robotronik@damnson:~/jetsonMV/build$ ./Robotronikaruco -c
/dev/gpiochip0 at u_� is attached to
/dev/tegra_camera_ctrl
vi-output, imx219 7-0010 at platform:54080000.vi:0 is attached to
/dev/video0
[ERROR:0@0.377] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration//dev/gpiochip0.yaml' in read mode
Failed to read camera parameters for ../calibration//dev/gpiochip0
[ERROR:0@0.377] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration/vi-output, imx219 7-0010.yaml' in read mode
Failed to read camera parameters for ../calibration/vi-output, imx219 7-0010
Starting calibration of camera index0
Opening device at "nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink" with API id 1800
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: ERROR OCCURRED
[ WARN:0@2.650] global /home/robotronik/opencv/modules/videoio/src/cap_gstreamer.cpp (2409) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module nvarguscamerasrc0 reported: CANCELLED
GST_ARGUS: Cleaning up
^C
robotronik@damnson:~/jetsonMV/build$ 

As you can see, and as I said before, it only runs once, then argus doesn’t recover. Restarting the daemon doesn’t help, as can be seen, and restarting the jetson doesn’t either. There seems to be something in OpenCV that doesn’t correctly call the gstreamer destructors; and that argus doesn’t like…
Fixing this would fix everything for me.

I hope the above log can help.

Regards.

Seems the first error is related to some calibration file failed to be found at that path.

Don’t worry about that, it’s not related.

Long answer is that the calibration file is for undistorting the image, but the mode I have started my program in doesn’t undistort, so disregard the “Failed to read camera parameters” and the “Can’t open file”, as these are both the same error.

Log order after any launch of my program is basically :
-Small dump of v4l2 detect data
-Warnings for missing calibration files
-Start the cameras, using gstreamer via VideoCapture.

During the log that I sent earlier, the program is launched 3 times : first time it works, but leave two threads of argus and 6 objects, that subsequent launches can’t get rid of.

For anyone willing to try, here is some code to reproduce the issue :

#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <iostream>

using namespace std;
using namespace cv;

int main(int argc, char** argv )
{
	cuda::setDevice(0);
	ocl::setUseOpenCL(true);

	VideoCapture* cap = new VideoCapture();
	cout << "Created video capture instance" << endl;
	cap->open("nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink", CAP_GSTREAMER);
	cout << "Opened camera" << endl;
	int keypressed = 0;
	do
	{
		UMat image;
		cap->read(image);
		imshow("ArgusCamera", image);
		keypressed = waitKey(1);
	}
	while (keypressed != 27);
	return EXIT_SUCCESS;
}

Yes the camera isn’t getting released, but that doesn’t cause any issues for other cameras (USB and such). SIGTERM aren’t usually handled, so i believe that this use case is realistic, where users don’t clean up afterwards.

Did you verify by gst-launch-1.0 command line to open/close to confirm without problem?

Yes, it’s exactly the same as for the first messages in this conversation.

Are there any plugins registered in the gstreamer plugin blacklist?
show the blacklist:

gst-inspect-1.0 -b

Thanks for you reply.
Could you remind me the gst-launch-1.0 get the same failed?

Yes, I have the same results for the gst-launch.
Also the blacklist doesn’t have any plugins inside.

OK, that could be the sensor problem.
I would suggest consult to arducam for this issue.

You may also experiment with this test code. You’ll see it has a signal handler for termination, but it may be omitted if you add property drop=1 to appsink.

OK, that could be the sensor problem.
I would suggest consult to arducam for this issue.

It’s clearly a software issue though, as it works once, then complains that the objects haven’t been released, then doesn’t work. Other ways to access the camera work…
Is there a file somewhere that it’s writing to? Like a lock file? That would explain the persistence of the issue, and deleting it could partially fix the issue for me between reboots.

Please verify the sensor driver by v4l2-ctl like below command. Press CTRL + C to terminal it and run again much times to confirm it.

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap

I get 21.2 fps, consistently, but no image is displayed.