Nvarguscamerasrc issues, related to OpenCV

moddingear · April 27, 2022, 7:22pm

Hello,
I’m having trouble getting nvarguscamerasrc to run reliably: it seems to have a mind of it’s own.
These are the manipulations I’ve done, in chronological order : (I’m ommiting the formats as these aren’t necessary to have a pipeline working and would bloat this tremendously, and don’t mind the typos if there are some, i’m writing this from head)
gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! videoconvert ! nvoverlaysink -e : Worked until I launched once the following command in OpenCV’ VideoCapture.
nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink which worked only once.
Since, any commands using nvoverlaysink or appsink with nvarguscamerasrc as a source no longer work.

What is frustrating is that the following pipelines still work:
gst-launch-1.0 nvarguscamerasrc ! nvegltransform ! nveglglessink
gst-launch-1.0 nvarguscamerasrc ! xvimagesink
gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! videoconvert ! gtksink

And I also have verified that GStreamer works with my OpenCV installation using a v4l2 pipeline:
v4l2src device=/dev/video0 ! nvvidconv ! videoconvert ! appsink

The VPI examples (Notably VPI Remap) also manage to make the CSI camera work.

Here’s my setup :
-Nvidia Jetson Nano 4GB
-IMX219 with fisheye lens
-JetPack 4.6, L4T 32.7.1
-Opencv 4.5.5 compiled with Cuda, VTK and GStreamer on the Jetson itself.

I’m just looking for a fast way to get it working with OpenCV, either using libargus or a gstreamer pipeline. The thing is that I haven’t found any good tutorial for libargus.
I’ve attached the terminal output for the nvoverlaysink pipeline and the nveglglessink pipeline
eglpipeline.txt (1.8 KB)
nvpipeline.txt (1.8 KB)

Thanks for reading !

ShaneCCC · April 28, 2022, 7:09am

The nvoverlaysink was not support from r32.6.1

moddingear · April 28, 2022, 7:32am

I didn’t know that…
That’s weird because nvoverlaysink still works with v4l2src.

ShaneCCC · April 28, 2022, 7:59am

Sorry, looks like J4.x still support nvoverlaysink but J5.x
Could you try below sample.

moddingear · April 28, 2022, 9:33am

I don’t use python, but I’ve integrated the gstreamer pipeline into my own code : It basically prints the v4l2-ctl device of lists, then prints the pipeline it’s going to use with API ID, then tries to start the camera. After that it tries to read the frames, and after 10 read failures it aborts and shuts down.
The following text file is the console output of running the script :
opencvpipeline.txt (4.2 KB)

Short answer: It didn’t work, and it has roughly the same errors as the nvoverlaysink pipeline.

Also here is my OpenCV build info, just in case:
buildinfo.txt (7.2 KB)

ShaneCCC · April 28, 2022, 11:02am

Add this ‘video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12’ to your pipeline like below to try.

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12' ! nvvidconv ! videoconvert ! nvoverlaysink -e

moddingear · April 28, 2022, 12:34pm

I’ve tried it, still no luck :/
Here’s the console output below: It just waited doing nothing so I had to terminate it
consoleoutput.txt (1.9 KB)

DaneLLL · April 28, 2022, 10:43pm

Hi
For imx219, do you use Raspberry Pi camera v2? This is supported by default. If you use imx219 from other vendor, the driver may not be ready and you would need to contact the vendor for support.

moddingear · April 29, 2022, 6:20am

I use the arducam version.

What’s weird is that there are pipelines that work fully, and the Nvidia provided examples work too…

ShaneCCC · April 29, 2022, 7:20am

Could you check if nvgstcapture-1.0 able see the preview screen?

moddingear · April 29, 2022, 8:11am

nvgstcapture-1.0 can’t start a video or capture an image.
Here is the console output :
nvgstout (5.2 KB)

Is there something I can run to verify the files, or something to reinstall possibly ?
What about possible alternatives ?

moddingear · May 3, 2022, 7:57pm

Hello, can you point me towards some possible resolution, either using gstreamer, libargus, or something else I don’t know ?

ShaneCCC · May 5, 2022, 12:56am

From your experiment any reason need to use nvoverlaysink?

moddingear · May 5, 2022, 7:58am

There’s no reason, but the symptoms are the same as those shown by OpenCV, so I’m thinking that those two are related in implementation, by having some conversion from EGL to NVMM that’s not working. Also they don’t seem to allow the stream to be freed.

kayccc · May 18, 2022, 1:17am

Sorry for the late response, have you managed to get issue resolved or still need the support? Thanks

moddingear · May 18, 2022, 8:44pm

I haven’t resolved it, and i’m currently trying to get Argus to work. I will check again with a clean install in a few days.

kayccc · May 25, 2022, 5:26am

Any result can be shared? Thanks

moddingear · May 25, 2022, 4:27pm

I don’t have physical access to a Jetson right now, but my code with Argus can be seen here: Argus : STATUS_DISCONNECTED when trying to acquire frame - #5 by ShaneCCC
It’s not working.
I’ll make a tutorial for eveything (CMake setup, includes and usage) once I get something working.

moddingear · September 12, 2022, 8:55pm

I have done a fresh install on my jetson : I have compiled opencv 4.5 myself, same as before. When starting a capture with nvarguscamerasrc, this happens :

robotronik@damnson:~/jetsonMV/build$ ./Robotronikaruco -c
/dev/gpiochip0 at 5;� is attached to
/dev/tegra_camera_ctrl
vi-output, imx219 7-0010 at platform:54080000.vi:0 is attached to
/dev/video0
[ERROR:0@0.477] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration//dev/gpiochip0.yaml' in read mode
Failed to read camera parameters for ../calibration//dev/gpiochip0
[ERROR:0@0.477] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration/vi-output, imx219 7-0010.yaml' in read mode
Failed to read camera parameters for ../calibration/vi-output, imx219 7-0010
Starting calibration of camera index0
Opening device at "nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink" with API id 1800
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0@1.857] global /home/robotronik/opencv/modules/videoio/src/cap_gstreamer.cpp (1412) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
CONSUMER: Done Success
(Argus) Error InvalidState: Argus client is exiting with 2 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 357)
WARNING Argus: 6 client objects still exist during shutdown:
	548155724480 (0x7f24000ee0)
	548160724704 (0x7f24000c70)
	548160725056 (0x7f24000fe0)
	548160730736 (0x7f240010f0)
	548160732032 (0x7f24002fe0)
	548160740120 (0x7f240030f8)
robotronik@damnson:~/jetsonMV/build$ ./Robotronikaruco -c
/dev/gpiochip0 at �p� is attached to
/dev/tegra_camera_ctrl
vi-output, imx219 7-0010 at platform:54080000.vi:0 is attached to
/dev/video0
[ERROR:0@0.380] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration//dev/gpiochip0.yaml' in read mode
Failed to read camera parameters for ../calibration//dev/gpiochip0
[ERROR:0@0.380] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration/vi-output, imx219 7-0010.yaml' in read mode
Failed to read camera parameters for ../calibration/vi-output, imx219 7-0010
Starting calibration of camera index0
Opening device at "nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink" with API id 1800
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: ERROR OCCURRED
[ WARN:0@2.552] global /home/robotronik/opencv/modules/videoio/src/cap_gstreamer.cpp (2409) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module nvarguscamerasrc0 reported: CANCELLED
GST_ARGUS: Cleaning up
^C
robotronik@damnson:~/jetsonMV/build$ systemctl restart nvargus-daemon.service 
robotronik@damnson:~/jetsonMV/build$ ./Robotronikaruco -c
/dev/gpiochip0 at u_� is attached to
/dev/tegra_camera_ctrl
vi-output, imx219 7-0010 at platform:54080000.vi:0 is attached to
/dev/video0
[ERROR:0@0.377] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration//dev/gpiochip0.yaml' in read mode
Failed to read camera parameters for ../calibration//dev/gpiochip0
[ERROR:0@0.377] global /home/robotronik/opencv/modules/core/src/persistence.cpp (505) open Can't open file: '../calibration/vi-output, imx219 7-0010.yaml' in read mode
Failed to read camera parameters for ../calibration/vi-output, imx219 7-0010
Starting calibration of camera index0
Opening device at "nvarguscamerasrc ! nvvidconv ! videoconvert ! appsink" with API id 1800
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: ERROR OCCURRED
[ WARN:0@2.650] global /home/robotronik/opencv/modules/videoio/src/cap_gstreamer.cpp (2409) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module nvarguscamerasrc0 reported: CANCELLED
GST_ARGUS: Cleaning up
^C
robotronik@damnson:~/jetsonMV/build$

As you can see, and as I said before, it only runs once, then argus doesn’t recover. Restarting the daemon doesn’t help, as can be seen, and restarting the jetson doesn’t either. There seems to be something in OpenCV that doesn’t correctly call the gstreamer destructors; and that argus doesn’t like…
Fixing this would fix everything for me.

I hope the above log can help.

Regards.

Honey_Patouceul · September 12, 2022, 9:03pm

Seems the first error is related to some calibration file failed to be found at that path.