Problems showing Allied Vision camera with Gstreamer / OpenCV

Aleks-CST · June 29, 2022, 10:16am

JetPack 4.6 (rev. 3)
Allied Vision 1800 C-319c camera
Camera is connected to the Jetson via “ADAPTER CSI-2 JETSON TX2 / XAVIER” board.

Using:

“gst-launch-1.0 nvarguscamerasrc ! nvoverlaysink”

I get the following error:

Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:725 No cameras available

ls /dev/video* shows:
/dev/video0

and lsmod shows the driver is installed:
avt_csi2 92085 0

v4l2-ctl -d /dev/video0 --list-formats-ext shows:

ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘TP31’
Name : 0x31 MIPI DATATYPE
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'GREY'
Name        : 8-bit Greyscale
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 2
Type        : Video Capture
Pixel Format: 'RGGB'
Name        : 8-bit Bayer RGRG/GBGB
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 3
Type        : Video Capture
Pixel Format: 'JXY0'
Name        : 10-bit/16-bit Greyscale
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 4
Type        : Video Capture
Pixel Format: 'JXR0'
Name        : 10-bit/16-bit Bayer RGRG/GBGB
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 5
Type        : Video Capture
Pixel Format: 'JXY2'
Name        : 12-bit/16-bit Greyscale
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 6
Type        : Video Capture
Pixel Format: 'JXR2'
Name        : 12-bit/16-bit Bayer RGRG/GBGB
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 7
Type        : Video Capture
Pixel Format: 'BX24'
Name        : 32-bit XRGB 8-8-8-8
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 8
Type        : Video Capture
Pixel Format: 'XR24'
Name        : 32-bit BGRX 8-8-8-8
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

Index       : 9
Type        : Video Capture
Pixel Format: 'VYUY'
Name        : VYUY 4:2:2
	Size: Discrete 2064x1544
		Interval: Discrete 0.019s (51.968 fps)

I tried using this longer command with edited resolution:

gst-launch-1.0 nvarguscamerasrc sensor_id=0 ! ‘video/x-raw(memory:NVMM),width=2064, height=1544, framerate=51/1’ ! nvvidconv flip-method=0 ! ‘video/x-raw,width=960, height=540’ ! nvvidconv ! nvegltransform ! nveglglessink -e

Output:

Setting pipeline to PAUSED …

Using winsys: x11
Pipeline is live and does not need PREROLL …
Got context from element ‘eglglessink0’: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING …
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:725 No cameras available
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.049963076
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

My goal is to show the camera output using OpenCV so I’ve been looking at the JetsonHacks python script “simple_camera.py” but no luck.

Any help appriciated!

JerryChang · June 30, 2022, 3:24am

hello Aleks-CST,

please narrow down the issue by Applications Using V4L2 IOCTL Directly to verify basic functionality.

Aleks-CST · June 30, 2022, 7:22am

Hi Jerry, thanks for your reply.

Using:
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --stream-mmap --stream-count=1 -d /dev/video0 --stream-to=ov5693.raw

I get a 0 byte file opening up with a text editor.

Changing the resolutions and format (I picked VYUY from the v4l2-ctl --list-formats-ext list), using:
v4l2-ctl --set-fmt-video=width=2064,height=1544,pixelformat=VYUY–stream-mmap --stream-count=1 -d /dev/video0 --stream-to=ov5693.raw

I get a 12.7 MB file but cannot open it with any media player.

Hope this makes sense

Regards
Aleks

Update:

Using:
v4l2-ctl --set-fmt-video=width=2064,height=1544,pixelformat=VYUY–stream-mmap --stream-count=1000 -d /dev/video0

gives the following output:

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 52.00 fps

JerryChang · June 30, 2022, 8:01am

hello Aleks-CST,

is this expect sensor outputs?
if yes, you cannot use nvarguscamerasrc plugin since it doesn’t support with YUV formats.

Aleks-CST · June 30, 2022, 8:27am

VYUY is a format I used for a different project with the same camera. I can choose JXR2 as well. I’m not sure how to set up a pipeline to use in VideoCapture using OpenCV.

Edit:
I’ve tried different “pixelformat=***” and no matter what I put, I get:

VIDIOC_S_FMT: failed: Invalid argument
<<<<<<<<<<<<<<<<<<<<<<<<<<<< 53.45 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<< 53.50 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<< 53.48 fps
…

JerryChang · June 30, 2022, 8:34am

hello Aleks-CST,

may I know what’s the sensor mode/formats reported by v4l utility.
for example, $ v4l2-ctl -d /dev/video0 --list-formats-ext

Aleks-CST · June 30, 2022, 8:37am

ioctl: VIDIOC_ENUM_FMT

Index : 0
Type : Video Capture
Pixel Format: ‘TP31’
Name : 0x31 MIPI DATATYPE
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 1
Type : Video Capture
Pixel Format: ‘GREY’
Name : 8-bit Greyscale
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 2
Type : Video Capture
Pixel Format: ‘RGGB’
Name : 8-bit Bayer RGRG/GBGB
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 3
Type : Video Capture
Pixel Format: ‘JXY0’
Name : 10-bit/16-bit Greyscale
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 4
Type : Video Capture
Pixel Format: ‘JXR0’
Name : 10-bit/16-bit Bayer RGRG/GBGB
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 5
Type : Video Capture
Pixel Format: ‘JXY2’
Name : 12-bit/16-bit Greyscale
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 6
Type : Video Capture
Pixel Format: ‘JXR2’
Name : 12-bit/16-bit Bayer RGRG/GBGB
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 7
Type : Video Capture
Pixel Format: ‘BX24’
Name : 32-bit XRGB 8-8-8-8
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 8
Type : Video Capture
Pixel Format: ‘XR24’
Name : 32-bit BGRX 8-8-8-8
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Index : 9
Type : Video Capture
Pixel Format: ‘VYUY’
Name : VYUY 4:2:2
Size: Discrete 2064x1544
Interval: Discrete 0.019s (51.968 fps)

Aleks-CST · June 30, 2022, 8:49am

The above output was copied from my original post where I provided the same info.

Printing the formats again today, I notice the Size and fps have changed… Not sure how this happened…

Output is now:

Size: Discrete 2048x1544
Interval: Discrete 0.019s (53.432 fps)

for all formats. I don’t really understand how this changed from yesterday.

-Aleks

Update:
The above values seem to change when I try to run:
v4l2-ctl --set-fmt-video=width=2064,height=1544,pixelformat=VYUY–stream-mmap --stream-count=1000 -d /dev/video0

That somehow changes the output of v4l2-ctl --list-formats-ext to
Size: Discrete 2048 x1544
Interval: Discrete 0.019s (53.432 fps )

It’s a bit confusing but not the biggest issue right now. I’m still not sure how I’d set up a pipeline that could work with OpenCV and VideoCapture.

DaneLLL · June 30, 2022, 1:04pm

Hi,
VYUY is not supported in v4l2src You may try GREY. Please try to capture frame data in GREY8 through v4l2-ctl. If the frame data looks good, please try

gst-launch-1.0 v4l2src ! video/x-raw, format=GRAY8, width=2064, height=1544 ! videoconvert ! video/x-raw, format=I420 ! xvimagesink

And check if you can see camera preview.

Aleks-CST · June 30, 2022, 2:15pm

Hi Dane, thanks for your reply!

A few formats that seem to work:
v4l2-ctl --set-fmt-video=width=2064,height=1544,pixelformat=BX24–stream-mmap -d /dev/video0
v4l2-ctl --set-fmt-video=width=2064,height=1544,pixelformat=BGRx–stream-mmap -d /dev/video0
v4l2-ctl --set-fmt-video=width=2064,height=1544,pixelformat=XR24–stream-mmap -d /dev/video0

pixelformat=GREY8 doesn’t work.

If I set pixelformat=GREY it works, but it lowers width to 2048 like I described above and I have to reload the driver before I can get back to 2064.

gst-launch-1.0 v4l2src ! video/x-raw, format=GRAY8, width=2064, height=1544 ! videoconvert ! video/x-raw, format=I420 ! xvimagesink

doesn’t work and gives me the following message:
Tried to capture at 2064x1544, but device returned size 2048x1544

When changing width to 2048 it works…!

DaneLLL · July 1, 2022, 12:32am

Hi,
v4l2src supports BGRx so please try

gst-launch-1.0 v4l2src ! video/x-raw, format=BGRx, width=2064, height=1544 ! videoconvert ! video/x-raw, format=I420 ! xvimagesink

If it works, you can apply this string:

v4l2src ! video/x-raw, format=BGRx, width=2064, height=1544 ! videoconvert ! video/x-raw, format=BGR ! appsink

to the sample and try:
V4l2src using OpenCV Gstreamer is not working in Jetson Xavier NX - #3 by DaneLLL

Aleks-CST · July 1, 2022, 7:54am

Hi again, it’s finally working. Thank you so much, I’ve been stuck with this problem for days…

The following code seems to be working, however, it’s running really really slowly on my Jetson, using Visual Studio Code. Any idea what could cause this?

import cv2

cap = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw, format=BGRx, width=2064, height=1544 ! videoconvert ! video/x-raw, format=BGR ! appsink")

while True:
    ret_val, img = cap.read()
    cv2.imshow('v4l2-source',img)
    if cv2.waitKey(1) ==  ord('q'):
        break

cv2.destroyAllWindows()

Thanks for your help so far!

DaneLLL · July 1, 2022, 9:39am

Hi,
It takes significant CPU usage. Please execute sudo nvpmodel -m 0 and sudo jetson_clocks. To run CPU cores at maximum clock and try again.

Aleks-CST · July 1, 2022, 9:55am

That seemed to do the trick. Thank you!

system · July 20, 2022, 4:58am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Gstreamer nvv4l2camerasrc plugin resolution problem Jetson Xavier NX camera , gstreamer	18	2269	May 17, 2023
Convert Image Jetson AGX Xavier camera	10	2143	October 18, 2021
Jetson L4T 35.1 - CSI camera's issue with gstreamer Jetson AGX Xavier camera	19	1826	November 2, 2022
Python OpenCV video capture problem Jetson Xavier NX opencv , gstreamer , python	11	18533	October 18, 2021
CSI-camera is not work on Jetson Xavier nx Jetson Xavier NX camera , opencv , gstreamer	15	1524	October 18, 2021
Recording video from camera Jetson AGX Xavier python	38	4672	October 18, 2021
Recorded with Gstreamer videos compression question on Jetson Xavier NX Jetson Xavier NX camera , ubuntu , gstreamer	5	1395	October 18, 2021
Jetson is struggling with high resolution camera Jetson AGX Orin camera , opencv	11	1517	September 14, 2022
Gstreamer fail on Jetson AGX - Internal data error, Jetson AGX Xavier opencv , gstreamer	9	2139	October 18, 2021
Jetson Xavier yuv422 camera driver [one camera] no reply from camera processor Jetson AGX Xavier camera	35	3499	October 17, 2021

Problems showing Allied Vision camera with Gstreamer / OpenCV

Related topics