Csi camera with python and cv2 not working jetpack 5.02

paulrrh · October 31, 2022, 11:51am

I have some simple python programs using csi camera opencv and gstreamer. They now do not work jetpack 5.02
I have found examples on the internet that also don’t work e,g, Jetson Hacks
Can sombody give me a clue:-

ERROR MESSAGE:-
4.6.0
Traceback (most recent call last):
File “/home/paul/pypro/opencv-1.py”, line 24, in
cv2.imshow(‘mywindow’,frame)
cv2.error: OpenCV(4.6.0) /io/opencv/modules/highgui/src/window.cpp:967: error: (-215:Assertion failed) size.width>0 && size.height>0 in function ‘imshow’

PYTHON CODE PRODUCING THE ABOVE ERROR MESSAGE.

import cv2
print (cv2.version)
width=800
height=600
flip=2

camSet0=“nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12 ! nvvidconv flip-method=2 ! video/x-raw,format=BGRx, width=816, height=616, pixel-aspect-ratio=1/1 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1”
#camSet1=“nvarguscamerasrc sensor-id=1 ! video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12 ! nvvidconv flip-method=2 ! video/x-raw,format=BGRx, width=816, height=616, pixel-aspect-ratio=1/1 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1”

cam0=cv2.VideoCapture(camSet0)
#cam1=cv2.VideoCapture(camSet1)

#cam2=cv2.VideoCapture(0)

while True:

#csi camera 1 does not work since changing to jetpack 5.02

_,frame= cam0.read()
cv2.imshow('mywindow',frame)
cv2.moveWindow('myCam0',0,0)

csi camera 2 does not work since changing to jetpack 5.02

_, frame1= cam1.read()

cv2.imshow(‘mywindow’,frame1)

cv2.moveWindow(‘myCam1’,0,0)

Usb camera works ok when not commented out

_,frame2= cam2.read()

cv2.imshow(‘mywindow’,frame2)

cv2.moveWindow(‘myCam2’,0,0)

if cv2.waitKey(1)==ord('q'):
    break

cam0.release()
#cam1.release()
#cam2.release()

cv2.destroyAllWindows()

THANKS FOR ANY HELP

DaneLLL · November 1, 2022, 1:46am

Hi,
You may need to port sensor driver to Jetpack 5.0.2. Please try gst-launch-1.0 command:

$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12' ! fakesink

If it fails to run, the sensor driver shall not be ready.

paulrrh · November 1, 2022, 11:08am

Thanks very much for your reply.

The command comes back:-
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12 ! fakesink
bash: syntax error near unexpected token `(’

Altered the following command found on internet works for both camera
changing the sensor-id

gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! “video/x-raw(memory:NVMM),width=3264,height=2464,format=NV12,framerate=21/1” ! nvvidconv flip-method=2 ! video/x-raw,width=640,height=480 ! autovideoconvert ! ximagesink

Just a note to say :

Using jetson-inference containers the camera’s work fine.

paulrrh · November 2, 2022, 4:23pm

There seems to be no solutiion for using python and csi cameras with jetson xavier nx since moving to jetpack 5.02.
I searched for three days on the internet and all solutions given come to the same answer. The problem seems to be no image when using
cv2-imshow('Window",image)
I would be grateful if you could give me a clue.
gst-launch-1.0 command works fine as terminal command.
Thanks

Honey_Patouceul · November 2, 2022, 10:06pm

You may try:

import cv2
print (cv2.__version__)

camSet0="nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12 ! nvvidconv flip-method=2 ! video/x-raw,format=BGRx, width=816, height=616, pixel-aspect-ratio=1/1 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1"
cam0=cv2.VideoCapture(camSet0, cv2.CAP_GSTREAMER)
if not cam0.isOpened():
    print("Failed to open cam0")
    exit(-1)

camSet1="nvarguscamerasrc sensor-id=1 ! video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12 ! nvvidconv flip-method=2 ! video/x-raw,format=BGRx, width=816, height=616, pixel-aspect-ratio=1/1 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1"
cam1 = cv2.VideoCapture(camSet1, cv2.CAP_GSTREAMER)
if not cam1.isOpened():
    print("Failed to open cam1")
    exit(-2)

while True:
   ret, frame0 = cam0.read()
   if not ret:
       print('Failed to read from cam0')
       break

   ret, frame1 = cam1.read()
   if not ret:
       print('Failed to read from cam1')
       break

   cv2.imshow('frame0', frame0)
   cv2.imshow('frame1', frame1)
   cv2.waitKey(1)


cam0.release()
cam1.release()

paulrrh · November 3, 2022, 10:16am

Thanks for your help:-

failed to open cam0
My system not connectiong to csi camera when using python with visual code or Thonney.
Using gstreamer as terminal command I have connection and picture.
Its a shame that all python programes worked with previous version of jetpack but not now. Jetson OS is not upward compatible .

Thanks for your input but I do not know where or what to do next
Cheers

DaneLLL · November 3, 2022, 10:20am

Hi,
Please put the caps in ' ' like

'video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12'

And check if you can launch the two cameras in gst-launch-1.0 command.

paulrrh · November 3, 2022, 1:53pm

Thanks for you input.

This is command I used and works
gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! ‘video/x-raw(memory:NVMM),width=3264,height=2464,format=NV12,framerate=21/1’ ! nvvidconv flip-method=2 ! video/x-raw,width=640,height=480 ! autovideoconvert ! ximagesink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 3280 H = 2464
seconds to Run = 0
Frame Rate = 21.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
WARNING: from element /GstPipeline:pipeline0/GstXImageSink:ximagesink0: Pipeline construction is invalid, please add queues.
Additional debug info:
gstbasesink.c(1209): gst_base_sink_query_latency (): /GstPipeline:pipeline0/GstXImageSink:ximagesink0:
Not enough buffering available for the processing deadline of 0:00:00.015000000, add enough queues to buffer 0:00:00.015000000 additional data. Shortening processing latency to 0:00:00.000000000.
ERROR: from element /GstPipeline:pipeline0/GstXImageSink:ximagesink0: Output window was closed
Additional debug info:
ximagesink.c(696): gst_x_image_sink_handle_xevents (): /GstPipeline:pipeline0/GstXImageSink:ximagesink0
Execution ended after 0:00:12.356572413
Setting pipeline to NULL …
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Freeing pipeline …

This is the output from your command line

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM),width=800, height=600,framerate=21/1, format=NV12’ ! fakesink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 4
Output Stream W = 1280 H = 720
seconds to Run = 0
Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:33.870784335
Setting pipeline to NULL …
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Freeing pipeline …

There is no video output.

I am no totally confused as to what to do next.

Is this version for jetpack the latest ?pool/main/n/nvidia-jetpack/nvidia-jetpack_5.0.2-b231_arm64.deb

Once again thanks for your help.

Thanks for you help

Honey_Patouceul · November 3, 2022, 3:18pm

You may check if the opencv library installed in python has GStreamer SUPPORT with:

import cv2
print(cv2.getBuildInformation())

paulrrh · November 3, 2022, 4:06pm

This is output
Looks like answer no.

General configuration for OpenCV 4.6.0 =====================================
Version control: 4.6.0-dirty

Extra modules:
Location (extra): /io/opencv_contrib/modules
Version control (extra): 4.6.0

Platform:
Timestamp: 2022-06-07T14:25:15Z
Host: Linux 4.9.140-tegra aarch64
CMake: 3.22.5
CMake generator: Unix Makefiles
CMake build tool: /bin/gmake
Configuration: Release

CPU/HW features:
Baseline: NEON FP16

C/C++:
Built as dynamic libs?: NO
C++ standard: 11
C++ Compiler: /opt/rh/devtoolset-10/root/usr/bin/c++ (ver 10.2.1)
C++ flags (Release): -Wl,-strip-all -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -Wl,-strip-all -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
C Compiler: /opt/rh/devtoolset-10/root/usr/bin/cc
C flags (Release): -Wl,-strip-all -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -Wl,-strip-all -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release): -L/root/ffmpeg_build/lib -Wl,–gc-sections -Wl,–as-needed -Wl,–no-undefined
Linker flags (Debug): -L/root/ffmpeg_build/lib -Wl,–gc-sections -Wl,–as-needed -Wl,–no-undefined
ccache: YES
Precompiled headers: NO
Extra dependencies: /lib64/libopenblas.so Qt5::Test Qt5::Concurrent /usr/local/lib/libpng.so /lib64/libz.so Qt5::Core Qt5::Gui Qt5::Widgets Iconv::Iconv dl m pthread rt
3rdparty dependencies: libprotobuf ade ittnotify libjpeg-turbo libwebp libtiff libopenjp2 IlmImf quirc tegra_hal

OpenCV modules:
To be built: aruco barcode bgsegm bioinspired calib3d ccalib core cvv datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
Disabled: world
Disabled by dependency: -
Unavailable: alphamat cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev freetype hdf java julia matlab ovis python2 sfm ts viz
Applications: -
Documentation: NO
Non-free algorithms: NO

GUI: QT5
QT: YES (ver 5.15.0 )
QT OpenGL support: NO
GTK+: NO
VTK support: NO

Media I/O:
ZLib: /lib64/libz.so (ver 1.2.7)
JPEG: libjpeg-turbo (ver 2.1.2-62)
WEBP: build (ver encoder: 0x020f)
PNG: /usr/local/lib/libpng.so (ver 1.6.37)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.4.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES

Video I/O:
DC1394: NO
FFMPEG: YES
avcodec: YES (58.134.100)
avformat: YES (58.76.100)
avutil: YES (56.70.100)
swscale: YES (5.9.100)
avresample: NO
GStreamer: NO
v4l/v4l2: YES (linux/videodev2.h)

Parallel framework: pthreads

Trace: YES (with Intel ITT)

Other third-party libraries:
Lapack: YES (/lib64/libopenblas.so)
Eigen: NO
Custom HAL: YES (carotene (ver 0.0.1))
Protobuf: build (3.19.1)

OpenCL: YES (no extra features)
Include path: /io/opencv/3rdparty/include/opencl/1.2
Link libraries: Dynamic load

Python 3:
Interpreter: /opt/python/cp36-cp36m/bin/python3.6 (ver 3.6.15)
Libraries: libpython3.6m.a (ver 3.6.15)
numpy: /opt/python/cp36-cp36m/lib/python3.6/site-packages/numpy/core/include (ver 1.19.3)
install path: python/cv2/python-3

Python (for build): /bin/python2.7

Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO

Install to: /io/_skbuild/linux-aarch64-3.6/cmake-install

Honey_Patouceul · November 3, 2022, 4:37pm

So now you know why it fails. You may install an other version of OpenCV supporting Gstreamer in your python env.

paulrrh · November 4, 2022, 10:20am

Thanks again for everything

I now have a version of open cv which I achieved by
pip uninstall opencv-python
Rerun

print(cv2.getBuildInformation())

Now I have a differenct version of cv2 4.2.0
with gtreamer set YES

re-run my original program and now works

open cv2 4.6.0 was the problem.
It seems odd you can unistall cv2 but leaves a previous version live!!
Thanks once again

paulrrh · November 4, 2022, 10:23am

Solved I now have a version of open cv which I achieved by
pip uninstall opencv-python

Rerun

print(cv2.getBuildInformation())