Opencv alongwith raspbeery pi cam connected with Jetson Nano

Sir,
I’m trying a python program for object detection using raspberry pi cam. But it is showing the following error. The python program is given below .

import time
import serial
import numpy as np
import jetson.inference
import jetson.utils
import cv2
#from adafruit_servokit import ServoKit
#myKit=ServoKit(channels=16)
#arm
pan=0
#tilt=90
#myKit.servo[1].angle=armq
#myKit.servo[0].angle=pan
#myKit.servo[2].angle=tilt
dispW=640
dispH=480
flip=2

net = jetson.inference.detectNet(argv=[“–model=/home/manpreet/jetson-inference/python/training/detection/ssd/models/rice3/ssd-mobilenet.onnx”, “–labels=/home/manpreet/jetson-inference/python/training/detection/ssd/models/rice3/labels.txt”, “–input-blob=input_0”, “–output-cvg=scores”, “–output-bbox=boxes”],threshold=0.5)
cam = cv2.VideoCapture(‘csi://0’)
#camset= (‘gst-launch-1.0 nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1920,height=1080,format=NV12 ! nvvidconv ! fpsdisplaysink video-sink=fakesink --verbose’)
#camset=(‘gst-launch-1.0 -v v4l2src device=csi://0 ! video/x-raw,framerate=30/1,width=1280,height=720 ! nvvidconv ! xvimagesink’)
#cam = cv2.VideoCapture(camset)
#cam = cv2.VideoCapture(“nvarguscamerasrc ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink”,)
width=cam.get(cv2.CAP_PROP_FRAME_WIDTH)
height=cam.get(cv2.CAP_PROP_FRAME_HEIGHT)
print(‘width:’,width,‘height:’,height)
#display = jetson.utils.videoOutput(“display://0”)
while True:
#while display.IsStreaming() :
_,img = cam.read()
print(img)
frame=cv2.cvtColor(img,cv2.COLOR_BGR2RGBA).astype(np.float32)
frame=jetson.utils.cudaFromNumpy(frame)
detections = net.Detect(frame)
print(“detected {:d} objects in image”.format(len(detections)))
#myKit.servo[1].angle=90
print(“loop1”)
arduino = serial.Serial(‘/dev/ttyACM0’,115200)
#arduino = serial.Serial(‘/dev/ttyUSB0’,115200)
for detection in detections:
#print(detection)
top=int(detection.Top)
left=int(detection.Left)
bottom=int(detection.Bottom)
right=int(detection.Right)
center=detection.Center
#item=net.GetClassDesc(ID)
print(top,left,bottom,right,center)
cv2.rectangle(img,(left,top),(right,bottom),(0,255,0),2)
objX=left+(right-left)/2
errorPan=objX-width/2
print(pan)
print(objX)
print(errorPan)
arduino.write((str(errorPan)).encode(‘utf-8’))
if abs(errorPan)>15:
pan=pan-errorPan/75
print(“if loop 1 >15”)
print(pan)
#else:
#time.sleep(4)
if pan>180:
pan=180
print(“pan out of range”)
if pan<0:
pan=0
print(“pan out of range”)
#myKit.servo[0].angle=pan
print(“loop2”)
#time.sleep(4)
break
#time.sleep(3)
#display.Render(frame)
#display.SetStatus(“Object Detection | Network {:.0f} FPS” .format(net.GetNetworkFPS()))

cv2.imshow('detCam',img)
cv2.moveWindow('detCam',0,0)
if cv2.waitKey(1)==ord('q'):
    break

cam.release()
cv2.destroyAllWindows()

And the error shown is…
width: 0.0 height: 0.0
None
Traceback (most recent call last):
File “/home/manpreet/jetson-inference/python/training/detection/ssd/contour and detection2.py”, line 33, in
frame=cv2.cvtColor(img,cv2.COLOR_BGR2RGBA).astype(np.float32)
cv2.error: OpenCV(4.6.0) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function ‘cvtColor’

Please kindly reply if anyone could find any solution.

Hi,
Please run the gstreamer command to ensure the camera source is good first:

$ gst-launch-1.0 nvarguscamerasrc ! nvoverlaysink

By defaultRaspberry pi V2 camera is supported. And you should see camera preview when running the command.

When I run these code It is showing
'WARNING: errorneous pipeline: syntax error

Hi,
There was an additional ! in the command. Have removed it. Please try again.

I got the following result.
manpreet@manpreet-desktop:~$ gst-launch-1.0 nvarguscamerasrc ! nvoverlaysink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 2
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

And it is showing up to these only . Nothing else is coming …

Hi @jino.joy.m, does that show you the video? You could also try nvgstcapture-1.0 command to test video capture and make sure your CSI camera is working.

This is the syntax for constructing a jetson.utils.videoSource interface for CSI camera, not the cv2.VideoCapture. I think cam = cv2.VideoCapture(“nvarguscamerasrc ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink”,) is more akin to how you would do it with OpenCV. However I recommend just using jetson.utils.videoSource like you would see in detectnet.py

The video is not coming. And when I try these code of cam = cv2.VideoCapture(“nvarguscamerasrc ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink”,) it is showing the same error as above mentioned like
File “/home/manpreet/jetson-inference/python/training/detection/ssd/contour and detection2.py”, line 33, in
frame=cv2.cvtColor(img,cv2.COLOR_BGR2RGBA).astype(np.float32)
cv2.error: OpenCV(4.6.0) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function ‘cvtColor’

Hi @jino.joy.m, if nvgstcapture-1.0 isn’t working, then it’s recommended to check the cabling connection with the camera or the camera hardware itself. Until then, OpenCV/ect probably won’t work to capture the video either.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.