I am trying to capture a vide stream coming from a rapberry zero. I can stream the video via gstreamer using below command on terminal. gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=-1 ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.178.84 port=5200
And I can capture the stream from nano using the terminal command. gst-launch-1.0 -v udpsrc port=5200 ! application/x-rtp, media=video, clock-rate=90000, payload=96 ! rtpjpegdepay ! jpegdec ! videoconvert ! autovideosink
How can I integrate those commands into a python script so that I can process the stream with opencv. I want to use a python script both on the sender raspberry and on the receiving jetson.
Everything works fine on linux terminal but could not succeed to make it work on python. Any assitance?
import time
import Jetson.GPIO as GPIO
import jetson.inference
import jetson.utils
import cv2
LED_PIN = 11
GPIO.setmode(GPIO.BOARD)
GPIO.setup(LED_PIN, GPIO.OUT)
net = jetson.inference.detectNet(model='models/fruit/ssd-mobilenet.onnx',
labels='models/fruit/labels.txt',
input_blob='input_0',
output_cvg='scores',
output_bbox='boxes',
threshold=0.5)
# Open the CSI camera using OpenCV
camera = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER)
# Set the desired width and height of the camera
width, height = 1280, 720
camera.set(cv2.CAP_PROP_FRAME_WIDTH, width)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
# Create a window to display the camera feed
window_title = "CSI Camera"
cv2.namedWindow(window_title, cv2.WINDOW_AUTOSIZE)
def main():
while True:
try:
ret, frame = camera.read()
if not ret:
continue
# Convert BGR image to RGBA for jetson_inference
img_rgba = cv2.cvtColor(frame, cv2.COLOR_BGR2RGBA)
# Get the width and height of the image
width, height = frame.shape[1], frame.shape[0]
detections = net.Detect(img_rgba, width, height)
# Display the image with detected objects
cv2.imshow(window_title, frame)
GPIO.output(LED_PIN, GPIO.LOW)
Apple_detected = False
for detection in detections:
if detection.ClassID == 1:
GPIO.output(LED_PIN, GPIO.HIGH)
print("Apple detected")
Apple_detected = True
break
if not Apple_detected:
GPIO.output(LED_PIN, GPIO.LOW)
if cv2.waitKey(10) & 0xFF == 27:
break
except Exception as e:
print("Error capturing image from camera:", str(e))
break
# Release the camera and close the window when done
camera.release()
cv2.destroyAllWindows()
if __name__ == "__main__":
main()
I’ve tried using the provided code, but I’m encountering an issue. The camera starts up as expected, but it doesn’t seem to detect any fruits, and I don’t see any bounding boxes being drawn around them. Could you please assist me in resolving this problem? I’m not sure why the fruit detection and bounding box drawing are not functioning as intended.