Streaming and recording camera feed using python, opencv and gstreamer

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, , ,
1

Hi,
I am trying to send camera feed as opencv frames through udp port and receive the frames and save it as a video, below is the Producer program and Receiver program that I am using,

Producer.py

import cv2
from multiprocessing import Process
def savefile():
    cap_feed = cv2.VideoCapture('nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! appsink',cv2.CAP_GSTREAMER)
    frame_width = int(cap_feed.get(3))
    frame_height = int(cap_feed.get(4))
    size = (frame_width, frame_height)
    cap_send = cv2.VideoWriter('appsrc ! videoconvert ! x264enc tune = zerolatency bitrate = 500 ! rtph264pay ! udpsink host = 127.0.0.1 port = 5000',cv2.CAP_GSTREAMER, 0, 20, size, True)
    if not cap_feed.isOpened():
        print('VideoCapture not opened')
        exit(0)
    while True:
        ret, frame = cap_feed.read()
        if not ret:
            print('empty frame')         
            break
        cap_send.write(frame)
    cap_feed.release()
    cap_send.release()
if __name__ == '__main__':
    savefile()

Receiver.py

import cv2
from multiprocessing import Process
import time
def receive_n_save():
    cap_receive = cv2.VideoCapture('udpsrc port = 5000 caps = "application/x-rtp, media = (string)video, clock-rate = (int)90000, encoding-name = (string)H264, payload = (int)96" ! rtph264depay ! decodebin ! videoconvert ! appsink',cv2.CAP_GSTREAMER)
    if not cap_receive.isOpened():
        print('receive_n_save : Sending not opened')
        exit(0)
    frame_width = int(cap_receive.get(3))
    frame_height = int(cap_receive.get(4))
    size = (frame_width, frame_height)
    print(cap_receive, size)    
    # Below VideoWriter object will create a frame of above defined The output is stored in 'feed_filename_fourcc_codec.avi' file.
    result = cv2.VideoWriter('PIM1'+int(time.time())+'.avi',
                             cv2.VideoWriter_fourcc(*'PIM1'), #available options : MJPG, PIM1 etc.
                             20, size)
    while True:
        ret, frame = cap_receive.read()
        if not ret:
            print('empty frame')
            break
        else:
            # Write the frame into the file 'feed_filename_fourcc_codec.avi'
            result.write(frame)
    cap_receive.release()
if __name__ == '__main__':
    receive_n_save()

the following is the error message that I get trying to run these programs,

Error message in Receiver.py

Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NVMEDIA: NVMEDIABufferProcessing: 1507: NvMediaParserParse Unsupported Codec

do anyone know how to solve this issue?

2

Try using HW encoder:

cap_send = cv2.VideoWriter('appsrc ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 idrinterval=30 ! rtph264pay ! udpsink host = 127.0.0.1 port = 5000', cv2.CAP_GSTREAMER, 0, 20.0, size, True)

Also, on receiver you may use :

cap_receive = cv2.VideoCapture('udpsrc port=5000 caps="application/x-rtp, media = (string)video, clock-rate = (int)90000, encoding-name = (string)H264, payload = (int)96" ! rtph264depay ! decodebin ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1',cv2.CAP_GSTREAMER)
3

Thank Patouceul for your response,
But I’m getting this error message.

[ WARN:0] global /root/opencv/modules/videoio/src/cap_gstreamer.cpp (734) open OpenCV | GStreamer warning: Error opening bin: could not link nvvconv0 to videoconvert0, nvvconv0 can't handle caps video/x-raw, format=(string)BGR
[ WARN:0] global /root/opencv/modules/videoio/src/cap_gstreamer.cpp (501) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
receive_n_save : Sending not opened

However I got the code to work, but still facing issue with the capture quality. The recevier program seems to miss few frames while writing video.

Here is the revised code:
Kindly have a look at it and let me know what’s missing / what can be improved
Sender.py

import cv2
from multiprocessing import Process

def sendCapture():
    cap_feed = cv2.VideoCapture('nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! appsink',cv2.CAP_GSTREAMER)
    # We need to set resolutions. so, convert them from float to integer.
    frame_width = int(cap_feed.get(3))
    frame_height = int(cap_feed.get(4))
    size = (frame_width, frame_height)
    cap_send = cv2.VideoWriter('appsrc ! videoconvert ! x264enc ! rtph264pay ! udpsink host = 127.0.0.1 port = 5000',cv2.CAP_GSTREAMER, 0, 20, size, True)
    if not cap_feed.isOpened():
        print('VideoCapture not opened')
        exit(0)
    if not cap_send.isOpened():
        print('sendCapture: Sending not opened')
        exit(0)
    while True:
        ret, frame = cap_feed.read()
        #print("Caturing...")
        if not ret:
            print('empty frame')         
            break
        cap_send.write(frame)
    cap_feed.release()
    cap_send.release()

if __name__ == '__main__':
    sendCapture()

Receiver.py

import cv2
from multiprocessing import Process
import time

def receive_n_save():
    cap_receive = cv2.VideoCapture('udpsrc port=5000 ! application/x-rtp,media=video,encoding-name=H264 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! appsink',cv2.CAP_GSTREAMER)
    if not cap_receive.isOpened():
        print('receive_n_save : Capturing not opened')
        exit(0)
    # We need to set resolutions. so, convert them from float to integer.
    frame_width = int(cap_receive.get(3))
    frame_height = int(cap_receive.get(4))
    size = (frame_width, frame_height)
    # Below VideoWriter object will create a frame of above defined The output is stored in 'feed_filename_fourcc_codec.avi' file.
    result = cv2.VideoWriter('PIM1'+str(time.time())+'.avi',
                             cv2.VideoWriter_fourcc(*'PIM1'), #available options : MJPG, PIM1 etc.
                             20, size)
    print("Starting capture...")
    while True:
        try:
            ret, frame = cap_receive.read()
            print("capturing...")
            if not ret:
                print('empty frame')
                break
            else:
                # Write the frame into the file 'feed_filename_fourcc_codec.avi'
                result.write(frame)
        except KeyboardInterrupt:
            cap_receive.release()
            result.release()
            break
    cap_receive.release()
    result.release()

if __name__ == '__main__':
    receive_n_save()

Sender.py Log

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0] global /root/opencv/modules/videoio/src/cap_gstreamer.cpp (961) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1

Receive.py Log

[ WARN:0] global /root/opencv/modules/videoio/src/cap_gstreamer.cpp (961) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=162, duration=-1
Starting capture...
capturing...

Regards

4

Sorry I replied in hurry and edited my post just later. You may try again. This would leverage HW encoding and decoding, and may better perform than x264enc/avdec_h264 for higher resolutions or fps and may save CPU load.

I also notice that you’re using ffmpeg backend in receiver writer for saving into AVI file. If using PIM1 codec is not a requirement, you may also encode into h264 with HW and save to AVI with:

appsrc ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 idrinterval=30 ! avimux ! filesink location=test_h264.avi
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.