How to stream video to network from python?

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, ,
1

Hi.

The problem:

I need to receive video from IP camera, decode it, do some work with frames and streaming it to network.
I read the forums and wrote this code:

import cv2
gst_in = "rtspsrc location="rtsp://login:password@ipadress:port" latency=20 ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw, format=(string)BGRx! videoconvert ! appsink"
cap = cv2.VideoCapture(gst_in, cv2.CAP_GSTREAMER)
gst_out = "appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=RGBA ! nvvidconv ! nvv4l2h264enc insert-sps-pps=true ! h264parse ! rtph264pay pt=96 ! tcpserversink host=ipadress port=8001 sync=false"
out = cv2.VideoWriter(gst_out, cv2.CAP_GSTREAMER, 0, float(25), (1920, 1080))
while True:
    _,frame = cap.read()
    ....... do some work with frame.....
    out.write(frame)

That’s running without errors. I received decoded frame from camera and do some operations with them.

But when I try to open this stream in VLC on another client, the connection is established and stops after 10 seconds with the error " live555 error: no data received in 10s, aborting"

For opening this stream in VLC i create SDP file:

The questions:

  1. The idea to use OpenCV and Gstreamer for this task is correct?

  2. Why i have error “no data received” when i send frame to GStreamer?

  3. Is my pipelins for VideoCapture and VideoWriter correct?

Additional information:

  1. I tryed to use

cap.get(cv2.CAP_PROP_FSP)

for getting FPS from camera, but everytime that returns 0

  1. If i use this pipline:

appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=RGBA ! nvvidconv ! nvv4l2h264enc insert-sps-pps=true ! h264parse ! qtmux ! filesink location=test.mov

then the data is written to the file successfully.

  1. For opening this stream in VLC i create SDP file:

v=0
m=video 8001 RTP/AVP 96
c=IN IP4 192.168.1.43
a=rtpmap:96 H264/90000

2

I think the root problem is trying RTP over TCP. TCP won’t maintain the packetization RTP relies upon. You may use another container, such as gdppay/gdpdepay if using gstreamer on both sides. I’ve also been using matroskamux for that purpose. There might also be options such as RTSPT (rtsp over tcp).

I’d suggest to use UDP for now.

I first assume you can connect to your camera from Jetson and display with something like:

gst-launch-1.0 -v rtspsrc location=rtsp://login:password@cam_ip:cam_port latency=200 ! application/x-rtp, media=video, encoding-name=H264 ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! xvimagesink

Then an input pipeline for opencv capture in python would be:

gst_in = "rtspsrc location=rtsp://login:password@cam_ip:cam_port latency=200 ! application/x-rtp, media=video, encoding-name=H264 ! rtph264depay ! h264parse ! omxh264dec ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! queue ! video/x-raw, format=BGR ! appsink "
cap = cv2.VideoCapture(gst_in, cv2.CAP_GSTREAMER)
if not cap.isOpened() :
    print("capture failed")
    exit()
print('cap opened')
w = cap.get(cv2.CAP_PROP_FRAME_WIDTH)
h = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)
fps = cap.get(cv2.CAP_PROP_FPS)
print('Src opened, %dx%d @ %d fps' % (w, h, fps))

If this displays 0 fps, probably the stream lacks VUI. You may add resolution and framerate into caps after rtph264depay:

gst_in = "rtspsrc location=rtsp://login:password@cam_ip:cam_port latency=200 ! application/x-rtp, media=video, encoding-name=H264 ! rtph264depay ! video/x-h264, width=1920, height=1080, framerate=25/1 ! h264parse ! omxh264dec ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! queue ! video/x-raw, format=BGR ! appsink "

and it should find 25 fps (assuming this is a true framerate for your ip cam)

You would then use such writer to encode into H264 → RTP → UDP localhost port 5000:

gst_out = "appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! omxh264enc insert-sps-pps=true ! h264parse ! rtph264pay pt=96 ! queue ! application/x-rtp, media=video, encoding-name=H264 ! udpsink host=127.0.0.1 port=5000 sync=false"
out = cv2.VideoWriter(gst_out, cv2.CAP_GSTREAMER, 0, float(fps), (int(w), int(h)))
if not out.isOpened() :
    print("Writer failed")
    exit()
print('Writer opened')

and adding loop:

while True:
    ret,frame = cap.read()
    if not ret:
        break

    # Some processing
    out.write(frame)

out.release()
cap.release()

…it should be ready.
You would check with gstreamer on Jetson:

gst-launch-1.0 -ev udpsrc port=5000 ! application/x-rtp, media=video, encoding-name=H264, clock-rate=90000 ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! xvimagesink sync=false

If working you would try VLC on Jetson. It can be installed through APT, but be aware that for Jetson you would have to remove a plugin. Then prepare a file test.sdp with that content:

c=IN IP4 127.0.0.1
m=video 5000 RTP/AVP 96
a=rtpmap:96 H264/90000

save it and run :

vlc -v test.sdp

If all works, you would adapt for streaming to another address.

2 Likes
3

Thanks alot! It’s working locally on jatson.(from gst-launch and vlc)

But when i try to open stream in vlc on another pc i recieve error:

live555 error: no data received in 10s, aborting

P.S.
I changed address in gst_out pipeline. I tried adresses: “0.0.0.0” and “192.168.1.43”(ip address of jetson in my local network)

4

Use the address you want to stream to in udpsink.
You may also remove sync=false.

Be also sure that no firewall prevents frame from jetson to target at port 5000.

5

when I removed the sync=false everything worked.
Thanks!

1 Like
6

Hi, can I try this in any VLC?
I try to open the video in VLC using udp://127.0.0.1:5000, it didn’t work.

7

You would make a sdp file as in post#2.
For multicast, you may also check this post.