Encountering issues when attempting to stream RTMP with Jetson Nano using Gstreamer and OpenCV

1735756353 · February 27, 2024, 12:39pm

My device is a jetson nano.
I want to use opencv and Gstreamer to push the processed frame image to a computer on the same LAN with the corresponding GPU acceleration enabled, which uses VLC to pull and display. But something went wrong. The corresponding code and log are as follows:

import time
import cv2

# Cam properties
fps = 25

frame_width = 1920
frame_height = 1080
# Create capture
address = 'rtspsrc location=rtsp://admin:wzu123456@192.168.0.64:554/h264/ch1/main/av_stream latency=0  ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! appsink '
cap = cv2.VideoCapture(address)

# Set camera properties
#cap.set(cv2.CAP_PROP_FRAME_WIDTH, frame_width)
#cap.set(cv2.CAP_PROP_FRAME_HEIGHT, frame_height)
#cap.set(cv2.CAP_PROP_FPS, fps)

**# Define the gstreamer sink**
**gst_str_rtp = "appsrc ! videoconvert ! nvvidconv ! nvv4l2h264enc tune=zerolatency bitrate=500 speed-preset=superfast ! h264parse ! flvmux ! rtmpsink location='rtmp://127.0.0.1:8554'"**

# Check if cap is open
if cap.isOpened() is not True:
    print ("Cannot open camera. Exiting.")
    quit()

# Create videowriter as a SHM sink
**out = cv2.VideoWriter(gst_str_rtp, 0, fps, (frame_width, frame_height), True)**

# Loop it
while True:
    # Get the frame
    ret, frame = cap.read()
    # Check
    if ret is True:
        cv2.imshow('frame1', frame)
        if cv2.waitKey(1) == ord('q'):
            break
        # Flip frame
        # frame = cv2.flip(frame, 1)
        # Write to SHM
        **out.write(frame)**
    else:
        print("Camera error.")
        time.sleep(10)

cap.release()



(ocr) wzu@wzu-desktop:~/jjh/demo$ python gst_device_to_rtp.py

(python:13353): GStreamer-CRITICAL **: 20:20:24.125: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(python:13353): GStreamer-CRITICAL **: 20:20:24.125: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed

(python:13353): GStreamer-CRITICAL **: 20:20:24.125: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(python:13353): GStreamer-CRITICAL **: 20:20:24.125: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(python:13353): GStreamer-CRITICAL **: 20:20:24.125: gst_structure_get_string: assertion 'structure != NULL' failed

(python:13353): GStreamer-CRITICAL **: 20:20:24.125: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Allocating new output: 2560x1440 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 2560, nFrameHeight = 1440 
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1063) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1100) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1673) open OpenCV | GStreamer warning: error opening writer pipeline: no property "tune" in element "nvv4l2h264enc0"

Nano address is 192.168.0.166, pull flow is normal, but another computer via VLC display screen, VLC using address is RTMP: / / 192.168.0.166:8554

DaneLLL · February 29, 2024, 3:16am

Hi,
We are uncertain if RTMP works in VLC. This would need other users to share experience.

Generally, we use test-launch to launch RTSP server and VLC player can open the network stream successfully. Please refer to

Jetson Nano FAQ
Q: Is there any example of running RTSP streaming?

1735756353 · February 29, 2024, 7:58am

I tried what you said, and it worked. But now I want to push the processed image through rtmp, just like the code I wrote above, how I can modify it. If VLC doesn’t support RTMP, try RTSP first. Here is the key code:

gst_str_rtp = "appsrc ! videoconvert ! nvvidconv ! nvv4l2h264enc tune=zerolatency bitrate=500 speed-preset=superfast ! h264parse ! flvmux ! rtmpsink location='rtmp://127.0.0.1:8554'"

out = cv2.VideoWriter(gst_str_rtp, 0, fps, (frame_width, frame_height), True)

 if ret is True:
        cv2.imshow('frame1', frame)
        if cv2.waitKey(1) == ord('q'):
            break
  
        out.write(frame)

DaneLLL · March 1, 2024, 1:28am

Hi,
We are uncertain how to launch RTMP. This would need other users to share experience and knowledge.

Try the command on Jetson device:

$ gst-launch-1.0 videotestsrc is-live=1 ! queue ! videoconvert ! x264enc ! flvmux streamable=true ! queue ! rtmpsink location='rtmp://10.19.100.135:1234/test'

10.19.100.135 is IP address of Windows laptop. The command can be run. But I try to open VLC player on Windows laptop and access the stream, it fails. We don’t know what we are missing in setting up RTMP. Please check and see if you have any idea.

1735756353 · March 8, 2024, 8:17am

I changed some of the code and found that the pipeline works fine under the following test commands.

gst-launch-1.0 -v videotestsrc ! video/x-raw,width=1920,height=1080,format=BGRx,framerate=25/1 ! nvvidconv ! omxh264enc ! flvmux ! rtmpsink location='rtmp://192.168.0.2:1935/live'

But the following code does not work properly.

import time
import cv2

# Cam properties
fps = 25

frame_width = 1920
frame_height = 1080
# Create capture
address = 'rtspsrc location=rtsp://admin:wzu123456@192.168.0.64:554/h264/ch1/main/av_stream latency=0  ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! appsink '
cap = cv2.VideoCapture(address)

# Set camera properties
#cap.set(cv2.CAP_PROP_FRAME_WIDTH, frame_width)
#cap.set(cv2.CAP_PROP_FRAME_HEIGHT, frame_height)
#cap.set(cv2.CAP_PROP_FPS, fps)

# Define the gstreamer sink
gst_str_rtp = "appsrc ! video/x-raw,width=1920,height=1080,format=BGRx,framerate=25/1 ! nvvidconv ! omxh264enc !  flvmux ! rtmpsink location='rtmp://192.168.0.2:1935/live'"

# Check if cap is open
if cap.isOpened() is not True:
    print ("Cannot open camera. Exiting.")
    quit()

# Create videowriter as a SHM sink
out = cv2.VideoWriter(gst_str_rtp, 0, fps, (frame_width, frame_height), True)

# Loop it
while True:
    # Get the frame
    ret, frame = cap.read()
    # Check
    if ret is True:
        cv2.imshow('frame1', frame)
        if cv2.waitKey(1) == ord('q'):
            break
        # Flip frame
        # frame = cv2.flip(frame, 1)
        # Write to SHM
        out.write(frame)
    else:
        print("Camera error.")
        time.sleep(10)

cap.release()

The following error occurred:

(ocr) wzu@wzu-desktop:~/jjh/demo$ python gst_device_to_rtp.py

(python:6300): GStreamer-CRITICAL **: 16:02:33.673: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(python:6300): GStreamer-CRITICAL **: 16:02:33.673: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed

(python:6300): GStreamer-CRITICAL **: 16:02:33.673: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(python:6300): GStreamer-CRITICAL **: 16:02:33.673: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(python:6300): GStreamer-CRITICAL **: 16:02:33.673: gst_structure_get_string: assertion 'structure != NULL' failed

(python:6300): GStreamer-CRITICAL **: 16:02:33.674: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Allocating new output: 2560x1440 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 2560, nFrameHeight = 1440 
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1063) open OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1100) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=0, duration=-1
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (2076) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module appsrc0 reported: Internal data stream error.
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1967) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1967) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1967) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1967) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1967) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1967) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
[ WARN:0] global /home/wzu/jjh/opencv/opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1967) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline

What should I do to make the code work?

1735756353 · March 8, 2024, 8:18am

In addition, the video stream can be obtained normally.

Topic		Replies	Views
Jetson Nano: streaming opencv frames via RTSP gstreamer pipeline Jetson Nano camera , opencv , gstreamer , python	3	335	July 17, 2024
Gstreamer with Opencv for rtsp stream Jetson Nano gstreamer	8	9576	December 22, 2021
Python opencv gstreamer RTSP stream error Jetson Nano gstreamer	18	6237	April 6, 2022
IP camera streaming on Jetson TX2 Jetson TX2	15	6861	October 18, 2021
Streaming an openCV capture with gstreamer in Python Jetson AGX Xavier opencv , gstreamer	17	4948	October 18, 2021
opencv problem with capturing rtsp stream using gstreamer on tx2 Jetson TX2	16	18856	October 18, 2021
RTSP streaming with cv::VideoWriter Jetson Nano	6	7828	October 13, 2021
Reading RTSP stream using Gsteamer in Opencv in C++ Jetson Nano	5	4008	October 18, 2021
Create RTSP server on Jetson Nano with Gstreamer python Jetson Nano camera , opencv , gstreamer	3	8119	October 15, 2021
Multiple stream from one camera via gstreamer Jetson Nano camera , gstreamer	11	6453	October 15, 2021

Encountering issues when attempting to stream RTMP with Jetson Nano using Gstreamer and OpenCV

Related topics