RTSP streaming with cv::VideoWriter

I want to stream images processed by OpenCV
in RTSP.

My program image is as below.
1.capture avi/mp4 file using cv::VideoCapture
2.any processing
3.stream processed image in RTSP

cv::VideoCapture and processing did work.
But VideoWriter did not.

I use gstreamer pipeline as argument of VideoWriter, and gst-rtsp-server.

I concerned about fps.
In detail, input fps is 30fps.
But output fps is 3 fps I want to set.
So I implemented program that processes image per 10 frames.

Where should I check ?

Not sure you can easily/efficiently stream RTSP from opencv videoWriter.

Maybe building a recent gstreamer version with rtspsink could be a solution, but I can’t tell if this would really work out.

The easiest way would be to install v4l2loopback, then use a videowriter with gstreamer pipeline to v4l2sink into your v4l2loopback node (assuming here it created /dev/video1):

cv::VideoWriter("appsrc ! videoconvert ! video/x-raw,format=BGRx ! identity drop-allocation=true ! v4l2sink device=/dev/video1", 0, fps, cv::Size(width, height));

then use test-launch to read the v4l2loopback node and stream it as rtsp:

./test-launch "v4l2src device=/dev/video1 ! nvvidconv ! nvv4l2h265enc ! h265parse ! rtph265pay name=pay0 pt=96 config-interval=1 "

Check with local client:

gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/test ! rtph265depay ! h265parse ! nvv4l2decoder ! nvegltransform ! nveglglessink

Honey_Patouceul, thank you for your reply.

I’ll try your proposal.
And I iave known about test-iaunch.
So, I’ll check whether my program works good.

I tried the way using v4l2loopback. But my program failed.
v4l2loopback installation had probably succeeded.

[s]My program error message is as followings.

OpenCV Error: Unspecified error (GStreamer: cannot put pipeline to play) in CvVideoWriter_GStreamer::open, 
file /home/nvidia/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp, line 1696
VIDEOIO(cvCreateVideoWriter_GStreamer (filename, fourcc, fps, frameSize, is_color)): raised OpenCV exception:

/home/nvidia/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp:1696: 
error: (-2) GStreamer: cannot put pipeline to play in function CvVideoWriter_GStreamer::open

Could not open the output video file for write

And pseudo cmd message is as followings.

$ gst-launch-1.0 videotestsrc ! videoconvert ! video/x-raw,format=BGRx ! identity drop-allocation=true ! v4l2sink
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0: Cannot identify device '/dev/video1'.
Additional debug info:
v4l2_calls.c(609): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0:
system error: No such file or directory
Setting pipeline to NULL ...
Freeing pipeline ...

[/s]

Above error occured by forgetting “sudo modprobe v4l2loopback”.

I have done test-launch and check client.
But no image displayed.
So I tested “gst-launch-1.0 v4l2src device=/video0 ! nvvidconv ! queue ! nvelgltransform ! nveglglessink”.
This displayed an image(not movie).

–02/18/2020
Finaly, I could stream and receive.
The error is caused by sync of sink is true.

Hello, I’m getting green display in VLC. Could you please help me? It is working when I use only test-launch to stream to vlc, but doesnt work with opencv

This is part of my code:

out = cv2.VideoWriter(“appsrc ! videoconvert ! video/x-raw,format=BGRx,width=1920,height=1080,framerate=30/1 ! identity drop-allocation=true ! v4l2sink device=/dev/video1” ,cv2.VideoWriter_fourcc(*“H264”), 30,(frame_width,frame_height))

./test-launch “v4l2src device=/dev/video1 ! nvvidconv ! omxh264enc ! h264parse ! rtph264pay name=pay0 pt=96 config-interval=1”

and opening in vlc with:

rtsp://192.168.2.242:9999/test

Not sure using v4l2loopback would be the best solution for later RTSP streaming.
You may try a videoWriter encoding into H264 and using RTP/UDP to localhost for test-launch:
opencv python example:

import cv2
cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1", cv2.CAP_GSTREAMER)
print(cap.isOpened())

out = cv2.VideoWriter("appsrc ! video/x-raw,format=BGR,width=1920,height=1080,framerate=30/1 ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 insert-sps-pps=1 ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=5000", cv2.CAP_GSTREAMER, 0, 30, (1920,1080))
print(out.isOpened())

while True:
	ret,frame = cap.read()
	out.write(frame)

out.release()
cap.release()

when running, you would launch RTSP server with:

./test-launch "udpsrc port=5000 ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! rtph264pay name=pay0"

And you may check RTSP client from localhost with:

gst-launch-1.0 -e rtspsrc location=rtsp://127.0.0.1:8554/test latency=500 ! application/x-rtp, media=video, encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=xvimagesink -v

If this doesn’t work, please create a new topic for your issue.