OpenCV version: 4.8.0
By compiling opencv4.8 with gstreamer, At Nvidia jestson Xavier NX uses mat data as rtsp server, the client to pull the flow, the code is as follows:
void* InitRtspServerModules()
{
std::string gstreamerOutputUrl ="appsrc ! x264enc ! rtph264pay name=pay0 pt=96";
cv::VideoWriter outWriter = cv::VideoWriter(gstreamerOutputUrl, cv::CAP_GSTREAMER,cv::VideoWriter::fourcc('H','2','6','4'), 25.0, cv::Size(320, 320));
if(!outWriter.isOpened())
{
std::cout << "InitRtspServerModules failed "<<std::endl;
return NULL;
}
rtsp_server_instance *_instance = new rtsp_server_instance();
if (NULL == _instance) {
return NULL;
}
_instance->_param = std::make_shared<output_server_param>();
_instance->_param->writer = outWriter;
_instance->_param->bParamIsOk = true;
return (void*)_instance;
}
FACE_ERROR_E WriteStreamForRtspServer( void* pOutputInstance, cv::Mat frame)
{
struct rtsp_server_instance* _instance =
reinterpret_cast<struct rtsp_server_instance*>(pOutputInstance);
if (!_instance || !_instance->_param->bParamIsOk) {
std::cout << "param is NULL" << std::endl;
return FACE_ERR_INVALID_PARAM;
}
if (frame.empty())
{
std::cout << "frame is empty." << std::endl;
return FACE_ERR_INVALID_PARAM;
}
//_instance->_param->writer<<frame;
_instance->_param->writer.write(frame);
return FACE_OK;
}
However, the following problems were reported:[ WARN:0@24.097] global cap_gstreamer.cpp:2676 writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
However, it is normal for me to refer to the rtsp-server source code to create the service：
/* GStreamer
* Copyright (C) 2008 Wim Taymans <wim.taymans at gmail.com>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/
#include <gst/gst.h>
show original
1. After compiling, switch to the examples directory: cd examples 2. Build the Rtsp Server: ./test-launch "( videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96 )" 3, Play rtsp stream: GST - launch - 1.0 playbin uri = RTSP: / / 127.0.0.1:8554 / test
Compile gstreamer version opencv：cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=install -D INSTALL_C_EXAMPLES=OFF -D WITH_FFMPEG=ON -D WITH_GSTREAMER=ON -D WITH_GTK_2_X=ON -D WITH_GTHREAD=ON -D WITH_TBB=ON -D WITH_OPENGL=ON -DBUILD_opencv_python3=ON -D OPENCV_GENERATE_PKGCONFIG=1 ..
gstreamer part info is:
Compile the interface code to run the program.
@DaneLLL
My current problem may be different, my input is to decode the RTSP stream and then push the stream to the default address in the way of the RTSP service, and the client pulls the stream through the address。
This is my modified gstreamer code, but the client can’t pull streams through rtsp://x.x.x.x:8554/test:
Hi,
For running UDP, please refer toStream processed video with OpenCV on Jetson TX2 - #5 by DaneLLL OpenvCV, Gstreamer, Python camera capture/access and streaming to RTP
@DaneLLL
gstreamerUrl = format("rtspsrc location=%s latency=200 ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert n-threads=4 ! video/x-raw, format=(string)BGR ! appsink sync=false", strRtspUrl.c_str());
videocapcfg = cv::VideoCapture(gstreamerUrl,cv::CAP_GSTREAMER);
The following code is used to transfer to the RTSP server:
std::string gstreamerOutputUrl ="appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc ! h264parse ! rtph264pay name=pay0 pt=96 config-interval=1 ! udpsink host=0.0.0.0 port=8554";
cv::VideoWriter outWriter = cv::VideoWriter(gstreamerOutputUrl, cv::CAP_GSTREAMER,cv::VideoWriter::fourcc('H','2','6','4'), 25.0, cv::Size(320, 320));
Code to run nor any error, I am a 192.168.101.79 server IP, udpsink address from 0.0.0.0 modification for 192.168.101.79 also, but I’m from the network, through the VLC from RTSP: / / 192.168.101.79:8554 / test can’t flow, What is the reason for this?
Hi,nvv4l2decoder since omx plugins are deprecated. Please try this sample and check if video preview is good:Doesn't work nvv4l2decoder for decoding RTSP in gstreamer + opencv - #3 by DaneLLL
And then it should work by applying the URI to this sample:Stream processed video with OpenCV on Jetson TX2 - #5 by DaneLLL
@DaneLLL
gstreamerUrl = format("rtspsrc location=%s latency=200 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert n-threads=4 ! video/x-raw, format=(string)BGR ! appsink", strRtspUrl.c_str());
videocapcfg = cv::VideoCapture(gstreamerUrl,cv::CAP_GSTREAMER);
std::string gstreamerOutputUrl ="appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! h264parse ! rtph264pay config-interval=1 pt=96 ! tcpserversink host=0.0.0.0 port=8554";
cv::VideoWriter outWriter = cv::VideoWriter(gstreamerOutputUrl, cv::CAP_GSTREAMER,cv::VideoWriter::fourcc('H','2','6','4'), 25.0, cv::Size(320, 320));
Hi,
$ DISPLAY=:0 gst-launch-1.0 rtspsrc location=RTSP_URI latency=200 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvegltransform ! nveglglessink
@DaneLLL
Hi,Stream processed video with OpenCV on Jetson TX2 - #5 by DaneLLL
I am not sure if tcpserversink works in OpenCV. Using UDP should be more promising.
@DaneLLL
I tried udp and couldn’t open the video：
"appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=0.0.0.0 port=5000
Hi,host=0.0.0.0 is valid. We would suggest set to the IP address of the device which is receiving UDP stream.
And it looks like VLC player does not enable UDP by default. You may refer to the post to manually enable it:Gstreamer TCPserversink 2-3 seconds latency - #13 by Bazziil
It is simpler to run a gstreamer command like:Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL
@DaneLLL
appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.101.79 port=5000
@DaneLLL
Hi,Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL
We run the server command on Jetson device and client command on host PC in Ubuntu 18.04 or 20.04. It works well in the set up. Please give it a try.
When using gstreamer backend for a VideoWriter, the 4CC code would be 0 (RAW).
For VideoCapture or VideoWriter, first check with method isOpened() that it is successfully opened before using it.
@Honey_Patouceul
674316183:
cv::VideoWriter outWriter = cv::VideoWriter(gstreamerOutputUrl, cv::CAP_GSTREAMER,cv::VideoWriter::fourcc('H','2','6','4'), 25.0, cv::Size(320, 320));
You would use instead:
cv::VideoWriter outWriter = cv::VideoWriter(gstreamerOutputUrl, cv::CAP_GSTREAMER, 0, 25.0, cv::Size(320, 320));
@Honey_Patouceul
appsrc ! videoconvert ! queue ! nvvidconv ! nvv4l2h264enc maxperf-enable=1 ! h264parse ! matroskamux ! rtspserversink port=8554 service-prefix=test[ WARN:0@1.360] global cap_gstreamer.cpp:2382 open OpenCV | GStreamer warning: error opening writer pipeline: no element "rtspserversink"
