IP Camera RTSP + GSTREAMER + C++ (Stream Latency 4-5 seconds)

[b]Hi,

I have a the application reads rtsp stream address from the xml file where we mention the i.e rtsp or webcam and streams the rtsp camera with a latency of 4-5 seconds whereas usb webcam is in realtime.

I am using HPE DL385 servers Ubuntu 18.04 server edition OS with cuda 10.1 and two Tesla T4 GPU cards & opencv 3.4.1

I also tried the gstreamer pipeline which gives 0.3sec latency and thats almost a realtime stream

gst-launch-1.0 rtspsrc location=rtsp://root:Glueck321@10.0.1.36:554/axis-media/media.amp?streamprofile=H264  latency=200 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

I wanted to merge the gstreamer pipeline to my c++ code where I capture the rtsp stream. I just added the whole pipeline into the c++ code as

// RTSP or Video File
        void open(const std::string url_){
//              this->url = url_;
//              cap.open(this->url);
                this->url="rtspsrc location=";
                this->url.append(url_);
                this->url.append(" latency=100 ! rtph264depay ! h264parse ! nvv4l2decoder !  nvvideoconvert ! video/x-raw , format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink$
//              this->url.append(" latency =100 ! nvvidconv ! video/x-raw(memory:NVMM),format=RGBA ! nvoverlaysink ");
                cout<<url<<endl;
                cap.open(this->url,CV_CAP_GSTREAMER);   
                if (url.find("http") != std::string::npos && url.find("@") != std::string::npos){
                        this->isCamera_ = true;
                        this->restart = true;
                }
                else{
                        this->restart = false;
                        this->isCamera_ = false;
                }

        }

I have also attached my old VideoManager part where i added the pipe along with this

class VideoManager{
public:
	VideoManager(){this->isCamera_ = false;}
	~VideoManager(){cap.release();}
	// WebCam
	void open(const int camera_id){
		cap.open(camera_id);
		this->restart = false;
		this->isCamera_ = true;
	}
	// RTSP or Video File
	void open(const std::string url_){
		this->url = url_;
	//	cap.open(this->url);
			cap(this->url);
		if (url.find("http") != std::string::npos && url.find("@") != std::string::npos){
			this->isCamera_ = true;
			this->restart = true;
		}
		else{
			this->restart = false;
			this->isCamera_ = false;
		}
	}
	// Retrieve all camera settings
	void retrieve(cv::Mat &frame){
		if (this->restart){
			cap.open(this->url);
			std::map<int, double>::iterator iter = this->camera_params.begin();
			while (iter != this->camera_params.end()){
				cap.set(iter->first, iter->second);
				iter++;
			}
		}
		cap >> frame;
	}
	// Set camera setting
	void set(int propId, double value){
		if (camera_params.find(propId) == camera_params.end())
			camera_params.insert(std::pair<int, double>(propId, value));
		else{
			camera_params[propId] = value;
		}
		this->cap.set(propId, value);
	}
	// Get camera setting
	double get(int propId){return this->cap.get(propId);}
	bool isOpened(){return cap.isOpened();}
	void release(){this->cap.release();}
	bool isCamera(){return isCamera_;}
	std::map<int, double> camera_params;
	std::string url;
	cv::VideoCapture cap;
	bool isCamera_;
private:
	bool restart;
};

I also tried changing the pipelines. But while compiling and running the code it gives a error’s as


<ul>
VIDEOIO ERROR: V4L: device rtspsrc location=rtsp://root:root@10.0.1.3:554/axis-media/media.amp?streamprofile=H264 latency=100 ! rtph264depay ! h264parse ! nvv4l2decoder !  nvvideoconvert ! video/x$
</ul>

<ul>
OpenCV(3.4.1) Error: Unspecified error (GStreamer: unable to start pipeline
) in icvStartPipeline, file /home/glueck/Downloads/opencv/modules/videoio/src/cap_gstreamer.cpp, line 450
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(3.4.1) /home/glueck/Downloads/opencv/modules/videoio/src/cap_gstreamer.cpp:450: error: (-2) GStreamer: unable to start pipeline
 in function icvStartPipeline
Aborted (core dumped)
</ul>

I have installed Gstreamer 1.14.8 and also have compiled it along with opencv 3.4.1 by -DWITH_GSTREAMER=ON -DWITH_LIBV4L=ON -DWITH_V4L=ON

I am stuck at this place where I have to merge the pipeline into my application. Can anyone suggest a way to run the pipeline for the GPU through the c++ code mentioned above.

Can anyone help me on this?

[/b]

Can you run the exact same gstreamer command line with gst-launch-1.0 as you are using in the opencv gstreamer command line?

Yes I can but cannot run when integrated with the application.

Hi,
Firstly, you need to use appsink instead of autovideosink, since opencv need to get frame from GStreamer pipeline, but autovideosink can’t support it.

After you change to appsink, if it still does not work, I think it may be related to your opencv installation. I tried below sample on Jetson which has OpenCV4 pre-installed, I can run the pipeline and capture the frame.

#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
using namespace cv;

#include <iostream>
using namespace std;

int main()
{
    std::string pipe = "rtspsrc location=rtsp://freja.hiof.no:1935/rtplive/definst/hessdalen03.stream ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! appsink";

    VideoCapture cap(pipe, CAP_GSTREAMER);

    if (!cap.isOpened()) {
        cerr <<"VideoCapture not opened"<<endl;
        exit(-1);
    }

    while (true) {
        Mat frame;

        cap.read(frame);

        imwrite("receiver.png", frame);

        getchar();
    }

    return 0;
}
1 Like

Thanks… the above script works!