VisionWorks Gstreamer and Opencv

Hello all,

I would like to implement gstreamer pipeline in order to get output of Visionworks sample main_opencv_npp_interop.cpp and pass it to RTP via using Opencv videowriter function. Yet, I do not know where I am missing through the process.

//initilization of videowriter
cv::VideoWriter writer("appsrc ! videoconvert ! video/x-raw,format=BGR,width=1920,height=480,framerate=30/1 ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=1234", 0, (double)30, cv::Size(3*cv_src1.cols, cv_src1.rows), true);
//main loop
renderer->putImage(demoImg);
cv::Mat bigframe_cv = nvx_cv::VXImageToCVMatMapper(demoImg, 0, NULL, VX_READ_AND_WRITE, VX_MEMORY_TYPE_HOST).getMat();
cv::imshow(“asd”,bigframe_cv);
cv::waitKey(1);
cv::Mat bigframe_cvBGR;
cv::cvtColor(bigframe_cv,bigframe_cvBGR,cv::COLOR_RGB2BGR);
writer << bigframe_cv;
I make the conversion for Opencv and show the frames in Opencv frame via imshow to get the sense of progress and sanity check.

By using gstreamer command line as below I try to get the output:(appsrc sends BGR but I would like to have with format h264 preferably)

gst-launch-1.0 -v udpsrc port=1234 \
 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! \
 rtph264depay ! decodebin ! videoconvert ! autovideosink

In addition, I get this warning repeatedly while sample is running:

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1629) writeFrame OpenCV | GStreamer warning: cvWriteFrame() needs images with depth = IPL_DEPTH_8U and nChannels = 3.

Hi,

The error indicates the format for videoWriter is incorrect.
Could you check the below issue to see if it helps first?

Thanks.

Hi AastaLLL,
thank you for the link.

I no longer have the warning below with the Opencv formating below:
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1629) writeFrame OpenCV | GStreamer warning: cvWriteFrame() needs images with depth = IPL_DEPTH_8U and nChannels = 3.

        `cv::Mat I(512,1536, CV_8UC3);`
        `cv::randu(I, cv::Scalar(0, 0, 0), cv::Scalar(255, 255, 255));`
        `cv::imshow("randu",I);`

my writer initialization as follows:

cv::VideoWriter writer(“appsrc ! videoconvert ! video/x-raw,format=BGR,width=1536,height=512,framerate=30/1 ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=1234”, 0, (double)30, cv::Size(3*cv_src1.cols, cv_src1.rows), true);

I would like to get it via but no success yet :

gst-launch-1.0 -v udpsrc port=1234 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink

Do you have suggestions how to get the stream via gst-launch?

Thank you

If you are using JPG encoding and rtpjpegpay, then your receiver pipeline is not correct.
Try instead something like:

gst-launch-1.0 -v udpsrc port=1234 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG, payload=(int)26" ! rtpjpegdepay ! jpegparse ! jpegdec ! videoconvert ! xvimagesink