Rtsp server with gpu accelerating (pushing opencv mats)

Hello ,

After long efforts I have finally managed to run rtsp server by feeding opencv mats . But i couldn’t turn it for using gpu accelerator. How would I proceed to make it happen.
I am following https://developer.download.nvidia.com/embedded/L4T/r31_Release_v1.0/Docs/Accelerated_GStreamer_User_Guide.pdf?w5asL-pMqZ9IdsavPmEYuUUckNgBRRUcmOlklBpA0mt9giEpqBwgIaV1Vd9HCcZO6xu-qQlNpxVZ_Te1vpnVtKN5unX3sNDRZx-4mMi5jXncUWjdmBK4WKI0qjUN7W40gXhg5fl0Nc6LigovcAX4qQjfklDImSUq00JNnVKlPxajbqzkduo this guide to turn my pipeline but, couldn’t be succesful yet.
I am using gtx1060 for testing this, i will planning to use it in tx2 or xavier too.
These are my works for now.

// g++ test.cpp -o test `pkg-config --cflags --libs opencv4 gstreamer-1.0 gstreamer-rtsp-server-1.0`


#include <iostream>
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"

int main()
{
    cv::VideoCapture cap("/home/ibrahimdell/Videos/test.MOV");

    int fps = cap.get(cv::CAP_PROP_FPS);
    int w = cap.get(cv::CAP_PROP_FRAME_WIDTH);
    int h = cap.get(cv::CAP_PROP_FRAME_HEIGHT);
    cv::Size size(w,h);

    cv::VideoWriter out("appsrc ! videoconvert ! x264enc ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=5000",cv::CAP_GSTREAMER,0,fps,size,true);

    if(!cap.isOpened() || !out.isOpened())
    {
        std::cout<<"VideoCapture or VideoWriter not opened"<<std::endl;
        exit(-1);
    }

    cv::Mat frame;
    while(true) {
        cap.read(frame);
        if(frame.empty())
            break;
        out.write(frame);
        cv::waitKey(50);
    }
}

// gcc rtspServer.c -o server $(pkg-config --cflags --libs gstreamer-1.0 gstreamer-rtsp-server-1.0)
#include <gst/gst.h>
#include <gst/rtsp-server/rtsp-server.h>

// gst_rtsp_server_set_service()
int main (int argc, char *argv[])
{
    gst_init (&argc, &argv);

    /* make a mainloop for the default context */
    GMainLoop *loop;
    loop = g_main_loop_new (NULL, FALSE);

    /* create a server instance */
    GstRTSPServer *server;
    server = gst_rtsp_server_new ();
  
    /* get the mount points for this server, every server has a default object
    * that be used to map uri mount points to media factories */
    GstRTSPMountPoints *mounts;
    mounts = gst_rtsp_server_get_mount_points (server);
     
     guint64 udp_buffer_size = 512 * 1024;
    int updsink_port_num = 5000;
    char udpsrc_pipeline[512];
     sprintf (udpsrc_pipeline,
    "( udpsrc name=pay0 port=%d buffer-size=%lu caps=\"application/x-rtp, media=video, "
    "clock-rate=90000, encoding-name=H264, payload=96 \" )",
    updsink_port_num, udp_buffer_size);

    /* make a media factory for a test stream. The default media factory can use
    * gst-launch syntax to create pipelines. 
    * any launch line works as long as it contains elements named pay%d. Each
    * element with pay%d names will be a stream */
    GstRTSPMediaFactory *factory;
    factory = gst_rtsp_media_factory_new ();
    gst_rtsp_media_factory_set_launch (factory, udpsrc_pipeline);
    gst_rtsp_media_factory_set_shared (factory, TRUE);


    /* attach the video test signal to the "/test" URL */
    gst_rtsp_mount_points_add_factory (mounts, "/test", factory);
    
    /* don't need the ref to the mapper anymore */
    g_object_unref (mounts);

    /* attach the server to the default maincontext */
    gst_rtsp_server_attach (server, NULL);

    /* start serving */
    g_print ("stream ready at rtsp://127.0.0.1:8554/test\n");
    g_main_loop_run (loop);

}

Which part do you want to accelerate by HW? Why do you use opencv to read *.mp4 file while you decide to use gstreamer? I think the https://developer.download.nvidia.com/embedded/L4T/r31_Release_v1.0/Docs/Accelerated_GStreamer_User_Guide.pdf?w5asL-pMqZ9IdsavPmEYuUUckNgBRRUcmOlklBpA0mt9giEpqBwgIaV1Vd9HCcZO6xu-qQlNpxVZ_Te1vpnVtKN5unX3sNDRZx-4mMi5jXncUWjdmBK4WKI0qjUN7W40gXhg5fl0Nc6LigovcAX4qQjfklDImSUq00JNnVKlPxajbqzkduo has mentioned how to read *.mp4 video with gstreamer in page 4. Please dig into the whole document carefully.

Hello , @Fiona.Chen

It is my mistake to clarify which part to accelerate . Actually for the time being I am trying to use hw accelerating on making rtsp part.
appsrc ! videoconvert ! x264enc ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=5000 this part. I should use nvv4l2h264enc instead of x264enc but couldn’t able to write correct pipeline yet.

Please try appsrc ! videoconvert ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=5000