qt-gstreamer dynamic pipeline

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

I am using qt-gstreamer 1.2.1 on jetson TX1 with ubuntu 14.04 and Qt 5.2.1.

My application is to preview camera feed with Record / Snapshot / ETH Streaming / Wi-Fi Streaming. any function can be start/stop while preview is running.

I am using tee for different function like record or streaming.
Below is my code for preview with Ethernet Streaming Start/Stop functions.

'                                                     queue1 ! nvvideosink
                                                     /
gst-launch-1.0 v4l2src ! caps ! nvvidconv ! caps ! tee ! queue2 ! fakesink
                                                     \
                                                      queue 3 ! omxh264enc ! h264parse ! rtph264pay ! udpsink

void gstPipe::init()
{
    m_pipeline  =   QGst::Pipeline::create();
    m_source    =   QGst::ElementFactory::make("v4l2src", "v4l2src");
    m_filter1   =   QGst::ElementFactory::make("capsfilter", "capsfilter");
    m_convert   =   QGst::ElementFactory::make("nvvidconv", "videoconvert");
    m_filter2   =   QGst::ElementFactory::make("capsfilter", "capsfilter1");
    m_filter3   =   QGst::ElementFactory::make("capsfilter", "capsfilter2");
    m_tee       =   QGst::ElementFactory::make("tee", "tee");
    m_enc       =   QGst::ElementFactory::make("omxh264enc", "omxh264enc");
    m_parse     =   QGst::ElementFactory::make("h264parse", "h264parse");
    m_queue1    =   QGst::ElementFactory::make("queue", "queue1");
    m_queue2    =   QGst::ElementFactory::make("queue", "queue2");
    m_queue3    =   QGst::ElementFactory::make("queue", "queue3");
    m_rtppay1   =   QGst::ElementFactory::make("rtph264pay", "rtph264pay");
    m_udpsink   =   QGst::ElementFactory::make("udpsink", "udpsink");
    m_videoSink =   QGst::ElementFactory::make("nvoverlaysink", "sink");
    m_fakesink  =   QGst::ElementFactory::make("fakesink", "fakesink");

    m_filter1->setProperty("caps", QGst::Caps::fromString("video/x-raw, width=1920, height=1080, format=UYVY, framerate=60/1"));
    m_filter2->setProperty("caps", QGst::Caps::fromString("video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=60/1"));
    m_filter3->setProperty("caps", QGst::Caps::fromString("video/x-h264, stream-format=(string)byte-stream"));

    m_videoSink->setProperty("sync", false);
    m_videoSink->setProperty("async", false);
    m_videoSink->setProperty("overlay-x", 0);
    m_videoSink->setProperty("overlay-y", 35);
    m_videoSink->setProperty("overlay-w", 1920);
    m_videoSink->setProperty("overlay-h", 1080);

    m_pipeline->setProperty("video-sink", m_videoSink);
    m_videoSink->setProperty("enable-last-sample", true);
    m_enc->setProperty("bitrate", 19000000);
    m_enc->setProperty("low-latency", 1);
    m_enc->setProperty("control-rate", 2);
    m_rtppay1->setProperty("mtu", 1400);

    m_udpsink->setProperty("port", 8554);
    m_udpsink->setProperty("sync", false);
    m_udpsink->setProperty("async", false);
    m_udpsink->setProperty("host", "127.0.0.1");

    m_pipeline->add(m_source, m_filter1, m_convert, m_filter2, m_tee, m_videoSink);
    m_pipeline->add(m_queue1, m_queue2);
    m_pipeline->add(m_fakesink);

    m_source->link(m_filter1);
    m_filter1->link(m_convert);
    m_convert->link(m_filter2);
    m_filter2->link(m_tee);

    m_tee->link(m_queue1);
    m_queue1->link(m_videoSink);

    m_tee->link(m_queue2);
    m_queue2->link(m_fakesink);

    // Connect to pipeline's bus
    QGst::BusPtr bus = m_pipeline->bus();
    bus->addSignalWatch();
    QGlib::connect(bus, "message", this, &gstPipe::onBusMessage);
    bus.clear();
}

void gstPipe::startETH()
{
    m_pipeline->setState(QGst::StatePaused);

    m_pipeline->add(m_queue3, m_enc, m_filter3, m_parse, m_rtppay1, m_udpsink);

    m_tee->link(m_queue3);
    m_queue3->link(m_enc);
    m_enc->link(m_filter3);
    m_filter3->link(m_parse);
    m_parse->link(m_rtppay1);
    m_rtppay1->link(m_udpsink);

    m_pipeline->setState(QGst::StatePlaying);
}

void gstPipe::stopETH()
{
    m_queue3->sendEvent(QGst::EosEvent::create());

    m_pipeline->setState(QGst::StatePaused);

    m_queue3->setState(QGst::StateNull);
    m_enc->setState(QGst::StateNull);
    m_filter3->setState(QGst::StateNull);
    m_parse->setState(QGst::StateNull);
    m_rtppay1->setState(QGst::StateNull);
    m_udpsink->setState(QGst::StateNull);

    m_tee->unlink(m_queue3);

    m_pipeline->remove(m_queue3);
    m_pipeline->remove(m_enc);
    m_pipeline->remove(m_filter3);
    m_pipeline->remove(m_parse);
    m_pipeline->remove(m_rtppay1);
    m_pipeline->remove(m_udpsink);

    m_pipeline->setState(QGst::StatePlaying);
}

This works fine. I can dynamically start/stop Ethernet streaming.
But after several start/stop event some times pipeline hangs on start event. If i used single tee like in posted code it takes 10-15 mins to occur this problem. But if i added multiples tee for all recording/ snapshot and other pipeline hangs after 3-5 min.
Note: This issue occurs only on start/stop event. If i continue to event without start/stop for several hours its work fine.

So what could be the issue?
On some forum i found need to use Pads to add/remove element dynamically.
So Do i need to use pads and its blocking property or something is missing in my code?

2

Hi RiteshPanchal,
Are you on r24.2.1?
We don’t have much experience in using qt-gstreamer 1.2.1. Is it based on gstreamer 1.2.1?
In your implementation, there are three queues running simultaneously. Does the issue also happen if you run only one queue at one time?

3

Single queue doesn’t creating the issue. Adding or removing another queue simultaneously creating this issue…

4

Everytime when Gstreamer Pipeline hangs i am getting QGst::MessageError error…

"No valid frames found before end of stream"
5

Hi RiteshPanchal,
Please try to put ‘nvtee’ between queue and NV components(omxh264enc, nvoverlaysink) and check whether it helps.
Command for reference:

gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=90 ! 'video/x-raw,width=1280,height=720' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! tee name=t t. ! queue ! nvtee ! nvoverlaysink t. ! queue ! nvtee ! omxh264enc ! filesink location=a.264

If it doesn’t help, please share a method to reproduce it on open-source app nvgstcapture-1.0 on r24.2.1. If we can reproduce it, we can check further.

6

Hi RiteshPanchal,

Have your clarified and fixed this issue?
Any further update?

Thanks

7

Actually i found it’s Pausing/Playing the pipeline which creating the issue. Not Adding or removing the elements from the pipeline.

Actually in my pipeline if i add all tee and queue for Recording, Eth & Wifi Stream at init. and just PAUSE and PLAY the pipeline on key event after some trials pipeline hangs.

So I added all elements at init and figure out a way to dynamically change property of element so that it start or stop recording/streaming without pausing the pipeline.

Its a temporary solution.
But according to Gstreamer forum i need to use PADs and its blocking capabilities to dynamically add/remove elements from pipeline so i can decrease load on TX1 while recording/streaming is not started ON.