Gstreamer pipeline modification at runtime

I have two gstreamer issues im trying to solve:

  1. Modifying a running pipeline.
  2. Loss of timing data within a captured pipeline, this means I have a transmission and display/recording but the recorded file does not have the ability to seek in the file. It can play from start to finish in VLC.

To address 1, it appears that adding a capsfilter is the right way to handle this, but since its not a native gstreamer app I’m not sure if its possible in ocv4.4.

https://gstreamer.freedesktop.org/documentation/coreelements/capsfilter.html?gi-language=c

I’m not sure where to look to address 2.

Here are my pipelines in and out of ocv.

def gstreamer_pipeline_in():
return (

	"v4l2src device=/dev/video0 ! "
    "video/x-raw, format=BGRx ! "
"nvvidconv flip-method=2 ! "
"video/x-raw(memory:NVMM), format=(string)BGRx ! "
"nvvidconv ! "
"video/x-raw, format=(string)BGRx ! "
    "videoconvert ! "
    "video/x-raw, format=BGR ! "
    "appsink"        
)

def gstreamer_pipeline_out():
return (

	"appsrc ! "
    "video/x-raw, format=BGR ! "
    "videoconvert ! "
    "video/x-raw, format=BGRx ! "
    "nvvidconv ! "
    "omxh265enc ! "
    "rtph265pay pt=96 config-interval=1 ! "
    "udpsink host=127.0.0.1 port=8554"
    
)

Thanks for any support, even if its just a review of pipeline efficiency!

Hi,
In realtime UDP streaming, it may not be able to seeking the content. If you need the function, you should save it to a local file such as mp4 or mkv.

@DaneLLL. Thanks for the reply I should have been more clear. On the receive end I am saving to an mp4 via a matroska mux, but my saved MP4 doesn’t have any timing info. It plays back in vlc but doesn’t seek. I’m only trying to seek in the recording.

What I’d like to adjust in real time is the exposure, but v4l2 doesn’t allow me to modify the in pipe line once it’s running even if I call it as a totally separate process call to v4l2-ctl

Hi,
You should be able to seek mkv files. It is a bit strange it fails. Please check if you can seek the generated file:

$ gst-launch-1.0 videotestsrc num-buffers=600 ! nvvidconv ! nvv4l2h264enc ! h264parse ! matroskamux ! filesink location=a.mkv

Hey @DaneLLL,

I get the same problem and it is solved by adding -e in the running pipeline.
I saw it somewhere that it has something to do with abrupt stop when recording the mp4 file.
However I don’t know how to add it in python code using the python gst binding.
Also, I have similar question on modifying running pipeline, in my case I want to be able to start and stop the recording of the mp.4 file.

@andreaswidy thanks for the tip. I am using -e on my receive end. It the -e forces the End of Signal on the source which closes the file out cleanly. So it is required in my application but it isn’t allowing seeking. I’ll try record and playback local to the nano to see if that improves the behavior. If it does, then it points to something being lost when its sent over UDP. Maybe additional flags in the rtppay are required.

@andreaswidy @DaneLLL SUCCESS!!

I needed to combine both of your recommendations change to .mkv and ±e, I was using the -e option but I noticed my teed pipeline lacked the -e on the recording branch. Adding -e to the end worked! I need to read up on what’s in the MKV format.

This just leaves the realtime change to a pipeline to be solved.

I’ve tried running

v4l2-ctl -d 0 --set-ctrl exposure_absolute=10000

direct from the terminal and it shows the resource busy error. I think gstreamer is going to block unless the running process calls, or I can somehow reduce the priority of the call from python.

Hi @spacebaseone and @andreaswidy,

If you need to modify pipelines on runtime it is possible by using our free RidgeRun’s GStreamer Daemon (Gstd).

For example:

# Create the pipeline
$ pipeline_create testpipe videotestsrc name=vts ! autovideosink

# Play the pipeline
$ pipeline_play testpipe

# Change a property
$ element_set testpipe vts pattern ball

# Stop the pipeline
$ pipeline_stop testpipe

# Destroy the pipeline 
$ pipeline_delete testpipe

It is possible to change plugins properties as well. You can find some examples here:

We have an API for C/C++, Python, Javascript and HTTP requests.

Regards,
Fabian
www.ridgerun.com

@fabian.solano
I cheked the GST daemon and it looks promising, I can send the EOS.
However do you know how if I want the pipeline to be branched out using tee, one to rtsp and one to mp4 streaming.
But the mp4 streaming I need to be able to start and stop the recording at arbitrary time and increment the filenaming?
I think from the gst dameon i can stop and start the pipeline and then change the filename while it is stopped. but the rtsp stream will be stopped as well when i stop the pipeline, do you know how to solve for the use case?

Hi @andreaswidy,

You can use Gstd along with our other plugin called gst-interpipes. It is a more powerful kind of ‘tee’, You can stop the only section of the pipeline that is recording to mp4.

Here is an example with camera switching: GstInterpipe - Example 2: Digital Camera

Regards,
Fabian
www.ridgerun.com

@fabian.solano I’ve installed and tested the gstd. I am using opencv for image processing and currently bring the image data in through the gstreamer appsink. Any recommendations on the best method to connect my current gstreamer appsink to the running gstd pipeline? I was thinking from gstd I could either create a named pipe through a file or try v4l2-loopback to create a new device, then connect my opencv to the loopback device.

Thanks for any input.