Gstreamer (python) with appsrc need-data is called more times than FPS is set

[hlacik]

Hi guys,

I do use appsrc which grabs image from redis.
This pipeline is used in video , and therefore I want appsrc to grab image only 30times a second.
Even though I have in CAPS framerate 30/1 , need-data callback calls function at least 80times/second making my compute resources on jetson nano exhausted. Why is that and how can I fix it?

This is my appsrc element

 appsrc = Gst.ElementFactory.make("appsrc", "source")
    appsrc.set_property("format", Gst.Format.TIME)
    appsrc.set_property("do-timestamp", True)
    appsrc.set_property("is-live", True)

    appsrc.set_property(
        "caps",
        Gst.Caps.from_string(
            "video/x-raw,width=(int){width},height=(int){height},format=(string){format},framerate={framerate}/1".format(
                width=frame.shape[1],
                height=frame.shape[0],
                format=FLAGS.format,
                framerate=30,
            )
        ),
    )
    appsrc.connect("need-data", get_frame_buffer, (db, FLAGS.input_key))

and this is my get_frame_buffer function which just grabs image, converts it to bytes and then wraps it to GstBuffer

# python callback for the 'need-data' signal
def get_frame_buffer(appsrc, length, udata):
    tic = time()
    # grab frame from basler
    frame = get_frame(*udata)
    if frame is None:
        appsrc.emit("end-of-stream")
    else:
        appsrc.emit("push-buffer", Gst.Buffer.new_wrapped(frame.tobytes()))
    toc = time()
    logging.debug("Framerate: {:.1f} fps".format(1 / (toc - tic)))

thanks in advance

[hlacik]

I mean, I am aware that i could do like blocking sleep in my get_frame_buffer function

# python callback for the 'need-data' signal
def get_frame_buffer(appsrc, length, udata):
    tic = time()
    # grab frame from basler
    frame = get_frame(*udata)
    if frame is None:
        appsrc.emit("end-of-stream")
    else:
        appsrc.emit("push-buffer", Gst.Buffer.new_wrapped(frame.tobytes()))
        # sleep for the desired fps
        sleep(1 / FLAGS.fps - (time() - tic))
    logging.debug("Framerate: {:.1f} fps".format(1 / (time() - tic)))
    return Gst.FlowReturn.OK

but i believe there should be more reasonable way

[DaneLLL]

Hi,
We don’t have experience about running this in python. Have tried this C sample and it works fine:
gst-rtsp-server/test-appsrc.c at 1.14.5 · GStreamer/gst-rtsp-server · GitHub

Not sure if it helps, but please set timestamp to GstBuffer for a try.

[hlacik]

Hello @DaneLLL ,
I have modified my appsrc to similar behavior as in shared code

# python callback for the 'need-data' signal
def get_frame_buffer(appsrc, length, udata):

    # grab frame from basler
    frame = get_frame(*udata)
    if frame["frame"] is None:
        appsrc.emit("end-of-stream")
    else:
        # convert frame["frame"] to buffer and add frame["timestamp"] to metadata
        buf = Gst.Buffer.new_wrapped(frame["frame"].tobytes())
        buf.pts = frame["timestamp"]
        # push buffer to appsrc
        appsrc.emit("push-buffer", buf)

    return Gst.FlowReturn.OK

aka adding timestamp to each buffer, and to be precise adding timestamp from frame as it was captured in camera. However it is not changing gstreamers behaviour , still pooling that get_frame_buffer as crazy :)

Hope someone will see this and be able to help.

[DaneLLL]

Hi,
Search online and find a sample similar to this use-case:
send opencv frame over gstreamer pipe using python · GitHub

You may try it and check if the issue is observed. If it happens in both you sample and this sample, it may be a bug in python gstreamer.

[DaneLLL]

Hi,
The issue look specific to python(python 2). Please try python3.