How to Get Start on **Gst-python** for beginner to create custom plugins?

[elyaabbas9]

Hi hope you all doing good.
i am kind of beginner to Deep Stream and i have follow official doc so i have install TAO tool kit and trained model on custom data. and integrated those models with deep stream app and run through deepstream command.
I also run sample deepstream-python-app so now i am looking for creating plugins with gst-python . i also find two notebook on deepstream-python-app that also show some basic , i am also looking the official doc of Gstream python tutorials but my question is , the way i am following is right or here might be something i am missing that is very fundamental for learining gstream plugins for deep stream app with python

or any other resource for learning ?

Thanks

[yingliu]

Moved to DeepStream for better support.

[yuweiw]

1.If you are not familiar with the configuration of the environment, we recommend you use docker.
https://catalog.ngc.nvidia.com/containers?filters=&orderBy=scoreDESC&query=deepstream
2.When your env is set up, you can try to run the deepstream_test_1.py and referring it to learn how to use the plugin.
3.If you want to create custom plguins, you should use C/C++ to write a new plugin and bind it to python.
https://gstreamer.freedesktop.org/documentation/plugin-development/basics/index.html?gi-language=c

[elyaabbas9]

Thanks @yuweiw for reply.

i have already installed deep stream and tested its python sample 1 and 2 its working.

Sorry to say that i was asking about creating pipline as we have already build in plugins in C/C++ so we just learn GST-Python it that enough to make use of plugins by the help of Gst-python

?

[yuweiw]

Now that you have already build in plugins in C/C++, could you just try to use Gst.ElementFactory.make to build your own plguin in python?

[miguel.taylor]

@elyaabbas9 We have quite a bit of experience developing custom plugins on GstPython using DeepStream.

DeepStream

On DeepStream version 5.0 the python bindings are included with the DeepStream SDK installation. However, you still need to install the bindings for GStreamer and GObject and then run the install script in DeepStream:

cd /opt/nvidia/deepstream/deepstream-5.0/lib
sudo python3 setup.py install

For more information check the Python bindings API reference another excellent source is the deepstream_python_apps repo.

GStreamer

The GStreamer python bindings can be installed with apt:

sudo apt install python-gi-dev python-gst-1.0 gstreamer1.0-python3-plugin-loader

The plugin support in GStreamer is not stable in GStreamer 1.14.5. There is no documentation for creating elements, but your best bet is to check the examples in the GitHub repo. These bindings only provide structures to map GStreamer buffers and other elements in python, some of the functionalities and the support to create plugins. Most of the GStreamer objects and their methods are defined in the GObject bindings.

GObject

The documentation for the GStreamer modules of the GObject python bindings is available here. The API is similar to GStreamer on C, but some of the functionalities are missing.

Know Issues

These issues are specific to GStreamer 1.14 and I know for a fact that they are fixed on 1.16:

• Can’t map the input buffer for writting
• Installing multiple properties causes a segfault

I can provide you our workarounds for these issues if you are interested

Example
Finally here is an empty example plugin:

import gi
import os
import platform
import sys

gi.require_version('Gst', '1.0')
gi.require_version('GstBase', '1.0')
gi.require_version('GstVideo', '1.0')
gi.require_version('GLib', '2.0')

def is_aarch64():
    return platform.uname()[4] == 'aarch64'

sys.path.append('/opt/nvidia/deepstream/deepstream-5.0/sources/python/bindings/' +
                ('jetson' if is_aarch64() else 'x86_64'))

from gi.repository import Gst, GObject, GLib, GstBase, GstVideo

Gst.init(None)
FIXED_CAPS = Gst.Caps.from_string(
    'video/x-raw,format={ (string)RGBA, (string)I420 },width=[1,2147483647],height=[1,2147483647],framerate=[ 0/1, 2147483647/1 ]')
FIXED_CAPS.append(Gst.Caps.from_string(
    'video/x-raw(memory:NVMM),format={ (string)RGBA, (string)NV12 },width=[1,2147483647],height=[1,2147483647],framerate=[ 0/1, 2147483647/1 ]'))


class GstPycustom(GstBase.BaseTransform):
    __gstmetadata__ = (
        'PyCustom',
        'Generic',
        'Custom python element',
        'Miguel Taylor <miguel.taylor@ridgerun.com>')

    __gsttemplates__ = (
        Gst.PadTemplate.new(
            "src",
            Gst.PadDirection.SRC,
            Gst.PadPresence.ALWAYS,
            FIXED_CAPS),
        Gst.PadTemplate.new(
            "sink",
            Gst.PadDirection.SINK,
            Gst.PadPresence.ALWAYS,
            FIXED_CAPS))

    __gproperties__ = {}

    def __init__(self):
        GstBase.BaseTransform.__init__(self)
        GstBase.BaseTransform.set_in_place(self, True)

    def do_transform_ip(self, buf):
        Gst.debug("transform_ip")
        #custom operation
        return Gst.FlowReturn.OK

GObject.type_register(GstPycustom)
__gstelementfactory__ = ("pycustom", Gst.Rank.NONE, GstPycustom)

To install python plugins simply copy the source into /usr/lib/aarch64-linux-gnu/gstreamer-1.0/python/ or any other path in GST_PLUGIN_PATH. The sources must to be inside a folder named python and the element name must coincide with the file name, for example for pycustom the file name should be gstpycustom.py

I hope this is enough information to get you started.

[yuweiw]

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

We do not recommend that you write plug-ins in python, which will affect performance and is very difficult to locate the problem. If you use it only for debugging, you can refer to the following link:https://mathieuduponchelle.github.io/2018-02-01-Python-Elements.html