Hi,
I understand the brief concept of Gstreamer pipeline and plugin, but I still don’t know how to develop my own deepstream plugin.
I’ve taken a look at gst-dsexample plugin but it doesn’t help much. Shall I start from Gstreamer plugin tutorial or any other recommandation?
Thanks for any suggestion!
Hi,
You may refer to the document to try/run deeptream-app first.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html
Sorry I forgot to mention. I can run deepstream sample already. I’ve tried objectDetector_Yolo and some sample apps like deepstream-test4 with kafka. I’ve also tried to modify the config file to make deepstream get sources stream from my own rtsp.
The thing I want to do is to make my own plugins like maybe some pre-process stuff on input frames, or post-process on the output metdata by the neural network.
Pre-processing frames and working with a metadata is what I’m intending on doing with this plugin below. It’s designed for you to use pure CUDA. If you want to use OpenCV, gst-dsexample is the way to go:
https://github.com/mdegans/gst-cuda-plugin/tree/dev?files=1
The dev branch is in the working state. It builds, installs, and has basic tests using GstCheck and GstHarness. It requires Meson (pip3 install meson) and Ninja (sudo apt install ninja-build) to build. I’m using Meson and Ninja because that’s what GStreamer uses. Also ninja is just about the fastest build system out there and scales to any project size. It’s used to build Chrome.
The idea is that you subclass BaseCudaFilter in the libdsfilter submodule and override one of the methods appropriate to your use case (batch, frame and/or objects). The preliminary documentation is in the headers and the base class has default implementations you’ll want to take a look at.
The API reference you will need is here.
I welcome any contributions to either the plugin, libdsfilter, or both. Make sure you initialize the sub module if you clone the plugin repo.
git submodule update --init
There’s also a test Dockerfile, however due to differences between Nvidia Docker on Tegra and x86 it only works on x86 for now. The project itself should build on any Nvidia platform however (untested on Tegra, however that is coming shortly).
Are there any pure CUDA preprocess/postprocess examples ? Like image frame manipulation with something like NPP ?
I hope to add some soon, but a social distancing project has me busy. In the meantime you might check out gst-dsexample. You can tear out the OpenCV parts where the comments tell you to. My recollection is in the latest DeepStream, It is now permissively licensed, but make sure you check that. I am away from my workstations right now so can’t double check.
I believe there’s also some work done on a cuda plugin in the freedesktop gitlab that uses npp, but a quick glance over it seems it may not be designed to work with DeepStream:
As soon as I get a chance I will update gst-cuda-plugin with examples. There is some WIP I haven’t committed yet. If it crosses paths with what I’m working on now it could be updated sooner rather than later. If and when I do, I will update this thread to let you know as well.