I have a requirement for running a classifier on the bottom 20% of a given bounding box inferred by my object detection model. I plan to run the object detection model as the PGIE and the classifier as the SGIE. What are my options to do this ?
You can customize the SGIE preprocess with Gst-nvdspreprocess (Alpha) — DeepStream documentation. The gst-nvdspreprocess source code graph may help you to understand the structure of nvdspreprocess library implementation DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums
I dont want to start with making changes to the nvdspreprocess plugin, since it would require time investment to learn how the plugin works and adding relevant changes to the code. Is there an alternative, I was thinking if we could add a probe to the SGIE input buffer and within the probe we make changes to the buffer. This would require only adding a probe function to my current python codebase.
You don’t need to change the code of nvdspreprocess plugin. The customization is done with the nvdspreprocess library. There are “custom-input-transformation-function” and “custom-tensor-preparation-function” configurations for you to customize your own preprocessing algorithm. Surely you need to know how the plugin works with nvinfer before you implement your own preprocessing algorithm with nvdspreprocess plugin. Gst-nvdspreprocess (Alpha) — DeepStream documentation
It is not easier than nvdspreprocess customization. You need to design your own way to transfer the processed data to the nvinfer and you need to change nvinfer to identify the data you send to it too. nvinfer and nvinferserver are all GStreamer in-place transform plugins. If you don’t tarnsfer the non-video data through DeepStream interfaces, you need to design you own way and customize all related DeepStream plugins to adapt to your way.
Ok, thanks for clarifying. Looking at the source code diagram given here DeepStream SDK FAQ , I should prepare a 'custom_transform ’ function I am guessing this relates to the “custom-input-transformation-function” parameter in the configuration file. Can you help me out with an example implementation of a ‘custom_transform’ function and I am guessing this will be a C code implementation.
Please refer to /opt/nvidia/deepstream/deepstream-7.1/sources/gst-plugins/gst-nvdspreprocess/nvdspreprocess_lib
I added my “custom-input-transformation-function” to /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/nvdspreprocess_lib/nvdspreprocess_lib.cpp . I ran make inside its parent folder /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess . Ran my application with custom-lib-path as /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/nvdspreprocess_lib/libcustom2d_preprocess.so and the original value /opt/nvidia/deepstream/deepstream/lib/gst-plugins/libcustom2d_preprocess.so . In both cases I am getting the following error
Error: gst-stream-error-quark: Custom Transformation function not found (1): gstnvdspreprocess.cpp(498): gst_nvdspreprocess_start (): /GstPipeline:2_1_1_2024-12-26T10-41-15/GstNvDsPreProcess:preprocess-plugin
Tell me how to rightly add a custom-input-transformation-function
The code is totally open source, please debug with your customized code.
You may also refer to deepstream_tao_apps/apps/tao_others/deepstream-pose-classification at master · NVIDIA-AI-IOT/deepstream_tao_apps for how to customized the nvpreprocess library. An empty customized trasformation function is implemented in this sample.deepstream_tao_apps/apps/tao_others/deepstream-pose-classification/nvdspreprocess_lib/nvdspreprocess_lib.cpp at master · NVIDIA-AI-IOT/deepstream_tao_apps, the nvpreprocess plugin configuration file is in deepstream_tao_apps/configs/nvinfer/bodypose_classification_tao/config_preprocess_bodypose_classification.txt at master · NVIDIA-AI-IOT/deepstream_tao_apps.
There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.