How does a custom rotated box model work or how to customize the model output type in DeepStream?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU) Jetson nx
• DeepStream Version 6.0
• JetPack Version 4.6
• TensorRT Version 8001
Hello mate,
I have a custom rotating box detection model, which has been verified to work properly on the NX platform. Now, I want to write a parser following the examples and integrate it into the DeepStream pipeline.
The current issue is that all parser examples use the NvDsInferObjectDetectionInfo type for detection boxes. For rotating boxes, I need to add a rotation angle, theta, based on this type.
I’ve already tried adding theta to the definition of NvDsInferObjectDetectionInfo (/opt/nvidia/deepstream/deepstream-6.0/sources/includes/nvdsinfer.h), but the program doesn’t work as expected.
How should I proceed to customize a model output type within the DeepStream process? I am using DeepStream version 6.0.

3

You can add a cuatomized user meta in the object meta to contain the rotation information. There is “obj_user_meta_list” in NvDsObjectMeta.

You can refer to deepstream_tao_apps/apps/tao_others/deepstream-faciallandmark-app/deepstream_faciallandmark_meta.cpp at release/tao4.0_ds6.3ga · NVIDIA-AI-IOT/deepstream_tao_apps (github.com) for how to add new customized user meta to the object meta.

Please do not change NvDsInferObjectDetectionInfo since it will be used by many DeepStream modules which are not open source.

4

Thank you for your response. I’ve noticed that the Python SDK supports custom struct binding from version 6.2 onwards, and I believe this could solve the issue of passing rotated bounding box data.

Unfortunately, I’m constrained to using version 6.0. I’d like to know if your method is dependent on the DeepStream version? Is it exclusive to C++ or can it also be utilized with Python?

5

No. No limitation.

It can be implemented in python too. We have sample for how to customize new metadata python binding. https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/master/bindings/CUSTOMUSERMETAGUIDE.md

6

Thank you very much, this method is effective.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.