Populate NvDsObjectMeta with added landmarks field in NvDsInferObjectDetectionInfo

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform: RTX 2080
• DeepStream Version: 5.0
• NVIDIA GPU Driver Version (valid for GPU only): 440.33.01

Hi,
I have a test case where I run face detection model and populate corresponding detection Info followed by a face recognition model which give out some fixed c feature. I have added a field of float landmarks[c] in NvDsInferObjectDetectionInfo(nvdsinfer.h) and changed the custom parse function so that the landmarks[c] is pushed into the objectList as suggested Face detection with deepstream with landmarks - #16 by gmgmishra94 .

I added a field of float landmarks[c] in NvDsObjectMeta struck(nvdsmeta.h). But how do I populate the landmarks[c] field in objectmeta ?
I later want to access the field landmarks[c] in object meta and modify by the values given by face recog model.

3

@duttaneil16

We are now evaluating feasibility of your solution.
I will reply with our suggestions soon. Thank you for your patience.

4

@duttaneil16

The struct NvDsInferObjectDetectionInfo is a type shared by both NvInfer and NvInferServer.
Addtionally, NvDsObjectMeta is a general type that is probably shared by the entire DeepStream pipeline.

Adding new fields to these types will cause size and address changes in fields of these objects, which may cause undefined behavor.
I think a better way is to extend list type fields in NvDsObjectMeta with your own information. For example, you can extend NvDsClassifierMetaList or NvDsObjectMetaList in NvDsObjectMeta.

Similarly, as I know NvDsInferObjectDetectionInfo is usually handled as a list like std::vector<NvDsInferObjectDetectionInfo>, is it possible to extend the list with your own specific data without changing field definitions in NvDsInferObjectDetectionInfo itself?

6

Hi @ersheng,
I removed those extra fields from NvDSInferObjectDetectionInfo and NvDsObjectMeta. Now I am retrieving the feature vector given by FR(output_tensor_meta) and filling NvDsUserMeta and attaching it to NvDsObjectMeta. This works fine in this case where I need to add one field to my metadata(landmarks) and even add those in extMsg field for msgbroker.
It will be quite helpful if there is some way to define custom objects and their metadata and different custom objType(and corresponding sub-structure) in the present or future versions of deepstream.

Thanks.