This is regarding appending additional information apart from the bounding box in custom bounding box parsing function.
I have implemented a custom bbox parser library for my detector and its working well.
As this object detector along with bounding box provides five-point landmarks but currently there no mechanism in Deesptrream to pass this data along with bounding box information to elements downstream.
Please suggest some pointers on what could be the best way to implement the same.
Is it need to be done in nvdsinfer , or it could be achieved somehow by adding probe to sink pad of gst-nvinfer ??
You can create your own metadata to save the customized information.
Please find /opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/deepstream-user-metadata-test for more information.
Thanks for your reply. This metadata is object level and mapping of the object bounding box and corresponding metadata(the 5 landmark point) is only available inside detectors parser bounding box callback implementation via NvDsInferLayerInfo (std::vector const &outputLayersInfo )
Now It seems I cannot add Object level user meta from inside my implementation of CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE as a pointer to GstBuffer can not be derived from inside custom parse function,
And suppose if I want to do it like explained in opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/deepstream-user-metadata-test , to do it externally from the pipeline by adding probe and appending the USEr Meta then also it’s not possible as the detector output layer information is not available here so that I can map for which object what is the metadata.
One option could be to change in nvds_infer NvDsInferParseCustomFunc to take additional metadata along with bounding box, and then inside nvds_infer it can be added as user meta.
I have the same problem. I’m trying to get the detection’s confidence from the custom bbox parser and use them to define the obj_meta->confidence (in “attach_metadata_detector” function in file : “gst-nvinfer/gstnvinfer_meta_utils.cpp”) which is equal to zero by default.
Is there any way to access NvDsInferObjectDetectionInfo object from “attach_metadata_detector” function ??
For getting detection confidence you have to parse your layerInfo (NvDsInferLayerInfo) and then set that value in NvDsInferObjectDetectionInfo object’s detectionConfidence member in your custom parser.
Please see other parserbbox implementation for more clarity.
The issue which I am facing is If anything other than bounding box and confidence is generated by the output layer of the detector there is no direct way to append that information so that downstream elements can use it.
I’m using “/deepstream-4.0/sources/objectDetector_Yolo/nvdsinfer_custom_impl_Yolo/nvdsparsebbox_Yolo.cpp” as a custom parser which does exactly what you said. But the problem is in “attach_metadata_detector” function in file : “gst-nvinfer/gstnvinfer_meta_utils.cpp” #Ln83 :
obj_meta->confidence = 0.0;
How can I change this to get the confidence calculated before in “nvdsparsebbox_Yolo.cpp” ??
Hi anassamar8,
Your issue is different from the issue discussion happening in this post.
Its better if you can raise separate post , We can continue discussion of your issue in that post
The issue disscused here is the support of DNN which have additional output apart from bounding box.
Regards
Pallab Sarkar
Is tensor meta test example (/sources/apps/sample_apps/deepstream-infer-tensor-meta-test/) will be useful
for handling DNN network which outputs additional information apart from object bounding box ??
Using tensor meta example and user meta example done changes in deepstream-app and dsexample to flow additional output(apart from bounding box) from my network to the dsexample plugin.