Obj_user_meta_list is erased after nvvideoconvert and is not populated by a sgie after that

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 5.1
**• JetPack Version (valid for Jetson only)**4.5.1
• Issue Type( questions, new requirements, bugs) question

I have a system where a frame is processed by nvinfer PGIE (detector), then 1st nvinfeserver SGIE is run on detections of one specific class from PGIE. I use a probe to get this sgie’s results. Then I have a nvvideoconvert to convert frame to RGBA as I need access to frame data and NV12 is not supperted by pyds.get_nvds_buf_surface. I get the frame in a probe before 2nd nvinfeserver sgie and use 1st sgie’s output to modify (rotate) the frame data. At this point, if I check ‘obj_user_meta_list’, it is always None, but e.g. ‘class_id’ is present. Then I run 2nd nvinfeserver sgie on the frame, and try to intercept its output in a probe pretty much like with 1st sgie, but at this point obj_user_meta_list is also always None. I tried removing the nvvideoconvert from the pipeline, it would break the access to frame data because it is not RGBA now, but just before this step I get all the obj_user_meta_list from 1st sgie. It seems that after running nvvideoconvert, obj_user_meta_list from 1st gie is removed and not added by 2nd sgie afterwards. At the same time other object info like rect_params or class_ids remain intact. How can I keep it there?

Suppose nvvideoconvert will keep all meta data. Will check internally and feedback.

Can you share the detail of which meta data? Can you check if this meta data implement meta data copy function?

https://docs.nvidia.com/metropolis/deepstream/4.0/dev-guide/DeepStream_Development_Guide/baggage/group__metadata__structures.html#gad20590cd18e480d2c42d67d95dab6939

These metadata are created by nvinferserver plugins, and they are output tensors of the neural networks. I do not know whether nvinferserver internally uses this function or not.

If you found the meta data copy function for your montioned meta data in below link, nvvideoconvert will copy the meta data when convert.

https://docs.nvidia.com/metropolis/deepstream/4.0/dev-guide/DeepStream_Development_Guide/baggage/group__metadata__structures.html#gad20590cd18e480d2c42d67d95dab6939

where should I have found it? tensormeta is created by nvinferserver, I have no control over what is going on there. nvvideoconvert does not copy the meta data when convert.

I will check with it internally and feedback later.

Conformed it is a issue on DS 5.1. The issue is fixed on DS 6.0 GA. Please try it on DS 6.0 GA when the DS 6.0 GA is available. Thanks for reporting the issue.