How to get a metadata in python

vlukin · September 30, 2020, 1:51pm

• Hardware Platform (Jetson / GPU)
Jetson Nx
• DeepStream Version
5.0.0-1
• JetPack Version (valid for Jetson only)
4.4-b144

Hello,
I have a couple of questions regarding getting inference results (metadata) in python. I’m working with examples: https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/

Do I need to use nvdsosd for getting metadata? I see that it’s used in examples but seems like nvdsosd is needed mostly for drawing, not for getting metadata. It seems like I can take this metadata right from nvinfer but is it possible in python?
Is it possible to get additional attributes in python (for example, car color and vehicle type for networks in deepstream_test_2.py) ? If so, what is the preferred method for this?

vlukin · October 2, 2020, 5:18pm

I I looked through C examples and I found it in deepstream-test5.
There is no such an example in python bindings but you can see how it’s done in the schema_fill_sample_sgie_vehicle_metadata function.

Right now I modified ods_sink_bad_buffer_probe function:

    while l_obj is not None:
        try:
            # Casting l_obj.data to pyds.NvDsObjectMeta
            obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
            if (obj_meta.class_id == PGIE_CLASS_ID_VEHICLE):
                cls_obj = obj_meta.classifier_meta_list
                while cls_obj is not None:
                    try:
                        cls_meta=pyds.NvDsClassifierMeta.cast(cls_obj.data)
                        #print("NN inference ID", cls_meta.unique_component_id)
                        #gie-unique-id=2 in dstest2_sgie1_config.txt
                        if cls_meta.unique_component_id==2: #2 - carcolor, 3-carMake, 4-vehicleType
                            cls_meta_lbl = cls_meta.label_info_list
                            while cls_meta_lbl is not None:
                                try:
                                    # getting attributes (car color, carMake, vehicle type)
                                    """
                                    cls_meta_lbl_info=pyds.NvDsLabelInfo.cast(cls_meta_lbl.data)
                                    #print((cls_meta_lbl_info.result_label) #doesn't work (it gives an array)
                                    #print(pyds.get_string(cls_meta_lbl_info.result_label)) #doesn't work (error)
                                    #print(pyds.get_string(cls_meta_lbl_info.pResult_label)) #doesn't work (error)
                                    print("label id:", cls_meta_lbl_info.label_id) #doesn't work (always 0)
                                    print("n_classes:", cls_meta_lbl_info.num_classes) #doesn't work (always 0)
                                    print("result_class_id:", cls_meta_lbl_info.result_class_id) #works 
                                    print("result_prob:", cls_meta_lbl_info.result_prob) #works
                                    """
                                    cls_meta_lbl = cls_meta_lbl.next
                                except StopIteration:
                                    break

                        
                        cls_obj = cls_obj.next
                    except StopIteration:
                        break
                
        except StopIteration:
            break
        obj_counter[obj_meta.class_id] += 1
        try: 
            l_obj=l_obj.next
        except StopIteration:
            break

For some reason labels doesn’t work (see comments). Is it an expected behavior? Or there is some flaws in the code?

mjhuria · October 3, 2020, 7:53am

Hi,

Could you please also add a log for these errors?

aas5670 · October 6, 2020, 6:01pm

• Hardware Platform (Jetson / GPU)
Jetson Nano
• DeepStream Version
5.0
• JetPack Version (valid for Jetson only)
R32 (release), REVISION: 4.3

Sorry if I am hijacking the thread but thought it was appropriate. I have encountered the same issue with retrieving secondary inference results (specifically the result_label) from the metadata using python and haven’t been able to find a solution. I have modified the deepstream-test2 example from https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/ using the modifications (with slight adjustments) that that @vlukin provided to the ods_sink_bad_buffer_probe function. The log output can be found here metadata-06-10-20-18-36-trunc.txt (54.1 KB). The modified function can be found below:

    while l_obj is not None:
        try:
            # Casting l_obj.data to pyds.NvDsObjectMeta
            obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
            if (obj_meta.class_id == PGIE_CLASS_ID_VEHICLE):
                cls_obj = obj_meta.classifier_meta_list
                while cls_obj is not None:
                    try:
                        cls_meta=pyds.NvDsClassifierMeta.cast(cls_obj.data)
                        #print("NN inference ID", cls_meta.unique_component_id)
                        #gie-unique-id=2 in dstest2_sgie1_config.txt
                        if cls_meta.unique_component_id==2: #2 - carcolor, 3-carMake, 4-vehicleType
                            cls_meta_lbl = cls_meta.label_info_list
                            while cls_meta_lbl is not None:
                                try:
                                    # getting attributes (car color, carMake, vehicle type)

                                    cls_meta_lbl_info=pyds.NvDsLabelInfo.cast(cls_meta_lbl.data)
                                    print("result_label:", cls_meta_lbl_info.result_label) #doesn't work (it gives an array)
                                    try:
                                        print("result_label_pyds:", pyds.get_string(cls_meta_lbl_info.result_label)) #doesn't work (error)
                                    except Exception as e:
                                        print("result_label_pyds_ex:", e)
                                    try:
                                        print("result_label_ptr:", pyds.get_string(cls_meta_lbl_info.pResult_label)) #doesn't work (error)
                                    except Exception as e:
                                        print("result_label_ptr_ex:", e)
                                    print("label id:", cls_meta_lbl_info.label_id) #doesn't work (always 0)
                                    print("n_classes:", cls_meta_lbl_info.num_classes) #doesn't work (always 0)
                                    print("result_class_id:", cls_meta_lbl_info.result_class_id) #works
                                    print("r:sult_prob:", cls_meta_lbl_info.result_prob) #works

                                    cls_meta_lbl = cls_meta_lbl.next
                                except StopIteration:
                                    break
                        cls_obj = cls_obj.next
                    except StopIteration:
                        break
        except StopIteration:
            break
        obj_counter[obj_meta.class_id] += 1
        try:
            l_obj=l_obj.next
        except StopIteration:
            break

vlukin · October 6, 2020, 6:02pm

Logs are below

print(pyds.get_string(cls_meta_lbl_info.pResult_label))
log:

 File "deepstream_test_2.py", line 122, in osd_sink_pad_buffer_probe
    print(pyds.get_string(cls_meta_lbl_info.pResult_label)) #doesn't work
TypeError: get_string(): incompatible function arguments. The following argument types are supported:
    1. (arg0: int) -> str
Invoked with: None

print(pyds.get_string(cls_meta_lbl_info.result_label))
log:

Traceback (most recent call last):
  File "deepstream_test_2.py", line 123, in osd_sink_pad_buffer_probe
    print(pyds.get_string(cls_meta_lbl_info.result_label)) #doesn't work
TypeError: get_string(): incompatible function arguments. The following argument types are supported:
    1. (arg0: int) -> str
Invoked with: array( <huge array as in the log below> )

print(cls_meta_lbl_info.result_label)

output:

[1953065079        101          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0         10
          0 1065047953          0        117          0 -402489280
        126 -402489200        126 -402489200        126  924601824
          0        186          0          0          0  924601280
          0          0          0          0          0          0
          0          0          0          0          0        112
          0        565          0 -402482544        126          3
          0          0          0          0          0 2064861648
        127          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0          0          0          0          0
          0          0]

vlukin · October 6, 2020, 6:14pm

Wow.
Fast update:
I tried
print(cls_meta_lbl_info.result_label.tobytes())
and output is:

b'white\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x00\x02\xb1y?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u\x00\x00\x00\x00\x00\x00\x00\xb0p\x03$\x7f\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x06\x00\x00\x00\x01\x00\x00\x00\x06\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x06\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x00E\x00\x00\x00\x00\x00\x00\x00\x00d\x00$\x7f\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x88*F\xab\x7f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00E\x00\x00\x00\x00\x00\x00\x00@d\x00$\x7f\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x88*F\xab\x7f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00E\x00\x00\x00\x00\x00\x00\x00\x00?\x00$\x7f\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x88*F\xab\x7f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x005\x02\x00\x00\x00\x00\x00\x00\x80n\x00$\x7f\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

So, seems like I need to get bytes and then get everything until the first zero byte, like in C strings.
But I’m afraid that it will broke something: pyds.get_string must be there for a reason.

May be the safest way to get a readable label is to read a label index and then select a label (which can be found in the labels file)

aas5670 · October 6, 2020, 7:12pm

Following on from the tobytes(), I modified the result_label processing to the following:

cls_meta_lbl_info=pyds.NvDsLabelInfo.cast(cls_meta_lbl.data)
result_str = str(cls_meta_lbl_info.result_label.tobytes().decode(‘iso-8859-1’)) # Taken from https://stackoverflow.com/questions/56282563/how-to-decode-bytes-object-that-contains-invalid-bytes-python3
print(“result_decode”, result_str)
print(“result_strip:”, result_str.split(’\x00’))
print(“result_one:”, result_str.split(’\x00’)[0])
print("-----------------------------------------")

WIth this log output metadata-06-10-20-19-50.txt (765.3 KB)

mjhuria · October 17, 2020, 12:14am

Hello,
Thanks for providing the log. This is a known issue and fixed in DeepStream SDK 5.0.1
Please use the pyds.so from 5.0.1.

Thanks!

machalen · November 25, 2020, 5:22pm

Hi. I am using DeepStream SDK 5.0.1 and I have the same results mentioned before with that code. This is not resolved this way.

Please, could you help me.

kayccc · November 30, 2020, 8:02am

Hi machalen,

Please help to open a new topic for your issue. Thanks