Unable to Get Segmentation Masks Values and Colors

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) A10
• DeepStream Version 6.2
• Issue Type( questions, new requirements, bugs) Questions?
Hi, I was working on getting the Nvidia Deepstream for Instance Segmentation, I am unable to fetch the mask values and unable to change the color, While fetching the mask values I referred

  1. deepstream_python_apps/apps/deepstream-segmentation/deepstream_segmentation.py at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub
  2. deepstream_python_apps/apps/deepstream-segmask at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub
  3. GitHub - marcoslucianops/DeepStream-Yolo-Seg: NVIDIA DeepStream SDK 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 implementation for YOLO-Segmentation models
    I am getting this error
    Traceback (most recent call last):
    File “main.py”, line 986, in tracker_src_pad_buffer_probe
    masks_meta = pyds.NvOSD_MaskParams.get_mask_array(l_obj.data)
    AttributeError: type object ‘pyds.NvOSD_MaskParams’ has no attribute ‘get_mask_array’

Could anyone help me out on this.

Did you use the right version of v1.1.6 for DeepStream 6.2?

I am using v1.1.8, does it not support for deepstream 6.2?
I did change it to v1.1.6 I am still getting this error
deepstream | 2024-01-24T11:59:29.501322547Z Traceback (most recent call last):
deepstream | 2024-01-24T11:59:29.501372387Z File “main.py”, line 1055, in osd_sink_pad_buffer_probe
deepstream | 2024-01-24T11:59:29.501709395Z masks = pyds.NvOSD_MaskParams.get_mask_array(mask_data)
deepstream | 2024-01-24T11:59:29.501739345Z AttributeError: type object ‘pyds.NvOSD_MaskParams’ has no attribute ‘get_mask_array’
Here is mask part in my Code
batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))

    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)
        except StopIteration:
            break
        l_user = frame_meta.frame_user_meta_list
        frame_number = frame_meta.frame_num
        l_obj = frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
                mask_data = pyds.NvOSD_MaskParams.cast(l_obj.data)
                # masks = mask_data.get_mask_array(mask_data)
                masks = pyds.NvOSD_MaskParams.get_mask_array(mask_data)
                print(masks)
                # obj_meta.rect_params.has_bg_color = 1
                # obj_meta.rect_params.bg_color.set( 0.0, 1.0, 0.0, 0.2 )
            except StopIteration:
                error("Iteration Stopped from OnScreenDisplay due to an Exception in masks values")
                break
            # obj_meta.rect_params.border_color.set(0.0, 0.0, 1.0, 0.0)
            try:
                l_obj = l_obj.next
            except StopIteration:
                break

Adding to the above code
for print(dir(mask_data), dir(pyds.NvOSD_MaskParams))
I am getting this

deepstream  | 2024-01-24T12:08:01.923371068Z ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'cast', 'data', 'height', 'size', 'threshold', 'width'],

 ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'cast', 'data', 'height', 'size', 'threshold', 'width']

According to the source code it should have this function right:

py::class_<NvOSD_MaskParams>(m, "NvOSD_MaskParams",
                                     pydsdoc::NvOSD::NvOSD_MaskParams::descr)
                .def(py::init<>())
                .def_readwrite("data", &NvOSD_MaskParams::data)
                .def_readwrite("size", &NvOSD_MaskParams::size)
                .def_readwrite("threshold", &NvOSD_MaskParams::threshold)
                .def_readwrite("width", &NvOSD_MaskParams::width)
                .def_readwrite("height", &NvOSD_MaskParams::height)

                .def("get_mask_array", 
                     [](NvOSD_MaskParams &self) -> py::array

NO. There are no input parameters for the binding API. But you used that with a input parameter. Please refer to our use case: deepstream_segmask.py.

Are there any formulations to extract the Pixel values of mask, in a 1280, 720 resolution image and mask output tensor dimension from the enigne is 100, 160, 160

1.You can just get the mask data by referring the deepstream_segmask.py.
2. You can use some formulations of OpenCV to convert the data to 1280x720.

Would that give me corresponding pixel values for each of data

Yes. The pixel values are stored in the numpy struct.