Missing unique_id in NvDsInferSegmentationMeta for DeepStream Python Bindings

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

• Hardware Platform (Jetson / GPU) : NVIDIA Jetson AGX Orin
• DeepStream Version : 7.1
• JetPack Version (valid for Jetson only) : 6.1
• TensorRT Version : 8.6.2.3
• Issue Type( questions, new requirements, bugs) : question
Hello,

I am working with a DeepStream inference pipeline in Python with the following setup:

nvstreammux -> queue0 -> PGIE0 -> queue1 -> PGIE1 -> fakesink

Both PGIE0 and PGIE1 are segmentation models. I have attached a probe function to fakesink to process the inference results from both models. It looks like this:

 pyds.nvds_acquire_meta_lock(batch_meta)

    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        # Process all user metadata to collect masks
        l_user = frame_meta.frame_user_meta_list
        while l_user is not None:
            try:
                user_meta = pyds.NvDsUserMeta.cast(l_user.data)
            except StopIteration:
                break

            if (
                user_meta
                and user_meta.base_meta.meta_type == pyds.NVDSINFER_SEGMENTATION_META
            ):
                try:
                    segmeta = pyds.NvDsInferSegmentationMeta.cast(
                        user_meta.user_meta_data
                    )
                except StopIteration:
                    break

                # Retrieve mask data in the numpy format from segmeta
                mask = np.array(
                    pyds.get_segmentation_masks(segmeta),
                    copy=True,
                    order="C",
                    dtype=np.float32,
                )

                unique_id = segmeta.unique_id
                if unique_id == 0: 
                    mask_edge = np.squeeze(mask)
                elif unique_id == 1:
                    mask_line = np.squeeze(mask)
                else:
                    logger.warning(f"Unknown unique_id")

            try:
                l_user = l_user.next
            except StopIteration:
                break
        try:
            l_frame = l_frame.next
        except StopIteration:
            break

    pyds.nvds_release_meta_lock(batch_meta)

To differentiate between them, I attempted to use the unique_id attribute in NvDsInferSegmentationMeta. However, I encountered the following error:

AttributeError: 'pyds.NvDsInferSegmentationMeta' object has no attribute 'unique_id'

I found a related discussion on the NVIDIA forum, but it appears that the issue remains unresolved. Upon reviewing the DeepStream Python bindings source code on GitHub NvDsInferSegmentationMeta definition, I realized that NvDsInferSegmentationMeta is missing the unique_id attribute, even though it is documented in the official DeepStream API reference DeepStream documentation. However in PyDs documentation unique_id property is not visible.

Is this a known issue, and is there a fix available? Would adding the following line in the Python bindings resolve the problem?

.def_readonly("unique_id", &NvDsInferSegmentationMeta::unique_id)

Is there any other way to determine from which segmentation model has user_meta came from?

Any insights or potential workarounds would be greatly appreciated.

4

I think this is a solution, or you can add probe function to the src pad of pgie0/pie1 respectively, which is more recommended, because sometimes meta will be released in the pipeline, and it is possible that the downstream element will not get the relevant metadata

5

@junshengy thank you for your response.

In my case, I need to process the outputs from both PGIE0 and PGIE1 within the same probe function. This is because, during post-processing, I extract the masks from both models and perform additional calculations that require both masks simultaneously.

However, without the unique_id property in NvDsInferSegmentationMeta, I am unable to distinguish between the two models, making it impossible to determine whether an incoming mask originates from PGIE0 or PGIE1.

Adding separate probe functions to PGIE0 and PGIE1 does not solve the issue, as I need both masks together for further processing. Is there an alternative approach or a workaround to differentiate between the models’ outputs? Is it possible to attach some additional data in probe functions of PGIE0 and PGIE1 that will later in sink probe function be extracted?

6

For your application, it may be quickest to simply add .def_readonly("unique_id", &NvDsInferSegmentationMeta::unique_id).

In addition, I think using the Python built-in function id(segmeta), distinguishing python object ids can also achieve similar goals.

7

Adding this line

.def_readonly("unique_id", &NvDsInferSegmentationMeta::unique_id)

to file bindnvdsinfer.cpp in deepstream_python_apps and compilation of this file resolved the issue.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.