Getting Segmentation Mask Coordinates from Secondary GIE in DeepStream Message Schema

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU): Jetson Orin NX

• DeepStream Version: 7.1 (running in multiarch Docker container)

• JetPack Version (valid for Jetson only): 6.2.1

• TensorRT Version: [You’ll need to check this - run dpkg -l | grep tensorrt in your container]

• NVIDIA GPU Driver Version (valid for GPU only): N/A (Jetson platform)

• Issue Type( questions, new requirements, bugs): Questions

Setup:

  • Primary GIE: Object detection model

  • Secondary GIE: Segmentation model (operates on class ID 0 detections from primary GIE)

  • Message converter: Using msg-conv-payload-type=1 (minimal schema)

  • Output: Kafka messages via libnvds_msgconv.so

Current Status: I’m successfully getting object detection results through the DeepStream message schema, but I’m only receiving bounding box coordinates and classification data. The secondary segmentation GIE is running and processing the cropped objects from the primary GIE detections.

What I Need: I want to include the segmentation mask coordinates or contour points generated by the secondary GIE in my Kafka message output. Currently, the message schema only contains the primary GIE results (bounding boxes, classes, confidence scores).

Question: Does the built-in DeepStream message converter (libnvds_msgconv.so) with minimal schema support segmentation metadata extraction, or do I need to implement a custom message schema to access the mask/contour coordinates from the secondary GIE output?

Configuration Details:

  • Secondary GIE config: network-type=3, output-tensor-meta=1

  • Message converter: msg-conv-payload-type=1

  • Both GIEs are processing successfully based on console output

Any guidance on whether this requires custom schema development or if there’s a configuration approach I’m missing would be appreciated.

3

which sample are you testing or referring to? Here is the guide for how to add custom information. if msg2p-newapi is 0, please refer to this topic.

4

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.