Jetson AGX Orin - Deepstream 6.1 - Python Bindings - PGIE Probe - Null Capsule

Hi,

I’m using the Deepstream USB source python sample app well but want to improve the appearance (bounding box color, thickness …). I trained a Detectnet_v2 model that added a label class using TAO and converted on the Orin with reasonable success (I also use the generate partial config file in TAO).

When I add a new probe to the DS pipeline, I add it to the SRC pad of the nvinfer element (I call it PGIE). I note that the call back is called and I use the boiler plate code to collect the buffer and iterate on the l_frame variable (batch_meta frame meta list)

def pgie_src_pad_buffer_probe(pad, info, u_data):
‘’’ Probe to collect meta from the output of the Primary Detector’’’
ptvsd.debug_this_thread()
gst_buffer = info.get_buffer()
if not gst_buffer:
print(“ERROR**** Unable to get GSTBuffer”)
return

batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
l_frame = batch_meta.frame_meta_list
while l_frame is not None:
    try:
        # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
        # The casting also keeps ownership of the underlying memory
        # in the C code, so the Python garbage collector will leave
        # it alone.
        frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
    except StopIteration:
        break

I’m using ptvsd to stop on the thread breakpoint.

Unfortunately the while l_frame loops forever because it appears the data capsule is NULL.

All the examples Ive seen refer to an SSD but I use Detectnet_v2. I believe I need to get hold of the tensors but am stuck.

Many thanks!!

Cheers

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

• Hardware Platform (Jetson / GPU) Jetson Orin AGX
• DeepStream Version 6.1
• JetPack Version (valid for Jetson only) 5.01
• TensorRT Version 8.x
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) Using python bindings insert prob into src of NVINFER element of pipeline to allow insertion of object into meta data relating to configuring bounding boxes for TAO trained (class added) Detectnet_v2 Object detection model. Callback function is fired but gets stuck in repeating loop in while l_frame (below, marked ****). It is noted that the data capsule is NULL. Callback function does not currently do anything except attempts to get the tensors.
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description) N/A

PGIE instantiation:

pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")
if not pgie:
    sys.stderr.write(" Unable to create pgie \n")

Insertion of Probe

pgiesrcpad = pgie.get_static_pad("src")
if not pgiesrcpad:
    sys.stderr.write("ERROR**** Unable to get SRC pad of PGIE \n")

pgie_src_pad_buffer_probe_label = partial(pgie_src_pad_buffer_probe)
pgiesrcpad.add_probe(Gst.PadProbeType.BUFFER, pgie_src_pad_buffer_probe_label,0)

(have tried without python partial too).

Callback function

def pgie_src_pad_buffer_probe(pad, info, u_data):
‘’’ Probe to collect meta from the output of the Primary Detector’’’
ptvsd.debug_this_thread()
gst_buffer = info.get_buffer()
if not gst_buffer:
print(“ERROR**** Unable to get GSTBuffer”)
return

batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
l_frame = batch_meta.frame_meta_list
while l_frame is not None: ****
    try:
        # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
        # The casting also keeps ownership of the underlying memory
        # in the C code, so the Python garbage collector will leave
        # it alone.
        frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
    except StopIteration:
        break

l_user = frame_meta.frame_user_meta_list
**while l_user is not None:**
    try:
        # Note that l_user.data needs a cast to pyds.NvDsUserMeta
        # The casting also keeps ownership of the underlying memory
        # in the C code, so the Python garbage collector will leave
        # it alone.
        user_meta = pyds.NvDsUserMeta.cast(l_user.data)
    except StopIteration:
            break

    if (
        user_meta.base_meta.meta_type
        != pyds.NvDsMetaType.NVDSINFER_TENSOR_OUTPUT_META
        ):
        continue

    tensor_meta = pyds.NvDsInferTensorMeta.cast(user_meta.user_meta_data)
               
return Gst.PadProbeReturn.OK

Thank you.

Do you mean you want to get tensor output with probe but failed? Please refer to the sample of deepstream_python_apps/apps/deepstream-ssd-parser at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub and Welcome to the DeepStream Documentation — DeepStream 6.1 Release documentation

the answer was figuring out the boxes are drawn in the OSD probe, where you can change the appearance of the boxes and fonts etc. Thanks for your help.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.