In Deepstream lpr app obj_meta.parent is always None

[gbetsos]

• Hardware Platform (Jetson / GPU) Orin AGX
• DeepStream Version 6.1.1
• JetPack Version (valid for Jetson only) 5.0.2
• TensorRT Version 8.4.1.5
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

I have ported the DeepStream LPR app (GitHub - NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream) from C++ to python. I can get the lpd bounding boxes displayed, so I guess my port is successful.

I’m facing an issue with getting the parent object of a plate detection. This is the corresponding part of the C++ app:

  if (obj_meta->unique_component_id == SECONDARY_DETECTOR_UID) {
    if (obj_meta->class_id == SGIE_CLASS_ID_LPD) {
      lp_count++;
      /* Print this info only when operating in secondary model. */
      if (obj_meta->parent)
          g_print ("License plate found for parent object %p (type=%s)\n", obj_meta->parent, pgie_classes_str[obj_meta->parent->class_id]);

The message “License plate found for parent object …” never appears after executing the app, allthough plates are detected in most of the frames.

In Python, I am trying with the following part of code:

if obj_meta.unique_component_id == self.SECONDARY_DETECTOR_UID:
print(obj_meta.rect_params.top, obj_meta.rect_params.left, obj_meta.rect_params.width, obj_meta.rect_params.height)
print(obj_meta.parent)
if obj_meta.parent:
print(obj_meta.parent.class_id, obj_meta.parent.object_id)

Unfortunately obj_meta.parent is always None. I’ve tried moving the probe function before the tiler, as suggested in some other post I found, but it didn’t work.

[yuweiw]

You can try to set the probe function to secondary_classifier sink.

[gbetsos]

My pipeline is as follows:

uridecodebin → streammux → queue → pgie (trafficamnet) → queue → nvtracker → queue → sgie (lpdnet) → queue → nvvideoconvert → queue → capsfilter (“video/x-raw(memory:NVMM), format=RGBA”) → nvtiler → queue → nvosd → fakesink

If I place the probe function on the src pad of sgie then I do get a value for obj_meta.parent. But if I do so then the rest of my code breaks.

I also need to include in the probe function:

n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)            
frame_copy = np.array(n_frame, copy=True, order='C')

But this code won’t execute before nvvideoconvert. I get the following error:

n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)
RuntimeError: get_nvds_buf_Surface: Currently we only support RGBA color Format
Traceback (most recent call last):
File “/srv/ivms_v2/detect/core/pipeline.py”, line 485, in sgie_src_pad_buffer_probe

[yuweiw]

According to your own pipeline, there are some differences with our demo. Let’s discuss our demo code first. So you want to get the parent object on osd source pad? Do you mean that once datas get through the nvvideoconvert plug-in, the parent object will be NULL?

[gbetsos]

@yuweiw

So you want to get the parent object on osd source pad?

Yes. obj_meta->parent in line

if (obj_meta->parent)

is always null, since I never get to see the message “License plate found for parent object …”

I’m using the following command to execute the app:

./deepstream-lpr-app 1 1 0 infer /opt/nvidia/deepstream/deepstream/samples/streams/sample_qHD.mp4 out.mp4

Sample output:

Frame Number = 1793 Vehicle Count = 10 Person Count = 1 License Plate Count = 3
Frame Number = 1794 Vehicle Count = 9 Person Count = 1 License Plate Count = 3
Frame Number = 1795 Vehicle Count = 8 Person Count = 0 License Plate Count = 3
Frame Number = 1796 Vehicle Count = 8 Person Count = 0 License Plate Count = 3
Frame Number = 1797 Vehicle Count = 9 Person Count = 0 License Plate Count = 4
Frame Number = 1798 Vehicle Count = 8 Person Count = 0 License Plate Count = 3
End of stream
Returned, stopping playback
[NvMultiObjectTracker] De-initialized
Average fps 154.557418
Totally 2701 plates are inferred
Deleting pipeline

[yuweiw]

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Ok. The parent para is NULL after the tiler plugin cause the tiler will merge multiple pictures. So you can’t get it from the osd probe function. We need to comment the print out in our demo. If you want to get the para, you can only get it before the tiler.