I’m using the Deepstream USB source python sample app well but want to improve the appearance (bounding box color, thickness …). I trained a Detectnet_v2 model that added a label class using TAO and converted on the Orin with reasonable success (I also use the generate partial config file in TAO).
When I add a new probe to the DS pipeline, I add it to the SRC pad of the nvinfer element (I call it PGIE). I note that the call back is called and I use the boiler plate code to collect the buffer and iterate on the l_frame variable (batch_meta frame meta list)
def pgie_src_pad_buffer_probe(pad, info, u_data):
‘’’ Probe to collect meta from the output of the Primary Detector’’’
ptvsd.debug_this_thread()
gst_buffer = info.get_buffer()
if not gst_buffer:
print(“ERROR**** Unable to get GSTBuffer”)
return
batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
l_frame = batch_meta.frame_meta_list
while l_frame is not None:
try:
# Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
# The casting also keeps ownership of the underlying memory
# in the C code, so the Python garbage collector will leave
# it alone.
frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
except StopIteration:
break
I’m using ptvsd to stop on the thread breakpoint.
Unfortunately the while l_frame loops forever because it appears the data capsule is NULL.
All the examples Ive seen refer to an SSD but I use Detectnet_v2. I believe I need to get hold of the tensors but am stuck.
Please provide complete information as applicable to your setup. • Hardware Platform (Jetson / GPU) • DeepStream Version • JetPack Version (valid for Jetson only) • TensorRT Version • NVIDIA GPU Driver Version (valid for GPU only) • Issue Type( questions, new requirements, bugs) • How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) • Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
• Hardware Platform (Jetson / GPU) Jetson Orin AGX • DeepStream Version 6.1 • JetPack Version (valid for Jetson only) 5.01 • TensorRT Version 8.x • NVIDIA GPU Driver Version (valid for GPU only) • Issue Type( questions, new requirements, bugs) question • How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) Using python bindings insert prob into src of NVINFER element of pipeline to allow insertion of object into meta data relating to configuring bounding boxes for TAO trained (class added) Detectnet_v2 Object detection model. Callback function is fired but gets stuck in repeating loop in while l_frame (below, marked ****). It is noted that the data capsule is NULL. Callback function does not currently do anything except attempts to get the tensors. • Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description) N/A
PGIE instantiation:
pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")
if not pgie:
sys.stderr.write(" Unable to create pgie \n")
Insertion of Probe
pgiesrcpad = pgie.get_static_pad("src")
if not pgiesrcpad:
sys.stderr.write("ERROR**** Unable to get SRC pad of PGIE \n")
pgie_src_pad_buffer_probe_label = partial(pgie_src_pad_buffer_probe)
pgiesrcpad.add_probe(Gst.PadProbeType.BUFFER, pgie_src_pad_buffer_probe_label,0)
(have tried without python partial too).
Callback function
def pgie_src_pad_buffer_probe(pad, info, u_data):
‘’’ Probe to collect meta from the output of the Primary Detector’’’
ptvsd.debug_this_thread()
gst_buffer = info.get_buffer()
if not gst_buffer:
print(“ERROR**** Unable to get GSTBuffer”)
return
batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
l_frame = batch_meta.frame_meta_list
while l_frame is not None: ****
try:
# Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
# The casting also keeps ownership of the underlying memory
# in the C code, so the Python garbage collector will leave
# it alone.
frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
except StopIteration:
break
l_user = frame_meta.frame_user_meta_list
**while l_user is not None:**
try:
# Note that l_user.data needs a cast to pyds.NvDsUserMeta
# The casting also keeps ownership of the underlying memory
# in the C code, so the Python garbage collector will leave
# it alone.
user_meta = pyds.NvDsUserMeta.cast(l_user.data)
except StopIteration:
break
if (
user_meta.base_meta.meta_type
!= pyds.NvDsMetaType.NVDSINFER_TENSOR_OUTPUT_META
):
continue
tensor_meta = pyds.NvDsInferTensorMeta.cast(user_meta.user_meta_data)
return Gst.PadProbeReturn.OK
the answer was figuring out the boxes are drawn in the OSD probe, where you can change the appearance of the boxes and fonts etc. Thanks for your help.