DeepStream app - Running yolov8 with TensorRT and DeepStream SDK, not able to extract information regarding class_id, bounding box coordinates

[wajid.ali]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson orin nano
• DeepStream Version version 6.3
• JetPack Version (valid for Jetson only) jetpack version 5.1.2
• TensorRT Version version 8.5.1
• CUDA version version 11.4

I have used this repository Deploy YOLOv8 with TensorRT and DeepStream SDK | Seeed Studio Wiki to create files
deepstream_app_config.txt and config_infer_primary.txt

File deepstream_app_config.txt
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=1
rows=1
columns=1
width=1920
height=1080
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=2
uri=file:///home/kodifly/DeepStream-Yolo/4_classes_yolov9m/DeepStream-Yolo/IMG_7640.MOV
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=2
sync=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[tests]
file-loop=0

File config_infer_primary.txt

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
custom-network-config=yolov8m.cfg
model-file=yolov8m.wts
model-engine-file=model_b1_gpu0_fp32.engine
#int8-calib-file=calib.table
labelfile-path=labels.txt
batch-size=1
network-mode=0
num-detected-classes=4
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=0
symmetric-padding=1
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

after that I am using a python file to run them
File run_deepstream_app.py

import subprocess
import re

deepstream_config_path = ‘/home/kodifly/DeepStream-Yolo/4_classes_yolov9m/DeepStream-Yolo/deepstream_app_config.txt’

def run_deepstream():
command = f’deepstream-app -c {deepstream_config_path}’
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
return process

def process_output(process):
try:
# Regular expression to capture class_id, class name, and bbox coordinates
detection_pattern = re.compile(r’Detected class ID: (\d+), name: (\w+), bbox: [(.*?)]')

    for line in process.stdout:
        if 'Detected class ID:' in line:
            match = detection_pattern.search(line)
            if match:
                class_id, class_name, bbox = match.groups()
                print(f'Class ID: {class_id}, Name: {class_name}, Bounding Box: {bbox}')
                
        # Optionally, handle other log messages or errors
        if 'Error' in line:
            print(f'Error: {line.strip()}')
            
except Exception as e:
    print(f"Error occurred while processing output: {e}")

if name == ‘main’:
# Run DeepStream pipeline
process = run_deepstream()

# Process the output from DeepStream
process_output(process)

# Optionally, wait for DeepStream to complete (if it runs until completion)
process.wait()

It is opening the video window and performing detection on the video window. I am trying to access the class_id, class_name and bounding_box coordinate for further post processing. I have tried it and not been able to extract it. Please suggest some solution or any refrence.

[yuweiw]

@wajid.ali

It is opening the video window and performing detection on the video window.

Do you mean that the bbox can display normally?

[wajid.ali]

yes bounding box are displaying normally, but not able to retrieve them and class information.

[yuweiw]

Since you are using the deepstream-app app, so you need to get the information in the deepstream-app source code. You can refer to our source code below to learn how to get the information.

sources\apps\sample_apps\deepstream-app\deepstream_app.c
static void
write_kitti_output (AppCtx * appCtx, NvDsBatchMeta * batch_meta)

[wajid.ali]

Updated the file deepstream_app_config.txt and gave path for gie-kitti-output

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

gie-kitti-output-dir=/home/kodifly/DeepStream-Yolo/4_classes_yolov9m/bbox_output_dir

[tiled-display]
enable=1
rows=1
columns=1
width=1920
height=1080
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=2
uri=file:///home/kodifly/DeepStream-Yolo/4_classes_yolov9m/DeepStream-Yolo/IMG_7641.MOV
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=2
sync=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=5
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[tests]
file-loop=0

File config_infer_primary is

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
custom-network-config=yolov8m.cfg
model-file=yolov8m.wts
model-engine-file=model_b1_gpu0_fp32.engine
#int8-calib-file=calib.table
labelfile-path=labels.txt
batch-size=1
network-mode=0
num-detected-classes=4
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=0
symmetric-padding=1
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

the python file run_deepstream_app.py is

import subprocess
import os
import time
import threading
import cv2
import numpy as np
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
from ultralytics import YOLO

deepstream_config_path = ‘/home/kodifly/DeepStream-Yolo/4_classes_yolov9m/DeepStream-Yolo/deepstream_app_config.txt’

kitti_output_dir = “/home/kodifly/DeepStream-Yolo/4_classes_yolov9m/bbox_output_dir”

frames_dir = ‘/home/kodifly/DeepStream-Yolo/frames’

license_plate_detector = YOLO(‘license_plate_detector.pt’)

vehicles_class_labels = [“vehicle”] # Adjust this if needed

def get_current_frame():
“”“Mock function to get the current frame.”“”
# Replace this with your actual method to get the current frame
# For example, if frames are saved in a directory, get the latest file
frame_files = sorted([os.path.join(frames_dir, f) for f in os.listdir(frames_dir) if f.endswith(‘.jpg’)],
key=os.path.getmtime)
if frame_files:
return cv2.imread(frame_files[-1])
return None

class KITTIOutputHandler(FileSystemEventHandler):
def on_created(self, event):
“”“Handles the creation of new .txt files in the monitored directory.”“”
if event.src_path.endswith(“.txt”):
extract_and_display_bbox_info(event.src_path)

def extract_and_display_bbox_info(file_path):
“”“Extracts and displays class label and bounding box information, and handles ALERT mechanism.”“”
with open(file_path, ‘r’) as file:
for line in file:
data = line.strip().split()
if len(data) < 8:
continue # Skip if the line does not have enough parameters

        class_label = data[0]
        bbox_left = float(data[4])
        bbox_top = float(data[5])
        bbox_right = float(data[6])
        bbox_bottom = float(data[7])

        # Display extracted information
        print(f"Class Label: {class_label}, BBox [Left: {bbox_left}, Top: {bbox_top}, Right: {bbox_right}, Bottom: {bbox_bottom}]")

        # Check if the detected class is a vehicle
        if class_label.lower() in vehicles_class_labels:
            # Crop the vehicle area from the frame
            check_license_plate(bbox_left, bbox_top, bbox_right, bbox_bottom)

def check_license_plate(bbox_left, bbox_top, bbox_right, bbox_bottom):
“”“Check if the license plate is detected within the vehicle’s bounding box.”“”
# Get the current frame
frame = get_current_frame()

if frame is None:
    print("Error: Unable to read the current frame.")
    return

# Crop the vehicle area
vehicle_crop = frame[int(bbox_top):int(bbox_bottom), int(bbox_left):int(bbox_right)]

# Detect license plates within the vehicle bounding box
license_plates = license_plate_detector(vehicle_crop)[0]

if not license_plates.boxes.data.tolist():
    print("ALERT: No license plate detected within the vehicle's bounding box!")
    # Optionally, add code to overlay ALERT text on the vehicle bounding box

def monitor_kitti_output():
“”“Monitors the KITTI output directory for new .txt files.”“”
event_handler = KITTIOutputHandler()
observer = Observer()
observer.schedule(event_handler, kitti_output_dir, recursive=False)
observer.start()

print(f"Monitoring KITTI output directory: {kitti_output_dir}")

try:
    while True:
        time.sleep(1)  # Keep the script running
except KeyboardInterrupt:
    observer.stop()

observer.join()

def run_deepstream():
“”“Runs the DeepStream pipeline using the provided configuration file.”“”
command = f’deepstream-app -c {deepstream_config_path}’
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
return process

if name == ‘main’:
# Start the DeepStream pipeline first
process = run_deepstream()
print(“DeepStream pipeline started. Waiting for KITTI output…”)

# Wait a moment to ensure DeepStream starts generating files
time.sleep(5)

# Start monitoring KITTI output in a separate thread
monitor_thread = threading.Thread(target=monitor_kitti_output)
monitor_thread.daemon = True
monitor_thread.start()

# Optionally, wait for DeepStream to complete
process.wait()

the .txt file are being generated and have class_label and bbox information, but they are stored in a directory and I have to read them from there to check for any particular class label and if its present then start post processing on it. But firstly its not available on runtime and delay is induced secondly its not displaying directly at run time . I am not able to have access to video stream and when I give particular bounding box , then not able to have access to images. Also once a video is run and kitti-output is generated in .txt format , then for next video I have had to delete all files from the directory so that they are generated for new video. Please help me with deepstream app implementation according to the desired functionality.

[yuweiw]

@wajid.ali

deepstream-app -c {deepstream_config_path}

If you are using the method above in your python code, you should implement the functionality you want in our source code yourself.

sources\apps\sample_apps\deepstream-app

You can also learn our deepstream_python_apps demo and use python to implement your whole pipeline.

[wajid.ali]

Can you share or refer to any example implementation using deepstream_app -c where bbox information is being extracted for further processing. or integrated with data broker on nvidia jetson device

[yuweiw]

@wajid.ali

Can you share or refer to any example implementation using deepstream_app -c

If you are using deepstream-app, you need to implement that in our open source code for your own need.

About the broker demo, you can refer to our demo5.

sources\apps\sample_apps\deepstream-test5

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks