No Detections with YOLOv8 Model

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1
  • Jetson Orin Nano
  • DeepStream 7.1
  • JetPack 6.2.1
  • TensorRT 10.3.0

I am trying to use a custom YOLOv8 model with DeepStream. I use

$ yolo export model=bush_yolov8.pt format=‘engine’

to convert the model to an engine file, and then I run a script to remove the Ultralytics metadata from the engine file,

import os

engines = [f for f in os.listdir() if '.engine' in f]
for engine in engines:
	with open(engine, "rb") as f:
		meta_len = int.from_bytes(f.read(4), byteorder="little")
		f.seek(meta_len + 4)
		data = f.read()
	with open('new'+engine, "wb") as f:
		f.write(data)

so it works with DeepStream. Now when I run

$ deepstream-app -c app_configs/deepstream_app_config_bush.txt

DeepStream starts successfully but there are no detections on screen, and none being saved from kitti-track-output. I have other custom YOLOv8 models, and the same thing happens with those. Why am I not getting any detections?

2

Probably the output isn’t in the format that postprocessing requires.

You can check this for DeepStream

3

I have used the marcoslucianops/DeepStream-Yolo repository before. However, I want to be able to use an OBB model. The export_yoloV8 script doesn’t work for OBB models, and there isn’t an OBB implementation yet. I was hoping I could bypass this since YOLO offers a way to export to TensorRT. (I know DeepStream doesn’t natively support OBBs, but I want the models to at least give some detections instead of none.)

4

You would need to modify the postprocessing function to process the OBB output since it’s different from regular detection.

If you just want a normal detection, you can convert the OBB model to normal detection model:

6

please refer to NV yolov8 sample. can the model works well with other tools? if so, please make sure the preprocess and post postprocess are consistent with other tools. Please refer to this faq for accuracy Issue.

7

I still get no detections when first using ‘$ yolo export …‘ to make an onnx file then using ‘$ trtexec …‘ to make an engine file. If this is a problem with postprocess formatting, how would I fix this?

8

By their instructions, they say to use ‘yolo export‘ to get an onnx file, run their ‘append_transpose‘ script on the onnx file, and then generate an engine file with trtexec. When I try this and load the engine in DeepStream, there ARE detections now, but not only are they completely wrong, it crashes after a few frames (segmentation fault).

Edit: I was actually able to get an engine with this method such that running it with DeepStream did not crash after a few frames, but the detections were still broken.