Repacing PeopleNet model with yolo model

Hi I want to replace PeopleNet model with my custom yolo model in nvidia metropolis, with jetpack 6.0.2(rev.2). But I’m unable to do so.I’m using Nvidia Agx Orin board. I did some changes according to this ticket. But it’s not working with latest metropolis release.
I’ve successfully generated onnx and engine file

ds-config-1_agx.txt (7.5 KB)
ds-config-0_agx.txt (7.4 KB)
config_infer_primary_RN34_PN26_960x544_dla1_orin_unprune_agx.txt (1.4 KB)
config_infer_primary_RN34_PN26_960x544_dla0_orin_unprune_agx.txt (1.4 KB)

Can you provide some more details on what specifically is not working? Is DeepStream crashing when trying to launch or is some other part of the pipeline not functioning? Are you pre-generating the engine file before running DeepStream? If so, what steps are you taking to generate it?

I also recommend using the provided YOLOv8s configs with the latest AI-NVR sample as part of JPS as a base rather than the ones from that post since the AI-NVR configs are more up to date. Docs for that can be found here: DeepStream Perception — Metropolis on Jetson documentation 0.1.0 documentation and the latest AI-NVR is here: Jetson Platform Services Reference Workflow & Resources | NVIDIA NGC.

I’m getting some error by running below command I’ve attached the log file
logs.txt (1.0 KB)
:

sudo docker run -v ./yolov8s:/yolov8s -v ./config/deepstream:/ds-config-files --runtime nvidia nvcr.io/nvidia/jps/deepstream:7.0-jps-v1.1.1 /yolov8s-files/yolov8s_setup.sh --agx

Your log show error when parse below DeepStream configure file. Please ensure DeepStream application can access it in the container.

./config/deepstream/yolov8s/setup_config_agx.txt

The DeepStream configure file(ai_nvr-1.1.0.tar.gz) can be download from: Jetson Platform Services Reference Workflow & Resources | NVIDIA NGC

I’ve downloaded files from that link only, Actually it has downloaded .pt file and converted that into onnx format but while converting it into engine file I’m getting error

I can get below log when parse /ds-config-files/yolov8s/setup_config_agx.txt in DeepStream container:

root@ubuntu:~# export GST_DEBUG=NVDS_CFG_PARSER:6
root@ubuntu:~# /yolov8s-files/yolov8s_setup.sh --agx
NOTE: For each dataset an user elects to use, the user is responsible for checking if the dataset license is fit for the intended purpose.
Using AGX so batch size set to 8…
yolov8s-dependencies directory exists - skipping download/init. If there was a failure last time, please delete this folder and retry.
Running AGX setup…

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.420: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstmpg123.so’: libmpg123.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.517: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstmpeg2enc.so’: libmpeg2encpp-2.1.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.572: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstopenmpt.so’: libmpg123.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.610: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstchromaprint.so’: libavcodec.so.58: cannot open shared object file: No such file or directory

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.653: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstmpeg2dec.so’: libmpeg2.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.687: adding type GstEvent multiple times
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 204)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 106)

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.745: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so’: librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:42): GStreamer-WARNING **: 05:57:24.849: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so’: libtritonserver.so: cannot open shared object file: No such file or directory
0:00:00.553999677 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: application
0:00:00.554152861 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: tiled-display
0:00:00.554172893 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: source0
0:00:00.554208541 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: sink0
0:00:00.554228061 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: osd
0:00:00.554274813 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: streammux
0:00:00.554291293 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: primary-gie
0:00:00.554344509 41 0xaaaaf6be67e0 DEBUG NVDS_CFG_PARSER deepstream_app_config_parser.c:425:parse_config_file: Parsing group: tests

Have you modified any of the configs under ./config/deepstream/yolov8s/ from what is included in the bundle?

Also ensure that the ./yolov8s/yolov8s-dependencies/yolov8s.onnx file exists on the host system. If ./yolov8s/calib.table or any .engine files exist under the ./yolov8s/ or ./yolov8s/yolov8s-dependencies/ folders, try deleting them and rerun that command.

Thanks @hfarooq @kesong for your reply. I’ve reinstalled docker again and now I can visulize the yolov8 overlay output by doing some modification in “yolov8s-ds-config_agx.txt” file. Now I have a custom trained “.pt” file of yoloV7 that I wanted to use with metropolis, so could you please suggest me with the steps for the same.

Here is the DeepStream yoloV7 project: yolo_deepstream/deepstream_yolo at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub
Need porting this projcet into JPS.

Is deepstream 7.0 compatible with yolov7?

Yes. You only need the PGIE configure file and the customized post-process for porting YoloV7 into DeepStream.

I have trained yolov8 model on my custom datasets and trying to convert it into engine format with “yolov8s_setup.sh” script but I’m unable to do so. It’s successfully converting into onnx format but not converting to engine format.
best.zip (19.8 MB)

yolov8s_setup.zip (1.6 KB)

I did setup of “yolov8s_setup.sh” script outside of docker container, but while converting to engine I’m getting error

File does not exist: /yolov8s/calib.table

Can you generate engine file with default yolov8? Is there any layer change for your trained yolov8?

No its not generating with default yolov8 as well outside of docker.
No layer change for my trained yolov8.

Does it mean you generated yolov8 engine and it works fine?

Yes, I generated the engine of the default YOLOv8 using the DeepStream Docker container. However, to generate the engine for my trained model, I need to set it up on the host. The engine file is not being generated on the host for the default YOLOv8. I have installed all the required dependencies on the host as well.

Does the issue resolved? Can you generate engine file on the Jetson?

No Kesong.

Can you share the full log? You also can compare the log between internal container and outside container. So we can know the difference.

Since you are running this outside the container: have you created the /yolov8s directory on the system before running the script?

And you have installed all dependencies, correct? This being OpenCV (with pkgconfig and extramodules) and the YOLO DS function to generate the TRT engine from the 3rd party repo.

You will also need to modify the DS YOLO initialization configs to point to the location of the .so file for TRT engine generation and the location of your ONNX file.