Hello, folks!
I’m experiencing performance issues with YOLOv8 using DeepStream6.2. I’m using the default yolov8s.pt file and the generated cfg, wts, and labels.txt files (source: https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt ) and performing inference on the sample_1080p_h264.mp4 video (path: /opt/nvidia/deepstream/deepstream/samples/streams/). The indicated performance is around 25FPS on a 1920x1080 monitor operating at 60Hz. How can I increase this frame rate?
System Specifications:
Output:
**PERF: FPS 0 (Avg)
Opening in BLOCKING MODE
**PERF: 34.75 (31.86)
File deepstream_app_config.txt:
[application]
[tiled-display]
[source0]
[sink0]
[osd]
[streammux]
[primary-gie]
[tests]
Thank you!
yingliu
September 18, 2023, 1:31pm
3
Please also share your PGIE config file, thanks.
Okay, here is the PGIE config file:
File config_infer_primary_yoloV8.txt:
[property]
[class-attrs-all]
Why did you set nvstreammux batch size as 1000? Frequently Asked Questions — DeepStream 6.3 Release documentation
batched-push-timeout should be 1/framerate(ms)
Please measure the model performance of the “model_b1_gpu0_fp32.engine” by the “trtexec” tool.
Hi, @Fiona.Chen . I’m sorry for the delay. I tried to measure the performance using the command:
/usr/src/tensorrt/bin/trtexec --loadEngine=model_b1_gpu0_fp32.engine
but it resulted the following errors:
[09/25/2023-09:29:01] [E] Error[1]:[pluginV2Runner.cpp::load::299] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
How should I procedure?
How and where did you generate the engine file?
I’ve measured the model engine performance by trtexec in my Xavier NX board with max power.
[09/26/2023-14:40:55] [I] === Performance summary ===
[09/26/2023-14:40:55] [I] Throughput: 21.5936 qps
[09/26/2023-14:40:55] [I] Latency: min = 45.8785 ms, max = 46.2848 ms, mean = 46.0246 ms, median = 46.0143 ms, percentile(90%) = 46.147 ms, percentile(95%) = 46.1866 ms, percentile(99%) = 46.2848 ms
[09/26/2023-14:40:55] [I] Enqueue Time: min = 2.44128 ms, max = 4.10165 ms, mean = 2.77255 ms, median = 2.69226 ms, percentile(90%) = 2.96558 ms, percentile(95%) = 3.55359 ms, percentile(99%) = 4.10165 ms
[09/26/2023-14:40:55] [I] H2D Latency: min = 0.278076 ms, max = 0.387558 ms, mean = 0.289538 ms, median = 0.283432 ms, percentile(90%) = 0.306519 ms, percentile(95%) = 0.31073 ms, percentile(99%) = 0.387558 ms
[09/26/2023-14:40:55] [I] GPU Compute Time: min = 45.5718 ms, max = 45.9738 ms, mean = 45.7089 ms, median = 45.6978 ms, percentile(90%) = 45.8298 ms, percentile(95%) = 45.8577 ms, percentile(99%) = 45.9738 ms
[09/26/2023-14:40:55] [I] D2H Latency: min = 0.0153809 ms, max = 0.0288086 ms, mean = 0.0261959 ms, median = 0.0263672 ms, percentile(90%) = 0.0280762 ms, percentile(95%) = 0.0283203 ms, percentile(99%) = 0.0288086 ms
[09/26/2023-14:40:55] [I] Total Host Walltime: 3.10278 s
[09/26/2023-14:40:55] [I] Total GPU Compute Time: 3.0625 s
[09/26/2023-14:40:55] [I] Explanations of the performance metrics are printed in the verbose logs.
The compute time is more than 45ms. So the model is the bottleneck. Yolov8 may be too heavy for Xavier NX, you need to optimize the model first.
There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks
If your platform can get similar trtexec performance, you may need to set “interval=4” to gst-nvinfer to skip some frames inferencing to achieve the 60 FPS speed.
system
October 17, 2023, 1:41am
10
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.