cpu:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 40
On-line CPU(s) list: 0-39
Thread(s) per core: 2
Core(s) per socket: 10
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz
When I run the Deepstream app with the following configuration, the CPU usage will reach 100% in a few minutes.
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl
[tiled-display]
enable=0
rows=2
columns=2
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=rtmp://127.0.0.1:10936/live/yace201
num-sources=60
drop-frame-interval=4
gpu-id=0
# (0): memtype_device - Memory type Device
# (1): memtype_pinned - Memory type Host Pinned
# (2): memtype_unified - Memory type Unified
cudadec-memtype=0
[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink/nv3dsink (Jetson only) 3=File
type=1
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0
[sink1]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=1
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
# set profile only for hw encoder, sw encoder selects profile based on sw-preset
profile=0
output-file=out.mp4
source-id=0
[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
#sw-preset=1 #for SW enc=(0)None (1)ultrafast (2)superfast (3)veryfast (4)faster
#(5)fast (6)medium (7)slow (8)slower (9)veryslow (10)placebo
sync=0
#iframeinterval=10
bitrate=400000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
# set profile only for hw encoder, sw encoder selects profile based on sw-preset
profile=0
# set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400
[osd]
enable=0
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0
[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
buffer-pool-size=4
batch-size=60
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
## If set to TRUE, system timestamp will be attached as ntp timestamp
## If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached
# attach-sys-ts-as-ntp=1
# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=/home/runone/program/folder/yolov8s.engine
batch-size=12
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=/home/runone/deepstream-implatform/deepstream-common/test_config_infer_primary_yoloV8.txt
[tracker]
enable=1
# For NvDCF and NvDeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively
tracker-width=960
tracker-height=544
ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
# ll-config-file required to set different tracker types
ll-config-file=config_tracker_IOU.yml
#ll-config-file=config_tracker_NvSORT.yml
#ll-config-file=config_tracker_NvDCF_perf.yml
# ll-config-file=config_tracker_NvDCF_accuracy.yml
# ll-config-file=config_tracker_NvDeepSORT.yml
gpu-id=0
display-tracking-id=1
[secondary-gie0]
enable=0
#model-engine-file=../../models/Secondary_VehicleTypes/resnet18_vehicletypenet.etlt_b16_gpu0_int8.engine
model-engine-file=../../models/Secondary_VehicleTypes/resnet18_vehicletypenet.etlt_b32_gpu0_int8.engine
gpu-id=0
batch-size=32
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_vehicletypes.txt
[secondary-gie1]
enable=0
#model-engine-file=../../models/Secondary_VehicleMake/resnet18_vehiclemakenet.etlt_b16_gpu0_int8.engine
model-engine-file=../../models/Secondary_VehicleTypes/resnet18_vehicletypenet.etlt_b32_gpu0_int8.engine
batch-size=32
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_vehiclemake.txt
[tests]
file-loop=1
The partial output is as follows:
************BATCH-NUM = 1951**************
Comp name = nvv4l2decoder7 in_system_timestamp = 1733645017937.278076 out_system_timestamp = 1733645017938.025879 component latency= 0.747803
Comp name = nvstreammux-src_bin_muxer source_id = 0 pad_index = 0 frame_num = 7804 in_system_timestamp = 1733645017938.246094 out_system_timestamp = 1733645017979.398926 component_latency = 41.152832
Comp name = primary_gie in_system_timestamp = 1733645017979.668945 out_system_timestamp = 1733645018014.274902 component latency= 34.605957
Comp name = tracking_tracker in_system_timestamp = 1733645018014.449951 out_system_timestamp = 1733645018035.341064 component latency= 20.891113
Comp name = demuxer in_system_timestamp = 1733645018035.454102 out_system_timestamp = 1733645018037.031982 component latency= 1.577881
Source id = 0 Frame_num = 7804 Frame latency = 100.702881 (ms)
************BATCH-NUM = 1952**************
Comp name = nvv4l2decoder7 in_system_timestamp = 1733645019413.281982 out_system_timestamp = 1733645019414.123047 component latency= 0.841064
Comp name = nvstreammux-src_bin_muxer source_id = 0 pad_index = 0 frame_num = 7808 in_system_timestamp = 1733645019414.270996 out_system_timestamp = 1733645019418.333984 component_latency = 4.062988
Comp name = primary_gie in_system_timestamp = 1733645019418.606934 out_system_timestamp = 1733645019496.678955 component latency= 78.072021
Comp name = tracking_tracker in_system_timestamp = 1733645019496.852051 out_system_timestamp = 1733645019523.608887 component latency= 26.756836
Comp name = demuxer in_system_timestamp = 1733645019523.732910 out_system_timestamp = 1733645019524.583008 component latency= 0.850098
Source id = 0 Frame_num = 7808 Frame latency = 111.863037 (ms)
************BATCH-NUM = 1953**************
Comp name = nvv4l2decoder7 in_system_timestamp = 1733645020701.476074 out_system_timestamp = 1733645020703.510010 component latency= 2.033936
Comp name = nvstreammux-src_bin_muxer source_id = 0 pad_index = 0 frame_num = 7812 in_system_timestamp = 1733645020703.639893 out_system_timestamp = 1733645020755.684082 component_latency = 52.044189
Comp name = primary_gie in_system_timestamp = 1733645020757.172119 out_system_timestamp = 1733645020863.929932 component latency= 106.757812
Comp name = tracking_tracker in_system_timestamp = 1733645020863.985107 out_system_timestamp = 1733645020882.616943 component latency= 18.631836
Comp name = demuxer in_system_timestamp = 1733645020882.697998 out_system_timestamp = 1733645020883.471924 component latency= 0.773926
Source id = 0 Frame_num = 7812 Frame latency = 182.752930 (ms)
**PERF: 2.82 (6.49) 2.82 (6.49) 3.07 (6.49) 2.77 (6.49) 2.91 (6.49) 3.61 (6.49) 2.77 (6.49) 3.46 (6.49) 2.90 (6.49) 3.04 (6.49) 2.86 (6.49) 3.40 (6.49) 2.82 (6.49) 3.04 (6.49) 3.37 (6.49) 3.33 (6.49) 3.20 (6.49) 2.77 (6.49) 3.61 (6.49) 3.48 (6.49) 3.04 (6.49) 3.50 (6.48) 3.24 (6.49) 3.69 (6.49) 3.71 (6.49) 3.50 (6.49) 2.79 (6.49) 3.61 (6.49) 3.08 (6.49) 3.48 (6.49) 2.82 (6.49) 3.69 (6.49) 3.31 (6.49) 2.98 (6.49) 2.60 (6.49) 2.87 (6.49) 3.31 (6.49) 3.61 (6.49) 2.77 (6.49) 3.48 (6.49) 2.86 (6.49) 2.86 (6.49) 3.48 (6.49) 3.48 (6.49) 3.04 (6.49) 2.90 (6.49) 3.48 (6.49) 2.77 (6.49) 3.50 (6.49) 3.20 (6.49) 2.77 (6.49) 5.25 (6.48) 3.73 (6.48) 2.79 (6.49) 2.80 (6.49) 3.48 (6.49) 2.79 (6.49) 3.37 (6.49) 2.79 (6.49) 3.31 (6.49)
************BATCH-NUM = 1954**************
Comp name = nvv4l2decoder7 in_system_timestamp = 1733645022259.010986 out_system_timestamp = 1733645022261.749023 component latency= 2.738037
Comp name = nvstreammux-src_bin_muxer source_id = 0 pad_index = 0 frame_num = 7816 in_system_timestamp = 1733645022262.114990 out_system_timestamp = 1733645022271.552002 component_latency = 9.437012
Comp name = primary_gie in_system_timestamp = 1733645022279.949951 out_system_timestamp = 1733645022314.012939 component latency= 34.062988
Comp name = tracking_tracker in_system_timestamp = 1733645022314.072021 out_system_timestamp = 1733645022332.966064 component latency= 18.894043
Comp name = demuxer in_system_timestamp = 1733645022333.052979 out_system_timestamp = 1733645022333.731934 component latency= 0.678955
Source id = 0 Frame_num = 7816 Frame latency = 75.187988 (ms)
************BATCH-NUM = 1955**************
Comp name = nvv4l2decoder7 in_system_timestamp = 1733645024323.956055 out_system_timestamp = 1733645024325.158936 component latency= 1.202881
Comp name = nvstreammux-src_bin_muxer source_id = 0 pad_index = 0 frame_num = 7820 in_system_timestamp = 1733645024325.587891 out_system_timestamp = 1733645024353.639893 component_latency = 28.052002
Comp name = primary_gie in_system_timestamp = 1733645024354.030029 out_system_timestamp = 1733645024452.125000 component latency= 98.094971
Comp name = tracking_tracker in_system_timestamp = 1733645024452.270020 out_system_timestamp = 1733645024516.207031 component latency= 63.937012
Comp name = demuxer in_system_timestamp = 1733645024516.221924 out_system_timestamp = 1733645024516.398926 component latency= 0.177002
Source id = 0 Frame_num = 7820 Frame latency = 192.571045 (ms)
As shown in the above figure, the CPU starts to skyrocket a few minutes after the program runs.
In the config of deepstream app, most of them are default settings with two variables.
One is the model, which is the open-source YOLOV8 on GitHub. The configuration is as follows:
[property]
net-scale-factor=0.0039215697906911373
gpu-id=0
batch-size=16
#0=RGB, 1=BGR
model-color-format=0
model-engine-file=/home/runone/program/folder/yolov8s.engine
labelfile-path=/home/runone/program/folder/yolov8_test_classes.txt
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=80
gie-unique-id=1
is-classifier=0
maintain-aspect-ratio=1
output-blob-names=num_dets;bboxes;scores;labels
parse-bbox-func-name=NvDsInferParseCustomYoloV8
custom-lib-path=/home/zhangxj/YOLOv8-TensorRT/csrc/deepstream/build/libnvdsinfer_custom_bbox_yoloV8.so
The above mentioned libnvdsinferr_custom_fbox_yoloV8.so comes from:
YOLOv8-TensorRT/csrc/deepstream at main · triple-Mu/YOLOv8-TensorRT
The other one is the source. We have selected a more complex multi vehicle scenario, as shown in the following figure:
So is this kind of CPU surge normal? Is there any solution available? thank you
