Hi eveyone,I am using deepstream to inference batch30 model ,GPU and FPS is low
Image
nvcr.io/nvidia/deepstream:5.1-21.02-devel
Deepstream of NGC on X86
GPU
3080 * 1
software version
deepstream-app version 5.1.0
DeepStreamSDK 5.1.0
CUDA Driver Version: 11.1
CUDA Runtime Version: 11.1
TensorRT Version: 7.2
cuDNN Version: 8.0
libNVWarp360 Version: 2.0.1d3
***In container ***
cd /opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app
deepstream-app -c source30_1080p_dec_infer-resnet_tiled_display_int8.txt
***some logs****
**PERF: 26.15 (25.95) 25.15 (24.97) 25.83 (25.63) 26.73 (26.53) 24.70 (24.53) 28.82 (28.56) 27.54 (26.69) 23.81 (23.64) 22.92 (22.76) 23.36 (23.20) 27.54 (26.69) 26.73 (26.53) 28.22 (27.96) 23.73 (23.57) 26.91 (26.69) 25.78 (25.58) 24.20 (24.02) 28.82 (28.56) 25.83 (25.63) 25.83 (25.63) 24.67 (24.48) 24.65 (24.47) 28.22 (27.96) 24.17 (23.99) 25.71 (25.51) 28.82 (28.56) 26.91 (26.69) 25.27 (25.08) 24.67 (24.48) 23.72 (23.55)
nvidia-smi
GPU Memory 2728MiB
GPU Volatile 15 %
The utilization rate of GPU very is low
I tried to modify the batch to improve the FPS,but change batch-size 30 to 40, FPS is lower .
When changed batch-size 30 to 40,The results are like this
**PERF: 18.98 (19.29) 18.78 (18.39) 18.78 (19.22) 18.93 (19.14) 18.98 (19.29) 18.98 (18.72) 18.98 (18.39) 18.98 (18.52) 18.98 (18.72) 18.78 (18.32) 18.98 (18.78) 18.68 (19.22) 18.98 (18.68) 18.94 (18.22) 18.78 (18.32) 18.68 (19.22) 18.93 (19.14) 18.93 (19.14) 18.98 (18.72) 18.68 (19.22) 18.68 (19.22) 18.78 (18.76) 18.98 (19.29) 18.93 (19.14) 18.98 (19.29) 18.98 (18.67) 18.98 (19.29) 18.98 (18.62) 18.93 (19.14) 18.98 (19.29) 18.98 (18.46) 18.68 (19.22) 18.68 (19.22) 18.98 (18.72) 18.98 (18.46) 18.98 (18.25) 18.93 (19.14) 18.94 (18.57) 18.98 (18.47) 18.93 (19.14)
How to raise FPS