Hi, i am getting “buffers are being dropped” message sooner than what you tested and given as performance achieved at 56 streams.?
Can you share the config settings to achieve below
Realtime 30 fps for 56 streams for Tesla T4
• Average CPU Utilization - ~5%
• Average GPU Utilization - ~72%
By the way, i am using n1-standard-16 (16 vCPUs, 60 GB memory) as opposed to the configuration that you reported.
Is this the bottleneck / reason for dropping buffers sooner?
Thanks
HI,
Can you provide the configuration you used when running deepstream? how about the stream resolution? also
how many gpu cards in your system? can you provide “nvidia-smi dmon” data when running deepstream?
I am using 24 core cpu with 1 t4.
I am able to run 36 h.264 streams with streammux (36), Primary detector batch as 36 & secondary classifiers as batch 16.
as of now Sinko is set to Fakesink
Beyond 36 i am getting the below warning (does this mean dropping frames)
WARNING from sink_sub_bin_sink1: A lot of buffers are being dropped.
Debug info: gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/GstFakeSink:sink_sub_bin_sink1:
0 63 72 - 52 25 0 0 5000 1575
0 59 72 - 58 20 0 0 5000 1590
0 47 71 - 50 21 0 0 5000 1590
0 45 71 - 32 15 0 2 5000 1590
0 34 71 - 2 1 0 14 5000 585
0 35 71 - 5 4 0 22 5000 750
0 38 71 - 29 11 0 26 5000 870
0 49 71 - 25 10 0 22 5000 870
0 64 72 - 28 13 0 28 5000 1080
0 78 73 - 38 19 0 39 5000 1440
0 65 73 - 58 28 0 48 5000 1530
0 58 73 - 34 19 0 45 5000 1470
0 76 74 - 47 25 0 49 5000 1215
0 65 75 - 60 30 0 52 5000 1485
0 54 75 - 63 33 0 59 5000 1440
0 80 75 - 37 20 0 39 5000 1275
0 62 75 - 62 32 0 59 5000 1470
0 49 75 - 66 33 0 53 5000 1380
0 65 75 - 56 29 0 61 5000 1395
0 90 76 - 57 28 0 55 5000 1500
0 66 76 - 58 29 0 53 5000 1185
0 49 76 - 60 31 0 63 5000 1245
0 57 77 - 62 31 0 58 5000 1350
0 53 77 - 65 31 0 61 5000 1335
0 75 77 - 59 29 0 53 5000 1080
0 75 77 - 58 27 0 55 5000 1425
0 56 78 - 60 28 0 54 5000 1395
0 86 78 - 54 27 0 54 5000 975
0 66 78 - 65 30 0 42 5000 1515
0 64 79 - 53 25 0 48 5000 1305
0 84 79 - 59 27 0 47 5000 1350
0 76 79 - 54 25 0 53 5000 1365
0 62 79 - 40 19 0 41 5000 1545
thanks.
Below is the config
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl
[tiled-display]
enable=0
rows=2
columns=2
width=1280
height=720
gpu-id=0
cuda-memory-type=1
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://…/…/streams/sample_720p.h264
num-sources=40
gpu-id=0
cuda-memory-type=1
[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=1
sync=1
source-id=0
gpu-id=0
cuda-memory-type=1
[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=3
sync=0
bitrate=2000000
output-file=out.mp4
source-id=0
[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=mp4 2=mkv
container=2
#1=h264 2=h265 3=mpeg4
codec=1
sync=0
bitrate=4000
rtsp-port=8554
udp-port=5400
[osd]
enable=0
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
cuda-memory-type=1
[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=40
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
cuda-memory-type=1
[primary-gie]
enable=1
gpu-id=0
model-engine-file=…/…/models/Primary_Detector/resnet10.caffemodel_b4_int8.engine
labelfile-path=…/…/models/Primary_Detector/labels.txt
batch-size=40
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
cuda-memory-type=1
config-file=config_infer_primary.txt
[tracker]
enable=1
tracker-width=640
tracker-height=368
gpu-id=0
[secondary-gie0]
enable=1
model-engine-file=…/…/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_int8.engine
labelfile-path=…/…/models/Secondary_VehicleTypes/labels.txt
gpu-id=0
batch-size=16
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_vehicletypes.txt
[secondary-gie1]
enable=1
model-engine-file=…/…/models/Secondary_CarColor/resnet18.caffemodel_b16_int8.engine
labelfile-path=…/…/models/Secondary_CarColor/labels.txt
batch-size=16
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carcolor.txt
[secondary-gie2]
enable=1
model-engine-file=…/…/models/Secondary_CarMake/resnet18.caffemodel_b16_int8.engine
labelfile-path=…/…/models/Secondary_CarMake/labels.txt
batch-size=16
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carmake.txt
[tests]
file-loop=0
what about the stream resolution, 720p or 1080p?
It is 720p.
I have one more question. Can i run decode on one gpu and ml on other gpu to increase the overall performance. If yes, please send me the changes that i need to make to the config file.
I tried to put source0, Sink0, Streammux on GPU0, and all ml on GPU1 using the Fakesink(as i dont have a display). The result does not even match the performance of single GPU set up. I must be doing something wrong.
Does this make sense.
Thanks
Hi,
the perf data based on configuration, refer to this, within documentation, Deepstream_tesla\DeepStream Tesla Development Guide/deepstream_performance.html, check part 'System Configuration, Application Configuration
’
Can you try tiler off, tracker algorithm set to IOU, disabled all sink, and try again?
here is one configuration for your reference
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl
[tiled-display]
enable=0
rows=5
columns=6
width=1280
height=720
gpu-id=0
cuda-memory-type=1
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://…/…/streams/sample_720p.mp4
#uri=file://…/…/streams/sample_outdoor_car_1080p_10fps.h264
num-sources=56
gpu-id=0
cuda-memory-type=1
[source1]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://…/…/streams/sample_720p.mp4
num-sources=1
gpu-id=0
cuda-memory-type=1
[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=1
sync=0
source-id=0
gpu-id=1
cuda-memory-type=1
[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=3
sync=1
bitrate=2000000
output-file=out.mp4
source-id=0
[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
cuda-memory-type=1
[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=56
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
cuda-memory-type=1
[primary-gie]
enable=1
gpu-id=0
batch-size=56
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;1;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
cuda-memory-type=1
config-file=config_infer_primary.txt
labelfile-path=…/…/Model/IVAPrimary_resnet10_DeepstreamRel_V2_ivalarge_its_phase1/labels.txt
model-engine-file=…/…/Model/IVAPrimary_resnet10_DeepstreamRel_V2_ivalarge_its_phase1/resnet10.caffemodel_b4_int8.engine
[tracker]
enable=1
tracker-width=640
tracker-height=368
#0 - Default, 1 - KLT 2 - IOU
tracker-algorithm=2
iou-threshold=0.1
gpu-id=0
[secondary-gie0]
enable=1
gpu-id=0
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
batch-size=32
config-file=config_infer_secondary_cartype.txt
labelfile-path=…/…/Model/V4/cartype/V4_TRT4/labels.txt
model-engine-file=…/…/Model/V4/cartype/V4_TRT4/resnet18.caffemodel_b16_int8.engine
[secondary-gie1]
enable=1
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
batch-size=32
config-file=config_infer_secondary_carcolor.txt
labelfile-path=…/…/Model/V4/carcolor/V4_TRT4/labels.txt
model-engine-file=…/…/Model/V4/carcolor/V4_TRT4/resnet18.caffemodel_b16_int8.engine
[secondary-gie2]
enable=1
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
batch-size=32
config-file=config_infer_secondary_carmake.txt
labelfile-path=…/…/Model/V4/carmake/V4_TRT4/labels.txt
model-engine-file=…/…/Model/V4/carmake/V4_TRT4/resnet18.caffemodel_b16_int8.engine
[tests]
file-loop=1