Explanation about batch-size

[marcoslucianops]

Hi,

I’m running 1000 cameras in my custom DeepStream code. About nvinfer and nvstreammux, should I set the batch-size = 1000 for both? I’m facing a high GPU memory usage with high values of batch-size. Can you explain about how I need to set the batch-size for this situation (high cameras number) without reaching the GPU memory limit?

I’m using:

• Python custom code
• Streammux resolution: 704x480
• NvDCF Tracker (640x384 resolution, with enable_batch_process = 1)
• YoloV4-Tiny model (from my custom YOLO: GitHub - marcoslucianops/DeepStream-Yolo: NVIDIA DeepStream SDK 6.1 / 6.0.1 / 6.0 configuration for YOLO models)
• FP16
• Camera resolution: 704x480
• Camera FPS: 1
• Kafka output message

[kayccc]

Sorry for the late response, will get back to you soon.

Thanks

[kesong]

Can you have a try with small batch-size?

Gst-nvinfer — DeepStream 5.1 Release documentation (nvidia.com)

Gst-nvstreammux — DeepStream 5.1 Release documentation (nvidia.com)

[marcoslucianops]

Yes, I’m testing with lower and higher batch-size for both. But, is there some problem by using, for example, batch-size 1 for both (nvinfer and nvstreammux) with 1000 cameras (besides inferior performance)?

[kesong]

You can set right batch size to balance performance and memory footprint. such as 32.