Delay gathered over time

You may need to measure the pipeline delay to exclude the network delay first. You can refer to DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums.