Inference Throughput(FPS) Scaling with Interval Property

Hi,

I am trying to understand how the inference performance scales by manipulating the interval property (Ignoring the effectiveness of the use case) and what would be the potential limiters?

FYI, the pipeline was ran with YOLOv4-608 INT8 model.

gst-launch-1.0 -v \
nvstreammux name=m batch-size=1 width=1920 height=1080 ! \
nvinfer config-file-path= $CONFIG_FILE_PATH batch-size=1 unique-id=1 ! \
nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! nvdsosd !  nvegltransform ! fpsdisplaysink video-sink=nveglglessink text-overlay=false sync=false \
filesrc location=$VIDEO_0 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 \

Interval = 0
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1082, dropped: 0, current: 25.35, average: 24.84

Interval = 5
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 519, dropped: 0, current: 54.33, average: 53.30

Interval=10
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 364, dropped: 0, current: 53.69, average: 54.98

As you can see, the FPS doesnt scale linearly along with the interval value.
I have looked at the CPU Usage, GPU Usage, Memory Bandwidth with tegrastats and it seems there is plenty of headroom available.

RAM 5564/7766MB (lfb 108x4MB) SWAP 1099/3883MB (cached 42MB) CPU [26%@1420,26%@1420,32%@1420,23%@1420,14%@1420,29%@1420] EMC_FREQ 32%@1600 GR3D_FREQ 12%@1109 NVDEC 192 NVDEC1 192 VIC_FREQ 56%@204 APE 150 MTS fg 0% bg 11% AO@42.5C GPU@43C PMIC@100C AUX@40.5C CPU@42C thermal@41.7C VDD_IN 8081/7773 VDD_CPU_GPU_CV 3973/3727 VDD_SOC 1720/1699
RAM 5560/7766MB (lfb 108x4MB) SWAP 1099/3883MB (cached 42MB) CPU [23%@1420,33%@1420,29%@1420,33%@1420,25%@1420,28%@1420] EMC_FREQ 32%@1600 GR3D_FREQ 12%@1109 NVDEC 192 NVDEC1 192 VIC_FREQ 83%@115 APE 150 MTS fg 0% bg 11% AO@42.5C GPU@43C PMIC@100C AUX@40.5C CPU@42C thermal@41.7C VDD_IN 7753/7766 VDD_CPU_GPU_CV 3686/3713 VDD_SOC 1679/1692
RAM 5561/7766MB (lfb 108x4MB) SWAP 1099/3883MB (cached 42MB) CPU [28%@1420,31%@1420,33%@1420,21%@1420,46%@1420,26%@1420] EMC_FREQ 31%@1600 GR3D_FREQ 15%@1109 NVDEC 192 NVDEC1 192 VIC_FREQ 76%@115 APE 150 MTS fg 0% bg 9% AO@42.5C GPU@43C PMIC@100C AUX@40.5C CPU@42C thermal@42.05C VDD_IN 7589/7722 VDD_CPU_GPU_CV 3604/3686 VDD_SOC 1679/1689
RAM 5561/7766MB (lfb 108x4MB) SWAP 1099/3883MB (cached 42MB) CPU [20%@1420,21%@1420,33%@1420,23%@1420,21%@1420,15%@1420] EMC_FREQ 32%@1600 GR3D_FREQ 98%@1109 NVDEC 192 NVDEC1 192 VIC_FREQ 54%@115 APE 150 MTS fg 0% bg 10% AO@42.5C GPU@43.5C PMIC@100C AUX@41C CPU@42C thermal@41.7C VDD_IN 7999/7777 VDD_CPU_GPU_CV 3850/3718 VDD_SOC 1720/1695
RAM 5561/7766MB (lfb 108x4MB) SWAP 1099/3883MB (cached 42MB) CPU [21%@1420,23%@1420,30%@1420,26%@1420,16%@1420,18%@1420] EMC_FREQ 31%@1600 GR3D_FREQ 9%@1109 NVDEC 192 NVDEC1 192 VIC_FREQ 87%@115 APE 150 MTS fg 0% bg 10% AO@42.5C GPU@43C PMIC@100C AUX@41C CPU@41.5C thermal@41.9C VDD_IN 7753/7773 VDD_CPU_GPU_CV 3645/3706 VDD_SOC 1679/1692
RAM 5561/7766MB (lfb 108x4MB) SWAP 1099/3883MB (cached 42MB) CPU [21%@1420,14%@1420,30%@1420,20%@1420,13%@1420,18%@1420] EMC_FREQ 31%@1600 GR3D_FREQ 17%@1109 NVDEC 192 NVDEC1 192 VIC_FREQ 84%@115 APE 150 MTS fg 0% bg 10% AO@42.5C GPU@43C PMIC@100C AUX@41C CPU@42C thermal@41.75C VDD_IN 7507/7735 VDD_CPU_GPU_CV 3563/3686 VDD_SOC 1679/1690
RAM 5562/7766MB (lfb 108x4MB) SWAP 1099/3883MB (cached 42MB) CPU [17%@1420,17%@1420,28%@1420,19%@1420,14%@1420,20%@1420] EMC_FREQ 31%@1600 GR3D_FREQ 16%@1109 NVDEC 192 NVDEC1 192 VIC_FREQ 49%@115 APE 150 MTS fg 0% bg 10% AO@42.5C GPU@43C PMIC@100C AUX@41C CPU@42C thermal@42.05C VDD_IN 7425/7696 VDD_CPU_GPU_CV 3399/3650 VDD_SOC 1679/1689

• Hardware Platform (Jetson / GPU): Nvidia Xavier NX (15W 6CORE)
• DeepStream Version: 5.1
• JetPack Version (valid for Jetson only): 4.5.1
• TensorRT Version: 7.1.3.0-1+cuda10.2
• Issue Type( questions, new requirements, bugs): questions

Looking forward for your insights.

Thanks,
Vincent

There other components decoder, streammux, osd, nvvideoconverter, display sink affect the performance, it did not scale with the interval value.