video playing not smooth caused by calling gst_nvvconv_transform() sometimes very time consumed

Robotics & Edge Computing Jetson Systems Jetson TX2
1

I have a pipeline decoding 8 IP cameras’ h264 video stream of 1920x1080@25fps simultaneously, and get RGBA video frames via nvvideosink’s EGL stream. Each camera pipeline like this: rtspsrc ! decodebin ! nvvidconv ! nvvideosink.

After profiling with nvvp tool, I find that gst_nvvconv_transform sometimes very time consumed. Normally, it takes within 1ms, but sometimes it takes tens of milliseconds.

By examining the tegrastat’s report, the usages of CPU/GPU/EMC/NVDEC all are not overload.

RAM 4498/7851MB (lfb 5x4MB) cpu [42%@498,off,off,32%@499,40%@499,34%@499] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 13%@522
RAM 4498/7851MB (lfb 5x4MB) cpu [45%@345,off,off,32%@345,34%@348,35%@348] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 31%@420
RAM 4500/7851MB (lfb 5x4MB) cpu [40%@805,off,off,39%@806,48%@807,35%@808] EMC 48%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 13%@318
RAM 4500/7851MB (lfb 5x4MB) cpu [41%@652,off,off,35%@652,36%@653,39%@652] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 74%@522
RAM 4500/7851MB (lfb 5x4MB) cpu [41%@345,off,off,36%@345,35%@344,31%@345] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 46%@420
RAM 4500/7851MB (lfb 5x4MB) cpu [44%@345,off,off,31%@345,37%@345,36%@345] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 51%@420
RAM 4498/7851MB (lfb 5x4MB) cpu [54%@2031,off,off,33%@2035,35%@2030,34%@2034] EMC 48%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 86%@726
RAM 4499/7851MB (lfb 5x4MB) cpu [46%@807,off,off,35%@807,34%@806,32%@807] EMC 48%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 80%@420
RAM 4499/7851MB (lfb 5x4MB) cpu [42%@1420,off,off,35%@1421,31%@1419,36%@1420] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 83%@216
RAM 4499/7851MB (lfb 5x4MB) cpu [38%@806,off,off,40%@806,32%@806,34%@808] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 41%@522
RAM 4498/7851MB (lfb 5x4MB) cpu [43%@960,off,off,33%@960,37%@960,34%@964] EMC 49%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 3%@624
RAM 4498/7851MB (lfb 5x4MB) cpu [43%@653,off,off,32%@652,47%@652,35%@656] EMC 48%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 88%@318
RAM 4498/7851MB (lfb 5x4MB) cpu [40%@805,off,off,31%@806,35%@652,38%@653] EMC 48%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 65%@624
RAM 4499/7851MB (lfb 5x4MB) cpu [43%@499,off,off,33%@499,35%@499,37%@499] EMC 47%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 62%@420
RAM 4464/7851MB (lfb 5x4MB) cpu [47%@806,off,off,31%@806,37%@806,34%@806] EMC 49%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 10%@522
RAM 4500/7851MB (lfb 5x4MB) cpu [54%@653,off,off,36%@652,48%@652,38%@652] EMC 48%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 76%@420
RAM 4500/7851MB (lfb 5x4MB) cpu [45%@345,off,off,35%@345,34%@345,36%@345] EMC 49%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 59%@522

I try running jetson_clocks.sh to maximize the system clock, and the issue disappears. I was wondering is there a problem with the system scheduling. How can i fix this, and any suggestions would be appreciate, thanks.

The attachments are the nvvp profile report, and screenshot of the issue’s timeline.

NewSession1.xml (3.01 MB)

timeline.png
timeline.png1666×861 118 KB

2

Hi SongjianSu,

Could you also provide the tegrastat’s report by running jetson_clocks.sh to maximize the system clock on your use case again as reference?
And, are you on r28.1?

Thanks

3

Hi kayccc，

Yes, r28.1, the TX2 was flashed via jetson 3.1.

Below is the tegrastat’s report on maximizing system clock.

RAM 4513/7851MB (lfb 5x4MB) cpu [45%@498,off,off,36%@502,38%@499,34%@502] EMC 53%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 34%@522
RAM 4478/7851MB (lfb 5x4MB) cpu [42%@653,off,off,32%@652,33%@652,32%@653] EMC 50%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 8%@522
RAM 4513/7851MB (lfb 5x4MB) cpu [40%@805,off,off,35%@810,54%@810,35%@810] EMC 50%@1866 APE 150 NVDEC 793 MSENC 217 GR3D 71%@624
RAM 4514/7851MB (lfb 5x4MB) cpu [45%@499,off,off,33%@498,32%@499,39%@498] EMC 49%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 54%@624
//<----- after start runnig jetson_clocks.sh
RAM 4515/7851MB (lfb 5x4MB) cpu [39%@2034,off,off,27%@2034,32%@2035,37%@2035] EMC 48%@1866 APE 150 NVDEC 576 MSENC 217 GR3D 56%@1134
RAM 4515/7851MB (lfb 5x4MB) cpu [21%@2027,off,off,8%@2027,15%@2024,15%@2028] EMC 55%@1600 APE 150 NVDEC 576 MSENC 217 GR3D 61%@1134
RAM 4515/7851MB (lfb 5x4MB) cpu [25%@2023,off,off,14%@2024,31%@2033,14%@2033] EMC 54%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 65%@1134
RAM 4514/7851MB (lfb 5x4MB) cpu [21%@2024,off,off,8%@2022,12%@2016,18%@2020] EMC 53%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 70%@1134
RAM 4514/7851MB (lfb 5x4MB) cpu [23%@2024,off,off,15%@2014,16%@2024,14%@2027] EMC 55%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 74%@1134
RAM 4514/7851MB (lfb 5x4MB) cpu [18%@2031,off,off,6%@2030,17%@2030,15%@2033] EMC 54%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 77%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [23%@2025,off,off,18%@2021,26%@2020,36%@2026] EMC 56%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 72%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [18%@2034,off,off,9%@2035,9%@2034,16%@2025] EMC 54%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 83%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [23%@2027,off,off,13%@2031,12%@2034,18%@2028] EMC 55%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 79%@1134
RAM 4514/7851MB (lfb 5x4MB) cpu [22%@2028,off,off,18%@2031,11%@2031,18%@2029] EMC 55%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 78%@1134
RAM 4514/7851MB (lfb 5x4MB) cpu [30%@2031,off,off,20%@2030,39%@2031,24%@2032] EMC 56%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 58%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [24%@2032,off,off,13%@2032,19%@2031,16%@2008] EMC 55%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 72%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [40%@2031,off,off,51%@2032,56%@2031,35%@2033] EMC 62%@1600 APE 150 NVDEC 576 MSENC 217 GR3D 90%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [47%@2033,off,off,34%@2033,40%@2033,32%@2034] EMC 65%@1600 APE 150 NVDEC 576 MSENC 217 GR3D 82%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [27%@2032,off,off,29%@2034,25%@2033,25%@2032] EMC 63%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 0%@1134
RAM 4514/7851MB (lfb 5x4MB) cpu [31%@2034,off,off,28%@2032,17%@2035,25%@2034] EMC 59%@1600 APE 150 NVDEC 576 MSENC 217 GR3D 0%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [19%@2030,off,off,8%@2034,8%@2034,14%@2034] EMC 56%@1600 APE 150 NVDEC 576 MSENC 217 GR3D 0%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [20%@2030,off,off,11%@2024,15%@2021,10%@2024] EMC 54%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 0%@1134
RAM 4486/7851MB (lfb 5x4MB) cpu [30%@2012,off,off,18%@2030,13%@2027,15%@2023] EMC 54%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 0%@1134
RAM 4513/7851MB (lfb 5x4MB) cpu [24%@2030,off,off,28%@2030,18%@2028,17%@2031] EMC 55%@1600 APE 150 NVDEC 1164 MSENC 217 GR3D 4%@1134
4

Hi SongjianSu,
It looks to be CPU scheduling issue. You may refer to jetson_clocks.sh and configure one or two CPUs always online.