BufferPool exceeds max expected | nvargus-daemon crashed?

When I run the capture process on Jetson TX2 (Jetpack 4.2.1) for more than 3 hours receiving {BufferPool exceeds max expected} errors

Captured using opencv.

nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160, format=(string)NV12, framerate=(fraction)8/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink

Got this error after ~3 hours.

PowerServiceCore:handleRequests: timePassed = 740
PowerServiceCore:handleRequests: timePassed = 1388
PowerServiceCore:handleRequests: timePassed = 691
PowerServiceCore:handleRequests: timePassed = 1297
...
SCF: Error OverFlow: Temp BufferPool exceeds max expected size: 7 (3864x2180 BL U8_V8_ER 420SP) (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 218)
SCF: Error OverFlow: Possibly due to pipeline slowdown, can cause memory bloat. (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 220)

Then below errors are coming indefinitely.

(Argus) Error InvalidState:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
(Argus) Error InvalidState:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
(Argus) Error InvalidState:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
(Argus) Error InvalidState:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
(Argus) Error InvalidState:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)

What these errors really indicate?
Is it because nvargus-daemon is crashed?

Any leads to resolve this would be great.

Thanks.

Replace the /usr/lib/aaxxx64/tegra/libnvscf.so by attached file and reboot to try.
libnvscf.so.txt (8.1 MB)

Hello ShaneCCC,

Replaced libnvscf.so as you suggested and ran process again after rebooting Jetson TX2.

But still receiving errors after ~4 hours.

Logs:

OpenCV: FFMPEG: tag 0x5634504d/'MP4V' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
aec **************** Converge took 91 frames
aec **************** Converge took 90 frames
aec **************** Converge took 99 frames
aec **************** Converge took 89 frames
aec **************** Converge took 61 frames
aec **************** Converge took 4 frames
aec **************** Converge took 129 frames
aec **************** Converge took 68 frames
aec **************** Converge took 46 frames
aec **************** Converge took 54 frames
aec **************** Converge took 74 frames
aec **************** Converge took 62 frames
PowerServiceCore:handleRequests: timePassed = 696
aec **************** Converge took 93 frames
aec **************** Converge took 81 frames
aec **************** Converge took 75 frames
aec **************** Converge took 144 frames
aec **************** Converge took 80 frames
aec **************** Converge took 101 frames
aec **************** Converge took 73 frames
aec **************** Converge took 50 frames
aec **************** Converge took 51 frames
aec **************** Converge took 80 frames
aec **************** Converge took 85 frames
aec **************** Converge took 85 frames
aec **************** Converge took 82 frames
aec **************** Converge took 61 frames
aec **************** Converge took 51 frames
aec **************** Converge took 106 frames
aec **************** Converge took 72 frames
aec **************** Converge took 62 frames
aec **************** Converge took 167 frames
aec **************** Converge took 75 frames
aec **************** Converge took 82 frames
aec **************** Converge took 74 frames
aec **************** Converge took 71 frames
aec **************** Converge took 68 frames
aec **************** Converge took 68 frames
aec **************** Converge took 77 frames
aec **************** Converge took 71 frames
aec **************** Converge took 76 frames
aec **************** Converge took 142 frames
aec **************** Converge took 71 frames
aec **************** Converge took 72 frames
aec **************** Converge took 78 frames
aec **************** Converge took 326 frames
aec **************** Converge took 73 frames
aec **************** Converge took 79 frames
aec **************** Converge took 61 frames
aec **************** Converge took 36 frames
aec **************** Converge took 37 frames
aec **************** Converge took 34 frames
aec **************** Converge took 35 frames
aec **************** Converge took 45 frames
aec **************** Converge took 36 frames
aec **************** Converge took 33 frames
PowerServiceCore:handleRequests: timePassed = 1368
aec **************** Converge took 42 frames
aec **************** Converge took 1671 frames
aec **************** Converge took 85 frames
aec **************** Converge took 15 frames
OpenCV: FFMPEG: tag 0x5634504d/'MP4V' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
aec **************** Converge took 1490 frames
aec **************** Converge took 72 frames
PowerServiceCore:handleRequests: timePassed = 768
PowerServiceCore:handleRequests: timePassed = 687
aec **************** Converge took 80 frames
aec **************** Converge took 90 frames
aec **************** Converge took 84 frames
aec **************** Converge took 74 frames
aec **************** Converge took 78 frames
PowerServiceCore:handleRequests: timePassed = 878
aec **************** Converge took 91 frames
aec **************** Converge took 82 frames
aec **************** Converge took 38 frames
SCF: Error OverFlow: Temp BufferPool exceeds max expected size: 7 (3864x2180 BL U8_V8_ER 420SP) (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 218)
SCF: Error OverFlow: Possibly due to pipeline slowdown, can cause memory bloat. (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 220)
Session 0 has 10 CCs in-flight:
  CC Id=118813 ApiId=118713 InstructionListId=0    5:TempBufferAcquireStage
  CC Id=118814 ApiId=118714 InstructionListId=0    5:TempBufferAcquireStage
  CC Id=118812 ApiId=118712 InstructionListId=0    9:SensorISPCaptureStage
  CC Id=118811 ApiId=118711 InstructionListId=0   15:BlitStage
  CC Id=118810 ApiId=118710 InstructionListId=0   15:BlitStage
  CC Id=118809 ApiId=118709 InstructionListId=0   15:BlitStage
  CC Id=118808 ApiId=118708 InstructionListId=0   15:BlitStage
  CC Id=118807 ApiId=118707 InstructionListId=0   15:BlitStage
  CC Id=118806 ApiId=118706 InstructionListId=0   15:BlitStage
  CC Id=118805 ApiId=118705 InstructionListId=0   15:BlitStage
SCF: Error Timeout:  (propagating from src/components/CaptureContainerImpl.cpp, function assignAllBuffersFromStream(), line 230)
SCF: Error Timeout:  (propagating from src/components/stages/CCDataSetupStage.cpp, function doHandleRequest(), line 68)
SCF: Error Timeout:  (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
SCF: Error Timeout: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 990)
SCF: Error OverFlow: Temp BufferPool exceeds max expected size: 7 (3864x2180 BL U8_V8_ER 420SP) (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 218)
SCF: Error OverFlow: Possibly due to pipeline slowdown, can cause memory bloat. (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 220)
Session 0 has 11 CCs in-flight:
  CC Id=118815 ApiId=118715 InstructionListId=0    5:TempBufferAcquireStage
  CC Id=118813 ApiId=118713 InstructionListId=0    9:SensorISPCaptureStage
  CC Id=118814 ApiId=118714 InstructionListId=0    5:TempBufferAcquireStage
  CC Id=118812 ApiId=118712 InstructionListId=0   19:BufferReturnStage
  CC Id=118811 ApiId=118711 InstructionListId=0   19:BufferReturnStage
  CC Id=118810 ApiId=118710 InstructionListId=0   19:BufferReturnStage
  CC Id=118809 ApiId=118709 InstructionListId=0   19:BufferReturnStage
  CC Id=118808 ApiId=118708 InstructionListId=0   19:BufferReturnStage
  CC Id=118807 ApiId=118707 InstructionListId=0   19:BufferReturnStage
  CC Id=118806 ApiId=118706 InstructionListId=0   19:BufferReturnStage
  CC Id=118805 ApiId=118705 InstructionListId=0   19:BufferReturnStage
SCF: Error OverFlow: Temp BufferPool exceeds max expected size: 7 (3864x2180 BL U8_V8_ER 420SP) (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 218)
SCF: Error OverFlow: Possibly due to pipeline slowdown, can cause memory bloat. (in src/services/buffermanager/BufferPool.cpp, function allocBuffer(), line 220)
PowerServiceCore:handleRequests: timePassed = 529

Could you please see if it is possible for Nvidia to reproduce it?
Any other solution to resolve this?

Thanks

Could you update to the latest BSP to confirm and we can repo it locally if you can provide the APP due to we didn’t see the problem with gst-launch-1.0 pipeline.

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), format=NV12' ! nvvidconv flip-method=0 ! videoconvert ! xvimagesink sync=false