Nvarguscamerasrc missing video frames

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
1

Hi,

ENV:
JP 4.5.2, Nano production module

I’m using gstreamer with nvarguscamerasrc to reading camera and I observed video frame missing in nvarguscamerasrc.

In code of nvarguscamerasrc element, I added some lines to check if the video frame is continuous or not:

// In function: bool StreamConsumer::threadExecute(GstNvArgusCameraSrc *src)
      if (!src->silent)
        GST_INFO_OBJECT(src, "Acquired Frame: %llu, time %llu\n",
                   static_cast<unsigned long long>(iFrame->getNumber()),
                   static_cast<unsigned long long>(iFrame->getTime()));
      // printf("LLL: frame %llu, time %llu\n", static_cast<unsigned long long>(iFrame->getNumber()), static_cast<unsigned long long>(iFrame->getTime()));
      if (((m_lastFrameIdx + 1) != iFrame->getNumber()) && (m_lastFrameIdx != 0)) {
        // Not continous frame
        printf("%d LLL: Frame index abnormal changed %lu -> %lu, time %lu -> %lu\n", __LINE__, m_lastFrameIdx, iFrame->getNumber(), m_lastFrameTime, iFrame->getTime());
      } else if (m_lastFrameIdx == 0) {
        printf("%d LLL: First frame %lu at %lu\n", __LINE__, iFrame->getNumber(), iFrame->getTime());
      }
      m_lastFrameIdx = iFrame->getNumber();
      m_lastFrameTime = iFrame->getTime();

Enabling nvargus-daemon log by:

export enableCamPclLogs=5
export enableCamScfLogs=5

When I run gstreamer pipeline, some video frame is missing in nvarguscamerasrc but looks normal in nvargus-daemon’s log.
On gstreamer pipeline:

LLL: Frame index abnormal changed 3675 -> 3983, time 7110775373000 -> 7121082582000
LLL: Frame index abnormal changed 3983 -> 3985, time 7121082582000 -> 7121117011000

On nvargus-daemon log, the frames from 3676 → 3982 are missing, but still available in nvargus-daemon.

3675 session 1
Thread 12 is working on CC 3675 session 1 globalID 3675 step 0
CC 3675 session 1 completed step 0 in fiber 0x7f380012f0
cc 3675(3575) session 1 runCount=3574 runIspOut=3569, latest ccId=3669
cc 3675(3575) session 1 waiting
AfAnalysis cc 3670 push FK_ISP_RUN_NUMBER=3570.
CC 3670 session 1 completed step 14 in fiber 0x7f38000fb0
cc 3675(3575) session 1 wait completed by ispOut cc 3670
AC Stats Delay Count:5
CC 3670 session 1 completed step 15 in fiber 0x7f38000fb0
CC 3675 session 1 completed step 1 in fiber 0x7f380012f0
NV AE and AfApply algorithms are active.
CC 3670 session 1 completed step 16 in fiber 0x7f38000fb0
CC 3670 session 1 completed step 17 in fiber 0x7f38000fb0
CC 3670 session 1 completed step 18 in fiber 0x7f38000fb0
CC 3670 session 1 completed step 19 in fiber 0x7f38000fb0
CC 3670 session 1 completed step 20 in fiber 0x7f38000fb0
FiberScheduler: cc 3670, session 1, fiber 0x7f38000fb0 complete

FiberScheduler: fiber 0x7f38000fb0 exiting
Thread 5 getting next capture
disposing CC 3670 Session 1
Thread 5 is waiting
Thread 10 is waiting
CC 3675 session 1 completed step 2 in fiber 0x7f380012f0
CC 3675 session 1 completed step 3 in fiber 0x7f380012f0
CC 3675 session 1 completed step 4 in fiber 0x7f380012f0
CC 3675 session 1 completed step 5 in fiber 0x7f380012f0
CC 3675 session 1 completed step 6 in fiber 0x7f380012f0
CC 3675 session 1 processing step 7 in fiber 0x7f380012f0
FiberScheduler: cc 3675, session 1, fiber 0x7f380012f0 in progress...

Thread 12 getting next capture
Thread 12 is waiting
Thread 1 is waiting
Thread 4 is waiting
FiberScheduler: cc 3671, session 1, fiber 0x7f38001b10 succeeded async operation

Thread 6 is working on CC 3671 session 1 globalID 3671 step 8
CC 3671 session 1 completed step 8 in fiber 0x7f38001b10
CC 3671 session 1 completed step 9 in fiber 0x7f38001b10
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.775
CC 3671 session 1 completed step 10 in fiber 0x7f38001b10
CC 3671 session 1 completed step 11 in fiber 0x7f38001b10
CC 3671 session 1 completed step 12 in fiber 0x7f38001b10
CC 3671 session 1 completed step 13 in fiber 0x7f38001b10
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38001970 for CC 3676 globalID 3676 session 1
Thread 7 is working on CC 3676 session 1 globalID 3676 step 0
CC 3676 session 1 completed step 0 in fiber 0x7f38001970
cc 3676(3576) session 1 runCount=3575 runIspOut=3570, latest ccId=3670
cc 3676(3576) session 1 waiting
AfAnalysis cc 3671 push FK_ISP_RUN_NUMBER=3571.
CC 3671 session 1 completed step 14 in fiber 0x7f38001b10
CC 3671 session 1 completed step 15 in fiber 0x7f38001b10cc 3676(3576) session 1 wait completed by ispOut cc 3671
AC Stats Delay Count:5

CC 3671 session 1 completed step 16 in fiber 0x7f38001b10
CC 3671 session 1 completed step 17 in fiber 0x7f38001b10
CC 3676 session 1 completed step 1 in fiber 0x7f38001970
NV AE and AfApply algorithms are active.
CC 3671 session 1 completed step 18 in fiber 0x7f38001b10
CC 3671 session 1 completed step 19 in fiber 0x7f38001b10
CC 3671 session 1 completed step 20 in fiber 0x7f38001b10
FiberScheduler: cc 3671, session 1, fiber 0x7f38001b10 complete

FiberScheduler: fiber 0x7f38001b10 exiting
Thread 6 getting next capturedisposing CC 3671 Session 1

Thread 11 is waiting
Thread 8 is waiting
Thread 6 is waiting
CC 3676 session 1 completed step 2 in fiber 0x7f38001970
CC 3676 session 1 completed step 3 in fiber 0x7f38001970
CC 3676 session 1 completed step 4 in fiber 0x7f38001970
CC 3676 session 1 completed step 5 in fiber 0x7f38001970
CC 3676 session 1 completed step 6 in fiber 0x7f38001970
CC 3676 session 1 processing step 7 in fiber 0x7f38001970
FiberScheduler: cc 3676, session 1, fiber 0x7f38001970 in progress...

Thread 7 getting next capture
Thread 3 is waiting
Thread 7 is waiting
FiberScheduler: cc 3672, session 1, fiber 0x7f38000b60 succeeded async operation

Thread 9 is working on CC 3672 session 1 globalID 3672 step 8
CC 3672 session 1 completed step 8 in fiber 0x7f38000b60
CC 3672 session 1 completed step 9 in fiber 0x7f38000b60
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.781
CC 3672 session 1 completed step 10 in fiber 0x7f38000b60
CC 3672 session 1 completed step 11 in fiber 0x7f38000b60
CC 3672 session 1 completed step 12 in fiber 0x7f38000b60
CC 3672 session 1 completed step 13 in fiber 0x7f38000b60
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38001e50 for CC 3677 globalID 3677 session 1
AfAnalysis cc 3672 push FK_ISP_RUN_NUMBER=3572.
CC 3672 session 1 completed step 14 in fiber 0x7f38000b60
CC 3672 session 1 completed step 15 in fiber 0x7f38000b60
CC 3672 session 1 completed step 16 in fiber 0x7f38000b60
CC 3672 session 1 completed step 17 in fiber 0x7f38000b60
Thread 2 is working on CC 3677 session 1 globalID 3677 step 0
CC 3677 session 1 completed step 0 in fiber 0x7f38001e50
CC 3672 session 1 completed step 18 in fiber 0x7f38000b60
CC 3672 session 1 completed step 19 in fiber 0x7f38000b60
CC 3672 session 1 completed step 20 in fiber 0x7f38000b60
FiberScheduler: cc 3672, session 1, fiber 0x7f38000b60 complete

FiberScheduler: fiber 0x7f38000b60 exiting
cc 3677(3577) session 1 runCount=3576 runIspOut=3572, latest ccId=3672
Thread 5 is waiting
AC Stats Delay Count:5
Thread 9 getting next capturedisposing CC 3672 Session 1

Thread 10 is waiting
CC 3677 session 1 completed step 1 in fiber 0x7f38001e50
Thread 9 is waiting
NV AE and AfApply algorithms are active.
CC 3677 session 1 completed step 2 in fiber 0x7f38001e50
CC 3677 session 1 completed step 3 in fiber 0x7f38001e50
CC 3677 session 1 completed step 4 in fiber 0x7f38001e50
CC 3677 session 1 completed step 5 in fiber 0x7f38001e50
CC 3677 session 1 completed step 6 in fiber 0x7f38001e50
CC 3677 session 1 processing step 7 in fiber 0x7f38001e50
FiberScheduler: cc 3677, session 1, fiber 0x7f38001e50 in progress...

Thread 2 getting next capture
Thread 2 is waiting
Thread 12 is waiting
FiberScheduler: cc 3673, session 1, fiber 0x7f38002110 succeeded async operation

Thread 1 is working on CC 3673 session 1 globalID 3673 step 8
CC 3673 session 1 completed step 8 in fiber 0x7f38002110
CC 3673 session 1 completed step 9 in fiber 0x7f38002110
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.798
CC 3673 session 1 completed step 10 in fiber 0x7f38002110
CC 3673 session 1 completed step 11 in fiber 0x7f38002110
CC 3673 session 1 completed step 12 in fiber 0x7f38002110
CC 3673 session 1 completed step 13 in fiber 0x7f38002110
NV AF analysis algorithm is active.
AfAnalysis cc 3673 push FK_ISP_RUN_NUMBER=3573.
CC 3673 session 1 completed step 14 in fiber 0x7f38002110
CC 3673 session 1 completed step 15 in fiber 0x7f38002110
CC 3673 session 1 completed step 16 in fiber 0x7f38002110
CC 3673 session 1 completed step 17 in fiber 0x7f38002110
CC 3673 session 1 completed step 18 in fiber 0x7f38002110
CC 3673 session 1 completed step 19 in fiber 0x7f38002110
CC 3673 session 1 completed step 20 in fiber 0x7f38002110
FiberScheduler: cc 3673, session 1, fiber 0x7f38002110 complete

FiberScheduler: fiber 0x7f38002110 exiting
Thread 1 getting next capture
disposing CC 3673 Session 1
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------Thread 4 is waiting

Thread 11 is waiting
NvPclSettingsApply: Reading PCL settings
Thread 1 is waiting
Created fiber 0x7f38001ff0 for CC 3678 globalID 3678 session 1
Thread 8 is working on CC 3678 session 1 globalID 3678 step 0
CC 3678 session 1 completed step 0 in fiber 0x7f38001ff0
cc 3678(3578) session 1 runCount=3577 runIspOut=3573, latest ccId=3673
AC Stats Delay Count:5
CC 3678 session 1 completed step 1 in fiber 0x7f38001ff0
NV AE and AfApply algorithms are active.
CC 3678 session 1 completed step 2 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 3 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 4 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 5 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 6 in fiber 0x7f38001ff0
CC 3678 session 1 processing step 7 in fiber 0x7f38001ff0
FiberScheduler: cc 3678, session 1, fiber 0x7f38001ff0 in progress...

Thread 8 getting next capture
Thread 6 is waiting
Thread 8 is waiting
FiberScheduler: cc 3674, session 1, fiber 0x7f38001630 succeeded async operation

Thread 3 is working on CC 3674 session 1 globalID 3674 step 8
CC 3674 session 1 completed step 8 in fiber 0x7f38001630
CC 3674 session 1 completed step 9 in fiber 0x7f38001630
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.830
CC 3674 session 1 completed step 10 in fiber 0x7f38001630
CC 3674 session 1 completed step 11 in fiber 0x7f38001630
CC 3674 session 1 completed step 12 in fiber 0x7f38001630
CC 3674 session 1 completed step 13 in fiber 0x7f38001630
NV AF analysis algorithm is active.
AfAnalysis cc 3674 push FK_ISP_RUN_NUMBER=3574.
CC 3674 session 1 completed step 14 in fiber 0x7f38001630
CC 3674 session 1 completed step 15 in fiber 0x7f38001630
CC 3674 session 1 completed step 16 in fiber 0x7f38001630
CC 3674 session 1 completed step 17 in fiber 0x7f38001630
CC 3674 session 1 completed step 18 in fiber 0x7f38001630
CC 3674 session 1 completed step 19 in fiber 0x7f38001630
CC 3674 session 1 completed step 20 in fiber 0x7f38001630
FiberScheduler: cc 3674, session 1, fiber 0x7f38001630 complete

FiberScheduler: fiber 0x7f38001630 exiting
Thread 3 getting next capture
disposing CC 3674 Session 1
Thread 3 is waiting
Thread 5 is waiting
Thread 7 is waiting
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
Created fiber 0x7f380017d0 for CC 3679 globalID 3679 session 1Thread 10 is working on CC 3679 session 1 globalID 3679 step 0apply:+++++++++++++++++++++++


apply:-----------------------
NvPclSettingsApply: Reading PCL settings
CC 3679 session 1 completed step 0 in fiber 0x7f380017d0
cc 3679(3579) session 1 runCount=3578 runIspOut=3574, latest ccId=3674
AC Stats Delay Count:5
CC 3679 session 1 completed step 1 in fiber 0x7f380017d0
NV AE and AfApply algorithms are active.
CC 3679 session 1 completed step 2 in fiber 0x7f380017d0
CC 3679 session 1 completed step 3 in fiber 0x7f380017d0
CC 3679 session 1 completed step 4 in fiber 0x7f380017d0
CC 3679 session 1 completed step 5 in fiber 0x7f380017d0
CC 3679 session 1 completed step 6 in fiber 0x7f380017d0
CC 3679 session 1 processing step 7 in fiber 0x7f380017d0
FiberScheduler: cc 3679, session 1, fiber 0x7f380017d0 in progress...

Thread 10 getting next capture
Thread 9 is waiting
Thread 10 is waiting
FiberScheduler: cc 3675, session 1, fiber 0x7f380012f0 succeeded async operation

Thread 2 is working on CC 3675 session 1 globalID 3675 step 8
CC 3675 session 1 completed step 8 in fiber 0x7f380012f0
CC 3675 session 1 completed step 9 in fiber 0x7f380012f0
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.838
CC 3675 session 1 completed step 10 in fiber 0x7f380012f0
CC 3675 session 1 completed step 11 in fiber 0x7f380012f0
CC 3675 session 1 completed step 12 in fiber 0x7f380012f0
CC 3675 session 1 completed step 13 in fiber 0x7f380012f0
NV AF analysis algorithm is active.
AfAnalysis cc 3675 push FK_ISP_RUN_NUMBER=3575.
CC 3675 session 1 completed step 14 in fiber 0x7f380012f0
CC 3675 session 1 completed step 15 in fiber 0x7f380012f0
CC 3675 session 1 completed step 16 in fiber 0x7f380012f0
CC 3675 session 1 completed step 17 in fiber 0x7f380012f0
CC 3675 session 1 completed step 18 in fiber 0x7f380012f0
CC 3675 session 1 completed step 19 in fiber 0x7f380012f0
CC 3675 session 1 completed step 20 in fiber 0x7f380012f0
FiberScheduler: cc 3675, session 1, fiber 0x7f380012f0 complete

FiberScheduler: fiber 0x7f380012f0 exiting
Thread 2 getting next capture
disposing CC 3675 Session 1
Thread 2 is waiting
Thread 12 is waiting
Thread 4 is waiting
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38001490 for CC 3680 globalID 3680 session 1
Thread 11 is working on CC 3680 session 1 globalID 3680 step 0
CC 3680 session 1 completed step 0 in fiber 0x7f38001490
cc 3680(3580) session 1 runCount=3579 runIspOut=3575, latest ccId=3675
AC Stats Delay Count:5
CC 3680 session 1 completed step 1 in fiber 0x7f38001490
NV AE and AfApply algorithms are active.
CC 3680 session 1 completed step 2 in fiber 0x7f38001490
CC 3680 session 1 completed step 3 in fiber 0x7f38001490
CC 3680 session 1 completed step 4 in fiber 0x7f38001490
CC 3680 session 1 completed step 5 in fiber 0x7f38001490
CC 3680 session 1 completed step 6 in fiber 0x7f38001490
CC 3680 session 1 processing step 7 in fiber 0x7f38001490
FiberScheduler: cc 3680, session 1, fiber 0x7f38001490 in progress...

Thread 11 getting next capture
Thread 11 is waiting
Thread 1 is waiting
FiberScheduler: cc 3676, session 1, fiber 0x7f38001970 succeeded async operation

Thread 6 is working on CC 3676 session 1 globalID 3676 step 8
CC 3676 session 1 completed step 8 in fiber 0x7f38001970
CC 3676 session 1 completed step 9 in fiber 0x7f38001970
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.866
CC 3676 session 1 completed step 10 in fiber 0x7f38001970
CC 3676 session 1 completed step 11 in fiber 0x7f38001970
CC 3676 session 1 completed step 12 in fiber 0x7f38001970
CC 3676 session 1 completed step 13 in fiber 0x7f38001970
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f380020d0 for CC 3681 globalID 3681 session 1
Thread 8 is working on CC 3681 session 1 globalID 3681 step 0
CC 3681 session 1 completed step 0 in fiber 0x7f380020d0
cc 3681(3581) session 1 runCount=3580 runIspOut=3575, latest ccId=3675
cc 3681(3581) session 1 waiting
AfAnalysis cc 3676 push FK_ISP_RUN_NUMBER=3576.
CC 3676 session 1 completed step 14 in fiber 0x7f38001970
cc 3681(3581) session 1 wait completed by ispOut cc 3676
AC Stats Delay Count:5
CC 3676 session 1 completed step 15 in fiber 0x7f38001970
CC 3681 session 1 completed step 1 in fiber 0x7f380020d0
NV AE and AfApply algorithms are active.
CC 3676 session 1 completed step 16 in fiber 0x7f38001970
CC 3676 session 1 completed step 17 in fiber 0x7f38001970
CC 3676 session 1 completed step 18 in fiber 0x7f38001970
CC 3676 session 1 completed step 19 in fiber 0x7f38001970
CC 3676 session 1 completed step 20 in fiber 0x7f38001970
FiberScheduler: cc 3676, session 1, fiber 0x7f38001970 complete

FiberScheduler: fiber 0x7f38001970 exiting
Thread 6 getting next capture
disposing CC 3676 Session 1
Thread 3 is waiting
Thread 6 is waiting
Thread 5 is waiting
CC 3681 session 1 completed step 2 in fiber 0x7f380020d0
CC 3681 session 1 completed step 3 in fiber 0x7f380020d0
CC 3681 session 1 completed step 4 in fiber 0x7f380020d0
CC 3681 session 1 completed step 5 in fiber 0x7f380020d0
CC 3681 session 1 completed step 6 in fiber 0x7f380020d0
CC 3681 session 1 processing step 7 in fiber 0x7f380020d0
FiberScheduler: cc 3681, session 1, fiber 0x7f380020d0 in progress...

Thread 8 getting next captureThread 7 is waiting

Thread 8 is waiting
FiberScheduler: cc 3677, session 1, fiber 0x7f38001e50 succeeded async operation

Thread 9 is working on CC 3677 session 1 globalID 3677 step 8
CC 3677 session 1 completed step 8 in fiber 0x7f38001e50
CC 3677 session 1 completed step 9 in fiber 0x7f38001e50
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.874
CC 3677 session 1 completed step 10 in fiber 0x7f38001e50
CC 3677 session 1 completed step 11 in fiber 0x7f38001e50
CC 3677 session 1 completed step 12 in fiber 0x7f38001e50
CC 3677 session 1 completed step 13 in fiber 0x7f38001e50
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38001cb0 for CC 3682 globalID 3682 session 1
Thread 10 is working on CC 3682 session 1 globalID 3682 step 0milliseconds
CC 3682 session 1 completed step 0 in fiber 0x7f38001cb0
cc 3682(3582) session 1 runCount=3581 runIspOut=3576, latest ccId=3676
cc 3682(3582) session 1 waiting
AfAnalysis cc 3677 push FK_ISP_RUN_NUMBER=3577.
CC 3677 session 1 completed step 14 in fiber 0x7f38001e50
CC 3677 session 1 completed step 15 in fiber 0x7f38001e50
cc 3682(3582) session 1 wait completed by ispOut cc 3677
AC Stats Delay Count:5
CC 3677 session 1 completed step 16 in fiber 0x7f38001e50
CC 3677 session 1 completed step 17 in fiber 0x7f38001e50
CC 3682 session 1 completed step 1 in fiber 0x7f38001cb0
NV AE and AfApply algorithms are active.
CC 3677 session 1 completed step 18 in fiber 0x7f38001e50
CC 3677 session 1 completed step 19 in fiber 0x7f38001e50
CC 3677 session 1 completed step 20 in fiber 0x7f38001e50
FiberScheduler: cc 3677, session 1, fiber 0x7f38001e50 complete

FiberScheduler: fiber 0x7f38001e50 exiting
Thread 9 getting next capture
disposing CC 3677 Session 1
Thread 9 is waiting
Thread 12 is waiting
Thread 2 is waiting
CC 3682 session 1 completed step 2 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 3 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 4 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 5 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 6 in fiber 0x7f38001cb0
CC 3682 session 1 processing step 7 in fiber 0x7f38001cb0
FiberScheduler: cc 3682, session 1, fiber 0x7f38001cb0 in progress...

Thread 10 getting next capture
Thread 4 is waiting
Thread 10 is waiting
FiberScheduler: cc 3678, session 1, fiber 0x7f38001ff0 succeeded async operation

Thread 11 is working on CC 3678 session 1 globalID 3678 step 8
CC 3678 session 1 completed step 8 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 9 in fiber 0x7f38001ff0
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.874
CC 3678 session 1 completed step 10 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 11 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 12 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 13 in fiber 0x7f38001ff0
NV AF analysis algorithm is active.
AfAnalysis cc 3678 push FK_ISP_RUN_NUMBER=3578.
CC 3678 session 1 completed step 14 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 15 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 16 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 17 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 18 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 19 in fiber 0x7f38001ff0
CC 3678 session 1 completed step 20 in fiber 0x7f38001ff0
FiberScheduler: cc 3678, session 1, fiber 0x7f38001ff0 complete

FiberScheduler: fiber 0x7f38001ff0 exiting
Thread 11 getting next capture
disposing CC 3678 Session 1
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
Thread 3 is waiting
apply:+++++++++++++++++++++++
apply:-----------------------
Thread 1 is waiting
NvPclSettingsApply: Reading PCL settings
Thread 11 is waiting
Created fiber 0x7f38001ff0 for CC 3683 globalID 3683 session 1
Thread 6 is working on CC 3683 session 1 globalID 3683 step 0
CC 3683 session 1 completed step 0 in fiber 0x7f38001ff0
cc 3683(3583) session 1 runCount=3582 runIspOut=3578, latest ccId=3678
AC Stats Delay Count:5
CC 3683 session 1 completed step 1 in fiber 0x7f38001ff0
NV AE and AfApply algorithms are active.
CC 3683 session 1 completed step 2 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 3 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 4 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 5 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 6 in fiber 0x7f38001ff0
CC 3683 session 1 processing step 7 in fiber 0x7f38001ff0
FiberScheduler: cc 3683, session 1, fiber 0x7f38001ff0 in progress...

Thread 6 getting next capture
Thread 5 is waiting
Thread 6 is waiting
FiberScheduler: cc 3679, session 1, fiber 0x7f380017d0 succeeded async operation

Thread 7 is working on CC 3679 session 1 globalID 3679 step 8
CC 3679 session 1 completed step 8 in fiber 0x7f380017d0
CC 3679 session 1 completed step 9 in fiber 0x7f380017d0
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.886
CC 3679 session 1 completed step 10 in fiber 0x7f380017d0
CC 3679 session 1 completed step 11 in fiber 0x7f380017d0
CC 3679 session 1 completed step 12 in fiber 0x7f380017d0
CC 3679 session 1 completed step 13 in fiber 0x7f380017d0
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38001b10 for CC 3684 globalID 3684 session 1
Thread 8 is working on CC 3684 session 1 globalID 3684 step 0
CC 3684 session 1 completed step 0 in fiber 0x7f38001b10
cc 3684(3584) session 1 runCount=3583 runIspOut=3578, latest ccId=3678
cc 3684(3584) session 1 waiting
AfAnalysis cc 3679 push FK_ISP_RUN_NUMBER=3579.
CC 3679 session 1 completed step 14 in fiber 0x7f380017d0
cc 3684(3584) session 1 wait completed by ispOut cc 3679
AC Stats Delay Count:5
CC 3679 session 1 completed step 15 in fiber 0x7f380017d0
CC 3684 session 1 completed step 1 in fiber 0x7f38001b10
NV AE and AfApply algorithms are active.
CC 3679 session 1 completed step 16 in fiber 0x7f380017d0
CC 3679 session 1 completed step 17 in fiber 0x7f380017d0
CC 3679 session 1 completed step 18 in fiber 0x7f380017d0
CC 3679 session 1 completed step 19 in fiber 0x7f380017d0
CC 3679 session 1 completed step 20 in fiber 0x7f380017d0
FiberScheduler: cc 3679, session 1, fiber 0x7f380017d0 complete

FiberScheduler: fiber 0x7f380017d0 exiting
Thread 7 getting next capture
disposing CC 3679 Session 1
Thread 9 is waiting
CC 3684 session 1 completed step 2 in fiber 0x7f38001b10
CC 3684 session 1 completed step 3 in fiber 0x7f38001b10
Thread 7 is waiting
Thread 12 is waiting
CC 3684 session 1 completed step 4 in fiber 0x7f38001b10
CC 3684 session 1 completed step 5 in fiber 0x7f38001b10
CC 3684 session 1 completed step 6 in fiber 0x7f38001b10
CC 3684 session 1 processing step 7 in fiber 0x7f38001b10
FiberScheduler: cc 3684, session 1, fiber 0x7f38001b10 in progress...

Thread 8 getting next captureThread 2 is waiting

Thread 8 is waiting
FiberScheduler: cc 3680, session 1, fiber 0x7f38001490 succeeded async operation

Thread 4 is working on CC 3680 session 1 globalID 3680 step 8
CC 3680 session 1 completed step 8 in fiber 0x7f38001490
CC 3680 session 1 completed step 9 in fiber 0x7f38001490
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.902
CC 3680 session 1 completed step 10 in fiber 0x7f38001490
CC 3680 session 1 completed step 11 in fiber 0x7f38001490
CC 3680 session 1 completed step 12 in fiber 0x7f38001490
CC 3680 session 1 completed step 13 in fiber 0x7f38001490
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38000b60 for CC 3685 globalID 3685 session 1
Thread 10 is working on CC 3685 session 1 globalID 3685 step 0
CC 3685 session 1 completed step 0 in fiber 0x7f38000b60
cc 3685(3585) session 1 runCount=3584 runIspOut=3579, latest ccId=3679
cc 3685(3585) session 1 waiting
AfAnalysis cc 3680 push FK_ISP_RUN_NUMBER=3580.
CC 3680 session 1 completed step 14 in fiber 0x7f38001490
CC 3680 session 1 completed step 15 in fiber 0x7f38001490
cc 3685(3585) session 1 wait completed by ispOut cc 3680
AC Stats Delay Count:5
CC 3680 session 1 completed step 16 in fiber 0x7f38001490
CC 3680 session 1 completed step 17 in fiber 0x7f38001490
CC 3685 session 1 completed step 1 in fiber 0x7f38000b60
NV AE and AfApply algorithms are active.
CC 3685 session 1 completed step 2 in fiber 0x7f38000b60
CC 3685 session 1 completed step 3 in fiber 0x7f38000b60
CC 3680 session 1 completed step 18 in fiber 0x7f38001490
CC 3680 session 1 completed step 19 in fiber 0x7f38001490
CC 3680 session 1 completed step 20 in fiber 0x7f38001490
FiberScheduler: cc 3680, session 1, fiber 0x7f38001490 complete

FiberScheduler: fiber 0x7f38001490 exiting
Thread 4 getting next capture
disposing CC 3680 Session 1
Thread 3 is waiting
Thread 4 is waiting
Thread 1 is waiting
CC 3685 session 1 completed step 4 in fiber 0x7f38000b60
CC 3685 session 1 completed step 5 in fiber 0x7f38000b60
CC 3685 session 1 completed step 6 in fiber 0x7f38000b60
CC 3685 session 1 processing step 7 in fiber 0x7f38000b60
FiberScheduler: cc 3685, session 1, fiber 0x7f38000b60 in progress...

Thread 10 getting next capture
Thread 11 is waiting
Thread 10 is waiting
FiberScheduler: cc 3681, session 1, fiber 0x7f380020d0 succeeded async operation

Thread 5 is working on CC 3681 session 1 globalID 3681 step 8
CC 3681 session 1 completed step 8 in fiber 0x7f380020d0
CC 3681 session 1 completed step 9 in fiber 0x7f380020d0
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.840
CC 3681 session 1 completed step 10 in fiber 0x7f380020d0
CC 3681 session 1 completed step 11 in fiber 0x7f380020d0
CC 3681 session 1 completed step 12 in fiber 0x7f380020d0
CC 3681 session 1 completed step 13 in fiber 0x7f380020d0
NV AF analysis algorithm is active.
AfAnalysis cc 3681 push FK_ISP_RUN_NUMBER=3581.
CC 3681 session 1 completed step 14 in fiber 0x7f380020d0
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCLCC 3681 session 1 completed step 15 in fiber 0x7f380020d0

apply:+++++++++++++++++++++++
apply:-----------------------Created fiber 0x7f38002110 for CC 3686 globalID 3686 session 1Thread 6 is working on CC 3686 session 1 globalID 3686 step 0

CC 3686 session 1 completed step 0 in fiber 0x7f38002110
CC 3681 session 1 completed step 16 in fiber 0x7f380020d0
cc 3686(3586) session 1 runCount=3585 runIspOut=3581, latest ccId=3681
AC Stats Delay Count:5
CC 3681 session 1 completed step 17 in fiber 0x7f380020d0

CC 3686 session 1 completed step 1 in fiber 0x7f38002110
NvPclSettingsApply: Reading PCL settings
NV AE and AfApply algorithms are active.CC 3681 session 1 completed step 18 in fiber 0x7f380020d0

CC 3681 session 1 completed step 19 in fiber 0x7f380020d0
CC 3681 session 1 completed step 20 in fiber 0x7f380020d0
FiberScheduler: cc 3681, session 1, fiber 0x7f380020d0 complete

FiberScheduler: fiber 0x7f380020d0 exiting
Thread 5 getting next capture
disposing CC 3681 Session 1
Thread 7 is waiting
Thread 5 is waiting
Thread 9 is waiting
CC 3686 session 1 completed step 2 in fiber 0x7f38002110
CC 3686 session 1 completed step 3 in fiber 0x7f38002110
CC 3686 session 1 completed step 4 in fiber 0x7f38002110
CC 3686 session 1 completed step 5 in fiber 0x7f38002110
CC 3686 session 1 completed step 6 in fiber 0x7f38002110
CC 3686 session 1 processing step 7 in fiber 0x7f38002110
FiberScheduler: cc 3686, session 1, fiber 0x7f38002110 in progress...

Thread 6 getting next capture
Thread 12 is waiting
Thread 6 is waiting
FiberScheduler: cc 3682, session 1, fiber 0x7f38001cb0 succeeded async operation

Thread 2 is working on CC 3682 session 1 globalID 3682 step 8
CC 3682 session 1 completed step 8 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 9 in fiber 0x7f38001cb0
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.837
CC 3682 session 1 completed step 10 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 11 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 12 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 13 in fiber 0x7f38001cb0
NV AF analysis algorithm is active.
AfAnalysis cc 3682 push FK_ISP_RUN_NUMBER=3582.
CC 3682 session 1 completed step 14 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 15 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 16 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 17 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 18 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 19 in fiber 0x7f38001cb0
CC 3682 session 1 completed step 20 in fiber 0x7f38001cb0
FiberScheduler: cc 3682, session 1, fiber 0x7f38001cb0 complete

FiberScheduler: fiber 0x7f38001cb0 exiting
Thread 2 getting next capture
disposing CC 3682 Session 1
Thread 8 is waiting
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38001630 for CC 3687 globalID 3687 session 1
Thread 4 is working on CC 3687 session 1 globalID 3687 step 0
CC 3687 session 1 completed step 0 in fiber 0x7f38001630
cc 3687(3587) session 1 runCount=3586 runIspOut=3582, latest ccId=3682
AC Stats Delay Count:5
CC 3687 session 1 completed step 1 in fiber 0x7f38001630
NV AE and AfApply algorithms are active.
Thread 3 is waiting
Thread 2 is waiting
CC 3687 session 1 completed step 2 in fiber 0x7f38001630
CC 3687 session 1 completed step 3 in fiber 0x7f38001630
CC 3687 session 1 completed step 4 in fiber 0x7f38001630
CC 3687 session 1 completed step 5 in fiber 0x7f38001630
CC 3687 session 1 completed step 6 in fiber 0x7f38001630
CC 3687 session 1 processing step 7 in fiber 0x7f38001630
FiberScheduler: cc 3687, session 1, fiber 0x7f38001630 in progress...

Thread 4 getting next capture
Thread 1 is waiting
Thread 4 is waiting
FiberScheduler: cc 3683, session 1, fiber 0x7f38001ff0 succeeded async operation

Thread 11 is working on CC 3683 session 1 globalID 3683 step 8
CC 3683 session 1 completed step 8 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 9 in fiber 0x7f38001ff0
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.833
CC 3683 session 1 completed step 10 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 11 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 12 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 13 in fiber 0x7f38001ff0
NV AF analysis algorithm is active.
AfAnalysis cc 3683 push FK_ISP_RUN_NUMBER=3583.
CC 3683 session 1 completed step 14 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 15 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 16 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 17 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 18 in fiber 0x7f38001ff0
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
CC 3683 session 1 completed step 19 in fiber 0x7f38001ff0
CC 3683 session 1 completed step 20 in fiber 0x7f38001ff0
FiberScheduler: cc 3683, session 1, fiber 0x7f38001ff0 complete

FiberScheduler: fiber 0x7f38001ff0 exiting
Thread 11 getting next capturedisposing CC 3683 Session 1

Thread 10 is waiting
Thread 7 is working on CC 3688 session 1 globalID 3688 step 0
Created fiber 0x7f380012f0 for CC 3688 globalID 3688 session 1
Thread 11 is waiting
Thread 5 is waiting
CC 3688 session 1 completed step 0 in fiber 0x7f380012f0
cc 3688(3588) session 1 runCount=3587 runIspOut=3583, latest ccId=3683
AC Stats Delay Count:5
CC 3688 session 1 completed step 1 in fiber 0x7f380012f0
NV AE and AfApply algorithms are active.
CC 3688 session 1 completed step 2 in fiber 0x7f380012f0
CC 3688 session 1 completed step 3 in fiber 0x7f380012f0
CC 3688 session 1 completed step 4 in fiber 0x7f380012f0
CC 3688 session 1 completed step 5 in fiber 0x7f380012f0
CC 3688 session 1 completed step 6 in fiber 0x7f380012f0
CC 3688 session 1 processing step 7 in fiber 0x7f380012f0
FiberScheduler: cc 3688, session 1, fiber 0x7f380012f0 in progress...

Thread 7 getting next capture
Thread 9 is waiting
Thread 7 is waiting
FiberScheduler: cc 3684, session 1, fiber 0x7f38001b10 succeeded async operation

Thread 12 is working on CC 3684 session 1 globalID 3684 step 8
CC 3684 session 1 completed step 8 in fiber 0x7f38001b10
CC 3684 session 1 completed step 9 in fiber 0x7f38001b10
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.828
CC 3684 session 1 completed step 10 in fiber 0x7f38001b10
CC 3684 session 1 completed step 11 in fiber 0x7f38001b10
CC 3684 session 1 completed step 12 in fiber 0x7f38001b10
CC 3684 session 1 completed step 13 in fiber 0x7f38001b10
NV AF analysis algorithm is active.
AfAnalysis cc 3684 push FK_ISP_RUN_NUMBER=3584.
CC 3684 session 1 completed step 14 in fiber 0x7f38001b10
CC 3684 session 1 completed step 15 in fiber 0x7f38001b10
CC 3684 session 1 completed step 16 in fiber 0x7f38001b10
CC 3684 session 1 completed step 17 in fiber 0x7f38001b10
CC 3684 session 1 completed step 18 in fiber 0x7f38001b10
CC 3684 session 1 completed step 19 in fiber 0x7f38001b10
CC 3684 session 1 completed step 20 in fiber 0x7f38001b10
FiberScheduler: cc 3684, session 1, fiber 0x7f38001b10 complete

FiberScheduler: fiber 0x7f38001b10 exiting
Thread 12 getting next capture
disposing CC 3684 Session 1
Thread 12 is waiting
Thread 8 is waiting
Thread 6 is waiting
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
Created fiber 0x7f38001970 for CC 3689 globalID 3689 session 1
Thread 3 is working on CC 3689 session 1 globalID 3689 step 0
CC 3689 session 1 completed step 0 in fiber 0x7f38001970
cc 3689(3589) session 1 runCount=3588 runIspOut=3584, latest ccId=3684
AC Stats Delay Count:5
CC 3689 session 1 completed step 1 in fiber 0x7f38001970
NV AE and AfApply algorithms are active.
CC 3689 session 1 completed step 2 in fiber 0x7f38001970
CC 3689 session 1 completed step 3 in fiber 0x7f38001970
CC 3689 session 1 completed step 4 in fiber 0x7f38001970
CC 3689 session 1 completed step 5 in fiber 0x7f38001970
CC 3689 session 1 completed step 6 in fiber 0x7f38001970
CC 3689 session 1 processing step 7 in fiber 0x7f38001970
FiberScheduler: cc 3689, session 1, fiber 0x7f38001970 in progress...

Thread 3 getting next capture
Thread 2 is waiting
Thread 3 is waiting
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
FiberScheduler: cc 3685, session 1, fiber 0x7f38000b60 succeeded async operation

Thread 1 is working on CC 3685 session 1 globalID 3685 step 8
CC 3685 session 1 completed step 8 in fiber 0x7f38000b60
CC 3685 session 1 completed step 9 in fiber 0x7f38000b60
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.825
CC 3685 session 1 completed step 10 in fiber 0x7f38000b60
CC 3685 session 1 completed step 11 in fiber 0x7f38000b60
CC 3685 session 1 completed step 12 in fiber 0x7f38000b60
CC 3685 session 1 completed step 13 in fiber 0x7f38000b60
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38001e50 for CC 3690 globalID 3690 session 1
Thread 4 is working on CC 3690 session 1 globalID 3690 step 0
CC 3690 session 1 completed step 0 in fiber 0x7f38001e50
cc 3690(3590) session 1 runCount=3589 runIspOut=3584, latest ccId=3684
cc 3690(3590) session 1 waiting
AfAnalysis cc 3685 push FK_ISP_RUN_NUMBER=3585.
CC 3685 session 1 completed step 14 in fiber 0x7f38000b60
cc 3690(3590) session 1 wait completed by ispOut cc 3685CC 3685 session 1 completed step 15 in fiber 0x7f38000b60

AC Stats Delay Count:5
CC 3685 session 1 completed step 16 in fiber 0x7f38000b60
CC 3690 session 1 completed step 1 in fiber 0x7f38001e50
CC 3685 session 1 completed step 17 in fiber 0x7f38000b60
NV AE and AfApply algorithms are active.
CC 3685 session 1 completed step 18 in fiber 0x7f38000b60
CC 3685 session 1 completed step 19 in fiber 0x7f38000b60
CC 3685 session 1 completed step 20 in fiber 0x7f38000b60
FiberScheduler: cc 3685, session 1, fiber 0x7f38000b60 complete

FiberScheduler: fiber 0x7f38000b60 exiting
Thread 1 getting next capture
disposing CC 3685 Session 1
Thread 10 is waiting
Thread 1 is waiting
Thread 11 is waiting
CC 3690 session 1 completed step 2 in fiber 0x7f38001e50
CC 3690 session 1 completed step 3 in fiber 0x7f38001e50
CC 3690 session 1 completed step 4 in fiber 0x7f38001e50
CC 3690 session 1 completed step 5 in fiber 0x7f38001e50
CC 3690 session 1 completed step 6 in fiber 0x7f38001e50
CC 3690 session 1 processing step 7 in fiber 0x7f38001e50
FiberScheduler: cc 3690, session 1, fiber 0x7f38001e50 in progress...

Thread 4 getting next captureThread 5 is waiting

Thread 4 is waiting
FiberScheduler: cc 3686, session 1, fiber 0x7f38002110 succeeded async operation

Thread 9 is working on CC 3686 session 1 globalID 3686 step 8
CC 3686 session 1 completed step 8 in fiber 0x7f38002110
CC 3686 session 1 completed step 9 in fiber 0x7f38002110
exposureTime=0.033330 analogGain=1.678902 digitalGain=1.000000 commonGain=0.055958 expComp=1.000000

lux in Statsupdate: isAohdrEnable=0 currentSceneLux 262.832
CC 3686 session 1 completed step 10 in fiber 0x7f38002110
CC 3686 session 1 completed step 11 in fiber 0x7f38002110
CC 3686 session 1 completed step 12 in fiber 0x7f38002110
CC 3686 session 1 completed step 13 in fiber 0x7f38002110
NV AF analysis algorithm is active.
NvPclSettingsUpdate: Sending Updated Settings through PCL
NvPclSettingsApply: Applying last settings through PCL
apply:+++++++++++++++++++++++
apply:-----------------------
NvPclSettingsApply: Reading PCL settings
Created fiber 0x7f38000b60 for CC 3691 globalID 3691 session 1
Thread 7 is working on CC 3691 session 1 globalID 3691 step 0
CC 3691 session 1 completed step 0 in fiber 0x7f38000b60
cc 3691(3591) session 1 runCount=3590 runIspOut=3585, latest ccId=3685
cc 3691(3591) session 1 waiting
AfAnalysis cc 3686 push FK_ISP_RUN_NUMBER=3586.
CC 3686 session 1 completed step 14 in fiber 0x7f38002110
CC 3686 session 1 completed step 15 in fiber 0x7f38002110
cc 3691(3591) session 1 wait completed by ispOut cc 3686
AC Stats Delay Count:5
CC 3686 session 1 completed step 16 in fiber 0x7f38002110CC 3691 session 1 completed step 1 in fiber 0x7f38000b60

NV AE and AfApply algorithms are active.
CC 3686 session 1 completed step 17 in fiber 0x7f38002110
CC 3686 session 1 completed step 18 in fiber 0x7f38002110
CC 3686 session 1 completed step 19 in fiber 0x7f38002110
CC 3686 session 1 completed step 20 in fiber 0x7f38002110
FiberScheduler: cc 3686, session 1, fiber 0x7f38002110 complete

FiberScheduler: fiber 0x7f38002110 exiting
Thread 9 getting next capture
disposing CC 3686 Session 1
Thread 12 is waiting
Thread 9 is waiting
Thread 8 is waiting
CC 3691 session 1 completed step 2 in fiber 0x7f38000b60
CC 3691 session 1 completed step 3 in fiber 0x7f38000b60
CC 3691 session 1 completed step 4 in fiber 0x7f38000b60
CC 3691 session 1 completed step 5 in fiber 0x7f38000b60
CC 3691 session 1 completed step 6 in fiber 0x7f38000b60
CC 3691 session 1 processing step 7 in fiber 0x7f38000b60
FiberScheduler: cc 3691, session 1, fiber 0x7f38000b60 in progress...

Any thoughts about the reason of the frame missing?
Thank you

3

Could you attached the lib here for verify.

Thanks

4

Also please attached your source code would be great.

5

Hi, please find source code and lib/binary files attached.

The pipeline uses RidgeRun interpipe version 1.1.7, the library attached.
Code of test pipeline in pipeline.c, compiled binary is camera-rtsp-test, I’m using camera with resolution 3864x2180, you may change the resolution in code and recompile. Compile the pipeline code using some libaries:

$(pkg-config --cflags --libs gstreamer-1.0 gstreamer-rtsp-1.0 glib-2.0 gstreamer-plugins-base-1.0 gstreamer-rtsp-server-1.0)

The source code and library of nvarguscamera attached also.

If you have 3864x2180 camera, you can run the app command below, else you must edit the resolution in pipeline.c to run:

./camera-rtsp-test -w 3864 -h 2180

The nvarguscamerasrc frame missing issue will happen when you try to connect to RTSP, URL:

rtsp://User1:TestUser1@<board_ip>:554/video1

Thank you.

libgstinterpipe.so (244.2 KB)
gst-nvarguscamera_r32.5.2.tgz (563.9 KB)
pipeline.c (15.3 KB)
camera-rtsp-test (69.7 KB)
libgstnvarguscamerasrc.so (656.7 KB)

6

Does the problem show for “gst-launch-1.0 nvarguscamerasrc …” in local device instead of RTSP?

7

I don’t see the problem when running simple pipeline likes

gst-launch-1.0 nvarguscamerasrc name=argussrc exposuretimerange="27000 33333000" gainrange="1 16" ispdigitalgainrange="1 256" tnr-mode=0 ee-mode=0 aeantibanding=0 silent=0 \
 ! video/x-raw\(memory:NVMM\),width=3864,height=2180,format=NV12,framerate=30/1 \
 ! nvvidconv left=12 right=3852 top=10 bottom=2170 ! video/x-raw\(memory:NVMM\),width=3840,height=2160,format=NV12 \
 ! queue leaky=2 max-size-buffers=3 \
 ! nvv4l2h264enc name=p0enc insert-sps-pps=true profile=4 iframeinterval=30 idrinterval=30 preset-level=2 insert-sps-pps=true bitrate=30000000 \
 ! rtph264pay name=pay0 config-interval=-1 \
 ! fakesink
8

Can try lower sensor output size or check with test-launch

9

From my testing, the issue looks like only happening on some special conditions.

It can run good with test-launch, even with max resolution or lower resolution.

But it happens more often when using interpipe together with RTSP.
I guess the nvarguscamerasrc receiving thread is affected by other elements while its state is still PLAYING.

11

Hi,
Please contact RidgeRun to get support about using interpipe. We don’t have much experience about using it. Would suggest contact the vendor to get further suggestion.

One suggestion is to try max performance mode by executing sudo nvpmodel -m 0 and sudo jetson_clocks. And can run VIC at max clock by referring to
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL

12

It is maybe the interpipe problem, I’m not sure.

But how it can affect nvarguscamerasrc to not receiving the frame while nvargus-daemon still having frame?

13

Hi,
One possible rootcause is the buffers are still in use in upper application layer and do not return to nvargus-daemon for capturing frames. In the mechanism buffers are allocated and used by application layer and nvargus-daemon. When frame data is captured to the buffer, it is sent to application layer. When application layer processes the buffer, it is returned to nvargus-daemon for next capture. If upper application layer does not return the buffer fast enough, the frames will be dropped.

1 Like
14

Hi Dane, thanks for your reply.

Can this be checked from nvargus-daemon log? or in nvarguscamerasrc debug log?
Or need to add debug code in nvarguscamerasrc plugin?
Thank you.

15

Hi,
The source code of nvarguscamerasrc is public. You can add debug prints for further debugging. When next element requests for buffer gst_nv_argus_camera_create() is called. You can add prints to check if it is not called in desired framerate. For 30fps it should be called 30 times in one second.

1 Like
16

Hi,

Added error print nvmm_buffers->length into the function gst_nv_argus_camera_create() and consumer_thread() when the buffer is pushed into nvmm_buffer again. Result below, can it proves your idea about the application is too slow processing?:
At 0:02:37.27 offset time, the consumer_thread adds the buffer back to queue, but there is no new buffer requested.

Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.069525019  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0         
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.100456681  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1             
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.103968788  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0         
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.130870424  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1             
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.134696386  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0         
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.165243411  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1             
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.169838389  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0         
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.198295192  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1             
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.201775162  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0         
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.228040127  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.230688478  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.266229494  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.270663378  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.300322688  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.328980012  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.374805523  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 2    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.403733995  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 3    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.430170316  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 4    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.474105189  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 5    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.523595672  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 6    
Nov 26 11:47:51 orbit-nano camera-streamer[4864]: 0:02:37.547204370  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 7    
Nov 26 11:47:54 orbit-nano camera-streamer[4864]:           s_bus_thread_handler() Feed watchdog                                                                                                             
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.832616685  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 7
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.836813900  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 7    
Nov 26 11:48:02 orbit-nano camera-streamer[4864]:             new_state_callback() RTSP state 1                                                                                                              
Nov 26 11:48:02 orbit-nano camera-streamer[4864]:       session_removed_callback() RTSP client session 0x7f0c029ae0 is removed                                                                      
Nov 26 11:48:02 orbit-nano camera-streamer[4864]:    rtsp_client_closed_callback() RTSP client 0x7f642f4450 rtsp://192.168.0.29:60088(null) closed                                                  
Nov 26 11:48:02 orbit-nano camera-streamer[4864]:    rtsp_client_closed_callback() client 0 rtsp://192.168.0.29:60088(null) removed                                                                 
Nov 26 11:48:02 orbit-nano camera-streamer[4864]:           s_rtsp_connection_cb() RTSP no of connection 0                                                                                          
Nov 26 11:48:02 orbit-nano camera-streamer[4864]:           s_rtsp_connection_cb() Timeout live 60, rtsp 301                                                                                        
Nov 26 11:48:02 orbit-nano camera-streamer[4864]:           s_rtsp_connection_cb() No RTSP connection, timeout started                                                                              
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.840836530  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 7
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.845792032  4864   0x7f2c01be00 ERROR                default gstnvarguscamerasrc.cpp:593:threadExecute: 593 LLL: NO CONTINUOUS frame 4687 -> 4995
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.849657057  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 6
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.855308918  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 6    
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.859300038  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 6
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.865413776  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 5
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.874748317  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 4
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.878765375  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 4    
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.882597223  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 4
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.885364845  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 3
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.887996217  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 2
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.891188321  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 1
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.896769295  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.899939525  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1    
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.902839023  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.931182231  4864   0x7f5c0031e0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1634:consumer_thread: nvmm_buffers added, length 1    
Nov 26 11:48:02 orbit-nano camera-streamer[4864]: 0:02:47.934050583  4864   0x5564b2d8f0 ERROR       nvarguscamerasrc gstnvarguscamerasrc.cpp:1651:gst_nv_argus_camera_create: nvmm_buffers length 0

Thank you

17

Hi Shane and Dane,

I think I found the cause. I’m using wrong setting for interpipesrc timestamp. After updated, the frame missing is gone.

Thank you for your support.

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.