Gstreamer works at 4K but freeze the system at 2K

pietro.assenza · July 7, 2020, 5:14pm

Hi All

looking for help about a strange issue: system works well at 4K, while gstreamer freezes the system at 2K.

We have a board Auvidea JN30B with a Jetson Nano (production) and a Framos IMX415 sensor;
we ported the original Framos driver to this platform, configured the sensor as CSI-2 serial / 4lane and we can successfully capture images 3840x2160 at 25-35 fps or more, both with gstreamer and V4L2;
when we try to configure the system to work with 1944x1096 (H/V 2/2-line binning mode) at 30fps, we can still capture with V4L2 but if we run gstreamer the system freezes and then reboots

This is how we configure the DT (we edited the working one for 4K and always have only one mode at the time):

        mode0 {
            mclk_khz = "37125";
            num_lanes = "4";
            tegra_sinterface = "serial_c";
            discontinuous_clk = "no";
            cil_settletime = "0";
            active_w = "1944";
            active_h = "1096";

            dynamic_pixel_bit_depth = "12";
            csi_pixel_bit_depth = "12";
            mode_type = "bayer";
            pixel_phase = "gbrg";

            readout_orientation = "0";
            line_length = "4400";
            inherent_gain = "1";
            pix_clk_hz = "297000000";

            min_gain_val = "1.0";
            max_gain_val = "72.0";
            min_hdr_ratio = "1";
            max_hdr_ratio = "64";
            min_framerate = "1.462526";
            max_framerate = "30";
            min_exp_time = "1";
            max_exp_time = "65535";
            embedded_metadata_height = "1";
        };

Here how we capture with V4L2:

    v4l2-ctl -d /dev/video1 --set-ctrl=data_rate=4
    v4l2-ctl -d /dev/video1 --verbose --set-fmt-video=width=1944,height=1096,pixelformat=GB12 --set-ctrl bypass_mode=0 --stream-mmap --stream-skip=0 --stream-count=1 --stream-to=frame.raw

This is the log produced:

        VIDIOC_QUERYCAP: ok
        VIDIOC_S_EXT_CTRLS: ok
        VIDIOC_G_FMT: ok
        VIDIOC_S_FMT: ok
        Format Video Capture:
                Width/Height      : 1952/1096
                Pixel Format      : 'GB12'
                Field             : None
                Bytes per Line    : 3904
                Size Image        : 4278784
                Colorspace        : sRGB
                Transfer Function : Default (maps to sRGB)
                YCbCr/HSV Encoding: Default (maps to ITU-R 601)
                Quantization      : Default (maps to Full Range)
                Flags             :
        VIDIOC_REQBUFS: ok
        VIDIOC_QUERYBUF: ok
        VIDIOC_QBUF: ok
        VIDIOC_QUERYBUF: ok
        VIDIOC_QBUF: ok
        VIDIOC_QUERYBUF: ok
        VIDIOC_QBUF: ok
        VIDIOC_QUERYBUF: ok
        VIDIOC_QBUF: ok
        VIDIOC_STREAMON: ok
                Index    : 0
                Type     : Video Capture
                Flags    : mapped
                Field    : None
                Sequence : 0
                Length   : 4278784
                Bytesused: 4278784
                Timestamp: 0.000000s (Monotonic, End-of-Frame)

        VIDIOC_STREAMOFF: ok

This the gstreamer command:

        v4l2-ctl -d /dev/video1 --set-ctrl=data_rate=4
        gst-launch-1.0 --gst-debug-level=2 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=1944, height=1096, format=(string)NV12, framerate=(fraction)30/1' ! nvoverlaysink overlay-y=100 overlay-x=200 overlay-w=1558 overlay-h=876 overlay=1 -e

Here its log:

        0:00:00.607358627  7506   0x5582fff780 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/maselettronica/.config:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
        Setting pipeline to PAUSED ...
        Pipeline is live and does not need PREROLL ...
        Setting pipeline to PLAYING ...
        New clock: GstSystemClock
        GST_ARGUS: Creating output stream
        CONSUMER: Waiting until producer is connected...
        GST_ARGUS: Available Sensor modes :
        GST_ARGUS: 1944 x 1096 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 72,000000; Exposure Range min 1000, max 65535000;

        GST_ARGUS: Running with following settings:
        Camera index = 0
        Camera mode  = 0
        Output Stream W = 1944 H = 1096
        seconds to Run    = 0
        Frame Rate = 29,999999
        GST_ARGUS: PowerService: requested_clock_Hz=13982220
        GST_ARGUS: Setup Complete, Starting captures for 0 seconds
        GST_ARGUS: Starting repeat capture requests.
        CONSUMER: Producer has connected; continuing.

Because of the freezing/reboot I cannot provide a trace and I have only a partial log from nvargus-daemon:

        ==== NVIDIA Libargus Camera Service (0.97.3)=== Listening for connections...
        === gst-launch-1.0[7506]: Connection established (7F7A5E31D0)Thread 1 getting next capture
        Thread 1 is waiting
        Thread 2 getting next capture
        Thread 2 is waiting
        Thread 3 getting next capture
        Thread 3 is waiting
        Thread 4 getting next capture
        Thread 4 is waiting
        Thread 5 getting next capture
        Thread 5 is waiting
        Thread 6 getting next capture
        Thread 6 is waiting
        Thread 7 getting next capture
        Thread 7 is waiting
        Thread 8 getting next capture
        Thread 8 is waiting
        Thread 9 getting next capture
        Thread 9 is waiting
        Thread 10 getting next capture
        Thread 10 is waiting
        Thread 11 getting next capture
        Thread 11 is waiting
        Thread 12 getting next capture
        Thread 12 is waiting
        Starting services...
        Worker thread IspHw statsComplete start
        Worker thread IspHw frameComplete start
        Worker thread IspHw statsComplete start
        Worker thread IspHw frameComplete start
        Worker thread CaptureScheduler checkFramePending start
        Worker thread CaptureScheduler frameStart start
        Worker thread V4L2CaptureScheduler checkCaptureComplete start
        Worker thread V4L2CaptureScheduler issueCaptures start
        Worker thread PS handleRequests start
        getInstance: s_instance(0x7f747d95b0)
        getInstance: s_instance(0x7f747d95b0)
        subscribe: SensorType(gyroscope)
        subscribe: create SensorType(gyroscope) sensor(0x7f747dbbb0)
        subscribe: SensorType(accelerometer)
        subscribe: create SensorType(accelerometer) sensor(0x7f747e1110)
        AC plugin not present: dlopen "acplugin.so", acplugin.so: cannot open shared object file: No such file or directory
        Services are started
        OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
        NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
        NvPclHwGetModuleList: No module data found
        OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
        ---- imager: No override file found. ----
        LSC: LSC surface is not based on full res!
        === gst-launch-1.0[7506]: CameraProvider initialized (0x7f7482a6a0)sourceRegistry[0] assigned

        ispRegistry[0] assigned

        Using Source GUID 0
        Worker thread ViCsiHw frameComplete start
        Worker thread ViCsiHw frameStart start
        Using ISP A
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=2, m_value=4294967295, m_timeout_ms=1000

        LSC: LSC surface is not based on full res!
        AC plugin not present: dlopen "acplugin.so", acplugin.so: cannot open shared object file: No such file or directory
        No library found, disabling AC plugin.
        Worker thread CaptureDispatcher start
        Session::updatePerfHints()
        updatePerfHints(): guid=0ll, CaptureIntent=1
        updatePerfHints(): NvCameraCoreUseCase_Preview
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

        No output buffers for 2
        InstructionList:
        + GraphSettings
            | SensorMode: 1944x1096 BayerS16GBRG 30.0fps
            | output 0: 1944x1096 BL U8_V8_ER 420SP
            | inputClipRect 0: (0.00,0.00, 1.00,1.00)

        + Instruction List
            | id: 0
            +  0: CCDataSetupStage
                | EstimatedIspOutLatencyFrames: 5
                | NumConcurrentCaptures: 1
                | UnprocessedYuvBufferMask: 0
            +  1: ACSynchronizeStage
            +  2: AeAfApplyStage
            +  3: AcPluginStage
                | operation: opApply
            +  4: AcMergeStage
                | IspIn: [1944, 1096]
                | IspOut0: [1944, 1096]
                | IspOut1: [0, 0]
                | IspOut2: [0, 0]
            +  5: TempBufferAcquireStage
                | Buffer Index: 2
                | BufferRequirements: 1944x1 Pitch NonColor8
            +  6: TempBufferAcquireStage
                | Buffer Index: 3
                | BufferRequirements: 524288x1 Pitch NonColor8
            +  7: SensorISPCaptureStage
                | Source GUID: 0
                | Output A Buffer: 0
                | SensorMetadata Buffer: 2
                | Output IspStats Buffer: 3
            +  8: StatsBufferAcquireStage
                | Buffer Index: 1
                | BufferRequirements: 640x362 Pitch Y8_ER 420
            +  9: BlitStage
                | Input Buffer: 0
                | Output Buffer: 1
                | Filter: Nearest
                | Transform: None
                | Src Rect: Not used
                | Dst Rect: Not used
            + 10: StatsUpdateStage
                | Output Meta Buffer: 2
                | Output IspStats Buffer: 3
            + 11: BufferReturnStage
                | Output A Buffer: 2
            + 12: BufferReturnStage
                | Output A Buffer: 3
            + 13: AcPluginStage
                | operation: opAnalyze
            + 14: AfAnalysisStage
            + 15: MonitorStage
            + 16: ExifStage
            + 17: MakerNoteStage
            + 18: BufferReturnStage
                | Output A Buffer: 0
            + 19: MetadataReturnStage
            + 20: PerfStatsStage

        Created fiber 0x7f24000b20 for CC 101 globalID 101 session 0
        Thread 1 is working on CC 101 session 0 globalID 101 step 0
        CC 101 session 0 completed step 0 in fiber 0x7f24000b20
        cc 101(1) session 0 runCount=0 runIspOut=0, latest ccId=0
        CC 101 session 0 completed step 1 in fiber 0x7f24000b20
        NV AE and AfApply algorithms are active.
        CC 101 session 0 completed step 2 in fiber 0x7f24000b20
        CC 101 session 0 completed step 3 in fiber 0x7f24000b20
        CC 101 session 0 completed step 4 in fiber 0x7f24000b20
        CC 101 session 0 completed step 5 in fiber 0x7f24000b20
        CC 101 session 0 completed step 6 in fiber 0x7f24000b20
        CC 101 session 0 processing step 7 in fiber 0x7f24000b20
        FiberScheduler: cc 101, session 0, fiber 0x7f24000b20 in progress...

        Thread 1 getting next capture
        Session::updatePerfHints()
        updatePerfHints(): guid=0ll, CaptureIntent=1
        updatePerfHints(): NvCameraCoreUseCase_Preview
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

        Thread 2 is waiting
        Session::updatePerfHints()
        updatePerfHints(): guid=0ll, CaptureIntent=1
        updatePerfHints(): NvCameraCoreUseCase_Preview
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

        Thread 1 is waiting
        Created fiber 0x7f24000dd0 for CC 102 globalID 102 session 0
        Thread 3 is working on CC 102 session 0 globalID 102 step 0
        CC 102 session 0 completed step 0 in fiber 0x7f24000dd0
        cc 102(2) session 0 runCount=1 runIspOut=0, latest ccId=0
        CC 102 session 0 completed step 1 in fiber 0x7f24000dd0
        NV AE and AfApply algorithms are active.
        CC 102 session 0 completed step 2 in fiber 0x7f24000dd0
        CC 102 session 0 completed step 3 in fiber 0x7f24000dd0
        CC 102 session 0 completed step 4 in fiber 0x7f24000dd0
        CC 102 session 0 completed step 5 in fiber 0x7f24000dd0
        CC 102 session 0 completed step 6 in fiber 0x7f24000dd0
        CC 102 session 0 processing step 7 in fiber 0x7f24000dd0
        FiberScheduler: cc 102, session 0, fiber 0x7f24000dd0 in progress...

        Thread 3 getting next capture
        Thread 4 is waiting
        Thread 3 is waiting
        PowerServiceHw:addRequest: table size: before: 0, after:1
                request table for ISP 0:
                req[0]: output1 width=0, output1 height=0, output1 Bpp=0
                req[0]: output2 width=0, output2 height=0, output2 Bpp=0
                req[0]: input width=1944, input height=1096, input Bpp=12
                req[0]: output width=130, output height=74, output Bpp=16
                req[0]: input fps=30
                req[0]: guID=0, stage type=SensorIspCapture
                req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
                req[450]: timeout=0
        PowerServiceHwIsp:setLaBw: m_bwVal_Iso=54151 and m_bwVal_NonIso=0 KBpS

        PowerServiceCore:setCameraBw: totalIsoBw=54151
        PowerServiceHw:addRequest: table size: before: 1, after:2
                request table for ISP 0:
                req[0]: output1 width=0, output1 height=0, output1 Bpp=0
                req[0]: output2 width=0, output2 height=0, output2 Bpp=0
                req[0]: input width=1944, input height=1096, input Bpp=12
                req[0]: output width=130, output height=74, output Bpp=16
                req[0]: input fps=30
                req[0]: guID=0, stage type=SensorIspCapture
                req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
                req[415]: timeout=0
                req[1]: output1 width=0, output1 height=0, output1 Bpp=0
                req[1]: output2 width=0, output2 height=0, output2 Bpp=0
                req[1]: input width=1944, input height=1096, input Bpp=12
                req[1]: output width=1944, output height=1096, output Bpp=12
                req[1]: input fps=30
                req[1]: guID=0, stage type=SensorIspCapture
                req[1]: clock=0 Hz, iso bw=595438 KB/s, non-iso bw=0 KB/s
                req[450]: timeout=0
        PowerServiceHw:updateRequests: table size: before: 2, after:2
                request table for ISP 0:
                req[0]: output1 width=0, output1 height=0, output1 Bpp=0
                req[0]: output2 width=0, output2 height=0, output2 Bpp=0
                req[0]: input width=1944, input height=1096, input Bpp=12
                req[0]: output width=130, output height=74, output Bpp=16
                req[0]: input fps=30
                req[0]: guID=0, stage type=SensorIspCapture
                req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
                req[414]: timeout=0
                req[1]: output1 width=0, output1 height=0, output1 Bpp=0
                req[1]: output2 width=0, output2 height=0, output2 Bpp=0
                req[1]: input width=1944, input height=1096, input Bpp=12
                req[1]: output width=1944, output height=1096, output Bpp=12
                req[1]: input fps=30
                req[1]: guID=0, stage type=SensorIspCapture
                req[1]: clock=0 Hz, iso bw=595438 KB/s, non-iso bw=0 KB/s
                req[449]: timeout=0
        PowerServiceHwIsp:setLaBw: m_bwVal_Iso=595438 and m_bwVal_NonIso=0 KBpS

        PowerServiceCore:setCameraBw: totalIsoBw=595438
        Created fiber 0x7f24000f70 for CC 103 globalID 103 session 0
        Thread 5 is working on CC 103 session 0 globalID 103 step 0
        Session::updatePerfHints()
        updatePerfHints(): guid=0ll, CaptureIntent=1
        updatePerfHints(): NvCameraCoreUseCase_Preview
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

        CC 103 session 0 completed step 0 in fiber 0x7f24000f70
        cc 103(3) session 0 runCount=2 runIspOut=0, latest ccId=0
        CC 103 session 0 completed step 1 in fiber 0x7f24000f70
        NV AE and AfApply algorithms are active.
        CC 103 session 0 completed step 2 in fiber 0x7f24000f70
        CC 103 session 0 completed step 3 in fiber 0x7f24000f70
        CC 103 session 0 completed step 4 in fiber 0x7f24000f70
        CC 103 session 0 completed step 5 in fiber 0x7f24000f70
        CC 103 session 0 completed step 6 in fiber 0x7f24000f70
        CC 103 session 0 processing step 7 in fiber 0x7f24000f70
        FiberScheduler: cc 103, session 0, fiber 0x7f24000f70 in progress...

        Thread 5 getting next capture
        Thread 5 is waiting
        Thread 6 is waiting

If I switch discontinuous_clk the system doesn’t freeze, still gstremer doesn’t capture and nvargus-daemon logs as follow:

        === NVIDIA Libargus Camera Service (0.97.3)=== Listening for connections...
        === gst-launch-1.0[8099]: Connection established (7F9B9461D0)Thread 1 getting next capture
        Thread 1 is waiting
        Thread 2 getting next capture
        Thread 2 is waiting
        Thread 3 getting next capture
        Thread 3 is waiting
        Thread 4 getting next capture
        Thread 4 is waiting
        Thread 5 getting next capture
        Thread 5 is waiting
        Thread 6 getting next capture
        Thread 6 is waiting
        Thread 7 getting next capture
        Thread 7 is waiting
        Thread 8 getting next capture
        Thread 8 is waiting
        Thread 9 getting next capture
        Thread 9 is waiting
        Thread 10 getting next capture
        Thread 10 is waiting
        Thread 11 getting next capture
        Thread 11 is waiting
        Thread 12 getting next capture
        Thread 12 is waiting
        Starting services...
        Worker thread IspHw statsComplete start
        Worker thread IspHw frameComplete start
        Worker thread IspHw statsComplete start
        Worker thread IspHw frameComplete start
        Worker thread CaptureScheduler checkFramePending start
        Worker thread CaptureScheduler frameStart start
        Worker thread V4L2CaptureScheduler checkCaptureComplete start
        Worker thread V4L2CaptureScheduler issueCaptures start
        Worker thread PS handleRequests start
        getInstance: s_instance(0x7f947d9720)
        getInstance: s_instance(0x7f947d9720)
        subscribe: SensorType(gyroscope)
        subscribe: create SensorType(gyroscope) sensor(0x7f9476e190)
        subscribe: SensorType(accelerometer)
        subscribe: create SensorType(accelerometer) sensor(0x7f947e11d0)
        AC plugin not present: dlopen "acplugin.so", acplugin.so: cannot open shared object file: No such file or directory
        Services are started
        OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
        NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
        NvPclHwGetModuleList: No module data found
        OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
        ---- imager: No override file found. ----
        LSC: LSC surface is not based on full res!
        === gst-launch-1.0[8099]: CameraProvider initialized (0x7f94829ba0)sourceRegistry[0] assigned

        ispRegistry[0] assigned

        Using Source GUID 0
        Worker thread ViCsiHw frameComplete start
        Worker thread ViCsiHw frameStart start
        Using ISP A
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=2, m_value=4294967295, m_timeout_ms=1000

        LSC: LSC surface is not based on full res!
        AC plugin not present: dlopen "acplugin.so", acplugin.so: cannot open shared object file: No such file or directory
        No library found, disabling AC plugin.
        Worker thread CaptureDispatcher start
        Session::updatePerfHints()
        updatePerfHints(): guid=0ll, CaptureIntent=1
        updatePerfHints(): NvCameraCoreUseCase_Preview
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

        No output buffers for 2
        InstructionList:
        + GraphSettings
            | SensorMode: 1944x1096 BayerS16GBRG 30.0fps
            | output 0: 1944x1096 BL U8_V8_ER 420SP
            | inputClipRect 0: (0.00,0.00, 1.00,1.00)

        + Instruction List
            | id: 0
            +  0: CCDataSetupStage
                | EstimatedIspOutLatencyFrames: 5
                | NumConcurrentCaptures: 1
                | UnprocessedYuvBufferMask: 0
            +  1: ACSynchronizeStage
            +  2: AeAfApplyStage
            +  3: AcPluginStage
                | operation: opApply
            +  4: AcMergeStage
                | IspIn: [1944, 1096]
                | IspOut0: [1944, 1096]
                | IspOut1: [0, 0]
                | IspOut2: [0, 0]
            +  5: TempBufferAcquireStage
                | Buffer Index: 2
                | BufferRequirements: 1944x1 Pitch NonColor8
            +  6: TempBufferAcquireStage
                | Buffer Index: 3
                | BufferRequirements: 524288x1 Pitch NonColor8
            +  7: SensorISPCaptureStage
                | Source GUID: 0
                | Output A Buffer: 0
                | SensorMetadata Buffer: 2
                | Output IspStats Buffer: 3
            +  8: StatsBufferAcquireStage
                | Buffer Index: 1
                | BufferRequirements: 640x362 Pitch Y8_ER 420
            +  9: BlitStage
                | Input Buffer: 0
                | Output Buffer: 1
                | Filter: Nearest
                | Transform: None
                | Src Rect: Not used
                | Dst Rect: Not used
            + 10: StatsUpdateStage
                | Output Meta Buffer: 2
                | Output IspStats Buffer: 3
            + 11: BufferReturnStage
                | Output A Buffer: 2
            + 12: BufferReturnStage
                | Output A Buffer: 3
            + 13: AcPluginStage
                | operation: opAnalyze
            + 14: AfAnalysisStage
            + 15: MonitorStage
            + 16: ExifStage
            + 17: MakerNoteStage
            + 18: BufferReturnStage
                | Output A Buffer: 0
            + 19: MetadataReturnStage
            + 20: PerfStatsStage

        Session::updatePerfHints()
        Thread 1 is working on CC 101 session 0 globalID 101 step 0
        CC 101 session 0 completed step 0 in fiber 0x7f44000b20
        cc 101(1) session 0 runCount=0 runIspOut=0, latest ccId=0
        CC 101 session 0 completed step 1 in fiber 0x7f44000b20
        NV AE and AfApply algorithms are active.
        CC 101 session 0 completed step 2 in fiber 0x7f44000b20
        CC 101 session 0 completed step 3 in fiber 0x7f44000b20
        Created fiber 0x7f44000b20 for CC 101 globalID 101 session 0
        updatePerfHints(): guid=0ll, CaptureIntent=1
        updatePerfHints(): NvCameraCoreUseCase_Preview
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

        Created fiber 0x7f44000dd0 for CC 102 globalID 102 session 0
        Thread 2 is working on CC 102 session 0 globalID 102 step 0
        Session::updatePerfHints()
        CC 101 session 0 completed step 4 in fiber 0x7f44000b20
        CC 102 session 0 completed step 0 in fiber 0x7f44000dd0
        CC 101 session 0 completed step 5 in fiber 0x7f44000b20updatePerfHints(): guid=0ll, CaptureIntent=1
        cc 102(2) session 0 runCount=1 runIspOut=0, latest ccId=0

        CC 101 session 0 completed step 6 in fiber 0x7f44000b20
        CC 102 session 0 completed step 1 in fiber 0x7f44000dd0
        NV AE and AfApply algorithms are active.
        updatePerfHints(): NvCameraCoreUseCase_Preview
        CC 101 session 0 processing step 7 in fiber 0x7f44000b20
        FiberScheduler: cc 101, session 0, fiber 0x7f44000b20 in progress...

        Thread 1 getting next capture
        Thread 3 is waiting
        Thread 1 is waiting
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

        CC 102 session 0 completed step 2 in fiber 0x7f44000dd0
        CC 102 session 0 completed step 3 in fiber 0x7f44000dd0
        CC 102 session 0 completed step 4 in fiber 0x7f44000dd0
        CC 102 session 0 completed step 5 in fiber 0x7f44000dd0
        CC 102 session 0 completed step 6 in fiber 0x7f44000dd0
        CC 102 session 0 processing step 7 in fiber 0x7f44000dd0
        FiberScheduler: cc 102, session 0, fiber 0x7f44000dd0 in progress...

        Thread 2 getting next capture
        Thread 4 is waiting
        Thread 2 is waiting
        PowerServiceHw:addRequest: table size: before: 0, after:1
                request table for ISP 0:
                req[0]: output1 width=0, output1 height=0, output1 Bpp=0
                req[0]: output2 width=0, output2 height=0, output2 Bpp=0
                req[0]: input width=1944, input height=1096, input Bpp=12
                req[0]: output width=130, output height=74, output Bpp=16
                req[0]: input fps=30
                req[0]: guID=0, stage type=SensorIspCapture
                req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
                req[450]: timeout=0
        PowerServiceHwIsp:setLaBw: m_bwVal_Iso=54151 and m_bwVal_NonIso=0 KBpS

        PowerServiceCore:setCameraBw: totalIsoBw=54151
        PowerServiceHw:addRequest: table size: before: 1, after:2
                request table for ISP 0:
                req[0]: output1 width=0, output1 height=0, output1 Bpp=0
                req[0]: output2 width=0, output2 height=0, output2 Bpp=0
                req[0]: input width=1944, input height=1096, input Bpp=12
                req[0]: output width=130, output height=74, output Bpp=16
                req[0]: input fps=30
                req[0]: guID=0, stage type=SensorIspCapture
                req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
                req[411]: timeout=0
                req[1]: output1 width=0, output1 height=0, output1 Bpp=0
                req[1]: output2 width=0, output2 height=0, output2 Bpp=0
                req[1]: input width=1944, input height=1096, input Bpp=12
                req[1]: output width=1944, output height=1096, output Bpp=12
                req[1]: input fps=30
                req[1]: guID=0, stage type=SensorIspCapture
                req[1]: clock=0 Hz, iso bw=595438 KB/s, non-iso bw=0 KB/s
                req[450]: timeout=0
        PowerServiceHw:updateRequests: table size: before: 2, after:2
                request table for ISP 0:
                req[0]: output1 width=0, output1 height=0, output1 Bpp=0
                req[0]: output2 width=0, output2 height=0, output2 Bpp=0
                req[0]: input width=1944, input height=1096, input Bpp=12
                req[0]: output width=130, output height=74, output Bpp=16
                req[0]: input fps=30
                req[0]: guID=0, stage type=SensorIspCapture
                req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
                req[409]: timeout=0
                req[1]: output1 width=0, output1 height=0, output1 Bpp=0
                req[1]: output2 width=0, output2 height=0, output2 Bpp=0
                req[1]: input width=1944, input height=1096, input Bpp=12
                req[1]: output width=1944, output height=1096, output Bpp=12
                req[1]: input fps=30
                req[1]: guID=0, stage type=SensorIspCapture
                req[1]: clock=0 Hz, iso bw=595438 KB/s, non-iso bw=0 KB/s
                req[448]: timeout=0
        PowerServiceHwIsp:setLaBw: m_bwVal_Iso=595438 and m_bwVal_NonIso=0 KBpS

        PowerServiceCore:setCameraBw: totalIsoBw=595438
        SCF: Error Timeout: ISP Stats timed out! (in src/services/capture/NvIspHw.cpp, function waitIspStatsFinished(), line 561)
        SCF: Error Timeout: ISP port 0 timed out! (in src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 478)
        SCF: Error Timeout:  (propagating from src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 519)
        SCF: Error Timeout:  (propagating from src/common/Utils.cpp, function workerThread(), line 116)
        SCF: Error Timeout: Worker thread IspHw frameComplete failed (in src/common/Utils.cpp, function workerThread(), line 133)
        Error: waitCsiFrameStart timeout guid 0
        ************VI/CSI Debug Registers**********
        VI_CFG_INTERRUPT_MASK_0 = 0x00000000
        VI_CFG_INTERRUPT_STATUS_0 = 0x00000000
        VI_CSI_0_ERROR_STATUS_0 = 0x00000000
        VI_CSI_0_ERROR_INT_MASK_0 = 0x00000000
        VI_CSI_1_ERROR_STATUS_0 = 0x00000000
        VI_CSI_1_ERROR_INT_MASK_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_A_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_A_STATUS_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_B_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_B_STATUS_0 = 0x00000000
        CSI_CSI_CIL_A_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_CIL_A_STATUS_0 = 0x00000000
        CSI_CSI_CILA_STATUS_0 = 0x00000000
        CSI_CSI_CIL_B_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_CIL_B_STATUS_0 = 0x00000000
        CSI_CSI_CILB_STATUS_0 = 0x00000000
        CSI_CSI_READONLY_STATUS_0 = 0x00000000
        CSI_DEBUG_CONTROL_0 = 0x2f752a00
        CSI_DEBUG_COUNTER_0_0 = 0x00000000
        CSI_DEBUG_COUNTER_1_0 = 0x00000000
        CSI_DEBUG_COUNTER_2_0 = 0x00000000
        *****************************************
        Error: waitCsiFrameStart Something went wrong with waiting on frame start
        Error: waitCsiFrameStart Something went wrong with waiting on frame start
        Created fiber 0x7f44000f70 for CC 103 globalID 103 session 0
        Thread 5 is working on CC 103 session 0 globalID 103 step 0
        PowerServiceCore:handleRequests: timePassed = 1507
        Session::updatePerfHints()
        updatePerfHints(): guid=0ll, CaptureIntent=1
        updatePerfHints(): NvCameraCoreUseCase_Preview
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

        CC 103 session 0 completed step 0 in fiber 0x7f44000f70
        cc 103(3) session 0 runCount=2 runIspOut=0, latest ccId=0
        CC 103 session 0 completed step 1 in fiber 0x7f44000f70
        NV AE and AfApply algorithms are active.
        CC 103 session 0 completed step 2 in fiber 0x7f44000f70
        CC 103 session 0 completed step 3 in fiber 0x7f44000f70
        CC 103 session 0 completed step 4 in fiber 0x7f44000f70
        CC 103 session 0 completed step 5 in fiber 0x7f44000f70
        CC 103 session 0 completed step 6 in fiber 0x7f44000f70
        CC 103 session 0 processing step 7 in fiber 0x7f44000f70
        FiberScheduler: cc 103, session 0, fiber 0x7f44000f70 in progress...

        Thread 5 getting next capture
        Thread 6 is waiting
        NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000
        Thread 5 is waiting



        SCF: Error Timeout: ISP Stats timed out! (in src/services/capture/NvIspHw.cpp, function waitIspStatsFinished(), line 561)
        Error: waitCsiFrameStart timeout guid 0
        ************VI/CSI Debug Registers**********
        VI_CFG_INTERRUPT_MASK_0 = 0x00000000
        VI_CFG_INTERRUPT_STATUS_0 = 0x00000000
        VI_CSI_0_ERROR_STATUS_0 = 0x00000000
        VI_CSI_0_ERROR_INT_MASK_0 = 0x00000000
        VI_CSI_1_ERROR_STATUS_0 = 0x00000000
        VI_CSI_1_ERROR_INT_MASK_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_A_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_A_STATUS_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_B_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_PIXEL_PARSER_B_STATUS_0 = 0x00000000
        CSI_CSI_CIL_A_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_CIL_A_STATUS_0 = 0x00000000
        CSI_CSI_CILA_STATUS_0 = 0x00000000
        CSI_CSI_CIL_B_INTERRUPT_MASK_0 = 0x00000000
        CSI_CSI_CIL_B_STATUS_0 = 0x00000000
        CSI_CSI_CILB_STATUS_0 = 0x00000000
        CSI_CSI_READONLY_STATUS_0 = 0x00000000
        CSI_DEBUG_CONTROL_0 = 0x2f752a00
        CSI_DEBUG_COUNTER_0_0 = 0x00000000
        CSI_DEBUG_COUNTER_1_0 = 0x00000000
        CSI_DEBUG_COUNTER_2_0 = 0x00000000
        *****************************************
        SCF: Error Timeout: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 990)
        Created fiber 0x7f44001110 for CC 104 globalID 104 session 0
        Thread 7 is working on CC 104 session 0 globalID 104 step 0
        CC 104 session 0 completed step 0 in fiber 0x7f44001110
        cc 104(4) session 0 runCount=3 runIspOut=0, latest ccId=0
        CC 104 session 0 completed step 1 in fiber 0x7f44001110
        NV AE and AfApply algorithms are active.
        CC 104 session 0 completed step 2 in fiber 0x7f44001110
        CC 104 session 0 completed step 3 in fiber 0x7f44001110
        CC 104 session 0 completed step 4 in fiber 0x7f44001110
        CC 104 session 0 completed step 5 in fiber 0x7f44001110
        CC 104 session 0 completed step 6 in fiber 0x7f44001110
        CC 104 session 0 processing step 7 in fiber 0x7f44001110
        FiberScheduler: cc 104, session 0, fiber 0x7f44001110 in progress...
        SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
        (Argus) Error InvalidState:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
        SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
        (Argus) Error InvalidState:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
        SCF: Error InvalidState: Session has suffered a critical failure (in src/api/Session.cpp, function capture(), line 667)
        .....

In this case this is the trace:

        # tracer: nop
        #
        # entries-in-buffer/entries-written: 110/110   #P:4
        #
        #                              _-----=> irqs-off
        #                             / _----=> need-resched
        #                            | / _---=> hardirq/softirq
        #                            || / _--=> preempt-depth
        #                            ||| /     delay
        #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
        #              | |       |   ||||       |         |
                v4l2-ctl-8246  [003] ....  1290.475237: tegra_channel_open: vi-output, imx415 0-001a
                v4l2-ctl-8246  [000] ....  1290.478702: camera_common_s_power: status : 0x1
                v4l2-ctl-8246  [002] ....  1290.481966: camera_common_s_power: status : 0x0
                v4l2-ctl-8246  [003] ....  1290.482191: camera_common_s_power: status : 0x1
                v4l2-ctl-8246  [001] ....  1290.661073: camera_common_s_power: status : 0x0
                v4l2-ctl-8246  [001] ....  1290.662408: tegra_channel_set_power: imx415 0-001a : 0x1
                v4l2-ctl-8246  [001] ....  1290.662434: camera_common_s_power: status : 0x1
                v4l2-ctl-8246  [001] ....  1290.694773: tegra_channel_set_power: nvcsi--1 : 0x1
                v4l2-ctl-8246  [001] ....  1290.694786: csi_s_power: enable : 0x1
                v4l2-ctl-8246  [001] ....  1290.698657: tegra_channel_close: vi-output, imx415 0-001a
                v4l2-ctl-8246  [001] ....  1290.698731: tegra_channel_set_power: imx415 0-001a : 0x0
                v4l2-ctl-8246  [001] ....  1290.698755: camera_common_s_power: status : 0x0
                v4l2-ctl-8246  [001] ....  1290.699844: tegra_channel_set_power: nvcsi--1 : 0x0
                v4l2-ctl-8246  [001] ....  1290.699852: csi_s_power: enable : 0x0
        nvargus-daemon-8259  [001] ....  1293.494515: tegra_channel_open: vi-output, nvtc358743 6-000f
        nvargus-daemon-8259  [001] ....  1293.495667: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [000] ....  1293.499882: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [000] ....  1293.499942: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [000] ....  1293.666600: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [000] ....  1293.666980: tegra_channel_set_power: nvtc358743 6-000f : 0x1
        nvargus-daemon-8259  [000] ....  1293.666989: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [000] ....  1293.667086: tegra_channel_set_power: nvcsi--2 : 0x1
        nvargus-daemon-8259  [000] ....  1293.667090: csi_s_power: enable : 0x1
        nvargus-daemon-8259  [000] ....  1293.667358: tegra_channel_close: vi-output, nvtc358743 6-000f
        nvargus-daemon-8259  [000] ....  1293.667381: tegra_channel_set_power: nvtc358743 6-000f : 0x0
        nvargus-daemon-8259  [000] ....  1293.667391: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [000] ....  1293.667515: tegra_channel_set_power: nvcsi--2 : 0x0
        nvargus-daemon-8259  [000] ....  1293.667519: csi_s_power: enable : 0x0
        nvargus-daemon-8259  [000] ....  1293.667846: tegra_channel_open: vi-output, imx415 0-001a
        nvargus-daemon-8259  [000] ....  1293.667920: tegra_channel_set_power: imx415 0-001a : 0x1
        nvargus-daemon-8259  [000] ....  1293.667930: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [002] ....  1293.699782: tegra_channel_set_power: nvcsi--1 : 0x1
        nvargus-daemon-8259  [002] ....  1293.699789: csi_s_power: enable : 0x1
        nvargus-daemon-8259  [002] ....  1293.700085: tegra_channel_close: vi-output, imx415 0-001a
        nvargus-daemon-8259  [002] ....  1293.700107: tegra_channel_set_power: imx415 0-001a : 0x0
        nvargus-daemon-8259  [002] ....  1293.700118: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [001] ....  1293.700811: tegra_channel_set_power: nvcsi--1 : 0x0
        nvargus-daemon-8259  [001] ....  1293.700817: csi_s_power: enable : 0x0
        nvargus-daemon-8259  [001] ....  1293.707813: tegra_channel_open: vi-output, nvtc358743 6-000f
        nvargus-daemon-8259  [001] ....  1293.707881: tegra_channel_set_power: nvtc358743 6-000f : 0x1
        nvargus-daemon-8259  [001] ....  1293.707892: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [003] ....  1293.708177: tegra_channel_set_power: nvcsi--2 : 0x1
        nvargus-daemon-8259  [003] ....  1293.708180: csi_s_power: enable : 0x1
        nvargus-daemon-8259  [003] ....  1293.708277: tegra_channel_close: vi-output, nvtc358743 6-000f
        nvargus-daemon-8259  [003] ....  1293.708288: tegra_channel_set_power: nvtc358743 6-000f : 0x0
        nvargus-daemon-8259  [003] ....  1293.708292: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [003] ....  1293.708358: tegra_channel_set_power: nvcsi--2 : 0x0
        nvargus-daemon-8259  [003] ....  1293.708359: csi_s_power: enable : 0x0
        nvargus-daemon-8259  [003] ....  1293.708525: tegra_channel_open: vi-output, imx415 0-001a
        nvargus-daemon-8259  [003] ....  1293.708569: tegra_channel_set_power: imx415 0-001a : 0x1
        nvargus-daemon-8259  [003] ....  1293.708575: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [001] ....  1293.740138: tegra_channel_set_power: nvcsi--1 : 0x1
        nvargus-daemon-8259  [001] ....  1293.740140: csi_s_power: enable : 0x1
        nvargus-daemon-8259  [001] ....  1293.740280: tegra_channel_close: vi-output, imx415 0-001a
        nvargus-daemon-8259  [001] ....  1293.740292: tegra_channel_set_power: imx415 0-001a : 0x0
        nvargus-daemon-8259  [001] ....  1293.740297: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [000] ....  1293.740663: tegra_channel_set_power: nvcsi--1 : 0x0
        nvargus-daemon-8259  [000] ....  1293.740667: csi_s_power: enable : 0x0
        nvargus-daemon-8259  [000] ....  1293.769959: tegra_channel_open: vi-output, imx415 0-001a
        nvargus-daemon-8259  [000] ....  1293.769994: tegra_channel_set_power: imx415 0-001a : 0x1
        nvargus-daemon-8259  [000] ....  1293.770008: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [000] ....  1293.801368: tegra_channel_set_power: nvcsi--1 : 0x1
        nvargus-daemon-8259  [000] ....  1293.801372: csi_s_power: enable : 0x1
        nvargus-daemon-8259  [000] ....  1293.801398: tegra_channel_close: vi-output, imx415 0-001a
        nvargus-daemon-8259  [000] ....  1293.801407: tegra_channel_set_power: imx415 0-001a : 0x0
        nvargus-daemon-8259  [000] ....  1293.801411: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [000] ....  1293.801705: tegra_channel_set_power: nvcsi--1 : 0x0
        nvargus-daemon-8259  [000] ....  1293.801706: csi_s_power: enable : 0x0
        nvargus-daemon-8259  [000] ....  1293.801803: tegra_channel_open: vi-output, imx415 0-001a
        nvargus-daemon-8259  [000] ....  1293.801828: tegra_channel_set_power: imx415 0-001a : 0x1
        nvargus-daemon-8259  [000] ....  1293.801832: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [002] ....  1293.833307: tegra_channel_set_power: nvcsi--1 : 0x1
        nvargus-daemon-8259  [002] ....  1293.833315: csi_s_power: enable : 0x1
        nvargus-daemon-8259  [002] ....  1293.833620: tegra_channel_close: vi-output, imx415 0-001a
        nvargus-daemon-8259  [002] ....  1293.833637: tegra_channel_set_power: imx415 0-001a : 0x0
        nvargus-daemon-8259  [002] ....  1293.833646: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [002] ....  1293.834165: tegra_channel_set_power: nvcsi--1 : 0x0
        nvargus-daemon-8259  [002] ....  1293.834168: csi_s_power: enable : 0x0
        nvargus-daemon-8259  [002] ....  1293.834351: tegra_channel_open: vi-output, imx415 0-001a
        nvargus-daemon-8259  [002] ....  1293.834407: tegra_channel_set_power: imx415 0-001a : 0x1
        nvargus-daemon-8259  [002] ....  1293.834415: camera_common_s_power: status : 0x1
        nvargus-daemon-8259  [001] ....  1293.866145: tegra_channel_set_power: nvcsi--1 : 0x1
        nvargus-daemon-8259  [001] ....  1293.866157: csi_s_power: enable : 0x1
        nvargus-daemon-8259  [001] ....  1293.866529: tegra_channel_close: vi-output, imx415 0-001a
        nvargus-daemon-8259  [001] ....  1293.866568: tegra_channel_set_power: imx415 0-001a : 0x0
        nvargus-daemon-8259  [001] ....  1293.866581: camera_common_s_power: status : 0x0
        nvargus-daemon-8259  [000] ....  1293.867153: tegra_channel_set_power: nvcsi--1 : 0x0
        nvargus-daemon-8259  [000] ....  1293.867159: csi_s_power: enable : 0x0
        CaptureSchedule-8278  [002] ....  1294.016282: tegra_channel_open: vi-output, imx415 0-001a
        CaptureSchedule-8278  [002] ....  1294.016319: tegra_channel_set_power: imx415 0-001a : 0x1
        CaptureSchedule-8278  [002] ....  1294.016325: camera_common_s_power: status : 0x1
        CaptureSchedule-8278  [002] ....  1294.047830: tegra_channel_set_power: nvcsi--1 : 0x1
        CaptureSchedule-8278  [002] ....  1294.047833: csi_s_power: enable : 0x1
        CaptureSchedule-8278  [002] ....  1294.051540: tegra_channel_set_stream: enable : 0x1
        CaptureSchedule-8278  [002] ....  1294.052711: tegra_channel_set_stream: nvcsi--1 : 0x1
        CaptureSchedule-8278  [002] ....  1294.052714: csi_s_stream: enable : 0x1
        CaptureSchedule-8278  [002] ....  1294.052715: tegra_channel_set_stream: imx415 0-001a : 0x1
        nvargus-daemon-8314  [002] ....  1299.408895: tegra_channel_close: vi-output, imx415 0-001a
        nvargus-daemon-8314  [002] ....  1299.408912: tegra_channel_set_stream: enable : 0x0
        nvargus-daemon-8314  [002] ....  1299.408913: tegra_channel_set_stream: imx415 0-001a : 0x0
        nvargus-daemon-8314  [003] ....  1299.421526: tegra_channel_set_stream: nvcsi--1 : 0x0
        nvargus-daemon-8314  [003] ....  1299.421542: csi_s_stream: enable : 0x0
        nvargus-daemon-8314  [000] ....  1299.425874: tegra_channel_set_power: imx415 0-001a : 0x0
        nvargus-daemon-8314  [000] ....  1299.425884: camera_common_s_power: status : 0x0
        nvargus-daemon-8314  [001] ....  1299.426342: tegra_channel_set_power: nvcsi--1 : 0x0
        nvargus-daemon-8314  [001] ....  1299.426346: csi_s_power: enable : 0x0
            kworker/2:0-8247  [002] ....  1304.445196: camera_common_s_power: status : 0x1
            kworker/2:0-8247  [002] ....  1304.448677: camera_common_s_power: status : 0x0
            kworker/2:0-8247  [002] ....  1304.448770: camera_common_s_power: status : 0x1
            kworker/2:0-8247  [002] ....  1304.622291: camera_common_s_power: status : 0x0

Any idea is welcome.
Thanks
Pietro

ShaneCCC · July 8, 2020, 3:07am

Have below command to confirm can capture continuously for vi mode.

v4l2-ctl --set-fmt-video=width=1944,height=1096,pixelformat=GB12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=300 -d /dev/video0

pietro.assenza · July 8, 2020, 11:13am

Hello ShaneCCC
this the output of the command:

v4l2-ctl --set-fmt-video=width=1944,height=1096,pixelformat=GB12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=300 -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.02 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.01 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<

ShaneCCC · July 8, 2020, 11:16am

Could you modify the sensor driver to have those gain/frame rate/exposure function as dummy doing not things to clarify those function cause the problem.

pietro.assenza · July 8, 2020, 1:55pm

To implement your suggestion, I made dummy call in functions linked to TEGRA_CAMERA_CID_GAIN and TEGRA_CAMERA_CID_COARSE_TIME; frame rate is already fixed by defining the data_rate

The behaviour of the system seems unchanged: when I run gstreamer the system freezes and after a while it eventually reboots

ShaneCCC · July 8, 2020, 2:12pm

That’s weird,
From the log there’s no any error from the CSI_CSI_CIL_xxx reg that tells CSI/VI didn’t receive any data from the bus.

pietro.assenza · July 8, 2020, 3:39pm

We just did this test:

configure the data_rate
run gstreamer
while the system is frozen, check by oscilloscope data signal on sensor CSI

We see the signal pattern that indicates the sensor is sending frames, and we can also confirm that the frame-rate is 30fps as expected.

I could switch to the 4K configuration to provide some log of the working case. If you think they can be useful, please let me know which log to produce.

ShaneCCC · July 9, 2020, 1:44am

I don’t think there more information from the log of working case.
Maybe be you can try the cil_settletime from 2 - 15 one by one.

pietro.assenza · July 10, 2020, 5:49pm

Sorry ShaneCCC I still have to try the changes you asked, but in the mean time I did a test on a custom board where we use the same Nano and run the same driver: also on this board the sytem freezes when we run gstremer at 2K (but 4K works fine) and I got some dmesg that could be useful.

When the system freezes, this log appear:

[  177.783356] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  177.790987] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  177.797779] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  177.805236] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  177.811349] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080
[  177.817635] tegra-vii2c 546c0000.i2c: I2C_INT_MASK - 0xac
[  177.823505] tegra-vii2c 546c0000.i2c: I2C_INT_STATUS - 0xc2
[  177.829444] tegra-vii2c 546c0000.i2c: msg->len - 1
[  177.834517] tegra-vii2c 546c0000.i2c: is_msg_write - 1
[  177.839914] tegra-vii2c 546c0000.i2c: buf_remaining - 0
[  177.845417] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x08
[  178.871038] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  178.878204] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  178.883845] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  178.891191] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  178.897218] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080
[  178.903508] tegra-vii2c 546c0000.i2c: I2C_INT_MASK - 0xac
[  178.909175] tegra-vii2c 546c0000.i2c: I2C_INT_STATUS - 0xc2
[  178.915032] tegra-vii2c 546c0000.i2c: msg->len - 1
[  178.920089] tegra-vii2c 546c0000.i2c: is_msg_write - 1
[  178.925490] tegra-vii2c 546c0000.i2c: buf_remaining - 0
[  178.930999] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x00
[  178.939487] nvtc358743 14-000f: i2c_rd: reading register 0x14 from 0xf failed
[  179.959529] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  179.967082] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  179.972632] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  179.981165] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  179.988357] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080
[  179.995885] tegra-vii2c 546c0000.i2c: I2C_INT_MASK - 0xac
[  180.002650] tegra-vii2c 546c0000.i2c: I2C_INT_STATUS - 0xc2
[  180.010064] tegra-vii2c 546c0000.i2c: msg->len - 1
[  180.016542] tegra-vii2c 546c0000.i2c: is_msg_write - 1
[  180.023532] tegra-vii2c 546c0000.i2c: buf_remaining - 0
[  180.030416] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x08
[  181.047109] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  181.054308] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  181.062362] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  181.071360] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  181.079064] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080
[  181.087083] tegra-vii2c 546c0000.i2c: I2C_INT_MASK - 0xac
[  181.094218] tegra-vii2c 546c0000.i2c: I2C_INT_STATUS - 0xc2
[  181.101501] tegra-vii2c 546c0000.i2c: msg->len - 1
[  181.108075] tegra-vii2c 546c0000.i2c: is_msg_write - 1
[  181.115046] tegra-vii2c 546c0000.i2c: buf_remaining - 0
[  181.121967] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x00
[  181.132094] nvtc358743 14-000f: i2c_rd: reading register 0x14 from 0xf failed
[  183.190970] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  183.198162] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  183.203933] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  183.211465] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  183.218959] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080
[  183.225231] tegra-vii2c 546c0000.i2c: I2C_INT_MASK - 0xac
[  183.232479] tegra-vii2c 546c0000.i2c: I2C_INT_STATUS - 0xc2
[  183.240060] tegra-vii2c 546c0000.i2c: msg->len - 1
[  183.246975] tegra-vii2c 546c0000.i2c: is_msg_write - 1
[  183.253957] tegra-vii2c 546c0000.i2c: buf_remaining - 0
[  183.260945] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x08
[  184.279354] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  184.286923] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  184.292470] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  184.301519] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  184.308116] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080
[  184.314487] tegra-vii2c 546c0000.i2c: I2C_INT_MASK - 0xac
[  184.320169] tegra-vii2c 546c0000.i2c: I2C_INT_STATUS - 0xc2
[  184.326006] tegra-vii2c 546c0000.i2c: msg->len - 1
[  184.331063] tegra-vii2c 546c0000.i2c: is_msg_write - 1
[  184.336462] tegra-vii2c 546c0000.i2c: buf_remaining - 0
[  184.341953] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x00
[  184.350437] nvtc358743 14-000f: i2c_rd: reading register 0x14 from 0xf failed
[  185.367549] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  185.374994] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  185.380689] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  185.388050] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  185.394084] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080
[  185.400353] tegra-vii2c 546c0000.i2c: I2C_INT_MASK - 0xac
[  185.406019] tegra-vii2c 546c0000.i2c: I2C_INT_STATUS - 0xc2
[  185.411846] tegra-vii2c 546c0000.i2c: msg->len - 1
[  185.416906] tegra-vii2c 546c0000.i2c: is_msg_write - 1
[  185.422435] tegra-vii2c 546c0000.i2c: buf_remaining - 0
[  185.428045] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x08
[  185.979553] host1x 50000000.host1x: cdma_handle_timeout: timeout: 22 (54680000.isp_0) client 5, HW thresh 0, done 1
[  185.990371] host1x 50000000.host1x: cdma_handle_timeout: timeout: 23 (54680000.isp_1) client 5, HW thresh 0, done 1
[  186.001150] host1x 50000000.host1x: cdma_handle_timeout: timeout: 25 (54680000.isp_3) client 5, HW thresh 1, done 1
[  186.011845] host1x 50000000.host1x: cdma_handle_timeout: timeout: 24 (54680000.isp_2) client 5, HW thresh 7, done 7
[  186.455498] tegra-vii2c 546c0000.i2c: --- register dump for debugging ----
[  186.462683] tegra-vii2c 546c0000.i2c: I2C_CNFG - 0x22c00
[  186.468671] tegra-vii2c 546c0000.i2c: I2C_PACKET_TRANSFER_STATUS - 0x1010001
[  186.476122] tegra-vii2c 546c0000.i2c: I2C_FIFO_CONTROL - 0xe0
[  186.482159] tegra-vii2c 546c0000.i2c: I2C_FIFO_STATUS - 0x800080

then after a while:

[  192.327124] ------------[ cut here ]------------
[  192.331938] WARNING: CPU: 3 PID: 0 at /home/user/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_JETSON_NANO/Linux_for_Tegra/sources/kernel/kernel-4.9/kernel/watchdog_hld.c:143 watchdog_check_hardlockup_other_cpu+0xf8/0x120
[  192.352794] ---[ end trace 1b5b50cc0e94b389 ]---

ShaneCCC · July 11, 2020, 7:49am

Looks like the i2c transfer failed. Does 4k sensor i2c REG setting don’t have this problem?
If 4K setting didn’t have problem it could be the REG setting lose something may be sw reset or some others.

Also curious v4l2-ctl capture have the same i2c error?

sme21ab · July 11, 2020, 12:27pm

Hello,
I’m the Hardware designer of the board.
Obviosly in the 4K acquisition we do not have any errors on the I2C.
In the v412-ctl 2k acquisition we do not have this problems.
The failure of the i2c is on a different bus than the one used for the Sensor. The failure address is 0x0070 (that is internal to the Jetson Nano Module), the sensor address is 0x1a.
What worries me is the dma Error :

[  185.979553] host1x 50000000.host1x: cdma_handle_timeout: timeout: 22 (54680000.isp_0) client 5, HW thresh 0, done 1
[  185.990371] host1x 50000000.host1x: cdma_handle_timeout: timeout: 23 (54680000.isp_1) client 5, HW thresh 0, done 1
[  186.001150] host1x 50000000.host1x: cdma_handle_timeout: timeout: 25 (54680000.isp_3) client 5, HW thresh 1, done 1
[  186.011845] host1x 50000000.host1x: cdma_handle_timeout: timeout: 24 (54680000.isp_2) client 5, HW thresh 7, done 7

Any Ideas?
Sandro

ShaneCCC · July 11, 2020, 4:02pm

Looks like the failed should be the nvtc358743 slave address is could be 0x0f and connect to bus 14.
It’s could be a HMDI-IN chip driver been enabled cause this error message.

[ 184.350437] nvtc358743 14-000f: i2c_rd: reading register 0x14 from 0xf failed

d.kristofic · July 13, 2020, 2:20pm

Hi,

I’ve tested FSM-IMX415 with provided configuration (CSI-2 4lanes, mode0) on our prototype carrier board with Jetson Nano production and devKit module.
We can get the stream with v4l2 and nvarguscamerasrc for both modes, 4K and 2K.

L4T used: 32.4.3

Could you dump the current device tree(not only mode of image sensor) of your configuration?

From your log of the custom board, it seems to me that you have some kind of mux where i2c transaction failed.

[ 184.341953] tegra-vii2c 546c0000.i2c: i2c transfer timed out, addr 0x0070, data 0x00
[ 184.350437] nvtc358743 14-000f: i2c_rd: reading register 0x14 from 0xf failed

HW related:
Do you use FSA module in the stack or you have your own board for interface conversion and power supply? Which version of FSA do you use (if applicable)?
What do you use for interface conversion? Could you provide schematics?

sme21ab · July 13, 2020, 4:26pm

Hello d.kristofic,
We are using the FSM-IMX415 with our FSA I could Share schematics,if you need, my mail is
sm@maselettronica.com Plese write here so I can answer to you.
In any case the board works well in the 4K configuration so i’m sure there are no hardware Issues.
For the other questions my colleague will answer to you. Thanks.
Sandro

pietro.assenza · July 13, 2020, 4:48pm

@ShaneCCC: I went back to the Auvidea board and managed to disable in the dts the device nvtc358743; then I repeated the test and got the same result: 4K is ok, 2K fails.
This time logging the dmesg on the serial I get this:

[   80.43538[   90.720693] host1x 50000000.host1x: cdma_handle_timeout: timeout: 30 (54080000.vi_0) client 10, HW thresh 0, done 1
[   90.731366] host1x 50000000.host1x: cdma_handle_timeout: timeout: 35 (54080000.vi_4) client 10, HW thresh 1, done 1
[   90.748635] host1x 50000000.host1x: cdma_handle_timeout: timeout: 32 (54080000.vi_2) client 10, HW thresh 2, done 2
[   96.560488] ------------[ cut here ]------------
[   96.565218] WARNING: CPU: 3 PID: 1895 at /home/user/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_JETSON_NANO/Linux_for_Tegra/sources/kernel/kernel-4.9/kernel/watchdog_hld.c:143 watchdog_check_hardlockup_other_cpu+0xf8/0x120
[   96.585262] ---[ end trace 639d909c9e517c7f ]---
[  101.636366] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  101.642129] 	0-...: (1 GPs behind) idle=f17/140000000000001/0 softirq=13781/13782 fqs=2531
[  101.650495] 	(detected by 2, t=5255 jiffies, g=1515, c=1514, q=16780)
[  105.548513] INFO: rcu_sched detected stalls on CPUs/tasks:
[  105.554100] 	0-...: (1 GPs behind) idle=f17/140000000000001/0 softirq=13660/13782 fqs=2549
[  105.562461] 	(detected by 3, t=5252 jiffies, g=608, c=607, q=20)
Dump for CPU0:
pid: 4291  comm: sugov:0
  x0 0000000000052204   x1 0000000000000046
  x2 0000000000000000   x3 0000000000000000
  x4 ffffffc0fa09d050   x5 ffffffc0fa09cff8
  x6 0000000000000000   x7 ffffffc0fa09d020
  x8 ffffffc0fa09cff8   x9 ffffff8009e94440
 x10 0000000000000040  x11 0000000002000111
 x12 0000000000000000  x13 0000000002000000
 x14 0000000080a10330  x15 0000000080000330
 x16 0000000000000000  x17 0000000000000001
 x18 0000000000000085  x19 ffffffc0f9496c20
 x20 ffffffc0f2590018  x21 ffffffc0f9496c9c
 x22 ffffff8009077000  x23 ffffff8009e76e48
 x24 ffffff8009835018  x25 ffffffc0fefcef3c
 x26 ffffffc0f9496c00  x27 ffffff800a0a5000
 x28 ffffffc0f92dc600  x29 ffffffc0fefcedb0
 x30 ffffff80081236dc   sp ffffffc0fefcedb0
  pc ffffff8008123588 cpsr 800001c5 (EL1h)

__synchronize_hardirq+0x28/0x70:
  pc ffffff8008123588   sp ffffffc0fefcedb0   fp ffffffc0fefcedb0
synchronize_irq+0x2c/0x90:
  pc ffffff80081236dc   sp ffffffc0fefcedc0   fp ffffffc0fefcede0
disable_irq+0x2c/0x38:
  pc ffffff8008123c84   sp ffffffc0fefcedf0   fp ffffffc0fefcee30
vi_disable_irq+0x34/0xa8:
  pc ffffff8008b52e14   sp ffffffc0fefcee40   fp ffffffc0fefcee50
vi_isr+0x94/0x188:
  pc ffffff8008b53104   sp ffffffc0fefcee60   fp ffffffc0fefcee80
__handle_irq_event_percpu+0x68/0x288:
  pc ffffff8008123260   sp ffffffc0fefcee90   fp ffffffc0fefceeb0
handle_irq_event_percpu+0x28/0x60:
  pc ffffff80081234a8   sp ffffffc0fefceec0   fp ffffffc0fefcef10
handle_irq_event+0x50/0x80:
  pc ffffff8008123530   sp ffffffc0fefcef20   fp ffffffc0fefcef40
handle_fasteoi_irq+0xc8/0x1b8:
  pc ffffff80081272f8   sp ffffffc0fefcef50   fp ffffffc0fefcef70
generic_handle_irq+0x34/0x50:
  pc ffffff800812224c   sp ffffffc0fefcef80   fp ffffffc0fefcefa0
__handle_domain_irq+0x68/0xc0:
  pc ffffff8008122930   sp ffffffc0fefcefb0   fp ffffffc0fefcefc0
gic_handle_irq+0x5c/0xb0:
  pc ffffff8008080d44   sp ffffffc0fefcefd0   fp ffffffc0fefcf000
el1_irq+0xe8/0x18c:
  pc ffffff8008082be8   sp ffffffc0fefcf010   fp ffffffc0fefcf040
_raw_spin_unlock_irqrestore+0x30/0x60:
  pc ffffff8008f5b828   sp ffffffc0ee91f890   fp ffffffc0ee91f9c0
tegra210_emc_set_rate+0x1b4/0x560:
  pc ffffff8008cbb0bc   sp ffffffc0ee91f9d0   fp ffffffc0ee91f9e0
clk_emc_set_rate+0x78/0x158:
  pc ffffff800868fc68   sp ffffffc0ee91f9f0   fp ffffffc0ee91fa40
clk_change_rate+0xd0/0x5d0:
  pc ffffff800867b720   sp ffffffc0ee91fa50   fp ffffffc0ee91fa80
clk_core_set_rate_nolock+0x60/0xd8:
  pc ffffff800867bc80   sp ffffffc0ee91fa90   fp ffffffc0ee91fad0
clk_set_rate+0x58/0xa0:
  pc ffffff800867bd88   sp ffffffc0ee91fae0   fp ffffffc0ee91fb00
_simple_shared_update+0x38/0x48:
  pc ffffff8008691b78   sp ffffffc0ee91fb10   fp ffffffc0ee91fb20
clk_shared_bus_update+0x40/0x60:
  pc ffffff800868ff68   sp ffffffc0ee91fb30   fp ffffffc0ee91fb40
clk_shared_set_rate+0x40/0x98:
  pc ffffff8008690f20   sp ffffffc0ee91fb50   fp ffffffc0ee91fb60
clk_change_rate+0xd0/0x5d0:
  pc ffffff800867b720   sp ffffffc0ee91fb70   fp ffffffc0ee91fb80
clk_core_set_rate_nolock+0x60/0xd8:
  pc ffffff800867bc80   sp ffffffc0ee91fb90   fp ffffffc0ee91fbd0
clk_set_rate+0x58/0xa0:
  pc ffffff800867bd88   sp ffffffc0ee91fbe0   fp ffffffc0ee91fc00
bwmgr_update_clk+0x16c/0x200:
  pc ffffff8008c7271c   sp ffffffc0ee91fc10   fp ffffffc0ee91fc20
tegra_bwmgr_set_emc+0xf4/0x340:
  pc ffffff8008c73634   sp ffffffc0ee91fc30   fp ffffffc0ee91fc70
set_cpu_to_emc_freq+0x9c/0x180:
  pc ffffff8008ca21e4   sp ffffffc0ee91fc80   fp ffffffc0ee91fcd0
set_cpu_freq+0x17c/0x1a8:
  pc ffffff8008b9b714   sp ffffffc0ee91fce0   fp ffffffc0ee91fcf0
__cpufreq_driver_target+0x1d0/0x610:
  pc ffffff8008b8d848   sp ffffffc0ee91fd00   fp ffffffc0ee91fd50
sugov_work+0x34/0x50:
  pc ffffff8008114904   sp ffffffc0ee91fd60   fp ffffffc0ee91fdb0
kthread_worker_fn+0x98/0x160:
  pc ffffff80080dbfc8   sp ffffffc0ee91fdc0   fp ffffffc0ee91fdd0
kthread+0xec/0xf0:
  pc ffffff80080dbee4   sp ffffffc0ee91fde0   fp ffffffc0ee91fe20
ret_from_fork+0x10/0x40:
  pc ffffff8008083850   sp ffffffc0ee91fe30   fp 0000000000000000
debug>   x0 0000000000052204   x1 0000000000000046
  x2 0000000000000000   x3 0000000000000000
  x4 ffffffc0fa09d050   x5 ffffffc0fa09cff8
  x6 0000000000000000   x7 ffffffc0fa09d020
  x8 ffffffc0fa09cff8   x9 ffffff8009e94440
 x10 0000000000000040  x11 0000000002000111
 x12 0000000000000000  x13 0000000002000000
 x14 0000000080a10330  x15 0000000080000330
 x16 0000000000000000  x17 0000000000000001
 x18 0000000000000085  x19 ffffffc0f9496c20
 x20 ffffffc0f2590018  x21 ffffffc0f9496c9c
 x22 ffffff8009077000  x23 ffffff8009e76e48
 x24 ffffff8009835018  x25 ffffffc0fefcef3c
 x26 ffffffc0f9496c00  x27 ffffff800a0a5000
 x28 ffffffc0f92dc600  x29 ffffffc0fefcedb0
 x30 ffffff80081236dc   sp ffffffc0fefcedb0
  pc ffffff8008123588 cpsr 800001c5 (EL1h)
 sp_el0   ffffffc0f92dc600
 elr_el1  ffffff8008f5b828
 spsr_el1 60000045
debug> Dump for CPU2:
pid: 1895  comm: systemd-journal
  x0 ffffffc0f238d4a8   x1 ffffffc0f27fbe58
  x2 ffffffc0ec09a000   x3 000000000270c542
  x4 ffffffc0ff00e1c0   x5 ffffffc0f9b72a00
  x6 ffffffc0ec09c000   x7 4362ef2706d0488b
  x8 9cf9d26eff5a34ad   x9 e5c1e4ecbb002fd7
 x10 d0d0d0d0d0d0d0bd  x11 00000000821eb000
 x12 0000000000000018  x13 000000005f0c809d
 x14 0006c701d8a92426  x15 000047921aa82620
 x16 0000000000000000  x17 0000000000000000
 x18 0000007fc04115c6  x19 ffffffc0f27fbe10
 x20 ffffffc0f27fbe58  x21 ffffffc0f27fbe10
 x22 0000007f7b4e5c10  x23 ffffffc0f27fbe58
 x24 0000000000000015  x25 0000000000000123
 x26 000000000000004f  x27 ffffff8008f72000
 x28 ffffffc0f9b72a00  x29 ffffffc0f27fbd80
 x30 ffffff800826392c   sp ffffffc0f27fbd80
  pc ffffff80083f6288 cpsr 60000145 (EL1h)

security_inode_getattr+0x20/0x88:
  pc ffffff80083f6288   sp ffffffc0f27fbd80   fp ffffffc0f27fbd80
vfs_getattr+0x24/0x40:
  pc ffffff800826392c   sp ffffffc0f27fbd90   fp ffffffc0f27fbdb0
vfs_fstatat+0x7c/0xc0:
  pc ffffff8008263a34   sp ffffffc0f27fbdc0   fp ffffffc0f27fbdd0
SyS_newfstatat+0x3c/0x60:
  pc ffffff8008263e3c   sp ffffffc0f27fbde0   fp ffffffc0f27fbe20
__sys_trace_return+0x0/0x4:
  pc ffffff800808391c   sp ffffffc0f27fbe30   fp 0000000000000000
debug>   x0 ffffffc0f238d4a8   x1 ffffffc0f27fbe58
  x2 ffffffc0ec09a000   x3 000000000270c542
  x4 ffffffc0ff00e1c0   x5 ffffffc0f9b72a00
  x6 ffffffc0ec09c000   x7 4362ef2706d0488b
  x8 9cf9d26eff5a34ad   x9 e5c1e4ecbb002fd7
 x10 d0d0d0d0d0d0d0bd  x11 00000000821eb000
 x12 0000000000000018  x13 000000005f0c809d
 x14 0006c701d8a92426  x15 000047921aa82620
 x16 0000000000000000  x17 0000000000000000
 x18 0000007fc04115c6  x19 ffffffc0f27fbe10
 x20 ffffffc0f27fbe58  x21 ffffffc0f27fbe10
 x22 0000007f7b4e5c10  x23 ffffffc0f27fbe58
 x24 0000000000000015  x25 0000000000000123
 x26 000000000000004f  x27 ffffff8008f72000
 x28 ffffffc0f9b72a00  x29 ffffffc0f27fbd80
 x30 ffffff800826392c   sp ffffffc0f27fbd80
  pc ffffff80083f6288 cpsr 60000145 (EL1h)
 sp_el0   ffffffc0f9b72a00
 elr_el1  ffffff8008cac008
 spsr_el1 20000145
debug> [  144.527982] ------------[ cut here ]------------
[  144.532693] WARNING: CPU: 2 PID: 1895 at /home/user/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_JETSON_NANO/Linux_for_Tegra/sources/kernel/kernel-4.9/kernel/watchdog_hld.c:143 watchdog_check_hardlockup_other_cpu+0xf8/0x120
[  144.552726] ---[ end trace 639d909c9e517c80 ]---
[  144.557705] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [systemd-journal:1895]
[  172.527682] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [systemd-journal:1895]

Please note that I repeated the test multiple times, the higher part of the stack dump is always the same:

__synchronize_hardirq+0x28/0x70:
 pc ffffff8008123588   sp ffffffc0fefcedb0   fp ffffffc0fefcedb0
synchronize_irq+0x2c/0x90:
 pc ffffff80081236dc   sp ffffffc0fefcedc0   fp ffffffc0fefcede0
disable_irq+0x2c/0x38:
 pc ffffff8008123c84   sp ffffffc0fefcedf0   fp ffffffc0fefcee30
vi_disable_irq+0x34/0xa8:
 pc ffffff8008b52e14   sp ffffffc0fefcee40   fp ffffffc0fefcee50
vi_isr+0x94/0x188:
 pc ffffff8008b53104   sp ffffffc0fefcee60   fp ffffffc0fefcee80
__handle_irq_event_percpu+0x68/0x288:
 pc ffffff8008123260   sp ffffffc0fefcee90   fp ffffffc0fefceeb0

@d.kristofic: Hello and thanks for helping.
Please find attached the dts dump for the Auvidea board, in the version with nvtc358743 disabled:
runtime-auv2k-nohdmi-dts.txt (261.9 KB)
We have kernel 4.9, Jetpack 4.2.2 rev1 and L4T-7.3.1

ShaneCCC · July 14, 2020, 2:47am

Have a check the log from daemon.

pietro.assenza · July 14, 2020, 7:02am

this is the daemon log until the system freezes:

=== gst-launch-1.0[6496]: Connection established (7F8A26F1D0)Thread 1 getting next capture
Thread 1 is waiting
Thread 2 getting next capture
Thread 2 is waiting
Thread 3 getting next capture
Thread 3 is waiting
Thread 4 getting next capture
Thread 4 is waiting
Thread 5 getting next capture
Thread 5 is waiting
Thread 6 getting next capture
Thread 6 is waiting
Thread 7 getting next capture
Thread 7 is waiting
Thread 8 getting next capture
Thread 8 is waiting
Thread 9 getting next capture
Thread 9 is waiting
Thread 10 getting next capture
Thread 10 is waiting
Thread 11 getting next capture
Thread 11 is waiting
Thread 12 getting next capture
Thread 12 is waiting
Starting services...
Worker thread IspHw statsComplete start
Worker thread IspHw frameComplete start
Worker thread IspHw statsComplete start
Worker thread IspHw frameComplete start
Worker thread CaptureScheduler checkFramePending start
Worker thread CaptureScheduler frameStart start
Worker thread V4L2CaptureScheduler checkCaptureComplete start
Worker thread V4L2CaptureScheduler issueCaptures start
Worker thread PS handleRequests start
getInstance: s_instance(0x7f847d9600)
getInstance: s_instance(0x7f847d9600)
subscribe: SensorType(gyroscope)
subscribe: create SensorType(gyroscope) sensor(0x7f847b71c0)
subscribe: SensorType(accelerometer)
subscribe: create SensorType(accelerometer) sensor(0x7f847e10b0)
AC plugin not present: dlopen "acplugin.so", acplugin.so: cannot open shared object file: No such file or directory
Services are started
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
LSC: LSC surface is not based on full res!
=== gst-launch-1.0[6496]: CameraProvider initialized (0x7f8482d1a0)sourceRegistry[0] assigned

ispRegistry[0] assigned

Using Source GUID 0
Worker thread ViCsiHw frameComplete start
Worker thread ViCsiHw frameStart start
Using ISP A
NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=2, m_value=4294967295, m_timeout_ms=1000

LSC: LSC surface is not based on full res!
AC plugin not present: dlopen "acplugin.so", acplugin.so: cannot open shared object file: No such file or directory
No library found, disabling AC plugin.
Worker thread CaptureDispatcher start
Session::updatePerfHints()
updatePerfHints(): guid=0ll, CaptureIntent=1
updatePerfHints(): NvCameraCoreUseCase_Preview
NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

No output buffers for 2
InstructionList:
  + GraphSettings
      | SensorMode: 1944x1096 BayerS16GBRG 30.0fps
      | output 0: 1944x1096 BL U8_V8_ER 420SP
      | inputClipRect 0: (0.00,0.00, 1.00,1.00)

  + Instruction List
      | id: 0
      +  0: CCDataSetupStage
          | EstimatedIspOutLatencyFrames: 5
          | NumConcurrentCaptures: 1
          | UnprocessedYuvBufferMask: 0
      +  1: ACSynchronizeStage
      +  2: AeAfApplyStage
      +  3: AcPluginStage
          | operation: opApply
      +  4: AcMergeStage
          | IspIn: [1944, 1096]
          | IspOut0: [1944, 1096]
          | IspOut1: [0, 0]
          | IspOut2: [0, 0]
      +  5: TempBufferAcquireStage
          | Buffer Index: 2
          | BufferRequirements: 1944x1 Pitch NonColor8
      +  6: TempBufferAcquireStage
          | Buffer Index: 3
          | BufferRequirements: 524288x1 Pitch NonColor8
      +  7: SensorISPCaptureStage
          | Source GUID: 0
          | Output A Buffer: 0
          | SensorMetadata Buffer: 2
          | Output IspStats Buffer: 3
      +  8: StatsBufferAcquireStage
          | Buffer Index: 1
          | BufferRequirements: 640x362 Pitch Y8_ER 420
      +  9: BlitStage
          | Input Buffer: 0
          | Output Buffer: 1
          | Filter: Nearest
          | Transform: None
          | Src Rect: Not used
          | Dst Rect: Not used
      + 10: StatsUpdateStage
          | Output Meta Buffer: 2
          | Output IspStats Buffer: 3
      + 11: BufferReturnStage
          | Output A Buffer: 2
      + 12: BufferReturnStage
          | Output A Buffer: 3
      + 13: AcPluginStage
          | operation: opAnalyze
      + 14: AfAnalysisStage
      + 15: MonitorStage
      + 16: ExifStage
      + 17: MakerNoteStage
      + 18: BufferReturnStage
          | Output A Buffer: 0
      + 19: MetadataReturnStage
      + 20: PerfStatsStage

Session::updatePerfHints()
updatePerfHints(): guid=0ll, CaptureIntent=1
updatePerfHints(): NvCameraCoreUseCase_Preview
NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

Created fiber 0x7f2c000b20 for CC 101 globalID 101 session 0
Thread 1 is working on CC 101 session 0 globalID 101 step 0
Session::updatePerfHints()
updatePerfHints(): guid=0ll, CaptureIntent=1
updatePerfHints(): NvCameraCoreUseCase_Preview
NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=1, m_value=30, m_timeout_ms=1000

NvPHSSendThroughputHints: sensorId=0, m_usecase=4, m_type=15, m_value=0, m_timeout_ms=1000

CC 101 session 0 completed step 0 in fiber 0x7f2c000b20
cc 101(1) session 0 runCount=0 runIspOut=0, latest ccId=0
CC 101 session 0 completed step 1 in fiber 0x7f2c000b20
NV AE and AfApply algorithms are active.
CC 101 session 0 completed step 2 in fiber 0x7f2c000b20
CC 101 session 0 completed step 3 in fiber 0x7f2c000b20
Created fiber 0x7f2c000dd0 for CC 102 globalID 102 session 0
Thread 2 is working on CC 102 session 0 globalID 102 step 0
CC 102 session 0 completed step 0 in fiber 0x7f2c000dd0
cc 102(2) session 0 runCount=1 runIspOut=0, latest ccId=0
CC 102 session 0 completed step 1 in fiber 0x7f2c000dd0
NV AE and AfApply algorithms are active.
CC 102 session 0 completed step 2 in fiber 0x7f2c000dd0
CC 102 session 0 completed step 3 in fiber 0x7f2c000dd0
CC 102 session 0 stalled step 4 in fiber 0x7f2c000dd0
FiberScheduler: cc 102, session 0 fiber 0x7f2c000dd0 not ready to execute stalled on 0x7f84824170 stage...

Thread 2 getting next capture
Thread 3 is waiting
Thread 2 is waiting
CC 101 session 0 completed step 4 in fiber 0x7f2c000b20
Thread 4 is working on CC 102 session 0 globalID 102 step 4
CC 101 session 0 completed step 5 in fiber 0x7f2c000b20
CC 102 session 0 completed step 4 in fiber 0x7f2c000dd0
CC 102 session 0 completed step 5 in fiber 0x7f2c000dd0
CC 101 session 0 completed step 6 in fiber 0x7f2c000b20
CC 101 session 0 processing step 7 in fiber 0x7f2c000b20
FiberScheduler: cc 101, session 0, fiber 0x7f2c000b20 in progress...

Thread 1 getting next capture
Thread 5 is waiting
Thread 1 is waiting
CC 102 session 0 completed step 6 in fiber 0x7f2c000dd0
CC 102 session 0 processing step 7 in fiber 0x7f2c000dd0
FiberScheduler: cc 102, session 0, fiber 0x7f2c000dd0 in progress...

Thread 4 getting next capture
Thread 6 is waiting
Thread 4 is waiting
PowerServiceHw:addRequest: table size: before: 0, after:1
        request table for ISP 0:
        req[0]: output1 width=0, output1 height=0, output1 Bpp=0
        req[0]: output2 width=0, output2 height=0, output2 Bpp=0
        req[0]: input width=1944, input height=1096, input Bpp=12
        req[0]: output width=130, output height=74, output Bpp=16
        req[0]: input fps=30
        req[0]: guID=0, stage type=SensorIspCapture
        req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
        req[450]: timeout=0
PowerServiceHwIsp:setLaBw: m_bwVal_Iso=54151 and m_bwVal_NonIso=0 KBpS

PowerServiceCore:setCameraBw: totalIsoBw=54151
PowerServiceHw:addRequest: table size: before: 1, after:2
        request table for ISP 0:
        req[0]: output1 width=0, output1 height=0, output1 Bpp=0
        req[0]: output2 width=0, output2 height=0, output2 Bpp=0
        req[0]: input width=1944, input height=1096, input Bpp=12
        req[0]: output width=130, output height=74, output Bpp=16
        req[0]: input fps=30
        req[0]: guID=0, stage type=SensorIspCapture
        req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
        req[409]: timeout=0
        req[1]: output1 width=0, output1 height=0, output1 Bpp=0
        req[1]: output2 width=0, output2 height=0, output2 Bpp=0
        req[1]: input width=1944, input height=1096, input Bpp=12
        req[1]: output width=1944, output height=1096, output Bpp=12
        req[1]: input fps=30
        req[1]: guID=0, stage type=SensorIspCapture
        req[1]: clock=0 Hz, iso bw=595438 KB/s, non-iso bw=0 KB/s
        req[450]: timeout=0
PowerServiceHw:updateRequests: table size: before: 2, after:2
        request table for ISP 0:
        req[0]: output1 width=0, output1 height=0, output1 Bpp=0
        req[0]: output2 width=0, output2 height=0, output2 Bpp=0
        req[0]: input width=1944, input height=1096, input Bpp=12
        req[0]: output width=130, output height=74, output Bpp=16
        req[0]: input fps=30
        req[0]: guID=0, stage type=SensorIspCapture
        req[0]: clock=0 Hz, iso bw=54151 KB/s, non-iso bw=0 KB/s
        req[404]: timeout=0
        req[1]: output1 width=0, output1 height=0, output1 Bpp=0
        req[1]: output2 width=0, output2 height=0, output2 Bpp=0
        req[1]: input width=1944, input height=1096, input Bpp=12
        req[1]: output width=1944, output height=1096, output Bpp=12
        req[1]: input fps=30
        req[1]: guID=0, stage type=SensorIspCapture
        req[1]: clock=0 Hz, iso bw=595438 KB/s, non-iso bw=0 KB/s
        req[445]: timeout=0
PowerServiceHwIsp:setLaBw: m_bwVal_Iso=595438 and m_bwVal_NonIso=0 KBpS

PowerServiceCore:setCameraBw: totalIsoBw=595438
Created fiber 0x7f2c000f70 for CC 103 globalID 103 session 0
Thread 7 is working on CC 103 session 0 globalID 103 step 0

this the dmesg for this daemon log:

[  252.323606] ------------[ cut here ]------------
[  252.328362] WARNING: CPU: 3 PID: 0 at /home/user/nvidia/nvidia_sdk/JetPack_4.2.2_Linux_GA_JETSON_NANO/Linux_for_Tegra/sources/kernel/kernel-4.9/kernel/watchdog_hld.c:143 watchdog_check_hardlockup_other_cpu+0xf8/0x120
[  252.348366] ---[ end trace edbfab487a1612cb ]---
Dump for CPU0:
pid: 6516  comm: SCF Execution
  x0 ffffffc0f948ec00   x1 0000000000000046
  x2 0000000000000000   x3 0000000000000000
  x4 ffffffc0fa09d050   x5 ffffffc0fa09cff8
  x6 0000000000000000   x7 ffffffc0fa09d020
  x8 ffffffc0fa09cff8   x9 ffffff8009e94440
 x10 0000000000000040  x11 0000000000000400
 x12 0000000000000020  x13 0000000000000000
 x14 0000000000000000  x15 000000000000000d
 x16 0000000000000000  x17 0000000000000000
 x18 0000007f827f9007  x19 ffffffc0f948ec20
 x20 ffffffc0f51ec018  x21 ffffffc0f948ec9c
 x22 ffffff8009077000  x23 ffffff8009e76e48
 x24 ffffff8009835018  x25 ffffffc0fefcef3c
 x26 ffffffc0f948ec00  x27 ffffff800a0a5000
 x28 ffffffc0c55c1c00  x29 ffffffc0fefcedb0
 x30 ffffff80081236dc   sp ffffffc0fefcedb0
  pc ffffff8008123590 cpsr 800001c5 (EL1h)

__synchronize_hardirq+0x30/0x70:
  pc ffffff8008123590   sp ffffffc0fefcedb0   fp ffffffc0fefcedb0
synchronize_irq+0x2c/0x90:
  pc ffffff80081236dc   sp ffffffc0fefcedc0   fp ffffffc0fefcede0
disable_irq+0x2c/0x38:
  pc ffffff8008123c84   sp ffffffc0fefcedf0   fp ffffffc0fefcee30
vi_disable_irq+0x34/0xa8:
  pc ffffff8008b52e14   sp ffffffc0fefcee40   fp ffffffc0fefcee50
vi_isr+0x94/0x188:
  pc ffffff8008b53104   sp ffffffc0fefcee60   fp ffffffc0fefcee80
__handle_irq_event_percpu+0x68/0x288:
  pc ffffff8008123260   sp ffffffc0fefcee90   fp ffffffc0fefceeb0
handle_irq_event_percpu+0x28/0x60:
  pc ffffff80081234a8   sp ffffffc0fefceec0   fp ffffffc0fefcef10
handle_irq_event+0x50/0x80:
  pc ffffff8008123530   sp ffffffc0fefcef20   fp ffffffc0fefcef40
handle_fasteoi_irq+0xc8/0x1b8:
  pc ffffff80081272f8   sp ffffffc0fefcef50   fp ffffffc0fefcef70
generic_handle_irq+0x34/0x50:
  pc ffffff800812224c   sp ffffffc0fefcef80   fp ffffffc0fefcefa0
__handle_domain_irq+0x68/0xc0:
  pc ffffff8008122930   sp ffffffc0fefcefb0   fp ffffffc0fefcefc0
gic_handle_irq+0x5c/0xb0:
  pc ffffff8008080d44   sp ffffffc0fefcefd0   fp ffffffc0fefcf000
el1_irq+0xe8/0x18c:
  pc ffffff8008082be8   sp ffffffc0fefcf010   fp ffffffc0fefcf040
_raw_spin_unlock_irqrestore+0x30/0x60:
  pc ffffff8008f5b828   sp ffffffc0b5017b00   fp ffffffc0b5017c30
pty_write+0x78/0xb8:
  pc ffffff80086da7b8   sp ffffffc0b5017c40   fp ffffffc0b5017c50
do_output_char+0x138/0x1b0:
  pc ffffff80086d1538   sp ffffffc0b5017c60   fp ffffffc0b5017c80
n_tty_write+0x1d0/0x3f8:
  pc ffffff80086d2a30   sp ffffffc0b5017c90   fp ffffffc0b5017cb0
tty_write+0x128/0x2e8:
  pc ffffff80086cd020   sp ffffffc0b5017cc0   fp ffffffc0b5017d50
__vfs_write+0x48/0x118:
  pc ffffff800825c228   sp ffffffc0b5017d60   fp ffffffc0b5017db0
vfs_write+0xac/0x1b0:
  pc ffffff800825d21c   sp ffffffc0b5017dc0   fp ffffffc0b5017e40
SyS_write+0x54/0xb0:
  pc ffffff800825e854   sp ffffffc0b5017e50   fp ffffffc0b5017e80
el0_svc_naked+0x34/0x38:
  pc ffffff80080838c0   sp ffffffc0b5017e90   fp 0000000000000000
debug>   x0 ffffffc0f948ec00   x1 0000000000000046
  x2 0000000000000000   x3 0000000000000000
  x4 ffffffc0fa09d050   x5 ffffffc0fa09cff8
  x6 0000000000000000   x7 ffffffc0fa09d020
  x8 ffffffc0fa09cff8   x9 ffffff8009e94440
 x10 0000000000000040  x11 0000000000000400
 x12 0000000000000020  x13 0000000000000000
 x14 0000000000000000  x15 000000000000000d
 x16 0000000000000000  x17 0000000000000000
 x18 0000007f827f9007  x19 ffffffc0f948ec20
 x20 ffffffc0f51ec018  x21 ffffffc0f948ec9c
 x22 ffffff8009077000  x23 ffffff8009e76e48
 x24 ffffff8009835018  x25 ffffffc0fefcef3c
 x26 ffffffc0f948ec00  x27 ffffff800a0a5000
 x28 ffffffc0c55c1c00  x29 ffffffc0fefcedb0
 x30 ffffff80081236dc   sp ffffffc0fefcedb0
  pc ffffff8008123590 cpsr 800001c5 (EL1h)
 sp_el0   ffffffc0c55c1c00
 elr_el1  ffffff8008f5b828
 spsr_el1 60000145
debug>

ShaneCCC · July 14, 2020, 8:10am

Sorry, still no see any useful message.

Could you try have nvargus-daemon run without timeout.

sudo service nvargus-daemon stop
sudo enableCamInfiniteTimeout=1 nvargus-daemon

pietro.assenza · July 14, 2020, 10:52am

Here the daemon log you asked for; BTW with this option set the system doesn’t freeze.

sudo enableCamInfiniteTimeout=1 enableCamScfLogs=5 nvargus-daemon
=== NVIDIA Libargus Camera Service (0.97.3)=== Listening for connections...
=== gst-launch-1.0[7876]: Connection established (7F8D2541D0)Thread 1 getting next capture
Thread 1 is waiting
Thread 2 getting next capture
Thread 2 is waiting
Thread 3 getting next capture
Thread 3 is waiting
Thread 4 getting next capture
Thread 4 is waiting
Thread 5 getting next capture
Thread 5 is waiting
Thread 6 getting next capture
Thread 6 is waiting
Thread 7 getting next capture
Thread 7 is waiting
Thread 8 getting next capture
Thread 8 is waiting
Thread 9 getting next capture
Thread 9 is waiting
Thread 10 getting next capture
Thread 10 is waiting
Thread 11 getting next capture
Thread 11 is waiting
Thread 12 getting next capture
Thread 12 is waiting
Starting services...
Worker thread IspHw statsComplete start
Worker thread IspHw frameComplete start
Worker thread IspHw statsComplete start
Worker thread IspHw frameComplete start
Worker thread CaptureScheduler checkFramePending start
Worker thread CaptureScheduler frameStart start
Worker thread V4L2CaptureScheduler checkCaptureComplete start
Worker thread V4L2CaptureScheduler issueCaptures start
Worker thread PS handleRequests start
getInstance: s_instance(0x7f887d9370)
getInstance: s_instance(0x7f887d9370)
subscribe: SensorType(gyroscope)
subscribe: create SensorType(gyroscope) sensor(0x7f887db970)
subscribe: SensorType(accelerometer)
subscribe: create SensorType(accelerometer) sensor(0x7f887e0ed0)
AC plugin not present: dlopen "acplugin.so", acplugin.so: cannot open shared object file: No such file or directory
Services are started
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
LSC: LSC surface is not based on full res!
=== gst-launch-1.0[7876]: CameraProvider initialized (0x7f88829830)sourceRegistry[0] assigned

ispRegistry[0] assigned

Using Source GUID 0
Worker thread ViCsiHw frameComplete start
Worker thread ViCsiHw frameStart start
SCF: Error BadParameter:  (propagating from src/services/capture/NvViCsiHw.cpp, function openViCsi(), line 116)
Worker thread ViCsiHw frameComplete stop
Worker thread ViCsiHw frameStart stop
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function open(), line 307)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function openSource(), line 355)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function openSource(), line 478)
SCF: Error BadParameter:  (propagating from src/api/Session.cpp, function initialize(), line 262)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function createSession(), line 569)
(Argus) Error BadParameter:  (propagating from src/api/CaptureSessionImpl.cpp, function initialize(), line 120)
(Argus) Error BadParameter:  (propagating from src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 250)
(NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/argus/src/api/CaptureSessionImpl.cpp:197) (in Mutex.cpp, function lock(), line 79)
(Argus) Error InvalidState: Element not found (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function remove(), line 172)
(NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
=== gst-launch-1.0[7876]: CameraProvider destroyed (0x7f88829830)Stopping services...
Worker thread CaptureScheduler frameStart stop
Worker thread CaptureScheduler checkFramePending stop
Worker thread IspHw statsComplete stop
Worker thread IspHw frameComplete stop
Worker thread IspHw statsComplete stop
Worker thread IspHw frameComplete stop
Worker thread V4L2CaptureScheduler issueCaptures stop
Worker thread V4L2CaptureScheduler checkCaptureComplete stop
Worker thread PS handleRequests stop
unsubscribe: SensorType(gyroscope)
unsubscribe: gyroscope sensor(0x7f887db970)
unsubscribe: SensorType(accelerometer)
unsubscribe: accelerometer sensor(0x7f887e0ed0)
Services are stopped
closeInstance: ms_instance((nil))
FiberScheduler: shutting down; 0 fibers still in map

FiberScheduler: thread 8 exiting
FiberScheduler: thread 4 exiting
FiberScheduler: thread 6 exiting
FiberScheduler: thread 12 exiting
FiberScheduler: thread 3 exiting
FiberScheduler: thread 1 exiting
FiberScheduler: thread 12 has been joined
FiberScheduler: thread 10 exiting
FiberScheduler: thread 2 exiting
FiberScheduler: thread 5 exiting
FiberScheduler: thread 11 exiting
FiberScheduler: thread 9 exiting
FiberScheduler: thread 11 has been joined
FiberScheduler: thread 7 exitingFiberScheduler: thread 10 has been joined

FiberScheduler: thread 9 has been joined
FiberScheduler: thread 8 has been joined
FiberScheduler: thread 7 has been joined
FiberScheduler: thread 6 has been joined
FiberScheduler: thread 5 has been joined
FiberScheduler: thread 4 has been joined
FiberScheduler: thread 3 has been joined
FiberScheduler: thread 2 has been joined
FiberScheduler: thread 1 has been joined
=== gst-launch-1.0[7876]: Connection closed (7F8D2541D0)=== gst-launch-1.0[7876]: Connection cleaned up (7F8D2541D0)

ShaneCCC · July 14, 2020, 11:16am

Sorry, forget Nano/TX1 didn’t support infinite mode.
Could you add set_mode_delay_ms in the device tree to try. Maybe set it as 1000 to try.

Topic		Replies	Views
New custom camera bring-up - v4l2-ctl working but gstreamer fail Jetson TX2 camera , nvbugs	30	1912	March 20, 2020
Nvcamera daemon crashing when running two gstreamer captures Jetson TX2	13	1289	October 18, 2021
Jetson Nano v4l2 capture working, nvarguscamerasrc capture not working Jetson Nano camera , gstreamer	6	1615	December 15, 2021
How can i solve this problem in Libargus? "Failed to get IFrame interface" Jetson Xavier NX camera	8	1341	September 19, 2021
Gstreamer nvarguscamerasrc not working at TX2 Custom Board with Serdes Gmsl Camera Jetson TX2 camera , gstreamer , device-tree	2	1416	October 18, 2021
Nvarguscamerasrc does not work Jetson AGX Orin camera	5	783	October 25, 2023
Release 28.2.1 nvcamera-daemon Jetson TX2 camera	27	1315	October 18, 2021
Jetson Nano with XGS12000 Imager not working with gstreamer Jetson Nano board-design , gstreamer	7	508	August 10, 2022
nvargus-deamon problem Jetson TX2	8	1425	October 18, 2021
IMX390 image sensor cannot get image on TX2 r28.2 Jetson TX2	2	1294	October 18, 2021

Gstreamer works at 4K but freeze the system at 2K

Related topics