The imx297 camera on the Jetson Nano crashes Linux when running gstreamer

Robotics & Edge Computing Jetson Systems Jetson Nano
1

Jetson Nano R32.7.6
The camera is connected via FPD link.

The imx297 is detected in the system.
On startup

v4l2-ctl -d /dev/video0 --set-fmt-video=width=720,height=540,pixelformat=RG10 --set-ctrl=bypass_mode=0,sensor_mode=0 --stream-mmap

Everything works.

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 61.38 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 61.19 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 61.12 fps

When I start capturing an image, the Jetson Nano module freezes and reboots after 10 seconds.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1408,height=(int)1088, framerate=60/1' ! nvvidconv flip-method=0 ! 'video/x-raw, format=(string)I420' ! xvimagesink sync=false --verbose
gst-launch-1.0 output

Setting pipeline to PAUSED …

Pipeline is live and does not need PREROLL …

Setting pipeline to PLAYING …

New clock: GstSystemClock

/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)720, height=(int)540, format=(string)NV12, framerate=(fraction)60/1

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int) 720, height=(int) 540, format=(string)NV12, framerate=(fraction)60/1

/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw, width=(int) 720, height=(int) 540, framerate=(fraction)60/1, format=(string)I420

/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int) 720, height=(int) 540, framerate=(fraction)60/1, format=(string)I420

/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw, width=(int) 720, height=(int) 540, framerate=(fraction)60/1, format=(string)I420

/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int) 720, height=(int) 540, framerate=(fraction)60/1, format=(string)I420

/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int) 720, height=(int) 540, format=(string)NV12, framerate=(fraction)60/1

/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int) 720, height=(int) 540, format=(string)NV12, framerate=(fraction)60/1

GST_ARGUS: Creating output stream

CONSUMER: Waiting until producer is connected…

GST_ARGUS: Available Sensor modes :

GST_ARGUS: 720 x 540 FR = 59,999999 fps Duration = 16666667 ; Analog Gain range min 0,000000, max 48,000000; Exposure Range min 0, max 1048575000;

GST_ARGUS: Running with following settings:

Camera index = 0

Camera mode = 0

Output Stream W = 720 H = 540

seconds to Run = 0

Frame Rate = 59,999999

GST_ARGUS: Setup Complete, Starting captures for 0 seconds

GST_ARGUS: Starting repeat capture requests.

CONSUMER: Producer has connected; continuing.

nvbuf_utils: dmabuf_fd -1 mapped entry NOT found

nvbuf_utils: Can not get HW buffer from FD… Exiting…

CONSUMER: ERROR OCCURRED

ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: CANCELLED

Additional debug info:

Argus Error Status

Execution ended after 0:00:02.214725033

Setting pipeline to PAUSED …

Setting pipeline to READY …

Log nvargus-daemon

# pkill nvargus-daemon
# nvargus-daemon 2>&1 | tee argus-output.log
log nvargus-daemon

PuTTY X11 proxy: Unsupported authorisation protocol

=== NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections…=== gst-launch-1.0[6153]: Connection established (7F79DED1D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0

OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree

-— imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----

CAM: serial no file already exists, skips storing againLSC: LSC surface is not based on full res!

-— imager: No override file found. ----

initializeDevNode: Failed to open dev node ‘/dev/camera/video1’; No such file or directory, trying alternate default location

(NvCamV4l2) Error BadParameter: Control 10094858 not found (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findControlById(), line 1874)

(NvCamV4l2) Error BadParameter: (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function getControlRange(), line 331)

initialize: focus query failed

=== gst-launch-1.0[6153]: CameraProvider initialized (0x7f74ab9c20)CAM: serial no file already exists, skips storing againLSC: LSC surface is not based on full res!

PuTTY X11 proxy: Unsupported authorisation protocol

SCF: Error Timeout: ISP port 0 timed out! (in src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 492)

SCF: Error InvalidState: Something went wrong with waiting on Isp frame end (in src/services/capture/NvIspHw.cpp, function waitIspFrameEnd(), line 550)

SCF: Error InvalidState: (propagating from src/common/Utils.cpp, function workerThread(), line 116)

SCF: Error InvalidState: Worker thread IspHw frameComplete failed (in src/common/Utils.cpp, function workerThread(), line 133)

Error: waitCsiFrameStart timeout guid 0

************VI/CSI Debug Registers**********

VI_CFG_INTERRUPT_MASK_0 = 0x00000000

VI_CFG_INTERRUPT_STATUS_0 = 0x00000000

VI_CSI_0_ERROR_STATUS_0 = 0x00000000

VI_CSI_0_ERROR_INT_MASK_0 = 0x00000000

VI_CSI_1_ERROR_STATUS_0 = 0x00000000

VI_CSI_1_ERROR_INT_MASK_0 = 0x00000000

CSI_CSI_PIXEL_PARSER_A_INTERRUPT_MASK_0 = 0x00000000

CSI_CSI_PIXEL_PARSER_A_STATUS_0 = 0x00000000

CSI_CSI_PIXEL_PARSER_B_INTERRUPT_MASK_0 = 0x00000000

CSI_CSI_PIXEL_PARSER_B_STATUS_0 = 0x00000000

CSI_CSI_CIL_A_INTERRUPT_MASK_0 = 0x00000000

CSI_CSI_CIL_A_STATUS_0 = 0x00000000

CSI_CSI_CILA_STATUS_0 = 0x00000000

CSI_CSI_CIL_B_INTERRUPT_MASK_0 = 0x00000000

CSI_CSI_CIL_B_STATUS_0 = 0x00000000

CSI_CSI_CILB_STATUS_0 = 0x00000000

CSI_CSI_READONLY_STATUS_0 = 0x00000000

CSI_DEBUG_CONTROL_0 = 0x312a1b00

CSI_DEBUG_COUNTER_0_0 = 0x00000000

CSI_DEBUG_COUNTER_1_0 = 0x00000000

CSI_DEBUG_COUNTER_2_0 = 0x00000000

*****************************************

SCF: Error Timeout: (propagating from src/common/Utils.cpp, function workerThread(), line 116)

SCF: Error Timeout: Worker thread ViCsiHw frameStart failed (in src/common/Utils.cpp, function workerThread(), line 133)

dmesg --follow

[ 1793.774510] Caller is tegra_channel_fmt_align+0x94/0x148
[ 1793.779919] Caller is tegra_channel_update_format+0x6c/0x168
[ 1793.785648] Caller is tegra_channel_fmt_align+0x94/0x148
[ 1793.855511] imx296_set_exposure: oneH_Period 15, frame_per_sencond 60, frame_legth 1118[line], val 16666[us], shs_time 7[lines]
[ 1793.867003] imx296_set_frame_rate : val=60000000 frame_length=1125 : mode->signal_properties.pixel_clock.val=118800000 : mode->control_properties.framerate_factor=1000000 : mode->image_properties.line_length=1760
[ 1795.557506] fence timeout on [ffffffc0f3eb6a80] after 1500ms
[ 1795.557540] name=[nvhost_sync:14], current value=0 waiting value=1
[ 1795.557564] ---- mlocks ----

[ 1795.557606] ---- syncpts ----
[ 1795.557637] id 1 (disp0_a) min 180 max 181 refs 1 (previous client : )
[ 1795.557659] id 2 (disp0_b) min 1 max 1 refs 1 (previous client : )
[ 1795.557680] id 3 (disp0_c) min 1 max 1 refs 1 (previous client : )
[ 1795.557700] id 4 (disp1_a) min 1 max 1 refs 1 (previous client : )
[ 1795.557721] id 5 (disp1_b) min 1 max 1 refs 1 (previous client : )
[ 1795.557740] id 6 (disp1_c) min 1 max 1 refs 1 (previous client : )
[ 1795.557765] id 8 (gm20b_507) min 17862 max 17866 refs 1 (previous client : )
[ 1795.557786] id 9 (gm20b_506) min 14 max 14 refs 1 (previous client : )
[ 1795.557812] id 11 (gm20b_505) min 920 max 924 refs 1 (previous client : gm20b_505)
[ 1795.557836] id 13 (54600000.isp_0) min 0 max 3 refs 4 (previous client : )
[ 1795.557857] id 14 (54600000.isp_1) min 0 max 3 refs 4 (previous client : )
[ 1795.557879] id 15 (54600000.isp_2) min 7 max 15 refs 10 (previous client : )
[ 1795.557901] id 16 (54600000.isp_3) min 1 max 3 refs 4 (previous client : )
[ 1795.557930] id 21 (gm20b_504) min 704 max 704 refs 1 (previous client : )
[ 1795.557951] id 22 (gm20b_503) min 60 max 60 refs 1 (previous client : )
[ 1795.557973] id 23 (gm20b_502) min 2 max 2 refs 1 (previous client : )
[ 1795.557994] id 24 (gm20b_501) min 2 max 2 refs 1 (previous client : )
[ 1795.558014] id 25 (gm20b_500) min 6 max 6 refs 1 (previous client : )
[ 1795.558036] id 26 (vblank0) min 107646 max -2 refs 1 (previous client : )
[ 1795.558062] id 29 (54080000.vi_0) min 0 max 3 refs 4 (previous client : 54080000.vi_0)
[ 1795.558085] id 31 (54080000.vi_2) min 2 max 7 refs 7 (previous client : 54080000.vi_2)
[ 1795.558112] id 34 (54080000.vi_4) min 1 max 3 refs 4 (previous client : 54080000.vi_4)
[ 1795.558136] id 37 (gm20b_499) min 6 max 6 refs 1 (previous client : )
[ 1795.558156] id 38 (gm20b_498) min 8 max 8 refs 1 (previous client : )
[ 1795.558177] id 39 (gm20b_497) min 6 max 6 refs 1 (previous client : )
[ 1795.558198] id 40 (gm20b_496) min 6 max 6 refs 1 (previous client : )
[ 1795.558218] id 41 (gm20b_495) min 6 max 6 refs 1 (previous client : )
[ 1795.558239] id 42 (gm20b_494) min 6 max 6 refs 1 (previous client : )

[ 1795.558511] ---- channels ----
[ 1795.558544]
channel 0 - 54600000.isp

[ 1795.558568] 0-54600000.isp (18):
[ 1795.558591] active class 01, offset 0050, val 0000000d
[ 1795.558611] DMAPUT 000001b0, DMAGET 00000128, DMACTL 00000000
[ 1795.558627] CBREAD 0000000d, CBSTAT 00010050
[ 1795.558659]
000000007633ef16: JOB, syncpt_id=13, syncpt_val=1, first_get=000000a8, timeout=10000, num_slots=3, num_handles=1
[ 1795.558702] GATHER at ffd10000+4c48, 8 words
[ 1795.558717] 20000001
[ 1795.558732] 0000040d
[ 1795.558746] 20000001
[ 1795.558760] 0000050e
[ 1795.558772] 20000001
[ 1795.558786] 00000610
[ 1795.558798] 200c0001
[ 1795.558812] 00000005

[ 1795.558847] GATHER at ffd10000+4c68, 2 words
[ 1795.558860] 20000001
[ 1795.558874] 0000000f

[ 1795.558921]
channel 1 - 54080000.vi

[ 1795.558944] 1-54080000.vi (12):
[ 1795.558963] active class 01, offset 0050, val 0000001d
[ 1795.558982] DMAPUT 000000e8, DMAGET 000000e8, DMACTL 00000000
[ 1795.558998] CBREAD 0000001d, CBSTAT 00010050
[ 1795.559026]
000000005b7cf141: JOB, syncpt_id=29, syncpt_val=1, first_get=00000000, timeout=10000, num_slots=7, num_handles=1
[ 1795.559056] GATHER at ffd00000+0000, 35 words
[ 1795.559069] 11620001
[ 1795.559083] 0000001f
[ 1795.559096] 11630002
[ 1795.559109] 00000000
[ 1795.559121] 00000000
[ 1795.559135] 103a0001
[ 1795.559147] 10100010
[ 1795.559160] 103c0001
[ 1795.559172] 10101010
[ 1795.559185] 11420001
[ 1795.559198] 00000001
[ 1795.559211] 11430006
[ 1795.559223] 00000000
[ 1795.559237] 001c984c
[ 1795.559249] 00000000
[ 1795.559262] 00000000
[ 1795.559274] 00000000
[ 1795.559287] 00000000
[ 1795.559299] 11590001
[ 1795.559312] 00000003
[ 1795.559325] 160e0006
[ 1795.559338] 007f0014
[ 1795.559351] 080301f0
[ 1795.559364] 00000000
[ 1795.559376] 00140000
[ 1795.559389] 0000f005
[ 1795.559402] 00000000
[ 1795.559415] 16420001
[ 1795.559427] 00000001
[ 1795.559440] 164b0003
[ 1795.559452] 00000000
[ 1795.559465] 00000000
[ 1795.559478] 00000040
[ 1795.559491] 168b0001
[ 1795.559503] 00000000

[ 1795.559535] GATHER at ffd00000+008c, 3 words
[ 1795.559548] 304e0005
[ 1795.559561] 00000007
[ 1795.559574] 0000000f

[ 1795.559605] GATHER at ffd00000+0098, 17 words
[ 1795.559617] 11430006
[ 1795.559631] 00200002
[ 1795.559643] 001c984c
[ 1795.559656] 00000000
[ 1795.559669] 021c02d0
[ 1795.559682] 00000384
[ 1795.559695] 0000002b
[ 1795.559708] 164b0003
[ 1795.559720] 00000000
[ 1795.559733] 00000000
[ 1795.559745] 0000004a
[ 1795.559758] 20000001
[ 1795.559771] 00001c1d
[ 1795.559784] 20000001
[ 1795.559797] 00001522
[ 1795.559810] 21410001
[ 1795.559822] 00000001

[ 1795.559852] GATHER at ffd00000+00dc, 2 words
[ 1795.559865] 20000001
[ 1795.559878] 0000001f

[ 1795.559919]
---- host general irq ----

[ 1795.559935] sync_hintmask_ext = 0xc0000000
[ 1795.559951] sync_hintmask = 0x80000000
[ 1795.559966] sync_intc0mask = 0x00000001
[ 1795.559980] sync_intmask = 0x00000011
[ 1795.559994]
---- host syncpt irq mask ----

[ 1795.560011] syncpt_thresh_int_mask(0) = 0x54410001
[ 1795.560028] syncpt_thresh_int_mask(1) = 0x44000001
[ 1795.560044] syncpt_thresh_int_mask(2) = 0x00000010
[ 1795.560060] syncpt_thresh_int_mask(3) = 0x00000000
[ 1795.560075] syncpt_thresh_int_mask(4) = 0x00000000
[ 1795.560091] syncpt_thresh_int_mask(5) = 0x00000000
[ 1795.560106] syncpt_thresh_int_mask(6) = 0x00000000
[ 1795.560122] syncpt_thresh_int_mask(7) = 0x00000000
[ 1795.560137] syncpt_thresh_int_mask(8) = 0x00000000
[ 1795.560152] syncpt_thresh_int_mask(9) = 0x00000000
[ 1795.560168] syncpt_thresh_int_mask(10) = 0x00000000
[ 1795.560185] syncpt_thresh_int_mask(11) = 0x00000000
[ 1795.560198]
---- host syncpt irq status ----

[ 1795.560215] syncpt_thresh_cpu0_int_status(0) = 0x00010000
[ 1795.560231] syncpt_thresh_cpu0_int_status(1) = 0x00000000

We have a solution based on the imx296, which is similar to the imx297.
The imx296 operates using the same FPD link.

This is very strange behavior; we’ve never encountered anything like it.
Usually, the camera may not work, but this doesn’t cause the system to reboot.
Therefore, we can’t retrieve the logs, since everything is clean after a reboot.

Can you tell me what the problem might be?

3

The problem could be the CID function write incorrect config to output invalidate data to Nano cause the problem.

Could you modify the CID like xxx_set_exposure to dummy function to confirm the problem.

4

Thanks for the reply.
We tried removing the imx296_set_exposure function, its callback, and the CID from the driver.
But that didn’t work.

Any other ideas?

5

Dump device tree to confirm the tegra_sinterface is correct.

6

We’re attaching a DTS dump.

But this is a fully working DTS for the IMX296.

For the IMX297, some resolution changes have been made.
The changes were made to both the DTS and the IMX296 driver.
The IMX296 and IMX297 are completely identical except for the resolution.
We understand the problem isn’t with the driver or DTS, but rather with calls from

We’re currently trying to run it on an Original NX system.

DTS for imx297

include <dt-bindings/media/camera.h>

include <dt-bindings/platform/t210/t210.h>

include <dt-bindings/gpio/gpio.h>

define CAM1_PWDN TEGRA_GPIO(S,7)

define CAM2_PWDN TEGRA_GPIO(T,0)

define CAM3_PWDN_GPIO TEGRA_GPIO(V,0)

define CAM4_PWDN TEGRA_GPIO(H,6)

define CAMERA_I2C_MUX_BUS(x) (0x1E + x)

/ {

host1x {

gpio@6000d000 {

camera-control-output-low {

        gpio-hog;

        output-low;

        gpios = < CAM1_PWDN 0  CAM2_PWDN 0>;

        label = "cam1-pwdn", "cam2-pwdn";

    };

};

vi{

    status = "okay";

    num-channels = <1>;

ports {

        #address-cells = <1>;

        #size-cells = <0>;

port@0{

            status = "okay";

            reg = <0>;

ds90ub95x_vi_in0: endpoint {

                status = "okay";

                port-index = <4>;  // MIPI0 -> CSI-A-B

                bus-width = <1>;   // Use CSI-A and CSI-B

                remote-endpoint = <&ds90ub95x_csi_out0>;

            };

        };

    };//end port@1

};//end vi

nvcsi{

    status = "okay";

    num-channels = <1>;

    #address-cells = <1>;

    #size-cells = <0>;

channel@0 {

        status = "okay";

        reg = <0>;

ports {

            #address-cells = <1>;

            #size-cells = <0>;

port@0 {

                status = "okay";

                reg = <0>;

ds90ub95x_i2c_csi_in0: endpoint@0 {

                    status = "okay";

                    port-index = <4>; // MIPI4 -> CSI-C

                    bus-width = <1>;  // Use CSI-C and CSI-D

                    remote-endpoint = <&ds90ub95x_i2c_out0>;

                };

            };

port@1 {

                status = "okay";

                reg = <1>;

ds90ub95x_csi_out0: endpoint@1 {

                    status = "okay";

                    remote-endpoint = <&ds90ub95x_vi_in0>;

                };

            };

        };

    };//end chanel@0       

};//end nvcsi

};//end host1x

i2c@7000c500 {

fpdlink: ds90ub954@30 {

    status = "okay";

    compatible = "ti,ds90ub954";

    reg = <0x30>;

    csi-lane-count = <0x01>;

    csi-lane-speed = <1600>;

    continuous-clock;

//test-pattern;

// pdb-gpio = <&gpio TEGRA_GPIO(E, 0) 0>;

// pass-gpio = <&gpio TEGRA_GPIO(S, 7) 0>;

// lock-gpio = <&gpio TEGRA_GPIO(T, 0) 0>;

i2c_address_s0: num-alias-pairs0 {list-cells = <1>;};

serializers {

            status = "okay";

            num-channels = <0x01>;

            #address-cells = <1>;

            #size-cells = <0>;

serializer@0 {

                status = "okay";

                rx-channel=<0x00>;

                i2c-address=<0x18>;

                csi-lane-count = <0x01>;

                continuous-clock;

//i2c-pass-through-all;1

                virtual-channel-map = <0x0>;

// test-pattern;

                gpio0-output-enable = <1>;

                gpio0-control = <0x9>;

                i2c-slave = <&i2c_address_s0 0x1a>;// 0x11>;

                slave-alias = <&i2c_address_s0 0x1a>;// 0x21>;

// i2c-slave = <&i2c_address_s0 0x6c>;

// slave-alias = <&i2c_address_s0 0x6c>;

            };

        };

    };

//---------------------------------------------------------------------------------

imx296_single_cam0: rbpcv2_imx296_f@1a {

            status = "okay";

            compatible = "nvidia,imx296";

/* I2C device address */

            reg = <0x1a>;

/* V4L2 device node location */

            devnode = "video0";

/* Physical dimensions of sensor */

//physical_w = “4.968”;

//physical_h = “3.726”;

            physical_w = "3.680";

            physical_h = "2.760";

            sensor_model = "imx296";

            use_sensor_mode_id = "true";

            reset-gpios = <&gpio CAM4_PWDN GPIO_ACTIVE_HIGH>;

mode0 {

                mclk_khz = "37125";

                num_lanes = "1";

                tegra_sinterface = "serial_e";

                phy_mode = "DPHY";

                discontinuous_clk = "no";

                dpcm_enable = "false";

                cil_settletime = "0";

                active_w = "720"; //1408";

                active_h = "540"; //"1088";

                pixel_t = "bayer_rggb";

                readout_orientation = "90";

                line_length = "1760";

                inherent_gain = "1";

                mclk_multiplier = "4";

                pix_clk_hz = "118800000";

                gain_factor = "10";

                framerate_factor = "1000000";

                exposure_factor = "1000000";

                min_gain_val = "0"; // 1.00x 

                max_gain_val = "480"; // 48x 

                step_gain_val = "1";

                default_gain = "10"; //1 1.00x 

                min_hdr_ratio = "1";

                max_hdr_ratio = "1";

                min_framerate = "10000000"; 

                max_framerate = "60000000";

                step_framerate = "1";

                default_framerate = "60000000";

                min_exp_time = "0"; // us 

                max_exp_time = "1048575"; // us 

                step_exp_time = "1";

                default_exp_time = "29"; // us 

                embedded_metadata_height = "0";//"2";

            };//mode0

ports {

                #address-cells = <1>;

                #size-cells = <0>;

port@0 {

                    reg = <0>;

ds90ub95x_i2c_out0: endpoint {

                        port-index = <4>;

                        bus-width = <1>;

                       remote-endpoint = <&ds90ub95x_i2c_csi_in0>;

                    };

                };

            };

        };//rbpcv2_imx296_f

};

tegra-camera-platform {

    compatible = "nvidia, tegra-camera-platform";

    num_csi_lanes = <1>;  // Changed 2 -> 4

    max_lane_speed = <1500000>;

    min_bits_per_pixel = <10>; //<16>;  

    vi_peak_byte_per_pixel = <2>;

    vi_bw_margin_pct = <25>;

    max_pixel_rate = <240000>;

    isp_peak_byte_per_pixel = <5>;

    isp_bw_margin_pct = <25>;

/**

    \* The general guideline for naming badge_info contains 3 parts, and is as follows,

    \* The first part is the camera_board_id for the module; if the module is in a FFD

    \* platform, then use the platform name for this part.

    \* The second part contains the position of the module, ex. rear or front.

    \* The third part contains the last 6 characters of a part number which is found

    \* in the module's specsheet from the vender.

    \*/

modules {

module0 {

            status = "okay";

            badge = "porg_rear_RBPCV2";

            position = "rear";

            orientation = "1";

drivernode0 {

                status = "okay";

                pcl_id = "v4l2_sensor";

                devname = "imx296 2-001a";

                proc-device-tree = "/proc/device-tree/i2c@7000c500/rbpcv2_imx296_f@1a";

            };

        };

};

};

};

7

Do you verify by 720x540

8

My apologies.
I copied it incorrectly for the forum.
Actually, the resolution is always correct.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)720,height=(int)540, framerate=60/1' ! nvvidconv flip-method=0 ! 'video/x-raw, format=(string)I420' ! xvimagesink sync=false --verbose
9

Get the trace log.

modprobe rtcpu_debug

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 3 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)720,height=(int)540, framerate=60/1' ! nvvidconv flip-method=0 ! 'video/x-raw, format=(string)I420' ! xvimagesink sync=false --verbose


cat /sys/kernel/debug/tracing/trace
10

Thanks for the reply.

I’d like to continue this thread.

The thing is, you switched to an Orin NX platform.
In this case, we were able to start image capture.
But capture doesn’t work correctly.
If the scene is static, the image is fine.
When a moving object enters the frame, lines in the image start to break.
Moreover, the lines break in the middle of the image.

It’s also interesting that when I run

v4l2-ctl -d /dev/video0 --set-fmt-video=width=720,height=540,pixelformat=RG10 --set-ctrl bypass_mode=0,sensor_mode=0 --stream-mmap --stream-count=100

Nothing happens, there’s no frame count.

When I start the capture, we get an image, but with atifacts.

st-launch-1.0 nvarguscamerasrc aelock=true sensor-id=0 wbmode=9 ! 'video/x-raw(memory:NVMM), width=(int)728,height=(int)540,framerate=60/1' ! nvvidconv flip-method=0 ! 'video/x-raw, format=(string)I420' ! xvimagesink sync=false -e

Please review the startup debug.

trace-log.txt (359.1 KB)

We think that by understanding this, we can figure out the cause of the artifacts.

DTS for IMX297 via FPD link

//include <dt-bindings/media/camera.h>
//include <dt-bindings/platform/t210/t210.h>
//include <dt-bindings/gpio/gpio.h>

include <dt-bindings/tegra234-p3767-0000-common.h>
include <dt-bindings/gpio/gpio.h>
include <dt-bindings/clock/tegra234-clock.h>

//define CAM1_PWDN TEGRA234_MAIN_GPIO(S,7)
//define CAM2_PWDN TEGRA234_MAIN_GPIO(T,0)
//define CAM3_PWDN_GPIO TEGRA234_MAIN_GPIO(V,0)
//define CAM4_PWDN TEGRA234_MAIN_GPIO(H,6)

define CAM0_PWDN TEGRA234_MAIN_GPIO(H, 6)
define CAM1_PWDN TEGRA234_MAIN_GPIO(AC, 0)
define CAM3_PWDN TEGRA234_MAIN_GPIO(G, 6)

define CAMERA_I2C_MUX_BUS(x) (0x1E + x)

/ {
//host1x {

gpio@6000d000 {
	camera-control-output-low {
		gpio-hog;
		output-low;

// gpios = < CAM1_PWDN 0 CAM2_PWDN 0>;
label = “cam1-pwdn”, “cam2-pwdn”;
};
};

//vi_base: vi {
tegra-capture-vi{
    status = "okay";
    num-channels = <1>;
    ports {
        #address-cells = <1>;
        #size-cells = <0>;
        port@0{
            status = "okay";
            reg = <0>;
            ds90ub95x_vi_in0: endpoint {
                status = "okay";
                // IMX477
                port-index = <2>;  // MIPI0 -> CSI-A-B
                bus-width = <1>;   // Use CSI-A and CSI-B
                remote-endpoint = <&ds90ub95x_csi_out0>;
            };
        };
    };//end port@1
};//end vi

bus@0 {
host1x@13e00000 {
nvcsi@15a00000 {
status = “okay”;
num-channels = <1>;
#address-cells = <1>;
#size-cells = <0>;

    channel@0 {
        status = "okay";
        reg = <0>;
        ports {
            #address-cells = <1>;
            #size-cells = <0>;
            port@0 {
                status = "okay";
                reg = <0>;
                ds90ub95x_i2c_csi_in0: endpoint@0 {
                    status = "okay";
                    port-index = <2>; // MIPI4 -> CSI-C
                    bus-width = <1>;  // Use CSI-C and CSI-D
                    remote-endpoint = <&ds90ub95x_i2c_out0>;
                };
            };
            port@1 {
                status = "okay";
                reg = <1>;
                ds90ub95x_csi_out0: endpoint@1 {
                    status = "okay";
                    remote-endpoint = <&ds90ub95x_vi_in0>;
                };
            };
        };
    };//end chanel@0       
};//end nvcsi

};//end host1x

//i2c@7000c500 {
i2c@3160000 {
fpdlink: ds90ub954@30 {
status = “okay”;
compatible = “ti,ds90ub954”;
reg = <0x30>;
csi-lane-count = <0x01>;
csi-lane-speed = <1600>;
//continuous-clock;

    //test-pattern;

// pdb-gpio = <&gpio TEGRA_GPIO(E, 0) 0>;
// pass-gpio = <&gpio TEGRA_GPIO(S, 7) 0>;
// lock-gpio = <&gpio TEGRA_GPIO(T, 0) 0>;

    i2c_address_s0: num-alias-pairs0 {list-cells = <1>;};

        serializers {
            status = "okay";
            num-channels = <0x01>;
            #address-cells = <1>;
            #size-cells = <0>;

            serializer@0 {
                status = "okay";
                rx-channel=<0x00>;
                i2c-address=<0x18>;
                csi-lane-count = <0x01>;
                //continuous-clock;
                //i2c-pass-through-all;1

                virtual-channel-map = <0x0>;
                
            // test-pattern;
                
                gpio0-output-enable = <1>;
                gpio0-control = <0x9>;

                
                i2c-slave = <&i2c_address_s0 0x1a>;// 0x11>;
                slave-alias = <&i2c_address_s0 0x1a>;// 0x21>;
//                               i2c-slave = <&i2c_address_s0 0x6c>;
//                               slave-alias = <&i2c_address_s0 0x6c>;
            };
        };
    };

//---------------------------------------------------------------------------------
imx296_single_cam0: rbpcv2_imx296_f@1a {
status = “okay”;
compatible = “nvidia,imx296”;
/* I2C device address */
reg = <0x1a>;

            /* V4L2 device node location */
            devnode = "video0";

            /* Physical dimensions of sensor */
            //physical_w = "4.968";
            //physical_h = "3.726";
            
            physical_w = "3.680";
            physical_h = "2.760";

            sensor_model = "imx296";

            use_sensor_mode_id = "true";

            //reset-gpios = <&gpio CAM2_PWDN GPIO_ACTIVE_HIGH>;
            reset-gpios = <&gpio CAM3_PWDN GPIO_ACTIVE_HIGH>;

clocks = <&bpmp TEGRA234_CLK_EXTPERIPH1>, <&bpmp TEGRA234_CLK_EXTPERIPH1>;
clock-names = “extperiph1”, “pllp_grtba”;
mclk = “extperiph1”;
clock-frequency = <37125000>;
set_mode_delay_ms = “1000”;

            mode0 { // IMX296_MODE_1408x1088_60FPS 
                mclk_khz = "37125";
                num_lanes = "1";

// tegra_sinterface = “serial_f”;
tegra_sinterface = “serial_c”;
phy_mode = “DPHY”;
discontinuous_clk = “yes”;
dpcm_enable = “false”;
cil_settletime = “0”;

                active_w = "720";//1408";
                active_h = "540"; //1088";
                //pixel_t = "bayer_rggb";
                readout_orientation = "90";
                line_length = "720";//"1760";
                inherent_gain = "1";
                mclk_multiplier = "4";

pix_clk_hz = “118800000”;
//pix_clk_hz = “72000000”;
//serdes_pix_clk_hz = “200000000”;

csi_pixel_bit_depth = “10”;
mode_type = “bayer”;
pixel_phase = “rggb”;
lane_polarity = “0”;
gain_factor = “10”;
framerate_factor = “1000000”;
exposure_factor = “1000000”;

// min_gain_val = “10”; // 1.00x
min_gain_val = “0”; // 1.00x
max_gain_val = “480”; // 48x
step_gain_val = “1”;
default_gain = “10”; //1 1.00x

                min_hdr_ratio = "1";
                max_hdr_ratio = "1";

                min_framerate = "10000000";

// i2c@0{

                max_framerate = "120000000";
                step_framerate = "1";
                default_framerate = "120000000";

                //min_exp_time = "29"; // us 
                min_exp_time = "0"; // us

// max_exp_time = “16518”; // us
max_exp_time = “1048575”; // us
step_exp_time = “1”;
default_exp_time = “29”; // us

                embedded_metadata_height = "0";//"2";
            };//mode0

            ports {
                #address-cells = <1>;
                #size-cells = <0>;
                port@0 {
                    reg = <0>;
                    ds90ub95x_i2c_out0: endpoint {
                        port-index = <2>;
                        bus-width = <1>;
                       remote-endpoint = <&ds90ub95x_i2c_csi_in0>;
                    };
                };
            //};

            };
        };//rbpcv2_imx296_f

//---------------------------------------------------------------------------------

};

};//bus@0

tegra-camera-platform {
  //compatible = "nvidia,tegra-camera-platform";
    compatible = "nvidia, tegra-camera-platform";

// status = “okay”;
num_csi_lanes = <1>; // Changed 2 → 4
max_lane_speed = <1500000>;
min_bits_per_pixel = <10>; //<16>;
vi_peak_byte_per_pixel = <2>;
vi_bw_margin_pct = <25>;
max_pixel_rate = <240000>;
// max_pixel_rate = <480000>; //<430000>;
isp_peak_byte_per_pixel = <5>;
// isp_peak_byte_per_pixel = <2>;
isp_bw_margin_pct = <25>;

	/**
	* The general guideline for naming badge_info contains 3 parts, and is as follows,
	* The first part is the camera_board_id for the module; if the module is in a FFD
	* platform, then use the platform name for this part.
    * The second part contains the position of the module, ex. rear or front.
	* The third part contains the last 6 characters of a part number which is found
	* in the module's specsheet from the vender.
	*/
	modules {
        module0 {
            status = "okay";
            badge = "porg_rear_RBPCV2";
            position = "rear";
            orientation = "1";
            drivernode0 {
                status = "okay";
                pcl_id = "v4l2_sensor";

// devname = “imx477 30-0010”
// devname = “imx296 2-001a”;
// proc-device-tree = “/proc/device-tree/i2c@7000c500/rbpcv2_imx296_f@1a”;
sysfs-device-tree = “/sys/firmware/devicetree/base/bus@0/i2c@3160000/rbpcv2_imx296_f@1a”;
};
};
};
};
};

We use Ser/Des for one channel, so virtual channels are not needed at this stage.
I’d like to point out again that we have two other identical solutions based on the IMX296 monochrome and IMX296 color sensors.
They are fully functional on all platforms.

The only issue with the IMX297 is that it differs from the IMX296 only in resolution.

11

The trace log tell the short frame that tell the output size less than expected.

vi_tstamp:145236571296 data:0x0000000000000000
     kworker/2:4-981     [002] .......   121.087716: rtcpu_vinotify_event: tstamp:4538681119 cch:0 vi:1 tag:CHANSEL_FAULT channel:0x23 frame:1 vi_tstamp:145236580768 data:0x00000000021c41c0
     kworker/2:4-981     [002] .......   121.087716: rtcpu_vinotify_event: tstamp:4538902438 cch:0 vi:1 tag:FE channel:0x00 frame:1 vi_tstamp:145244692864 data:0x0000000100000022
     kworker/2:4-981     [002] .......   121.087716: rtcpu_vinotify_event: tstamp:4538902574 cch:0 vi:1 tag:CHANSEL_SHORT_FRAME channel:0x04 frame:1 vi_tstamp:145244692864 data:0x0002200000000000