Jetson Nano MIPI CSI-2 without I2C from FPGA

hello @JerryChang ,
we did a lot of things !

We found a problem in the IP of the FPGA, and corrected it !
Now we can receive some frame with a correct frame rate and create a .raw file of the capture. We can convert it to observe the actual image sent and it is correct !
fabiengray

Here is the capture’s trace :

[  237.912666] Allumage de l'ati...
[  237.912669] Ati powered on
[  237.912718] debut du steam
[  237.912724] vi 54080000.vi: Calibrate csi port 0
[  237.912846] arret du stream
[  237.912850] Extinction de l'ati...
[  237.912870] Ati powered off
[  237.912932] Allumage de l'ati...
[  237.912934] Ati powered on
[  237.915280] vi 54080000.vi: cil_settingtime was autocalculated
[  237.915286] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[  237.915315] debut du steam
[  237.915405] pb of csi ports in single thread
[  237.915410] 0
[  237.922360] pb of csi ports in single thread
[  237.922365] 0
[  237.929169] pb of csi ports in single thread
[  237.929172] 0
[  237.936315] pb of csi ports in single thread
[  237.936318] 0
[  237.942889] pb of csi ports in single thread
[  237.942893] 0
[  237.949723] pb of csi ports in single thread
[  237.949727] 0
[  237.956539] pb of csi ports in single thread
[  237.956543] 0
[  237.963396] pb of csi ports in single thread
[  237.963399] 0
[  237.970263] pb of csi ports in single thread
[  237.970268] 0
[  237.977139] pb of csi ports in single thread
[  237.977144] 0
[  237.983985] pb of csi ports in single thread
[  237.983988] 0
[  237.990892] pb of csi ports in single thread
[  237.990896] 0
[  237.997753] pb of csi ports in single thread
[  237.997757] 0
[  238.004564] pb of csi ports in single thread
[  238.004569] 0
[  238.011411] pb of csi ports in single thread
[  238.011414] 0
[  238.018271] pb of csi ports in single thread
[  238.018275] 0
[  238.025138] pb of csi ports in single thread
[  238.025143] 0
[  238.031989] pb of csi ports in single thread
[  238.031994] 0
[  238.038866] pb of csi ports in single thread
[  238.038874] 0
[  238.045760] pb of csi ports in single thread
[  238.045765] 0
[  238.052661] pb of csi ports in single thread
[  238.052664] 0
[  238.059485] pb of csi ports in single thread
[  238.059489] 0
[  238.066279] pb of csi ports in single thread
[  238.066283] 0
[  238.073153] pb of csi ports in single thread
[  238.073158] 0
[  238.080054] pb of csi ports in single thread
[  238.080058] 0
[  238.086909] pb of csi ports in single thread
[  238.086913] 0
[  238.093743] pb of csi ports in single thread
[  238.093748] 0
[  238.100594] pb of csi ports in single thread
[  238.100597] 0
[  238.107508] pb of csi ports in single thread
[  238.107513] 0
[  238.114354] pb of csi ports in single thread
[  238.114358] 0
[  238.121154] pb of csi ports in single thread
[  238.121158] 0
[  238.128129] pb of csi ports in single thread
[  238.128135] 0
[  238.134871] pb of csi ports in single thread
[  238.134876] 0
[  238.141812] pb of csi ports in single thread
[  238.141818] 0
[  238.148666] pb of csi ports in single thread
[  238.148671] 0
[  238.155606] pb of csi ports in single thread
[  238.155610] 0
[  238.162352] pb of csi ports in single thread
[  238.162357] 0
[  238.169157] pb of csi ports in single thread
[  238.169160] 0
[  238.176048] pb of csi ports in single thread
[  238.176053] 0
[  238.182973] pb of csi ports in single thread
[  238.182978] 0
[  238.189804] pb of csi ports in single thread
[  238.189811] 0
[  238.196632] pb of csi ports in single thread
[  238.196700] 0
[  238.203466] pb of csi ports in single thread
[  238.203472] 0
[  238.210389] pb of csi ports in single thread
[  238.210395] 0
[  238.217256] pb of csi ports in single thread
[  238.217260] 0
[  238.224166] pb of csi ports in single thread
[  238.224173] 0
[  238.230972] pb of csi ports in single thread
[  238.230976] 0
[  238.237848] pb of csi ports in single thread
[  238.237854] 0
[  238.244624] pb of csi ports in single thread
[  238.244629] 0
[  238.251544] pb of csi ports in single thread
[  238.251548] 0
[  238.258348] pb of csi ports in single thread
[  238.258355] 0
[  238.265192] pb of csi ports in single thread
[  238.265198] 0
[  238.272051] pb of csi ports in single thread
[  238.272055] 0
[  238.278921] pb of csi ports in single thread
[  238.278926] 0
[  238.285786] pb of csi ports in single thread
[  238.285791] 0
[  238.292642] pb of csi ports in single thread
[  238.292648] 0
[  238.299494] pb of csi ports in single thread
[  238.299501] 0
[  238.306336] pb of csi ports in single thread
[  238.306339] 0
[  238.313220] pb of csi ports in single thread
[  238.313226] 0
[  238.320057] pb of csi ports in single thread
[  238.320062] 0
[  238.327275] pb of csi ports in single thread
[  238.327282] 0
[  238.333777] pb of csi ports in single thread
[  238.333781] 0
[  238.340628] pb of csi ports in single thread
[  238.340631] 0
[  238.347565] pb of csi ports in single thread
[  238.347570] 0
[  238.354426] pb of csi ports in single thread
[  238.354431] 0
[  238.361250] pb of csi ports in single thread
[  238.361258] 0
[  238.368138] pb of csi ports in single thread
[  238.368144] 0
[  238.374931] pb of csi ports in single thread
[  238.374935] 0
[  238.381830] pb of csi ports in single thread
[  238.381834] 0
[  238.388648] pb of csi ports in single thread
[  238.388654] 0
[  238.395680] pb of csi ports in single thread
[  238.395688] 0
[  238.402348] pb of csi ports in single thread
[  238.402352] 0
[  238.409211] pb of csi ports in single thread
[  238.409215] 0
[  238.416089] pb of csi ports in single thread
[  238.416095] 0
[  238.422999] pb of csi ports in single thread
[  238.423004] 0
[  238.429875] pb of csi ports in single thread
[  238.429880] 0
[  238.436676] pb of csi ports in single thread
[  238.436682] 0
[  238.443521] pb of csi ports in single thread
[  238.443528] 0
[  238.450454] pb of csi ports in single thread
[  238.450459] 0
[  238.457308] pb of csi ports in single thread
[  238.457313] 0
[  238.464173] pb of csi ports in single thread
[  238.464182] 0
[  238.470955] pb of csi ports in single thread
[  238.470991] 0
[  238.477814] pb of csi ports in single thread
[  238.477820] 0
[  238.484671] pb of csi ports in single thread
[  238.484675] 0
[  238.491539] pb of csi ports in single thread
[  238.491546] 0
[  238.498459] pb of csi ports in single thread
[  238.498465] 0
[  238.505237] pb of csi ports in single thread
[  238.505241] 0
[  238.512109] pb of csi ports in single thread
[  238.512115] 0
[  238.519038] pb of csi ports in single thread
[  238.519044] 0
[  238.525887] pb of csi ports in single thread
[  238.525892] 0
[  238.532723] pb of csi ports in single thread
[  238.532728] 0
[  238.539527] pb of csi ports in single thread
[  238.539531] 0
[  238.546398] pb of csi ports in single thread
[  238.546404] 0
[  238.553247] pb of csi ports in single thread
[  238.553251] 0
[  238.560132] pb of csi ports in single thread
[  238.560136] 0
[  238.567054] pb of csi ports in single thread
[  238.567060] 0
[  238.573811] pb of csi ports in single thread
[  238.573815] 0
[  238.580684] pb of csi ports in single thread
[  238.580689] 0
[  238.587534] pb of csi ports in single thread
[  238.587538] 0
[  238.594482] pb of csi ports in single thread
[  238.594487] 0
[  238.601296] pb of csi ports in single thread
[  238.601303] 0
[  238.608107] pb of csi ports in single thread
[  238.608112] 0
[  238.614980] pb of csi ports in single thread
[  238.614986] 0
[  238.621962] pb of csi ports in single thread
[  238.621967] 0
[  238.628802] pb of csi ports in single thread
[  238.628806] 0
[  238.635571] pb of csi ports in single thread
[  238.635578] 0
[  238.642401] pb of csi ports in single thread
[  238.642405] 0
[  238.649260] pb of csi ports in single thread
[  238.649264] 0
[  238.656253] pb of csi ports in single thread
[  238.656258] 0
[  238.663117] pb of csi ports in single thread
[  238.663121] 0
[  238.669909] pb of csi ports in single thread
[  238.669914] 0
[  238.676695] pb of csi ports in single thread
[  238.676699] 0
[  238.683566] pb of csi ports in single thread
[  238.683571] 0
[  238.690496] pb of csi ports in single thread
[  238.690500] 0
[  238.697302] pb of csi ports in single thread
[  238.697308] 0
[  238.704167] pb of csi ports in single thread
[  238.704175] 0
[  238.711011] pb of csi ports in single thread
[  238.711019] 0
[  238.717853] pb of csi ports in single thread
[  238.717858] 0
[  238.724716] pb of csi ports in single thread
[  238.724722] 0
[  238.731656] pb of csi ports in single thread
[  238.731661] 0
[  238.738527] pb of csi ports in single thread
[  238.738534] 0
[  238.745300] pb of csi ports in single thread
[  238.745305] 0
[  238.752170] pb of csi ports in single thread
[  238.752176] 0
[  238.759003] pb of csi ports in single thread
[  238.759008] 0
[  238.765932] pb of csi ports in single thread
[  238.765937] 0
[  238.772781] pb of csi ports in single thread
[  238.772788] 0
[  238.779584] pb of csi ports in single thread
[  238.779589] 0
[  238.786443] pb of csi ports in single thread
[  238.786447] 0
[  238.793297] pb of csi ports in single thread
[  238.793303] 0
[  238.800159] pb of csi ports in single thread
[  238.800164] 0
[  238.807050] pb of csi ports in single thread
[  238.807056] 0
[  238.813863] pb of csi ports in single thread
[  238.813869] 0
[  238.820735] pb of csi ports in single thread
[  238.820742] 0
[  238.827610] pb of csi ports in single thread
[  238.827614] 0
[  238.834453] pb of csi ports in single thread
[  238.834458] 0
[  238.841447] pb of csi ports in single thread
[  238.841454] 0
[  238.848200] pb of csi ports in single thread
[  238.848204] 0
[  238.855038] pb of csi ports in single thread
[  238.855043] 0
[  238.861989] pb of csi ports in single thread
[  238.861993] 0
[  238.868822] pb of csi ports in single thread
[  238.868826] 0
[  238.875610] pb of csi ports in single thread
[  238.875615] 0
[  238.882450] pb of csi ports in single thread
[  238.882454] 0
[  238.889334] pb of csi ports in single thread
[  238.889340] 0
[  238.896289] pb of csi ports in single thread
[  238.896293] 0
[  238.903143] pb of csi ports in single thread
[  238.903150] 0
[  238.910070] pb of csi ports in single thread
[  238.910352] 0
[  238.916747] pb of csi ports in single thread
[  238.916751] 0
[  238.923612] pb of csi ports in single thread
[  238.923618] 0
[  238.930542] pb of csi ports in single thread
[  238.930547] 0
[  238.937328] pb of csi ports in single thread
[  238.937334] 0
[  238.944266] pb of csi ports in single thread
[  238.944294] 0
[  238.951079] pb of csi ports in single thread
[  238.951084] 0
[  238.957978] pb of csi ports in single thread
[  238.957983] 0
[  238.958035] video4linux video0: Syncpoint already enabled at capture done!0
[  238.964798] capture done error, single_thread
[  238.964803] 0
[  238.964841] arret du stream
[  238.966393] Extinction de l'ati...
[  238.966448] Ati powered off

Now we are stuck at the step of capturing the video stream…we tried to use gstreamer :

pi21@pi21-desktop:~/Téléchargements$ gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, format=GRAY8, width=320, height=240, framerate=147/1' ! xvimagesink -ev
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

And what we get using dmesg --follow :

[  260.263408] ati320 open
[  260.265727] Allumage de l'ati...
[  260.265731] Ati powered on
[  260.265755] debut du steam
[  260.265762] vi 54080000.vi: Calibrate csi port 0
[  260.265886] arret du stream
[  260.265891] Extinction de l'ati...
[  260.265914] Ati powered off
[  260.265978] Allumage de l'ati...
[  260.265979] Ati powered on
[  260.266019] Extinction de l'ati...
[  260.266041] Ati powered off
[  260.551334] ati320 open
[  260.551626] Allumage de l'ati...
[  260.551628] Ati powered on
[  260.551662] Extinction de l'ati...
[  260.551719] Ati powered off
[  260.571136] Allumage de l'ati...
[  260.571140] Ati powered on
[  260.622162] Extinction de l'ati...
[  260.622203] Ati powered off

Do you have any suggestions ?
Thank you by advance
Adrien

hello adrien.leroy2,

cool, it’s a good news.
assume you had already verify the streaming by v4l2 standard controls, and the frame-rate report correctly.

please check the sensor capability via v4l2, i.e. $ v4l2-ctl -d /dev/video0 --list-formats-ext
are you also complete the implementation in VI driver to add GRAY8 supports?
thanks

Hello @JerryChang

Yes, I verified it and we can see the correct framerate :

sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=320,height=240,pixelformat=GREY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=450
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 146.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 146.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 146.00 fps
<<<<<<<<<<<

Here is the result of the command v4l2-ctl -d /dev/video0 --list-formats-ext :

ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'GREY'
        Name        : 8-bit Greyscale
                Size: Discrete 320x240
                        Interval: Discrete 0.007s (147.000 fps)

In my opinion, I well implemented the GRAY8 support in VI driver. But maybe that I made a mistake, here the modifications I did :

vi2_formats.h :

static const struct tegra_video_format vi2_video_formats[] = {
        /* RAW 6: TODO */

        /* RAW 7: TODO */

        /* GRAYSCALE */
        TEGRA_VIDEO_FORMAT(RAW8, 8, Y8_1X8, 1, 1, T_L8, RAW8, GREY, "GRAY8"),

camera_common.c :

static const struct camera_common_colorfmt camera_common_color_fmts[]
        {        
                MEDIA_BUS_FMT_Y8_1X8, //GRAYSCALE 8 BITS
                V4L2_COLORSPACE_RAW, //RAW
                V4L2_PIX_FMT_GREY,
        },

sensor_common.c :

static int extract_pixel_format
//grayscale format
        else if (strncmp(pixel_t, "gray", size) == 0)
                *format = V4L2_PIX_FMT_GREY;

I saw in the dtsi files that there are a lot af fields status = "disabled";. Maybe shall I have to put them in "okay". What do think about it ?

tegra210-porg-camera.dtsi
/*
 * Copyright (c) 2018-2019, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */
#include "tegra210-porg-camera-ati320.dtsi"

/ {
        tegra-camera-platform {
                compatible = "nvidia, tegra-camera-platform";

                modules {
                        cam_module0: module0 {
                                status = "disabled";
                                cam_module0_drivernode0: drivernode0 {
                                        status = "disabled";
                                };
                                cam_module0_drivernode1: drivernode1 {
                                        status = "disabled";
                                        pcl_id = "v4l2_lens";
                                };
                        };
                        cam_module1: module1 {
                                status = "disabled";
                                cam_module1_drivernode0: drivernode0 {
                                        status = "disabled";
                                };
                                cam_module1_drivernode1: drivernode1 {
                                        status = "disabled";
                                        pcl_id = "v4l2_lens";
                                };
                        };
                };
        };

        /* all vi and csi ports are disabled by default */
        host1x {
                vi_base {
                        ports {
                                vi_port0 {
                                        status = "disabled";
                                        vi_in0: endpoint {
                                                status = "disabled";
                                        };
                                };
                                vi_port1 {
                                        status = "disabled";
                                        vi_in1: endpoint {
                                                status = "disabled";
                                        };
                                };
                        };
                };
                csi_base{
                        csi_chan0 {
                                status = "disabled";
                                ports {
                                        csi_chan0_port0 {
                                                status = "disabled";
                                                csi_in0: endpoint@0 {
                                                        status = "disabled";
                                                };
                                        };
                                        csi_chan0_port1 {
                                                status = "disabled";
                                                csi_out0: endpoint@1 {
                                                        status = "disabled";
                                                };
                                        };
                                };
                        };
                        csi_chan1 {
                                status = "disabled";
                                ports {
                                        csi_chan1_port0 {
                                                status = "disabled";
                                                csi_in1: endpoint@2 {
                                                        status = "disabled";
                                                };
                                        };
                                        csi_chan1_port1 {
                                                status = "disabled";
                                                csi_out1: endpoint@3 {
                                                        status = "disabled";
                                                };
                                        };
                                };
                        };
                };
        };

};

Thanks.

Adrien

hello adrien.leroy2,

yes, please revise the device tree definition to configure status = "okay" for the port bindings,

I don’t have grayscale sensor to validate the capture path on Nano,
according to below topic, there’s forum developer has RAW8 camera sensor (i.e. RGGB8), and they had confirmed it works with v4l2src.
i.e. RAW8 capture settings for nvcamerasrc.
so, how about have a try to report the video format instead of grayscale?

Hi @JerryChang

I already did a thing like that :

tegra210-porg-camera-ati320.dtsi

#include "tegra210-camera-ati320.dtsi"

#define CAM1_PWDN                TEGRA_GPIO(S, 7)

/ {
    host1x {
                i2c@546c0000 {
                        ati320_a@13 {
                                status = "okay";
                                reset-gpios = <&gpio CAM1_PWDN GPIO_ACTIVE_HIGH>;
                        };
                };
        };

    gpio@6000d000 {
                camera-control-output-low {
                        gpio-hog;
                        output-low;
                        gpios = < CAM1_PWDN 0 >;
                        label = "cam1-pwdn";
                };
        };
};

Is it what you would liked me to do ?

Yes we can try it.

Thanks

Adrien

yes, please further have a try to report the video format as default supported RAW8 instead of grayscale. thanks

@JerryChang

I don’t understand well what I have to do. I switched from grayscale to rggb8 in the dtsi but what are next steps ?

Thanks

Adrien

hello adrien.leroy2,

it my assumption to have several approaches to narrow down this,
for example,

  1. could fakesink workable to test the sensor streaming? for example, $ gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, format=GRAY8, width=320, height=240, framerate=147/1' ! fakesink

  2. if fakesink works, it’s due to xvimagesink it supports YUV format types, you may include conversion to change the color formats to adapt the output buffer.

  3. here’s my another assumption to cheat driver side as a RAW8 camera sensor (i.e. RGGB8), for testing the camera streaming.

hello Jerrychang,
I am gonna try the gstreamer command as soon as I can !
But concerning your 3rd assumption, didn’t you forget something ?

Thank you !
Adrien

Hello @JerryChang !
I got some good news !

I tested with your command (which works) but I wanted an ouput so I had to use xvimagesink…

Here is the command : gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-bayer, format=rggb, width=320,height=240' ! bayer2rgb ! videoconvert ! xvimagesink

Thanks again !
Adrien

hello adrien.leroy2,

let’s have summarize and close this topic, initial anther new topic if you need further supports.
so, it is confirmed you’re able to see video stream by including video converter?

Yes It is confirmed, and it is also working in GRAY8 mode with this command :

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, format=GRAY8, width=320,height=240' ! glimagesink

Thanks

Adrien

1 Like