Using 6 Raspberry Pi cameras with Jetson Xavier NX

[majtanm]

Hey,
I was just wondering whether anyone managed to make 6 Raspberry Pi v2 cameras work with the new Jetson Xavier NX. I have the production module (as the devkit wasn’t available when we were ordering it) and I have a custom board made by following Jetson_Xavier_NX_Product_Design_Guide_v1.1.pdf document provided by Nvidia … however I am struggling with setting up the device tree for the board as we are using TCA9548AMRGER in order to control I2C switching for the Camera I2C. However the original device tree for the board uses some I2C switch that uses GPIO to control the output. The TCA9548AMRGER uses I2C registers that will switch between the outputs. Please let me know whether it is possible to make it work and if not I would appreciate a suggestion for an I2C switch that could be used with the device trees and can support 6 outputs.

Thanks,
Matej

[ShaneCCC]

You can reference to tegra186-camera-e3322-a00.dtsi for i2c mux design.

[majtanm]

Hey Shane,
I have looked at the file you suggested and I think it helped me for the most part. However there is a small part that I am not sure about. In the beginning of the i2c mux block it says i2c@3180000 as I understand this is the i2c node with 3180000 being the address of the i2c bus. But as tegra186-camera-e3322-a00.dtsi is meant for TX2 and not for Xavier NX, I wondering what is the correct address for this i2c bus on Xavier NX and where can I find this information? I have tested the hardware and I get the correct i2c addresses (address of the i2c switch and the camera module) on i2c bus 2.

Thanks,
Matej

[ShaneCCC]

You can change the address follow the @ for Xavier. Like i2c@xxxx, vi@xxxx nvhost@xxxx

[majtanm]

Hey Shane,
So I have tried to use i2c@3180000 as the node for i2c and it doesn’t seem to work. I found this value in tegra194-p2822-0000-camera-e3333-a00.dtsi. I also ran i2cdetect -l to check for the address and it does seem to be correct, if you look at the bus i2c-2:

Also when I scan through the i2c-2 bus I get the address 0x70 which is the address of the TCA9548 i2c switch. I can also send 0x01 to that address and that way enable the switch to use one of the buses as shown on the picture after sending the command addresses 0x10 and 0x64 are available (these belong to the Raspberry Pi v2 camera).

However my dmesg says that there was a failure at creating the device:

So I am wondering whether I am doing something wrong when configuring the device tree or whether it could be a hardware problem. In my opinion it doesn’t seem like hardware issue though.

Thanks,
Matej

[ShaneCCC]

Did you modify below to 70?

                tca9548_77: tca9548@77 {
                        compatible = "nxp,pca9548";
                        reg = <0x77>;
                        #address-cells = <1>;
                        #size-cells = <0>;
                        vcc-supply = <&vdd_1v8_cvb>;
                        skip_mux_detect;
                        force_bus_start = <CAMERA_I2C_MUX_BUS(0)>;
                        i2c@0 {
                                reg = <0>;
                                i2c-mux,deselect-on-exit;
                                #address-cells = <1>;
                                #size-cells = <0>;

[majtanm]

Hey Shane,

Thanks, that helped. I missed that whole setup … now I get it to recognize the TCA9548 chip but I have problems with the Camera GPIO.

I have checked the physical connection on the PWDN pin as well as the MCLK pin for cam0 (I am testing now with only one camera) and it seems fine. I guess it could be the problem with the PWDN pin. When I checked it the pin was not triggered high, so I guess the camera was off.

This is my device tree:

#define CAM0_PWDN	TEGRA194_MAIN_GPIO(P, 4)
#define CAMERA_I2C_MUX_BUS(x) (0x1E + x)

/ {

    gpio@2200000 {
        camera-control-output-low {
            gpio-hog;
            output-low;
            gpios = <CAM0_PWDN 0>;
            label = "cam0-pwdn";
        };
    };

    i2c@3180000 {
        #address-cells = <1>;
        #size-cells = <0>;
        tca9548@70 {
            compatible = "nxp,pca9548";
            reg = <0x70>;
            #address-cells = <1>;
            #size-cells = <0>;
            vcc-supply = <&p3509_vdd_3v3_cvb>;
            skip_mux_detect;
            force_bus_start = <CAMERA_I2C_MUX_BUS(0)>;
            i2c@0 {
                reg = <0>;
                i2c-mux,deselect-on-exit;
                reset-gpios = <&tegra_main_gpio CAM0_PWDN GPIO_ACTIVE_HIGH>;
                #address-cells = <1>;
                #size-cells = <0>;
            };

I also checked the physical pin 114 corresponds to the CAM0_PWDN pin which is on port P pin 4, following the Pinmux documentation.

So I am not sure whether I am again missing some crucial configuration of the GPIO or what could be the problem. Is the PWDN pin even considered as reset-gpios?

It would be nice if you could guide me in the correct direction.

Thanks,
Matej

[fabian.solano]

Hello @majtanm,

I faced a similar problem in the past with a custom carrier board and the TCA9548 in TX2, however it might give you a clue to find the same in the Xavier NX. So, as far as I know for the first two cameras the GPIO definition for reset should work fine as (this is in TX2):

#define CAM0_RST_L  TEGRA_MAIN_GPIO(R, 5)
#define CAM1_RST_L  TEGRA_MAIN_GPIO(R, 1)

But for the other 4 cameras support I had to do the gpio-hog definition as follows:

            camera-c-rst{
                gpio-hog;
                output-low;
                gpios = <1 0>;
                label = "cam_c_rst";
            };

            camera-d-rst{
                gpio-hog;
                output-low;
                gpios = <3 0>;
                label = "cam_d_rst";

            };

            camera-e-rst{
                gpio-hog;
                output-low;
                gpios = <5 0>;
                label = "cam_e_rst";

            };

            camera-f-rst{
                gpio-hog;
                output-low;
                gpios = <7 0>;
                label = "cam_f_rst";

            };

Then you can define the pin as:

#define CAM2_RST_L  1
#define CAM3_RST_L  3
#define CAM4_RST_L  5
#define CAM5_RST_L  7

And use them in the device tree:

reset-gpios = <&gpio_i2c_0_74 CAM3_RST_L GPIO_ACTIVE_HIGH>;

Hope this helps, this kind of issues are annoying and hard to debug.

Regards,
Fabian
www.ridgerun.com

[ShaneCCC]

You need to add the reset-gpios depends on you HW connection. Have a reference to below dt.

./galen/kernel-dts/common/tegra194-p2822-0000-camera-e3333-a00.dtsi

[majtanm]

Hey @fabian.solano and @ShaneCCC,

Thank you for the help. I got it kind of working. However only one of my cameras works for now if I use the nvarguscamerasrc in gstreamer. That is the cam0. If I use the UVC device then I get:

And the console gets stuck without showing any image. My application doesn’t require the UVC drivers to work so the bigger issue is that if I for example try to get video from sensor 1 using the nvarguscamerasrc pipeline I get this:

And again the console freezes and doesn’t show anything. I am sure that the camera modules are ok as the one that worked on cam0 doesn’t work on cam1.

I am also attaching my device tree files:

https://gist.github.com/bambooSocks/41b432b01cf2ae42ab07c9d97178ca86

https://gist.github.com/bambooSocks/0379ca2b30d8ca2aac49cf49423ec66d

Hope you can help me again

Thanks,
Matej

[fabian.solano]

Hi @majtanm

Please test with the following pipeline just to verify correct video stream

gst-launch-1.0  nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv ! xvimagesink

Regards,
Fabian
www.ridgerun.com

[ShaneCCC]

Definitely not working for UVC because your sensor is Bayer sensor can’t output YUV without ISP.
Please have a try below to confirm if those two camera are working normally first.

v4l2-ctl --stream-mmap–set-ctrl bypass_mode=0 --stream-count=100 -d /dev/video0
v4l2-ctl --stream-mmap–set-ctrl bypass_mode=0 --stream-count=100 -d /dev/video1

[majtanm]

Hey @fabian.solano and @ShaneCCC

I have tried both of you suggestions

The output of the gst-launch-1.0 looks like:

and it seems to open a new window that is stuck without image.

When I tried to run the v4l2-ctl commands I got an output when using the video0 device and it got stuck and not print if I used video1 device. I tested it with the camera modules swapped and it worked on video0 in both cases and never in video1. The output when using video0 is:

Any suggestions what could be the problem?

Thanks,
Matej

[fabian.solano]

Hi @majtanm,

From the output of gst-launch-1.0 the results seems to be good, and ran without errors.

And the output from v4l2-ctl just confirmed that you are correctly receiving buffers from the camera.

The problem about video1 might be related to the reset-gpio, misconfiguration or something at the driver that is not enabling the camera therefore it does not send buffers.

Regards,
Fabian
www.ridgerun.com

[majtanm]

Hi @fabian.solano,

I am not at work anymore, but I will go through the configuration once again tomorrow and let you know if I find something.

Thanks,
Matej

[ShaneCCC]

You may need to probe the signal to compare to figure the problem.

[majtanm]

Hey,

Sorry for the delay, we have had some hardware issues that took a while to fix but now I can confirm that all the physical connection for CSI buses, reset pins as well as the I2C buses are ok. If I connect all six cameras to the board I get an error in dmesg with the last two cameras (I think this is connected to the reset_gpio as it doesn’t correctly reset and the I2C fails).

Moreover when I try to run gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv ! xvimagesink with sensor-id set to 0 then the video shows up and if I use any other sensor I get a gst-launch window with the desktop image instead of the actual video stream. I was wondering whether it has anything to do with the clocks because that is one of the last things missing, but after more research I found out that the Raspberry Camera v2 actually doesn’t have a clock input (at least the cable doesn’t have a pin dedicated for it). So I would like to ask whether there is anything else that I could be doing wrong.

Only change from last time in the device tree is :

#define CAM0_PWDN	TEGRA194_MAIN_GPIO(P, 4)
#define CAM1_PWDN	TEGRA194_MAIN_GPIO(P, 5)
#define CAM2_PWDN	TEGRA194_MAIN_GPIO(Q, 1)
#define CAM3_PWDN	TEGRA194_MAIN_GPIO(Q, 3)
#define CAM4_PWDN	TEGRA194_MAIN_GPIO(R, 0)
#define CAM5_PWDN	TEGRA194_MAIN_GPIO(Q, 2)
#define CAMERA_I2C_MUX_BUS(x) (0x1E + x)

Also @ShaneCCC what do you mean by probing the signal? I only did the continuity check and that is fine. Are you suggesting to hook it up to oscilloscope? In that case what do I expect to see.

With Regards,
Matej Majtan

[ShaneCCC]

You may need to have oscilloscope to confirm the power/rest pin are the correct status.

[majtanm]

Hey @ShaneCCC,

I have probed the reset pins in the case of cam0 up to cam3 the gpio went high when I started the camera stream, but only the cam0 had image. for cam4 and cam5 it seems like the pin goes high when booting the system but that seems like it’s not related to my configuration.

Reset GPIO connection for reference:

CAM0_PWDN connected to pin 114 (GPIO_PP.04)
CAM1_PWDN connected to pin 120 (GPIO_PP.05)
CAM2_PWDN connected to pin 212 (GPIO_PQ.01)
CAM3_PWDN connected to pin 124 (GPIO_PQ.03)
CAM4_PWDN connected to pin 206 (GPIO_PR.00)
CAM5_PWDN connected to pin 208 (GPIO_PQ.02)

Does this seem correct to you? The first four seem to work but I am not sure whether the last two are right. I tried to follow the Xavier NX pinmux file provided by Nvidia.

[ShaneCCC]

I think you may need HW guide to help on your design.