IMX477 behind TCA9548 on TX2-NX

[Buanderie]

Hello,
I’m currently trying to port our BSP from Jetson Nano (prod. module) to TX2-NX. We used the same custom carrier board. I’m currently facing difficulties adapting the DTS to support our imaging devices.
We use 3 IMX477 with the I2C muxed using a TCA9548.
As far as I understand, the codename for the TX2-NX hw is “lanai”, so I started to modify the DTS to match our carrier board.
I’m stuck here, as I can’t get the tca9548 to work, and thus the cameras…
The tca9548 can is succesfully detected using i2cdetect on i2c-2 ( i2c@3180000 ), with address 0x70:

i2cdetect -y -r 2
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: UU -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: 70 -- -- -- -- -- -- --   

But, I get this error at boot time:

[    1.846348] i2c i2c-2: of_i2c: modalias failure on /i2c@3180000/tca9548@70
[    1.853782] i2c i2c-2: Failed to create I2C device for /i2c@3180000/tca9548@70
[    5.000288] i2c /dev entries driver

So my assumption of the DTS being faulty seems correct. But I can’t figure out what I did wrong.
Any help would be very welcome.

Please find attached to this post the complete extracted DTS used by the system.

Thanks
extracted.dts (243.4 KB)

[ShaneCCC]

Try reference to tegra186-quill-camera-e3333-a00.dtsi to check the dts context.

[Buanderie]

Alright, after checking the reference DTS, I finally managed to get the I2C mux working. Problem was that I defined the I2C muxing parts inside host1x (as I did for the Jetson Nano version).

Problem is now… I can’t capture any frames using GStreamer and nvarguscamerasrc.
I tried running:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! "video/x-raw(memory:NVMM),width=4032,height=3040" ! fakesink dump=true

And got:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 4032 x 3040 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 11000, max 500000000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 354.000000; Exposure Range min 5000, max 352112000;

GST_ARGUS: 2028 x 1520 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 354.000000; Exposure Range min 5000, max 352112000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 4032 H = 3040 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: Done Success
Got EOS from element "pipeline0".
Execution ended after 0:00:04.001036864
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up

Please also find the dmesg output attached to this reply… (dmesg_output.txt)
dmesg_output.txt (138.1 KB)

Although, capturing raw frames using v4l2-ctl does work:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=4032,height=3040,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test_output.raw

I tried to look for similar issues on the forum, but can’t quite get useful insights.

[ShaneCCC]

Does the 1080p mode the same?
Have a check the port-index and tegra_sinterface are sync up
Also boost the clocks to try.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

[Buanderie]

Boosting the clocks got me the 1080p mode working (up to 60fps)

As a matter of fact, the 1080p mode was already working without modifications to the clocks (I rebooted and tested it as is and it worked)

No luck with the 4032x3040 mode, though.
The port-index and tegra_sinterface were left similar to those used on Jetson Nano (as stated in the Sensor Dev. Guide, the mapping should be similar, right ? → https://docs.nvidia.com/jetson/l4t/#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/camera_sensor_prog.47.1.html#wwpID0E0Z10HA

The mapping used is:

• CAM_0: port_index=0, bus_width=2, tegra_sinterface=serial_a
• CAM_1: port_index=4, bus_width=2, tegra_sinterface=serial_e
• CAM_2: port_index=2, bus_width=2, tegra_sinterface=serial_c

[ShaneCCC]

Yes, it’s the same with Nano.
How about 2029x1520?

[Buanderie]

How about 2029x1520?

No luck, either. Same error as with the full-sensor mode.

[ShaneCCC]

Sorry for typo it’s 2028 instead of 2029 due to unable support odd resolution.

[Buanderie]

I guessed it, yes… Not working either, anyway :)

[ShaneCCC]

If any mode able working that confirm the lane configure is correct.
Could you modify the sensor driver to make the set gain/exposure/frame rate function as dummy function to make sure the sensor REG configure the same with v4l2-ctl capture.

[Buanderie]

Hum…
Okay so, I after setting the driver functions as dummy, I still have the same problem.

It appears though, that the sequence of driver function calls are not the same between v4l2-ctl capture and Argus capture.

Please find attached the dmesg from v4l2-ctl capture
v4l2_capture.txt (1018 Bytes)
and Argus (Gstreamer)
argus_capture.txt (23.3 KB)
capture sequences.

[Buanderie]

Hum…
Maybe, you could review my dts to check for any errors in the 4K mode (maybe wrong pixel_clk_hz, line_length…) ?

[ShaneCCC]

If you boost the clock didn’t help you can except the pix_clk_hz cause the problem.
Also if modify all of control function as dummy the line_length also can’t be the root cause.

[Buanderie]

Okay…
But how could that be ? The pix_clk_hz is sensor/mode dependent, right ? Or am I mistaken ? If it was faulty it wouldn’t have worked on Nano in the first place.

[ShaneCCC]

The original imx477 driver have below sensor mode. Looks like it’s different with yours. Could you have a try it and reference to the dts configure from that.

nvidia@nvidia-desktop:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘RG10’
Name : 10-bit Bayer RGRG/GBGB
Size: Discrete 3840x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)

[Buanderie]

Hum… I believe I already tried the latest driver. I tested the 4K 16:9 resolution, as proposed, with no luck.

[ShaneCCC]

What I mean is the output resolution in default sensor driver like 3840x2160 or 1080p