Bypass check using FPGA (HDMI to CSI-2)

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hello,

I am getting a lot of help from the forum.
Thank you for always.

I want to receive input as follows using FPGA.
HDMI → FPGA → CSI-2 → NX

A message is displayed in the KERNEL with the following contents and is not bypassed.

[  324.861556] tegra194-vi5 15c10000.vi: no reply from camera processor
[  324.861723] tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms
[  324.861881] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel
[  324.864241] tegra194-vi5 15c10000.vi: err_rec: successfully reset the capture channel

My development environment is as follows.

  • Hardware Platform - Jetson Xavier NX devkit P3518
  • Module Info - NX EMMC
  • JetPack Version - 4.5
  • L4T - 32.5
  • HDMI resolution 1920x1080 30frame

And I modified the source with the following contents.

  • Write FPGA driver without I2C communication.
  • Writing DTS for FPGA.

And I checked the devnode.

nvidia@linux:~/log$ ls -al /dev/vi*
crw-rw----+ 1 root video 81, 0 Nov 30 08:33 /dev/video0

Bypass was performed as follows.

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1

The Kernel message is:
Behavior for bypass is included.
fpga_bypass_dmesg.log (74.6 KB)

I also checked the trace.

	echo 1 > /sys/kernel/debug/tracing/tracing_on
	echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
	echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
	echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
	echo 2 > /sys/kernel/debug/camrtc/log-level
	echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
	echo > /sys/kernel/debug/tracing/trace
	cat /sys/kernel/debug/tracing/trace

fpga_bypass_trace.log (32.2 MB)

When the bypass command is input, the frame does not flow.

May I know what is the problem?
Can I get help?
Thanks in advance for the reply.

3

Looks like the pix_clk_hz define in dts too smaller, have below command boost the clocks to try if working then modify the pix_clk_hz in dts

sudo su

echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
4

Thanks for the reply, ShaneCCC

I did as you said, same thing.

root@linux:/home/nvidia# echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
root@linux:/home/nvidia# echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
root@linux:/home/nvidia# echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
root@linux:/home/nvidia# cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
460800000
root@linux:/home/nvidia# cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
576000000
root@linux:/home/nvidia# cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
314000000

fpga_20211201_dmesg.log (72.7 KB)
fpga_20211201_trace.log (43.7 KB)

For reference, for a simple test, the FPGA is outputting a colorbar.
The CSI-2 related settings on the FPGA are:

  • Lane = 2
  • Ref Clock Frequency = 27MHz
  • active h = 1920
  • total h = 2200
  • active v = 1080
  • total v = 1125
  • front porch h = 100
  • synch h = 30
  • back porch h = 5
  • front porch v = 320
  • synch v = 200
  • frame = 30

The DTS for this is as follows:

/ {
	host1x {
		vi@15c10000  {
			num-channels = <1>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				vi_port0: port@0 {
					reg = <0>;
					rbpcv3_fpga_vi_in0: endpoint {
						status = "okay";
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&rbpcv3_fpga_csi_out0>;
					};
				};
			};
		};

		nvcsi@15a00000 {
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;
			csi_chan0: channel@0 {
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					csi_chan0_port0: port@0 {
						reg = <0>;
						status = "okay";
						rbpcv3_fpga_csi_in0: endpoint@0 {
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&rbpcv3_fpga_out0>;
						};
					};
					csi_chan0_port1: port@1 {
						reg = <1>;
						status = "okay";
						rbpcv3_fpga_csi_out0: endpoint@1 {
							status = "okay";
							remote-endpoint = <&rbpcv3_fpga_vi_in0>;
						};
					};
				};
			};
		};
	};

	cam_i2cmux {
		i2c_0:i2c@0 {
			fpga_cam0: rbpcv3_fpga_a@1a {
				compatible = "hdmi2csi,fpga";
				/* I2C device address */
				reg = <0x1a>;

				/* V4L2 device node location */
				devnode = "video0";

				status = "okay";

				/* Physical dimensions of sensor */
				physical_w = "3.674";
				physical_h = "2.738";

				sensor_model = "hdmi2csi_fpga";

				use_sensor_mode_id = "true";

				mode0 { /* MODE_1920X1080 */
					mclk_khz = "27000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					phy_mode = "DPHY";
					discontinuous_clk = "yes";
					dpcm_enable = "false";
					cil_settletime = "0";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "2025";
					inherent_gain = "1";
					// 85050000 / 27000000 = 3.15
					mclk_multiplier = "3.15";
					// (100+1920+5)x(320+1080)x30=2025x1400x30=
					pix_clk_hz = "85050000";

					gain_factor = "1000000";
					min_gain_val = "1000000";
					max_gain_val = "354000000";
					step_gain_val = "1";
					default_gain = "1000000";
					min_hdr_ratio = "1";
					max_hdr_ratio = "64";
					framerate_factor = "1000000";
					min_framerate = "1420000";
					max_framerate = "60000000";
					step_framerate = "1";
					default_framerate = "60000000";
					exposure_factor = "1000000";
					min_exp_time = "16";
					max_exp_time = "704225";
					step_exp_time = "1";
					default_exp_time = "8000"; /* us */
					embedded_metadata_height = "2";
				};

				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						rbpcv3_fpga_out0: endpoint {
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&rbpcv3_fpga_csi_in0>;
						};
					};
				};
			};
		};
	};
};
/ {
	tcp: tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		num_csi_lanes = <2>;
		max_lane_speed = <1500000>;
		min_bits_per_pixel = <10>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		max_pixel_rate = <240000>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		modules {
			cam_module0: module0 {
				badge = "jakku_front_RBP194";
				position = "front";
				orientation = "1";
				status = "okay";
				cam_module0_drivernode0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
					devname = "ktnc_fpga 9-001a";
					proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@0/rbpcv3_fpga_a@1a";
				};
			};
		};
	};
};

The formula to obtain the pix_clk_hz value and the mclk_multiplier value in the contents is as follows.

([front porch h] + [active h] + [back porch h]) x 30 frame

// (100+1920+5)x(320+1080)x30=2025x1400x30=
pix_clk_hz = 85050000;

pix_clk_hz / mclk_khz

// 85050000 / 27000000 = 3.15
mclk_multiplier = 3.15;

Is the calculation wrong?

5

The trace log get from the boosting clocks?
It’s still show err_intr_cil_data_lane_rxfifo_full_err0_a but a litter different with previous log.
If you have boost the clocks you can ignore the pix_clk_hz in device tree, Maybe enable test pattern to try.

    kworker/1:3-1749  [001] ....    71.414684: rtcpu_nvcsi_intr: tstamp:2792765467 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x00000010
6

Hello ShaneCCC
Thank you for your interest.

“The trace log get from the boosting clocks?”
A: Logs obtained after applying max_rate .

What is err_intr_cil_data_lane_rxfifo_full_err0_a?
It is not checked in my log.

In the mention
“Maybe enable test pattern to try.”

What kind of “test pattern” are there?

7

The err_intr_cil_data_lane_rxfifo_full_err0_a is the from TRM.
If FPGA able to configure output test pattern.

8

Hello ShaneCCC

I still don’t know err_intr_cil_data_lane_rxfifo_full_err0_a.
Does TRM mean TECHNICAL REFERENCE MANUAL?
Document “Xavier_TRM_DP09253002_v1.4p.pdf” is viewed, But couldn’t find err_intr_cil_data_lane_rxfifo_full_err0_a .

9

Search _intr_cil_data_lane_rxfifo_full_err0 by this strings

10

Hello ShaneCCC

I can’t find it in my log.
Can you give me a link about _intr_cil_data_lane_rxfifo_full_err0?

11

Check this REG NVCSI_PHY_0_CILA_INTR_0_STATUS_CILA_0 from TRM.
Below log is this REG status.

rtcpu_nvcsi_intr: tstamp:2792765467 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x00000010
13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.