Embedded Data from MIPI in Xavier NX

sebin.thankachan · December 21, 2023, 7:21am

Hi,

This is my previous query related to same topic.

Here, I got output while applying the below settings.

mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_d";
				discontinuous_clk = "no";
				dpcm_enable = "false";
				cil_settletime = "0";
				dynamic_pixel_bit_depth = "16";
				csi_pixel_bit_depth = "16";
				mode_type = "yuv";
				pixel_phase = "yuyv";

				active_w = "1920";
				active_h = "1081";
				readout_orientation = "0";
				line_length = "2468";

				embedded_metadata_height = "0";

But as per the other forum solutions, to extract embedded data we need to set embedded_metadata_height = “1”. But while applying this, I’m facing some stuck in streaming.

Please check the output while I had set the height as 1081 and embedded data as 0. How to extract the embedded data from the above 1920*1081 buffer or Is there any files which I can take as reference?. Also, Is there any issues in the ISP binary which I’m using?

testcamembedded.zip (2.3 MB)

sebin.thankachan · December 22, 2023, 6:58am

@ShaneCCC Can you please check this issue?.

ShaneCCC · December 25, 2023, 8:20am

The embedded data line shouldn’t include to the height. Just add correct line for the embedded_metadata_height and keep report the output exclude the embedded data line.

sebin.thankachan · December 28, 2023, 12:31pm

Hi @ShaneCCC

I had applied the below modifications in device tree as well as in the driver.

DEVICE TREE

vi@15c10000 {
	num-channels = <1>;
	ports {
			port@0 {
			reg = <0>;
			ox03a10_vi_in2: endpoint {
					port-index = <3>;
					bus-width = <2>;
					remote-endpoint = <&ox03a10_csi_out2>;
			};
		};
	};
};

nvcsi@15a00000 {
	num-channels = <1>;
	channel@0 {
		reg = <0>;
		ports {
			port@0 {
				reg = <0>;
				ox03a10_csi_in2: endpoint@0 {
					port-index = <3>;
					bus-width = <2>;
					remote-endpoint = <&ox03a10_ox03a10_out2>;
				};
			};
			port@1 {
				reg = <1>;
				ox03a10_csi_out2: endpoint@1 {
					remote-endpoint = <&ox03a10_vi_in2>;
				};
			};
		};
	};
};

ox03a10_d@1c {
	compatible = "nvidia,ox03a10";
	reg = <0x1c>;
	devnode = "video2";
	physical_w = "15.0";
	physical_h = "12.5";
	sensor_model ="ox03a10";
	post_crop_frame_drop = "0";
	use_decibel_gain = "true";
	delayed_gain = "true";
	use_sensor_mode_id = "true";
	sensor_slave_mode = "true";
	mode0 {
		mclk_khz = "24000";
		num_lanes = "2";
		tegra_sinterface = "serial_d";
		discontinuous_clk = "no";
		dpcm_enable = "false";
		cil_settletime = "0";
		dynamic_pixel_bit_depth = "16";
		csi_pixel_bit_depth = "16";
		mode_type = "yuv";
		pixel_phase = "yuyv";
		active_w = "1920";
		active_h = "1080";
		readout_orientation = "0";
		line_length = "2468";
		inherent_gain = "1";
		pix_clk_hz = "209681280";
		serdes_pix_clk_hz = "833333333";
		gain_factor = "16";
		min_gain_val = "16"; /* 0dB */
		max_gain_val = "511"; /* 48dB */
		step_gain_val = "1"; /* 0.1 */
		default_gain = "16";
		min_hdr_ratio = "1";
		max_hdr_ratio = "1";
		framerate_factor = "1000000";
		min_framerate = "20000000"; /* 20fps */
		max_framerate = "20000000"; /* 20fps */
		step_framerate = "1";
		default_framerate = "20000000";
		exposure_factor = "1000000";
		min_exp_time = "12"; /* us */
		max_exp_time = "50000"; /* us */
		step_exp_time = "1";
		default_exp_time = "1500";/* us */
		embedded_metadata_height = "1";
	};
	ports {
		port@0 {
			reg = <0>;
			ox03a10_ox03a10_out2: endpoint {
				port-index = <3>;
				bus-width = <2>;
				remote-endpoint = <&ox03a10_csi_in2>;
			};
		};
	};
};

DRIVER FILE

/*
 * WARNING: frmfmt ordering need to match mode definition in
 * device tree!
 */
static const struct camera_common_frmfmt ox03a10_frmfmt[] = {
	{{1920, 1080}, ox03a10_20fps, 1, 0, OX03A10_MODE_1920X1080_20FPS},
	/* Add modes with no device tree support after below */
};

While executing the below pipeline in order to get streaming, I’m getting a stuck in the streaming.

gst-launch-1.0 v4l2src -v device=/dev/video2 ! identity silent=FALSE ! videoconvert ! xvimagesink

Output Streaming Video :
stuck_streaming_embedded.zip (16.3 MB)

I had also collected trace from another terminal as mentioned here.
Trace Logs :
errlog_embedded_data.txt (1.7 MB)

Can you please check this issue and correct me if I’m doing anything wrong.

ShaneCCC · January 2, 2024, 5:51am

Looks the embedded data type incorrect. The embedded data type should be 0x12

 kworker/1:3-1629  [001] ....  6273.002251: rtcpu_vinotify_event: tstamp:196452047629 tag:CHANSEL_FAULT channel:0x23 frame:0 vi_tstamp:196451589255 data:0x04380040

sebin.thankachan · January 2, 2024, 6:35am

Hi @ShaneCCC

Can you suggest me what might be the reason for the wrong embedded data type?. We’re getting the stream from an external ISP. Should we need to alter the ISP Image in order to resolve this issue?

ShaneCCC · January 2, 2024, 8:47am

You may need consult with sensor vendor to confirm the embedded data type. Current Jetson device only support 0x12 embedded data type only.

Thanks

sebin.thankachan · January 3, 2024, 5:28am

Hi @ShaneCCC

Thanks for the reply.

Can you help me to figure out what’s the current value of embedded data type from the trace what I shared.

Also, to make it clear the sensor vendor asked about what does this Embedded Data Type=0x12 means?. Is that the Generic 8 bit long packet data types mentioned in the below document?(page 73)

ShaneCCC · January 3, 2024, 5:32am

Yes, it’s

sebin.thankachan · January 3, 2024, 4:59pm

Hi @ShaneCCC

I checked with the sensor vendor and modified the ISP binary. Right now the data_type of embedded data is 0x12.

Also, I modified the device tree parameters like this.

active_w = "1920";
active_h = "1080";
embedded_metadata_height = "1";

Also, changed the driver parameters to

static const struct camera_common_frmfmt ox03a10_frmfmt[] = {
	{{1920, 1080}, ox03a10_20fps, 1, 0, OX03A10_MODE_1920X1080_20FPS},
	/* Add modes with no device tree support after below */
};

In this scenario, I’m facing similar stuck issue again, but with some irregularities in the stream.
Streaming Error : stuck_streaming_embedded_new-2024-01-03_17.09.39.zip (20.6 MB)
Trace Logs : tracelog_newbin_447.txt (3.6 MB)

Can you guide me to debug on this issue?

ShaneCCC · January 4, 2024, 5:46am

Make sure how many lines of embedded data output.

sebin.thankachan · January 4, 2024, 6:40am

Hi @ShaneCCC

Sure, I’ll recheck with the vendor on the same.

I also wanted to add one more observation. When I tried to set embedded_data_height as 0, I’m getting streaming without any stuck. But still some irregularities in the streaming.

active_w = "1920";
active_h = "1080";
embedded_metadata_height = "0";

Driver End

static const struct camera_common_frmfmt ox03a10_frmfmt[] = {
	{{1920, 1080}, ox03a10_20fps, 1, 0, OX03A10_MODE_1920X1080_20FPS},
	/* Add modes with no device tree support after below */
};

Streaming Error : stuck_streaming_embedded_new_new-2024-01-03_22.53.41.zip (18.9 MB)
Trace Logs : tracelog.txt (1.4 MB)

Can you please check whether it’s an issue with Image Size or Embedded Data Size?

ShaneCCC · January 4, 2024, 7:11am

The CHANSEL_NOMATCH could be incorrect embedded data lines cause it.

kworker/0:2-8348 [000] … 372.957563: rtcpu_vinotify_event: tstamp:12079508852 tag:CHANSEL_NOMATCH channel:0x08 frame:0 vi_tstamp:12079133832 data:0x000003c9

sebin.thankachan · January 4, 2024, 10:30am

Hi @ShaneCCC

The datatype expected for Embedded Data is 0x12 and for Image data it needs to be 0x1E If I’m planning to collect Embedded data and Image Data(YUV422 8-bit) simultaneously, right?

system · February 13, 2024, 9:17am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Embedded Meta Data from MIPI CSI2 in Xavier NX Jetson Xavier NX camera	4	442	November 24, 2023
Problem with embedded data Jetson TX2	4	1078	October 18, 2021
Capture Image with meta data from IMX566 on TX2 NX Jetson TX2 camera	8	600	February 21, 2023
Trace log mistakes mipi Jetson Xavier NX camera	4	860	January 19, 2023
Problem with embedded metadata Jetson Xavier NX camera , kernel	4	579	May 4, 2022
Metadata Capture - TX2 Jetson TX2	14	1925	October 18, 2021
Get sensor embedded data in Xavier Jetson AGX Xavier camera , nvbugs	17	1194	June 2, 2022
Inotify_event: tag:CHANSEL_NOMATCH channel:0x01 Jetson AGX Xavier camera , device-tree , vi	17	1414	August 4, 2023
Xavier NX support of 4 MIPI CSI camera Jetson Xavier NX camera	8	1277	May 4, 2022
Embedded data causes CHANSEL_NOMATCH? Jetson AGX Xavier	9	866	August 30, 2023

Embedded Data from MIPI in Xavier NX

Related topics