Camera Sensor 1lane and v4l2src

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello,
we are developing a camera for jetson nano.
the camera runs with mipi 1lane (RAW).
With nvarguscamerasrc and gstreamer the camera works without a problem, but when we try to capture an image with v4l2src and gstreamer or with v4l2-ctl it does not work, we cannot capture an image.
If we use a mipi 2 lanes sensor board then all of them work (nvarguscamerasrc, v4l2src, v4l2-ctl).
why can’t we use v4l2-ctl for capturing images with a camera sensor in 1lane mode?

I assume that everything is set correctly in the device tree and sensor driver, otherwise it will not work with nvarguscamerasrc.

Best regards,
Osama

3

May be sensor initial timing cause it. Have some delay after i2c sensor mode table initial.

4

We make sure that everything is correctly initialized with enough delay.
We think that the video input is not set correctly for mipi 1lane and is always set as default in mipi 2lane if v4l2-ctl or v4l2src is used directly.
with nvidia isp pipeline everything works (nvarguscamerasrc)

5

You can print the port/lanes configure in the vi2_fops.c/csi2_fops.c to confirm it for VI mode(v4l2-ctl)

6

in fact it was always set to 2 lane.
I changed to 1lane directly in csi2_fops.c (hardcoded) and then it also worked with v4l2-ctl and v4l2src.
why are the number of lanes not taken from the device tree?

7

It’s could be some incorrect in the dts. Have print message to the parser dts in the driver to confirm it.

8

I did that and noticed that when I use nvidia isp “nvarguscamerasrc” with gstreamer the lane number is set correctly but when I use v4l2-ctl or v4l2src 2-lane is always set.
the device tree is correct but where is the lanes set when v4l3src is used?

Best regards.

9

v4l2 driver is different with nvarguscamerasrc. Not sure below link is from your colleague or not. You need to modify the VI driver for v4l2src to load the lane configure during running time.

11

Hey ba.osama, where in the csi2_fops.c did you hardcode to 1 lane? I’d like to try that as well.

12

I found the problem v4l2 drivers do not take the lane number from devicetree like the nvidia isp. nvidia isp uses the num_lane from the mode table, but v4l2 uses the device tree lane number setting “bus-width”.
you have to set this correctly in the devicetree so that it works.
we want one lane and two lane to support at the same time.
We have to use v4l2 because the nvidia ISP does not support 8bit and raw capture.
how can we fix it so that the lane settings of mode table (num_lane) are taken?

Best regards,
Osama

13

We are basically in the same boat, so have been trying to solve the same issue. Maybe we can work together to figure this out. ShaneCCC, ba.osama do not know each other, haha, but thanks for linking the questions together!

Let me dig deeper into the v4l2 side, I am uncertain where and how that code is using bus-width to do what it needs to do.

14

Looking at dts file, I see we have this info in multiple locations:

  1. vi

Is this where v4l2 is parsing? Here I see a bus-width setting

  1. nvcsi

Here I also see a bus-width = 2

I also see stuff under the i2c section, but to me, those settings would indicate things we’d send to the i2c communication and not deal with the Tegra CSI port specifically.

15

After lots of tracing tonight, I see that in the csi.c file, this is where we get the parsing of the port information from the dts file.

What I don’t understand is both the nvcsi and vi probes BOTH call “tegra_csi_media_controller_init”, which doesn’t make sense to me. I was under the impression that nvcsi is basically the driver for the “CSI Unit” and VI for the VI init that CSI feeds into. All I can think is that both units (CSI and VI) use the same type of CSI structure to match settings between both sides.

So I imagine if you hard coded the dt reading to override the lanes at this point, you were able to switch them, in the csi.c code.

What I haven’t been able to determine yet is how we can get access to the variables holding the CSI information that we need to modify in where I would assume the code in the driver the I am working with that is changing the lanes on the CSI transmitter chip. All the functions that I found so far are expecting structs inside of structs that we don’t have, so that doesn’t work.

Basically we want to access the “tegra_csi_device” structures we want to modify. And, know which port we are in for the driver calls we are getting.

In my case, this is for an embedded device, so if I had to modify csi.c that isn’t a problem, although I’d rather JUST modify the driver if possible.

16

Okay, progress

In the csi2_fops.c code, where we start streaming (csi2_start_streaming) I se the csi_lanes = 1 and this forced the capture to 1 lane.

In other drivers, normally in this streaming code it would make a call to “g_mbus_config” which is not happening here. In the Raspberry Pi version of this code, it is happening there, so I need to figure out how I can make that call to get the same setup happening (or at least, get a call into tc358743_num_csi_lanes_in_use to find out how many lanes. If we do that, I think we can get it dynamically switching.

EDIT

Success!

I am not sure if this code is done in the correct way, but my v4l calls and setting new timings are all working now.

In csi2_fops.c, I added (after TEST):

(beginning of function)
struct v4l2_mbus_config mbus_config = { 0 };
int ret;

(later in function)
csi_port = !chan->pg_mode ? port->csi_port : port->stream_id;
csi_lanes = port->lanes;
// - TEST
ret = v4l2_subdev_call(chan->sensor_sd,video,g_mbus_config,&mbus_config);

csi_lanes = (mbus_config.flags & V4L2_MBUS_CSI2_LANE_MASK) >> __ffs(V4L2_MBUS_CSI2_LANE_MASK);

I also added this define above the function:

#define V4L2_MBUS_CSI2_LANE_MASK (0xf << 10)

Over in tc358743, I added to make look like this:

#define V4L2_MBUS_CSI2_LANE_MASK (0xf << 10)

static int tc358743_g_mbus_config(struct v4l2_subdev *sd,
struct v4l2_mbus_config *cfg)
{
u32 mask = V4L2_MBUS_CSI2_LANE_MASK;
v4l2_info(sd, “Calling %s\n”, FUNCTION);

cfg->type = V4L2_MBUS_CSI2;
cfg->flags = (tc358743_num_csi_lanes_in_use(sd) << __ffs(mask)) & mask;

/* In DT mode, only report the number of active lanes */
if (sd->dev->of_node)
	return 0;

Finally, search for the following set (only happens in one place in the driver) and set to 374 to match the reference driver.

state->pdata.fifo_level = 374;

I’ve only tested 640x480 at the moment, but this should work for all single lane variants from what I can tell.

EDIT 2

Upon further inspection, I am getting empty frames in the buffer and sometimes there is tearing in frame 0.

Doing some reading in other threads, this has been seen when the line counts don’t match, between what the device is sending and what is being expected. Although I am not seeing any errors thrown in DMESG. So still digging deeper. Getting closer anyways.

640x480p = Only writes into buffer 0, all other buffers empty
720x480p = Only writes into buffer 0, all other buffers empty
800x600p = Works
1024x768 = Works

EDIT

Okay, found the problem. When the number of lines is 480 and the height is 525, it’s sending 1 less line for some reason. I hacked the code to subtract 1 whole line and it works now for 640x480. Very strange.

17

Hi enc0der,

great work I’ll try that too

Best regards,
Osama

18

Verified this works perfectly! Not sure why we are getting 1 less line but my issues were other code issues not related to these changes I made. So you should be good to go implementing this.

The other weird thing is 720x480 mode actually has an active area of 736 in the buffer it builds. No clue what that is about but tested with 2 different sources that output at that resolution.

19

Another update, in the csi2_fops.c, I turned on the padding for line and frame, this is much easier to deal with. (I duplicated the lines so I could remember what they were before, but removed both the NOPAD defines)

  /* CSI pixel parser registers setup */
pp_write(port, TEGRA_CSI_PIXEL_STREAM_PP_COMMAND,
		(0xF << CSI_PP_START_MARKER_FRAME_MAX_OFFSET) |
		CSI_PP_SINGLE_SHOT_ENABLE | CSI_PP_RST);
pp_write(port, TEGRA_CSI_PIXEL_PARSER_INTERRUPT_MASK, 0x0);
pp_write(port, TEGRA_CSI_PIXEL_STREAM_CONTROL0,
		CSI_PP_PACKET_HEADER_SENT |
		CSI_PP_DATA_IDENTIFIER_ENABLE |
		CSI_PP_WORD_COUNT_SELECT_HEADER |
		CSI_PP_CRC_CHECK_ENABLE |  CSI_PP_WC_CHECK |
		CSI_PP_OUTPUT_FORMAT_STORE |
		CSI_PP_HEADER_EC_DISABLE | 
		(csi_port & 1));
		//CSI_PP_OUTPUT_FORMAT_STORE | CSI_PPA_PAD_LINE_NOPAD |
		//CSI_PP_HEADER_EC_DISABLE | CSI_PPA_PAD_FRAME_NOPAD |
pp_write(port, TEGRA_CSI_PIXEL_STREAM_CONTROL1,
		(0x1 << CSI_PP_TOP_FIELD_FRAME_OFFSET) |
		(0x1 << CSI_PP_TOP_FIELD_FRAME_MASK_OFFSET));
pp_write(port, TEGRA_CSI_PIXEL_STREAM_GAP,
		0x14 << PP_FRAME_MIN_GAP_OFFSET);
pp_write(port, TEGRA_CSI_PIXEL_STREAM_EXPECTED_FRAME, 0x0);
pp_write(port, TEGRA_CSI_INPUT_STREAM_CONTROL,
		(0x3f << CSI_SKIP_PACKET_THRESHOLD_OFFSET) |
		(csi_lanes - 1));
1 Like