Jetson Nano MIPI CSI-2 without I2C from FPGA

hello adrien.leroy2,

I would suggest you probe and review the sensor signaling again,
is it actual follow MIPI CSI-2 specification, especially, end-of-frame signaling.
thanks

Hello @JerryChang,

I am not sure to understand what you mean. I have analysed the signals and they seem good. I don’t have a MIPI analyser so I can’t verify all the datas sent accurately.

I tested to reduce the framerate (from 147,7 to 60 fps) of my sensor and it modified the result of the previous command :

sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=320,height=240,pixelformat=RGGB --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1

I now have a framerate which vary from 5000 to 6000 (Before it was from 11000 to 15000). So it seems to have an impact but it’s always not good).

Furthermore, I tested to modify a lot of parameters of the device tree (line length, pix_clk_hz, tegra_sinterface, port-index…) and each time I have the same problem with the framerate. However, the device tree is well compiled. When I use the commands you gave me :

cat /proc/device-tree/host1x/i2c@546c0000/rbpcv2_imx219_a@10/mode0/active_h

After the flash of DTB I can see the parameters was been well updated. Could it be that the device tree is not read correctly ? (The device tree is well connected to the driver with the field compatible and the driver is well loaded)

What do you think about this ?

Thank you in advance,

Adrien.

hello adrien.leroy2,

the device tree blob has updated successfully if you checked and confirmed it via sysnode.

please check the frame-length settings of your sensor stream, frame-length is defined in the sensor init table,
there’s a formula to examine your line_length, frame_length, and also FPS.
i.e. FPS = pixel_clk_hz / (Line length * Frame length)
thanks

Hello @JerryChang ,
Currently, we are not working with a sensor but a FPGA which simulates its behaviour. So the Verylog/VHDL code allows me to set the parameters of the stream.
I checked this input stream using an oscilloscope to verify the framerate, the pixel clock, the image dimensions etc.
Everything seems correct.

In my opinion the problem would more likely come from the Jetson side, such as device tree, mode_tbls or driver but I don’t know where precisely…

I join to this post my codes for the driver and the mode_tbls file.

Here is the driver code :

#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/gpio.h>
#include <linux/module.h>
#include <linux/debugfs.h>

#include <linux/seq_file.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/of_gpio.h>

#include <media/tegra_v4l2_camera.h>
#include <media/tegracam_core.h>
#include <media/ati320.h>

#include "../platform/tegra/camera/camera_gpio.h"
#include "ati320_mode_tbls.h"
#define CREATE_TRACE_POINTS
#include <trace/events/ati320.h>

/* ati320 - sensor parameter limits */
//#define ATI320_MIN_GAIN				0x0000

/* ati320 sensor register address */
//#define ATI320_MODEL_ID_ADDR_MSB		0x0000

static const struct of_device_id ati320_of_match[] = {
	{ .compatible = "lynred,ati320", },
	{ },
};
MODULE_DEVICE_TABLE(of, ati320_of_match);

static const u32 ctrl_cid_list[] = {
	TEGRA_CAMERA_CID_GAIN,
	TEGRA_CAMERA_CID_EXPOSURE,
	TEGRA_CAMERA_CID_FRAME_RATE,
	TEGRA_CAMERA_CID_SENSOR_MODE_ID,
};

struct ati320 {
	struct i2c_client		*i2c_client;
	struct v4l2_subdev		*subdev;
	struct mutex            streaming_lock;
	bool                    streaming;
	u16				fine_integ_time;
	u32				frame_length;
	struct camera_common_data	*s_data;
	struct tegracam_device		*tc_dev;
};

static const struct regmap_config sensor_regmap_config = {
	.reg_bits = 16,
	.val_bits = 16,
	.cache_type = REGCACHE_RBTREE,
	.use_single_rw = true,
};



static inline int ati320_read_reg(struct camera_common_data *s_data,
	u16 addr, u8 *val)
{
	int err = 0;
	u32 reg_val = 0;

	err = regmap_read(s_data->regmap, addr, &reg_val);
	*val = reg_val & 0xff;

	return err;
}

static inline int ati320_write_reg(struct camera_common_data *s_data,
	u16 addr, u8 val)
{
	int err = 0;

	err = regmap_write(s_data->regmap, addr, val);
	if (err)
		dev_err(s_data->dev, "%s: i2c write failed, 0x%x = %x",
			__func__, addr, val);

	return err;
}

static int ati320_write_table(struct ati320 *priv, const ati320_reg table[])
{
	return 0;
}

static int ati320_set_group_hold(struct tegracam_device *tc_dev, bool val)
{
	return 0;
}

static int ati320_set_gain(struct tegracam_device *tc_dev, s64 val)
{
	return 0;
}

static int ati320_set_frame_rate(struct tegracam_device *tc_dev, s64 val)
{
	return 0;
}

static int ati320_set_exposure(struct tegracam_device *tc_dev, s64 val)
{
	return 0;
}


static struct tegracam_ctrl_ops ati320_ctrl_ops = {
	.numctrls = ARRAY_SIZE(ctrl_cid_list),
	.ctrl_cid_list = ctrl_cid_list,
	.set_gain = ati320_set_gain,
	.set_exposure = ati320_set_exposure,
	.set_frame_rate = ati320_set_frame_rate,
	.set_group_hold = ati320_set_group_hold,
};

static int ati320_power_on(struct camera_common_data *s_data)
{
	int err = 0;
	struct camera_common_power_rail *pw = s_data->power;
	struct camera_common_pdata *pdata = s_data->pdata;
	struct device *dev = s_data->dev;

	printk("Allumage de l'ati...\n");

	dev_dbg(dev, "%s: power on\n", __func__);
	if (pdata && pdata->power_on) {
		err = pdata->power_on(pw);
		if (err)
			dev_err(dev, "%s failed.\n", __func__);
		else
			pw->state = SWITCH_ON;
		return err;
	}

	// Ici met la sequence de démarrage de l ati

	/*
	if (pw->reset_gpio) {
		if (gpio_cansleep(pw->reset_gpio))
			gpio_set_value_cansleep(pw->reset_gpio, 0);
		else
			gpio_set_value(pw->reset_gpio, 0);
	}

	if (unlikely(!(pw->avdd || pw->iovdd || pw->dvdd)))
		goto skip_power_seqn;

	usleep_range(10, 20);

	if (pw->avdd) {
		err = regulator_enable(pw->avdd);
		if (err)
			goto ati320_avdd_fail;
	}

	if (pw->iovdd) {
		err = regulator_enable(pw->iovdd);
		if (err)
			goto ati320_iovdd_fail;
	}

	if (pw->dvdd) {
		err = regulator_enable(pw->dvdd);
		if (err)
			goto ati320_dvdd_fail;
	}

	usleep_range(10, 20);

skip_power_seqn:
	if (pw->reset_gpio) {
		if (gpio_cansleep(pw->reset_gpio))
			gpio_set_value_cansleep(pw->reset_gpio, 1);
		else
			gpio_set_value(pw->reset_gpio, 1);
	}
	*/
	/* Need to wait for t4 + t5 + t9 time as per the data sheet */
	/* t4 - 200us, t5 - 21.2ms, t9 - 1.2ms */
	//usleep_range(23000, 23100);
	
	//printk("ati start streaming");
	//ati320_start_streaming(s_data->tegracam_ctrl_hdl->tc_dev);

	pw->state = SWITCH_ON;
	printk("Ati powered on\n");
	return 0;

/*ati320_dvdd_fail:
	regulator_disable(pw->iovdd);

ati320_iovdd_fail:
	regulator_disable(pw->avdd);

ati320_avdd_fail:
	dev_err(dev, "%s failed.\n", __func__);

	return -ENODEV;*/
}

static int ati320_power_off(struct camera_common_data *s_data)
{
	int err = 0;
	struct camera_common_power_rail *pw = s_data->power;
	struct camera_common_pdata *pdata = s_data->pdata;
	struct device *dev = s_data->dev;
	
	printk("Extinction de l'ati...\n");

	dev_dbg(dev, "%s: power off\n", __func__);

	if (pdata && pdata->power_off) {
		err = pdata->power_off(pw);
		if (err) {
			dev_err(dev, "%s failed.\n", __func__);
			return err;
		}
	} else {
		if (pw->reset_gpio) {
			if (gpio_cansleep(pw->reset_gpio))
				gpio_set_value_cansleep(pw->reset_gpio, 0);
			else
				gpio_set_value(pw->reset_gpio, 0);
		}

		usleep_range(10, 10);

		if (pw->dvdd)
			regulator_disable(pw->dvdd);
		if (pw->iovdd)
			regulator_disable(pw->iovdd);
		if (pw->avdd)
			regulator_disable(pw->avdd);
	}

	pw->state = SWITCH_OFF;

	printk("Ati powered off\n");
	return 0;
}

static int ati320_power_put(struct tegracam_device *tc_dev)
{
	struct camera_common_data *s_data = tc_dev->s_data;
	struct camera_common_power_rail *pw = s_data->power;

	if (unlikely(!pw))
		return -EFAULT;

	if (likely(pw->dvdd))
		devm_regulator_put(pw->dvdd);

	if (likely(pw->avdd))
		devm_regulator_put(pw->avdd);

	if (likely(pw->iovdd))
		devm_regulator_put(pw->iovdd);

	pw->dvdd = NULL;
	pw->avdd = NULL;
	pw->iovdd = NULL;

	if (likely(pw->reset_gpio))
		gpio_free(pw->reset_gpio);

	return 0;
}

static int ati320_power_get(struct tegracam_device *tc_dev)
{
	struct device *dev = tc_dev->dev;
	struct camera_common_data *s_data = tc_dev->s_data;
	struct camera_common_power_rail *pw = s_data->power;
	struct camera_common_pdata *pdata = s_data->pdata;
	struct clk *parent;
	int err = 0;

	if (!pdata) {
		dev_err(dev, "pdata missing\n");
		return -EFAULT;
	}

	/* Sensor MCLK (aka. INCK) */
	if (pdata->mclk_name) {
		pw->mclk = devm_clk_get(dev, pdata->mclk_name);
		if (IS_ERR(pw->mclk)) {
			dev_err(dev, "unable to get clock %s\n",
				pdata->mclk_name);
			return PTR_ERR(pw->mclk);
		}

		if (pdata->parentclk_name) {
			parent = devm_clk_get(dev, pdata->parentclk_name);
			if (IS_ERR(parent)) {
				dev_err(dev, "unable to get parent clock %s",
					pdata->parentclk_name);
			} else
				clk_set_parent(pw->mclk, parent);
		}
	}

	/* Reset or ENABLE GPIO */
	pw->reset_gpio = pdata->reset_gpio;
	/*err = gpio_request(pw->reset_gpio, "cam_reset_gpio");
	if (err < 0) {
		dev_err(dev, "%s: unable to request reset_gpio (%d)\n",
			__func__, err);
		goto done;
	}

done:*/
	pw->state = SWITCH_OFF;

	return err;
}

static struct camera_common_pdata *ati320_parse_dt(
	struct tegracam_device *tc_dev)
{
	struct device *dev = tc_dev->dev;
	struct device_node *np = dev->of_node;
	struct camera_common_pdata *board_priv_pdata;
	const struct of_device_id *match;
	struct camera_common_pdata *ret = NULL;
	int err = 0;
	int gpio;
	printk("lancement du parsing\n");

	if (!np)
		return NULL;

	match = of_match_device(ati320_of_match, dev);
	if (!match) {
		dev_err(dev, "Failed to find matching dt id\n");
		return NULL;
	}

	board_priv_pdata = devm_kzalloc(dev,
		sizeof(*board_priv_pdata), GFP_KERNEL);
	if (!board_priv_pdata)
		return NULL;

	gpio = of_get_named_gpio(np, "reset-gpios", 0);
	if (gpio < 0) {
		if (gpio == -EPROBE_DEFER)
			ret = ERR_PTR(-EPROBE_DEFER);
		dev_err(dev, "reset-gpios not found\n");
		//goto error;
	}
	board_priv_pdata->reset_gpio = (unsigned int)gpio;

	err = of_property_read_string(np, "mclk", &board_priv_pdata->mclk_name);
	if (err)
		dev_dbg(dev, "mclk name not present, "
			"assume sensor driven externally\n");

	return board_priv_pdata;

/*error:
	devm_kfree(dev, board_priv_pdata);

	return ret;*/
}

static int ati320_set_mode(struct tegracam_device *tc_dev)
{
	struct ati320 *priv = (struct ati320 *)tegracam_get_privdata(tc_dev);
	struct camera_common_data *s_data = tc_dev->s_data;

	int err = 0;

	err = ati320_write_table(priv, mode_table[s_data->mode_prop_idx]);
	if (err)
		return err;

	err = ati320_write_table(priv, mode_table[s_data->mode]);
	if (err)
		return err;

	return 0;
}

static int ati320_start_streaming(struct tegracam_device *tc_dev)
{
	struct ati320 *priv = (struct ati320 *)tegracam_get_privdata(tc_dev);

	mutex_lock(&priv->streaming_lock);
	priv->streaming = true;
	//set pin 76 a 1 pour demarrer la mire
	gpio_request(76, "stream");
	gpio_direction_output(76, 1);
	gpio_set_value(76, 1);
	printk("debut du steam\n");
	mutex_unlock(&priv->streaming_lock);

	return 0;
}

static int ati320_stop_streaming(struct tegracam_device *tc_dev)
{
	struct ati320 *priv = (struct ati320 *)tegracam_get_privdata(tc_dev);
	u32 frame_time;

	mutex_lock(&priv->streaming_lock);
	priv->streaming = false;
	//set pin 76 a 0
	gpio_direction_output(76, 0);
	gpio_set_value(76, 0);
	printk("arret du stream\n");
	mutex_unlock(&priv->streaming_lock);

	frame_time = priv->frame_length*1480/52500/1000;

	usleep_range(frame_time, frame_time + 1000);


	return 0;
}

static struct camera_common_sensor_ops ati320_common_ops = {
	.numfrmfmts = ARRAY_SIZE(ati320_frmfmt),
	.frmfmt_table = ati320_frmfmt,
	.power_on = ati320_power_on,
	.power_off = ati320_power_off,
	.write_reg = ati320_write_reg,
	.read_reg = ati320_read_reg,
	.parse_dt = ati320_parse_dt,
	.power_get = ati320_power_get,
	.power_put = ati320_power_put,
	.set_mode = ati320_set_mode,
	.start_streaming = ati320_start_streaming,
	.stop_streaming = ati320_stop_streaming,
};

/*static int ati320_board_setup(struct ati320 *priv)
{
	struct camera_common_data *s_data = priv->s_data;
	struct camera_common_pdata *pdata = s_data->pdata;
	struct device *dev = s_data->dev;
	//u8 reg_val[2];
	int err = 0;

	if (pdata->mclk_name) {
		err = camera_common_mclk_enable(s_data);
		if (err) {
			dev_err(dev, "error turning on mclk (%d)\n", err);
			goto done;
		}
	}

	err = ati320_power_on(s_data);
	if (err) {
		dev_err(dev, "error during power on sensor (%d)\n", err);
		goto err_power_on;
	}*/

	/* Probe sensor model id registers */
	/*err = ati320_read_reg(s_data, ATI320_MODEL_ID_ADDR_MSB, &reg_val[0]);
	if (err) {
		dev_err(dev, "%s: error during i2c read probe (%d)\n",
			__func__, err);
		goto err_reg_probe;
	}
	err = ati320_read_reg(s_data, ATI320_MODEL_ID_ADDR_LSB, &reg_val[1]);
	if (err) {
		dev_err(dev, "%s: error during i2c read probe (%d)\n",
			__func__, err);
		goto err_reg_probe;
	}
	if (!((reg_val[0] == 0x02) && reg_val[1] == 0x19))
		dev_err(dev, "%s: invalid sensor model id: %x%x\n",
			__func__, reg_val[0], reg_val[1]);*/

	/* Sensor fine integration time */
	/*err = ati320_get_fine_integ_time(priv, &priv->fine_integ_time);
	if (err)
		dev_err(dev, "%s: error querying sensor fine integ. time\n",
			__func__);

err_reg_probe:
	ati320_power_off(s_data);

err_power_on:
	if (pdata->mclk_name)
		camera_common_mclk_disable(s_data);

done:
	return err;
}*/

static int ati320_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
{
	struct i2c_client *client = v4l2_get_subdevdata(sd);
	
	printk("ati320 open \n");
	dev_dbg(&client->dev, "%s:\n", __func__);

	return 0;
}

static const struct v4l2_subdev_internal_ops ati320_subdev_internal_ops = {
	.open = ati320_open,
};

static int ati320_probe(struct i2c_client *client,
	const struct i2c_device_id *id)
{
	struct device *dev = &client->dev;
	struct tegracam_device *tc_dev;
	struct ati320 *priv;
	int err;
	printk("ati320 probe start !\n");
	dev_dbg(dev, "probing v4l2 sensor at addr 0x%0x\n", client->addr);

	if (!IS_ENABLED(CONFIG_OF) || !client->dev.of_node)
		return -EINVAL;

	priv = devm_kzalloc(dev,
			sizeof(struct ati320), GFP_KERNEL);
	if (!priv)
		return -ENOMEM;

	tc_dev = devm_kzalloc(dev,
			sizeof(struct tegracam_device), GFP_KERNEL);
	if (!tc_dev)
		return -ENOMEM;

	priv->i2c_client = tc_dev->client = client;
	tc_dev->dev = dev;
	strncpy(tc_dev->name, "ati320", sizeof(tc_dev->name));
	tc_dev->dev_regmap_config = &sensor_regmap_config;
	tc_dev->sensor_ops = &ati320_common_ops;
	tc_dev->v4l2sd_internal_ops = &ati320_subdev_internal_ops;
	tc_dev->tcctrl_ops = &ati320_ctrl_ops;

	err = tegracam_device_register(tc_dev);
	if (err) {
		dev_err(dev, "tegra camera driver registration failed\n");
		return err;
	}
	priv->tc_dev = tc_dev;
	priv->s_data = tc_dev->s_data;
	priv->subdev = &tc_dev->s_data->subdev;
	tegracam_set_privdata(tc_dev, (void *)priv);
	mutex_init(&priv->streaming_lock);

	/*err = ati320_board_setup(priv);
	if (err) {
		tegracam_device_unregister(tc_dev);
		dev_err(dev, "board setup failed\n");
		return err;
	}*/

	err = tegracam_v4l2subdev_register(tc_dev, true);
	if (err) {
		dev_err(dev, "tegra camera subdev registration failed\n");
		return err;
	}

	dev_dbg(dev, "detected ati320 sensor\n");

	printk("ati320 probe stop !\n");
	return 0;
}

static int ati320_remove(struct i2c_client *client)
{
	struct camera_common_data *s_data = to_camera_common_data(&client->dev);
	struct ati320 *priv = (struct ati320 *)s_data->priv;

	tegracam_v4l2subdev_unregister(priv->tc_dev);
	tegracam_device_unregister(priv->tc_dev);

	mutex_destroy(&priv->streaming_lock);

	printk("ati320 removed !\n");
	return 0;
}

static const struct i2c_device_id ati320_id[] = {
	{ "ati320", 0 },
	{ }
};
MODULE_DEVICE_TABLE(i2c, ati320_id);

static struct i2c_driver ati320_i2c_driver = {
	.driver = {
		.name = "ati320",
		.owner = THIS_MODULE,
		.of_match_table = of_match_ptr(ati320_of_match),
	},
	.probe = ati320_probe,
	.remove = ati320_remove,
	.id_table = ati320_id,
};
module_i2c_driver(ati320_i2c_driver);

MODULE_DESCRIPTION("Media Controller driver for Sony ATI320");
MODULE_AUTHOR("NVIDIA Corporation");
MODULE_LICENSE("GPL v2");

and the mode_tbls code :

/*
 * ati320_tables.h - sensor mode tables for ati320 HDR sensor.
 *
 * Copyright (c) 2015-2020, NVIDIA CORPORATION, All Rights Reserved.
 *
 * This program is free software; you can redistribute it and/or modify it
 * under the terms and conditions of the GNU General Public License,
 * version 2, as published by the Free Software Foundation.
 *
 * This program is distributed in the hope it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

#ifndef __ATI320_I2C_TABLES__
#define __ATI320_I2C_TABLES__

#define ATI320_TABLE_WAIT_MS	0
#define ATI320_WAIT_MS	10
#define ATI320_TABLE_END	1

#define ati320_reg struct reg_8

static ati320_reg ati320_start_stream[] = {
	{0x0100, 0x01},
	{ATI320_TABLE_END, 0x00}
};

static ati320_reg ati320_stop_stream[] = {
	{0x0100, 0x00},
	{ATI320_TABLE_END, 0x00}
};

static ati320_reg ati320_mode_320x240_147fps[] = {
	{ATI320_TABLE_WAIT_MS, ATI320_WAIT_MS},
	{ATI320_TABLE_END, 0x0000}
};

enum {
	ATI320_MODE_320x240_147FPS,

	ATI320_START_STREAM,
	ATI320_STOP_STREAM,
};

static ati320_reg *mode_table[] = {
	[ATI320_MODE_320x240_147FPS] = ati320_mode_320x240_147fps,

	[ATI320_START_STREAM]  = ati320_start_stream,
	[ATI320_STOP_STREAM]  = ati320_stop_stream,
};

static const int ati320_147fps[] = {
	60,
};

/*
 * WARNING: frmfmt ordering need to match mode definition in
 * device tree!
 */
static const struct camera_common_frmfmt ati320_frmfmt[] = {
	{{320, 240},	ati320_147fps, 1, 0, ATI320_MODE_320x240_147FPS},
};

#endif /* __ATI320_I2C_TABLES__ */

Maybe you could spot an error in those files.

Thank you,
Adrien

hello adrien.leroy2,

don’t you need to program some registers to enable test-pattern, or, this is the device sending signaling continuously when it power-on?
is actual signaling coming immediately, how you determine this 10ms waiting time for mode initialization?
for example,

had you repot frame_length before senor streaming?
please have a try to hack the value for testing.

hello @JerryChang ,
we did a lot of things !

We found a problem in the IP of the FPGA, and corrected it !
Now we can receive some frame with a correct frame rate and create a .raw file of the capture. We can convert it to observe the actual image sent and it is correct !
fabiengray

Here is the capture’s trace :

[  237.912666] Allumage de l'ati...
[  237.912669] Ati powered on
[  237.912718] debut du steam
[  237.912724] vi 54080000.vi: Calibrate csi port 0
[  237.912846] arret du stream
[  237.912850] Extinction de l'ati...
[  237.912870] Ati powered off
[  237.912932] Allumage de l'ati...
[  237.912934] Ati powered on
[  237.915280] vi 54080000.vi: cil_settingtime was autocalculated
[  237.915286] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[  237.915315] debut du steam
[  237.915405] pb of csi ports in single thread
[  237.915410] 0
[  237.922360] pb of csi ports in single thread
[  237.922365] 0
[  237.929169] pb of csi ports in single thread
[  237.929172] 0
[  237.936315] pb of csi ports in single thread
[  237.936318] 0
[  237.942889] pb of csi ports in single thread
[  237.942893] 0
[  237.949723] pb of csi ports in single thread
[  237.949727] 0
[  237.956539] pb of csi ports in single thread
[  237.956543] 0
[  237.963396] pb of csi ports in single thread
[  237.963399] 0
[  237.970263] pb of csi ports in single thread
[  237.970268] 0
[  237.977139] pb of csi ports in single thread
[  237.977144] 0
[  237.983985] pb of csi ports in single thread
[  237.983988] 0
[  237.990892] pb of csi ports in single thread
[  237.990896] 0
[  237.997753] pb of csi ports in single thread
[  237.997757] 0
[  238.004564] pb of csi ports in single thread
[  238.004569] 0
[  238.011411] pb of csi ports in single thread
[  238.011414] 0
[  238.018271] pb of csi ports in single thread
[  238.018275] 0
[  238.025138] pb of csi ports in single thread
[  238.025143] 0
[  238.031989] pb of csi ports in single thread
[  238.031994] 0
[  238.038866] pb of csi ports in single thread
[  238.038874] 0
[  238.045760] pb of csi ports in single thread
[  238.045765] 0
[  238.052661] pb of csi ports in single thread
[  238.052664] 0
[  238.059485] pb of csi ports in single thread
[  238.059489] 0
[  238.066279] pb of csi ports in single thread
[  238.066283] 0
[  238.073153] pb of csi ports in single thread
[  238.073158] 0
[  238.080054] pb of csi ports in single thread
[  238.080058] 0
[  238.086909] pb of csi ports in single thread
[  238.086913] 0
[  238.093743] pb of csi ports in single thread
[  238.093748] 0
[  238.100594] pb of csi ports in single thread
[  238.100597] 0
[  238.107508] pb of csi ports in single thread
[  238.107513] 0
[  238.114354] pb of csi ports in single thread
[  238.114358] 0
[  238.121154] pb of csi ports in single thread
[  238.121158] 0
[  238.128129] pb of csi ports in single thread
[  238.128135] 0
[  238.134871] pb of csi ports in single thread
[  238.134876] 0
[  238.141812] pb of csi ports in single thread
[  238.141818] 0
[  238.148666] pb of csi ports in single thread
[  238.148671] 0
[  238.155606] pb of csi ports in single thread
[  238.155610] 0
[  238.162352] pb of csi ports in single thread
[  238.162357] 0
[  238.169157] pb of csi ports in single thread
[  238.169160] 0
[  238.176048] pb of csi ports in single thread
[  238.176053] 0
[  238.182973] pb of csi ports in single thread
[  238.182978] 0
[  238.189804] pb of csi ports in single thread
[  238.189811] 0
[  238.196632] pb of csi ports in single thread
[  238.196700] 0
[  238.203466] pb of csi ports in single thread
[  238.203472] 0
[  238.210389] pb of csi ports in single thread
[  238.210395] 0
[  238.217256] pb of csi ports in single thread
[  238.217260] 0
[  238.224166] pb of csi ports in single thread
[  238.224173] 0
[  238.230972] pb of csi ports in single thread
[  238.230976] 0
[  238.237848] pb of csi ports in single thread
[  238.237854] 0
[  238.244624] pb of csi ports in single thread
[  238.244629] 0
[  238.251544] pb of csi ports in single thread
[  238.251548] 0
[  238.258348] pb of csi ports in single thread
[  238.258355] 0
[  238.265192] pb of csi ports in single thread
[  238.265198] 0
[  238.272051] pb of csi ports in single thread
[  238.272055] 0
[  238.278921] pb of csi ports in single thread
[  238.278926] 0
[  238.285786] pb of csi ports in single thread
[  238.285791] 0
[  238.292642] pb of csi ports in single thread
[  238.292648] 0
[  238.299494] pb of csi ports in single thread
[  238.299501] 0
[  238.306336] pb of csi ports in single thread
[  238.306339] 0
[  238.313220] pb of csi ports in single thread
[  238.313226] 0
[  238.320057] pb of csi ports in single thread
[  238.320062] 0
[  238.327275] pb of csi ports in single thread
[  238.327282] 0
[  238.333777] pb of csi ports in single thread
[  238.333781] 0
[  238.340628] pb of csi ports in single thread
[  238.340631] 0
[  238.347565] pb of csi ports in single thread
[  238.347570] 0
[  238.354426] pb of csi ports in single thread
[  238.354431] 0
[  238.361250] pb of csi ports in single thread
[  238.361258] 0
[  238.368138] pb of csi ports in single thread
[  238.368144] 0
[  238.374931] pb of csi ports in single thread
[  238.374935] 0
[  238.381830] pb of csi ports in single thread
[  238.381834] 0
[  238.388648] pb of csi ports in single thread
[  238.388654] 0
[  238.395680] pb of csi ports in single thread
[  238.395688] 0
[  238.402348] pb of csi ports in single thread
[  238.402352] 0
[  238.409211] pb of csi ports in single thread
[  238.409215] 0
[  238.416089] pb of csi ports in single thread
[  238.416095] 0
[  238.422999] pb of csi ports in single thread
[  238.423004] 0
[  238.429875] pb of csi ports in single thread
[  238.429880] 0
[  238.436676] pb of csi ports in single thread
[  238.436682] 0
[  238.443521] pb of csi ports in single thread
[  238.443528] 0
[  238.450454] pb of csi ports in single thread
[  238.450459] 0
[  238.457308] pb of csi ports in single thread
[  238.457313] 0
[  238.464173] pb of csi ports in single thread
[  238.464182] 0
[  238.470955] pb of csi ports in single thread
[  238.470991] 0
[  238.477814] pb of csi ports in single thread
[  238.477820] 0
[  238.484671] pb of csi ports in single thread
[  238.484675] 0
[  238.491539] pb of csi ports in single thread
[  238.491546] 0
[  238.498459] pb of csi ports in single thread
[  238.498465] 0
[  238.505237] pb of csi ports in single thread
[  238.505241] 0
[  238.512109] pb of csi ports in single thread
[  238.512115] 0
[  238.519038] pb of csi ports in single thread
[  238.519044] 0
[  238.525887] pb of csi ports in single thread
[  238.525892] 0
[  238.532723] pb of csi ports in single thread
[  238.532728] 0
[  238.539527] pb of csi ports in single thread
[  238.539531] 0
[  238.546398] pb of csi ports in single thread
[  238.546404] 0
[  238.553247] pb of csi ports in single thread
[  238.553251] 0
[  238.560132] pb of csi ports in single thread
[  238.560136] 0
[  238.567054] pb of csi ports in single thread
[  238.567060] 0
[  238.573811] pb of csi ports in single thread
[  238.573815] 0
[  238.580684] pb of csi ports in single thread
[  238.580689] 0
[  238.587534] pb of csi ports in single thread
[  238.587538] 0
[  238.594482] pb of csi ports in single thread
[  238.594487] 0
[  238.601296] pb of csi ports in single thread
[  238.601303] 0
[  238.608107] pb of csi ports in single thread
[  238.608112] 0
[  238.614980] pb of csi ports in single thread
[  238.614986] 0
[  238.621962] pb of csi ports in single thread
[  238.621967] 0
[  238.628802] pb of csi ports in single thread
[  238.628806] 0
[  238.635571] pb of csi ports in single thread
[  238.635578] 0
[  238.642401] pb of csi ports in single thread
[  238.642405] 0
[  238.649260] pb of csi ports in single thread
[  238.649264] 0
[  238.656253] pb of csi ports in single thread
[  238.656258] 0
[  238.663117] pb of csi ports in single thread
[  238.663121] 0
[  238.669909] pb of csi ports in single thread
[  238.669914] 0
[  238.676695] pb of csi ports in single thread
[  238.676699] 0
[  238.683566] pb of csi ports in single thread
[  238.683571] 0
[  238.690496] pb of csi ports in single thread
[  238.690500] 0
[  238.697302] pb of csi ports in single thread
[  238.697308] 0
[  238.704167] pb of csi ports in single thread
[  238.704175] 0
[  238.711011] pb of csi ports in single thread
[  238.711019] 0
[  238.717853] pb of csi ports in single thread
[  238.717858] 0
[  238.724716] pb of csi ports in single thread
[  238.724722] 0
[  238.731656] pb of csi ports in single thread
[  238.731661] 0
[  238.738527] pb of csi ports in single thread
[  238.738534] 0
[  238.745300] pb of csi ports in single thread
[  238.745305] 0
[  238.752170] pb of csi ports in single thread
[  238.752176] 0
[  238.759003] pb of csi ports in single thread
[  238.759008] 0
[  238.765932] pb of csi ports in single thread
[  238.765937] 0
[  238.772781] pb of csi ports in single thread
[  238.772788] 0
[  238.779584] pb of csi ports in single thread
[  238.779589] 0
[  238.786443] pb of csi ports in single thread
[  238.786447] 0
[  238.793297] pb of csi ports in single thread
[  238.793303] 0
[  238.800159] pb of csi ports in single thread
[  238.800164] 0
[  238.807050] pb of csi ports in single thread
[  238.807056] 0
[  238.813863] pb of csi ports in single thread
[  238.813869] 0
[  238.820735] pb of csi ports in single thread
[  238.820742] 0
[  238.827610] pb of csi ports in single thread
[  238.827614] 0
[  238.834453] pb of csi ports in single thread
[  238.834458] 0
[  238.841447] pb of csi ports in single thread
[  238.841454] 0
[  238.848200] pb of csi ports in single thread
[  238.848204] 0
[  238.855038] pb of csi ports in single thread
[  238.855043] 0
[  238.861989] pb of csi ports in single thread
[  238.861993] 0
[  238.868822] pb of csi ports in single thread
[  238.868826] 0
[  238.875610] pb of csi ports in single thread
[  238.875615] 0
[  238.882450] pb of csi ports in single thread
[  238.882454] 0
[  238.889334] pb of csi ports in single thread
[  238.889340] 0
[  238.896289] pb of csi ports in single thread
[  238.896293] 0
[  238.903143] pb of csi ports in single thread
[  238.903150] 0
[  238.910070] pb of csi ports in single thread
[  238.910352] 0
[  238.916747] pb of csi ports in single thread
[  238.916751] 0
[  238.923612] pb of csi ports in single thread
[  238.923618] 0
[  238.930542] pb of csi ports in single thread
[  238.930547] 0
[  238.937328] pb of csi ports in single thread
[  238.937334] 0
[  238.944266] pb of csi ports in single thread
[  238.944294] 0
[  238.951079] pb of csi ports in single thread
[  238.951084] 0
[  238.957978] pb of csi ports in single thread
[  238.957983] 0
[  238.958035] video4linux video0: Syncpoint already enabled at capture done!0
[  238.964798] capture done error, single_thread
[  238.964803] 0
[  238.964841] arret du stream
[  238.966393] Extinction de l'ati...
[  238.966448] Ati powered off

Now we are stuck at the step of capturing the video stream…we tried to use gstreamer :

pi21@pi21-desktop:~/Téléchargements$ gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, format=GRAY8, width=320, height=240, framerate=147/1' ! xvimagesink -ev
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

And what we get using dmesg --follow :

[  260.263408] ati320 open
[  260.265727] Allumage de l'ati...
[  260.265731] Ati powered on
[  260.265755] debut du steam
[  260.265762] vi 54080000.vi: Calibrate csi port 0
[  260.265886] arret du stream
[  260.265891] Extinction de l'ati...
[  260.265914] Ati powered off
[  260.265978] Allumage de l'ati...
[  260.265979] Ati powered on
[  260.266019] Extinction de l'ati...
[  260.266041] Ati powered off
[  260.551334] ati320 open
[  260.551626] Allumage de l'ati...
[  260.551628] Ati powered on
[  260.551662] Extinction de l'ati...
[  260.551719] Ati powered off
[  260.571136] Allumage de l'ati...
[  260.571140] Ati powered on
[  260.622162] Extinction de l'ati...
[  260.622203] Ati powered off

Do you have any suggestions ?
Thank you by advance
Adrien

hello adrien.leroy2,

cool, it’s a good news.
assume you had already verify the streaming by v4l2 standard controls, and the frame-rate report correctly.

please check the sensor capability via v4l2, i.e. $ v4l2-ctl -d /dev/video0 --list-formats-ext
are you also complete the implementation in VI driver to add GRAY8 supports?
thanks

Hello @JerryChang

Yes, I verified it and we can see the correct framerate :

sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=320,height=240,pixelformat=GREY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=450
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 146.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 146.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 146.00 fps
<<<<<<<<<<<

Here is the result of the command v4l2-ctl -d /dev/video0 --list-formats-ext :

ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'GREY'
        Name        : 8-bit Greyscale
                Size: Discrete 320x240
                        Interval: Discrete 0.007s (147.000 fps)

In my opinion, I well implemented the GRAY8 support in VI driver. But maybe that I made a mistake, here the modifications I did :

vi2_formats.h :

static const struct tegra_video_format vi2_video_formats[] = {
        /* RAW 6: TODO */

        /* RAW 7: TODO */

        /* GRAYSCALE */
        TEGRA_VIDEO_FORMAT(RAW8, 8, Y8_1X8, 1, 1, T_L8, RAW8, GREY, "GRAY8"),

camera_common.c :

static const struct camera_common_colorfmt camera_common_color_fmts[]
        {        
                MEDIA_BUS_FMT_Y8_1X8, //GRAYSCALE 8 BITS
                V4L2_COLORSPACE_RAW, //RAW
                V4L2_PIX_FMT_GREY,
        },

sensor_common.c :

static int extract_pixel_format
//grayscale format
        else if (strncmp(pixel_t, "gray", size) == 0)
                *format = V4L2_PIX_FMT_GREY;

I saw in the dtsi files that there are a lot af fields status = "disabled";. Maybe shall I have to put them in "okay". What do think about it ?

tegra210-porg-camera.dtsi
/*
 * Copyright (c) 2018-2019, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */
#include "tegra210-porg-camera-ati320.dtsi"

/ {
        tegra-camera-platform {
                compatible = "nvidia, tegra-camera-platform";

                modules {
                        cam_module0: module0 {
                                status = "disabled";
                                cam_module0_drivernode0: drivernode0 {
                                        status = "disabled";
                                };
                                cam_module0_drivernode1: drivernode1 {
                                        status = "disabled";
                                        pcl_id = "v4l2_lens";
                                };
                        };
                        cam_module1: module1 {
                                status = "disabled";
                                cam_module1_drivernode0: drivernode0 {
                                        status = "disabled";
                                };
                                cam_module1_drivernode1: drivernode1 {
                                        status = "disabled";
                                        pcl_id = "v4l2_lens";
                                };
                        };
                };
        };

        /* all vi and csi ports are disabled by default */
        host1x {
                vi_base {
                        ports {
                                vi_port0 {
                                        status = "disabled";
                                        vi_in0: endpoint {
                                                status = "disabled";
                                        };
                                };
                                vi_port1 {
                                        status = "disabled";
                                        vi_in1: endpoint {
                                                status = "disabled";
                                        };
                                };
                        };
                };
                csi_base{
                        csi_chan0 {
                                status = "disabled";
                                ports {
                                        csi_chan0_port0 {
                                                status = "disabled";
                                                csi_in0: endpoint@0 {
                                                        status = "disabled";
                                                };
                                        };
                                        csi_chan0_port1 {
                                                status = "disabled";
                                                csi_out0: endpoint@1 {
                                                        status = "disabled";
                                                };
                                        };
                                };
                        };
                        csi_chan1 {
                                status = "disabled";
                                ports {
                                        csi_chan1_port0 {
                                                status = "disabled";
                                                csi_in1: endpoint@2 {
                                                        status = "disabled";
                                                };
                                        };
                                        csi_chan1_port1 {
                                                status = "disabled";
                                                csi_out1: endpoint@3 {
                                                        status = "disabled";
                                                };
                                        };
                                };
                        };
                };
        };

};

Thanks.

Adrien

hello adrien.leroy2,

yes, please revise the device tree definition to configure status = "okay" for the port bindings,

I don’t have grayscale sensor to validate the capture path on Nano,
according to below topic, there’s forum developer has RAW8 camera sensor (i.e. RGGB8), and they had confirmed it works with v4l2src.
i.e. RAW8 capture settings for nvcamerasrc.
so, how about have a try to report the video format instead of grayscale?

Hi @JerryChang

I already did a thing like that :

tegra210-porg-camera-ati320.dtsi

#include "tegra210-camera-ati320.dtsi"

#define CAM1_PWDN                TEGRA_GPIO(S, 7)

/ {
    host1x {
                i2c@546c0000 {
                        ati320_a@13 {
                                status = "okay";
                                reset-gpios = <&gpio CAM1_PWDN GPIO_ACTIVE_HIGH>;
                        };
                };
        };

    gpio@6000d000 {
                camera-control-output-low {
                        gpio-hog;
                        output-low;
                        gpios = < CAM1_PWDN 0 >;
                        label = "cam1-pwdn";
                };
        };
};

Is it what you would liked me to do ?

Yes we can try it.

Thanks

Adrien

yes, please further have a try to report the video format as default supported RAW8 instead of grayscale. thanks

@JerryChang

I don’t understand well what I have to do. I switched from grayscale to rggb8 in the dtsi but what are next steps ?

Thanks

Adrien

hello adrien.leroy2,

it my assumption to have several approaches to narrow down this,
for example,

  1. could fakesink workable to test the sensor streaming? for example, $ gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, format=GRAY8, width=320, height=240, framerate=147/1' ! fakesink

  2. if fakesink works, it’s due to xvimagesink it supports YUV format types, you may include conversion to change the color formats to adapt the output buffer.

  3. here’s my another assumption to cheat driver side as a RAW8 camera sensor (i.e. RGGB8), for testing the camera streaming.

1 Like

hello Jerrychang,
I am gonna try the gstreamer command as soon as I can !
But concerning your 3rd assumption, didn’t you forget something ?

Thank you !
Adrien

Hello @JerryChang !
I got some good news !

I tested with your command (which works) but I wanted an ouput so I had to use xvimagesink…

Here is the command : gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-bayer, format=rggb, width=320,height=240' ! bayer2rgb ! videoconvert ! xvimagesink

Thanks again !
Adrien

hello adrien.leroy2,

let’s have summarize and close this topic, initial anther new topic if you need further supports.
so, it is confirmed you’re able to see video stream by including video converter?

Yes It is confirmed, and it is also working in GRAY8 mode with this command :

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw, format=GRAY8, width=320,height=240' ! glimagesink

Thanks

Adrien

1 Like

So, to summarize for people that may read this topic…
Thanks to the great support or @JerryChang I have been able to correct some mistakes in my .dtsi and have some useful tools to debug my whole setup. But finally the biggest problem was in the hardware part, now I am able to stream and capture frame from my custom RAW8 sensor.

Thank you again

Adrien

1 Like

Hi @adrien.leroy2

I am trying to implement the same FPGA pre-configured Test Pattern Generation with Xavier NX board. I have implemented the all the steps mentioned above but I am not able to capture the CSI data.
Could you please tell me how you resolved your issue,

Thanks,
Ajith.