How to display preview Dual Custom camera sensor on Jetson Tx2

Hello,

I developed Dual custom camera sensor on Jetson Tx2
I registered device drivers /dev/video0 , /dev/video1

I want to display dual camera images captured in preview at the same time.

but nvgstcapture-1.0 command is not allowed video driver select as below.

nvgstcapture-1.0 – cap-dev-node=1

and let me know how to use “gstreammer command” in dual camera mode.

Hello @jpchae,

You can use the videomixer GStreamer element to get both images at the same time. Here is an example using video test src

gst-launch-1.0 -e videomixer name=mix sink_1::xpos=640 ! xvimagesink \
videotestsrc ! video/x-raw, width=640, height=480, framerate=30/1 ! mix.sink_0 \
videotestsrc pattern="ball" ! video/x-raw, width=640, height=480, framerate=30/1 ! mix.sink_1

You will get the following result:

You can change the videotestsrc elements for nvarguscamerasrc, v4l2src or the capture element you are trying to use.

Regards,

Fabian
www.ridgerun.com

1 Like

Hello fabian.solano ,

I used the following command for a single CIS sensor.

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=2608, height=1960, format=(string)NV12, framerate=30/1' ! nvoverlaysink -e
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=1304, height=980, format=(string)NV12, framerate=60/1' ! nvoverlaysink -e

When i edited and executed the command you let know me, the following error occurred.

jpchae@jpchae-desktop:~/Downloads$ gst-launch-1.0 -e videomixer name=mix sink_1::xpos=640 ! xvimagesink nvarguscamerasrc ! video/x-raw, width=1304,height=980, framerate=60/1 ! mix.sink_0 nvarguscamerasrc ! video/x-raw, width =1304, height=980, framerate=50/1 ! mix.sink_1
WARNING: erroneous pipeline: could not link nvarguscamerasrc0 to mix, nvarguscamerasrc0 can't handle caps video/x-raw, width=(int)1304, height=(int)980, framerate=(fraction)60/1
jpchae@jpchae-desktop:~/Downloads$ gst-launch-1.0 -e videomixer name=mix sink_1::xpos=640 ! xvimagesink nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1304,height=980, framerate=60/1 ! mix.sink_0 nvarguscamerasrc ! video/x-raw, width =1304, height=980, framerate=50/1 ! mix.sink_1
bash: syntax error near unexpected token `('
jpchae@jpchae-desktop:~/Downloads$ gst-launch-1.0 -e videomixer name=mix sink_1::xpos=640 ! xvimagesink nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=1304,height=980, format=(string)NV12,framerate=60/1' ! mix.sink_0 nvarguscamerasrc ! video/x-raw, width =1304, height=980, framerate=50/1 ! mix.sink_1
WARNING: erroneous pipeline: could not link nvarguscamerasrc0 to mix, mix can't handle caps video/x-raw(memory:NVMM), width=(int)1304, height=(int)980, format=(string)NV12, framerate=(fraction)60/1

please, tell me how to fix it.

hello jpchae,

please refer to below sample commands to enable dual camera preview at different overlays; I’ve also resize the sources content by video converter due to display resolution. you may also stop gdm service to make multiple overlay works for rendering.
for example,

sudo systemctl stop gdm
sudo loginctl terminate-seat seat0

camera-1:
$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! "video/x-raw(memory:NVMM), format=I420, width=960, height=360, framerate=30/1" ! nvoverlaysink overlay=0 overlay-x=0 overlay-y=0 overlay-w=960 overlay-h=360
camera-2:
$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! "video/x-raw(memory:NVMM), format=I420, width=960, height=360, framerate=30/1" ! nvoverlaysink overlay=1 overlay-x=961 overlay-y=360 overlay-w=960 overlay-h=360
1 Like

Hello JerryChang,

I fixed hardware bug as shown blew link.

and I test gstreammer command as below .

gst-launch-1.0 nvarguscamerasrc sensor-id=0 wbmode=1 aelock=1 ! ‘video/x-raw(memory:NVMM) , width=2608 , height=1960, format=(string)NV12, frameratre=30/1’ ! nvvidconv ! xvimagesink -e

gst-launch-1.0 nvarguscamerasrc sensor-id=1 wbmode=1 aelock=1 ! ‘video/x-raw(memory:NVMM) , width=2608 , height=1960, format=(string)NV12, frameratre=30/1’ ! nvvidconv ! xvimagesink -e

sensor-id=0 (dev/video0) is good working.
but sensor-id=1 (dev/video1) is not working as shown below message.

jpchae@jpchae-desktop:~$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 wbmode=1 aelock=1 saturation=1.5 maxperf=1 ! ‘video/x-raw(memory:NVMM), width=2608, height=1960, format=(string)NV12, framerate=30/1’ ! nvvidconv ! xvimagesink -e
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
> Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:536 Invalid camera device specified 1 specified, 0 max index
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.005479785
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

please , give me advise.

hello jpchae,

it seems there’s only single camera registered.
could you please check $ ls /dev/video* for all camera nodes.

there should be couple of camera if sensor probing without failures.
please also looking kernel init messages for the camera sensor probing messages.
thanks

hello JerryChang,

i checked device register /dev/video* .
i found two camera ( video0 ,video1 ).

and also check media-ctl -p message.

media-ctl -p
Media controller API version 0.1.0

Media device information

driver tegra-vi4
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 0.0.0

Device topology

  • entity 1: 150c0000.nvcsi–2 (2 pads, 2 links)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev0
    pad0: Sink
    ← “s5k5e8 2-0010”:0 [ENABLED]
    pad1: Source
    → “vi-output, s5k5e8 2-0010”:0 [ENABLED]

  • entity 4: 150c0000.nvcsi–1 (2 pads, 2 links)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev1
    pad0: Sink
    ← “s5k5e8 2-0018”:0 [ENABLED]
    pad1: Source
    → “vi-output, s5k5e8 2-0018”:0 [ENABLED]

  • entity 7: s5k5e8 2-0010 (1 pad, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev2
    pad0: Source
    [fmt:SGRBG10_1X10/2608x1960 field:none colorspace:srgb]
    → “150c0000.nvcsi–2”:0 [ENABLED]

  • entity 9: vi-output, s5k5e8 2-0010 (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video0
    pad0: Sink
    ← “150c0000.nvcsi–2”:1 [ENABLED]

  • entity 23: s5k5e8 2-0018 (1 pad, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev3
    pad0: Source
    [fmt:SGRBG10_1X10/2608x1960 field:none colorspace:srgb]
    → “150c0000.nvcsi–1”:0 [ENABLED]

  • entity 25: vi-output, s5k5e8 2-0018 (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video1
    pad0: Sink
    ← “150c0000.nvcsi–1”:1 [ENABLED]

i checked dmesg and two driver was registered.

also i tested each sensor setting( single sensor) was good working.

please, give me advise.

i attached device tree.
video0 (i2c 0x10) csi-c video1 (i2c 0x18) csi-d

/ {
host1x {
vi@15700000 {
num-channels = <2>;
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;

  			e3327_vi_in0: endpoint {
  				port-index = <2>;
  				bus-width = <2>;
  				remote-endpoint = <&e3327_csi_out0>;
  			};
  		};
  		port@1 {
  			reg = <1>;
  	
  			e3327_vi_in1: endpoint {
  				port-index = <3>;
  				bus-width = <2>;
  				remote-endpoint = <&e3327_csi_out1>;
  			};
  		};

  	};
  };

  nvcsi@150c0000 {
  	num-channels = <2>;

  	channel@0 {
  		reg = <0>;
  		ports {
  			#address-cells = <1>;
  			#size-cells = <0>;
  			port@0 {
  				reg = <0>;
  				e3327_csi_in0: endpoint@0 {
  					port-index = <2>;
  					bus-width = <2>;
  					remote-endpoint = <&e3327_s5k5e8_out0>;
  				};
  			};
  			port@1 {
  				reg = <1>;
  				e3327_csi_out0: endpoint@1 {
  					remote-endpoint = <&e3327_vi_in0>;
  				};
  			};
  		};
  	};
  	channel@1 {
  		reg = <1>;
  		ports {
  			#address-cells = <1>;
  			#size-cells = <0>;
  			port@0 {
  				reg = <0>;
  				e3327_csi_in1: endpoint@2 {
  					port-index = <3>;
  					bus-width = <2>;
  					remote-endpoint = <&e3327_s5k5e8_out1>;
  				};
  			};
  			port@1 {
  				reg = <1>;
  				e3327_csi_out1: endpoint@3 {
  					remote-endpoint = <&e3327_vi_in1>;
  				};
  			};
  		};
  	};

  };

};

i2c@3180000 {
s5k5e8_c@10 {
compatible = “nvidia,s5k5e8”;
/* I2C device address */
reg = <0x10>;

  	/* V4L2 device node location */
  	devnode = "video0";

  	/* Physical dimensions of sensor */
  	physical_w = "4.675";
  	physical_h = "3.633";

  	/* Define any required hw resources needed by driver */
  	/* ie. clocks, io pins, power sources */
  	avdd-reg = "vana";
  	iovdd-reg = "vif";
  	dvdd-reg = "vdig";

  	/* Sensor output flip settings */
  	vertical-flip = "false";


  	mode0 { // S5K5E8_MODE_2608X1960
  		mclk_khz = "24000";
  		num_lanes = "2";
  		tegra_sinterface = "serial_c";
  		phy_mode = "DPHY";
  		discontinuous_clk = "no";
  		dpcm_enable = "false";
  		cil_settletime = "0";

  		active_w = "2608";
  		active_h = "1960";
  		mode_type = "bayer";
  		pixel_phase = "grbg";     
  		csi_pixel_bit_depth = "10";   
  		readout_orientation = "90";
  		line_length = "2856";
  		inherent_gain = "1";
  		mclk_multiplier = "7.1";
  		pix_clk_hz = "167200000";//"170000000";

  		gain_factor = "1";
  		min_gain_val = "0";/* 1DB*/
  		max_gain_val = "4";/* 16DB*/
  		step_gain_val = "1";
  		default_gain = "2";
  		min_hdr_ratio = "";
  		max_hdr_ratio = "";
  		framerate_factor = "1000000";
  		min_framerate = "20000000";/*1.816577 */
  		max_framerate = "30000000";/*30*/
  		step_framerate = "1";
  		default_framerate = "30000000";
  		exposure_factor = "1000000";
  		min_exp_time = "34";        //"16000";/* us */
  		max_exp_time = "550385";     //"30088";/* us */
  		step_exp_time = "1";
  		default_exp_time = "33334"; //"30088";/* us */
  		embedded_metadata_height = "0";
  	};


  	ports {
  		#address-cells = <1>;
  		#size-cells = <0>;

  		port@0 {
  			reg = <0>;
  			e3327_s5k5e8_out0: endpoint {
  				port-index = <2>;
  				bus-width = <2>;
  				remote-endpoint = <&e3327_csi_in0>;
  			};
  		};
  	};
  };


  s5k5e8_d@18 {
  	compatible = "nvidia,s5k5e8";
  	/* I2C device address */
  	reg = <0x18>;
  	
  	/* V4L2 device node location */
  	devnode = "video1";

  	/* Physical dimensions of sensor */
  	physical_w = "4.675";
  	physical_h = "3.633";

  	/* Define any required hw resources needed by driver */
  	/* ie. clocks, io pins, power sources */
  	avdd-reg = "vana";
  	iovdd-reg = "vif";
  	dvdd-reg = "vdig";

  	/* Sensor output flip settings */
  	vertical-flip = "false";


  	mode0 { // S5K5E8_MODE_2608X1960
  		mclk_khz = "24000";
  		num_lanes = "2";
  		tegra_sinterface = "serial_d";
  		phy_mode = "DPHY";
  		discontinuous_clk = "no";
  		dpcm_enable = "false";
  		cil_settletime = "0";

  		active_w = "2608";
  		active_h = "1960";
  		mode_type = "bayer";
  		pixel_phase = "grbg";     
  		csi_pixel_bit_depth = "10";   
  		readout_orientation = "90";
  		line_length = "2856";
  		inherent_gain = "1";
  		mclk_multiplier = "7.1";
  		pix_clk_hz = "167200000";//"170000000";

  		gain_factor = "1";
  		min_gain_val = "0";/* 1DB*/
  		max_gain_val = "4";/* 16DB*/
  		step_gain_val = "1";
  		default_gain = "2";
  		min_hdr_ratio = "";
  		max_hdr_ratio = "";
  		framerate_factor = "1000000";
  		min_framerate = "20000000";/*1.816577 */
  		max_framerate = "30000000";/*30*/
  		step_framerate = "1";
  		default_framerate = "30000000";
  		exposure_factor = "1000000";
  		min_exp_time = "34";        //"16000";/* us */
  		max_exp_time = "550385";     //"30088";/* us */
  		step_exp_time = "1";
  		default_exp_time = "33334"; //"30088";/* us */
  		embedded_metadata_height = "0";
  	};

  	mode1 { //S5K5E8_MODE_1304X980
  		mclk_khz = "24000";
  		num_lanes = "2";
  		tegra_sinterface = "serial_d";
  		phy_mode = "DPHY";
  		discontinuous_clk = "no";
  		dpcm_enable = "false";
  		cil_settletime = "0";

  		active_w = "1304";
  		active_h = "980";
  		mode_type = "bayer";
  		pixel_phase = "grbg";  
  		csi_pixel_bit_depth = "10";    
  		readout_orientation = "90";
  		line_length = "2856";
  		inherent_gain = "1";
  		mclk_multiplier = "7.1";
  		pix_clk_hz = "167200000";//"170000000";

  		gain_factor = "1";
  		min_gain_val = "0";/* 1DB*/
  		max_gain_val = "4";/* 16DB*/
  		step_gain_val = "1";
  		default_gain = "2";
  		min_hdr_ratio = "";
  		max_hdr_ratio = "";
  		framerate_factor = "1000000";
  		min_framerate = "20000000";/*1.816577 */
  		max_framerate = "60000000";/*30*/
  		step_framerate = "1";
  		default_framerate = "60000000";
  		exposure_factor = "1000000";
  		min_exp_time = "22";      //"16000";/* us */
  		max_exp_time = "358733";  //"30088";/* us */
  		step_exp_time = "1";
  		default_exp_time ="8334"; //"30088";/* us */
  		embedded_metadata_height = "0";
  	};

  	ports {
  		#address-cells = <1>;
  		#size-cells = <0>;

  		port@0 {
  			reg = <0>;
  			e3327_s5k5e8_out1: endpoint {
  				port-index = <3>;
  				bus-width = <2>;
  				remote-endpoint = <&e3327_csi_in1>;
  			};
  		};
  	};
  };

};

tegra-camera-platform {
compatible = “nvidia, tegra-camera-platform”;
modules {
module0 {
badge = “e3327_front_s5k5e8”;
position = “front”;
orientation = “1”;
drivernode0 {
/* Declare PCL support driver (classically known as guid) /
pcl_id = “v4l2_sensor”;
/
Driver v4l2 device name /
devname = “s5k5e8 2-0010”;
/
Declare the device-tree hierarchy to driver instance */
proc-device-tree = “/proc/device-tree/i2c@3180000/s5k5e8_c@10”;
};
};

  	module1 {
  		badge = "e3327_rear_s5k5e8";
  		position = "rear";
  		orientation = "1";
  		drivernode0 {
  			/* Declare PCL support driver (classically known as guid)  */
  			pcl_id = "v4l2_sensor";
  			/* Driver v4l2 device name */
  			devname = "s5k5e8 1-0018";
  			/* Declare the device-tree hierarchy to driver instance */
  			proc-device-tree = "/proc/device-tree/i2c@3180000/s5k5e8_d@18";
  		};
  	};

  };

};
};

hello jpchae,

could you please have another verification to access with v4l2 standard controls.
please refer to Applications Using V4L2 IOCTL Directly for sample commands, thanks

Hello JerryChang,

as your recommend.
i tested v4l2 standard controls was good working as shown below.

What more can i check?

these are not calibrate lens but raw data images captured.

Hello JerryChang,

Thank you for your advise.

I solved problem.
The cause of the problem is that the devname of the module in the device tree does not match.

devname = “s5k5e8 2-0010”

Looking at the contents of devname, it is expected that the following.
s5k5e8 is sensor type
0010 is i2c address
what is stand for “2” ?

		modules {
			module0 {
				badge = "e3327_front_s5k5e8";
				position = "front";
				orientation = "1";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "s5k5e8 2-0010";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/s5k5e8_c@10";
				};
			};

			module1 {
				badge = "e3327_rear_s5k5e8";
				position = "rear";
				orientation = "1";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "s5k5e8 2-0018";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/s5k5e8_d@18";
				};
			};

		};