Hi,
we are trying to capture video data on a Xavier NX from three cameras connected via MIPI CSI. Each camera has a resolution of 3840x2160 and should operate at 60 fps.
Everything is working so far, we can capture the video data from each individual camera at the full framerate. If I try to capture from two camera simultaneously, it is also working, but when I enable the third camera the framerate for all three cameras is dropping to about 40 fps.
So it looks like there is a limit in the Video Input pipeline of the Jetson module. I could not find any information in the Technical Reference Manual or datasheet which could explain this. Can you please tell me if we are above some hardware limitations or if there is a way to adjust this?
The issue can be reproduced quite easily for example with gstreamer. When running these commands at the same time, it gives an output of 60 fps until the third command starts. Then the fps is dropping to 40.
$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=3840,height=2160,framerate=(fraction)60/1, format=NV12' ! nvvidconv ! fpsdisplaysink video-sink=fakesink --verbose
$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM),width=3840,height=2160,framerate=(fraction)60/1, format=NV12' ! nvvidconv ! fpsdisplaysink video-sink=fakesink --verbose
$ gst-launch-1.0 nvarguscamerasrc sensor-id=2 ! 'video/x-raw(memory:NVMM),width=3840,height=2160,framerate=(fraction)60/1, format=NV12' ! nvvidconv ! fpsdisplaysink video-sink=fakesink --verbose
The same behavior can be observed with a custom application using libargus, which is just capturing the frames and outputs fps.
This is part of the configuration in the device tree:
i2c_mux@70 {
compatible = "nxp,pca9548";
reg = <0x70>;
#address-cells = <1>;
#size-cells = <0>;
vcc-supply = <&p3509_vdd_1v8_cvb>;
vcc_lp = "vcc";
skip_mux_detect = "yes";
force_bus_start = <CAMERA_I2C_MUX_BUS(0)>;
i2c@0 {
cam_imx334_a@37 {
compatible = "framos,imx334";
/* I2C device address */
#address-cells = <1>;
#size-cells = <0>;
reg = <0x37>;
reset-gpios = <&tegra_main_gpio CAM0_RST_L GPIO_ACTIVE_HIGH>;
/* V4L2 device node location */
devnode = "video0";
avdd-reg = "avdd-cam-2v8";
dvdd-reg = "vdd-1v8-cvb";
iovdd-reg = "vdd_sys_en";
/* Physical dimensions of sensor */
physical_w = "15.0";
physical_h = "12.5";
sensor_model = "imx334";
use_sensor_mode_id = "true";
mode0 { //
mclk_khz = "24000";
num_lanes = "4";
tegra_sinterface = "serial_a";
phy_mode = "DPHY";
discontinuous_clk = "no";
cil_settletime = "0";
active_w = "3840";
active_h = "2160";
dynamic_pixel_bit_depth = "10";
csi_pixel_bit_depth = "10";
mode_type = "bayer";
pixel_phase = "rggb";
readout_orientation = "0";
line_length = "5280";
inherent_gain = "1";
mclk_multiplier = "30";
pix_clk_hz = "712800000";
gain_factor = "10";
min_gain_val = "0"; /* 0dB */
max_gain_val = "720"; /* 72dB */
step_gain_val = "3"; /* 0.3 */
default_gain = "0";
min_hdr_ratio = "1";
max_hdr_ratio = "1";
framerate_factor = "1000000";
min_framerate = "1500000"; /* 1.5 */
max_framerate = "60000000"; /* 60 */
step_framerate = "1";
default_framerate= "60000000";
exposure_factor = "1000000";
min_exp_time = "7"; /* us */
max_exp_time = "15000";
step_exp_time = "1";
default_exp_time = "10000";/* us */
embedded_metadata_height = "1";
};
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
imx334_out0: endpoint {
port-index = <0>;
bus-width = <4>;
remote-endpoint = <&imx334_csi_in0>;
};
};
};
};
};
num_csi_lanes = <12>;
max_lane_speed = <2500000>;
min_bits_per_pixel = <10>;
vi_peak_byte_per_pixel = <2>;
vi_bw_margin_pct = <25>;
max_pixel_rate = <800000>;
isp_peak_byte_per_pixel = <5>;
isp_bw_margin_pct = <25>;
I’m not sure if it helps in finding the reason for this, but it looks like running v4l2-ctl
to capture does not have this limitation. So when running the following commands at the same time, the output reports 60 fps for all three streams.
$ v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1000
$ v4l2-ctl -d /dev/video1 --stream-mmap --stream-count=1000
$ v4l2-ctl -d /dev/video2 --stream-mmap --stream-count=1000
Thanks in advance.