Gstreamer reports Raspberry Pi camera streaming at 120fps when in reality it is only 60fps

But the 120fps section is already in the new (and old) DTS:

mode5 {
					default_framerate = "120000000";
					step_exp_time = [31 00];
					line_length = "3560";
					active_w = "1280";
					embedded_metadata_height = [32 00];
					pixel_t = "bayer_rggb";
					pix_clk_hz = "169600000";
					dpcm_enable = "false";
					num_lanes = [32 00];
					inherent_gain = [31 00];
					max_gain_val = "170";
					min_hdr_ratio = [31 00];
					discontinuous_clk = "yes";
					readout_orientation = "90";
					min_gain_val = "16";
					max_hdr_ratio = [31 00];
					tegra_sinterface = "serial_a";
					step_gain_val = [31 00];
					default_gain = "16";
					min_framerate = "2000000";
					default_exp_time = "2495";
					max_framerate = "120000000";
					exposure_factor = "1000000";
					step_framerate = [31 00];
					phy_mode = "DPHY";
					mclk_khz = "24000";
					cil_settletime = [30 00];
					gain_factor = "16";
					framerate_factor = "1000000";
					active_h = "720";
					max_exp_time = "683709";
					mclk_multiplier = "9.33";
					min_exp_time = "13";
				};

So how can I add it, if it is already present?

Thanks.

What’s the below command shows.

v4l2-ctl --all
v4l2-ctl --list-formats-ext

1 Like
nano@jetson-nano:~$ v4l2-ctl --all
Driver Info (not using libv4l2):
        Driver name   : tegra-video
        Card type     : vi-output, imx219 6-0010
        Bus info      : platform:54080000.vi:0
        Driver version: 4.9.253
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: no power)
Format Video Capture:
        Width/Height      : 3264/2464
        Pixel Format      : 'RG10'
        Field             : None
        Bytes per Line    : 6528
        Size Image        : 16084992
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             : 

Camera Controls

                     group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                    sensor_mode 0x009a2008 (int64)  : min=0 max=0 step=0 default=0 value=0 flags=slider
                           gain 0x009a2009 (int64)  : min=0 max=0 step=0 default=0 value=16 flags=slider
                       exposure 0x009a200a (int64)  : min=0 max=0 step=0 default=0 value=13 flags=slider
                     frame_rate 0x009a200b (int64)  : min=0 max=0 step=0 default=0 value=2000000 flags=slider
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
               write_isp_format 0x009a2068 (bool)   : default=0 value=0
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=6 flags=read-only
nano@jetson-nano:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'RG10'
        Name        : 10-bit Bayer RGRG/GBGB
                Size: Discrete 3264x2464
                        Interval: Discrete 0.048s (21.000 fps)
                Size: Discrete 3264x1848
                        Interval: Discrete 0.036s (28.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1640x1232
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)

That’s weird everything looks fine. Don’t know why can’t launch gst-launch after replace the dtb

Here is my uImage and dtb in case you’d like to test it.
test.dtb (223.3 KB)
Image-4.9.253-test (33.3 MB)

Also, here is the dmesg text that appears after I attempt gst-launch:

[14455.915414] gk20a 57000000.gpu: Direct firmware load for gm20b/nv_acr_ucode_prod.bin failed with error -2
[14455.925001] gk20a 57000000.gpu: Falling back to user helper
[14455.932857] nvgpu: 57000000.gpu            gm20b_bootstrap_hs_acr:1279 [ERR]  nv_acr_ucode_prod.bin ucode get fail for gm20b
[14455.944095] nvgpu: 57000000.gpu            gk20a_finalize_poweron:328  [ERR]  ACR bootstrap failed
[14456.903021] gk20a 57000000.gpu: Direct firmware load for gm20b/nv_acr_ucode_prod.bin failed with error -2
[14456.912882] gk20a 57000000.gpu: Falling back to user helper
[14456.920442] nvgpu: 57000000.gpu            gm20b_bootstrap_hs_acr:1279 [ERR]  nv_acr_ucode_prod.bin ucode get fail for gm20b
[14456.931786] nvgpu: 57000000.gpu            gk20a_finalize_poweron:328  [ERR]  ACR bootstrap failed
[14456.947785] gk20a 57000000.gpu: Direct firmware load for gm20b/nv_acr_ucode_prod.bin failed with error -2
[14456.957665] gk20a 57000000.gpu: Falling back to user helper
[14456.965063] nvgpu: 57000000.gpu            gm20b_bootstrap_hs_acr:1279 [ERR]  nv_acr_ucode_prod.bin ucode get fail for gm20b
[14456.976498] nvgpu: 57000000.gpu            gk20a_finalize_poweron:328  [ERR]  ACR bootstrap failed

What’s the “uname -a” showing?

nano@jetson-nano:~$ uname -a
Linux jetson-nano 4.9.253-test #1 SMP PREEMPT Wed Jun 22 21:54:13 BST 2022 aarch64 aarch64 aarch64 GNU/Linux

I attach the only file in the kernel source that I changed (as a result of one of the two patches - as I say, the other patch had no effect):

imx219_mode_tbls.h (8.9 KB)

I believe that you now have a replica of my setup. And I assume you have not got 120fps working on your Nano?

For completeness, I list the steps I made to compile this kernel:

1. Log in here: https://developer.nvidia.com/embedded/downloads
2. Check the “Jetson Nano” filter checkbox and follow the link to the latest “Jetson Linux Driver Package (L4T)” package.
3. Download “L4T Driver Package (BSP) Sources” and extract the downloaded “public sources.tbz2” file.
4. With the extracted file the file kernel_src.tbz2 should be extracted which will create a new folder named “kernel_src”.
5. Apply imx219_mode_tbls.h.patch (other patch, even after modifying for tegra210, has no effect): 

patch -p0 < imx219_mode_tbls.h.patch

6. Do the following to build the kernel:
export L4TK=$PWD
export SRC=$L4TK/kernel/kernel-4.9
export STAGE=$L4TK/build
rm -rf ${STAGE}
mkdir ${STAGE}
export TEGRA_KERNEL_OUT=${STAGE}/kernel
mkdir $TEGRA_KERNEL_OUT
export TEGRA_MODULES_OUT=${STAGE}/modules
mkdir $TEGRA_MODULES_OUT
sudo /usr/bin/jetson_clocks
cd $SRC
export LOCALVERSION=-test
make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfig
make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j6
make ARCH=arm64 O=$TEGRA_KERNEL_OUT modules_install INSTALL_MOD_PATH=${TEGRA_MODULES_OUT}/
sudo rm -rf /lib/modules/4.9.253${LOCALVERSION}
sudo cp -R ${TEGRA_MODULES_OUT}/lib/modules/4.9.253${LOCALVERSION} /lib/modules
sudo cp $TEGRA_KERNEL_OUT/arch/arm64/boot/Image /boot/Image-4.9.253${LOCALVERSION}
7. Find out which device tree is being used on the current board, then use this to copy the appropriate DTB file into place (tegra210-p3448-0002-p3449-0000-a02.dtb does NOT work):
cat /proc/device-tree/nvidia,dtsfilename
sudo cp ${L4TK}/build/kernel/arch/arm64/boot/dts/tegra210-p3448-0000-p3449-0000-a02.dtb /boot/test.dtb
8.  As root, create a new entry in /boot/extlinux/extlinux.conf, first copying your working default entry and in new entry modifying LABEL, MENU, LINUX for the new kernel image, adding FDT line for device tree, keeping INITRD and APPEND unchanged such as:
LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image-4.9.253-test
      FDT /boot/test.dtb
      INITRD /boot/initrd
      APPEND ${cbootargs etc etc etc

Have “export LOCALVERSION=-tegra” instead of " “export LOCALVERSION=-test” for the module loaded failed.

But if I do that it will overwrite my working official Nvidia setup of current modules.

Do you want me to change extlinux.conf too?

Please could you explain more what you mean. Am I booting the official Nvidia kernel with newly-built modules? Thank you.

If you change the LOCALVERSION that will cause some module like GPU … unable loaded due to version check, that could cause unknow problem.

1 Like

I tried this just now. I have this in my extlinux.conf:

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image-4.9.253-tegra
      FDT /boot/tegra.dtb
      INITRD /boot/initrd
      APPEND ${cbootargs} etc etc etc

It actually appears worse than before because I now see this repeating error in dmesg:

[   63.755079] gk20a 57000000.gpu: Direct firmware load for gm20b/nv_acr_ucode_prod.bin failed with error -2
[   63.764939] gk20a 57000000.gpu: Falling back to user helper
[   63.773329] nvgpu: 57000000.gpu            gm20b_bootstrap_hs_acr:1279 [ERR]  nv_acr_ucode_prod.bin ucode get fail for gm20b
[   63.784605] nvgpu: 57000000.gpu            gk20a_finalize_poweron:328  [ERR]  ACR bootstrap failed
[   63.794960] tegradc tegradc.0: blank - powerdown

I think the last line is different to previous dmesg errors. gstreamer does not work still. Here is my /lib/modules directory:


nano@jetson-nano:/boot$ ls -l /lib/modules
total 12
drwxr-xr-x 2 root root 4096 Sep  6  2021 4.9.140-tegra
drwxr-xr-x 3 root root 4096 Jul  7 11:51 4.9.253-tegra
drwxr-xr-x 3 root root 4096 Jun 22 22:35 4.9.253-test

If it is possibly a naming problem, could it be because the official Nvidia kernel modules is called “4.9.140-tegra” but when rebuilt it is called “4.9.253-tegra”???

It could be your source version didn’t match.
Check your version by below command. Looks likes your kernel source is r32.6.1 but BSP is not.

cat /etc/nv_tegra_release

1 Like

Here we go:

nano@jetson-nano:~$ cat /etc/nv_tegra_release 
# R32 (release), REVISION: 6.1, GCID: 27863751, BOARD: t210ref, EABI: aarch64, DATE: Mon Jul 26 19:20:30 UTC 2021
nano@jetson-nano:~$ uname -a
Linux jetson-nano 4.9.253-tegra #1 SMP PREEMPT Thu Jul 7 11:29:10 BST 2022 aarch64 aarch64 aarch64 GNU/Linux

This version is match. The 4.9.140-tegra is your customized kernel?

1 Like

“4.9.140-tegra” is the official Nvidia kernel. You can see the date is September 6th 2021.

My kernel that I rebuilt today (with “-test” replaced by “-tegra”) is called “4.9.253-tegra”

Really my device is r32.6.1 too.

Linux nvidia-desktop 4.9.253-tegra #1 SMP PREEMPT Mon Jul 26 12:19:28 PDT 2021 aarch64 aarch64 aarch64 GNU/Linux
nvidia@nvidia-desktop:~$ cat /etc/nv_tegra_release
# R32 (release), REVISION: 6.1, GCID: 27863751, BOARD: t186ref, EABI: aarch64, DATE: Mon Jul 26 19:36:31 UTC 2021

1 Like

Yes, but you have a Jetson Xavier.

This thread is about the Jetson Nano.

@jetsonnvidia

t186 should be TX2, that shares t194 Xavier SOC kernel sources. For TX1/Nano, t210 may have a different kernel source tree.

This seems weird. You may find where the video modes are defined for Nano IMX219 driver (you may grep Nano kernel sources for IMX219_MODE_1920x1080_30FPS or imx219_mode_1920x1080_30fps and if found adapt the patch for Nano.

@ShaneCCC ,
May you further explain the kernel local version impact on GPU driver loading ?

1 Like

The GPU driver will check the kernel version to load it.
Please check the DATE of the nv_tegra_release and uname -a