Nvarguscamerasrc doesn't show all available camera modes

Hi everyone,

1-2 months ago, I used nvarguscamerasrc in combination with gstreamer and OpenCV to read frames from a camera with 120FPS. It worked great. But the SD-Card died and I set up a new image with maybe a new Jetpack version, I sadly don’t know. With the new image the framerate maximums are not right, as the product page of my camera (Sony 8MP IMX219 Sensor) suggests Frame Rates of: 21fps@8MP, 60fps@1080P, 180fps@720P. The last image had one sensor mode more, which was 720P with 120FPS. I used this one, but now it is gone. Can you please help me find the problem?

I use a Jetson Xavier NX (Developer Kit Version) - Jetpack 4.4.1 [L4T 32.4.4], OpenCV Version 4.5.0, gst-launch-1.0 version 1.14.5 GStreamer 1.14.5

I also tried to use the .so file from this post, but there was no difference: >120 fps with nvarguscamerasrc? (TX2, Jetpack 4.2.1)

Greetings,
Fabio

v4l-ctl -all shows:

    Driver Info (not using libv4l2):
    	Driver name   : tegra-video
    	Card type     : vi-output, imx219 9-0010
    	Bus info      : platform:15c10000.vi:0
    	Driver version: 4.9.140
    	Capabilities  : 0x84200001
    		Video Capture
    		Streaming
    		Extended Pix Format
    		Device Capabilities
    	Device Caps   : 0x04200001
    		Video Capture
    		Streaming
    		Extended Pix Format
    Priority: 2
    Video input : 0 (Camera 0: ok)
    Format Video Capture:
    	Width/Height      : 1280/720
    	Pixel Format      : 'RG10'
    	Field             : None
    	Bytes per Line    : 2560
    	Size Image        : 1843200
    	Colorspace        : sRGB
    	Transfer Function : Default (maps to sRGB)
    	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
    	Quantization      : Default (maps to Full Range)
    	Flags             :

    Camera Controls

                         group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                        sensor_mode 0x009a2008 (int64)  : min=0 max=0 step=0 default=0 value=4 flags=slider
                               gain 0x009a2009 (int64)  : min=0 max=0 step=0 default=0 value=160 flags=slider
                           exposure 0x009a200a (int64)  : min=0 max=0 step=0 default=0 value=79999 flags=slider
                         frame_rate 0x009a200b (int64)  : min=0 max=0 step=0 default=0 value=60000000 flags=slider
               sensor_configuration 0x009a2032 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
             sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
          sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
                        bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=1
                    override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=1
                       height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                         size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
                   write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
           sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
            sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
          sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
                  sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
                   low_latency_mode 0x009a206d (bool)   : default=0 value=0
                   preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                       sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=5 flags=read-only

To reproduce the issue you can run:

$ gst-launch-1.0 nvarguscamerasrc ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 2
   Output Stream W = 1920 H = 1080
   seconds to Run    = 0
   Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

More plugin information:

$ gst-inspect-1.0 nvarguscamerasrc
Factory Details:
  Rank                     primary (256)
  Long-name                NvArgusCameraSrc
  Klass                    Video/Capture
  Description              nVidia ARGUS Camera Source
  Author                   Viranjan Pagar <vpagar@nvidia.com>, Amit Pandya <apandya@nvidia.com>

Plugin Details:
  Name                     nvarguscamerasrc
  Description              nVidia ARGUS Source Component
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvarguscamerasrc.so
  Version                  1.0.0
  License                  Proprietary
  Source module            nvarguscamerasrc
  Binary package           NvARGUSCameraSrc
  Origin URL               http://nvidia.com/

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSrc
                         +----GstNvArgusCameraSrc

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
                 format: { (string)NV12 }
              framerate: [ 0/1, 2147483647/1 ]

Could you roll back to J4.4(r32.4.3) to verify the problem.

Hi Fabio,

Any update? Is this still an issue to support?

Yeah it works with the old version. Will the issue be fixed in the upcoming version?

The 120fps mode was disabled from the 32.4.4 and later release won’t support it due to some internal test failed. However you can enable it from the dts by yourself.

Hi, I have two more questions. The camera also supports 1080P at 60FPS, but this is not in the list of available sensor modes. Am I right with guessing, that this also needs to be enabled from the DTS? Is there even a 1080P 60FPS mode? If so can you give me a hint how to enable it in the DTS because I never succeeded to build the DTS before, but tried it several times. Which files do I have to change?
The second question is, if you have an idea, why the 120FPS 720P mode in the 32.4.3 version shows up but only gives me 60FPS. Is that what you meant by the failed internal testing?

Have a reference to below link to build the kernel Image and dtb.
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/kernel_custom.html#

Yes, it could be the HW limitation to support 120fps with 2 lane configure.

Which files do I have to change in the DTS, and what changes are needed? If you could give me a hint I would be super pleased. And I also think, that the 120FPS is not a hardware limitation, as it worked before with the same hardware, or?

Modify below file.

…/hardware/nvidia/platform/t19x/jakku/kernel-dts/common/tegra194-camera-rbpcv2-imx219.dtsi

Not sure, did verify the 120fps of this sensor before.

I successfully compiled the kernel and changed these files:

diff --git a/common/tegra194-camera-rbpcv2-imx219.dtsi b/common/tegra194-camera-rbpcv2-imx219.dtsi
index 4997943..32b1df8 100644
--- a/common/tegra194-camera-rbpcv2-imx219.dtsi
+++ b/common/tegra194-camera-rbpcv2-imx219.dtsi
@@ -266,7 +266,7 @@

                                        embedded_metadata_height = "2";
                                };
-                               mode2 { /* IMX219_MODE_1920x1080_30FPS */
+                               mode2 { /* IMX219_MODE_1920x1080_60FPS */
                                        mclk_khz = "24000";
                                        num_lanes = "2";
                                        tegra_sinterface = "serial_a";
@@ -294,9 +294,9 @@
                                        min_hdr_ratio = "1";
                                        max_hdr_ratio = "1";
                                        min_framerate = "2000000"; /* 2.0 fps */
-                                       max_framerate = "30000000"; /* 30.0 fps */
+                                       max_framerate = "60000000"; /* 60.0 fps */
                                        step_framerate = "1";
-                                       default_framerate = "30000000"; /* 30.0 fps */
+                                       default_framerate = "60000000"; /* 60.0 fps */
                                        min_exp_time = "13"; /* us */
                                        max_exp_time = "683709"; /* us */
                                        step_exp_time = "1";
@@ -569,7 +569,7 @@

                                        embedded_metadata_height = "2";
                                };
-                               mode2 { /* IMX219_MODE_1920x1080_30FPS */
+                               mode2 { /* IMX219_MODE_1920x1080_60FPS */
                                        mclk_khz = "24000";
                                        num_lanes = "2";
                                        tegra_sinterface = "serial_c";
@@ -597,9 +597,9 @@
                                        min_hdr_ratio = "1";
                                        max_hdr_ratio = "1";
                                        min_framerate = "2000000"; /* 2.0 fps */
-                                       max_framerate = "30000000"; /* 30.0 fps */
+                                       max_framerate = "60000000"; /* 60.0 fps */
                                        step_framerate = "1";
-                                       default_framerate = "30000000"; /* 30.0 fps */
+                                       default_framerate = "60000000"; /* 60.0 fps */
                                        min_exp_time = "13"; /* us */
                                        max_exp_time = "683709"; /* us */
                                        step_exp_time = "1";

But sadly it doesn’t change anything. It still says 30FPS. I rebooted after every try. I tried compiling as described in these pages: https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/kernel_custom.html# and GitHub - jetsonhacks/buildJetsonXavierNXKernel: Build the linux kernel and modules on board the Jetson Xavier NX.

Did you check the context from the /proc/device-tree to confirm your modification?

Now it works. I needed to add my compiled DTB to the extlinux.conf file, which worked.