V4l2 on Tegra X1 or X2

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX1
1

I found it too difficult to get bayer raw with v4l2(mmap) on TX1(TX2), I just wanna get 10 bit raw datas from cameras, and the v4l2-ctl also cannot finish this work.
The only way to get raw data is nvgstcapture, and it is 14 bit bayer raw, and we cannot get many raws at a time.
If I use v4l2-ctl, it got the same problem as this topic:
https://devtalk.nvidia.com/default/topic/1032388/jetson-tx1/why-use-quot-v4l2-ctl-quot-command-get-raw-data-is-alway-zero-at-jetson-tx1-r28-2-/
And it cannot set exposure:

ubuntu@tegra-ubuntu:~$ v4l2-ctl  --list-ctrls

Camera Controls

                   frame_length (int)    : min=1154 max=92307 step=1 default=4562 value=4562 flags=slider
                    coarse_time (int)    : min=1 max=92295 step=1 default=4550 value=4550 flags=slider
              coarse_time_short (int)    : min=1 max=92295 step=1 default=4550 value=4550 flags=slider
                     group_hold (intmenu): min=0 max=1 default=0 value=0
                     hdr_enable (intmenu): min=0 max=1 default=0 value=0
                           gain (int)    : min=1 max=22 step=1 default=1 value=1 flags=slider
                    bypass_mode (intmenu): min=0 max=1 default=0 value=0
                override_enable (intmenu): min=0 max=1 default=0 value=0
                   height_align (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align (intmenu): min=0 max=2 default=0 value=0
               write_isp_format (bool)   : default=0 value=0

If I use ffmpeg to finish it with v4l2, it shows:

ubuntu@tegra-ubuntu:~$ ffmpeg -hide_banner -f v4l2 -list_formats all -i /dev/video0 
[video4linux2,v4l2 @ 0x4c6500] Raw       : bayer_rggb8 : 8-bit Bayer RGRG/GBGB : 3864x2174 1932x1094 1288x734 1288x546 3864x2202
[video4linux2,v4l2 @ 0x4c6500] Raw       : Unsupported : 10-bit Bayer RGRG/GBGB : 3864x2174 1932x1094 1288x734 1288x546 3864x2202
[video4linux2,v4l2 @ 0x4c6500] Raw       : Unsupported : 10-bit Bayer BGBG/GRGR : 3864x2174 1932x1094 1288x734 1288x546 3864x2202
[video4linux2,v4l2 @ 0x4c6500] Raw       : Unsupported : 10-bit Bayer BGGR(10-10-10-2) :
[video4linux2,v4l2 @ 0x4c6500] Raw       : Unsupported : 10-bit Bayer RGGB(10-10-10-2) : 3864x2174 1932x1094 1288x734 1288x546 3864x2202
[video4linux2,v4l2 @ 0x4c6500] Raw       : Unsupported : 12-bit Bayer RGRG/GBGB : 3864x2174 1932x1094 1288x734 1288x546 3864x2202
/dev/video0: Immediate exit requested

but WHEN I trying to get 8 bit bayer, it shows:

[video4linux2,v4l2 @ 0x4c6580] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
/dev/video0: Inappropriate ioctl for device

So, how to get some 10 bit raw datas?

2

Check the sensor cap by v4l2-ctl --list-formats-ext to make sure it support 10 bit.
I don’t think you can get the raw data by nvgstcapture! How?
Any kernel error message while use v4l2-ctl to dump data? TX1 or TX2?

3

You may also read this topic

4

It is the v4l2-ctl --list-formats-ext shows,

ioctl: VIDIOC_ENUM_FMT                                                                                                                                                                                                                                        
        Index       : 0                                                                                                                                                                                                                                       
        Type        : Video Capture                                                                                                                                                                                                                           
        Pixel Format: 'RGGB'                                                                                                                                                                                                                                  
        Name        : 8-bit Bayer RGRG/GBGB                                                                                                                                                                                                                   
                Size: Discrete 3864x2174                                                                                                                                                                                                                      
                        Interval: Discrete 0.017s (60.000 fps)                                                                                                                                                                                                
                Size: Discrete 1932x1094                                                                                                                                                                                                                      
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x734                                                                                                                                                                                                                       
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x546                                                                                                                                                                                                                       
                        Interval: Discrete 0.004s (240.000 fps)                                                                                                                                                                                               
                Size: Discrete 3864x2202                                                                                                                                                                                                                      
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                
                                                                                                                                                                                                                                                              
        Index       : 1                                                                                                                                                                                                                                       
        Type        : Video Capture                                                                                                                                                                                                                           
        Pixel Format: 'RG10'                                                                                                                                                                                                                                  
        Name        : 10-bit Bayer RGRG/GBGB                                                                                                                                                                                                                  
                Size: Discrete 3864x2174                                                                                                                                                                                                                      
                        Interval: Discrete 0.017s (60.000 fps)                                                                                                                                                                                                
                Size: Discrete 1932x1094                                                                                                                                                                                                                      
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x734                                                                                                                                                                                                                       
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x546                                                                                                                                                                                                                       
                        Interval: Discrete 0.004s (240.000 fps)                                                                                                                                                                                               
                Size: Discrete 3864x2202                                                                                                                                                                                                                      
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                
                                                                                                                                                                                                                                                              
        Index       : 2                                                                                                                                                                                                                                       
        Type        : Video Capture                                                                                                                                                                                                                           
        Pixel Format: 'BG10'                                                                                                                                                                                                                                  
        Name        : 10-bit Bayer BGBG/GRGR                                                                                                                                                                                                                  
                Size: Discrete 3864x2174                                                                                                                                                                                                                      
                        Interval: Discrete 0.017s (60.000 fps)                                                                                                                                                                                                
                Size: Discrete 1932x1094                                                                                                                                                                                                                      
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x734                                                                                                                                                                                                                       
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x546                                                                                                                                                                                                                       
                        Interval: Discrete 0.004s (240.000 fps)                                                                                                                                                                                               
                Size: Discrete 3864x2202                                                                                                                                                                                                                      
                        Interval: Discrete 0.033s (30.000 fps)                                                                                                                                                                                                
                                                                                                                                                                                                                                                              
        Index       : 3                                                                                                                                                                                                                                       
        Type        : Video Capture                                                                                                                                                                                                                           
        Pixel Format: 'XBGP'                                                                                                                                                                                                                                  
        Name        : 10-bit Bayer BGGR(10-10-10-2)                                                                                                                                                                                                           
                                                                                                                                                                                                                                                              
        Index       : 4                                                                                                                                                                                                                                       
        Type        : Video Capture                                                                                                                                                                                                                           
        Pixel Format: 'XRGP'                                                                                                                                                                                                                                  
        Name        : 10-bit Bayer RGGB(10-10-10-2)                                                                                                                                                                                                           
                Size: Discrete 3864x2174                                                                                                                                                                                                                      
                        Interval: Discrete 0.017s (60.000 fps)                                                                                                                                                                                                
                Size: Discrete 1932x1094                                                                                                                                                                                                                      
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x734                                                                                                                                                                                                                       
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x546                                                                                                                                                                                                                       
                        Interval: Discrete 0.004s (240.000 fps)                                                                                                                                                                                               
                Size: Discrete 3864x2202                                                                                                                                                                                                                      
                        Interval: Discrete 0.033s (30.000 fps)     

Index       : 5                                                                                                                                                                                                                                       
        Type        : Video Capture                                                                                                                                                                                                                           
        Pixel Format: 'RG12'                                                                                                                                                                                                                                  
        Name        : 12-bit Bayer RGRG/GBGB                                                                                                                                                                                                                  
                Size: Discrete 3864x2174                                                                                                                                                                                                                      
                        Interval: Discrete 0.017s (60.000 fps)                                                                                                                                                                                                
                Size: Discrete 1932x1094                                                                                                                                                                                                                      
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x734                                                                                                                                                                                                                       
                        Interval: Discrete 0.008s (120.000 fps)                                                                                                                                                                                               
                Size: Discrete 1288x546                                                                                                                                                                                                                       
                        Interval: Discrete 0.004s (240.000 fps)                                                                                                                                                                                               
                Size: Discrete 3864x2202                                                                                                                                                                                                                      
                        Interval: Discrete 0.033s (30.000 fps)

It looks normal, but the raw data is unnormal.
I tried it both on TX1 and TX2.

I use v4l2-ctl, it got the same problem as this topic:

I can get raw data by nvgstcapture :

nvgstcapture-1.0 -m 1 --sensor-id 0 -A --capture-auto 1 --dump-bayer --file-name L0 --image-res 8

and this raw data is normal

5

I think you are not upgrade to latest BSP 28.2.1
The v4l2-ctl --list-formats-ext show the wrong support at early BSP. I believe your sensor driver only support one type of pixel format.

6

I have upgraded it to BSP 28.2.1.(TX2)
The v4l2-ctl --list-formats-ext shows:

ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'RG10'
        Name        : 10-bit Bayer RGRG/GBGB
                Size: Discrete 3840x2160
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1280x540
                        Interval: Discrete 0.004s (240.000 fps)

Yes, it seems only support a type of pixel format.
The nvgscatpure-1.0 cannot get raw data now.
And v4l2-ctl also cannot get correct 10 bit raw data.

7

Check below link to get the trace log to know the root cause.

https://elinux.org/Jetson_TX2_Camera_BringUp

8

I’m running into a problem with reading raw data through v4l2-ctl.
My problem is that the raw data files created show 14 bits of information in the 16 bit data type saved. I get no problems when reading data and can max out my camera’s throughput at 64fps with no errors for long durations (8 hours or more). The data from the camera is sent as 10 bits with 4 packets of 8 bits followed by one 8 bit packet of 2 bits for each of the previous 4 8 bit packages. This last packet is LSBs. I’m guessing this is standard across Sony imx cameras.
Using the Nvargus/gstreamer pipeline the images look okay. I’m more worried about data being corrupted or changed.

I’m seeing this across two different cameras a IMX412 and IMX351.

Here is the v4l2-ctl argument:
v4l2-ctl --set-fmt-video=width=4656,height=1744,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=…/test.raw

To be clear if I crop the outputted data to 10 bits I get an image that is close to what I’m expecting, but again it would be almost the same with the first 8 bits as the last two are LSB’s.
This might be as simple as the data is only overwriting the 10 bits of useful information of the 16 bit packet, but getting some confirmation of how nvargus/v4l2-ctl handles 10 bit incoming data would be useful.

9

Could you check the TX2 TRM “27.10.6 RAW Memory Formats”

10

Where is this doc? All I have is the TLDP development guide.

After looking into this more with the help of a colleague, it looks like the raw data can be extracted by shifting it over right 4 bits. Which is what I was doing but what was weird is the left 2 bits are zeros and the last 4 bits are the first 4 bits of data repeated.

11

You can download the TRM from the download center.

https://developer.nvidia.com/embedded/downloads#