stride alignment problem when capture 720x288@50 video with csi

Hi,all

We are running into a problem when capture video of 720x288@50 with csi.

We found that when we capture video with width of 720 pixel (1440 bytes), the value of temp_bpl in driver roundup (function) to 1472, then bytesperline is also changed to 1472 bytes.

That means when we call driver of v4l2 with app ,the amount of bytes of pixel data of one frame is not 7202882
but 7362882. So the picture we captured is abnormal because of wrong stride.

And the additional 32 bytes appended by driver of per line is all zero.

Then we have tried to change TEGRA_STRIDE_ALIGNMNET ( default 64 ) to 32. Now the bytesperline is ok.

But the frame captured is still not normal.

So can TEGRA_STRIDE_ALIGNMNET be set to 32 ? Does the hardware has some restrictions?

If it can be set to 32, is there anything we are missing to capture normal frame with resolution of 720x288.

Thanks.

Did you use v4l2-ctl to capture?
The TEGRA_STRIDE_ALIGNMNET shouldn’t the root cause of the capture failed, it’s only for the memory alignment.
Any CSI error message?

Hi, ShaneCCC

There is no error message from debug serial port.

We didn't use v4l2-ctl to capture, but v4l2 demo from v4l2 official website.

We also find that if we set fmt.width with 736 in v4l2 app, we can see normal frame but it has interval frame displayed green on screen(no data) with fixed frequency. It may be caused by mismatch for tc358748 register value.

When we set  TEGRA_STRIDE_ALIGNMNET to 32 ,capture with arguments 720 288, it also display with abnormal frame which is similar with capture with improper arguments width.  

And the green bar is on bottom of the frame, seems stride is right.We try to save a frame data with fwrite but can't see hex value by ultraedite. When we change TEGRA_STRIDE_ALIGNMNET to 64, we can see hex value.

I feel there is also another factor in driver limiting some change.

Thanks.

Can you have a video clip to show it.
Can you try if any error for below command.

v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=30

There is no error with the above commandline.

Below is debug serial port:

5] v4l2_open entry                                                                                                   
[ 1381.952362] tegra_channel_open entry                                                                                          
[ 1381.955968] tegra_channel_set_format entry  the pix=1448695129,the width = 720  the hdight = 288  the maxframerate 0          
[ 1381.966630] tegra_channel_fmt_align  bpp = 2  width = 720  height = 288   bytesperline = 0                                    
[ 1381.975290] tegra_channel_fmt_align  bytesperline = 1440                                                                      
[ 1381.980652] tegra_channel_fmt_align  align = 2  min_width = 32  max_width = 32768   temp_width = 1440  width = 720  height = 8
[ 1381.992395] tegra_channel_fmt_align  min_bpl = 1440  max_bpl = 32768  temp_bpl = 1472   bytesperline = 1472  stride_align = 64
[ 1382.003870] tc358748_set_fmt:entry,format->which  0                                                                           
[ 1382.008746] camera_common_try_fmt entry                                                                                       
[ 1382.012599] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.020353] camera_common_try_fmt: s_data :width 4096 height 3072 maxframerate 25 hdr_en 0                                    
[ 1382.028692] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.036441] camera_common_try_fmt: s_data :width 1920 height 1080 maxframerate 30 hdr_en 0                                    
[ 1382.044797] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.052546] camera_common_try_fmt: s_data :width 1280 height 720 maxframerate 50 hdr_en 0                                     
[ 1382.061011] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.068752] camera_common_try_fmt: s_data :width 1280 height 720 maxframerate 60 hdr_en 0                                     
[ 1382.077029] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.084779] camera_common_try_fmt: s_data :width 1920 height 1080 maxframerate 25 hdr_en 0                                    
[ 1382.093137] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.100889] camera_common_try_fmt: s_data :width 720 height 288 maxframerate 50 hdr_en 0                                      
[ 1382.109072]                                                                                                                   
[ 1382.109072] camera_common_try_fmt: mf->code8200                                                                               
[ 1382.115074] tegra_channel_set_format entry  the ret=0 format->fmt.pix.maxframerate 0                                          
[ 1382.122821] __tegra_channel_set_format entry                                                                                  
[ 1382.127087] __tegra_channel_set_format code=8200 width=16 vf_code=12 img_dt=30 img_fmt=200                                    
[ 1382.135357] tc358748_set_fmt:entry,format->which  1                                                                           
[ 1382.140247] camera_common_s_fmt entry                                                                                         
[ 1382.143911] tc358748_mipi 6-000e.1: camera_common_s_fmt(8200) size 720 x 288                                                  
[ 1382.150968] camera_common_try_fmt entry                                                                                       
[ 1382.154805] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.162555] camera_common_try_fmt: s_data :width 4096 height 3072 maxframerate 25 hdr_en 0                                    
[ 1382.170913] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.178645] camera_common_try_fmt: s_data :width 1920 height 1080 maxframerate 30 hdr_en 0                                    
[ 1382.187000] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.194751] camera_common_try_fmt: s_data :width 1280 height 720 maxframerate 50 hdr_en 0                                     
[ 1382.203019] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.210768] camera_common_try_fmt: s_data :width 1280 height 720 maxframerate 60 hdr_en 0                                     
[ 1382.219034] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.226766] camera_common_try_fmt: s_data :width 1920 height 1080 maxframerate 25 hdr_en 0                                    
[ 1382.235121] camera_common_try_fmt: mf :width 720 height 288 maxframerate 0 hdr_en 0                                           
[ 1382.242867] camera_common_try_fmt: s_data :width 720 height 288 maxframerate 50 hdr_en 0                                      
[ 1382.251046]                                                                                                                   
[ 1382.251046] camera_common_try_fmt: mf->code8200                                                                               
[ 1382.257049] camera_common_s_fmt  the colorfm code=0x2008 space=0x2 pix=0x56595559                           
[ 1382.270550] v4l2_mmap entry                                                                                                   
[ 1382.273346] vb2_fop_mmap entry                                                                                                
[ 1382.276426] v4l2_mmap entry                                                                                                   
[ 1382.279237] vb2_fop_mmap entry                                                                                                
[ 1382.282317] v4l2_mmap entry                                                                                                   
[ 1382.285110] vb2_fop_mmap entry                                                                                                
[ 1382.288188] v4l2_mmap entry                                                                                                   
[ 1382.290999] vb2_fop_mmap entry                                                                                                
[ 1382.294084] tegra_channel_start_streaming entry                                                                               
[ 1382.298609] tegra_channel_ec_init:422 =========entry=========                                                                 
[ 1382.304648] tegra_channel_start_streaming complete                                                                            
[ 1382.304760]  csi_port = 2 mode = 5 fmt_wdth = 720 fmt_height = 288                            
[ 1382.304763]  csi_port = 2 mode = 5 fmt_wdth = 720 fmt_height = 288
[ 1382.304764] tc358748_s_stream  stream on                                                                                      
[ 1382.304765] tc358748_s_stream++                                                                                               
[ 1382.861834] tc358748_init_config: gpio-CSI[38]=0                                                                              
[ 1382.866454] HD mode[4]                                                                                                        
[ 1382.868813] *******************************************************                                                           
[ 1382.875092] *******************************************************                                                           
[ 1382.881425] *******************************************************                                                           
[ 1382.909272] tc358748_s_stream--                                                                                               
[ 1387.021524]  csi_port = 2 mode = 5 fmt_wdth = 720 fmt_height = 288                           
[ 1387.031721]  csi_port = 2 mode = 5 fmt_wdth = 720 fmt_height = 288   
[ 1387.041733] stream exit.

Attached is screenshot.

TEGRA_STRIDE_ALIGNMENT_64.png

If the v4l2-ctl show no errors that means csi/vi capture frame without problem. Could you try use v4l2-ctl to dump the data to check if the frame is normal.
Do you use v4l2 app? What’s the sensor output color format?
It’s could be the color format incorrect.

v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=1.raw

Hi, ShaneCCC.

We use v4l2 for video capture. We set colorfmt to V4L2_MBUS_FMT_YUYV8_2X8 in driver.

Below two file is data dumped with  "v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=1.raw"

raw_data.tar.gz (409 KB)

It is strange that we can read hex value from TEGRA_STRIDE_ALIGNMENT_64.raw by software ultraedit which can show file with hex format.

But show TEGRA_STRIDE_ALIGNMENT_32.raw with messy code.

Try to set alignment as 720’s multiple like 16.
Does the tc358748 output YUV422? Could it be MEDIA_BUS_FMT_UYVY8_1X16?

The tc358748 output is YUV422.

When set TEGRA_STRIDE_ALIGNMENT to 16. The result is similar with TEGRA_STRIDE_ALIGNMENT =32.

When I try to add MEDIA_BUS_FMT_YUYV8_1X16 in driver

The debug serial port is stuck in below

7.204835] usb-vbus1: disabling                                                                                              
[    7.208107] vdd-disp-3v0: disabling                                                                                           
[    7.211638] snsr_pm: disabling                                                                                                
[    7.214745] aux-3v3: disabling                                                                                                
[    7.217840] en-vdd-sd: disabling                                                                                              
[    7.225989] lcd-bl-en: disabling                                                                                              
[    7.229263] max77620-gpio7: disabling                                                                                         
[    7.244562] vdd-cpu-fixed: disabling                                                                                          
[    7.248186] regulator-dummy: disabling                                                                                        
[    7.252251] reg-fixed-sync-voltage 5.regulator: Consumer c1 does not have device name                                         
[    7.260283] platform 5.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                       
[    7.268749] platform 7.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                       
[    7.277061] platform d.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                       
[    7.285385] platform c9.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                      
[    7.293960] platform ca.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                      
[    7.302394] platform cb.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                      
[    7.310775] platform cc.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                      
[    7.319171] platform cd.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                      
[    7.327643] reg-fixed-sync-voltage ce.regulator: Consumer c0 does not have device name                                        
[    7.335801] platform ce.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                      
[    7.340280] tegradc tegradc.1: vrr_setup failed

TEGRA_STRIDE_ALIGNMENT_16.tar.gz (180 KB)

Our driver version is 24-2-1.

Check below topic it could be help for your case.

https://devtalk.nvidia.com/default/topic/946840

Hi,ShaneCCC

Bother to you again.

I add

{
    	V4L2_MBUS_FMT_YUYV8_1X16,
    	V4L2_COLORSPACE_SMPTE240M,
    	V4L2_PIX_FMT_YUYV,
    },
if (mf->code != V4L2_MBUS_FMT_SRGGB8_1X8 &&
		mf->code != V4L2_MBUS_FMT_SRGGB10_1X10 &&
		mf->code != V4L2_MBUS_FMT_YUYV8_2X8 &&
		mf->code != V4L2_MBUS_FMT_Y8_1X8 &&
		mf->code != V4L2_MBUS_FMT_RGB888_1X24 &&
		mf->code != V4L2_MBUS_FMT_UYVY8_2X8 &&
                <b>mf->code != V4L2_MBUS_FMT_YUYV8_1X16</b>) {
		mf->code = V4L2_MBUS_FMT_SRGGB10_1X10;
		err = -EINVAL;
	}
else if(mf->code == V4L2_MBUS_FMT_YUYV8_1X16)
	{
		mf->field = V4L2_FIELD_ANY;
		mf->colorspace = V4L2_COLORSPACE_SMPTE240M;
	}

and use it by initial

common_data->colorfmt       = camera_common_find_datafmt(V4L2_MBUS_FMT_YUYV8_1X16);
But after I flash them to system, the Tx1 can't boot and stuck below 
[    6.694142] usb usb3: Manufacturer: Linux 3.10.96+ ehci_hcd                                                                      
[    6.699720] usb usb3: SerialNumber: tegra-ehci.0                                                                                 
[    6.704692] hub 3-0:1.0: USB hub found                                                                                           
[    6.708451] hub 3-0:1.0: 1 port detected                                                                                         
[    6.768736] otg state changed: HOST --> SUSPEND                                                                                  
[    6.818747] tegra-ehci tegra-ehci.0: remove, state 1                                                                             
[    6.823718] usb usb3: USB disconnect, device number 1                                                                            
[    6.829501] tegra-ehci tegra-ehci.0: USB bus 3 deregistered                                                                      
[    6.948898] tegradc tegradc.1: probed                                                                                            
[    7.035772] tegradc tegradc.1: _tegra_dc_set_mode():MODE:3840x2160@60.005Hz pclk=533333000                                       
[    7.066421] tegradc tegradc.1: tegra_dc_program_mode():MODE:3840x2160@60.005Hz pclk=533333000                                    
[    7.066446] nominal-pclk:type[1] 533333000 parent:533278125 div:1.0 pclk:533278125 527999670~581332970                           
[    7.116350] Console: switching to colour frame buffer device 480x135                                                             
[    7.212522] tegradc tegradc.1: fb registered                                                                                     
[    7.217329] vdd_gpu: disabling                                                                                                   
[    7.220473] en-vdd-vcm-2v8: disabling                                                                                            
[    7.224195] usb-vbus3: disabling                                                                                                 
[    7.227465] usb-vbus1: disabling                                                                                                 
[    7.230745] vdd-disp-3v0: disabling                                                                                              
[    7.234281] snsr_pm: disabling                                                                                                   
[    7.237374] aux-3v3: disabling                                                                                                   
[    7.240504] en-vdd-sd: disabling                                                                                                 
[    7.248655] lcd-bl-en: disabling                                                                                                 
[    7.251937] max77620-gpio7: disabling                                                                                            
[    7.267219] vdd-cpu-fixed: disabling                                                                                             
[    7.270859] regulator-dummy: disabling                                                                                           
[    7.274897] reg-fixed-sync-voltage 5.regulator: Consumer c1 does not have device name                                            
[    7.282959] platform 5.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                          
[    7.291404] platform 7.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                          
[    7.299745] platform d.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                          
[    7.308028] platform c9.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.316608] platform ca.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.325043] platform cb.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.333440] platform cc.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.341857] platform cd.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.350433] reg-fixed-sync-voltage ce.regulator: Consumer c0 does not have device name                                           
[    7.358496] platform ce.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.366915] platform d1.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.375313] platform d3.regulator: Driver reg-fixed-sync-voltage requests probe deferral                                         
[    7.383653] tegradc tegradc.1: vrr_setup failed

And we can’t figure out vrr_setup failed. In kernel “vrr_setup failed” is concerned with hdmi.

We can’t find connection between this capture and this display.

We have reflashed the system several times and cp Image zImage dts to it. All can’t boot with above debug serial message.

Hi, ShaneCCC

after unpluging hdmi.The debug serial port printed error stack.

We find tegra_video_formats has no V4L2_MBUS_FMT_YUYV8_1X16 member.

So we switch to V4L2_MBUS_FMT_UYVY8_1X16.

But the result is same with V4L2_MBUS_FMT_YUYV8_2x8.

If the pixel format is correct the problem could be the device output have problem due to there’s no problem for CSI/VI capture.

Yes, and we also want to know why we can’t read hex value from TEGRA_STRIDE_ALIGNMENT_32.raw.

Would you mind sharing some opinion with us.

Thanks.

What do you mean can’t read hex value?
I can read it by Notepad++ normally.

Thank you.

It can be read with hex format by Notepad++.