Hello,
We are bring-up our csi camera on AGX Orin Developer Kit.
When we select YUV422 format, it works.
stereo@stereo-desktop:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
stereo@stereo-desktop:~$ sudo dmesg | grep csi
[ 9.516564] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: context isolation disabled due to no IOMMU
[ 9.525968] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: initialized
[ 9.532408] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e40000.host1x:nvcsi@15a00000- bound
[ 16.267772] OF: graph: no port node found in /i2c@c240000/ucsi_ccg@8/connector@0
[ 16.275653] ucsi_ccg 1-0008: Port-0: no role switch found
[ 16.302320] ucsi_ccg 1-0008: request_threaded_irq failed - -5
[ 16.309277] ucsi_ccg: probe of 1-0008 failed with error -5
[ 2109.219514] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[ 2109.237931] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
[ 2111.811599] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[ 2111.829994] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
But the NV16 format has a issue.
The catpure image is just green and UV data fills in ‘0’ values.
stereo@stereo-desktop:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[ 9.497281] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: context isolation disabled due to no IOMMU
[ 9.506849] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: initialized
[ 9.513319] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e40000.host1x:nvcsi@15a00000- bound
[ 15.958773] OF: graph: no port node found in /i2c@c240000/ucsi_ccg@8/connector@0
[ 15.966469] ucsi_ccg 1-0008: Port-0: no role switch found
[ 15.993299] ucsi_ccg 1-0008: request_threaded_irq failed - -5
[ 15.999882] ucsi_ccg: probe of 1-0008 failed with error -5
We prefer NV16 Multi-planar format for the work to extract Y data.
hello sumin.lee1,
may I know which sensor module it is. what’s the default output sensor format types?
NV16 is the YUV422 in semi-planar, they’re all 16-bits.
The mipi csi is connected with external ISP.
The ISP outputs a frame with NV16.
stereo@stereo-desktop:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'NV16' (Y/CbCr 4:2:2)
But, the supported format is changed with below code.
(the format is updated as YUYV by tegra_channel_fmts_bitmap_init() in channel.c)
Linux_for_Tegra/source/public/kernel/nvidia/drivers/media/platform/tegra/camera/camera_common.c
{ //for NV16 UYVY8_1X16
+ MEDIA_BUS_FMT_YUYV8_2X8, /*yuyv*/
- MEDIA_BUS_FMT_UYVY8_1X16, /*nv16*/
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_NV16,/*nv16*/
},
Anyway, if the supported format is ‘YUYV’, the dequeued buffer is transferred well.
But if the format is ‘NV16’, UV data area of the dequeued buffer is filled with ‘0’ values.
hello yt.jeon,
that’s the media bus pixel codes describe image formats as flowing over physical bus.
please see-also below to review the format elaboration,
https://www.kernel.org/doc/html/v4.15/media/uapi/v4l/subdev-formats.html#packed-yuv-formats
Dear Jerry,
Doses it mean that media bus format is set incorrectly?
The video format of NV16 is defined in vi5_formats.h.
TEGRA_VIDEO_FORMAT(YUV422, 16, UYVY8_1X16, 1, 1, T_Y8__V8U8_N422,
YUV422_8, NV16, “NV16”),
so I added a camera color format as like below,
Linux_for_Tegra/source/public/kernel/nvidia/drivers/media/platform/tegra/camera/camera_common.c
{
MEDIA_BUS_FMT_UYVY8_1X16, /*nv16*/
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_NV16,/*nv16*/
},
Is multi-plane format like nv16 considered and implemented in tegra camera driver?
Thanks,
hello yt.jeon,
may I know what’s the format types sensor has actually sending out. which sensor module it is.
The actual format that the sensor transmits is NV16.
As mentioned above, orin is connected to an external ISP and the ISP is connected to a camera module customized by LG.
Hi,
Please check if you can run v4l2-ctl to capture frame data:
$ v4l2-ctl --device /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=NV16 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=200
Please modify resolution( set to width=640,height=480 in the command ) per your device.
If the command fails to run, it may be still some issue in sensor driver.
If it runs successfully, you can try the gstreamer command and check if preview is good:
$ DISPLAY=:0 gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=NV16,width=640,height=480 ! videoconvert ! video/x-raw,formst=I420 ! xvimagesink
The v4l2-ctl command works well.
stereo@stereo-desktop:~/projects/stereo_camera_sdk/test$ v4l2-ctl --device /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=NV16 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=200
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<
I captured a frame after adding a option (–stream-to=frame.yuv ).
A half of the captured data is all ‘0’
v4l2-ctl --device /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=NV16 --set-ctrl bypass_mode=0 --stream-mmap --stream-to=frame.yuv --stream-count=1
And the gstreamer doesn’t work.
$gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=NV16,width=640,height=480 ! videoconvert ! video/x-raw,formst=I420 ! xvimagesink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.010077720
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
The preview is greenish, but below gsteamer command works well.
$gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! video/x-raw,formst=I420 ! xvimagesink
hello yt.jeon,
may I know what’s the sensor format dumps, please share the results of v4l2-ctl --list-formats-ext.
stereo@stereo-desktop:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'NV16' (Y/CbCr 4:2:2)
hello yt.jeon,
please revise driver side to use formats as following.
{
MEDIA_BUS_FMT_YUYV8_2X8,
V4L2_COLORSPACE_SRGB,
V4L2_PIX_FMT_NV16,
},
Dear Jerry,
It needs to change of vi5_video_formats(in vi5_format.h). How should I change that?
- TEGRA_VIDEO_FORMAT(YUV422, 16, UYVY8_1X16, 1, 1, T_Y8__V8U8_N422,
YUV422_8, NV16, “NV16”),
+ TEGRA_VIDEO_FORMAT(YUV422, 16, YUYV8_2X8, ?, ?, ???,
YUV422_8, NV16, “NV16”),
thanks,
hello yt.jeon,
hold-on, it looks driver side already support NV16.
you’re v4l test with NV16 also works with fetching the frames.
could you please attach the yuv file which shown a half of the captured data is all ‘0’ for checking,
i.e. v4l2-ctl --device /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=NV16 --set-ctrl bypass_mode=0 --stream-mmap --stream-to=frame.yuv --stream-count=1
and… is your sensor supported resolution at 640x480?
don’t sensor format dumps, v4l2-ctl --list-formats-ext also list the support resolutions?
Dear Jerry,
I’m uploading the dump.
The sensor only outputs 1902x2306.
stereo@stereo-desktop:~$ v4l2-ctl --all
Video input : 0 (Camera 0: ok)
Format Video Capture:
Width/Height : 1920/2306
Pixel Format : ‘NV16’ (Y/CbCr 4:2:2)
Field : None
Bytes per Line : 1920
Size Image : 8855040
Colorspace : sRGB
Thanks,
Dear Jerry, @yt.jeon
Here it is the stream data.
frame.yuv (8.4 MB)
hello sumin.lee1,
is it the YUV dumps with width/height setting to 640x480?
please examine the sensor supported resolutions, and revise the pipeline to dump the content with it.
Dear Jerry,
The dump was from below cmd.
v4l2-ctl --device /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=NV16 --set-ctrl bypass_mode=0 --stream-mmap --stream-to=frame.yuv --stream-count=1
And the sensor has only one mode as below. Only supports 1920x2306.
static const struct camera_common_frmfmt lgsc_frmfmt[ ] = {
{{1920, 2306}, lgsc_30_fr, 1, 0, LGSC_MODE_3840x2160_30FPS},
};
mode0 {
dynamic_pixel_bit_depth = “16”;
csi_pixel_bit_depth = “16”;
mode_type = “yuv”;
pixel_phase = “nv16”;
active_w = “1920”;
active_h = “2306”;
…
}
Thanks,
hello yt.jeon,
to clarify,
please configure --set-fmt-video=width=1920,height=2306 as same as sensor supported formats to dump the data.
Dear Jerry,
The dump is same (data size, filled 0 in half) with the configure( --set-fmt-video=width=1920,height=2306).
Thanks,