orin nano 4gb r35.3.1 bayer RGGB12 1920*1080 30fps
imx390 csi 4lane
nvgstcapture can’t display image.
The data can be captured normally via v4l2-ctl, I checked that the image can be decoded.Using opencv bayer2rgb, the image seems to have some problems, how to play the video with nvgstcapture ?
v4l2 normally log:
v4l2-ctl-3383 [005] .... 2743.075309: i2c_result: i2c-2 n=1 ret=1
v4l2-ctl-3383 [005] .... 2743.075310: i2c_result: i2c-10 n=1 ret=1
kworker/1:0-20 [001] .... 2743.130202: rtcpu_vinotify_event: tstamp:86412309791 cch:0 vi:1 tag:FS channel:0x00 frame:1 vi_tstamp:2765186841344 data:0x0000000100000012
kworker/1:0-20 [001] .... 2743.130203: rtcpu_vinotify_event: tstamp:86412310053 cch:0 vi:1 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:2765186841440 data:0x0000000800000000
kworker/1:0-20 [001] .... 2743.130204: rtcpu_vinotify_event: tstamp:86412310344 cch:0 vi:1 tag:CHANSEL_EMBED_SOF channel:0x23 frame:1 vi_tstamp:2765186895648 data:0x0000000000000004
kworker/1:0-20 [001] .... 2743.130204: rtcpu_vinotify_event: tstamp:86412310655 cch:0 vi:1 tag:CHANSEL_EMBED_EOF channel:0x23 frame:1 vi_tstamp:2765186902816 data:0x0000000000000008
kworker/1:0-20 [001] .... 2743.130205: rtcpu_vinotify_event: tstamp:86412310945 cch:0 vi:1 tag:ATOMP_EMB_DATA_DONE channel:0x23 frame:1 vi_tstamp:2765186904000 data:0x0000000000000000
kworker/1:0-20 [001] .... 2743.130206: rtcpu_vinotify_event: tstamp:86412311192 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:1 vi_tstamp:2765186926912 data:0x0000000003020001
kworker/1:0-20 [001] .... 2743.130206: rtcpu_vinotify_event: tstamp:86412311469 cch:0 vi:1 tag:CHANSEL_NOMATCH channel:0x04 frame:1 vi_tstamp:2765186984512 data:0x00000000000006e9
kworker/1:0-20 [001] .... 2743.130207: rtcpu_vinotify_event: tstamp:86412311712 cch:0 vi:1 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:2765187577120 data:0x0000000000000001
kworker/1:0-20 [001] .... 2743.130207: rtcpu_vinotify_event: tstamp:86412311994 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:1 vi_tstamp:2765187588032 data:0x0000000008020001
kworker/1:0-20 [001] .... 2743.130208: rtcpu_vinotify_event: tstamp:86413129734 cch:0 vi:1 tag:CHANSEL_PXL_EOF channel:0x23 frame:1 vi_tstamp:2765219849888 data:0x0000000004370002
kworker/1:0-20 [001] .... 2743.130209: rtcpu_vinotify_event: tstamp:86413130018 cch:0 vi:1 tag:ATOMP_FRAME_DONE channel:0x23 frame:1 vi_tstamp:2765219851136 data:0x0000000000000000
kworker/1:0-20 [001] .... 2743.130209: rtcpu_vinotify_event: tstamp:86413130266 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:1 vi_tstamp:2765219871200 data:0x0000000002020001
vi-output, imx3-3385 [000] .... 2743.130348: tegra_channel_capture_frame: sof:2765.187577120
vi-output, imx3-3385 [000] .... 2743.130350: tegra_channel_capture_frame: eof:2765.219881344
vi-output, imx3-3385 [000] .... 2743.163288: tegra_channel_capture_frame: sof:2765.220909408
vi-output, imx3-3385 [000] .... 2743.163290: tegra_channel_capture_frame: eof:2765.253213728
v4l2-ctl-3383 [005] .... 2743.163500: tegra_channel_set_stream: enable : 0x0
v4l2-ctl-3383 [005] .... 2743.163503: tegra_channel_set_stream: imx390 10-0020 : 0x0
v4l2-ctl-3383 [005] .... 2743.163530: i2c_write: i2c-10 #0 a=020 f=0000 l=3 [00-00-01]
v4l2-ctl-3383 [005] .... 2743.163539: i2c_write: i2c-2 #0 a=020 f=0000 l=3 [00-00-01]
v4l2-ctl-3383 [005] .... 2743.163714: i2c_result: i2c-2 n=1 ret=1
v4l2-ctl-3383 [005] .... 2743.163715: i2c_result: i2c-10 n=1 ret=1
v4l2-ctl-3383 [005] d..1 2743.163736: console: [ 2743.163727] imx390 10-0020: stop cam but nothing todo 0
v4l2-ctl-3383 [005] .... 2743.169147: tegra_channel_set_stream: 13e40000.host1x:nvcsi@15a00000- : 0x0
v4l2-ctl-3383 [005] .... 2743.169149: csi_s_stream: enable : 0x0
v4l2-ctl-3383 [005] .... 2743.173885: tegra_channel_set_power: imx390 10-0020 : 0x0
v4l2-ctl-3383 [005] .... 2743.173890: camera_common_s_power: status : 0x0
v4l2-ctl-3383 [005] .... 2743.173899: tegra_channel_set_power: 13e40000.host1x:nvcsi@15a00000- : 0x0
v4l2-ctl-3383 [005] .... 2743.173901: csi_s_power: enable : 0x0
kworker/1:0-20 [001] .... 2743.186016: rtcpu_vinotify_event: tstamp:86413611434 cch:0 vi:1 tag:FE channel:0x00 frame:1 vi_tstamp:2765219881248 data:0x0000000100000022
kworker/1:0-20 [001] .... 2743.186017: rtcpu_vinotify_event: tstamp:86413611695 cch:0 vi:1 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:2765219881344 data:0x0000000800000000
kworker/1:0-20 [001] .... 2743.186018: rtcpu_vinotify_event: tstamp:86413611977 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:1 vi_tstamp:2765219941280 data:0x0000000000020001
kworker/1:0-20 [001] .... 2743.186019: rtcpu_vinotify_event: tstamp:86413612221 cch:0 vi:1 tag:FS channel:0x00 frame:2 vi_tstamp:2765220173728 data:0x0000000200000012
kworker/1:0-20 [001] .... 2743.186020: rtcpu_vinotify_event: tstamp:86413612504 cch:0 vi:1 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:2765220173792 data:0x0000000800000000
kworker/1:0-20 [001] .... 2743.186020: rtcpu_vinotify_event: tstamp:86413612749 cch:0 vi:1 tag:CHANSEL_EMBED_SOF channel:0x23 frame:2 vi_tstamp:2765220227936 data:0x0000000000000004
kworker/1:0-20 [001] .... 2743.186021: rtcpu_vinotify_event: tstamp:86413613025 cch:0 vi:1 tag:CHANSEL_EMBED_EOF channel:0x23 frame:2 vi_tstamp:2765220235104 data:0x0000000000000008
kworker/1:0-20 [001] .... 2743.186022: rtcpu_vinotify_event: tstamp:86413613274 cch:0 vi:1 tag:ATOMP_EMB_DATA_DONE channel:0x23 frame:2 vi_tstamp:2765220236288 data:0x0000000000000000
kworker/1:0-20 [001] .... 2743.186022: rtcpu_vinotify_event: tstamp:86413613550 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:2 vi_tstamp:2765220259232 data:0x0000000003020002
kworker/1:0-20 [001] .... 2743.186024: rtcpu_vinotify_event: tstamp:86413613797 cch:0 vi:1 tag:CHANSEL_NOMATCH channel:0x04 frame:2 vi_tstamp:2765220316800 data:0x00000000000006e9
kworker/1:0-20 [001] .... 2743.186025: rtcpu_vinotify_event: tstamp:86413614077 cch:0 vi:1 tag:CHANSEL_PXL_SOF channel:0x23 frame:2 vi_tstamp:2765220909408 data:0x0000000000000001
kworker/1:0-20 [001] .... 2743.186025: rtcpu_vinotify_event: tstamp:86413614324 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:2 vi_tstamp:2765220920288 data:0x0000000008020002
kworker/1:0-20 [001] .... 2743.186025: rtcpu_vinotify_event: tstamp:86413614603 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2765221185856 data:0x719e300010000000
kworker/1:0-20 [001] .... 2743.186026: rtcpu_vinotify_event: tstamp:86413614848 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2765221229184 data:0x0000000031000003
kworker/1:0-20 [001] .... 2743.186027: rtcpu_vinotify_event: tstamp:86414171360 cch:0 vi:1 tag:CHANSEL_PXL_EOF channel:0x23 frame:2 vi_tstamp:2765253182272 data:0x0000000004370002
kworker/1:0-20 [001] .... 2743.186027: rtcpu_vinotify_event: tstamp:86414171613 cch:0 vi:1 tag:ATOMP_FRAME_DONE channel:0x23 frame:2 vi_tstamp:2765253183520 data:0x0000000000000000
kworker/1:0-20 [001] .... 2743.186029: rtcpu_string: tstamp:86414356753 id:0x04010000 str:"tegra_nvcsi_stream_close(vm0, stream=2, csi=2)
"
kworker/1:0-20 [001] d..1 2743.186041: console: [ 2743.172608] bwmgr API not supported
kworker/1:0-20 [001] d..1 2743.189634: console: [ 2743.186035] [RCE] tegra_nvcsi_stream_close(vm0, stream=2, csi=2)
kworker/1:0-20 [001] .... 2743.195815: rtcpu_vinotify_event: tstamp:86414499914 cch:-1 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:2 vi_tstamp:2765253211744 data:0x0000000002020002
kworker/1:0-20 [001] .... 2743.195816: rtcpu_vinotify_event: tstamp:86414500167 cch:-1 vi:1 tag:FE channel:0x00 frame:2 vi_tstamp:2765253213632 data:0x0000000200000022
kworker/1:0-20 [001] .... 2743.195816: rtcpu_vinotify_event: tstamp:86414500456 cch:-1 vi:1 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:2765253213728 data:0x0000000800000000
kworker/1:0-20 [001] .... 2743.195816: rtcpu_vinotify_event: tstamp:86414500699 cch:-1 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:2 vi_tstamp:2765253273632 data:0x0000000000020002
kworker/1:0-20 [001] .... 2743.195816: rtcpu_vinotify_event: tstamp:86414500989 cch:-1 vi:1 tag:FS channel:0x00 frame:3 vi_tstamp:2765253506016 data:0x0000000300000012
kworker/1:0-20 [001] .... 2743.195816: rtcpu_vinotify_event: tstamp:86414501234 cch:-1 vi:1 tag:ATOMP_FS channel:0x00 frame:3 vi_tstamp:2765253506080 data:0x0000000800000000
kworker/1:0-20 [001] .... 2743.195817: rtcpu_vinotify_event: tstamp:86414501507 cch:-1 vi:1 tag:CHANSEL_EMBED_SOF channel:0x23 frame:3 vi_tstamp:2765253560256 data:0x0000000000000004
kworker/1:0-20 [001] .... 2743.195817: rtcpu_vinotify_event: tstamp:86414501744 cch:-1 vi:1 tag:CHANSEL_EMBED_EOF channel:0x23 frame:3 vi_tstamp:2765253567424 data:0x0000000000000008
kworker/1:0-20 [001] .... 2743.195817: rtcpu_vinotify_event: tstamp:86414502011 cch:-1 vi:1 tag:ATOMP_EMB_DATA_DONE channel:0x23 frame:3 vi_tstamp:2765253568608 data:0x0000000000000000
kworker/1:0-20 [001] .... 2743.195817: rtcpu_vinotify_event: tstamp:86414502247 cch:-1 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:3 vi_tstamp:2765253591520 data:0x0000000003020003
kworker/1:0-20 [001] .... 2743.195817: rtcpu_vinotify_event: tstamp:86414502522 cch:-1 vi:1 tag:CHANSEL_NOMATCH channel:0x04 frame:3 vi_tstamp:2765253649120 data:0x00000000000006e9
kworker/1:0-20 [001] .... 2743.195818: rtcpu_vinotify_event: tstamp:86414502758 cch:-1 vi:1 tag:CHANSEL_PXL_SOF channel:0x23 frame:3 vi_tstamp:2765254241696 data:0x0000000000000001
kworker/1:0-20 [001] .... 2743.195818: rtcpu_vinotify_event: tstamp:86414503025 cch:-1 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:3 vi_tstamp:2765254252576 data:0x0000000008020003
kworker/1:0-20 [001] .... 2743.195818: rtcpu_vinotify_event: tstamp:86414503261 cch:-1 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2765254513056 data:0x719e2d0010000000
kworker/1:0-20 [001] .... 2743.195818: rtcpu_vinotify_event: tstamp:86414503527 cch:-1 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:2765254561056 data:0x0000000031000004
kworker/1:0-20 [001] .... 2743.195819: rtcpu_vinotify_event: tstamp:86414503766 cch:-1 vi:1 tag:FE channel:0x00 frame:3 vi_tstamp:2765263287968 data:0x0000000000000022
kworker/1:0-20 [001] .... 2743.195819: rtcpu_vinotify_event: tstamp:86414810456 cch:-1 vi:1 tag:CHANSEL_SHORT_FRAME channel:0x04 frame:3 vi_tstamp:2765263287968 data:0x00af200001000000
kworker/1:0-20 [001] .... 2743.195819: rtcpu_vinotify_event: tstamp:86414810753 cch:-1 vi:1 tag:ATOMP_FE channel:0x00 frame:3 vi_tstamp:2765263288000 data:0x0000000800000000
v4l2-ctl-3383 [005] .... 2743.220320: tegra_channel_close: vi-output, imx390 10-0020
hello gaoming.song,
may I know how many cameras in your system. please also check… $ ls /dev/video*
it may usually due to device tree settings, please share your tegra-camera-platform for examination.
Only one camera is actually connected。 10-0020
root@ytmipi:/usr/local/lib/python3.8/dist-packages/camset# media-ctl -p /dev/media0
Media controller API version 5.10.104
Media device information
------------------------
driver tegra-camrtc-ca
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 5.10.104
Device topology
- entity 1: 13e40000.host1x:nvcsi@15a00000- (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
<- "imx390 9-0020":0 [ENABLED]
pad1: Source
-> "vi-output, imx390 9-0020":0 [ENABLED]
- entity 4: 13e40000.host1x:nvcsi@15a00000- (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
<- "imx390 10-0020":0 [ENABLED]
pad1: Source
-> "vi-output, imx390 10-0020":0 [ENABLED]
- entity 7: imx390 9-0020 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SRGGB12_1X12/1920x1080 field:none colorspace:srgb]
-> "13e40000.host1x:nvcsi@15a00000-":0 [ENABLED]
- entity 9: vi-output, imx390 9-0020 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "13e40000.host1x:nvcsi@15a00000-":1 [ENABLED]
- entity 23: imx390 10-0020 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev3
pad0: Source
[fmt:SRGGB12_1X12/1920x1080 field:none colorspace:srgb]
-> "13e40000.host1x:nvcsi@15a00000-":0 [ENABLED]
- entity 25: vi-output, imx390 10-0020 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "13e40000.host1x:nvcsi@15a00000-":1 [ENABLED]
tegra-camera-platform {
compatible = "nvidia, tegra-camera-platform";
num_csi_lanes = <0x04>;
max_lane_speed = <0x16e360>;
min_bits_per_pixel = <0x0a>;
vi_peak_byte_per_pixel = <0x02>;
vi_bw_margin_pct = <0x19>;
max_pixel_rate = <0x7270e0>;
isp_peak_byte_per_pixel = <0x05>;
isp_bw_margin_pct = <0x19>;
modules {
module0 {
badge = "imx390_front";
position = "front";
orientation = [31 00];
drivernode0 {
pcl_id = "v4l2_sensor";
devname = "imx390 9-0020";
proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@1/cam_0@20";
};
};
module1 {
badge = "imx390_rear";
position = "rear";
orientation = [31 00];
drivernode0 {
pcl_id = "v4l2_sensor";
devname = "imx390 10-0020";
proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@0/cam_1@20";
};
};
};
};
opencv can display video, but there’s a problem with the image display
root@ytmipi:/usr/local/lib/python3.8/dist-packages/camset# cat test_gstream.py
import cv2
import numpy as np
# Create a VideoCapture object and read from input file
# If the input is the camera, pass 0 instead of the video file name
cap = cv2.VideoCapture("/dev/video1")
cap.set(3,1920)
cap.set(4,1080)
data = ['RG12',"BA12","BG12","RG16"]
s = data[0]
fourcode = cv2.VideoWriter.fourcc(*s)
# also set resolution to cap, otherwise cv2 will use default and not the res set by v4l2
cap.set(cv2.CAP_PROP_FOURCC, fourcode)
# Check if camera opened successfully
if (cap.isOpened()== False):
print("Error opening video stream or file")
# Read until video is completed
while(cap.isOpened()):
# Capture frame-by-frame
ret, frame = cap.read()
if ret == True:
#cv2.cvtColor(frame, cv2.COLOR_BayerRG2RGB)
# Display the resulting frame
cv2.imshow('Frame',frame)
# Press Q on keyboard to exit
if cv2.waitKey(25) & 0xFF == ord('q'):
break
# Break the loop
else:
break
# When everything done, release the video capture object
cap.release()
# Closes all the frames
cv2.destroyAllWindows()
root@ytmipi:/home/yt# ls /dev/video*
/dev/video0 /dev/video1
hello gaoming.song,
you should configure sensor-id=0 for testing since you’ve only single camera connected.
sensro-id=0 Still wrong.
log:
# entries-in-buffer/entries-written: 12/12 #P:6
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
kworker/2:1-6153 [002] .... 7557.460079: rtcpu_string: tstamp:236859547316 id:0x04010000 str:"VM0 deactivating."
kworker/2:1-6153 [002] .... 7558.300044: rtcpu_string: tstamp:236887349574 id:0x04010000 str:"VM0 activating."
kworker/2:1-6153 [002] .... 7558.300048: rtcpu_string: tstamp:236887429461 id:0x04010000 str:"NVCSILP clock rate = 204000000 Hz.
"
kworker/2:1-6153 [002] d..1 7558.300064: console: [ 7558.300056] [RCE] VM0 deactivating.VM0 activating.NVCSILP clock rate = 204000000 Hz.
nvargus-daemon-8064 [005] .... 7558.305768: tegra_channel_open: vi-output, imx390 9-0020
nvargus-daemon-8064 [005] .... 7558.305836: tegra_channel_close: vi-output, imx390 9-0020
nvargus-daemon-8064 [005] .... 7558.305850: tegra_channel_open: vi-output, imx390 10-0020
nvargus-daemon-8064 [005] .... 7558.305886: tegra_channel_close: vi-output, imx390 10-0020
nvargus-daemon-8064 [005] .... 7558.306761: tegra_channel_open: vi-output, imx390 9-0020
nvargus-daemon-8064 [005] .... 7558.306801: tegra_channel_close: vi-output, imx390 9-0020
nvargus-daemon-8064 [005] .... 7558.306809: tegra_channel_open: vi-output, imx390 10-0020
nvargus-daemon-8064 [005] .... 7558.306839: tegra_channel_close: vi-output, imx390 10-0020
as you can see… camera service trying to enable dual IMX390s.
please try connect all your cameras, or, please moving the node definition in tegra-camera-platform for testing.
hello gaoming.song,
please share the disassembler the dtb file into text file for examination.
for example, $ dtc -I dtb -O dts -o temp.txt tegra234-xxx.dtb
BTW,
please also check Jetson Orin NX Series and Orin Nano Series Design Guide for [Figure 10-2. CSI 4-Lane Connection Options].
there’s polarity swap, please also check you’ve used device tree property, lane_polarity to configure a polarity swap on any lane.
imx390_all_4lane.zip (53.7 KB)
This is the configuration I’m using, because of the hardware characteristics of the nano mipi interface, cis 4lane can only be used on cam_1, and only cam_1 is connected to the camera, how do I change it?
just for confirmation, is this a developer kit?
yes, development environment is r35.3.1 ，original development board
hello gaoming.song,
it looks there’re some erroneous.
it should be cam_0 assign to CSI-B due to the port-index has defined as 0x1.
however, CSI-B does not support 4-lane configuration.
for example,
cam_i2cmux
i2c@0 {
cam_0@20 {
compatible = "sensing,imx390_x";
mode0 {
num_lanes = [34 00];
tegra_sinterface = "serial_c";
endpoint {
port-index = <0x01>;
bus-width = <0x04>;
i2c@1 {
cam_1@20 {
compatible = "sensing,imx390_1";
mode0 {
tegra_sinterface = "serial_c";
endpoint {
port-index = <0x02>;
bus-width = <0x04>;
so…
it should be i2c@1/cam_0@20 available for your system.
please revise the tegra-camera-platform to leave only single node, and updating position property for testing.
for example,
modules {
module0 {
badge = "imx390_front";
position = "front"; <== revise it as rear.
orientation = [31 00];
drivernode0 {
pcl_id = "v4l2_sensor";
devname = "imx390 9-0020";
proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@1/cam_0@20";
};
};
imx390_all_4lane.zip (53.7 KB)
thanks a lot.Instead of prompting for no cameras, it now prompts for the wrong parameter. nvgstcapture-1.0 --sensor-id=0
camera is: 1920*1080, RGGB12 * 30FPS，video0.
Can you help with the commands to display the video, seeing as the commands on the forum are rather complex.