1080p60 capture works on 5.1.2 but not 6.1 (tc358743, custom carrier)

Hello again,

We are having a problem capturing 1080p60 video w/JetPack 6.1 on our custom board. We can capture 720p60 with no issues. We were able to capture 1080p60 with the same hardware runing JetPack 5.1.2.

Here is the trace :
orin-trace1080p.txt (18.6 KB)

This is what I see when I run the capture command:

v4l2-ctl -d /dev/video1 --stream-count=1 --stream-mmap --stream-to=rickHD-v0-1080.raw --verbose --set-fmt-video=width=1920,height=1080,pixelformat=1
VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'UYVY' (UYVY 4:2:2)
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : SMPTE 170M
	Transfer Function : Default (maps to Rec. 709)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Limited Range)
	Flags             :
New timings found
		VIDIOC_REQBUFS returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_STREAMON returned 0 (Success)
cap dqbuf: 0 seq:      0 bytesused: 4147200 ts: 3446.942415 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      1 bytesused: 4147200 ts: 3446.959053 delta: 16.638 ms (error, ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq:      2 bytesused: 4147200 ts: 3446.975736 delta: 16.683 ms (error, ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq:      3 bytesused: 4147200 ts: 3446.992419 delta: 16.683 ms (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq:      4 bytesused: 4147200 ts: 3447.009102 delta: 16.683 ms (error, ts-monotonic, ts-src-eof)

And this is an excerpt from dmesg output when the capture fails :

[ 3418.801503] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- brick_config.phy_mode : 0
[ 3418.801508] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- brick_config.lane_polarity[index] : 0
[ 3418.801511] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- csi5_stream_set_config : before s_data
[ 3418.801515] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- using hardcode 5940
[ 3418.801518] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- mipi_clock_rate is : 297000
[ 3418.801521] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- sizeof(msg) is : 280
[ 3418.801525] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- valid ports is : 1
[ 3418.801527] t194-nvcsi 13e00000.host1x:nvcsi@15a00000:    X-- vi port is : 0
[ 3418.801930] tc358743 10-000f: Calling tc358743_s_stream - start of attempt
[ 3418.801933] tc358743 10-000f: enable_stream: enable
[ 3418.802177] tc358743 10-000f: I2C write 0x0238 = 0x00000000
[ 3418.802394] tc358743 10-000f: I2C write 0x0238 = 0x00000001
[ 3418.802534] tc358743 10-000f: I2C write 0x857f = 0xc0
[ 3418.802908] tc358743 10-000f: I2C write 0x0004 = 0x0cd7
[ 3418.802910] tc358743 10-000f: after mutex_unlock - 691:enable_stream: end
[ 3418.803556] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 18432, flags: 0, err_data 512
[ 3418.803573]   jc-- vi5_ops.c : in rel_buf, releasing buffer
[ 3418.803579]   jc-- vi5_ops.c : vi5_capture_dequeue : using vi_port 0
[ 3418.803580]   jc-- vi5_ops.c : vi5_capture_dequeue : buf->vb2_state 4
[ 3418.803581]   jc-- vi5_ops.c : vi5_capture_dequeue : VB2_BUF_STATE_ACTIVE  4
[ 3418.803583]   jc-- vi5_ops.c : vi5_capture_dequeue : before vi_capture_status
[ 3418.803584]   jc- in vi_capture_status
[ 3418.803585]   jc- in vi_capture_status : capture initialized
[ 3418.803586]   jc- in vi_capture_status : valid channel ID found
[ 3418.803587]   jc- in vi_capture_status : after wait timeout...
[ 3418.820398] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 18432, flags: 0, err_data 512
[ 3418.820410]   jc-- vi5_ops.c : in rel_buf, releasing buffer
[ 3418.820415]   jc-- vi5_ops.c : vi5_capture_dequeue : using vi_port 0
[ 3418.820416]   jc-- vi5_ops.c : vi5_capture_dequeue : buf->vb2_state 4
[ 3418.820418]   jc-- vi5_ops.c : vi5_capture_dequeue : VB2_BUF_STATE_ACTIVE  4
[ 3418.820420]   jc-- vi5_ops.c : vi5_capture_dequeue : before vi_capture_status
[ 3418.820421]   jc- in vi_capture_status
[ 3418.820422]   jc- in vi_capture_status : capture initialized
[ 3418.820423]   jc- in vi_capture_status : valid channel ID found
[ 3418.820424]   jc- in vi_capture_status : after wait timeout...

Can you tell why this isn’t working?

Thank you.

Doesn’t look like receive any data from the tc358743 from the trace log.
You may need to check the configuration or probe the signal to confirm it.

From the trace log:

 vi-output, tc35-3916    [004] .......  2910.119147: tegra_channel_capture_frame: sof:2938.258002240
 vi-output, tc35-3916    [004] .......  2910.119148: tegra_channel_capture_frame: eof:2938.258056896
 vi-output, tc35-3915    [004] .......  2910.119238: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:2388 pid:3915 tid:3915
 vi-output, tc35-3916    [004] .......  2910.122155: tegra_channel_capture_frame: sof:2938.261009728
 vi-output, tc35-3916    [004] .......  2910.122156: tegra_channel_capture_frame: eof:2938.261064384
 vi-output, tc35-3915    [004] .......  2910.122209: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:2388 pid:3915 tid:3915

sof = Start of Frame
eof = End of Frame

Doesn’t that mean it is receiving frames ?

And the difference in timestamp is 16ms which adds up to 60fps.

What configuration?

Probe what signal? The incoming HDMI signal is working correctly, tested with two different input devices.

In the output of dmesg I see the incoming signal correctly identified:

[ 3418.758071] tc358743 10-000f: line 500 : tc358743_get_detected_timings: width 1920 heigh 1080 interlaced 0
[ 3418.758075] tc358743 10-000f: tc358743_query_dv_timings: 1920x1080p60.00 (2200x1125)

It is receiving a 1080p60 signal, the dtb is configured for four lane communication.

Why are there no errors in the trace log?

Thank you.

The other thing to note, when we run the capture command, the seq runs fast by the screen. We don’t see the 2500ms wait you’d get when no signal is present (or incomplete frame). The ts (delta) is 16.683ms which is correct for 60p signal frame rate. So it seems like it is getting something from the CSI lanes, but we aren’t getting the normal notify lines in the trace to tell us what it doesn’t like.

Sorry I didn’t notice the SOF/EOF from the trace.
Looks like there’s no problem for the capturing. Could you try by v4l2src to check if able get the preview.

Hello @ShaneCCC -

I ran the command:

gst-launch-1.0 -v v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,framerate=60/1,width=1920,height=1080 ! xvimagesink

and received the following output :

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Signal lost
Additional debug info:
../sys/v4l2/gstv4l2src.c(556): gst_v4l2src_query_preferred_size (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
No input source was detected - video frames invalid
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string)2:3:5:4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string)2:3:5:4
/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string)2:3:5:4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1920, height=(int)1080, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string)2:3:5:4
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Output window was closed
Additional debug info:
../sys/xvimage/xvimagesink.c(568): gst_xv_image_sink_handle_xevents (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0
Execution ended after 0:00:59.646976279
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason error (-5)
Freeing pipeline ...

The input only displayed in the top left corner of the screen and only had green and purples as colors. The signal was unstable and glitchy. If I increased the output fade to +20% brightness, the frame would stretch to almost the full height (but not width) of the window.

I can capture 720p60 normally.

Thank you.

Could you boost the clocks to try.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

I always boost clocks before posting here. :)

cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
550400000
729600000
214300000
2133000000

Thank you.

Could you modify the driver to report 720p only to try.

Thanks

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,framerate=60/1,width=1280,height=720 ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1280, height=(int)720, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string)2:3:5:4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1280, height=(int)720, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string)2:
3:5:4
/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1280, height=(int)720, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string
)2:3:5:4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, framerate=(fraction)60/1, width=(int)1280, height=(int)720, format=(string)UYVY, interlace-mode=(string)progressive, colorimetry=(string)2
:3:5:4
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:32.164293930
Setting pipeline to NULL ...
Freeing pipeline ...

720p60 video was captured correctly and without error. No modifications to the driver or dtb were necessary. It is 1080p60 that is failing here.

@ShaneCCC looking back at this thread (post #100 by @Connor.B)

I remembered that he got 1080p60 working by setting the pll_fbd value to 176 for 1080p. Well, I dug deeper into that and what I am seeing is yes, we got one channel to work on our customer board with this, but the other is struggling to get full frames.

Looking into the user guide, this is setting the PLL at 594M x 2 which means the Orin is actually doing 1080p60 on 2 lanes and I believe there was a reason that wasn’t to be used (but I don’t exactly remember who the limiter was on that). Clearly one CSI port is struggling with this setting.

So, that mean 4 lanes is not working for us. From what @anomad has shown above, given the color shifting, and the way the CSI lanes are split, maybe we’re just not getting 4 lane recording on the Orin SoM side. If it’s missing 2 lanes, that would account for half the screen getting captured.

So again, I suspect the Orin is staying in 2 lane mode no matter what we put in the DTB, and the TC35 driver is setting the TC35 to 4 lanes when asked (based on bandwidth requirements)

Any idea of what we can check to see what the Orin itself is doing lane wise with the CSI ports?

I don’t think the CSI driver would have problem for 4 lanes.
You can confirm the setting by csi5_fops.c to clarify.