Problem of getting data from MIPI CSI Jetson TX2

Quang_OpenStack · May 29, 2019, 3:58am

Hi JerryChang,
Additionally, when saving video input to file, the timestamp does not change, you can see as following

./v4l2-ctl -d /dev/video0 -w --verbose --set-fmt-video=width=3840,height=2160,pixelformat=UYVY --stream-mmap --stream-count=30 --stream-to=test.raw
VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
        Width/Height      : 3840/2160
        Pixel Format      : 'UYVY'
        Field             : None
        Bytes per Line    : 7680
        Size Image        : 16588800
        Colorspace        : SMPTE 170M
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Flags             :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
        Index    : 0
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 0
        Length   : 16588800
        Bytesused: 16588800
        Timestamp: 163.298596s (Monotonic, End-of-Frame)

        Index    : 1
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 1
        Length   : 16588800
        Bytesused: 16588800
        Timestamp: 163.298596s (Monotonic, End-of-Frame)

 0.50 fps
        Index    : 2
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 2
        Length   : 16588800
        Bytesused: 16588800
        Timestamp: 163.298596s (Monotonic, End-of-Frame)
....

Thank,
Best regard.

JerryChang · May 29, 2019, 6:11am

hello Quang_OpenStack,

please refer to below two topics, you’ll need to confirmed you had include these kernel patches.

Topic 1038067: fix FE syncpt wait
Topic 1020202: fix missing timestamp for vi4
thanks

Quang_OpenStack · May 29, 2019, 7:30am

Dear JerryChang,
I included those patches; some parameters are missing but finally it does not work; the same error happen.

Additionally, as I did before, I tested our custom FPGA board with Jetpack 3.3 that also include those patches; it has same error.
Do you have other suggestion?
Because half of screen sometime has still image; I think only data from 1 MIPI come to CSI port.
So, I want to know method to check whether or not data is came in CSI input and come out CSI output

Thank,
Best regards.

JerryChang · May 29, 2019, 8:51am

hello Quang_OpenStack,

let’s check the raw dump files,

$ v4l2-ctl -d /dev/video0 -w --verbose --set-fmt-video=width=3840,height=2160,pixelformat=UYVY --stream-mmap --stream-count=30 --stream-to=test.raw

please use the 3rdparty tools (such as 7yuv) to check the raw dump files.
thanks

Quang_OpenStack · May 29, 2019, 9:03am

Hi JerryChang,
I attached the link record file.
Using 7yuv, i can see sometime half of screen which has still image

https://drive.google.com/open?id=1z_3MrIGjIva3-1U_M5bgEGJLbLrHqdhk

Thank

JerryChang · May 29, 2019, 9:34am

hello Quang_OpenStack,

could you please check the signaling, please confirm it’s actually sending 8-lane MIPI signaling.
thanks

Quang_OpenStack · May 29, 2019, 11:24am

Dear JerryChang,
Our FPGA engineer said that, MIPI signal from FPGA was generated and sent to CSI port.
But I am not clear how to check CSI status on TX2 side.

Our device driver and device tree is base on tc358840 device
Thank.

JerryChang · May 30, 2019, 3:08am

hello Quang_OpenStack,

several suggestions as below,

you may probe the FPGA output signaling to check if they’re following MIPI CSI-2 specification.
could you please have an alternative way to configure your device as 4-lane device (without using gang mode), to ensure your streaming individually.
it’s always helpful to enable test-mode or color-bar to output pseudo CSI signaling for triage the root-cause.
thanks

Quang_OpenStack · May 30, 2019, 4:04am

Dear JerryChang,
Thank for your reply.
How can I enable test-mode or color-bar to output pseudo CSI signaling?

Best regards.

JerryChang · May 30, 2019, 7:09am

hello Quang_OpenStack,

you should have implementation in your kernel driver,
please also contact your vendor about the configuration,
thanks

Quang_OpenStack · September 3, 2019, 1:15pm

Hi JerryChang,
It seems that, on VI module we only receive the first PXL_SOF of 1 channel.
And then timeout happen because of nvhost_syncpt_wait_timeout_ext() function. But our FPGA engineer confirm about sending SOF frequently.
Do you have any suggestion for this trouble?
Thank in advance.

Quang_OpenStack · September 3, 2019, 11:26pm

Hi,
By the way, in case of 4K@60fps, we use 2 MIPI to send data from FPGA. So should we set SOF for each frame for each MIPI?
Best regards.

JerryChang · September 4, 2019, 3:12am

hello Quang_OpenStack,

VI engine is passively waiting for SOF and EOF to catch one frame.
hence, you should check your sensor stream output as SOF/EOF/SOF/…/EOF.
thanks

Quang_OpenStack · September 4, 2019, 5:35am

Hi JerryChang,
Should we set SOF and EOF for each MIPI?
Thank

Quang_OpenStack · September 5, 2019, 4:38am

Hi JerryChang,
Following this patch

When the “PXL_SOF syncpt timeout” happens, I reset NvCsi, re-initialize VI and capture context; then I can get data frame.
But because timeout happen, the frame is drop in the middle and screen is not smooth.
Do you have any idea for solving this problem?
Best regards.

JerryChang · September 5, 2019, 6:48am

hello Quang_OpenStack,

that patch was adding error handling mechanism, driver side simply reset the broken capture request and waiting for another frame coming.
you should tune your drivers, please check your stream also follow the MIPI specification.
thanks

Quang_OpenStack · September 5, 2019, 9:00am

Hi JerryChang,
In case of there are 2 MIPI, how signal will be sent? simultaneously or sequentially? and how about FS & FE on each MIPI?
Best regards

Quang_OpenStack · September 6, 2019, 5:34am

Hi JerryChang,
Following is driver source code for notification when PXL_SOF came to VI module:

/*
         * Wait for PXL_SOF syncpt
         *
         * Use the syncpt max value we just set as threshold
         */
        for (i = 0; i < chan->valid_ports; i++) {
                err = nvhost_syncpt_wait_timeout_ext(chan->vi->ndev,
                                chan->syncpt[i][SOF_SYNCPT_IDX], thresh[i],
                                250, NULL, NULL);
                if (unlikely(err))
                        dev_err(chan->vi->dev,
                                "PXL_SOF syncpt timeout! err = %d\n", err);
                else {
                        struct vi_capture_status status;
 
                        err = vi_notify_get_capture_status(chan->vnc[i],
                                        chan->vnc_id[i],
                                        thresh[i], &status);
                        if (unlikely(err))
                                dev_err(chan->vi->dev,
                                        "no capture status! err = %d\n", err);
                        else
                                *ts = ns_to_timespec((s64)status.sof_ts);
                }

Because we are using two MIPI for sending data and only one PXL_SOF is get on the first channel port. Do you think that the CSI second port did not initialize correctly or something wrong on device tree (please check attached file)
P.S. We make device driver and device tree base on TC358840
tegra186-quill-p3310-1000-c03-00-base-hdmi2csi.zip (3.11 KB)

JerryChang · September 6, 2019, 5:58am

hello Quang_OpenStack,

am I understand correctly that you’re working a 8-lane device, which need to combine two CSI bricks for enough bandwidth.
hence, please update the num-channels device tree property since you actually only having one device connected.
for example,

vi@15700000 {
    num-channels = <1>;
nvcsi@150c0000 {
    num-channels = <1>;

you may also refer to below kernel sources for handling more than one CSI port.

<i>$l4t-r32.2/kernel_src/kernel/nvidia/drivers/media/platform/tegra/camera/vi/channel.c</i>

static void update_gang_mode(struct tegra_channel *chan){...}

Quang_OpenStack · September 6, 2019, 7:24am

Hi JerryChang,
I udpated num-channels on device tree as your suggestion but there are no lucky,
I still cannot get PXL_SOF on the second channel port (timeout still happen); only left screen get data, right screen is green.
I also added some debug message on update_gang_mode and get some result like bellowing (input video is 4K)

chan->format.width: 3840
chan->format.height: 2160
chan->gang_mode: CAMERA_GANG_L_R
chan->valid_ports: 2

Do you have other idea for this?
We are splitting data into 2 MIPI and send with SOF/EOF on each MIPI
Thank in advance.
Best regards,