Camera Mipi CSI-2 Data with repetition of one correct frame and 3 empty frames

Hi,

we implemeted a Camera on the Jetson nano.

when we test the Camera with:

“v4l2-ctl --set-fmt-video=width=640,height=539,pixelformat=RG10 --stream-mmap --stream-count=20 -d /dev/video0”

we get a repetition of one correct frame and three empty frames (filled with 0).
we get the correct framerate and no errors.

when we test the camera with:
“gst-launch-1.0 nvarguscamerasrc ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink -v”
we get not empty frames with the correct framerate.

Do you have any idea where the diffrence come from and how we can fix this?

thanks and best regards,

Michel

Can you check which mode is reported by nvarguscamerasrc when running gst-launch and set that mode into v4l2-ctl with -set-ctrl=sensor_mode=X? Also could you share the output of v4l2-ctl -d /dev/video0 -D --list-formats-ext?

Hi

we are Using mode 0 in both cases. We already checked this.

here is the output of

jetson@localhost:~$ v4l2-ctl -d /dev/video0 -D --list-formats-ext
Driver Info (not using libv4l2):
        Driver name   : tegra-video
        Card type     : vi-output, IMX991 6-0036
        Bus info      : platform:54080000.vi:0
        Driver version: 4.9.201
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'RG10'
        Name        : 10-bit Bayer RGRG/GBGB
                Size: Discrete 640x539
                        Interval: Discrete 0.004s (240.000 fps)
                Size: Discrete 640x512
                        Interval: Discrete 0.004s (240.000 fps)
                Size: Discrete 640x512
                        Interval: Discrete 0.004s (240.000 fps)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'RG12'
    Name        : 12-bit Bayer RGRG/GBGB
            Size: Discrete 640x539
                    Interval: Discrete 0.004s (240.000 fps)
            Size: Discrete 640x512
                    Interval: Discrete 0.004s (240.000 fps)
            Size: Discrete 640x512
                    Interval: Discrete 0.004s (240.000 fps)

hello reifenrath.michel,

FYI, VI engine doesn’t support with odd resolutions, could you please revise this 640x539 as even numbers?
also, what’s the fps reported of v4l and gst pipeline? are they reporting the same?

I treid it but it shows the same behaviour (just one more line 640x540).

yes they report the same framerate.

hello reifenrath.michel,

may I also confirm the Jetpack release you’re using.
is this a regression? it this worked normally on previous release version?

it shows:
Version: 4.5-b129

no it didnt work before. its the first test with this Version

please move to JetPack 4.6.3 if that’s possible,
at least, I don’t this issue (i.e. one correct frame and three empty frames) on the latest rel-32 code-line.

we cant do this easily.
so you dont have any other idea where this could come from?

I am taking over for my colleague Michel from now on.
Let’s start with some background. We are using FPGA from Lattice to bridge a sensor from SLVS to MIPI-CSI. The FPGA is cutting away the first line, so that is why we have the weird resolution of 539 height.
We managed to control the FPGA ip core, so that we now have good correct images, except for the 3 empty ones in between. Those seem to not happen with GStreamer since GStreamer display is correct, but using fpssink displays the correct amount of frames (so they dont seem to be just removed).

Now I wanted to try out on XAVIER NX instead of NANO and we get a different result. V4L2-ctl and GStreamer do not get any frames. On dmesg I get the following output:

[  +0.017136] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 6, err_data 256
[  +0.008259] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 0, err_data 256
[  +0.008196] tegra194-vi5 15c10000.vi: corr_err: discarding frame 2, flags: 0, err_data 256
[  +0.008196] tegra194-vi5 15c10000.vi: corr_err: discarding frame 1, flags: 0, err_data 256

Activating more debug and reading the trace gives me these results:

     kworker/0:0-8071  [000] ....  1549.953865: rtcpu_nvcsi_intr: tstamp:48813517562 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x00000089
     kworker/0:0-8071  [000] ....  1549.953866: rtcpu_nvcsi_intr: tstamp:48813517562 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x00000088
     kworker/0:0-8071  [000] ....  1549.953869: rtcpu_vinotify_event: tstamp:48813935881 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50201286944 data:0x10000000
     kworker/0:0-8071  [000] ....  1549.953870: rtcpu_vinotify_event: tstamp:48813936023 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50201295168 data:0x31000001
     kworker/0:0-8071  [000] ....  1549.953871: rtcpu_vinotify_event: tstamp:48813936185 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:50201297280 data:0x07020001
     kworker/0:0-8071  [000] ....  1549.953872: rtcpu_vinotify_event: tstamp:48813936320 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50201330976 data:0x10000000
     kworker/0:0-8071  [000] ....  1549.953873: rtcpu_vinotify_event: tstamp:48813936477 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50201339264 data:0x31000002
     kworker/0:0-8071  [000] ....  1550.009740: rtcpu_vinotify_event: tstamp:48815523552 tag:FS channel:0x00 frame:1 vi_tstamp:48815390852 data:0x00000010
     kworker/0:0-8071  [000] ....  1550.009746: rtcpu_vinotify_event: tstamp:48815523716 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:48815390855 data:0x00000000
     kworker/0:0-8071  [000] ....  1550.009747: rtcpu_vinotify_event: tstamp:48815523856 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:48815391333 data:0x00000001
     kworker/0:0-8071  [000] ....  1550.009748: rtcpu_vinotify_event: tstamp:48815524017 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:50264038720 data:0x08020001
     kworker/0:0-8071  [000] ....  1550.009749: rtcpu_vinotify_event: tstamp:48815524152 tag:CHANSEL_FAULT channel:0x23 frame:1 vi_tstamp:48815391415 data:0x00000100
     kworker/0:0-8071  [000] ....  1550.009750: rtcpu_vinotify_event: tstamp:48815524309 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:50264065088 data:0x01020001
     kworker/0:0-8071  [000] ....  1550.009751: rtcpu_vinotify_event: tstamp:48815524444 tag:CHANSEL_FAULT channel:0x23 frame:1 vi_tstamp:48815520298 data:0x021a0182
     kworker/0:0-8071  [000] ....  1550.009752: rtcpu_vinotify_event: tstamp:48815524599 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:50268169344 data:0x01020001
     kworker/0:0-8071  [000] ....  1550.009753: rtcpu_vinotify_event: tstamp:48815524734 tag:ATOMP_FRAME_DONE channel:0x23 frame:1 vi_tstamp:48815520324 data:0x00000000
     kworker/0:0-8071  [000] ....  1550.009754: rtcpu_vinotify_event: tstamp:48815524891 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:50268174176 data:0x02020001
     kworker/0:0-8071  [000] ....  1550.009755: rtcpu_vinotify_event: tstamp:48815525031 tag:FE channel:0x00 frame:1 vi_tstamp:48815520449 data:0x00000020
     kworker/0:0-8071  [000] ....  1550.009757: rtcpu_vinotify_event: tstamp:48815779895 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:48815520451 data:0x00000000
     kworker/0:0-8071  [000] ....  1550.009758: rtcpu_vinotify_event: tstamp:48815780035 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:50268196640 data:0x07020002
     kworker/0:0-8071  [000] ....  1550.009759: rtcpu_vinotify_event: tstamp:48815780195 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50268311648 data:0x10000000
     kworker/0:0-8071  [000] ....  1550.009760: rtcpu_vinotify_event: tstamp:48815780329 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50268340768 data:0x31000003
     kworker/0:0-8071  [000] ....  1550.009761: rtcpu_vinotify_event: tstamp:48815780486 tag:FS channel:0x00 frame:2 vi_tstamp:48815647191 data:0x00000010
     kworker/0:0-8071  [000] ....  1550.009762: rtcpu_vinotify_event: tstamp:48815780625 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:48815647193 data:0x00000000
     kworker/0:0-8071  [000] ....  1550.009763: rtcpu_vinotify_event: tstamp:48815780781 tag:CHANSEL_PXL_SOF channel:0x23 frame:2 vi_tstamp:48815647672 data:0x00000001
     kworker/0:0-8071  [000] ....  1550.009765: rtcpu_vinotify_event: tstamp:48815780916 tag:RESERVED_19 channel:0x23 frame:2 vi_tstamp:50272241600 data:0x08020002
     kworker/0:0-8071  [000] ....  1550.009766: rtcpu_vinotify_event: tstamp:48815781072 tag:CHANSEL_FAULT channel:0x23 frame:2 vi_tstamp:48815647754 data:0x00000100
     kworker/0:0-8071  [000] ....  1550.009767: rtcpu_vinotify_event: tstamp:48815781207 tag:RESERVED_19 channel:0x23 frame:2 vi_tstamp:50272267936 data:0x01020002
     kworker/0:0-8071  [000] ....  1550.009768: rtcpu_vinotify_event: tstamp:48815781364 tag:CHANSEL_FAULT channel:0x23 frame:2 vi_tstamp:48815776637 data:0x021a0182
     kworker/0:0-8071  [000] ....  1550.009769: rtcpu_vinotify_event: tstamp:48815781499 tag:RESERVED_19 channel:0x23 frame:2 vi_tstamp:50276372224 data:0x01020002
     kworker/0:0-8071  [000] ....  1550.009770: rtcpu_vinotify_event: tstamp:48815781653 tag:ATOMP_FRAME_DONE channel:0x23 frame:2 vi_tstamp:48815776663 data:0x00000000
     kworker/0:0-8071  [000] ....  1550.009771: rtcpu_vinotify_event: tstamp:48815781788 tag:RESERVED_19 channel:0x23 frame:2 vi_tstamp:50276377056 data:0x02020002
     kworker/0:0-8071  [000] ....  1550.009772: rtcpu_vinotify_event: tstamp:48815781945 tag:FE channel:0x00 frame:2 vi_tstamp:48815776787 data:0x00000020
     kworker/0:0-8071  [000] ....  1550.009773: rtcpu_vinotify_event: tstamp:48815782081 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:48815776790 data:0x00000000
     kworker/0:0-8071  [000] ....  1550.065718: rtcpu_vinotify_event: tstamp:48816036289 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:50276399520 data:0x07020003
     kworker/0:0-8071  [000] ....  1550.065744: rtcpu_vinotify_event: tstamp:48816036428 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50276509504 data:0x10000000
     kworker/0:0-8071  [000] ....  1550.065745: rtcpu_vinotify_event: tstamp:48816036587 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:50276538624 data:0x31000004
     kworker/0:0-8071  [000] ....  1550.065746: rtcpu_vinotify_event: tstamp:48816036725 tag:FS channel:0x00 frame:1 vi_tstamp:48815903530 data:0x00000010
     kworker/0:0-8071  [000] ....  1550.065746: rtcpu_vinotify_event: tstamp:48816036884 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:48815903534 data:0x00000000
     kworker/0:0-8071  [000] ....  1550.065747: rtcpu_vinotify_event: tstamp:48816037020 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:48815904011 data:0x00000001
     kworker/0:0-8071  [000] ....  1550.065748: rtcpu_vinotify_event: tstamp:48816037177 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:50280444384 data:0x08020003
     kworker/0:0-8071  [000] ....  1550.065749: rtcpu_vinotify_event: tstamp:48816037312 tag:CHANSEL_FAULT channel:0x23 frame:1 vi_tstamp:48815904094 data:0x00000100

Can you see any hints on this what could be the problem?
Do you think the 539 pixel height is a big problem? So I could try to improve sensor output by 1.

Do you have any more ideas on why is it kinda working on NANO but not NX. And why is it working on GStreamer but not V4L2?

Any suggestions would help!
Best regards,
Johannes

I updated our Jetson Nano to this version and it is the same behaviour.

Thank you for your support! I was able to solve my problem. In the end, the FPGA was adding 16 columns I did not expect so I set the resolution in the device tree and driver were wrong. After correcting it, the Xavier NX was able to stream without errors and with correct frames. The same goes for the Nano.

2 things are important to mention:

  • NANO was streaming without errors, even when the frames were corrupted while XAVIER NX was not streaming because of errors ( XAVIER NX VI Debug Codes ).
  • On all systems jetson was receiving frames in the resolution I was giving him on the driver and device tree, not the solution the sensor was sending
  • Wrong resolution in the driver seems to produce such errors we have been seeing.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.