Improper frames in PAL

Robotics & Edge Computing Jetson Systems Jetson TX2
1

Hi,
I am getting an interlaced PAL video source and it can be displayed but the gstreamer identifies the signal as progressive while the signal is interlaced, moreover I’m getting fake 120fps where actually there supposed to be 25fps:

gst-launch-1.0 -vvv v4l2src device=“/dev/video0” ! videoconvert ! glupload ! glimagesink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Got context from element ‘sink’: gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)“(GstGLDisplayX11)\ gldisplayx11-0”;
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstGLUploadElement:gluploadelement0.GstPad:src: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLUploadElement:gluploadelement1.GstPad:src: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLColorConvertElement:glcolorconvertelement0.GstPad:src: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)RGBA, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLColorBalance:glcolorbalance0.GstPad:src: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)RGBA, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink.GstPad:sink: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)RGBA, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLColorBalance:glcolorbalance0.GstPad:sink: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)RGBA, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLColorConvertElement:glcolorconvertelement0.GstPad:sink: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLUploadElement:gluploadelement1.GstPad:sink: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0.GstGhostPad:sink: caps = video/x-raw(memory:GLMemory), width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive, texture-target=(string)2D
/GstPipeline:pipeline0/GstGLUploadElement:gluploadelement0.GstPad:sink: caps = video/x-raw, width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, width=(int)720, height=(int)576, framerate=(fraction)120/1, format=(string)UYVY, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.

I have set in the DT timing to interlaced.

When I run the v4l2-ctl I get the half framerate:

v4l2-ctl --device /dev/video2 --set-fmt-video=width=720,height=576,pixelformat=UYVY,field=interlaced --stream-mmap --stream-count=200
<<<<<<<<<<<<<< 12.62 fps
<<<<<<<<<<<<< 12.56 fps
<<<<<<<<<<<< 12.54 fps
<<<<<<<<<<<<< 12.53 fps
<<<<<<<<<<<< 12.52 fps
<<<<<<<<<<<<< 12.52 fps
<<<<<<<<<<<< 12.51 fps
<<<<<<<<<<<<< 12.51 fps
<<<<<<<<<<<< 12.51 fps
<<<<<<<<<<<<< 12.51 fps
<<<<<<<<<<<< 12.51 fps
<<<<<<<<<<<<< 12.51 fps
<<<<<<<<<<<< 12.50 fps
<<<<<<<<<<<<< 12.50 fps
<<<<<<<<<<<< 12.50 fps
<<<<<<<<<<<

Any suggestions were might be the problem?

5

Hi,
Since we don’t have plugin to support hardware deinterlacing, please check if you can run command like:

$ gst-launch-1.0 v4l2src ! deinterlace ! videoconvert ! xvimagesink sync=0

And if the framerate is expected.

6

@DaneLLL, hi,

thanks for the reply. It actually worked. The thing that pazels me is the fact the debug info provided by the gstreamer is the fact that there are 120fps, does it mean that the CPU is more utilized than with 25fps?

Other things are:

  1. why the same video works fine with v4l2src but not with nvv432camerasrc (whether I set the input format to progressive or interlaced)
  2. why the fps is so low 12.5 fps and not 25.
  3. why when running the above line the gstreamer identifies the input video as progressive, and thus why is the deinterlace entity required?
  4. “Since we don’t have plugin to support hardware…” what is the meaning of “Interlaced Capture” in TRM section 27.3 ?
  5. Another thing that I’ve noticed was that removing the deinterlace entity affects nothing and the only feature that actually remove the “There may be a timestamping problem, or this computer is too slow” message was the sync=0 which ignores not in-time frames.

Regards.

7

Hi,
There is no plugin supporting deinterlacing on hardware block, so you wound need to use deinterlace plugin. If you need video encoding, need to run like:

$ gst-launch-1.0 v4l2src ! deinterlace ! nvvidconv ! nvv4l2h264enc ! ...

There is synchronization mechanism in gstreamer frameworks and in certain use-cases it may not work properly. Since v4l2 source is live source and generate frames in steady framerate(fixed interval), it should be fine to run in sync=0.

For half framerate, please check the sensor driver to make it work in v4l2-ctl command. Generally if v4l2-ctl works, it works the same in gstreamer command.

8

@DaneLLL, hi,

as I said the gstreamer’s pipeline copes OK without the deinterlace entity.

Though I’ve started debugging the incoming stream and I see the reason for the low frame rate there are some failed frames. Though, I’m confused by the data depicted there could you please enlight my eyes what is going on there, and advise how to identify the core reason for the failures.
I can see a good sequence of frame messages there:

  • FS
  • CHANSEL_PXL_SOF
  • ATOMP_FS
  • CHANSEL_LOAD_FRAMED
  • CHANSEL_PXL_EOF
  • ATOMP_FRAME_DONE
  • FE
  • ATOMP_FE

but once in a while I do not see the tegra_channel_capture_frame message which indicates a good frame was received (I guess) on the other hand I can the rtos_queue_peek_from_isr_failed message indicating a bad frame was received. But I think that I did not turn on enough debug features to understand the reason for the failures:
a good frame:

kworker/4:2-1629 [004] … 316.962405: rtcpu_vinotify_event: tstamp:10271301713 tag:FS channel:0x00 frame:2 vi_tstamp:10271301345 data:0x00000012
kworker/4:2-1629 [004] … 316.962409: rtcpu_vinotify_event: tstamp:10271308176 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:10271307696 data:0x00000001
kworker/4:2-1629 [004] … 316.962420: rtcpu_vinotify_event: tstamp:10271308338 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:10271307752 data:0x00000000
kworker/4:2-1629 [004] … 316.962424: rtcpu_vinotify_event: tstamp:10271310572 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:2 vi_tstamp:10271310218 data:0x08000000
vi-output, ov23-8697 [004] … 316.983325: tegra_channel_capture_frame: sof:316.845144857
kworker/4:2-1629 [004] … 317.018336: rtcpu_vinotify_event: tstamp:10272509767 tag:CHANSEL_PXL_EOF channel:0x00 frame:2 vi_tstamp:10272509238 data:0x023f0002
kworker/4:2-1629 [004] … 317.018341: rtcpu_vinotify_event: tstamp:10272509882 tag:ATOMP_FRAME_DONE channel:0x00 frame:2 vi_tstamp:10272509244 data:0x00000000
kworker/4:2-1629 [004] … 317.018342: rtcpu_vinotify_event: tstamp:10272513789 tag:FE channel:0x00 frame:2 vi_tstamp:10272513238 data:0x00000022
kworker/4:2-1629 [004] … 317.018343: rtcpu_vinotify_event: tstamp:10272513923 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:10272513238 data:0x00000000
a good frame:
kworker/4:2-1629 [004] … 317.018345: rtcpu_vinotify_event: tstamp:10272551607 tag:FS channel:0x00 frame:1 vi_tstamp:10272551234 data:0x00000012
kworker/4:2-1629 [004] … 317.018346: rtcpu_vinotify_event: tstamp:10272558060 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:10272557586 data:0x00000001
kworker/4:2-1629 [004] … 317.018348: rtcpu_vinotify_event: tstamp:10272558217 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:10272557641 data:0x00000000
kworker/4:2-1629 [004] … 317.018349: rtcpu_vinotify_event: tstamp:10272560886 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:1 vi_tstamp:10272560532 data:0x08000000
vi-output, ov23-8697 [004] … 317.063309: tegra_channel_capture_frame: sof:316.925137721
kworker/4:2-1629 [004] … 317.074116: rtcpu_vinotify_event: tstamp:10273759697 tag:CHANSEL_PXL_EOF channel:0x00 frame:1 vi_tstamp:10273759127 data:0x023f0002
kworker/4:2-1629 [004] … 317.074118: rtcpu_vinotify_event: tstamp:10273759890 tag:ATOMP_FRAME_DONE channel:0x00 frame:1 vi_tstamp:10273759151 data:0x00000000
kworker/4:2-1629 [004] … 317.074119: rtcpu_vinotify_event: tstamp:10273763714 tag:FE channel:0x00 frame:1 vi_tstamp:10273763127 data:0x00000022
kworker/4:2-1629 [004] … 317.074120: rtcpu_vinotify_event: tstamp:10273763919 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:10273763127 data:0x00000000
a bad frame?:
kworker/4:2-1629 [004] … 317.074120: rtcpu_vinotify_event: tstamp:10273801537 tag:FS channel:0x00 frame:2 vi_tstamp:10273801123 data:0x00000012
kworker/4:2-1629 [004] … 317.074121: rtcpu_vinotify_event: tstamp:10273807993 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:10273807474 data:0x00000001
kworker/4:2-1629 [004] … 317.074122: rtcpu_vinotify_event: tstamp:10273808251 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:10273807530 data:0x00000000
kworker/4:2-1629 [004] … 317.074123: rtcpu_vinotify_event: tstamp:10273810917 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:2 vi_tstamp:10273810525 data:0x08000000
kworker/4:2-1629 [004] … 317.074125: rtos_queue_peek_from_isr_failed: tstamp:10274408535 queue:0x0b4b4500
kworker/4:2-1629 [004] … 317.074126: rtcpu_vinotify_event: tstamp:10275009538 tag:CHANSEL_PXL_EOF channel:0x00 frame:2 vi_tstamp:10275009016 data:0x023f0002
kworker/4:2-1629 [004] … 317.074126: rtcpu_vinotify_event: tstamp:10275009653 tag:ATOMP_FRAME_DONE channel:0x00 frame:2 vi_tstamp:10275009022 data:0x00000000
kworker/4:2-1629 [004] … 317.074127: rtcpu_vinotify_event: tstamp:10275013565 tag:FE channel:0x00 frame:2 vi_tstamp:10275013015 data:0x00000022
kworker/4:2-1629 [004] … 317.074128: rtcpu_vinotify_event: tstamp:10275013706 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:10275013015 data:0x00000000
a bad frame?:
kworker/4:2-1629 [004] … 317.074128: rtcpu_vinotify_event: tstamp:10275051376 tag:FS channel:0x00 frame:1 vi_tstamp:10275051012 data:0x00000012
kworker/4:2-1629 [004] … 317.074129: rtcpu_vinotify_event: tstamp:10275057840 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:10275057363 data:0x00000001
kworker/4:2-1629 [004] … 317.074130: rtcpu_vinotify_event: tstamp:10275057998 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:10275057419 data:0x00000000
kworker/4:2-1629 [004] … 317.074130: rtcpu_vinotify_event: tstamp:10275060445 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:1 vi_tstamp:10275060093 data:0x08000000
kworker/4:2-1629 [004] … 317.130278: rtcpu_vinotify_event: tstamp:10276259453 tag:CHANSEL_PXL_EOF channel:0x00 frame:1 vi_tstamp:10276258905 data:0x023f0002
kworker/4:2-1629 [004] … 317.130305: rtcpu_vinotify_event: tstamp:10276259606 tag:ATOMP_FRAME_DONE channel:0x00 frame:1 vi_tstamp:10276258920 data:0x00000000
kworker/4:2-1629 [004] … 317.130309: rtcpu_vinotify_event: tstamp:10276263474 tag:FE channel:0x00 frame:1 vi_tstamp:10276262904 data:0x00000022
kworker/4:2-1629 [004] … 317.130313: rtcpu_vinotify_event: tstamp:10276263642 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:10276262905 data:0x00000000
a good frame:
kworker/4:2-1629 [004] … 317.130316: rtcpu_vinotify_event: tstamp:10276301287 tag:FS channel:0x00 frame:2 vi_tstamp:10276300901 data:0x00000012
kworker/4:2-1629 [004] … 317.130320: rtcpu_vinotify_event: tstamp:10276307757 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:10276307252 data:0x00000001
kworker/4:2-1629 [004] … 317.130323: rtcpu_vinotify_event: tstamp:10276307964 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:10276307308 data:0x00000000
kworker/4:2-1629 [004] … 317.130327: rtcpu_vinotify_event: tstamp:10276311094 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:2 vi_tstamp:10276310723 data:0x08000000
vi-output, ov23-8697 [004] … 317.143440: tegra_channel_capture_frame: sof:317.5130617
kworker/4:2-1629 [004] … 317.186221: rtcpu_vinotify_event: tstamp:10277509635 tag:CHANSEL_PXL_EOF channel:0x00 frame:2 vi_tstamp:10277508794 data:0x023f0002
kworker/4:2-1629 [004] … 317.186230: rtcpu_vinotify_event: tstamp:10277510131 tag:ATOMP_FRAME_DONE channel:0x00 frame:2 vi_tstamp:10277508890 data:0x00000000
kworker/4:2-1629 [004] … 317.186232: rtcpu_vinotify_event: tstamp:10277513532 tag:FE channel:0x00 frame:2 vi_tstamp:10277512793 data:0x00000022
kworker/4:2-1629 [004] … 317.186233: rtcpu_vinotify_event: tstamp:10277514027 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:10277512793 data:0x00000000
a bad frame?:
kworker/4:2-1629 [004] … 317.186235: rtcpu_vinotify_event: tstamp:10277551363 tag:FS channel:0x00 frame:1 vi_tstamp:10277550790 data:0x00000012
kworker/4:2-1629 [004] … 317.186236: rtcpu_vinotify_event: tstamp:10277557830 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:10277557141 data:0x00000001
kworker/4:2-1629 [004] … 317.186238: rtcpu_vinotify_event: tstamp:10277558476 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:10277557197 data:0x00000000
kworker/4:2-1629 [004] … 317.186239: rtcpu_vinotify_event: tstamp:10277565285 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:1 vi_tstamp:10277564742 data:0x08000000
kworker/4:2-1629 [004] … 317.186240: rtcpu_vinotify_event: tstamp:10278759253 tag:CHANSEL_PXL_EOF channel:0x00 frame:1 vi_tstamp:10278758682 data:0x023f0002
kworker/4:2-1629 [004] … 317.186242: rtcpu_vinotify_event: tstamp:10278759442 tag:ATOMP_FRAME_DONE channel:0x00 frame:1 vi_tstamp:10278758707 data:0x00000000
kworker/4:2-1629 [004] … 317.186243: rtcpu_vinotify_event: tstamp:10278763270 tag:FE channel:0x00 frame:1 vi_tstamp:10278762682 data:0x00000022
kworker/4:2-1629 [004] … 317.186245: rtcpu_vinotify_event: tstamp:10278763472 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:10278762683 data:0x00000000
a bad frame?:
kworker/4:2-1629 [004] … 317.186246: rtcpu_vinotify_event: tstamp:10278801283 tag:FS channel:0x00 frame:2 vi_tstamp:10278800679 data:0x00000012
kworker/4:2-1629 [004] … 317.186247: rtcpu_vinotify_event: tstamp:10278807715 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:10278807030 data:0x00000001
kworker/4:2-1629 [004] … 317.186249: rtcpu_vinotify_event: tstamp:10278808337 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:10278807086 data:0x00000000
kworker/4:2-1629 [004] … 317.186250: rtcpu_vinotify_event: tstamp:10278815230 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:2 vi_tstamp:10278814689 data:0x08000000
vi-output, ov23-8697 [004] … 317.223506: tegra_channel_capture_frame: sof:317.85123513
kworker/4:2-1629 [004] … 317.242373: rtos_queue_peek_from_isr_failed: tstamp:10279408774 queue:0x0b4b4500
kworker/4:2-1629 [004] … 317.242395: rtcpu_vinotify_event: tstamp:10280009424 tag:CHANSEL_PXL_EOF channel:0x00 frame:2 vi_tstamp:10280008572 data:0x023f0002
kworker/4:2-1629 [004] … 317.242398: rtcpu_vinotify_event: tstamp:10280009924 tag:ATOMP_FRAME_DONE channel:0x00 frame:2 vi_tstamp:10280008666 data:0x00000000
kworker/4:2-1629 [004] … 317.242402: rtcpu_vinotify_event: tstamp:10280013311 tag:FE channel:0x00 frame:2 vi_tstamp:10280012571 data:0x00000022
kworker/4:2-1629 [004] … 317.242405: rtcpu_vinotify_event: tstamp:10280013807 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:10280012571 data:0x00000000
a bad frame?:
kworker/4:2-1629 [004] … 317.242410: rtcpu_vinotify_event: tstamp:10280051173 tag:FS channel:0x00 frame:1 vi_tstamp:10280050568 data:0x00000012
kworker/4:2-1629 [004] … 317.242413: rtcpu_vinotify_event: tstamp:10280057589 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:10280056919 data:0x00000001
kworker/4:2-1629 [004] … 317.242416: rtcpu_vinotify_event: tstamp:10280058245 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:10280056975 data:0x00000000
kworker/4:2-1629 [004] … 317.242419: rtcpu_vinotify_event: tstamp:10280068053 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:1 vi_tstamp:10280067503 data:0x08000000
kworker/4:2-1629 [004] … 317.298324: rtcpu_vinotify_event: tstamp:10281259294 tag:CHANSEL_PXL_EOF channel:0x00 frame:1 vi_tstamp:10281258460 data:0x023f0002
kworker/4:2-1629 [004] … 317.298350: rtcpu_vinotify_event: tstamp:10281259885 tag:ATOMP_FRAME_DONE channel:0x00 frame:1 vi_tstamp:10281258557 data:0x00000000
kworker/4:2-1629 [004] … 317.298354: rtcpu_vinotify_event: tstamp:10281263196 tag:FE channel:0x00 frame:1 vi_tstamp:10281262460 data:0x00000022
kworker/4:2-1629 [004] … 317.298357: rtcpu_vinotify_event: tstamp:10281263686 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:10281262460 data:0x00000000
a good frame?:
kworker/4:2-1629 [004] … 317.298361: rtcpu_vinotify_event: tstamp:10281301059 tag:FS channel:0x00 frame:2 vi_tstamp:10281300457 data:0x00000012
kworker/4:2-1629 [004] … 317.298365: rtcpu_vinotify_event: tstamp:10281307472 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:10281306808 data:0x00000001
kworker/4:2-1629 [004] … 317.298368: rtcpu_vinotify_event: tstamp:10281308110 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:10281306864 data:0x00000000
kworker/4:2-1629 [004] … 317.298371: rtcpu_vinotify_event: tstamp:10281316726 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:2 vi_tstamp:10281316143 data:0x08000000
vi-output, ov23-8697 [004] … 317.303494: tegra_channel_capture_frame: sof:317.165116409
kworker/4:2-1629 [004] … 317.354299: rtcpu_vinotify_event: tstamp:10282509194 tag:CHANSEL_PXL_EOF channel:0x00 frame:2 vi_tstamp:10282508350 data:0x023f0002
kworker/4:2-1629 [004] … 317.354332: rtcpu_vinotify_event: tstamp:10282509839 tag:ATOMP_FRAME_DONE channel:0x00 frame:2 vi_tstamp:10282508443 data:0x00000000
kworker/4:2-1629 [004] … 317.354336: rtcpu_vinotify_event: tstamp:10282513084 tag:FE channel:0x00 frame:2 vi_tstamp:10282512349 data:0x00000022
kworker/4:2-1629 [004] … 317.354341: rtcpu_vinotify_event: tstamp:10282513578 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:10282512349 data:0x00000000

Another thing that I’ve noticed was the fact that if I reduce the time stamp of each frame (let’s say of the FS token) I am getting the following time difference (~msec): 56,56,0,56,56,0,56,56, while I expect the time differnce between PAL (this is the video input signal) frames to be 40msec or PAL fields to be 20msec.

Regards.

9

Hi,
Would like to have more information. So now you can successfully run:

v4l2-ctl --device /dev/video2 --set-fmt-video=width=720,height=576,pixelformat=UYVY,field=interlaced --stream-mmap --stream-count=200

But if you dump the UYVY frames:

v4l2-ctl --device /dev/video2 --set-fmt-video=width=720,height=576,pixelformat=UYVY,field=interlaced --stream-mmap --stream-count=200 --stram-to=dump.UYVY

The data is wrong?

Not sure but probably the source runs in 720x288p25. So if you treat it as 720x576 input, it becomes 12.5fps.

10

Hi,

Data is OK. the problem is that it comes at a lower rate. Moreover when I run

gst-launch-1.0 v4l2src ! videoconvert ! xvimagesink sync=0

I can see an image but because the video source is static I do not experience the low rate artifacts.

Are the frames good in my previous message? should the time of FS received indicate of the problem with the arrival of the frames?

11

hello igal.kroyter,

could you please examine and share the format dumps by v4l utility.
for example, $ v4l2-ctl -d /dev/video0 --list-formats-ext

12

@JerryChang , hi,

following is the dump per your request:

ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2

13

hello igal.kroyter,

don’t there show the available sensor modes? it’ll include the active resolution and fps.

14

hello @JerryChang,

no at the moment that’s all. Is my driver missing something?
Is it possible that you meant? :

v4l2-ctl --all -d /dev/video0 --verbose

VIDIOC_QUERYCAP: ok
Driver Info (not using libv4l2):
Driver name : tegra-video
Card type : vi-output, ov23850 2-0036
Bus info : platform:15700000.vi:0
Driver version: 4.9.201
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: no power)
Format Video Capture:
Width/Height : 720/576
Pixel Format : ‘UYVY’
Field : None
Bytes per Line : 1536
Size Image : 1769472
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :

Camera Controls

               frame_length 0x009a2000 (int)    : min=0 max=32767 step=1 default=4806 value=4806 flags=slider
                coarse_time 0x009a2001 (int)    : min=2 max=32735 step=1 default=4774 value=4774 flags=slider
          coarse_time_short 0x009a2002 (int)    : min=2 max=32735 step=1 default=4774 value=4774 flags=slider
                 group_hold 0x009a2003 (intmenu): min=0 max=1 default=0 value=0
                 hdr_enable 0x009a2004 (intmenu): min=0 max=0 default=0 value=0
                eeprom_data 0x009a2005 (str)    : min=0 max=2048 step=2 value='' flags=volatile, has-payload
                   otp_data 0x009a2006 (str)    : min=0 max=64 step=2 value='' flags=read-only, has-payload
                    fuse_id 0x009a2007 (str)    : min=0 max=32 step=2 value='' flags=read-only, has-payload
                       gain 0x009a2009 (int)    : min=256 max=4096 step=1 default=256 value=256 flags=slider
                bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
            override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
               height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                 size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
           write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
   sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
    sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
  sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
          sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
           low_latency_mode 0x009a206d (bool)   : default=0 value=0
           preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
               sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only
17

hello igal.kroyter,

is it an OmniVision’s OV23850?
according to its supports output formats, https://www.ovt.com/sensors/OV23850, it’s pixel formats should be RAW instead of YUV.
are you having external ISP to process debayer process? could you please also have confirmation.
thanks

18

hello @JerryChang,

we’re using a Latice FPGA with an embedded CSI-2 engine. I’ve ustilized the OV23850 driver as a reference.

Regards.

19

hello igal.kroyter,

I see.

  1. could you please double check you’re given enough clock rate with this CSI data rate calculation formula.
    you should include 15% protocol overhead, it should be considered for CSI data rate calculation.
    i.e. CSI data rate = width * height * bpp * fps * 1.15

  2. is this an interlaced input? for example, for a 1920x1080i capture, you should configure height as 540 instead of 1080.
    you should also have, is_interlaced = 1; property assign in the sensor mode of the device tree.

  3. following above, by default the interlaced captures will be top and bottom.

20

Hello @JerryChang,

here after my DT definition:

i2c@3180000 {
ov23850_a@36 {
compatible = “nvidia,ov23850”;
reg = <0x36>;

		clocks					= <&tegra_car TEGRA186_CLK_EXTPERIPH1>, <&tegra_car TEGRA186_CLK_PLLP_OUT0>;
		clock-names				= "extperiph1", "pllp_grtba";
		mclk					= "extperiph1";
		**clock-frequency			= <24000000>;**
		status					= "okay";
		mode0 {
			**mclk_khz			= "24000";**
			num_lanes			= "1";
			tegra_sinterface 	= "serial_a";
			discontinuous_clk	= "no";
			dpcm_enable			= "false";
			cil_settletime		= "0";
			phy_mode			= "DPHY";

			mode_type			= "yuv";
			pixel_phase			= "uyvy";
			csi_pixel_bit_depth = "16";
			dynamic_pixel_bit_depth = "16";

			active_w			= "720";
			**active_h			= "576";**
			**is_interlaced		= "1";**

** interlaced_type = “1”; // 1 - Interleaved ; 0 - Top_Bottom**
readout_orientation = “0”;
line_length = “864”;
inherent_gain = “1”;
mclk_multiplier = “6”;
pix_clk_hz = “13500000”;

			gain_factor			= "10";
			min_gain_val		= "0"; /* 0dB */
			max_gain_val		= "480"; /* 48dB */
			step_gain_val		= "3"; /* 0.3 */
			default_gain		= "0";
			min_hdr_ratio		= "1";
			max_hdr_ratio		= "1";
			framerate_factor	= "1000000";
			min_framerate		= "1500000"; /* 1.5 */
			max_framerate		= "60000000"; /* 60 */
			step_framerate		= "1";
			default_framerate	= "25000000";
			exposure_factor		= "1000000";
			min_exp_time		= "30"; /* us */
			max_exp_time		= "660000"; /* us */
			step_exp_time		= "1";
			default_exp_time	= "33334";/* us */
		
			horz_front_porch	= "12";
			horz_sync			= "63";
			horz_back_porch		= "69";
			vert_front_porch	= "2";
			vert_sync			= "19";
			vert_back_porch		= "3";				
		};

		ports {
			#address-cells	= <1>;
			#size-cells		= <0>;

			port@0 {
				reg		= <0>;
				no_ov23850_out0: endpoint {
					port-index		= <0>;
					bus-width		= <1>;
					remote-endpoint = <&ov23850_csi_in0>;
				};
			};
		};
	};

I coudn’t find any CSI specific clocks in the DT, though 24MHz>13.5MHz (I think these are the relevant clocks?). I have tried both options the interleaved and top_bottom.
I’ve tried to set the actual height to 288 instead of 576, but it was only sampling the first field.

Regards.

21

hello igal.kroyter,

did you see framerate improvements by having interlaced type and setting the actual height to 288 instead of 576?

22

hello @JerryChang,

when I modified the interlace type the fields were one above the other. When reducing the height I sa no improvement.

23

hello igal.kroyter,

it’s pixel clock to calculate the exposure and frame rate of the sensor.
per the previous calculation formula, it’s… 720 * 576 * 25 * 16 * 1.15 = 190,771,200
hence, please have a try to increase pix_clk_hz and check the results.
thanks

24

Hello @JerryChang,

could you please verify my calculation: as the video also incorporates the VBI lines and horizontal syncs thus the clock calculation should be as following:
864 (720) x 625 x 25 x16 x 1.15 = 248,400,000[Hz]
Moreover, the name of the field is pixel_clk_Hz, why should bits per pixel be involved? if so it should have been called bit_clk_hz, no?

25

hello igal.kroyter,

please check developer guide, Sensor Pixel Clock for the several ways to check the correct sensor pixel clock rate,