I am using a RPi 2 camera (IMX219) with my Jetson Nano and I am trying to get high framerate 640x480 video. I am targeting 180fps as I’ve seen people achieve this with this sensor at this resolution on other forums. I modified the stock Nvidia V4L2 driver to add this resolution mode (adding stuff to
tegra210-camera-rbpcv2-imx219.dtsi), and I am able to start a stream from a C program.
imx219_mode_tbls.h, I needed to define a new set of register values for my new resolution mode. I copied the values from
imx219_mode_1280x720_60fps and modified the ones relevant to the sensor cropping, binning, and output resolution (same for the stuff in the device tree file).
When I run my application, the video output is glitchy because the
VIDIOC_DQBUF IOCTL seems to be returning empty or partially-filled frame buffers (i.e. half the scan lines of the buffer are updated, the rest are old). I am using four user pointer buffers. It seems like one of the four framebuffers is updated, but only part of it is filled. When I change the
frame_rate control while my application is running using
v4l2-ctl, I found that lower framerates did not produce this problem.
With the default pixel clock settings (182.4MHz as defined in the device tree and sensor registers, again copied from the 720p60 mode), the video stream is fine up until about 110fps. If I changed the registers and device tree to 169.6MHz (as used in the 720p120 line-doubled mode), the video works up to about 100fps. If I increase the pixel clock to 224MHz (extrapolating the PLL multiplier register values from the ones in the other modes), it works up to about 130fps. In all cases, it works fine until you increase the framerate by 1 and it immediately starts glitching out.
I have also tried using the nonblocking driver mode and increasing the number of framebuffers to 10, but this didn’t change the behavior. In no case does the IOCTL return any errors (flags in the
v4l2_buffer struct or errno).
640x480 180fps video has fewer pixels per second than some of the other stock modes, which all work fine (including 720p 120fps), so I think the clock speed isn’t truly the limiting factor here. I don’t want to just keep pushing the clock speed up, since other drivers (for the Raspberry Pi) that supposedly achieve 180fps at 480p are using 182.4MHz. I am not familiar with the details of the CSI interface or how the V4L2 driver is implemented, so does anyone recognize what might be going on here and how I can fix it?