[Custom Board] FPS control by editing uvcvideo driver, but the result is not correct

I edit and compile the driver to filtering out some frames to get the FPS I want. For example, my camera only support 60 FPS, and I filter one in every two frames, then I will get a 30 FPS.

However, here is the results:

[16019.383083] Frame 1188 Duration: 15 ms, Total Frames: 3562
[16019.418196] Frame 1189 Start time recorded: 1732537841.015229658 seconds
[16019.433480] Frame 1189 Completion time (bulk): 1732537841.030515028 seconds
[16019.433491] Frame 1189 Duration: 15 ms, Total Frames: 3565
[16019.468593] Frame 1190 Start time recorded: 1732537841.065625954 seconds
[16019.483870] Frame 1190 Completion time (bulk): 1732537841.080905084 seconds
[16019.483881] Frame 1190 Duration: 15 ms, Total Frames: 3568
[16019.518984] Frame 1191 Start time recorded: 1732537841.116020811 seconds
[16019.534272] Frame 1191 Completion time (bulk): 1732537841.131310053 seconds
[16019.534283] Frame 1191 Duration: 15 ms, Total Frames: 3571
[16019.569385] Frame 1192 Start time recorded: 1732537841.166420915 seconds
[16019.584666] Frame 1192 Completion time (bulk): 1732537841.181704013 seconds
[16019.584677] Frame 1192 Duration: 15 ms, Total Frames: 3574
[16019.619783] Frame 1193 Start time recorded: 1732537841.216819196 seconds
[16019.635058] Frame 1193 Completion time (bulk): 1732537841.232098038 seconds
[16019.635069] Frame 1193 Duration: 15 ms, Total Frames: 3577

At the end I only get 1193 frames of 3577 for 60 seconds, it is nearly 20 FPS.

I edit the uvc_video.c as follows:

static int uvc_video_decode_start(struct uvc_streaming *stream,
		struct uvc_buffer *buf, const u8 *data, int len)
{
	u8 fid;

	static unsigned int frame_counter = 0;  
    unsigned int interval = get_camera_fps() / get_user_fps();  

	total_frames++;

	/* Sanity checks:
	 * - packet must be at least 2 bytes long
	 * - bHeaderLength value must be at least 2 bytes (see above)
	 * - bHeaderLength value can't be larger than the packet size.
	 */
	if (len < 2 || data[0] < 2 || data[0] > len) {
		stream->stats.frame.nb_invalid++;
		return -EINVAL;
	}

	fid = data[1] & UVC_STREAM_FID;

	/* Skip current frames for FPS control */
	if (frame_counter % interval != 0) {
		frame_counter++;
		return -ENODATA;  // Skip current frame
	}


	/* Increase the sequence number regardless of any buffer states, so
	 * that discontinuous sequence numbers always indicate lost frames.
	 */
	if (stream->last_fid != fid) {
		stream->sequence++;
		if (stream->sequence)
			uvc_video_stats_update(stream);
	}

	uvc_video_clock_decode(stream, buf, data, len);
	uvc_video_stats_decode(stream, data, len);

	/* Store the payload FID bit and return immediately when the buffer is
	 * NULL.
	 */
	if (buf == NULL) {
		stream->last_fid = fid;
		return -ENODATA;
	}

	/* Mark the buffer as bad if the error bit is set. */
	if (data[1] & UVC_STREAM_ERR) {
		uvc_trace(UVC_TRACE_FRAME, "Marking buffer as bad (error bit "
			  "set).\n");
		buf->error = 1;
	}

	/* Synchronize to the input stream by waiting for the FID bit to be
	 * toggled when the the buffer state is not UVC_BUF_STATE_ACTIVE.
	 * stream->last_fid is initialized to -1, so the first isochronous
	 * frame will always be in sync.
	 *
	 * If the device doesn't toggle the FID bit, invert stream->last_fid
	 * when the EOF bit is set to force synchronisation on the next packet.
	 */
	if (buf->state != UVC_BUF_STATE_ACTIVE) {
		if (fid == stream->last_fid) {
			uvc_trace(UVC_TRACE_FRAME, "Dropping payload (out of "
				"sync).\n");
			if ((stream->dev->quirks & UVC_QUIRK_STREAM_NO_FID) &&
			    (data[1] & UVC_STREAM_EOF))
				stream->last_fid ^= UVC_STREAM_FID;
			return -ENODATA;
		}

		buf->buf.field = V4L2_FIELD_NONE;
		buf->buf.sequence = stream->sequence;
		buf->buf.vb2_buf.timestamp = ktime_to_ns(uvc_video_get_time());
		buf->idx = ++frame_idx;

		ktime_get_real_ts64(&buf->start_time);
		pr_info("Frame %u Start time recorded: %ld.%09ld seconds\n", 
				buf->idx, (long)buf->start_time.tv_sec, buf->start_time.tv_nsec);

		/* TODO: Handle PTS and SCR. */
		buf->state = UVC_BUF_STATE_ACTIVE;
	}

	/* Mark the buffer as done if we're at the beginning of a new frame.
	 * End of frame detection is better implemented by checking the EOF
	 * bit (FID bit toggling is delayed by one frame compared to the EOF
	 * bit), but some devices don't set the bit at end of frame (and the
	 * last payload can be lost anyway). We thus must check if the FID has
	 * been toggled.
	 *
	 * stream->last_fid is initialized to -1, so the first isochronous
	 * frame will never trigger an end of frame detection.
	 *
	 * Empty buffers (bytesused == 0) don't trigger end of frame detection
	 * as it doesn't make sense to return an empty buffer. This also
	 * avoids detecting end of frame conditions at FID toggling if the
	 * previous payload had the EOF bit set.
	 */
	if (fid != stream->last_fid && buf->bytesused != 0) {
		uvc_trace(UVC_TRACE_FRAME, "Frame complete (FID bit "
				"toggled).\n");
		buf->state = UVC_BUF_STATE_READY;
		return -EAGAIN;
	}

	stream->last_fid = fid;

	frame_counter++; 

	return data[0];
}

Does anyone know how to solve the problem? Thank you so much!

Help please!! Thank you so much!

USB camera?
Do you verify the real output is 60fps by v4l2-ctl?

Yes, my camera is Intel Realsense D435if.
I print the timestamp in the driver for each frame by pr_info(), and it shows totally 3564 timestamp are printed in 60 second, which is nearly 60 FPS. I think this can verify.
Here is the screenshot of such command:

sudo dmesg -w

v4l2-ctl --device=/dev/video4 --get-parm

You try get the frame from without interval to confirm the frame rate first.
Then break down to know where cause the frame drop.

Yes, I tried. Thanks for your answer and I have solved the problem by update the fid in each filter.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.