Issue with Corrupted Frames and Frame Discarding in GStreamer 1.16 on Xavier NX (L4T 35.5.0, Kernel 5.10)

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
,
1

Dear NVIDIA Community,

I’m encountering an issue related to handling corrupted video frames after porting my code from a 4.9 kernel to a 5.10 kernel on a Xavier NX platform (L4T 35.5.0).

In the older kernel (4.9) using the videobuf2 framework, I believe there was a “requeue” option that allowed for the re-submission of buffers. However, this feature seems to be removed in kernel 5.4 and later, as I understand it.

Currently, with the 5.10 kernel, when a frame received from the kernel is in an error state (specifically, the VB2_BUF_STATE_ERROR flag is set in the kernel space), GStreamer 1.16 still passes this frame to the userspace pipeline. This results in errors and the corrupted frame being potentially displayed, which is undesirable.

My goal is to discard these corrupted frames within the GStreamer pipeline before they reach the display.

My questions are:

  1. Is there a recommended approach in GStreamer 1.16 to detect and discard frames that have been flagged with VB2_BUF_STATE_ERROR by the kernel?
  2. Are there specific GStreamer elements or properties that can be used to achieve this frame discarding based on metadata or buffer flags?
  3. Could the removal of the “requeue” option in newer kernels be directly contributing to this behavior in GStreamer?
  4. Are there any known issues or best practices related to handling VB2_BUF_STATE_ERROR flags within GStreamer on the Xavier NX platform with L4T 35.5.0 and a 5.10 kernel?
  5. Looking ahead, do future versions of GStreamer offer more explicit support for discarding frames reported as erroneous from kernel space? If so, could you provide some details or point me towards relevant documentation?

Any insights, suggestions, or pointers on how to effectively discard these corrupted frames within my GStreamer 1.16 pipeline on the Xavier NX would be greatly appreciated.

Thank you for your time and assistance.

Sincerely,
Hari

3

Hi,
For information, which source plugin is used? Would like to know whether you use nvarguscamerasrc or v4l2src.

4

Hi,

I am using nvv4l2camerasrc in gstreamer pipeline.

More specifically, in the file kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi5_fops.c, I suspect this line below in the vi5_capture_dequeue(...) function is the reason for not truncating the frames in the kernel space:

if (frame_err)
#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 4, 0)
		buf->vb2_state = VB2_BUF_STATE_REQUEUEING;
#else
		buf->vb2_state = VB2_BUF_STATE_ERROR;
#endif
	else
		buf->vb2_state = VB2_BUF_STATE_DONE;

Is there any way I could stop the frame on the kernel side and requeue it to userspace, or is there any way on the GStreamer side I can handle this?

Thanks.

6

Hi,
The source code of v4l2src plugin is public, so you may check the source code and see if it receives the error. If the error can be reported to v4l2src plugin, you can ty to add code to drop the frames.

The nvv4l2camerasrc plugin is also public. You can download it and check:

Jetson Linux | NVIDIA Developer
Driver Package (BSP) Sources