Nvv4l2h264enc dropping frames on live input

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

I am currently working on a project involving recording from an HDMI v4l2 capture device using a Jetson Xavier NX on a custom carrier board. When the video source is set to 1280x720@60Hz, I use the following pipeline to capture:

gst-launch-1.0 nvv4l2camerasrc device=“/dev/video8” ! ‘video/x-raw(memory:NVMM), width=1280, height=720, format=UYVY, framerate=60/1’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=60/1’ ! nvv4l2h264enc control-rate=0 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! h264parse ! avimux ! filesink location=/remote/Videos/gstlaunch.avi

/remote is an NFS share and the device is /dev/video8 because there is an 8-channel SD capture device (currently unused).

For 1920x1080 capture, I use:

gst-launch-1.0 nvv4l2camerasrc device=“/dev/video8” ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, format=UYVY, framerate=30/1’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=30/1’ ! nvv4l2h264enc control-rate=0 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! h264parse ! avimux ! filesink location=/remote/Videos/gstlaunch.avi

I am finding that when encoding, I lose approximately 38 frames out of every 5000 for 1080p30 and 114 frames out of every 5000 for 720p60. Frame drops at 1920x1080 seem to be singular, whereas at 1280x720, they seem to happen in groups of 7-8.

I am using a signal generator that embeds a timestamp in each frame so that I can detect gaps between frames at the recording and capture levels. I have confirmed that there are no gaps when capturing directly to memory from video4linux, so it looks as if the H.264 encoder is doing this.

I have seem similar behaviour with the omxh264enc plugin as well as nvv4l2h265enc. I have also noticed that when using a transport stream or Matroska mux instead of AVI, it will inject duplicate frames to make up for the dropped ones.

I would be grateful for any suggestions about how to eliminate frame drops.

2

perhaps H264/265 is inherently lossy?

4

Hi,
Please try this:
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL

And set the property to nvv4l2h264enc:

  maxperf-enable      : Enable or Disable Max Performance mode
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false

Furthermore, please execute sudo nvpmodel -m 2 and sudo jetson_clocks. The commands can enable CPU/GPU/EMC at maximum clocks.

5

Thank you for the suggestion. Unfortunately, maxperf-enable doesn’t seem to have yielded a significant improvement (the drops went down to 108 frames per 5000).

The additional changes have, if anything, made matters slightly worse. After setting sudo nvpmodel -m 2 and sudo jetson_clocks and following the instructions in the link, the drops per 5000 frames have increased to 140.

6

Hi,
After the execution the system is in maximum throughput. A bit strange you see worse performance. Probably the camera source does not generate frames steadily. If the source is connected to NVCSI port, you may try this:
https://elinux.org/Jetson/l4t/Camera_BringUp#Tips_for_debugging

See if boosting the clocks helps.

7

I do not see the same frame dropping issue when using a V4L application to capture the raw frames to memory. The HDMI signal generator puts a machine-readable timestamp in the picture and the V4L application can read that timestamp and log to a file.

I have also observed that simplifying the test pattern causes a reduction in the number of frames dropped.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.