Occasional loss of frame data in video stream

orin nx 8GB,Jetpack 6.0DP,R36.2
I build the pipeline using the following command:

gst-launch-1.0 videotestsrc num-buffers=300 animation-mode=2 horizontal-speed=10 ! \
     'video/x-raw, width=(int)1280, height=(int)720, \
     format=(string)I420, framerate=(fraction)30/1' ! nvvidconv ! \
     'video/x-raw(memory:NVMM), format=(string)NV12' ! nvv4l2h264enc \
     bitrate=6000000 peak-bitrate=6500000 ! h264parse ! qtmux ! \
     filesink location=filename_h264.mp4 -e

occasional loss of frame data, making the video look jumpy, occurring about once every second.
I use the above example to facilitate the reproduction, “occasional loss of frame data” this problem has been plaguing us for a long time, whether we use C language to create RTSP, or in various ways to build the pipeline, we can observe this phenomenon, sometimes feel very obvious, sometimes not easy to detect, only a small number of colleagues can detect this phenomenon, I’m not sure whether it is a GSTREAMER or JETSON problem, or we are missing a key piece of code.


This one is a video recorded via the above command. The strange thing is that sometimes the recording looks fine.

Hi,
6.0DP is developer preview and there may be certain issues. You may try 5.1.3 and see if the issue is still present.


I am using a Xavier NX 8g development board with L4T version 32.7.2.
With this video, still the same phenomenon.
The command at the beginning of the article has a symbolic error, please change it to the following command
sudo gst-launch-1.0 videotestsrc num-buffers=300 animation-mode=2 horizontal-speed=10 !" video/x-raw, width=1280, height=720, format=(string)I420, framerate=(fraction)30/1" ! nvvidconv !" video/x-raw(memory:NVMM), format=(string)NV12" ! nvv4l2h264enc bitrate=6000000 peak-bitrate=6500000 ! h264parse ! qtmux ! filesink location=filename_h264.mp4 -e

Hi,
Please add is-live=1 to videotestsrc for a try. It looks like videotestsrc does not generate frame data correctly. we have never seen this in real camera sources.

ORIN NX 16G development board, L4T version 35.3.1, JetPack 5.1.1.
Still the same phenomenon.
sudo gst-launch-1. 0 videotestsrc num-buffers=300 animation-mode=2 horizontal-speed=10 !" video/x-raw, width=1280, height=720, format=(string)I420, framerate=(fraction)30/1" !nvvidconv !" video/x-raw(memory: NVMM, format=(string)NV12"! nvv4l2h264enc bitrate=6000000 peak-bitrate=6500000! h264parse! qtmux! filesink location=filename_h264. mp4 -e

Okay, I’ll try.

ORIN NX 16G development board, L4T version 35.3.1, JetPack 5.1.1.
sudo gst-launch-1.0 videotestsrc is-live=1 animation-mode=2 horizontal-speed=10 ! “video/x-raw, width=1280, height=720, format=(string)I420, framerate=(fraction)30/1” ! nvvidconv ! “video/x-raw(memory:NVMM), format=(string)NV12” ! nvv4l2h264enc bitrate=6000000 peak-bitrate=6500000 ! h264parse ! qtmux ! filesink location=filename_h264.mp4 -e
I added is-live=1. Still the same phenomenon.
We are also experiencing this issue with a product we are developing at Xavier NX that uses v4l2src to acquire video data and then pushes the stream through RTSP. The pipeline is similar to the above command, except that it is built in C and is much more complex.

Hi,
Please try CBR + virtual buffer size:
Random blockiness in the picture RTSP server-client -Jetson TX2 - #5 by DaneLLL

This is the setup which balances video quality+bitrate. Please try and see if there is improvement.

I don’t have 5.1.3 orin nx at the moment, I used orin nx 16GB, Jetpack 6.0DP, R36.2.
I followed the instructions above and set vbv-size to: baud rate/frame rate = 6000000/30 = 200000 and EnableTwopassCBR = 1, but I don’t feel any improvement.

sudo gst-launch-1.0 videotestsrc is-live=1 num-buffers=300 animation-mode=2 horizontal-speed=10 ! "video/x-raw, width=1280, height=720, format=(string)I420, framerate=(fraction)30/1" ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)NV12" ! nvv4l2h264enc bitrate=6000000 peak-bitrate=6500000 vbv-size=200000 EnableTwopassCBR=1 ! h264parse ! qtmux ! filesink location=filename_h264.mp4 -e

Hi,
Please try software encoder x264enc and see if the issue is present or not. It still looks like the source doesn’t generate the frames in a steady interval, so it looks like certain frames are lost.

I don’t have 5.1.3 orin nx at the moment, I used orin nx 16GB, Jetpack 6.0DP, R36.2.

sudo gst-launch-1.0 videotestsrc is-live=1 num-buffers=300 animation-mode=2 horizontal-speed=10 ! "video/x-raw, width=1280, height=720, format=(string)I420, framerate=(fraction)30/1" ! x264enc bitrate=2000000 ! h264parse ! qtmux ! filesink location=filename_h264.mp4 -e

I used x264enc and the problem is still there.

Since the discontinuity also occurs in using software encoder, it looks to be the behavior of videotestsrc that does not generate frames smoothly. Not likely to be an issue in software or hardware encoder.

And please upgrade to JP 6.0 GA SW if you would like to try latest version.

Are you absolutely sure there are dropped frames?
When I look at this video’s metadata, I don’t see any evidence that there are dropped frames. It’s exactly 300 frames long, and with num-buffers=300, suggests it hasn’t dropped any frames. All of the PTSs look correct as well, each being 0.033333s apart.

Perhaps whatever you’re using to play out the video is dropping frames.