Saving video to file gives poor fps

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Nvidia agx xavier 32GB
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only) 4.5
• TensorRT Version
• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

I’m trying to make a pipeline to do both inference and also record the mjpeg video stream from from a camera into a file.
I’m currently stuck at saving the video to file where it produces video-files with errors that are not playable or poor framerate.

I’ve made one pipeline that works by playing the video output and it gives good fps.

gst-launch-1.0 v4l2src device=/dev/video0 do-timestamp=true ! image/jpeg, framerate=30/1, width=1920, height=1080 ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! ‘video/x-raw’,format=NV12 ! nveglglessink

When I try to save it to file:

gst-launch-1.0 v4l2src device=/dev/video0 do-timestamp=true ! image/jpeg, framerate=30/1, width=1920, height=1080 ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! ‘video/x-raw’,format=NV12 ! x264enc ! qtmux ! filesink location=./out4.mp4

The mp4 file is generated but gives the error “moov atom not found” which indicates there is an error in the metadata about the time-line and it’s not possible to play it.

I’ve also tried to save it to mkv:

gst-launch-1.0 v4l2src device=/dev/video0 do-timestamp=true ! image/jpeg, framerate=30/1, width=1920, height=1080 ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! ‘video/x-raw’,format=NV12 ! omxh264enc qp-range=‘15,30:5,20:-1,-1’ preset-level=2 bitrate=8000000 ! matroskamux ! filesink location=test.mkv

This video is playable but gives about 1 frame every 5 seconds or so but the video-properties tells me it’s 30fps.

The video recording seems to work when reading out the raw-video from the camera but this gives too low framrate at higher resolutions, so I need to access the mjpeg stream somehow.

Any idea why I get corrupted video-files?

Best regards/
Daniel Grafström

3

For the mp4 issue:
How much do you know about mp4 file format(ISO - ISO/IEC 14496-14:2003 - Information technology — Coding of audio-visual objects — Part 14: MP4 file format) and the work flow of qtmux(gst/isomp4/gstqtmux.c · 1.14 · GStreamer / gst-plugins-good · GitLab)?

You must send EOS event to end the pipeline in order to insert the necessary header. gst-launch tool can not do it. You need to write your own application.

This is just a basic gstreamer programming skill. Please refer to gstreamer resources in internet.

It is not deepstream related.

4

Do you mean when you playback the saved mkv file, it plays with 0.2 fps? How do you get “the video-properties”?

5

I’ve used the gstreamer documentation to find the commands. eg:
https://gstreamer.freedesktop.org/documentation/isomp4/qtmux.html?gi-language=c
This example is the same except the file
ending is mov and not mp4

It seems like I can send EOS by adding to -e to gst-launch in the end of the line. anyway, it still does not work.

EOS on shutdown enabled – Forcing EOS on the pipeline
Waiting for EOS…
Got EOS from element “pipeline0”.
EOS received - stopping pipeline…
Execution ended after 0:00:06.428340518

I get the camera settings from video4linux: v4l2-ctl -d /dev/video0 --list-formats-ext

v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘MJPG’ (compressed)
Name : Motion-JPEG
Size: Discrete 4656x3496
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 4208x3120
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 4160x3120
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 4000x3000
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 3264x2448
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 2592x1944
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 2320x1744
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 2304x1728
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 2048x1536
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1600x1200
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x960
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1024x768
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 4656x3496
Interval: Discrete 0.100s (10.000 fps)

6

Please read line 1817 in gst/isomp4/gstqtmux.c · 1.14 · GStreamer / gst-plugins-good · GitLab. You can enable qtmux debug message to check whether qtmux handles the EOS correctly.

Gstreamer is open source.

7

There is no problem with the EOS.
The problem is in the ubuntu default video-player(totem) that seems to use software decoding on only one core which is overloaded. VLC has similar problem but mpv works great or by playing the video using deepsteram/gstream command:

gst-launch-1.0 filesrc location=test.mkv ! matroskademux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=I420 ! nveglglessink

8

Thank you for the information. Can we close this topic now?

1 Like