Jetson L4t multimedia Camera Recording. FPS Stuck at 30

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hi,

I am running the Camera Recording on my Jetson Xavier NX (Jetson Linux API Reference: Main Page | NVIDIA Docs)

Unfortunately, I am not able to get more than 30 fps. Changing the DEFAULT_FPS macro from 30 to 60 didnt make any difference.

I noticed the following lines

ret = m_VideoEncoder->setIFrameInterval(30);

ret = m_VideoEncoder->setFrameRate(30, 1);

I replaced 30 with 60 here as well, but it still gave a 30fps video output.

I am using ArduCam IMX477, and using the gstreamer command at this link I can record a video in 60fps

How can I make the example code record the video at 60 fps.

3

Hi,
For having fps information in h264 stream you need to enable VUI. Please check the option in 01_video_encode:

        --insert-vui          Insert VUI [Default = disabled]

And add the code to 10_camera_recording.

4

Thanks for the reply. I enabled VUI and it does seem to have added FPS information in the stream, but its still recording at 30fps.

Case 1:
VUI Enabled, and recording at 30fps. The playback works fine, there is no difference here whether VUI is enabled or not.

Case 2:
VUI enabled and recording at 60fps.
To play I use command
gst-launch-1.0 filesrc location=/home/omer/output60vui.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! videoconvert ! videorate ! video/x-raw,framerate=60/1 ! autovideosink

The video plays 2x faster and a 10 second video ends in 5 seconds

if I give the framerate=30/1, the video seems to be be skipping frames but still ends in 5 seconds

Case 3:
VUI not enabled and recording at 60fps.
To play I use command
gst-launch-1.0 filesrc location=/home/omer/output60.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! videoconvert ! videorate ! video/x-raw,framerate=60/1 ! autovideosink

The video plays 2x faster and a 10 second video ends in 5 seconds

if I give the framerate=30/1, the video plays exactly as if it was recorded at 30 fps, and runs for 10 seconds.

TLDR:
So inserting vui has just added fps information to video, but its still recording at 30fps. What am I missing

5

Hi,
Please check if the VUI is correct through ffprobe:

$ ffprobe -i filename.h264

And raw h264 stream may not be played in correct framerate in gstreamer command. Suggest you mux it into mp4 or mkv. You can try

$ ffmpeg -i filename.h264 -c copy out.mp4
6

Hi DaneLLL,

Thank you for helping me out. Unfortunately I’m still stuck at this issue.

Doing ffprobe on 30fps file I get

Input #0, h264, from 'output_1080p_30fps_vui.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt470bg/reserved/reserved, progressive), 1920x1080, 30 fps, 30 tbr, 1200k tbn, 60 tbc

While doing ffprobe on 60fps file I get

Input #0, h264, from 'output_1080p_60fps_vui.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt470bg/reserved/reserved, progressive), 1920x1080, 60 fps, 60 tbr, 1200k tbn, 120 tbc

I muxed it into mp4 and played in VLC, but 60fps video runs for just ~5 seconds, whereas I recorded it for 10 seconds.

Also strangely the size of 60fps file is 2.4mb whereas size of 30fps file is 5mb

7

Hi,
Please mux it into mp4:

$ ffmpeg -i output_1080p_60fps_vui.h264 -c copy out_60fps.mp4

And try to play out_60fps.mp4 through VLC.

8

I muxed it into mp4 using ffmpeg and played in VLC, but 60fps video runs for just ~5 seconds, whereas I recorded it for 10 seconds.

Also strangely the size of 60fps file is 2.4mb whereas size of 30fps file is 5mb

9

Hi,
The issue is not in encoding but in the source. It does not output in 60fps. Please check if you correctly run the source in 60fps. Probably you don’t set to the correct mode or miss some setting to Argus.

10

In the example I do not see any settings for Camera. Although when creating output stream I see this

UniqueObj<OutputStreamSettings> streamSettings(
        iCaptureSession->createOutputStreamSettings(STREAM_TYPE_BUFFER));

where going to STREAM_TYPE_BUFFER definition shows this

DEFINE_UUID(StreamType, STREAM_TYPE_BUFFER, c723d960,5231,11e7,9598,18,00,20,0c,9a,66);

I can notice an 1E in there which translates to 30. Is it something thats limiting the output frame rate ?

11

Hi,
You can set the sensor mode. It is demonstrated in 09_camera_jpeg_capture. Please refer to it and apply to 10_camera_recording.

Run the command with Pi camera V2:

$ ./camera_jpeg_capture --sensor-mode 3 --fps 60 --disable-jpg --cap-time 10 -s

Profiling result:

----------- Element = renderer0 -----------
Total Profiling time = 9.93072
Average FPS = 59.5123
Total units processed = 592
Num. of late units = 168
-------------------------------------
12

Thank you so much Dane. Finally this resolved my issue.

My sensor had just two modes:

[0] W=4032 H=3040 minduration=33333334 maxduration=500000224
[1] W=1920 H=1080 minduration=16666667 maxduration=500000224

and by default it was using mode 0, due to which the fps was getting limited to 30. On selecting sensor mode 1 it worked like a charm.

ISensorMode *iSensorMode;
std::vector<SensorMode*> sensorModes;
iCameraProperties->getBasicSensorModes(&sensorModes);
SensorMode *sensorMode = sensorModes[1];
iSensorMode = interface_cast<ISensorMode>(sensorMode);
iSourceSettings->setSensorMode(sensorMode);

Once again thanks a lot Dane :)

1 Like