Dyanamic FPS/bitrate with nvv4l2h264enc & nvv4l2h265enc

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier
1

I have a live video feed whose FPS is variable (it changes from time to time). Unfortunately it seems that nvv4l2h264enc/nvv4l2h265enc do not support dynamic updates to the stream’s FPS (updating the source caps FPS seems to halt the encoding completely).

Additionally, it seems that specifying a particular bitrate on nvv4l2h264enc/nvv4l2h265enc, depends on the FPS being set correctly (otherwise the output stream will be larger or smaller than the specified bitrate).

Is there any way to properly encode a stream with a dynamic FPS at a somewhat consistent bitrate?

Thanks!

2

Hi,
Please share steps to reproduce the issue.

3

I have a pipeline such as:

appsrc -> nvvidconv -> nvv4l2h264enc -> h264parse -> matroskamux -> filesync

After playing the pipeline, I try to modify the source caps, as in:

// Pause the pipeline
gst_element_set_state(pipeline, GST_STATE_NULL);

// Update the caps
auto caps = gst_caps_new_simple(
    "video/x-raw",
    "format",       G_TYPE_STRING,  "RGBA",
    "bpp",          G_TYPE_INT,     32,
    "depth",        G_TYPE_INT,     32,
    "width",        G_TYPE_INT,     m_width,
    "height",       G_TYPE_INT,     m_height,
    "framerate",    GST_TYPE_FRACTION, NEW_FPS, 1,
    NULL);

g_object_set(appsrc, "caps", caps, NULL);
gst_caps_unref(caps);

// Continue the pipeline
gst_element_set_state(pipeline, GST_STATE_PLAYING);

In the GStreamer logs I see that after trying to continue the pipeline (set to PLAYING), the elements start playing up until nvv4l2h264enc, which never gets back to the playing state. Seems to be silently stuck:

state-changed el:sink from:PAUSED to:PLAYING
state-changed el:muxer from:PAUSED to:PLAYING
state-changed el:queue0 from:PAUSED to:PLAYING
state-changed el:filesync0 from:PAUSED to:PLAYING
state-changed el:h264parse3 from:PAUSED to:PLAYING
state-changed el:capsfilter7 from:PAUSED to:PLAYING
state-changed el:nvv4l2h264enc0 from:PAUSED to:PLAYING
state-changed el:capsfilter6 from:PAUSED to:PLAYING
state-changed el:nvvconv3 from:PAUSED to:PLAYING
state-changed el:appsrc0 from:PAUSED to:PLAYING
state-changed el:pipeline3 from:PAUSED to:PLAYING

...

// gst_element_set_state(pipeline, GST_STATE_NULL);

state-changed el:sink from:PLAYING to:PAUSED
state-changed el:muxer from:PLAYING to:PAUSED
state-changed el:queue0 from:PLAYING to:PAUSED
state-changed el:filesync0 from:PLAYING to:PAUSED
state-changed el:h264parse3 from:PLAYING to:PAUSED
state-changed el:capsfilter7 from:PLAYING to:PAUSED
state-changed el:nvv4l2h264enc0 from:PLAYING to:PAUSED
state-changed el:capsfilter6 from:PLAYING to:PAUSED
state-changed el:nvvconv3 from:PLAYING to:PAUSED
state-changed el:appsrc0 from:PLAYING to:PAUSED
state-changed el:pipeline3 from:PLAYING to:PAUSED

// set caps..
// gst_element_set_state(pipeline, GST_STATE_PLAYING);

state-changed el:sink from:PAUSED to:READY
state-changed el:muxer from:PAUSED to:READY
stream-status
state-changed el:queue0 from:PAUSED to:READY
state-changed el:filesync0 from:PAUSED to:READY
state-changed el:h264parse3 from:PAUSED to:READY
state-changed el:capsfilter7 from:PAUSED to:READY
4

Hi,
We would need your help to give a full test code so that we can simply build/run it to reproduce the issue. Please kindly help. Thanks.
For more information, do you run r32.2?

5

I’ve uploaded test code here:
https://github.com/Adjacence/nvv4l2h264enc-fps-bug

I am running r32.2.

There seems to be a lot of issues here. As indicated in the README, I am not even able to stop the pipeline before performing the FPS change. Moreover, these messages are continuously logged:

"GStreamer-CRITICAL **: 12:47:31.574: gst_buffer_resize_range: assertion 'bufmax >= bufoffs + offset + size' failed"
6

Hi,
We checked the sample but don’t see code about setting bitrate. For setting new bitrate, you should call gst_object_set():

(skip...)
  const char* const pipeline_launch_str
    = "videotestsrc pattern=ball is-live=TRUE "
      "! capsfilter name=src_caps_filter "
      " caps=video/x-raw,format=RGBA,width=1280,height=720,framerate=15/1 "
      "! nvvidconv "
      "! video/x-raw(memory:NVMM),format=I420"
      "! nvv4l2h264enc <b>name=myencoder</b> "
      "! fakesink num-buffers=150";
(skip...)
GstElement *encoder = gst_bin_get_by_name(GST_BIN(pipeline), "myencoder");
g_object_set(GST_OBJECT(encoder), "bitrate", _NEW_BITRATE_VALUE_, NULL);
(skip...)

For setting new framerate, we will check how to do it in gst-v4l2. It is supported and verified in tegra_multimedia_api 01_video_encode. gst-v4l2 is open source package and we need to check if this function is supported in default code flow, or we need to implement it.

Xavier AGX Video Encoder realtime questions
7

I understand the bitrate property can be changed - my sample was just meant to demonstrate changing the framerate dynamically.

Thanks for looking into the issue.

8

Hi, @connorniz ,

Did you manage to find a solution to the problem you’ve described in the first post? We’ve been observing similar behavior in our pipeline and would be thankful if you could share your insights.