Adaptive change bitrate of encoder nvv4l2h264enc in runtime

[cooi0611]

• Hardware Platform (Jetson / GPU) GPU - Nvidia GeForce RTX 4090

• DeepStream Version DeepStream 6.1

• TensorRT Version • NVIDIA GPU Driver Version (valid for GPU only) Driver Version: 535.129.03

• Issue Type (questions, new requirements, bugs)

Hello, everyone. I am trying to use DeepStream and GStreamer to change the bitrate of nvv4l2h264enc at runtime. According to the output of gst-inspect-1.0 nvv4l2h264enc, it seems that the bitrate can only be changed when the pipeline is in the NULL or READY state:

bitrate : Set bitrate for v4l2 encode ** flags: readable, writable, changeable only in NULL or READY state** ** Unsigned Integer. Range: 0 - 4294967295 Default: 4000000**

I tried to change it at runtime, no error occurred, but the bitrate change did not take effect.

Is there any method to change the bitrate of the nvv4l2h264enc encoder at runtime?

Thank you!

[cooi0611]

Here is a part of my code:

import gi
gi.require_version('Gst', '1.0')
from gi.repository import Gst, GObject

bitrate_queue = Queue();

class RtpStreaming:
    def __init__(self, VIDEO_SSRC, VIDEO_PT, ip, port, rtcpPort):
        # Initialize GStreamer
        Gst.init(None)
        
        # Create pipeline
        self.pipeline = Gst.parse_launch(
            "rtpbin name=rtpbin rtp-profile=avpf "
            "appsrc name=source format=time is-live=true block=false "
            " caps=video/x-raw,format=RGB,width=1920,height=1080,framerate=0/1 "
            " ! videoconvert name=videoconvert "
            " ! nvvideoconvert name=nvvideoconvert compute-hw=0 "
            " ! video/x-raw(memory:NVMM),format=NV12,framerate=0/1 "
            " ! nvv4l2h264enc name=video_enc profile=0 bitrate=500000 "
            " ! video/x-h264,stream-format=byte-stream "
            " ! rtph264pay name=pay0 mtu=1400 ssrc={} pt={} timestamp-offset=0 "
            " ! rtprtxqueue max-size-time=1000 max-size-packets=0 "
            " ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 "
            " ! udpsink host={} port={} "
            " rtpbin.send_rtcp_src_0 "
            " ! udpsink host={} port={} "
            " sync=false async=false".format(VIDEO_SSRC, VIDEO_PT, ip, port, ip, rtcpPort)
        )

        # Get encoder element reference
        self.video_enc = self.pipeline.get_by_name("video_enc")
        
    def set_bitrate(self, bitrate):
        if self.video_enc:
            self.video_enc.set_property("bitrate", bitrate)
            print(f"Bitrate set to: {bitrate}")
        
    def run(self):
        self.pipeline.set_state(Gst.State.READY)
        self.pipeline.set_state(Gst.State.PLAYING)
        self.loop = GLib.MainLoop()
        self.loop.run()

    def monitor_bitrate_queue(self):
        while True:
            new_bitrate = bitrate_queue.get()
            if self.current_bitrate != new_bitrate:
                    self.video_enc.set_property('bitrate', int(new_bitrate))
                    self.current_bitrate = new_bitrate
                    print("New bitrate set to:", self.video_enc.get_property('bitrate'))

rtp_stream = RtpStreaming(VIDEO_SSRC=1234, VIDEO_PT=96, ip="127.0.0.1", port=5004, rtcpPort=5005)

thread = threading.Thread(target=rtpstreamer.monitor_bitrate_queue)
thread.daemon = True
thread.start()

rtp_stream.start()

[fanzh]

why do you need to change the bitrate at runtime? could you share the use case? Thanks!

[cooi0611]

I am designing an algorithm to study the transmission issues in low-latency real-time interactive scenarios. In this scenario, the client predicts bandwidth and sends a request to the server. The server then encodes video frames at the bitrate requested by the client. Therefore, I need to dynamically modify the bitrate while GStreamer is in the playing state. How can I achieve this?

[fanzh]

Thanks for the sharing! currently nvv4l2h264enc only support changing bitrate in the NULL or READY state. you can adjust peak-bitrate in PLAYING state.

[cooi0611]

ok, thanks for your reply. In my knowledge, peak-bitrate is a parameter used to set the peak bitrate in VBR mode. It can only set the peak bitrate for encoder in VBR mode but is difficult to control the video bitrate as precisely as CBR mode, all right?

Besides, I’ve seen some forum posts discussing modifying the NVENC encoder bitrate at runtime, for example:

Allocating a GstBuffer of type "memory:NVMM"

Hi, It may not be shown correctly in gst-launch-1.0. You can change bitrate runtime. Please check the attachment and give it a try $ g++ -Wall -std=c++11 test2.cpp -o test $(pkg-config --cflags --libs gstreamer-app-1.0) nvidia@nvidia-desktop:~$ ./test Using launch string: nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1,format=(string)NV12 ! nvv4l2h264enc control-rate=1 name=video_enc ! video/x-h264,stream-format=byte-stream ! appsink name=mysink Default b…

This confuses me. Is it because other frameworks have implemented this feature, but DeepStream does not support it yet?

[fanzh]

1. yes, peak-bitrate is only a peak of bitrate. it can’t control the average bitrate.
2. Thanks for the sharing! did you validate the solution test2.zip? you can create a thread to test other values when the state is playing.

[cooi0611]

I am sorry for not testing it myself as I rarely use C++ and prefer Python. But, following the test2, I implemented it in Python as shown in my previous post. After modifying the encoder’s bitrate in this way, I successfully modified the encoder’s bitrate attribute without any errors at runtime. However, the encoded video did not change visually. For example, when I reduced the bitrate from 10 Mbps to 500 kbps, the encoder’s bitrate attribute changed accordingly, but there was no visible change in the video output. It’s emmm… amazing :).

[fanzh]

the topic above is on Jetson. I tested test2.zip on dgpu with DS7.0. after the first bitrate setting(2Mbps), the bitrate is close to 2Mbps. After the second bitrate setting(4Mbps), the bitrate is close to 3Mbps, not 4Mbps. here is the log:
Default bit rate is 2Mbps
bitrate = 2348432 bps
bitrate = 2002560 bps
bitrate = 1558120 bps
bitrate = 1446424 bps
bitrate = 1940664 bps
bitrate = 1930560 bps
bitrate = 2035224 bps
Change bit rate to 4Mbps
bitrate = 1985696 bps
bitrate = 2343280 bps
bitrate = 2832752 bps
bitrate = 2586352 bps
bitrate = 2466888 bps
bitrate = 2720472 bps
bitrate = 2608872 bps
bitrate = 2635504 bps
bitrate = 2447008 bps
bitrate = 2350016 bps
bitrate = 3051416 bps
app sink receive eos
going to exit

[cooi0611]

Thanks for testing! Based on the results shown in the log, it seems that DeepStream is able to change the bitrate at runtime, but it cannot set the bitrate precisely？

[fanzh]

Yes. On dgpu with DS7.0, resetting bitrate at runtime will not report any error, but the actual bitrate does not match the set value.

[cooi0611]

Wow! I think I should update my DeepStream from 6.1 to 7.0 :), and try using GStreamer in Python again. Thanks for your patient replies!

[cooi0611]

Good news! I tested my code in DS7.0:
I successfully modified the encoder’s bitrate attribute without any errors at runtime, and the encoded video changed visually!

[fanzh]

Thanks for the sharing! two questions:

1. on DeepStream 6.1, did changing bitrate at runtime report error?
2. On DS7.0, after changing bitrate at runtime, did the actual bitrate match the set value?

[cooi0611]

1. In DeepStream 6.1, changing the bitrate at runtime did not report any errors, but as I say in previous post, didn’t work.

@cooi0611

But, following the test2, I implemented it in Python as shown in my previous post. After modifying the encoder’s bitrate in this way, I successfully modified the encoder’s bitrate attribute without any errors at runtime. However, the encoded video did not change visually.

2. I’ll test this and provide an update on the results here.