Jetson Multimedia API Zero-frame Latency Streaming

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
,
1

Hi, I’m doing a H265 real-time streaming application with Jetson Multimedia API on a Xavier for the encoding side and Nvidia Video Codec SDK on a x86 machine for the decoding side. I noticed that the application has around 300ms delay and it’s probably because I was not achieving zero-frame-latency encoding and decoding. Although we have the examples in the Jetson Multimedia API folder, those are applications with user-set parameters as inputs. Is there an example of the optimal parameter setting we should use for the Jetson Multimedia API encoder in order to achieve Zero-frame Latency? Thanks!

2

Hi,
Please refer to this post:
TX2 tegra_multimedia_api encode/decode issue - #16 by DaneLLL

For low latency, please set disable-dpb.

3

Hi, @DaneLLL ! Thanks for your reply. I saw that it says I have to use ‘–disable-dpb’, and I checked that ‘–disable-dpb’ is only an option for the jetson api decoder, but I’m using the jetson api as encoder and I use the nvidia video codec sdk as the decoder. Is there a equivalent ‘–disable-dpb’ setting on the jetson multimedia api encoder side, or is there a equivalent setting on the nvidia video codec sdk decoder side? Thanks again!

5

Hi,
For h265 encoding, please try to set the properties to encode a frame into multiple slices:

        -slt <type>           Slice length type (1 = Number of MBs, 2 = Bytes) [Default = 1]
        -slen <length>        Slice length [Default = 0]
        --sle                 Slice level encode output [Default = disabled]

So that encoder outputs a slice instead of a whole frame. It should reduce certain latency.

And please set max-perf=1:

        --max-perf            Enable maximum Performance
6

Hi @DaneLLL. Thanks for the reply. I tried your suggestion and it seems that there is little difference that I can notice. With some additional testing, I think the issue is not on the performance side (encoding too slow), but rather on the setting side. I tried to log a message when inputting the frame to the encoder and outputting the packet from the encoder, and I found that I must input 3 frames to the encoder at the beginning before the encoder can output one packet. It’s something looking like this:

LOG: frame input
LOG: frame input
LOG: frame input
LOG: packet output
LOG: frame input
LOG: packet output
…

It looks like it is not achieving zero-frame-latency. Even if I set the slice as you suggested, it still requires 3 frames before outputting anything. Do you know if it is possible for the encoder to achieve zero-frame-latency on Jetson multimedia API? Thanks!

7

Hi,
For encoding into I P P P… frames, encoder needs to keep one reference frame, so you need to feed at least 2 frames. Could you check if you can get encoded stream by feeding 2 frames?

In default 01_video_encode, it queues all output-plane buffers before starting encoding. We try to queue only 2 frame and can get encoded stream:

    /* Read video frame and queue all the output plane buffers. */
    for (uint32_t i = 0; i < 2/*ctx.enc->output_plane.getNumBuffers()*/; i++)

Encoding command:

01_video_encode$ ./video_encode 1080.yuv 1920 1080 H265 a.265 --max-perf

Seems not right you have to feed 3 frames and then encoder starts to output encoded stream.

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.