How to use the Reference Picture Selection(RPS) model for video encoding in Jetson Multimedia API

Hi,

I’m trying to use the Reference Picture Selection(RPS) mode for video encoding. My goal is to set the long-term reference frame (LTR frame) as the reference, instead of using the previous frame as the reference.

The default mode: frame 1 is I frame, frame 2 uses frame 1 as the reference frame, frame 3 uses frame 2 as the reference frame, frame 4 uses frame 3 as the reference frame, frame 5 uses frame 4 as the reference frame…

What I want: frame 1 is I frame, frame 2 is an I frame as well, then frame 3 uses frame 1 as reference frame, then frame 4 uses frame 2 as the reference frame, then frame 5 uses frame 3 as the reference frame, then frame 6 uses frame 4 as reference frame…

The video encodes sample project can let me input a file contains the rps parameter file.

-rpsf <rps_file_path> Specify external rps param file

However, it is very hard to understand how to use the RPS mode. The rps mode provides these parameters to set:

typedef struct v4l2_enc_frame_ext_rps_ctrl_params_
{
    /** unique Id of current frame. */
    __u32 nFrameId;
    /** Boolean value indicating if current frame referenced or non-referenced. */
    __u8 bRefFrame;
    /** Boolean value indicating if current frame long Term Ref Flag. */
    __u8 bLTRefFrame;
   /** Max Number of reference frames to use for inter-motion search. */
    __u32 nMaxRefFrames;
    /** # of valid entries in RPS, 0 means IDR. */
    __u32 nActiveRefFrames;;
    /**  frame id of reference frame to be used for motion search, ignored for IDR. */
    __u32 nCurrentRefFrameId;
    /** Array of RPS */
    v4l2_enc_frame_prop RPSList[V4L2_MAX_REF_FRAMES];
}v4l2_enc_frame_ext_rps_ctrl_params;

I tried several numbers but all of them fails. For my use case, how should I set for each frame?

Or could you provide me with a sample file that uses the long-term reference frame in this mode?

Thank you so much!

Hi liuluyang,
This is advanced functionality and we need to do further check. Is your usecase real time video streaming?

Hi DaneLLL,

Yes, the use case I provided is just a simple way to validate this RPS mode. my real use case is still to assign LTR frame for better video encoding performance. How long will you need to check for this issue?

Hi liuluyang,
Because this is advanced functionality. If you are in contact with NVIDIA region salesperson, we encourage you to go through business process to prioritize this request.

Hi DaneLLL,

Sure, can you let me know their contacts? I’m in the northeast of US. Thanks.

Please go to https://www.nvidia.com/en-us/contact/sales/