Using invalidateReferenceFrames

Gaming and Visualization Technologies Video Processing & Optical Flow
1

I’m building a remote desktop streaming application. One feature that will be critically helpful is to not send any frames if the remote desktop is idle. The idea I had to solve this was to look at each encoded h.264 frame, decide if there are changes (not yet solved), and if there are no changes, then “undo” the encoder, and not send the frame to the client. The purpose of this “undo” operation is that I don’t want future frames referencing this frame, because it’s not going to be sent to the client.

To implement this “undo” operation, I used invalidateReferenceFrames (I think this may not exist on newer SDKs). Here’s a screenshot of docs talking about this: Dropbox - Screenshot 2020-04-03 13.15.03.png - Simplify your life

However, it seems to not work for my use case. Even if a frame is invalidated, it still counts towards the “maxNumRefFrames”. So, say I were to make 10 frames, where the last 9 are invalidated. My 11th frame won’t be able to reference the first frame if maxNumRefFrames is less than 10.

I drew out a picture to describe this situation in another form: Dropbox - Screenshot 2020-04-03 13.06.14.png - Simplify your life

Is this an issue with invalidateReferenceFrames, or am I using the API incorrectly?

Thanks!

2

Hi.
NvIFROpenGL API has been deprecated (https://developer.nvidia.com/capture-sdk).
I recommend you to use NVIDIA Video Codec SDK directly instead of using Capture SDK to generate compressed bit-stream.

Thanks.

3

Yup, I know it’s been deprecated but it’s the only tool that captures a specific window instead of the entire desktop. The Capture SDK captures the entire desktop.

I’m guessing they both, as well as NVENC, have this same sort of issue anyways – so do you have a general answer to invalidated reference frames?