I am trying to encode video at ultra low latency settings and I am seeing effect for which I don’t have good explanation.
Environment is: NVIDIA 1080 Ti GPU, 1920x1080@60 video signal, NV_ENC_PRESET_LOW_LATENCY_DEFAULT_GUID and other low latency targeting settings (I am under impression fine tuning with settings is not so much important).
I am encoding in Windows 10, I am using async version of the API and I wait for event that signals completion of bitstream. (It is not directly related, but I also did polling and subframe readback and the behavior is consistent through all the modes).
The problem is this: when I am doing encoding I see encoding latency of around 3 ms per frame during a few first seconds of encoding, then it goes up to 7 ms, then in a while it stabilizes on 10 ms level. Sometimes it can go back to 7 ms for some time and then jumps back again to 10 ms.
When I refer to encoding latency it is about time between NvEncEncodePicture call and event triggered as a result of encoding completion.
The encoding is rock solid in terms of stability and so is this latency pattern. I see that hardware can do 3 ms encoding but it switches to slower mode, such as [I am guessing] a result of thermal self-balancing. I wonder if there is any way programmatically or via interactive setup to affect this behavior?
Other things I tried:
Same code on GTX 750 system does encoding at constant 15 ms latency without going up and down
Synthetic test producing video stream using Direct2D with encoding that on 1080 Ti ASAP (effective frame rate is much higher than 60! - 10x or so) results in 6-7 ms latency.
Synthetic test producing video stream using Direct2D on another GPU with cross-GPU transfer and encoding that on 1080 Ti ASAP (effective frame rate is still much higher than 60 - 8x or so) results in 4 ms latency of the encoding step.
I see that GPU encoder is doing well at stable low latency when it’s idle and the encoder is basically capable to operate fast. However in scenario with live 60 fps signal, mixed load (encoding and frontend application activity such as 3D app or video playback application) and underloaded GPU in general I see that encoder degrades its performance to 10 ms latency for no clear reason.
Is there something I could try or maybe there is a well known reason for such behavior?
Things go better with pictures, so here we go - blue line indicates how encoding latency changes as encoding goes: