NVENC - timestamps are getting swapped

Gaming and Visualization Technologies Video Processing & Optical Flow
1

Hi all,

Recently started working with the NVENC API. I’m successfully decoding N distinct h264 bits streams into NV12, resize and stitch all the frames into a final 1080p or 4K frame, and then pass this final frame to the encoder.

Some background (not really relevant to this issue, but offers some context)
For this purpose, I select one of those N input h264 streams as the “driver”: whenever the final 4K frame gets changed because of that select input stream, I shoot out a request to the NVENC to encode the fat frame (1080p or 4k).

What I do

  • For each NV12 frame about to be encoded, I associate a timestamp with the help of NV_ENC_PIC_PARAMS::inputTimeStamp
  • The encoder is setup in such a way that it does not produce B frames (NV_ENC_CONFIG::frameIntervalP == 1)
  • The timestamps are monotonically increasing in a strict way. In other words, the next-timestamp is strictly bigger (not less or equal) then previous-timestamp.
  • I inspect the return value of nvEncEncodePicture and consider the call successful if the value is either one of NV_ENC_SUCCESS or NV_ENC_ERR_NEED_MORE_INPUT
  • If the value is NV_ENC_SUCCESS, then I consider that I have an output encoded frame. I maintain a queue with what was passed to the encoder, and in what order. It is a FIFO basically. I take the oldest NV_ENC_OUTPUT_PTR, lock it, extract te bits stream and timestamp, unlock it

    • The problem
    All the above works as expected, except on sporadic occasions where I get the timestamps messed up. Here is a real log dump:
    On input I feed in the following timestamps:

    37037652351
37037658351
37037664351
37037670351
37037676351
37037682351

    On output I get:

    37037652351
37037658351
37037664351
37037682351 <<<<<BANG!!!!
37037670351

    Notice how 37037670351 and 37037676351 were skipped.

    Any hints on what might go wrong?

    Best regards,
    Andrei

    2

    Hi Andrei,

    Have you enabled lookahead NV_ENC_RC_PARAMS:: enableLookahead? If yes, can you try disabling lookahead if you see the same problem?

    Secondly, are all NvEncEncodePicture() calls returning NV_ENC_SUCCESS or NV_ENC_ERR_NEED_MORE_INPUT? OR are you getting any other return values from NvEncEncodePicture().

    Thanks,
    Ryan Park

    3

    Hi Ryan,

    Thanks for getting into this issue. Will test the disabled LA and come back with the results. As for the return values, no, I do not get any other return values. Only those 2