Possible bug? FORCEIDR to resend SPS/PPS does not work in D3D11 encoder API, but works in CUDA

I have hardware H.264 encoding using the CUDA API working now, and I want to expand my solution to cover D3D11.

For some reason, the D3D11 encoder does not allow me to resend the SPS/PPS headers. This makes it impossible for clients to hook into existing streams.

Here’s the code for the CUDA encoder which works perfectly:

// Send out an IDR frame every n frames (or when explicitly requested)
NV_ENC_PIC_PARAMS picParams = { NV_ENC_PIC_PARAMS_VER };
if (nFrame % refreshInterval == 0 || forceRefresh)
{
	picParams.encodePicFlags = NV_ENC_PIC_FLAG_FORCEIDR;
	forceRefresh = false;
}
else
{
	picParams.encodePicFlags = 0;
}

const NvEncInputFrame* encoderInputFrame = enc->GetNextInputFrame();
NvEncoderCuda::CopyToDeviceFrame(cuContext, inputBytes, 0, (CUdeviceptr)encoderInputFrame->inputPtr,
	(int)encoderInputFrame->pitch,
	enc->GetEncodeWidth(),
	enc->GetEncodeHeight(),
	CU_MEMORYTYPE_HOST,
	encoderInputFrame->bufferFormat,
	encoderInputFrame->chromaOffsets,
	encoderInputFrame->numChromaPlanes);

enc->EncodeFrame(vPacket, &picParams);

And here is the equivalent code of my D3D11 encoder which does not work:

// Send out an IDR frame every n frames (or when explicitly requested)
NV_ENC_PIC_PARAMS picParams = { NV_ENC_PIC_PARAMS_VER };
if (nFrame % refreshInterval == 0 || forceRefresh)
{
	picParams.encodePicFlags = NV_ENC_PIC_FLAG_FORCEIDR;
	forceRefresh = false;
}
else
{
	picParams.encodePicFlags = 0;
}

// Get an input buffer for the next frame from the encoder
const NvEncInputFrame* encoderInputFrame = enc->GetNextInputFrame();

// Copy the texture data into the new input frame
ID3D11Texture2D* pTexBgra = reinterpret_cast<ID3D11Texture2D*>(encoderInputFrame->inputPtr);
pContext->CopyResource(pTexBgra, inputTexture);

// Tell the encoder to encode the data
enc->EncodeFrame(vPacket, &picParams);

I am playing the video streams in VLC, via raw H.264 over TCP/IP. Both clients are receiving the exact same data, but only the first client to connect will get the correct headers (because the encoder only starts when the first client connects). The first client plays the stream, but any clients afterwards never get the SPS/PPS. The screen stays black and the VLC logs constantly give out warnings that the player is waiting for SPS/PPS headers.

Any ideas?

This seems like a bug in NVIDIA’s encoding API. For the CUDA encoder, passing the NV_ENC_PIC_FLAG_FORCEIDR flag is enough. But for the D3D11 encoder, you also need all the other flags:

picParams.encodePicFlags = NV_ENC_PIC_FLAG_FORCEIDR | NV_ENC_PIC_FLAG_FORCEINTRA | NV_ENC_PIC_FLAG_OUTPUT_SPSPPS;

However, it seems that on some devices (currently testing on a GRID M10-2B), this isn’t enough and the SPS/PPS is not resent at all.

Hi.
We are not able to reproduce the behavior you mentioned using sample application.

In general,
‘picParams.encodePicFlags = NV_ENC_PIC_FLAG_FORCEIDR’ ensures that the current pic is IDR.
SPS/PPS are not sent for this IDR frame irrespective of D3D11/CUDA apps.

‘picParams.encodePicFlags = NV_ENC_PIC_FLAG_FORCEIDR |NV_ENC_PIC_FLAG_OUTPUT_SPSPPS’ ensures IDR frame and SPS/PPS will be sent for this frame.

In the context of sample application, above flag should be set in NvEncoder::DoEncode().
Can you confirm flag is set correctly?

You can also get IDR and SPS/PPS by adding following lines in NvEncoder::CreateEncoder()
m_encodeConfig.encodeCodecConfig.h264Config.idrPeriod = 5;
m_encodeConfig.encodeCodecConfig.h264Config.repeatSPSPPS = 1;

If you continue to see the problem, please provide an exact reproducer code along with instructions to help reproduce this issue internally.

Thanks.