Using the Nvidia Encoder SDK

I’m not sure if this is the correct forum to post, I’m new in NVidia Development. Windows Media Foundation can use the Video Codec SDK but not in 10-bit RGB, so I want to create my own Media Foundation Transform to be then used in my professional Windows Video Sequencer in order to create HEVC 10-bit HDR videos.

I’m following advice from NVENC Video Encoder API Programming Guide :: NVIDIA Video Codec SDK Documentation

I’m trying at this time to use the H264 codec in the slightest configuration possible to see if it works. My problem is that, when locking the output bitstream buffer, I receive the error NV_ENC_ERR_INVALID_DEVICE.

Code resume:

  • Create a D3D11 device with D3D11_CREATE_DEVICE_BGRA_SUPPORT and at feature level D3D_FEATURE_LEVEL_11_1
  • I don’t create any texture (the NVidia SDK sample does create a texture)
  • Init the SDK
    		NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS e = {};
		e.apiVersion = NVENCAPI_VERSION;
		e.deviceType = NV_ENC_DEVICE_TYPE_DIRECTX;
		e.device = d2d.device;
		e.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER;
		auto res = fl.nvEncOpenEncodeSessionEx(&e, &encoder); // OK
  • Initialize the encoder
                        NV_ENC_INITIALIZE_PARAMS params = {};
			params.version = NV_ENC_INITIALIZE_PARAMS_VER;
    		        params.encodeGUID = NV_ENC_CODEC_H264_GUID;
			
			// First preset
			params.presetGUID = ... // a preset GUID from the enumerator;
   		        params.encodeWidth = 1920;
			params.encodeHeight = 1080;
//			params.enableOutputInVidmem = 1;
//			params.enablePTD = 1;
			params.darWidth = wi2;
			params.darHeight = he2;
			params.frameRateDen = 1;
			params.frameRateNum = 30;
			e.fl.nvEncInitializeEncoder(e.encoder, &params); // OK
  • Create buffers
                     	NV_ENC_CREATE_INPUT_BUFFER bu = {};
			bu.version = NV_ENC_CREATE_INPUT_BUFFER_VER;
			bu.width = 1920;
			bu.height = 1080;
     		        bu.bufferFmt = NV_ENC_BUFFER_FORMAT_ARGB;
			e.fl.nvEncCreateInputBuffer(e.encoder, &bu); // OK

                 	NV_ENC_CREATE_BITSTREAM_BUFFER bu2 = {};
			bu2.version = NV_ENC_CREATE_BITSTREAM_BUFFER_VER;
			e.fl.nvEncCreateBitstreamBuffer(e.encoder, &bu2); // OK
  • Encode
	       NV_ENC_LOCK_INPUT_BUFFER l1 = {};
		l1.inputBuffer = in_buffer->inputBuffer;
		l1.version = NV_ENC_LOCK_INPUT_BUFFER_VER;
                e.fl.nvEncLockInputBuffer(e.encoder, &l1); // OK
		memcpy(l1.bufferDataPtr, by, cl); // buffer I have from the raw video frame
		res = e.fl.nvEncUnlockInputBuffer(e.encoder, in_buffer->inputBuffer); // OK

		NV_ENC_PIC_PARAMS params = {};
		params.version = NV_ENC_PIC_PARAMS_VER;
		params.inputWidth = 1920;
		params.inputHeight = 1080;
		params.inputPitch = 1920;
		params.inputBuffer = in_buffer->inputBuffer;
		params.pictureStruct = NV_ENC_PIC_STRUCT_FIELD_TOP_BOTTOM;
		params.pictureType = NV_ENC_PIC_TYPE_P;
		params.bufferFmt = NV_ENC_BUFFER_FORMAT_ARGB;
		params.outputBitstream = out_buffer->bitstreamBuffer;
		res = e.fl.nvEncEncodePicture(e.encoder, &params); // Success
		NV_ENC_LOCK_BITSTREAM l2 = {};
		l2.version = NV_ENC_LOCK_BITSTREAM_VER;
		l2.outputBitstream = out_buffer->bitstreamBuffer;
		res = e.fl.nvEncLockBitstream(e.encoder, &l2);

This last call fails. What did I miss so far?

Thanks a lot.

Now something weird. The method returns failure, but it actually succeeds. The buffer is locked and I 'm able to copy bytes and unlock it.
What’s happening?