I build a desktop capturing and encoding to h264 system with nvFBC. It worked well if the output resolution is in 1920x1080 same as the desktop resolution. I added the feature of rescale from the resolution of 1920x1080 to 1280x720 with following codes:
m_fbcHwEncGrabFrameParams.eGMode = NVFBC_HW_ENC_SOURCEMODE_SCALE;
m_fbcHwEncGrabFrameParams.dwStartX = 0;
m_fbcHwEncGrabFrameParams.dwStartY = 0;
m_fbcHwEncGrabFrameParams.dwTargetHeight = 720;
m_fbcHwEncGrabFrameParams.dwTargetWidth = 1280;
It worked some how, but occasionally, nvfbc encoder crashed. I traced the crash point and found the crash occurred when NvFBCHWEncGrabFrame() return error: NVFBC_ERROR_INVALIDATED_SESSION or NVFBC_ERROR_HW_ENC_FAILURE, and then I called NvFBCHWEncRelease(), my program crashed.
It seems it is the procedure inside nvfbc DLL or nvidia driver level, or above my codes wrong ?
My OS is Windows 10, GRID K520 on AWS G2, visual Studio 2015, nvidia capture SDK 5.0/6.0/6.0.6. I also verified with a physical Windows 10 with K520, same result.
I also have a log of nvfbc failure as: