Tesla M60


I have been using the Nvenc 6.0 API and using the RGB input formats to skip converting to NV12. It works great on a Quadro 3100M laptop and a Titan X desktop PC. However on a Tesla M60 for some reason the same unmodified code runs without errors but only a quarter of the video (on the left side) is visible the other 3/4 of each frame of compressed video is all black. Anyone else seeing this? I have tried experimenting with changing the pitch but the same behavior still happens. This is using a d3d11 device with a texture registered and mapped as input.

Can you please tell us which version of the NVIDIA driver you are using and can you please also try the latest driver and see if the issue still occurs?

The issue occurs on 356.54. At the time I had to use the windows 10 driver since the Windows Server 2016 wasnt available. So the driver that is being used has this file name: 356.54-tesla-desktop-win10-64bit-international-whql.exe

I tried to use 376.33-tesla-desktop-winserver2016-international-whql.exe and I get an error now whether or not I use direct RGB or NV12 conversion myself. The error is:


I am using a directx 11.0 device.

So I had to switch back to the 356.54 driver.

The following simple initialization code fails for a Tesla M60 on Windows Server 2016 using driver version 376.33:

#include <iostream>
#include <windows.h>
#include <d3d11_2.h>
#include <atlcomcli.h>
#include "nvEncodeAPI.h"

CComPtr<ID3D11Device> CreateDevice();

int main()
    auto const& device(CreateDevice());

#if defined (_WIN64)
    auto const dllName("nvEncodeAPI64.dll");
    auto const dllName("nvEncodeAPI.dll");

    HMODULE mModule = LoadLibraryA(dllName);

    if (nullptr == mModule)
        std::cout << "cannot load dll" << std::endl;
        return 1;

    typedef NVENCSTATUS(NVENCAPI *NvEncodeAPICreateInstanceFunctionPointer)(NV_ENCODE_API_FUNCTION_LIST*);

    auto nvEncodeAPICreateInstance(
        GetProcAddress(mModule, "NvEncodeAPICreateInstance")));

    if (nullptr == nvEncodeAPICreateInstance)
        std::cout << "Cannot find dll entry point" << std::endl;
        return 1;

    NVENCSTATUS status;
    memset(&mAPIFunctions, 0, sizeof(mAPIFunctions));
    mAPIFunctions.version = NV_ENCODE_API_FUNCTION_LIST_VER;

    if (NV_ENC_SUCCESS != (status = nvEncodeAPICreateInstance(&mAPIFunctions)))
        std::cout << "nvEncodeAPICreateInstance: failed " << status << std::endl;
        return 1;

    void* mSession;
    memset(&openSessionExParams, 0, sizeof(openSessionExParams));
    openSessionExParams.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER;
    openSessionExParams.apiVersion = NVENCAPI_VERSION;
    openSessionExParams.deviceType = NV_ENC_DEVICE_TYPE_DIRECTX;
    openSessionExParams.device = device.p;

    if (NV_ENC_SUCCESS != (status = mAPIFunctions.nvEncOpenEncodeSessionEx(&openSessionExParams, &mSession)))
        std::cout << "nvEncOpenEncodeSessionEx: failed " << status << std::endl;

        return 1;

    std::cout << "Success" << std::endl;

    return 0;

CComPtr<ID3D11Device> CreateDevice()
    uint32_t createDeviceFlags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;

#if defined( DEBUG ) || defined( _DEBUG )
    createDeviceFlags |= D3D11_CREATE_DEVICE_DEBUG;

    D3D_FEATURE_LEVEL const featureLevels[] = {

    D3D_FEATURE_LEVEL featureLevel;
    HRESULT           hr = E_FAIL;
    CComPtr<ID3D11Device> device;

    hr = ::D3D11CreateDevice(nullptr, // default adapter
                             0, // no software device

    if (FAILED(hr))
        std::cout << "D3D11CreateDevice: failed " << hr << std::endl;
        return nullptr;

    std::cout << "Device feature level: " << featureLevel << std::endl;

    if (featureLevel < D3D_FEATURE_LEVEL_11_0)
        throw std::runtime_error("Direct3D Feature Level 11 unsupported.");

    return device;

Thank you trying with the latest driver and sending the code sample.
We are looking into the issue.

If you are passing the debug flag, make sure Windows SDK is installed.

You can either remove the debug flag or install Windows SDK.

The test was run in release mode so _DEBUG and DEBUG are not defined. I have run into the issue the link talks about but that is not what is happening in this case. That same code works on the same machine with nvidia driver 356.54 but not 376.33. Also to be clear the failure occurs when calling nvEncOpenEncodeSessionEx.

Do you have TCC mode enabled on your GPU?


For driver version 356.54 this is the outout of nvidia-smi.exe:

| NVIDIA-SMI 356.54 Driver Version: 356.54 |
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| 0 Tesla M60 WDDM | F241:00:00.0 Off | Off |
| N/A 41C P8 15W / 150W | 222MiB / 8192MiB | 0% Default |

For 376.33:

Failed to initialize NVML: Unknown Error

This issue has been fixed in 376.84.

I ran into the issue mentioned in #3 with the newest drivers.

Some info about my setup:

  • 376.84 drivers
  • Windows 2016 server
  • Tesla K80 GPU (on Google Cloud Engine)
  • I’m using the 32bit dll (nvEncodeAPI.dll).

I’m using a ID3D11Device* as NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS::device and calling nvEncOpenEncodeSessionEx() returns NV_ENC_ERR_UNSUPPORTED_DEVICE.

Here is a snippet of the initialization code I have:

memset(&openSessionExParams, 0, sizeof(openSessionExParams));

openSessionExParams.device = (void*) d3dDevice;
openSessionExParams.deviceType = NV_ENC_DEVICE_TYPE_DIRECTX;
openSessionExParams.apiVersion = NVENCAPI_VERSION;

nvStatus = m_NVEncAPI->nvEncOpenEncodeSessionEx(&openSessionExParams, &m_encoderHandle);

Any ideas what could be wrong here? I tried with older drivers (376.33), but encountered the same issue.

@Osurac On your Tesla M60 this works with the newest (376.84) drivers, right?

Yes the initialization works on M60 with driver version 376.84.