NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running

[lolcocks123]

Hello,

I am trying to get YOLOv3 running on the Jetson Nano using the darknet library.

However, darknet is unable to detect my GPU as the GPU is not detected.

When I try running “nvidia-smi” I get the following error:

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

On the side note, this is the code darknet is using to find the GPU:

int cuda_get_device()
{
    int n = 0;
    cudaError_t status = cudaGetDevice(&n);
    check_error(status);
    return n;
}

void check_error(cudaError_t status)
{
    //cudaDeviceSynchronize();
    cudaError_t status2 = cudaGetLastError();
    if (status != cudaSuccess)
    {   
        const char *s = cudaGetErrorString(status);
        char buffer[256];
        printf("CUDA Error: %s\n", s);
        assert(0);
        snprintf(buffer, 256, "CUDA Error: %s", s);
        error(buffer);
    } 
    if (status2 != cudaSuccess)
    {   
        const char *s = cudaGetErrorString(status);
        char buffer[256];
        printf("CUDA Error Prev: %s\n", s);
        assert(0);
        snprintf(buffer, 256, "CUDA Error Prev: %s", s);
        error(buffer);
    } 
}

Anyone knows how to fix this issue? Perhaps the GPU drivers are not loaded? How can I confirm / load them?

[dusty_nv]

@lolcocks123

When I try running “nvidia-smi” I get the following error:

Hi @lolcocks123, I’m not familiar with running the darknet library myself, but nvidia-smi isn’t supported on Jetson (so that is unrelated to the issue)

Are you able to compile/run the deviceQuery sample from the CUDA Toolkit?

cd /usr/local/cuda/samples/1_Utilities/deviceQuery
sudo make
./deviceQuery

That should confirm if CUDA is installed/functioning on your device.

[linuxdev]

You are using software which needs a discrete GPU over PCI. The Jetson GPU is an integrated GPU directly wired to the memory controller, and thus PCI functions have no possibility of working. I do not know what versions of Yolo are out there, but perhaps someone can suggest a version which works with the iGPU.

[Andrey1984]

@lolcocks123

However, darknet is unable to detect my GPU as the GPU is not detected.

@lolcocks123
As per my experience, typically Jetson GPU is detected by yolov3, but it may require to export CUDA paths, opencv paths or check the default configuration file to confirm applicable paths are exact
As it was pointed out already - the thing you did run [ nvidia smi] is for x86_64. for arm64[aarch64]/Jetson there is [kind of] equivalent the device query app mentioned above.
AV