CUDA driver version is insufficient... on Ubuntu 16.04

I have a Linux 16.04 machine, and a Jetson TX1, connected to USB on Linux host.
I installed JetPack 2.3, which included CUDA 8.0.

uname -a:
Linux faith 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:39:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Trying to run a small CUDA C program from the host.

I get this, upon running the C code:

GPUassert: CUDA driver version is insufficient for CUDA runtime version

I found a suggestion to upgrade the CUDA driver on host, using

sudo apt-get install nvidia-367 nvidia-modprobe

and that completed.

Earlier, prior to a reboot, I had seen a different error from the C code, talking about “no NVIDIA devices found”.

What will fix this?

Is this a host logged in remotely to the Jetson? If so, then the CUDA insufficient message may have been referring to the host instead of the Jetson. Remote login can end up setting some of the GPU or video as being performed by host instead of Jetson. When using local login the CUDA and GPU would always refer to the Jetson.

Thanks for looking at this, @linuxdev.

The host is connected to my router; and so is the Jetson; and then the Jetson’s micro-USB is connected to host USB.
I set that up because that’s how I did it when I flashed the memory on the Jetson, from the host.

I tried the Nvidia 370 driver, updated to it, rebooted. It seemed to be more current.

I’m still getting the same error with the small CPU program.

Unless the Jetson is being flashed the micro-B USB won’t be seen as a device by the host (the Jetson is only a device in recovery mode…the rest of the time it is host mode…two hosts cannot see each other). USB is essentially inert.

The drivers you mention on host (such as 375.20) are irrelevant unless the GPU computation is actually offloaded to the host. Any indication that this host’s driver version matters implies all CUDA has bypassed the Jetson. If the display is going on entirely on the Jetson then the Jetson’s GPU is used…if remote display via X11 forwarding goes to the host, then the host does the GPU part…to remote display to an outside host without GPU being offloaded you’d need something like a virtual desktop program.

If you do not log in remotely, but instead at the local Jetson GUI, you should find it is the Jetson’s CUDA version which matters because it will have switched to using that GPU.

@linuxdev,
there’s something I don’t understand, then.
When getting a CUDA C program going, on the host, aren’t I trying to send the problem to the Jetson?
I’ve got directives like global, and I do a “kernel launch,”
doesn’t the host send that calculation to the Jetson?

I assumed the host is my Linux box - Ubuntu 16.04
and the GPU is the Jetson

Is that picture not correct?

Thanks-

Joel

Here’s another way to phrase my question…

Suppose I have a CUDA C program on my Linux host, 192.168.1.115.
I compile the CUDA C program on that Linux host.
Then I execute the executable on that Linux host.
It complained about the CUDA driver being insufficient.

However – my Jetson Linux is reachable at 192.168.1.102, I run ssh.

How would the CUDA C program, on 192.168.1.115, typically reach the Jetson GPU on 192.168.1.102?
This morning I was assuming that somehow the problem went over the USB link between them.
I’m learning that’s not the case.
How do people do that?

I see…the program run on the host would use GPU on the host. I was under the impression you wanted this to run on the Jetson. Correct me if I’m wrong, but it seems now what you want is to display on Jetson and get CUDA enhancement via the desktop host (reverse of what most people do).

To get CUDA to run on the Jetson one of these conditions would need to be true:

  • Run on Jetson, display on Jetson;
  • Run on host, remote display via X11 forwarding on Jetson;
  • Run on Jetson to a virtual desktop, remote display to anything monitoring the virtual X11 server.

So for example, ssh from the Jetson to the host via “ssh -Y” while in the Jetson GUI. Run the command that way and it forwards to Jetson using X11 mechanisms. This should provide host CUDA enhancing performance of a CUDA program displayed on Jetson.

Conversely, to get CUDA to run on the host one of these conditions would need to be true:

  • Run on host, display on host;
  • Run on Jetson, remote display via X11 forwarding on host;
  • Run on host to a virtual desktop, remote display to anything monitoring the virtual X11 server.

Thanks @linuxdev. I do want to use the Jetson’s GPU, and display on host.
I didn’t understand before that the Jetson Development Kit has a host.
I’ll write again soon - will give this some time to sink in - and am going to use it happily in the meantime!