Keep in mind that X11 is what talks to the GPU driver. Many people mistake the memory used for a desktop login as somehow differing from what the GPU uses for CUDA. If you break the X11 API you break CUDA because the video driver has nothing to talk to it. Even though the buffer might be named via the DISPLAY environment variable the driver doesn’t care whether an end monitor attaches to it or not.
You can remove all end user login and desktop applications and still run X for no purpose other than allowing CUDA (this would be much faster boot than also running a bunch of large programs). Almost nothing you see from a login is actually from X…it is the display manager or window manager application you are looking at. X by itself has almost no functionality…it is a buffer with organized (X11 protocol) access and nothing more. Maybe one day the newer Vulkan will have a server which can run without X…but it’d still be the same story…the window manager would be what the end user sees…Vulkan itself would be just a buffer attached to the GPU (though a more efficient buffer than X11 when rendering locally).
A virtual desktop with no window manager or login manager software is what you want…it could be started directly from the command line with some variation of the “startx” command, and then you could remove lightdm and unity…viewing this with a monitor would be nothing but an ugly gray background and a large ugly mouse cursor (or no mouse cursor). Everything else is a product of the window manager and is not needed for pure CUDA. You would have a DISPLAY environment variable associated with the buffer…CUDA would be satisfied.
As an example of part of this being removed just log in to a console (e.g., CTRL-ALT-F2) and run (as user ubuntu) “startx”. The defaults will give you a minimal desktop and not run much of the software. If on another login you can run “killall -9 Xorg” you’ll end that session (you own that session, so you can kill it…you’ll get an operation not permitted which applies to the system’s X login). It’ll drop back to a console.
Note that “/usr/bin/startx” is a human readable script. The man page for xinit will give you information about what startx uses under the covers.
Next, in a console, run “export DISPLAY=:1”. Then, from a remote ssh login, run this while logged in to the same user (assuming “ubuntu”, but “nvidia” would work…login must be root or in group “video” and must be consistently the same user for all uses):
X -nolisten tcp :1
The command issued in the ssh prompt should cause the console where you had exported :1 to basically go blank (but in graphical mode…the actual console F1/F2/F3/F4, so on, depends on how many servers are currently running unless your command specifies). Now, in another ssh login:
You’ll see the xterm show on that display. No window manager, no login manager, no fancy mouse cursor. It’s purely X, and if you were to run a CUDA program with DISPLAY set to :1 it would use this video buffer. None of the login or other programs people associat with X would be running.
CUDA uses an associated DISPLAY the same as xterm would…only the results are computations which don’t show up in the user’s desktop. Setting “DISPLAY=:1” and starting X, followed by executing a program, is nothing special. It could just as well be xterm. The two programs have different results, but exist due to the driver talking the X protocol. It’s the window manager and display manager which are complicated and CPU/memory hungry.