Who creates /dev/nvidia0 and /dev/nvidiactl?

I’m playing around with systemd-nspawn containers, and one of the containers needs access to the GPU.

On boot (of the headless server) /dev/nvidia0 and /dev/nvidiactl don’t exist until I run something like nvidia-smi which seems to create them.

However, I can’t figure out how to encode this into my systemd unit file - I tried adding the following to the unit file:

[Unit]
Requires=graphical.target

[Service]
ExecStartPre=/usr/bin/modprobe -abq nvidia
ExecStartPre=/usr/bin/nvidia-smi

But it fails: nvidia-smi[3389]: NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver.

In order to bring the container up I need to:

1/ ssh into the server
2/ manually run nvidia-smi
3/ restart systemd-nspawn

Then it works fine.

So, my question is, what dependency to I need to have before starting the container in order to ensure the driver is ready to go?

Okay I managed to figure it out… by using nvidia-persistenced.service

I documented it all here: https://github.com/ioquatix/github-actions