No graphics output Quadro RTX 5000 Ubuntu 20.04

Here is my problem. I have a Nvidia Quadro RTX 5000 installed on my ubuntu 20.04 server (DELL PowerEdge R7425 AMD EPYC™ 7551). I do not manage to get a graphics output from the nvidia card, neither from VGA output from the onboard graphics card. After I installed nvidia-440 drivers from the ubuntu repository. Other symptoms are that after a while the server hangs totally and I have to force restart from the iDRAC interface.
The propose I need thous GPU-s is for HPC cluster and not for graphical use.
~$ nvidia-smi
Wed Sep 2 10:53:12 2020
±----------------------------------------------------------------------------+
| NVIDIA-SMI 440.100 Driver Version: 440.100 CUDA Version: 10.2 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Quadro RTX 5000 Off | 00000000:41:00.0 Off | Off |
| 31% 39C P0 48W / 230W | 0MiB / 16125MiB | 0% Default |
±------------------------------±---------------------±---------------------+
| 1 Quadro RTX 5000 Off | 00000000:C1:00.0 Off | Off |
| 31% 36C P0 19W / 230W | 0MiB / 16125MiB | 0% Default |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
±----------------------------------------------------------------------------

~$ lspci -k | grep -EA3 ‘VGA|3D|Display’
04:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. Integrated Matrox G200eW3 Graphics Controller (rev 04)
DeviceName: Embedded Video
Kernel driver in use: mgag200
Kernel modules: mgag200

41:00.0 VGA compatible controller: NVIDIA Corporation TU104GL [Quadro RTX 5000] (rev a1)
Subsystem: NVIDIA Corporation TU104GL [Quadro RTX 5000]
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

c1:00.0 VGA compatible controller: NVIDIA Corporation TU104GL [Quadro RTX 5000] (rev a1)
Subsystem: NVIDIA Corporation TU104GL [Quadro RTX 5000]
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

Regards,
Perparim