Run the following command and recive error output below:
docker run --gpus=1 --name triton --rm -p8000:8000 -p8001:8001 -p8002:8002 -v/home/roni/models:/models nvcr.io/nvidia/tritonserver:24.10-py3 tritonserver --model-repository=/models
Same applies with tritonserver:23.05
Also below is my full hardware setup.
What do I still need to do?
Many thanks
Receive output:
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as ‘legacy’
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.
Hardware: Ubuntu 24.04 with Geforce RTX 3060
cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 550.120 Fri Sep 13 10:10:01 UTC 2024
sudo apt-cache policy linux-modules-nvidia-550-$(uname -r)
[sudo] password for roni:
linux-modules-nvidia-550-6.8.0-49-generic:
Installed: 6.8.0-49.49
Candidate: 6.8.0-49.49
Version table:
*** 6.8.0-49.49 500
500 Index of /ubuntu noble-updates/restricted amd64 Packages
500 Index of /ubuntu noble-security/restricted amd64 Packages
100 /var/lib/dpkg/status
nvidia-smi
Tue Nov 26 12:57:13 2024
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.120 Driver Version: 550.120 CUDA Version: 12.4 |
|-----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3060 Off | 00000000:01:00.0 Off | N/A |
| 0% 24C P8 9W / 170W | 158MiB / 12288MiB | 0% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 1710 G /usr/lib/xorg/Xorg 101MiB |
| 0 N/A N/A 2037 G /usr/bin/gnome-shell 50MiB |
±----------------------------------------------------------------------------------------+