Hi,
I have been trying to use podman compose to pass through GPU inside container with WSL2.
podman run --rm --device nvidia.com/gpu=all --security-opt=label=disable - this command working fine for GPU pass through
However updating the same configuration in compose file which seems not working
devices:
- nvidia.com/gpu=all
security_opt:
- "label=disable"
Note: podman-compose command working fine with these configuration, wondering why its not working with podman compose (copied the docker-compose installer file to podman installer path)
Same problem here on ubuntu 22.04 with podman 5.3.2.
I’m experiencing the same with Podman Compose under WSL2 Podman machine.
Found this fresh GitHub issue reporting the same:
opened 10:58PM - 02 Feb 25 UTC
kind/bug
remote
### Issue Description
I'm running podman desktop on Windows 11 23H2.
Creating a… container using `podman run` detects my gpu correctly.
Creating a container using `podman compose` does not.
### Steps to reproduce the issue
1. run
`podman run -p 11434:11434 -v ollama_ollama_data:/root/.ollama --device nvidia.com/gpu=all --name ollama ollama/ollama:latest`
2. create a compose file
```
services:
ollama:
image: ollama/ollama:latest
container_name: ollama_compose
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
restart: unless-stopped
devices:
- nvidia.com/gpu=all
volumes:
ollama_data:
```
3. run
`podman compose up`
### Describe the results you received
Running the container using `podman run` shows
`level=INFO source=types.go:131 msg="inference compute" id=GPU-2e3db6bb-6d29-dd8d-c4cb-2e891458ba6c library=cuda variant=v12 compute=8.9 driver=12.8 name="NVIDIA GeForce RTX 4090" total="24.0 GiB" available="22.5 GiB"`
Running the container using the compose file shows
`level=INFO source=gpu.go:392 msg="no compatible GPUs were discovered"`
### Describe the results you expected
Detect the gpu for `podman run` and `podman compose up`
### podman info output
```yaml
host:
arch: amd64
buildahVersion: 1.38.0
cgroupControllers:
- cpuset
- cpu
- cpuacct
- blkio
- memory
- devices
- freezer
- net_cls
- perf_event
- net_prio
- hugetlb
- pids
- rdma
- misc
cgroupManager: cgroupfs
cgroupVersion: v1
conmon:
package: conmon-2.1.12-2.fc40.x86_64
path: /usr/bin/conmon
version: 'conmon version 2.1.12, commit: '
cpuUtilization:
idlePercent: 99.39
systemPercent: 0.1
userPercent: 0.51
cpus: 16
databaseBackend: sqlite
distribution:
distribution: fedora
variant: container
version: "40"
eventLogger: journald
freeLocks: 2041
hostname: Ericsons-Desktop
idMappings:
gidmap: null
uidmap: null
kernel: 5.15.167.4-microsoft-standard-WSL2
linkmode: dynamic
logDriver: journald
memFree: 15965605888
memTotal: 16725131264
networkBackend: netavark
networkBackendInfo:
backend: netavark
dns:
package: aardvark-dns-1.13.1-1.fc40.x86_64
path: /usr/libexec/podman/aardvark-dns
version: aardvark-dns 1.13.1
package: netavark-1.13.1-1.fc40.x86_64
path: /usr/libexec/podman/netavark
version: netavark 1.13.1
ociRuntime:
name: crun
package: crun-1.19.1-1.fc40.x86_64
path: /usr/bin/crun
version: |-
crun version 1.19.1
commit: 3e32a70c93f5aa5fea69b50256cca7fd4aa23c80
rundir: /run/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
os: linux
pasta:
executable: /usr/bin/pasta
package: passt-0^20241211.g09478d5-1.fc40.x86_64
version: |
pasta 0^20241211.g09478d5-1.fc40.x86_64
Copyright Red Hat
GNU General Public License, version 2 or later
<https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
remoteSocket:
exists: true
path: unix:///run/podman/podman.sock
rootlessNetworkCmd: pasta
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: false
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: true
slirp4netns:
executable: ""
package: ""
version: ""
swapFree: 4294967296
swapTotal: 4294967296
uptime: 0h 53m 44.00s
variant: ""
plugins:
authorization: null
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
- ipvlan
volume:
- local
registries:
search:
- docker.io
store:
configFile: /usr/share/containers/storage.conf
containerStore:
number: 4
paused: 0
running: 0
stopped: 4
graphDriverName: overlay
graphOptions:
overlay.imagestore: /usr/lib/containers/storage
overlay.mountopt: nodev,metacopy=on
graphRoot: /var/lib/containers/storage
graphRootAllocated: 1081101176832
graphRootUsed: 30164312064
graphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "false"
Supports d_type: "true"
Supports shifting: "false"
Supports volatile: "true"
Using metacopy: "true"
imageCopyTmpDir: /var/tmp
imageStore:
number: 3
runRoot: /run/containers/storage
transientStore: false
volumePath: /var/lib/containers/storage/volumes
version:
APIVersion: 5.3.1
Built: 1732147200
BuiltTime: Thu Nov 21 01:00:00 2024
GitCommit: ""
GoVersion: go1.22.7
Os: linux
OsArch: linux/amd64
Version: 5.3.1
```
### Podman in a container
No
### Privileged Or Rootless
Privileged
### Upstream Latest Release
Yes
### Additional environment details
WSL-Version: 2.3.26.0
Kernelversion: 5.15.167.4-1
WSLg-Version: 1.0.65
MSRDC-Version: 1.2.5620
Direct3D-Version: 1.611.1-81528511
DXCore-Version: 10.0.26100.1-240331-1435.ge-release
Windows-Version: 10.0.22631.4830
### Additional information
_No response_
Comment pointed to this merge that suggests it might be fixed but needs testing:
main ← giuseppe:pass-device-requests-to-cli
opened 08:23PM - 30 Jan 25 UTC
Disclaimer: I am not very familiar with all the details of CDI, so there might s… till be something missing, but the following patch seems to be enough for the test cases that were reported so I open a PR to gather some feedback.
Pass down the devices specifies in the resources block so that CDI devices in the compose file are honored.
Tested manually with the following compose file:
services:
testgpupodman_count:
image: ubuntu:latest
command: ["nvidia-smi"]
profiles: [gpu]
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
testgpupodman_deviceid:
image: docker.io/ubuntu:latest
command: ["nvidia-smi"]
deploy:
resources:
reservations:
devices:
- driver: cdi
device_ids: ['nvidia.com/gpu=all']
capabilities: [gpu]
#### Does this PR introduce a user-facing change?
```release-note
Now Podman honors CDI devices passed from the api
```