m
HJere is how I now reinstalled docker :
sudo apt update
sudo apt install -y nvidia-container curl
curl https://get.docker.com | sh && sudo systemctl --now enable docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
sudo usermod -aG docker $USER
newgrp docker
sudo apt install -y jq
sudo jq '. + {"default-runtime": "nvidia"}' /etc/docker/daemon.json | \
sudo tee /etc/docker/daemon.json.tmp && \
sudo mv /etc/docker/daemon.json.tmp /etc/docker/daemon.json
sudo systemctl daemon-reload && sudo systemctl restart docker
when i run docker info, i get
Client: Docker Engine - Community
Version: 27.5.1
Context: desktop-linux
Debug Mode: false
Plugins:
ai: Docker AI Agent - Ask Gordon (Docker Inc.)
Version: v1.1.7
Path: /usr/lib/docker/cli-plugins/docker-ai
buildx: Docker Buildx (Docker Inc.)
Version: v0.23.0-desktop.1
Path: /usr/lib/docker/cli-plugins/docker-buildx
cloud: Docker Cloud (Docker Inc.)
Version: v0.3.0
Path: /usr/lib/docker/cli-plugins/docker-cloud
compose: Docker Compose (Docker Inc.)
Version: v2.35.1-desktop.1
Path: /usr/lib/docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.38
Path: /usr/lib/docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Docker Inc.)
Version: v0.1.8
Path: /usr/lib/docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /usr/lib/docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.27
Path: /usr/lib/docker/cli-plugins/docker-extension
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.4.0
Path: /usr/lib/docker/cli-plugins/docker-init
mcp: Docker MCP Plugin (Docker Inc.)
Version: dev
Path: /usr/lib/docker/cli-plugins/docker-mcp
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /usr/lib/docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.17.1
Path: /usr/lib/docker/cli-plugins/docker-scout
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 28.1.1
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
CDI spec directories:
/etc/cdi
/var/run/cdi
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
runc version: v1.2.5-0-g59923ef
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.10.14-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 6
Total Memory: 1.788GiB
Name: docker-desktop
ID: 57b75178-4000-485d-a29f-a82845ae8586
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///home/marc/.docker/desktop/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
::1/128
127.0.0.0/8
Live Restore Enabled: false
WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW is set
WARNING: daemon is not using the default seccomp profile
when i run the command
docker info | grep -e “Runtime” -e “Root”
i get
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Docker Root Dir: /var/lib/docker
however, when I run
sudo docker info | grep -e “Runtime” -e “Root”
I get
Runtimes: runc io.containerd.runc.v2 nvidia
Default Runtime: nvidia
Docker Root Dir: /var/lib/docker
Now, I tried to run
sudo jetson-containers run --name ollama $(autotag ollama)
and I get the following result
Namespace(packages=['ollama'], prefer=['local', 'registry', 'build'], disable=[''], user='dustynv', output='/tmp/autotag', quiet=False, verbose=False)
-- L4T_VERSION=36.4.3 JETPACK_VERSION=6.2 CUDA_VERSION=12.6
-- Finding compatible container image for ['ollama']
Found compatible container dustynv/ollama:0.6.8-r36.4-cu126-22.04 (2025-05-06, 5.1GB) - would you like to pull it? [Y/n] Y
dustynv/ollama:0.6.8-r36.4-cu126-22.04
V4L2_DEVICES:
### ARM64 architecture detected
### Jetson Detected
SYSTEM_ARCH=tegra-aarch64
/home/marc/jetson-containers/run.sh: line 326: /tmp/nv_jetson_model: Permission denied
+ docker run --runtime nvidia --env NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics -it --rm --network host --shm-size=8g --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /home/marc/jetson-containers/data:/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --device /dev/snd --device /dev/bus/usb --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-7 --device /dev/i2c-9 -v /run/jtop.sock:/run/jtop.sock --name ollama dustynv/ollama:0.6.8-r36.4-cu126-22.04
Starting ollama server
OLLAMA_HOST 0.0.0.0
OLLAMA_LOGS /data/logs/ollama.log
OLLAMA_MODELS /data/models/ollama/models
I run :, on a terminal window on the jetson:
sudo docker ps
and I see my container running
I then try to run, in the docker shell :
ollama run gemma3
in jtop, i see a heavy load on the CPU’s, but not much GPU load