NvRmMemInitNvmap failed / NVMAP permission denied when launching nvcr.io/nvidia/vllm:25.11-py3 container on Jetson Orin NX + JetPack 6.2 (L4T 36.4.3)

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
, , , , , , ,
1

NvRmMemInitNvmap failed _ NVMAP permission denied when launching nvcr.io_nvidia_vllm_25.11-py3 container on Jetson Orin NX + JetPack 6.2 (L4T 36.4.3).pdf (1.2 MB)

Hi!

When I run the nvcr.io/nvidia/vllm:25.11 container with GPU support, I get the following error message. Even when I start the container with --privileged, the same error still occurs.
（I have also compiled the text and images below into a PDF, hoping it will be helpful to you.）

📌

sudo docker run -d -t \

--net=host \

--gpus all \

--ipc=host \

--name vllm \

-v /home/unitree/models:/workspace/models \

--restart=unless-stopped \

nvcr.io/nvidia/vllm:25.11-py3

1f47c643e288aeecfe37efa3d9691a801e4670321c10572c3c76975b9292c8ff

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

nvidia-container-cli: detection error: nvml error: unknown error

Run ‘docker run --help’ for more information

sudo docker run -d -t \

--net=host \

--gpus all \

–privileged \

--ipc=host \

--name vllm \

-v /home/unitree/models:/workspace/models \

--restart=unless-stopped \

nvcr.io/nvidia/vllm:25.11-py3

[sudo] password for unitree:

d1d87ba2a83a8c2e2d6e6d53ecdc8f997a5b1275284bfe968bd94fc72b037f7f

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

nvidia-container-cli: detection error: nvml error: unknown error

Run ‘docker run --help’ for more information

This is “id ${USER}”.

📌

unitree@unitree-g1-nx:~/xp/jetson-containers$ id ${USER}

uid=1000(unitree) gid=1000(unitree) groups=1000(unitree),4(adm),24(cdrom),27(sudo),29(audio),30(dip),44(video),46(plugdev),104(render),116(i2c),120(lpadmin),135(gdm),999(gpio),996(weston-launch),136(sambashare),1001(jtop),994(docker)

This is “docker inspect nvcr.io/nvidia/vllm:25.11-py3”.

📌

unitree@unitree-g1-nx:~$ docker inspect nvcr.io/nvidia/vllm:25.11-py3

[

{

“Id”: “sha256:c94cef75a33ff40dd34d45429809d840b64f416f957fbe73af11944213b6b994”,

“RepoTags”: [

“nvcr.io/nvidia/vllm:25.11-py3”

],

“RepoDigests”: [

“nvcr.io/nvidia/vllm@sha256:c94cef75a33ff40dd34d45429809d840b64f416f957fbe73af11944213b6b994”

],

“Comment”: “buildkit.dockerfile.v0”,

“Created”: “2025-11-08T11:13:00.160179947Z”,

“Config”: {

“Env”: [

“PATH=/usr/local/lib/python3.12/dist-packages/torch_tensorrt/bin:/usr/local/cuda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/mpi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin:/opt/amazon/efa/bin:/opt/tensorrt/bin”,

“NVIDIA_REQUIRE_JETPACK_HOST_MOUNTS=”,

“GDRCOPY_VERSION=2.5.1”,

“HPCX_VERSION=2.24.1”,

“MOFED_VERSION=5.4-rdmacore56.0”,

“OPENUCX_VERSION=1.19.0”,

“OPENMPI_VERSION=4.1.7”,

“RDMACORE_VERSION=56.0”,

“EFA_VERSION=1.43.1”,

“AWS_OFI_NCCL_VERSION=1.17.0”,

“OPAL_PREFIX=/opt/hpcx/ompi”,

“OMPI_MCA_coll_hcoll_enable=0”,

“CUDA_VERSION=13.0.2.006”,

“CUDA_DRIVER_VERSION=580.95.05”,

“NVVM_VERSION=13.0.88”,

“DOCA_VERSION=3.1.0”,

“_CUDA_COMPAT_PATH=/usr/local/cuda/compat”,

“ENV=/etc/shinit_v2”,

“BASH_ENV=/etc/bash.bashrc”,

“SHELL=/bin/bash”,

“NVIDIA_REQUIRE_CUDA=cuda>=9.0”,

“NCCL_VERSION=2.28.8”,

“CUBLAS_VERSION=13.1.0.3”,

“CUFFT_VERSION=12.0.0.61”,

“CURAND_VERSION=10.4.0.35”,

“CUSPARSE_VERSION=12.6.3.3”,

“CUSPARSELT_VERSION=0.8.1.1”,

“CUSOLVER_VERSION=12.0.4.66”,

“NPP_VERSION=13.0.1.2”,

“NVJPEG_VERSION=13.0.1.86”,

“CUFILE_VERSION=1.15.1.6”,

“NVJITLINK_VERSION=13.0.88”,

“NVFATBIN_VERSION=13.0.85”,

“CUBLASMP_VERSION=0.6.0.84”,

“NVSHMEM_VERSION=3.4.5”,

“CUDLA_VERSION=13.0.2.006”,

“NVPTXCOMPILER_VERSION=13.0.88”,

“CUDNN_VERSION=9.15.0.58”,

“CUDNN_FRONTEND_VERSION=1.15.0”,

“TRT_VERSION=10.14.1.48”,

“TRTOSS_VERSION=”,

“NSIGHT_SYSTEMS_VERSION=2025.5.1.121”,

“NSIGHT_COMPUTE_VERSION=2025.3.1.4”,

“DALI_VERSION=1.52.0”,

“DALI_BUILD=”,

“DALI_URL_SUFFIX=130”,

“POLYGRAPHY_VERSION=0.49.26”,

“TRANSFORMER_ENGINE_VERSION=2.9”,

“MODEL_OPT_VERSION=0.37.0”,

“CUDA_ARCH_LIST=8.0 8.6 9.0 10.0 11.0 12.0”,

“MAXSMVER=121”,

“NVRX_VERSION=0.4.1+cuda13”,

“LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/torch/lib:/usr/local/lib/python3.12/dist-packages/torch_tensorrt/lib:/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64”,

“NVIDIA_VISIBLE_DEVICES=all”,

“NVIDIA_DRIVER_CAPABILITIES=compute,utility,video”,

“NVIDIA_PRODUCT_NAME=vLLM”,

“CUDA_COMPONENT_LIST=cccl crt nvrtc driver-dev culibos-dev cudart cudart-dev nvcc”,

“LIBRARY_PATH=/usr/local/cuda/lib64/stubs:”,

“VLLM_VERSION=0.11.0+582e4e37”,

“PIP_CONSTRAINT=/etc/pip/constraint.txt”,

“CUDA_HOME=/usr/local/cuda”,

“CPATH=/usr/local/cuda/include:”,

“TORCH_CUDA_ARCH_LIST=8.0 8.6 9.0 10.0 11.0 12.0+PTX”,

“TRITON_PTXAS_PATH=/usr/local/cuda/bin/ptxas”,

“TRITON_CUOBJDUMP_PATH=/usr/local/cuda/bin/cuobjdump”,

“TRITON_NVDISASM_PATH=/usr/local/cuda/bin/nvdisasm”,

“TRITON_CUDACRT_PATH=/usr/local/cuda/include”,

“TRITON_CUDART_PATH=/usr/local/cuda/include”,

“TRITON_CUPTI_PATH=/usr/local/cuda/include”,

“MAX_JOBS=12”,

“PIP_BREAK_SYSTEM_PACKAGES=1”,

“PIP_NO_BUILD_ISOLATION=1”,

“PYTORCH_TRITON_VERSION=3.5.0+git8daff01a”,

“TIKTOKEN_CACHE_DIR=/root/.cache/tiktoken_cache”,

“TIKTOKEN_RS_CACHE_DIR=/root/.cache/tiktoken_cache”,

“NVIDIA_VLLM_VERSION=25.11”,

“NVIDIA_BUILD_ID=231063344”

],

“Entrypoint”: [

“/opt/nvidia/nvidia_entrypoint.sh”

],

“WorkingDir”: “/workspace”,

“Labels”: {

“com.nvidia.build.id”: “231063344”,

“com.nvidia.build.ref”: “70a7e4b3c87282a4cb66684ed2ae82174aa991a2”,

“com.nvidia.cublas.version”: “13.1.0.3”,

“com.nvidia.cuda.version”: “9.0”,

“com.nvidia.cudnn.version”: “9.15.0.58”,

“com.nvidia.cufft.version”: “12.0.0.61”,

“com.nvidia.curand.version”: “10.4.0.35”,

“com.nvidia.cusolver.version”: “12.0.4.66”,

“com.nvidia.cusparse.version”: “12.6.3.3”,

“com.nvidia.cusparselt.version”: “0.8.1.1”,

“com.nvidia.nccl.version”: “2.28.8”,

“com.nvidia.npp.version”: “13.0.1.2”,

“com.nvidia.nvjpeg.version”: “13.0.1.86”,

“com.nvidia.vllm.version”: “0.11.0+582e4e37”,

“com.nvidia.volumes.needed”: “nvidia_driver”,

“org.opencontainers.image.ref.name”: “ubuntu”,

“org.opencontainers.image.version”: “24.04”

}

},

“Architecture”: “arm64”,

“Os”: “linux”,

“Size”: 6105891268,

“RootFS”: {

“Type”: “layers”,

“Layers”: [

“sha256:ab34259f9ca5d315bec1b17d9f1ca272e84dedd964a8988695daf0ec3e0bbc2e”,

…

“sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef”

]

},

“Metadata”: {

“LastTagTime”: “2025-12-09T06:58:27.878443023Z”

},

“Descriptor”: {

“mediaType”: “application/vnd.docker.distribution.manifest.list.v2+json”,

“digest”: “sha256:c94cef75a33ff40dd34d45429809d840b64f416f957fbe73af11944213b6b994”,

“size”: 743

}

}

]

This is “jetson_release”.

📌

unitree@unitree-g1-nx:~/xp/temp$ sudo jetson_release

Software part of jetson-stats 4.3.2 - (c) 2024, Raffaello Bonghi

Model: NVIDIA Jetson Orin NX Engineering Reference Developer Kit - Jetpack 6.2 [L4T 36.4.3]

NV Power Mode[2]: 15W

Serial Number: [XXX Show with: jetson_release -s XXX]

Hardware:

  • P-Number: p3767-0000

  • Module: NVIDIA Jetson Orin NX (16GB ram)

Platform:

  • Distribution: Ubuntu 22.04 Jammy Jellyfish

  • Release: 5.15.148-tegra

jtop:

  • Version: 4.3.2

  • Service: Active

Libraries:

  • CUDA: Not installed

  • cuDNN: Not installed

  • TensorRT: Not installed

  • VPI: Not installed

  • Vulkan: 1.3.204

  • OpenCV: 4.5.4 - with CUDA: NO

This is “cat /etc/nv_tegra_release”.

📌

unitree@unitree-g1-nx:~/xp/temp$ cat /etc/nv_tegra_release

R36 (release), REVISION: 4.3, GCID: 38968081, BOARD: generic, EABI: aarch64, DATE: Wed Jan 8 01:49:37 UTC 2025

KERNEL_VARIANT: oot

TARGET_USERSPACE_LIB_DIR=nvidia

TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

This is “nvidia-smi”.

898×475 25.9 KB

This is “jtop”.

975×586 45.7 KB

969×585 31.8 KB

980×583 42.7 KB

This is “docker --version”.

📌

unitree@unitree-g1-nx:~/xp/temp$ docker --version

Docker version 29.1.2, build 890dcca

unitree@unitree-g1-nx:~/xp/temp$ sudo systemctl status docker

● docker.service - Docker Application Container Engine

Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)

Active: active (running) since Wed 2025-12-10 17:13:43 CST; 10min ago

TriggeredBy: ● docker.socket

Docs: https://docs.docker.com

Main PID: 158962 (dockerd)

Tasks: 18

Memory: 43.4M

CPU: 1.630s

CGroup: /system.slice/docker.service

└─158962 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

This is “docker info”. As you can see, I’ve added mirror sources to the “registry-mirrors” section. This is because I’m in China and can’t pull images from docker.io directly. However, even with these mirrors, I still can’t use them.

📌

unitree@unitree-g1-nx:~/xp/temp$ docker info

Client: Docker Engine - Community

Version: 29.1.2

Context: default

Debug Mode: false

Plugins:

buildx: Docker Buildx (Docker Inc.)

Version: v0.30.1

Path: /usr/libexec/docker/cli-plugins/docker-buildx

compose: Docker Compose (Docker Inc.)

Version: v5.0.0

Path: /usr/libexec/docker/cli-plugins/docker-compose

Server:

Containers: 11

Running: 1

Paused: 0

Stopped: 10

Images: 2

Server Version: 29.1.2

Storage Driver: overlayfs

driver-type: io.containerd.snapshotter.v1

Logging Driver: json-file

Cgroup Driver: systemd

Cgroup Version: 2

Plugins:

Volume: local

Network: bridge host ipvlan macvlan null overlay

Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog

CDI spec directories:

/etc/cdi

/var/run/cdi

Swarm: inactive

Runtimes: io.containerd.runc.v2 nvidia runc

Default Runtime: nvidia

Init Binary: docker-init

containerd version: 1c4457e00facac03ce1d75f7b6777a7a851e5c41

runc version: v1.3.4-0-gd6d73eb8

init version: de40ad0

Security Options:

seccomp

Profile: builtin

cgroupns

Kernel Version: 5.15.148-tegra

Operating System: Ubuntu 22.04.5 LTS

OSType: linux

Architecture: aarch64

CPUs: 4

Total Memory: 15.29GiB

Name: unitree-g1-nx

ID: 99da6a17-d471-4816-b7f9-8457dfc89da9

Docker Root Dir: /var/lib/docker

Debug Mode: false

Experimental: false

Insecure Registries:

::1/128

127.0.0.0/8

Registry Mirrors:

https://hub-mirror.c.163.com/

https://mirror.aliyuncs.com/

https://docker.mirrors.ustc.edu.cn/

https://nvcr.io-mirror.nvidia.com/

Live Restore Enabled: false

Firewall Backend: iptables

unitree@unitree-g1-nx:~/xp/temp$ docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Unable to find image ‘nvidia/cuda:12.4.1-base-ubuntu22.04’ locally

docker: Error response from daemon: failed to resolve reference “docker.io/nvidia/cuda:12.4.1-base-ubuntu22.04”: failed to do request: Head "https://hub-mirror.c.163.com/v2/nvidia/cuda/manifests/12.4.1-base-ubuntu22.04?ns=docker.io": dial tcp: lookup hub-mirror.c.163.com on 223.5.5.5:53: no such host

Run ‘docker run --help’ for more information

This is “nvidia-ctk -h”

📌

unitree@unitree-g1-nx:~/xp/temp$ nvidia-ctk -h

NAME:

NVIDIA Container Toolkit CLI - Tools to configure the NVIDIA Container Toolkit

USAGE:

nvidia-ctk [global options] command [command options] [arguments…]

VERSION:

1.13.5

commit: 6b8589dcb4dead72ab64f14a5912886e6165c079

COMMANDS:

hook A collection of hooks that may be injected into an OCI spec

runtime A collection of runtime-related utilities for the NVIDIA Container Toolkit

info Provide information about the system

cdi Provide tools for interacting with Container Device Interface specifications

system A collection of system-related utilities for the NVIDIA Container Toolkit

help, h Shows a list of commands or help for one command

GLOBAL OPTIONS:

--debug, -d Enable debug-level logging (default: false) [$NVIDIA_CTK_DEBUG]

--help, -h show help (default: false)

--version, -v print the version (default: false)

This is “nvidia-container-runtime --version”

📌

unitree@unitree-g1-nx:~/xp/temp$ nvidia-container-runtime --version

NVIDIA Container Runtime version 1.13.5

commit: 6b8589dcb4dead72ab64f14a5912886e6165c079

spec: 1.1.0-rc.2

runc version 1.3.4

commit: v1.3.4-0-gd6d73eb8

spec: 1.2.1

go: go1.24.9

libseccomp: 2.5.3

This is “jetson-containers run $(autotag vllm)”. I also tried installing vllm with jetson-containers, but it didn’t work either.

📌

unitree@unitree-g1-nx:~$ jetson-containers run $(autotag vllm)

Namespace(packages=[‘vllm’], prefer=[‘local’, ‘registry’, ‘build’], disable=[‘’], user=‘dustynv’, output=‘/tmp/autotag’, quiet=False, verbose=False)

-- L4T_VERSION=36.4.3 JETPACK_VERSION=6.2 CUDA_VERSION=12.6

-- Finding compatible container image for [‘vllm’]

nvcr.io/nvidia/vllm:25.11-py3

V4L2_DEVICES:

  • docker run --runtime nvidia -it --rm --network host --shm-size=8g --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /home/unitree/xp/jetson-containers/data:/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --device /dev/snd -e PULSE_SERVER=unix:/run/user/1000/pulse/native -v /run/user/1000/pulse:/run/user/1000/pulse --device /dev/bus/usb --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-7 --device /dev/i2c-9 -v /run/jtop.sock:/run/jtop.sock --name jetson_container_20251210_174314 nvcr.io/nvidia/vllm:25.11-py3

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

NvRmMemInitNvmap failed with Permission denied

356: Memory Manager Not supported

NvRmMemMgrInit failed error type: 196626

libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626

nvidia-container-cli: detection error: nvml error: unknown error

Run ‘docker run --help’ for more information

This is “jetson-containers commit hash”.

📌

unitree@unitree-g1-nx:~/xp/jetson-containers$ git rev-parse HEAD

62bc36d9efb3056b67403ed096c5df8edf7f6f8f

Project source and local modification about jetson-containers:

I cloned the jetson-containers repository through the China mirror

git clone https://gitclone.com/github.com/dusty-nv/jetson-containers.git

and the only file I changed is jetson-containers/requirements.txt.

The original line

git+https://github.com/Granulate/DockerHub-API.git

was unreachable from China, so I replaced it with

git+https://gitclone.com/github.com/Granulate/DockerHub-API.git

(all other lines remain identical).

After that I simply ran bash install.sh to set up the tool.

4

Hi,

The container requires the SBSA GPU driver but Orin uses nvgpu instead.
Please try the container below that was built for the Orin series:

Thanks.

6

Hi~

I executed the following command on my local Linux x86 machine:

docker pull --platform linux/arm64 dustynv/vllm:r36.4-cu129-24.04

Then I ran:

docker inspect dustynv/vllm:r36.4-cu129-24.04

Here is the output:

【

root@unitree-g1-nx:/home/unitree# docker inspect dustynv/vllm:r36.4-cu129-24.04
[
{
"Id": "sha256:14b99e86bd00e940626d61919c93723bbd77a87ff3e61640a11ca55210b0dd88",
"RepoTags": [
"dustynv/vllm:r36.4-cu129-24.04"
],
"RepoDigests": [
"dustynv/vllm@sha256:14b99e86bd00e940626d61919c93723bbd77a87ff3e61640a11ca55210b0dd88"
],
"Created": "2025-07-07T02:48:58.933436399Z",
"Config": {
"Env": [
"PATH=/root/.cargo/bin:/opt/venv/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"DEBIAN_FRONTEND=noninteractive",
"LANGUAGE=en_US:en",
"LANG=en_US.UTF-8",
"LC_ALL=en_US.UTF-8",
"WGET_FLAGS=--quiet --show-progress --progress=bar:force:noscroll --no-check-certificate",
"MULTIARCH_URL=https://apt.jetson-ai-lab.dev/multiarch",
"TAR_INDEX_URL=https://apt.jetson-ai-lab.dev/jp6/cu129/24.04",
"PIP_INDEX_URL=https://pypi.jetson-ai-lab.dev/jp6/cu129",
"PIP_TRUSTED_HOST=",
"TWINE_REPOSITORY_URL=http://localhost/jp6/cu129",
"TWINE_USERNAME=jp6",
"TWINE_PASSWORD=none",
"SCP_UPLOAD_URL=localhost:/dist/jp6/cu129/24.04",
"SCP_UPLOAD_USER=None",
"SCP_UPLOAD_PASS=None",
"CUDA_HOME=/usr/local/cuda",
"NVCC_PATH=/usr/local/cuda/bin/nvcc",
"NVIDIA_VISIBLE_DEVICES=all",
"NVIDIA_DRIVER_CAPABILITIES=all",
"CUDAARCHS=87",
"CUDA_ARCHITECTURES=87",
"CUDNN_LIB_PATH=/usr/lib/aarch64-linux-gnu",
"CUDNN_LIB_INCLUDE_PATH=/usr/include",
"CMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc",
"CUDA_NVCC_EXECUTABLE=/usr/local/cuda/bin/nvcc",
"CUDACXX=/usr/local/cuda/bin/nvcc",
"TORCH_NVCC_FLAGS=-Xfatbin -compress-all",
"CUDA_BIN_PATH=/usr/local/cuda/bin",
"CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda",
"LD_LIBRARY_PATH=/usr/local/cuda/compat:/usr/local/cuda/lib64:",
"PYTHON_VERSION=3.12",
"PYTHONFAULTHANDLER=1",
"PYTHONUNBUFFERED=1",
"PYTHONIOENCODING=utf-8",
"PYTHONHASHSEED=random",
"PIP_NO_CACHE_DIR=true",
"PIP_CACHE_PURGE=true",
"PIP_ROOT_USER_ACTION=ignore",
"PIP_DISABLE_PIP_VERSION_CHECK=on",
"PIP_DEFAULT_TIMEOUT=100",
"PIP_WHEEL_DIR=/opt/wheels",
"PIP_VERBOSE=1",
"TWINE_NON_INTERACTIVE=1",
"OPENBLAS_CORETYPE=ARMV8",
"DYNAMIC_ARCH=1",
"NUMPY_PACKAGE=numpy",
"NUMPY_VERSION_MAJOR=2",
"TORCH_CUDA_ARCH_LIST=8.7",
"PIP_EXTRA_INDEX_URL=",
"TORCH_HOME=/data/models/torch",
"TRANSFORMERS_CACHE=/data/models/huggingface",
"HUGGINGFACE_HUB_CACHE=/data/models/huggingface",
"HF_HOME=/data/models/huggingface",
"TRANSFORMERS_PACKAGE=",
"TRANSFORMERS_VERSION=",
"DIFFUSERS_FORCE_DISABLE_TRITON=1",
"XFORMERS_FORCE_DISABLE_TRITON=1"
],
"Cmd": [
"/bin/bash"
],
"WorkingDir": "/",
"Labels": {
"org.opencontainers.image.ref.name": "ubuntu",
"org.opencontainers.image.version": "24.04"
}
},
"Architecture": "arm64",
"Variant": "v8",
"Os": "linux",
"Size": 17649327685,
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:ccd2cfe57c25db3597f049070e10ffe286c34a12ccaadcf69ce28543ce350979",
.....many sha256 layers.....             "sha256:222616c338ab81efc5688bc4b53e83d65099a9d0098b86326220dedec64d3235"
]
},
"Metadata": {
"LastTagTime": "2025-12-17T03:21:00.257241877Z"
},
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:14b99e86bd00e940626d61919c93723bbd77a87ff3e61640a11ca55210b0dd88",
"size": 12309,
"annotations": {
"io.containerd.image.name": "docker.io/dustynv/vllm:r36.4-cu129-24.04",
"org.opencontainers.image.ref.name": "r36.4-cu129-24.04"
}
}
}
]

】

After that, I performed the following operations on the image dustynv/vllm:r36.4-cu129-24.04 :

  • Export the image :
docker save -o /mnt/sda-20T/xupeng/docker_images/vllm-jetson.tar dustynv/vllm:r36.4-cu129-24.04
  • Transfer the image to Jetson orin nx :
rsync -avz /mnt/sda-20T/xupeng/docker_images/vllm-jetson.tar unitree@172.16.22.27:/home/unitree/xp/docker_images/
  • Change permissions :
sudo chmod -R 777 /home/unitree/xp/docker_images/
  • Load the image :
docker load -i /home/unitree/xp/docker_images/vllm-jetson.tar
  • Run the container :
docker run --rm -it --shm-size 2g --ipc host --gpus all --ulimit memlock=-1 --ulimit stack=67108864 -v  /home/unitree/models:/workspace/models -p 8000:8000 dustynv/vllm:r36.4-cu129-24.04

However, when I executed the docker run command, I encountered an error. The error message is as follows:

【

unitree@unitree-g1-nx:~$ docker run --rm -it --shm-size 2g --ipc host --gpus all --ulimit memlock=-1 --ulimit stack=67108864 -v  /home/unitree/models:/workspace/m
odels -p 8000:8000 dustynv/vllm:r36.4-cu129-24.04
WARNING: IPv4 forwarding is disabled. Networking will not work.
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported
NvRmMemMgrInit failed error type: 196626
libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported
NvRmMemMgrInit failed error type: 196626
libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported
NvRmMemMgrInit failed error type: 196626
libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
nvidia-container-cli: detection error: nvml error: unknown error
Run 'docker run --help' for more information

】

What could be the issue? I look forward to your reply. Thank you!

7

Hi~
I would be very grateful if you could help me with my question. I really need your help, thanks.