Starting text-generation-webui from jetson-containers on Jetson Orin Nano throws error

I am trying to follow this tutorial to run text-generation-webui on a Jetson Orin Nano Developer kit, but when i run ./run.sh $(./autotag text-generation-webui) it shows the following output:

albert@albert-desktop:~/jetson-containers$ ./run.sh $(./autotag text-generation-webui)
Namespace(packages=['text-generation-webui'], prefer=['local', 'registry', 'build'], disable=[''], user='dustynv', output='/tmp/autotag', quiet=False, verbose=False)
-- L4T_VERSION=36.2.0  JETPACK_VERSION=6.0  CUDA_VERSION=12.2.140
-- Finding compatible container image for ['text-generation-webui']
dustynv/text-generation-webui:r36.2.0
localuser:root being added to access control list
xauth:  unable to rename authority file /tmp/.docker.xauth, use /tmp/.docker.xauth-n
chmod: changing permissions of '/tmp/.docker.xauth': Operation not permitted
./run.sh: line 45: /tmp/nv_jetson_model: Permission denied
+ docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /home/albert/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb -e DISPLAY=:0 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /tmp/.docker.xauth:/tmp/.docker.xauth -e XAUTHORITY=/tmp/.docker.xauth dustynv/text-generation-webui:r36.2.0
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported



****NvRmMemMgrInit failed**** error type: 196626


libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported



****NvRmMemMgrInit failed**** error type: 196626


libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported



****NvRmMemMgrInit failed**** error type: 196626


libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
nvidia-container-cli: detection error: nvml error: unknown error: unknown.

I have tried running run.sh with sudo, but the error is roughly the same.

System info

albert@albert-desktop:~/jetson-containers$ docker info
Client: Docker Engine - Community
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: nvidia
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.122-tegra
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.442GiB
 Name: albert-desktop
 ID: ffa49c59-2078-488d-801e-ba553cddef28
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

albert@albert-desktop:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:08:11_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0
albert@albert-desktop:~$ hostnamectl
 Static hostname: albert-desktop
       Icon name: computer
      Machine ID: 5dbfb12414a3456d9014d88183e338b1
         Boot ID: 09b7528436754432b0ef89b2132e5f56
Operating System: Ubuntu 22.04.3 LTS                   
          Kernel: Linux 5.15.122-tegra
    Architecture: arm64
 Hardware Vendor: Unknown
  Hardware Model: NVIDIA Jetson Orin Nano Developer Kit

Hi,

The error indicates some permission issues.
Could you try to run the script with sudo to see if it helps?

Thanks.

@albert.unn are you able to run other containers like l4t-jetpack?

Update

I had not properly set up jetpack and the problem was probably something with CUDA compatability. I just reflashed my Jetson Orin Nano (with Jetpack 5.1.3 this time) and the problem did not rise anymore.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.