Starting text-generation-webui from jetson-containers on Jetson Orin Nano throws error

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
, ,
1

I am trying to follow this tutorial to run text-generation-webui on a Jetson Orin Nano Developer kit, but when i run ./run.sh $(./autotag text-generation-webui) it shows the following output:

albert@albert-desktop:~/jetson-containers$ ./run.sh $(./autotag text-generation-webui)
Namespace(packages=['text-generation-webui'], prefer=['local', 'registry', 'build'], disable=[''], user='dustynv', output='/tmp/autotag', quiet=False, verbose=False)
-- L4T_VERSION=36.2.0  JETPACK_VERSION=6.0  CUDA_VERSION=12.2.140
-- Finding compatible container image for ['text-generation-webui']
dustynv/text-generation-webui:r36.2.0
localuser:root being added to access control list
xauth:  unable to rename authority file /tmp/.docker.xauth, use /tmp/.docker.xauth-n
chmod: changing permissions of '/tmp/.docker.xauth': Operation not permitted
./run.sh: line 45: /tmp/nv_jetson_model: Permission denied
+ docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /home/albert/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb -e DISPLAY=:0 -v /tmp/.X11-unix/:/tmp/.X11-unix -v /tmp/.docker.xauth:/tmp/.docker.xauth -e XAUTHORITY=/tmp/.docker.xauth dustynv/text-generation-webui:r36.2.0
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported



****NvRmMemMgrInit failed**** error type: 196626


libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported



****NvRmMemMgrInit failed**** error type: 196626


libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
NvRmMemInitNvmap failed with Permission denied
356: Memory Manager Not supported



****NvRmMemMgrInit failed**** error type: 196626


libnvrm_gpu.so: NvRmGpuLibOpen failed, error=196626
nvidia-container-cli: detection error: nvml error: unknown error: unknown.

I have tried running run.sh with sudo, but the error is roughly the same.

System info

albert@albert-desktop:~/jetson-containers$ docker info
Client: Docker Engine - Community
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: nvidia
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.122-tegra
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.442GiB
 Name: albert-desktop
 ID: ffa49c59-2078-488d-801e-ba553cddef28
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false


albert@albert-desktop:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:08:11_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0

albert@albert-desktop:~$ hostnamectl
 Static hostname: albert-desktop
       Icon name: computer
      Machine ID: 5dbfb12414a3456d9014d88183e338b1
         Boot ID: 09b7528436754432b0ef89b2132e5f56
Operating System: Ubuntu 22.04.3 LTS                   
          Kernel: Linux 5.15.122-tegra
    Architecture: arm64
 Hardware Vendor: Unknown
  Hardware Model: NVIDIA Jetson Orin Nano Developer Kit
3

Hi,

The error indicates some permission issues.
Could you try to run the script with sudo to see if it helps?

Thanks.

4

@albert.unn are you able to run other containers like l4t-jetpack?

7

Update

I had not properly set up jetpack and the problem was probably something with CUDA compatability. I just reflashed my Jetson Orin Nano (with Jetpack 5.1.3 this time) and the problem did not rise anymore.