Error while running Docker Services example on Drive Orin

arpit.agarkar · September 26, 2023, 1:09pm

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

I am trying to run Docker Services | NVIDIA Docs example on my Drive Orin. While running locally, it’s working but while running on Docker, getting the below error.

sudo docker run --rm --runtime nvidia --gpus all -v $(pwd):$(pwd) -w $(pwd) ubuntu:20.04 ./deviceQuery

docker: Error response from daemon: unknown or invalid runtime name: nvidia.
See ‘docker run --help’.

SivaRamaKrishnaNV · September 26, 2023, 1:20pm

Dear @arpit.agarkar
Please re install nvidia rutime and check if fixes issue

arpit.agarkar · September 26, 2023, 1:46pm

Please provide detailed steps

SivaRamaKrishnaNV · September 26, 2023, 2:08pm

Dear @arpit.agarkar,
Could you try if it fixes?

sudo apt-get remove nvidia-container-runtime
sudo apt-get install nvidia-container-runtime

poweroff/on device via aurix console

arpit.agarkar · September 27, 2023, 7:02am

While running sudo apt-get remove nvidia-container-runtime , we are getting

Reading package lists… Done
Building dependency tree
Reading state information… Done
Package ‘nvidia-container-runtime’ is not installed, so not removed
The following packages were automatically installed and are no longer required:
libnvidia-container-tools libnvidia-container0 libnvidia-container1
Use ‘sudo apt autoremove’ to remove them.
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.

And while sudo apt-get install nvidia-container-runtime,

Reading package lists… Done
Building dependency tree
Reading state information… Done
Package nvidia-container-runtime is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
nvidia-container-toolkit

E: Package ‘nvidia-container-runtime’ has no installation candidate

So, I tried this sudo apt-get install nvidia-container-toolkit
Reading package lists… Done
Building dependency tree
Reading state information… Done
Package nvidia-container-toolkit is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package ‘nvidia-container-toolkit’ has no installation candidate

Please suggest what needs to be done now

arpit.agarkar · September 28, 2023, 4:08am

Any update on this issue @SivaRamaKrishnaNV

SivaRamaKrishnaNV · September 28, 2023, 5:30am

Dear @arpit.agarkar,
I notice below output when running on my target.

nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/1_Utilities/deviceQuery$ sudo docker run --rm --runtime nvidia --gpus all -v $(pwd):$(pwd) -w $(pwd) ubuntu:20.04 ./deviceQuery
WARNING: IPv4 forwarding is disabled. Networking will not work.
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "Orin"
  CUDA Driver Version / Runtime Version          11.8 / 11.4
  CUDA Capability Major/Minor version number:    8.7
  Total amount of global memory:                 28458 MBytes (29840424960 bytes)
  (016) Multiprocessors, (128) CUDA Cores/MP:    2048 CUDA Cores
  GPU Max Clock rate:                            1275 MHz (1.27 GHz)
  Memory Clock rate:                             1275 Mhz
  Memory Bus Width:                              128-bit
  L2 Cache Size:                                 4194304 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total shared memory per multiprocessor:        167936 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  1536
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 2 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            Yes
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device supports Managed Memory:                Yes
  Device supports Compute Preemption:            Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 0 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 11.8, CUDA Runtime Version = 11.4, NumDevs = 1
Result = PASS
nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/1_Utilities/deviceQuery$ cat /etc/nvidia/
00_nvidia_lib.conf             oem-config/                    ubi-eula.sh
DRIVE_EULA.txt                 persistent_partitions_resized  ubi-slf.sh
configs/                       rootfilesystem-manifest/       version-ubuntu-rootfs.txt
efs/                           run-once/                      version_checker/
network/                       serial_getty_enabled           weston_lib.conf
nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/1_Utilities/deviceQuery$ cat /etc/nvidia/version-ubuntu-rootfs.txt
6.0.6.0-32441545

nvidia@tegra-ubuntu:/usr/local/cuda-11.4/samples/1_Utilities/deviceQuery$ sudo dpkg -l | grep nvidia
ii  libnvidia-container-tools            1.10.0-1                                arm64        NVIDIA container runtime library (command-line tools)
ii  libnvidia-container0:arm64           0.11.0+jetpack                          arm64        NVIDIA container runtime library
ii  libnvidia-container1:arm64           1.10.0-1                                arm64        NVIDIA container runtime library
ii  nvidia-container-toolkit             1.10.0-1                                arm64        NVIDIA container runtime hook
ii  nvidia-docker2                       2.11.0-1                                all          nvidia-docker CLI wrapper

Please check installing nvidia-container toolkit.

arpit.agarkar · September 28, 2023, 6:05am

I tried this

sudo apt-get install nvidia-container-toolkit

Reading package lists… Done
Building dependency tree
Reading state information… Done
Package nvidia-container-toolkit is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package ‘nvidia-container-toolkit’ has no installation candidate

VickNV · September 28, 2023, 4:00pm

Please follow the ‘DRIVE OS 6.0.8.1 Installation Guide for NVIDIA Developers’ to upgrade to the latest version, and see if you still encounter this issue. If the problem persists after the upgrade, feel free to reach out for further assistance.

arpit.agarkar · October 17, 2023, 1:23pm

Can you please confirm if this example wont run on Drive Os 6.0.6 ?

VickNV · October 17, 2023, 6:53pm

We haven’t come across any reports of developers facing this issue. To help diagnose the problem, I just reinstalled my system with 6.0.6 and executed the following commands (from the document) without any problems:

cd /usr/local/cuda-11.4/samples/1_Utilities/deviceQuery/ && sudo make
sudo docker run --rm --runtime nvidia --gpus all -v $(pwd):$(pwd) -w $(pwd) ubuntu:20.04 ./deviceQuery

system · November 15, 2023, 8:27am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Docker on top of AGX orin hardware with 6.0.6 DRIVE AGX Orin General docker	21	1147	June 28, 2023
[BUG] failed to start docker container in orin target with error: failed to create endpoint on network bridge, operation not supported DRIVE AGX Orin General docker	7	1807	September 5, 2024
Docker Support DriveOS 6.0.8.1 DRIVE AGX Orin General docker	27	1789	May 3, 2024
Running Docker Containers Directly on NVIDIA DRIVE AGX Orin Technical Blog	17	1686	April 25, 2023
Docker can not run Jetson AGX Orin docker	6	87	November 7, 2024
Error run docker image nvcr.io/nvidia/l4t-base:r32.6.1 on jetson AGX Jetson AGX Xavier docker , jetson	9	2469	November 10, 2021
Docker exec fails on xavier DRIVE AGX Xavier General docker	14	4416	October 12, 2021
Upgrading CUDA for Autoware Compatibility and tensorrt libs not Accessible Inside the l4t-jetpack DRIVE AGX Orin General driveos-cuda	10	792	January 22, 2024
Nvidia driver-container does not work after restart Docker and NVIDIA Docker	7	6015	March 24, 2022
Unable to to install Nvidia Driver on Drive AGX Orin DRIVE AGX Orin General driveos-cuda	11	998	November 29, 2023

Error while running Docker Services example on Drive Orin

Related topics